<%BANNER%>

Probabilistic Analysis and Results of Combinatorial Problems with Military Applications


PAGE 4

IwishtoexpressmyheartfeltthankstoProfessorPanosPardalosforhisguidanceandsupport.Hisextraordinaryenergeticpersonalityinspiresallthosearoundhim.WhatIappreciatemostaboutProfessorPardalosishesetshighgoalsforhimselfandhisstudentsandthentirelesslystrivestoreachthosegoals.IamgratefultotheUnitedStatesAirForceforitsnancialsupportandforallowingmetopursuemylifelonggoal.WithintheAirForce,IoweadebtofgratitudetoDr.DavidJecoatforhiscounselandassistancethroughoutmyPhDeorts.MyappreciationalsogoestomycommitteemembersStanUryasev,JosephGe-unes,andWilliamHagerfortheirtimeandthoughtfulguidance.IwouldliketothankmycollaboratorsAnthonyOkafor,CarlosOliveira,PavloKrakhmal,andLewisPasil-iao.Finally,tomyfamily,Bonnie,AndrewandErin,whohavebeenextremelysup-portive{Icouldnothavecompletedthisworkwithouttheirloveandunderstanding. iv

PAGE 5

page ACKNOWLEDGMENTS ............................. iv LISTOFTABLES ................................. viii LISTOFFIGURES ................................ x ABSTRACT .................................... xii 1INTRODUCTION .............................. 1 1.1ProbabilisticAnalysisofCombinatorialProblems ......... 1 1.2MainContributionsandOrganizationoftheDissertation ..... 3 2SURVEYOFTHEMULTIDIMENSIONALASSIGNMENTPROBLEM 5 2.1Formulations .............................. 5 2.2Complexity .............................. 7 2.3Applications .............................. 8 2.3.1WeaponTargetAssignmentProblem ............. 8 2.3.2ConsideringWeaponCostsintheWeaponTargetAssign-mentProblem ........................ 11 2.4Summary ............................... 12 3CHARACTERISTICSOFTHEMEANOPTIMALSOLUTIONTOTHEMAP .................................... 13 3.1Introduction .............................. 13 3.1.1BasicDenitionsandResults ................. 14 3.1.2Motivation ........................... 15 3.1.3AsymptoticStudiesandResults ............... 16 3.1.4ChapterOrganization ..................... 19 3.2MeanOptimalCostsforaSpecialCaseoftheMAP ........ 20 3.3BranchandBoundAlgorithm .................... 23 3.3.1Procedure ........................... 24 3.3.2Sorting ............................. 27 3.3.3LocalSearch .......................... 27 3.4ComputationalExperiments ..................... 28 3.4.1ExperimentalProcedures ................... 28 3.4.2MeanOptimalSolutionCosts ................ 29 3.4.3CurveFitting ......................... 33 v

PAGE 6

......... 38 3.5.1ImprovementofB&B ..................... 39 3.5.2ComparisonofB&BImplementations ............ 42 3.6Remarks ................................ 42 4PROOFSOFASYMPTOTICCHARACTERISTICSOFTHEMAP .. 44 4.1Introduction .............................. 44 4.2GreedyAlgorithms .......................... 44 4.2.1GreedyAlgorithm1 ...................... 45 4.2.2GreedyAlgorithm2 ...................... 46 4.3MeanOptimalCostsofExponentiallyandUniformlyDistributedRandomMAPs ........................... 47 4.4MeanOptimalCostsofNormal-DistributedRandomMAPs .... 53 4.5RemarksonFurtherResearch .................... 55 5PROBABILISTICAPPROACHTOSOLVINGTHEMULTISENSORMULTITARGETTRACKINGPROBLEM ............... 56 5.1Introduction .............................. 56 5.2DataAssociationFormulatedasanMAP .............. 58 5.3MinimumSubsetofCostCoecients ................ 62 5.4GRASPforaSparseMAP ...................... 64 5.4.1GRASPComplexity ...................... 64 5.4.2SearchTreeDataStructure .................. 65 5.4.3GRASPvsSparseGRASP .................. 67 5.5Conclusion ............................... 68 6EXPECTEDNUMBEROFLOCALMINIMAFORTHEMAP ..... 69 6.1Introduction .............................. 69 6.2SomeCharacteristicsofLocalNeighborhoods ............ 73 6.3ExperimentallyDeterminedNumberofLocalMinima ....... 74 6.4ExpectedNumberofLocalMinimaforn=2 ............ 77 6.5ExpectedNumberofLocalMinimaforn3 ............ 80 6.6NumberofLocalMinimaEectsonSolutionAlgorithms ..... 85 6.6.1RandomLocalSearch ..................... 85 6.6.2GRASP ............................. 86 6.6.3SimulatedAnnealing ..................... 86 6.6.4Results ............................. 87 6.7Conclusions .............................. 88 7MAPTESTPROBLEMGENERATOR .................. 92 7.1Introduction .............................. 92 7.1.1TestProblemGenerators ................... 94 7.1.2TestProblemLibraries .................... 96 vi

PAGE 7

....................... 98 7.2.1ProposedAlgorithm ...................... 98 7.2.2ProofofUniqueOptimum .................. 102 7.2.3Complexity ........................... 103 7.3MAPTestProblemQuality ..................... 104 7.3.1DistributionofAssignmentCosts .............. 105 7.3.2RelativeDicultlyofSolvingTestProblems ........ 106 7.4TestProblemLibrary ......................... 109 7.5Remarks ................................ 109 8CONCLUSIONS ............................... 111 REFERENCES ................................... 113 BIOGRAPHICALSKETCH ............................ 122 vii

PAGE 8

Table page 3{1MeanoptimalsolutioncostsobtainedfromtheclosedformequationforMAPsofsizesn=2,3d10andwithcostcoecientsthatareindependentexponentiallydistributedwithmeanone. ........ 23 3{2Numberofrunsforeachexperimentwithuniformorexponentialas-signmentcosts. ............................. 29 3{3Numberofrunsforeachexperimentstandardnormalassignmentcosts. 30 3{4MeanoptimalcostsfordierentsizesofMAPswithindependentas-signmentcoststhatareuniformin[0;1]. ............... 31 3{5MeanoptimalcostsfordierentsizesofMAPswithindependentas-signmentcoststhatareexponentialwithmean1. .......... 31 3{6MeanoptimalcostsfordierentsizesofMAPswithindependentas-signmentcoststhatarestandardnormal. ............... 31 3{7Curvettingresultsforttingtheform(An+B)CtothemeanoptimalcostsforMAPswithuniformassignmentcosts. ............ 35 3{8Curvettingresultsforttingtheform(An+B)CtothemeanoptimalcostsforMAPswithexponentialassignmentcosts. .......... 35 3{9CurvettingresultsforttingtheformA(n+B)CtothemeanoptimalcostsforMAPswithstandardnormalassignmentcosts. ....... 36 3{10EstimatedandactualmeanoptimalcostsfromtenrunsforvariouslysizedMAPsdevelopedfromdierentdistributions.Includedaretheaveragedierenceandlargestdierencebetweenestimatedmeanop-timalcostandoptimalcost. ...................... 37 3{11Resultsshowingcomparisonsbetweenthreeprimalheuristicsandthenumericalestimateofoptimalcostforseveralproblemsizesandtypes.Shownaretheaveragefeasiblesolutioncostsfrom50runsofeachprimalheuristiconrandominstances. ............. 40 3{12Averagetimetosolutioninsecondsofsolvingeachofverandomlygeneratedproblemsofvarioussizesandtypes.Theexperimentin-volvedusingtheB&Bsolutionalgorithmwithdierentstartingupperboundsdevelopedinthreedierentways. ............... 43 viii

PAGE 9

................................... 63 5{2Tableofexperimentalresultsofcomparingsolutionqualityandtime-to-solutionforGRASPinsolvingfullydenseandreducedsimulatedMSMTTproblems.Fiverunsofeachalgorithmwereconductedagainsteachproblem. .......................... 68 6{1Averagenumberoflocalminima(2-exchangeneighborhood)fordier-entsizesofMAPswithindependentassignmentcosts. ........ 75 6{2Averagenumberoflocalminima(3-exchangeneighborhood)fordier-entsizesofMAPswithi.i.d.standardnormalassignmentcosts. .. 76 6{3ProportionoflocalminimatototalnumberoffeasiblesolutionsfordierentsizesofMAPswithi.i.d.standardnormalcosts. ...... 76 7{1Timedresultsofproducingtestproblemsofvarioussizes. ....... 105 7{2Chi-squaregoodness-of-ttestfornormaldistributionofassignmentcostsforsixrandomlyselected5x5x5testproblems. ......... 106 7{3Numberofdiscretelocalminimaper106feasiblesolutions.Therangeisa95-percentcondenceintervalbasedonproportionatesampling. 108 7{4Comparisonofsolutiontimesinsecondsusinganexactsolutionalgo-rithmofthebranch-and-boundvariety. ................ 109 7{5ComparisonofsolutionresultsusingaGRASPalgorithm. ....... 109 ix

PAGE 10

Figure page 3{1BranchandBoundontheIndexTree. .................. 24 3{2PlotsofmeanoptimalcostsforfourdierentsizedMAPswithexpo-nentialassignmentcosts. ........................ 30 3{3Surfaceplotsofmeanoptimalcostsfor3d10and2n10sizedMAPswithexponentialassignmentcosts. ........... 32 3{4PlotsofmeanoptimalcostsforfourdierentsizedMAPswithstandardnormalassignmentcosts. ........................ 32 3{5PlotsofstandarddeviationofmeanoptimalcostsforfourdierentsizedMAPswithexponentialassignmentcosts. ........... 33 3{6PlotsofstandarddeviationofmeanoptimalcostsforfourdierentsizedMAPswithstandardnormalassignmentcosts. ........ 34 3{7ThreedimensionalMAPwithexponentialassignmentcosts.Plotin-cludesbothobservedmeanoptimalcostvaluesandttedvalues.Thetwolinesarenearlyindistinguishable. .............. 36 3{8PlotsofttedandmeanoptimalcostsfromtenrunsofvariouslysizedMAPsdevelopedfromtheuniformdistributionon[10;20].Notethattheobserveddataandtteddataarenearlyindistinguishable. 38 3{9PlotsofttedandmeanoptimalcostsfromtenrunsofvariouslysizedMAPsdevelopedfromtheexponentialdistributionwithmeanthree. 38 3{10Plotsofttedandmeanoptimalcostsfromten0runsofvariouslysizedMAPsdevelopedfromanormaldistribution,N(=5;=2). ... 39 3{11Branchandboundontheindextree. .................. 41 5{1Exampleofnoisysensormeasurementsoftargetlocations. ...... 57 5{2Exampleofnoisysensormeasurementsofclosetargets.Inthiscasethereisfalsedetectionandmissedtargets. .............. 57 5{3Searchtreedatastructureusedtondacostcoecientordetermineacostcoecientdoesnotexist. .................... 66 5{4SearchtreeexampleofasparseMAP. .................. 67 x

PAGE 11

................................. 76 6{2Plotsofsolutionqualityversusnumberoflocalminimawhenusingthe2-exchangeneighborhood.TheMAPhasasizeofd=4;n=5withcostcoecientsthatarei.i.d.standardnormal. ........... 89 6{3Plotsofsolutionqualityversusnumberoflocalminimawhenusinga3-exchangeneighborhood.TheMAPhasasizeofd=4;n=5withcostcoecientsthatarei.i.d.standardnormal. ........... 90 7{1Treegraphfor3x4x4MAP. ....................... 98 7{2Initialtreegraphwithassignmentcostsandlowerboundpathcosts. 101 7{3Treegraphwithoptimalpathandcosts. ................ 102 7{4Treegraphusedtoconsiderallfeasiblenodesatlevel3fromtherstnodeinlevel2. ............................. 103 7{5Finaltreegraphfora3x4x4MAP. ................... 104 7{6Typicalnormalprobabilityplotfora5x5x5testproblem. ....... 106 7{7Typicalhistogramof20x30x40testproblem. .............. 107 xi

PAGE 12

Theworkinthisdissertationexaminescombinatorialproblemsfromaprobabilis-ticapproachinaneorttoimproveexistingsolutionmethodsorndnewalgorithmsthatperformbetter.Applicationsaddressedherearefocusedonmilitaryusessuchasweapon-targetassignment,pathplanningandmultisensormultitargettracking;however,thesemaybeeasilyextendedtothecivilianenvironment. Aprobabilisticanalysisofcombinatorialproblemsisaverybroadsubject;how-ever,thecontexthereisthestudyofinputdataandsolutionvalues. Weinvestigatecharacteristicsofthemeanoptimalsolutionvaluesforrandommultidimensionalassignmentproblems(MAPs)withaxialconstraints.Costcoe-cientsaretakenfromthreedierentrandomdistributions:uniform,exponentialandstandardnormal.Inthecaseswherecostcoecientsareindependentuniformorexponentialrandomvariables,experimentaldataindicatethattheaverageoptimalvalueoftheMAPconvergestozeroastheMAPsizeincreases.Wegiveashortproofofthisresultforthecaseofexponentiallydistributedcostswhenthenumberofelementsineachdimensionisrestrictedtotwo.Inthecaseofstandardnormal xii

PAGE 13

Usinganovelprobabilisticapproach,weprovidegeneralizedproofsoftheasymp-toticcharacteristicsofthemeanoptimalcostsofMAPs.Theprobabilisticapproachisthenusedtoimprovetheeciencyofthepopulargreedyrandomizedadaptivesearchprocedure. Asmanysolutionapproachestocombinatorialproblemsrely,atleastpartly,onlocalneighborhoodsearches,itiswidelyassumedthenumberoflocalminimahasimplicationsonsolutiondiculty.WeinvestigatetheexpectednumberoflocalminimaforrandominstancesoftheMAP.Wereportonempiricalndingsthattheexpectednumberoflocalminimadoesimpacttheeectivenessofthreedierentsolutionalgorithmsthatrelyonlocalneighborhoodsearches. AprobabilisticapproachisusedtodevelopanMAPtestproblemgeneratorthatcreatesdicultproblemswithknownuniquesolutions. xiii

PAGE 14

Combinatorialoptimizationproblemsarefoundineverydaylife.Theyarepar-ticularlyimportantinmilitaryapplicationsastheymostoftenconcernmanagementandecientuseofscarceresources.Applicationsofcombinatorialproblemsareinaperiodofrapiddevelopmentwhichfollowsfromthewidespreaduseofcomputersandthedataavailablefrominformationsystems.Althoughcomputershaveallowedexpandedcombinatorialapplications,mostoftheseproblemsremainveryhardtosolve.Thepurposeoftheworkinthisdissertationistoexaminecombinatorialprob-lemsfromaprobabilisticapproachinaneorttoimproveexistingsolutionmethodsorndnewalgorithmsthatperformbetter.Mostapplicationsaddressedherearefo-cusedonmilitaryapplications;however,mostmaybeeasilyextendedtothecivilianenvironment. AnexampleofthestudyofsolutionvaluesisbyBarvinokandStephen[ 13 ],wheretheauthorsobtainanumberofresultsregardingthedistributionofsolutionvalues 1

PAGE 15

ofthequadraticassignmentproblem.Inthepaper,theauthorsconsiderquestionssuchas,howwelldoestheoptimumofasampleofrandompermutationsapproximatethetrueoptimum?Theyexploreaninterestingapproachinwhichtheyconsiderthe\k-thsphere"aroundthetrueoptimum.Thek-thsphere,insimpleterms,quantiesthenearnessofpermutationstotheoptimumpermutation.Byallowingthetrueoptimumtorepresentabullseye,theauthorsobserveasthek-thspherecontractstotheoptimalpermutation,theaveragesolutionvalueofasampleofpermutationssteadilyimproves. Astudyofthequadraticassignmentproblem(QAP)isfoundworkbyAbreuetal.[ 1 ]wheretheauthorsconsiderusingaverageandvarianceofsolutioncoststoestablishthedicultyofaparticularinstance. SanchisandSchnabl[ 103 ]studythe\landscape"ofthetravelingsalesmanprob-lem.Consideredarenumberoflocalminimaandautocorrelationfunctions.TheconceptoflandscapewasintroducedbyWright[ 111 ]andcanbethoughtofasamapofsolutionvaluessuchthattherearepeaksandvalleys.Landscaperoughnesscangiveanindicationofproblemdiculty. Inastudyofcostinputs,Reilly[ 94 ]suggeststhatthedegreeofcorrelationamonginputdatamayinuencethedicultyofndingasolution.Itissuggestedthatanextremelevelofcorrelationcanproduceverychallengingproblems. Inthisdissertation,weuseaprobabilisticapproachtoconsiderhowinputcostsaectsolutionvaluesinanimportantclassofproblemscalledthemultidimensionalassignmentproblem.Wealsoconsiderthemeanoptimalcostsofvariousprobleminstancestoincludesomeasymptoticcharacteristics.Weincludeanotherinterestingprobabilisticanalysiswhichisourstudyoflocalminimaandhowthenumberoflocalminimaaectssolutionmethods.Finally,weuseaprobabilisticapproachtodesignandanalyzeatestproblemgenerator.

PAGE 16

2 .Inthischapter,weprovidealternativeformulationsandapplicationsforthisimportantanddicultproblem. 3 wereportexper-imentallydeterminedvaluesofthemeanoptimalsolutioncostsofMAPswithcostcoecientsthatareindependentrandomvariablesthatareuniformly,exponentiallyornormallydistributed.Usingtheexperimentaldata,wethenndcurvettingmodelsthatcanbeusedtoaccuratelydeterminetheirmeanoptimalsolutioncosts.Finally,weshowhowthenumericalestimatescanbeusedtoimproveatleasttwosolutionmethodsoftheMAP. 4 weprovesomeasymptoticcharacteristicsofthemeanoptimalcostsusinganovelprobabilisticapproach. 4 ,weextendtheapproachinChapter 5 tomoreecientlysolvethedataassociationproblemthatresultsfromthemultisensormultitargettrackingproblem.Inthemultisensormultitargetproblemnoisymeasurementsaremadewithanarbitrarynumberofspatiallydiversesensorsregardinganarbitrarynumberoftargetswiththegoalofestimatingthetrajectoriesofallthetargetspresent.Furthermore,thenumberoftargetsmaychangebymovingintoandoutofdetectionrange.Theprobleminvolvesadataassociationofsensormeasurementstotargetsandestimatesthecurrentstateofeachtarget.Thecombi-natorialnatureoftheproblemresultsfromthedataassociationproblem;thatishow

PAGE 17

doweoptimallypartitiontheentiresetofmeasurementssothateachmeasurementisattributedtonomorethanonetargetandeachsensordetectsatargetnomorethanonce? 6 exploresthenumberoflocalminimaintheMAPandthenconsiderstheimpactofthenumberoflocalminimaonthreesolutionmethods. 7 wedevelopanMAPtestproblemgeneratorandusesomeprobabilisticanalysestodeterminethegenerator'seectivenessincreatingqualitytestproblemswithknownuniqueoptimalsolutions.Alsoincludedisabriefsurveyofsourcesofcombinatorialtestproblems.

PAGE 18

TheMAPisahigherdimensionalversionofthestandard(two-dimensional,orlinear)assignmentproblem.TheMAPisstatedasfollows:givend,nsetsA1;A2;:::;Ad,thereisacostforeachd-tuple 86 ].Solutionmethodshaveincludedbranchandbound[ 87 10 84 ],GreedyRandomizedAdap-tiveSearchProcedure(GRASP)[ 4 74 ],Lagrangianrelaxation[ 90 85 ],ageneticalgorithmbasedheuristic[ 25 ],andsimulatedannealing[ 27 ]. 5

PAGE 19

formulatedasminniXi=1njXj=1nkXk=1cijkxijks.t.njXj=1nkXk=1xijk=1foralli=1;2;:::;ni,niXi=1nkXk=1xijk1forallj=1;2;:::;nj,niXi=1njXj=1xijk1forallk=1;2;:::;nk,xijk2f0;1gforalli;j;k2f1;:::;ng,ninjnk; minn1Xi1=1ndXid=1ci1idxi1ids.t.n2Xi2=1ndXid=1xi1id=1foralli1=1;2;:::;n1,n1Xi1=1nk1Xik1=1nk+1Xik+1=1ndXid=1xi1id1forallk=2;:::;d1,andik=1;2;:::;nk,n2Xi2=1nd1Xid1=1xi1id1forallid=1;2;:::;nd,xi1id2f0;1gforalli1;i2;:::;id2f1;:::;ng,n1n2nd;

PAGE 20

Ifweallown1=n2=nd=n,anequivalentformulationstatestheMAPintermsofpermutations1;:::;d1ofnumbers1ton.Usingthisnotation,theMAPisequivalenttomin1;:::;d12nnXi=1ci;1(i);:::;d1(i); (nin1)!: 44 ].Eveninthecasewhencoststakeonaspecialstructureoftriangleinequalities,CramaandSpieksma[ 31 ]provethethree-dimensionalproblemremainsNP-hard.However,specialcasesthatarenotNP-harddoexist. Burkard,Rudolf,andWoeginger[ 23 ]investigatethethree-dimensionalproblemswithdecomposablecostcoecients.Giventhreen-elementsequencesai;biandci,i=1;:::;n,acostcoecientdijkisdecomposablewhendijk=aibjck.Burkard[ 23 ]ndstheminimizationandmaximizationofthethree-dimensionalassignmentproblemhavedierentcomplexities.Whilethemaximizationproblemissolvableinpolynomialtime,theminimizationproblemremainsNP-hard.Ontheotherhand,Burkard[ 23 ]identiesseveralstructureswheretheminimizationproblemispolyno-miallysolvable. ApolynomiallysolvablecaseoftheMAPoccurswhenthecostcoecientsaretakenfromaMongematrix[ 22 ].AnmnmatrixCiscalledaMongematrixifcij+crscis+crjforall1i
PAGE 21

theMongearrayistoagainconsiderthematrixC.Anytworowsandtwocolumnsmustintersectatexactlyfourelements.TherowsandcolumnssatisfytheMongepropertyifthesumoftheupper-leftandlower-rightelementsisatmostthesumoftheupper-rightandlower-leftelements.Thiscaneasilybeextendedtohigherdimensions.BecauseofthespecialstructureoftheMongematrix,theMAPbecomespolynomiallysolvablewithalexicographicalgreedyalgorithmandtheidentitypermutationisanoptimalsolution. 8 ],schedulingteachingpractices[ 42 ],productionofprintedcircuitboards[ 30 ],placementofdistributionwarehouses[ 87 ],multisensormultitargetproblems[ 74 91 ],trackingelementaryparticles[ 92 ]andmultiagentpathplanning[ 84 ].Moreexamplesandanextensivediscussionsofthesubjectcanbefoundintwoextensivesurveys[ 81 19 ].AparticularmilitaryapplicationoftheMAPistheWeaponTargetAssignmentproblemwhichisdiscussedinthefollowingsubsection. 81 ]considersop-timallyassigningWweaponstoTtargetssothatthetotalexpecteddamagetothetargetsismaximized.Thetermtarget-basedisusedtodistinguishtheseproblemsfromtheasset-basedordefense-basedproblemswherethegoaloftheseproblemsistoassignweaponstoincomingmissilestomaximizethesurvivingassets.Thetarget-basedproblemsprimarilyapplytooensivestrategies. Assumeataparticularinstantintimethenumberandlocationofweaponsandtargetsareknownwithcertainty.Thenasingleassignmentmaybemadeatthatinstant.ConsiderWweaponsandTtargetsanddenexij;i=1;2;:::;W;j=

PAGE 22

1;2;:::;Tas:xij=8>><>>:1ifweaponiassignedtotargetj,0otherwise. Giventhatweaponiengagestargetj,theoutcomeisrandom.P(targetjisdestroyedbyweaponi)=PijP(targetjisnotdestroyedbyweaponi)=1Pij NowassignVjtoindicateavalueforeachtargetj.Theobjectiveistomaximizethedamagetotargetsorminimizethevalueofthetargetswhichmaybeformulated minimizeTXj=1VjWYi=1qxijij subjecttoTXj=1xij=1;i=1;2;:::;Wxij=f0;1g: AtransformationofthisformulationtoanMAPmaybeaccomplished.Usingatwoweapon,twotargetexample,thetransformationfollows.Firstobservethattheobjectivefunctionof( 2.1 )maybewrittenas minimizeV1[qx1111qx2121]+V2[qx1212qx2222]:(2.2)

PAGE 23

Obviously,theindividualprobabilitiesofsurvival,qij,gotooneifweaponidoesnotengagetargetj.Therefore,usingthersttermoftheobjectivefunctioninequation( 2.2 )asanexample,thersttermbecomesV1[q11q21]ifx11=1andx21=1;orV1[q11]ifx11=1andx21=0;orV1[q21]ifx11=0andx21=1;orV1ifx11=0andx21=0: and,12j=8>><>>:1therstbutnotthesecondweaponengagestargetj,0else. Thecostvaluesmaynowberepresentedbycj.Forexample,c111=V1[q11q21]andc121=V1[q11].Usingtheserepresentations,thersttermofobjectivefunction( 2.2 )becomesc111111+c121121+c211211+c221221:

PAGE 24

Forthetwoweapon,twotargetscenario,( 2.1 )mayreformulatedtoathreedimensionalMAPasfollows.min2X=12X=12Xj=1cjjs.t.2Xj=12X=1j=18=1;22X=12X=1j=18j=1;22X=12Xj=1j=18=1;2j2f0;1g8;;j: 2.1 )willresultinaW+1dimensionalMAP.ThenumberofindiceswillbeT.Asmentionedabove,weaponcostsarenotconsideredinthisformulationwhichresultsinallweaponsbeingassigned.Amorerealisticformulationthatconsidersweaponcostsisdevelopedinthenextsubsection. 2.1 )as minimizeWXi=1TXj=1CixijWXi=1;j=T+1Cixij+TXj=1VjWYi=1qxijij subjecttoT+1Xj=1xij=1;i=1;2;:::;Wxij=f0;1g:

PAGE 25

Therstsummationtermconsidersthecostsofweaponsassignedtoactualtargets.Thesecondsummationtermconsidersthesavingsbyapplyingweaponstothedummytarget. Followingasimilardevelopmentasintheprevioussubsection,weobtainagen-eralizedMAPformulationthatincorporatesweaponcosts.minT+1Xw1=1T+1Xw2=1T+1Xj=1cw1w2jw1w2js.t.T+1Xw2=1T+1Xj=1w1w2j=18w1=1;2;:::;T+1T+1Xw1=1T+1Xwk1=1T+1Xwk+1=1T+1Xj=1w1w2j=18k=1;:::;W1;andwk=1;2;:::;T+1T+1Xw1=1T+1XwW=1w1w2j=18j=1;2;:::;T+1w1w2j2f0;1g8w1;w2;:::;j:

PAGE 26

Inthischapter,weinvestigatecharacteristicsofthemeanoptimalsolutionvaluesforrandomMAPswithaxialconstraints.Throughoutthestudy,weconsidercostcoecientstakenfromthreedierentrandomdistributions:uniform,exponentialandstandardnormal.Inthecasesofuniformandexponentialcosts,experimentaldataindicatethatthemeanoptimalvalueconvergestozerowhentheproblemsizeincreases.Wegiveashortproofofthisresultforthecaseofexponentiallydistributedcostswhenthenumberofelementsineachdimensionisrestrictedtotwo.Inthecaseofstandardnormalcosts,experimentaldataindicatethemeanoptimalvaluegoestonegativeinnitywithincreasingproblemsize.Usingcurvettingtechniques,wedevelopnumericalestimatesofthemeanoptimalvalueforvarioussizedproblems.TheexperimentsindicatethatnumericalestimatesarequiteaccurateinpredictingtheoptimalsolutionvalueofarandominstanceoftheMAP. Oneofthesimplestand,insomecases,mostusefulwaysofcreatingprobleminstancesconsistsofdrawingvaluesfromarandomdistribution.Usingthisprocedure,onewishestocreateaproblemthatisdicult\onaverage,"butthatcanalsoappearastheoutcomeofsomenaturalprocess. 13

PAGE 27

Thus,oneofthequestionsthatarisesishowarandomproblemwillbehaveintermsofsolutionvalue,givensomedistributionfunctionandparametersfromwhichvaluesaretaken.Thisquestionturnsouttobeverydiculttosolveingeneral.Asanexample,fortheLinearAssignmentProblem(LAP),resultshavenotbeeneasytoprove,despiteintenseresearchinthiseld[ 5 28 29 55 82 ]. InthischapterweperformacomputationalstudyoftheasymptoticbehaviorforinstancesoftheMAP. 2 providesanoverviewoftheMAPtoincludeformulationsandapplications. Letz(I)bethevalueoftheoptimumsolutionforaninstanceIoftheMAP.Wedenotebyztheexpectedvalueofz(I),overallinstancesIconstructedfromarandomdistribution(thecontextwillmakeclearwhatspecicdistributionwearetalkingabout).Intheprobleminstancesconsideredinthischapter,wehaven1=n2=nd=n. OurmaincontributioninthischapteristhedevelopmentofnumericalestimatesofthemeanoptimalcostsforrandomlygeneratedinstancesoftheMAP.Theexperi-mentsperformedshowthatforuniform[0;1]andexponentiallydistributedcosts,theoptimumvalueconvergestozeroastheproblemsizeincreases.Theseresultsarenotsurprisingforanincreaseindsincethenumberofcostcoecientsincreasesexponen-tiallywithd.However,convergencetozeroforincreasingnisnotasobvioussincetheobjectivefunctionisthesumofncostcoecients.Experimentswithstandardnormallydistributedcostsshowthattheoptimumvaluegoestoastheproblem

PAGE 28

sizeincreases.Moreinterestingly,theexperimentsshowconvergenceevenforsmallvaluesofnandd. Thethreedistributions(exponential,uniformandnormal)werechosenforanal-ysisastheyareveryfamiliartomostpractitioners.Althoughwewouldnotexpectreal-worldproblemstohavecostcoecientsthatfollowexactlythesedistributions,webelievethatourresultsmaybeextendedtoothercostcoecientdistributions. Anothermotivationforthisworkhasbeenthepossibleuseofasymptoticresultsinthepracticalsettingofheuristicalgorithms.WhenworkingwithMAPs,oneofthegreatestdicultiesistheneedtocopewithalargenumberofentriesinthemultidimensionalvectorofcosts.Forexample,inaninstancewithddimensionsandminimumdimensionsizen,therearendcostelementsthatmustbeconsideredfortheoptimumassignment.SolvinganMAPcanbecomeveryhardwhenallelementsofthecostvectormustbereadandconsideredduringthealgorithmexecution.Thishappensbecausethetimeneededtoreadndvaluesmakesthealgorithmexponentialond.Apossibleuseoftheresultsshowninthischapterallowsone,havinggoodestimatesoftheexpectedvalueofanoptimalsolutionandthedistributionofcosts,todiscardalargenumberofentriesinthecostvector,whichhavelowprobabilityofbeingpartofthesolution.Bydoingthis,wecanimprovetherunningtimeofmostalgorithmsfortheMAP.

PAGE 29

Finally,whilesomecomputationalstudieshavebeenperformedfortherandomLAP,suchasbyPardalosandRamakrishnan[ 82 ],therearelimitedpracticalandtheoreticalresultsfortherandomMAP.InthischapterwetrytoimproveinthisrespectbypresentingextensiveresultsofcomputationalexperimentsfortheMAP. 14 ]onthetravelingsalesmanprob-lem(TSP).Otherworkincludesstudiesoftheminimumspanningtree[ 41 105 ],QuadraticAssignmentProblem(QAP)[ 21 ]and,mostnotably,studiesoftheLinearAssignmentProblem(LAP)[ 5 28 55 64 83 76 82 109 ].AmoregeneralanalysiswasmadeonrandomgraphsbyLueker[ 69 ]. InthecaseoftheTSP,theproblemistoletXi,Xi=1;:::;n,beindependentrandomvariablesuniformlydistributedontheunitsquare[0;1]2,andletLndenotethelengthoftheshortestclosedpath(usualEuclidiandistance)whichconnectseachelementoffX1;X2;:::;Xng.TheclassicresultprovedbyBeardwoodetal.[ 14 ]islimn!1Ln 104 ],becauseitiskeytoKarp'salgorithm[ 54 ]forsolvingtheTSP.Karpusesacellulardissectionalgorithmfortheapproximatesolution.Theaboveresultmaybesummarizedasimplyingthattheoptimaltourthroughnpointsissharplypredictablewhennislargeandthedissectionmethodtendstogivenear-optimalsolutionswhennislarge.Thispointstoanideaofusingasymptoticresultstodevelopeectivesolutionalgorithms. Intheminimumspanningtreeproblem,consideranundirectedgraphG=(N;A)denedbythesetNofnnodesandasetAofmarcs,withalengthcijassociatedwitheacharc(i;j)2A.TheproblemistondaspanningtreeofG,calledaminimum

PAGE 30

spanningtree(MST),thathasthesmallesttotallength,LMST,ofitsconstituentarcs[ 3 ].Ifweleteacharclengthcijbeanindependentrandomvariabledrawnfromtheuniformdistributionon[0;1],Frieze[ 41 ]showedthatE[LMST]!(3)=1j=11 105 ],wheretheTuttepolynomialforaconnectedgraphisusedtodevelopanexactformulafortheexpectedvalueofLMSTforanitegraphwithuniformlydistributedarccosts.Additionalworkconcerningthedirectedminimumspanningtreeisalsoavailable[ 17 ]. FortheSteinertreeproblemwhichisanNP-hardvariantoftheMST,Bollobas,etal.[ 18 ]provedthatwithhighprobabilitytheweightoftheSteinertreeis(1+O(1))(k1)(lognlogk)=nwhenk=O(n)andn!1andwherenisthenumberofverticesinacompletegraphwithedgeweightschosenasi.i.d.randomvariablesdistributedasexponentialwithmeanone.Intheproblem,kisthenumberofverticescontainedintheSteinertree. AfamousresultthatsomecalltheBurkard-FinckeconditionrelatestotheQAP.TheQAPwasintroducedbyKoopmansandBeckmann[ 60 ]in1957asamodelforthelocationofasetofindivisibleeconomicalactivities.QAPapplications,extensionsandsolutionmethodsarewellcoveredinworkbyHorstetal.[ 51 ].TheBurkard-Finckecondition[ 21 ]isthattheratiobetweenthebestandworstsolutionvaluesapproachesoneasthesizeoftheproblemincreases. Anotherwaytothinkofthisisforalargeproblemanypermutationisclosetooptimal.AccordingtoBurkardandFincke[ 21 ]thisconditionappliestoallproblemsintheclassofcombinatorialoptimizationproblemswithsum-andbottleneckobjec-tivefunctions.TheLinearOrderingProblem(LOP)[ 26 ]fallsintothiscategoryaswell.BurkardandFinckesuggestthatthisresultmeansthatverysimpleheuristicalgorithmscanyieldgoodsolutionsforverylargeproblems.

PAGE 31

RecentworkbyAldousandSteele[ 6 ]providespartsurvey,parttutorialontheobjectivemethodinunderstandingasymptoticcharacteristicsofcombinatorialproblems.Theyprovidesomeconcreteexamplesoftheapproachandpointoutsomeunavoidablelimitations. Intermsoftheasymptoticnatureofcombinatorialproblems,themostexploredproblemhasbeentheLAP.IntheLAPwearegivenamatrixCnnwithcoecientscij.Theobjectiveistondaminimumcostassignment;i.e.,nelementsc1j1;:::;cnjn,suchthatjp6=jqforallp6=q,withji2f1;:::;ng,andPni=1cijiisminimum. AwellknownconjecturebyMezardandParisi[ 71 72 ]statesthattheopti-malsolutionforinstanceswherecostscijaredrawnfromanexponentialoruniformdistribution,approaches2=6whenn(thesizeoftheinstance)approachesinnity.PardalosandRamakrishnan[ 82 ]provideadditionalempiricalevidencethatthecon-jectureisindeedvalid.TheconjecturewasexpandedbyParisi[ 83 ],whereinthecaseofcostsdrawnfromanexponentialdistributiontheexpectedvalueoftheoptimalsolutionofaninstanceofsizenisgivenby Moreover,nXi=11 28 ].Theauthorsconjecturethattheexpectedvalueoftheoptimumk-assignment,foraxedmatrixofsizenm,isgivenbyXi;j0;i+j
PAGE 32

Theyalsopresentedproofsofthisconjectureforsmallvaluesofn,mandk.Theconjectureisconsistentwithpreviouswork[ 71 83 ],sinceitcanbeprovedthatform=n=kthisissimplytheexpressionin( 3.1 ) Althoughuntilrecentlytheproofsoftheseconjectureshaveeludedmanyre-searchers,therehasbeenprogressinthedeterminationofupperandlowerbounds.Walkup[ 109 ]provedanupperboundof3ontheasymptoticvalueoftheobjectivefunction,whentheproblemsizeincreases.ThiswasimprovedbyKarp[ 55 ],whoshowedthatthelimitisatmost2.Ontheotherhand,Lazarus[ 64 ]provedalowerboundof1+1=e1:3679.MorerecentlythisresultwasimprovedbyOlin[ 76 ]tothetighterlowerboundvalueof1:51. Finally,recentpapersbyLinussonandWastlund[ 67 ]andNairetal.[ 75 ]havesolvedtheconjecturesofMezardandParisi,andCoppersmithandSorkin. ConcerningtheMAP,notmanyresultsareknownabouttheasymptoticbehav-ioroftheoptimumsolutionforrandominstances.However,oneexampleofresentworkisthatbyHuanget.al.[ 52 ].Inthisworktheauthorsconsiderthecompleted-partitegraphwithnverticesineachofdsets.Ifalledgesinthisgraphareassignedindependentweightsthatareuniformlydistributedon[0,1],thentheexpectedmini-mumweightperfectd-dimensionalmatchingisatleast3 16n12=d.Theyalsodescribearandomizedalgorithmtosolvethisproblemwheretheexpectedsolutionhasweightatmost5d3n12=d+d15foralld3.However,notethatforevenamoderatesizeford,thisupperboundisnottight. 3.3 ,tondexactsolu-tionstotheproblem.Then,inSection 3.4 wepresentthecomputationalresultsand

PAGE 33

curvettingmodelstoestimatethemeanoptimalcosts.Followingthis,weprovidesomemethodstousethenumericalmodelstoimprovetheeciencyoftwosolutionalgorithms.Finally,concludingremarksandfutureresearchdirectionsarepresentedinSection 3.6 4 .Initially,weemploythepropertystatedinthefollowingproposition. Accordingtotheproofabove,itisclearwhyinstanceswithn3donothavethesameproperty.Dierentfeasiblesolutionsshareelementsofthecostvector,andthereforethefeasiblesolutionsarenotindependentofeachother.Forexample,consideraproblemofsized=3,n=3.Afeasiblesolutiontothisproblemis

PAGE 34

SupposethatX1;X2;:::;Xkarekindependentgammadistributedvariables.LetX(i)betheithsmallestofthese.Applyingorderstatistics[ 33 ],wehavethefollowingexpressionfortheexpectedminimumvalueofkindependentidenticallydistributedrandomvariablesE[X(1)]=Z10kxf(x)(1F(x))k1dx; Theproblemofndingzforthespecialcasewhenn=2andd3correspondstondingtheexpectedminimumcostE[X(1)],fork=2d1independentgammadistributedfeasiblesolutioncosts,withparameters=2,and=1(notethatkisthenumberoffeasiblesolutions).Throughsomeroutinecalculus,andnotingaresultingpatternaskisincreased,wendthefollowingrelationshipz=k1Xj=0k1jj+2Yi=1i k: 106 ].Asanalternativeapproach,weusetheaboveequationtoprovethefollowingtheorem.

PAGE 35

z=k1Xj=0k1jj+2Yi=1i k=k1Xj=0k1j(j+2)! =k1Xj=0(k1)! (k1j)!(j+2)(j+1) =k1Xj=0(k1)! Equality( 3.4 )isfoundbyachangeofvariable.UsingStirling'sapproximationn!(n=e)np zk1Xj=0k1 =e(k1)k1p (3.6) NotethatthesummationinFormula( 3.7 )isexactlyE[(kj)(kj+1)]foraPoissondistributionwithparameterk,whichthereforehasvaluek.Thus, zep andas(k1)k1=2 AswillbeshowninSection 3.4 ,experimentalresultssupporttheseconclusions,evenforrelativelysmallvaluesofd.Table 3{1 providesthevalueofzforMAPsofsizesn=2,3d10.Wenotethatasimilarapproachandresultsmaybeobtainedforotherdistributionsofcostcoecients.Forexample,wehavesimilarresultsifthe

PAGE 36

costcoecientsareindependentgammadistributedrandomvariables,sincethesumofgammarandomvariablesisagainagammarandomvariable. Table3{1: MeanoptimalsolutioncostsobtainedfromtheclosedformequationforMAPsofsizesn=2,3d10andwithcostcoecientsthatareindependentexponentiallydistributedwithmeanone. 30.80440.53050.35660.24270.16780.11690.080100.056 87 ]whereanindextreedatastructureisusedtorep-resentthecostcoecients.Therearenlevelsintheindextreewithnd1nodesoneachlevelforatotalndnodes.Eachleveloftheindextreehasthesamevalueintherstindex.Afeasiblesolutioncanbeconstructedbyrststartingatthetoplevelofthetree.Thepartialsolutionisdevelopedbymovingdownthetreeonelevelatatimeandaddinganodethatisfeasiblewiththepartialsolution.Thenumberofnodesthatarefeasibletoapartialsolutiondevelopedatleveli,fori=1;2;:::;nis(ni)d1.Acompletefeasiblesolutionisobtaineduponreachingthebottomornth-levelofthetree.DeeperMAPtreerepresentationsprovidemoreopportunitiesfor

PAGE 37

B&Balgorithmstoeliminatebranches.Therefore,wewouldexpecttheindex-basedB&Btobemoreeectiveforalargernumberofelementsineachdimension. 3{1 1fori=1;:::;ndoki0 2S; 4whilei>0do 8ii1 9else 11ifFeasible(S;Li;ki)then 17else 19else Figure3{1: BranchandBoundontheIndexTree. Thealgorithminitializesthetreelevelmarkerski,thesolutionsetS,andthecurrenttreeleveliinSteps1{3.Thevalueofthebest-knownsolutionsetSisdenotedasz.Levelmarkersareusedtotrackthelocationofcostcoecientson

PAGE 38

thetreelevelsandLiisthesetofcoecientsateachleveli.ThesolutionsetScontainsthecostcoecientstakenfromthedierenttreelevels.Steps4{21performanimplicitenumerationofeveryfeasiblepathintheindex-basedtree.Theprocedureinvestigateseverypossiblepathbelowagivennodebeforemovingontothenextnodeinthesametreelevel.Onceallthenodesinagivenlevelaresearchedoreliminatedfromconsiderationthroughtheuseofupperandlowerbounds,thealgorithmmovesuptothepreviouslevelandmovestothenextnodeinthenewlevel.Step11checksifagivencostcoecientLi;ki,whichistheki-thnodeonleveli,isfeasibletothepartialsolutionset.Ifthecostcoecientisfeasibleandifitsinclusiondoesnotcausethelowerboundoftheobjectivefunctiontosurpassthebest-knownsolution,thenthecoecientiskeptinthesolutionset.Otherwise,itisremovedfromSinStep20. Alowerboundthatmaybeimplementedtotrytoremovesomeofthetreebranchesisgivenby:LB(S)=rXi=1Si+nXi=r+1min8jmci;j2;:::;jd; Beforestartingthealgorithm,aninitialfeasiblesolutionisneededforanupperbound.AnaturalselectionwouldbeS=fci;j2;j3;:::;jdji=jmform=2;3;:::;d;i=1;2;:::;ng:

PAGE 39

Thealgorithminitiallypartitionsthecostarrayintongroupsortreelevelswithrespecttothevalueoftheirrstindex.Therstcoecienttobeanalyzedisthenodefurthesttotheleftatleveli=1.Ifthelowerboundofthepartialsolutionthatincludesthatnodeislowerthantheinitialsolution,thepartialsolutioniskept.Itthenmovestothenextlevelwithi=2andagainanalyzesthenodefurthesttotheleft.Thealgorithmkeepsmovingdownthetreeuntiliteitherreachesthebottomorndsanodethatresultsinapartialsolutionhavingalowerboundvaluehigherthantheinitialsolution.Ifitdoesreachthebottom,afeasiblesolutionhasbeenfound.Ifthenewsolutionhasalowerobjectivevaluethantheinitialsolution,thelatestsolutioniskeptasthecurrentbest-knownsolution.Ontheotherhandifthealgorithmdoesencounteranodewhichhasalowerboundgreaterthanthebest-knownsolution,thenthatnodeandallthenodesunderneathitareeliminatedfromthesearch.Thealgorithmthenanalyzesthenextnodetotherightofthenodethatdidnotmeetthelowerboundcriteria.Onceallnodesatagivenlevelhavebeenanalyzed,thealgorithmmovesuptothepreviouslevelandbeginssearchingonthenextnodetotherightofthelastnodeanalyzedonthatlevel. WediscussdierentmodicationsthatmaybeimplementedontheoriginalB&Balgorithmtohelpincreasetherateofconvergence.TheB&Balgorithm'sperformanceisdirectlyrelatedtothetightnessoftheupperandlowerbounds.Therestofthissectionaddressestheproblemofobtainingatighterupperbound.Theobjectiveistoobtainagoodsolutionasearlyaspossible.Byhavingalowupperboundearlyintheprocedure,weareabletoeliminatemorebranchesandguaranteeanoptimalsolutioninashorteramountoftime.Themodicationsthatweintroducearesortingthenodesinallthetreelevelsandperformingalocalsearchalgorithmthatguaranteeslocaloptimality.

PAGE 40

Thesecondwaytosortthetreeistoperformasortprocedureeverytimeacostcoecientischosen.Atagiventreelevel,asetofcoecientsthatarestillfeasibletothepartialsolutioniscreatedandsorted.Findingcoecientsthatarefeasibleiscomputationallymuchlessdemandingthancheckingifaparticularcoecientisstillfeasible.Thedrawbackwiththesecondmethodisthehighnumberofsortingproce-duresthatneedtobeperformed.Forourtestproblems,wehavechosentoimplementtherstapproach,whichistoperformasingleinitialsortingofthecoecientsforeachtreelevel.Thischoicewasmadebecausetherstmethodperformedbestinpracticefortheinstanceswetested. Becauseanoptimalsolutioninoneneighborhooddenitionisnotusuallyop-timalinotherneighborhoods,weimplementavariableneighborhoodapproach.Adescriptionofthismetaheuristicanditsapplicationstodierentcombinatorialopti-mizationproblemsisgivenbyHansenandMladenovic[ 47 ].Variableneighborhoodworksbyexploringmultipleneighborhoodsoneatatime.Forourbranchandbound

PAGE 41

algorithm,weimplementtheintrapermutation2-andn-exchangesandtheinterper-mutation2-exchangepresentedbyPasiliao[ 84 ].Startingfromaninitialsolution,wedeneandsearchtherstneighborhoodtondalocalminimum.Fromthatlocalminimum,weredeneandsearchanewneighborhoodtondanevenbettersolution.Themetaheuristiccontinuesuntilallneighborhoodshavebeenexplored. 3{2 and 3{3 provideasummaryofthesizeofeachexperimentforthevarioustypesofproblems. Thetimetakenbyanexperimentrangedfromaslowasafewsecondstoashighas20hoursona2.2GHzPentium4processor.Weobservedthatprobleminstanceswithstandardnormalassignmentcoststookconsiderablylongertimetosolve;there-fore,problemsizesandnumberofrunsperexperimentaresmaller.Theassignmentcostsci1idforeachprobleminstanceweredrawnfromoneofthreedistributions.The

PAGE 42

Table3{2: Numberofrunsforeachexperimentwithuniformorexponentialassign-mentcosts. 21000100010001000100010001000500310001000100010001000100010001004100010001000100010005005005051000100010005002002001002061000100050050010050207100010005002005020158100010002005020159100010005020151010001000201511500500201250050020132002001514100501510016501750183019202015 63 ]asfollows: 1. GenerateU1andU2,forU1;U2U[0;1]. 2. LetV1=2U11,V2=2U21,andW=V21+V22. 3. IfW>1,gobackto1,elseci1id=V1q W. 3{4 3{5 ,and 3{6 .WeobservethatinallcasesthemeanoptimalcostgetssmallerasthesizeoftheMAPincreases.Figure 3{2 showstheplotsforproblemswithdimensiond=3,d=5,d=7andd=10,asexamplesfortheexponentialcase(plotsfortheuniformcasearesimilar).Weobservethatplotsforhigherdimensionalproblemsconvergetozeroforsmallervaluesofn.Thisisemphasizedinthesurfaceplot,Figure 3{3 ,ofasubsetofthedata.Figure 3{4 showsplotsforproblemswiththesamenumberofdimensions

PAGE 43

Table3{3: Numberofrunsforeachexperimentstandardnormalassignmentcosts. 2100010001000100010001000100050031000100010001000100010001000500410001000100010001000500501551000100010005002002015610001000500100501571000100050201581000100020159100010015101000201150015125013151415 3{2 ,butforthestandardnormalcase.Dierentfromtheuniformandexponentialcases,themeanoptimalsolutioncostsappeartoapproachwithincreasingn. Figure3{2: PlotsofmeanoptimalcostsforfourdierentsizedMAPswithexponen-tialassignmentcosts. WeobservethatintheuniformandexponentialcasesthestandarddeviationofoptimalcostsconvergestozeroasthesizeoftheMAPgetslarger.Clearly,thisjustconrmstheasymptoticcharacteristicoftheresults.However,atrendisdicultto

PAGE 44

Table3{4: MeanoptimalcostsfordierentsizesofMAPswithindependentassign-mentcoststhatareuniformin[0;1]. 20.5845090.419550.3080780.21852 20.159980.11245830.540780.2955780.1551890.0853185 30.04554280.02486240.4808250.2092720.08847390.0386061 40.01710490.007550850.417160.1515430.05490550.019867 50.007360.00259660.3740460.1145510.03574280.011516 60.0035420.0009470.3348050.08979280.024650.0067875 70.0017240.000280.303290.07240170.01751950.004348 80.000815090.2771390.05872190.0129820.00253 90.0002733100.2521560.04861180.009610.00168 10 110.2378840.04193660.00762 120.2162870.0356170.006025 20.07920260.0543526130.2055520.0310225 30.01347170.007492140.1857690.02696 40.00321060.001232150.180002 50.000820.00013160.16832 60.0001170.162104 70180.14787 8190.14583 9200.129913 10 MeanoptimalcostsfordierentsizesofMAPswithindependentassign-mentcoststhatareexponentialwithmean1. 20.80468750.5329070.3539810.241978 20.1672960.11027930.639590.3191880.1651220.0829287 30.0463460.025188440.5311260.2129840.09035520.0391171 40.01704340.007712650.4543080.1558330.05483370.020256 50.0072720.002694560.3969760.1164690.03550340.0116512 60.0034910.00094470.3495430.09091230.02518560.0068995 70.0014820.00032580.3104890.07235510.01757450.0044 80.000660.00001390.283930.05951480.012330.002665 90.0001667100.2634870.04935350.0091650.0019 10 110.2389540.0418090.007305 120.2186660.03546240.005385 20.07896260.0562926130.2033970.0309670.0044667 30.01326250.007394140.1938670.0279 40.00318520.001248150.181644 50.0007230.0001160.172359 60.00003170.161126 70180.15081 8190.144787 9200.134107 10 MeanoptimalcostsfordierentsizesofMAPswithindependentassign-mentcoststhatarestandardnormal. 2-1.52566-2.04115-2.46001-2.91444 2-3.29715-3.680933-3.41537-4.59134-5.57906-6.44952 3-7.22834-7.915874-5.6486-7.52175-9.05299-10.3701 4-11.5257-12.59165-8.00522-10.6145-12.6924-14.5221 5-16.128-17.46766-10.6307-13.9336-16.5947-18.7402 6-20.9121-22.71787-13.2918-17.2931-20.6462-23.4246 7-25.8241 8-16.1144-20.8944-24.7095-28.1166 9-18.9297-24.5215-28.7188 2-4.04084-4.2947710-21.7916-28.6479 3-8.57385-9.1470711-24.7175-31.9681 4-13.5045-14.432812-27.9675 5-18.887313-30.9362 614-34.4204 7

PAGE 45

Figure3{3: Surfaceplotsofmeanoptimalcostsfor3d10and2n10sizedMAPswithexponentialassignmentcosts. Figure3{4: PlotsofmeanoptimalcostsforfourdierentsizedMAPswithstandardnormalassignmentcosts.

PAGE 46

detectforstandarddeviationofoptimalcostsinthestandardnormalcase.Figure 3{5 showstheplotsforthethree,ve,sevenandtendimensionalproblems,asexamples,fortheexponentialcase(plotsfortheuniformcasearesimilar). Figure3{5: PlotsofstandarddeviationofmeanoptimalcostsforfourdierentsizedMAPswithexponentialassignmentcosts. NocleartrendisgiveninFigure 3{6 whichshowstheplotsforthesamedimensionalproblemsbutforthestandardnormalcase.

PAGE 47

Figure3{6: PlotsofstandarddeviationofmeanoptimalcostsforfourdierentsizedMAPswithstandardnormalassignmentcosts. 3{7 3{8 ,and 3{9 .Weobservethatcurvestsurprisinglywelltothecollecteddata.Figure 3{7 isaplotofthecurvettingmodelandobserveddatafortheexponentialcasewhered=3.Notethatthecurvesarenearlyindistinguishable.Thisistypicalformostproblemsizes.AcloseranalysisofthecurvettingparametersforbothuniformandexponentialtypeproblemsindicatesthatasthedimensionoftheMAPincreases,thecurvettingparameterCapproaches(d2).Aheuristicargumentofwhythisissoisgiveninthefollowing. Considerthecaseofuniformlydistributedcostcoecients.ForeachleveloftheindextreerepresentationoftheMAP,theexpectedvalueoftheminimumorder

PAGE 48

Table3{7: Curvettingresultsforttingtheform(An+B)CtothemeanoptimalcostsforMAPswithuniformassignmentcosts. 30.1021.133-1.7648.80E-0440.1830.977-2.9327.74E-0550.3190.782-3.3598.28E-0760.3000.776-4.7735.77E-0770.4080.627-4.9976.28E-0780.4080.621-6.0007.91E-0790.4080.621-7.0003.44E-07100.4080.621-8.0009.50E-07 Curvettingresultsforttingtheform(An+B)CtothemeanoptimalcostsforMAPswithexponentialassignmentcosts. 30.3000.631-1.0455.26E-0540.4180.550-1.9301.07E-0550.4060.601-3.0092.40E-0660.4200.594-3.9428.39E-0870.4140.601-5.0019.42E-0780.4130.617-5.9999.45E-0790.4180.600-7.0001.94E-07100.4140.607-8.0006.68E-07 Againusingaleastsquaresapproach,ifwerebuildthecurvettingmodelsfortheuniformandexponentialcasesbyxingC=2d,wend,asexpected,thelowerdimensionmodelsresultinhighersumofsquares.Theworstttingmodelisthatoftheuniformcasewithd=3.Inthiscasethesumofsquaresincreasesfrom8:80E04to3:32E03andthedierenceinthemodelestimateandactualresultsforn=3increasesfrom2:3%to5%.AlthoughwebelievexingC=2dcanprovideadequatettingmodels,intheremainderofthischapterwecontinuetousethemoreaccuratemodels(whereCisnotxedtoC=2d);however,itisobviousthehigherdimensionproblemsareunaected.

PAGE 49

Table3{9: CurvettingresultsforttingtheformA(n+B)CtothemeanoptimalcostsforMAPswithstandardnormalassignmentcosts. 3-1.453-0.9801.2327.27E-024-1.976-0.9861.2111.54E-015-2.580-1.0531.1642.85E-026-2.662-0.9151.2041.68E-027-3.124-0.9561.1741.20E-038-3.230-0.8821.1943.13E-039-3.307-0.8191.2181.71E-0310-3.734-0.8741.1871.52E-04 ThreedimensionalMAPwithexponentialassignmentcosts.Plotincludesbothobservedmeanoptimalcostvaluesandttedvalues.Thetwolinesarenearlyindistinguishable. Anobviousquestiontoaskiswhathappenswithvariationsofthedistributionparameters.Forexample,whatisthenumericalestimationofzwhenthecostcoecientsaredistributedasuniformon[a;b]orexponentialwithmean?Weproposewithoutproofthefollowingnumericalmodelstoestimatez. Forcostcoecientsthatareuniformon[a;b],thecurvetornumericalesti-mationiszze=an+(ba)(An+B)C,usingthecurvetparametersfortheuniformcaseon[0;1]foundinTable 3{7 .Forcostcoecientsthatareexponentialwithmean,thecurvetiszze=(An+B)Cusingthecurvetparametersfortheexponentialcasewith=1foundinTable 3{8

PAGE 50

Table3{10: EstimatedandactualmeanoptimalcostsfromtenrunsforvariouslysizedMAPsdevelopedfromdierentdistributions.Includedaretheaveragedierenceandlargestdierencebetweenestimatedmeanoptimalcostandoptimalcost. 312Uniformon[5,10]61.161.10.1430.428320Expo,=30.4150.4040.06180.15459N(=0;=3)-86.4-86.51.623.48512Uniformon[-1,1]-12-121.65E-033.16E-0375N(=5;=2)-7.24-7.270.4480.7377Expo,=101.90E-021.95E-022.62E-035.47E-0386Uniformon[10,20]60600.0030.00888Expo,=1:54.13E-043.07E-041.15E-042.30E-0495N(=5;=2)-62.8-63.20.9442.2697Uniformon[-10,-5]-70-703.60E-046.70E-04104N(=1;=4)-53.8-53.30.8312.12105Expo,=27.57E-048.00E-041.10E-044.03E-04 3{9 Toassistinvalidatingthenumericalestimationmodelsdiscussedabove,experi-mentswereconductedtocomparethenumericalestimatesofthemeanoptimalcostsandresultsofsolvedproblems.Theexperimentscreatedteninstancesofdierentproblemsizesandofdierentdistributionsandsolvedthemtooptimality.Avarietyofparameterswereusedforeachdistributioninaneorttoexercisetheestimationmodels.Intherstexperiment,wereportmeanoptimalsolution,estimatedmeanoptimalsolution,themax,andmeanwhere=jzez(I)j.Thatis,foraprobleminstanceisthedierencebetweenthepredictedorestimatedmeanopti-malcostandtheactualoptimalcost.ResultsoftheseexperimentsareprovidedinTable 3{10 .Weobservethatthenumericalestimatesofthemeanoptimalcostsarequiteclosetoactualresults.

PAGE 51

SimilartoFigure 3{7 ,Figures 3{8 3{9 and 3{10 haveplottedresultsofzandze(tteddata)forrandominstancesofdierentsizedproblems.Asintheaboveexperiments,thenumberofrunsislimitedtotenforeachproblemsize.Astheplotsofzandzeareclosetoeachother,thisfurthervalidatesthenumericalmodelsforestimatingz. Figure3{8: PlotsofttedandmeanoptimalcostsfromtenrunsofvariouslysizedMAPsdevelopedfromtheuniformdistributionon[10;20].Notethattheobserveddataandtteddataarenearlyindistinguishable. Figure3{9: PlotsofttedandmeanoptimalcostsfromtenrunsofvariouslysizedMAPsdevelopedfromtheexponentialdistributionwithmeanthree.

PAGE 52

Figure3{10: Plotsofttedandmeanoptimalcostsfromten0runsofvariouslysizedMAPsdevelopedfromanormaldistribution,N(=5;=2). Inthissection,weinvestigatewhetherthenumericalestimatescanbeusedtoimproveabranchandbound(B&B)exactsolutionmethod. 3.3 .RecallthattheB&Bperformsbestbyestablishmentofatightupperboundearlyintheprocess.Atightupperboundallowssignicantpruningofthebranchesofthesearchtree.Weconsidertheuseofthenumericalestimatestosettighterupperboundsthanwouldbeavailablethroughotherprimalheuristics.Anadvantageoftheprimalheuristicis,ofcourse,asolutionisathand;whereas,thenumericalestimateisaboundonlywithnosolution.TheheuristicusedinSection 3.3 randomlyselectsastartingsolutionandthenperformsavariablelocalneighborhoodsearchtondalocalminimum.Alternatively,wealsoconsidertheglobalgreedyandavariationofthemaximumregretapproachesassuggestedbyBalasandSaltzman[ 10 ].Intheglobalgreedyapproach,astartingsolutionisconstructedstep-by-stepbyselectingthesmallestfeasiblecostcoecientthenavariablelocalneighborhoodsearchisappliedtondalocalminimum.Formaximumregret,afeasiblesolutionisconstructedasfollows.Thedierencebetweenthetwosmallestfeasiblecostsassociatedwitheachleveloftheindextreeiscalculated.Thisdierenceiscalledtheregretasitrepresentsthepenaltyfornotchoosingthesmallestcostintherow.

PAGE 53

Table3{11: Resultsshowingcomparisonsbetweenthreeprimalheuristicsandthenumericalestimateofoptimalcostforseveralproblemsizesandtypes.Shownaretheaveragefeasiblesolutioncostsfrom50runsofeachprimalheuristiconrandominstances. 610Uniformon[0,1]0.5300.2160.1650.0017777Uniformon[0,1]0.4330.2010.1820.0019586Uniformon[0,1]0.4290.1860.1680.001994Uniformon[0,1]0.3200.2180.2140.00341104Uniformon[0,1]0.2830.2190.2160.00152610Expo,=10.6110.2260.24260.0025177Expo,=10.4900.2440.2160.0019086Expo,=10.4300.2170.1750.0011494Expo,=10.3850.2670.2700.00318104Expo,=10.3200.2240.2150.0014567N(=0;=1)-12.91-21.29-21.57-23.4076N(=0;=1)-12.91-18.51-18.97-20.8985N(=0;=1)-8.99-15.77-16.08-17.5194N(=0;=1)-6.99-11.67-11.883-13.53104N(=0;=1)-7.00-12.60-12.67-14.44 10 ]wheretheyconsidereveryrowinthemulti-dimensionalcostmatrix,whereasweconsideronlythenrowsintheindextree.Wetookthisapproachasatrade-obetweencomplexityandqualityofthestartingsolution.Table 3{11 providesacomparisonofstartingsolutioncostvaluesforthethreeprimalheuristicsdescribedabovealongwithacomparisonofthenumericalestimateoftheoptimalcostforvariousproblemsizesanddistributiontypes.Thetableshowstheresultsoftheaverageof50randomgeneratedinstances. Intermsofanupperbound,theresultsofTable 3{11 indicatethat,generally,thegreedyprimalheuristicisbetterthantherandomheuristicandmaxregretisbetterthangreedy.Fortheuniformandexponentialcases,thenumericalestimateofoptimalcostsisclearlysmallerthananyoftheresultsoftheheuristics.Inthenormalcases,thenumericalestimateisnotsignicantlysmaller.Fortheuniformandexponentialcases,itappearsmuchistobegainedbysomehowincorporatingthenumericalestimateintoanupperbound.

PAGE 54

Weproposeusingafactor>1ofthenumericalestimateastheupperbound.Ifafeasiblesolutionisfound,thenewsolutionservesastheupperbound.Ifafeasiblesolutionisnotfound,thentheestimatedupperboundisincrementedupwardsuntilafeasiblesolutionisfound.Thisprocessguaranteesanoptimalsolutionwillbefound. Figure 3{11 isfundamentallythesameasFigure 3{1 exceptfortheoutsideloopwhichincrementstheestimatedupperboundupwarduntilafeasiblesolutionisfound. 1solution found=false 2whilesolution found=falsedo 5S; 7whilei>0do 11ii1 12else 14ifFeasible(S;Li;ki)then 20solution found=true 17else 19else Figure3{11: Branchandboundontheindextree. Thetrade-owhichmustbeconsideredisiftheupperboundisestimatedtoolowandincrementedupwardstooslow,thenitmaytakemanyiterationsoverthe

PAGE 55

indextreebeforeafeasiblesolutionisfound.However,nobenetisgainedbysettingtheupperboundtoohigh.Wefoundthroughless-than-rigorousanalysisthatsettoavaluesuchthattheupperboundisincrementedupwardbyonestandarddeviationoftheoptimalcost(seeFigures 3{5 and 3{6 )isanicecompromise. 3{12 comparestheperformanceoftheB&Balgorithmusingtherandomprimalheuristicforastartingupperboundversususingthemaximumregretheuris-ticversususinganumericalestimatefortheupperbound.Thetableshowstheaveragetimestosolutionofverunsonrandominstancesofvariousproblemsizesanddistributiontypes.Intheuniformandexponentialcases,weobservethatB&Busingmaximumregretgenerallydoesslightlybetterthanusingarandomstartingsolution.Wealsoobservetheapproachofusinganumericallyestimatedupperboundsignicantlyoutperformstheotherapproachesinsolvingproblemswithuniformlyorexponentiallydistributedcosts.However,thereisnocleardierencebetweentheapproacheswhensolvingproblemswithnormallydistributedcosts.Thisisexplainedbythesmalldierencesinthestartingupperboundsforeachapproach. 4

PAGE 56

Table3{12: Averagetimetosolutioninsecondsofsolvingeachofverandomlygeneratedproblemsofvarioussizesandtypes.TheexperimentinvolvedusingtheB&Bsolutionalgorithmwithdierentstartingupperboundsdevelopedinthreedierentways. 610Uniformon[0,1]1305131179577Uniformon[0,1]19.119.213.986Uniformon[0,1]20.520.413.194Uniformon[0,1]0.30.290.13104Uniformon[0,1]1.151.120.4610Expo,=112791285120177Expo,=125.525.817.886Expo,=121.824.513.494Expo,=10.240.230.1104Expo,=11.671.660.5767N(=0;=1)54.947.354.276N(=0;=1)89.989.689.285N(=0;=1)24.724.624.694N(=0;=1)1.251.231.24104N(=0;=1)30.730.230.7

PAGE 57

3 leadstoconjecturesconcerningtheasymptoticcharacteristicsofthemeanoptimalcostsofrandomlygeneratedinstancesoftheMAPwherecostsareassignedindependentlytoassignments.Inthischapter,weprovideproofsofmoregeneralizedinstancesofConjecture 3.3 andproveConjec-ture 3.4 .Theproofsarebasedonbuildinganindextree[ 87 ]torepresentthecostcoecientsoftheMAPandthenselectingaminimumsubsetofcostcoecientssuchthatatleastonefeasiblesolutioncanbeexpectedfromthissubset.Thenanupperboundonthecostofthisfeasiblesolutionisestablishedandusedtocompletetheproofs.ThroughoutthischapterweconsiderMAPswithnelementsineachoftheddimensions. Beforepresentingthetheoremsandtheirproofsconcerningtheasymptoticna-tureoftheseproblems,werstconsideranaiveapproach[ 28 ]toestablishingtheasymptoticcharacteristicsbasedonsomegreedyalgorithms. 3.3 themeanoptimalcostsarethoughttogotozerowithincreasingproblemsize.Supposeweconsiderthesolutionfromagreedyalgorithm.Asthesolutionservesasanupperboundtotheoptimalsolution,wecantrytoprovetheconjectureifwecanshowthemeanofthesub-optimalsolutionsgoestozerowithincreasingproblemsize.However,aswillbeshownthisisdicultwithtwocommongreedyalgorithms. 44

PAGE 58

87 ]torepresentthecostcoecientsoftheMAP.Therearenlevelsintheindextreewithnd1nodesoneachlevelforatotalndnodes.Eachleveloftheindextreehasthesamevalueintherstindex.Afeasiblesolutioncanbeconstructedbyrststartingatthetoplevelofthetree.Thepartialsolutionisdevelopedbymovingdownthetreeonelevelatatimeandaddinganodethatisfeasiblewiththepartialsolution.Thenumberofnodesatlevelithatarefeasibletoapartialsolutiondevelopedfromlevels1;2;:::;i1is(ni+1)d1.Acompletefeasiblesolutionisobtaineduponreachingthebottomornth-levelofthetree. Theproposedgreedyalgorithmisasfollows: WewishtocalculatetheexpectedsolutioncostfromthisalgorithmfortheMAPconstructedfromi.i.d.exponentialrandomvariableswithmean1.Letthemeansolutioncostresultingfromthealgorithmberepresentedbyzu.SupposethatX1;X2;:::;Xkareki.i.d.exponentialrandomvariableswithmean1.LetX(i)betheithsmallestofthese.Applyingorderstatistics[ 33 ],wehavethefollowingexpressionfortheexpectedminimumvalueofkindependentidenticallydistributedrandomvariables:E[X(1)]=1=k. Wemaynowconstructafeasiblesolutionusingtheabovegreedyalgorithm.Wedosobyrecallingthatthenumberofnodesthatarefeasibleatleveli+1toapartial

PAGE 59

solutiondevelopeddowntoleveli,fori=1;2;:::;nis(ni)d1.Consideringthisandthefactthatcostcoecientsareindependent,theexpectedsolutioncostofS1is1 (n1)d1andsoforth.Therefore,wend z=n1Xi=01 (ni)d1 whereequation( 4.2 )holdsbecausethen-thtermofequation( 4.1 )isone. Sincez>0,weconcludethisgreedyapproachcannotbeusedtoproveConjec-ture 3.3 .However,maybeamoreglobalapproachwillwork. 10 ]astheGREEDYheuristic.Thealgorithmisasfollows: Usingthiscoveringapproach,weseethenumberofnodesthatarefeasibletoapartialsolutiondevelopeduptoiterationi,fori=1;2;:::;nis(ni)d.Forexample,allndcostcoecientsareconsideredintherstiteration.Thenextiterationhas(n1)dnodesforconsideration.TheexpectedsolutioncostofS1is1=nd.TheexpectedsolutioncostofS2is1=nd+1=nd+1=(n1)d.Theextra1=ndappearsintheexpressionbecause,ingeneral,theexpectedminimumvalueoftheuncoverednodesisatleastasmuchastheexpectedminimumvaluefoundinthepreviousiteration.Wecouldnowdeveloptheexpressionforz;however,wenotethatthealgorithm'slastiterationwillconsideronlyonecostcoecient.Therefore,again,wehavetheresultthatz>1whenusingthisalgorithm.

PAGE 60

WeconcludethatthesesimplegreedyapproachescannotbeusedtoprovetheconjecturesconcerningtheasymptoticcharacteristicsoftheMAP.Inthenextsec-tions,weresorttoanovelprobabilisticapproach. 7 ].Basically,weshowthat,forasubsetoftheindextree,theexpectedvalueofthenumberoffeasiblepathsinthissubsetisatleastone.Thus,suchasetmustcontainafeasiblepathandthisfactcanbeusedtogiveanupperboundonthecostoftheoptimumsolution.Thisisexplainedinthenextproposition. CreateasetAofnodestorepresentareducedindextreebyselectingnodesrandomlyfromeachleveloftheoverallindextreeandplacingthemonacorresponding

PAGE 61

levelinthereducedindextree.ThenumberofnodesinAisobviouslyn.ForthisreducedindextreeofA,therearenpaths(notnecessarilyfeasibletotheMAP)fromthetopleveltothebottomlevel.SincethesetofnodesinAwereselectedrandomly,wemaynowuse%todeterminetheexpectednumberoffeasiblepathsinAbysimplymultiplying%bythenumberofallpathsinthereducedtreeofA.ThatisE[numberfeasiblepathsinA]=%n: 4.4 )wegetnd1 4.3 ). Wenowtakeamomenttodiscusstheconceptoforderstatistics.Formorecompleteinformation,refertostatisticsbookssuchasbyDavid[ 33 ].SupposethatX1;X2;:::;Xkarekindependentidenticallydistributedvariables.LetX(i)bethei-thsmallestofthese.ThenX(i)iscalledthei-thorderstatisticforthesetfX1;X2;:::;Xkg.

PAGE 62

Intherestofthesection,wewillconsiderboundsforthevalueofthe-thorderstatisticofi.i.d.variablesdrawnfromarandomdistribution.Thisvaluewillbeusedtoderiveanupperboundonthecostoftheoptimalsolutionforrandominstances,whennordincreases.Notethat,insomeplaces(e.g.,Equation( 4.6 )),weassumethat=nd1=n!d1 (a) ifnisxedandd!1,then!1,andthereforethereisnodierencebetweenandnd1=n!d1 (b) ifdisxedandn!1,then!ed1.Thisisnotdiculttoderive,sincen n!1 e)n(2n)1 2i1 2n: 2n=(2elogn)1 2n=(2)1 2nelogn

PAGE 63

hasthesamenumberofindependentandidenticallydistributedcostvalues,wemayconcludethatE[X()]isthesameforeachlevelintheindextree.Byrandomlyselectingcostvaluesfromeachofthenlevelsoftheindextree,weexpecttohaveatleastonefeasiblesolutiontotheMAPbyProposition 4.1 .Thus,itisclearthatanupperboundcostfortheexpectedfeasiblesolutioniszu=nE[X()]. kj: Notethat( 4.5 )hastermsandthetermoflargestmagnitudeisthelastterm.Usingthelastterm,anupperboundon( 4.5 )isdevelopedasE[X()]u1Xj=0 k(1)= k+1: 4.1 and 4.2 ,theupperboundforthemeanoptimalsolutiontotheMAPwithexponentialcostsmaybedevelopedas zu=n k+1n k=n 1; 4.6 ),whichgives zun

PAGE 64

Letn=andn!=,whereandaresomexednumbers.Then( 4.6 )becomeszu d1 d1 d1 4.6 )becomeszu=n n1n n; nn n=n n(+ n(2n+ = n(2n(1)+ where( 4.7 )holdsbecause(2) 4.8 )willapproachzeroasn!1.Therefore,fortheexponentialcaselimn!1zu=0andlimd!1zu=0fromabove:

PAGE 65

k+1: 4.1 and 4.2 ,theupperboundonthemeanoptimalsolutionforanMAPwithuniformcostsin[0;1]is zu=n k+1n k; wherek=nd1isthenumberofcostelementsoneachleveloftheindextree.Wemustnowsubstitutethevaluesofkandinto( 4.9 ),whichbecomes zun Applyingto( 4.10 )Stirling'sapproximation,inthesamewayasusedinTheorem 4.3 ,weseethatzu!0asn!1ord!1.Noteagainthatzisboundedfrombelowbyzerobecausethelowerboundofanycostcoecientiszero(acharacteristicoftheuniformrandomvariablein[0;1]).Since0zzu,thiscompletestheproof. 33 ]E[X()]=a+(ba) k+1:

PAGE 66

Therefore,usingPropositions 4.1 and 4.2 ,theupperboundonthemeanoptimalsolutionforanMAPwithuniformcostsin[a;b]is zu=na+(ba) k+1na+(ba) k=na+(ba)n k; wherek=nd1isthenumberofcostelementsoneachleveloftheindextree.Wemustnowsubstitutevaluesofkandinto( 4.11 ),whichresults zuna+(ba)n Itbecomesimmediatelyobviousfrom( 4.12 )thatforaxednandasd!1,zu!n.AszzuandnaisanobviouslowerboundforthisinstanceoftheMAPweconcludethat,forxedn,z!naasd!1. 3.4 .AboundonthecostoftheoptimalsolutionfornormaldistributedrandomMAPscanbefound,usingatechniquesimilartotheoneusedintheprevioussection.However,inthiscaseareasonableboundisgivenbythemedianorderstatistics,asdescribedintheproofofthefollowingtheorem. 2(X(r)+X(r+1)).Obviously,theexpectedvalueofthemedianinbothcasesiszero.Letk=nd1andnotethat,asnordgetlarge,rforeitheroddorevencase.Thereforewemayimmediatelyconclude

PAGE 67

4.1 and 4.2 ,weseethatzzu=nE[X()]andz!asn!1. 32 ]toestablishtheexpectedvalueoftheithorderstatisticofkindependentstandardnormalvariables.Withik=2wehave 2p logk); whereS1=1 1+1 2++1 4.13 )maybedropped.Inaddition,aslightrearrangementof( 4.13 )isuseful: 2p 2p Itisnotdiculttoseethatask!1,thesumoftherstthreetermsof( 4.14 )goesto.Therefore,weconsiderthelasttermof( 4.14 )ask!1. (S1C)

PAGE 68

Notingthatthesecondtermof( 4.15 )goestozeroask!1,andalsomakingthesubstitutionsi==nd1=n!d1 (S1C) p p p p Itisclearthatforaxedn,andasd!1,therighthandsideof( 4.16 )approacheszero.Therefore,usingPropositions 4.1 and 4.2 wehavezzu=nE[X()]andE[X()]!foraxednandd!1.Theproofiscomplete. 5 appliestheprobabilisticapproachtoreducethecardinalityoftheMAPwhich,inturn,isthensolvedbyGRASP.Weshowthisprocesscanresultinbettersolutionsinlesstimeforthedataassociationprobleminthemultisensormultitargettrackingproblem.

PAGE 69

TheMSMTTproblemisageneralizationofthesinglesensorsingletargettrack-ingproblem.IntheMSMTTproblemnoisymeasurementsaremadefromanarbitrarynumberofspatiallydiversesensors(forexamplecooperatingremoteagents)regard-inganarbitrarynumberoftargetswiththegoalofestimatingthestateofallthetargetspresent.SeeFigure 5{1 forvisualrepresentationoftheproblem.Becauseofnoise,measurementsareimperfect.Theproblemisexacerbatedwithmanyclosetargetsandnoisymeasurements.Furthermore,thenumberoftargetsmaychangebymovingintoandoutofdetectionrangeandthereareinstancesoffalsedetectionsasshowninFigure 5{2 .TheMSMTTsolvesadataassociationproblemonthesen-sormeasurementsandestimatesthecurrentstateofeachtargetbasedonthedataassociationproblemforeachsensorscan. ThecombinatorialnatureoftheMSMTTproblemresultsfromthedataasso-ciationproblem;thatis,givendsensorswithntargetmeasurementseach,howdoweoptimallypartitiontheentiresetofmeasurementssothateachmeasurementisattributedtonomorethanonetargetandeachsensordetectsatargetnomorethan 56

PAGE 70

Figure5{1: Exampleofnoisysensormeasurementsoftargetlocations. Figure5{2: Exampleofnoisysensormeasurementsofclosetargets.Inthiscasethereisfalsedetectionandmissedtargets. once?Thedataassociationproblemmaximizesthelikelihoodthateachmeasurementisassignedtothepropertarget.InMSMTT,ascanismadeatdiscrete,periodicmo-mentsintime.Inpracticalinstances,thedataassociationproblemshouldbesolvedinrealtime-particularlyinthecaseofcooperatingagentssearchingforandidenti-fyingtargets.Combiningdatafrommorethanonesensorwiththegoalofimprovingdecision-makingistermedsensorfusion. Solvingevenmoderate-sizedinstancesoftheMAPhasbeenadiculttask,sincealinearincreaseinthenumberofdimensions(inthiscase,sensors)bringsan

PAGE 71

exponentialincreaseinthesizeoftheproblem.Assuch,severalheuristicalgorithms[ 74 90 ]havebeenappliedtothisproblem.However,duetothesizeandcomplexityoftheproblem,eventheheuristicsstruggletoachievesolutionsinrealtime.Inthischapterweproposeasystematicapproachtoreducethesizeandcomplexityofthedataassociationproblem,yetachievehigherqualitysolutionsinfastertimes. Thischapterisorganizedasfollows.WerstgivesomebackgroundondataassociationfortheMSMTTproblem.Wethenintroduceatechniquethatmaybeusedtoreducethesizeoftheproblem.Followingthat,wediscusstheheuristic,GreedyRandomizedAdaptiveSearchProcedure(GRASP),andhowGRASPcanbemodiedtoworkeectivelyonasparseproblem.Finally,weprovidesomecomparativeresultsofthesesolutionmethods. 62 ],forexample.However,thethree-dimensionalassignmentproblemisageneralizationofthethreedimensionalmatchingproblemwhichisshownbyGareyandJohnson[ 44 ]tobeNP-hard. AreviewofthemultitargetmultisensorproblemformulationandalgorithmsisprovidedbyPoore[ 89 ].Bar-Shalom,Pattipati,andYeddanapudi[ 11 ]alsopresentacombinedlikelihoodfunctioninmultisensorairtracsurveillance.

PAGE 72

SupposethatwehaveSsensorsobservinganunknownnumberoftargetsT.TheCartesiancoordinatesofsensorsisknowntobe!s=[xs;ys;zs]0,whiletheunknownpositionoftargettisgivenby!t=[xt;yt;zt]0.Sensorstakesnsmeasurements,zs;is.Sincethemeasurementsoftargetlocationsarenoisy,wehavethefollowingexpressionformeasurementisfromsensors:zs;is=8><>:hs(!t;!s)+!s;isifmeasurementisisproducedbytargetts;isifmeasurementisisafalsealarm Themeasurementnoise,!s;is,isassumedtobenormallydistributedwithzeromeanandcovariancematrixRs.ThenonlineartransformationofmeasurementsfromthesphericaltoCartesianframeisgivenbyhs(!t;!s). ConsidertheS-tupleofmeasurementsZi1;i2;:::;iS,eachelementisproducedbyadierentsensor.Usingdummymeasurementszs;0tomakeacompleteassignment,thelikelihoodthateachmeasurementoriginatesfromthesametargettlocatedat!tisgiven. (Zi1;i2;:::;iSj!t)=SYs=1[PDsp(zs;isj!t)]s;is[1PDs]1s;is;(5.1) wheres;is=8><>:0ifis=0(dummymeasurement)1ifis>0(actualmeasurement) andPDs1isthethedetectionprobabilityforsensorm.ThelikelihoodthatthesetofmeasurementsZi1;i2;:::;iScorrespondstoafalsealarmisasfollows. (Zi1;i2;:::;iSjt=0)=SYs=1[PFs]s;is;(5.2) wherePFs0istheprobabilityoffalsealarmforsensors.

PAGE 73

ThecostofassociatingasetofmeasurementsZi1;i2;:::;iStoatargettisgivenbythelikelihoodratio: (Zi1;i2;:::;iSjt=0)=SYs=1PDsp(zs;isj!t) ThisisthelikelihoodthatZi1;i2;:::;iScorrespondstoanactualtargetandnotafalsealarm. Multiplyingalargesetofsmallnumbersleadstoroundoerrorsastheproductapproacheszero.Toavoidthisproblem,weapplythelogarithmfunctiononbothsides.ThecostofassigningasetofmeasurementsZi1;i2;:::;iStoatargettisgivenbythenegativelogarithmsofthelikelihoodratio. (Zi1;i2;:::;iSjt=0)(5.4) Insteadofmaximizingthelikelihoodfunction,wenowtrytominimizethenegativelog-likelihoodratio.Agoodassociationwould,therefore,havealargenegativecost. Inpractice,theactuallocationoftargettisnotknown.Ifitwere,thenobtainingmeasurementswouldbeuseless.Wedeneanestimateofthetargetpositionas^!t=argmax!t(Zi1;i2;:::;iSj!t):

PAGE 74

Thegeneralizedlikelihoodratioutilizesanestimatedtargetposition.Ourneg-ativelog-likelihoodratiotakesthefollowingform Wecandoatypeofgating minXZi1;i2;:::;iSci1;i2;:::;iSi1;i2;:::;iSs.t.Xi2;i3;:::;iSi1;i2;:::;iS=18i1=1;2;:::;n1Xi1;:::;is1;is+1;:::;iSi1;i2;:::;iS18is=1;2;:::;ns;8s=2;3;:::;S1Xi1;i2;:::;iS1i1;i2;:::;iS18iS=1;2;:::;nS;(5.6)

PAGE 75

wherei1;i2;:::;iS=8>><>>:1ifthetupleZi1;i2;:::;iSisassignedtothesametarget0otherwiseZi1;i2;:::;iS=fz1;i1;z2;i2;:::;zS;iSgn1=minsns8s=1;2;:::;Szs;is2<3 ThetotalnumberofpossiblepartitionsofPSs=1nsmeasurementsintoTtargetsisgivenby M=8>>>>><>>>>>:"nSXi=0TnSinS! (nS+i)!#fornS>T(5.7) wherenSmaxfn1;n2;:::;nS1g. 4 (specicallyProposition 4.1 )whereweusetheindextreerepresentationoftheMAPandrandomlyselectnodesfromeachleveloftreewhere

PAGE 76

WhenthesenodesfromeachlevelarecombinedintosetA,wecanexpectthissettocontainatleastonefeasiblesolutiontotheMAP.ForthegeneralizedMAPwithdimensiondandnielementsineachdimensioni,i=1;2;:::;d,andn1n2nd,wecaneasilyextendequation( 5.8 )bynotingthenumberoffeasiblesolutionsisQdi=2ni! (nin1)!.Usingthiswend (nin1)!1 ConsideranMAPwherethecostcoecientsoftheindextreearesortedinnon-decreasingorderforeachlevelofthetree.Ifthecostcoecientsareindependentidenticallydistributedrandomvariablesthentherstcostcoecientsarefromrandomlocationsateachlevel.Therefore,wemayuseProposition 4.1 andconcludewecanexpectatleastonefeasiblesolutioninA.ThecardinalityofthissetAissubstantiallysmallerthantheoriginalMAPwhichmayresultinfastersolutiontimes.Table 5{1 showsacomparisonofthesizeofAtothesizeofthethreeoriginalproblems.Sincethereducedsetismadeupofthesmallestcostcoecientsweexpectgoodsolutionvalues. Table5{1: ComparisonsofthenumberofcostcoecientsoforiginalMAPtothatinA. NumberofCostCoecientsProblemOriginalMAPA NowconsideranMAPwherecostcoecientsarenotindependentandidenticallydistributed.Inrealworldapplications,costcoecientswillmostlikelybedependent.Consider,forexample,amultisensormultitargettrackingsituationwhereasmalltargetistrackedamongotherlargetargets.Wecanexpectahighernoise/signalratioforthesmallertarget.Thus,costcoecientsassociatedwithmeasurements

PAGE 77

ofthesmallertargetinthedataassociationwillbecorrelatedtoeachother.Inthecaseofdependentcostcoecients,Proposition 4.1 cannotbedirectlyappliedbecausethesmallestcostcoecientswillnotberandomlydistributedacrosseachleveloftheindextree.However,usingProposition 4.1 asastartingpoint,considerselectingsomemultiple,>1,ofcostcoecientsfromeachlevelofasortedindextree.Forexample,selecttherstcostcoecientsfromeachofthesortedlevelsoftheindextreetoformasmallerindextreeA.Asisincreased,thecardinalityofAobviouslyincreasesbutsodoestheopportunitythatafeasiblesolutionexistsinA.ThebestvalueofdependsupontheparticularMAPinstance,butwecanempiricallydetermineasuitableestimate.Inthischapter,weuseaconsistentvalueof=10wherevertheprobabilisticapproachisused. 36 37 38 4 ]isamulti-startoriterativeprocessinwhicheachGRASPiterationconsistsoftwophases.Inaconstructionphase,arandomadaptiveruleisusedtobuildafeasiblesolutiononecomponentatatime.Inalocalsearchphase,alocaloptimumintheneighborhoodoftheconstructedsolutionissought.Thebestoverallsolutioniskeptastheresult. 4 ],asmallerNwilldirectlyreducethetimeittakesforeachconstructionphase.AsitiseasytoseethatreducingtheproblemsizetosomethinglessthanNhelpsintheconstructionphase,itremainstobeseenhowthelocalsearchphaseiseected. ThelocalsearchphaseofGRASPfortheMAPoftenreliesonthe2-exchangeneighborhood[ 74 4 ].AthoroughexaminationofotherneighborhoodsfortheMAP

PAGE 78

isprovidedintheworkbyPasiliao[ 84 ].Thelocalsearchprocedureisasfollows.Startfromacurrentfeasiblesolution,examineoneneighboratatime.Ifalowercostisfoundadopttheneighborasthecurrentsolutionandstartthelocalsearchprocedureagain.Continuetheprocessuntilnobettersolutionisfound.Thesizeofthe2-exchangeneighborhoodisdn12.AsthesizeoftheneighborhoodisnotdirectlydependentuponNthereappears,atrst,tobenoadvantageordisadvantageofreducingthenumberofcostcoecientsintheproblem.However,anobstaclesurfacesinthelocalsearchprocedurebecause,astheconstructionphaseproducesafeasiblesolution,wehavenoguaranteeaneighborofthissolutionevenexistsinthesparseproblem.Afeasiblesolutionconsistsofn1costcoecients.Aneighborinthe2-exchangeneighborhoodhasthesamen1costcoecientsexceptfortwo.InasparseMAP,mostcostcoecientsaretotallyremovedfromtheproblem.Therefore,thelocalsearchphaserstgeneratesapotentialneighborandthenmustdeterminewhethertheneighborexists.InacompleteMAP,theproceduremayaccessthecostmatrixdirectly;however,thesparseproblemcannotbeaccesseddirectlyinthesameway.Asimpleprocedureistosimplyscanallcostcoecientsinthesparseproblemtondthetwonewcostcoecientsortodeterminethatonedoesnotexist.Thisisanexpensiveprocedure.Weproposeadatastructurewhichprovidesaconvenient,inexpensivewayofevaluatingexistingcostcoecientsordeterminingthattheydonotexist.

PAGE 79

toconstructthetreeisO(N).AnexampleofthissearchtreeisgiveninFigure 5{3 foracomplete3x3x3MAP.Whensearchingforaparticularcostcoecient,startatleveli=1andtraversedownbranchy;y=0;:::;niwhereyistheelementoftheithdimensionforthecostcoecient.Continuethisprocessuntileitherleveli=d+1isreached,inwhichcasethecostcoecientexists,oranullpointerisreached,inwhichcasewemayconcludethecostcoecientdoesnotexist.ItisobviousthesearchtimeisO(d). Figure5{3: Searchtreedatastructureusedtondacostcoecientordetermineacostcoecientdoesnotexist. AsearchtreebuiltfromsparsedataisshowninFigure 5{4 .Asanexampleofsearchingforcostcoecient(001),startatlevel1andtraversedownbranchlabelled\0"tothenodeatlevel2.Fromlevel2,traverseagaindownbranchlabelled\0"tothenodeatlevel3.Fromlevel3,traversedownbranchlabelled\1"tothecostcoecient.Anotherexampleissearchingforcostcoecient(222).Startatlevel1andtraversedownbranchlabelled\2"tothenodeatlevel2.Fromlevel2,traverseagaindownbranchlabelled\2"tonditisanullpointer.ThenullpointerindicatesthecostcoecientdoesnotexistinthesparseMAP.

PAGE 80

Figure5{4: SearchtreeexampleofasparseMAP. TheGRASPalgorithmcanbenetfromthissearchtreedatastructureiftheproblemissparse.Inadenseproblem,itwouldbebesttoputcostcoecientsinamatrixwhichcanbedirectlyaccessed{thiswouldbenetthelocalsearchphase.However,inthesparseproblem,completelyeliminatingcostcoecientsreducesstor-ageandbenetstheconstructionphase.Itremainsamatterofexperimentationandcloserexaminationtondthelevelofsparsenesswherethesearchtreedatastructurebecomesmorebenecial. 74 ].Theproblemsrangedinsizefromvetosevensensors.Thosewithvesensorshadvetoninetargets.Problemswithsixandsevensensorshadjustvetargets.Twoproblemsofeachsizeweretested.Theproblemsizeisindicatedbytheproblemtitle.Forexample,\s5t6rm1"meansproblemonewithvesensorsandsixtargets.Theexperimentconductedverunsofeachsolutionalgorithmandreportstheaveragetime-to-solution,theaveragesolutionvalueandthebestsolutionvaluefromtheveruns.Thesolution

PAGE 81

timescanbeeasilyadjustedforeachalgorithmbysimplyadjustingthenumberofiterations.Anobviousconsequenceisthatasthenumberofiterationsgoesdown,thesolutionqualitygenerallygetsworse.Tocreatesparseinstancesofeachproblem,theprobabilisticapproachdescribedaboveinSection 5.3 wasusedwhere=10.Table 5{2 showstheresultsoftheexperiment.Exceptforproblemss5t8rm1ands5t8rm2,reducingtheproblemsizeincreasedsolutionqualitywithlesstime-to-solution. Table5{2: Tableofexperimentalresultsofcomparingsolutionqualityandtime-to-solutionforGRASPinsolvingfullydenseandreducedsimulatedMSMTTproblems.Fiverunsofeachalgorithmwereconductedagainsteachproblem. SparseGrasp ProblemOptSolAveSolBestSolAveTime(sec)AveSolBestSolAveTime(sec) s5t5rm1-50-49.2-500.026-50-500.022s5t5rm2-44-38-410.024-43.8-440.024s5t6rm1-57-54-51.40.044-49.4-520.044s5t6rm2-45-38.6-410.0462-45-450.04s5t7rm1-63-52.6-590.0902-61.2-620.0962s5t7rm2-66-59.2-620.0862-61.8-620.0822s5t8rm1-74-64.8-670.1322-71.2-720.1262s5t8rm2-33-20.6-320.1402-17-250.1542s5t9rm1-84-74.6-781.7044-74.4-771.8326s5t9rm2-65-59-611.6664-60.6-631.5702s6t5rm1-48-44.4-480.9676-48-480.9194s6t5rm2-45-42-420.9754-45-450.8392s7t5rm1-51-41.6-441.378-50.4-511.0556s7t5rm2-52-44.8-471.4804-52-521.0916

PAGE 82

Asdiscussedinpreviouschapters,theMAPisanNP-hardcombinatorialop-timizationproblemoccurringinmanyapplications,suchasdataassociation.Asmanysolutionapproachestothisproblemrely,atleastpartly,onlocalneighborhoodsearches,itiswidelyassumedthenumberoflocalminimahasimplicationsonsolutiondiculty.Inthischapter,weinvestigatetheexpectednumberoflocalminimaforrandominstancesofthisproblem.Both2-exchangeand3-exchangeneighborhoodsareconsidered.Wereportonexperimentalndingsthatexpectednumberoflocalminimadoesimpacteectivenessofthreedierentsolutionalgorithmsthatrelyonlocalneighborhoodsearches. 4 74 27 ].Althoughthe2-exchangeismostcommonintheliterature,weincludeinthischaptersomeanalysisofthe3-exchangeneighborhoodforcomparisonpurposes. ThemotivationofthischapteristhatthenumberofdistinctlocalminimaofanMAPmayhaveimplicationsforheuristicsthatrely,atleastpartly,onrepeatedlocalsearchesinneighborhoodsoffeasiblesolutions[ 112 ].Ingeneral,ifthenumberoflocalminimaissmallthenwemayexpectbetterperformancefrommeta-heuristicalgo-rithmsthatrelyonlocalneighborhoodsearches.Asolutionlandscapeisconsideredtoberuggedifthenumberoflocalminimaisexponentialwithrespecttothesizeoftheproblem[ 78 ].EvidencebyAngelandZissimopoulos[ 9 ]showedthatruggedness 69

PAGE 83

ofthesolutionlandscapehasadirectimpactontheeectivenessofthesimulatedan-nealingheuristicinsolvingatleastoneotherhardproblem,thequadraticassignmentproblem. TheconceptofsolutionlandscapeswasrstintroducedbyWright[ 111 ]asanon-mathematicalwaytodescribetheactionduringevolutionofselectionandvariation[ 102 ].Theideaistoimaginethespaceinwhichevolutionoccursasalandscape.Inonedimensionthereisthegenotypeandinanotherdimensionthereisaheightortness.Evolutioncanbeviewedasthemovementofthepopulation,representedasasetofpoints(genotypes),towardshigher(tter)areasofthelandscape.Inananalogousway,asolutionprocessforacombinatorialproblemcanbeviewedasthemovementfromsomefeasiblesolutionwithitsassociatedcost(tness)towardsbettercost(tter)areaswithinthesolutionlandscape.AspointedoutbySmithetal.[ 102 ],thedicultyofsearchinginagivenproblemisrelatedtothestructureofthelandscape,however,theexactrelationshipbetweendierentlandscapefeaturesandthetimetakentondgoodsolutionsisnotclear.Tonameacoupleofthelandscapefeaturesofinterestarenumberlocaloptimaandbasinsofattraction. ReidysandStadler[ 93 ]describesomecharacteristicsoflandscapesandexpressthatlocaloptimaplayanimportantrolesincetheymightbeobstaclesonthewaytotheoptimalsolution.Fromaminimizationperspective,if^xisafeasiblesolutionofsomeoptimizationproblemandf(^x)isthesolutioncost,then^xisalocalmin-imaif(^x)f(^y)forall^yintheneighborhoodof^x.Obviouslythesizeoftheneighborhooddependsuponthedenitionoftheneighborhood.AccordingtoReidysandStadler[ 93 ]thereisnosimplewayofcomputingthenumberoflocalminimawithoutexhaustivelygeneratingthesolutionlandscape.However,thenumbercanbeestimatedasdoneinsomerecentworks[ 43 45 ]. Rummukainen[ 98 ]describessomeaspectsoflandscapetheorywhichhavebeenusedtoproveconvergenceofsimulatedannealing.Ofparticularinterestaresome

PAGE 84

resultsonthebehavioroflocaloptimizationonafewdierentrandomlandscapeclasses.Forexample,theexpectednumberoflocalminimaisgivenfortheNklandscape. AssociatedwithlocalminimaisabasinB(^x)denedbymeansofasteepestdescentalgorithm[ 93 ].Letf(xi)bethecostofsomefeasiblesolutionxi.Startingfromxi;i=0,recordforallyneighborsthesolutioncostf(y).Letxi+1=yforneighborywheref(y)isthesmallestforallneighborsandf(y)
PAGE 85

themoreatisthelandscape{andso,aspostulatedbytheauthors,themoresuitedistheproblemforanylocal-search-basedheuristic.AngelandZissimopoulos[ 9 ]calculatetheautocorrelationcoecientfortheQAPasbeingnosmallerthann=4andnolargerthann=2whichisconsideredrelativelylarge.Theydeveloptheparameter,ruggednesscoecient,,whichisindependentofproblemsizeandliesbetween0to100.Closeto100meansthethelandscapeisverysteep.TheygoontoshowexperimentallythatincreasingforthesameproblemsizeresultsinhigherrelativesolutionerrorandhighernumberofstepswhenusingasimulatedannealingalgorithmbyJohnsonetal.[ 53 ].TheconclusionsAngelandZissimopoulos[ 9 ]arearelativelylowruggednesscoecientfortheQAPgivestheoreticaljusticationoftheeectivenessoflocal-search-basedheuristicsfortheQAP. ThischapterwillfurtherinvestigatetheassumptionthatnumberoflocalminimaimpactstheeectivenessofalgorithmssuchassimulatedannealinginsolvingtheMAP. Thenextsectionprovidessomeadditionalbackgroundonthe2-exchangeand3-exchangelocalsearchneighborhoods.TheninSection 6.3 ,weprovideexperimen-talresultsontheaveragenumberoflocalminimaforvariouslysizedproblemswithassignmentcostsindependentlydrawnfromdierentdistributions.Section 6.4 de-scribestheexpectednumberoflocalminimaforMAPsofsizeofn=2andd3wherethecostelementsareindependentidenticallydistributedrandomvariablesfromanyprobabilitydistribution.Section 6.5 describeslowerandupperboundsfortheexpectednumberoflocalminimaforallsizesofMAPswhereassignmentcostsareindependentstandardnormalrandomvariables.TheninSection 6.6 ,weinvestigatewhethertheexpectednumberoflocalminimaimpactstheperformanceofvariousalgorithmsthatrelyonlocalsearches.Someconcludingremarksaregiveninthelastsection.

PAGE 86

66 ].Ifzkisthesolutioncostofthek-thsolution,thenzkisadiscretelocalminimumizkzjforallj2Np(k).Asanexampleofa2-exchangeneighbor,considerthefollowingfeasiblesolutiontoanMAPwithd=3;n=3:f111;222;333g.Aneighborisf111;322;233g.Thesolutionf111;222;333gisalocalminimumifitssolutioncostislessthanorequaltoallofitsneighbor'ssolutioncosts. Theformulaforthenumberofneighbors,J,ofafeasiblesolutioninthe2-exchangeneighborhoodofanMAPwithdimensiondandnelementsineachdimen-sionisasfollows Itisobviousthatforaxedn,Jislinearind.Ontheotherhandforaxedd,Jisquadraticinn.IfwedeneaatMAPasonewithrelativelysmallnanddeneadeepMAPasonewithrelativelylargen,thenweexpectlargerneighborhoodsindeepproblems. Similarly,forn>2thesizeofthe3-exchangeneighborhoodisasfollows Similartoaboveforthe2-exchange,itbecomesclearJislinearwithrespecttodandcubicwithrespectton. Theminimumnumberoflocalminimaforanyinstanceisone-theglobalmini-mum.Attheotherextreme,themaximumnumberoflocalminimais(n!)d1which

PAGE 87

isthenumberoffeasiblesolutionsofanMAP.Thisoccursifallcostcoecientsareequal. 63 ]. Table 6{1 showstheaveragenumberoflocalminimaforrandomlygeneratedinstancesoftheMAPwhenconsideringa2-exchangeneighborhood.Forsmallsizedproblems,thestudywasconductedbygeneratinganinstanceofanMAPandcount-ingnumberoflocalminimathroughcompleteenumerationofthefeasiblesolutions.Thevaluesinthetablesaretheaveragenumberoflocalminimafrom100probleminstances.Forlargerproblems(indicatedby*inthetable),theaveragenumberoflocalminimawasfoundbyexaminingalargenumber

PAGE 88

Table 6{2 showssimilarresultsforthe3-exchangeneighborhoodandwhencostcoecientsarei.i.d.standardnormal.Wenote,asexpected,evidenceindicatesE[M]issmallerinthe3-exchangecaseversusthe2-exchangecaseforthesamesizedproblems. Table6{1: Averagenumberoflocalminima(2-exchangeneighborhood)fordierentsizesofMAPswithindependentassignmentcosts. 2 NumberofLocalMinima,Exponential=1 2 NumberofLocalMinima,StandardNormalCosts 2 6{3 showstheaverageproportionoffeasiblesolutionsthatarelocalminimaforboththe2-exchangeand3-exchangeneighborhoodswherecostsarei.i.d.standardnormalrandomvariables.ThetableisfollowedbyFigure 6{1 whichincludesplotsoftheproportionoflocalminimatonumberoffeasiblesolutions.Weobservethatfor

PAGE 89

Table6{2: Averagenumberoflocalminima(3-exchangeneighborhood)fordierentsizesofMAPswithi.i.d.standardnormalassignmentcosts. 3 Table6{3: Proportionoflocalminimatototalnumberoffeasiblesolutionsfordif-ferentsizesofMAPswithi.i.d.standardnormalcosts. 2 Proportionoflocalminimatofeasiblesolutionsusingstandardnormalcostsand3-exchangennd 3 Proportionoffeasiblesolutionsthatarelocalminimawhenconsideringthe2-exchangeneighborhoodandwherecostsarei.i.d.standardnormal.

PAGE 90

InthespecialcaseofanMAPwheren=2,d3,andcostelementsareinde-pendentidenticallydistributedrandomvariablesfromsomecontinuousdistributionwithc.d.fF(),onecanestablishaclosedformexpressionfortheexpectednum-beroflocalminima.Toshowthis,werecallthatdistributionFX+YofthesumoftwoindependentrandomvariablesXandYisdeterminedbytheconvolutionoftherespectivedistributionfunctions,FX+Y=FXFY. WenowborrowfromProposition 3.1 toconstructthefollowingproposition. Oneotherpropositionisimportanttothisdevelopment.

PAGE 91

whosec.d.f.andp.d.f.arecomputedtriviallyasF(j1)(x)=P[X(j1)x]=1(1F(x))j1;f(j1)(x)=d dxF(j1)(x)=(j1)(1F(x))j2f(x): HereFY()isthec.d.f.ofrandomvariableY=XrX(j1),and,byconvolutionrule,itequalstoFY(x)=Z+1F(xy)(j1)(1F(y))j2f(y)dy: 6.3 )canimmediatelybecalculatedasP[XrX(j1)]=Z+1F(y)(j1)(1F(y))j2f(y)dy=1 TheobviousconsequenceofProposition 6.2 isthatgivenasequenceofindepen-dentrandomvariablesfromacontinuousdistribution,positionoftheminimumvalueisuniformlylocatedwithinthesequenceregardlessoftheparentdistribution. Wearenowreadytoprovethemainresultofthissection.

PAGE 92

(6.5) onecanwriteMasthesumofindicatorvariables:M=NXk=1Yk: whereP[Yk=1]istheprobabilitythatthecostofk-thfeasiblesolutiondoesnotexceedthecostofanyofitsneighbors.AnyfeasiblesolutionhasJ=dn2=dneighborswhosecosts,byvirtueofProposition 6.1 ,arei.i.d.continuousrandomvariables.Then,Proposition 6.2 impliesthattheprobabilityofthecostofk-thfeasiblesolutionbeingminimalamongitsneighborsisequaltoP[Yk=1]=1 6.6 ),yieldsthestatementofthetheorem( 6.4 ). 6.4 )impliesthatthenumberoflocalminimainann=2;d3MAPisexponentialindwhenthecostcoecientsareindependentlydrawnfromanycontinuousdistribution. 6.4 )canbeusedtoderivetheexpectedratioofthenumberoflocalminimaMtothetotalnumberoffeasiblesolutionsNinann=2;d3MAP:E[M=N]=E[M]

PAGE 93

Thisshowsthatthenumberoflocalminimainann=2MAPbecomesinnitelysmallcomparingtothenumberoffeasiblesolutions,whendimensiondoftheproblemincreases.Thisasymptoticcharacteristicisreectedinthenumericaldataaboveandmaybeusefulforthedevelopmentofnovelsolutionmethods. Ourabilitytoderiveaclosed-formexpression( 6.4 )fortheexpectednumberoflocalminimaE[M]intheprevioussectionhasreliedontheindependenceofcostsoffeasiblesolutionsinann=2MAP.Asitiseasytoverifydirectly,incaseofn3thecostscoecientsaregenerallynotindependent.Thiscomplicatessignicantlytheanalysisifanarbitrarycontinuousdistributionforassignmentcostsci1idisassumed.However,asweshowbelow,onecanderiveupperandlowerboundsforE[M]inthecasewhenthecostscoecientsof( 2.1 )arenormallydistributedrandomvariables. First,weintroduceaproposition,whichfollowsasimilardevelopmentbyBeck[ 16 ]. 6.5 ),whichconsequentlyleadsto AsYk=1meanszkzjforallj2N2(k),itisobviousthatP[Yk=1]=P[zkzj0;8j2N2(k)],whichprovestheproposition.

PAGE 94

Ifweallowthendcostcoecientsci1idN(0;1)oftheMAPtobeindependentstandardnormalN(0;1)randomvariables,thenforanytwofeasiblesolutionsthedierenceoftheircostsZij=zizjisanormalvariablewithmeanzero. Withoutlossofgenerality,considerthek=1feasiblesolutionto( 2.1 )whosecostisz1=c11+c22+:::+cnn: Inthe2-exchangeneighborhoodN2(1),thecostofafeasiblesolutiondiersfrom( 6.9 )bytwocostcoecients,e.g.,z2=c211+c122+c33+:::+cnn: 6.9 )andthatofanyneighborl2N2(1)hastheformZrsq=crr+csscrrsrrcssrss; wherethelasttwocoecientshave\switched"indicesintheq-thposition,q=1;:::;d.ObservingthatZrsq=Zsrqforr;s=1;:::;n;q=1;:::;d;

PAGE 95

VectorZhasnormaldistributionN(0;),withcovariancematrixdenedasCov(Zrsq;Zijk)=8>>>>>>><>>>>>>>:4;ifi=r;j=s;q=k;2;ifi=r;j=s;q6=k;1;if(i=r;j6=s)or(i6=r;j=s);0;ifi6=r;j6=s: Forexample,incasen=3;d=3thecovariancematrixhastheform 6.7 )canbeexpressedasPh\j2N2(k)zkzj0i=F(0); whereFisthec.d.f.oftheJ-dimensionalmultivariatenormaldistributionN(0;).WhilethevalueofF(0)in( 6.13 )isdiculttocomputeexactlyforlargedandn,lowerandupperboundscanbeconstructedusingSlepianinequality[ 107 ].Tothis

PAGE 96

end,letusintroducecovariancematrices (6.14a) (6.14b) sothat 6.12 ).Then,theSlepianinequalityclaimsthatF whereF ),respectively. AsthevariableX 6.15 )canbeexpressedbytheone-dimensionalintegral(see,e.g.,[ 107 ])Z+1(az)Jd(z);a=r where()isthec.d.f.ofstandardnormaldistribution:(z)=Zz1 2dt; 2,whichallowsforasimpleclosed-formexpressionfortheupperboundin( 6.15 )F

PAGE 97

Thisimmediatelyyieldsthevalueoftheupperbound 6.15 ).Accordingtothecovariancematrix 6.14a ),thevectorX 2groupsofvariables,eachconsistingfromdelements,X 6.14b ).Thus,onecanexpressthelowerboundF 6.15 )asaproductF 2Yi=1PX(i1)d+10;:::;Xid0: 2,eachprobabilityterminthelastequalitycanbecomputedsimilarlytoevaluationofthelower-boundprobability( 6.17 ),i.e.,F 2Yi=1Z+1(z)dd(z)=1 2: =NXk=1F

PAGE 98

6.18 )itfollowsthatforxedn3,theexpectednumberoflocalminimaisexponentialind. 6.18 )canbeusedtoestimatetheexpectedratioofthenumberoflocalminimaMtothetotalnumberoffeasiblesolutionsNinann3;d3MAP:1 (d+1)n(n1)=2E[M=N]2 Theheuristicsdescribedinthefollowingthreesubsectionswereexercisedagainstvarioussizedproblemsthatwererandomlygeneratedfromthestandardnormaldis-tribution.

PAGE 99

capturestheoverallbestsolutionandreportsitafterexecutingthemaximumnumberofiterations.Thefollowingisamoredetaileddescriptionofthestepsinvolved. 1. Setiterationnumbertozero,Iter=0. 2. Randomlyselectacurrentsolution,xcurrent. 3. Whilenotallneighborsofxcurrentexamined,selectaneighbor,xnew,ofthecurrentsolution. Ifzxnew
PAGE 100

forsimulatedannealingusedinthischapteraretakenfromworkbyGosavi[ 46 ].SimulatedannealingwasrecentlyappliedtotheMAPbyClemmonsetal.[ 27 ]. 1. GeneraterandomMAPinstancewithcostcoecientsthatarei.i.d.standardnormalrandomvariables. 2. ObtainMbycheckingeachfeasiblesolutionforbeingalocalminimum. 3. SolvethegeneratedMAPinstanceusingeachoftheaboveheuristics100timesandreturntheaveragesolutionquality, ProblemsizeswerechosenbasedonthedesiretotestavarietyofsizesandthepracticalamountoftimetodetermineM(ascountingMistheobviousbottleneckintheexperiment).Fourproblemsizeschosenwered=3;n=6;d=4;n=5;d=4;n=6;andd=6;n=4.Forproblemsized=4;n=6whichhasthelargestN,asingleruntookapproximatelyfourhoursona2.2GHzPentium4machine.Thenumberofrunsofeachexperimentvariedforeachproblemsizewithfewerrunsforlargerproblems.Thenumberofrunswere100,100,30,and50,respectively,fortheproblemsizeslistedabove. Figure 6{2 displaysplotsoftheaveragesolutionqualityforeachofthethreeheuristicsversusthenumberoflocalminima.Theplotsaretheresultsfromproblemsized=4;n=5andaretypicalfortheotherproblemsizes.Includedineachplotisabest-tlinearleast-squareslinethatprovidessomeinsightontheeectofMonsolutionquality.Acloseexaminationoftheguresindicatesthatthesolution

PAGE 101

qualityimproveswithsmallerMforeachheuristicsolutionmethod.ThisconclusionwasveriedusingaregressionanalysistodeterminethattheeectofMonaveragesolutionqualityisstatisticallysignicant(p-valueaveragedapproximately0.01). Wehavealsoinvestigatedthe3-exchangeneighborhood.Figure 6{3 displaysplotsoftheaveragesolutionqualityofthethreeheuristicsversusthenumberoflocalminimawhenusingthelargerneighborhood.TheparametersineachheuristicsuchasnumberofiterationsforrandomlocalsearchandGRASPwerekeptthesameforeachheuristic.Theonlychangemadewastheneighborhooddenition.TheplotsforrandomlocalsearchandGRASPindicatethatMaectssolutionquality(regressionanalysisshowsaveragep-valueof0.05).However,theeectofMisnotstatisticallysignicantinthecaseofsimulatedannealingwhenusingthe3-exchangeneighborhood(p-valueof0.4). WenoteacoupleinterestingaspectswhencomparingFigures 6{2 and 6{3 .ThesolutionqualitywhenusingrandomlocalsearchorGRASPimproveswhenusingthelargerneighborhood.Thisistobeexpected,butattheexpenseoflongersolu-tiontimes.Wefoundonaveragetherandomlocalsearchtookapproximately30%longerandGRASPtookabout15%longertocompletethesamenumberofitera-tions.Simulatedannealing'sperformanceintermsofsolutionqualitydroppedwhentheneighborhoodsizewasincreased.Thisisnotsurprisingastheoptimalstartingtemperatureandcoolingratearefunctionsoftheproblem-instancecharacteristicssuchassize,neighborhooddenition,costcoecientvalues,etc.Ourexperimentalresultsforsimulatedannealingreiteratethenecessityforproperlytuningheuristicparameterswhentheneighborhooddenitionischanged.

PAGE 102

Figure6{2: Plotsofsolutionqualityversusnumberoflocalminimawhenusingthe2-exchangeneighborhood.TheMAPhasasizeofd=4;n=5withcostcoecientsthatarei.i.d.standardnormal.

PAGE 103

Figure6{3: Plotsofsolutionqualityversusnumberoflocalminimawhenusinga3-exchangeneighborhood.TheMAPhasasizeofd=4;n=5withcostcoecientsthatarei.i.d.standardnormal.

PAGE 104

whenn>2.WeprovedaclosedformexpressionforE[M]whenn=2;d3.Theexpressionholdsforanycasewhenthecostcoecientsarei.i.d.fromacontinuousdistribution.UpperandlowerboundstoE[M]aregiveninthecaseofn>2;d3andcostcoecientsarei.i.d.standardnormalvariables.TheboundsshowthatE[M]isexponentialwithrespecttod.Finally,experimentalresultsareprovidedthatshowatleastthreeheuristicsthatrely,atleastpartly,onlocalsearchareadverselyimpactedbyincreasingnumberoflocalminima.

PAGE 105

Testproblemsareoftenusedincomputationalstudiestoestablishtheeciencyofsolutionmethods,or,aspointedoutbyYongandPardalos[ 112 ],testproblemsareimportantforcomparingnewsolutionmethodsagainstexistingalgorithms.Alongwithacollectionofcombinatorialtestproblems,thebookbyFloudasetal.[ 39 ]emphasizestheimportanceofwelldesignedproblems.AsBarretal.[ 12 ]pointout,thereisadeniteneedforavarietyoftestproblemstochecktherobustnessandaccuracyofproposedalgorithms.Aprobabilisticapproachinthedevelopmentandstudyoftestproblemsmayresultinhigherqualitytestinstances. InthischapterwedevelopatestproblemgeneratorfortheMAPanduseaprobabilisticanalysistodetermineitseectivenessingeneratinghardproblems. 1. Realworldproblems(e.g.[ 95 ]). 2. Librariesofstandardtestproblems[ 20 59 95 ]. 3. Randomlygeneratedproblemssuchasthosewithcostcoecientsdrawninde-pendentlyfromsomedistributionsuchasuniformon[0,1]. 4. Problemsgeneratedfromanalgorithmsuchasthequadratictestproblemgen-erator[ 112 ]. AsnotedbyReilly[ 94 ],realworldproblemshavetheadvantageofprovidingresultsconsistentwithatleastsomeproblemsencounteredinpractice.However,inmostcasesthereisnotasizeablesetofrealworldproblemstoconstituteasat-isfactoryexperiment.Librariesofstandardtestproblemsserveasagoodsourceofproblems;however,againtheremaynotbeenoughoftheright-sizedproblems. 92

PAGE 106

Randomlygeneratedproblemsprovidevirtuallyaninnitesupplyoftestproblems;however,theoptimalsolutiontolargeproblemsmayremainunknown.Anadditionalhazardwithrandomlygeneratedproblemsistheyareoften\easy"tosolve[ 21 101 ].Thesemaybesignicantissueswhenevaluatingtheperformanceofanewalgorithm.Generatedtestproblemswithknownsolutionscanalsobeinvirtuallyinnitesupplyand,importantly,auniqueknownsolutioncanbeveryusefulinfullyevaluatingasolutionalgorithm'sperformance.Carefulstudyofgeneratedproblemsisnecessarytodeterminetherelativeusefulnessoftheproblemsintermsofdiculty,realism,etc.Sanchis[ 99 ]mentionsgeneratedproblemsshouldhavethefollowingproperties Sanchisgoesontosaythatmeetingallthreerequirementscanbedicult.Forex-ample,meetingtherstrequirementcanbequitesimplebycreatingatrivialinstance;however,atrivialinstancewouldmostlikelyviolatethesecondproperty.Also,itisrecognizedbymanyresearchersthatthereisaneedforstandardizedrepresentationsofprobleminstances[ 40 ].ApopulartechniqueindesigningnonlinearprogrammingtestproblemsistheuseofKarush-Kuhn-TuckeroptimalityconditionsasproposedbyRosenandSuzuki[ 97 ].Testproblemgeneratorsforinteger-programmingproblemsarediculttoconstructandrequireadeepinsightintotheproblemstructure[ 56 ]. AninterestingapproachtotestproblemgenerationisthatoftheDiscreteMath-ematicsandTheoreticalComputerScience(DIMACS)challenges[ 34 ].Overthepastdecade,thechallengeshavehadthegoalofencouragingtheexperimentalevaluationofalgorithms.Itisrecognizedthatcomparisonsmustbemadeonstandardtestprob-lemsthatareincludedaspartofthechallenges.ChallengeshavebeenheldforTSP,cliques,coloring,andsatisability.

PAGE 107

Tosummarizesourcesoftestproblems,thefollowingtwosectionsdescribeavail-abletestproblemgeneratorsandlibrariesthatincludeready-madetestproblems. Steinerproblemingraphs. 56 ]presentabinary-programmingformulationfortheSteinerproblemingraphswhichiswellknowntobeNP-hard.Theyusethisformulationtogeneratetestproblemswithknownoptimalsolutions.ThetechniqueusestheKKToptimalityconditionsonthecorrespondingquadrati-callyconstrainedoptimizationproblem. 48 ],theauthorsconsiderseveralinterestingproblems.Theyintroducedierenttestproblemgeneratorsthatarisefromavarietyofpracticalapplicationsaswellastheproblemofmaximumclique.Applicationsincludecodingtheoryproblems,problemsfromKeller'sConjecture,andproblemsinfaultdiagnosis.WorkbySanchisandJagota[ 100 ]discussesatestproblemgeneratorthatbuildsthecomplementaryminimumvertexcoverproblem.Thehardnessoftheirgeneratedproblemsreliesonconstructionparameters.Sanchis[ 99 ]providesanalgorithmtogenerateminimumvertexcoverproblemthatisdiverse,hardandofknownsolution. 79 ]providedamethodforconstructingtestproblemsforconstrainedbivalentquadraticprogramming.Alsoprovidedisastandardizedrandomtestproblemgeneratorfortheunconstrainedquadraticzero-oneprogrammingproblem.YongandPardalos[ 112 ]provideeasymethodstogeneratetestproblemswithknownoptimalsolutionsforgen-eraltypesofQAPs.Acodeisavailableat 24 ]describeatechniqueforgeneratingconvex,strictlyconcaveandindenite(bilinearornot)quadraticprogrammingproblems.Theirapproach

PAGE 108

involvescombiningmtwo-variableproblemstoconstructaseparablequadraticprob-lem.Palubeckis[ 77 ]providesamethodforgeneratinghardrectilinearQAPswithknownoptimalsolutions. 99 ]. 99 ]. 2 ]proposeageneratorthatonlyoutputssatisableprobleminstances.Theyshowhowtonelycontrolthehardnessoftheinstancesbyestablishingaconnectionbetweenproblemhardnessandakindofphasetransitionphenomenoninthespaceofprobleminstances.Uchidaetal.[ 108 ]provideawebpagededicatedtotwomethodsofgeneratinginstancesofthe3-satisabilityproblem. 73 ].ThesiteprovidesinformationconcerningresearchingenerationofinstancesofTSPswithknownoptimalsolution.AnapproachtogeneratingdiscreteproblemswithknownoptimabasedonapartialdescriptionofthesolutionpolytopeisprovidedbyPilcherandRardin[ 88 ].Theapproachisusedtogenerateinstancesofthesymmetrictravelingsalesmanproblem. 61 ]providesanappropriatetestproblemgenerator.

PAGE 109

HandbookoftestProblems. 39 ]presentacollectionoftestproblemsarisinginliteraturestudiesandawidespectrumofapplications.Applica-tionsinclude:pooling/blendingoperations,heatexchangernetworksynthesis,phaseandchemicalreactornetworksynthesis,parameterestimationanddatareconcilia-tion,clustersofatomsandmolecules,pumpnetworksynthesis,trimlossminimiza-tion,homogeneousazeotropicseparation,dynamicoptimizationandoptimalcontrolproblems. 15 ]isanextensivecollectionoftestinstances. 20 ]alsoprovidesotherusefulinformationconcerningthisdicultproblem. 50 ].Ithasacollectionofbenchmarkproblems,solvers,andtoolsrelatedtosatisabilityresearch. 49 ]. 96 ]containsseveraltypesofTSPinstances. 95 ].

PAGE 110

59 ].Typesofproblemsfoundthereareassignment,min-cutclustering,linearprogram-ming,integerandmixed-integerprogramming,matrixdecomposition,matching,max-imumowindirectedandundirectedgraphs,minimumcostnetworkowandtrans-portation,setpartitioning,Steinertree,travelingsalesman,andcapacitatedvehiclerouting. 58 ]thatcol-lectsavailableinstancesofSteinertreeproblemsingraphsandprovidessomeinfor-mationabouttheirorigins,solvabilityandcharacteristics. 35 ].Alongwithtestproblems,thelibraryprovidesavastbibliographyofworkconcerningthisimportantclassofproblem. Assumingavarietyofhardanddiverseproblemsexist,carefuluseoftheprob-lemsisofcoursenecessary.Anicetreatmentconcerningexperimenting/reportingonsolutionalgorithmperformanceisprovidedbyBarretal.[ 12 ].Theauthorsdescribeconditionsofdesigningcomputationalexperimentstocarefullyexamineheuristics.Theyalso,givereportingguidelines. SeveralpreviousworksonsolutionmethodsfortheMAPusedtestproblemsthatweregeneratedusingsomerandommethod.Thesetestproblemsmaybeclassiedintothreecategories.TherstcategoryofproblemsasusedbyBalasandSaltzman[ 10 ]andPierskalla[ 87 ]drawsintegercostsfromauniformdistribution.AsecondcategoryoftestproblemsasbyFrieze[ 42 ]usedcostcoecientscijk=aij+aik+ajkwhereaij,aik,andajkarerandomlygeneratedintegersfromauniformdistribution.ThelastcategoryoftestproblemsincludesproblemsgeneratedforaspecicMAPapplicationsuchasformultitargetmultisensordataassociation[ 74 ]orforsomespecialcoststructuresuchasdecomposablecostsasexaminedbyBurkardetal.[ 23 ].

PAGE 111

87 ]basedonthedesiredsizeoftheproblem.Figure 7{1 showsatreegraphforathree-dimensionalproblemwheren1=3;n2=4andn3=4(or3x4x4asaconvenientnotationwhichmaybeextendedtootherdimensions). Figure7{1: Treegraphfor3x4x4MAP. Eachnodeofthetreerepresentsanassignmentxijkandhasanassignmentcostcijk.Ingeneral,therearen1levelsinthetreegraph,wheretherootisatlevel0.ThenumberofnodesonanylevelotherthantherootisQdi=2ni.Branchesinthetreegrapharefeasiblepathsfromatripleatoneleveltoatripleatthenextleveldown.Apathfromlevel0toleveln1isafeasiblesolutiontotheMAP.UsingFigure 7{1 asanexample,if121istheassignmentchosenfromthe16availableassignmentsatlevel

PAGE 112

1,thenthefeasibleassignmentsatthenextlevelare212,213,214,232,233,234,242,243,244.If232isthenchosenatlevel2,then313,314,343,344areavailableatlevel3.Usingthisprocedure,afeasiblepathis121,232,313. Inadditiontotheassignmentcost,cijk,associatedwitheachnode,thereisanothercostcalledLowerBoundPathCost,lbijk.Thisisthecostoftheassignmentplustheminimumlowerboundpathcostofanyfeasiblenode(assignment)atthenextleveldown.Alowerboundpathcostforanynodeprovidesalowerboundfortheadditionalcostofgoingthroughthatparticularnodeonapathtothelowestlevel.Thealgorithmidentiesafeasiblesolutionthatwillremaintheuniqueoptimalsolution.Oncethisoptimalsetofassignmentsisidentied,randomcostsareappliedtoeachoftheassignmentsintheoptimalset.Thenworkingfromleveln1tothehighestlevelofthetree,applyassignmentcosts,cijk,andlowerboundpathcosts,lbijk,toeachnode(exceptforthoseontheoptimalpaththatalreadyhavethesecostsassigned). ThefollowingareproceduralstepsforgeneratinganMAPofcontrollablesizewithaknownuniqueoptimalsolution. Step1:Basedonthedimensionandnumberofelementsforeachalternative,buildatreegraphofallpossibleassignmentssuchthatci1id=1;lbi1id=1;8i1;i2;:::;id: Step3:Applyrandomassignmentcoststhatareuniformbetweensomeloweranduppervaluestoeachnodeontheoptimalpathandupdatetheirlowerbound

PAGE 113

pathcostssuchthatlbn1id=cn1id;lbkid=ckid+minffeasiblelbk+1idg;8k=1;:::;n11: Step5:Applyrandomcoststhatareuniformbetweensomeloweranduppervaluesfortherestofnodesatleveln1. Step6:Workupthetreegraphfromleveln11.Foreachnodeatlevelk,considerallfeasiblenodesatlevelk+1.Setnode'scostsuchthatitslowerboundpathcostisatleastgreaterthanthelowerboundpathcostoftheoptimalpathnodeatlevelk.Thatis,setitslowerboundpathcostsuchthatlbkid=ckid+minffeasiblelbk+1idg>minflbkidg;8k=1;:::;n11: Step1:TreegraphisshowninFigure 7{2 Step2.Anoptimalsolutionsetof141,222,334ischosenbyrandomlychoosinganodeateachlevel. Step3.Integercostsuniformin[1,10]areappliedtotheoptimalsolutionsetsuchthatc141=2;c222=1;c334=4; 7{3

PAGE 114

Figure7{2: Initialtreegraphwithassignmentcostsandlowerboundpathcosts. Steps4and5.Integercostsuniformin[1,10]areappliedtonodesatlevel3suchthatthelowerboundpathcostoftheoptimalpathnodeatlevel2isnotreduced. Step6.Startingatlevel2,applyrandomcoststoeachnode(otherthannodesontheoptimalpath)suchthatitslowerboundpathcostisatleastgreaterthanthelowerboundpathcostoftheoptimalpathnodeatthesamelevel.ConsiderFigure 7{4 andthefollowingcalculation.lb211=c211+minflb322;lb323;lb324;lb332;lb333;lb334;lb342;lb343;lb344g>lb222;=c211+lb323>lb222;8=7+1>5:

PAGE 115

Figure7{3: Treegraphwithoptimalpathandcosts. Continuingstep6untilallassignmentcostsareidentiedresultsintheMAPshowninFigure 7{5

PAGE 116

Figure7{4: Treegraphusedtoconsiderallfeasiblenodesatlevel3fromtherstnodeinlevel2. nodeintheoptimalassignmentset.Sincenootherfeasiblepaththroughanodeatlevel1tolevel2canhaveacostlessthantheoptimallowerboundpathcost,opti-malityismaintained.Nowassumethealgorithmresultsinaknownuniqueoptimalsolutionforn1=k.Bystep6ofthealgorithm,theminimumlowerboundpathcostatanylevelismaintainedatthenodeintheoptimalassignmentset.Therefore,theadditionallevelwillstillmaintaintheminimumlowerboundpathcostatanodeintheoptimalassignmentsetandaknownuniqueoptimalsolutionresults.

PAGE 117

Figure7{5: Finaltreegraphfora3x4x4MAP. Usinga2.2GHz,Pentium4machine,theresultsoftimedexperimentalrunsproducingdierentsizedMAPtestproblemsusingtheproposedalgorithmarepro-videdinTable 7{1 .Theresultsshowthatlargeproblemswithknownsolutionscanbegeneratedinareasonableamountoftime.

PAGE 118

Table7{1: Timedresultsofproducingtestproblemsofvarioussizes. ProblemSizeTotalNumberofApproximateMachineRunNodesNumberofTime(sec)FeasibleSolutions 20x30x402:41042:451055250x50x501:251059:25101286760x70x702:941042:45101873096x7x8x9x92:71043:721017529x9x9x9x95:91041:73102217010x10x11x11x121:331052:311029809 usefulinexercisingsolutionmethods?Althoughthedenitionofproblemqualityissomewhatsubjective,weanalyzeseveralimportantqualitycharacteristicsinthefollowingparagraphs. 7{6 .Thisplotisthatofatypicalsetofassign-mentcostsfromageneratedtestproblemusingtheproposedalgorithm.Theplotindicatesthecostsmaybenormallydistributed.Usingchi-squaregoodness-of-t,ananalysisofsixrandomlyselected5x5x5testproblemsthatweregeneratedusingtheproposedalgorithmyieldedtheresultsshowninTable 7{2 .Usingap-valuestatisticof15-percent,weconcludeassignmentcostsarenormallydistributedforveofthesixcases.Thegoodness-of-ttestforrunfourdoesnotindicatetheassignmentcostsarenormallydistributed. ForlargerMAPtestproblems,Chi-squaregoodness-of-ttestsfailedtoconrmthattheassignmentcostsarenormallydistributed.However,asshowninFigure 7{7 ,atypicalhistogramofassignmentcostsfora20x30x40testproblemshowsthecostsappeartobewellbehaved.

PAGE 119

Figure7{6: Typicalnormalprobabilityplotfora5x5x5testproblem. Table7{2: Chi-squaregoodness-of-ttestfornormaldistributionofassignmentcostsforsixrandomlyselected5x5x5testproblems. Run AssignmentCosts Goodness-of-Fit Mean StandardDeviation TestValue,2 1 13.18 5.95 19.10 0.06 2 13.13 5.53 16.60 0.12 3 11.53 5.60 31.19 0.001 4 12.21 4.93 13.14 0.28 5 10.10 5.30 19.48 0.05 6 13.86 6.10 16.98 0.11 ThenumberoflocalminimaofanMAPhasimplicationsforheuristicsthatrely,atleastpartly,onrepeatedlocalsearchesinneighborhoodsoffeasiblesolutions[ 112 ].Inouranalysistheneighborhoodofafeasiblesolutionisdenedasall2-elementexchangepermutations[ 74 ].Usingthisdenition,thesizeoftheneighborhoodis

PAGE 120

Figure7{7: Typicalhistogramof20x30x40testproblem. 65 ]categoriesoftheMAPmaybecompared.Theproposedsam-plingapproachistorandomlyselectasampleoffeasiblesolutionsanddeterminethefractionofthesamplethatarelocalminima.A95-percentcondenceintervalonthefractionoflocalminimatototalnumberofsolutionsmaybecalculated.Table 7{3 comparesthenumberoflocalminimaper106feasiblesolutionsforvariousproblemsizesandcategories.Therstsetcontainsproblemsgeneratedfromtheproposedal-gorithm.Thenextset,CategoryI,containsproblemsgeneratedwithintegerrandomassignmentcoststhatareuniformon[1,25].CategoryII,usingathree-dimensionalproblemasanexample,areproblemsgeneratedwithcijk=aij+aik+ajk,whereaij,aik,andajkarerandomlygeneratedintegersfromauniformdistributionon[1,7].Theseparameterswerechosenbasedonapproximatespreadofminimumandmaximumassignmentcostsfortestproblemsgeneratedbytheproposedalgorithm.

PAGE 121

AnalysisofresultsinTable 7{3 suggeststhenumberoflocalminimaforthegener-atedMAPiscomparabletootherMAPswithdierentassignmentcostsstructures.Anotherinterestingaspectisitappearsproblemswithrelativelysmalldandlargen1haveasmallerfractionoflocalminima.Thissuggeststhatalgorithmsusinglocalsearchtechniquescomparedtothosethatdonotmayconvergetoaglobalminimumfasterforthesetypeproblems.However,additionalresearchisneededconcerningthenumberoflocalminimaforMAPsanditsimpactsondierentsolutionmethods. Table7{3: Numberofdiscretelocalminimaper106feasiblesolutions.Therangeisa95-percentcondenceintervalbasedonproportionatesampling. ProblemSizeTestProblemCategoryICategoryIIUniform,[1,25]SumofUniform,[1,7] 3989,800to250,12082,230to237,77052,510to187,4905x6x7x87681to115199036to131647501to112996x7x8x9x10501to819518to842422to7189x10x100.4to0.70.2to1.20.0to0.8 Consideringtherelativedicultyinsolvingtheproposedtestproblems,twodif-ferentexperimentswererun.Therstexperimentmeasuredthetimetosolvethedierentproblemcategoriesusingabranch-and-boundexactsolutionalgorithmassuggestedbyPierskalla[ 87 ].Theexperimentwasrunona2.2GHz,PentiumIVmachine.FiverunswereconductedoneachoftheMAPsizes9x10x10and5x6x7x8forthecategoriesdescribedaboveforatotalof15runs.TheresultsasshowninTable 7{4 indicatetheproposedtestproblemstakelongertosolvethantheran-domlygeneratedproblems.ThesecondexperimentusedaversionofGRASPtosolve20x30x40and6x7x8x9x10MAPsasdescribedabove.Unlikeintherstexper-imentwheretime-to-solvewasusedasameasureofdiculty,thisexperimentxedthetimethealgorithmwasallowedtorunandthencomparedtheresultingsolutionwiththeoptimalsolution.Fiveexperimentswererunoneachproblemcategoryandsize.TheresultsinTable 7{5 showthat,forthissolutionmethod,theproposedtest

PAGE 122

problemsaremorediculttosolve.Theseexperimentsindicatethetestproblemswouldbeusefulinexercisingatleastsomeexactandnon-exactsolutionmethods. Table7{4: Comparisonofsolutiontimesinsecondsusinganexactsolutionalgorithmofthebranch-and-boundvariety. 9x10x105x6x7x8 Run TestProblemCatICatII TestProblemCatICatII 1 40<11 4<1<12 53<1<1 17<1<13 26<1<1 13<1<14 44<1<1 2<1<1 2<1<1 Mean 33<1<1 7.2<1<1 Table7{5: ComparisonofsolutionresultsusingaGRASPalgorithm. Averagepercentagedierencefromoptimal

PAGE 123

distributionofassignmentcosts,numberoflocalminimaanddicultytosolve.Basedonthesefewcharacteristics,thegeneratedMAPtestproblemsappeartoberealisticandchallengingforexercisingexactandheuristicsolutionmethods.

PAGE 124

Theworkinthisdissertationexaminedcombinatorialproblemsfromaproba-bilisticapproachinaneorttoimproveexistingsolutionmethodsorndnewalgo-rithmsthatperformbetter.Aprobabilisticanalysisofcombinatorialproblemsisaverybroadsubject;however,thecontexthereisthestudyofinputdataandsolutionvalues. Weinvestigatedcharacteristicsofthemeanoptimalsolutionvaluesforrandommultidimensionalassignmentproblems(MAPs)withaxialconstraints.Inthecasesofuniformandexponentialcosts,experimentaldataindicatesthatthemeanoptimalvalueconvergestozerowhentheproblemsizeincreases.Inthecaseofstandardnormalcosts,experimentaldataindicatesthemeanoptimalvaluegoestonegativeinnitywithincreasingproblemsize.Usingcurvettingtechniques,wedevelopednumericalestimatesofthemeanoptimalvalueforvarioussizedproblems.Theexper-imentsindicatethatnumericalestimatesarequiteaccurateinpredictingtheoptimalsolutionvalueofarandominstanceoftheMAP. OurexperimentalapproachtotheMAPcanbeeasilyextendedtootherhardproblems.Forexample,solutionapproachestotheQAPmaybenetfromnumericalestimatesoftheoptimalvalues.Additionally,futureworkisneededusingreal-worlddata.Otherinterestingworkincludescloserstudyofthenumericalmodels.Itiscleartheparametersasymptoticallyapproachparticularvalues.Questionsremainonwhatthesevaluesareandwhytheyexist. Furtherresearchandthoughtareneededtoseehowthenumericalestimatesofthemeanoptimalvaluescanbeusedtoimproveexistingsolutionalgorithmsordevelopingnewalgorithms. 111

PAGE 125

Usinganovelprobabilisticapproach,weprovedtheasymptoticcharacteristicsofthemeanoptimalcostsofMAPs.Furtherworkisneededtodevelopandprovemoreglobaltheoremsontheasymptoticcharacteristicsofcombinatorialproblems.IntheexampleoftheMAP,itappearsthelowersupportoftheparentdistributionhassomebearingonthemeanoptimalcosts. WeinvestigatedtheexpectednumberoflocalminimaforrandominstancesoftheMAPandreportedontheirimpactonthreedierentsolutionalgorithmsthatrelyonlocalneighborhoodsearches.WealsoprovidedaclosedformrelationshipfortheaveragenumberoflocalminimainaspecialcaseoftheMAP.WeprovidedboundsontheaverageinmoregeneralcasesoftheMAP.Moreworkinneededinthisarea.Forexample,aninterestingstudyistoconsiderthedistributionoflocalminimaacrossthesolutionlandscapeandthedistancebetweentheselocalminima.Ananswertothisquestionmayleadtonovelsolutionapproaches. AprobabilisticapproachwasusedtodevelopanMAPtestproblemgeneratorthatcreatesdicultproblemswithknownuniquesolutions.Testproblemgeneratorsareoftenveryusefultoresearchers.Additionalworkisnecessarytocreateothergeneratorsandtouseaprobabilisticapproachtoensurethegeneratorsproducehardproblemsthatareusefulinexercisingsolutionalgorithms. Finally,continuedexploitationofdual-useapplications(militaryandcivilian)isofgreatinterest.Cross-fertilizationofideasbenetspractitionersinallareasofresearch.

PAGE 126

[1] N.M.M.Abreu,P.O.B.Netto,T.M.Querido,andE.F.Gouvea,\Classesofquadraticassignmentprobleminstances:Isomorphismanddicultymeasureusingastatisticalapproach,"DiscreteAppliedMathematics124:103{116,2002. [2] D.Achlioptas,C.Gomes,H.Kautz,andB.Selman,\Generatingsatisableprobleminstances,"AmericanAssociationforArticialIntelligence,pp.256{261,2000. [3] R.K.Ahuja,T.L.Magnanti,andJ.B.Orlin,NetworkFlows:Theory,Algo-rithms,andApplications,Prentice-Hall,EnglewoodClis,NJ,1993. [4] R.M.Aiex,M.G.C.Resende,P.M.Pardalos,andG.Toraldo,\GRASPwithpathrelinkingforthethree-indexassignmentproblem,"Technicalreport,AT&TLabsResearch,FlorhamPark,NJ07733,2001. [5] D.Aldous,\Asymptoticsintherandomassignmentproblem,"Probab.Th.RelatedFields,93:507{534,1992. [6] D.AldousandJ.M.Steele,\Theobjectivemethod:probabilisticcombinatorialoptimizationandlocalweakconvergence,"DiscreteandCombinatorialProba-bility,H.Kesten(ed.),Springer-Verlag,2003. [7] N.AlonandJ.Spencer,TheProbabilisticMethod,2ndEdition,InterscienceSeriesinDiscreteMathematicsandOptimization,JohnWiley,NewYork,2000. [8] S.M.AndrijichandL.Caccetta,\Solvingthemultisensordataassociationproblem,"NonlinearAnalysis,47:5525{5536,2001. [9] E.AngelandV.Zissimopoulos,\Onthelandscaperuggednessofthequadraticassignmentproblem,"TheoreticComputerScience,263:159{172,2001. [10] E.BalasandM.J.Saltzman,\Analgorithmforthethree-indexassignmentproblem,"OperationsResearch,39:150{161,1991. [11] Y.Bar-Shalom,K.Pattipati,andM.Yeddanapudi,\IMMestimationformultitarget-multisensorairtracsurveillance,"ProceedingsoftheIEEE,85(1):80{94,January1997. [12] R.S.Barr,B.L.Golden,J.P.Kelly,M.G.C.ResendeandW.R.Stewart,\De-signingandreportingoncomputationalexperimentswithheuristicmethods,"J.Heuristics,1:9{32,1995. 113

PAGE 127

[13] A.BarvinokandT.Stephen,\Onthedistributionofvaluesinthequadraticassignmentproblem,"inNovelApproachestoHardDiscreteOptimization,P.PardalosandHenryWolkowicz(eds.)FieldsInstituteCommunications,vol.37,2003. [14] J.Beardwood,J.H.Halton,andJ.M.Hammersley,\Theshortestpaththroughmanypoints."Proc.CambridgePhilosophicalSociety55:299{327,1959. [15] J.E.Beasley,OR-Library:Distributingtestproblemsbyelectronicmail,Jour-naloftheOperationalResearchSociety,41(11):1069{1072,1990,webbasedlibraryaccessedat [16] A.Beck,\Onthenumberoflocalmaximaforthemax-cutandbisectionprob-lems,"SchoolofMathematicalSciences,Tel-AvivUniversity,June2003. [17] A.G.BhattandR.Roy,\Onarandomdirectedspanningtree,"IndianStatis-ticalInstitute,DelhiCentre,NewDelhiIndia,2May2003. [18] B.Bollobas,D.Gamarnik,O.Riordan,andB.Sudakov,\OnthevalueofarandomminimumweightSteinertree,"Combinatorica,24(2):187{207,2004. [19] R.E.Burkard,\Selectedtopicsonassignmentproblems,"DiscreteAppliedMathematics,123:257{302,2002. [20] R.Burkard,E.Cela,S.KarischandF.Rendlqaplib,AQuadraticAssignmentProblemLibrary,accessedat [21] R.BurkardandU.Fincke,\Probabilisticasymptoticpropertiesofsomecom-binatorialoptimizationproblems,"DiscreteAppl.Math.,12:21{29,1985. [22] R.Burkard,B.Klinz,andR.Rudolf,\PerspectivesofMongepropertiesinoptimization,"DiscreteAppliedMathematics,70:95{161,1996. [23] R.Burkard,R.Rudolf,andG.J.Woeginger,\Three-dimensionalaxialassign-mentproblemswithdecomposiblecostcoecients,"DiscreteAppliedMathe-matics,65:123{139,1996. [24] P.H.Calamai,L.N.VicenteandJ.J.Judice,\Anewtechniqueforgeneratingquadraticprogrammingtestproblems,"MathematicalProgramming,61:215{231,1993. [25] G.ChenandL.Hong,\Ageneticalgorithmbasedmulti-dimensionaldataasso-ciationalgorithmformulti-sensormulti-targettracking,"Mathl.Comput.Mod-elling,26(4):57{69,1997.

PAGE 128

[26] B.H.Chiarini,W.Chaovalitwongse,andP.Pardalos,\Anewalgorithmforthetriangulationofinput-outputtablesineEconomics,"inSupplyChainandFinance,P.Pardalos,A.Migdalas,andG.Baourakis(eds.),WorldScientic,NewJersey,2004. [27] W.Clemons,D.Grundel,andD.Jecoat,\Applyingsimulatedannealingonthemultidimensionalassignmentproblem,"inTheoryandAlgorithmsforCo-operativeSystems,D.Grundel,R.Murphey,andP.Pardalos(eds.),WorldScientic,NewJersey,(toappear)2004. [28] D.CoppersmithandG.Sorkin,\Constructiveboundsandexactexpecta-tionsfortherandomassignmentproblem,"RandomStructuresandAlgorithms,15:133{144,1999. [29] D.CoppersmithandG.Sorkin,\Ontheexpectedincrementalcostofamini-mumassignment,"Technicalreport,IBMT.J.WatsonResearchCenter,1999. [30] Y.Crama,A.Kolen,A.Oerlemans,andF.Spieksma,\Throughputrateop-timizationintheautomatedassemblyonprintedcircuitboards,"AnnalsofOperationResearch,26:455{480. [31] Y.CramaandF.Spieksma,\Approximatealgorithmsforthree-dimensionalas-signmentproblemswithtriangleinequalities,"EuropeanJournalofOperationalResearch,60:273{279,1992. [32] H.Cramer,MathematicalMethodsofStatistics,PrincetonUniversityPress,Princeton,NewJersey,1957. [33] H.David,OrderStatistics,JohnWileyandSons,Inc.,NewYork,1970. [34] DIMACSCenter,RutgersUniversity,NJ,accessedat [35] A.EisenblatterandA.Koster,FAPweb,Awebsiteaboutfrequencyassignmentproblems,accessedat [36] T.A.FeoandM.G.C.Resende,\Aprobabilisticheuristicforacomputationallydicultsetcoveringproblem,"OperationsResearchLetters,8:67{71,1989. [37] T.A.FeoandM.G.C.Resende,\Greedyrandomizedadaptivesearchproce-dures,"JournalofGlobalOptimization,6:109{133,1995. [38] P.FestaandM.G.C.Resende,\GRASP:anannotatedbibliography,"Technicalreport,AT&TLabsResearch,FlorhamPark,NJ07733,2000. [39] C.A.Floudas,P.M.Pardalos,C.S.Adjiman,W.R.Esposito,Z.H.Gms,S.T.Harding,J.L.Klepeis,C.A.Meyer,andC.A.Schweiger,HandbookofTestProb-lemsinLocalandGlobalOptimization,Kluwer,Dordrecht,Netherlands,1999.

PAGE 129

[40] R.Fourer,L.Lopes,andK.Martin,\LPFML:aW3CXMLschemaforlin-earprogramming,"workingpaperaccessedat [41] A.M.Frieze,\Onthevalueofarandomminimumspanningtreeproblem,"DiscreteAppl.Math.,10:47{56,1985. [42] A.M.FriezeandJ.Yadegar,\Analgorithmforsolving3-dimensionalassign-mentproblemswithapplicationtoschedulingateachingpractice,"JournalofOperationalResearchSociety,32:989{995,1981. [43] R.GarcaandP.Stadler,\Correlationlength,isotropyandmeta-stablestates,"PhysicaD,107:240{254,1997. [44] M.R.GareyandD.S.Johnson,ComputersandIntractability:AGuidetotheTheoryofNP-Completeness,W.H.Freeman,NewYork,1979. [45] J.GarnierandL.Kallel,\Eciencyoflocalsearchwithmultiplelocaloptima,"SIAMJ.DiscreteMath,15(1):122-141,2002. [46] A.Gosavi,Simulation-BasedOptimization:ParametricOptimizationTech-niquesandReinforcementLearning,KluwerAcademicPublishing,Norwell,MA,pp.110{117,2003. [47] P.HansenandN.Mladenovic,\Variableneighborhoodsearch,"inHandbookofAppliedOptimization,P.M.PardalosandM.G.C.Resende(eds.),OxfordUniversityPress,NewYorkNY,Ch.3.6.9,pp.221{234,2002. [48] J.Hasselberg,P.M.PardalosandG.Vairaktarakis,\Testcasegeneratorsandcomputationalresultsforthemaximumcliqueproblem,"JournalofGlobalOptimization,3:463{482,1993. [49] J.Homberger,ExtendedSOLOMON'svehicleroutingproblemswithtimewin-dows,accessedat [50] H.HoosandT.Sttzle,SATLIB-Thesatisabilitylibrary,DarmstadtUniver-sityofTechnology,accessedat [51] R.Horst,P.M.Pardalos,andN.V.Thoai.IntroductiontoGlobalOptimization,2nded.KluwerAcademicPublishers,Dordrecht,Boston,London,2000. [52] B.Huang,L.Perkovicc,andE.Schmutz,\Inexpensived-dimensionalmatch-ings,"RandomStructuresandAlgorithms,20(1):50{58,2002. [53] D.Johnson,C.Aragon,L.McGeoch,andC.Schevon,\Optimizationbysimu-latedannealing:anexperimentalevalution;partI,graphpartitioning,"Oper-ationsResearch,37(6):865{892,1989.

PAGE 130

[54] R.M.Karp,\Theprobabilisticanalysisofsomecombinatorialsearchalgo-rithms,"inAlgorithmsandComplexity:NewDirectionsandRecentResults,J.F.Taub(ed.),AcademicPress,NewYork,pp.1{19,1976. [55] R.M.Karp,\Anupperboundontheexpectedcostofanoptimalassignment,"inDiscreteAlgorithmsandComplexity,D.Johnsonetal.(eds.),AcademicPress,pp.1{4,1987. [56] B.N.Khoury,P.M.ParadalosandD.-ZDu,\Atestproblemgeneratorforthesteinerproblemingraphs,"ACMTransactionsonMathematicalSoftware,19:4:509-522,1993. [57] S.Kirkpatrick,C.Gelatt,andM.Vecchi.\Optimizationbysimulatedanneal-ing,"Science,220:671{680,1983. [58] T.Koch,A.MartinandS.Vo,SteinLib:anupdatedlibraryonsteinertreeproblemsingraphs,accessedat [59] Konrad-Zuse-ZentrumfrInformationstechnikBerlin(ZIB),1994-2002,accessedat [60] T.C.KoopmansandM.J.Beckmann,\Assignmentproblemsandthelocationofeconomicactivities,"Econometrica,25:53{76,1957. [61] B.Krishnamurthy,\Constructingtestcasesforpartitioningheuristics,"IEEETransactionsonComputers,36(9):1112{1114,September1987. [62] H.W.Kuhn,\TheHungarianmethodforsolvingtheassignmentproblem,"NavalResearchLogisticsQuarterly,2:83{97,1955. [63] A.M.LawandW.D.Kelton,SimulationModelingandAnalysis,2nded.,McGraw-Hill,Inc.,NewYork,1991. [64] A.J.Lazarus,\Theassignmentproblemwithuniform(0,1)costmatrix,"Mas-ter'sThesis,Dept.ofMathematics,PrincetonUniversity,1979. [65] E.E.Lewis,IntroductiontoReliabilityEngineering,2nded.,JohnWileyandSons,NewYork,pp.25{30,1996. [66] S.LinandB.W.Kernighan,\Aneectiveheuristicalgorithmforthetravelingsalesmanproblem,"OperationsResearch,21:498{516,1973. [67] S.LinussonandJ.Waastlund,\AproofofParisi'sconjectureontherandomassignmentproblem,"accessedat

PAGE 131

[68] J.LongandA.C.Williams,\Onthenumberoflocalmaximainquadratic0-1programs,"OperationsResearchLetters,13:75{78,1993. [69] G.S.Lueker,\Optimizationproblemsongraphswithindependentrandomedgeweights,"SIAMJournalofComputing,10(2):338{351,May1981. [70] N.Metropolis,A.Rosenbluth,M.Rosenbluth,A.Teller,andE.Teller,\Equa-tionofstatecalculationsbyfastcomputingmachines,"J.Chem.Phys.,21(6):1087{1092,1953. [71] M.MezardandG.Parisi,\Replicasandoptimization,"JPhysLetters,46:771{778,1985. [72] M.MezardandG.Parisi,\Onthesolutionoftherandomlinkmatchingprob-lems,"JPhysLetters,48:1451{1459,1987. [73] P.Moscato,\Fractalinstancesofthetravelingsalesmanproblem,"Densis,FEEC,UNICAMPUniversidadeEstadualdeCampinas,accessedat [74] R.Murphey,P.Pardalos,andL.Pitsoulis,\Agreedyrandomizedadaptivesearchprocedureforthemultitargetmultisensortrackingproblem,"inNetworkdesign:ConnectivityandFacilitiesLocation,volume40ofDIMACSSeriesonDiscreteMathematicsandTheoreticalComputerScience,P.M.PardalosandD.-Z.Du(eds.),pp.277{302,1998. [75] C.Nair,B.Prabhakar,andM.Sharma,\AproofoftheconjectureduetoParisifortheniterandomassignmentproblem,"accessedat [76] B.Olin,\Asymptoticpropertiesofrandomassignmentproblems,"PhDthesis,KunglTekniskaHogskolan,Stockholm,Sweden,1992. [77] G.Palubeckis,\Generatinghardtestinstanceswithknownoptimalsolutionfortherectilinearquadraticassignmentproblem,"JournalofGlobalOptimization,15:127{156,1999. [78] R.Palmer,\Optimizationonruggedlandscapes,"MolecularEvolutiononRuggedLadscapes:Proteins,RNA,andtheImmuneSystem,A.PerelsonandS.Kauman(eds.),AddisonWesley,RedwoodCityCA,pp.3{25,1991. [79] P.Pardalos,\Constructionoftestproblemsinquadraticbivalentprogram-ming,"ACMTransactionsonMathematicalSoftware,17(1):74{87,March1991. [80] P.PardalosandS.Jha,\Parallelsearchalgorithmsforquadraticzero-oneprogramming,"ComputerScienceDepartment,PennsylvaniaStateUniversity,UniversityPark,PA,1988.

PAGE 132

[81] P.PardalosandL.Pitsoulis(eds.),NonlinearAssignmentProblems,AlgorithmsandApplications,Kluwer,Dordrecht,pp.1{12,2000. [82] P.M.PardalosandK.G.Ramakrishnan,\Ontheexpectedoptimalvalueofrandomassignmentproblems:experimentalresultsandopenquestions,"Com-putationalOptimizationandApplications,2:261{271,1993. [83] G.Parisi,\Aconjectureonrandombipartitematching,"Physicse-Printarchive,Jan1998,accessedat [84] E.L.Pasiliao,\AlgorithmsforMultidimensionalAssignmentProblems,"Ph.D.Dissertation,UniversityofFlorida,2003. [85] K.Pattipati,S.Deb,Y.Bar-Shalom,andR.Washburn,\Anewrelaxational-gorithmandpassivesensordataassociation,"IEEETransactionsonAutomaticControl,37(2):198{213February1992. [86] W.Pierskalla,\Thetri-substitutionmethodforthethree-dimensionalassign-mentproblem,"CORSJournal,5:71{81,1967. [87] W.Pierskalla,\Themultidimensionalassignmentproblem,"OperationsRe-search,16:422{431,1968. [88] M.PilcherandR.Rardin,\Partialpolyhedraldescriptionandgenerationofdiscreteoptimizationproblemswithknownoptima,"NavalResearchLogistics,39:839{858,1992. [89] A.Poore,\MDassignmentofdataassociation,"inNonlinearAssignmentProb-lems,AlgorithmsandApplications,P.PardalosandL.Pitsoulis(eds.),Kluwer,Dordrecht,pp.13{38,2000. [90] A.PooreandN.Rijavec,\ALagrangianrelaxationalgorithmformultidimen-sionalassignmentproblemsarisingfrommultitargettracking,"SIAMJournalonOptimization,3:544{563,1993. [91] A.Poore,N.Rijavec,M.Liggins,andV.Vannicola,\Dataassociationprob-lemsposedasmultidimensionalassignmentproblems:problemformulation,"InSignalandDataProcessingofSmallTargets,O.E.Drummond(ed.),pp.552{561.SPIE,Bellingham,WA,1993. [92] J.Pusztaszeri,P.Rensing,andT.Liebling,\Trackingelementaryparticlesneartheirprimaryvertex:acombinatorialapproach,"JournalofGlobalOptimiza-tion,16:422{431,1995. [93] C.ReidysandP.Stadler,\Combinatoriallandscapes,"TheSantaFeInstitute,WorkingPaperswithNumber01-03-014,2001.

PAGE 133

[94] C.H.Reilly,\Inputmodelsforsyntheticoptimizationproblems,"Proceedingsofthe1999WinterSimulationConference,P.A.Farrington,H.B.Nembhard,D.T.Sturrock,andG.W.Evans(eds.)1999. [95] G.Reinelt,LinearOrderingLibrary(LOLIB),accessedat [96] G.ReineltandB.Bixby,TSPLibrary,accessedat [97] J.RosenandS.Suzuki,\Constructionofnonlinearprogrammingtestprob-lems,"CommunicationsoftheACM,8(2):113,Feb1965. [98] H.Rummukainen,\Dynamicsonlandscapes,"NotesfromPostgraduateCourseinTheoreticalComputerScience,LandscapeTheory,Spring2002,accessedat [99] L.Sanchis,\GeneratinghardanddiversetestsetsforNP-hardgraphprob-lems,"DiscreteAppliedMathematics58:35{66,1995. [100] L.SanchisandA.Jagota,\Someexperimentalandtheoreticalresultsontestcasegeneratorsforthemaximumcliqueproblem,"INFORMSJournalonCom-puting,8:2:87{102,Spring1996. [101] B.Selman,D.Mitchell,andH.Levesque,\Generatinghardsatisabilityprob-lems,"ArticialIntelligence,81:17{29,1996. [102] T.Smith,P.Husbands,P.Layzell,andM.O'Shea,\Fitnesslandscapesandevolvability,"EvolutionaryComputation,10(1):1{34,2001. [103] P.StadlerandW.Schnabl,\Thelandscapeofthetravelingsalesmanproblem,"PhysicsLettersA,161:337{344,1992. [104] J.M.Steele,\CompleteconvergenceofshortpathsandKarp'salgorithmfortheTSP,"MathematicsofOperationsResearch,6(3):374{378,1981. [105] J.M.Steele,\Minimalspanningtreesforgraphswithrandomedgelengths,"inMathematicsandComputerScienceII:Algorithms,Trees,CombinatoricsandProbabilities,B.Chauvin,Ph.Flajolet,D.Gardy,andA.Mokkadem(eds.),Birkhuser,Boston,2002. [106] W.Szpankowski,\Averagecaseanalysisofalgorithms,"inHandbookofAlgo-rithmsandTheoryofComputation,M.Atallah(ed.),CRCpress,BocaRaton,FL,pp.14.1{14.38,1998. [107] Y.L.Tong,TheMultivariateNormalDistribution,SpringerVerlag,Berlin,1990.

PAGE 134

[108] T.Uchida,M.Motoki,andO.Watanabe,\SATinstancegenerationpage,"WatanabeResearchGroupofDept.ofMath.andComputingSciences,TokyoInst.ofTechnology,accessedat [109] D.W.Walkup,\Ontheexpectedvalueofarandomassignmentproblem,"SIAMJournalofComputing,8:440{442,1979. [110] E.D.Weinberger,\Correlatedanduncorrelatedtnesslandscapesandhowtotellthedierence,"BiologyCybernet,63:325{336,1990. [111] S.Wright,\Therolesofmutation,inbreeding,andselectioninevolution,"ProceedingsoftheSixthCongressonGenetics,1:365{366,1932. [112] L.YongandP.M.Pardalos,\Generatingquadraticassignmenttestproblemswithknownoptimalpermutations,"ComputationalOptimizationandApplica-tions,1(2):163{184,1992.

PAGE 135

Theauthor,DonA.Grundel,wasborninFresno,California,in1963.HegrewupinOcala,Florida,wherehemethislovelywifeBonnie.Heandhiswifehavetwowonderfulchildren,AndrewandErin.HegraduatedfromtheUniversityofFloridain1986withaBachelorofMechanicalEngineeringandwenttoworkatEglinAFB,Florida,asadesignandconstructionengineerforthebase'scivilworksdepartment.In1994,heobtainedanMBAfromtheUniversityofWestFlorida.HewentbacktoschoolattheUniversityofFlorida,GraduateEngineeringandResearchCenterandobtainedamaster'sinindustrialandsystemsengineeringin2001.HeearnedhisPhDinAugust2004. 122


Permanent Link: http://ufdc.ufl.edu/UFE0005640/00001

Material Information

Title: Probabilistic Analysis and Results of Combinatorial Problems with Military Applications
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0005640:00001

Permanent Link: http://ufdc.ufl.edu/UFE0005640/00001

Material Information

Title: Probabilistic Analysis and Results of Combinatorial Problems with Military Applications
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0005640:00001


This item has the following downloads:


Full Text











PROBABILISTIC ANALYSIS AND RESULTS
OF COMBINATORIAL PROBLEMS
WITH MILITARY APPLICATIONS














By

DON A. GRUNDEL


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2004

































Copyright 2004

by

Don A. Grundel
















I dedicate this work to Bonnie, Andrew and Erin.















ACKNOWLEDGMENTS

I wish to express my heartfelt thanks to Professor Panos Pardalos for his guidance

and support. His extraordinary energetic personality inspires all those around him.

What I appreciate most about Professor Pardalos is he sets high goals for himself and

his students and then tirelessly strives to reach those goals.

I am grateful to the United States Air Force for its financial support and for

allowing me to pursue my lifelong goal. Within the Air Force, I owe a debt of

gratitude to Dr. David Jeffcoat for his counsel and assistance throughout my PhD

efforts.

My appreciation also goes to my committee members Stan Uryasev, Joseph Ge-

unes, and William Hager for their time and thoughtful guidance. I would like to thank

my collaborators Anthony Okafor, Carlos Oliveira, Pavlo Krakhmal, and Lewis Pasil-

iao.

Finally, to my family, Bonnie, Andrew and Erin, who have been extremely sup-

portive -I could not have completed this work without their love and understanding.















TABLE OF CONTENTS
page

ACKNOWLEDGMENTS ................... ...... iv

LIST OF TABLES ................... .......... viii

LIST OF FIGURES ................................ x

ABSTRACT ...................... ............ xii

1 INTRODUCTION ........................... 1

1.1 Probabilistic Analysis of Combinatorial Problems ........ 1
1.2 Main Contributions and Organization of the Dissertation ..... 3

2 SURVEY OF THE MULTIDIMENSIONAL ASSIGNMENT PROBLEM 5

2.1 Formulations .............................. 5
2.2 Com plexity . . . . . . . 7
2.3 Applications .............................. 8
2.3.1 Weapon Target Assignment Problem. . . ... 8
2.3.2 Considering Weapon Costs in the Weapon Target Assign-
ment Problem .................. ... .. 11
2.4 Summary .................. ............ .. 12

3 CHARACTERISTICS OF THE MEAN OPTIMAL SOLUTION TO THE
MAP ..... .............. ............... .. 13

3.1 Introduction .................. ........... .. 13
3.1.1 Basic Definitions and Results ................ .. 14
3.1.2 M otivation ........... . . ...... 15
3.1.3 Asymptotic Studies and Results .............. .. 16
3.1.4 Chapter Organization . . ........ 19
3.2 Mean Optimal Costs for a Special Case of the MAP . ... 20
3.3 Branch and Bound Algorithm ............ ... .. .. 23
3.3.1 Procedure .................. ...... .. .. 24
3.3.2 Sorting . . . . . . ... .. 27
3.3.3 Local Search .................. ..... .. .. 27
3.4 Computational Experiments .................. .. 28
3.4.1 Experimental Procedures .................. .. 28
3.4.2 Mean Optimal Solution Costs ............... .. 29
3.4.3 Curve Fitting .................. ..... .. 33









3.5 Algorithm Improvement Using Numerical Models .. .......
3.5.1 Improvement of B&B .. .................
3.5.2 Comparison of B&B Implementations .. ..........
3.6 R em arks . . . . . . . .

4 PROOFS OF ASYMPTOTIC CHARACTERISTICS OF THE MAP .

4.1 Introduction . . . . . . . .
4.2 Greedy Algorithm s .. .....................
4.2.1 Greedy Algorithm 1 .. ..................
4.2.2 Greedy Algorithm 2 .. ..................
4.3 Mean Optimal Costs of Exponentially and Uniformly Distributed
Random M APs .. ......................
4.4 Mean Optimal Costs of Normal-Distributed Random MAPs ..
4.5 Remarks on Further Research .. ................

5 PROBABILISTIC APPROACH TO SOLVING THE MULTISENSOR
MULTITARGET TRACKING PROBLEM .. ............

5.1 Introduction . . . . . . . .
5.2 Data Association Formulated as an MAP .. ............
5.3 Minimum Subset of Cost Coefficients .. .............
5.4 GRASP for a Sparse MAP .. ..................
5.4.1 GRASP Complexity .. ..................
5.4.2 Search Tree Data Structure .. ...............
5.4.3 GRASP vs Sparse GRASP .. ...............
5.5 C conclusion . . . . . . . .

6 EXPECTED NUMBER OF LOCAL MINIMA FOR THE MAP .....

6.1 Introduction . . . . . . . .
6.2 Some ('! i o '.teristics of Local Neighborhoods .. ..........
6.3 Experimentally Determined Number of Local Minima .......
6.4 Expected Number of Local Minima for n 2 ............
6.5 Expected Number of Local Minima for n > 3 .. ..........
6.6 Number of Local Minima Effects on Solution Algorithms .....
6.6.1 Random Local Search .. .................
6.6.2 G R A SP . . . . . . . .
6.6.3 Simulated Annealing .. .................
6.6.4 R results . . . . . . . .
6.7 Conclusions . . . . . . . .

7 MAP TEST PROBLEM GENERATOR .. ...............

7.1 Introduction . . . . . . . .
7.1.1 Test Problem Generators .. ................
7.1.2 Test Problem Libraries .. ................









7.2 Test Problem Generator ............... .... 98
7.2.1 Proposed Algorithm ...... .......... ...... 98
7.2.2 Proof of Unique Optimum ............. .. 102
7.2.3 Complexity ............... .... .. 103
7.3 MAP Test Problem Quality ............. . 104
7.3.1 Distribution of Assignment Costs . . 105
7.3.2 Relative Difficultly of Solving Test Problems ....... 106
7.4 Test Problem Library ................ ... 109
7.5 Remarks ............... ........... 109

8 CONCLUSIONS ............... ........... ..111

REFERENCES ................... ... ........ 113

BIOGRAPHICAL SKETCH .................. ......... 122















LIST OF TABLES
Table page

3-1 Mean optimal solution costs obtained from the closed form equation for
MAPs of sizes n = 2, 3 < d < 10 and with cost coefficients that are
independent exponentially distributed with mean one. . ... 23

3-2 Number of runs for each experiment with uniform or exponential as-
signment costs. ............... ......... 29

3-3 Number of runs for each experiment standard normal assignment costs. 30

3-4 Mean optimal costs for different sizes of MAPs with independent as-
signment costs that are uniform in [0,1]. .. . ..... 31

3-5 Mean optimal costs for different sizes of MAPs with independent as-
signment costs that are exponential with mean 1. . .... 31

3-6 Mean optimal costs for different sizes of MAPs with independent as-
signment costs that are standard normal. .. . ..... 31

3-7 Curve fitting results for fitting the form (An+B)c to the mean optimal
costs for MAPs with uniform assignment costs. . . 35

3-8 Curve fitting results for fitting the form (An+B)c to the mean optimal
costs for MAPs with exponential assignment costs. . .... 35

3-9 Curve fitting results for fitting the form A(n+B)c to the mean optimal
costs for MAPs with standard normal assignment costs. ...... .. 36

3-10 Estimated and actual mean optimal costs from ten runs for variously
sized MAPs developed from different distributions. Included are the
average difference and largest difference between estimated mean op-
timal cost and optimal cost. .............. ...... 37

3-11 Results showing comparisons between three primal heuristics and the
numerical estimate of optimal cost for several problem sizes and
types. Shown are the average feasible solution costs from 50 runs
of each primal heuristic on random instances. ............ ..40

3-12 Average time to solution in seconds of solving each of five randomly
generated problems of various sizes and types. The experiment in-
volved using the B&B solution algorithm with different starting upper
bounds developed in three different v-i--. ............. 43









5-1 Comparisons of the number of cost coefficients of original MAP to that
in A. ..... .............. .............. 63

5-2 Table of experimental results of comparing solution quality and time-
to-solution for GRASP in solving fully dense and reduced simulated
MSMTT problems. Five runs of each algorithm were conducted
against each problem. .................. ..... 68

6-1 Average number of local minima (2-exchange neighborhood) for differ-
ent sizes of MAPs with independent assignment costs. . ... 75

6-2 Average number of local minima (3-exchange neighborhood) for differ-
ent sizes of MAPs with i.i.d. standard normal assignment costs. 76

6-3 Proportion of local minima to total number of feasible solutions for
different sizes of MAPs with i.i.d. standard normal costs. . 76

7-1 Timed results of producing test problems of various sizes. ...... ..105

7-2 Chi-square goodness-of-fit test for normal distribution of assignment
costs for six randomly selected 5x5x5 test problems. . ... 106

7-3 Number of discrete local minima per 106 feasible solutions. The range
is a 95-percent confidence interval based on proportionate sampling. 108

7-4 Comparison of solution times in seconds using an exact solution algo-
rithm of the branch-and-bound variety. .. . . .... 109

7-5 Comparison of solution results using a GRASP algorithm . 109















LIST OF FIGURES
Figure page

3-1 Branch and Bound on the Index Tree. ................. 24

3-2 Plots of mean optimal costs for four different sized MAPs with expo-
nential assignment costs. ............... .... 30

3-3 Surface plots of mean optimal costs for 3 < d < 10 and 2 < n < 10
sized MAPs with exponential assignment costs. ......... ..32

3-4 Plots of mean optimal costs for four different sized MAPs with standard
normal assignment costs. ............... .... 32

3-5 Plots of standard deviation of mean optimal costs for four different
sized MAPs with exponential assignment costs. ......... ..33

3-6 Plots of standard deviation of mean optimal costs for four different
sized MAPs with standard normal assignment costs. . ... 34

3-7 Three dimensional MAP with exponential assignment costs. Plot in-
cludes both observed mean optimal cost values and fitted values.
The two lines are nearly indistinguishable. . . ..... 36

3-8 Plots of fitted and mean optimal costs from ten runs of variously sized
MAPs developed from the uniform distribution on [10, 20]. Note
that the observed data and fitted data are nearly indistinguishable. 38

3-9 Plots of fitted and mean optimal costs from ten runs of variously sized
MAPs developed from the exponential distribution with mean three. 38

3-10 Plots of fitted and mean optimal costs from ten0 runs of variously sized
MAPs developed from a normal distribution, N(p = 5, a = 2). 39

3-11 Branch and bound on the index tree. ................... .. 41

5-1 Example of noisy sensor measurements of target locations. . 57

5-2 Example of noisy sensor measurements of close targets. In this case
there is false detection and missed targets. . . ..... 57

5-3 Search tree data structure used to find a cost coefficient or determine
a cost coefficient does not exist. ................ .. .. 66

5-4 Search tree example of a sparse MAP. ................. 67









6-1 Proportion of feasible solutions that are local minima when considering
the 2-exchange neighborhood and where costs are i.i.d. standard
norm al. . . . . . .. . . 76

6-2 Plots of solution quality versus number of local minima when using the
2-exchange neighborhood. The MAP has a size of d = 4, n = 5 with
cost coefficients that are i.i.d. standard normal. ......... ..89

6-3 Plots of solution quality versus number of local minima when using a
3-exchange neighborhood. The MAP has a size of d = 4, n = 5 with
cost coefficients that are i.i.d. standard normal. ......... ..90

7-1 Tree graph for 3x4x4 MAP. .................. .... 98

7-2 Initial tree graph with assignment costs and lower bound path costs.. 101

7-3 Tree graph with optimal path and costs. .............. ..102

7-4 Tree graph used to consider all feasible nodes at level 3 from the first
node in level 2. .................. .. ...... 103

7-5 Final tree graph for a 3x4x4 MAP. ................. 104

7-6 Typical normal probability plot for a 5x5x5 test problem ..... ..106

7-7 Typical histogram of 20x30x40 test problem. . . 107















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

PROBABILISTIC ANALYSIS AND RESULTS
OF COMBINATORIAL PROBLEMS
WITH MILITARY APPLICATIONS

By

Don A. Grundel

August 2004

C('! In: Panagote M. Pardalos
Major Department: Industrial and Systems Engineering

The work in this dissertation examines combinatorial problems from a probabilis-

tic approach in an effort to improve existing solution methods or find new algorithms

that perform better. Applications addressed here are focused on military uses such

as weapon-target assignment, path planning and multisensor multitarget tr 1.:ii-n -:

however, these may be easily extended to the civilian environment.

A probabilistic analysis of combinatorial problems is a very broad subject; how-

ever, the context here is the study of input data and solution values.

We investigate characteristics of the mean optimal solution values for random

multidimensional assignment problems (\. APs) with axial constraints. Cost coeffi-

cients are taken from three different random distributions: uniform, exponential and

standard normal. In the cases where cost coefficients are independent uniform or

exponential random variables, experimental data indicate that the average optimal

value of the MAP converges to zero as the MAP size increases. We give a short

proof of this result for the case of exponentially distributed costs when the number

of elements in each dimension is restricted to two. In the case of standard normal









costs, experimental data indicate the average optimal value of the MAP goes to neg-

ative infinity as the MAP size increases. Using curve fitting techniques, we develop

numerical estimates of the mean optimal value for various sized problems. The exper-

iments indicate that numerical estimates are quite accurate in predicting the optimal

solution value of a random instance of the MAP.

Using a novel probabilistic approach, we provide generalized proofs of the .- i-mp-

totic characteristics of the mean optimal costs of MAPs. The probabilistic approach

is then used to improve the efficiency of the popular greedy randomized adaptive

search procedure.

As many solution approaches to combinatorial problems rely, at least partly,

on local neighborhood searches, it is widely assumed the number of local minima

has implications on solution difficulty. We investigate the expected number of local

minima for random instances of the MAP. We report on empirical findings that the

expected number of local minima does impact the effectiveness of three different

solution algorithms that rely on local neighborhood searches.

A probabilistic approach is used to develop an MAP test problem generator that

creates difficult problems with known unique solutions.















CHAPTER 1
INTRODUCTION

Combinatorial optimization problems are found in everyday, life. They are par-

ticularly important in military applications as they most often concern management

and efficient use of scarce resources. Applications of combinatorial problems are in

a period of rapid development which follows from the widespread use of computers

and the data available from information systems. Although computers have allowed

expanded combinatorial applications, most of these problems remain very hard to

solve. The purpose of the work in this dissertation is to examine combinatorial prob-

lems from a probabilistic approach in an effort to improve existing solution methods

or find new algorithms that perform better. Most applications addressed here are fo-

cused on military applications; however, most may be easily extended to the civilian

environment.

1.1 Probabilistic Analysis of Combinatorial Problems

In general, probabilistic analysis of combinatorial problems is a very broad sub-

ject; however, the context being used here is the study of problem input values and

solution values of combinatorial problems. An obvious goal is to determine if param-

eters (e.g., mean, standard deviation, etc.) of these values can be used to improve

the efficiency of a solution algorithm. Alternatively, parameters of these values may

be useful in selecting an appropriate solution algorithm. Although problem instance

size is directly correlated with the difficulty of determining a solution, we often face

problems of similar size that have far different computing times. One can conclude

from this that characteristics of the problem data are significant factors.

An example of the study of solution values is by Barvinok and Stephen [13], where

the authors obtain a number of results regarding the distribution of solution values









of the quadratic assignment problem. In the paper, the authors consider questions

such as, how well does the optimum of a sample of random permutations approximate

the true optimum? They explore an interesting approach in which they consider the

"k-th sphere" around the true optimum. The k-th sphere, in simple terms, quantifies

the nearness of permutations to the optimum permutation. By allowing the true

optimum to represent a bullseye, the authors observe as the k-th sphere contracts

to the optimal permutation, the average solution value of a sample of permutations

steadily improves.

A study of the quadratic assignment problem (QAP) is found work by Abreu

et al. [1] where the authors consider using average and variance of solution costs to

establish the difficulty of a particular instance.

Sanchis and Schnabl [103] study the "landscape" of the traveling salesman prob-

lem. Considered are number of local minima and autocorrelation functions. The

concept of landscape was introduced by Wright [111] and can be thought of as a map

of solution values such that there are peaks and valleys. Landscape roughness can

give an indication of problem difficulty.

In a study of cost inputs, Reilly [94] I--. -i that the degree of correlation among

input data may influence the difficulty of finding a solution. It is -r.-.ii, -I. 1 that an

extreme level of correlation can produce very challenging problems.

In this dissertation, we use a probabilistic approach to consider how input costs

affect solution values in an important class of problems called the multidimensional

assignment problem. We also consider the mean optimal costs of various problem

instances to include some .i-i! i, 1I.l ic characteristics. We include another interesting

probabilistic analysis which is our study of local minima and how the number of local

minima affects solution methods. Finally, we use a probabilistic approach to design

and analyze a test problem generator.









1.2 Main Contributions and Organization of the Dissertation

The main contributions and organization of this dissertation are briefly discussed

in the following paragraphs.

Survey of the multidimensional assignment problem. A brief survey of

the multidimensional assignment problem (1 AP) is provided in C'! Ilpter 2. In this

chapter, we provide alternative formulations and applications for this important and

difficult problem.

Mean optimal solution values of the MAP. In ('!, lpter 3 we report exper-

imentally determined values of the mean optimal solution costs of MAPs with cost

coefficients that are independent random variables that are uniformly, exponentially

or normally distributed. Using the experimental data, we then find curve fitting

models that can be used to accurately determine their mean optimal solution costs.

Finally, we show how the numerical estimates can be used to improve at least two

solution methods of the MAP.

Proof of asymptotic characteristics of the MAP. In ('!i lpter 4 we prove

some .-i-, !! ii '1 ic characteristics of the mean optimal costs using a novel probabilistic

approach.

Probabilistic approach to solving the data association problem. Us-

ing the probabilistic approach introduced in ('!i lpter 4, we extend the approach in

('!i lpter 5 to more efficiently solve the data association problem that results from the

multisensor multitarget tracking problem. In the multisensor multitarget problem

noisy measurements are made with an arbitrary number of spatially diverse sensors

regarding an arbitrary number of targets with the goal of estimating the trajectories

of all the targets present. Furthermore, the number of targets may change by moving

into and out of detection range. The problem involves a data association of sensor

measurements to targets and estimates the current state of each target. The combi-

natorial nature of the problem results from the data association problem; that is how









do we optimally partition the entire set of measurements so that each measurement

is attributed to no more than one target and each sensor detects a target no more

than once?

Expected number of local minima for the MAP. The number of local

minima in a problem may provide insight to more appropriate solution methods.

('!i lpter 6 explores the number of local minima in the MAP and then considers the

impact of the number of local minima on three solution methods.

MAP test problem generator. As examined in the first five chapters, a

probabilistic analysis can be used to develop a priori knowledge of problem instance

hardness. In ('!i lpter 7 we develop an MAP test problem generator and use some

probabilistic analyses to determine the generator's effectiveness in creating Il;,oil.:l

test problems with known unique optimal solutions. Also included is a brief survey

of sources of combinatorial test problems.














CHAPTER 2
SURVEY OF THE MULTIDIMENSIONAL ASSIGNMENT PROBLEM

The MAP is a higher dimensional version of the standard (two-dimensional,

or linear) assignment problem. The MAP is stated as follows: given d, n-sets

A1, A2,..., Ad, there is a cost for each d-tuple1 in A, x A2 x ... x Ad. The problem

is to minimize the cost of n tuples such that each element in A1 U A2 U ... U Ad is

in exactly one tuple. The problem was first introduced by Pierskalla [86]. Solution

methods have included branch and bound [87, 10, 84], Greedy Randomized Adap-

tive Search Procedure (GRASP) [4, 74], Lagrangian relaxation [90, 85], a genetic

algorithm based heuristic [25], and simulated annealing [27].

2.1 Formulations

A well-known instance of the MAP is the three-dimensional assignment problem

(3DAP). An example of the 3DAP consists of minimizing the total cost of assigning

ni items to nj locations at nk points in time. The three-dimensional MAP can be



1 Tuple an abstraction of the sequence: single, double, triple,..., d-tuple. Tuple is
used in denote a point in a multidimensional coordinate system.










formulated as

ni j nk 1
min cijkXijk

nj nk
s.t. Xijk = 1 for all i = 1,2,..., ii,
j=1 k=1


i=l k 1

1 i Xijk < 1 for all k = 1,2,..., nk,
i=1 j=1
Xijk e {0,1} for all i,j, k {1,... ,n},

ni

where Cijk is the cost of assigning item i to location j at time k. In this formulation,

the variable xijk is equal to 1 if and only if the i-th item is assigned to the j-th

location at time k and zero otherwise. If we consider additional dimensions for this

problem, the formulation can be similarly extended in the following way:


min c~l il id
il=1 id=1
n2 nd
s.t. Xil ...i 1 for all i = 1,2,..., ni,
i21= id 1
ni nk-1 nk+1 nd
S E E ** ... id- < 1
ii 1 ik-l l +l=l 1 d 1
for all k = 2,... d 1, and ik = 1, 2,..., nk,
n2 nd-1
.. > il "id < 1 for all id 1, 2,... nd,
i21= id 1=1
Xil...id {0, 1} for all i, i2, .. d {1, .. },

ni < n2 < rd,


where d is the dimension of the MAP.









If we allow n1 = n2 =" nd = n, an equivalent formulation states the MAP in

terms of permutations 61,..., 6d-i of numbers 1 to n. Using this notation, the MAP

is equivalent to

rain 1 Ci,61() ,...,6d 1(i)W
.... ...,..d iE^l
i=1
where HI" is the set of all permutations of {1,..., n}.

2.2 Complexity

Solving even moderate sized instances of the MAP is a difficult task. A linear

increase in the number of dimensions brings an exponential increase in the number

of cost coefficients in the problem and the number of feasible solutions, N, is given

by the relation
d ,i
N f nj!
i= 2
In general, the MAP is known to be NP-hard, a fact which follows from results work

by Garey and Johnson [44]. Even in the case when costs take on a special structure of

triangle inequalities, Crama and Spieksma [31] prove the three-dimensional problem

remains NP-hard. However, special cases that are not NP-hard do exist.

Burkard, Ridolf, and Woeginger [23] investigate the three-dimensional problems

with decomposable cost coefficients. Given three n-element sequences ai, bi and ci,

i = 1,...,n, a cost coefficient dijk is decomposable when dijk = 7 .Ck. Burkard

[23] finds the minimization and maximization of the three-dimensional assignment

problem have different complexities. While the maximization problem is solvable in

polynomial time, the minimization problem remains NP-hard. On the other hand,

Burkard [23] identifies several structures where the minimization problem is polyno-

mially solvable.

A polynomially solvable case of the MAP occurs when the cost coefficients are

taken from a Monge matrix [22]. An m x n matrix C is called a Monge matrix if

cij + crs < cis + crj for all 1 < i < r < m, 1 < j < s < n. Another way to describe









the Monge array is to again consider the matrix C. Any two rows and two columns

must intersect at exactly four elements. The rows and columns satisfy the Monge

property if the sum of the upper-left and lower-right elements is at most the sum of the

upper-right and lower-left elements. This can easily be extended to higher dimensions.

Because of the special structure of the Monge matrix, the MAP becomes polynomially

solvable with a lexicographical greedy algorithm and the identity permutation is an

optimal solution.

2.3 Applications

The MAP has applications in numerous areas such as, data association [8],

scheduling teaching practices [42], production of printed circuit boards [30], placement

of distribution warehouses [87], multisensor multitarget problems [74, 91], tracking

elementary particles [92] and multiagent path planning [84]. More examples and an

extensive discussions of the subject can be found in two extensive surveys [81, 19]. A

particular military application of the MAP is the Weapon Target Assignment problem

which is discussed in the following subsection.

2.3.1 Weapon Target Assignment Problem

The target-based Weapon Target Assignment (WTA) problem [81] considers op-

timally assigning W weapons to T targets so that the total expected damage to the

targets is maximized. The term target-based is used to distinguish these problems

from the asset-based or defense-based problems where the goal of these problems

is to assign weapons to incoming missiles to maximize the surviving assets. The

target-based problems primarily apply to offensive strategies.

Assume at a particular instant in time the number and location of weapons and

targets are known with certainty. Then a single assignment may be made at that

instant. Consider W weapons and T targets and define xij, i = 1,2,..., W, j









1,2,. ,T as:

1 if weapon i assigned to target j,
xij c 0
O otherwise.

Given that weapon i engages target j, the outcome is random.

P(target j is destroyed by weapon i) = Pij

P(target j is not destroyed by weapon i) = 1 Pij

If one assumes that each weapon engagement is independent of every other en-

gagement, then the outcomes of the engagements are independent and Bernoulli dis-

tributed. Note that we let qij = (1 Pi) which is the probability that target j

survives an encounter with weapon i.

Now assign Vj to indicate a value for each target j. The objective is to maximize

the damage to targets or minimize the value of the targets which may be formulated

T W
minimize V ij qJ j (2.1)
j=1 i=1
T
subject to Xij 1, i= ,2,...,W
j=1
Xij {0, 1}.


This is a nonlinear assignment problem and is known to be NP-complete. Notice a

few characteristics of the above problem.

Since there is no cost for employing a weapon, all weapons will be used.

The solution may result in some targets not being targeted because they are

relatively worthless and/or because they are very difficult to defeat.

A transformation of this formulation to an MAP may be accomplished. Using a

two weapon, two target example, the transformation follows. First observe that the

objective function of (2.1) may be written as


minimize Vi[q qX21 ] + V2[q2 q 22


(2.2)









Obviously, the individual probabilities of survival, qij, go to one if weapon i does not

engage target j. Therefore, using the first term of the objective function in equation

(2.2) as an example, the first term becomes


V [qliq21] if X1 = 1 and x21 1= or

Vi[qll] if xl = 1 and X21 = 0, or

V [q21] if 11 = 0 and x21 1 or

V1 if xll = 0 and x21 = 0.


Notice these terms are now constant cost values. A different decision variable, paj,

may be introduced that represents the status of engaging the different weapons on

target j. a = 1, 2} represents weapon l's status of engagement on target j, where

a = 1 means weapon 1 engages target j and a = 2 otherwise. Similarly, {1, 2}

represents weapon 2's status of engagement of target j. For example,


1 both the first and second weapon engage target j,

0 else,

and,

S1 the first but not the second weapon engages target j,
P12) =
0 else.

The cost values may now be represented by c,3j. For example, cl = V [qllq21]

and 121 = VI[qul]. Using these representations, the first term of objective function

(2.2) becomes


CllPlll + C121P121 + C211P211 + C221P221.










For the two weapon, two target scenario, (2.1) may reformulated to a three

dimensional MAP as follows.

2 2 2
min ZZZ "* C
a=l 13 j=1
2 2
s.t. Z a IV a = 1,2
j= 1 =1

j Pa8 tV-1,2

2 2
Pa8 tV/3 -1,2
a=1 j=1
Papje {0, 1} V a, j.


In general, reformulation of (2.1) will result in a W + 1 dimensional MAP. The

number of indices will be T. As mentioned above, weapon costs are not considered

in this formulation which results in all weapons being assigned. A more realistic

formulation that considers weapon costs is developed in the next subsection.

2.3.2 Considering Weapon Costs in the Weapon Target Assignment Prob-
lem

The formulation in the previous subsection excludes weapon costs which can

result in overkill or poor use of expensive weapons on low valued targets. A more

realistic formulation includes weapon costs. Let Ci be the cost of the i-th weapon

and let j =T + 1 be a dummy target. We may now reformulate (2.1) as

W T W T W
minimize C Tx + V q I (2.3)
i=1 j=1 i=l,j=T+1 j=1 i=1
T+1
subject to Xj 1, i 1,2,...,W
j=1
Xi, {0, 1}.










The first summation term considers the costs of weapons assigned to actual targets.

The second summation term considers the savings by applying weapons to the dummy

target.

Following a similar development as in the previous subsection, we obtain a gen-

eralized MAP formulation that incorporates weapon costs.

T+1 T+1 T+I
minm Cwiw2...jPwiw2...
w=1 w21= j=1
T+1 T+1
s.t. Pwwa2 ... 1 Vwi = 1, 2, ,T + 1
w2 1 j1=
T+1 T+1 T+1 T+1

wi=1 Wk- =1wk+l 1 j1=
Vk= 1,...,W-1, and",, 1,2,...,T+1
T+1 T+I
I -- pV 2...J = 1V t,2,...,T + 1
wi=1 ww=1
PW1w2j E {0, 1} V WI, 2, .. ,j.


This formulation results in a W + 1 dimensional MAP with T + 1 elements in each

dimension.

2.4 Summary

The MAP has been studied extensively in the last couple of decades and its appli-

cations in both military and civilian arenas has been rapidly expanding. The difficult

nature of the problem requires researchers to continuously consider novel solution

methods and a probabilistic approach provides some needed insight in developing

these solution methods.















CHAPTER 3
CHARACTERISTICS OF THE MEAN OPTIMAL SOLUTION TO THE MAP

In this chapter, we investigate characteristics of the mean optimal solution values

for random MAPs with axial constraints. Throughout the study, we consider cost

coefficients taken from three different random distributions: uniform, exponential

and standard normal. In the cases of uniform and exponential costs, experimental

data indicate that the mean optimal value converges to zero when the problem size

increases. We give a short proof of this result for the case of exponentially distributed

costs when the number of elements in each dimension is restricted to two. In the case

of standard normal costs, experimental data indicate the mean optimal value goes

to negative infinity with increasing problem size. Using curve fitting techniques, we

develop numerical estimates of the mean optimal value for various sized problems.

The experiments indicate that numerical estimates are quite accurate in predicting

the optimal solution value of a random instance of the MAP.

3.1 Introduction

NP-hard problems present important challenges to the experimental researcher

in the field of algorithms. That is because, being difficult to solve in general, careful

restrictions must be applied to a combinatorial optimization problem in order to

solve some of its instances. However, it is also difficult to create instances that are

representative of the problem, suitable for the technique or algorithm being used, and

at the same time interesting from the practical point of view.

One of the simplest and, in some cases, most useful v--~i- of creating problem

instances consists of drawing values from a random distribution. Using this procedure,

one wishes to create a problem that is difficult "on average," but that can also appear

as the outcome of some natural process.









Thus, one of the questions that arises is how a random problem will behave in

terms of solution value, given some distribution function and parameters from which

values are taken. This question turns out to be very difficult to solve in general. As

an example, for the Linear Assignment Problem (LAP), results have not been easy

to prove, despite intense research in this field [5, 28, 29, 55, 82].

In this chapter we perform a computational study of the .-i-~!,lil ic behavior

for instances of the MAP.

3.1.1 Basic Definitions and Results

The MAP is an NP-hard combinatorial optimization problem, which extends the

Linear Assignment Problem (LAP) by adding more sets to be matched. The number

d of sets corresponds to the dimension of the MAP. In the special case of the LAP,

we have d = 2. ('! Ilpter 2 provides an overview of the MAP to include formulations

and applications.

Let z(I) be the value of the optimum solution for an instance I of the MAP.

We denote by z* the expected value of z(I), over all instances I constructed from

a random distribution (the context will make clear what specific distribution we

are talking about). In the problem instances considered in this chapter, we have

fl = n2 = nrd = n.

Our main contribution in this chapter is the development of numerical estimates

of the mean optimal costs for randomly generated instances of the MAP. The experi-

ments performed show that for uniform [0, 1] and exponentially distributed costs, the

optimum value converges to zero as the problem size increases. These results are not

surprising for an increase in d since the number of cost coefficients increases exponen-

tially with d. However, convergence to zero for increasing n is not as obvious since

the objective function is the sum of n cost coefficients. Experiments with standard

normally distributed costs show that the optimum value goes to -oo as the problem









size increases. More interestingly, the experiments show convergence even for small

values of n and d.

The three distributions (exponential, uniform and normal) were chosen for anal-

ysis as they are very familiar to most practitioners. Although we would not expect

real-world problems to have cost coefficients that follow exactly these distributions,

we believe that our results may be extended to other cost coefficient distributions.

3.1.2 Motivation

The study of.,-i, :!il II ic values for MAPs has important motivations arising from

theory and from practical applications. First, there are few theoretical results on this

subject, and therefore, practical experiments are a good method for determining how

MAPs behave for instances with random values. Determining .i-vmptotic values for

such problems is a 1 ii 'i"r open question in combinatorics, which can be made clear

by careful experimentation.

Another motivation for this work has been the possible use of .- i~! ,ll1 ic results

in the practical setting of heuristic algorithms. When working with MAPs, one of

the greatest difficulties is the need to cope with a large number of entries in the

multidimensional vector of costs. For example, in an instance with d dimensions and

minimum dimension size n, there are nd cost elements that must be considered for

the optimum assignment. Solving an MAP can become very hard when all elements

of the cost vector must be read and considered during the algorithm execution. This

happens because the time needed to read nd values makes the algorithm exponential

on d. A possible use of the results shown in this chapter allows one, having good

estimates of the expected value of an optimal solution and the distribution of costs,

to discard a large number of entries in the cost vector, which have low probability of

being part of the solution. By doing this, we can improve the running time of most

algorithms for the MAP.









Finally, while some computational studies have been performed for the random

LAP, such as by Pardalos and Ramakrishnan [82], there are limited practical and

theoretical results for the random MAP. In this chapter we try to improve in this

respect by presenting extensive results of computational experiments for the MAP.

3.1.3 Asymptotic Studies and Results

Asymptotical studies of random combinatorial problems can be traced back to

the work of Beardwood, Halton and Hammersley [14] on the traveling salesman prob-

lem (TSP). Other work includes studies of the minimum spanning tree [41, 105],

Quadratic Assignment Problem (QAP) [21] and, most notably, studies of the Linear

Assignment Problem (LAP) [5, 28, 55, 64, 83, 76, 82, 109]. A more general analysis

was made on random graphs by Lueker[69].

In the case of the TSP, the problem is to let Xi, Xi 1,..., n, be independent

random variables uniformly distributed on the unit square [0, 1]2, and let L, denote

the length of the shortest closed path (usual Euclidian distance) which connects each

element of {X1,X2,...,X,.}. The classic result proved by Beardwood et al. [14] is

lim --


with probability one for a finite constant P. This becomes significant, as addressed by

Steele [104], because it is key to Karp's algorithm [54] for solving the TSP. Karp uses

a cellular dissection algorithm for the approximate solution. The above result may be

summarized as implying that the optimal tour through n points is sharply predictable

when n is large and the dissection method tends to give near-optimal solutions when

n is large. This points to an idea of using .,i~!,l')itic results to develop effective

solution algorithms.

In the minimum spanning tree problem, consider an undirected graph G = (N, A)

defined by the set N of n nodes and a set A of m arcs, with a length ci associated with

each arc (i,j) e A. The problem is to find a spanning tree of G, called a minimum









spanning tree (\!ST), that has the smallest total length, LMST, of its constituent arcs

[3]. If we let each arc length ci be an independent random variable drawn from the

uniform distribution on [0, 1], Frieze [41] showed that


E[LMsT] ((3) 3 = 1.202 as n i oo

This was followed by Steele [105], where the Tutte polynomial for a connected graph is

used to develop an exact formula for the expected value of LMST for a finite graph with

uniformly distributed arc costs. Additional work concerning the directed minimum

spanning tree is also available [17].

For the Steiner tree problem which is an NP-hard variant of the MST, Bollobds,

et al. [18] proved that with high probability the weight of the Steiner tree is (1 +

O(1))(k 1)(log n log k)/n when k = O(n) and n -i o and where n is the number

of vertices in a complete graph with edge weights chosen as i.i.d. random variables

distributed as exponential with mean one. In the problem, k is the number of vertices

contained in the Steiner tree.

A famous result that some call the Burkard-Fincke condition relates to the QAP.

The QAP was introduced by Koopmans and Beckmann [60] in 1957 as a model for

the location of a set of indivisible economical activities. QAP applications, extensions

and solution methods are well covered in work by Horst et al. [51]. The Burkard-

Fincke condition [21] is that the ratio between the best and worst solution values

approaches one as the size of the problem increases.

Another way to think of this is for a large problem any permutation is close to

optimal. According to Burkard and Fincke [21] this condition applies to all problems

in the class of combinatorial optimization problems with sum- and bottleneck objec-

tive functions. The Linear Ordering Problem (LOP) [26] falls into this category as

well. Burkard and Fincke -i-i- -1 that this result means that very simple heuristic

algorithms can yield good solutions for very large problems.










Recent work by Aldous and Steele [6] provides part survey, part tutorial on

the objective method in understanding .,-i-vi ill. lic characteristics of combinatorial

problems. They provide some concrete examples of the approach and point out some

unavoidable limitations.

In terms of the .,- i-i!1 .I ,tic nature of combinatorial problems, the most explored

problem has been the LAP. In the LAP we are given a matrix C"x' with coefficients

cij. The objective is to find a minimum cost assignment; i.e., n elements cl1,..., cCni,

such that jp / j, for all p / q, with ji E {1,..., n}, and E I cij is minimum.

A well known conjecture by M6zard and Parisi [71, 72] states that the opti-

mal solution for instances where costs cij are drawn from an exponential or uniform

distribution, approaches 7r2/6 when n (the size of the instance) approaches infinity.

Pardalos and Ramakrishnan [82] provide additional empirical evidence that the con-

jecture is indeed valid. The conjecture was expanded by Parisi [83], where in the case

of costs drawn from an exponential distribution the expected value of the optimal

solution of an instance of size n is given by


1 (3.1)
i2.
i= 1

Moreover,

1 7r2
as n o.
i= 1

This conjecture has been further strengthened by Coppersmith and Sorkin [28]. The

authors conjecture that the expected value of the optimum k-assignment, for a fixed

matrix of size n x m, is given by


.1
i,j>0, i+j








They also presented proofs of this conjecture for small values of n, m and k. The

conjecture is consistent with previous work [71, 83], since it can be proved that for

m = n = k this is simply the expression in (3.1)

Although until recently the proofs of these conjectures have eluded many re-

searchers, there has been progress in the determination of upper and lower bounds.

Walkup [109] proved an upper bound of 3 on the ..-iiiiil ,l ic value of the objective

function, when the problem size increases. This was improved by Karp [55], who

showed that the limit is at most 2. On the other hand, Lazarus [64] proved a lower

bound of 1 + 1/e t 1.3679. More recently this result was improved by Olin [76] to

the tighter lower bound value of 1.51.

Finally, recent papers by Linusson and Wastlund [67] and Nair et al. [75] have

solved the conjectures of M6zard and Parisi, and Coppersmith and Sorkin.

Concerning the MAP, not many results are known about the ..i-mptotic behav-

ior of the optimum solution for random instances. However, one example of resent

work is that by Huang et. al. [52]. In this work the authors consider the complete

d-partite graph with n vertices in each of d sets. If all edges in this graph are assigned

independent weights that are uniformly distributed on [0,1], then the expected mini-

mum weight perfect d-dimensional matching is at least n1-2/d. They also describe

a randomized algorithm to solve this problem where the expected solution has weight

at most 5d3 1-2/d + d15 for all d > 3. However, note that for even a moderate size

for d, this upper bound is not tight.

3.1.4 Chapter Organization

This chapter is organized as follows. In the next section, we give a closed form

result on the mean optimal costs for a special case of the MAP when the number

of elements in each dimension is equal to 2. The method used to solve the MAP

employs a branch-and-bound algorithm, described in Section 3.3, to find exact solu-

tions to the problem. Then, in Section 3.4 we present the computational results and









curve fitting models to estimate the mean optimal costs. Following this, we provide

some methods to use the numerical models to improve the efficiency of two solution

algorithms. Finally, concluding remarks and future research directions are presented

in Section 3.6.

3.2 Mean Optimal Costs for a Special Case of the MAP

In this section we present a result regarding the i-mptotical behavior of 2z in

the special case of the MAP where n = 2, d > 3, and cost elements are independent

exponentially distributed with mean one. This is done to give a flavor of how these

results can be obtained. For proofs of a generalization of this theorem, including

normal distributed costs, refer to C'i plter 4. Initially, we employ the property stated

in the following proposition.


Proposition 3.1 In an instance of the MAP with n = 2 and i.i.d. exponential cost

coefficients with mean 1, the cost of each feasible solution is an independent .ri,,i,,.

distributed random variable with parameters a = 2, and A = 1.

Proof: Let I be an instance of MAP with n = 2. Each feasible solution for I

is an assignment al = CI,(),...,d 1(1), a2 = C2,61(2),...,6d 1(2), with cost z = al + a2.

The important feature of such assignments is that for each fixed entry C,i(1),...,Sd (i),

there is just one remaining possibility, namely c2,6S(2),...,6d-1(2), since each dimension has

only two elements. This implies that different assignments cannot share elements in

the cost vector, and therefore different assignments have independent costs z. Now,

a, and a2 are independent exponential random variables with parameter 1. Thus

z = al + a2 is a Gamma(a, A) random variable, with parameters a = 2 and A = 1.

According to the proof above, it is clear why instances with n > 3 do not have

the same property. Different feasible solutions share elements of the cost vector,

and therefore the feasible solutions are not independent of each other. For example,

consider a problem of size d = 3, n = 3. A feasible solution to this problem is









cill, c232, and c323. Another feasible solution is cll,, c223, and c332. Note that both

solutions share the cost coefficient cll, and are not independent.

Suppose that X, X2,... Xk are k independent gamma distributed variables. Let

X(i) be the ith smallest of these. Applying order statistics [33], we have the following

expression for the expected minimum value of k independent identically distributed

random variables

E[X(1)] j kxf(x)( F(x))k- dx
JO
where f(x) and F(x) are, respectively, the density and distribution functions of the

gamma random variable.

The problem of finding z* for the special case when n = 2 and d > 3 corresponds

to finding the expected minimum cost E[X(1)], for k =2d-1 independent gamma

distributed feasible solution costs, with parameters a = 2, and A = 1 (note that k

is the number of feasible solutions). Through some routine calculus, and noting a

resulting pattern as k is increased, we find the following relationship

k-l /1 j+2.

j=0 i=1

The above equation can be used to prove the .iil. l 1ic characteristics of the

mean optimal cost of the MAP as d increases. We also note that this special result for

the MAP follows directly from Lemma2(ii) by Szpankowski [106]. As an alternative

approach, we use the above equation to prove the following theorem.


Theorem 3.2 For the MAP with n = 2, and i.i.d. exponential cost coefficients with

mean one, z* 0 as d oo..









Proof: When d -- oo, then 2d-

k -- o. We have


(k
j=0

k-1 (k
j=0
k-1 (k -

j=0


k oc as well. So we prove the result when


-Yk- (y + 2)!
i1 j0 j+

1)! ( + 2)(j + 1)
1- )! T kj+2

1)! (k- j)(k + 1)
kk-j+1


Equality (3.4) is found by a change of variable. Using Stirling's approximation n!

(n/e)"gv2Tn, we have


(3.5)


(3.6)


k-1 k 1 k-1 27(k 1) (k j)(k j + 1)
e j! kk-j
j=
e( )k 1 27(k 1) k-1 k
e(k+1 (k- )(k- j + 1)e-
j=o

< e(- 1)k1 2(k 1)
Sk+ ( j)(k 1)
j=0


Note that the summation in Formula (3.7) is exactly E[(k -j)(k j+1)] for a Poisson

distribution with parameter k, which therefore has value k. Thus,


(3.8)


and as


(k- 1)k-1/2
kk


- 0 when k oo,


the theorem is proved.

As will be shown in Section 3.4, experimental results support these conclusions,

even for relatively small values of d. Table 3-1 provides the value of z* for MAPs of

sizes n = 2, 3 < d < 10. We note that a similar approach and results may be obtained

for other distributions of cost coefficients. For example, we have similar results if the


(3.2)


(3.3)


(3.4)


S/2 (k 1)-1/2
2z < eV ,,









cost coefficients are independent gamma distributed random variables, since the sum

of gamma random variables is again a gamma random variable.

Table 3-1: Mean optimal solution costs obtained from the closed form equation for
MAPs of sizes n 2, 3 < d < 10 and with cost coefficients that are independent
exponentially distributed with mean one.

d\n 2
3 0.804
4 0.530
5 0.356
6 0.242
7 0.167
8 0.116
9 0.080
10 0.056


3.3 Branch and Bound Algorithm

This section describes the Branch and bound (B&B) algorithm used in the ex-

periments to optimally solve the MAPs. Branch and bound is essentially an implicit

enumeration algorithm. The worst-case scenario for the algorithm is to have to cal-

culate every single feasible solution. However, by using a bounding technique, the

algorithm is typically able to find an optimal solution by only searching a limited

number of solutions. The index-based B&B is an extension of the three dimensional

B&B proposed by Pierskalla [87] where an index tree data structure is used to rep-

resent the cost coefficients. There are n levels in the index tree with nd-1 nodes on

each level for a total nd nodes. Each level of the index tree has the same value in the

first index. A feasible solution can be constructed by first starting at the top level

of the tree. The partial solution is developed by moving down the tree one level at

a time and adding a node that is feasible with the partial solution. The number of

nodes that are feasible to a partial solution developed at level i, for i = 1, 2,..., n

is (n i)-1. A complete feasible solution is obtained upon reaching the bottom or

nth-level of the tree. Deeper MAP tree representations provide more opportunities for










B&B algorithms to eliminate branches. Therefore, we would expect the index-based

B&B to be more effective for a larger number of elements in each dimension.

3.3.1 Procedure

The B&B approach proposed here finds the optimal solution by moving through

the index-based tree representation of the MAP. The algorithm avoids having to

check every feasible solution by eliminating branches with lower bounds that are

greater than the best-known solution. The approach is presented as a pseudo-code

in Figure 3-1.


procedu
1 for
2 S-
3 i-
4 whi:
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21 end
22 reti
end Inde


re IndexBB(L)
i = 1,..., n do ki -- 0
- 0
1
le i > 0 do
if ki = Lil then
S S\{sj}
ki 00
i- i-1
else
ki = ki + 1
if Feasible(S, Li,k7) then
S -- S U Li,k,
if LB(S) < z* then
if i= n then
S-S
S <---S
z <- Objective(S)
else
i+-i+1
else
S S\{ss}

urn(S, z)
xBB


Figure 3-1: Branch and Bound on the Index Tree.


The algorithm initializes the tree level markers ki, the solution set S, and the

current tree level i in Steps 1-3. The value of the best-known solution set S is

denoted as z. Level markers are used to track the location of cost coefficients on









the tree levels and Li is the set of coefficients at each level i. The solution set S

contains the cost coefficients taken from the different tree levels. Steps 4-21 perform

an implicit enumeration of every feasible path in the index-based tree. The procedure

investigates every possible path below a given node before moving on to the next node

in the same tree level. Once all the nodes in a given level are searched or eliminated

from consideration through the use of upper and lower bounds, the algorithm moves

up to the previous level and moves to the next node in the new level. Step 11 checks

if a given cost coefficient Li,kw, which is the ki-th node on level i, is feasible to the

partial solution set. If the cost coefficient is feasible and if its inclusion does not cause

the lower bound of the objective function to surpass the best-known solution, then

the coefficient is kept in the solution set. Otherwise, it is removed from S in Step 20.

A lower bound that may be implemented to try to remove some of the tree

branches is given by:
r n
LB(S) S, + m in Ci-...d
i= i=r+1

where r = S1 is the size of the partial solution and Si is the cost coefficient selected

from level i of the index-based MAP representation. This expression finds a lower

bound by summing the values of all the cost coefficients that are already in the partial

solution and the minimum cost coefficient at each of the tree levels underneath the

last level searched. The lower bound consists of n elements, one from each level. If

a cost coefficient from a given level is in the partial solution, then that coefficient is

used in the calculation of the lower bound. If none of the coefficients from a given

level is found in the partial solution, then the smallest coefficient from that level is

used.

Before starting the algorithm, an initial feasible solution is needed for an upper

bound. A natural selection would be


S = { ci,J2,3,..,jd I 1 for m = 2, 3,..., d; i = 1,2,..., n} .









The algorithm initially partitions the cost array into n groups or tree levels with

respect to the value of their first index. The first coefficient to be analyzed is the

node furthest to the left at level i = 1. If the lower bound of the partial solution

that includes that node is lower than the initial solution, the partial solution is kept.

It then moves to the next level with i = 2 and again analyzes the node furthest

to the left. The algorithm keeps moving down the tree until it either reaches the

bottom or finds a node that results in a partial solution having a lower bound value

higher than the initial solution. If it does reach the bottom, a feasible solution has

been found. If the new solution has a lower objective value than the initial solution,

the latest solution is kept as the current best-known solution. On the other hand

if the algorithm does encounter a node which has a lower bound greater than the

best-known solution, then that node and all the nodes underneath it are eliminated

from the search. The algorithm then a"n Ji. the next node to the right of the node

that did not meet the lower bound criteria. Once all nodes at a given level have been

analyzed, the algorithm moves up to the previous level and begins searching on the

next node to the right of the last node analyzed on that level.

We discuss different modifications that may be implemented on the original B&B

algorithm to help increase the rate of convergence. The B&B algorithm's performance

is directly related to the tightness of the upper and lower bounds. The rest of this

section addresses the problem of obtaining a tighter upper bound. The objective is to

obtain a good solution as early as possible. By having a low upper bound early in the

procedure, we are able to eliminate more branches and guarantee an optimal solution

in a shorter amount of time. The modifications that we introduce are sorting the

nodes in all the tree levels and performing a local search algorithm that guarantees

local optimality.









3.3.2 Sorting

There are two v--v to sort the index-based tree. The first is to sort every

level of the tree once before the branch and bound algorithm begins. By using this

implementation, the sorting complexity is minimized. However, the drawback is that

infeasible cost coefficients are mixed in with the feasible ones. The algorithm would

have to perform a large number of feasibility checks whenever a new coefficient is

needed from each level.

The second way to sort the tree is to perform a sort procedure every time a cost

coefficient is chosen. At a given tree level, a set of coefficients that are still feasible

to the partial solution is created and sorted. Finding coefficients that are feasible is

computationally much less demanding than checking if a particular coefficient is still

feasible. The drawback with the second method is the high number of sorting proce-

dures that need to be performed. For our test problems, we have chosen to implement

the first approach, which is to perform a single initial sorting of the coefficients for

each tree level. This choice was made because the first method performed best in

practice for the instances we tested.

3.3.3 Local Search

The local search procedure improves upon the best-known solution by searching

within a predefined neighborhood of the current solution to see if a better solution

can be found. If an improvement is found, this solution is then stored as the current

solution and a new neighborhood is searched. When no better solution can be found,

the search is terminated and a local minimum is returned.

Because an optimal solution in one neighborhood definition is not usually op-

timal in other neighborhoods, we implement a variable neighborhood approach. A

description of this metaheuristic and its applications to different combinatorial opti-

mization problems is given by Hansen and Mladenovi6 [47]. Variable neighborhood

works by exploring multiple neighborhoods one at a time. For our branch and bound









algorithm, we implement the intrapermutation 2- and n-exchanges and the interper-

mutation 2-exchange presented by Pasiliao [84]. Starting from an initial solution, we

define and search the first neighborhood to find a local minimum. From that local

minimum, we redefine and search a new neighborhood to find an even better solution.

The metaheuristic continues until all neighborhoods have been explored.

3.4 Computational Experiments

In this section, the computational experiments performed are explained. In the

first subsection, we describe the experimental procedures employ, -1 Then, in latter

subsections, the results from the experiments are presented and discussed. The results

include mean optimal costs and their standard deviation, for each type of problem

and size. In the last subsection we present some interesting results, based on curve

fitting models.

3.4.1 Experimental Procedures

The experimental procedures involved creating and exactly solving MAPs using

the B&B algorithm described in the preceding section. There were at least 15 runs

for each experiment where the number of runs was selected based on the practical

amount of time to complete the experiment. Generally, as the size of the problem

increased, the number of runs in the experiment had to be decreased. Also, as the

dimension, d, of the MAP increased, the maximum number elements, n, decreased.

Tables 3-2 and 3-3 provide a summary of the size of each experiment for the various

types of problems.

The time taken by an experiment ranged from as low as a few seconds to as high

as 20 hours on a 2.2 GHz Pentium 4 processor. We observed that problem instances

with standard normal assignment costs took considerably longer time to solve; there-

fore, problem sizes and number of runs per experiment are smaller. The assignment

costs cl...id for each problem instance were drawn from one of three distributions. The










Table 3-2: Number of runs for each experiment with uniform or exponential assign-
ment costs.

n\d 3 4 5 6 7 8 9 10
2 1000 1000 1000 1000 1000 1000 1000 500
3 1000 1000 1000 1000 1000 1000 1000 100
4 1000 1000 1000 1000 1000 500 500 50
5 1000 1000 1000 500 200 200 100 20
6 1000 1000 500 500 100 50 20
7 1000 1000 500 200 50 20 15
8 1000 1000 200 50 20 15
9 1000 1000 50 20 15
10 1000 1000 20 15
11 500 500 20
12 500 500 20
13 200 200 15
14 100 50
15 100
16 50
17 50
18 30
19 20
20 15


first distribution of assignment costs used was the uniform U[O, 1]. The next distribu-

tion used was the exponential with mean one, being determined by cil... = In U.

Finally, the third distribution used was the standard normal, N(0, 1), with values

determined by the polar method [63] as follows:

1. Generate U1 and U2, for U1, U2 ~ U[0, 1].

2. Let V1 = 2U1 1,V2 2U2 1, and W V12 + V22.

3. If W > 1, go back to 1, else c,...i = VI.

3.4.2 Mean Optimal Solution Costs

A summary of results for MAPs is provided in Tables 3-4, 3-5, and 3-6. We

observe that in all cases the mean optimal cost gets smaller as the size of the MAP

increases. Figure 3-2 shows the plots for problems with dimension d = 3, d = 5,

d = 7 and d = 10, as examples for the exponential case (plots for the uniform case

are similar). We observe that plots for higher dimensional problems converge to zero

for smaller values of n. This is emphasized in the surface plot, Figure 3-3, of a subset

of the data. Figure 3-4 shows plots for problems with the same number of dimensions











Table 3-3: Number

n\d
2
3
4
5
6
7
8
9
10
11
12
13
14


of runs

3
1000
1000
1000
1000
1000
1000
1000
1000
1000
500
50
15
15


for each experiment standard normal assignment costs.

4 5 6 7 8 9 10
1000 1000 1000 1000 1000 1000 500
1000 1000 1000 1000 1000 1000 500
1000 1000 1000 1000 500 50 15
1000 1000 500 200 20 15
1000 500 100 50 15
1000 50 20 15
1000 20 15
100 15
20
15


as the problems in Figure 3-2, but for the standard normal case. Different from the

uniform and exponential cases, the mean optimal solution costs appear to approach

-oc with increasing n.


0.9

0.8

0.7
.n1
0


cV
S0.6

E 0.5

0-0.4
0
S0.3
()
E 0.2

0.1

0


2 7 12 17 2
I number of elements


-3 DAP
--5 DAP
-7 DAP
-x-10 DAP


Figure 3-2: Plots of mean optimal costs for four different sized MAPs with exponen-
tial assignment costs.


We observe that in the uniform and exponential cases the standard deviation of

optimal costs converges to zero as the size of the MAP gets larger. Clearly, this just

confirms the .,-''iiii.l i ic characteristic of the results. However, a trend is difficult to













Table 3-4: Mean < :.i': : costs for dilfrernti sizes of MA[Ps with independent assign-

nient costs that are uniform in [0, 11.


0.54078
0.480825
0.41716
0.374046
0.334805
0.30329
0.277139
0.252156
0.237884
0.216287
0.205552
0.185769
0.180002
0.16832
0.162104
0.14787
0.14583
0.129913


0.295578
0.209272
0.151543
0.114551
0.0897928
0.0724017
0.0587219
0.0486118
0.0419366
0.035617
0.0310225
0.02696


0.155189
0.0884739
0.0549055
0.0357428
0.02465
0.0175195
0.012982
0.00961
0.00762
0.006025


0.0853185
0.0386061
0.019867
0.011516
0.0067875
0.0041348
0.00253
0.00168


Table 3 5: :: optimal costs for different sizes of MAPs with independent assign-

ment costs that are exponential with mean 1.


0.63959
0.531126
0.454308
0.396976
0.349543
0.310489
0.28393
0.263187
0.238954
0.218666
0.203397
0.193867
0.181644
0.172359
0.161126
0.15081
0.144787
0.134107


0.319188
0.212984
0.155833
0.116469
0.0909123
0.0723551
0.0595148
0.0149353
0.041809
0.03546241
0.030967
0.0279


0.165122
0.0903552
0.0548337
0.0355034
0.0251856
0.0175745
0.01233
0.009165
0.007305
0.005385
0.004,4667


0.0829287
0.0391171
0.020256
0.0116512
0.0068995
0.0044
0.002665
0.0019


Table 3-6: Mean < :.i': : costs for different sizes of MAPs with independent assign-

nient costs that are standard normal.


-3.41537
-5.6486
-8.00522
-10.6307
-13.2918
-16.1144
-18.9297
-21.7916
-24.7175
-27.9675
-30.9362
-34.4204


-4.59134
-7.52175
-106145
-13.9336
-17.2931
-20.8944
-24.5215
-28.6479
-31.9681


-5.57906
-9.05299
-12.6924
-16.5947
-20.6462
-24.7095
-28.7188


-6.44952
-10.3701
-145221
-18.7402
-23.4246
-28.1166











Exponentially Distributed Cost Coefficients


0.9


0.8
0.7
0.6
0.5
0.4


d, dimension


9


n, number of elements


Figure 3-3: Surface plots of mean optimal costs for 3 < d < 10 and 2 < n < 10 sized
MAPs with exponential assignment costs.


5

-10

-15

-20

25

-30

35

-An


n, number of elements
2 4 6 8 10


12 14


--3 DAP
---5 DAP
-A-7 DAP
-X-10 DAP


Figure 3-4: Plots of mean optimal costs for four different sized MAPs with standard
normal assignment costs.


Mean Optimal Cost


/










detect for standard deviation of optimal costs in the standard normal case. Figure 3-5

shows the plots for the three, five, seven and ten dimensional problems, as examples,

for the exponential case (plots for the uniform case are similar).


0.6


0 0.5
0
S0.4
w -4-3 DAP
*' ,0.3 ---5 DAP
o ---7 DAP
** -X--10 DAP
0 0.1
0.


^ 01 i "------


2 4 6 8 10 12 14 16 18 20
n, number of elements



Figure 3-5: Plots of standard deviation of mean optimal costs for four different sized
MAPs with exponential assignment costs.


No clear trend is given in Figure 3-6 which shows the plots for the same dimensional

problems but for the standard normal case.

3.4.3 Curve Fitting

Curve fits for the mean optimal solution costs were performed for the three types

of problems using a least squares approach. The solver tool in Microsoft's Excel was

used to minimize the sum of squares. Several nonlinear models were tested for the

purpose of developing a model to estimate the mean optimal cost, z.. The tested

models include the following

Power Fit, z = An

Shifted Power Fit, = A(n + B)c

Scaled Power Fit, (An + B)c











1.4

1.2


0
o -4-3 DAP
S0.8 --5 DAP
E ---7 DAP
0.6 --10 DAP

o 0.4

0.2


2 4 6 8 10 12 14
n, number of elements



Figure 3-6: Plots of standard deviation of mean optimal costs for four different sized
MAPs with standard normal assignment costs.


Exponential, = Ae"B

Reciprocal Quadratic, z* A + Bn + Cn2

In each case the fit was calculated by fixing d and varying n. For the uniform

and exponential problems the Scaled Power Fit was found to be the best model.

For the standard normal problems the Shifted Power Fit was used. The results of

curve fitting are shown in Tables 3-7, 3-8, and 3-9. We observe that curves fit

surprisingly well to the collected data. Figure 3-7 is a plot of the curve fitting model

and observed data for the exponential case where d = 3. Note that the curves are

nearly indistinguishable. This is typical for most problem sizes. A closer analysis of

the curve fitting parameters for both uniform and exponential type problems indicates

that as the dimension of the MAP increases, the curve fitting parameter C approaches

-(d 2). A heuristic argument of why this is so is given in the following.

Consider the case of uniformly distributed cost coefficients. For each level of

the index tree representation of the MAP, the expected value of the minimum order







35

Table 3 7: Curve fitting results for fitting the form (A! + B3) to the mean optimal
costs for MAPs with uniform assignment costs.

d A B C Sum of Squares
3 0.102 1.133 -1.764 8.80E-04
4 0.183 .* -2.932 7.74E-05
5 0.319 0.782 -3.359 i ::7
6 0.300 0.776 -4.773 5.77E-07
7 0.408 0.627 -4.997 .
080.08 0621 -6.000 7.91E-07
9 0.408 0.621 -7.000 3.44E-07
10 0.408 0.621 -8.000 9.50E-07

Table 3 8: Curve fitting results for fitting the form (. + B)c to the mean optimal
costs for MAPs with i onential assignment costs.

d A B C Sum of Squares
3 0.300 0.631 -1.045 5.26E-05
4 0.418 0.550 -1.930 :'i
5 0.406 0.601 -3.009 2.40E-06
6 0.420 0.594 -3.942 8.39E-08
7 0.414 0601 -5.001 9.42E-07
8 0.413 0.617 -5.999 9.45E-07
9 0.418 0.600 -7.000 1.94E-07
10 0.114 0.607 -8.000 6.68E-07


statistic is given by E[X(1)] = l/(n- + 1) as there are nd-1 coefficients on each level

of the tree. And as there is one coefficient from each of the n levels in a feasible

solution we may expect = O(n n-(d-1)) O(n-(d-2)). The same argument can

be made for the exponential case where E[X(1)] = 1/n 1

Again using a least squares approach, if we rebuild the curve fitting models for

the uniform and exponential cases by fixing C = 2 d, we find, as expected, the

lower dimension models result in higher sum of squares. The worst fitting model is

that of the uniform case with d = 3. In this case the sum of squares increases from

8.80E 04 to 3.32E 03 and the difference in the model estimate and actual results

for n = 3 increases from 2.;:'. to 5'. Although we believe fixing C = 2 d can

provide adequate fitting models, in the remainder of this chapter we continue to use

the more accurate models (where C is not fixed to C = 2 d); however, it is obvious

the higher dimension problems are unaffected.








36

Table 3-9: Curve fitting results for fitting the form A(n + B)c to the mean optimal
costs for MAPs with standard normal assignment costs.

d A B C Sum of Squares
3 -1.453 -0.980 1.232 7.27E-02
4 -1.976 -0.986 1.211 1.54E-01
5 -2.580 -1.053 1.164 2.85E-02
6 -2.662 -0.915 1.204 1.68E-02
7 -3.124 -0.956 1.174 1.20E-03
8 -3.230 -0.882 1.194 3.13E-03
9 -3.307 -0.819 1.218 1.71E-03
10 -3.734 -0.874 1.187 1.52E-04


0.9
0.8
0.7
0.6
o
0.5
S0.4 observed data
0 0.3
0 0.3 -... fitted data
0.2
0.1
0
7 12 17
n, number of elements



Figure 3-7: Three dimensional MAP with exponential assignment costs. Plot includes
both observed mean optimal cost values and fitted values. The two lines are nearly
indistinguishable.


An obvious question to ask is what happens with variations of the distribution

parameters. For example, what is the numerical estimation of z* when the cost

coefficients are distributed as uniform on [a, b] or exponential with mean A? We

propose without proof the following numerical models to estimate z*.

For cost coefficients that are uniform on [a, b], the curve fit or numerical esti-

mation is z* z = an + (b a)(An + B)c, using the curve fit parameters for the

uniform case on [0, 1] found in Table 3-7. For cost coefficients that are exponential

with mean A, the curve fit is z* ~ = A(An + B)c using the curve fit parameters

for the exponential case with A = 1 found in Table 3-8.







37

Table 3-10: Estimated and actual mean optimal costs from ten runs for variously sized
MAPs developed from different distributions. Included are the average difference and
largest difference between estimated mean optimal cost and optimal cost.

d n Distribution Z* z* Ave A Max A
with Parameters
3 12 Uniform on [5,10] 61.1 61.1 0.143 0.428
3 20 Expo, A 3 0.415 0.404 0.0618 0.154
5 9 N(p 0, a 3) -86.4 -86.5 1.62 3.48
5 12 Uniform on [-1,1] -12 -12 1.65E-03 3.16E-03
7 5 N(p =5, a 2) -7.24 -7.27 0.448 0.73
7 7 Expo, A 10 1.90E-02 1 '-.-' 2.62E-03 5.47E-03
8 6 Uniform on [10,20] 60 60 0.003 0.008
8 8 Expo, A 1.5 4.13E-04 3.07E-04 1.15E-04 2.30E-04
9 5 N(t -5, a 2) -62.8 -63.2 0.944 2.26
9 7 Uniform on [-10,-5] -70 -70 3.60E-04 6.70E-04
10 4 N(p 1 ,a 4) -53.8 -53.3 0.831 2.12
10 5 Expo, A 2 7.57E-04 8.00E-04 1.10E-04 4.03E-04


The situation is just a bit more involved for the normal case. Consider when the

mean of the standard normal is changed from 0 by an amount p and the standard

deviation is changed by a factor a. That is the cost coefficients have the distribution

N(p,, a). Then z* z = nup + aA(n + B)c using the curve fit parameters found in

Table 3-9.

To assist in validating the numerical estimation models discussed above, experi-

ments were conducted to compare the numerical estimates of the mean optimal costs

and results of solved problems. The experiments created ten instances of different

problem sizes and of different distributions and solved them to optimality. A variety

of parameters were used for each distribution in an effort to exercise the estimation

models. In the first experiment, we report mean optimal solution, estimated mean

optimal solution, the max A, and mean A where A = Iz z(I)l. That is, A for

a problem instance is the difference between the predicted or estimated mean opti-

mal cost and the actual optimal cost. Results of these experiments are provided in

Table 3-10. We observe that the numerical estimates of the mean optimal costs are

quite close to actual results.











Similar to Figure 3-7, Figures 3-8, 3-9 and 3-10 have plotted results of z* and

z* (fitted data) for random instances of different sized problems. As in the above

experiments, the number of runs is limited to ten for each problem size. As the plots

of z* and z* are close to each other, this further validates the numerical models for

estimating z*.


4 dimension, Uniform on [10,20]


--observed data
......... fitted data


0 5 10
n, number of elements


8 dimension, Uniform on [10,20]


--observed data
.........fitted data


0 2 4 6 8 10
n, number of elements


Figure 3-8: Plots of fitted and mean optimal costs from ten runs of variously sized
MAPs developed from the uniform distribution on [10, 20]. Note that the observed
data and fitted data are nearly indistinguishable.


4 dimension, Exponential, mean=3


5 10
n, number of elements


8 dimension, Exponential, mean=3


- observed data
f......itted data


0 2 4 6
n, number of elements


Figure 3-9: Plots of fitted and mean optimal costs from ten runs of variously sized
MAPs developed from the exponential distribution with mean three.



3.5 Algorithm Improvement Using Numerical Models

The numerical estimates of the mean optimal cost can be used to accurately

predict the optimal solution cost of a random instance of an MAP that is constructed

from a uniform, exponential or normal distribution. However, we still lack a solution.


--observed data
........ fitted data











4 dimension, Normal [5,2] 8 dimension, Normal [5,2]
8 4
6 2
-4 0
S2 2 4
05 10 -observed-- (
S 10 1 observeddata observed data
'. fitted data '. -- fitted data

E -8 1
12 -12
n, number of elements n, number of elements


Figure 3-10: Plots of fitted and mean optimal costs from ten0 runs of variously sized
MAPs developed from a normal distribution, N(1 = 5, a = 2).


In this section, we investigate whether the numerical estimates can be used to improve

a branch and bound (B&B) exact solution method.

3.5.1 Improvement of B&B

The B&B solution method under consideration is that described in this chapter,

Section 3.3. Recall that the B&B performs best by establishment of a tight upper

bound early in the process. A tight upper bound allows significant pruning of the

branches of the search tree. We consider the use of the numerical estimates to set

tighter upper bounds than would be available through other primal heuristics. An

advantage of the primal heuristic is, of course, a solution is at hand; whereas, the

numerical estimate is a bound only with no solution. The heuristic used in Section 3.3

randomly selects a starting solution and then performs a variable local neighborhood

search to find a local minimum. Alternatively, we also consider the global greedy and

a variation of the maximum regret approaches as -i-i.:: -1. I by Balas and Saltzman

[10]. In the global greedy approach, a starting solution is constructed step-by-step

by selecting the smallest feasible cost coefficient then a variable local neighborhood

search is applied to find a local minimum. For maximum regret, a feasible solution

is constructed as follows. The difference between the two smallest feasible costs

associated with each level of the index tree is calculated. This difference is called

the regret as it represents the penalty for not choosing the smallest cost in the row.







40

Table 3-11: Results showing comparisons between three primal heuristics and the
numerical estimate of optimal cost for several problem sizes and types. Shown are
the average feasible solution costs from 50 runs of each primal heuristic on random
instances.
d n Distribution Random Greedy Max Regret Numerical
with Parameters Estimate
6 10 Uniform on [0,1] 0.530 0.216 0.165 0.00177
7 7 Uniform on [0,1] 0.433 0.201 0.182 0.00195
8 6 Uniform on [0,1] 0.429 0.186 0.168 0.0019
9 4 Uniform on [0,1] 0.320 0.218 0.214 0.00341
10 4 Uniform on [0,1] 0.283 0.219 0.216 0.00152
6 10 Expo, A 1 0.611 0.226 0.2426 0.00251
7 7 Expo, A 1 0.490 0.244 0.216 0.00190
8 6 Expo, A 1 0.430 0.217 0.175 0.00114
9 4 Expo, A 1 0.385 0 _'.7 0.270 0.00318
10 4 Expo, A 1 0.320 0.224 0.215 0.00145
6 7 N( = 0, = 1) -12.91 -21.29 -21.57 -23.40
7 6 N( = 0, a 1) -12.91 -18.51 -18.97 -20.89
8 5 N( = 0, a= 1) -8.99 -15.77 -16.08 -17.51
9 4 N( = 0, a 1) -6.99 -11.67 -11.883 -13.53
10 4 N( = 0, a 1) -7.00 -12.60 -12.67 -14.44


The smallest feasible cost in the row with the largest regret is selected. This differs

from the approach by Balas and Saltzman [10] where they consider every row in the

multi-dimensional cost matrix, whereas we consider only the n rows in the index tree.

We took this approach as a trade-off between complexity and quality of the starting

solution. Table 3-11 provides a comparison of starting solution cost values for the

three primal heuristics described above along with a comparison of the numerical

estimate of the optimal cost for various problem sizes and distribution types. The

table shows the results of the average of 50 random generated instances.

In terms of an upper bound, the results of Table 3-11 indicate that, generally,

the greedy primal heuristic is better than the random heuristic and max regret is

better than greedy. For the uniform and exponential cases, the numerical estimate

of optimal costs is clearly smaller than any of the results of the heuristics. In the

normal cases, the numerical estimate is not significantly smaller. For the uniform

and exponential cases, it appears much is to be gained by somehow incorporating the

numerical estimate into an upper bound.









We propose using a factor r > 1 of the numerical estimate as the upper bound. If

a feasible solution is found, the new solution serves as the upper bound. If a feasible

solution is not found, then the estimated upper bound is incremented upwards until a

feasible solution is found. This process guarantees an optimal solution will be found.

Figure 3-11 is fundamentally the same as Figure 3-1 except for the outside loop

which increments the estimated upper bound upward until a feasible solution is found.

procedure IndexBB(L)
1 solutionfound = false
2 while solutionfound = false do
3 Z* = Z* 7
4 for i =,...,n do ki 0
5 S --0
6 i-1
7 while i > 0 do
8 if ki = ILil then
9 S <-- S\{sj}
10 ki 0
11 i--i-1
12 else
13 k = ki + 1
14 if Feasible(S, Li,k ) then
15 S SU Li,ki
16 if LB(S) < z* then
17 if i= n then
18 S -S
19 z <- Objective(S)
20 solutionfound = true
17 else
18 i --i+1
19 else
20 S S\{sj}
21 end
22 end
22 return(S, Z)
end IndexBB

Figure 3-11: Branch and bound on the index tree.


The trade-off which must be considered is if the upper bound is estimated too

low and incremented upwards too slow, then it may take many iterations over the









index tree before a feasible solution is found. However, no benefit is gained by setting

the upper bound too high. We found through less-than-rigorous ,i i,', J-i that r set to

a value such that the upper bound is incremented upward by one standard deviation

of the optimal cost (see Figures 3-5 and 3-6) is a nice compromise.

3.5.2 Comparison of B&B Implementations

Table 3-12 compares the performance of the B&B algorithm using the random

primal heuristic for a starting upper bound versus using the maximum regret heuris-

tic versus using a numerical estimate for the upper bound. The table shows the

average times to solution of five runs on random instances of various problem sizes

and distribution types. In the uniform and exponential cases, we observe that B&B

using maximum regret generally does slightly better than using a random starting

solution. We also observe the approach of using a numerically estimated upper bound

significantly outperforms the other approaches in solving problems with uniformly or

exponentially distributed costs. However, there is no clear difference between the

approaches when solving problems with normally distributed costs. This is explained

by the small differences in the starting upper bounds for each approach.

3.6 Remarks

In this chapter we presented experimental results for the .- i-! ,i1 Iic value of the

optimal solution for random instances of the MAP. The results lead to the following

conjectures which will be addressed in detail in C'i ipter 4.


Conjecture 3.3 Given a d-dimensional MAP with n elements in each dimension,

if the nd cost coefficients are independent exponr ,lI ll;i distributed random variables

with mean 1 or independent unifoti,,li distributed random variables in [0,1], z* 0

as n --- o or d -- oo.










Table 3 12: Average time to solution in seconds of solving each of five randomly
generated problems of various sizes and I :. TI: experiment involved : '::" the


F solution algorithm with

Distribution
with ..
Uniform on [0,1
Uni form on 10,1
Uniform on [0,1
Uniform on [0,11
Uniform on [0,1]
Expo, A = 1
Expo, A = 1
Expo, A 1
Expo, A 1
Expo, A = 1
N(p 0, a 1)
N( 0, a 1)
N(p 0, 1)
N(pf 0, .. 1)
N(y aO, 1)


.: starting upper bounds developed in three


.. Max : _,et

1 1311
19.1 19.2
20.5 ',: 4
0.3 0.29
1.15 1.12
1279 1285
25.5 25.8
21.8 24.5
0.24 0.23
1.67 1.66
54.9 47.3
89.9 .
24.7 24.6
1.25 1.23
30.7 :


Numerical

795
13.9
13.1
0.13
0.4
1201
17.8
13.4
0.1
0.57
54.2
89.2
24.6
1.24
30.7


Conjecture 3.4 Given a d-dimensional MAP with n elements in each dimension, if

the nd cost coefficients are independent standard normal random variables, z* -- -oo

as n oo or d -- oo.


We also presented in this chapter curve fitting results to accurately estimate the

mean optimal costs of variously sized problems constructed with cost coefficients in-

dependently drawn from the uniform, exponential or normal distributions. Of high

interest of course is how numerical estimates of mean optimal cost can be used to

improve existing solution algorithms or is they can be used to find new solution algo-

rithms. To this end, we have shown that using numerical estimates can significantly

improve the performance of a B&B exact solution method.















CHAPTER 4
PROOFS OF ASYMPTOTIC CHARACTERISTICS OF THE MAP

4.1 Introduction

The experimental work detailed in C!i ipter 3 leads to conjectures concerning the

.Ii-mptotic characteristics of the mean optimal costs of randomly generated instances

of the MAP where costs are assigned independently to assignments. In this chapter,

we provide proofs of more generalized instances of Conjecture 3.3 and prove Conjec-

ture 3.4. The proofs are based on building an index tree [87] to represent the cost

coefficients of the MAP and then selecting a minimum subset of cost coefficients such

that at least one feasible solution can be expected from this subset. Then an upper

bound on the cost of this feasible solution is established and used to complete the

proofs. Throughout this chapter we consider MAPs with n elements in each of the d

dimensions.

Before presenting the theorems and their proofs concerning the .i-vmptotic na-

ture of these problems, we first consider a naive approach [28] to establishing the

.,-vmptotic characteristics based on some greedy algorithms.

4.2 Greedy Algorithms

Consider the case of the MAP where cost coefficients are independent exponen-

tially distributed random variables with mean 1. By Conjecture 3.3 the mean optimal

costs are thought to go to zero with increasing problem size. Suppose we consider

the solution from a greedy algorithm. As the solution serves as an upper bound to

the optimal solution, we can try to prove the conjecture if we can show the mean of

the sub-optimal solutions goes to zero with increasing problem size. However, as will

be shown this is difficult with two common greedy algorithms.









4.2.1 Greedy Algorithm 1

The first algorithm that we consider uses the index tree data structure proposed

by Pierskalla [87] to represent the cost coefficients of the MAP. There are n levels

in the index tree with nd-1 nodes on each level for a total nd nodes. Each level

of the index tree has the same value in the first index. A feasible solution can be

constructed by first starting at the top level of the tree. The partial solution is

developed by moving down the tree one level at a time and adding a node that is

feasible with the partial solution. The number of nodes at level i that are feasible to

a partial solution developed from levels 1, 2,..., i 1 is (n i + l)d-1. A complete

feasible solution is obtained upon reaching the bottom or nth-level of the tree.

The proposed greedy algorithm is as follows:

Input MAP of dimension d and n elements in each dimension in the form of an index

tree.

Build a partial solution, S, i = 1, by choosing the smallest cost coefficient from row

1 of the tree.

For i = 2,...,n, continue to construct a solution by choosing the smallest cost

coefficient in row i of the tree that is feasible with Si-_ constructed from rows 1,..., i-

1.

We wish to calculate the expected solution cost from this algorithm for the

MAP constructed from i.i.d. exponential random variables with mean 1. Let the

mean solution cost resulting from the algorithm be represented by z*. Suppose that

X, X2,..., Xk are k i.i.d. exponential random variables with mean 1. Let X(i) be the

ith smallest of these. Applying order statistics [33], we have the following expression

for the expected minimum value of k independent identically distributed random

variables: E[X(1)]= 1/k.

We may now construct a feasible solution using the above greedy algorithm. We

do so by recalling that the number of nodes that are feasible at level i + 1 to a partial









solution developed down to level i, for i = 1, 2,... n is (n i)d1. Considering this

and the fact that cost coefficients are independent, the expected solution cost of S1

is the expected solution cost of S2 is I- + and so forth. Therefore, we

find
n- 1
S= ( )d-1 (4.1)

> (4.2)


where equation (4.2) holds because the n-th term of equation (4.1) is one.

Since z* > 0, we conclude this greedy approach cannot be used to prove Conjec-

ture 3.3. However, maybe a more global approach will work.

4.2.2 Greedy Algorithm 2

The following algorithm is described by Balas and Saltzman [10] as the GREEDY

heuristic. The algorithm is as follows:

Input MAP of dimension d and n elements in each dimension as matrix A.

For i 1,... n, construct the partial solution Si by choosing the smallest element

in matrix A and then exclude the d rows covered by this element.

Using this covering approach, we see the number of nodes that are feasible to a

partial solution developed up to iteration i, for i = 1, 2,..., n is (n- )d. For example,

all nd cost coefficients are considered in the first iteration. The next iteration has

(n )d nodes for consideration. The expected solution cost of S1 is 1/nd. The

expected solution cost of S2 is 1/nd + 1/nd + 1/(n )d. The extra 1/nd appears

in the expression because, in general, the expected minimum value of the uncovered

nodes is at least as much as the expected minimum value found in the previous

iteration. We could now develop the expression for z*; however, we note that the

algorithm's last iteration will consider only one cost coefficient. Therefore, again, we

have the result that z* > 1 when using this algorithm.









We conclude that these simple greedy approaches cannot be used to prove the

conjectures concerning the .,-iiiiil .1 ic characteristics of the MAP. In the next sec-

tions, we resort to a novel probabilistic approach.

4.3 Mean Optimal Costs of Exponentially and Uniformly Distributed
Random MAPs

To find the .,-vmptotic cost when the costs are uniformly or exponentially dis-

tributed, we use an argument based on the probabilistic method [7]. Basically, we

show that, for a subset of the index tree, the expected value of the number of feasible

paths in this subset is at least one. Thus, such a set must contain a feasible path and

this fact can be used to give an upper bound on the cost of the optimum solution.

This is explained in the next proposition.


Proposition 4.1 Using an index tree to represent the cost coefficients of the MAP,

Tr IJ..i;,, J select a different nodes from each level of the tree and combine these nodes

from each level into set A. A is expected to produce at least one feasible solution to

the MAP when

d-1 and A na (4.3)


Proof: Consider there are nd-1 cost coefficients on each of the n levels of the index

tree representation of an MAP of dimension d and with n elements in each dimension.

Now consider there are (n d-)" paths (not necessarily feasible to the MAP) in the

index tree from the top level to the bottom level. The number of feasible paths (or

feasible solutions to the MAP) in the index tree is (n!)d-1. Therefore, the proportion

Q of feasible paths to all paths in the entire index tree is

(n!)d-1 (4.4)
(nd-I)n

Create a set A of nodes to represent a reduced index tree by selecting a nodes

randomly from each level of the overall index tree and placing them on a corresponding









level in the reduced index tree. The number of nodes in A is obviously na. For this

reduced index tree of A, there are a" paths (not necessarily feasible to the MAP)

from the top level to the bottom level. Since the set of nodes in A were selected

randomly, we may now use Q to determine the expected number of feasible paths in

A by simply multiplying Q by the number of all paths in the reduced tree of A. That

is


E[number feasible paths in A] = Qa".


We wish to ensure that the expected number of feasible paths A is at least one. Thus,

over all possible choices of the n subsets of a elements, there must be one choice such

that there is one feasible path (in fact there may be many since the expected value

gives only the average over all possible solutions). Therefore,


Qa" > 1,


which results


a > -
(1

Incorporating the value of Q from (4.4) we get

d-1
a > (n!)dn


Therefore, since a must be an integer, we get (4.3).

We now take a moment to discuss the concept of order statistics. For more

complete information, refer to statistics books such as by David [33]. Suppose

that X1,X2,...,Xk are k independent identically distributed variables. Let X()

be the i-th smallest of these. Then X() is called the i-th order statistic for the set

{Xi,X2,..., Xk}.









In the rest of the section, we will consider bounds for the value of the a-th order

statistic of i.i.d. variables drawn from a random distribution. This value will be used

to derive an upper bound on the cost of the optimal solution for random instances,

when n or d increases. Note that, in some places (e.g., Equation (4.6)), we assume

that a = nd-1/n!d This is a good approximation in the following formulas because

(a) if n is fixed and d oo, then a -- oc, and therefore there is no difference

between a and nd- //n! d

(b) if d is fixed and n -- o, then a -+ ed-1. This is not difficult to derive, since

n n e
n! [() (2r)n (27rn)

But

(27n)' (27elog) n (27) 2n e 2n

and both factors in the right have limit equal to 1. However, ed-1 is a constant

value, and will not change the limit of the whole formula, as n -- co.


Proposition 4.2 Let z* = nE[X(a)], where E[X(a)] is the expected value of the ath

order statistic for each level of the index tree representation of the MAP. Then, zu

is an upper bound to the mean optimal solution cost of an instance of an MAP with

independent .:. ,.,/..ll'; distributed cost coefficients.

Proof: Consider any level j of the index tree and select the a elements with lowest

cost on that level. Let Aj be the set composed by the selected elements. Since the cost

coefficients are independent and identically distributed, the nodes in Aj are randomly

distributed across the level j. Now, pick the maximum node v E Aj, i.e.,


v = max{w Iw Aj}.


The expected value of v is the same as the expected value of the ath order statistic

among nd-1 cost values for this level of the tree. Since each level of the index tree









has the same number of independent and identically distributed cost values, we may

conclude that E[X(,)] is the same for each level in the index tree. By randomly

selecting a cost values from each of the n levels of the index tree, we expect to have

at least one feasible solution to the MAP by Proposition 4.1. Thus, it is clear that

an upper bound cost for the expected feasible solution is z, nE[X(a)].


Theorem 4.3 Given a d-dimensional MAP with n elements in each dimension, if

the nd cost coefficients are independent exponr ,i.: all distributed random variables

with mean A > 0, then z* 0 as n oo or d oo.

Proof: We first note that for independent exponentially distributed variables the

expected value of the ath order statistic for k i.i.d. variables is given by
E } A (4.5)
a-1
E[X(a)] = (4.5)
j=0

Note that (4.5) has a terms and the term of largest magnitude is the last term. Using

the last term, an upper bound on (4.5) is developed as

a-1
E[x(a)]u C (a t)
< k- (a -1)
j=o
aA
k- a+1

Now, using Propositions 4.1 and 4.2, the upper bound for the mean optimal solution

to the MAP with exponential costs may be developed as

SaA aA n
z -a+ 1 k-a k -

where k = nd- is the number of cost elements on each level of the index tree. To

prove z* 0, we must first substitute the values of k and a into (4.6), which gives

nA
z -< n (4.6)
(!) -










Let n = 7 and n! = 6, where 7 and 6 are some fixed numbers. Then (4.6) becomes


< 7A 7A
S d ~ d 1
6- ] 6-

as d gets large. Therefore,

lim z < lim = 0.
d Uc dc -
d->oo d-*oo d->

Now, let d 1 = 7, where 7 is some fixed number. Then (4.6) becomes

nA n\
z (n!)F 1 (n!)


as n gets large. Using Stirling's approximation n! \ (n/e)"f/2n,

n\ n\


nA
((n/e) (27n) '
nA


< r(4.7)


S )+(4.8)


where (4.7) holds because (27r) approaches one from the right as n 0- o. Con-

sidering that (1)7 is a constant and that the exponent to n is greater than one for

any 7 > 2, which holds because d > 3, then (4.8) will approach zero as n -+ oo.

Therefore, for the exponential case


lim z = 0 and lim z = 0 from above.
n->oo d- oo

Note that z* is bounded from below by zero because the lower bound of any cost

coefficient is zero (a characteristic of the exponential random variable with A > 0).

Since 0 < z* < z*, the proof is complete.









Theorem 4.4 Given a d-dimensional MAP with n elements in each dimension, if the

nd cost coefficients are independent unifoc 'in,,l distributed random variables in [0, 1],

then z* -+ 0 as n oo or d -+ oo.

Proof: For the case of the uniform variable in [0, 1], the expected value of the ath

order statistic for k i.i.d. variables is given by


E[Xa)] k +

Therefore, using Propositions 4.1 and 4.2, the upper bound on the mean optimal

solution for an MAP with uniform costs in [0, 1] is

nao na
-- < (4.9)
U k+1 k '

where k n d-1 is the number of cost elements on each level of the index tree. We

must now substitute the values of k and a into (4.9), which becomes


I < _n! (4.10)


Applying to (4.10) Stirling's approximation, in the same way as used in Theorem 4.3,

we see that z* -- 0 as n -- oo or d -- oc. Note again that z* is bounded from below

by zero because the lower bound of any cost coefficient is zero (a characteristic of the

uniform random variable in [0, 1]). Since 0 < z* < z*, this completes the proof.


Theorem 4.5 Given a d-dimensional MAP with n elements in each dimension, for

some fixed n, if the nd cost coefficients are independent, ; ,,'.: ,,,1 ..i distributed random

variables in [a, b], then z* -+ na as d -+ oo.

Proof: For the case of the uniform variable in [a, b], the expected value of the ath

order statistic for k i.i.d. variables is given by David [33]

(b- a)a
E[X(a)]= a+
k+1









Therefore, using Propositions 4.1 and 4.2, the upper bound on the mean optimal

solution for an MAP with uniform costs in [a, b] is

( (b a)a ( (b a)a
z = n a + k+ tn a+ k
k+1 k
(b a)na
Sna+ k (4.11)
k

where k = n'-1 is the number of cost elements on each level of the index tree. We

must now substitute values of k and a into (4.11), which results

S(b a)n
z _< na+ n (4.12)

It becomes immediately obvious from (4.12) that for a fixed n and as d oo, z* -

na. As z* < z- and na is an obvious lower bound for this instance of the MAP we

conclude that, for fixed n, z* na as d -- oo.

4.4 Mean Optimal Costs of Normal-Distributed Random MAPs

We want to now prove results similar to the theorems above, for the case where

cost values are taken from a normal distribution. This will allow us to prove Conjec-

ture 3.4. A bound on the cost of the optimal solution for normal distributed random

MAPs can be found, using a technique similar to the one used in the previous section.

However, in this case a reasonable bound is given by the median order statistics, as

described in the proof of the following theorem.


Theorem 4.6 Given a d-dimensional MAP, for a fixed d, with n elements in each

dimension, if the nd cost coefficients are independent standard normal random vari-

ables, then z* -oo as n o-o.

Proof: First note that for odd k = 2r + 1, X(r+1) is the median order statistic and

for even k = 2r, we define the median as ((X(r) + X(r+1)). Obviously, the expected

value of the median in both cases is zero. Let k nd-1 and note that, as n or d get

large, a < r for either odd or even case. Therefore we may immediately conclude









E[X(a)] < 0. Using Propositions 4.1 and 4.2, we see that z* < z = nE[X(,)] and
z* oo as n oo.


Theorem 4.7 Given a d-dimensional MAP with n elements in each dimension, for a
fixed n, if the nd cost coefficients are independent standard normal random variables,
then z* -oo as d oo.

Proof: We use the results from Cramir [32] to establish the expected value of the
ith order statistic of k independent standard normal variables. With i < k/2 we have

E[X() -2ogk+ log(log k) + log(4r) + 2(S1 C) 1
E[X( =-2 log- O( (4.13)
2 2logk log k

where S = + + + and C denotes Euler's constant, C 0.57722. As
d o o, k oc and the last term of (4.13) may be dropped. In addition, a slight

rearrangement of (4.13) is useful:

E[2ogk+ log(log k) log(4w) (S C) (4.14)
E[Xp 21o + + + (4.14)
2 2 1ogk 2 /2logk V/2logk

It is not difficult to see that as k oo, the sum of the first three terms of (4.14) goes

to -oo. Therefore, we consider the last term of (4.14) as k oo.
(Si -C) -C+ C+ log(i1)C
(Si c) -c + -' j j log(i t) C
2log k 2 log k /2 log k 2 log k
log(i ) C (4.
gk g(4.15)
2/2go v/log k









Noting that the second term of (4.15) goes to zero as k o0, and also making the

substitutions i = a = n /n and k = n1, we have


log d I 1 log dI
2/log k 21log -1 l og -1
log(nd- ) -log((n!) d1)
2 log -1
(d- 1)log() (d 1)log(n!A)
2log nd- 1

It is clear that for a fixed n, and as d oo, the right hand side of (4.16) approaches

zero. Therefore, using Propositions 4.1 and 4.2 we have < z* = nE[X(,)] and

E[X(a)] -oo for a fixed n and d -- oo. The proof is complete.

4.5 Remarks on Further Research

In this chapter, we proved some ..i-mptotic characteristics of random instances

of the MAP. This was accomplished using a probabilistic approach. An interest-

ing direction of research is how the probabilistic approach can be used to improve

the performance of existing solution algorithms. C'! lpter 5 applies the probabilistic

approach to reduce the cardinality of the MAP which, in turn, is then solved by

GRASP. We show this process can result in better solutions in less time for the data

association problem in the multisensor multitarget tracking problem.















CHAPTER 5
PROBABILISTIC APPROACH TO SOLVING THE MULTISENSOR
MULTITARGET TRACKING PROBLEM

5.1 Introduction

The data association problem arising from multisensor multitarget tracking ( S \ lTT)

can be formulated as an MAP. Although the MAP is considered a hard problem, a

probabilistic approach to reducing problem cardinality may be used to accelerate the

convergence rate. With the use of MSMTT simulated data sets, we show that the

data association problem can be solved faster and with higher quality solutions due

to these exploitations.

The MSMTT problem is a generalization of the single sensor single target track-

ing problem. In the MSMTT problem noisy measurements are made from an arbitrary

number of spatially diverse sensors (for example cooperating remote agents) regard-

ing an arbitrary number of targets with the goal of estimating the state of all the

targets present. See Figure 5-1 for visual representation of the problem. Because

of noise, measurements are imperfect. The problem is exacerbated with many close

targets and noisy measurements. Furthermore, the number of targets may change by

moving into and out of detection range and there are instances of false detections as

shown in Figure 5-2. The MSMTT solves a data association problem on the sen-

sor measurements and estimates the current state of each target based on the data

association problem for each sensor scan.

The combinatorial nature of the MSMTT problem results from the data asso-

ciation problem; that is, given d sensors with n target measurements each, how do

we optimally partition the entire set of measurements so that each measurement is

attributed to no more than one target and each sensor detects a target no more than










d sensors -- each have n measurements, not
necessarily the same
X sensor measurement 1, from sensor I
L1


Xi
Xj1 Xkl


Xk2

X2


Xk
- Xa-


9k


Figure 5-1: Example of noisy sensor measurements of target locations.


6Xil Xk2
Xj
Xi k


( ,i


False detection by sensors

'4
Xi Xi


Missed detection by sensor k

%1


Figure 5-2: Example of noisy sensor measurements of close targets. In this case there
is false detection and missed targets.

once? The data association problem maximizes the likelihood that each measurement

is assigned to the proper target. In MSMTT, a scan is made at discrete, periodic mo-

ments in time. In practical instances, the data association problem should be solved

in real time particularly in the case of cooperating agents searching for and identi-

fying targets. Combining data from more than one sensor with the goal of improving

decision-making is termed sensor fusion.

Solving even moderate-sized instances of the MAP has been a difficult task,

since a linear increase in the number of dimensions (in this case, sensors) brings an









exponential increase in the size of the problem. As such, several heuristic algorithms

[74, 90] have been applied to this problem. However, due to the size and complexity

of the problem, even the heuristics struggle to achieve solutions in realtime. In this

chapter we propose a systematic approach to reduce the size and complexity of the

data association problem, yet achieve higher quality solutions in faster times.

This chapter is organized as follows. We first give some background on data

association for the MSMTT problem. We then introduce a technique that may be used

to reduce the size of the problem. Following that, we discuss the heuristic, Greedy

Randomized Adaptive Search Procedure (GRASP), and how GRASP can be modified

to work effectively on a sparse problem. Finally, we provide some comparative results

of these solution methods.

5.2 Data Association Formulated as an MAP

Data association is formulated as an MAP where the cost coefficients are derived

from a computationally expensive negative log-likelihood function. The data asso-

ciation problem for the MSMTT problem is to match sensor measurements in such

a way that no measurement is matched more than once and overall matching is the

most likely association of measurements to targets. In the MAP, elements from d dis-

joint sets are matched in such a way that the total cost associated with all matching

is minimized. It is an extension of the two-dimensional assignment problem where

there are only two disjoint sets. For sets of size n, the two-dimensional assignment

problem has been demonstrated to be solvable in O(n3) arithmetic operations using

the Hungarian method [62], for example. However, the three-dimensional assignment

problem is a generalization of the three dimensional matching problem which is shown

by Garey and Johnson [44] to be NP-hard.

A review of the multitarget multisensor problem formulation and algorithms is

provided by Poore [89]. Bar-Shalom, Pattipati, and Yeddanapudi [11] also present a

combined likelihood function in multisensor air traffic surveillance.










Suppose that we have S sensors observing an unknown number of targets T. The

Cartesian coordinates of sensor s is known to be uw = [x,, ys, z,]', while the unknown

position of target t is given by ut = [xt, t, zt]'. Sensor s takes n, measurements, z,i,.

Since the measurements of target locations are noisy, we have the following expression

for measurement is from sensor s:


i h(wct, ws) + ws,is if measurement is is produced by target t
zs,is =
Vs,i, if measurement is is a false alarm

The measurement noise, u,ijs, is assumed to be normally distributed with zero mean

and covariance matrix Rs. The nonlinear transformation of measurements from the

spherical to Cartesian frame is given by hs((t, cs).

Consider the S-tuple of measurements Zil,i2,...is, each element is produced by a

different sensor. Using dummy measurements zs,o to make a complete assignment,

the likelihood that each measurement originates from the same target t located at Wt

is given.
S
A(Zi ,i2,...,is It) J [PD p(zs,, t)] 1 [1 PD1- (5.1)
S=l
where
S 0 if is = 0 (dummy measurement)
JS,is i
1 if 1i > 0 (actual measurement)

and PD, < 1 is the the detection probability for sensor m. The likelihood that the

set of measurements Zi1,i2...,is corresponds to a false alarm is as follows.

S
A( Z,i2,...,is 0) [Pp S"'i (5.2)
sl
ss1

where PF, > 0 is the probability of false alarm for sensor s.









The cost of associating a set of measurements Zi1,i2,...,i to a target t is given by

the likelihood ratio:

/ A (Zil,i2,...,is 't)
il,i2, ...,i A (Zil,i2,...,is lt 0)

sI P -(Zi, It) t PDI- (5.3)


This is the likelihood that Zi,,i,..., corresponds to an actual target and not a false

alarm.

Multiplying a large set of small numbers leads to round off errors as the product

approaches zero. To avoid this problem, we apply the logarithm function on both

sides. The cost of assigning a set of measurements Zi1,i2,...,i to a target t is given by

the negative logarithms of the likelihood ratio.


cI In A(Zi,i2 ...is I L]t) (5.4)
,\i, A(zi,i2...,isg t 0)

Instead of maximizing the likelihood function, we now try to minimize the negative

log-likelihood ratio. A good association would, therefore, have a large negative cost.

In practice, the actual location of target t is not known. If it were, then obtaining

measurements would be useless. We define an estimate of the target position as


Ljt = arg max A(Zi,,i2... is t).


The estimated target position maximizes the likelihood of a given set of measure-

ments.










The generalized likelihood ratio utilizes an estimated target position. Our neg-

ative log-likelihood ratio takes the following form


Cil,2,...,iS


6s,i In



s-1


2 PD, k [ -h(t, )' R [zi, -h(wit, us)


ln (1-PDs)


(5.5)

We can do a type of gating' by simply dropping any association with ci1,i2...,s > 0. A

feasible solution of the MT\ [ST problem assigns each measurement to no more than

one S-tuple or association Zi1,i2,...,i In other words, each measurement may not be

associated with more than one target. The result is a multidimensional assignment

problem that chooses tuples of measurements minimizing the negative log likelihood.

This is formally given as a 0-1 integer program.


min m
Zil,i2,...,iS


Cil,i2,...,is Pil,i2,...,is


S Pi,i2,..,is 1 V 1


Pi2,i2,..., < 1 V 1
i2 ,ik,.,isPS1

Vs


Pi,i2.is < 1 V is-
il~i2,..,is 1


- 1,2,... ,ni


1, 2, ... n ;

2,3,... ,S -1

= 1, 2, ... ns,


1 Gating is a process of initially excluding some measurement-target assignments
because of an arbitrarily large distance between the measurement and target.


(5.6)










w1 if the tuple Zi,,i2 ....is is assigned to the same target
where pi,,i2,,is =
0 otherwise

Zil,i2,..,i2 s = L{zi, z2,i2 zS,is

ni = min n, Vs = 1,2,..., S
S

Zs,, E R3

The objective is to find n, measurement associations so that the sum of all the neg-

ative log-likelihood costs are minimized. Measurements are assigned to a maximum

of one association or S-tuple. We define the Boolean decision variable piI,i2,...is, to be

zero when not all measurements {zi, 1, z2,i ... zs ,is are assigned to the same target.

The total number of possible partitions of s 1 n, measurements into T targets

is given by
n S i i S
M T .) .S!] for ns 5 T
WM = (5.7)

i= (ns-T (n-S+i)! r ns
where ns > max {ni, n2,..., ns-1}.

5.3 Minimum Subset of Cost Coefficients

Our objective is to preprocess the fully-dense data association problem by re-

ducing the size of the problem to a smaller subset. We would expect advantages such

as reduced storage requirements and less complexity for some algorithms. The devel-

opment of a minimum subset of cost coefficients is based on the work in C'! Ipter 4

(specifically Proposition 4.1) where we use the index tree representation of the MAP

and randomly select a nodes from each level of tree where

a d- (5.8)









When these a nodes from each level are combined into set A, we can expect this set

to contain at least one feasible solution to the MAP. For the generalized MAP with

dimension d and ni elements in each dimension i, i 1 2,..., d, and nl < n2 < <

nd, we can easily extend equation (5.8) by noting the number of feasible solutions is

ni(2 (nj2 Using this we find


a -H 2 ni (5.9)

Hi=2 (ni-n1)!

Consider an MAP where the cost coefficients of the index tree are sorted in non-

decreasing order for each level of the tree. If the cost coefficients are independent

identically distributed random variables then the first a cost coefficients are from

random locations at each level. Therefore, we may use Proposition 4.1 and conclude

we can expect at least one feasible solution in A. The cardinality of this set A

is substantially smaller than the original MAP which may result in faster solution

times. Table 5 1 shows a comparison of the size of A to the size of the three original

problems. Since the reduced set is made up of the smallest cost coefficients we expect

good solution values.

Table 5-1: Comparisons of the number of cost coefficients of original MAP to that
in A.
Number of Cost Coefficients
Problem Original MAP A
5x5x5 125 20
O1xl0xl0x10 10000 110
8x9xl0xl0x10 72000 72


Now consider an MAP where cost coefficients are not independent and identically

distributed. In real world applications, cost coefficients will most likely be dependent.

Consider, for example, a multisensor multitarget tracking situation where a small

target is tracked among other 1.',., targets. We can expect a higher noise/signal

ratio for the smaller target. Thus, cost coefficients associated with measurements









of the smaller target in the data association will be correlated to each other. In

the case of dependent cost coefficients, Proposition 4.1 cannot be directly applied

because the a smallest cost coefficients will not be randomly distributed across each

level of the index tree. However, using Proposition 4.1 as a starting point, consider

selecting some multiple, r > 1, of a cost coefficients from each level of a sorted index

tree. For example, select the first ra cost coefficients from each of the sorted levels

of the index tree to form a smaller index tree A. As r is increased, the cardinality

of A obviously increases but so does the opportunity that a feasible solution exists

in A. The best value of r depends upon the particular MAP instance, but we can

empirically determine a suitable estimate. In this chapter, we use a consistent value

of 7 = 10 wherever the probabilistic approach is used.

5.4 GRASP for a Sparse MAP

A greedy randomized adaptive search procedure (GRASP) [36, 37, 38, 4] is a

multi-start or iterative process in which each GRASP iteration consists of two phases.

In a construction phase, a random adaptive rule is used to build a feasible solution one

component at a time. In a local search phase, a local optimum in the neighborhood

of the constructed solution is sought. The best overall solution is kept as the result.

5.4.1 GRASP Complexity

It is easy to see that GRASP can benefit in terms of solution times for the MAP

by reducing the size of the problem. This can be seen by noting there are N cost

coefficients in the complete MAP where N H= 1 ni. As the complexity of the

construction phase can be shown to be 0(N) [4], a smaller N will directly reduce

the time it takes for each construction phase. As it is easy to see that reducing the

problem size to something less than N helps in the construction phase, it remains to

be seen how the local search phase is effected.

The local search phase of GRASP for the MAP often relies on the 2-exchange

neighborhood [74, 4]. A thorough examination of other neighborhoods for the MAP









is provided in the work by Pasiliao [84]. The local search procedure is as follows.

Start from a current feasible solution, examine one neighbor at a time. If a lower

cost is found adopt the neighbor as the current solution and start the local search

procedure again. Continue the process until no better solution is found. The size

of the 2-exchange neighborhood is d(n). As the size of the neighborhood is not

directly dependent upon N there appears, at first, to be no advantage or disadvantage

of reducing the number of cost coefficients in the problem. However, an obstacle

surfaces in the local search procedure because, as the construction phase produces

a feasible solution, we have no guarantee a neighbor of this solution even exists in

the sparse problem. A feasible solution consists of n1 cost coefficients. A neighbor in

the 2-exchange neighborhood has the same n1 cost coefficients except for two. In a

sparse MAP, most cost coefficients are totally removed from the problem. Therefore,

the local search phase first generates a potential neighbor and then must determine

whether the neighbor exists. In a complete MAP, the procedure may access the cost

matrix directly; however, the sparse problem cannot be accessed directly in the same

way. A simple procedure is to simply scan all cost coefficients in the sparse problem

to find the two new cost coefficients or to determine that one does not exist. This is

an expensive procedure. We propose a data structure which provides a convenient,

inexpensive way of evaluating existing cost coefficients or determining that they do

not exist.

5.4.2 Search Tree Data Structure

We propose to use a search tree data structure to find a particular cost coefficient

or determine that one does not exist in the sparse problem. The search tree has d + 1

levels. The tree is constructed such that there are ni branches extending from each

node at level i,i =1,2,... d. The bottom level, i = d + 1, (leaves of the tree)

contains each of the cost coefficients (if they exist). The maximum number of nodes

in the tree including the leaves is equal to 1 + Y7i 1 j1 nj and therefore, the time










to construct the tree is 0(N). An example of this search tree is given in Figure 5-3

for a complete 3x3x3 MAP. When searching for a particular cost coefficient, start at

level i = 1 and traverse down branch y, y = 0,..., ni where y is the element of the th

dimension for the cost coefficient. Continue this process until either level i d + 1 is

reached, in which case the cost coefficient exists, or a null pointer is reached, in which

case we may conclude the cost coefficient does not exist. It is obvious the search time

is 0(d).


Level 1
Dimension 1

0 1 2


Level 2
Dimension 2

0 1 2 0 1 \ 0 1 1


Level 3
Dimension 3

00 21200 1 0 0 \20 2 0 7 0 ,1
000 001 002 010 011 012 020 021 022 100 101 102 110 111 112 120 121 122 200 201 202 210 211 212 220 221 222


Figure 5-3: Search tree data structure used to find a cost coefficient or determine a
cost coefficient does not exist.


A search tree built from sparse data is shown in Figure 5-4. As an example of

searching for cost coefficient (001), start at level 1 and traverse down branch labelled

"0" to the node at level 2. From level 2, traverse again down branch labelled "0"

to the node at level 3. From level 3, traverse down branch labelled "1" to the cost

coefficient. Another example is searching for cost coefficient (222). Start at level 1

and traverse down branch labelled "2" to the node at level 2. From level 2, traverse

again down branch labelled "2" to find it is a null pointer. The null pointer indicates

the cost coefficient does not exist in the sparse MAP.



























Figure 5-4: Search tree example of a sparse MAP.


The GRASP algorithm can benefit from this search tree data structure if the

problem is sparse. In a dense problem, it would be best to put cost coefficients in

a matrix which can be directly accessed -this would benefit the local search phase.

However, in the sparse problem, completely eliminating cost coefficients reduces stor-

age and benefits the construction phase. It remains a matter of experimentation and

closer examination to find the level of sparseness where the search tree data structure

becomes more beneficial.

5.4.3 GRASP vs Sparse GRASP

To compare the performance of GRASP to solve a fully dense problem against

the performance of GRASP to solve a sparse problem, we used simulated data from

a multisensor multitarget tracking problem [74]. The problems ranged in size from

five to seven sensors. Those with five sensors had five to nine targets. Problems

with six and seven sensors had just five targets. Two problems of each size were

tested. The problem size is indicated by the problem title. For example, "s5t6rml"

means problem one with five sensors and six targets. The experiment conducted

five runs of each solution algorithm and reports the average time-to-solution, the

average solution value and the best solution value from the five runs. The solution











times can be easily adjusted for each algorithm by simply adjusting the number of

iterations. An obvious consequence is that as the number of iterations goes down, the

solution quality generally gets worse. To create sparse instances of each problem, the

probabilistic approach described above in Section 5.3 was used where r = 10. Table

5-2 shows the results of the experiment. Except for problems s5t8rml and s5t8rm2,

reducing the problem size increased solution quality with less time-to-solution.

Table 5-2: Table of experimental results of comparing solution quality and time-to-
solution for GRASP in solving fully dense and reduced simulated MSMTT problems.
Five runs of each algorithm were conducted against each problem.

Ordinary Grasp Sparse Grasp
Problem Opt Sol Ave Sol Best Sol Ave Time (sec) Ave Sol Best Sol Ave Time (sec)
s5t5rml -50 -49.2 -50 0.026 -50 -50 0.022
s5t5rm2 -44 -38 -41 0.024 -43.8 -44 0.024
s5t6rml -57 -54 -51.4 0.044 -49.4 -52 0.044
s5t6rm2 -45 -38.6 -41 0.0462 -45 -45 0.04
s5t7rml -63 -52.6 -59 0.0902 -61.2 -62 0.0962
s5t7rm2 -66 -59.2 -62 0.0862 -61.8 -62 0.0822
s5t8rml -74 -64.8 -67 0.1322 -71.2 -72 0.1262
s5t8rm2 -33 -20.6 -32 0.1402 -17 -25 0.1542
s5t9rml -84 -74.6 -78 1.7044 -74.4 -77 1.8326
s5t9rm2 -65 -59 -61 1.6664 -60.6 -63 1.5702
s6t5rml -48 -44.4 -48 0.9676 -48 -48 0.9194
s6t5rm2 -45 -42 -42 0.9754 -45 -45 0.8392
s7t5rml -51 -41.6 -44 1.378 -50.4 -51 1.0556
s7t5rm2 -52 -44.8 -47 1.4804 -52 -52 1.0916


5.5 Conclusion

In this chapter, we implemented techniques to reduce the size of the data associ-

ation problem that is linked to the MSMTT problem. Empirical results indicate that

probabilistically reducing the cardinality generally increases the solution quality and

decreases the time-to-solution for heuristics such as GRASP. We -i-i :. -1 that further

research is needed to study this approach on problems that are initially sparse in the

first place which is a common occurrence in real-world problems. Additionally, we

believe the probabilistic approach to reducing MAP size could be extended to other

solution algorithms such as simulated annealing.















CHAPTER 6
EXPECTED NUMBER OF LOCAL MINIMA FOR THE MAP

As discussed in previous chapters, the MAP is an NP-hard combinatorial op-

timization problem occurring in many applications, such as data association. As

many solution approaches to this problem rely, at least partly, on local neighborhood

searches, it is widely assumed the number of local minima has implications on solution

difficulty. In this chapter, we investigate the expected number of local minima for

random instances of this problem. Both 2-exchange and 3-exchange neighborhoods

are considered. We report on experimental findings that expected number of local

minima does impact effectiveness of three different solution algorithms that rely on

local neighborhood searches.

6.1 Introduction

In this chapter we develop relationships for the expected number of local minima.

The 2-exchange local neighborhood appears as the most commonly used neighborhood

in meta-heuristics such as GRASP that are applied to the MAP as evidenced in

several different works [4, 74, 27]. Although the 2-exchange is most common in the

literature, we include in this chapter some analysis of the 3-exchange neighborhood

for comparison purposes.

The motivation of this chapter is that the number of distinct local minima of an

MAP may have implications for heuristics that rely, at least partly, on repeated local

searches in neighborhoods of feasible solutions [112]. In general, if the number of local

minima is small then we may expect better performance from meta-heuristic algo-

rithms that rely on local neighborhood searches. A solution landscape is considered

to be rugged if the number of local minima is exponential with respect to the size of

the problem [78]. Evidence by Angel and Zissimopoulos [9] showed that r-1.- '--nl. --









of the solution landscape has a direct impact on the effectiveness of the simulated an-

nealing heuristic in solving at least one other hard problem, the quadratic assignment

problem.

The concept of solution landscapes was first introduced by Wright [111] as a non-

mathematical way to describe the action during evolution of selection and variation

[102]. The idea is to imagine the space in which evolution occurs as a landscape.

In one dimension there is the .-, i.1 ripe and in another dimension there is a height

or fitness. Evolution can be viewed as the movement of the population, represented

as a set of points (._.- r ir. epes), towards higher (fitter) areas of the landscape. In

an analogous way, a solution process for a combinatorial problem can be viewed as

the movement from some feasible solution with its associated cost (fitness) towards

better cost (fitter) areas within the solution landscape. As pointed out by Smith et al.

[102], the difficulty of searching in a given problem is related to the structure of the

landscape, however, the exact relationship between different landscape features and

the time taken to find good solutions is not clear. To name a couple of the landscape

features of interest are number local optima and basins of attraction.

Reidys and Stadler [93] describe some characteristics of landscapes and express

that local optima pl i, an important role since they might be obstacles on the way

to the optimal solution. From a minimization perspective, if x- is a feasible solution

of some optimization problem and f(x-) is the solution cost, then x is a local min-

ima iff f(x) < f(y) for all y in the neighborhood of x. Obviously the size of the

neighborhood depends upon the definition of the neighborhood. According to Reidys

and Stadler [93] there is no simple way of computing the number of local minima

without exhaustively generating the solution landscape. However, the number can

be estimated as done in some recent works [43, 45].

Rummukainen [98] describes some aspects of landscape theory which have been

used to prove convergence of simulated annealing. Of particular interest are some









results on the behavior of local optimization on a few different random landscape

classes. For example, the expected number of local minima is given for the N k

landscape.

Associated with local minima is a basin B(x) defined by means of a steepest

descent algorithm [93]. Let f(xi) be the cost of some feasible solution xi. Starting

from xi, i = 0, record for all y-neighbors the solution cost f(y). Let xi + 1 = y for

neighbor y where f(y) is the smallest for all neighbors and f(y) < f(xi). Stop when

xi is a local minima. It becomes apparent; however, that a basin may have more than

one local minima because of the definition of local minima is not strict. The basin

sizes becomes important for simple meta-heuristics. For example, consider selecting

a set of feasible solutions that are uniformly distributed in the solution landscape and

performing a steepest descent. A question is what is the probability of starting in the

basin with the global minima? This question is partially addressed by Gamier and

Kaller [45].

Long and Williams [68] mention that problems are generally easier to solve when

the number of local optima is small, but the difficulty can increase significantly when

the number of local optima is large. The authors consider the quadratic 0-1 problem

where instances are randomly generated over integers symmetric about 0. For such

problems, the authors show the expected number of local maxima increases expo-

nentially with respect to n, the size of the problem. They also reconcile this result

with Pardalos and Jha [80] who showed when test data are generated from a normal

distribution, the expected number of local maxima approaches 1 as n gets large.

Angel and Zissimopoulos [9] introduce a ruggedness coefficient which measures

the r-i-.- i--. of the QAP solution landscape. They conclude that because the QAP

landscape is rather flat, this gives theoretical justification for the effectiveness of local

search algorithms. The r-i. n 'l. d-n s coefficient is an extension of the autocorrelation

coefficient introduced by Weinberger [110]. The larger the autocorrelation coefficient









the more flat is the landscape -and so, as postulated by the authors, the more

suited is the problem for any local-search-based heuristic. Angel and Zissimopoulos

[9] calculate the autocorrelation coefficient for the QAP as being no smaller than

n/4 and no larger than n/2 which is considered relatively large. They develop the

parameter, r-i---'-dn-si coefficient, (, which is independent of problem size and lies

between 0 to 100. Close to 100 means the the landscape is very steep. They go on

to show experimentally that increasing ( for the same problem size results in higher

relative solution error and higher number of steps when using a simulated annealing

algorithm by Johnson et al. [53]. The conclusions Angel and Zissimopoulos [9] are a

relatively low r-i~-~---dn .- coefficient for the QAP gives theoretical justification of the

effectiveness of local-search-based heuristics for the QAP.

This chapter will further investigate the assumption that number of local minima

impacts the effectiveness of algorithms such as simulated annealing in solving the

MAP.

The next section provides some additional background on the 2-exchange and

3-exchange local search neighborhoods. Then in Section 6.3, we provide experimen-

tal results on the average number of local minima for variously sized problems with

assignment costs independently drawn from different distributions. Section 6.4 de-

scribes the expected number of local minima for MAPs of size of n = 2 and d > 3

where the cost elements are independent identically distributed random variables

from any probability distribution. Section 6.5 describes lower and upper bounds for

the expected number of local minima for all sizes of MAPs where assignment costs are

independent standard normal random variables. Then in Section 6.6, we investigate

whether the expected number of local minima impacts the performance of various

algorithms that rely on local searches. Some concluding remarks are given in the last

section.









6.2 Some Characteristics of Local Neighborhoods

A first step is to establish the definition of a neighborhood of a feasible solution.

Let Ap(k) be the p-exchange neighborhood of the k-th feasible solution, k = 1,..., N,

where N is the number of feasible solutions to the MAP. The p-exchange neighborhood

is all p- or less element exchange permutations in each dimension of the feasible

solution. The neighborhood is developed from the work by Lin and Kernighan [66].

If zk is the solution cost of the k-th solution, then Zk is a discrete local minimum

iff k < zj for all j E AVp(k). As an example of a 2-exchange neighbor, consider the

following feasible solution to an MAP with d = 3, n = 3: {111, 222, 333}. A neighbor

is {111, 322, 233}. The solution {111, 222, 333} is a local minimum if its solution cost

is less than or equal to all of its neighbor's solution costs.

The formula for the number of neighbors, J, of a feasible solution in the 2-

exchange neighborhood of an MAP with dimension d and n elements in each dimen-

sion is as follows


J |i(k) = d ( (6.1)

It is obvious that for a fixed n, J is linear in d. On the other hand for a fixed d, J

is quadratic in n. If we define a flat MAP as one with relatively small n and define

a deep MAP as one with relatively large n, then we expect larger neighborhoods in

deep problems.

Similarly, for n > 2 the size of the 3-exchange neighborhood is as follows


J = W(k) = d + 2 (n (6.2)

Similar to above for the 2-exchange, it becomes clear J is linear with respect to d

and cubic with respect to n.

The minimum number of local minima for any instance is one the global mini-

mum. At the other extreme, the maximum number of local minima is (n!)d-1 which









is the number of feasible solutions of an MAP. This occurs if all cost coefficients are

equal.

6.3 Experimentally Determined Number of Local Minima

Studies were made of randomly produced instances of MAPs to empirically de-

termine E[M]. The assignment costs ci,...i for each problem instance were drawn

from one of three distributions. The first distribution of assignment costs used is the

uniform, U[0, 1]. The next distribution used is the exponential with mean one, being

determined by ci..., = In U. Finally, the third distribution used was the standard

normal, N(0, 1), with values determined by the polar method [63].

Table 6-1 shows the average number of local minima for randomly generated

instances of the MAP when considering a 2-exchange neighborhood. For small sized

problems, the study was conducted by generating an instance of an MAP and count-

ing number of local minima through complete enumeration of the feasible solutions.

The values in the tables are the average number of local minima from 100 problem

instances. For larger problems (indicated by in the table), the average number

of local minima was found by examining a large number1 of generated problem in-

stances. For each instance of a problem we randomly selected a feasible solution and

determined whether it was a local minimum. This technique gives an estimate of the

probability that any feasible solution is a local minima. This relationship was then

used to estimate the average number of local minima by multiplying the probability

by the number of feasible solutions. This technique showed to have results consistent

with the complete enumeration method mentioned above for small problems. Re-

gardless of the distribution that cost coefficients were drawn, a standard deviation

of 40-percent and 5-percent were observed for problems of sizes d = 3, n = 3 and



1 The number examined depends on problem size. The number ranged from 106
to 107.










d = 5, n = 5, respectively. It is clear from the tables that E[M] is effected by the dis-

tribution from which assignment costs are drawn. For example, problems generated

from the exponential distribution have more local minima than problems generated

from the normal distribution.

Table 6-2 shows similar results for the 3-exchange neighborhood and when cost

coefficients are i.i.d. standard normal. We note, as expected, evidence indicates

E[M] is smaller in the 3-exchange case versus the 2-exchange case for the same sized

problems.

Table 6-1: Average number of local minima (2-exchange neighborhood) for different
sizes of MAPs with independent assignment costs.


Number of Local Minima, Uniform on [0,1]
n\ d 3 4 5 6
2 1 1.68 2.66 4.56
3 2 7.69 33.5 159
4 5.60 77.8 1230 2.1E+4
5 21.0 1355 9.58E+04* 7.60E+6*
6 116 3.62E+04* 1.30E+07* 6.56E+9*

Number of Local Minima, Exponential A 1
n\ d 3 4 5 6
2 1 1.54 2.66 4.71
3 2.06 7.84 35.8 165
4 5.47 80.6 1290 2.18E+4
5 22.7 1400 1.01E+5* 7.67E+06*
6 122 3.91E+4* 1.53E+07* (. ; in*

Number of Local Minima, Standard Normal Costs
n\ d 3 4 5 6
2 1 1.56 2.58 4.66
3 1.82 7.23 30.3 141
4 4.54 62.2 949 1.58E+04
5 16.3 939 6.5E+4* 4.75E+06*
6 75.6 2.36E+4* 7.90E+06* 3.48E+09*


Table 6-3 shows the average proportion of feasible solutions that are local minima

for both the 2-exchange and 3-exchange neighborhoods where costs are i.i.d. standard

normal random variables. The table is followed by Figure 6-1 which includes plots of

the proportion of local minima to number of feasible solutions. We observe that for











Table 6-2: Average number of local minima (3-exchange neighborhood) for different
sizes of MAPs with i.i.d. standard normal assignment costs.

n\d 3 4 5 6


3 1.55
4 3.27
5 8.27
6 28.8


5.98
43.1
516
8710W


26.0 124
670 1.11E+4
3.48E+4* 2.65E+06'
3.06E+06* 1.22E+09'


either fixed dimension and increasing number of elements or visa versa, the proportion

of local minima approaches zero.

Table 6-3: Proportion of local minima to total number of feasible solutions for dif-
ferent sizes of MAPs with i.i.d. standard normal costs.


Proportion of local minima to feasible solutions
using standard normal costs and 2-exchange
n\ d 3 4 5 6


2.50E-01
4.88E-02
8.02E-03
1.13E-03
1.50E-04


2.00E-01
3.27E-02
4.50E-03
5.43E-04
6.32E-05


1.67E-01
2.37E-02
2.87E-03
3.14E-04
2.94E-05


1.43E-01
2.11E-02
2.81E-03
1.91E-04
1.80E-05


Proportion of local minima to feasible solutions
using standard normal costs and 3-exchange
n\ d 3 4 5 6


4.23E-02
5.60E-03
6.40E-04
5.94E-05


2.87E-02
3.13E-03
2.99E-04
2.33E-05


2.05E-02
2.03E-03
1.68E-04
1.14E-05


1.58E-02
1.40E-03
1.06E-04
6.31E-06


Figure 6-1: Proportion of feasible solutions that are local minima when
the 2-exchange neighborhood and where costs are i.i.d. standard normal.


considering


Ratio of Local Minima to Feasible Solutions,
Dimension, d = 6

16
S14

5 10 -
2
0




1 2 3 4 5 6 7
Number of Elements, n


Ratio of Local Minima to Feasible Solutions,
Number of Elements, n = 6

16
14
12
0 1
. 06-
.04-
02-

2 3 4 5 6 7
Dimension, d









6.4 Expected Number of Local Minima for n = 2

In the special case of an MAP where n = 2, d > 3, and cost elements are inde-

pendent identically distributed random variables from some continuous distribution

with c.d.f F(.), one can establish a closed form expression for the expected num-

ber of local minima. To show this, we recall that distribution Fx+y of the sum of

two independent random variables X and Y is determined by the convolution of the

respective distribution functions, Fx+y = Fx Fy.

We now borrow from Proposition 3.1 to construct the following proposition.


Proposition 6.1 In an instance of the MAP with n=2 and with cost coefficients that

are i.i.d. random variables with continuous distribution F, the costs of all feasible

solutions are independent distributed random variables with distribution F F.

Proof: Let I be an instance of MAP with n = 2. Each feasible solution for I

is an assignment al = cl,,(1),..., d (1), a2 C2,61(2),...,6d-(2), with cost z = al + a2.

The important feature of such assignments is that for each fixed entry c1,61(1),...,d (1),

there is just one remaining possibility, namely C2,61(2),...,6d-(2), since each dimension

has only two elements. This implies that different assignments cannot share elements

in the cost vector, and therefore different assignments have independent costs z. Now,

a, and a2 are independent variables from F. Thus z = a, + a2 is a random variable

with distribution F F.

One other proposition is important to this development.


Proposition 6.2 Given j i.i.d. random variables with continuous distributions, the

1',./',1/.:],'/; that the rth, r = 1,... ,j, variable is the minimum value is 1/j.

Proof: Consider j i.i.d. random variables, Xi,i = 1,...,j, with c.d.f. F(.) and

p.d.f. f(.). Let X(j_1) be the minimum of j 1 of these variables,


X(j-) = min{Xi I i 1,... ,j, i / r},









whose c.d.f. and p.d.f. are computed trivially as


P[X(j_1) < x] =

d F(j_)() (j-


- (1 F(x))-1,

1)(1 F(x))j-2f(X).


Then, the probability that the rth random variable is minimal among j i.i.d. contin-
uous variables, is


P[rth r.v. is minimal] = P[X, < X(j_)] = P[Y < 0] = Fy(0).


(6.3)


Here Fy(.) is the c.d.f. of random variable Y
rule, it equals to


Fy(x)


I +00
00


Hence, the probability (6.3) can immediately be calculated as


- F(-y))j-f(-y)dy


j- f(-y)dy


X, X(j_I), and, by convolution


P[X, < X(j-l)] F(-y)(J- -)(]

1 **
t j(1 F(-y))

since j(1 F(-y))J-lf(-y) is the p.d.f. of -X(j)
equality yields the statement of the proposition.


-min{Xi,...,Xj}. The last


The obvious consequence of Proposition 6.2 is that given a sequence of indepen-
dent random variables from a continuous distribution, position of the minimum value
is uniformly located within the sequence regardless of the parent distribution.
We are now ready to prove the main result of this section.

Theorem 6.3 In an MAP with cost coefficients that are i.i.d. continuous random
variables where n = 2 and d > 3, the expected number of local minima is given by

2d- (6.4)
E [M] = (6.4)
d +1


F(j-1)(x))

f(j-1)(X)


F(x- y)(j- 1)(1 F(-y))-2f(-y)dy.







79

Proof: Let N be the number of feasible solutions to an n = 2 MAP, N = 2d-1

Introducing indicator variables


k 1, kth solution, k =1,..., N, is a local minimum; (6.5)
0, otherwise,

one can write M as the sum of indicator variables:
N
M Yk-
k=1

From the elementary properties of expectation it follows that

N N
E[M] E [Yk] P[Y= 1], (6.6)
k=1 k=1

where P[Yk = 1] is the probability that the cost of k-th feasible solution does not

exceed the cost of any of its neighbors. Any feasible solution has J = d() d

neighbors whose costs, by virtue of Proposition 6.1, are i.i.d. continuous random

variables. Then, Proposition 6.2 implies that the probability of the cost of k-th

feasible solution being minimal among its neighbors is equal to

1
P[Yk 11] =-d
d+1

which, upon substitution into (6.6), yields the statement of the theorem (6.4).


Remark 6.3.1 Equality (6.4) implies that the number of local minima in an n

2, d > 3 MAP is exponential in d when the cost coefficients are independently drawn

from ,in continuous distribution.


Corollary 6.4 The proved relation (6.4) can be used to derive the expected ratio

of the number of local minima M to the total number of feasible solutions N in an

n= 2, d> 3 MAP:

E [M] 1
E [M/N]
N d+1









This shows that the number of local minima in an n = 2 MAP becomes infinitely

small comparing to the number of feasible solutions, when dimension d of the problem

increases. This i-mptotic characteristic is reflected in the numerical data above and

may be useful for the development of novel solution methods.

6.5 Expected Number of Local Minima for n > 3

Our ability to derive a closed-form expression (6.4) for the expected number of

local minima E[M] in the previous section has relied on the independence of costs of

feasible solutions in an n = 2 MAP. As it is easy to verify directly, in case of n > 3

the costs coefficients are generally not independent. This complicates significantly the

analysis if an arbitrary continuous distribution for assignment costs il...id is assumed.

However, as we show below, one can derive upper and lower bounds for E[M] in the

case when the costs coefficients of (2.1) are normally distributed random variables.

First, we introduce a proposition, which follows a similar development by Beck

[16].


Proposition 6.5 Consider an n > 3, d > 3 MAP whose costs are i.i.d. continuous

random variables. Then the expected number of local minima can be represented as
N
E[M] =J P[ n z,-z, k 1 jENA(2k)

where A2(k) is the 2-, ,. hli,, c, neighborhood of the k-th feasible solution, and zi is the

cost of the i-th feasible solution.

Proof: As before, M can be written as the sum of indicator variables Yk (6.5), which

consequently leads to
N N
E[M] = E[Yk] P[Y = 1]. (6.8)
k=1 k=1

As Yk = 1 means Zk < zj for all j cE A(k), it is obvious that P[Yk 1] P[zk zj <

0, Vj E cA2(k)], which proves the proposition.









If we allow the nd cost coefficients ci...i, ~ N(0, 1) of the MAP to be independent

standard normal N(0, 1) random variables, then for any two feasible solutions the

difference of their costs Zj = zi zj is a normal variable with mean zero.

Without loss of generality, consider the k = 1 feasible solution to (2.1) whose

cost is


Zi = ci...i + c2...2 + + c+...,. (6.9)


In the 2-exchange neighborhood A/2(l), the cost of a feasible solution differs from

(6.9) by two cost coefficients, e.g.,


z2 = 21...1 + C12...2 + C3...3 + .. + Cn...n.


Generally, the difference zl z, of costs of (6.9) and that of any neighbor 1 E A2(l)

has the form


Zrsq Cr...r + Cs...s Crr..rsr..r Cs...srs...s, (6.10)


where the last two coefficients have .iiched" indices in the q-th position, q =

1,..., d. Observing that


Zrsq Zsrq for r, 1,. .. n; q= 1,.. ,d,


consider the J-dimensional random vector


Z= (Z111,... Zl, Z121, ... Z12d, ".

..., Zrs ..., Z rsd ,* Znn1 Znnd) r < s. (6.11)









Vector Z has normal distribution N(0, E), with covariance matrix E defined as


Cov(Z,,,q, Zij~k)


if i =r, j =s, q = k,

if i = r, j =s, q k,

if (i r, j s) or ( / r,j

if i/r, j s.


For example, in case n = 3, d = 3 the covariance matrix E has the form


4

2

2

1

= 1

1

1

1

1

Now, the probability in (6.7)


2 2 1 1 1 1 1

42 1 1 1 1 1

241 1 1 1 1

1 1 4 2 1 1

1 1 2 4 2 1 1

1 1 2241 1

1 1 1 1 142

1 1 1 1 1 2 4

1 1 1 1 1 2 2

can be expressed as


(6.13)


[nQ Zk i E A2 (k


where Fe is the c.d.f. of the J-dimensional multivariate normal distribution N(0, E).

While the value of Fv(0) in (6.13) is difficult to compute exactly for large d and n,

lower and upper bounds can be constructed using Slepian inequality [107]. To this


(6.12)









end, let us introduce covariance matrices = (o i) and E = (oay)

4, if i j,

aij= 2, if i j and (i -1)divd (j -1)divd, (6.14a)

0, otherwise,

S4, if (6.14b)
2, otherwise,

so that c ij < ayj < rij holds for all 1 < i, j < J, with aiy being the components of

the covariance matrix Z (6.12). Then, the Slepian inequality claims that


FE(O) < FW(O) < Fr(0), (6.15)

where Fr(O) and F-(0) are c.d.f.'s of random variables X_ ~ N(O, E) and Xr ~

N(0, E), respectively.

As the variable Xv is equicorrelated, the upper bound in (6.15) can be expressed

by the one-dimensional integral (see, e.g., [107])


S [(az)] Jd(z), a 1 (6.16)

where 4(<.) is the c.d.f. of standard normal distribution:

S 1 t2
N(z)= e 2 dt,

and p = aij/ /aoi jj is the correlation coefficient of distinct components of the cor-

responding random vector. The correlation coefficient of the components of vector

X, is p which allows for a simple closed-form expression for the upper bound

in (6.15)

Fr(O) = [ (z)] d4(z) = (6.17)
-oo J+1









This immediately yields the value of the upper bound E[M] for the expected number

of local minima E[M]:

N 2 (!)d-1
E[M] = F(O) = ,
[ ( n(n 1)d+2
k 1

Let us now calculate the lower bound for E[M] using (6.15). According to the

covariance matrix E (6.14a), the vector X_ is comprised of n" 1) groups of variables,

each consisting from d elements,


XE= (Xi,...,X, Xd X(i_)d+i,... ,Xid, ''

X(n(n-1)/2-1)d, ... X(n(n-l)/2)d


such that the variables from different groups are independent, whereas variables

within a group have d x d covariance matrix defined as in (6.14b). Thus, one can

express the lower bound Fr(O) in (6.15) as a product

n(n-1)
2
F-(0)= P n [ X(i_l)id+1< 0,..,Xd< 0 ].
i=i

Since variables X(i 1)d+1,... ,Xd, are equicorrelated with correlation coefficient p

, each probability term in the last equality can be computed similarly to evaluation

of the lower-bound probability (6.17), i.e.,

n(n2 1) n(n )
F(0) n [()] d4(z), =
^ -oo [X)]d rd +t

This finally leads to a lower bound for the number of expected minima:

N (n!)d_-1
E [M] =- Fe(0)= .
0)-(d + )n(-1)/2
k- 1


In such a way, we have proved the following









Theorem 6.6 In an n > 3, d > 3 MAP with i.i.d. standard normal cost coefficients,

the expected number of local minima is bounded as

(n!)d-1 2 (n!)d- 1
< E [M] < (6.18)
(d + 1)n(n- 1)2 E n(n 1)d + 2

Remark 6.6.1 From (6.18) it follows that for fixed n > 3, the expected number of

local minima is exponential in d.


Corollary 6.7 Similarly to the case n = 2, the developed lower and upper bounds

can (6.18) can be used to estimate the expected ratio of the number of local minima

M to the total number of feasible solutions N in an n > 3, d > 3 MAP:

1 2
< 2 [ M/NI n(n] )d+2
(d + 1)n(n-1)/2 n(n 1)d + 2

6.6 Number of Local Minima Effects on Solution Algorithms

In this section, we examine the question of whether number of local minima in

the MAP has an impact on heuristics that rely, at least in part, on local neighborhood

searches. We consider three heuristics

Random Local Search

Greedy Randomized Adaptive Search (GRASP)

Simulated Annealing

The heuristics described in the following three subsections were exercised against

various sized problems that were randomly generated from the standard normal dis-

tribution.

6.6.1 Random Local Search

The random local search procedure simply steps through a given number of iter-

ations. Each iteration begins by selecting a random starting solution. The algorithm

then conducts a local search until no better solution can be found. The algorithm









captures the overall best solution and reports it after executing the maximum number

of iterations. The following is a more detailed description of the steps involved.

1. Set iteration number to zero, Iter = 0.

2. Randomly select a current solution, current.

3. While not all neighbors of current examined, select a neighbor, xne., of the

current solution.

If zXne < zx ent then current -- xnew.

4. If zc ..rent < zXbest then Xbest -- current-

5. If Iter < Iterma,, increment Iter by one and go to Step 2. Otherwise, end.

6.6.2 GRASP

A greedy randomized adaptive search procedure (GRASP) [36, 37, 38] is a multi-

start or iterative process, in which each GRASP iteration consists of two phases. In

a construction phase, a random adaptive rule is used to build a feasible solution one

component at a time. In a local search phase, a local optimum in the neighborhood

of the constructed solution is sought. The best overall solution is kept as the result.

The neighborhood search is conducted similar to that in the random local search

algorithm above. That is neighbors of a current solution are examined one at a time

and if an improving solution is found, it is adopted as the current solution and local

search starts again. Local search ends when no improving solution is found.

GRASP has been used in many applications and specifically in solving the MAP

[4, 74].

6.6.3 Simulated Annealing

Simulated Annealing is a popular heuristic used in solving a variety of problems

[57, 70]. Simulated annealing uses a local search procedure, but the process allows

uphill moves. The probability of moving uphill is higher at high temperatures, but

as the process cools, there is less probability of moving uphill. The specific steps









for simulated annealing used in this chapter are taken from work by Gosavi [46].

Simulated annealing was recently applied to the MAP by Clemmons et al. [27].

6.6.4 Results

The heuristic solution quality, Q, which is the relative difference from the optimal

value, is reported and compared for the same sized problems with assignment costs

independently drawn from the standard normal distribution. The purpose of our

analysis is not to compare the efficiency of the heuristic algorithms, but to determine

the extent to which the number of local minima affects the performance of these

algorithms. Each run of an experiment involved the following general steps:

1. Generate random MAP instance with cost coefficients that are i.i.d. standard

normal random variables.

2. Obtain M by checking each feasible solution for being a local minimum.

3. Solve the generated MAP instance using each of the above heuristics 100 times

and return the average solution quality, Q, for each heuristic method.

Problem sizes were chosen based on the desire to test a variety of sizes and the

practical amount of time to determine M (as counting M is the obvious bottleneck

in the experiment). Four problem sizes chosen were d = 3, n = 6; d = 4, n = 5; d =

4, n = 6; and d = 6, n = 4. For problem size d = 4, n = 6 which has the largest N,

a single run took approximately four hours on a 2.2 GHz Pentium 4 machine. The

number of runs of each experiment varied for each problem size with fewer runs for

larger problems. The number of runs were 100, 100, 30, and 50, respectively, for the

problem sizes listed above.

Figure 6-2 di-pl'- plots of the average solution quality for each of the three

heuristics versus the number of local minima. The plots are the results from problem

size d = 4, n = 5 and are typical for the other problem sizes. Included in each plot

is a best-fit linear least-squares line that provides some insight on the effect of M

on solution quality. A close examination of the figures indicates that the solution