Combinatorial and Nonlinear Optimization Methods with Applications in Data Clustering, Biclustering and Power Systems

MISSING IMAGE

Material Information

Title:
Combinatorial and Nonlinear Optimization Methods with Applications in Data Clustering, Biclustering and Power Systems
Physical Description:
1 online resource (136 p.)
Language:
english
Creator:
Fan,Neng
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Industrial and Systems Engineering
Committee Chair:
Pardalos, Panagote M
Committee Members:
Smith, Jonathan
Uryasev, Stanislav
Lan, Guanghui
Hager, William W

Subjects

Subjects / Keywords:
biclustering -- clustering -- combinatorial -- graph -- integer -- islanding -- nonlinear
Industrial and Systems Engineering -- Dissertations, Academic -- UF
Genre:
Industrial and Systems Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
With the increasing number of databases appearing in computational biology, biomedical engineering, consumers' behavior survey, social networks, power systems, and many other areas, finding the useful information and discovering the knowledge within these databases are important issues nowadays. Data mining is one of the most important techniques to extract information from large data sets. Data clustering and biclustering are widely used tools in data mining. The graph partitioning problem is a classic combinatorial optimization problem and it is known to be NP-complete. In the dissertation, for the graph partitioning problem, instead of obtaining two subsets, we use integer programming formulation to obtain more than two subsets directly. Besides its deterministic version, we study the uncertain versions for changes on weights of edges by robust and stochastic optimization approaches. All these models and approaches are for the classic graph partitioning problem, which is stated as graph partitioning by minimum cut in this dissertation. Besides minimum cut, which is the direct summation of weights of edges connecting different subsets, we also study the graph partitioning problem by Ratio cut and Normalized cut. Both Ratio cut and Normalized cut were introduced for balancing the cardinalities of all subsets, and they are quite useful for data clustering. In the dissertation, we use graph partitioning method to obtain nonoverlap and exclusive clusters. The graph partitioning methods by Ratio cut and Normalized cut for clustering are formulated as mixed integer nonlinear programs (MINLPs). We prove that the spectral methods, which were widely used for clustering, are solving relaxations of our integer programming models. We also present the semidefinite programming (SDP) relaxations for these integer programming models, and our numerical experiments show that better solutions than spectral methods can be obtained. Since SDP is a subfield of convex optimization concerned with the optimization of a linear objective function over the intersection of the cone of positive semidefinite matrices, and it can be solved efficiently by interior point methods and many other methods. Our SDP relaxations can find many applications in data clustering. Additionally, we present equivalent integer linear programming formulations for our MINLPs, and they can be applied for small data cases to obtain exact solutions. Since a bipartite graph is a special case of a general graph, the bipartite graph partitioning problem can be easily transformed into the graph partitioning problem. For data biclustering, bipartite graph partitioning models with Ratio cut and Normalized cut are also studied from the adapted results of graph partitioning models for clustering. In the dissertation, we also use graph partitioning methods to form islands in a power grid and formulate these problems as mixed integer nonlinear programs. With these mathematical programming models, optimal formation of islands can be obtained and the different approaches can be compared. Through simulations, experimental results are analyzed and compared to provide insight for power grid islanding.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Neng Fan.
Thesis:
Thesis (Ph.D.)--University of Florida, 2011.
Local:
Adviser: Pardalos, Panagote M.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-02-29

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043200:00001


This item is only available as the following downloads:


Full Text

PAGE 1

COMBINATORIALANDNONLINEAROPTIMIZATIONMETHODSWITHAPPLICATIONSINDATACLUSTERING,BICLUSTERINGANDPOWERSYSTEMSByNENGFANADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2011

PAGE 2

c2011NengFan 2

PAGE 3

Tomyparents,familyandfriends 3

PAGE 4

ACKNOWLEDGMENTS Firstandforemost,Iwouldliketothankmysupervisor,Dr.PanosM.Pardalos,forhisguidanceandsupport.Hisknowledge,experience,enthusiasmandsenseofhumorhavebeentrulyvaluable.Withouthishelpandsuggestions,thisdissertationcannotbecompleted.Ithankmydissertationsupervisorycommitteemembers,Dr.WilliamHager,Dr.GuanghuiLan,Dr.J.ColeSmith,andDr.StanislavUryasevforprovidingsuggestionsattheverybeginningofmyresearchandalsoevaluatingmydissertation.Iwouldliketothankallmycollaborators:AltannarChinchuluun,QipengZheng,NikitaBoyko,PetrosXanthopoulos,HongshengXu,JicongZhang,SyedMujahid,PandoGeorgiev,SteffenRebennackandAshwinArulselvan.Iamgratefulfortheireffortsandsuggestions.IwanttothankallmyfellowstudentsinDepartmentofIndustrialandSystemsEngineeringatUniversityofFlorida,especiallyfriendsintheCenterforAppliedOptimization,fortheirhelpfuldiscussionsandtheirfriendship.IamverygratefultoDr.FengPanforbringingmetoD-6groupatLosAlamosNationalLaboratoryfromAugust2010toMay2011.Especially,theD-6groupandCenterforNonlinearStudiesprovidedmeaverygreatenvironmenttoconductmyresearchinthesmartgridandpowersystems.IwanttothankDr.DavidIzraelevitz,Dr.MichaelChertkovandDr.AlexanderGutfraindfortheirsupportanddiscussions.Ofcourse,Iamgratefultomyparentsfortheirpatienceandsupport.Finally,IwouldliketothankallmyfriendsduringmyPhDstudyinGainesville,Florida. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 9 CHAPTER 1INTRODUCTION ................................... 11 1.1Background ................................... 11 1.2Contributions .................................. 13 1.3OutlineoftheDissertation ........................... 15 2REVIEWofCLUSTERING,BICLUSTERINGANDGRAPHPARTITIONING .. 16 2.1ReviewofDataClusteringandBiclustering ................. 16 2.1.1ReviewofClustering .......................... 17 2.1.2ReviewofBiclustering ......................... 19 2.2ReviewoftheGraphPartitioningProblem .................. 22 2.3PreliminaryofGraphPartitioningandClustering .............. 23 3MULTI-WAYCLUSTERINGANDBICLUSTERINGBYTHERATIOCUTANDNORMALIZEDCUT ................................. 31 3.1ClusteringandGraphPartitioning ....................... 31 3.2SpectralRelaxationApproaches ....................... 36 3.2.1SpectralMethods ............................ 36 3.2.2RelaxedSolutionstoIntegerOnes .................. 39 3.3SemideniteProgrammingRelaxations ................... 41 3.3.1SemideniteProgrammingMethods ................. 41 3.3.2Algorithms ................................ 45 3.4QuadraticallyConstrainedProgrammingApproaches ............ 45 3.5BiclusteringandBipartiteGraphModels ................... 49 3.6NumericalExperiments ............................ 52 3.7Discussion ................................... 54 4GRAPHPARTITIONINGWITHMINIMUMCUT .................. 56 4.1GeneralGraphPartitioningProblem ..................... 57 4.1.1QuadraticProgrammingApproaches ................. 60 4.1.2LinearProgrammingApproaches ................... 63 4.1.3NumericalExperiments ........................ 69 4.1.4Discussion ................................ 71 5

PAGE 6

4.2RobustOptimizationofGraphPartitioningInvolvingIntervalUncertainty 71 4.2.1GraphPartitioningwithUncertainWeights .............. 74 4.2.2DecompositionMethodsforRobustGraphPartitioningProblem .. 76 4.2.3BipartiteGraphPartitioningInvolvingUncertainty .......... 81 4.2.4NumericalExperiments ........................ 83 4.2.5Discussion ................................ 84 4.3Two-StageStochasticGraphPartitioningProblem ............. 87 4.3.1TheModeloftheTwo-StageStochasticGraphPartitioningProblem 90 4.3.2EquivalentIntegerLinearProgrammingFormulations ........ 94 4.3.3NumericalExperiments ........................ 98 4.3.4Discussion ................................ 99 5APPLICATIONSINPOWERSYSTEMSFORISLANDING ............ 101 5.1Introduction ................................... 101 5.2OptimalPowerFlowModels .......................... 105 5.2.1Notations ................................ 105 5.2.2PowerFlowModels ........................... 106 5.3PowerGridIslandingModels ......................... 108 5.3.1ModelforCompleteIslanding ..................... 108 5.3.2IslandingwithMinimumSize ...................... 111 5.3.3ModelsforWeaklyConnectedIslanding ............... 111 5.3.4ModelPreprocessing .......................... 113 5.4NumericalExperiments ............................ 115 5.5Discussion ................................... 121 6CONCLUSIONS ................................... 124 REFERENCES ....................................... 125 BIOGRAPHICALSKETCH ................................ 136 6

PAGE 7

LISTOFTABLES Table page 3-1Multi-wayclusteringresultsforcaseN=10 .................... 52 3-2Multi-wayclusteringresultsforseveralcases ................... 53 3-3Multi-wayclusteringresultsforcaseN=100 ................... 54 4-1Comparisonsofformulationsforgraphpartitioning ................ 70 4-2Computationalsecondsforgeneralgraphpartitioning .............. 70 4-3Computationalsecondsforbipartitegraphpartitioning .............. 71 4-4Computationalresultsandsecondsforrobustgraphpartitioning(1) ....... 85 4-5Computationalresultsandsecondsforrobustgraphpartitioning(2) ....... 86 4-6Computationalresultsandsecondsfortwo-stagestochasticgraphpartitioning 100 5-1GenerationandloaddataofIEEE-30-Busnetwork ................ 116 5-2TransmissionlinedataofIEEE-30-Busnetwork .................. 117 5-3Islandingresultsbycompleteislandingmodel ................... 123 7

PAGE 8

LISTOFFIGURES Figure page 2-1ComparisonsofMinimum,RatioandNormalizedCuts .............. 29 3-1DifferencesforRatiocutandNormalizedcutsolvedbySDP2andSDP3 .... 55 4-1Robustgraphpartitioningobjectivevaluesregarding)]TJ ET 0 G 0 g BT /F1 11.955 Tf 319.22 -102.61 Td[(.............. 87 4-2Anexamplefortwo-stagestochasticgraphpartitioningproblem ......... 91 5-1IEEE-30-Busnetwork ................................ 115 5-2Loadsheddingcostvs.thenumberKofislands ................. 118 5-3IEEE-30-Busnetworkwith2islands ........................ 119 5-4IEEE-30-Busnetworkwith3islands ........................ 119 5-5IEEE-30-Busnetworkwith4islands ........................ 120 5-6Loadsheddingcostvs.MinSize ........................... 120 5-7Loadsheddingcostvs." .............................. 121 8

PAGE 9

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyCOMBINATORIALANDNONLINEAROPTIMIZATIONMETHODSWITHAPPLICATIONSINDATACLUSTERING,BICLUSTERINGANDPOWERSYSTEMSByNengFanAugust2011Chair:PanosM.PardalosMajor:IndustrialandSystemsEngineering Withtheincreasingnumberofdatabasesappearingincomputationalbiology,biomedicalengineering,consumers'behaviorsurvey,socialnetworks,powersystems,andmanyotherareas,ndingtheusefulinformationanddiscoveringtheknowledgewithinthesedatabasesareimportantissuesnowadays.Dataminingisoneofthemostimportanttechniquestoextractinformationfromlargedatasets.Dataclusteringandbiclusteringarewidelyusedtoolsindatamining.ThegraphpartitioningproblemisaclassiccombinatorialoptimizationproblemanditisknowntobeNP-complete. Inthedissertation,forthegraphpartitioningproblem,insteadofobtainingtwosubsets,weuseintegerprogrammingformulationtoobtainmorethantwosubsetsdirectly.Besidesitsdeterministicversion,westudytheuncertainversionsforchangesonweightsofedgesbyrobustandstochasticoptimizationapproaches.Allthesemodelsandapproachesarefortheclassicgraphpartitioningproblem,whichisstatedasgraphpartitioningbyminimumcutinthisdissertation. Besidesminimumcut,whichisthedirectsummationofweightsofedgesconnectingdifferentsubsets,wealsostudythegraphpartitioningproblembyRatiocutandNormalizedcut.BothRatiocutandNormalizedcutwereintroducedforbalancingthecardinalitiesofallsubsets,andtheyarequiteusefulfordataclustering.Inthedissertation,weusegraphpartitioningmethodtoobtainnonoverlapandexclusiveclusters.ThegraphpartitioningmethodsbyRatiocutandNormalizedcutforclustering 9

PAGE 10

areformulatedasmixedintegernonlinearprograms(MINLPs).Weprovethatthespectralmethods,whichwerewidelyusedforclustering,aresolvingrelaxationsofourintegerprogrammingmodels.Wealsopresentthesemideniteprogramming(SDP)relaxationsfortheseintegerprogrammingmodels,andournumericalexperimentsshowthatbettersolutionsthanspectralmethodscanbeobtained.SinceSDPisasubeldofconvexoptimizationconcernedwiththeoptimizationofalinearobjectivefunctionovertheintersectionoftheconeofpositivesemidenitematrices,anditcanbesolvedefcientlybyinteriorpointmethodsandmanyothermethods.OurSDPrelaxationscanndmanyapplicationsindataclustering.Additionally,wepresentequivalentintegerlinearprogrammingformulationsforourMINLPs,andtheycanbeappliedforsmalldatacasestoobtainexactsolutions.Sinceabipartitegraphisaspecialcaseofageneralgraph,thebipartitegraphpartitioningproblemcanbeeasilytransformedintothegraphpartitioningproblem.Fordatabiclustering,bipartitegraphpartitioningmodelswithRatiocutandNormalizedcutarealsostudiedfromtheadaptedresultsofgraphpartitioningmodelsforclustering. Inthedissertation,wealsousegraphpartitioningmethodstoformislandsinapowergridandformulatetheseproblemsasmixedintegernonlinearprograms.Withthesemathematicalprogrammingmodels,optimalformationofislandscanbeobtainedandthedifferentapproachescanbecompared.Throughsimulations,experimentalresultsareanalyzedandcomparedtoprovideinsightforpowergridislanding. 10

PAGE 11

CHAPTER1INTRODUCTION Inthischapter,themotivationandthebackgroundforresearchofthedissertationwillbeintroducedrst.Then,themaincontributionofthedissertationisstated.Afterthat,theorganizationofthedissertationisoutlined. 1.1Background Withtheamountnumberofdatabasesappearingincomputationalbiology,biomedicalengineering,consumers'behaviorsurveyandsocialnetworks,ndingtheusefulinformationbehindthesedataandgroupingthedataareimportantissuesnowadays.Clusteringisamethodtoclassifyofobjectsintodifferentgroups,suchthattheobjectsinthesamegroupsharesomecommontraitsoraresimilarinsomesense.Itisamethodofunsupervisedclassication,andacommontechniquefordataanalysisinmanyelds,includingdatamining,machinelearning,patternrecognition,imageanalysisandbioinformatics.Clusteringhasbeenwidelystudiedinthepast20years.ThemethodsforclusteringincludeK-meansclustering,fuzzyc-meansclustering,andspectralclustering. However,clusteringonlygroupsobjectswithoutconsideringthefeaturesofeachobjectmayhave.Inotherwords,clusteringcomparestwoobjectsbythefeaturesthattwoshare,withoutdepictingthedifferentfeaturesofthetwo.Amethodsimultaneouslygroupstheobjectsandfeaturesiscalledbiclusteringsuchthataspecicgroupofobjectshasaspecialkindgroupoffeatures.Moreprecisely,abiclusteringistondasubsetofobjectsandfeaturessatisfyingthattheseobjectsarerelatedtothosefeaturestosomelevel.Suchkindofsubsetsarecalledbiclusters.Meantime,biclusteringdoesnotrequireobjectsinthesamebiclustertobehavesimilarlyoverallpossiblefeatures,buttohighlyhavespecicfeaturesinthisbicluster.Biclusteringalsohasabilitiestondhidefeaturesandspecifythemtosomesubsetsofobjects.Thebiclusteringproblemis 11

PAGE 12

tondbiclustersindatasets,anditmayhavedifferentnamessuchasco-clustering,two-modeclusteringinsomeliteratures. Bothclusteringandbiclusteringareperformedondatamatrices.Thedatamatrixforclusteringisasymmetricnonnegativematrixanditisalwayschosenasasimilaritymatrix,whoseentriesdenotethesimilarityofobjects.Thematrixforbiclusteringisarectangularmatrixwithrowscorrespondingtoobjects,columnstofeatures,andentriestotheexpressionlevelofcorrespondingfeaturesintheobjects.Thismatrixiscalledexpressionmatrix. TheGraphPartitioningProblem(GPP)consistsofdividingthevertexsetofagraphintoseveraldisjointsubsetssothatthecutamongthesedisjointsubsetsisminimized.ThethreemostwidelyusedcutsforclusteringandbiclusteringareRatiocut,NormalizedcutandMinimumcut.TheMinimumcutisthesumweightoftheedgeswithendsindistinctsubsets.TheRatiocutisafractionofthecutstothesizesofthecorrespondingsubsets,andtheNormalizedcutisafractionofthecutstotheweighteddegreesofvertices.TheRatiocutandNormalizedcutareintroducedtobalancethethecardinalitiesofsubsetsforgraphpartitioning,andtheyareusedfordifferentpurposesofclusteringandbiclustering. Inthegraphpartitioningmodelsforclustering,theobjectsarecorrespondingtovertices,andthesimilaritymatrixforclusteringischosenastheweightmatrixofthegraph.Inthebipartitegraphpartitioningmodelforbiclustering,thesetofobjectscorrespondstotherstvertexsetandthesetoffeaturestotheotherone.Theexpressionmatrixischosenasthebiadjacencyweightmatrixofthebipartitegraph.Bythegraphpartitioningmethodforclusteringandbiclustering,weobtainnonoverlapandexclusiveclustersandbiclusters. Thegraphpartitioningproblemisaclassiccombinatorialoptimizationproblem,anditisknowntobeNP-complete.Withdifferentcuts,thegraphpartitioningproblemcanbeformulatedasanintegerprograminmathematicalprogramming.Fortheproblemwith 12

PAGE 13

Minimumcut,whichhasbeenstudiedforalongtimeinhistory,aquadraticprogrammingformulationisalwaysused.FortheproblemwithRatiocutorNormalizedcut,itcanbeformulatedasamixedintegernonlinearprogram(MINLP)asmodeledinChapter 3 .TheMINLPisadifcultclassofoptimizationproblemswithmanyapplications.Initsmostgeneralform,aMINLPproblemcanbeformulatedasming0(x)s.t.gi(x)0,i=1,,mx2ZpRn)]TJ /F9 7.97 Tf 6.58 0 Td[(p, wheregi:Rn!Risanonlinearfunctionofxforalli=0,1,,m,andsometimesnonconvex.GlobaloptimaofMINLPproblemscanbeobtainedbylinearrelaxationforlowerbounds.ForsomeclassofMINLPproblems,somelinearizationtechniquescanbeusedtotransformthemintoequivalentintegerlinearprogrammingproblems. Recently,optimizationmethodshavefoundmanyapplicationsinelectricpowersystems.Forexample,optimizationmodelsareusedintheseproblems:unitcommitmentforautomaticgenerationcontrol,optimalpowerowandeconomicdispatch,transmissionlineswitchingandloadsheddingforpowertransmissionanddistribution,theN)]TJ /F4 11.955 Tf 12.44 0 Td[(1orN)]TJ /F3 11.955 Tf 10.15 0 Td[(kcontingencyanalysisforsystemsecurity,maximumloadpointandvoltagecollapsepointforsystemstability,andetc.Someotherproblemsinpowersystemsarestillunderdiscussionforreliability,securityandstabilityinordertoconstructanefcient,economicandreliablesmartgrid. 1.2Contributions ThegraphpartitioningmethodforclusteringbyRatiocutorNormalizedcuthasbeenstudiedformanyapplicationsinnetworkanalysis,imagesegmentation,biomedicalengineering,textminingandetc.Thespectralmethodisusedinbothclusteringandbiclustering.ThismethodusestheeigenvectorsandeigenvaluesoftheLaplacianmatrix 13

PAGE 14

ofthecorrespondinggraphforclusteringorbiclustering.However,thismethodwasuseddirectlyindataminingwithouttheproofofeffectivenessandvalidity. Inthisdissertation,weprovespectralmethodsarerelaxationsofourMINLPoptimizationmodelsforclusteringbygraphpartitioningmethodwithRatiocutorNormalizedCut.Inthemeantime,westudythesemideniteprogramming(SDP)relaxationsoftheseMINLPoptimizationmodels.NumericalexperimentsshowthatSDPrelaxationscanobtainbetterresultsthanspectralmethods.Wealsopresentequivalentintegerlinearprogrammingformulationstoobtainexactsolutionsforsmalldatacases.Similarly,modelsofbipartitegraphpartitioningbyRatiocutandNormalizedarealsostudiedforbiclustering.Additionally,allmethodsweproposedcanobtainmorethananynumberofclustersorbiclustersdirectly,whilepastresearchresultsusuallyobtainedjusttwoclusters. Inpreviousresearch,thegraphpartitioningbyminimumcutdividesthevertexsetintosubsetsofequalordifferentby1cardinalities.Linear,quadraticandsemideniteprogrammingmethodsareusedforsuchcases.Mostoftheseapproachesarestillbasedonrelaxationsofexactformulations.Inthisdissertation,consideringthecardinalitywithinalooserangeandobtainingmorethantwosubsetsdirectly,wepresentthreeequivalentzero-oneintegerlinearprogrammingreformulations.Forthegraphpartitioningproblemwithintervaluncertainweightsofedges,robustoptimizationmodelswithtwodecompositionalgorithmsareintroduced.Wealsointroducethetwo-stagestochasticgraphpartitioningproblemandpresentthestochasticmixedintegerprogrammingformulationforthisproblemwithniteexplicitscenarios.Forsolvingthisproblem,wepresentanequivalentintegerlinearprogrammingformulationwheresomebinaryvariablesarerelaxedtocontinuousones.Additionally,forsomespecicgraphs,wepresentamoresimpliedlinearprogrammingformulation.Allformulationsforgraphpartitioningproblemwithminimumcut,includingrobustandstochasticoptimization 14

PAGE 15

versions,aretestedonrandomlygeneratedgraphswithdifferentdensitiesanddifferentnumbersofscenarios. Apowergridislandisaself-sufcientsubnetworkinalarge-scalepowersystem.Inweaklyconnectedislands,limitedinter-islandpowerowsareallowed.Inthisdissertation,weusegraphpartitioningmethodstoformislandsinapowergridandformulatetheseproblemsasMINLPs.Withthesemathematicalprogrammingmodels,optimalformationofislandscanbeobtainedandthedifferentapproachescanbecompared.Throughthesimulations,experimentalresultsareanalyzedandcomparedtoprovideinsightforpowergridislanding. 1.3OutlineoftheDissertation Inthisdissertation,weusegraphpartitioningbasedoptimizationmethodsfordataclusteringandbiclustering,discussmanyaspectsofthegraphpartitioningbyminimumcut,andalsoapplythiscombinatorialandnonlinearoptimizationapproachinpowersystems. Chapter 2 brieyreviewspreviousmethodsforclusteringandbiclustering,aswellassomepreliminaryresultsforthegraphpartitioningproblem. Chapter 3 studiesquadraticallyconstrainedprogrammingapproach,semideniteprogrammingandspectralrelaxationmethodsforclusteringandbiclusteringbygraphpartitioningmethodwithRatiocutandNormalizedcut. Chapter 4 introducesseverallinearandquadraticalprogrammingapproachesforgraphpartitioningbyminimumcut,anditsuncertainformsbyrobustandstochasticoptimizationmethods. Chapter 5 usesthiscombinatorialandnonlinearoptimizationapproachtostudytheislandingprobleminpowersystems.ManynumericalexperimentsareperformedonIEEEtestpowersystems. Chapter 6 concludesthedissertationandpresentssomefutureresearchdirections. 15

PAGE 16

CHAPTER2REVIEWOFCLUSTERING,BICLUSTERINGANDGRAPHPARTITIONING Inthischapter,literaturereviewsforbothclusteringandbiclusteringwillbebrieystatedrst,andoptimizationmethodswillbereviewedwithmoredetails.Then,themathematicalprogrammingmethodsforthegraphpartitioningproblemwillbereviewedbriey.Becauseofrelationshipsbetweenclusteringandgraphpartitioning,somepreliminaryresultsforbothwillbestudiedinSection 2.3 ,andtheseresultswillbecitedthroughoutChapter 3 andChapter 4 2.1ReviewofDataClusteringandBiclustering Asmentionedin 1.1 ,bothclusteringandbiclusteringareperformedondatamatrices:clusteringonsimilaritymatrixandbiclusteringonexpressionmatrix. SimilarityMatrix.Thisdatamatrixhasbothrowsandcolumnscorrespondingtoasetofsamples(objects),withentrymeasuringthesimilaritybetweentwocorrespondingsamples.Ithassamenumbersofrowsandcolumns,anditissymmetric.Thismatrixcanbecalledsample-by-samplematrix.Throughoutthedissertation,W=(wij)NNisusedtodenotethismatrixwithNsamples.Sinceweareusinggraphpartitioningtostudyclustering,thismatrixiscorrespondingtotheweightmatrixofagraphandhassomerequirementsasdiscussedinDenition 2.3 Notethematrixforclusteringcanalsobedissimilaritymatrixwithentrydenotingthedissimilaritybetweenapairofsamples.Therearemanysimilaritymeasurementfunctionstocomputethe(dis)similarityentries[ 132 ],suchasEuclideandistance,Mahalanobisdistance,etc.Sothesimilaritymatrixcanbecomputedfromtheexpressionmatrix. ExpressionMatrix.Thisdatamatrixhasrowscorrespondingtosamples(objects),columnstofeatures,withentrymeasuringtheexpressionlevelofafeatureinasample.Eachrowiscalledafeaturevectorofthesample.Wecancallthismatrixassample-by-featurematrix.Throughoutthedissertation,A=(aij)NMisusedto 16

PAGE 17

denotethismatrixwithNsampleswithMfeatures.MoredetailsandapplicationsofthismatrixforbiclusteringwillbediscussedinSections 3.5 and 4.1 Sometimes,thematrixisformedfromallsamples'featurevectors,andthefeatures'levelinthissamplewillbeobserveddirectly.Generallywejustscaleandthenputthesevectorstogethertoformamatrixifallvectorshavethesamelength,whichmeanstheyhavethesamesetoffeatures.However,thefeaturevectorsmaynotconformeachother.Underthiscase,weshouldaddvalues(maybe0)tovectorswithnocorrespondingfeaturesinordertoformsame-lengthvectors.Insomeapplications,therearealwayslargesetofsampleswithlimitedfeatures.Sincethedevelopmentsofbiclusteringareincludingsometimeseriesmodels[ 90 119 ],andanotherkindoftimeseriesdataarealsousedinbiclustering.Thisdataalsocanbeviewedasstoredinamatrixwiththatrowsdenotesampleswhilecolumnsfromlefttorightdenoteobservedtimepoints.Forsomequalitativefeaturesorsomecases,thedatamatrixisakindofsignmatrix.Somebiclusteringalgorithmscanworkunderthissituation. Sometimes,beforeprocessingalgorithmsonthematrix,somestepsareused,suchasnormalization,discretization,valuemappingandaggression,andthedetailsofthesedatapreparationoperationsareavailableat[ 29 ]. 2.1.1ReviewofClustering Clusteringistheassignmentofobjectsintodifferentgroups(calledclusters),sothattheobjectsinthesamegroupsharesomecommontraitsoraresimilarinsomesense.Itisamethodofunsupervisedclassication,andacommontechniquefordataanalysisinmanyelds,includingdatamining,machinelearning,patternrecognition,imageanalysisandbioinformatics. Clusteringhasbeenwidelystudiedinthepast20years.MostwidelyusedmethodsforclusteringincludeK-meansclustering[ 88 ],fuzzyc-meansclustering[ 10 ],andspectralclustering[ 101 ].AgeneralreviewofclusteringisgivenbyJainetal.in[ 68 ].Thispaperpresentedanoverviewofpatternclusteringmethodsfromastatistical 17

PAGE 18

patternrecognitionperspective.Thetechniquestheyreviewedincludehierarchicalclusteringalgorithms,partitionalgorithms,mixture-resolvingandmode-seekingalgorithms,nearestneighborclustering,fuzzyclustering,articialneuralnetworksforclustering,evolutionaryandsearch-basedapproachesforclustering,andetc.Theyalsodiscussedapplicationsofclusteringinimagesegmentation,objectrecognition,andinformationretrieval. Inthesurveypaper[ 132 ],XuandWunschlistedsimilarityanddissimilaritymeasureforquantitativefeatures,suchasMinkowskidistance,Eulideandistance,city-blockdistance,supdistance,Mahalanobisdistance,Pearsoncorrelation,pointsymmetrydistance,andCosinesimilarity.Algorithmswereclassiedintohierarchical,squarederror-based,mixturedensities-based,graph-theorybased,combinatorialsearchtechniques-based,fuzzy,neuralnetworks-based,andkernel-basedclusteringmethods.Theyalsoreviewedsomedatasetsforapplications,suchasgeneexpressiondata,DNAorproteinsequencesforclustering. Recentlyin[ 7 ],Berkhinreviewedrecentadvancesmethodsinclustering,andalsodiscussedclusteringofhighdimensionaldatabydimensionallyreductionandgeneralissuesofclustering,suchasassessmentofresults,choiceofappropriatenumberofclusters,datapreparation,proximitymeasures,andhandlingoutliers. Graphmodelsarealwaysusedforclusteringasreviewedin[ 110 ].ThespectralclusteringmethodisbasedontheLaplacianmatrixofthegraph[ 47 ]andsometimesitiscalledspectralgraphpartitioning[ 101 ].Twodifferentcutsingraphpartitioningaredenedforclustering:Ratiocut[ 56 ]andNormalizedcut[ 115 ].AtutorialonspectralclusteringisgivenbyDing[ 34 ].Thespectralmethodalwaysbeginswithbipartitionofagraph[ 47 56 101 115 ]andhierarchicalclustersofamulti-waypartitioningcanbeobtainedrecursivelybythisapproach.Thedirectmulti-waygraphpartitioningforclusteringhasbeenstudiedin[ 21 ]forRatiocutand[ 55 ]forNormalizedcut.Recently,asemideniterelaxationforthemulti-wayNormalizedcutwasstudiedin[ 131 ].Inthese 18

PAGE 19

papers,thetermsclusteringandgraphpartitioningarenotdistinguishedclearly.InChapter 3 ,weareconcentratingonthegraphpartitioningmethodsbytheRatiocutandNormalizedcutandclaimthatourmethodscanbeusedforclustering. 2.1.2ReviewofBiclustering Biclustering,orcalledco-clustering,two-modeclustering,isanextensionofclustering.Itallowssimultaneousgroupingoftheobjectsandfeaturessuchthattheobjectsinthesamegroup(calledbicluster)arehighlyrelatedtoagroupoffeaturestosomelevel,andviceversa.Biclusteringhasfoundmanyapplicationsinmicroarrayandgeneexpressionanalysis[ 2 5 19 20 25 26 54 77 81 90 93 96 103 105 113 119 121 136 142 ],biomedicalengineering[ 16 77 98 ],computationalbiology[ 33 67 89 ],textmining[ 30 ]andetc.Ithasbeenstudiedrecentlyandseveralreviewscanbefoundin[ 18 38 89 122 ].Biclusteringisrelatedtobipartitegraphpartitioning[ 30 39 104 141 ].TherecentandmostwidelyusedmethodsincludeChengandChurch'salgorithm[ 25 135 136 ],randomwalkbiclusteringalgorithm[ 2 ],order-preservingsubmatrixalgorithm[ 5 ],iterativesignaturealgorithm[ 67 ],xMotif[ 93 ],optimalre-orderingalgorithm[ 33 ],algorithmbasedonnonsmoothnonnegativematrixfactorization[ 20 100 ],binaryinclusionmaximalbiclusteringalgorithm(Bimax)[ 103 ],spectralbiclusteringbasedonbipartitegraphs[ 30 39 115 ],statisticalalgorithmicmethodforbiclusteranalysis(SAMBA)[ 120 121 ],informationtheoreticbiclusteringalgorithm[ 31 ],Bayesianbiclusteringmodel[ 54 81 113 ],cMonkey[ 105 ],andetc.Mostofbiclusteringalgorithmsareunsupervisedclassication.Itdoesnotneedtohaveanytrainingsetstosupervisebiclustering.Butsupervisedbiclusteringmethodsarealsousefulinsomecasesofbiomedicineapplications[ 17 18 96 ]. Obviously,theobjectiveofbiclusteringistondbiclustersindata.Inclustering,theobtainedclustersshouldhavethepropositionsthatthesimilaritiesamongthesampleswithineachclusteraremaximizedandthesimilaritiesbetweensamplesfromdifferentclustersareminimized.Forbiclustering,thesamplesandfeaturesin 19

PAGE 20

eachbiclusterarehighlyrelated.Butthisdoesnotmeanthesamplesinthisbiclusterdonothaveotherfeatures,theyjusthavethefeaturesinthisbiclustermoreobviousandtheystillshareotherfeatures.Thus,ineachbicluster,therelationsbetweenthesamplesandfeaturesaremaximizedcloserratherthanrelationsbetweensamples(features)fromthisbiclusterandfeatures(samples)fromanotherbicluster.Thereisnostandardforjustifyingthealgorithms.Indistinctapplicationsofbiclustering,aspecicorseveralobjectivesshouldbemetsosomealgorithmsaredesignedtosatisfytheserequirements.Therearesomemethodsaretryingtocomparedifferentalgorithms,andwereferto[ 89 103 106 142 ]. Somebiclusteringalgorithmsallowthatonesampleorfeaturecanbelongtoseveralbiclusters(calledoverlapping)whilesomeothersproduceexclusivebiclusters.Inaddition,somealgorithmshavethepropertythateachsampleorfeaturemusthaveitscorrespondingbiclusterwhilesomeothersneednottobeexhaustiveandcanallowonlyndonesub-matrixorseveralonesfromdatamatrixtoformthebiclusters. TherstapproachtobiclusteringisdirectclusteringofdatamatrixbyHartigan[ 60 ]in1972.ButthetermbiclusteringwasfamousafterChengandChurch[ 25 ]usingthistechniquetodogeneexpressionanalysis.Afterthat,manybiclusteringalgorithmsaredesignedindifferentareas'applications,suchasbiologicalnetwork,microarraydata,word-documentco-clustering,biomedicalengineering,etc.,ofwhichthemostpopularapplicationsareinmicroarraydataandgeneexpressiondata. In2004,MadeiraandOliveira[ 89 ]surveyedthebiclusteringalgorithmsforbiologicaldataanalysis.Inthissurvey,theyidentiedthebiclustersintofourmajorclasses:biclusterswithconstantvalues,withconstrantvaluesonrowsorcolumns,withcoherentvalues,andwithcoherentevolutions.Thebiclusteringstructuresofadatamatrixareclassiedintoninegroupsaccordingtoalgorithms:singlebicluster,exclusiverowandcolumnbiclusters,checkerboardstructure,exclusiverowsbiclusters,exclusivecolumnsbiclusters,nonoverlappingbiclusterswithtreestructure,nonoverlappingnonexclusive 20

PAGE 21

biclusters,overlappingbiclusterswithhierarchicalstructure,andarbitrarilypositionedoverlappingbiclusters.Inaddition,theydividedthealgorithmsintoveclasses:iterativerowandcolumnclusteringcombination,divideandconquer,greedyiterativesearch,exhaustivebiclusterenumeration,anddistributionparameteridentication.Acomparisonofthesealgorithmsaccordingtotheabovethreeclasseswasgiveninthissurvey. AnotherreviewaboutbiclusteringalgorithmsisbyTanay,SharanandShamirin[ 122 ]in2004.Inthissurvey,ninewidelyusedalgorithmsarereviewedandgivenwiththeirpseudocodes.OnereviewofbiclusteringisbyBusygin,ProkopyevandPardalosin[ 18 ],and16algorithmsarereviewedwiththeirapplicationsinbiomedicineandtextmining.Recentlyin[ 38 ],Fan,BoykoandPardalosreviewedthebackgrounds,motivation,datainput,objectivetasks,andhistoryofdatabiclustering.Thebiclustertypesandbiclusteringstructuresofdatamatrixaredenedmathematically.Mostrecentalgorithms,includingOREO,nsNMF,BBC,cMonkey,etc.,arereviewedwithformalmathematicalmodels. Sincethedevelopmentofbiclusteringalgorithms,manysoftwaresaredesignedtoincludeseveralalgorithms,includingBicAT[ 4 ],BicOverlapper[ 109 ],BiVisu[ 24 ],toolboxbyR(biclust)[ 69 ]andetc.Thesesoftwareorpackagesallowtododataprocessing,biclusteranalysisandvisualizationofresults,andcanbeuseddirectlytoconstructimages. InthetoolboxnamedBicAT[ 4 ],itprovidesdifferentfacilitiesfordatapreparation,inspectionandpostprocessingsuchasdiscretization,lteringofbiclustersaccording.SeveralalgorithmsofbiclusteringsuchasBimax,CC,XMotifs,OPSMareincluded,andthreemethodsofviewingdataincludingmatrix(heatmap),expressionandanalysis.ThesoftwareBicOverlapper[ 109 ]isatoolforoverlappingbiclustersvisualization.Itcanusethreedifferentkindsofdatalesoforginaldatamatrixandresultedbiclusterstoconstructbeautifulandcolorfulimagessuchasheatmaps,parallelcoordinates,TRN 21

PAGE 22

graph,bubblemapandoverlapper.TheBiVisu[ 24 ]isalsoasoftwaretoolforbiclusterdetectionandvisualization.Besidesbiclusterdetection,BiVisualsoprovidesfunctionsforpre-processing,lteringandbiclusteranalysis.AnothersoftwareisapackagewrittenbyR[ 69 ],biclust,whichcontainsacollectionofbiclusteralgorithms,suchasBimax,CC,plaid,spectral,xMotifs,etc,preprocessingmethodsfortwowaydata,andvalidationandvisualizationtechniquesforbiclusterresults.Forindividualbiclusteringsoftware,therearealsosomepackageavailable[ 18 122 ]. 2.2ReviewoftheGraphPartitioningProblem Inthissection,thegraphpartitioningproblemactuallydenotesthegraphpartitioningproblemwithminimumcut,whichisthedirectsummationofweightsofedgesconnectingdifferentsubsets.ThisproblemisanNP-completecombinatorialoptimizationproblem[ 51 ].ApplicationsofgraphpartitioningcanbefoundinVLSIdesign[ 72 ],datamining[ 21 30 35 56 104 110 141 ],biologicalorsocialnetworks[ 41 44 ],powersystems[ 11 83 111 123 139 ],parallelcomputing[ 61 ]andetc. AveryearlymethodforgraphpartitioningproblemistheKernighan-Linalgorithm[ 75 ],whichisaheuristicalgorithm.Thereareseveralsurveys[ 1 37 48 ]regardingheuristicandexactalgorithmsforgraphpartitioning.Someheuristicalgorithmscancomputeapproximatesolutionsveryfast,forexample,spectralmethodsin[ 21 30 34 39 55 63 101 ],multilevelalgorithmsin[ 62 72 74 117 127 ],optimization-basedmethodsin[ 46 ],andetc.SoftwarebasedonheuristicmethodsincludeMETIS,Chaco,Party,PaToH,SCOTCH,Jostle,ZoltanandHUNDasreviewedin[ 59 ]. Somemathematicalprogrammingmethods,includinglinearprogramming[ 14 42 84 ],quadraticprogramming[ 42 57 59 ],andmanysemideniteprogrammingrelaxations[ 3 70 71 84 129 ],werealsousedforgraphpartitioning. Somemethods([ 3 57 59 71 ])decomposethevertexsetintotwosubsets,orcalledbisection,whilesomeones([ 42 55 58 91 ])obtainmorethantwosubsetsdirectly.Inpreviousresearch,thegraphispartitionedintoequalordifferentby1 22

PAGE 23

cardinalitiesforallsubsetsbylinearprogramming[ 84 ]orsemideniteprogramming[ 70 84 ].Someapproaches,forexample,quadratic[ 57 58 ]andsemidenite[ 129 ]programmingmethods,requirethegivencardinalitiesofallsubsets. Throughthewholedissertation,allmethodscanobtainmorethantwosubsetsdirectly.InChapter 3 ,weusealooserestrictionforgraphpartitioningandclusteringthatallcardinalitiesofsubsetstakevaluesintheintegerrange[Cmin,Cmax].ThecardinalitiesofsubsetsareonereasonthatRatiocutandNormalizedwereintroducedforgraphpartitioning. 2.3PreliminaryofGraphPartitioningandClustering AsdiscussedinSection 1.2 ,weareconcentratingondirectmulti-wayclustering.Becauseofconnectionsbetweenclusteringandgraphpartitioning,ourmethodisbasedongraphpartitioningwithrespecttodifferentcuts.Inthissection,werstdenethepartitionmatrixfordirectmulti-waypartitioning.Thedenitionsofcutsaswellassomeotherconceptsarealsostudiedinthesection.Attheendofthissection,weestablishtheoptimizationmodelsofgraphpartitioningforclustering.AssumethatthematrixforclusteringisW=(wij)NN.Throughoutthissection,wearendingKclustersinthedatamatrixWaspresentedinSection 2.1 PartitionMatrixX Denition2.1. ThepartitionmatrixisdenedasarectangularNKmatrixX=(xik)NK,wherexik2f0,1g,rowsumsarePkk=1xik=1foralli=1,,N,andcolumnsumsarePNi=1xik2[Cmin,Cmax]forallk=1,,K. Thismatrixincludesalldecisionvariablesinourgraphpartitioningmodelsforclustering.ThenumberNisthenumberofobjectstoperformclusteringandKisthenumberformulti-wayorK-wayclustering.TheclusteringbythismatrixXisnonoverlappingandexclusive. TheconstraintofthesumPkk=1xik=1requirestoputtheithobjectintoexactlyonecluster.ThecolumnsumPNi=1xik:=nkdenesthesizeorcardinalityconstraintofthe 23

PAGE 24

kthcluster.Sincexik2f0,1g,therowsumstakeintegervaluesbetweenthelowersizeboundCminandupperboundCmax.Thesetwoboundsareknownparametersandcanbechosenroughlyfromf1,,Ng.MorediscussionsofthecardinalityconstraintwillbestatedinSection 4.1 Remark1. Allrowsumsandcolumnsumsensurethateachobjectbelongstoexactlyoneclusterandallobjectshavecorrespondingclusters.ThisisguaranteedbythefactthatKXk=1nk=KXk=1NXi=1xik=NXi=1KXk=1xik=NXi=11=N,whichmeansthatnkcantakeanyintegervaluesin[Cmin,Cmax]buttheirsumisxedasN.Thus,thepartitionmatrixhasexactlyN1's. GraphPartitioning Denition2.2. LetG=(V,E)beanundirectedgraphwithasetofverticesV=fv1,v2,,vNgandasetofedgesE=f(vi,vj):edgebetweenverticesviandvj,1i,jNg,whereNisthenumberofvertices.TheK-graphpartitioningconsistsofdividingthevertexsetVintoKdisjointsubsetsV1,V2,,VK,i.e.,V1[V2[[VK=VandVk\Vk0=;forallk,k0=1,2,,K,k6=k0. ThegraphGisundirectedandnoloopsareallowed.Inthedissertation,weassumethateachedgeofGhasnonnegativeweightsandthevertexhasnoweight.Always,thisisaweightedgraph.LetbeapartitionofthegraphG,i.e.,=(V1,V2,,VN),wherethesubsetsV1,V2,,VNsatisfytheDenition 2.2 .Now,weconstructabijectionbetweenthepartitionmatrixXandthegraphpartitioning. LettheithrowofXcorrespondtovertexviandthekthcolumncorrespondtosubsetVkof.TheentryxikdenesthatwhetheravertexviisinsubsetVkifxik=1ornotifxik=0.ThecolumnsumPNi=1xikiscorrespondingtothenumberofverticesinsubsetVk.TheconstraintPKk=1xik=1ofXensuresthatvertexvicanbelongtoexactonesubset,andthisalsoensuresthatthesubsetsaredisjoint.AssumethatjVkj=nk=PNi=1xikfork=1,,K.ThuswehavePKk=1nk=jVj,whichensuresthat 24

PAGE 25

theunionofthesesubsetsisV.Usually,thenksareunknownbeforepartitioningandweassumethattheytakeintegervaluesinarangeasthatinthepartitionmatrix.Wehavetransformedthepartitionmatrixintoagraphpartitioning.Thereversedirectionofthebijectioncanbeconstructedeasily. Therefore,thefeasibleregionforthegraphpartitioningintoKsubsetscanbeexpressedas FK=f(xik)NK:xik2f0,1g,KXk=1xik=1,CminNXi=1xikCmaxg. (2) Remark2. Generally,thenumberKcanbechosenfromthesetf2,3,,N)]TJ /F4 11.955 Tf 12.16 0 Td[(1g.ThevalueK=1istoputallobjectsintooneclusterandK=Nmeansoneclusterforeachobject.Thesearetrivialcases.Forminimumcut(Denition 2.5 )ofgraphpartitioning,thedeterminationofKisstudiedinSection 4.1 byintroducingapenaltyfunction. Remark3. ThesizeorcardinalitynkofthesubsetVkisalwaysunknownbeforepartitioning.Forminimumcut(Denition 2.5 )ofgraphpartitioning,thecardinalitiesarealwayschosenasequalsN=K[ 70 84 ],pre-givenintegers[ 58 129 ]orvaluesinarange[ 42 43 ].InChapter 3 ,weusealooserestrictionforgraphpartitioningandclusteringthatallthecardinalitiesofsubsetstakevaluesintheintegerrange[Cmin,Cmax].ThecardinalitiesofsubsetsareonereasonthatRatiocutandNormalizedwereintroducedforgraphpartitioning.AsshowninSection 3.2 andSection 3.3 ,therelaxationsdonotneedanyinformationoftherangeforclustering. MatricesonGraphs Denition2.3. TheweightmatrixW=(wij)NNofthegraphG=(V,E)isasymmetricmatrixwithnonnegativeentries.Theentrywij>0denotestheweightofedge(vi,vj)andwij=0ifnoedge(vi,vj)existsbetweenverticesviandvj. 25

PAGE 26

Theweighteddegreediofthevertexviisdenedasthesumofweightsoftheedgesincidentwithitself,i.e.,di=PNj=1wij.Theweighteddegreematrixisadiagonalmatrixformedbythedegreesofallvertices,i.e.,D=diag(d1,d2,,dN). TheLaplacianmatrixLisdenedasthedifferenceofDandW,i.e.,L=D)]TJ /F3 11.955 Tf 11.96 0 Td[(W. ForundirectedgraphsGwithoutloops,thematrixWissymmetricandwii=0fori=1,,N.ItistheadjacencymatrixofGifwij=1denotestheexistenceofedge(vi,vj)andotherwisewij=0.Thus,thematrixWisalsocalledadjacencyweightedmatrixforG. Thenotationdiagdenotestoformadiagonalmatrixfromthevector,andwealsouseitinthefollowingasthesteptoformavectorfromthediagonalentriesofthematrix. Cut Denition2.4. Thecutsetofthepartitionincludestheedgeswithtwoendsindifferentsubsets.ThecutsetbetweenVkandVk0isthesetofedges(vi,vj)swithvi2Vk,vj2Vk0fork6=k0.ThecutbetweensubsetsVkandVk0isthesumofweightsoftheedgesinthecorrespondingcutset,i.e.,fork6=k0,cut(Vk,Vk0)=Xi,j:vi2Vk,vj2Vk0wij. Theedgeinthecutsetiscalledcutedge.Inthedenitionofcut,wedonotrequirethat(vi,vj)2Esincewij=0ifnoedgeexists.Thefactthatcut(Vk,Vk0)=cut(Vk0,Vk)canbeeasilydrawnfromthesymmetryoftheweightmatrixW. Inthefollowing,thenotationcut(Va,Vb)isusedtodenotethesumofweightsofedgeswithoneendinVaandanotherinVb,whetherVaandVbaredisjointornot(thetwovertexsetscanbethesame,oroneisasubsetofanother).Forexample,thenotationcut(V1,V1)isthesumofweightsofedgeswithtwoendsinvertexsetV1,andthefactcut(V1,V)=cut(V1,V1[V2[[VK)=2cut(V1,V1)+cut(V1,V2)++cut(V1,VK)isobtainedbythefollowingproposition. 26

PAGE 27

Proposition2.1. Forthegraphpartitioning=(V1,,VK),thecutscanbeexpressedbythepartitionmatrixXinthefollowing:cut(Vk,Vk0)=NXi=1NXj=1wijxikxjk0,k6=k0,cut(Vk,Vk)=1 2NXi=1NXj=1wijxikxjk,cut(Vk,V)=NXi=1NXj=1wijxik. TherstoneisfromtheDenition 2.4 ofcut.ThesecondoneisthesumofweightsoftheedgeswithtwoendsinVkandeachedgecontributesonce.AssumethatdVk=Pi:vi2Vkdi=Pi:vi2VkPNj=1wij,thuswehavedVk=cut(Vk,V).Iftheseexpressionsofcutspresentinthematrixforms,wehavethefollowingproposition. Proposition2.2. BasedonthepartitionmatrixX,wehaveXTWX=0BBBBBBB@2cut(V1,V1)cut(V1,V2)cut(V1,VK)cut(V2,V1)2cut(V2,V2)cut(V2,VK)............cut(VK,V1)cut(VK,V2)2cut(VK,VK)1CCCCCCCA,XTDX=diag(dV1,,dVK),XTLX=0BBBBBBB@PKk=2cut(V1,Vk))]TJ /F3 11.955 Tf 9.3 0 Td[(cut(V1,V2))]TJ /F3 11.955 Tf 47.27 0 Td[(cut(V1,VK))]TJ /F3 11.955 Tf 9.3 0 Td[(cut(V2,V1)PKk=1,k6=2cut(V2,Vk))]TJ /F3 11.955 Tf 47.27 0 Td[(cut(V2,VK)............)]TJ /F3 11.955 Tf 9.3 0 Td[(cut(VK,V1))]TJ /F3 11.955 Tf 9.3 0 Td[(cut(VK,V2)PK)]TJ /F8 7.97 Tf 6.59 0 Td[(1k=1cut(VK,Vk)1CCCCCCCA, whereWistheweightmatrix,Distheweighteddegreematrix,andListheLaplacianmatrix. 27

PAGE 28

ThispropositioncanbeprovedbythefactsfromProposition 2.1 .Threedifferentcutsofgraphpartitioningforclusteringarepresentedbasedonthisproposition. Minimumcut,RatioCutandNormalizedCut Denition2.5. Forthegraphpartitioning=(V1,,VK),theminimumcutisthesumofweightsoftheedgeswithendsindistinctsubsets,i.e.,Mcut=KXk=1KXk0=k+1cut(Vk,Vk0).TheRatiocutofthegraphpartitioning=(V1,,VK)isafractionofthecutstothesizesofthecorrespondingsubsets,anditisdenedasRcut=KXk=1KXk0=1,k06=kcut(Vk,Vk0) jVkj.TheNormalizedcutofthegraphpartitioning=(V1,,VK)isafractionofthecutstotheweighteddegreesofvertices,anditisdenedasNcut=KXk=1KXk0=1,k06=kcut(Vk,Vk0) dVk. Usually,thetermgraphpartitioningproblem(GPP)referstondingthepartitioningamongallpossiblepartitionsofGwithminimumcut.TheRatiocut[ 56 ]andNormalizedcut[ 115 ]wereintroducedforclustering. IfWisthedatamatrixtoperformclusteringandwijmeasuresthesimilarityorclosenessoftheithandjthobjects,wecandenetheintersimilarityandintrasimilarityofclusteringbythegraphpartitioning=(V1,,VK)inthefollowingdenition. IntersimilarityandIntrasimilarity Denition2.6. AssumethatthepartitionmatrixXisperformedforclusteringofNobjectswithsimilaritymatrixW,theintersimilarityofclustersbythegraphpartitioning=(V1,,VK)isthesumofsimilaritiesoftheobjectsfromdifferentclustersanditcanbedenotedby1 2tr(XTLX). 28

PAGE 29

Theintrasimilarityofclustersisthesumofsimilaritiesoftheobjectsfromthesameclustersanditcanbedenotedby1 2tr(XTWX). Herethenotationtrdenotesthetraceofthematrix.AsWisasimilaritymatrix,thegraphpartitioningproblemofminimizingminimumcutndsthebestpartitionsuchthatobjectsindifferentgroups(subsets)haveasleastsimilarityaspossible.Thisistherequirementofclustering.However,asshowninFigure 2-1 ,theminimumcutalwaysleadtounnaturalbias[ 115 ]forpartitioningoutsmallsubsetsofvertices.ThisisthereasonthatRatiocutandNormalizedcutareintroducedforclustering.Ratiocutistobalancethecardinalitiesofeachsubsetsorclusters,whileNormalizedcuttriestobalancetheweightsamongthesubsetsorclusters.ThenumeratorsoffractionsinbothRatiocutandNormalizedcutmeasurethesimilaritiesamongdifferentsubsets,andthesetwocutsshouldbeminimizedingraphpartitioningforclustering.BesidesRatiocutandNormalizedcut,someothercutsarealsointroducedforclustering,suchasmin-maxcut[ 35 ],ICA(Isoperimetricco-clustering)cut[ 104 ]. GraphMinimumcut RatioCutNormalizedCut (inthisgraphN=12,K=2) Figure2-1. ComparisonsofMinimum,RatioandNormalizedCuts 29

PAGE 30

ByDenition 2.5 andDenition 2.6 ,theminimumcutofgraphpartitioning=(V1,,VK),andintersimilarityforclusteringofWbasedonthesamepartitionmatrixX,areequaltoeachother,i.e.,Mcut=1 2tr(XTLX). Inclustering,wewanttominimizetheintersimilarityamongtheseKclusters.Correspondingly,wehavethegraphpartitioningproblemtominimizetheminimumcutandtheformulationisminX2FK1 2tr(XTLX). (2) Remark4. Tominimizeintersimilarity1 2tr(XTLX)isequivalenttomaximizeintrasimilar-ity1 2tr(XTWX).Thisisobtainedfromthefacttr(XTLX)+tr(XTWX)=tr(XTDX)=PKk=1dVk=PNi=1di=PNi=1PNj=1wij,axedvalueforagivenmatrixW.Thusbothmin1 2tr(XTLX)andmax1 2tr(XTWX)canbeusedforgraphpartitioningbyminimumcutinChapter 4 Sinceabipartitegraphisaspecialcaseofageneralgraph,thebipartitegraphpartitioningproblemcanbeeasilytransformedintographpartitioningproblem.Forbiclustering,thebipartitegraphpartitioningbyminimumcut,RatiocutandNormalizedcutisused.Forconceptionsandnotationsofbiclustering,bipartitegraphpartitioning,partitionmatrix,expressionmatrix,cuts,andetc,werefertoSection 2.1.2 ,Section 3.5 ,Section 4.1 andSection 4.2.3 30

PAGE 31

CHAPTER3MULTI-WAYCLUSTERINGANDBICLUSTERINGBYTHERATIOCUTANDNORMALIZEDCUT Inthischapter,weconsiderthemulti-wayclusteringproblembasedongraphpartitioningmodelsbytheRatiocutandNormalizedcut.Weformulatetheproblemusingnewquadraticmodels.Spectralrelaxations,newsemideniteprogrammingrelaxationsandlinearizationtechniquesareusedtosolvetheseproblems.Ithasbeenshownthatourproposedmethodscanobtainimprovedsolutions.Wealsoadaptourproposedtechniquestothebipartitegraphpartitioningproblemforbiclustering. Thischapterisorganizedasfollows:InSection 3.1 ,WepresenttheoptimizationmodelsforgraphpartitioningwithRatiocutandNormalizedcut;Section 3.2 isabriefreviewofthespectralrelaxationapproaches;InSection 3.3 ,wepresentthesemideniteprogrammingapproaches;Section 3.4 includesthequadraticallyconstrainedprogrammingapproacheswithlinearizationtechniques;InSection 3.5 ,wediscussthebipartitegraphpartitioningandbiclustering;InSection 3.6 ,wepresentthenumericalexperimentsofallproposedalgorithms;Section 3.7 concludesthechapter. 3.1ClusteringandGraphPartitioning FollowingnotationsandpreliminaryresultsinSection 2.3 ,weareconcentratingontheRatiocutandNormalizedcutofgraphpartitioningforclusteringinthischapter.ThegraphpartitioningwithMinimumcutwillbediscussedinChapter 4 Lemma3.1. DeningamatrixYbasedonpartitionmatrixXasY=Xdiag(1=p n1,,1=p nK),wherenk=PNi=1xikfork=1,,K,theRatiocutcanbeexpressedintheformofYandLaplacianmatrixLasRcut=tr(YTLY). 31

PAGE 32

Proof.AssumethatP=diag(1=p n1,,1=p nK).FromthedenitionofY,wehavethefollowingrelationstr(YTLY)=tr((XP)TL(XP))=tr(P(XTLX)P). FromtheProposition 2.2 ,wehavethattr(YTLY)=KXk=1(XTLX)kk nk=KXk=1KXk0=1,k06=kcut(Vk,Vk0) jVkj, whichnishestheproofbytheDenition 2.5 fortheRatiocut. Thislemmawaspresentedin[ 21 ].Herethenotation(XTLX)kkdenotestheentryinthekthrowandkthcolumnofthematrixXTLX.Thus,theproblemofminimizingRatiocutofgraphpartitioningforclusteringcanbeexpressedinthematrixformasfollows:mintr(YTLY) (3)s.t.Y=XP,X2FK,P=diag(1 p (XTX)11,,1 p (XTX)KK). Bysolvingthisproblem,wecanndthegraphpartitioning=(V1,,VK)withsmallestRcutbytheoptimalsolutionX.TheassignmentofvertexvitosubsetVkisdecidedbyxik.However,wecanusethematrixY=(yik)NKdirectlytopartitionthevertexset.Ifyik=0,thevertexvidoesnotbelongtosubsetVk;Ifyik>0,thevertexvibelongstosubsetVk.ThematrixYshouldsatisfysomespecicconstraints,andwepresenttheminthefollowingtheorem. Theorem3.1. YandXareequivalentforgraphpartitioningwithrespecttoRatiocutbythefollowingformulationbasedonY:mintr(YTLY) (3)s.t.YTY=I, (3) 32

PAGE 33

YSe=e, (3)CminSYTeCmax, (3)S=diag(s1,,sK),Y=(yik)NK, (3)yik2f0,s)]TJ /F8 7.97 Tf 6.58 0 Td[(1kg,sk>0, (3)i=1,,N,k=1,,K, whereeisavectorofallelementsbeing1withconformdimension. Proof.AsshowninLemma 3.1 ,theobjective( 3 )istominimizetheRatiocut.Weneedtoshowthattheconstraints( 3 )-( 3 )willpresentagraphpartitioning=(V1,,VK)asdenedinDenition 2.2 Fromconstraints( 3 ),thekthcolumnofYcanbeeither0orapositivenumbers)]TJ /F8 7.97 Tf 6.59 0 Td[(1k.ThepositivenessofyikdecideswhethervertexvibelongstosubsetVkornot. Ifyik>0,theelement(YS)ikinconstraints( 3 ),( 3 )is1;Otherwise,ifyik=0,(YS)ik=0.ThisisobtainedbythestructuresofYandSinconstraints( 3 ),( 3 ).Thus,YSisapartitionmatrix.Theconstraint( 3 )ensuresthateveryrowofYShasthesum1,i.e.,everyvertexbelongstoexactlyonesubset.Theconstraint( 3 )guaranteesthateverycolumnsumofYSisintherange[Cmin,Cmax],thesizeconstraintsofeachsubset. Then,thesubsetscanbedenotedbyVk=fvi:yik>0,i=1,,Ng.Assumethatnk=jVkj.SinceYTY=diag(n1=s21,,nK=s2K),fromtheconstraint( 3 ),wehavenk=s2k=1orsk=p nkforallk=1,,K. Therefore,wehaveusedtheformulation( 3 )-( 3 )toobtainagraphpartitioning.Thereversedirectioncanbeprovedsimilarly. Remark5. In[ 115 ],theNormalizedassociationforbipartitionofgraphsisdened.Toextendformulti-wayorK-waypartitioning,theNormalizedassociationforgraph 33

PAGE 34

partitioningisdenedasNassoc=KXk=1cut(Vk,Vk) dVk.ByProposition 2.2 andNormalizedcutdenedinDenition 2.5 forthesamepartitioningandX,wehavetherelation2Nassoc+Ncut=K,whichisapre-givenparameter.Thus,wecanalsomaximizeNormalizedassociationinsteadofminimizingNormalizedcut,asthatforminimumcut. ForNormalizedcutofgraphpartitioning,wehavesimilarresultsasRatiocuttodeneZinthefollowinglemma.AssumethatIisanidentitymatrix. Lemma3.2. LetthematrixW0beW0=D)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2WD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2andthematrixL0beL0=I)]TJ /F3 11.955 Tf 12.06 0 Td[(W0.DeningamatrixZbasedonpartitionmatrixXasZ=D1=2Xdiag(1=p dV1,,1=p dVK), wheredVk(k=1,,K)isdenedinProposition 2.1 ,theNormalizedcutcanbeexpressedintheformofZandLaplacianmatrixLasNcut=tr(ZTL0Z). Proof.AssumethatQ=diag(1=p dV1,,1=p dVK).FromthedenitionofZ,wehavetherelationsZTL0Z=(D1=2XQ)T(I)]TJ /F3 11.955 Tf 11.96 0 Td[(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2WD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2)(D1=2XQ)=QXTD1=2(I)]TJ /F3 11.955 Tf 11.96 0 Td[(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2WD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2)(D1=2XQ)=Q(XT(D)]TJ /F3 11.955 Tf 11.96 0 Td[(W)X)Q=Q(XTLX)Q. 34

PAGE 35

FromtheProposition 2.2 ,wehavethattr(ZTL0Z)=KXk=1(XTLX)kk dVk=KXk=1KXk0=1,k06=kcut(Vk,Vk0) dVk, whichnishestheproofbyDenition 2.5 fortheNormalizedcut. Thislemmawaspresentedin[ 55 ].Thus,theprogramofminimizingNormalizedcutofgraphpartitioningforclusteringcanbeexpressedinthematrixformasfollows:mintr(ZTL0Z) (3)s.t.Z=D1=2XQ,X2FK,Q=diag(1 p (XTDX)11,,1 p (XTDX)KK). Theorem3.2. ZandXareequivalentforgraphpartitioningwithrespecttoNormalizedcutbythefollowingprogrambasedonZ:mintr(ZTL0Z) (3)s.t.ZTZ=I, (3)ZTe=diag(D1=2), (3)CminTZTD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2eCmax, (3)T=diag(t1,,tK), (3)(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2Z)ik2f0,t)]TJ /F8 7.97 Tf 6.59 0 Td[(1kg,tk>0, (3)i=1,,N,k=1,,K. Thistheoremwaspresentedin[ 131 ]andcanbeprovedusingthemethodsinTheorem 3.1 .ThesubsetVkcanbedenotedbyVk=fvi:zik>0,i=1,,Ng.Theconstraint( 3 )ensuresthateachvertexcanbelongtoexactlyonesubsetandconstraint( 3 )guaranteesthateachsubsethasthesizeintherange[Cmin,Cmax].ThematrixD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2ZTcanbeconsideredasapartitionmatrix. 35

PAGE 36

3.2SpectralRelaxationApproaches 3.2.1SpectralMethods AsmentionedinSection 2.1.1 ,mostgraphpartitioningbasedclusteringmethodsusethespectralmethod.WewillshowthatthespectralmethodofgraphpartitioningforclusteringisarelaxationofourproposedoptimizationmodelsinSection 3.1 .Thismethodisbasedonspectralgraphtheory[ 27 ]andtheeigenvaluesandeigenvectorsofamatrixareusedforpartitioning.Formulti-wayclustering,thespectralmethodshavebeenstudiedin[ 21 ]and[ 55 ].Inthissection,webrieyintroducethismethodandwillcompareitwithotherproposedmethodsinSection 3.6 FromTheorem 3.1 ,thedecisionmatrixYhasthepropertyYTY=I,andsimilarly,matrixZinTheorem 3.2 hasthepropertyZTZ=I.Inthefollowing,weusespectralrelaxationstosolvethegraphpartitioningwithRatiocutandNormalizedcut.Letx1,x2,,xkbethekrowsofpartitionmatrixX,i.e.,X=(x1,x2,,xK).Similarly,Y=(y1,,yK)andZ=(z1,,zK). Therefore,arelaxationformoftheprograminTheorem 3.1 tominimizeRatiocutismintr(YTLY) (3)s.t.YTY=I. Inaddition,bythecolumnsy1,,yK,theRatiocutcanbeexpressedasRcut=tr(YTLY)=KXk=1yTkLyk, andtheconstraintsin( 3 )areyTkyk=1fork=1,,K. ThedimensionofmatrixYTLYisKK.ByK.Fan'stheorem[ 27 ],optimalsolutiontotheprogram( 3 )iseigenvectors1,,KsuchthatLk=kkwithalowerboundPKk=1kforobjectivefunction,i.e.,KXk=1kminY:Formulation( 3 )-( 3 )Rcut. 36

PAGE 37

Roughly,wechooseyk=kfork=1,,NasthesolutionsforRatiocutofgraphpartitioning.SinceksareeigenvectorsofL,thismethodiscalledspectralrelaxations. Similarly,arelaxationformoftheformulationinTheorem 3.2 tominimizeNormalizedcutismintr(ZTL0Z) (3)s.t.ZTZ=I. ForNormalizedcut,wehavethefollowingpropertyfortheobjectivefunctioninTheorem 3.2 Lemma3.3. BythepartitionmatrixX=(x1,x2,,xk),theNormalizedcutcanbeexpressedasNcut=tr(ZTL0Z)=KXk=1xTkLxk xTkDxk. Proof.ByLemma 3.2 ,thematrixL0isI)]TJ /F3 11.955 Tf 12.38 0 Td[(W0,whereIcanexpressedasD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2DD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2andW0=D)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2WD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2.Thus,wehavetherelationsZTL0Z=ZT(I)]TJ /F3 11.955 Tf 11.96 0 Td[(W0)Z=ZT(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2DD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2)]TJ /F3 11.955 Tf 11.95 0 Td[(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2WD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2)Z=ZTD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2(D)]TJ /F3 11.955 Tf 11.95 0 Td[(W)D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2Z=ZTD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2LD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2Z=(D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2Z)TL(D)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2Z)=(XQ)TL(XQ)=(X(XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1=2)TL(X(XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1=2)=((XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1=2)XTLX((XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1=2), whereQ=(XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1=2isfromLemma 3.2 .Then,tr(ZTL0Z)=tr((XTDX))]TJ /F8 7.97 Tf 6.59 0 Td[(1(XTLX)). Thisresultwaspresentedin[ 131 ].ByK.Fan'stheorem[ 27 ]andLemma 3.3 ,theoptimalsolutiontothisprogramiseigenvectors'1,,'Kfromthegeneralized 37

PAGE 38

eigenvalueproblemL'k=0kD'k,withalowerboundPKk=10kforobjectivefunctionKXk=10kminZ:Formulation( 3 )-( 3 )Ncut.Similarly,theeigenvectors'kscanbechosenasthesolutionszksforNormalizedcutofgraphpartitioning.Insteadofsolvingthisgeneralizedeigenvalueproblem,wecansolvetheeigenvalueproblemD)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2(D)]TJ /F3 11.955 Tf 11.95 0 Td[(W)D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2'k=0k'k. AlgorithmSpectralRelaxations Input:DataMatrixW=(wij)NNandPositiveIntegerKOutput:PartitionMatrixX=(xik)NKandtheMinimizedRatiocutRcutandNormalizedcutNcutwithrespecttoXStep1:ConstructtheLaplacianmatrixLandtheweighteddegreematrixDfromWasdenedinDenition 2.3 ;Step2:ForRatiocut,solvetheeigenvalueproblemLk=kktoobtain;ForNormalizedcut,solvetheproblemD)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2(D)]TJ /F3 11.955 Tf 11.96 0 Td[(W)D)]TJ /F8 7.97 Tf 6.59 0 Td[(1=2'k=0k'ktoobtain;Step3:UsethedirectionalcosinemethodtoobtainXsfrom,forRatiocutandNormalizedcut,respectively;(oruserandomizedprojectionheuristicmethodorclusteringroundingtoobtainXfrom,directly)Step4:ObtainthepartitionmatrixXfromXsandcomputethecorrespondingRatiocutandNormalizedcut. Assumethat=(1,,K)and=('1,,'K),whereisarelaxedsolutionforYandisarelaxedsolutionforZ.Aftersolvingtheeigenvalueproblems,weobtaintherelaxedsolutionsand,whichmaynotbefeasiblefortheoriginalpartitioningproblem.Next,wepresentthreeapproachesforobtainingthefeasiblepartitionmatrixXfrom,:directionalcosinemethod[ 21 ],randomizedprojectionheuristicmethod[ 50 ] 38

PAGE 39

andclusteringrounding.Werstpresentthealgorithmforsolvinggraphpartitioningforclusteringbyspectralrelaxationsinabovetable. 3.2.2RelaxedSolutionstoIntegerOnes AssumethatXs=XXT.Thus,theentry(Xs)ijdenoteswhethervertexviandvjareinthesamesubset((Xs)ij=1)ornot((Xs)ij=0)foralli,j=1,,N.OncewehavethematrixXs,thecorrespondingpartitionmatrixXcanbeconstructedeasily.Beforepresentingtheapproaches,werststatethefollowingtheorem. Theorem3.3. GiventhematricesY,ZasdenedinTheorem 3.1 andTheorem 3.2 ,wehavethecorrespondingmatrixXsasXs=DyYYTDy, (3)Xs=DzZZTDz, (3) whereDy=diag(,1 p Pky2ik,),Dz=diag(,1 p Pkz2ik,),respectively. Proof.Werstprove( 3 ).AssumethatYhasNrowvectorsasyr1,,yrN.SinceDyisadiagonalmatrix,theentry(Xs)ijcanbeexpressedasyriyrTj p Pky2ikPky2jk.AsshowninTheorem 3.1 ,eachrowhasonlyonenone-zeroelements.Assumethattheithrowyrihasthenon-zeroentryyi,kandjthrowyrjhasthenon-zeroentryyj,k0.Ifk=k0,wehave(Xs)ij=1;otherwise,(Xs)ij=0.BythedenitionofXs,weproved( 3 ).TheproofofZtoXsissimilar. Directionalcosinemethod[ 21 ].Letthematrixbe=(ik)NK.Assumethat~XsistheapproximatesolutionofXsobtainedfrom,byTheorem 3.3 ,wehave~Xs=diag(,1 p Pk2ik,,)Tdiag(,1 p Pk2ik,,), andthus,wehavetheentry(~Xs)ij=Pkikjk q (Pk2ik)(Pk2jk). 39

PAGE 40

Thisentryisexactlythecosineoftheanglebetweentheithrowandjthrowvectorsof.Bythepropertyofcosinefunction,(~Xs)ij=1ifthetworowvectorsareinthesamedirection;(~Xs)ij=0iftheyareorthogonaltoeachother.Fromtheapproximationmatrix~XstoX,usinganotherefcientclusteringalgorithm,suchasK-meansclusteringonXs,canfulllit.In[ 21 ],aheuristicmethodwaspresented.ThismethodrstselectsKverticesastheprototypesoftheKsubsets;theverticeswithdirectionalcosinewithincos(=8)ofaprototypeareaddedtothatprototype'ssubset;andtheremainingverticesaremergedintosubsetsbycomputinghyperedgecutsfromeachleftvertexandexistingsubsets. ForNormalizedcut,thematrixisapproximatedbyandtheapproximationofXscanbeobtainedbytheequation( 3 ). Randomizedprojectionheuristicmethod[ 50 ].Thismethodwasintroducedin[ 50 ]forsolvingthemaxK-cutproblem.Assumethattheeigenvectors01,,0Narenormalizedrowvectorsof.Thestepsforthismethodinclude:obtaintheapproximatedsolutions01,,0N;chooseKrandomvectorsr1,,rK;obtainxikaccordingtowhichr1,,rKisclosesttoeach0i,i.e.,xik=1ifandonlyif(0i)Trk(0i)Trk0forallk6=k0,k0=1,,K.ThestepsforNormalizedcutaresimilar. Clusteringrounding.Inaddition,consideringeachrowof~asavertex,aclusteringmethod,suchasK-meansclustering,canbeusedtoobtainthepartitionmatrixXinsteadofthesetwomethods.Thismethodwasusedin[ 131 ]. Inthischapter,weconsiderthedirectK-waypartitioningforclustering.Manyresearchpapers[ 30 115 ]arebasedonbipartitionorbisectionofgraphs.Thespectralmethodforbisectionusessecondsmallesteigenvalueandthesignofitscorrespondingvectorisusedtopartitionthevertexsetintotwosubsets.Therecursivetwo-waypartitioningisusedtoobtainmulti-waypartitioning.However,ateachiterationoftwo-waypartitioning,thejudgementisneededtocheckwhetherfurtherpartitioningis 40

PAGE 41

needed.Themethodsdiscussedinthischapterareobtainingmulti-waypartitioningdirectly. 3.3SemideniteProgrammingRelaxations 3.3.1SemideniteProgrammingMethods Semideniteprogramming(SDP)isasubeldofconvexoptimizationconcernedwiththeoptimizationofalinearobjectivefunctionovertheintersectionoftheconeofpositivesemidenitematriceswithanafnespace.AlinearprimalSDPhastheformmintr(CX)s.t.tr(AiX)=bi,i=1,,m,X0, whereX0denotesthatmatrixXispositivesemidenite.Inaddition,ABdenotesthatA)]TJ /F3 11.955 Tf 12.3 0 Td[(Bispositivesemidenite.Inthefollowing,WeuseA0todenotethateveryentryofAisnonnegative. AsdenedinSection 2.3 ,thepartitionmatrixisX2FK.AssumethatXs=XXT,theobjectiveofminimumcuttr(XTLX)canbereplacedbytr(LXs).HereweconsiderseveralrelaxationsofXsbasedonFK.FK=fX:Xe=e,XTe2[Cmine,Cmaxe],xik2f0,1gg,TK=fXs:9Xs2FKsuchthatXs=XXTg,E=fXs:Xs=XTs,diag(Xs)=e,Xse2[Cmine,Cmaxe]g,N=fXs:Xs0g,P=fXs:Xs=XTs,Xs0g,CK=fXs:Xs=XTs,Xi
PAGE 42

Thesesetsareallreviewedin[ 70 ].ThesetTKisincludedinthesetsE,N,P,CK,D.TheconvexhullofTKisequivalenttoFKforpartitioning[ 70 ].ThesetCKdenotestheindependentsetconditions[ 70 ],whichrequirethatifXs2TK,thegraphwithadjacencymatrixXshasnoindependentsetofsizeK+1.ThesetDisthetriangleconstraints[ 70 ],whichmeanthatifpairs(vi,vj),(vj,vk)areinthesamesubset,thepair(vi,vk)isalsointhesubset.Theproblemofminimizingtheminimumcutofgraphpartitioninghasbeenstudiedbysemideniteprogrammingbasedonthesesets.Theseapproachesarestudiedandreviewedin[ 70 84 129 ]. ForthematrixYs=YYT,itsatisesTK,N,PbychangingX,XstoY,Ysinthesesets,respectively.TheEcanbechangedtoEy=fYs:Ys=YTs,tr(Ys)=K,Yse=eg.Weareprovingthisresultinthefollowingtheorem. Theorem3.4. ThematrixYinformulation( 3 )-( 3 )andYs=YYTsatisfythetheconstraintstr(Ys)=K,Yse=e. Proof.AsshowninTheorem 3.1 ,thematrixYcanbeexpressedasY=XP,wherePisgivenin( 3 ).Thus,wehavetherelations:Ys=YYT=(XP)(XP)T=XPPXT=Xdiag(1=n1,,1=nK)XT=(xik=nk)NKXT. Therefore,tr(Ys)=tr(XPPXT)=NXi=1KXk=1xik nkxik=KXk=1PNi=1x2ik nk=KXk=1nk nk=K, 42

PAGE 43

andforthepartYse,Yse=(xik=nk)NKXTe=(xik=nk)NK(n1,,nK)T=(,KXk=1(xik nknk),)T=e. Inaddition,thesetconvfYYT:YTY=IgisequaltothesetOK=convfYYT:YTY=Ig=fYs:Ys=YTs,tr(Ys)=K,0YsIg. Thisresultwasprovedin[ 94 ].ForthematrixZs=ZZT,itsatisesTK,N,P,OKandtheEcanbechangedtoEz=fZs:Zs=ZTs,tr(Zs)=K,Zsdiag(D1=2)=diag(D1=2)g,bythefollowingtheorem. Theorem3.5. ThematrixZinformulation( 3 )-( 3 )andZs=ZZTsatisfythetheconstraintstr(Zs)=K,Zsdiag(D1=2)=diag(D1=2). Thistheoremwaspresentedin[ 131 ]andcanbeprovedsimilarlyasthatinTheorem 3.4 Remark6. ConsideringtheserelaxationsforRatiocutandNormalizedcut,thesizeconstraints,expressedinthematrixformYTe,ZTe,arenotincludedinanyrelaxedsets.AsstatedinRemark 3 ,therange[Cmin,Cmax]ofthesizeforeachsubsetcanbechosenloosely.Infact,althoughtheseconstraintsarenotincludedinourfollowingSDPrelaxations,theyarestillinarange[1,N)]TJ /F4 11.955 Tf 12.67 0 Td[(1],whichmeansthatnosubsetisempty.Otherwise,ifthekthsubsetisempty,thesizewillbejVkj=0andtheweighteddegreewillbedVk=0,whichwillcausetheRatiocutandNormalizedcutto+1.Similarly,inthespectralrelaxations,therange[1,N)]TJ /F4 11.955 Tf 12.44 0 Td[(1]stillsatisfyforeachsubset.Inthenextsectionofquadraticallyconstrainedprograms,if[Cmin,Cmax]ischosenas[1,N)]TJ /F4 11.955 Tf 12.17 0 Td[(1],thesizeconstraintscanbedroppedfromtheprogramswithoutinuencingthesolutions. 43

PAGE 44

Inthefollowing,weusetheserelaxationstoconstructthesemideniteprogrammingrelaxations.ThesetsCKandDdonotworkforRatiocutandNormalizedcutofgraphpartitioning.Forminimumcutofgraphequipartition,threerelaxationsarepresentedin[ 70 ],andtheyaretheconstraintsP\E,P\E\NandP\E\N\D\CK.Basedonthesemethods,wealsoconsiderthefollowingrelaxationsforRatiocut:(RSDP1)minftr(LYs):Ys2P\Eyg,(RSDP2)minftr(LYs):Ys2P\Ey\Ng,(RSDP3)minftr(LYs):Ys2P\Ey\N\OKg, andforNormalizedcut:(NSDP1)minftr(L0Zs):Zs2P\Ezg,(NSDP2)minftr(L0Zs):Zs2P\Ez\Ng,(NSDP3)minftr(L0Zs):Zs2P\Ez\N\OKg. Therelaxation(NSDP3)hasbeenstudiedin[ 131 ].ThespectralrelaxationscanbeconsideredasaspecialformofSDPrelaxationswhichconsideronlyYs,Zs2OK. ThemostwidelyusedalgorithmforSDPisinteriorpointmethods.OthermethodsincludeBundlemethod,augmentedLagrangianmethod.Thesemethodscanbefoundin[ 76 125 ].ManycodesforsolvingSDPareavailable.Inthischapter,weusethepackageCVX,apackageforspecifyingandsolvingconvexprograms[ 52 53 ]. ThesolutionsYs,ZsfromtheseSDPrelaxationsmaynotbefeasiblefortheoriginalpartitioning,andwehavetotransformthemintofeasiblepartitionmatrixY,ZorX,Xs.InSection 3.2.2 ,wehavediscussedthreeapproachesforthistransformation.Thesemethodscanalsobeused. Assumethat~Ys,~ZsareobtainedfromSDPrelaxations,theyareapproximatedsolutionsofYs,Zs.Bynon-negativematrixfactorization[ 82 ],~Yisobtainedfrom~Ys=~Y~YT.The~Z0isalsoobtainedfrom~Zs=~Z0~Z0TbySVDand~Z=D)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2~Z0.Then,the 44

PAGE 45

directionalcosinemethod,randomizedprojectionheuristicorclusteringroundingasdiscussedinSection 3.2.2 areusedtoobtain~Xsfrom~Y,~Z. 3.3.2Algorithms ThealgorithmforgraphpartitioningbyRatiocutandNormalizedcutbasedSDPispresentedinthefollowing. AlgorithmRSDP/NSDP Input:DataMatrixW=(wij)NNandPositiveIntegerKOutput:PartitionMatrixX=(xik)NKandtheMinimizedRatiocutRcutandNormalizedcutNcutwithrespecttoXStep1:ConstructtheLaplacianmatrixLandweighteddegreematrixDfromWasdenedinDenition 2.3 ;Step2:ForRatiocut,solveoneoftheSDPrelaxationsRSDP1,RSDP2,RSDP3toobtain~Ys;ForNormalizedcut,solveoneoftheSDPrelaxationsNSDP1,NSDP2,NSDP3toobtain~Zs;Step3:Usenon-negativematrixfactorization(NNMF)toobtain~Yby~Ys=~Y~YT,and~Z0from~Zs=~Z0~Z0Tby~Z=D)]TJ /F8 7.97 Tf 6.58 0 Td[(1=2~Z0;Step4:Usethedirectionalcosinemethod,randomizedprojectionheuristicmethod,orclusteringroundingtoobtainXsorXfrom~Y,~ZforRatiocutandNormalizedcut,respectively;Step5:ObtainthepartitionmatrixXfromXsandcomputethecorrespondingRatiocutandNormalizedcut;Step6:Comparetheseresultstoobtainthebestsolution. 3.4QuadraticallyConstrainedProgrammingApproaches Theformulation( 3 )ortheformulation( 3 )-( 3 )forRatiocut,andtheformulation( 3 )ortheformulation( 3 )-( 3 )arenonlineardiscreteprogramming.Themethodsdiscussedaboveareallrelaxationsoftheseprograms.Inthefollowing,we 45

PAGE 46

reformulatethemasbinaryquadraticallyconstrainedprogramsbasedonthedecisionvariablesinX. BythedenitionofRatiocutinDenition 2.5 andProposition 2.2 ,wecanreformulate( 3 )forminimizingRatiocutofgraphpartitioningasfollows:minKXk=1KXk0=1,k06=kyk,k0 (3)s.t.yk,k0(NXi=1xik))]TJ /F9 7.97 Tf 17.3 14.95 Td[(NXi=1NXj=1wijxikxjk00,k=1,,K,k0=1,,K,k06=k,KXk=1xik=1,CminNXi=1xikCmax,xik2f0,1g,i=1,,N,k=1,,K. Here,weassumethatyk,k0=cut(Vk,vk0) jVkj,andusethefactjVkj=PNi=1xikandcut(Vk,Vk0)=PNi=1PNj=1wijxikxjk0.Similarly,assumethatzk,k0=cut(Vk,Vk0) dVk.BythedenitionofNormalizedcutinDenition 2.5 andProposition 2.2 withdVk=PNi=1PNj=1wijxik,wecanreformulate( 3 )forminimizingNormalizedcutofgraphpartitioningasfollows:minKXk=1KXk0=1,k06=kzk,k0 (3)s.t.zk,k0(NXi=1NXj=1wijxik))]TJ /F9 7.97 Tf 17.3 14.95 Td[(NXi=1NXj=1wijxikxjk00,k=1,,K,k0=1,,K,k06=k,KXk=1xik=1,CminNXi=1xikCmax,xik2f0,1g,i=1,,N,k=1,,K. Bothprograms( 3 ),( 3 )aremixedintegerquadraticallyconstrainedprograms(MIQCP)withlinearobjectivefunctions.However,thebinaryconstraintsxik2f0,1gcan 46

PAGE 47

bereplacedbyxik(xik)]TJ /F4 11.955 Tf 10.19 0 Td[(1)0andxik(1)]TJ /F3 11.955 Tf 10.19 0 Td[(xik)0.Theseprogramsbecomequadraticallyconstrainedprograms(QCP).Inthefollowing,weuselinearizationtechniquesonbothprogramsandobtaintheequivalentbinaryintegerlinearprograms. Lettheproductxikyk,k0beyi,k,k0,i.e.,yi,k,k0=xikyk,k0.Sincexikisbinary,thisproductcanbelinearized[ 124 ]bythefollowingconstraints8>>>>>>>>>><>>>>>>>>>>:yi,k,k0yk,k0)]TJ /F3 11.955 Tf 11.96 0 Td[(u(1)]TJ /F3 11.955 Tf 11.96 0 Td[(xik),yi,k,k0lxik,yi,k,k0yk,k0)]TJ /F3 11.955 Tf 11.96 0 Td[(l(1)]TJ /F3 11.955 Tf 11.95 0 Td[(xik),yi,k,k0uxik, (3) wherelyk,k0andtheycanbechosenassufcientsmall,sufcientlargeconstants,respectively. Theproductofthetwobinaryvariablesxik,xjk0canbelinearizedbyxi,j,k,k0usingthewellknowntechniques:8>>>>>>>>>><>>>>>>>>>>:xi,j,k,k0xik,xi,j,k,k0xjk0,xi,j,k,k0xik+xjk0)]TJ /F4 11.955 Tf 11.96 0 Td[(1,xi,j,k,k00. (3) Bythelinearizationtechniquesin( 3 )and( 3 ),wecanreformulate( 3 )forRatiocutinthefollowing:minKXk=1KXk0=1,k06=kyk,k0 (3)s.t.KXk=1xik=1,CminNXi=1xikCmax,NXi=1yi,k,k0)]TJ /F9 7.97 Tf 17.29 14.94 Td[(NXi=1NXj=1wijxi,j,k,k00, 47

PAGE 48

Constraints( 3 )and( 3 ),xik2f0,1g,i=1,,N,k=1,,K,k0=1,,K,k06=k. Thisisabinaryintegerlinearprogram,wherexikisbinary,yk,k0,xi,j,k,k0,yi,k,k0are(nonnegative)continuousvariables.Infact,undertheseconstraints,xi,j,k,k0isstillbinary. Fortheformulation( 3 )forNormalizedcut,wecanintroduceanothercontinuousvariableszi,k,k0=xikzk,k0bythesamelinearizationmethodsin( 3 ).Thus,theformulation( 3 )canbereformulatedasabinaryintegerlinearprogrambyzi,k,k0,xi,j,k,k0asfollows:minKXk=1KXk0=1,k06=kzk,k0 (3)s.t.KXk=1xik=1,CminNXi=1xikCmax,NXi=1NXj=1wijzi,k,k0)]TJ /F9 7.97 Tf 17.3 14.94 Td[(NXi=1NXj=1wijxi,j,k,k00,Constraints( 3 )(withchangesfromytoz)and( 3 ),xik2f0,1g,i=1,,N,k=1,,K,k0=1,,K,k06=k. Bothformulations( 3 )and( 3 )arebinaryintegerlinearprograms.ItcanbedecomposedintocontinuouslinearprogramsandpureintegerprogramsbyBenders'decompositionmethod.Themixedbinaryintegerlinearprogramcanalsobesolvedbycommercialsoftware,suchasCPLEX.Thesolutionofthisprogramistheexactsolutionforgraphpartitioning.However,forlargescaleproblems,thismethodmaytakelongcomputationaltimetoreachtheoptimalsolution.Inpractice,theanti-degeneracyconstraints[ 42 ]arealwaysaddedtoreducethecomputationaltime. 48

PAGE 49

3.5BiclusteringandBipartiteGraphModels Biclusteringsimultaneouslygroupstheobjectsandfeatures.Moreprecisely,biclusteringistondasubsetofobjectsandfeaturessatisfyingthattheobjectsinthissubsetaremuchmorehighlyrelatedtothefeaturesinthissubsetthanfeaturesfromanothersubset.Biclusteringisalwaysperformedonadatamatrix.AssumethatA=(aij)NMisthedatamatrixforbiclusteringwithnonnegativeentries,theithrowofthismatrixcorrespondstotheithobject,thejthcolumncorrespondstothejthfeatureandtheentryaijmeasurestheexpressionlevelfeaturejinobjecti.BiclusteringistondsubmatricesofAwithspecicconstraints.Inoutbipartitegraphpartitioningmodels,wearendingKnonoverlap,exclusive,exhaustivesubmatrices[ 38 ]inA.ThematrixAistheweightmatrixinthebipartitegraphmodel. ThebipartitegraphisdenedasG=(V,U,E)withvertexsetsV=fv1,,vNg,U=fu1,,uMgandedgesetE=f(vi,uj):edgebetweenverticesvianduj,1iN,1jMg,whereNandMarethenumbersofverticeswithintwosets,respectively.Usually,insteadofweightmatrix,thebiadjacencyweightmatrixA=(aij)NMisgivenwhereai,jistheweightofedge(vi,uj).AssumethatwestillwanttoobtainKnonemptysubsetsofbothVandU.Apartitioningbybipartitegraphis=(V1[U1,,VK[UK)suchthatVk[UkisaunionsubsetandV1[V2[[VK=V,Vk\Vk0=;,U1[U2[[UK=U,Uk\Uk0=;forallpairsk6=k0.Assumethatnk=jVkj,mk=jUkjandn=(n1,,nK)T,m=(m1,,mK)T. WedenetwopartitionmatricesXv=(xvik)NK,Xu=(xujk)MKforvertexsetsV,U,respectively.TherequirementsforbipartitegraphpartitioningisstatedinthesetFBK=fXv=(xvik)NK,Xu=(xujk)MK:Xve=e,XTve=n2[Cmine,Cmaxe],Xue=e,XTue=m2[cmine,cmaxe]g. 49

PAGE 50

LetthematrixXbeX=0B@XvXu1CA.FrommatrixA,wecanconstructthecorrespondingweightmatrixW=0B@0AAT01CA,theweighteddegreematrixDv=diag(Ae),Du=diag(ATe),andtheLaplacianmatrixL=0B@Dv)]TJ /F3 11.955 Tf 9.3 0 Td[(A)]TJ /F3 11.955 Tf 9.3 0 Td[(ATDu1CA.Theobjectiveofthebipartitegraphpartitioningbyminimumcutis1 2tr(XTLX)=1 2tr0B@0B@XvXu1CAT0B@Dv)]TJ /F3 11.955 Tf 9.29 0 Td[(A)]TJ /F3 11.955 Tf 9.29 0 Td[(ATDu1CA0B@XvXu1CA1CA=1 2tr(XTvDvXv+XTuDuXu)]TJ /F3 11.955 Tf 11.96 0 Td[(XTvAXu)]TJ /F3 11.955 Tf 11.95 0 Td[(XTuATXv)=eTAe)]TJ /F3 11.955 Tf 11.96 0 Td[(tr(XTvAXu), wheretr(XTvDvXv+XuDuXu)=2eTAecanbeobtainedbythemethodsusedinProposition 2.2 .Thus,asRemark 5 ,theobjectiveofthebipartitegraphpartitioningbyminimumcutcanchooseeithermin1 2tr(XTLX)ormaxtr(XTvAXu).TheformulationsforbipartitegraphpartitioningbyRatiocutandNormalizedcutcanbeconstructedsimilarlyasthosein( 3 ),( 3 )bychangingFKtoFBK.ThepartitionmatrixXhasthepropertyasfollows:XTX=0B@XvXu1CAT0B@XvXu1CA=0BBBB@n1+m1...0.........0...nK+mK1CCCCA, andthematricesDv,Du, XT0B@Dv00Du1CAX=0BBBB@dV1+dU10.........0dVK+dUK1CCCCA. 50

PAGE 51

AssumethatP=XTXandQ=XT0B@Dv00Du1CAX. TheminimumcutisthesumweightofedgesamongalldistinctunionsubsetsVk[Uk.TheRatiocutisthefractionofthecutamongdistinctunionsubsetstothesizesoftheunionsubset.TheNormalizedcutisthefractionofthecuttoweighteddegreesofverticesintheunionsubsets.TheRatiocutforbipartitegraphpartitioningisdenedasRBcut=KXk,k0=1,k06=kcut(Vk[Uk,Vk0[Uk0) jVk[Ukj (3)=KXk,k0=1,k06=kcut(Vk,Uk0)+cut(Vk0,Uk) jVkj+jUkj=KXk,k0=1,k06=kPNi=1PMj=1(aijxvikxujk0+aijxvik0xujk) PNi=1xvik+PMj=1xujk. TheNormalizedcutforbipartitegraphpartitioningisdenedasNBcut=KXk,k0=1,k06=kcut(Vk[Uk,Vk0[Uk0) dVk+dUk (3)=KXk,k0=1,k06=kPNi=1PMj=1(aijxvikxujk0+aijxvik0xujk) PNi=1PMj=1aijxvik+PNi=1PMj=1aijxujk. ThequadraticallyconstrainedprogramsforRatiocutandNormalizedcutforbipartitegraphpartitioningcanbeconstructedaccordingtothesedenitionsofcuts.ConsideringtheunionVk[UkasasubsetofV[U,spectralandSDPrelaxationsapproachesforgraphpartitioningcanbeusedonbipartitegraphsdirectly.ThesecanbeobtainedbythepropertiesofX=0B@XvXu1CA.AssumethatY=XP,Z=XQ,westillhavethatRBcut=tr(YTLY)andNBcut=tr(ZTL0Z).WecanassumethattherstNverticesbelongtoonevertexsetandtheremainingMverticesbelongtoanothervertexset.Inaddition,thesizeconstraints[Cmin,Cmax]and[cmin,cmax]arerelaxedintononemptyconstraintsforeachsubset,andtheycanbeeliminatedwithoutinuencingtheresults. 51

PAGE 52

3.6NumericalExperiments InSection3-5,wehavediscussedthreeapproachesforgraphpartitioningbyRatiocutandNormalizedcutforclustering.Thequadraticallyconstrainedprogramwithlinearizationtechniquesistheexactmethod,whichcanproduceexactsolutions.However,aswementioned,thismethodiscomputationallyexpensive.Inthefollowing,weuseCPLEXtosolveseveralcaseswithsmallnumberofvertices.WerstuseourproposedmethodstochecktheresultsmentionedinFig. 2-1 .Thisgraphhas12verticesand20edges,andwewanttopartitionthevertexsetintoK=2subsets.Byourproposedmethods,RSDP1,RSDP2andRSDP3canndthebestpartitioningasshowninFig. 2-1 withRatiocut0.67whilethespectralcannot.ForNormalizedcut,onlyNSDP2andNSDP3canndbestpartitioningwithminimumNormalizedcut0.22.TheQCPapproachcanndbothcutswithminimumvaluescorrectly. Inthefollowing,weuseMatlabtogeneratesomedatasets,anduseourrelaxationsmethods,includingspectralandSDPrelaxations,toobtainapproximateresults.InTable 3-1 ,weuseQCPtostudyseveralcaseswithsmallsizesofvertices,andalsolisttherelaxedsolutions.Therandomlygeneratedweightsofedgesareeither0or1. Table3-1. Multi-wayclusteringresultsforcaseN=10 RatioCut(Rcut)NormalizedCut(Ncut)KSpectralRSDP1RSDP2RSDP3QCPSpectralNSDP1NSDP2NSDP3QCP 26.403.753.334.443.330.970.890.730.730.73310.3310.607.839.007.832.092.291.591.591.59418.0015.0014.0012.8312.833.273.452.922.902.74521.5022.3318.3318.3318.334.634.283.923.923.92 Note:Inthistable,wepresentallproposedapproachesongraphswith10vertices. FromTable 3-1 ,theQCPapproachproducestheminimumRatiocutandNormalizedcut.However,aspointedinSection 3.5 ,thismethodiscomputationallyexpensive.Therelaxationmethods,NSDP2andNSDP3arequiteusefulforNormalizedcut.TheRSDP2andRSDP3canalsoworkwellinmostcases. 52

PAGE 53

Table3-2. Multi-wayclusteringresultsforseveralcases RatioCut(Rcut)NormalizedCut(Ncut)NKSpectralRSDP1RSDP2RSDP3SpectralNSDP1NSDP2NSDP3 1025.96514.48254.48254.48251.04341.02720.89550.895539.02509.56887.90867.90862.28722.36421.95351.9535412.023312.123910.221410.10163.53993.40262.80992.9347522.672719.771016.758318.02794.39884.68254.02434.0243 20210.00967.60147.60147.60141.01141.08710.86930.8693320.810818.129316.563417.15632.18341.95541.82131.8213423.663329.060022.193321.26223.26773.08452.67722.6772540.508639.711736.465836.28244.37834.20963.75023.7502 30214.870816.035012.596912.59690.94840.96140.87980.8798330.970724.680524.305024.05522.13862.07791.84291.8429444.810841.153234.426635.60523.17513.09742.76782.9321559.134063.621748.146951.82824.12014.06193.77353.9155 40220.823320.154315.669615.66961.03750.92490.90600.9060346.659741.072935.530836.24322.08092.07521.86341.8599462.281654.808151.361351.36133.14132.98492.91802.8006584.991878.122171.500973.99934.09044.10193.76953.7840 50226.189223.334021.528821.52881.05301.02390.93140.9327349.485344.830145.165941.86632.14461.98441.84521.8508477.396575.884667.098167.69383.03483.03452.80492.79145102.768795.719587.878488.64384.08514.03053.77823.8661 60229.612828.042123.561023.56101.00811.01590.92530.9240361.609656.978253.632751.03422.03112.03381.86091.8609485.112789.293980.177580.60993.08853.07252.81952.81445115.4614102.1540102.0608100.11194.04844.03643.83053.8458 70235.062734.849232.249632.24961.04830.93620.91380.9138370.073965.481963.338764.23921.99152.04421.86331.87424107.2466100.973791.840192.35313.06643.02632.82222.82225143.5539134.9686127.5178126.12044.06374.05843.80143.8275 80239.032138.492635.552237.24031.04110.93960.92900.9301379.812467.580869.074168.99262.06222.00801.87721.87664118.0029114.5897105.0521104.82073.07813.04702.82012.82065160.8420156.5950140.6994140.18694.06274.04793.81253.8195 Note:InTable 3-2 ,weperformourmethodsonthedatasetswithNverticesandpartitionthemintoKsubsets.Inthesecomputationalresults,alldatasetsforweightedmatricesaregeneratedbyMatlabwithintherange[0,1].Weusenonnegativematrixfactorizationmethodtoobtain~Y,~Zfrom~Ys,~Zs,respectively,with100iterationsatmost.Inaddition,weuseK-meansclusteringtotransforminfeasiblesolutionsintofeasiblesolutions. 53

PAGE 54

Table3-3. Multi-wayclusteringresultsforcaseN=100 RatioCut(Rcut)NormalizedCut(Ncut)KSpectralRSDP1RSDP2RSDP3SpectralNSDP1NSDP2NSDP3 250.699847.058847.443544.58730.98430.98250.93940.93763100.5659102.719186.774186.77412.02432.02001.88841.88654151.2269143.7174135.4610132.12992.99013.03812.87602.85305204.8972194.8411176.3037177.67864.06024.04503.82823.82906248.4688251.0488231.8624231.24375.10105.05444.80034.82837300.8281286.0585267.4994265.39076.04306.04625.76585.81318343.8683338.2903318.7011318.49487.08077.06346.77796.73309402.2044377.6955363.3538368.40708.10438.07667.79867.771310464.7659421.7296409.5492407.97569.07629.11268.76928.7478 Note:ThistablehasthesameassumptionsandusesamemethodsasthoseinTable 3-2 Fromthecomputationalresults,therelaxationsRSDP1orNSDP1canobtainbettersolutionsthanspectralrelaxations.TheRSDP2andRSDP3forRatiocut,andtheNSDP2andNSDP3forNormalizedcut,havethebestsolutionsamongtheserelaxations.However,theRSDP2andRSDP3havenoclearcomparisons.WeputthedifferenceoftheminFig. 3-1 .Fromthesetwogures,NSDP2andNSDP3havenobigdifferenceforNormalizedcut.In[ 131 ],theNSDP3isusedforclustering.However,asshowninourpracticalresults,theNSDP2haslessconstraintsandcanbechosenforgraphpartitioningofNormalizedcut. 3.7Discussion Inthischapter,wepresenttheoptimizationmodelsforgraphpartitioningbyRatiocutandNormalizedcut.Thesearemixedintegernonlinearprograms.Tosolvetheseproblems,wereformulatethemandusespectralrelaxationsandsemideniteprogrammingrelaxations.Inaddition,weusequadraticallyconstrainedprogramstoobtaintheexactsolutions.Theproposedmethodsarequiteusefulindataclustering.Inaddition,wediscussthreewaystoobtainfeasibleintegersolutionsfrominfeasiblerelaxedsolutions:thedirectionalcosinemethod,randomizedprojectionheuristicmethod,andclusteringrounding.Wealsouseourmethodstoobtainthebipartitegraphpartitioningfordatabiclustering.Wehaveperformedouralgorithmsondifferentdata 54

PAGE 55

Up:ResultsfromTable 3-2 (K=4);Bottom:ResultsfromTable 3-3 (N=100) Figure3-1. DifferencesforRatiocutandNormalizedcutsolvedbySDP2andSDP3 sets.IthasbeenshownthatthesecondandthethirdSDPrelaxationscanobtainbettersolutions.TheexactQCPmodelscanobtainexactsolutionsforsmalldatasets. Forquadraticallyconstrainedprograms,efcientalgorithmsforlargescaledatasetsarestillunderdiscussion.Possibleapplicationsofourmethodsincludeimagesegmentation,networkanalyzing,andtextmining.ResultsofthischapterandSection 2.3 arepublishedinourpaper[ 43 ]. 55

PAGE 56

CHAPTER4GRAPHPARTITIONINGWITHMINIMUMCUT Thegraphpartitioningproblemistopartitionthevertexsetofagraphintoanumberofnonemptysubsetssothatthetotalweightofedgesconnectingdistinctsubsetsisminimized.ThisreferstographpartitioningwithMinimumcutaswediscussedinSection 2.3 .Inthissection,allresearchresultsareforGPPwithMinimumcut.Thisproblemhasbeenstudiedforalongtime,anditisanNP-completecombinatorialoptimizationproblem. Previousresearchrequirestheinputofcardinalitiesofsubsetsorthenumberofsubsetsforequipartition.InSection 4.1 ,theproblemisformulatedasazero-onequadraticprogrammingproblemwithouttheinputofcardinalities.Wealsopresentthreeequivalentzero-onelinearintegerprogrammingreformulations.Becauseofitsimportanceindatabiclustering,thebipartitegraphpartitioningisalsostudied. InSection 4.2 ,robustoptimizationmodelswithtwodecompositionalgorithmsareintroducedtosolvethegraphpartitioningproblemwithintervaluncertainweightsofedges.Thebipartitegraphpartitioningproblemwithedgeuncertaintyisalsopresented.Throughoutthissection,wemakenoassumptionregardingtheprobabilityoftheuncertainweights. InSection 4.3 ,weintroducethetwo-stagestochasticgraphpartitioningproblemandpresentthestochasticmixedintegerprogrammingformulationforthisproblemwithniteexplicitscenarios.Forsolvingthisproblem,wepresentanequivalentintegerlinearprogrammingformulationwheresomebinaryvariablesarerelaxedtocontinuousones.Additionally,forsomespecicgraphs,wepresentamoresimpliedlinearprogrammingformulation.Allformulationsaretestedonrandomlygeneratedgraphswithdifferentdensitiesanddifferentnumbersofscenarios. 56

PAGE 57

4.1GeneralGraphPartitioningProblem LetG=(V,E)beanundirectedgraphwithasetofverticesV=fv1,v2,,vNgandasetofedgesE=f(vi,vj):edgebetweenverticesviandvj,1i,jNg,whereNisthenumberofvertices.TheweightsoftheedgesaregivenbyamatrixW=(wij)NN,wherewij(>0)denotestheweightofedge(vi,vj)andwij=0ifnoedge(vi,vj)existsbetweenverticesviandvj.ThismatrixissymmetricforundirectedgraphsGandistheadjacencymatrixofGifwij2f0,1g. LetKbethenumberofdisjointsetsthatwewanttopartitioninto,andn1,,nKbethenumbersofverticeswithineachsets.ThegeneralgraphpartitioningproblemcanbedescribedaspartitioningthevertexsetVintoKdisjointsubsetssothatthesumofweightsofedgesthatconnectverticesamongdifferentsubsetsisminimized.MethodsforgraphpartitioningproblemwerereviewedinSection 2.2 Inthissection,thecardinalitiesforeachsubsetsarenotrequiredandthenumberK(1
PAGE 58

Constraintsandobjectives.BasedonthegraphG=(V,E)describedabove,letxikbethedecisionvariablesdenotingthatvertexvibelongstokthsubsetifxik=1,otherwisexik=0.LetXbethematrixX=(xik)NKandtheconstraintsforgeneralgraphpartitioningproblemareconstructedasfollows: 1) Exclusiveconstraints.EveryvertexvicanonlybelongtoexactlyonesubsetofV,andsoeveryrowsumofXis1,R=fxik2RNK:KXk=1xik=1g. (4) Fori=1,,N,allverticesbelongtosomesubsets,andwehavepartitionedthevertexsetintoseveralsubsets. 2) Cardinality(size)constraints.Letthecardinalityofthekthsubsetbenk=PNi=1xik.Inordertoeliminatethecasethatallverticesbelongtoonlyonesubset,everysubsetshouldbenonempty(Sa).Somepartitioningresultsrequiretheorderofsubsets(Sb),andMaxCarddenotesthemaximumsizeofeachsubset(Sc).TheconstraintsSccanbeconsideredasanequipartitionifMaxCardischosenasanaveragecardinalityofN=K.Inthefollowingmodels,wealwayschoosethemostgeneralcaseSasothatthegraphcanbepartitionedaccordingtoitsstructure.Sincewehaverelaxedthesizeconstraintsforgraphpartitioning,thetermgeneralgraphpartitioningisusedinthissection. Sa=fxik2RNK:nk1g,(4a) Sb=fxik2RNK:n1n2nk1g,(4b) Sc=fxik2RNK:nkMaxCardg.(4c) 3) Anti-degeneracyconstraints.Thesubsetindexesoftheverticescanberestrictedthattherstvertexmustbelongtorstsubset,thesecondvertexmustbelongeithertorstortosecondsubset,andcontinually,thekthvertexmustbelongtooneoftherstksubsets.D=fxik2RNK:xik=0,i=1,K,k=i+1,,Kg. (4) Withtheseconstraints,theformulationsarelessdegeneratedbecauseoftheconstraintsonpermutationsofvertexassignment. 4) Binaryconstraints.Everyxikindicatesthatvertexvibelongstokthsubsetornot.B=fxik2RNK:xik=0or1g. (4) 58

PAGE 59

5) Nonnegativeconstraints.ThebinaryconstraintscanberelaxedtononnegativeoneswhichensurethepartitioningofGundersomeconditions.P=fxik2RNK:xik0g. (4) AsdiscussedinSection 2.3 ,thefeasibleset( 2 )basedonpartitionmatrixforgraphpartitioningisFK=f(xik)NK:xik2f0,1g,KXk=1xik=1,CminNXi=1xikCmaxg,wherethecardinalityconstraintisCminnk=PNi=1xikCmax.WehavefollowingresultsforcasesofSa,Sc:WhenCmin=1,Cmax=N)]TJ /F4 11.955 Tf 12.66 0 Td[(1,FK=R\Sa\B;WhenCmin=1,Cmax=MaxCard,wehaveFK=R\Sa\Sc\B. AsdenedinDenition 2.5 andDenition 2.6 ,theMinimumcutisexpressedMcut=1 2tr(XTLX).AsexplainedinRemark 4 ,twoobjectsmin1 2tr(XTLX)andmax1 2tr(XTWX)areequivalentforgraphpartitioningbyMinimumcut.InthefollowingSection 4.1.1 andSection 4.1.2 ,bothobjectivesareusedforgraphpartitioning. DeterminationofthenumberKofsubsects.ForcontrollingthecardinalitiesofsubsetsofV,theconstraintsSa,Sb,Scbynotationnk=PNi=1xikarechosenunderdifferentsituations.Theseconstraintscanbenamednonemptyconstraints,orderconstraintsandmaximalcardinalityconstraints,respectively. Inpreviousresearch[ 57 70 ],eithern1,,nKaregivenortheequalpartitionwithgivenK.Inthissection,previousformulationsrequiretheinputofK.However,thedeterminationtoinputwhichK2f2,3,,N)]TJ /F4 11.955 Tf 12.08 0 Td[(1gisstillopen.ThissectionwillpresentonemethodfordeterminingK. Letckbeabinaryvariablesuchthatck=1ifthekthsubsetofVisnonempty,andck=0otherwise.ThusthecardinalityconstraintscanbeexpressedasSd=fxik2RNK:nkckg. (4) 59

PAGE 60

ThetermPKk=1ckcanbeaddedtotheobjectivefunction(add+whentheoriginalobjectiveistominimize,and-otherwise),inordertorequiretheminimumpossiblenumberofsubsets,byforcingsomeoftheKclusterstobeempty.Thus,KcanbechosenasN)]TJ /F4 11.955 Tf 12.27 0 Td[(1generally.WhentheconstraintsSdischosentocontrolthenonemptysubsets,themaximalcardinalityconstraintsScmustbechosenatthesametime,andnonemptyconstraintsSacannotbechosen. Wehavediscussedconstraints,objectivefunctionsandthedeterminationofthenumberKforgraphpartitioningbyMinimumcutinthissection.TherestofSection 4.1 isorganizedasfollows:Insection 4.1.1 ,thequadraticprogrammingmodelanditsrelaxationsarestudied,andinaddition,thequadraticmodelsforbipartitegraphpartitioningarealsopresented;Insection 4.1.2 ,threelinearprogrammingapproachesareintroduced;Insection 4.1.3 ,numericalexperimentsformanygraphsunderdifferentconditionsareperformedaftercomparingtheseapproaches;Section 4.1.4 concludesSection 4.1 4.1.1QuadraticProgrammingApproaches FromRemark 4 ,theobjectivefunctioncanbeexpressedintheformmax1 2tr(XTWX).Asdescriptionsabove,theconstraints( 4 )and( 4 )areprerequisiteforgraphpartitioningproblem.Inourgeneralcase,theconstraints( 4a )arechosensothattherequirementofthecardinalityofeachsubsetisnotnecessarilytobegiven. Quadraticprogrammingmodelsandrelaxations.Thegraphpartitioningproblemcanbeformulatedasazero-onequadraticprogramasfollows:max1 2tr(XTWX) (4)s.t.xik2R\Sa,xik2B,i=1,,N,k=1,,K. Theobjectivein( 4 )isnotstandardinaquadraticprogramming.DenotingX=(xij)NK=(x1,,xk,,xK),wherexksarecolumnvectors,^X=(xT1,xT2,,xTK)T, 60

PAGE 61

and^W=0BBBBBBB@W0000W00...............000W1CCCCCCCA, withdimension(NK)(NK),where0isamatrixwithallelementsbeing0withproperdimension,thestandardformulationfortheobjectivein( 4 )is1 2tr(XTWX)=1 2KXk=1xTkWxk=1 2^XT^W^X. (4) Usually,theconstrainsDin( 4 )areaddedtoreducethedegeneracy,and( 4c )areaddedasarequirementofloosecardinalities,whereMaxCardcanbechosenloosely(forexample,MaxCard=N=(K)]TJ /F4 11.955 Tf 12.45 0 Td[(1)).LetD=(dij)NNbeadiagonalmatrix.Byproperlychoosingtheelementsdii,thebinaryconstraints( 4 )canberelaxedtononnegativeones( 4 )bythefollowingTheorem 4.1 andfortheproofdetails,wereferto[ 57 ].Thus,thefollowingcontinuousformulationcanbeobtained.max1 2tr(XT(W+D)X) (4)s.t.xik2R\Sa,xik2P,i=1,,N,k=1,,K. Theorem4.1. (Theorem6.1,[ 57 ])IfDischosentosatisfydii+djj2wijforeachiandj,thenthecontinuousproblem( 4 )hasamaximizercontainedinB,andhence,thismaximizerisasolutionofthediscreteproblem( 4 ).Conversely,everysolutionto( 4 )isalsoasolutionto( 4 ).Moreover,ifdii+djj>2wijforeachiandj,theneverylocalmaximizerfor( 4 )liesinB. Therearemanyapproachesforsolving( 4 ),includingheuristicandexactoptimizationonesasreviewedinthepaper[ 114 ]bySheraliandSmith.Theheursitcapproachesincludetherank-tworelaxationbyBureretal.[ 15 ],theevolutionary 61

PAGE 62

procedurebyLodietal.[ 86 ]andtheTabuSearchalgorithmbyKochenbergeretal.[ 78 ].Theexactoptimizationapproachesincludethebranch-and-boundalgorithmbyPardalosandRodgers[ 97 ],andtheLagrangiandecompositionalgorithmbyChardaireandSutter[ 23 ].Themostrecentsurveyincludingheuristicandexactoptimizationapproachescanbefoundforasimilarquadraticassignmentproblemin[ 87 ]byLoiolaetal.InSection 4.1 ,CPLEX11.0isusedtosolvethezero-onequadraticprogrammingproblem,andCPLEXusesmultipletypesofmostrecentalgorithms[ 28 ]. Quadraticapproachesforbipartitegraphs.LetG=(V,U,E)beabipartitegraphwithvertexsetsV=fv1,,vNg,U=fu1,,uMgandedgesetE=f(vi,uj):edgebetweenverticesvianduj,1iN,1jMg,whereNandMarethenumbersofverticeswithintwosets,respectively.Usually,insteadofweightedmatrix,thebiadjacencyweightedmatrixA=(aij)NMisgivenwhereai,jistheweightofedge(vi,uj),anditscorrespondingweightedmatrixWforGcanbeconstructedasW=0B@0AAT01CA. BytheweightedmatrixWandconsideringV[Uasthevertexset,thepreviousformulationscanbeused.However,someproblems(e.g.,biclustering,[ 39 ])basedonbipartitegraphmodelsrequirepartitioningofbothvertexsetVandUintoKdisjointsubsetssothatthekthsubsetsofVandalsoofUasawholesubset.Thesumofweightsofedgesamongtheunionsubsetshastobeminimized. DenotingX=0B@XvXu1CA,whereXv=(xvik)NKandXu=(xujk)MKareindicatorsforverticesfromVandU,respectively,theequivalentobjectivecanbeexpressedasmax1 2tr(XTWX)=maxtr(XTvAXu). (4) TheconstraintsforXvarestilltheones( 4 ),( 4a )and( 4 ),whiletheindexshouldbechangedtoj=1,,Minsteadofi=1,,NforXu.LetR0,S0aandB0be 62

PAGE 63

theconstraintsforXuwithchangedindexes.Thezero-onequadraticprogramcanbeformulatedasmaxtr(XTvAXu) (4)s.t.xvik2R\Sa\B,xujk2R0\S0a\B0,i=1,,N,j=1,,M,k=1,,K. Theorem 4.1 stillholdsinthiscasebyconsideringXvandXuassubmatricesofX,andWisalsosymmetric.Acontinuouscaseofprogram( 4 )canbeobtainedbychangingB,B0toP,P0byconstructingadiagonalmatrixDofdimension(N+M)(N+M).Inaddition,theanti-degeneracyconstraintsDcanonlybeputononeofXvandXuforbipartitegraphs. 4.1.2LinearProgrammingApproaches Directlinearprogrammingmodels.Theobjectivefunction1 2tr(XTWX)=1 2NXi=1NXj=1KXk=1wijxikxjkisnonlinear.Here,twoapproaches,similarlyaspresentedin[ 14 ],areusedtotransformthisnonlinearobjectiveintoalinearone. Letyijkdenoteindicatorthatedge(i,j)withtwoendsi,jbelongtosubsetkifyijk=1ornotifyijk=0.Thusyijk=xikxjk,i,j=1,,N,k=1,,K.Thelinearizationisasfollowsyijkxik,yijkxjk,yijkxik+xjk)]TJ /F4 11.955 Tf 11.95 0 Td[(1andyijk0. Sinceyijkisintheobjectivetomaximizeandthenonnegativityofwij,constraintsyijkxik+xjk)]TJ /F4 11.955 Tf 12.42 0 Td[(1andyijk0canbeeliminatedforwij0.Thelinearprogramming 63

PAGE 64

formationforgeneralgraphpartitioningproblemismax1 2NXi=1NXj=1KXk=1wijyijk (4)s.t.xik2R\Sa\B,8>><>>:yijkxik,yijkxjk,i,j=1,,N,k=1,,K. Thelinearizedformulation( 4 )introducesKmorevariablesand2Kmoreconstraintsforeachedgecomparingwiththeoriginalone( 4 ). Letzijbeabinaryvariablesuchthatzij=1iftheverticesvi,vjofedge(vi,vj)belongstothesamesubsetand0otherwise.Thuszij=PKk=1xikxjkandzij=1ifandonlyifforsomeksuchthatxik=xjk=1andallothersxik0=xjk0=0.Thelinearizationofzij=KXk=1xikxjk (4) canbeformulatedaszij1+xik)]TJ /F3 11.955 Tf 12.67 0 Td[(xjk,zij1)]TJ /F3 11.955 Tf 12.67 0 Td[(xik+xjk,foralli,j,kandallotherrequirementsofthisformulationsareeliminatedfortheobjectivetobemaximized.Therefore,weobtainanotherlinearprogrammingformulationasmax1 2NXi=1NXj=1wijzij (4)s.t.xik2R\Sa\B,8>><>>:zij1+xik)]TJ /F3 11.955 Tf 11.95 0 Td[(xjk,zij1)]TJ /F3 11.955 Tf 11.96 0 Td[(xik+xjk,i,j=1,,N,k=1,,K. 64

PAGE 65

Inpractice,theanti-degeneracyconstraintsDandthecardinalityconstraintsSccanbeaddedtobothprograms( 4 )and( 4 ). Anequivalentlinearprogrammingapproach.TheLaplacianmatrixforG=(V,E)withWisdenedasL=diag(Pjw1j,,PjwNj))]TJ /F3 11.955 Tf 12.61 0 Td[(W.AsdescribedinRemark 4 ,theobjectiveforpartitioningthegraphGcanalsobeexpressedas1 2tr(XTLX),andtheequivalentquadraticprogrammingformulationismin1 2tr(XTLX) (4)s.t.xik2R\Sa\B,i=1,,N,k=1,,K. Bythemethodsproposedin[ 22 ]and[ 114 ],deningS=(sik)NK=(s1,,sK),T=(tik)NK=(t1,,tK),wheresk,tkarecolumnvectors,theequivalentlinearprogrammingformulationismineTSe (4)s.t.xik2R\Sa\B,Lxk)]TJ /F3 11.955 Tf 11.96 0 Td[(tk)]TJ /F3 11.955 Tf 11.96 0 Td[(sk+Ce=0,tk2C(e)]TJ /F3 11.955 Tf 11.96 0 Td[(xk),tik,sik2P,i=1,,N,k=1,,K. wheretheconstantC=2maxNi=1PNj=1wij. Fromthemethodsin[ 22 ],theformulation( 4 )isequivalentto( 4 )aftersubtractingaconstanttheobjectiveof( 4 )bythefollowingtheorem. Theorem4.2. Formulations( 4 )and( 4 )areequivalent. 65

PAGE 66

Proof.LetX0beanoptimalsolutionoftheformulation( 4 ).WeclaimthatthereexistT,S(tik0,sik0)foralli,ksuchthatLx0k)]TJ /F3 11.955 Tf 11.95 0 Td[(tk)]TJ /F3 11.955 Tf 11.95 0 Td[(sk+Ce=0, (4)tTkx0k=0. (4) Theconstraintstk2C(e)]TJ /F3 11.955 Tf 12.18 0 Td[(x0k)in( 4 )isequivalentto( 4 )sincex0ikisbinaryandCisapositiveconstant.Inthefollowing,weuse( 4 )toprovetheequivalence. SinceC=2maxNi=1PNj=1wij=kLk1,Lxk+Ce0,andtherealwaysexistT,S(tik0,sik0)foralli,ksuchthatboth( 4 )and( 4 )hold.Theclaimisproved. ThevariablesS,TcanbechosenasS0,T0satisfying( 4 )and( 4 )sothateTSe,thesumofsikforalli,k,isminimized.Weclaimthat(X0,S0,T0)isanoptimalsolutionfortheformulation( 4 ). Multiplying( 4 )by(x0k)T,wehave(x0k)TLx0k)]TJ /F4 11.955 Tf 11.95 0 Td[((x0k)Tsk+(x0k)TCe=0, (4) byconsidering( 4 ).Takingthesumoverallk=1,,K,andconsideringtr(XTLX)=PKk=1xTkLxk,weobtaintr((X0)TLX0))]TJ /F9 7.97 Tf 17.13 14.94 Td[(KXk=1(x0k)Ts0k+KXk=1(x0k)TCe=0. (4) Inaddition,PKk=1(x0k)TCe=eTCe=CNisaconstantsincexik2Rinboth( 4 )and( 4 ).Theequation(x0k)Ts0k=eTs0k, (4) fork=1,,Kcanbeprovedbycontradiction.Assumethatforsomei,wehavex0ik=0ands0k>0,whereT0,S0arechosentominimizedeTSe.Deningthevectors~tk,~skandcorresponding~T,~Ssuchthat~tik=~t0ik+~s0ik,~sik=0forj6=i,~tjk=tjk,~sjk=sjk, 66

PAGE 67

inthiscase,(X0,~T,~S)satises( 4 )and( 4 ),andeT~Se
PAGE 68

Similarly,deningyijk=xvikxujk,thelinearprogrammingformulationforpartitioningbipartitegraphisasfollows,maxNXi=1MXj=1KXk=1aijyijk (4)s.t.xvik2R\Sa\B,xujk2R0\S0a\B0,8>><>>:yijkxvik,yijkxujk,i=1,N,j=1,M,k=1,,K. Thisformulation( 4 )introducesKmorevariablesand2Kmoreconstraintsforeachedgecomparingwiththeoriginalformulation( 4 ).Asintheformulation( 4 ),deningzij=PKk=1xvikxujk,anotherlinearprogrammingformulationwithlessvariablesandconstraintsisobtainedinthefollowing.maxNXi=1MXj=1aijzij (4)s.t.xvik2R\Sa\B,xujk2R0\S0a\B0,8>><>>:zij1+xvik)]TJ /F3 11.955 Tf 11.95 0 Td[(xujkzij1)]TJ /F3 11.955 Tf 11.96 0 Td[(xvik+xujki=1,N,j=1,M,k=1,,K. Asin( 4 ),deningXv=(xvik)NK=(xv1,,xvK),Xu=(xujk)MK=(xu1,,xuK),Sv=(svik)NK=(sv1,,svK),Su=(sujk)MK=(su1,,suK),Tv=(tvik)NK=(tv1,,tvK),Tu=(tujk)MK=(tu1,,tuK)andDv=diag(,PMj=1aij,),Du=diag(,PNi=1aij,),thelinearprogrammingformulationforbipartitegraphG= 68

PAGE 69

(V,U,E)withmatrixAismineTSve+eTSue (4)s.t.xvik2R\Sa\B,xujk2R0\S0a\B0,Dvxvk)]TJ /F3 11.955 Tf 11.96 0 Td[(Axuk)]TJ /F3 11.955 Tf 11.95 0 Td[(tvk)]TJ /F3 11.955 Tf 11.95 0 Td[(svk+Ce=0,Duxuk)]TJ /F3 11.955 Tf 11.96 0 Td[(ATxvk)]TJ /F3 11.955 Tf 11.96 0 Td[(tuk)]TJ /F3 11.955 Tf 11.95 0 Td[(suk+Ce=0,tvk2C(e)]TJ /F3 11.955 Tf 11.96 0 Td[(xvk),tuk2C(e)]TJ /F3 11.955 Tf 11.96 0 Td[(xuk),tvik,svik2P,tujk,sujk2P0,i=1,,N,j=1,,M,k=1,,K. wheretheconstantC=maxfmaxi2Pjaij,maxj2Piaijg.Theequivalenceof( 4 )withoriginalquadraticformulationforbipartitegraphcanbeprovedbysimilarmethodsinTheorem 4.2 TheconstraintsDcanalsobeaddedtoformulations( 4 ),( 4 )and( 4 )toreducethedegeneracyonXv. 4.1.3NumericalExperiments Comparisonsoftwoapproaches. ForthegraphG=(V,E)withNverticesandjEjedges,thediscretequadraticprogramming(DQ)formulation( 4 ),andthelinearprogrammingformulationsL1( 4 ),L2( 4 )andL3( 4 )arecomparedinTable1.ThenumberforRisNwhileforSaisK.ForL3( 4 ),thenumberofnonnegativeconstraintsforS,Tisnotadded. FromTable 4-1 ,thelinearformulationsintroducevariablesandconstraintstoeliminatequadraticvariablesintheobjective,andtheformulation( 4 )haslessvariablesandconstraints.Thenumbersofcontinuousvariablesandconstraintsin 69

PAGE 70

Table4-1. Comparisonsofformulationsforgraphpartitioning FormulationObjective#0-1decisions#cont.variables#constraints DQ( 4 )quadraticNK0N+KL1( 4 )linearNKjEjKN+K+2KjEjL2( 4 )linearNKjEjN+K+2KjEjL3( 4 )linearNK2NKN+K+2NK Table4-2. Computationalsecondsforgeneralgraphpartitioning GraphsVerticesNEdgesjEjSubsetsKL1L2L3DQ Rand10102320.030.020.020.0232.010.270.030.0843.790.730.250.30 Rand20209824.90.240.060.03Rand4040379212.295.440.150.05Rand100100245323456.692796.042.170.52 theformulationsL1andL2arerelatedtothenumberofedges,whilethenumberinformulationL3isrelatedtothenumberofvertices.Generally,L1andL2canbeusedforspareweightedmatrixW.Thecasesforbipartitegraphpartitioningcanbeanalyzedsimilarly. Computationalresults.Inthissection,allprogramsareimplementedusingCPLEX11.0[ 28 ]viaILOGConcertTechnology2.5,andallcomputationsareperformedonaSUNUltraSpace-IIIwitha900MHzprocessorand2.0GBRAM.ComputationaltimesarereportedinCPUseconds. GraphG=(V,E).InTable 4-2 ,discretequadraticandthreelinearprogrammingapproachesarecompared.ThegraphsarerandomlygeneratedinC++withallweightsbeingnonnegative.ThegapisdefaultinCPLEXandallapproachesobtainthesameoptimalresults.Underthecasesofthesegraphs,thediscretequadraticformulationhasthefastcomputationandthelinearformulation( 4 )isthefastestoneamongthreelinearformulationsthoughithasthemostvariables. BipartitegraphG=(V,U,E).Forthecaseofbipartitegraphs,thematrixAisalsorandomlygeneratedwithweightsbeingnonnegative.ThegapisdefaultinCPLEXandallapproachesobtainthesameoptimalresults.Table 4-3 showsthetime 70

PAGE 71

Table4-3. Computationalsecondsforbipartitegraphpartitioning Graphsvertices(N,M)EdgesjEjKBL1BL2BL3BDQ Rand10,10(10,10)4820.40.220.220.113374.1521.7528.544373.8733.51371.531432.63Rand5,15(5,15)4020.380.160.210.1034.350.791.646.9150.020.050.030.01Rand20,20(20,20)189240.43.552.101.07 comparisonsfordiscretequadraticprogrammingformulationBDQ( 4 )andthreelinearprogrammingformulations,BL1( 4 ),BL2( 4 )andBL3( 4 ).Theresultsshowthatthelinearprogram( 4 ),whichhasthefewestvariablesamongthreelinearprograms,isthemostefcientone,anditisalsobetterthanquadraticformulationsforthesegeneratedgraphs. 4.1.4Discussion Inthissection,azero-onequadraticprogrammingformulationisusedtomodelthegeneralgraphpartitioningproblem,andthecontinuousquadraticprogrammingmethodisalsoused.Twolinearprogrammingapproachescanbederivedfromthezero-onequadraticprogramming,andanotherlinearapproachwithintroducingsomevariablesinadifferentwayisalsopresented.Wehaveimplementedthealgorithmsoftheseformulationsfordifferentgraphsornetworksandcomparedthecomputationalseconds. Severalformulationsandtheirrelaxationstocontinuousformsmayworkfordifferentgraphs,andchoosingwhichofthemtouseforagivengraphisstillindiscussion.Inaddition,forthecomputationalcomplexityoflargegraphs,theparallelcomputingmaybeusefulandalgorithmsshouldbedesignedinsuchoccasionsforgraphpartitioning. ResultsofSection 4.1 arebasedonourpaper[ 42 ]. 4.2RobustOptimizationofGraphPartitioningInvolvingIntervalUncertainty LetG=(V,E)beanundirectedgraphwithasetofverticesV=fv1,v2,,vNgandasetofedgesE=f(vi,vj):edgebetweenverticesviandvj,1i,jNg,whereNisthenumberofvertices.TheweightsoftheedgesaregivenbyamatrixW=(wij)NN, 71

PAGE 72

wherewij(>0)denotestheweightofedge(vi,vj)andwij=0ifnoedge(vi,vj)existsbetweenverticesviandvj.ThismatrixissymmetricforundirectedgraphsGandistheadjacencymatrixofGifwij2f0,1g. AssumeKisthenumberofsubsetsthatwewantpartitionVinto,andCmin,Cmaxarelowerandupperboundsofthecardinalityofeachsubset,respectively.Usually,Kischosenfromf2,,N)]TJ /F4 11.955 Tf 12.06 0 Td[(1gandCmin,Cmaxcanbechosenroughlyfromf1,,NgsuchthatCminCmax. Letxikbetheindicatorsuchthatvertexvibelongstothekthsubsetifxik=1ornotifxik=0,andyijbetheindicatorsuchthattheedge(vi,vj)withverticesvi,vjareindifferentsubsetsifyij=1andvi,vjinthesamesubsetifyij=0.Thus,theobjectivefunctionofgraphpartitioningtominimizethesumofweightsofedgesconnectingdisjointsubsetscanbeexpressedasmin1 2PNi=1PNj=1wijyijorminPNi=1PNj=i+1wijyijbecauseofwij=wjiandwii=0fornon-existenceofloops.Eachvertexvihastobepartitionedintooneandonlyonesubset,i.e.,PKk=1xik=1,andthekthsubsethasthenumberofverticesinrange[Cmin,Cmax],i.e.,CminPNi=1xikCmax.Therelationbetweenxikandyijcanbeexpressedasyij=1)]TJ /F12 11.955 Tf 12.11 8.96 Td[(PKk=1xikxjk(similarto( 4 ))andthiscanbelinearizedas)]TJ /F3 11.955 Tf 9.3 0 Td[(yij)]TJ /F3 11.955 Tf 12.58 0 Td[(xik+xjk0,)]TJ /F3 11.955 Tf 9.3 0 Td[(yij+xik)]TJ /F3 11.955 Tf 12.59 0 Td[(xjk0fork=1,,Kundertheobjectiveofminimization.Therefore,thefeasiblesetofdeterministicformulationofgraphpartitioningproblemforagraphG=(V,E)withweightmatrixWisX=8>>>>>>>>>><>>>>>>>>>>:(xik,yij):PKk=1xik=1,CminPNi=1xikCmax,)]TJ /F3 11.955 Tf 9.3 0 Td[(yij)]TJ /F3 11.955 Tf 11.96 0 Td[(xik+xjk0,)]TJ /F3 11.955 Tf 9.3 0 Td[(yij+xik)]TJ /F3 11.955 Tf 11.95 0 Td[(xjk0,xik2f0,1g,yij2f0,1g,i=1,,N,j=i+1,,N,k=1,,K9>>>>>>>>>>=>>>>>>>>>>;, (4) 72

PAGE 73

andtheobjectivefunctionismin(xik,yij)2XNXi=1NXj=i+1wijyij (4) Thenominalgraphpartitioningproblemistosolvetheprogramwiththeobjective( 4 )andtheconstraintsin( 4 )ofX.Thisisabinaryintegerlinearprogram,whichisquitesimilartotheformulation( 4 ).MethodsforthenominalgraphpartitioningproblemarereviewedinSection 2.2 ThepreviousoptimizationmethodsareallbasedondeterminatedataWandignoretheuncertainty.However,theweightsofedgesarenotalwaysconstantandtheyareuncertain.Forexample,whenanalyzingthecommunitystructureinasocialnetwork[ 44 ],therelationshipbetweentwomembersischangingalongthetimeanditisuncertain.Therefore,thegraphpartitioningproblemwithuncertainweightsofedgesshouldbeconsidered.Therearetwomethodstoaddressdatauncertaintyinmathematicalprogrammingmodels:stochasticprogrammingandrobustoptimization.Thestochasticprogrammingmethodalwaysrequiretheknownprobabilisticdistributionsofuncertaindata,whilerobustoptimizationistooptimizeagainsttheworstcasesbyusingamin-maxobjective[ 8 ].Graphpartitioningisacombinatorialoptimizationproblem.Inthepast,robustversionofmanycombinatorialproblemshavebeenstudied,forexample,therobustshortestpath[ 92 ],therobustspanningtree[ 133 ]aswellasmanyotherproblemsin[ 80 ]. InthisSection 4.2 ,wefollowmethodsusedin[ 8 9 ],whichallowsomeviolationsandproducethefeasiblesolutionwithhighprobability.TheuncertaintyweaddressinthissectionistheintervaluncertaintyforweightmatrixW=(wij)NN.Eachentrywijismodeledasindependent,symmetricandboundedrandombutunknowndistributionvariable~wijthattakesvaluesin[wij)]TJ /F4 11.955 Tf 12.25 0 Td[(^wij,wij+^wij].Notethatwerequirewij=wjiandthus^wij=^wjifori,j=1,,N.Assume^wij0,wij^wijandwii=0foralli,j=1,,N. 73

PAGE 74

Inthissection,robustformulationsforgraphpartitioningwithuncertainWarediscussedandseveralalgorithmswillbeproposedbasedonthepropositionsofformulations.Inaddition,thecasesforbipartitegraphpartitioningarealsostudied. TherestofSection 4.2 isorganizedasfollows:Section 4.2.1 discussestheformulationsfortherobustgraphpartitioningproblem;InSection 4.2.2 ,twodecompositionmethodsforsolvingtherobustgraphpartitioningproblembysolvingaseriesofnominalgraphpartitioningproblemareconstructed;InSection 4.2.3 ,thebipartitegraphpartitioningprobleminvolvinguncertaintyisdiscussed;Section 4.2.4 includesthecomputationalresultsandanalysisoftheseapproaches;Section 4.2.5 istheconclusion. 4.2.1GraphPartitioningwithUncertainWeights Inthissection,therobustoptimizationistoaddresstheuncertaintyofweightmatrixWwith~wij2[wij)]TJ /F4 11.955 Tf 14.26 0 Td[(^wij,wij+^wij],wherewijisnominalvalueofedge(vi,vj).LetJbetheindexsetofWwithuncertainchanges,i.e.,J=f(i,j):^wij>0,i=1,,N,j=i+1,,Ng,whereweassumethatj>isinceWissymmetric.Let)]TJ /F1 11.955 Tf 10.1 0 Td[(beaparameter,notnecessarilyinteger,thattakesvaluesintheinterval[0,jJj].Thisparameter)]TJ /F1 11.955 Tf 10.1 0 Td[(isintroducedin[ 8 9 ]toadjusttherobustnessoftheproposedmethodagainstthelevelofconservatismofthesolution.Thenumberofcoefcientswijisallowedtochangeuptob)]TJ /F2 11.955 Tf 6.77 0 Td[(candanotherwit,jtchangesby()]TJ /F2 11.955 Tf 15 0 Td[()-280(b)]TJ /F2 11.955 Tf 6.77 0 Td[(c).Thusformulationfortherobustgraphpartitioningproblemcanbeestablishedasfollows: min(xik,yij)2X0BBBBBBBB@NXi=1NXj=i+1wijyij+max8>>><>>>:S:SJ,jSj)]TJ -77.53 -22.42 Td[((it,jt)2JnS9>>>=>>>;(X(i,j)2S^wijyij+()]TJ /F2 11.955 Tf 26.26 0 Td[()-222(b)]TJ /F2 11.955 Tf 6.78 0 Td[(c)^wit,jtyit,jt)1CCCCCCCCA (4) Sincethevalueyijtakesvaluefromf0,1g,jyijjinthemodel[ 9 ]isreducedtoyijhere.Dependingonthechosenof)]TJ /F1 11.955 Tf 6.77 0 Td[(,thereareseveralcases:if)-503(=0,nochangesare 74

PAGE 75

allowedandtheproblemreducestonominalproblem( 4 );if)]TJ /F1 11.955 Tf 10.09 0 Td[(ischosenasaninteger,themaximizingpartin( 4 )ismaxfSjSJ,jSj)]TJ /F11 7.97 Tf 4.82 0 Td[(gP(i,j)2S^wijyij;if)-451(=jJj,theproblemcansolvedbySoyster'smethod[ 116 ].TheindexsetJisequivalenttoedgesetEifallweightshaveuncertainty. Asshowninthefollowingtheorem,theproblem( 4 )canbereformulatedasanequivalentbinaryintegerlinearprogramming.Themethodusedinthisproofwasrstproposedin[ 9 ]. Theorem4.3. Theformulation( 4 )isequivalenttothefollowinglinearprogrammingformulation:minNXi=1NXj=i+1wijyij+)]TJ /F3 11.955 Tf 18.72 0 Td[(p0+X(i,j)2Jpij (4)s.t.p0+pij)]TJ /F4 11.955 Tf 13.64 0 Td[(^wijyij0,8(i,j)2Jpij0,8(i,j)2Jp00,(xik,yij)2X. Proof.Forgivenvalues(yij)i=1,,N,j=i+1,,N,thepartmax8>>><>>>:S:SJ,jSj)]TJ -77.53 -22.42 Td[((it,jt)2JnS9>>>=>>>;0@X(i,j)2S^wijyij+()]TJ /F2 11.955 Tf 26.27 0 Td[()-221(b)]TJ /F2 11.955 Tf 6.77 0 Td[(c)^wit,jtyit,jt1A, in( 4 )canbelinearizedbyintroducingzijforall(i,j)2JwiththeconstraintsP(i,j)2Jzij,0zij1,orequivalently,bythefollowingformulationmaxX(i,j)2J^wijyijzij (4)s.t.X(i,j)2Jzij,0zij1,8(i,j)2J. 75

PAGE 76

Theoptimalsolutionofthisformulationshouldhaveb)]TJ /F2 11.955 Tf 6.77 0 Td[(cvariableszij=1andonezij=)]TJ /F2 11.955 Tf 22.05 0 Td[()-222(b)]TJ /F2 11.955 Tf 6.78 0 Td[(c,whichisequivalenttotheoptimalsolutioninthemaximizingpartin( 4 ). Bystrongduality,forgivenvalues(yij)i=1,,N,j=i+1,,N,theproblem( 4 )islinearanditsdualitycanbeformulatedasmin)]TJ /F3 11.955 Tf 37.26 0 Td[(p0+X(i,j)2Jpijs.t.p0+pij)]TJ /F4 11.955 Tf 13.64 0 Td[(^wijyij0,8(i,j)2Jpij0,8(i,j)2Jp00. Combingthisformulationwith( 4 ),weobtaintheequivalentformulation( 4 ),whichnishestheproof. Ouralgorithm(MIP)isbasedonsolvingthebinarylinearprogram( 4 )directlybyCPLEXMIPsolver[ 28 ].ThereareNK+N(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1) 2binarydecisionvariableswithatmostN(N)]TJ /F8 7.97 Tf 6.59 0 Td[(1) 2+1continuousvariablesinthisformulation. 4.2.2DecompositionMethodsforRobustGraphPartitioningProblem Bendersdecompositiontoanominalproblemandalinearprogram.Intheformulation( 4 )fortherobustgraphpartitioningproblem,thevariablesp0,pijarecontinuouswhilexik,yijarebinary.Forthexedxik,yij,theformulation( 4 )canbereformulatedasfollows:min)]TJ /F3 11.955 Tf 37.26 0 Td[(p0+X(i,j)2Jpij+NXi=1NXj=i+1wijyijs.t.p0+pij^wijyij,8(i,j)2Jpij0,8(i,j)2Jp00. 76

PAGE 77

Thisisalinearprogram,andwecanobtainitsdualproblemasfollows:maxX(i,j)2J^wijyijzij+NXi=1NXj=i+1wijyij (4)s.t.X(i,j)2Jzij,0zij1,(i,j)2J. ByBendersdecompositionmethod[ 6 ],theprogram( 4 )presentsthesubproblem.Bysolvingthissubproblemforgivingvaluesofxik,yijattheiterationl,wecanobtainthevaluesz(l)ijandconstructtheoptimalitycutzNXi=1NXj=i+1wijyij+X(i,j)2J^wijyijz(l)ij (4) forthemasterproblem.Therefore,themasterproblemforBendersdecompositionmethodcanbeformulatedasfollows:minz (4)s.t.zNXi=1NXj=i+1wijyij+X(i,j)2J^wijyijz(l)ij,l=1,2,,L(xik,yij)2X. Theprogram( 4 )isalwaysfeasibleandboundedforanyfeasiblesolutionsxik,yijfrom( 4 ),andthus,thefeasibilitycutcanbeeliminatedinthemasterproblem( 4 ).Observing( 4 )and( 4 ),themasterproblemisabinaryintegerlinearprogramwithrespecttoxik,yijandthesubproblemisalinearprogramwithrespecttozij.Thus,byBendersdecompositionmethod,wedecomposethemixedintegerprogramforrobustgraphpartitioningproblemintoaseriesoflinearprogramsandmixedbinarylinearprograms.Additionally,thesubproblem( 4 )canbeeasilysolvedbyagreedyalgorithm:sortingthecoefcients^wijyijofzijfor(i,j)2Jintheobjectivefunctionin 77

PAGE 78

decreasingorder;assigningtherstb)]TJ /F2 11.955 Tf 6.78 0 Td[(ccorrespondingzijstobe1accordingtothisorder;andassigningthelastzijtobe)]TJ /F2 11.955 Tf 9.43 0 Td[()-222(b)]TJ /F2 11.955 Tf 6.77 0 Td[(candallotherszij=0. Theorem4.4. Solvingthemasterproblem( 4 )atiterationlisequivalenttosolvelnominalgraphpartitioningproblems. Proof.AttheiterationlofBendersdecompositionalgorithm,thereareladdedoptimalitycutsintheformof( 5 ),andthiscutisequivalenttozX(i,j)2J(wij+^wijz(l)ij)yij+X(i,j)=2Jwijyij. Therighthandsideofthiscutisinfacttheobjectiveofanominalgraphpartitioningproblemwithweightswij+^wijz(l)ijforedge(i,j)2Jandwijforedge(i,j)=2J.Therefore,solvingthemasterproblematiterationlisequivalenttosolvelnominalgraphpartitioningproblemandthenchoosetheonewithmaximumobjective. Thealgorithm(BD)basedonBendersdecompositionmethodispresentedinthefollowingtable. AlgorithmBD Step1:Initialization:xik,yij:=initialfeasiblesolutioninXforalli,j,k;LB:=,UB:=1;Step2:Whilethereisgaplargerthan"betweenUBandLB,i.e.,UB)]TJ /F3 11.955 Tf 11.96 0 Td[(LB>",dothefollowingsteps:Step2.1:Solvethesubproblem( 4 )toobtainpointzijfor(i,j)2J,andaddcutzPNi=1PNj=i+1wijyij+P(i,j)2J^wijyijzijtothemasterproblem( 4 );UB:=minfUB,PNj=i+1wijyij+P(i,j)2J^wijyijzijg;Step2.2:Solvethemasterproblemminfz:addedcuts,xik,yij2Xg;LB:=z,wherezistheobjectivevalueofmasterproblem;Step3:Outputtheoptimalsolutionxik,yijforalli,j,k. 78

PAGE 79

TheStep1ofthisalgorithmrequiresndingafeasiblesolution.Here,wepresentasimplemethodforndinginitialsolutionsofxiks:puttingverticesv1,v2,,vCminintotherstsubset,i.e.,x11=x21==xCmin,1=1;puttingtheverticesvCmin+1,vCmin+2,v2Cminintothesecondsubset,i.e.,xCmin+1,2==x2Cmin,2=1;repeatingthesestepsuntilwehavexKCmin,K=1;settingx(KCmin+1),K=1,x(KCmin+2),K=1,,xN,K=1andallotherunassignedxikstobe0.Theinitialsolutionforyijcanbeobtainedbyyij=1)]TJ /F12 11.955 Tf 10.04 8.96 Td[(PKk=1xikxjk. TheBendersdecompositionmethodcansolvetherobustgraphpartitioningproblembysolvingaseriesofnominalgraphpartitioningproblem.However,forsolvingthemasterproblematiterationl,itisequivalenttosolvelnominalgraphpartitioningproblem.AlthoughtheBendersdecompositionmethodscanconvergeinnitesteps,sayL,weneedtosolveL(L+1)=2nominalgraphpartitioningproblemstotally.Innextsection,wepresentanotherdecompositionmethod,whichcantakelesscomputationaltimeinsomecases. Algorithmbasedonthedecompositionofonevariable.Forall(i,j)2J,letel(l=1,,jJj)bethecorrespondingvalueof^wijintheincreasingorder.Forexample,e1=min(i,j)2J^wijandejJj=max(i,j)2J^wij.Let(il,jl)2Jbethecorrespondingindexofthelthminimumone,i.e.,^w(il,jl)=el.Inaddition,wedenee0=0.Thus,[e0,e1],[e1,e2],,[ejJj,1)isadecompositionof[0,1). Forl=0,1,,jJj,wedenetheprogramGlasfollows:Gl=)]TJ /F3 11.955 Tf 19.4 0 Td[(el+min(xik,yij)2X8<:NXi=1NXj=i+1wijyij+X(i,j):^wijel+1(^wij)]TJ /F3 11.955 Tf 11.95 0 Td[(el)yij9=;. (4) Totally,therearejJj+1ofGls.Inthefollowingtheorem,weprovethatthedecompositionmethodbasedonp0cansolvetheprogram( 4 ).Themethodintheproofwasrstproposedin[ 9 ]. Theorem4.5. Solvingrobustgraphpartitioningproblem( 4 )isequivalenttosolvejJj+1problemsGlsin( 4 )forl=0,1,,jJj. 79

PAGE 80

Proof.From( 4 )inTheorem 4.3 ,theoptimalsolution(xik,yij,p0,pij)satisespij=maxf^wijyij)]TJ /F3 11.955 Tf 11.95 0 Td[(p0,0g,andtherefore,theobjectivefunctionof( 4 )canbeexpressedasminfp00,(xik,yij)2Xg)]TJ /F3 11.955 Tf 6.77 0 Td[(p0+NXi=1NXj=i+1wijyij+X(i,j)2Jmaxf^wijyij)]TJ /F3 11.955 Tf 11.96 0 Td[(p0,0g=minfp00,(xik,yij)2Xg)]TJ /F3 11.955 Tf 6.77 0 Td[(p0+NXi=1NXj=i+1wijyij+X(i,j)2Jyijmaxf^wij)]TJ /F3 11.955 Tf 11.95 0 Td[(p0,0g, (4) wheretheequalityisobtainedbythefactyijisbinaryinthefeasiblesetX. Bythedecomposition[0,e1],[e1,e2],,[ejJj,1)of[0,1)forp0,wehaveX(i,j)2Jyijmaxf^wij)]TJ /F3 11.955 Tf 11.95 0 Td[(p0,0g=8>><>>:P(i,j):^wijel(^wij)]TJ /F3 11.955 Tf 11.96 0 Td[(p0)yij,ifp02[el)]TJ /F8 7.97 Tf 6.59 0 Td[(1,el],l=1,,jJj;0,ifp02[ejJj,1). Thus,theoptimalobjectivevalueof( 4 )isminl=1,,jJj,jJj+1fZlg,whereZl=minfp02[el)]TJ /F15 5.978 Tf 5.76 0 Td[(1,el],(xik,yij)2Xg0@)]TJ /F3 11.955 Tf 6.78 0 Td[(p0+NXi=1NXj=i+1wijyij+X(i,j):^wijel(^wij)]TJ /F3 11.955 Tf 11.95 0 Td[(p0)yij1A, (4) forl=1,,jJj,andZjJj+1=minfp0ejJj,(xik,yij)2Xg)]TJ /F3 11.955 Tf 6.78 0 Td[(p0+NXi=1NXj=i+1wijyij. Forl=1,,jJj,sincetheobjectivefunction( 4 )islinearovertheintervalp02[el)]TJ /F8 7.97 Tf 6.59 0 Td[(1,el],theoptimaliseitheratthepointp0=el)]TJ /F8 7.97 Tf 6.59 0 Td[(1orp0=el.Forl=jJj+1,ZlisobtainedatthepointejJjsince)]TJ /F2 11.955 Tf 10.1 0 Td[(0. Thus,theoptimalvalueminl=1,,jJj,jJj+1fZlgwithrespecttop0isobtainedamongthepointsp0=elforl=0,1,,jJj.LetGlbethevalueatpointp0=elin( 4 ),i.e.,Gl=)]TJ /F3 11.955 Tf 19.4 0 Td[(el+min(xik,yij)2X8<:NXi=1NXj=i+1wijyij+X(i,j):^wijel+1(^wij)]TJ /F3 11.955 Tf 11.95 0 Td[(el)yij9=;. 80

PAGE 81

Wenishtheproof. AsshowninTheorem 4.5 ,GjJj=)]TJ /F3 11.955 Tf 20.55 0 Td[(ejJj+PNi=1PNj=i+1wijyijistheoriginalnominalproblemwithanaddedconstant.OurAlgorithm(DP0)isbasedonthistheorem. AlgorithmDP0 Step1:Forall(i,j)2J,sort^wijinincreasingordertoobtaine0,e1,,ejJj;Step2:Forl=0,1,,jJj,solvingGlin( 4 );Step3:Letl=argminl=0,1,,jJjGlandobtaintheoptimalsolutionfxik,yijg=fxik,yijgl. Algorithm(DP0)isbasedonthedecompositionofp02[0,1)andeachsubproblemGlhasthesamecomputationalcomplexityasthenominalgraphpartitioningproblem.SincethenominalgraphpartitioningproblemisNP-complete,fromthedecompositionalgorithm(DP0),wecanconcludethattherobustgraphpartitioningproblemisalsoNP-complete. 4.2.3BipartiteGraphPartitioningInvolvingUncertainty ThebipartitegraphisdenedasG=(V,U,E)withvertexsetsV=fv1,,vNg,U=fu1,,uMgandedgesetE=f(vi,uj):edgebetweenverticesvianduj,1iN,1jMg,whereNandMarethenumbersofverticeswithintwosets,respectively.Usually,insteadofweightedmatrix,thebiadjacencyweightedmatrixA=(aij)NMisgivenwhereai,jistheweightofedge(vi,uj).In[ 42 43 ],therelationsforpartitioningbetweengraphsandbipartitegraphshavebeenpresented.AssumewestillwanttoobtainKsubsetsofbothVandU,andthecardinalityforsubsetsofVisintherange[Cmin,Cmax]andthecardinalityforsubsetsofUisintherange[cmin,cmax].Letthe 81

PAGE 82

constraintsofbipartitegraphpartitioningbeasetasfollows:Y=8>>>>>>>>>><>>>>>>>>>>:(xvik,xujk,yij):PKk=1xvik=1,CminPNi=1xvikCmax,PKk=1xujk=1,cminPMj=1xujkcmax,)]TJ /F3 11.955 Tf 9.3 0 Td[(yijxvikxujk0,xvik,xujk,yij2f0,1g,i=1,,N,j=1,,M,k=1,,K9>>>>>>>>>>=>>>>>>>>>>;, wherexvik,xujk,yijaretheindicatorsforvertexsetsV,U,andedgesetEasthesameexplanationsinsection1. Thebipartitegraphpartitioningproblemisformulatedasmin(xvik,xujk,yij)2YNXi=1MXj=1aijyij. BecauseofitssimilaritytothegraphpartitioningproblemasdiscussedinSection 4.2.1 ,wealsoconsidertheuncertainofmatrix~Awhere~aijtakesvaluesin[aij)]TJ /F4 11.955 Tf 9.38 0 Td[(^aij,aij+^aij]fori=1,,N,j=1,,M.Therobustoptimizationforuncertain~aijisasmin(xvik,xujk,yij)2YNXi=1MXj=1aijyij+max8>>><>>>:S:SJ,jSj)]TJ -77.53 -22.42 Td[((it,jt)2JnS9>>>=>>>;0@X(i,j)2S^aijyij+()]TJ /F2 11.955 Tf 26.27 0 Td[()-222(b)]TJ /F2 11.955 Tf 6.77 0 Td[(c)^ait,jtyit,jt1A, (4) whereJ=f(i,j):^aij>0gand)]TJ /F2 11.955 Tf 10.09 0 Td[(2[0,jJj]. AsprovedinTheorem 4.3 ,wecanobtainthelinearformulationas( 4 )forrobustbipartitegraphpartitioning( 4 )similarlyasfollows: 82

PAGE 83

minNXi=1MXj=1aijyij+)]TJ /F3 11.955 Tf 18.73 0 Td[(p0+X(i,j)2Jpij (4)s.t.p0+pij)]TJ /F4 11.955 Tf 11.86 0 Td[(^aijyij0,(i,j)2Jpij0,(i,j)2Jp00,(xvik,xujk,yij)2Y. Weomitothermethodsandalgorithmsheresincetherobustoptimizationforbipartitegraphpartitioningisquitesimilartographpartitioningproblems. 4.2.4NumericalExperiments Inthissection,allalgorithms(MIP,BD,DP0)areimplementedusingCPLEX11.0[ 28 ]viaILOGConcertTechnology2.5,andallcomputationsareperformedonaSUNUltraSpace-IIIwitha900MHzprocessorand2.0GBRAM.ComputationaltimesarereportedinCPUseconds. Alltestedgraphsarerandomlygenerated.Thedensityrofagraphistheratioofthenumberofedgesandthenumberofpossibleedges.Theuncertainvaluesof[wij)]TJ /F4 11.955 Tf 14.7 0 Td[(^wij,wij+^wij]arerandomlygenerated.Hereweassumewij2f0,1gand0<^wij=wij<1ifwij>0.InTable 4-4 ,weassumethecardinalityofeachsubsetisintherange[Cmin,Cmax]=[1,N)]TJ /F4 11.955 Tf 12.42 0 Td[(1].ThegapinCPLEXissettobe0.1.AllobjectivevaluesandcomputationalsecondsarepresentedinTable 4-4 .Fromthistable,wecanndthatthealgorithm(DP0)isthemostefcientone,andthealgorithm(BD)isleastefcient.AsdiscussedinSection 4.2.2 ,thealgorithm(BD)needstocomputelnominalgraphpartitioningproblemsatiterationl,whilethealgorithm(DP0)computesjJj+1nominalgraphpartitioningproblemstotally.Thus,iftheBendersdecompositionmethodcannotconvergesquicklyinsmallnumberofiterations,itusuallytakeslongertimethanthealgorithm(DP0). 83

PAGE 84

Insteadofloosecardinalities,weassumetheboundsCmin,CmaxtakevaluearoundN=K.AllobjectivevaluesandcomputationalsecondsfordifferentgraphsarepresentedinTable 4-5 withmoreconservativecardinalityconstraints.Fromthistable,wecanseeinthecaseofsamenumberofvertices,thecomputationaltimesincreaseasthedensityincreases.ComparingresultsinTable 4-4 andTable 4-4 ,inthesamegraph,thecasewithloosecardinalityconstraintstakesshortertimethantheonewithconservativebounds. AsdiscussedinSection 4.2.1 ,theparameter)]TJ /F1 11.955 Tf 10.1 0 Td[(isintroducedin[ 8 9 ]toadjusttherobustnessoftheproposedmethodagainstthelevelofconservatismofthesolution.InFig. 4-1 ,fortherandomgeneratedgraphwith20verticesanddensity0.4with76uncertainedges,therelationshipbetweentheobjectivevaluesandthevaluesof)]TJ /F1 11.955 Tf 10.1 0 Td[(ispresentedforobtaining4subsets.Fromthisgure,wecanseethatasthevalueof)]TJ /F1 11.955 Tf -439.98 -23.91 Td[(increasesforconsideringmoreuncertainties,theobjectivevaluesareincreasingaswell.Butwhen)]TJ /F1 11.955 Tf 10.1 0 Td[(islargeenough,theobjectivevalueisaconstant.Additionally,thevaluesforthecasesofconservativecardinalitiesarelargerthancorrespondingcasesofloosecardinalities. 4.2.5Discussion IntheSection 4.2 ,wepresentthreealgorithmsfortherobustgraphpartitioningproblemwithintervaluncertainweights.Werstpresenttheformulationforthisproblemandthengivetheequivalentmixedintegerlinearprogrammingformulation.Twodecompositionmethods,includingBendersdecompositionmethodanddecompositionononevariable,cansolvetherobustgraphpartitioningproblembysolvingaseriesofnominalgraphpartitioningproblems.Wecomparethesealgorithmsonrandomlygeneratedgraphswithuncertainweights.Inthissection,aparameter)]TJ /F1 11.955 Tf 6.77 0 Td[(,introducedby[ 8 9 ],ischosentoallowsomegapbetweentheoptimalvalueoftheexactformulationandtherobustsolutions.Additionally,westudythebipartitegraphpartitioningprobleminvolvinguncertainweights. 84

PAGE 85

Table4-4. Computationalresultsandsecondsforrobustgraphpartitioning(1) GraphsUncertaintyObjectiveValuesCPUSeconds NrKjJj)]TJ /F16 10.909 Tf 35.67 0 Td[(MIPBDDP0MIPBDDP0 100.13420000.010.010.010.2952.182.183.270.090.350.060.31371.031.031.670.100.230.050.41894.804.804.800.140.480.220.522117.687.688.590.390.650.200.627148.068.068.060.230.420.260.7301511.5811.5811.580.731.210.370.8351814.8514.8514.850.850.970.460.9402018.0818.0318.290.894.130.28 150.131050000.010.030.010.221114.944.944.980.341.010.280.331164.554.554.550.750.560.390.442219.509.509.501.185.720.430.5522611.8611.7811.862.162.200.60 200.1319100000.010.040.030.237192.692.692.640.781.390.430.357296.586.586.581.111.560.700.4763814.5214.4814.522.602.751.060.5954815.6515.6516.253.2511.591.48 200.1419100000.010.060.020.237196.506.506.741.566.990.920.3572912.0112.0113.113.329.961.540.4763821.8521.8521.8513.2441.879.070.5954825.6525.0525.6511.0521.044.74 300.1342211.041.041.460.860.510.070.287447.987.987.986.884.601.840.31306510.6410.6410.644.347.432.39 300.1442212.192.192.292.406.091.300.2874413.4313.2314.2420.8033.256.710.31306516.4416.4416.4412.6020.477.49 400.1378393.413.413.415.073.791.070.21557810.4610.4611.396.7513.993.830.323311720.6420.6422.0058.36134.2435.57 400.1478395.465.466.893.8311.832.490.21557816.2516.2516.25118.86303.0313.350.323311731.9431.6132.23229.26>3000259.06 500.13122611.731.731.732.021.460.120.224212111.0411.0413.0521.7323.1910.850.336618325.6525.6525.652088.19320.4518.56 500.14122613.513.513.655.507.822.110.224212119.6419.9421.651129.04>3000940.200.336618339.7139.7142.63>3000>3000>3000 500.15122615.435.435.4310.8020.764.650.224212128.1328.1331.01>3000>3000>3000 Note:Inthistable,thetheboundsforcardinalitiesarechosenas1,N)]TJ /F18 10.909 Tf 11.71 0 Td[(1,i.e.,Cmin=1,Cmax=N)]TJ /F18 10.909 Tf 10.91 0 Td[(1. 85

PAGE 86

Table4-5. Computationalresultsandsecondsforrobustgraphpartitioning(2) GraphsUncertaintyCardinalityObjectiveValuesCPUSeconds NrKjJj)-1978([Cmin,Cmax]MIPBDDP0MIPBDDP0 100.1342[3,5]0000.010.020.020.2952.552.552.550.200.260.140.31376.296.216.030.170.610.240.418914.0114.0114.130.592.140.420.5221118.9718.9719.190.8013.160.520.6271426.2225.4926.231.4137.690.660.7301527.6527.5928.801.9342.980.850.8351832.0832.0034.344.9352.671.930.9402038.5237.8239.635.01241.773.54 150.13105[4,6]1.311.311.690.070.120.070.221117.657.657.650.730.580.440.3311618.6717.9218.060.874.450.450.4422128.7528.7530.921.5918.500.710.5522640.5840.4940.9510.26479.571.78 200.131910[6,8]5.355.295.620.410.410.280.2371917.5917.5917.591.514.460.880.3572934.2734.0334.223.5442.342.100.4763857.5455.4556.9235.651358.5516.830.5954871.5871.5671.5676.80>300035.58 200.141910[4,6]6.506.357.700.602.180.550.2371922.0321.7822.1013.79115.282.980.3572943.2543.1943.8437.21>30009.000.4763865.3765.2765.27209.51>300046.650.5954884.0283.7585.891190.84>3000346.60 300.134221[9,11]13.8713.8715.121.725.261.210.2874447.1047.0849.5115.51434.3624.440.31306586.1684.5686.66358.65>3000113.79 300.144221[7,9]17.5917.5817.595.7185.382.670.2874459.2458.7260.36754.93>3000156.810.313065106.62104.48105.34>3000>30002222.75 Note:TheseresultsarebasedonsamedatasetsasthoseinTable 4-4 andsameparametersexceptthat[Cmin,Cmax]. 86

PAGE 87

(N=20,r=0.4,jJj=76,K=4,[Cmin,Cmax]=[1,19],[4,6]) Figure4-1. Robustgraphpartitioningobjectivevaluesregarding)]TJ ET BT /F1 11.955 Tf 23.41 -326.13 Td[(ThegraphpartitioningproblemisNP-complete,andtherobustgraphpartitioningproblemisalsoNP-completeasshowninSection 4.2.2 .Twodecompositionalgorithms,whicharesolvingaseriesofnominalgraphpartitioningproblems,canbeusedforfurtherresearch.Forexample,theapproximativesemideniteprogramming(SDP)methodisusefulfornominalgraphpartitioningproblem,andwecancombinethesedecompositionmethodsandtheSDPmethodtosolvelargerobustgraphpartitioningproblemsefciently. Resultsofsection 4.2 arebasedonourpaper[ 44 ]. 4.3Two-StageStochasticGraphPartitioningProblem LetG=(V,E)beanundirectedgraphwithasetofverticesV=fv1,v2,,vNgandasetofedgesE=f(vi,vj):edgebetweenverticesviandvj,1i,jNg,whereNisthenumberofvertices.TheweightsoftheedgesaregivenbyamatrixW=(wij)NN,wherewij(>0)denotestheweightofedge(vi,vj)andwij=0ifnoedge(vi,vj)existsbetweenverticesviandvj.ThismatrixissymmetricforundirectedgraphsG.Thus,the 87

PAGE 88

edgesetcanbeexpressedbyE=f(vi,vj):wij>0,1i
PAGE 89

areusedforthisproblem.Asdiscussedin[ 44 ]andSection 4.2 ,theweightsofedgesingraphG=(V,E)arealwaysuncertain.Inmathematicalprogramming,twomethodsarealwaysusedtodealwithsuchuncertainty.Therobustoptimizationmodelsforgraphpartitioning,istondoutabestpartitioningofvertexsetVamongalluncertainweightsofedgesintheworstcase.Inthissection,wewillintroducethetwo-stagestochasticgraphpartitioningproblemwithniteexplicitscenariostodealwiththeuncertainty. Thesetofcutedgesorthecutsetincludesalledgeswithendsindifferentsubsetsafterpartitioning.Thetwo-stagestochasticgraphpartitioningproblem(TSGP)consistsofndingabestpartitionofvertexsetintwostages:takingsomeedgesintothesetofcutedgesintherststagewithcertainweightsforedgesinmatrixW;assumingthattherearetotallySscenarioswithweightmatrixCs=(csij)NNinscenariosofprobabilityps,andthesecondstageinscenariosistochoosesomeedgesintothesetofcutedgesforsatisfyingtherequirementsofpartitioning.Theobjectiveistominimizethetotalexpectedweightofedgesinthesetofcutoverallscenarios.Undertheserequirementsanddescriptions,weformulatethetwo-stagestochasticgraphpartitioningproblemasastochasticmixedintegerprogram(SMIP)[ 12 ]. Similarly,manycombinatorialoptimizationproblemshavebeenextendedtotwo-stagestochasticformsrecently.Thetwo-stagemaximummatchingproblemisstudiedin[ 79 ]byanapproximationalgorithm,theminimumspanningtreeproblemisextendedtotwo-stageformsin[ 49 ],andetc. InSection 4.3 ,weassumethedistributionofweightshasniteexplicitscenarios.TherestofSection 4.3 isorganizedasfollows:Section 4.3.1 presentsthemodelforthetwo-stagestochasticgraphpartitioningproblem;InSection 4.3.2 ,wepresentequivalentintegerlinearformulations;InSection 4.3.3 ,wepresentnumericalexperimentsonrandomlygeneratedgraphswithdifferentnumbersofscenarios;Section 4.3.4 concludesSection 4.3 89

PAGE 90

4.3.1TheModeloftheTwo-StageStochasticGraphPartitioningProblem Thetwo-stagestochasticgraphpartitioningproblemcanbeformallystatedasfollows:GivenagraphG=(V,E)withtherst-stageedgeweightmatrixWandsecond-stageedgeweightmatrixCs=(csij)NNfors=1,,S,andtheprobabilitypsforscenarios,whereSisthenumberofscenarios.TheedgesetEnowisdenedasE=f(vi,vj):wij>0orcsij>0forsomes,j>ig,whichmeansifwij>0oroneofcsij>0,theedge(vi,vj)exists.Thatis,foredge(i,j)2E,therst-stageweightiswijandthesecondstageweightiscsijinscenarioswithprobabilityps.Inaddition,wearealsogiventhenumberKofsubsetsthatwewanttopartitionVintoandthecardinalitiesn1,,nKofallsubsets. Remark7. TheweightmatrixCsforscenarioshasthesamepregurementsasW:noloopsinthegraph,i.e.,csii=0fori=1,,N;symmetrically,i.e.,csij=csjifori,j=1,,N;nonnegativity,i.e.,csij0fori,j=1,,N. Remark8. Theprobabilitypsfors=1,,SandweightmatricesCs'saredenedonaprobabilityspace(,C,P),whereisthesamplespaceandcanbechosenasnonnegativerealspaceRNN+,Cisthesetofsubsetsin,andPistheprobabilitymeasure.Inthisproblem,weassumeniteexplicitscenarios. Thetwo-stagestochasticgraphpartitioningproblem(TSGP)istondasetofcutedgesECwiththeminimumsumofweightssothatthesubsetssatisfytherequirementsateachscenario.AssumethatE0isthesetofcutedgeschosenintherst-stage,andEsisthechosensetofcutedgesinthesecond-stagewithrespecttoscenariosfors=1,,S,thesetshavetherelationsE0[EsisthesetthatcancompletelyseparatetheverticesintoKsubsetswiththerequirementofcardinalities,andE0\Es=;.Inaddition,thecutsE0,E1,,ESshouldhavetheminimumexpectedsumofweightsX(vi,vj)2E0wij+SXs=1psX(vi,vj)2Escsij. 90

PAGE 91

Forexample,inFig. 4-2 ,theweightsforedge(vi,vj)2Eatststage(wij)andsecondstage(csij)fortwoscenarios(p1=0.6,p2=0.4)areshown.Theproblemistond3subsetswithcardinalitiesasn1=3,n2=4,n3=5.BytheSTGPmodel,twoedges(2,4),(7,12)areselectedintothecutsetattherststage,whileatthesecondstage,edges(3,7),(3,11),(6,8)areselectedintherstscenarios=1andedges(4,7),(5,7),(6,7),(8,9),(8,11),(8,12),(10,11),(11,12)areselectedinthesecondscenarios=2.Threesubsetsobtainedforscenarios=1aref1,2,3g,f4,5,6,7g,f8,9,10,11,12g,whilethreesubsetsforscenarios=2aref9,10,12g,f4,5,6,8g,f1,2,3,7,11g. Figure4-2. Anexamplefortwo-stagestochasticgraphpartitioningproblem Assumethatyij=1denotesthat(i,j)ischosentoE0andotherwiseyij=0,andzsij=1denotesedge(i,j)ischosentoEsinscenariosandotherwisezsij=0.Letxsik=1denotethattheithvertexbelongstothekthsubsetandotherwisexsik=0.Bythesedecisionvariables,thetwo-stagestochasticgraphpartitioningproblemcanbe 91

PAGE 92

formulatedasthefollowingtwo-stageprogram:[TSGP]:minX(i,j)2Ewijyij+SXs=1psf(y,s) (4)s.t.yij2f0,1g, (4)i2V,(i,j)2E. wherefors=1,,S,f(y,s)=minx,zX(i,j)2Ecsijzsij (4)s.t.KXk=1xsik=1,NXi=1xsik=nk (4)yij+zsij=1)]TJ /F9 7.97 Tf 17.13 14.95 Td[(KXk=1xsikxsjk, (4)xsik,zsij2f0,1g, (4)i2V,(i,j)2E,k=1,,K. Next,werstprovethisformulationisthecorrectformulationfortwo-stagestochasticgraphpartitioningproblem,andthendiscusstherelaxationsofthevariablesyij'sandzsij's. Theorem4.6. Theformulation( 4 )-( 4 )isthecorrectmodelforthetwo-stagestochasticgraphpartitioningproblem. Proof.Fromtheobjectivefunctionin( 4 ),thedecisionvariablesyijandzsijdecidewhethertheedge(vi,vj)isincludedinthesetofcutedgesforscenarioswithrespecttotherststageweightwijandsecondstageweightcsij,respectively.Theconstraintsxsik2f0,1gandtheconstraints( 4 )canguaranteethateachvertexbelongstoexactonesubsetandthekthsubsethasthecardinalitynkinthesecondstageofscenarios. 92

PAGE 93

Thus,thesetofcutedgesintherststageisE0=f(vi,vj)2E:yij=1gandthesetofcutedgesinthesecondstageofscenariosisEs=f(vi,vj)2E:zsij=1g.WehavetoprovethatE0[EsisthesetofcutedgesandE0\Es=;foranys=1,,S. Ifbothvertexviandvjbelongtosubsetkinscenarios,i.e.,xsik=xsjk=1andxsik0=xsjk0=0fork06=k,fromtheconstraint( 4 ),wehaveyij+zsij=0.Thus,yij=zsij=0sincebothofthemarebinary.Theedge(vi,vj)isnotinthesetofcutedges. Ifthevertexvibelongstosubsetk1andvjbelongstosubsetk2ofscenarios,i.e.,xsi,k1=1,xsik=0forallk6=k1andxsj,k2=1,xsj,k0=0forallk06=k2wherek16=k2,thuswehaveyij+zsij=1fromtheconstraint( 4 ).Thus,sincebothyij,zsij2f0,1g,eitheryij=1,whichmeansedge(vi,vj)ischosenincutedgesintherststage,orzsij=1,whichmeans(vi,vj)ischoseninthecutedgesinthesecondstageofscenarios.Consideringalledges,wehaveprovedthatE0[EsisthesetofcutedgesandE0\Es=;foranys=1,,S. Theobjectivefunctionistominimizeststageweightandtheexpectedsumweightofallscenarioss=1,,Sforedgeswithinthecut.Therefore,wehavecheckedtheobjectiveandallconstraints,andtheprogram( 4 )-( 4 )correctlyformulatesthetwo-stagestochasticgraphpartitioningproblem. Corollary4.1. Foredge(vi,vj)2E,ifwij>PSs=1pscsij,wehaveyij=0. Proof.Bycontradiction,ifyij=1,whichmeansthatedge(vi,vj)ischosenintothecutsetintherststage,andtheobjectivewithrespecttothisedgeiswij.However,bychoosingcsij=1foralls,thecorrespondingobjectiveisPSs=1pscsij,whichislessthanwij,acontradictiontominimizingtheobjective. However,forthecasewij=PSs=1pscsijforedge(vi,vj)2E,ifthisedgeischosenintothecutset,eitheryij=0,csij=1(8s)oryij=1,csij=0(8s)areoptimalsolutions;if 93

PAGE 94

thisedgeisnotchosenintothecutset,yij=0.Inordertoreducecomputationtime,wemakethefollowingassumptionwithoutlossofanyoptimality: AssumptionAForedge(vi,vj)2E,ifwij=PSs=1pscsij,assumeyij=0. 4.3.2EquivalentIntegerLinearProgrammingFormulations Intheconstraint( 4 ),thereisanonlineartermxsikxsjk,whichalwaysleadstohighcomputationalcomplexity.Inthissection,wepresentanapproachtolinearizethisterm.Additionally,weprovethatsomebinaryvariablesintheformulationforTSGPcanberelaxedtocontinuousones. Lemma4.1. ByCorollary 4.1 andunderAssumptionA,thedecisionvariablesyij,zsijinthetwo-stagestochasticgraphpartitioningproblem( 4 )-( 4 )canberelaxedtobecontinuousonessuchthat0yij1,zsij0. Proof.InthepartE0,EsoftheproofforTheorem 4.6 ,inthecaseofverticesvi,vjinthesamesubsetinscenarios,i.e.,yij+zsij=0,wehaveyij=zij=0ifyij,zsij0. Inthecaseofverticesvi,vjindifferentsubsetsinscenarios,i.e.,yij+zsij=1,wediscussinthefollowingthreecases: (a) wij>PSs=1pscsij.ByCorollary 4.1 ,yij=0andthenzsij=1. (b) wij=PSs=1pscsij.ByAssumptionA,yij=0andthenzsij=1. (c) wij
PAGE 95

Forthenonlineartermxsikxsjk,byintroducingusijk=xsikxsjk,wecanhavefollowinglinearizationmethods. Lemma4.2. Theconstraint( 4 )foredge(vi,vj)2Einscenariosisequivalenttofollowinglinearconstraints:8>>>>>>>>>>>>>><>>>>>>>>>>>>>>:yij+zsij=1)]TJ /F12 11.955 Tf 11.96 8.97 Td[(PKk=1usijkusijkxsikusijkxsjkusijkxsik+xsjk)]TJ /F4 11.955 Tf 11.95 0 Td[(1usijk0 (4) ByLemma 4.1 andLemma 4.2 ,wehavethefollowingtheorem,whichpresentstheequivalentintegerlinearprogrammingformulationforTSGP. Theorem4.7. Theformulationin( 4 )-( 4 )forTSGPunderAssumptionAisequivalenttothefollowingintegerlinearprograminextensiveform:minX(i,j)2Ewijyij+SXs=1psX(i,j)2Ecsijzsijs.t.( 4 ),( 4 )xsik2f0,1g,0yij1,zsij0i2V,(i,j)2E,k=1,,K,s=1,,S Forsomespeciccase,wecanhavemoresimpliedformulationforTSGPasshowninthefollowingcorollary. 95

PAGE 96

Corollary4.2. Ifwij>0andcsij>0holdforalledge(vi,vj)2Eforallscenarios,theformulationin( 4 )-( 4 )forTSGPisequivalentto:minX(i,j)2Ewijyij+SXs=1psX(i,j)2Ecsijzsij (4)s.t.KXk=1xsik=1,NXi=1xsik=nk (4))]TJ /F4 11.955 Tf 11.96 0 Td[((yij+zsij))]TJ /F3 11.955 Tf 11.96 0 Td[(xsik+xsjk0 (4))]TJ /F4 11.955 Tf 11.96 0 Td[((yij+zsij)+xsik)]TJ /F3 11.955 Tf 11.95 0 Td[(xsjk0 (4)xsik2f0,1g,0yij1,zsij0 (4)i2V,(i,j)2E,k=1,,K,s=1,,S Proof.Foredge(vi,vj)2Esuchthatwij>0andcsij>0inscenarios,differentfromLemma 4.1 ,ifverticesvi,vjareinthesamesubset,i.e.,xsik0=xsjk0=1andxsik=xsjk=0forallk6=k0,wehaveyij+zsij0withconsideringallk'sin( 4 )and( 4 );Similarly,ifverticesvi,vjaredifferentsubsets,wehaveyij+zsij1from( 4 )and( 4 ).AsmentionedinLemma 4.1 ,theobjectivefunction( 4 )withrespecttoedge(vi,vj)ismin(wijyij+PSs=1pscsijzsij). Forthecaseyij+zsij0,wewanttoprovethatyij=0andzij=0bytheformulation( 4 )-( 4 ).wehavethreesubcases: (a) wij>PSs=1pscsij.ByCorollary 4.1 ,yij=0andthenzsij0.Now,min(wijyij+PSs=1pscsijzsij)=minPSs=1pscsijzsijshouldobtaintheoptimalvalueatzsij=0sinceallcsij>0. (b) wij=PSs=1pscsij.ByAssumptionA,yij=0andthenzsij0.Thiscanbeprovedsimilarlytoabovecase. (c) wij0andcsij>0foralls. Forallsubcases,wehaveyij=0,zsij=0whenyij+zsij=0,whichisthesameasthecaseyij+zsij=0inLemma 4.1 96

PAGE 97

Forthecaseyij+zsij1,wealsodiscussthreesubcases: (d) wij>PSs=1pscsij.ByCorollary 4.1 ,yij=0andthenzsij1.Now,min(wijyij+PSs=1pscsijzsij)=minPSs=1pscsijzsijshouldobtaintheoptimalvalueatzsij=1sinceallcsij>0. (e) wij=PSs=1pscsij.ByAssumptionA,yij=0andthenzsij1.Thiscanbeprovedsimilarlytoabovecase. (f) wij0foralls;Ifyij=1,theobjectiveforedge(vi,vj)obtaintheoptimalvaluewijatzsij=0forallssincezsij0andcsij>0foralls;If0awij+(1)]TJ /F3 11.955 Tf 11.95 0 Td[(a)pscsij>wijforany00andcsij>0foralls.Forexample,incase(d),ifcsij=0,thetermmincsijzsijwithzij1canobtaintheminimumvalue0atanyvalueofnonnegativezsij;incase(c),wij=0willinuenceanynonnegativechoiceofyij. ComparingformulationsinTheorem 4.7 andCorollary 4.2 ,wecanseethattheformulationinCorollary 4.2 reducesalotofconstraintsandvariables,anditcanbe 97

PAGE 98

solvedmoreefciently.OurmethodforsolvingTSGPisusingCPLEX[ 28 ]tosolvethebinarylinearprogramsinextensiveforminTheorem 4.7 andCorollary 4.2 4.3.3NumericalExperiments Inthissection,weimplementallbinarylinearprogramsinTheorem 4.7 andCorollary 4.2 usingCPLEX11.0viaILOGConcertTechnology2.5.AllcomputationsareperformedonaSUNUltraSpace-IIIwitha900MHzprocessorand2.0GBRAM.ComputationaltimesarereportedinCPUseconds. Totestouralgorithms,wegeneraterandomgraphswithdifferentnumberofnodes,edgesandscenarios.Thenumberofedgesisdeterminedbythedensityrofagraph,whichistheratioofthenumberofedgesandthenumberofpossibleedges.Allgeneratedweightssatisfywij,csij2[2,3]for(vi,vj)2E.ThenumberN(N)]TJ /F4 11.955 Tf 12.48 0 Td[(1)r=2ofdecisionvariablesyijsisrelatedtothenumberofedges,whichisrelatedtoN,r;thenumberN(N)]TJ /F4 11.955 Tf 12.09 0 Td[(1)rS=2ofzsijsisdeterminedbyN,r,s;andthenumberNKSofxsiksisdeterminedN,S,K.InTheorem 4.7 ,thevariablesusijksareintroduced,andthenumberofusijksisN(N)]TJ /F4 11.955 Tf 11.96 0 Td[(1)rKS=2. InTable 4-6 ,theobjectivevaluesobtainedbysolvingformulationsinTheorem 4.7 andCorollary 4.2 arethesamewhilethegapinCPLEXissettingto0.01. FromTable 4-6 ,wecanseeallcomputationalsecondsbytheformulationinCorollary 4.2 arelessthanorequaltosecondsbytheformulationinTheorem 4.7 underthesametestcases.AsdiscussedinSection 4.3.2 ,theformulationinTheorem 4.7 introducesthevariablesusijksandthisinuencesthecomputationalcomplexity.ForthegraphwithN=10verticesandS=2scenarios,whenthedensityrisincreasing,thecomputationalsecondsareincreasingaswell;forthegraphN=10,S=2andthesamedensityr,whenthenumberKofsubsetsincreases,thecomputationalsecondsincreaseaswellforbothformulations.Similarly,whenthenumberSofscenariosandthenumberNofverticesareincreasing,computationalsecondsincreaseaswell. 98

PAGE 99

4.3.4Discussion Inthissection,wepresentthetwo-stagestochasticgraphpartitioningproblembythestochasticprogrammingapproach.Thisproblemisformulatedasanonlinearstochasticmixedintegerprogram,andwealsopresentalinearprogramming(SMIP)approachforsolvingthisproblembyCPLEX.Forsomecaseswithspecicrequirements,wepresentamoresimpliedlinearizationmethod,whichcansolvetheproblemmoreefciently.TheSMIPproblemisveryhardtosolve,andforfurtherresearch,moreefcientalgorithms,suchasmethodsbasedonBendersdecomposition[ 6 ],shouldbedesignedforsolvinglarge-scaletwo-stagestochasticgraphpartitioningproblems.ResultsofSection 4.3 arebasedonourpaper[ 45 ]. 99

PAGE 100

Table4-6. Computationalresultsandsecondsfortwo-stagestochasticgraphpartitioning GraphsProbabilityDist.CardinalitiesObjectiveValueCPUSeconds NrSp1++pSn1++nKThm 4.7 /Cor 4.2 Thm 4.7 Cor 4.2 100.120.6+0.45+50.000.010.010.413.490.100.060.731.610.500.33157.9117.3015.84 0.120.6+0.43+3+42.110.040.030.419.723.202.110.743.9423.0417.78176.772295.311828.23 0.120.6+0.43+2+3+22.110.050.040.422.184.954.040.752.08170.38120.70186.145088.863583.21 100.140.1+0.2+0.3+0.44+3+30.000.030.030.420.8090.6264.15 0.160.05+0.10+0.15+0.20+0.25+0.253+3+42.211.330.860.419.471938.17774.08 150.120.6+0.45+5+54.580.090.080.443.4242.3323.41 200.120.6+0.47+7+66.520.850.420.485.224354.522519.88 250.120.6+0.48+8+913.957.001.23300.120.6+0.410+10+1022.3311.231.44 100

PAGE 101

CHAPTER5APPLICATIONSINPOWERSYSTEMSFORISLANDING Apowergridislandisaself-sufcientsubnetworkinalarge-scalepowersystem.Inweaklyconnectedislands,limitedinter-islandpowerowsareallowed.Intentionalislandingdetectionishelpfulfortheanalysisofdistributedgenerationsystemsconnectedtoapowergrid,andalsovaluableforpowersystemreliabilityinextremeemergencystates.Inthischapter,weusegraphpartitioningmethodstoformislandsinapowergridandformulatetheseproblemsasmixedintegerprograms.Ourmodelsarebasedtheoptimalpowerowmodeltominimizetheloadsheddingcost.Withthesemathematicalprogrammingmodels,optimalformationofislandscanbeobtainedandthedifferentapproachescanbecompared.ThroughexperimentontheIEEE-30-Bussystem,computationalresultsareanalyzedandcomparedtoprovideinsightforpowergridintentionalislanding. Thischapterisorganizedasfollows:Section 5.1 introducessomebackgroundsaboutpowergridislandingandsomeotherapplicationsofclusteringandgraphpartitioninginpowersystems.InSection 5.2 ,optimalpowerowmodelsarereviewedforbothACandDCcases.InSection 5.3 ,weconstructapowergridislandingmodelbasedontheDC-OPF,anddiscussitsvariationswithminimumislandsizeandinter-islandows.InSection 5.4 ,ourproposedmodelsareappliedtotheIEEE-30-BusSystemTestCaseandtheresultsareanalyzedandcomparedforthedifferentmodels.Section 5.5 summarizesourresults. 5.1Introduction RecentreportsoflargescaleblackoutsinNorthAmerica,Europeandothercountriesshowthatpowersystemsmaybesubjecttooutagesbecauseofvulnerableoperatingconditions,suchassystemlimits,weakconnections,unexpectedevents,hiddenfailures,andhumanerrors.Insomeotherareas,suchasmanyAfrican,SouthAsianandLatinAmericancountries,therollingblackout,orloadsheddingis 101

PAGE 102

astapleofdailylife.Besidesreasonsofinsufcientgenerationcapacityandinadequatetransmissioninfrastructurefordeliveringsufcientpower,securityandstabilityissuesarevitalissuesofmodernpowersystems[ 65 ].Toimprovethereliabilityofpowersystemsbysolvingsecurityandstabilityproblems,themethodofsplittingalargepowernetworkintosubsystemscanpresentawaybywhichtheremainingsystemscanoperateproperlyinanacceptablecondition,whenproblemsaffectsomeotherpartofthesystem.Thisprovidesawaytopreventlarge-scaleblackouts.Powergridislandingisoneapproachofsplittingthepowernetworkintoself-sufcientislandsforemergencycontrols. Ontheotherhand,withthedevelopmentoftechnologiesforsolarandwindenergyresources,moreandmoredistributedgeneratorsareaddedtothecentralizedgenerationsystem.Becauseofuncertainissuesinsolarorwindenergy,thereliabilityofapowersystembecomesevenmoreimportantthanever,especiallyincaseoffailures.Currentmethodsforprotectingthedecentralizedgenerationpowersystemiseithertodisconnectalldistributedgeneratorsonceafaultoccursortoimpletmentanislandingdetectionalgorithm([ 140 ])thatdetectsanislandingsituationandinitiatesdisconnectionofdistributedgenerators[ 107 ]. Therefore,powergridintentionalislandingisonepromisedmethodtoimprovethepowersystemreliabilityforitsmoreexiblecontrolproperty.ThenewIEEEStandard1547-2003[ 66 ]providesgeneralguidelinesfordesignandoperationofintentionalislandsincaseofpoweroutages. Thereareseveralapproachestoformislandsinapowergrid.In[ 85 ],LiuandLiupresentedareviewofthemainaspectsforpowersystemislanding,andtheyoutlinedislandingschemesaccordingtographpartitioning,minimalcutsetenumerationandgeneratorgrouping.Sincethetopologyofthepowergridcanbedenotedbyagraphincludingvertices(buses)connectedbyedges(transmissionlines),graphpartitioningisadirectlyusedmethodtopartitionthevertexsetsintoseveralsubsets 102

PAGE 103

[ 11 83 111 123 ].Minimalcutsetsareusedinpartitioninganetworkintosmallersubnetworks[ 111 128 ].Theself-healingstrategy[ 138 ]candealwithcatastrophiceventswhenapowersystemisapproachinganextremeemergencyconditionbydividingitintosmallislandswithconsiderationofquickrestoration.In[ 134 137 139 ],thepowergridislandingmethodsarebasedonslowcoherency,whichhasthecapacitytodeterminesetsofgeneratorgroupsamongweakconnections.Recentworkincontrolledislandingusedthedecisiontreemethod[ 112 ]andParetooptimization[ 126 ].Someotherpartitioningmethodsbasedonmatriceswereusedtopartitionthepowergrid,forexample,spectralmethodonpowerowJacobianmatrix[ 123 ],andspectralandk-meansmethodsonrealpowerowmatrix[ 99 ]. Loadsheddingisalwaystakentoreducetheimbalanceofgenerationandloaddemandafterislanding[ 36 85 138 140 ].Inordertobeself-sufcient,eachislandshouldbebalanced.Afterislanding,islandsmayhaveeitherexcessofgenerationorofload.Excessgenerationcanbecontrolledbyadjustingtheoutputofgenerators,whilesomeloadshouldbecurtailedasloadshedding. Mostoftheaboveapproachescanonlyformtwoislandsinalargenetwork,althoughtheprocedurescanberepeatedtoformanevennumberofislands.In[ 11 83 ],approachesforsimultaneouslyobtainingmorethantwoislandsareproposedbyusinggraphpartitioning.Asstatedabove,islandingapowergridissimilartothegraphpartitioningproblemwhichisaclassicproblemincombinatorialoptimizationandcanformmultiplepartitionsingeneral. Someoptimizationapproaches([ 111 123 126 128 ])topowergridislandingreliedonheuristicandapproximationmethods.Withoutanexplicitmathematicalprogrammingformulation,theeffectivenessofislandingcanonlybeanalyzedwithapproximationratiosorempirically.Althoughsomeoptimizationmethodsareusedforislanding[ 111 140 ]bymathematicalprogramming,theyjustobtainedtwoislands.Methodsin[ 140 ]lackedofconnectivityconstraints,whicharecrucialforthereliabilityofislands,and 103

PAGE 104

becauseofthenonlinearproperty,theycanonlyworkonsmallcases.Modelsin[ 111 ]weremoreofgraphpartitioninganddidnotconsidertheloadshedding. Inthischapter,weusetheDirectCurrentOptimalPowerFlow(DC-OPF)modelandgraphpartitioningapproachestoformallyformulatepowergridislandingproblemsasmixedintegerprograms(MIPs).Ourmodelcanformmultipleislandssimultaneouslyandcanbeusedtoformweaklyconnectedislandsamongwhichinter-islandowsareallowed.Weadaptethesetofmeasuresofinter-islandowsfrom[ 111 123 ]inourformulation.Additionally,connectivityconstraintsareaddedtoensurebuses,includinggeneratorsandloadbuses,withineachislandareconnected.AlthoughourinitialformulationsresultinnonlinearMIPs,weprovidetheprocedurestotransformthemintolinearMIPs.Thesedifferentmodelsarecomparedinthesimulationexperiments.Inter-islandowsareviewedasawaytoimprovepowergridperformancesincetheyprovidealessrestrictivesolutionthancompletepartitiondoes.However,oursimulationresultsshowthatinter-islandowsdonotalwayshelppowergridperformanceandinmanycases,completelydisconnectedislandsperformmuchbetterthanweaklyconnectedislands.Theislandingschemeisforintentionalandpre-plannedislandingforemergencycontrols.Weassumethatgeneratorswithinoneislandcanbesynchronouseasily. Differentfrom[ 140 ],weaddconnectivityconstraintsforeachisland.Withoutthiskindofconstraint,anislandmaybeformedbyseveralparts,andsomegenerationcannotbeusedotherpartsofthissubsystem.Ourweaklyconnectedislandingmodelsallowowsamongdistinctislands,andthisistherealreectionofnowadays'interconnectedsystems.Additionally,ourmodelcanobtainanynumberofislandsdirectly. Besidestheapplicationofgraphpartitioninginpowersystems,clusteringisalsousedinpowergrids.Forexample,in[ 13 ],clusteringisusedtondPJMzonesfor 104

PAGE 105

multipleplanningandoperationsanalysis,andthisisquiteusefulforreliabilityandresourceadequacyassessment,aswellasidenticationandmitigationofmarketpower. 5.2OptimalPowerFlowModels Inthissection,westatetheoptimalpowerowmodels.First,westatethenotation,parametersanddecisionvariables,usedthroughoutthischapter. 5.2.1Notations IndicesandIndexSets I:setofbuses (i,j)2L:undirectedtransmissionlines,i,j2Iandi
PAGE 106

Pij:powerowonline(i,j)2L PSi:loadshedatnodei2I i:phaseangleatnodei2I xik:f0,1gvariabletodecidetheislandbelongings,e.g.,xik=1denotesthatnodeiisinislandk yk:variableequaltothetotalnumberofbusesinsubnetworkk yij,k:continuousdecisionvariableforline(i,j)inislandktoensuretheconnectivityofsubnetworkk zij:0iftheendnodesiandjareindifferentislandsand1iftheendnodesareinthesameisland. 5.2.2PowerFlowModels Thealternatingcurrentoptimalpowerow(AC-OPF)problemdealswiththenetworkowproblemfortheACelectrictransmissiongridtooptimalitydispatchgenerationtoloadconsumerssubjecttonetworkowconstraintsandreliabilityconstraints.TheowofelectricenergyamongthepowergridfollowsKirchhoff'slaws.TheAC-OPFproblemincludesanonlinearobjectivefunctionandconstraintswithtrigonometricfunctions.LetQij(inMVARs)denotethereactivepowerowonline(i,j).ThepowerowfrombusitojisexpressedasPij+p )]TJ /F4 11.955 Tf 9.3 0 Td[(1Qij.LetVi,Vjdenotethevoltagemagnitudesatbusiandj.Letgij,bijdenotetheconductanceandthesusceptanceforline(i,j),respectively,wheregij=rij=(r2ij+x2ij),bij=)]TJ /F3 11.955 Tf 9.3 0 Td[(xij=(r2ij+x2ij)andrijistheresistanceandxijisthereactanceofline(i,j).Bythesenotations,PijandQijareexpressedasPij=V2igij)]TJ /F3 11.955 Tf 11.96 0 Td[(ViVj[gijcos(i)]TJ /F5 11.955 Tf 11.96 0 Td[(j)+bijsin(i)]TJ /F5 11.955 Tf 11.95 0 Td[(j)]Qij=)]TJ /F3 11.955 Tf 9.3 0 Td[(V2ibij)]TJ /F3 11.955 Tf 11.95 0 Td[(ViVj[gijsin(i)]TJ /F5 11.955 Tf 11.95 0 Td[(j))]TJ /F3 11.955 Tf 11.95 0 Td[(bijcos(i)]TJ /F5 11.955 Tf 11.96 0 Td[(j)] Thedirectcurrentoptimalpowerowproblem(DC-OPF)isanapproximationforanunderlyingAC-OPFproblemunderthefollowingsimplifyingrestrictionsandassumptions: 106

PAGE 107

A.1 Theresistanceforeachlineisnegligiblecomparedtothereactanceandcanbesetto0; A.2 ThevoltagemagnitudeateachbusissettoequaltothebasevoltageV0; A.3 Thevoltageangledifferencei)]TJ /F5 11.955 Tf 11.99 0 Td[(jalongeachline(i,j)issufcientlysmallsothatcos(i)]TJ /F5 11.955 Tf 11.96 0 Td[(j)1andsin(i)]TJ /F5 11.955 Tf 11.95 0 Td[(j)i)]TJ /F5 11.955 Tf 11.96 0 Td[(j. Bysomesimplicationsin[ 118 ],wecanobtainPij=Bij(i)]TJ /F5 11.955 Tf 12.54 0 Td[(j),Qij=0,whereBij=)]TJ /F3 11.955 Tf 9.3 0 Td[(bij.Usually,onebusisselectedasreferencebuswithspeciedvoltageangle,andwehaveonemoreassumption. A.4 Bus1isthereferencebuswith1=0. Fortheobjectivefunctiontominimizethegeneratingcost,itisusuallyexpressedasaquadraticfunctionofthegenerationoutputPGi.Additionally,tomaximizethesocialwelfare,weaddtheloadsheddingcostPi2ICSiPSitobeminimized.Otherconstraintsincludethemaximumpowerowoneachline;thepowerbalanceateachnode,wheretheserveddemandatbusiis(Di)]TJ /F3 11.955 Tf 13.12 0 Td[(si);themaximumgeneratingoutput;andthelimitationofloadsheddingbythemaximumload.Therefore,theDC-OPFproblemismodeledasthefollowinglinearprogrammingformulation.minPG,PS,Pij,Xi2I(CGiPGi+CSiPSi)s.t.Pij=Bij(i)]TJ /F5 11.955 Tf 11.96 0 Td[(j),8(i,j)2LPGi+XjiPij,8i2I)]TJ /F3 11.955 Tf 11.95 0 Td[(PijmaxPijPijmax,8(i,j)2L0PGiPGimax,8i2I0PSiPDi,8i2I Manymethodsareusedfortheoptimalpowerowmodel,forexample,quadraticprogramming,interiorpointmethodandsomeheuristicmethods.Thesemethodsarereviewedby[ 64 ]andrecentlyby[ 95 ]. 107

PAGE 108

5.3PowerGridIslandingModels 5.3.1ModelforCompleteIslanding TheDC-OPFmodelweuseissimilartotheformulationin[ 108 130 ].Basedonthismodel,inthefollowing,wepresentamathematicalprogramforpowergridislanding(GI-DC-OPF). Mostcurrentresearchonpowergridislanding[ 111 123 128 ]dealwitha2-islandingproblembyusingthes)]TJ /F3 11.955 Tf 10.72 0 Td[(tgraphpartitioningapproach.Inthes)]TJ /F3 11.955 Tf 10.73 0 Td[(tapproach,apowernetworkN(I,L)ispartitionedintotwosubgraphsinducedbythenodesetsI1andI2,s2I1,t2I2,andbothsubgraphsareconnected.Inthischapter,weconsiderageneralK-islandingproblem.Letfi1,i2,,iKgIbeasetofKrootbuses.TheK-islandingproblemistoseparateapowergridintoKcomponentsI1,I2,,IKsuchthat[kIk=I,Ik\Ik0=;fork6=k0,ik2Ik,andeachcomponent,aninducedgraphbyIk,isconnected. Tokeeptheformulationlesscumbersome,weintroducetwoindicatorfunctions.Theindicatorfunctionsareusedtoshowwhichbushasgenerationcapacityorload.Let1gi=1fori2IifGi>0,and0otherwise,andlet1di=1fori2Iifdi>0,and0otherwise.ToformKislands,wesetfi1,i2,,iKgItobetherootnodeineachisland,i.e.,settingxik,k=1.Usually,wespecifyageneratorforeachisland. TheGI-DC-OPFmodeltondKislandsisformulatedasfollows:GI)]TJ /F23 11.955 Tf 11.95 0 Td[(DC)]TJ /F23 11.955 Tf 11.95 0 Td[(OPF:minPGi,PSi,Pij,,x,y,zXi2ICSiPSi (5) 108

PAGE 109

s.t.8>>>>>>>>>>>>>><>>>>>>>>>>>>>>:Pij=Bij(i)]TJ /F5 11.955 Tf 11.96 0 Td[(j)zij,8(i,j)2L)]TJ /F3 11.955 Tf 9.3 0 Td[(PijmaxPijPijmax,8(i,j)2LPGi+PjiPij,8i2I0PGiPGimax,8i2I0PSiPDi,8i2I (5)8>>>>>><>>>>>>:PKk=1xik=1,8i2VPi2I1gixik1,Pi2I1dixik1,8kzij=Pkxikxjk,8(i,j)2L (5)8>>>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>>>:xik,k=1,8kyk=Pi2Ixik,8kPjyikj,k=yk)]TJ /F4 11.955 Tf 11.96 0 Td[(1,8kP(i,j)2L0yij,k+xik=P(j,i)2L0yji,k,8k,8i2I,i6=ik0yij,kykxik,8(i,j)2L0,8k0yij,kykxjk,8(i,j)2L0,8k (5)xik2f0,1g,8i2I,8k (5) Theconstraints( 5 )areforDC-OPF,whichinclude(inorder):approximateactivepowerowontransmissionlines;themaximumpowerowoneachline;thepowerbalanceateachnode,wheretheserveddemandatbusiis(PDi)]TJ /F3 11.955 Tf 12.89 0 Td[(PSi);themaximumgeneratingoutput;andthelimitationofloadsheddingbythemaximumload.Constraints( 5 )areslightlydifferentfromthestandardDC-OPFbytheadditionofzij.InthestandardDC-OPF,Pij=0impliesthephaseanglesatendbuseshavetoequal.In( 5 ),ifzij=1,theconstraintisthesameasinstandardDC-OPFsinceline(i,j)isinsideofanisland.Ifzij=0,theconstraintbecomesPij=0sinceline(i,j)is 109

PAGE 110

betweentwoislandsandisremoved.Becausevariablezijrelaxesphaseanglesfromtheconstraint,thephaseangleiandjcantakeanyvaluewhenPij=0. Theconstraints( 5 )relatetographpartitioning,whichinclude:everynodemustbelongtoexactlyoneisland;andeveryislandmusthaveatleastonegeneratorandoneconsumer.Thegraphpartitioningapproachhasbeenstudiedforalongtime,buttheadditionalconstraintsguaranteeingatleastonegeneratorandloadineachislandmakeourgraphpartitiondifferentfromtheclassicalmodels.Recentlyin[ 42 ],theconstraintsforeachsubset(islandinourcase)hasbeenshowntorequireloosecardinalityrequirementsinsteadofequalityconstraints.Also,iftwobusesiandjareinthesameisland,thereexistsexactlyonek0(1k0K)suchthatxik0=xjk0=1,xik=xjk=0forallotherk's,andthuszij=1.Otherwise,zij=0. Theconstraints( 5 )aretoensurethateveryislandisconnected.Thatis,allbuseswithinoneislandareconnectedbytransmissionlineswithbothendsinthisisland.In[ 32 ],threewaysofenforcingthesubnetworkconnectivityaresummarizedviamathematicalprogrammingformulations.Here,themethodcalledsinglecommodityowisused.Inthissinglecommodityowproblem,thetotalnumberofnodesincludingikinsubnetworkkisyk,andyk)]TJ /F4 11.955 Tf 12.49 0 Td[(1unitsofowaresuppliedatnodeik.Everyothernodeactsasasinkbyconsumingoneunitofowandeverynodeshouldhaveapositiveincomingow.Theowoutfromikcannotbedistributedtoanyothersubnetwork.Theconstraints( 5 )areforthissinglecommodityowproblem.Ifallconstraintscanbesatised,everysubnetworkshouldbeconnected. Theobjective( 5 )ofGI-DC-OPFmodelistominimizeloadsheddingcost.Infact,bythisobjectivefunction,wecanminimizetheimbalanceofgenerationandloadineachisland.ThetotalgenerationofislandkisPi2IPGixik,whichisequaltothetotalsatisedloaddemandPi2I(PDi)]TJ /F3 11.955 Tf 12.09 0 Td[(PSi)xik.Theunsatisedloaddemandorloadsheddingneeded 110

PAGE 111

forislandkisPi2IPSixik,andsummingoverallislands,wehaveKXk=1Xi2IPSixik=Xi2IPSiKXk=1xik=Xi2IPSi,wherePKk=1xik=1isfrom( 5 ).Withthepenaltycostcoefcients,wehavetheobjectivefunction( 5 ). 5.3.2IslandingwithMinimumSize Thecompleteislandingmodel( 5 )-( 5 )partitionsapowergridintoKislands.Itispossiblethatthepartitionconsistsofsomesmallislands,e.g.,anislandofonegeneratorandoneloadbus.Anextremelysmallislandmayprovidelittlevalueinpracticeandmaybeconsideredasill-conditionedinsomerealapplications.OnewaytocontrolthesizeofislandsistoimposealowerboundMinsize.Wecanaddthefollowingconstraint Xi2IxikMinSize,(5) whichcontrolsthenumberoftotalbusesineachislandandcalltheislandingmodelwithlowerboundasGI-M-DC-OPF. 5.3.3ModelsforWeaklyConnectedIslanding InGI-DC-OPF,islandsarecompletelyseparated.Inthecaseofweaklyconnectedislands[ 111 ],inter-islandowscanbemodeledbyaddingtheconstraintsC(k,k0)",8k,k0,k6=k0, (5) where"(>0)istheamountoftheowallowedbetweendifferentislands.Theislandingwithweakconnectioncanbeformulatedbyaddingconstraint( 5 )andremovingvariableszijsandtheassociatedconstraintsfromGI-DC-OPF.TheresultingweaklyconnectedislandingmodelisGI-W-DC-OPF. ThetermC(k,k0)describesthepowercrossingbetweenislandkandk0andcanbemeasuredindifferentways.In[ 111 ],threemethodsareusedtomeasurepowerline 111

PAGE 112

capacitiesbetweenislands.Inthischapter,theactualpowerowbetweentwoislandsisusedtomeasuretheinter-islandpowerows.AssumeFkk0isthepositivepowerowfromislandktok0,andFk0kisthetotalpositivepowerowonlinesfromk0tok.Thethreemethodsformeasuringinter-islandowsC(k,k0)are I. C1(k,k0)=Fkk0+Fk0k II. C2(k,k0)=jFkk0)]TJ /F3 11.955 Tf 11.96 0 Td[(Fk0kj III. C3(k,k0)=maxfFkk0,Fk0,kg. Amongthesethreemeasures,C1(k,k0)isthemostconservativemeasure.Givenanyfeasiblepartition,therelationisC1(k,k0)C3(k,k0)C2(k,k0). ThemeasureI.andII.areadoptedfrom[ 111 ]byusingactualpowerowinsteadofthelinecapacity.Inthischapter,wefocusonC1(k,k0)andC2(k,k0)andaddthemtoGI-DC-OPFforweaklyconnectedislanding.ThemeasureC3(k,k0)countsmaximumdirectedow,whichcanbeusedtorestrictangledifferencesbetweentwoislands[ 40 ]. MeasureI.countstheabsoluteamountofowbetweentwoislandsinbothdirections,andwecanexplicitlyexpressconstraints( 5 )as8>>>>>><>>>>>>:C1(k,k0)=P(i,j)2Lij,kk0"ij,kk0Pij(xikxjk0+xik0xjk),8(i,j)2L,k6=k0ij,kk0)]TJ /F3 11.955 Tf 21.92 0 Td[(Pij(xikxjk0+xik0xjk),8(i,j)2L,k6=k0. (5) Variableij,kk0isusedtocounttheabsoluteamountofpowerowonline(i,j)crossingislandkandk0.For(i,j)betweenislandkandk0,effectively,ij,kk0jPijj.Effectively,P(i,j)2Lij,kk0"impliesP(i,j)2LjPijj",andP(i,j)2LjPijj"impliesthereexistsafeasibleassignmentofij,kk0suchthatP(i,j)2Lij,kk0".Therefore,constraints( 5 )isequivalenttotheconstraintsC1(k,k0)". 112

PAGE 113

InmeasureII.,thedifferenceinpowerowbetweentwodirectionsisusedtomeasurethepowertransferredbetweentwoislands.TheconstraintsforthismethodareC2(k,k0)=jX(i,j)2LPij(xikxjk0+xik0xjk)j" andequivalentto )]TJ /F5 11.955 Tf 11.96 0 Td[("X(i,j)2LPij(xikxjk0+xik0xjk)"(5) forpairsk,k0andk6=k0. 5.3.4ModelPreprocessing TheislandingmodelsdiscussedinSection 5.3.1 aremixedintegernonlinearprograms.Inthissection,weshowthatthesemodelscanbetransformedtolinearMIPswhichhaveawiderangeofsolutiontechniques. IntheformulationformodelGI-DC-OPF,therearethefollowingquadraticterms:xikxjk,ykxik,ykxjk Besidesthesenonlinearterms,themodelGI-W-DC-OPFhasadditionalnonlinearterms:xik0xjk,Pijxikxjk0,Pijxik0xjk Tolinearizetheseterms,wecandeneauxiliaryvariables,forexample,wijkk0=xikxjk0,uki=ykxikandvkj=ykxjk.Thevariablewijkk0istheproductoftwobinaryones,andanothertwouki,vkjaretheproductofabinaryvariableandacontinuousone.Theycanbelinearizedbythefollowing:8>>>>>>>>>><>>>>>>>>>>:wijkk0xikwijkk0xjk0wijkk0xik+xjk0)]TJ /F4 11.955 Tf 11.95 0 Td[(1wijkk00 (5) 113

PAGE 114

8>>>>>>>>>><>>>>>>>>>>:ukiyk)]TJ /F3 11.955 Tf 11.95 0 Td[(U(1)]TJ /F3 11.955 Tf 11.96 0 Td[(xik)ukiVxikukiyk)]TJ /F3 11.955 Tf 11.95 0 Td[(V(1)]TJ /F3 11.955 Tf 11.95 0 Td[(xik)ukiUxik (5) 8>>>>>>>>>><>>>>>>>>>>:vkjyk)]TJ /F3 11.955 Tf 11.95 0 Td[(U(1)]TJ /F3 11.955 Tf 11.95 0 Td[(xjk)vkjVxjkvkjyk)]TJ /F3 11.955 Tf 11.95 0 Td[(V(1)]TJ /F3 11.955 Tf 11.96 0 Td[(xjk)vkjUxjk (5) whereVykUandV,Ucanbechosenassufcientsmall,sufcientlargenumbers,respectively.Forexample,wecansetV=1andU=jIj. Thetermsxikxjk,xik0xjkcanbelinearizedanalogouslytoxikxjk0.Afterthelinearizationofxikxjk0bythevariablewijkk0,thecubictermPijxikxjk0becomesPijwijkk0whichitselfcanbelinearizedinasimilarwayvianewvariableqijkk0by8>>>>>>>>>><>>>>>>>>>>:qijkk0Pij)]TJ /F3 11.955 Tf 11.95 0 Td[(Pijmax(1)]TJ /F3 11.955 Tf 11.96 0 Td[(wijkk0)qijkk0)]TJ /F3 11.955 Tf 21.92 0 Td[(Pijmaxwijkk0qijkk0Pij+Pijmax(1)]TJ /F3 11.955 Tf 11.96 0 Td[(wijkk0)qijkk0Pijmaxwijkk0. (5) Similarly,wecanlinearizePijxik0xjk.Withtheabovelinearizations,theislandingproblemscannowbeformulatedasintegerlinearprograms.Averietyofcommercialsoftware,suchasCPLEX,cansolvesuchproblemsefciently.Inaddition,algorithmsforlargescaleoptimization,suchasbranch-and-cutorBenders'decompositioncanalsobeusedtosolveMIPs. 114

PAGE 115

Figure5-1. IEEE-30-Busnetwork 5.4NumericalExperiments TheMIPformulationfortheislandingproblemswereimplementedinC++andusingCPLEX11.0viaILOGConcertTechnology2.5,andallcomputationswereperformedonaSUNUltraSpace-IIIwitha900MHzprocessorand2.0GBRAM.Thegapwassettobe0.01inCPLEX. WeusetheIEEE-30-BusTestCase[ 102 ](seeFig. 5-1 )astheunderlyingpowergridinthesimulation.Thisnetworkhas30busesand41transmissionlines.ThedataforourGI-DC-OPF,GI-M-DC-OPF,andGI-W-DC-OPFmodelsareshowninTable 5-1 andTable 5-2 115

PAGE 116

Table5-1. GenerationandloaddataofIEEE-30-Busnetwork BusNo.LoadGenerationSheddingCostDiGiri 101502025032.40347.603501506000722.803801509000105.8031103001211.203130300146.203158.203163.50317903183.203199.503202.2032117.50322000233.203248.70325000263.5032700028000292.4033010.603 Note:Inthistable,ifthegenerationcapacityGiforsomebusispositive,thisbusisageneratingbus;similarly,ifDi>0,thisbusisaloadbus. BythecompleteislandingmodelGI-DC-OPFwelisttheresultsinTable 5-3 fordifferentcases.InthecaseofK=1,weconsiderthewholesystemasoneisland.Sincethereareatmost6generators,wecanobtainatmost6islandsinthissystem.InFig. 5-2 ,therelationshipbetweenloadsheddingcostandthenumberKofislandsispresented.Fromthisgure,wecanseethatasthenumberofislandsinthesame 116

PAGE 117

Table5-2. TransmissionlinedataofIEEE-30-Busnetwork FromBusToBusCapacitySusceptance 1213015.65131305.63241655.203413023.53251304.77261655.124619022.31571707.456713011.036813022.01691654.816101301.809111654.819101659.094121653.9112131657.1412141303.1712151306.1012161304.1014151602.2516171604.8415181603.6918191606.22192013011.7610201303.99101713010.32102113010.9810221305.40212213034.1315231603.9822241603.9523241602.9924251602.2925261601.8225271603.7628271652.5327291601.8827301601.2929301601.728281604.5462813015.46 117

PAGE 118

Figure5-2. Loadsheddingcostvs.thenumberKofislands systemincreases,thetotalloadsheddingcostincreasesaswell.FromTable 5-3 ,mostobtainedislandshavehighsatiseddemandrates. BasedontheresultsinTable 5-3 ,weshowthreecasesofcompletelydividedislandsforK=2,3,4inFig. 5-3 ,Fig. 5-4 ,andFig. 5-5 .Asshowninthesegures,eachislandisconnectedandincludesatleastonegenerator. AsdiscussedinSection 5.3.2 ,tocontrolthesizeofislands,thecontraint( 5 )isaddedtocompleteislandingmodeltoformthemodelGI-M-DC-OPF.Here,usingsameinputparametersasTable 5-3 ,therelationship,obtainedbythemodelGI-M-DC-OPF,betweenloadsheddingcostandMinSizeforK=2andK=3ispresentedinFig. 5-6 .ForK=2,themaximumMinSizeis15,andfork=3,themaximumMinSizeis10.AsshowninFig. 5-6 ,whenMinSizeissmall(eg.,MinSize7),bothcostsarethesame.WhenMinSizeincreases,theloadsheddingcostincreasesaswellbutmoresignicantlyforK=3. Next,wetestedthemodelGI-W-DC-OPFforislandingincaseofweakconnectionwithouttheMinSizeconstraint.Amongislands,asdiscussedinSection 5.3.3 ,limitedowsareallowed.Inthefollowing,wearegoingtotestthemodelGI-W-DC-OPFincase 118

PAGE 119

(K=2,RootBuses1,13) Figure5-3. IEEE-30-Busnetworkwith2islands (K=3,RootBuses1,8,13) Figure5-4. IEEE-30-Busnetworkwith3islands ofthemostconservativemeasureC1(k,k0)andthemostloosemeasureC2(k,k0)forweaklyconnectedislanding.InFig. 5-7 ,therelationshipbetweenloadsheddingcostand"undertwomeasuresC1(k,k0)andC2(k,k0)ispresentedforthecaseofK=4withrootbuses1,8,11,13,and4xedislandsasshowninFig. 5-5 .Additionally,the 119

PAGE 120

(K=4,RootBuses1,8,11,13) Figure5-5. IEEE-30-Busnetworkwith4islands Figure5-6. Loadsheddingcostvs.MinSize costforcompleteislandingispresentasadashedline.Fromthisgure,wecanseeloadsheddingcostsforbothcutsaremonotonicallydecreasingas"increases.When"islargeenoughtoallowlargeowsamongislands,costsareaconstantforbothcuts.ThecostunderC1(k,k0)ishigherthanC2(k,k0),sincehecutC2(k,k0)islooserthanC1(k,k0)asdiscussedin 5.3.3 120

PAGE 121

(K=4,RootBuses1,8,11,13) Figure5-7. Loadsheddingcostvs." Inaddition,fromFig. 5-7 ,when"<20,thecostunderbothcutsislargerthanthecostofcompleteislanding.Thisshowsthatinter-islandowsdonotalwayshelppowergridperformanceandinmanycases,completelydisconnectedislandsperformmuchbetterthanweaklyconnectedislands.IncompleteislandingmodelGI-DC-OPF,thereisaconstraintPij=Bij(i)]TJ /F5 11.955 Tf 12.28 0 Td[(j)forline(i,j).Whenthislineischosenintothecut,zij=0andthereisnorequirementforthedifferencei)]TJ /F5 11.955 Tf 12.14 0 Td[(j.However,intheweeklyconnectedislandingmodelGI-W-DC-OPF,theconstraintforline(i,j)becomesPij=Bij(i)]TJ /F5 11.955 Tf 12.37 0 Td[(j).Whenthislineischosenintothecut,onlylimitedowisallowedonthisline;when"becomescloserto0,thisforcesPij=0andi=j,whichwillinuenceotherpartofthesystemandincreasetheloadsheddingcost. 5.5Discussion Inthischapter,weusemixedintegerprogramstoformulatethepowergridislandingproblem.Ourmodelcanformmultipleislandssimultaneouslyandcanbeusedtoformweaklyconnectedislands.TosolvethenonlinearMIPs,wepresenttechniquestotransformthemintolinearMIPs.Ourmodelsproposeaschemetoformself-sufcient 121

PAGE 122

subnetworksinalarge-scalepowersystemandalsotominimizetheloadshedding.Futureresearchincludesfollowingdirectionsby: changingthebasedmodelfromDC-OPFtoAC-OPF. addingsecurityandstabilityconstraintstoourcurrentmodel,forexample,linepowerowbyN)]TJ /F4 11.955 Tf 12.52 0 Td[(1,N)]TJ /F3 11.955 Tf 12.52 0 Td[(kcontingencyanalysis,currentthermallimit,andbusvoltagestabilitylimittoensurethesystemworksnormally. addingthegenerationandloaddemandbalanceconstraints.Foreachisland,thereisaratioonloadsheddingtopreventblackouts. addingphysicallocationconstraints.Forexample,physicallyclosebusesshouldbedividedintooneislandtoreducetransmissioncost. 122

PAGE 123

Table5-3. Islandingresultsbycompleteislandingmodel KRootBusesObj.IslandsRealGenerationGen.CapacityLoadDemandSat.Demand 1122.5Island1:1-30130.0130.0137.594.5% 21,1322.5Island1:1,2,5-8,15,21-3070.070.076.991.0%Island2:3,4,9-14,16-2060.060.060.699.0% 31,8,1322.5Island1:1,2,5-7,9-11,14,15,18-2485.085.087.397.4%Island2:8,25-3015.015.016.590.9%Island3:3,4,12,13,16,1730.030.033.789.0% 41,8,11,1337.5Island1:1,3,410.015.010.0100.0%Island2:2,5-8,21,22,24-3055.055.065.584.0%Island3:9-11,16,17,19,2030.030.030.0100.0%Island4:12-15,18,2330.030.032.093.8% 51,5,8,11,1337.5Island1:1,3,410.015.010.0100.0%Island2:5,715.015.022.865.8%Island3:8,25-3015.015.016.590.9%Island4:2,6,9-11,17,19-2455.055.055.998.4%Island5:12-16,1830.030.032.392.9% 61,2,5,8,11,13112.5Island1:1,32.415.02.4100.0%Island2:2,47.625.07.6100.0%Island3:5,715.015.022.865.8%Island4:8,25-3015.015.016.590.9%Island5:6,9-11,21-2430.030.035.285.2%Island6:12-2030.030.053.056.6% Note:Inthistable,noowisallowedamongdifferentislands.ThevalueKisthenumberofislandstoform.TheObj.denotestheobjectivecostofthemodelGI-DC-OPFbasedonthedatainTable 5-1 andTable 5-2 .TheRootBusesdenotesthepickedrootbusforeachisland.RealGenerationisthetotalgenerationateachisland.Gen.CapacityandLoadDemandarethetotalgenerationca-pacity,totalloaddemandforeachdividedisland.TheSat.Demanddenotesthesatisedrate,whichistherateofsatiseddemanddividedbytotalloaddemandforeacheachisland. 123

PAGE 124

CHAPTER6CONCLUSIONS Inthedissertation,wemainlystudiedcombinatorialandnonlinearoptimizationmethodsandtheirapplicationsindataclustering,biclusteringandpowersystemsforislanding.Themathematicalprogrammingapproachweusedinthedissertationincludesintegerprogramming,linearprogramming,semideniteprogramming,quadraticallyconstrainedprogramming,stochasticprogrammingandetc.Spectral,interiorpoint,decompositionandlinearizationmethodsareallusedforsolvingtheseformulations. Inthefuture,moredecompositionmethods,forexample,BendersdecompositionmethodandL-shapedmethod,canbeusedforsolvingtherobustandstochasticoptimizationmodelsmentionedinthedissertation.Thesemideniteprogrammingmethodcanbecombinedtothesedecompositionapproachessothatmoreefcientalgorithmscanbeconstructed.Intheapplicationofpowersystemsislanding,ourproposedmodelisbasedonDCoptimalpowerowmodel.TheislandingmodelbasedonACoptimalpowerowbyaddingmoresecurityandstabilityconstraints,canbeconstructedasamixedintegernonlinearprogram.Thisrequiresmoreefcientalgorithmsforsolvingproblemsarisingfromlargepowersystems. 124

PAGE 125

REFERENCES [1] Alpert,C.J.,Kahng,A.B.:Recentdirectionsinnetlistpartitioning:asurvey.Integr.VLSIJ.19(1),1(1995) [2] Angiulli,F.,Cesario,E.,Pizzuti,C.:Randomwalkbiclusteringformicroarraydata.Inf.Sci.:Int.J.178(6),1479(2008) [3] Armbruster,M.,Fugenschuh,M.,Helmberg,C.,Martin,A.:Acomparativestudyoflinearandsemidenitebranch-and-cutmethodsforsolvingtheminimumgraphbisectionproblem.In:Lodi,A.,Panconesi,A.,Rinaldi,G.(eds.)Proceedingsofthe13thInternationalIntegerProgrammingandCombinatorialOptimizationConference,pp.112.LectureNotesinComputerScience5035,Springer(2008) [4] Barkow,S.,etal.:BicAT:Abiclusteringanalysistoolbox.Bioinformatics22,1282(2006) [5] Ben-Dor,A.,Chor,B.,Karp,R.,Yakhini,Z.:Discoveringlocalstructureingeneexpressiondata:Theorder-preservingsubmatrixproblem.J.Comput.Biol.10,373(2003) [6] Benders,J.F.:Partitioningproceduresforsolvingmixed-variablesprogrammingproblems.NumerischeMathematik4,238(1962) [7] Berkhin,P.:SurveyOfclusteringdataminingtechniques.TechnicalReport.AccrueSoftware(2002) [8] Bertsimas,D.,Sim,M.:Robustdiscreteoptimizationandnetworkows.Math.Program.Ser.B98,49(2003) [9] Bertsimas,D.,Sim,M.:Thepriceofrobustness.INFORMSOper.Res.52(1),35(2004) [10] Bezdek,J.C.:Patternrecognitionwithfuzzyobjectivefunctionalgorithms.Plenum,NewYork(1981) [11] Bi,T.,Ni,Y.,Shen,C.M.,Wu,F.F.:Efcientmultiwaygraphpartitioningmethodforfaultsectionestimationinlarge-scalepowernetworks.IEEProc.-Gener.Trunsm.Distrib.149(3),289(2002) [12] Birge,J.R.,Louveaux,F.:IntroductiontoStochasticProgramming,SpringerVerlag,NewYork(1997) [13] Blumsack,S.:Partitioningofelectricalnetworks.http://cnls.lanl.gov/chertkov/SmarterGrids/seminars.html(2009).Accessed6June2011 125

PAGE 126

[14] Boulle,M.:Compactmathematicalformulationforgraphpartitioning.Optim.Eng.5,315(2004) [15] Burer,S.,Monteiro,R.D.C.,Zhang,Y.:Rank-tworelaxationheuristicsforMAX-CUTandotherbinaryquadraticprograms.SIAMJ.Optim.12,503(2001) [16] Busygin,S.,Boyko,N.,Pardalos,P.,Bewernitz,M.,Ghacibehc,G.:BiclusteringEEGdatafromepilepticpatientstreatedwithvagusnervestimulation.In:AIPConferenceProceedingsoftheDataMining,SystemsAnalysisandOptimizationinBiomedicine,pp.220(2007) [17] Busygin,S.,Prokopyev,O.A.,Pardalos,P.M.:Featureselectionforconsistentbiclusteringviafractional0programming.J.Comb.Optim.10(1),7-21(2005) [18] Busygin,S.,Prokopyev,O.A.,Pardalos,P.M.:Biclusteringindatamining.Comput.Oper.Res.35,2964(2008) [19] Califano,A.,Stolovitzky,G.,Tu,Y.:Analysisofgeneexpressionmicroaraysforphenotypeclassication.In:ProceedingsofInternationalConferenceonComputationalMolecularBiology,75(2000) [20] Carmona-Saez,P.,Pascual-Marqui,R.D.,Tirado,F.,Carazo,J.M.,Pascual-Montano,A.:Biclusteringofgeneexpressiondatabynon-smoothnon-negativematrixfactorization.BMCBioinformatics7,78(2006) [21] Chan,P.K.,Schlag,M.,Zien,J.Y.:Spectralk-wayratio-cutpartitioningandclustering.IEEETrans.Comput-AidedDes.Integr.CircuitsSyst.13,1088(1994) [22] Chaovalitwongse,W.A.,Pardalos,P.M.,Prokopyev,O.A.:Anewlinearizationtechniqueformulti-quadratic0programmingproblems.Oper.Res.Lett.32,517(2004) [23] Chardaire,P.,Sutter,A.:Adecompositionmethodforquadraticzero-oneprogramming.Manag.Sci.41(4),704(1995) [24] Cheng,K.O.,etal.:Bivisu:Softwaretoolforbiclusterdetectionandvisualization.Bioinformatics23,2342(2007) [25] Cheng,Y.,Church,G.M.:Biclusteringofexpressiondata.In:Proceedingsofthe8thInternationalConferenceonIntelligentSystemsforMolecularBiology,pp.93(2000) [26] Cho,H.,Dhillon,I.S.:Coclusteringofhumancancermicroarraysusingminimumsum-squaredresiduecoclustering.IEEE/ACMTrans.Comput.Biol.Bioinform.5(3),385(2008) 126

PAGE 127

[27] Chung,F.R.K.:Spectralgraphtheory.In:Regionalconferenceseriesinmathematicsbyconferenceboardofthemathematicalsciences.AmericanMathematicalSociety,Providence(1997) [28] ILOGCPLEX11.0UsersManual,(2007) [29] DataTransformationSteps.http://www.dmg.org/v2-0/Transformations.html(2008).Accessed6June2011 [30] Dhillon,I.S.:Co-clusteringdocumentsandwordsusingbipartitespectralgraphpartitioning.In:Proceedingsofthe7thACMSIGKDDInternationalConferenceonKnowledgingDiscoveryandDataMining(KDD),pp.26(2001) [31] Dhillon,I.S.,Mallela,S.,Modha,D.S.:Informationtheoreticco-clustering.In:ProceedingsoftheNinthACMSIGKDDInternationalConferenceonKnowledgeDiscoveryandDataMining,pp.89(2003) [32] Dilkina,B.,Gomes,C.P.:Solvingconnectedsubgraphproblemsinwildlifeconservation.In:CPAIOR-10:7thInternationalConferenceonIntegrationofAIandORTechniquesinConstraintProgrammingforCombinatorialOptimizationProblems,Bologna,Italy(2010) [33] DiMaggio,P.A.,McAllister,S.R.,Floudas,C.A.,Feng,X.J.,Rabinowitz,J.D.,Rabitz,H.A.:Biclusteringviaoptimalre-orderingofdatamatricesinsystemsbiology:Rigorousmethodsandcomparativestudies.BMCBioinformatics9,458(2008) [34] Ding,C.:ATutorialonSpectralClustering.http://ranger.uta.edu/chqding/Spectral(2004).Accessed6June2011 [35] Ding,C.,He,X.,Zha,H.,Gu,M.,Simon,H.:Amin-maxcutalgorithmforgraphpartitioninganddataclustering.In:Proc.IEEEInt'lConf.DataMining,(2001) [36] Du,P.,Nelson,J.K.:Two-stepsolutiontooptimalloadsheddinginamicro-grid.IEEE/PESPowerSyst.Conf.andExpo.(2009) [37] Elsner,U.:GraphpartitioningAsurvey.TechnischeUniversitatChemnitz,SFB393-Preprint97(1997) [38] Fan,N.,Boyko,N.,Pardalos,P.M.:Recentadvancesofdatabiclusteringwithapplicationincomputationalneuroscience.In:Chaovalitwongse,W.A.,Pardalos,P.M.,Xanthopoulos,P.(eds.)ComputationalNeuroscience,pp.105.Springer,Berlin,(2010) [39] Fan,N.,Chinchuluun,A.,Pardalos,P.M.:Integerprogrammingofbiclusteringbasedongraphmodels.In:Chinchuluun,A.,Pardalos,P.M.,Enkhbat,R.,Tseveendorj,I.(eds.)OptimizationandOptimalControl:TheoryandApplications,pp.479.Springer,Berlin(2010) 127

PAGE 128

[40] Fan,N.,Pan,F.:Locatingphasormeasurementsanddetectingcutsetanglesinpowersystems.In:IEEEPESInnovativeSmartGridTechnologies(ISGT2011)Conference,Anaheim,CA(2011) [41] Fan,N.,Pardalos,P.M.,Chinchuluun,A.:Graphpartitioningapproachesforanalyzingbiologicalnetworks.In:R.P.Mondaini(eds.)BIOMAT2009-InternationalSymposiumonMathematicalandComputationalBiology,pp.250.WorldScientic,Singapore(2009) [42] Fan,N.,Pardalos,P.M.:Linearandquadraticprogrammingapproachesforthegeneralgraphpartitioningproblem.J.Glob.Optim.48(1),57(2010) [43] Fan,N.,Pardalos,P.M.:Multi-wayclusteringandbiclusteringbytheRatiocutandNormalizedcutingraphs.J.Comb.Optim.(2010).doi:10.1007/s10878-010-9351-5 [44] Fan,N.,Pardalos,P.M.:Robustoptimizationofgraphpartitioningandcriticalnodedetectioninanalyzingnetworks.In:Wu,W.,Daescu,O.(eds.)COCOA2010,PartI,pp.170.LectureNotesinComputerScienceVol.6508(2010) [45] Fan,N.,Zheng,Q.P.,Pardalos,P.M.:Onthetwo-stagestochasticgraphpartitioningproblem.In:Wang,W.,Zhu,X.,Du,D.-Z.(eds.)COCOA2011,pp.500,LectureNotesinComputerScienceVol.6831(2011) [46] Falkner,J.,Rendl,F.,Wolkowiz,H.:Acomputationalstudyofgraphpartitioning.Math.Program.66,211(1994) [47] Fiedler,M.:Apropertyofeigenvectorsofnon-negativesymmetricmatricesanditsapplicationtographtheory.Czech.Math.J.25,619(1975) [48] Fjallstrom,P.-O.:Algorithmsforgraphpartitioning:asurvey.Linkop.Elec.ArticlesComput.Inf.Sci.3,10(1998) [49] Frieze,A.,Flaxman,A.,Krivelevich,M.:Ontherandom2-stageminimumspanningtree.RandomStructuresandAlgorithms28,24(2006) [50] Frieze,A.,Jerrum,M.:Improvedapproximationalgorithmsformaxk-cutandmaxbisection.Algorithmica18,67(1997) [51] Garey,M.R.,Johnson,D.S.,Stockmeyer,L.:SomesimpliedNP-completegraphproblems.Theor.Comput.Sci.1,237(1976) [52] Grant,M.,Boyd,S.:Graphimplementationsfornonsmoothconvexprograms.In:Blondel,V.,Boyd,S.,Kimura,H.(eds.)RecentAdvancesinLearningandControl(atributetoM.Vidyasagar),pp.95.LectureNotesinControlandInformationSciences.Springer,Berlin(2008) 128

PAGE 129

[53] Grant,M.,Boyd,S.:CVX:Matlabsoftwarefordisciplinedconvexprogramming(webpageandsoftware).http://stanford.edu/boyd/cvx(2009).Accessed6June2011 [54] Gu,J.,Liu,J.S.:Bayesianbiclusteringofgeneexpressiondata.BMCGenom.9(Suppl1),S4(2008) [55] Gu,M.,Zha,H.,Ding,C.,He,X.,Simon,H.:Spectralrelaxationmodelsandstructureanalysisfork-waygraphClusteringandbi-clustering.PennStateUnivTechReportCSE-01-007(2001) [56] Hagen,L.,Kahng,A.B.:Newspectralmethodsforratiocutpartitioningandclustering.IEEETrans.onComputer-AidedDes.11(9),1074(1992) [57] Hager,W.,Krylyuk,Y.:Graphpartitioningandcontinuousquadraticprogramming.SIAMJ.Discret.Math.12(4),500(1999) [58] Hager,W.,Krylyuk,Y.:Multisetgraphpartitioning.Math.Meth.Oper.Res.55,1(2002) [59] Hager,W.,Phan,D.,Zhang,H.:Anexactalgorithmforgraphpartitioning.arXiv:0912.1664v1(2009) [60] Hartigan,J.A.:Directclusteringofadatamatrix.J.Am.Stat.Assoc.67,123(1972) [61] Hendrickson,B.,Kolda,T.G.:Graphpartitioningmodelsforparallelcomputing.ParallelComp.26,1519(1999) [62] Hendrickson,B.,Leland,R.:Amultilevelalgorithmforpartitioninggraphs.Tech.Rep.SAND93-1301,SandiaNationalLaboratory(1993) [63] Hendrickson,B.,Leland,R.:Animprovedspectralgraphpartitioningalgorithmformappingparallelcomputations.SIAMJ.Sci.Comput.16,452(1995) [64] Huneault,M.,Galiana,F.D.:Asurveyoftheoptimalpowerowliterature.IEEETrans.PowerSyst.6(2),762(1991) [65] IEEE/CIGREJointTaskForceonStabilityTermsandDenitions.:Denitionandclassicationofpowersystemstability.IEEETrans.PowerSyst.19(2),1387(2004) [66] IEEEStd1547-2003.IEEEStandardforInterconnectingDistributedResourceswithElectricPowerSystems(2003) [67] Ihmels,J.,Friedlander,G.,Bergmann,S.,Sarig,O.,Ziv,Y.,Barkai,N.:Revealingmodularorganizationintheyeasttranscriptionalnetwork.Nat.Genet.31(4),370(2002) 129

PAGE 130

[68] Jain,A.K.,Murty,M.N.,Flynn,P.J.:Dataclustering:Areview.ACMComput.Sur.31(3),264(1999) [69] Kaiser,S.,Leisch,F.:Atoolboxforbiclusteranalysisinr.Tech.Rep.028,Ludwing-Maximilians-UniversitatMnchen(2008) [70] Karisch,S.E.,Rendl,F.:Semideniteprogrammingandgraphequipartition.In:Pardalos,P.M.,Wolkowicz,H.(eds)TopicsinSemideniteandInterior-PointMethods,pp.77.AmericanMathmaticalSociety,Providence(1998) [71] Karisch,S.E.,Rendl,F.,Clausen,J.:Solvinggraphbisectionproblemswithsemideniteprogramming.INFORMSJ.Comp.12,177-191(2000) [72] Karypis,G.,Aggarwal,R.,Kumar,V.,Shekhar,S.:Multilevelhypergraphpartitioning:applicationsinVLSIdomain.IEEETrans.VeryLarg.ScaleIntegr.(VLSI)Syst.7(1),69(1999) [73] Karypis,G.,Kumar,V.:Parallelmultilevelgraphpartitioning.In:10thInternationalParallelProcessingSymposium(IPPS'96),314(1996) [74] Karypis,G.,Kumar,V.:Afastandhighqualitymultilevelschemeforpartitioningirregulargraphs.SIAMJ.Sci.Comput.20,359(1998) [75] Kernighan,B.W.,Lin,S.:Anefcientheuristicprocedureforpartitioninggraphs.BellSyst.Tech.J.49(1),291(1970) [76] Klerk,E.:Aspectsofsemideniteprogramming:Interiorpointalgorithmsandselectedapplications.KluwerAcademic,Dordrecht(2002) [77] Kluger,Y.,Basri,R.,Chang,J.T.,Gerstein,M.:Spectralbiclusteringofmicroarraycancerdata:Co-clusteringgenesandconditions.GenomeRes.13,703(2003) [78] Kochenberger,G.A.,Glover,F.,Alidaee,B.,Rego,C.:Anunconstrainedquadraticbinaryprogrammingapproachtothevertexcoloringproblem.Ann.Oper.Res.139(1),229(2005) [79] Kong,N.,Schaefer,A.J.:Afactor1/2approximationalgorithmfortwo-stagestochasticmatchingproblems.Eur.J.Oper.Res.172,740(2006) [80] Kouvelis,P.,Yu,G.:Robustdiscreteoptimizationanditsapplications.KluwerAcademicPublishers,London(1997) [81] Lazzeroni,L.,Owen,A.:Plaidmodelsforgeneexpressiondata.Stat.Sinica12,61(2002) [82] Lee,D.D.,Seung,H.S.:Learningthepartsofobjetsbynon-negativematrixfactorization.Nature,401,788(1999) 130

PAGE 131

[83] Li,J.,Liu,C.-C.:Powersystemrecongurationbasedonmultilevelgraphpartitioning.In:Proceedingsof2009IEEEBucharestPowerTechConference,Burcharest,Romania(2009) [84] Lisser,A.,Rendl,F.:Graphpartitioningusinglinearandsemideniteprogramming.Math.Program.Ser.B95,91(2003) [85] Liu,Y.,Liu,Y.:Aspectsonpowersystemislandingforpreventingwidespreadblackout.In:Proceedingsofthe2006IEEEInternationalConferenceonNetworking,SensingandControl(ICNSC'06),pp.1090(2006) [86] Lodi,A.,Allemand,K.,Liebling,T.M.:Anevolutionaryheuristicforquadratic0programming.Eur.J.Oper.Res.119(3),662(1999) [87] Loiola,E.M.,deAbreu,N.M.M.,Boaventura-Netto,P.O.,Hahn,P.,Querido,T.:Asurveyforthequadraticassignmentproblem.Eur.J.Oper.Res.176(2),657(2007) [88] MacQueen,J.B.:Somemethodsforclassicationandanalysisofmultivariateobservations.In:Proceedingsof5thBerkeleysymposiumonmathematicalstatisticsandprobability,pp.281.UniversityofCaliforniaPress,Berkeley(1967) [89] Madeira,S.C.,Oliveira,A.L.:Biclusteringalgorithmsforbiologicaldataanalysis:Asurvey.IEEETrans.Comput.Biol.Bioinf.1(1),24(2004) [90] Madeira,S.C.,Oliveira,A.L.:Alineartimebiclusteringalgorithmfortimeseriesgeneexpressiondata.Lect.NotesComput.Sci.3692,39,(2005) [91] Mitchell,J.:Branch-and-cutforthek-wayequipartitionproblem,Tech.Rep.,DepartmentofMathematicalSciences,RensselaerPolytechnicInstitute(2001) [92] Montemanni,R.,Gambardella,L.M.:Anexactalgorithmfortherobustshortestpathproblemwithintervaldata.Comput.Oper.Res.1(10),1667(2004) [93] Murali,T.M.,Kasif,S.:Extractingconservedgeneexpressionmotifsfromgeneexpressiondata.PacicSymp.Biocomput.8,77(2003) [94] Overton,M.L.,Womersley,R.S.:Onthesumoflargesteigenvaluesofsymmetricmatrix.SIAMJ.MatrixAnal.Appl.13,41(1992). [95] Pandya,K.S.,Joshi,S.K.:ASurveyofoptimalpowerowmethods.J.Theor.Appl.Inf.Tech.450(2008) [96] Pardalos,P.M.,Busygin,S.,Prokopyev,O.A.:Onbiclusteringwithfeatureselectionformicroarraydatasets.In:Mondaini,R.(ed.)BIOMAT2005-internationalSymposiumonMathematicalandComputationalBiology,pp.367.WorldScientic,Singapore(2006) 131

PAGE 132

[97] Pardalos,P.M.,Rodgers,G.P.:Computationalaspectsofabranchandboundalgorithmforquadraticzero-oneprogramming.Computing45,131(1990) [98] Pardalos,P.M.,Tomaino,V.,Xanthopoulos,P.:Optimizationanddatamininginmedicine.Top17,215(2009) [99] Peiavi,A.,Ildarabadi,R.:AfastalgorithmforintentionalislandingofpowersystemsusingthemultilevelKernelk-meansapproach.J.Appl.Sci.9(12),2247(2009) [100] Pascual-Montano,A.,Carazo,J.M.,Kochi,K.,Lehmann,D.,Pascual-Marqui,R.D.:NonsmoothNon-negativematrixfactorization(nsNMF).IEEETrans.PatternAnal.Mach.Intell.28,403(2006) [101] Pothen,A.,Simon,H.D.,Liou,K.P.:Partitioningsparsematriceswitheigenvectorsofgraph.SIAMJournalofMatrixAnal.Appl.,11,430(1990) [102] PowerSystemsTestCaseArchive.http://www.ee.washington.edu/research/pstca(2011).Accessed6June2011 [103] Prelic,A.,Bleuler,S.,Zimmermann,P.,Wille,A.,Buhlmann,P.,Gruissem,W.,Hennig,L.,Thiele,L.,Zitzler,E.:Asystematiccomparisonandevaluationofbiclusteringmethodsforgeneexpressiondata.Bioinformatics22(9),1122,(2006) [104] Rege,M.,Dong,M.,Fotouhi,F.:Bipartiteisoperimetricgraphpartitioningfordataco-clustering.DataMin.Knowl.Discov.16276(2008) [105] Reiss,D.J.,Baliga,N.S.,Bonneau,R.:Integratedbiclusteringofheterogeneousgenome-widedatasetsfortheinferenceofglobalregulatorynetworks.BMCBioinformatics7,280(2006) [106] Richards,A.L.,Holmans,P.A.,O'Donovan,M.C.,Owen,M.J.,Jones,L.:Acomparisonoffourclusteringmethodsforbrainexpressionmicroarraydata.BMCBioinformatics9,490(2008) [107] Ropp,M.,Bower,W.:Evaluationofislandingdetectionmethodsforphotovoltaicutilityinteractivepowersystems.In:Int.EnergyAgencyImplementingAgreementonPhotovoltaicPowerSyst.Tech.rep.IEAPVPST5-09(2002) [108] Salmeron,J.,Wood,K.,Baldick,R.:Analysisofelectricgridsecurityunderterroristthreat.IEEETrans.PowerSyst.19(2),905(2004) [109] Santamaria,R.,Theron,R.,Quintales,L.:BicOverlapper:AtoolforbiclustervisualizationRodrigo.Bioinformatics24,1212(2008) [110] Schaeffer,S.E.:Survey:graphclustering.Comput.Sci.Rev.1,27(2007) 132

PAGE 133

[111] Sen,A.,Ghosh,P.,Vittal,V.,Yang,B.:Anewmin-cutproblemwithapplicationtoelectricpowernetworkpartitioning.Eur.Trans.Electr.Power19(6),778(2008) [112] Senroy,N.,Heydt,G.T.,Vittal,V.:Decisiontreeassistedcontrolledislanding.IEEETrans.PowerSyst.21(4),1790(2006) [113] Sheng,Q.,Moreau,Y.,DeMoor,B.:BiclusteringmicroarraydatabyGibbssampling.Bioinformatics19,196(2003) [114] Sherali,H.D.,Smith,J.C.:Animprovedlinearizationstrategyforzero-onequadraticprogrammingproblems.Optim.Lett.1,33(2007) [115] Shi,J.,Malik,J.:Normalizedcutsandimagesegmentation.IEEETrans.onPatternAnal.Mach.Intell.22(8),888(2000) [116] Soyster,A.L.:Convexprogrammingwithset-inclusiveconstraintsandapplicationstoinexactlinearprogramming.Oper.Res.21,1154(1973) [117] Soper,A.J.,Walshaw,C.,Cross,M.:Acombinedmultilevelsearchandmultileveloptimizationapproachtograph-partition.J.GlobalOptim.29,225(2004) [118] Sun,J.,L.S.,Tesfatsion.:DCoptimalpowerowformulationandsolutionusingQuadProgJ.EconomicsDepartment,IowaStateUniversity6014,1(2006) [119] Supper,J.,Strauch,M.,Wanke,D.,Harter,K.,Zell,A.:EDISA:Extractingbiclustersfrommultipletime-seriesofgeneexpressionproles.BMCBioinformatics8,334(2007) [120] Tanay,A.,Sharan,R.,Kupiec,M.,Shamir,R.:Revealingmodularityandorganizationintheyeastmolecularnetworkbyintegratedanalysisofhighlyheterogeneousgenomewidedata.Proc.Natl.Acad.Sci.USA101,2981(2004) [121] Tanay,A.,Sharan,R.,Shamir,R.:Discoveringstatisticallysignicantbilcustersingeneexpressiondata.Bioinformatics18,S136S144(2002) [122] Tanay,A.,Sharan,R.,Shamir,R.:Biclusteringalgorithms:Asurvey.In:Aluru,S.(ed.)HandbookofComputationalMolecularBiology.ChapmanHall,London(2005) [123] Tiptipakorn,S.:Aspectralbisectionpartitioningmethodforelectricpowernetworkapplications.PhDDissertation,DepartmentofElectricalandComputerEngineering,UniversityofWisconsin(2001) [124] Torres,F.E.:Linearizationofmixed-integerproducts.Math.Program.49,427(1991) 133

PAGE 134

[125] Vandenberghe,L.,Boyd,S.:Semideniteprogramming,SIAMRev.38,49(1996) [126] Vittal,V.,Heydt,G.T.,TheproblemofinitiatingcontrolledislandingofalargeinterconnectedpowersystemsolvedasaParetooptimization.In:Proceedingsofthe2009IEEEPESPowerSystemConf.andExpo.,Seattle,WA(2009) [127] Walshaw,C.:Multilevelrenementforcombinatorialoptimizationproblems.Ann.Oper.Res.131,325(2004) [128] Wang,X.,Vittal,V.:Systemislandingusingminimalcutsetswithminimumnetow.In:Proceedingofthe2004IEEEPESPowerSystemConf.andExpo.,NewYork(2004) [129] Wolkowicz,H.,Zhao,Q.:Semideniteprogrammingrelaxationsforthegraphpartitioningproblem.DiscreteAppl.Math.96-97,461(1996) [130] Wood,A.J.,Wollenberg,B.F.:PowerGeneration,OperationandControl(2nded).Wiley,NewYork(1996) [131] Xing,E.P.,Jordan,M.I.:Onsemideniterelaxationfornormalizedk-cutandconnectionstospectralclustering.UCBerkeleyTechnicalReportCSD-03-1265(2003) [132] Xu,R.,Wunsch,D.II.:Surveyofclusteringalgorithms.IEEETrans.NeuralNetw.16(3),645(2005) [133] Yaman,H.,Karasan,O.E.,Pinar,M.C.:Therobustshortestspanningtreeproblemwithintervaldata.Oper.Res.Lett.29,31(2001) [134] Yang,B.,Vittal,V.,Heydt,G.T.:Slow-coherency-basedcontrolledislanding-AdemonstrationoftheapproachontheAugust14,2003blackoutscenario.IEEETrans.PowerSyst.21(4),1840(2006) [135] Yang,J.,Wang,W.,Wang,H.,Yu,P.:-Clusters:Capturingsubspacecorrelationinalargedataset.In:Proceedingsofthe18thIEEEInternationalConferenceonDataEngineering,pp.517(2002) [136] Yang,J.,Wang,W.,Wang,H.,Yu,P.:Enhancedbiclusteringonexpressiondata.In:ProceedingsoftheThirdIEEEConferenceonBioinformaticsandBioengineering,pp.321(2003) [137] You,H.,Vittal,V.,Wang,X.:Slowcoherencybasedislanding.IEEETrans.PowerSyst.19(1),483(2004) [138] You,H.,Vittal,V.,Yang,Z.:Self-healinginpowersystems:anapproachusingislandingandrateoffrequencydeclinebasedloadshedding.IEEETrans.PowerSyst.18(1),174(2003) 134

PAGE 135

[139] Yusof,S.B.,Rogers,G.J.,Alden,R.T.H.:Slowcoherencybasednetworkpartitioningincludingloadbuses.IEEETrans.PowerSyst.8(3),1375(1993) [140] Zeineldin,H.H.,Bhattacharya,K.,El-Saadany,E.F.,Salama,M.M.A.:Impactofintentionalislandingofdistributedgenerationonelectrocitymarketprices.IEEProc.Transm.Distrib.153(2),147(2006) [141] Zha,H.,He,X.,Ding,C.,Simon,H.,Gu,M.:Bipartitegraphpartitioninganddataclustering.In:ProceedingsofthetenthinternationalconferenceonInformationandknowledgemanagement,pp.25.ACMPress,NewYork,NY(2001) [142] Zhao,H.,Liew,A.W.-C.,Xie,X.,Yan,H.:AnewgeometricbiclusteringbasedontheHoughtransformforanalysisoflarge-scalemicroarraydata.J.Theor.Biol.251,264(2008) 135

PAGE 136

BIOGRAPHICALSKETCH NengFanobtainedhisbachelor'sdegreeininformationandcomputationalsciencefromWuhanUniversityinWuhan,Chinain2004.Then,hereceivedhismaster'sdegreeinappliedmathematicsfromNankaiUniversity,Tianjin,Chinain2007.InMay2009,heobtainedaMasterofScienceinindustrialandsystemsengineeringfromUniversityofFlorida.InAugust2011,hegraduatedfromtheUniversityofFloridawithaPhDdegreeunderthesupervisionofDr.PanosM.Pardalos.DuringAugust2010toMay2011,hewasagraduateresearchassistantatD-6RiskAnalysisandDecisionSupportSystems,LosAlamosNationalLaboratory. 136