<%BANNER%>

Total Bregman Divergence, a Robust Divergence Measure, and Its Applications

Permanent Link: http://ufdc.ufl.edu/UFE0043601/00001

Material Information

Title: Total Bregman Divergence, a Robust Divergence Measure, and Its Applications
Physical Description: 1 online resource (100 p.)
Language: english
Creator: Liu, Meizhu
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2011

Subjects

Subjects / Keywords: boosting -- bregman -- divergence -- dti -- metric -- robust
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Divergence measures provide a means to measure the pairwise dissimilarity between "objects", e.g., vectors and probability density functions (pdfs). Kullback-Leibler (KL) divergence and the square loss (SL) function are two examples of commonly used dissimilarity measures which along with others belong to the family of Bregman divergences (BD). In this thesis, we present a novel divergence dubbed the Total Bregman divergence (TBD), which is inherently very robust to outliers, a very desirable property in many applications. Further, we derive the TBD center, called the t-center (using the `1-norm), for a population of positive definite matrices is in closed form and show that it is invariant to transformations from the special linear group. This tcenter, which is also robust to outliers, is then used in shape retrieval, diffusion tensor imaging (DTI) estimation, interpolation and segmentation. Furthermore, TBD is used to regularize the conventional boosting algorithms, which have been applied to applications in pattern classification.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Meizhu Liu.
Thesis: Thesis (Ph.D.)--University of Florida, 2011.
Local: Adviser: Vemuri, Baba C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2011
System ID: UFE0043601:00001

Permanent Link: http://ufdc.ufl.edu/UFE0043601/00001

Material Information

Title: Total Bregman Divergence, a Robust Divergence Measure, and Its Applications
Physical Description: 1 online resource (100 p.)
Language: english
Creator: Liu, Meizhu
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2011

Subjects

Subjects / Keywords: boosting -- bregman -- divergence -- dti -- metric -- robust
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Divergence measures provide a means to measure the pairwise dissimilarity between "objects", e.g., vectors and probability density functions (pdfs). Kullback-Leibler (KL) divergence and the square loss (SL) function are two examples of commonly used dissimilarity measures which along with others belong to the family of Bregman divergences (BD). In this thesis, we present a novel divergence dubbed the Total Bregman divergence (TBD), which is inherently very robust to outliers, a very desirable property in many applications. Further, we derive the TBD center, called the t-center (using the `1-norm), for a population of positive definite matrices is in closed form and show that it is invariant to transformations from the special linear group. This tcenter, which is also robust to outliers, is then used in shape retrieval, diffusion tensor imaging (DTI) estimation, interpolation and segmentation. Furthermore, TBD is used to regularize the conventional boosting algorithms, which have been applied to applications in pattern classification.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Meizhu Liu.
Thesis: Thesis (Ph.D.)--University of Florida, 2011.
Local: Adviser: Vemuri, Baba C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2011
System ID: UFE0043601:00001


This item has the following downloads:


Full Text

PAGE 1

TOTALBREGMANDIVERGENCE,AROBUSTDIVERGENCEMEASURE,ANDITSAPPLICATIONSByMEIZHULIUADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2011

PAGE 2

c2011MeizhuLiu 2

PAGE 3

Idedicatethistomyparents. 3

PAGE 4

ACKNOWLEDGMENTS FirstandforemostIoffermysincerestgratitudetomysupervisor,ProfessorBabaC.Vemuri,whohassupportedmyPhDstudyalltheway.Heencouragedmewithhispatience,hiscreativityandhisrstclassleadingknowledge.IalsothanksmyothercommitteemembersincludingProfessorAnandRangarajan,ProfessorJefferyHo,ProfessorArunavaBanerjeeandProfessorWilliamHager.ProfessorAnandRangarajanwithhisgreatknowledgeandexplanation,taughtmemanymachinelearningtechniqueswhichareveryimportantbuildingblocksformyresearch.ProfessorJefferyHotaughtmealotofideasaboutconvexoptimizationwhichwasfundamentalformystudyinothertopics.ProfessorArunavaBanerjeegavemealotofsuggestionsduringthetalkswhichmotivatedmystudyalot.ProfessorWilliamHagertaughtmemanynumericalmethodsthatgavemethenecessarybackgroundtosolveseveraloptimizationproblemsencounteredinthisthesisefciently.SpecialthankstoProfessorShun-IchiAmariandProfessorFrankNielsenfortheircollaborationandtheirgreathelpformyresearch.Also,specialthankstoProfessorRachidDericheforhissupervisionbeforeandduringmyinternshipatINRIA.Hissupportencouragedmeimmensely.MyfriendlyandknowledgeablecolleaguesBing,Fei,Santhosh,Angelos,Adrian,Ajit,Ozlem,Ritwik,O'Neil,Ting,Yuchen,Wenxing,Sile,Guang,Yan,Yuanxiang,Dohyung,Hesam,discussedwithmealotaboutmyresearchandothercourses.Iwouldliketoexpressmysincerethankstooneandall.FinallyIoffermygreatestappreciationtomyparentsforbringingmetotheworld,feedme,educateme,supportmeandtheirconstanthelpinmydailylife.Icannotexpectbetterparentsthanthem.Ioffermyappreciationtomygrandparents,whoalwaysmadenicedishesformeandtoldinterestingstoriestomewhenIwasachild.Ialsooffermygreatthankstomysisterandmybrothersforgrowinguptogetherwithme,givingmesupportandbringingmegreathappiness. 4

PAGE 5

TheresearchinthisdissertationwasinpartsupportedbyNIHgrantsEB007082,NS066340toProfessorBabaC.Vemuri.IalsoreceivedtheUniversityofFloridaAlumniFellowship,manytravelgrantsfromNSF,theCISEdepartment,theGraduateStudentCouncilattheUniversityofFlorida,andtheINRIAInternshipsProgram.IgratefullyacknowledgethepermissiongrantedbyIEEEandSpringerformetoreusematerialsfrommypriorpublicationsinthisdissertation. 5

PAGE 6

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 8 LISTOFFIGURES ..................................... 9 ABSTRACT ......................................... 10 CHAPTER 1INTRODUCTION ................................... 11 1.1Motivation .................................... 11 1.2`1-normCenterofTBD ............................. 12 1.2.1Outline .................................. 13 2TOTALBREGMANDIVERGENCE ......................... 14 2.1DenitionofTBDandExamples ....................... 14 2.2TotalBregmanDivergenceCenters ...................... 17 2.2.1`1-normt-center ............................ 19 2.3Propertiesoft-center ............................. 19 2.3.1t-centerUniquelyExists ........................ 19 2.3.2t-centerisStatisticallyRobusttoOutliers .............. 23 2.3.3PropertiesofTBD ........................... 25 3APPLICATIONOFTBDTODTIANALYSIS .................... 26 3.1SPDTensorInterpolationApplications .................... 26 3.2PiecewiseConstantDTISegmentation .................... 29 3.3PiecewiseSmoothDTISegmentation .................... 30 3.4ExperimentalResults ............................. 31 3.5TensorInterpolationExperiments ....................... 31 3.6TensorFieldSegmentationExperiments ................... 33 3.6.1SegmentationofSyntheticTensorFields ............... 34 3.6.2SegmentationofDTIImages ..................... 34 3.7Discussions ................................... 38 4HIERARCHICALSHAPERETRIEVALUSINGt-CENTERBASEDSOFTCLUSTERING .................................... 39 4.1LiteratureReviewforShapeRetrieval .................... 39 4.2TotalBregmanDivergenceClustering .................... 41 4.2.1TotalBregmanDivergenceHardClustering ............. 41 4.2.2TotalBregmanDivergenceSoftClustering .............. 43 4.2.2.1Syntheticexperimentsforsoftclustering .......... 45 6

PAGE 7

4.3TBDApplicationsShapeRetrieval ...................... 47 4.3.1ShapeRepresentation ......................... 48 4.3.2ShapeDissimilarityComparisonUsingtSL .............. 49 4.3.3ShapeRetrievalinMPEG-7Database ................ 51 4.3.4BrownDatabase ............................ 55 4.3.5SwedishLeafDataSet ......................... 56 4.3.63DPrincetonShapeBenchmark ................... 57 4.4Discussions ................................... 57 5ROBUSTANDEFFICIENTREGULARIZEDBOOSTINGUSINGTOTALBREGMANDIVERGENCE ............................. 58 5.1IntroductiontoBoosting ............................ 58 5.2PreviousWork ................................. 61 5.3tBRLPBoost:TotalBregmanDivergenceRegularizedLPBoost ...... 63 5.3.1Computationofdtandw ........................ 66 5.3.2BoundingtheNumberofIterations .................. 68 5.3.3WeakClassiers ............................ 69 5.4Experiments .................................. 70 5.4.1UCIDatasets .............................. 70 5.4.2OASISDatasets ............................ 72 5.4.3OtherDatasets ............................. 74 5.5Discussions ................................... 75 6SIMULTANEOUSSMOOTHINGANDESTIMATIONOFDTIVIAROBUSTVARIATIONALNON-LOCALMEANS ........................ 76 6.1LiteratureReviewforDTIEstimation ..................... 76 6.2ProposedMethod ............................... 79 6.2.1ComputationoftheWeightw(x,y) .................. 79 6.2.2ComputationofthetKLDivergence .................. 80 6.2.3TheSPDConstraint .......................... 82 6.2.4NumericalSolution ........................... 82 6.3ExperimentalResults ............................. 83 6.3.1DTIEstimationonSyntheticDatasets ................ 84 6.3.2DTIEstimationonRealDatasets ................... 84 6.4Discussions ................................... 86 7CONCLUSIONS ................................... 88 REFERENCES ....................................... 89 BIOGRAPHICALSKETCH ................................ 100 7

PAGE 8

LISTOFTABLES Table page 2-1TBDfcorrespondingtof.x=1)]TJ /F5 11.955 Tf 12.17 0 Td[(x,y=1)]TJ /F5 11.955 Tf 12.17 0 Td[(y,xtisthetransposeofx.disd-simplex. ..................................... 17 2-2TBDfandthecorrespondingt-center._x=1)]TJ /F5 11.955 Tf 13.59 0 Td[(x,_y=1)]TJ /F5 11.955 Tf 13.59 0 Td[(y.cisthenormalizationconstanttomakeitapdf,wi=1=p 1+krf(xi)k2 Pj1=p 1+krf(xj)k2. .......... 22 3-1Thedifferencebetweenthevariousmeans/medians,andthegroundtruth. .. 33 3-2Time(seconds)spentinndingthemean/medianusingdifferentdivergences. ............................................. 34 3-3Time(seconds)comparisonforsegmentingtheratspinalcord,corpuscallosumandhippocampususingdifferentdivergences. .................. 37 4-1Theclusteringresultsforthe1D,2Dand5DdatasetsbyapplyingtheBregmanandTBDsoftclusteringalgorithms. ......................... 46 4-2NMIbeweentheoriginalclustersandtheclustersgotfromTBDandBDsoftclusteringalgorithms.~cisthepredictednumberofclusters. ........... 47 4-3RecognitionratesforshaperetrievalintheMPEG-7database. ......... 55 4-4RecognitionratesforshaperetrievalfromtheBrowndatabase. ......... 56 4-5RecognitionratesforshaperetrievalfromtheSwedishleafdatabase. ..... 56 4-6Retrievalcomparisonwithothermethods(CRSP[ 92 ],DSR[ 2 ],DBF[ 2 ],DBI[ 126 ],SIL[ 126 ],D2[ 89 ])onPSB. ......................... 57 5-1DescriptionoftheUCIdatasetsthatweuse. ................... 72 5-2Classicationaccuracy(meandeviation)fordifferentboostingalgorithms. .. 72 5-3ClassicationaccuracyofdifferentmethodsontheOASISdataset. ....... 74 5-4ClassicationaccuracyfordifferentmethodsontheEpilepsy,CNS,ColontumorandLeukemiadatasets. ............................... 75 6-1ErrorinestimatedDTIandS0,usingdifferentmethods,fromsyntheticDWIwithdifferentlevelsofnoise. ............................. 85 8

PAGE 9

LISTOFFIGURES Figure page 2-1df(x,y)isBD,f(x,y)isTBD.Thisshowsthedf(x,y)andf(x,y)beforeandafterrotatingthecoordinatesystem. ........................ 14 3-1TheisosurfacesofdF(P,I)=r,dR(P,I)=r,KLs(P,I)=randtKL(P,I)=rshownfromlefttoright.ThethreeaxesareeigenvaluesofP. .......... 29 3-2Fromlefttorightareinitialization,intermediatestepandnalsegmentation. .. 35 3-3DicecoefcientcomparisonfortKL,KLs,dR,dMandLEsegmentationofsynthetictensoreldwithincreasinglevel(x-axis)ofnoise. ................. 35 3-4DicecoefcientcomparisonfortKL,KLs,dR,dMandLEsegmentationofsynthetictensoreldwithincreasingpercentage(x-axis)ofoutliers. ............ 35 3-5ThesegmentationresultsusingtKL,KLs,dR,dMandLE. ............ 36 3-6tKLsegmentationofa3Dratcorpuscallosum. .................. 37 4-1k-treediagram.EverykeyisaGMM.Eachkeyintheinnernodesisthet-centerofallkeysinitschildrennodes. ........................... 42 4-2Lefttoright:originalshapes;alignedboundaries;GMMwith10components. 50 4-3ComparisonofclusteringaccuracyoftSL,2andSL,versusaveragenumberofshapespercluster. ................................ 53 4-4Retrievalresultsusingourproposedmethod. ................... 54 5-1Anillustrationofchoosingaweakclassierusingadecisionstump.xisasample,aisthethresholdandhistheweakclassier. ................... 70 5-2ClassicationusingtBRLPBoostandELPBoostonthetrainingandtestingsetsofthepima,spambase,irisandspectfdatasets. ............... 73 6-1GroundtruthsyntheticDTIeld,theoriginalDWI,theRiciannoiseaffectedDWI,estimationusingMRE,VF,NLMt,NLM,andtheproposedmethod. .... 85 6-2TheguresaretheonesliceoftheestimatedtensoreldsusingMRE,NLM,andtheproposedmethodrespectively. ....................... 86 6-3TheFAoftheestimatedtensoreldsusingMRE,NLM,theproposedmethod,andtheprincipaleigenvectorsoftheROIs. .................... 87 9

PAGE 10

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyTOTALBREGMANDIVERGENCE,AROBUSTDIVERGENCEMEASURE,ANDITSAPPLICATIONSByMeizhuLiuDecember2011Chair:BabaC.VemuriMajor:ComputerEngineeringDivergencemeasuresprovideameanstomeasurethepairwisedissimilaritybetweenobjects,e.g.,vectorsandprobabilitydensityfunctions(pdfs).Kullback-Leibler(KL)divergenceandthesquareloss(SL)functionaretwoexamplesofcommonlyuseddissimilaritymeasureswhichalongwithothersbelongtothefamilyofBregmandivergences(BD).Inthisthesis,wepresentanoveldivergencedubbedtheTotalBregmandivergence(TBD),whichisinherentlyveryrobusttooutliers,averydesirablepropertyinmanyapplications.Further,wederivetheTBDcenter,calledthet-center(usingthe`1-norm),forapopulationofpositivedenitematricesisinclosedformandshowthatitisinvarianttotransformationsfromthespeciallineargroup.Thist-center,whichisalsorobusttooutliers,isthenusedinshaperetrieval,diffusiontensorimaging(DTI)estimation,interpolationandsegmentation.Furthermore,TBDisusedtoregularizetheconventionalboostingalgorithms,whichhavebeenappliedtoapplicationsinpatternclassication. 10

PAGE 11

CHAPTER1INTRODUCTION 1.1MotivationInapplicationsthatinvolvemeasuringthedissimilaritybetweentwoobjects(numbers,vectors,matrices,functions,imagesandsoon)thedenitionofadistanceordivergencebecomesessential.Thestateofthearthasmanywidelyuseddivergences.Thesquareloss(SL)functionhasbeenusedwidelyforregressionanalysis;Kullback-Leibler(KL)divergence[ 61 ],hasbeenappliedtocomparetwoprobabilitydensityfunctions(pdfs);theMahalanobisdistanceisusedtomeasurethedissimilaritybetweentworandomvectorsfromthesamedistribution.AlltheaforementioneddivergencesarespecialcasesoftheBregmandivergencewhichwasintroducedbyBregmanin1967[ 18 ],andoflatehasbeenwidelyresearchedbothfromatheoreticalandpracticalviewpoint[ 4 10 47 84 140 ].Atthisjuncture,itwouldbeworthinquiring,whydoesoneneedyetanotherdivergence?Theanswerwouldbethatnoneoftheexisting c[2011]IEEE.Reprinted,withpermission,from[IEEETransactionsonMedicalImaging,TotalBregmanDivergenceanditsApplicationstoDTIAnalysis,BabaC.Vemuri,MeizhuLiu,Shun-IchiAmariandFrankNielsen].c[2010]IEEE.Reprinted,withpermission,from[IEEEConferenceonComputerVisionandPatternRecognition,TotalBregmanDivergenceanditsApplicationstoShapeRetrieval,MeizhuLiu,BabaC.Vemuri,Shun-IchiAmariandFrankNielsen].c[2011]IEEE.Reprinted,withpermission,from[IEEEConferenceonComputerVisionandPatternRecognition,RobustandEfcientRegularizedBoostingUsingTotalBregmanDivergence,MeizhuLiuandBabaC.Vemuri].c[2011]Springer-Verlag.Reprinted,withpermission,from[InternationalConferenceonMedicalImageComputingandComputerAssistedInterventionWorkshoponComputationalDiffusionMRI(CDMRI),SimultaneousSmoothing&EstimationofDTIviaRobustVariationalNon-localMeans,MeizhuLiu,BabaC.VemuriandRachidDeriche].c[2011]IEEE.Reprinted,withpermission,from[IEEEInternationalSymposiumonBiomedicalImaging,RBOOST:RiemannianDistancebasedRegularizedBoosting,MeizhuLiuandBabaC.Vemuri]. 11

PAGE 12

divergencesarestatisticallyrobustandonewouldneedtouseM-estimatorsfromrobuststatisticsliteraturetoachieverobustness.Thisrobustnesshowevercomesataprice,whichis,computationalcostandaccuracy.Moreover,someofthedivergenceslackinvariancetotransformationssuchassimilarity,afneetc.Suchinvariancebecomesimportantwhendealingwithforexample,segmentation,itisdesirabletoachieveinvariancetosimilarityorafnetransformationsthattwodifferentscansofthesamepatientmightberelatedby.Morerecently,severalmethodshaveadoptedthesquarerootdensityrepresentationforrepresentingshapeswhichthencanbetreatedaspointsonahypersphereandonecanusethemetriconthespheretocomparetheshapesefcientlysincethemetriconthesphereisinclosedform.Wereferthereaderto[ 96 111 ]formoredetails.Inthiswork,weproposeanewclassofdivergenceswhichallowustoperformpairwisecomparisonofobjectsthatareinvarianttorigidmotions(translationsandrotations)appliedtothegraphoftheconvexfunctionusedindeningthedivergence.Thisdivergencemeasurestheorthogonaldistancebetweenthevalueofaconvexanddifferentiablefunctionattherstargumentanditstangentatthesecondargument.WedubthisdivergencethetotalBregmandivergence(TBD). 1.2`1-normCenterofTBDBregmandivergencehasbeenwidelyusedinclustering,whereclustercentersaredenedusingthedivergence.WewillalsodeneaclustercenterusingtheTBDinconjunctionwiththe`1-normthatistermedthet-center.Thet-centercanbeviewedastheclusterrepresentativethatminimizesthe`1-normTBDbetweenitselfandthemembersofagivenpopulation.Wederiveananalyticexpressionforthet-centerwhichaffordsitanadvantageoveritsrivals(forexample,the21distancebasedmedianofapopulationofdensities).Thekeypropertyofthet-centeristhatitisaweightedmean 12:d(p,q)=R(p)]TJ /F7 7.97 Tf 6.58 0 Td[(q)2 R(p+q)2 12

PAGE 13

andtheweightisinverselyproportionaltothemagnitudeofthegradientoftheconvexfunctionusedindeningthedivergence.Andsincenoisydataandoutliershavegreatergradientmagnitudetheirinuenceisunderplayed.Inotherwords,t-centerputsmoreweightonthenormaldataandlessweightontheextraordinarydata.Inthissense,t-centerisarobustandstablerepresentativeandthispropertymakesthet-centerattractiveinmanyapplications.Since,theTBDcanbeviewedasaweightedBDwiththeweightbeinginverselyproportionaltothemagnitudeofthegradientoftheconvexfunction,theresultingt-centerthatweobtaincanbeviewedasaweightedmedianofsorts.Thisweightingschememakesthet-centerrobusttonoiseandoutliers,sinceitisinverselydependentonthegradientoftheconvexfunction.Anothersalientfeatureofthet-centeristhatitcanbecomputedveryefcientlyduetoitsanalyticformandthisleadstoefcientclustering. 1.2.1OutlineTherestofthedissertationisorganizedasfollows:Chapter 2 introducesthedenitionofTBDandt-centeraswellastheirproperties.Chapter 3 introducestheapplicationofTBDandt-centertoDTIinterpolationandsegmentation.Chapter 4 utilizesTBDinshaperetrievalapplicationsonvariousdatasets.Chapter 5 usesTBDtoregularizetheconventionallinearprogrammingbasedboosting(LPBoost)andusesitforclassication.Chapter 6 proposesavariationalframeworkforDTIestimation.FinallywepresentconclusionsinChapter 7 13

PAGE 14

CHAPTER2TOTALBREGMANDIVERGENCEInthischapter,wewillrstdenetheTBD,andthengivethedenitionofits`1-normbasedt-center.Finally,WewillexplorethepropertiesofTBDandt-center. 2.1DenitionofTBDandExamplesWewillrstrecallthedenitionoftheconventionalBregmandivergence[ 10 ]andthendenetheTBD.Bothdivergencesaredependentonthecorrespondingconvexanddifferentiablefunctionf:X!Rthatinducesthedivergences.Itisworthpointingoutthatiffisnotdifferentiable,onecanmimicthedenitionandproofsofpropertieswithgradientsubstitutedbyanyofitssubdifferentials[ 140 ].AgeometricalillustrationofthedifferencebetweenTBDandBDisgiveninFigure 2-1 .df(x,y)(thedottedline)isBregmandivergencebetweenxandybasedonaconvexanddifferentiablefunctionfwhereasf(x,y)(boldline)istheTBDbetweenxandybasedonaconvexanddifferentiablefunctionf.Thetwoarrowsindicatethecoordinatesystem.Wecanobservethatdf(x,y)willchangeifweapplyarotationtothecoordinatesystem,whilef(x,y)willnot. A B Figure2-1. df(x,y)isBD,f(x,y)isTBD.Thisshowsthedf(x,y)andf(x,y)beforeandafterrotatingthecoordinatesystem. 14

PAGE 15

Denition1. [ 10 ]TheBregmandivergencedassociatedwitharealvaluedstrictlyconvexanddifferentiablefunctionfdenedonaconvexsetXbetweenpointsx,y2Xisgivenby, df(x,y)=f(x))]TJ /F5 11.955 Tf 11.95 0 Td[(f(y))-221(hx)]TJ /F5 11.955 Tf 11.96 0 Td[(y,rf(y)i,(2)whererf(y)isthegradientoffatyandh,iistheinnerproductdeterminedbythespaceonwhichtheinnerproductisbeingtaken.df(,y)canbeseenasthedistancebetweentherstorderTaylorapproximationtofatyandthefunctionevaluatedatx.Bregmandivergencedfisnon-negativedenite,butitisnotsymmetricanddoesnotsatisfythetriangularinequalitythusmakingitadivergence.AsshowninFigure 2-1 ,Bregmandivergencemeasurestheordinatedistance.Additionally,itisnotinvarianttorigidtransformationsappliedto(x,f(x)),suchaninvarianceishoweverdesirableinmanyapplications.ThismotivatedthedevelopmentofanalternativethatwecallthetotalBregmandivergence(TBD). Denition2. ThetotalBregmandivergence(TBD)associatedwitharealvaluedstrictlyconvexanddifferentiablefunctionfdenedonaconvexsetXbetweenpointsx,y2Xisdenedas, f(x,y)=f(x))]TJ /F5 11.955 Tf 11.95 0 Td[(f(y))-222(hx)]TJ /F5 11.955 Tf 11.95 0 Td[(y,rf(y)i p 1+krf(y)k2,(2)h,iistheinnerproductasindenition 1 ,andkrf(y)k2=hrf(y),rf(y)igenerally.AsshowninFigure 2-1 ,df(,y)measurestheordinatedistance,andf(,y)measurestheorthogonaldistance.f(,y)canbeseenasahigherorderTaylorapproximationtofatyandthefunctionevaluatedatx.Since 1 p 1+krf(y)k2=1)]TJ 13.15 8.09 Td[(krf(y)k2 2+O(krf(y)k4)(2)then f(x,y)=df(x,y))]TJ 13.15 8.09 Td[(krf(y)k2 2df(x,y)+O(krf(y)k4)(2) 15

PAGE 16

whereO()istheBigOnotation,whichisusuallysmallcomparedtothersttermandthusonecanignoreitwithoutworryingabouttheaccuracyoftheresult.Also,wecanchoosethehigherorderTaylorexpansionifnecessary.ConsidertheBregmandivergencedf(,)=f())]TJ /F5 11.955 Tf 12.14 0 Td[(f())-237(h)]TJ /F3 11.955 Tf 12.14 0 Td[(,rf()iobtainedforstrictlyconvexanddifferentiablegeneratorf.Let^xdenotethepoint(x,z=f(x))ofXRlyingonthegraphF=f^x=(x,z=f(x))jx2Xg.WevisualizetheBregmandivergenceasthez-verticaldifferencebetweenthehyperplaneHtangentat^andthehyperplaneparalleltoHandpassingthrough^: df(,)=f())]TJ /F5 11.955 Tf 11.95 0 Td[(H(),(2)withH(x)=f()+hx)]TJ /F3 11.955 Tf 12.17 0 Td[(,rf()i.Ifinsteadoftakingtheverticaldistance,wechoosetheminimumdistancebetweenthosetwohyperplanesHandH,weobtainthetotalBregmandivergence(byanalogytototalleastsquarettingwheretheprojectionisorthogonal).Sincethedistancebetweentwoparallelhyperplaneshx,ai+b1=0andhx,ai+b2=0isjb1)]TJ /F7 7.97 Tf 6.58 0 Td[(b2j kak,lettinga=(rf(),)]TJ /F8 11.955 Tf 9.3 0 Td[(1),b1=f())-245(h,rf()i,andb2=f())-245(h,rf()i,wededucethatthetotalBregmandivergenceis f(,)=df(,) p 1+krf()k2(2)ComparedtotheBD,TBDcontainsaweightfactor(thedenominator)whichcomplicatesthecomputations.However,thisstructurebringsupmanynewandinterestingpropertiesandmakesTBDanadaptivedivergencemeasureinmanyapplications.Notethat,inpractice,Xcanbeaninterval,theEuclideanspace,ad-simplex,thespaceofnon-singularmatricesorthespaceoffunctions.Forinstance,intheapplicationtoshaperepresentation,weletpandqbetwopdfs,andf(p):=Rplogp,thenf(p,q)becomeswhatwewillcallthetotalKullback-Leibler 16

PAGE 17

X f(x) f(x,y) Remark R x2 (x)]TJ /F7 7.97 Tf 6.59 0 Td[(y)2 p 1+4y2 Totalsquareloss [0,1] xlogx+xlogx xlogx y+xlogx y p 1+y(1+logy)2+y(1+logy)2 Totallogisticloss R+ )]TJ /F8 11.955 Tf 11.29 0 Td[(logx x y)]TJ /F4 7.97 Tf 6.59 0 Td[(logx y)]TJ /F4 7.97 Tf 6.58 0 Td[(1 p 1+y)]TJ /F12 5.978 Tf 5.76 0 Td[(2 TotalItakura-Saitodistance R ex ex)]TJ /F7 7.97 Tf 6.58 0 Td[(ey)]TJ /F4 7.97 Tf 6.59 0 Td[((x)]TJ /F7 7.97 Tf 6.59 0 Td[(y)ey p 1+e2y Rd kxk2 kx)]TJ /F7 7.97 Tf 6.59 0 Td[(yk2 p 1+4kyk2 TotalsquaredEuclideandistance Rd xtAx (x)]TJ /F7 7.97 Tf 6.59 0 Td[(y)tA(x)]TJ /F7 7.97 Tf 6.58 0 Td[(y) p 1+4kAyk2 TotalMahalanobisdistance d Pdj=1xjlogxj Pdj=1xjlogxj yj q 1+Pdj=1yj(1+logyj)2 TotalKLdivergence Cmn kxk2F kx)]TJ /F7 7.97 Tf 6.58 0 Td[(yk2F p 1+4kyk2F TotalsquaredFrobeniusnorm Table2-1. TBDfcorrespondingtof.x=1)]TJ /F5 11.955 Tf 11.96 0 Td[(x,y=1)]TJ /F5 11.955 Tf 11.95 0 Td[(y,xtisthetransposeofx.disd-simplex. divergence(tKL).NotethatfortKL,wedenekrf(q)k2=R(1+logq)2qspecicallytomakeitintegrable.Table 2-1 listssomeTBDswithvariousassociatedconvexfunctions. 2.2TotalBregmanDivergenceCentersInmanyapplicationsofcomputervisionandmachinelearningsuchasimageandshaperetrieval,clusteringandclassicationetc.,itiscommontoseekarepresentativeortemplateforasetofobjectshavingsimilarfeatures.Thisrepresentativenormallyisaclustercenter,thus,itisdesirabletoseekacenterthatisintrinsicallyrepresentativeandeasytocompute.Inthissection,wewillintroducethetBD-basedcenters,includingthe`p-normmean,thegeometricandharmonicmeansrespectively.Specically,wewillfocusonthe`1-normclustercenterthatwecallthetotalcenter(t-centerforshort)andexploreitsproperties. Denition3. Letf:X!RbeaconvexanddifferentiablefunctionandE=fx1,x2,,xngbeasetofnpointsinX,then,the`p-normdistancebasedontBD,Apf,betweenapointx2XandEwithassociatedfandthe`p-normisdenedas Apf(x,E)= 1 nnXi=1(f(x,xi))p!1=p(2) 17

PAGE 18

The`p-normmeanxpofEisdenedas xpa=argminxApf(x,E).(2)Itiswellknownthattheconventionalgeometric,harmonicandarithmeticmeans(intheEuclideancasehaveastrongrelationship.ThisisalsothecaseforthetBDcenters.Whenp=1,thetBDcenteristhearithmeticmeanoff,andwhenp=)]TJ /F8 11.955 Tf 9.3 0 Td[(1,thetBDmeanbecomestheharmonicmean,andwhenp!0,themeanbecomesthegeometricmean[ 5 ].Thesemeansalsobearthenameofcircumcenter(p!1),centroid(p=2)andmedian(p=1)respectively.Inthischapter,wecallthemedian(p=1)thet-centerandwewillderiveananalyticformforthet-centerandfocusonitsapplicationstoshaperetrieval.Thecircumcenterxcinthelimitingcaseamountstosolvingforxc=argminxmaxif(x,xi).xcdoesnothaveananalyticform.InthecaseofBD,thecircumcenterwasshowntobecombinatoriallysolvableinrandomizedlinear-timeusingthelinearpropertyofBregmanbisectors[ 84 ].SimilarlyresultsholdfortotalBregmandivergences.However,themaindrawbackofthecircumcenteristhefactthatitishighlyvulnerabletooutliersinthedata,awell-knownfactandonewouldhavetousepreprocessingtechniquestogetridoftheseoutliers.The`2-normcenterisobtainedbysolvingthe`2-normminimizationxm=argminxq 1 nPni=1(f(x,xi))2.Similartothecircumcenter,`2-normmeandoesnothaveananalyticformandisnoteasytocompute.Themeanxmisalsosensitivetooutliersinthedatabutnottothesamedegreeasthecircumcenter.Inthefollowingwewilldenethe`1-normmean,whichwecallt-center. 18

PAGE 19

2.2.1`1-normt-centerGivenasetE=fx1,x2,,xng,wecanobtainthe`1-normt-centerxofEbysolvingthefollowingminimizationproblem x=argminx1f(x,E)=argminxnXi=1f(x,xi)(2)Usingthe`1-normt-centerxhasadvantagesoverothercenterssinceithasaclosedformwhichmakesitscomputationallyattractive.Theadvantageisevidentintheexperimentspresentedsubsequently.Thet-centeriscloselyrelatedtootherkindsoftBD-basedcenters,likethegeometricmeanandharmonicmean.Wewillshowinthenextsectionthat,basedontKL,thet-centerofasetofpdfsisaweightedgeometricmeanofallpdfs,andthet-centerofasetofsymmetricpositivedenitematricesistheweightedharmonicmeanofallmatrices. 2.3Propertiesoft-center 2.3.1t-centerUniquelyExistsIn[ 123 ],weshowedthatt-centerexists,isuniqueandcanbewritteninanexplicitformformostcommonlyusedcases.TheproofmadeuseoftheconvexityoffandthetheLegendredualspaceofTBD.Areal-valuedfunctionfdenedonanintervaloronanyconvexsubsetofvectorspacesissaidtobeconvexifthefollowingconditionholds f(tx+(1)]TJ /F5 11.955 Tf 11.95 0 Td[(t)y)tf(x)+(1)]TJ /F5 11.955 Tf 11.96 0 Td[(t)f(y),(2)x,y2dom(f),8t2[0,1].fisstrictlyconvexif f(tx+(1)]TJ /F5 11.955 Tf 11.96 0 Td[(t)y)
PAGE 20

Denition4. Letx2X,(XcanbeRn,Rn+,orthesetofprobabilitydistributionsRn+(Pni=1xi=1)andf(x)beaconvexfunction.WethenhavethedualcoordinatesthroughtheLegendretransformation x=rf(x),(2)andthedualconvexfunction f(x)=supxfhx,xi)]TJ /F5 11.955 Tf 19.26 0 Td[(f(x)g.(2)FortheLegendretransformation,thederivativeofthefunctionfbecomestheargumenttothefunctionf.Inaddition,iffisconvex,thenfsatisesthefunctionalequation f(rf(x))=hx,rf(x)i)]TJ /F5 11.955 Tf 19.26 0 Td[(f(x).(2)TheLegendretransformisitsowninverse,i.e.f=f.LikethefamiliarFouriertransform,theLegendretransformtakesafunctionf(x)andproducesafunctionofadifferentvariablex.However,whiletheFouriertransformconsistsofanintegrationwithakernel,theLegendretransformusesmaximizationasthetransformationprocedure.Thetransformisespeciallywellbehavediff(x))isaconvexfunction.Iffisaclosed(lower-continuous)convexfunction,thenfisalsoclosedandconvex.Wealreadyknowthatthegradientatthe`1-normt-centerxisaweightedEuclideanaverageofthegradientofalltheelementsinthesetE=fx1,x2,,xng[ 123 ],asgivenby rf(x)= nXi=1wirf(xi)!, nXi=1wi!,(2)Theweightwi=)]TJ /F8 11.955 Tf 5.48 -9.69 Td[(1+krf(xi)k2)]TJ /F4 7.97 Tf 6.59 0 Td[(1=2.UtilizingtheLegendredualtransform,andletgbetheLegendredualfunctionoff,i.e. g(y)=supxfhy,xi)]TJ /F5 11.955 Tf 19.26 0 Td[(f(x)g,(2) 20

PAGE 21

then x=rg(y0),(2)andy0=)]TJ 5.48 -.72 Td[(Pni=1wirf(xi))]TJ 19.86 -.72 Td[(Pni=1wi,whichisaconstant.Wewillprovemorepropertiesoft-center,whicharesummarizedinthefollowingtheorem. Theorem2.1. [ 123 ]Thet-centerofapopulationofobjects(densities,vectorsetc.)foragivendivergenceexists,andisunique. Proof. SupposegistheLegendredualfunctionfortheconvexanddifferentiablefunctionfinthespaceX,andthedualspaceisdenotedasY.Then8x2X,9y2Y: g(y)=supxfhy,xi)]TJ /F5 11.955 Tf 19.26 0 Td[(f(x)g,andf(x)=supyfhx,yi)]TJ /F5 11.955 Tf 19.26 0 Td[(g(y)g,(2)wherexandysatisfyy=rf(x)andx=rg(y).Since rf(x)= nXi=1wirf(xi)!, nXi=1wi!,(2)whichistosay y= nXi=1wirf(xi)!, nXi=1wi!,(2)and x=rg(y).(2) Actually,iffisgiven,thenwewillgettheexplicitformforyandg,andconsequentlyx.xdoesnothaveauniformexpressionforallconvexfunctions(thisisunliketheBregmandivergencecenter,whichisalwaysthemeanwhateverthegeneratingfunctionfis[ 10 ]),instead,xisdependentontheconvexfunctions(e.g.,fortSL,xistheweightedmean;fortKL,xistheweightedgeometricmean),whichisreasonablesincespecicfunctionshavespecicmeaningsandthusresultinspecicrepresentatives,i.e.t-centers. 21

PAGE 22

X f(x) f(x,y) t-center Remark R x2 (x)]TJ /F7 7.97 Tf 6.59 0 Td[(y)2 p 1+4y2 Piwixi totalsquareloss(tSL) R)]TJ /F17 11.955 Tf 11.95 0 Td[(R)]TJ ET q .398 w 50.92 -65.56 m 50.92 -40.61 l S Q BT /F6 11.955 Tf 74.33 -55.4 Td[(xlogx xlogx y+xlogx y p 1+y(1+logy)2+y(1+logy)2 Qi(xi)wi [0,1] )]TJ /F8 11.955 Tf 11.29 0 Td[(logx x y)]TJ /F4 7.97 Tf 6.58 0 Td[(logx y)]TJ /F4 7.97 Tf 6.59 0 Td[(1 p 1+y)]TJ /F12 5.978 Tf 5.76 0 Td[(2 Pi(xi=(1)]TJ /F4 7.97 Tf 6.59 0 Td[(xi))wi 1+Pi(xi=(1)]TJ /F4 7.97 Tf 6.59 0 Td[(xi))wi totallogisticloss R+ )]TJ /F8 11.955 Tf 11.29 0 Td[(logx x y)]TJ /F4 7.97 Tf 6.58 0 Td[(logx y)]TJ /F4 7.97 Tf 6.59 0 Td[(1 p 1+y)]TJ /F12 5.978 Tf 5.76 0 Td[(2 1 Piwi=xi totalItakura-Saitodistance R ex ex)]TJ /F7 7.97 Tf 6.59 0 Td[(ey)]TJ /F4 7.97 Tf 6.59 0 Td[((x)]TJ /F7 7.97 Tf 6.59 0 Td[(y)ey p 1+e2y Piwixi Rd kxk2 kx)]TJ /F7 7.97 Tf 6.59 0 Td[(yk2 p 1+4kyk2 Piwixi totalsquaredEuclidean Rd xtAx (x)]TJ /F7 7.97 Tf 6.58 0 Td[(y)tA(x)]TJ /F7 7.97 Tf 6.59 0 Td[(y) p 1+4kAyk2 Piwixi totalMahalanobisdistance d Pdj=1xjlogxj Pdj=1xjlogxj yj q 1+Pdj=1yj(1+logyj)2 cQi(xi)wi totalKLdivergence(tKL) Cmn kxk2F kx)]TJ /F7 7.97 Tf 6.59 0 Td[(yk2F p 1+4kyk2F kx)]TJ /F7 7.97 Tf 6.58 0 Td[(yk2F p 1+4kyk2F totalsquaredFrobenius Table2-2. TBDfandthecorrespondingt-center._x=1)]TJ /F5 11.955 Tf 11.96 0 Td[(x,_y=1)]TJ /F5 11.955 Tf 11.96 0 Td[(y.cisthenormalizationconstanttomakeitapdf,wi=1=p 1+krf(xi)k2 Pj1=p 1+krf(xj)k2. Theorem( 2.1 )revealsthatthet-centeruniquelyexists,thegradientatthet-centerhasaclosedformexpression,whichisaweightedaverage,andtheweightisinverselyproportionaltothemagnitudeofthegradientoffatthecorrespondingelement.Table 2-2 liststhet-centerscorrespondingwithvariousassociatedconvexfunctions.Forbetterillustration,weprovidethreeconcreteexamplesofTBDwiththeirt-centersinexplicitform. tSL:f(x)=x2,thet-center x= Xiwixi!, Xiwi!,wi=1 p 1+4x2i;(2) Exponentials:f(x)=ex,thet-centerx=)]TJ 5.48 -.72 Td[(Piwixi=)]TJ 5.48 -.72 Td[(Piwi,wherewi=)]TJ /F8 11.955 Tf 5.48 -9.68 Td[(1+e2xi)]TJ /F4 7.97 Tf 6.58 0 Td[(1=2; tKL:Letf(q)=Rqlogq,whichisthenegativeentropy[ 116 ],andE=fq1,q2,,qngbeasetofprobabilitydensityfunctions,thet-centeristhengivenbyAlso,asan`1-normmedian,t-centeriscloselyrelatedwithgeometriccenterandharmoniccenter.TherelationshipisobviouswhenusingthetKLbetweentwopdfs.Letf(q)=Rqlogq,whichisthenegativeentropy[ 116 ],and 22

PAGE 23

E=fq1,q2,,qngbeasetofpdfs,thet-centeristhengivenby q=cnYiqwi=Pjwji,wi=1 q 1+R(1+logqi)2qidx,(2)wherecisanormalizationconstanttomakeqapdf,i.e.RXq(x)dx=1.qisaweightedgeometricmeanoffqigni=1.Thisisveryusefulintensorinterpolation,whereatensorisasymmetricpositivedenite(SPD)matrix.TheTBDbetweentwotensorsQiandQjcanbetakenasthetKLbetweentwonormaldistributionsp(x;Qi)=1 p (2)ddetQiexp)]TJ /F8 11.955 Tf 10.5 8.08 Td[(1 2xtQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1ix, (2)q(x;Qj)=1 p (2)ddetQjexp)]TJ /F8 11.955 Tf 10.49 8.09 Td[(1 2xtQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1jx, (2)andtKL(Qi,Qj)=tKL(p,q)=log(det(Q)]TJ /F4 7.97 Tf 6.59 0 Td[(1iQj))+tr(Q)]TJ /F4 7.97 Tf 6.58 0 Td[(1jQi))]TJ /F5 11.955 Tf 11.96 0 Td[(d 2q c+(log(detQj))2 4)]TJ /F7 7.97 Tf 13.15 5.48 Td[(d(1+log2) 2log(detQj),wherec=3d 4+d2log2 2+(dlog2)2 4,anddisthenumberofrows/columnsofQi.Thet-centerQforfQigni=1isweightedharmonicmean: Q=(nXi=1wi Q)]TJ /F4 7.97 Tf 6.58 0 Td[(1i))]TJ /F4 7.97 Tf 6.59 0 Td[(1,(2)wherewi=2q c+(log(detQi))2 4)]TJ /F11 5.978 Tf 7.78 3.86 Td[(d(1+log2) 2log(detQi))]TJ /F12 5.978 Tf 5.76 0 Td[(1 Pj 2r c+(log(detQj))2 4)]TJ /F11 5.978 Tf 7.79 3.86 Td[(d(1+log2) 2log(detQj)!)]TJ /F12 5.978 Tf 5.76 0 Td[(1. 2.3.2t-centerisStatisticallyRobusttoOutliersBesidesclosedformexpression,anotherfundamentalpropertyoft-centeristhatitisrobusttooutliers.Wewillpresentitstheoreticalrobustnesshereandproveitspracticalrobustnessintheexperimentalsection. Theorem2.2. [ 123 ]Thet-centerisstatisticallyrobusttooutliers.Theinuencefunctionofthet-centerfromoutliersisupperbounded. 23

PAGE 24

Proof. Therobustnessoft-centerisanalyzedbytheinuencefunctionofanoutliery.Letxbethet-centerofE=fx1,,xng.When%(small)ofoutlieryismixedwithE,xisinuencedbytheoutliers,andthenewcenterbecomesto~x=x+z(y).Wecallz(y)theinuencefunction.Thecenterisrobustwhenz(y)doesnotgrowevenwhenyisverylarge.Theinuencecurveisgivenexplicitlyinthecaseof`1-norm.~xistheminimizerof (1)]TJ /F3 11.955 Tf 11.95 0 Td[()1 nnXi=1(x,xi)+(x,y),(2)Hence,bydifferentiatingtheabovefunction,settingitequaltozeroat~x=x+zandusingtheTaylorexpansion,wehave z(y)=G)]TJ /F4 7.97 Tf 6.59 0 Td[(1 w(y)(rf(y))-222(rf(x)),(2)whereG=1 nPni=1rrf(x)w(xi),andw(y)=p 1+krf(y)k2.Hence,thet-centerisrobust jz(y)j
PAGE 25

2.3.3PropertiesofTBDTBDhasmanygoodpropertieswhichmakeitanappropriatedivergenceinmanyapplications.Forexample,TBDisinvarianttospeciallineargroup(SL(n))transformationsonthedomainofdiffusiontensorelds.ThiswillbeprovedinChapter 3 .Thesepropertieswillbeprovedandexplainedatlengthinthefollowingchapters. 25

PAGE 26

CHAPTER3APPLICATIONOFTBDTODTIANALYSISThischapterisaboutusingTBD,mainlytotalKullback-Leiblerdivergence,andt-centertointerpolatediffusiontensorsinDT-MRIdataandsegmentationoftensorelds,specicallyDT-MRI. 3.1SPDTensorInterpolationApplicationsDenetKL,thetotalKullback-Leiblerdivergencebetweensymmetricpositivedenite(SPD)rank-2tensors(SPDmatrices),andshowthatitisinvarianttotransformationsbelongingtothespeciallineargroupSL(n).Further,wecomputethet-centerusingtKLforSPDmatrices,whichhasaclosedformexpression,astheweightedharmonicmeanofthepopulationofthetensors.tKLbetweenordertwoSPDtensors/matricesPandQisderivedusingthenegativeentropyofthezeromeanGaussiandensityfunctionstheycorrespondto.NotethatordertwoSPDtensorscanbeseenascovariancematricesofzeromeanGaussiandensities.Suppose, p(x;P)=1 p (2)ndetPexp)]TJ /F8 11.955 Tf 10.49 8.09 Td[(1 2xtP)]TJ /F4 7.97 Tf 6.59 0 Td[(1x, (3) q(x;Q)=1 p (2)ndetQexp)]TJ /F8 11.955 Tf 10.5 8.09 Td[(1 2xtQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1x, (3) then, tKL(P,Q)=Rplogp qdx q 1+R(1+logq)2qdx=log(det(P)]TJ /F4 7.97 Tf 6.59 0 Td[(1Q))+tr(Q)]TJ /F4 7.97 Tf 6.59 0 Td[(1P))]TJ /F5 11.955 Tf 11.96 0 Td[(n 2q c+(log(detQ))2 4)]TJ /F7 7.97 Tf 13.15 5.47 Td[(n(1+log2) 2log(detQ), (3) wherec=3n 4+n2log2 2+(nlog2)2 4.WhenanSL(n)transformationisappliedonx,i.e.,x7!Ax,thenP7!AtPAandQ7!AtQA.Itiseasytoseethat tKL(P,Q)=tKL(AtPA,AtQA),8A2SL(n),(3) 26

PAGE 27

whichmeansthattKLbetweenSPDtensorsisinvariantunderthegroupaction,whenthegroupmemberbelongstoSL(n).GivenanSPDtensorsetfQigmi=1,itst-centerPcanbeobtainedfrom( 2 )and P=(XiwiQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1i))]TJ /F4 7.97 Tf 6.58 0 Td[(1,wi=i Pjj,(3)wherei=2q c+(log(detQi))2 4)]TJ /F7 7.97 Tf 13.15 5.47 Td[(n(1+log2) 2log(detQi))]TJ /F4 7.97 Tf 6.59 0 Td[(1.Itcanbeseenthatwi(Qi)=wi(AtQiA),8A2SL(n).IfatransformationA2SL(n)isapplied,thenewt-centerwillbe^P=(Pwi(AtQiA))]TJ /F4 7.97 Tf 6.59 0 Td[(1))]TJ /F4 7.97 Tf 6.58 0 Td[(1=AtPA,whichmeansthatiffQigmi=1aretransformedbysomememberofSL(n),thenthet-centerwillundergothesametransformation.AlsowecancomputethetSLbetweenPandQfromthedensityfunctionsp,qandf(p)=Rp2dx,andtheresultistSL(P,Q)=R(p)]TJ /F5 11.955 Tf 11.95 0 Td[(q)2dx q 1+R(2q)2qdx=1=p det(2P)+1=p det(2Q))]TJ /F8 11.955 Tf 11.96 0 Td[(2=p det(P+Q) (2)n+4p (2)n=p det(3Q),also,tSL(P,Q)=tSL(AtPA,AtQA),8A2SL(n),whichmeansthattSLisalsoinvariantunderSL(n)transformations.Similarly,wecanalsoprovethatthetotalItakura-SaitodistanceandtotalsquaredEuclideandistancebetweenSPDmatricesareinvariantunderSL(n)transformations.Fortherestofthischapter,wewillfocusontKL.ThereareseveralwaystodenethedistancebetweenSPDmatrices,e.g.usingtheFrobeniusnorm[ 127 ],Riemannianmetric[ 57 64 79 94 ],symmetrizedKLdivergence[ 79 85 128 ]andthelog-Euclideandistance[ 103 ],respectivelydenedas dF(P,Q)=kP)]TJ /F5 11.955 Tf 11.96 0 Td[(QkF=p tr((P)]TJ /F5 11.955 Tf 11.95 0 Td[(Q)t(P)]TJ /F5 11.955 Tf 11.96 0 Td[(Q)), (3) dR(P,Q)=vuut nXi=1log2i, (3) 27

PAGE 28

wherei,i=1,,n,areeigenvaluesofP)]TJ /F4 7.97 Tf 6.59 0 Td[(1Q, KLs(P,Q)=1 4tr(Q)]TJ /F4 7.97 Tf 6.59 0 Td[(1P+P)]TJ /F4 7.97 Tf 6.59 0 Td[(1Q)]TJ /F8 11.955 Tf 11.95 0 Td[(2I). (3) LE(P,Q)=klog(P))]TJ /F8 11.955 Tf 11.95 0 Td[(log(Q)kF. (3) dF(.,.)isnotinvarianttotransformationsinSL(n),dR(.,.),KLs(.,.)andLE(.,.)areinvarianttotransformationsinGL(n),butnoneofthemarerobusttooutliersencounteredinthedatafore.g.,intensorinterpolation.FordR(.,.),neitheritsmeannoritsmedianisinclosedform,whichmakesitcomputationallyveryexpensiveasthepopulationsizeandthedimensionalityofthespaceincreases.Here,theKarchermeandenotedbyPMisdenedastheminimizerofthesumofsquaredRiemanniandistancesandthemedianPRisminimizerofthesumofRiemanniandistances[ 41 ]PM=argminPmXi=1d2R(P,Qi),andPR=argminPmXi=1dR(P,Qi).Forsimplicityinnotation,wedenoted2R(P,Qi)bydM(P,Qi),andPM=argminPPmi=1dM(P,Qi).Eventhoughtherearemanyalgorithms(forexample[ 40 41 64 94 ])tosolvethegeodesicmeanandmedian,mostofthemadoptaniterative(gradientdescent)method.Performinggradientdescentonmatrixmanifoldscanbetrickyandrathercomplicatedinthecontextofconvergenceissues(see[ 1 ])andhenceisnotpreferredoveraclosedformcomputation.KLsin( 3 )hasaclosedformmean[ 79 128 ],whichisG(A(Q1,,Qm),H(Q1,,Qm)),whereAisthearithmeticmeanoffQigmi=1,HistheharmonicmeanoffQigmi=1,andGisthegeometricmeanofAandH.However,itcanbeshownthatthemeancomputedusingsumofsquaredKLsdivergencesisnotstatisticallyrobustsincealltensorsaretreatedequally.Thisisbecause,neitherKLsnorthesumofsquaredKLssarerobustfunctions.Thisisdemonstratedintheexperimentalresults 28

PAGE 29

tofollow.Firstwewillobservethevisualdifferencebetweentheaforementioneddivergences/distances.Figure 3-1 showstheisosurfacescenteredattheidentitymatrixwithradiir=0.1,0.5and1respectively.FromlefttorightaredF(P,I)=r,dR(P,I)=r,KLs(P,I)=randtKL(P,I)=r.Theseguresindicatethedegreeofanisotropyinthedivergences/distances. Figure3-1. TheisosurfacesofdF(P,I)=r,dR(P,I)=r,KLs(P,I)=randtKL(P,I)=rshownfromlefttoright.ThethreeaxesareeigenvaluesofP. 3.2PiecewiseConstantDTISegmentationGivenanoisydiffusiontensorimage(DTI)T0(x)aeldofpositivedenitematrices,ourmodelforDTIsegmentationisbasedontheMumford-Shahfunctional[ 83 ], E(C,T)=Z(T0(x),T(x))dx+Z)]TJ /F7 7.97 Tf 6.58 0 Td[(CjrT(x)jdx+jCj,(3)whereandarecontrolparameters,isthetKLdivergencesameas( 3 ).istheregionofthetensoreld,T(x)isanapproximationtoT0(x),whichcanbediscontinuousonlyalongC.However,inasimpliedsegmentationmodel,aeldT0(x)canberepresentedbypiecewiseconstantregions[ 21 83 ].Therefore,weconsiderthefollowingbinarysegmentationmodelforDTI, E(C,T1,T2)=ZR(T0(x),T1)dx+ZRc(T0(x),T2)dx+jCj,(3) 29

PAGE 30

T1isthet-centerofDTIfortheregionRinsidethecurveCandT2isthet-centeroftheDTIfortheregionRcoutsideC, T1= XQi2RwiQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1i!)]TJ /F4 7.97 Tf 6.59 0 Td[(1,wi=i PQj2Rj.(3) T2= XQi2RcwiQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1i!)]TJ /F4 7.97 Tf 6.59 0 Td[(1,wi=i PQj2Rcj.(3)TheEulerLagrangeequationof( 3 )is ((T0(x),T1))]TJ /F3 11.955 Tf 11.95 0 Td[((T0(x),T2)+)N=0,(3)whereisthecurvature,=rrC jrCj,andNisthenormalofC,N=rC.Ccanbeupdatediterativelyaccordingtothefollowingequation @C @t=)]TJ /F8 11.955 Tf 9.3 0 Td[(((T0(x),T1))]TJ /F3 11.955 Tf 11.95 0 Td[((T0(x),T2)+)N.(3)Ateachiteration,wewillxC,updateT1andT2accordingto( 3 )and( 3 ),andthenfreezeT1andT2toupdateC.Inthelevelsetformulationoftheactivecontour[ 73 ],letbethesigneddistancefunctionofCandchooseittobenegativeinsideandpositiveoutside.Thenthecurveevolutionequation( 3 )canbereformulatedusingthelevelsetframework @ @t=((T0(x),T1))]TJ /F3 11.955 Tf 11.96 0 Td[((T0(x),T2)+rr jrj)jrj.(3) 3.3PiecewiseSmoothDTISegmentationForcomplicatedDTIimages,thepiecewiseconstantassumptiondoesnothold.Therefore,wehavetoturntothemoregeneralmodel,thepiecewisesmoothsegmentation.Inthischapter,wefollowWangetal's[ 128 ]modelbutreplacetheir 30

PAGE 31

divergence(KLs)withtKL,resultinginthefollowingfunctionalE(C,T)=ZR(T0(x),TR(x))dx+ZRc(T0(x),TRc(x))dx+Z)]TJ /F7 7.97 Tf 6.59 0 Td[(Cp(T)(x)dx+jCj,thethirdtermmeasuresthelackofsmoothnessoftheeldusingtheDirichletintegral[ 52 ],givingusthefollowingcurveevolutionequation:@C @t=0@(T0(x),TRc)+Xy2NRc(x)(TRc,TRc(y)1AN)]TJ /F19 11.955 Tf 19.26 24.03 Td[(0@(T0(x),TR)+Xy2NR(x)(TR,TR(y)))]TJ /F3 11.955 Tf 11.96 0 Td[(1AN.where,NRandNRcarethesetsofx'sneighboringpixelsinsideandoutsideoftheregionRrespectively.Inthediscretecase,onecanuseappropriateneighborhoodsfor2Dand3D.Weapplythetwostagepiecewisesmoothsegmentationalgorithmin[ 128 ]tonumericallysolvethisevolution. 3.4ExperimentalResultsWeperformedtwosetsofexperimentshere,(i)tensorinterpolationand(ii)tensoreldsegmentation.Theexperimentalresultsarecomparedwiththoseobtainedbyusingotherdivergencemeasuresdiscussedabove. 3.5TensorInterpolationExperimentsSPDtensorinterpolationisacrucialcomponentofDT-MRIanalysisinvolvingsegmentation,registrationandatlasconstruction[ 12 ],wherearobustdistance/divergencemeasureisverydesirable.First,weperformtensorinterpolationonasetoftensorswithnoiseandoutliers.WexanSPDtensorGasthegroundtruthtensorandgeneratenoisytensorsGfromitbyusingaMonteCarlosimulationmethodaswasdonein[ 90 93 ].Thisentails,givenab-value(weused1200s=mm2)azerogradientbaselineimageS0,andsixvectorsfqig6i=1,themagnitudeofthenoisefreecomplex-valued 31

PAGE 32

diffusionweightedsignalisgivenby,Si=S0exp()]TJ /F5 11.955 Tf 9.3 0 Td[(bqtiGqi).WeaddGaussiandistributed(meanzeroandvariance2=0.1)noisetotherealandimaginarychannelsofthiscomplexvalueddiffusionweightedsignalandtakeitsmagnitudetogetSi=S0exp()]TJ /F5 11.955 Tf 9.3 0 Td[(bqtiGqi).SithenhasaRiciandistributionwithparameters(,Si).GisobtainedbyttingthefSig6i=1usingLog-Euclideantensortting.Togenerateoutliers,werstgeneratea33matrixZ,witheachentrydrawnfromthenormaldistribution,andtheoutliertensorGisgivenbyG=exp(ZZt).Wecomputetheeigen-vectorsUandeigen-valuesVofG,i.e.G=Udiag(V)Ut,andthenrotateUbyarotationmatrixr(,,)in3D,togetUo,where,andareuniformlydistributedin[0,=2].ThentheoutlierisgivenbyGo=Uodiag(randperm(V))Uto.Togenerateanimage,thesameprocessisrepeatedateveryvoxel.ThetKLt-center,thegeometricmedian,thegeometricmean,theKLsmeanandthelog-EuclideanmeanMfor21SPDtensorsalongwith0,5,10,15,and20outliersarethencomputed.Thedifferencebetweenthevariousmeans/medians,andthegroundtruthGaremeasuredinfourways,usingtheFrobeniusnormkG)]TJ /F5 11.955 Tf 11.77 0 Td[(MkF,the`1distancekG)]TJ /F5 11.955 Tf 12.01 0 Td[(Mk1,theanglebetweentheprincipaleigenvectorsofMandG,andthedifferenceoffractionalanisotropyindex(FA)[ 91 97 ]forMandG,i.e.,jFA(M))]TJ /F5 11.955 Tf 12.82 0 Td[(FA(G)j.TheresultsareshowninTable 3-1 fromlefttoright,toptobottom.Fromthetables,wecanseethatthet-centeryieldsthebestapproximationtothegroundtruth,andfaithfullyrepresentsthedirectionalityandanisotropy.Therobustnessoft-centeroverotherssuchastheRiemannianmeanandmedian,KLsmean,andlog-Euclideanmeanisquiteevidentfromthistable.Eventhoughgeometricmedianseemstobecompetitivetothet-centerobtainedusingtKLinthecaseoflowerpercentageofoutliers,thegeometricmediancomputationhoweverismuchslowerthanthatoft-centercomputation.Thisisbecausethet-centerfortKLhasaclosedformwhilethegeometricmediandoesnot.Table 3-2 showstheCPUtimetondthetensormean/medianusingdifferentdivergences.Weuse1000SPDtensors,alongwith0,10, 32

PAGE 33

Table3-1. Thedifferencebetweenthevariousmeans/medians,andthegroundtruth. ErrorsincomputedmeanmeasuredbyFrobeniusnormdF ]ofoutliers 0 5 10 15 20 tKL 0.8857 1.0969 1.1962 1.3112 1.4455 dR 1.3657 1.4387 1.5348 1.6271 1.7277 dM 1.4291 1.5671 1.8131 2.1560 2.5402 KLs 3.4518 3.5744 3.7122 4.1040 4.3235 LE 1.0638 1.5721 1.6249 1.6701 1.8227 Errorsincomputedangle ]ofoutliers 0 5 10 15 20 tKL 0.1270 0.3875 0.6324 1.8575 2.9572 dR 0.4218 0.9058 8.9649 17.4854 44.9157 dM 0.8147 1.1985 10.1576 21.8003 43.7922 KLs 0.8069 0.9134 14.9706 26.1419 44.9595 LE 0.6456 0.9373 8.9937 17.0636 44.7635 Errorsincomputedmeanmeasuredby`1 tKL 0.6669 0.8687 1.0751 1.1994 1.5997 dR 0.8816 1.3498 1.5647 1.8826 2.1904 dM 1.3056 1.4160 1.6755 1.9390 2.5153 KLs 4.1738 4.3352 4.3463 4.3984 4.4897 LE 0.8768 1.2843 1.6947 1.9418 2.2574 ErrorsincomputedFA ]ofoutliers 0 5 10 15 20 tKL 0.0635 0.0738 0.0753 0.0792 0.0857 dR 0.0983 0.1529 0.1857 0.1978 0.2147 dM 0.1059 0.1976 0.2008 0.2193 0.2201 KLs 0.0855 0.1675 0.2005 0.2313 0.2434 LE 0.0972 0.1513 0.1855 0.1968 0.2117 100,500and800outliersrespectively.Alltensorsaregeneratedinthesamewayasdescribedintherstexperiment.Thetimeisaveragedbyrepeatingtheexperiment10timesonaPC,withIntel(R)Core(TM)2DuoCPUP7370,2GHz,4GBRAM,on32-bitWindowsVistaOS.Table 3-1 and 3-2 depictthesuperiorrobustnesstooutliersandthecomputationalefciencyinestimatingthetKLt-centerforSPDtensorinterpolationincomparisontoitsrivals. 3.6TensorFieldSegmentationExperimentsWenowdescribeexperimentsonsegmentationofsyntheticandrealDTIimages. 33

PAGE 34

Table3-2. Time(seconds)spentinndingthemean/medianusingdifferentdivergences. time Divergencesn]ofoutliers 0 10 100 500 800 tKL 0.02 0.02 0.02 0.03 0.03 KLs 0.03 0.03 0.03 0.04 0.04 dR 0.58 1.67 4.56 93.11 132.21 dM 0.46 1.42 3.13 72.74 118.05 LE 0.02 0.02 0.03 0.03 0.03 3.6.1SegmentationofSyntheticTensorFieldsTherstsynthetictensoreldiscomposedoftwotypesofhomogeneoustensors.Figure 3-2 depictsthesyntheticdataandthesegmentationresults.Weaddeddifferentlevelsofnoisetothetensoreldusingthemethoddescribedinsection 3.5 ,segmenteditusingtheaforementioneddivergencesandcomparedtheresultsusingthedicecoefcient[ 82 ].Wealsoaddeddifferentpercentagesofoutlierstothetensoreldsandsegmentedtheresultingtensorelds.Figure 3-3 depictsthecomparisonofsegmentationresultsfromdifferentmethodsusingthedicecoefcient,forvaryingnoiselevelswithvaryingfrom(0,0.02,0.04,,0.2).Figure 3-4 displaysthecomparisonofdicecoefcientwithdifferentpercentage(0,5,10,,50)ofoutliers.Theresultsshowthateveninthepresenceoflargeamountsofnoiseandoutliers,tKLyieldsverygoodsegmentationresultsincomparisontorivals.However,inourexperiments,weobservedthatthesegmentationaccuracyisinverselyproportionaltothevarianceoftheoutlierdistribution. 3.6.2SegmentationofDTIImagesInthissection,wepresentsegmentationresultsonrealDTIimagesfromaratspinalcord,anisolatedrathippocampusandaratbrain.ThedatawereacquiredusingaPGSEwithTR=1.5s,TE=28.3ms,bandwidth=35Khz,21diffusionweightedimageswithab-valueof1250s=mm2werecollected.A33diffusiontensorineachDTimageisillustratedasanellipsoid[ 128 ],whoseaxes'directions,andlengthscorrespondtoitseigen-vectors,andeigen-valuesrespectively.Thesameinitializationisusedforeach 34

PAGE 35

Figure3-2. Fromlefttorightareinitialization,intermediatestepandnalsegmentation. Figure3-3. DicecoefcientcomparisonfortKL,KLs,dR,dMandLEsegmentationofsynthetictensoreldwithincreasinglevel(x-axis)ofnoise. Figure3-4. DicecoefcientcomparisonfortKL,KLs,dR,dMandLEsegmentationofsynthetictensoreldwithincreasingpercentage(x-axis)ofoutliers. 35

PAGE 36

A B C D Figure3-5. ThesegmentationresultsusingtKL,KLs,dR,dMandLE. divergencebasedsegmentation.Weletallthedivergencebasedmethodsrununtilconvergence,recordtheirtimeandcomparetheresults.Weapplythepiecewiseconstantsegmentationmodelonasingleslice(108108)oftheratspinalcord,andapplythepiecewisesmoothsegmentationmodelonthemolecularlayerfromsingleslicesofsize(114108)forratcorpuscallosum(CC)and(9090)fortherathippocampusrespectively.Figure 3-5A B C showtheinitialization,Figure 3-5D showsthesegmentationresults,andTable 3-3 recordstheirexecutiontime.Theresultsconrmthatwhencomparedtootherdivergences,tKLyieldsamoreaccuratesegmentationinasignicantlyshorteramountofCPUtime. 36

PAGE 37

Table3-3. Time(seconds)comparisonforsegmentingtheratspinalcord,corpuscallosumandhippocampususingdifferentdivergences. Divergences tKL KLs dR dM LE TimeforCord 33 68 72 81 67 TimeforCC 87 183 218 252 190 Timeforhippocampus 159 358 545 563 324 Apartfromsegmentationin2Dslices,wealsodemonstrate3DDTIimagesegmentationusingtheproposeddivergence.Figure 3-6 depictstheprocessofsegmentingratcorpuscallosum(11410811)usingthepiecewiseconstantsegmentationmodel.Figure 3-6A B C D isa2Dsliceofthecorrespondingevolvingsurface,fromlefttorightareinitialization,intermediatestepsandnalsegmentation.Figure 3-6E isa3Dviewofthesegmentationresult.TheresultdemonstratesthattKLcansegmentthiswhitematterbundlequitewell. A B C D E Figure3-6. tKLsegmentationofa3Dratcorpuscallosum. 37

PAGE 38

3.7DiscussionsInthischapter,wedevelopedanapplicationofusingTBDforDTIinterpolationandsegmentation.Specically,wederivedanexplicitformulaforthet-centerwhichistheTBD-basedmedian,thatisrobusttooutliers.InthecaseofSPDtensors,thet-centerwasshowntobeSL(n)invariant.However,thestoryisnotyetcompleteandfurtherinvestigationsarecurrentlyunderway.Therobustness(tooutliers)propertyofTBDwasdemonstratedhereviaapplicationstoSPDtensoreldinterpolationandsegmentation.Theresultsfavorablydemonstratethecompetitivenessofournewlydeneddivergenceincomparisontoexistingmethodsnotonlyintermsofrobustness,butalsointermsofcomputationalefciencyandaccuracyaswell. 38

PAGE 39

CHAPTER4HIERARCHICALSHAPERETRIEVALUSINGT-CENTERBASEDSOFTCLUSTERINGInthischapter,weconsiderthefamilyoftotalBregmandivergences(tBDs)asanefcientandaccuratedistancemeasuretoquantifythe(dis)similaritybetweenshapes.Weusethet-centerastherepresentativeofasetofshapesandproposeanewclusteringtechniquenamely,thetotalBregmansoftclusteringalgorithm.WeevaluatethetBD,t-centerandthesoftclusteringalgorithmonshaperetrievalapplications.Theshaperetrievalframeworkiscomposedofthreesteps:(1)extractionoftheshapeboundarypoints(2)afnealignmentoftheshapesanduseofaGaussianmixturemodel(GMM)[ 24 54 ]torepresentthealignedboundaries,and(3)comparisonoftheGMMsusingtBDtondthebestmatchesgivenaqueryshape.Tofurtherspeeduptheshaperetrievalalgorithm,weperformhierarchicalclusteringoftheshapesusingourtotalBregmansoftclusteringalgorithm.Thisenablestocomparethequerywithasmallsubsetofshapeswhicharechosentobetheclustert-centers.Theproposedmethodisevaluatedonvariouspublicdomain2Dand3Ddatabases,anddemonstratecomparableorbetterresultsthanstate-of-the-artretrievaltechniques. 4.1LiteratureReviewforShapeRetrievalAsthenumberofimagesontheInternet,inpublicdatabasesandinbiometricsystemsgrowslargerandlarger,efcientandaccuratesearchalgorithmsforretrievalofthebestmatcheshavebecomecrucialforavarietyoftasks.Therefore,imageretrievalbecomesmoreandmorefundamentalincomputervisionandplaysanindispensableroleinmanypotentialapplications.Incontemporaryliterature,therearemainlytwotypesofalgorithmsforimageretrieval,key-wordsbasedandcontentbased.Key-wordsareanimportantandeasytousefeaturesforrepresentationandretrievalofimages.However,thoughefcient,key-wordsareverysubjective,sincedifferentpeoplemayusedifferentkey-wordstoindexthesameimage.Therefore,theaccuracyofkey-wordsbasedretrievalisverylimited.Hencethereisinterestintheideaofretrievalbased 39

PAGE 40

onimagefeatures[ 8 66 67 76 96 ]suchastexture,color,shape,andsoon.Ofthese,shapeisconsideredmoregenericandisoneofthebestforrecognitionasstudies[ 16 ]haveshown.Shapecomparisonandclassicationisveryoftenusedintheareasofobjectdetection[ 48 71 ]andactionrecognition[ 132 ].Thereforemanyresearchers[ 106 115 121 136 137 ]havebeendevelopingalgorithmsforimprovingtheperformanceofshaperetrieval.Anefcientmodernshaperetrievalschemehasthefollowingtwocomponents:anaccessibleandaccurateshaperepresentation,andanefcientaswellasrobustdistance/divergencemeasure.Therearemanywaystorepresentshapes,forexample,axialrepresentation[ 65 109 ],primitive-basedrepresentation[ 42 ],constructiverepresentation[ 36 ],referencepointsandprojectionbasedrepresentation[ 28 ],cover-basedrepresentation[ 98 ],histogramsoforientedgradients[ 72 ].Ofthese,contourbasedrepresentationinobjectrecognitionmethods[ 23 35 56 81 88 108 138 ]haveshowngreatperformance.Theprobabilitydensityfunction(pdf)hasemergedasasuccessfulrepresentationforshapecontours[ 32 70 96 ].Itisknowntobemathematicallyconvenientandrobusttorigidtransformations,noise,occlusionsandmissingdata.Bearingthisinmind,wechoosetorepresentshapesaspdfs.Againasthenumberofimagesinthewebsiteisincreasing,therequirementofretrievingspeedbecomesstronger(e.g.TinEyereverseimagesearchandGoogleimageretrieval,bothrequirerealtimeresponse).Anaccurateandfastretrievingmethodwillleadtogreatconvenienceforshaperetrievalinreallife.inthischapter,wewillbringupafastandaccurateshaperetrievalmethod,whichrepresentsshapesusingmixtureofGaussians(GMM),anddividetheshapesintosmallergroupsusingtotalBregmandivergencesoftclusteringalgorithm,eachclusterhavingarepresentativewhichisTBDbased`1-normcenter,notedast-center[ 70 ].Thet-centerisaweightedcombinationofallelementsinthecluster,hasaclosedformexpression,andisrobusttonoiseandoutliers.Westoretheshapesusingk-treestructure,withtheinnernodest-centers,and 40

PAGE 41

theleafnodesGMMsofshapes,whichwewillexplainatlengthlaterintheexperimentalpart.Duringretrieval,weonlyneedtocomparethequerywiththet-centers,andoncethebestmatchrepresentativeisobtained,wecomparethequerywiththet-centersoftherelativesub-clusters,recursivelydoingso,untiltherequirednumberofbestmatchesarefound. 4.2TotalBregmanDivergenceClusteringWhenretrievinginasmalldatabase,itispossibletoapplythebrute-forcesearchmethodbycomparingthequeryshapewitheachshapeinthedatabaseonebyone,however,inthecaseofretrievinginalargedatabase,itbecomesimpracticaltousethisbrute-forcesearchmethodbecauseoftheextremelyhighcomputationalcost.Tomakerealtimeretrievalinalargedatabase,weturntoafarmoreefcientstrategy,namelyadivideandconquerstrategy.First,utilizingthetopdownapproach,wesplitthewholedatabaseintosubclusters,andrepeatthesameapproachonthesubclusters.Thedivergencebetweenshapesfromthesameclustershouldbelessthanthedivergencebetweenimagesfromdifferentclusters,thenchoosearepresentativeforeachcluster,andassigneachshapetothenearestcluster.Therearetwowaysofassigningashapetoacluster,assignittoaclustercompletely,orassigntheshapetoaclusteraccordingtosomeprobability.Theformercorrespondstohardclustering,whilethelatercaseissoftclustering. 4.2.1TotalBregmanDivergenceHardClusteringTotalBregmandivergencehardclusteringassignsoneobjecttoaclusterwhosecenteristheclosesttotheobject.Theassignmentprocessisasfollowing.Wecomparetheobjectwiththeclustercenters,andpickthecenterthathassmallestdivergencetotheobject,andthenassigntheobjecttotherelativecluster.Afteralltheassignment,werecomputetheclustercenter,andthenrepeattheaforementionedsteps.Whenretrieving,weonlyneedtocomparethequerywitheachcluster'srepresentative,ifthedivergenceislargerthansomethreshold,wewillprunethiswholecluster,thus 41

PAGE 42

reducingthenumberofunnecessarycomparisonsandconsequentlyspeedinguptheretrieval.Oncetherepresentativethatbestmatchesthequeryisobtained,wewillrecursivelysplitthecorrespondingclusterintosmallerclusters,andrepeattheaforementionedprocessoneachsubcluster,andinthisway,successfullyseekingoutthebestmatches.Moreconveniently,thestepofsplittingcanbedoneoffline,whichsavesalotofcomputationtime.Topartitionthedatabaseintosmallerclustersefcientlyandaccurately,weutilizeanideasimilartothatofk-tree(ahybridoftheB-Treesandk-means[ 19 62 ]),byrstdividingthedatabaseintokclusters,calculatingthet-centerastherepresentativeofeachclusteraccordingtothetotalBregmanhardclusteringalgorithm(Algorithm 1 ),andrepeattheaboveprocessontheresultingclusterstogetksub-clusters,andaccordinglygetahierarchyofclusters.Thet-centersforthehierarchicalclustersformthek-tree.Tobeexplicit,inthek-tree,everykeyisamixtureofGaussians,everyinnernode(includingtheroot)has1tokkeys,eachofwhichisthet-centerofallkeysinitschildrennodes,andthekeyforaleafnodeisamixtureofGaussiansforanindividualshape.Thek-treeillustrationisshowninFigure 4-1 Figure4-1. k-treediagram.EverykeyisaGMM.Eachkeyintheinnernodesisthet-centerofallkeysinitschildrennodes. 42

PAGE 43

Duringretrieval,oneonlyneedstocomparethequerywiththerepresentatives,andoncethebestmatchrepresentativeisobtained,wecomparethequerywiththet-centersoftherelativesub-clusters,recursivelydoingso,untiltherequirednumberofbestmatchesarefound.Furthermore,thestepsofclusteringcanbeparallelizedusingmulti-core,multi-threadlevelparallelism. Algorithm1TotalBregmanHardClusteringAlgorithm Input:X=fxigni=1. Output:fCj,mjgkj=1,Cjisthejthclusterwithclustercentermj Initialization:RandomlychoosekelementsfromPasthet-centersfmjgkj=1andsetCj=; repeat fori=1tondo fComputexibelongstowhichclusterg Cj Cj[fxig,wherej=argmin~jf(xi,m~j) iftheradiusofclusterCjistoolarge,thenCjwillberandomlydividedintotwoclusters,andgotorepeat. endfor forj=1tokdo fUpdateclustercentersg mj t-centerforclusterCj(Equation( 2 )) endfor untilSomeconvergencecriterionismet(e.g.,unchangedassignment.) Thenumberofclusterscisdeterminedbytheradiusofthecluster,i.e.themaximumdivergencefromallelementstotheclustercenter.Iftheradiusoftheclusteristoolarge,thentheclusterwillbedividedintotwoclusters. 4.2.2TotalBregmanDivergenceSoftClusteringTotalBregmandivergencesoftclusteringisanotherclusteringstrategy,whichassignstheobjecttomorethanoneclustersaccordingtosomeprobabilitythatisrelatedwithTBD.Softclusteringassignsoneelementtomorethanoneclustersandeveryelementhasfractionalmembershipinseveralclusters.SoftClusteringisveryusefulinnumerousareas,likeparameterestimationformixtureofexponentialfamilydistributions(e.g.Gaussianmixturemodels,BinomialmixturemodelsandhiddenMarkovmodels).Also 43

PAGE 44

inmanycases,softclusteringismorereliableandaccuratethanhardclustering.Onepopularsoftclusteringalgorithmistheexpectation-maximization(EM)algorithm,whichhasbeenusedforndingmaximumlikelihoodestimationsofparametersinstatisticalmodels.EMisconceptuallyverysimple,easytoimplementandworkswellwhenthemissinginformationissmallandthedimensionofdataislow.However,theconvergenceratecanbeexcruciatinglyslowasapproachingalocaloptimaanditwillfailwhentheamountofmissingdataislargeorwhenthedimensionofdataishigh.TotalBregmansoftclusteringisaspecialkindofsoftclusteringwhichassignsanelementtoseveralclustersaccordingtotheprobabilitydeterminedbyTBD.ThestepsoftotalBregmansoftclusteringareverysimilartothoseofEMalgorithm,which,basicallyspeaking,iscomposedoftwoparts.Therstistoassigneveryelementtotheclustersbytheprobabilitythatmaximizestheintraclustersimilarityandminimizestheinnerclustersimilarity;thesecondistoupdatetheclustercentersandrelativeprobabilitiesofeachcluster.ThealgorithmfortotalBregmansoftclusteringisshowninAlgorithm 2 Algorithm2TotalBregmanSoftClusteringAlgorithm Input:X=fxigNi=1,numberofclustersc. Output:M=fmjgj=1andQ=fqjgj=1,mjistheclustercenterforthejthclusterwithprobabilityqj. Initialization:RandomlychoosecelementsfromXasMandtheircorrespondingprobabilitiesasQ. repeat fassignxitoclustersg fori=1toNdo forj=1tocdo q(jjxi) qjexp()]TJ /F9 7.97 Tf 6.59 0 Td[(f(mj,xi)) Pc~j=1q~jexp()]TJ /F9 7.97 Tf 6.59 0 Td[(f(m~j,xi)) endfor endfor fupdateclustercentersg forj=1tocdo qj 1 NPNi=1q(jjxi) mj t-centerforclusterj(Equation( 2 )) endfor untilThechangeoftheresultsbetweentwoconsecutiveiterationsisbelowsomesensitivitythreshold. 44

PAGE 45

TBDsoftclusteringconvergesveryfast,andcandoverywellinbothlowandhighdimensionalspace.WecomparedourTBDsoftclusteringalgorithmwithBregmansoftclusteringalgorithm[ 10 ]usingsyntheticexamplesandbyrealapplicationsofshaperetrieval. 4.2.2.1SyntheticexperimentsforsoftclusteringWedidfourexperimentsusingthesamedatasetsasBanerjeeetal.usedin[ 10 ].Therstoneisbasedonseveral1Ddatasetsof300sampleseach,generatedfrommixturesofGaussianandBinomialmodelsrespectively.Bothmixturemodelshadthreecomponentswithequalpriorscenteredat10,20and40.ThestandarddeviationoftheGaussiandistributionwassetto5andthenumberoftrialsoftheBinomialdistributionwassetto300soastomakethetwomodelssomewhatsimilartoeachother,intermsthatthevarianceisalmostthesameforallthemodels.Wealsousethesamemethodtogenerate2Dand5Ddatasetsandcomparethealgorithmsonthem.Theaccuracyofclusteringwasmeasuredbyusingthenormalizedmutualinformation(NMI)[ 113 ]betweenthepredictedclusters(thepredictedclusternumberis5)andoriginalclustersthatgeneratingthesamples,andtheresultswereaveragedover30trials.Table 4-1 liststheNMIresultedfromsoftclusteringusingBDandTBD.GaussianmixtureandBinomialmixturerepresentthemodelsthatgeneratedthedatasets.dGaussiananddBinomialrepresenttheBregmandivergenceforGaussianandBinomialdistributionswhileGaussianandBinomialrepresenttheTBDforGaussianandBinomialdistributions.Indetail, Gaussian(x1,x2)=f(x1))]TJ /F5 11.955 Tf 11.96 0 Td[(f(x2))-222(hx1)]TJ /F5 11.955 Tf 11.96 0 Td[(x2,rf(x2)i p 1+krf(x2)k2(4)f(x)=q(x)logq(x)andq(x)=1 p 2exp)]TJ /F4 7.97 Tf 10.49 5.47 Td[((x)]TJ /F9 7.97 Tf 6.59 0 Td[()2 22.Binomialisinthesameformatas( 4 )butq(x)=)]TJ /F7 7.97 Tf 5.52 -4.38 Td[(nxpx(1)]TJ /F5 11.955 Tf 11.95 0 Td[(p)n)]TJ /F7 7.97 Tf 6.58 0 Td[(x,andpistheprobabilityforasinglesuccess.ForTable 4-1 ,in( (a) ), (b) ),and (c) )),rows1and2correspondtotheNMIbetweentheoriginalandthepredictedclustersobtainedbyapplyingtheBregmanclustering 45

PAGE 46

(a) Measure GaussianMixture BinomialMixture dGaussian 0.736600.00142 0.549980.00035 dBinomial 0.543070.00066 0.729820.00379 Gaussian 0.750760.00193 0.550890.00018 Binomial 0.578990.00773 0.744500.00218 (b) Measure GaussianMixture BinomialMixture dGaussian 0.439220.03577 0.382460.05020 dBinomial 0.368050.05513 0.429870.04747 Gaussian 0.593850.11910 0.505180.07280 Binomial 0.521360.09187 0.570630.05173 (c) Measure GaussianMixture BinomialMixture dGaussian 0.398690.00049 0.382460.05020 dBinomial 0.197650.00025 0.142050.05619 Gaussian 0.523600.00018 0.415160.05320 Binomial 0.368830.00035 0.521640.04173 Table4-1. Theclusteringresultsforthe1D,2Dand5DdatasetsbyapplyingtheBregmanandTBDsoftclusteringalgorithms. algorithmusingtheBregmandivergencesdGaussiananddBinomial[ 10 ]respectively.Rows3and4correspondtotheNMIyieldedbytheTBDclusteringalgorithmusingGaussianandBinomialrespectively.ThenumbersinTable 4-1 illustratethatusingGaussiantoclusterdatasetsgeneratedbyGaussianmixtureandusingBinomialtoclusterdatasetsgeneratedbyBinomialmixturegivesbetterNMIthanusingdGaussiananddBinomial.Moreimportantly,usingGaussiantomeasurethedatasetsgeneratedbyBinomialmixturegivesmuchbetterNMIthanusingdGaussiantodothesamejob.ThisisalsotruewithBinomialanddBinomial.Thisisveryusefulinthereallife,becauseoftentimes,wedon'tknowthemodelthatgeneratesthedata,insteadwehavetoblindlychoosesomedivergencemeasure.ButonethingthatwearesurenowisthatTBDisalwaysbetterthanBregmandivergencewhenusingthesamegeneratingfunctions. 46

PAGE 47

~c dGaussian Gaussian 10 0.556270.00020 0.5563640.00034 8 0.584170.00122 0.585540.00109 6 0.599580.00112 0.702020.00154 5 0.736600.00142 0.750760.00193 3 0.861150.00141 0.986580.00120 Table4-2. NMIbeweentheoriginalclustersandtheclustersgotfromTBDandBDsoftclusteringalgorithms.~cisthepredictednumberofclusters. Remarks:FromTable 4-1 ,wecanseethatwiththedimensionincreasing,TBDsoftclusteringbecomesmoreandmoreaccuratethanBDclustering,andtheperformancedifferencebetweenTBDclusteringandBDclusteringbecomeslarger.Also,xthedimensionofdataandtheoriginalnumberofclustersc,andletthepredictedclusternumber~capproximatetoc,theNMIofTBDclusteringincreasesfasterthanthatofBDclustering.ItcanbeseenfromTable 4-2 usingGaussiangenerativemodelwheretheoriginalnumberofclustersc=3.Thispointisveryfundamentalinsegmentationintheprocessofpartitioninganimageintomultipleregionsorsetsandalsocantypicallybeusedtolocateobjectsandboundaries.FromTable 4-2 ,wecanseeTBDsoftclusteringbehavesverywelleventhepredictednumberofclustersisincorrect. 4.3TBDApplicationsShapeRetrievalThetaskofshaperetrievalistondthebestmatchfromadatabaseofshapestothequeryshape.Inthissection,weproposeanefcientandaccuratemethodforshaperetrievalthatincludesaneasytouseshaperepresentation,andananalyticalshapedissimilaritydivergencemeasure.Also,wepresentanefcientschemetosolvethecomputationallyexpensiveproblemencounteredwhenretrievingfromalargedatabase.Theschemeiscomposedofclusteringandefcientpruning,whichwillbeelaboratedoninSection 4.3.3 47

PAGE 48

4.3.1ShapeRepresentationAtimeandspaceefcientshaperepresentationisfundamentaltoshaperetrieval.Givenasegmentedshape(orabinaryimage),weuseamixtureofGaussians(GMM)[ 24 54 ]torepresentit.TheprocedureforobtainingtheGMMfromashapeiscomposedofthreesteps.First,weextractthepointsontheshapeboundaryorsurface(tomakeitrobust,foreachpointontheboundary,wealsopickeditsclosest2neighborsofftheboundary),sinceMPEG-7shapesarebinary,thepointsthathavenonzerogradientlieontheboundary(thisstepusesonelineMATLABcode).Aftergettingtheboundarypointsforeveryshape,weusetheafnealignmentproposedbyHoetal.[ 53 ]toalignthesepointstoremovetheeffectofrigidtransformations,e.g.,giventwosetsofpointsfxigmi=1andfyjgnj=1,wecanndafnealignment(A,b),A2GL(2)1,b2R2,suchthatg(A,b)=Piminjf(Axi+b)]TJ /F5 11.955 Tf 12.64 0 Td[(yj)2gachievesminimum,andthenweusethealignedfxijxi=Axi+bgmi=1torepresenttheoriginalpointsetfxigmi=1.Thisstepisalsoverysimpleduetotheexplicitsolutionof(A,b),anditonlytakesseverallinesofMATLABcodetoimplement.Finally,wecomputetheGMMfromthealignedboundarypoints.AparametricGMMisaweightedcombinationofGaussiankernels,whichcanbewrittenas p(x)=mXi=1aiN(x;i,i),0ai1,nXiai=1,(4)misthenumberofcomponents;N(x;i,i)istheGaussiandensityfunctionwithmeani,variancei,andweightaiinthemixturemodel.ThemixturemodelisobtainedthroughapplyingtheEMalgorithmanditerativelyoptimizingthecentersandwidthsoftheGaussiankernels.mshouldbeassmallaspossible,butmakethedeterminantofthecovarianceforeachcomponentnotlarge(wefoundthatm=10isagoodcompromiseforMPEG-7database).Theaboveprocessisportrayedusingtheow 1GL(2):Thesetof22invertiblematrices. 48

PAGE 49

chartshownbelow. Shape Boundarypointsextractionandalignment GMM SomeconcreteexamplesoftheapplicationoftheowchartareshowninFigure 4-2 .ThecolumnsfromlefttorightinFigure 4-2 areoriginalshapes,alignedboundaries,andGMMwith10components.ThedotinsideeachellipseisthemeanofthecorrespondingGaussiandensityfunction,andthetransverseandconjugatediametercorrespondtotheeigenvaluesofthecovariancematrix. 4.3.2ShapeDissimilarityComparisonUsingtSLAftergettingtheGMMrepresentationofeachshape,weusetSLtocomparetwoGMMs,andtakethedifferenceasthedissimilaritybetweenthecorrespondingshapes.NotethatthetSLfortwoGMMsisinclosedform(seebelow).SupposetwoshapeshavethefollowingGMMsp1andp2,p1(x)=mXi=1a(1)iN(x;(1)i,(1)i), (4)p2(x)=mXi=1a(2)iN(x;(2)i,(2)i). (4)SinceZN(x;1,1)N(x;2,2)dx=N(0;1)]TJ /F3 11.955 Tf 11.96 0 Td[(2,1+2),wecanarriveat tSL(p1,p2)=R(p1)]TJ /F5 11.955 Tf 11.95 0 Td[(p2)2dx q 1+R4p22dx=d1+d2)]TJ /F5 11.955 Tf 11.95 0 Td[(d1,2 p 1+4d2,(4) 49

PAGE 50

Figure4-2. Lefttoright:originalshapes;alignedboundaries;GMMwith10components. 50

PAGE 51

whered1=mXi,j=1a(1)ia(1)jN(0;(1)i)]TJ /F3 11.955 Tf 11.96 0 Td[((1)j,(1)i+(1)j), (4)d2=mXi,j=1a(2)ia(2)jN(0;(2)i)]TJ /F3 11.955 Tf 11.96 0 Td[((2)j,(2)i+(2)j), (4)d1,2=2mXi,j=1a(1)ia(2)jN(0;(1)i)]TJ /F3 11.955 Tf 11.96 0 Td[((2)j,(1)i+(2)j), (4)whereN(0;,)=1 (p 2)ep det(),andeisthedimensionof.GivenasetofGMMsfplgnl=1,pl=Pmi=1a(l)iN(x;(l)i,(l)i),itst-centercanbeobtainedfromequation( 2 ),whichis p=Pnl=1wlpl Pnl=1wl,wl=(1+4dl))]TJ /F4 7.97 Tf 6.59 0 Td[(1=2,(4) dl=mXi,j=1a(l)ia(l)jN(0;(l)i)]TJ /F3 11.955 Tf 11.96 0 Td[((l)j,(l)i+(l)j).(4)WeevaluatethedissimilaritybetweentheGMMofthequeryshapeandtheGMMsoftheshapesinthedatabaseusingtSL,andthesmallestdissimilaritiescorrespondtothebestmatches. 4.3.3ShapeRetrievalinMPEG-7DatabaseTheproposeddivergenceisevaluatedontheshaperetrievalproblemusingtheMPEG-7database[ 63 ],whichconsistsof70differentobjectswith20shapesperobject,foratotalof1400shapes.Thisisafairlydifcultdatabasetoperformshaperetrievalbecauseofitslargeintraclassvariability,and,formanyclasses,smallinterclassdissimilarity,andfurthermore,therearemissingpartsandocclusionsinmanyshapes.Weclusterthedatabaseintohierarchicalclusters,calculatetheirt-centersandcomparethequeryshapewiththet-centershierarchically.Fortheclusteringpart,weappliedbothhardclusteringandsoftclustering.Forhardclustering,weapplyavariationofk-treemethodbysettingk=10attherstlevelofclustering,7atthesecondlevel,5atthethirdleveland2atallfollowinglevels,sotheaveragenumberofshapesineach 51

PAGE 52

clusteris140,20,4,2,and1.Forsoftclustering,weappendasemi-hardassignmenttothesoftclusteringalgorithm,i.e.,afterthesoftclusteringconverges,wewillassigntheshapexitoclusterCj,ifp(Cjjxi)a.Weusea=1=2,sothatoneshapecanbeassignedtoatmost2clusters,buttheclustercentermaybedependentonxieventhoughxiisnotassignedtothatclusternally.Theclusteringprocessisacoarsetoneprocedure,whichgreatlyenhancesefciencywhileguaranteeingaccuracy.Also,wecomparetheclusteringaccuracyoftSL,2andSLsoftandhardclusteringbyareasonablemeasure,whichistheoptimalnumberofcategoriespercluster(denotedbyjCj,jCjrepresentsthecardinalityofC,i.e.,thenumberofcategoriesinC)dividedbytheaveragenumberofcategoriesineachcluster(denotedbyAvg(jCj)).Forexample,attherstlevelclustering,thereare10clustersfCig10i=1,withanaverageof140shapespercluster,andthus,jCj=140=20=7;Avg(jCj)=P10i=1jCij 10.Thesmallernumberofcategoriespercluster,thehighertheclusteringaccuracyis,andthemoreaccuratelywillthedifferentcategoriesbeseparated.Theoptimalclusteringaccuracyis1.Figure 4-3 comparestheclusteringaccuracyoftSL,2andSLsoftandhardclustering,whichshowsthattSLsoftclusteringhasastrikingclusteringaccuracy,implyingsubstantialcapabilitytodetectoutliers,occlusion,missingparts,andstrongabilitytodistinguishshapesfromdifferentcategories.WeputhereseveralgroupsofretrievalresultsinFigure 4-4 ,therstshapeineachgureisthequery,andtheothershapesareshownfromlefttoright,uptodownaccordingtotheascendingorderofthedivergencetothequery.Theresultsshowthatourmethodcandealverywellwithscale,rotation,pose,occlusion,missingparts,greatintraclassdissimilarityandlargeinterclasssimilarity.TheevaluationofaccuracyforretrievinginthewholeMPEG-7databaseisbasedonthewellrecognizedcriterion,recognitionrate[ 34 63 76 96 ].Eachshapeisusedasaqueryandthetop40matchesareretrievedfromall1400shapes.Themaximum 52

PAGE 53

Figure4-3. ComparisonofclusteringaccuracyoftSL,2andSL,versusaveragenumberofshapespercluster. possiblenumberofcorrectretrievalsforeachqueryis20,andhencethereareatotalof28,000possiblematcheswiththerecognitionratereectingthenumberofcorrectmatchesdividedbythistotal.Table 4-3 liststherecognitionrateweobtainedandcomparisonwithsomeotherexistingtechniques.Notethatourmethodgivesveryhighrecognitionrate,eventhoughitisnotasgoodas[ 7 115 ],butourmethoddoesnotneedanypreprocessingoftheshapesoranypostprocessingofthesimilarities.Thesimplicityandspeedofourmethodareincomparable. 53

PAGE 54

A B C Figure4-4. Retrievalresultsusingourproposedmethod. 54

PAGE 55

Technique Recognitionrate(%) GMM+softclustering+tSL 93.41 GMM+hardclustering+tSL[ 70 ] 89.1 Shape-tree[ 34 ] 87.7 IDSC+DP+EMD[ 67 ] 86.56 HierarchicalProcrustes[ 76 ] 86.35 IDSC+DP[ 66 ] 85.4 ShapeL'^AneRouge[ 96 ] 85.25 GenerativeModels[ 121 ] 80.03 CurveEdit[ 105 ] 78.14 SC+TPS[ 15 ] 76.51 VisualParts[ 63 ] 76.45 CSS[ 80 ] 75.44 PerceptualS.+IDSC+LCDP[ 115 ] 95.60 IDSC+MutualGraph[ 60 ] 93.4 IDSC+LCDP+unsupervisedGP[ 137 ] 93.32 IDSC+LCDP[ 137 ] 92.36 IDSC+LP[ 8 ] 91.61 ContourFlexibility[ 136 ] 89.31 Perceptual+IDSC[ 115 ] 88.39 SC+IDSC+Co-Transduction[ 7 ] 97.72 Table4-3. RecognitionratesforshaperetrievalintheMPEG-7database. 4.3.4BrownDatabaseAdditionally,weapplyourproposedmethodtotheBrowndatabase[ 106 ],whichcontains9shapecategories,whereeachshapecategoryhas11differentsegmentedbinaryshapesand99shapesintotal.WeuseGMMtorepresenteachshape,thenumberofcomponentsforeachGMMisdecidedusingthesamewayasintheMPEG-7experiment,andcomparethedifferenceofshapesusingthetSLbetweentheircorrespondingGMMs.Wetestedourmethodusingthecriteriaasin[ 8 34 106 115 121 ]:everyshapeistakenasthequery,andcompareitwithalltheshapesinthedatabase,ndthebest10matches,andcheckthenumberofcorrectmatches,i.e.,thenumberofshapeswhichbelongstothesamecategoryasthequeryshape.Thisprocessisrepeatedbytakingeachoneofthe99shapesinthewholedatasetasthequeryshape.Thenwecheckthetotalcorrectmatchesfortheithshape, 55

PAGE 56

Technique 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th GMM+tSL 99 99 99 99 99 99 99 99 99 99 PerceptualStrategy+IDSC+LCDP[ 115 ] 99 99 99 99 99 99 99 99 99 99 IDSC+LP[ 8 ] 99 99 99 99 99 99 99 99 97 99 Shape-tree[ 34 ] 99 99 99 99 99 99 99 97 93 86 IDSC[ 67 ] 99 99 99 98 98 97 97 98 94 79 Shock-GraphEdit[ 106 ] 99 99 99 98 98 97 96 95 93 82 GenerativeModels[ 121 ] 99 97 99 98 96 96 94 83 75 48 Table4-4. RecognitionratesforshaperetrievalfromtheBrowndatabase. i=1,2,,10(themaximumnumberofcorrectmatchesis99),whichareshowninTable 4-4 .Wecanseethatourmethodgivesperfectresult. 4.3.5SwedishLeafDataSetTheSwedishleafdataset[ 110 ]containsisolatedleavesfrom15differentSwedishtreespecies,with75leavesperspecies,with1125shapesintotal.Weusetheclassicationcriteriaasin[ 34 67 115 137 ],whichusedthe1-nearest-neighborapproachtomeasuretheclassicationperformance.Foreachleafspecies,25samplesareselectedasatemplateandtheother50areselectedastargets.WeuseGMMtorepresenteachshape,anduseTBDsoftclusteringalgorithmtoclustertheshapesintodifferentclusters.ShapeclassicationresultsonthisdatasetareshowninTable 4-5 ,fromwhichwecanseethatouraccessibleshaperepresentationplusTBDsoftclusteringalgorithmgivesthebestresults. Technique recognitionrate(%) GMM+softclustering+tSL 98.33 GMM+hardclustering+tSL 97.92 PerceptualStrategy+IDSC+LCDP[ 115 ] 98.27 IDSC+LCDP[ 137 ] 98.20 Shape-tree[ 34 ] 96.28 IDSC[ 67 ] 94.13 Table4-5. RecognitionratesforshaperetrievalfromtheSwedishleafdatabase. 56

PAGE 57

4.3.63DPrincetonShapeBenchmarkOurmethodperformedverywellinthedomainof2Dshaperetrievalanditcanbeextendedveryeasilytohigherdimensionalspace.WeevaluateourmethodonthePrincetonShapeBenchmark(PSB)[ 107 ]containing18143Dmodels,whichisdividedintothetrainingset(907modelsin90classes)andthetestingset(907modelsin92classes).Weevaluateourmethodonthetestingset,andcompareourresultswithothersinthreeways,NearestNeighbor(NN),DiscountedCumulativeGain(DCG)andNormalizedDCG(NDCG)usingthesoftwareprovidedinPSB[ 107 ].OurmethodoutperformsallothermethodswhenusingNNcriteria,andcanndtherstclosestmatchesthatbelongtothequeryclassmoreaccurately. Technique ours CRSP DSR DBF DBI SIL D2 NN 72.3 67.9 66.5 68.6 60.9 55.7 31.1 DCG 66.7 66.8 66.5 65.9 61.4 59.7 43.4 NDCG 16.1 16.4 15.9 14.9 7 4.1 -24.4 Table4-6. Retrievalcomparisonwithothermethods(CRSP[ 92 ],DSR[ 2 ],DBF[ 2 ],DBI[ 126 ],SIL[ 126 ],D2[ 89 ])onPSB. 4.4DiscussionsThischapterpresentsTBDhardandsoftclusteringalgorithms.WeuseGaussianmixturemodels(GMMs)torepresentshapes,clustertheshapesintosubclustersandstoretheshapesusingk-tree.TheclusteringisquitefastbecausetheclustercenterisinclosedformandisonlydependentontheGMMmeansandvariances.Thek-treemakesitefcienttoretrievalwithlogarithmiccomparisons,andfurthermore,eachcomparisonisvery-fastbecausetheTBDbetweentwoGMMsalsohasexplicitform.Ourmethodisfastandspaceefcient,canbeappliedtolowandhighdimensionalspaces,noneedtodoanypreprocessingabouttheshapes,oranypostprocessingonthedissimilarities,andisrobusttorigidtransformations,outliers,pose,occlusion,missingparts,andhasbetterorsimilarresultscomparedwiththosefromthestate-of-the-arttechniques. 57

PAGE 58

CHAPTER5ROBUSTANDEFFICIENTREGULARIZEDBOOSTINGUSINGTOTALBREGMANDIVERGENCEBoostingisawellknownmachinelearningtechniqueusedtoimprovetheperformanceofweaklearnersandhasbeensuccessfullyappliedtocomputervision,medicalimageanalysis,computationalbiologyandotherelds.Severalboostingmethodshavebeenproposedintheliterature,suchasAdaBoost,GentleBoost,SoftBoost,BrownBoost,LPBoost,RBoostandtheirvariations.Acriticalstepinboostingalgorithmsinvolvesupdateofthedatasampledistribution,however,mostexistingboostingalgorithmsuseupdatingmechanismsthatleadtooverttingandinstabilitiesduringevolutionofthedistributionwhichinturnresultsinclassicationinaccuracies.Regularizedboostinghasbeenproposedinliteratureasameanstoovercomethesedifculties.Inthischapter,weproposeanoveltotalBregmandivergence(tBD)regularizedLPBoost,termedtBRLPBoost[ 69 ].tBDwasdevelopedinChapter 2 ,whichisstatisticallyrobustandweprovethattBRLPBoostrequiresaconstantnumberofiterationstolearnastrongclassierandhenceiscomputationallymoreefcientcomparedtootherregularizedBoostingalgorithmsinliterature.Also,unlikeotherboostingmethodsthatareonlyeffectiveonahandfulofdatasets,tBRLPBoostworkswellonavarietyofdatasets.WepresentresultsoftestingouralgorithmonmanypublicdomaindatabasesincludingtheUCImachinelearningrepository,theOASISMRIbraindatabase,theCNSembryonaltumordataset,theColontumordataset,theLeukemiacancerdatasetandtheEpilepsydataset.Wealsopresentcomparisonstoseveralotherstate-of-the-artmethods.Numericalresultsshowthattheproposedalgorithmhasmuchimprovedperformanceinefciencyandaccuracyoverothermethods. 5.1IntroductiontoBoostingClassicationisaveryimportanttaskinnumerousareasincludingbutnotlimitedtocomputervision,patternrecognitionandimageprocessing.Ensembleclassiershave 58

PAGE 59

beeninvougeforhardclassicationproblemswhereuseofsingleclassiershavenotbeenverysuccessful.Boostingissuchanensembleclassicationtool.Itgeneratesastrongclassierthroughalinearconvexcombinationofweakclassiers.Aweakclassierisusuallyonlyrequiredtobebetterthanrandomguessing.However,thestrongclassiergreatlybooststheperformanceoftheweakclassiersandperformsquitewellonthewholedataset.Therefore,Boostinghasbecomeaverypopularmethodtoimprovetheaccuracyofclassicationandhasbeenwidelyusedincomputervision,patternrecognitionandotherareas,forexample,inimageandobjectcategorization[ 33 68 95 102 ],retrieval[ 117 ],tracking[ 87 139 ],facedetection[ 125 ],andobjectrecognition[ 11 88 118 ]etc.Themainideabehindtheboostingalgorithmisthatateachiteration,itwilllearnaweakclassierfromthetrainingsamplesthatfollowadistribution.Theweightedweakclassieristhenaddedtothestrongclassier.Thisweightistypicallyrelatedtotheweakclassier'saccuracy.Thehighertheaccuracy,thelargertheweight,andviseversa.Aftertheweakclassierisaddedtothestrongclassier,thedistributionofthesamplesisupdated.Thedataisreweightedfollowingtherulethatsamplesthataremisclassiedtendtogainweightandsamplesthatareclassiedcorrectlytendtoloseweight.Thustheweightwillbeverylargeifthesamplehasbeenmisclassiedbymanypreviouslylearnedweakclassiers,andtheweightwillbeverysmallifthesamplehasbeenclassiedcorrectlybymanypreviouslylearnedweakclassiers.Therefore,thefutureweakclassierstobelearnedwillbefocusedmoreonthesamplesthatmanypreviousweakclassiersmisclassied.Thismethodhasbeenproventobeeffectiveinvariousclassicationproblemsandthusmotivatedalotofresearchinthemachinelearningcommunity.SchapireandSinger[ 104 ]proposedAdaBoostminimizingtheexponentialhingeloss,followedbytheinceptionofLPBoost[ 29 ]whichmaximizestheminimummarginbetweenclasses,andothersproposedmanyvariationsandimprovementstoAdaBoost,namelyTotalBoost,SoftBoost,LogitBoost,GentleBoost 59

PAGE 60

[ 46 ],andentropyregularizedLPBoost.Theseaforementionedboostingalgorithmsaremainlyforbinaryclassication,i.e.,thedatabasecontainsonlytwoclasses.Tosolvethemulticlassclassicationproblems,severalresearchersextendedthebinaryboostingalgorithmstothemulticlasscase,seeforexample,[ 45 141 ].Asthedemandforclassifyinglargescaledatasets,dynamicenvironmentsandmultimediadatasetsincreased,variousonlineboostingtechniqueswereproposed[ 6 75 101 ]andusedonfacerecognitionandtrackingamongotherapplications.Inthischapter,wepresentaregularizedLPBoostthatisbasedonarecentlyintroducedrobustdivergencemeasure,forbinaryclassicationthatcaneasilybegeneralizedtothemulticlasscase.ThisdivergencemeasureiscalledtotalBregmandivergence(tBD)whichisbasedontheorthogonaldistancebetweentheconvexgeneratingfunctionofthedivergenceanditstangentapproximationatthesecondargumentofthedivergence.tBDisnaturallyrobustandleadstoefcientalgorithmsforsoftandhardclustering.Formoredetails,wereferthereaderto[ 123 ].BasedonearlierworkonanupperboundforthenumberofupdateiterationsintheentropyregularizedLPBoostalgorithm(ELPBoost),byWarmuthetal.[ 131 ],wepresentaconstantboundonthenumberofiterationsforourtBD-regularizedboostingalgorithm(tBRLPBoost).WealsoshowempiricalresultsthatdepicttheefciencyofourproposedclassicationalgorithmincomparisontoELPBoostandothers.Finally,duetoitscomputationalefciency,ourboostingmethodisapromisingcandidateforonlineclassication.Therestofthechapterisorganizedasfollows.InSection 5.2 ,webrieyreviewtheboostingliterature.Insection 5.3 ,weintroduceouralgorithm,thetotalBregmandivergenceregularizedLPBoost,dubbedastBRLPBoost,andinvestigateitsproperties,likerobustnessandefciency.Insection 5.4 ,weinvestigatethesamepropertiesexperimentally.Inthesamesection,wealsovalidate/testouralgorithmonanumberofdatasetsandcomparetheresultswithstate-of-the-artboostingalgorithmsandother 60

PAGE 61

competitiveclassiers.Finally,weconcludethechapteranddiscusspossiblefutureworkinsection 5.5 5.2PreviousWorkIntheliteratureofboosting,numeroustechniquesexistwhichsolvevariousoptimizationproblems.AdaBoostistheoriginforthemodernboostingalgorithms.Itsetsupaframeworkforallthefollowingboostingtechniques.AdaBoostperformsverywellinmanycases,however,itdoesnotdowellonnoisydatasets[ 31 ].BrownBoost[ 44 ]howeverovercomesthisnoiseproblem.BecauseinBrownBoost,itisassumedthatonlynoisyexampleswillbemisclassiedfrequently,thusthesamplesthatarerepeatedlymisclassiedaretakenasnoisydataandareremoved.Inthisway,thenalclassierislearnedfromthenoise-freesamples,andtherefore,thegeneralizationerrorofthenalclassiermaybemuchsmallerthanthatlearnedfromnoisyandnon-noisysamples.BrownBoostworkswellonavarietyofproblems,butitdoesnotmaximizethemarginbetweendifferentclasseswhichlimititsperformanceinmanycases.AnotherversionofboostingistheLPBoost[ 29 ]which,withasoftmarginisalinearprogrammingalgorithmthatcanmaximizetheminimumsoftmarginbetweendifferentclasses,andperformverywellonnaturaldata.However,itdoesnothaveanyiterationupperboundandsometimes,itrequireslineartime(O(N),Nisthesizeofthetrainingdataset)togetagoodstrongclassier,whichiscomputationallyexpensivewhenthedatasetislarge.SoftBoost[ 130 ]ontheotherhandcanmaximizetheminimummarginuptoaccuracy,withO(logN=2)numberofiterations.Thisisagreatimprovementexceptitsuffersfromthecommonannoyingproblemofslowstart,whichmakesSoftBoostratherunsatisfactory.Morerecently,theentropyregularizedLPBoost(ELPBoost)wasintroducedin[ 131 ],whichisaregularizedversionofLPBoost.Itusesrelativeentropy,theKullback-Leiblerdivergence(KL)betweentheupdateddistributionandtheorginaldistributiontoregularizetheconventionalLPBoost. 61

PAGE 62

ELPBoostovercomestheslowstartissueanditperformsaswellasLPBoostonnaturaldatasetswithaniterationboundofO(logN=2).Nevertheless,itisnotrobusttonoisydatasetsandO(logN=2)isstillanexpensivecomputationalrequirement.Toavoidalltheaforementionedproblems,wepresentarobustandefcientboostingalgorithminthischapter.Weshowthatithasconstantupperboundforthenumberofiterationsrequiredtoachieveagoodstrongclassier.ThealgorithmusesthetotalBregmandivergence(tBD)toregularizeLPBoost.tBDwasrecentlyproposedin[ 70 123 ]andithasthreesalientfeatures.First,itisinvarianttorigidtransformationsofthecoordinatesystemusedtospecifytheconvexgeneratingfunction.Secondly,itisintrinsicallyrobusttonoiseandoutliers.Finally,therepresentativeofasetofobjects(whethertheyarescalars,vectorsorfunctions)hasaclosedformexpression,whichmakesitcomputationallyattractive.Thesepropertieshavebeenveriedintheapplicationsofimageclustering,retrieval[ 70 ]andmedicalimagesegmentation[ 123 ].ThetBDassociatedwitharealvaluedstrictlyconvexanddifferentiablefunctionfdenedonaconvexsetXbetweenpointsd,~d2Xisdenedby, f(d,~d)=f(d))]TJ /F5 11.955 Tf 11.95 0 Td[(f(~d))-222(hd)]TJ /F8 11.955 Tf 12.48 2.66 Td[(~d,rf(~d)i q 1+krf(~d)k2,(5)where,h,idenotesthestandardinnerproduct,rf(y)isthegradientoffaty,andkrf(y)k2=hrf(y),rf(y)i.tBDisaclassofdivergenceswhichhasthespecicformatasin( 5 ).Inthischapter,wewillfocusonthetotalKullback-Leibler(tKL)divergenceanduseittoregularizetheLPBoost.WeshowthatthemaximumnumberofiterationsforthistBRLPBoosttolearnastrongclassierisaconstant.Inotherwords,theiterationupperboundisindependentofthenumberofthetrainingsamples,whichmakesiteasilyscalableandcomputationallyefcientincomparisontotheexistingmethods.Also,thisalgorithmcanperformverywellonnoisydatasetsduetotheintrinsicrobustnessoftBD. 62

PAGE 63

5.3tBRLPBoost:TotalBregmanDivergenceRegularizedLPBoosttBRLPBoostusestBDtoregularizetheconventionalLPBoost.Thepurposeofregularizationistomaketheboostingalgorithmconvergequicklyandsmoothly,andincreaseitsrobustnesstonoiseandoutliers.Wenowbrieypresentamathematicaldescriptionoftheconventionalboosting.GiventheinputXY,whereXisthedomainandYistherange,thegoalistolearnafunctionH:X7!Y.Inthebinaryclassicationproblem,Xisasetoffeaturevectors,andY=f)]TJ /F8 11.955 Tf 15.27 0 Td[(1,1g.Thetrainingsamplesaref(xn,yn)gNn=1,xn2Xisthefeaturevector,andyn2f1,)]TJ /F8 11.955 Tf 9.3 0 Td[(1gistheclasslabelforxn.Giventhetrainingsamples,thetaskofboostingistolearnthestrongclassierH(x)=sign(PTt=1wtht(x))toapproximateH.Here,htbelongstotheweakclassierspaceH,andht:X7!R.Inaddition,htislearnedatthetthiterationwithrespecttothedistributiondt)]TJ /F4 7.97 Tf 6.59 0 Td[(1.Furthermore,sign(ht(xn))predictstheclasslabelforxn,andjht(xn)jisthecondencefactorintheprediction.wtistheweightforht,andTisthenumberofweakclassiers.Giventhedistributionofthetrainingsamplesdt)]TJ /F4 7.97 Tf 6.58 0 Td[(1,theaccuracyoftheweakclassierhtisgivenbyt=PNn=1dnt)]TJ /F4 7.97 Tf 6.59 0 Td[(1sign(ynht(xn)).Foreaseofexpositionandavoidnotationclutter,weintroduceavectorutandcallittheedge.Welet utn=sign(ht(xn)yn),(5)i.e.,utmeasurestheaccuracyoftheweakclassierht.utn2f)]TJ /F8 11.955 Tf 26.75 0 Td[(1,1g,andut=1implieshtisaperfectclassierwhileut=)]TJ /F6 11.955 Tf 9.3 0 Td[(1implieshtisaverypoorclassier.Nowwecanrewritetheaccuracytofhtas t=NXn=1dt)]TJ /F4 7.97 Tf 6.59 0 Td[(1nutn=dt)]TJ /F4 7.97 Tf 6.58 0 Td[(1ut.(5)Duringboosting,ifthesamplesarelinearlyseparable,wewillmaximizethehardmarginbetweendifferentclasses.Thehardmarginisthewidthoftheareaseparatingthepositivefromthenegativesamples.Bymaximizingthehardmargin,theclassicationaccuracyofthestrongclassieronthetestingdatasetwillprobablybe 63

PAGE 64

maximized.TheLPBoostformulationtomaximizethehardmarginatiterationtisgivenby, maxw,s.t.tXi=1uin!i,n=1,,N,w2t,(5)where,istheminimumhardmarginbetweenthetwoclasses.tisthet-simplexandw2timpliesthatPtj=1wj=1andwj0,forj=1,,t.Theeffectofconstrainingw2tistopreventwfromscaling.Inotherwords,ifweremovetheconstraintw2t,thenwwillscaleto1makingtendto1.Whenthesamplescannotbelinearlyseparated,wecannotndahardmarginthatclearlyseparatesthepositivefromthenegativeclasses.Inthiscase,softboostingisanalternativechoice,wherewemaximizethesoftmarginateachiterationandallowsamplestofallbelowthemargin.Samplescanfallbelowthemarginuptosomeslackfactors,butwecanlevyapenaltyforfallingbelowthemargintomakesuretheslackfactorsdon'tbecomeextremelylarge.LPBoosttomaximizethesoftmargincanbeexpressedbythefollowingfunction, maxw,,)]TJ /F5 11.955 Tf 11.95 0 Td[(DNXn=1ns.t.tXi=1uinwi)]TJ /F3 11.955 Tf 11.96 0 Td[(n,n=1,,N,w2t,0,(5)where,istheslackvariablevector,andDistheconstantfactorwhichpenalizestheslackvariables.IfDisverylarge,say1,then( 5 )becomesthehardmarginmaximizationproblem( 5 );ifDissmallenough,thenitwillalwaysleadstofeasiblesolutionsfor( 5 ). 64

PAGE 65

Thedualproblemof( 5 )atsteptminimizesthemaximumaccuracyresultingfromtheweakclassierslearnedthusfar,andisgivenby, mind2N,dD1maxi=1,,tuid(5)NotethataddingtheweightDtotheslackvariablesoftheprimal( 5 )resultsindD1.Tomakesuchadexist,weshouldrequireD1=N.Itwasshownin[ 131 ]thatD=1=s,ands2f1,,Ngisafavorablechoice.LPBoostworkswellandhasbeenextensivelyused,however,itconvergesveryslowly,andthenumberofiterationsforthealgorithmisatbestO(logN)withalargeconstantfactor.WhenNislarge,itwillbecomputationallyexpensivetouseLPBoost.Also,theevolutionofdmighthaveseriousinstabilities,whichreducestheefciencysignicantlyandmightresultinovertting.Toovercomethesedownsides,weaddaregularizationtermbasedonthetotalKullback-Leibler(tKL)divergenceto( 5 ).Thisregularizationmakestheevolutionofdsmooth,andmoreinterestinglythenumberofiterationswillbereducedtoaconstant.Atthesametime,thesoftmargincanbemaximizedwithoutloss.TheregularizedLPBoost(tBRLPBoost)is mind2N,dD1(maxi=1,,tuid+(d,d0))(5)(.,.)isthetKLand (d,d0)= NXn=1dnlogdn d0n!,vuut 1+NXn=1d0n(1+logd0n)2(5)where,d0istheinitializeddistribution,whichissettoauniformdistribution,i.e.,d0n=1=N,n=1,,N.>0istheregularizationparameterforthetKLcomparedwiththemaximumaccuracy.When=0,theregularizationtermwillvanish,and( 5 )willbecome( 5 )maximizingthesoftmarginofLPBoost. 65

PAGE 66

Ateachiteration,tBRLPBoostwillcomputetheweakclassierht,alongwithitscorrespondingweightwtinbuildingthestrongclassier,andbasedontheperformanceoftheweakerclassier,tBRLPBoostwillupdatethedistributionofthetrainingsamples.Thedistributionisupdatedfromdt)]TJ /F4 7.97 Tf 6.58 0 Td[(1todt.ThealgorithmfortBRLPBoostisdepictedinAlgorithm 3 Algorithm3TotalBregmandivergenceregularizedLPBoost Input:f(xn,yn)gNn=1,xn2X,andyn2f1,)]TJ /F8 11.955 Tf 9.3 0 Td[(1g, Output:H(x)=sign(PTt=1wtht(x)),wt>0,fht(x)gTt=1aretheweakclassierstobelearned,Tisthenumberofiterations/weakclassiers. Initialization:d0n=1=N,n=1,,N fort=1toTdo fFindtheweakclassierg ht argmaxhPNn=1dt)]TJ /F4 7.97 Tf 6.58 0 Td[(1nsign(ynh(xn)). Updatethedistributionfromdt)]TJ /F4 7.97 Tf 6.59 0 Td[(1todtaccordingto( 5 )andtheweightwfortheweakclassiersaccordingto( 5 ) endfor ReturnH(x)=sign(PTt=1wtht(x)) Tomaketheboostingalgorithmevolvetoagoodstrongclassiermorequickly,someresearchersimposetheconstraintthattheweakclassiersfhtgTt=1shouldbelinearlyindependenttoremoveredundancy,i.e.,hthi=0,i=1,,t)]TJ /F8 11.955 Tf 12.82 0 Td[(1.Thiswillguaranteethattheredundancyfactoristheleastamongtheweakclassiersandthusleadingtohighefciency.However,thisisrathercomplicatedtoimplement,andinaddition,ourtBRLPBoostisalreadyabletoachieveveryhighefciency,sowedonotimposethiscondition. 5.3.1ComputationofdtandwTodirectlycomputedtfrom( 5 )iscomplicated,instead,wewillrstgettheLagrangiandualof( 5 )andaccordinglycomputedt.TondtheLagrangiandual,we 66

PAGE 67

rewrite( 5 )intothefollowingform min,d+(d,d0)s.t.uid,i=1,,td2N,dD1.(5)Now,theLagrangianof( 5 )iseasytosee,andgivenby, (d,,w,,)=+(d,d0)+tXi=1wi(uid)]TJ /F3 11.955 Tf 11.95 0 Td[()+NXn=1n(dn)]TJ /F5 11.955 Tf 11.95 0 Td[(D)+(d1)]TJ /F8 11.955 Tf 11.95 0 Td[(1).(5)where,wi,i=1,,t,n,n=1,,Nandarenon-negativeregularizers.Differentiatingwithrespectto,weget @ @=1)]TJ /F7 7.97 Tf 18.68 14.95 Td[(tXi=1wi=0,(5)andbyenforcingPti=1wi=1manually(donebynormalizingw),wecanremovefromtheLagrangiandualfunction( 5 ).Alsosince, @ @=d1)]TJ /F8 11.955 Tf 11.96 0 Td[(1=0,(5)byenforcingd1=1,wecaneliminatefrom( 5 ).Moreover,accordingtotheKKTcondition[ 17 ],n(dn)]TJ /F5 11.955 Tf 12.64 0 Td[(D)=0.Therefore,wecansimplify( 5 )andgetthepartialLagragian (d,w)=(d,d0)+tXi=1wiuid.(5)Nowdifferentiatingwithrespecttod,settingitto0,andnormalizingd,wecanget, dtn=d0nexp)]TJ /F2 11.955 Tf 5.48 -9.68 Td[()]TJ /F5 11.955 Tf 9.3 0 Td[(cPti=1uinwi Zt,(5) 67

PAGE 68

where,c=1 q 1+PNn=1d0n(1+logd0n)2,andZtisthenormalizationparametertomakePNn=1dtn=1.Theweightvectorwfortheweakclassiersshouldsatisfythelinearprogrammingproblem( 5 ),andthiscanbesolvedusingcolumngeneration[ 29 ]oragradientbasedmethod.Weusedthedelayedcolumngenerationtechniquetosolvew.Delayedcolumngeneration[ 29 ]isanefcientalgorithmtosolvelinearprogrammingproblems.Itutilizesthedivideandconquerstrategybysplittingthegoalproblemintosmallerproblems.Bysolvingthesmallerproblemsthroughtheirduals,itndsthesolutiontothebiggerproblem.Foradetailedexplanationaboutdelayedcolumngenerationthereaderisreferredto[ 29 ]. 5.3.2BoundingtheNumberofIterationsLetthetolerancebe,i.e.,weallowthemaximumsoftmarginresultingfromtBRLPBoosttobedifferentfromtheoptimalsoftmarginby.Iftheregularizationparameterin( 5 )issettop 1+(logN)]TJ /F4 7.97 Tf 6.58 0 Td[(1)2 2log(ND),thenthenumberofiterationsfortBRLPBoostisatmostb=32p 2=2+1. Theorem5.1. Letin( 5 )bep 1+(logN)]TJ /F4 7.97 Tf 6.59 0 Td[(1)2 2log(ND),thenthetBRLPBoostwillconvergeinconstant'b'numberofiterationsgivenbyb=32p 2=2+1. Proof. Sincetheinitializeddistributiond0issettod0n=1=N,n=1,,N,therefore,q 1+PNn=1d0n(1+logd0n)2=p 1+(logN)]TJ /F8 11.955 Tf 11.95 0 Td[(1)2.Alsoaccordingto( 5 ),dnD,n=1,,N,thus (d,d0)log(ND) p 1+(logN)]TJ /F8 11.955 Tf 11.96 0 Td[(1)2.(5)Consequently,if =p 1+(logN)]TJ /F8 11.955 Tf 11.95 0 Td[(1)2 2log(ND),(5)wehave (d,d0) 2.(5) 68

PAGE 69

Thismeansthatthedifferencebetweentheoptimalmarginoftheregularizedproblem( 5 )isatmostdifferentfromthemarginoftheunregularizedproblem( 5 ).Usingsimilarproofasin[ 131 ],wecansaythatthealgorithmwillconvergein T16 +1=32log(ND) 2p 1+(logN)]TJ /F8 11.955 Tf 11.95 0 Td[(1)2+1.(5)Sincep 1+(logN)]TJ /F8 11.955 Tf 11.96 0 Td[(1)2logN=p 2,N2Z+,andD1,thus,settingD=1,weget T32p 2=2+1.(5)From( 5 ),D=1meansthereisnoconstraintonthedistribution,inotherwords,dcangettoanypointintheN-simplex,sothealgorithmcanachievetheoptimum. TheupperboundforthenumberofiterationsfortBRLPBoostis32p 2=2+1,whichisaconstantandhenceindependentofthesizeofthetrainingdataset.Actually,thenumberofiterationsinrealapplicationscanbemuchsmallerthan32p 2=2+1.BecauseDisusuallynotsetto1manually,instead,itislearnedduringthetrainingandissettobethenumberthatcanmaximizetheclassicationaccuracyonthetrainingdataset.Therefore,Dcanbemuchsmallerthan1,Hence,theiterationnumberwillbemuchsmaller.Therefore,thealgorithmwillconvergeinmuchfeweriterations.Thiswasveriedinourexperiments. 5.3.3WeakClassiersThetypeofweakclassiersisveryimportantindeterminingtheaccuracyofthenalstrongclassier.Intheboostingcommunity,therearemanydifferentkindsofpopularweakclassiers,suchasdecisionstump,RIPPER[ 25 ],SLIPPER[ 26 ].ToemphasizetheperformancestrengthoftBRLPBoost,weusethesimplesttypeofweakclassier,namely,thedecisionstump,whichisatwolevelbinarydecisiontree.AnillustrationofchoosingaweakclassierusingthedecisionstumpisshowninFigure 5-1 .Theweakclassiersarecomputedinthefollowingway.Werandomlyselectonefeaturefromthefeaturevector,andputtogetherthevaluesofthisfeatureforalltraining 69

PAGE 70

Figure5-1. Anillustrationofchoosingaweakclassierusingadecisionstump.xisasample,aisthethresholdandhistheweakclassier. samples,sortthem,andgetthemeanofeverytwoconsequentvalues.Thesemeansandthesmallest,aswellasthelargestvalueswillconstitutethecandidatethresholdset.Givenatestsample,ifthevalueofitsfeatureisabovethethreshold,wewillassignthissampletooneclass,otherwiseassignittotheotherclass.Forexample,ifweselectedthekthfeatureofthetrainingsamples,andthevaluesofallthetrainingsamplesforthekthfeaturearefxk1,xk2,,xkNg.Wesortfxk1,xk2,,xkNginnon-decreasingorder,andifthesortedvaluesaredenotedbyf^xk1,^xk2,,^xkNg,thenthecandidatethresholdsetisf^xk1,(^xk1+^xk2)=2,,(^xkN)]TJ /F4 7.97 Tf 6.59 0 Td[(1+^xkN)=2,^xkNg.Wewillchoosethethresholdwhichgivesusthehighestclassicationaccuracyandtakeitasourweakclassier. 5.4ExperimentsWeevaluatedouralgorithmonnumerouspublicdomaindatasets,includingtheUCImachinelearningrepository[ 43 ],theOASISMRIbraindatabase[ 74 ],theEpilepsydataset[ 59 ],theCNSdataset[ 99 ],theColontumordataset[ 3 ]andLeukemiacancerdataset[ 50 ].Wecomparedourmethodwithseveralstate-of-the-artboostingtechniquesandclassiers. 5.4.1UCIDatasetsTheUCIrepository[ 43 ]isacollectionofdatabasesthathavebeenextensivelyusedforanalyzingmachinelearningtechniques.Therepositorycontainsverynoisydata(e.g.waveform)aswellasrelativelycleandata,whichisoptimalfortestingclassicationalgorithms.Weselected13datasetsfromtheUCIrepository.Theselecteddatasetsincludenoisyandcleandatasets,coversmallsizetolargesize 70

PAGE 71

datasetsintermsofnumberofsamplesinthedatasets,andrangefromlowdimensiontohighdimensionintermsofnumberofattributespersampleofthedatasets.ThedescriptionoftheselecteddatasetsisshowninTable 5-1 .Wecomparedourresultswiththosegeneratedfromothertechniquesintheliterature,includingAdaBoost,LPBoost,SoftBoost,BrownBoostandELPBoost.Fortheimplementationofmostmethods,weusedtheexistingcodesonthewebsite(http://sourceforge.net/projects/jboost/,http://www.kyb.mpg.de/bs/people/nowozin/gboost/#pubs),andforELPBoost,weadapteditfromLPBoost(http://www.kyb.mpg.de/bs/people/nowozin/gboost/#pubs).Weusethesameexperimentalsettingsasin[ 131 ].Eachdatasetisseparatedinto100predenedsplitsastrainingandtestingsets.Foreachofthesplits,weuse5-foldcross-validation,i.e.80%ofthesamplesarefortrainingandvalidation,and20%ofthesamplesarefortesting.Wedeterminetheparametersforeachoftheboostingalgorithmsduringthetrainingandvalidationperiod,andtheparametersaresettobethosemaximizingtheclassicationaccuracyofthetrainingsamples.Theresultsobtainedareanaveragetakenover100runsoftheestimationoftheclassicationaccuracyforeachalgorithmanddataset.ThemeansandstandarddeviationsarereportedinTable 5-2 ,fromwhichitisevidentthattBRLPBoosthasupto13%higheraccuracythanotherboostingalgorithmsonvariouskindsofdatasets.Wealsoreportthechangeoftheclassicationaccuracyasafunctionofthenumberofiterationsonthepima,spambase,irisandspectfdatasets,andcompareourresultswiththeresultsfromanapplicationofELPBoost[ 131 ].Theresultsareobtainedusing5-foldcross-validationandweshowtheaverageclassicationaccuracyvs.thenumberofiterationsinFigure 5-2 .ThegureshowsthattheaccuracyoftBRLPBoostincreasesmuchfasterduringtherstfewiterationsandgetstosatisfactoryaccuracyinfarlessnumberofiterationsincomparisontoELPBoost. 71

PAGE 72

dataset ]instance ]attribute description breastcancer 699 9 b-cancerdiagnosis diabetes 768 8 diabetesdiagnosis germancredit 1000 24 good/badcredit heartdisease 303 74 heartdiseases ionosphere 351 33 classifyradardata liverdisorders 345 6 bloodtestresults sonar 208 60 sonarsignals pendigits 1364 16 handwrittendigits waveform 5000 21 waveforms pima 768 8 signsofdiabetes spambase 4601 51 classifyspamemail iris 150 4 owerclassication spectf 267 44 (ab)normalheart Table5-1. DescriptionoftheUCIdatasetsthatweuse. dataset AdaBoost LPBoost BrownBoost ELPBoost tBRLPBoost breastcancer 0.7020.039 0.7340.050 0.6990.037 0.7280.042 0.8640.008 diabetes 0.7220.015 0.7650.018 0.7280.014 0.7560.018 0.7830.015 germancredit 0.7300.021 0.7540.023 0.7520.019 0.7580.223 0.8050.102 heartdisease 0.7980.025 0.8120.033 0.7930.027 0.8270.031 0.8950.016 ionosphere 0.8790.016 0.8430.011 0.8750.015 0.8380.014 0.9570.006 liverdisorders 0.7530.026 0.7650.032 0.8020.036 0.7830.027 0.8750.031 sonar 0.8610.053 0.8570.060 0.8610.042 0.8730.071 0.9020.018 pendigits 0.9210.010 0.9420.012 0.0930.012 0.9210.008 0.9850.003 waveform 0.8920.004 0.8990.005 0.9000.004 0.8950.006 0.9230.003 Table5-2. Classicationaccuracy(meandeviation)fordifferentboostingalgorithms. 5.4.2OASISDatasetsWealsoevaluatedouralgorithmontheOASISMRIbraindatabase[ 74 ].TheOASISdatabasecontainsacross-sectionalcollectionof416subjectsaged18to96.Outofthe416subjects,175subjectsareyoungerthan40thatwedesignatedasyoung(Y),and195subjectsareabove60thatwedesignatedasold(O).Theother66subjectsaredesignatedasmiddleaged(M).Eachsubjectisrepresentedusinga3Dhistogramdescribingthenon-rigidregistrationrequiredtoco-registeranemergingatlas(inagroupwiseregistrationprocess)toasubjectMRbrainscan.Thisgroupwiseregistrationwasaccomplishedbyusingthemethoddescribedin[ 55 ]ontheOASISdataset.The 72

PAGE 73

A B C D Figure5-2. ClassicationusingtBRLPBoostandELPBoostonthetrainingandtestingsetsofthepima,spambase,irisandspectfdatasets. numberofbinsineachdirectionwassetto(666)forconstructingthehistogramsofthedisplacementvectorsdescribingthenon-rigidregistration.Further,amongtheoldagedpeople,wetook70subjects,and35ofthemwerediagnosedwithverymildtomoderateAlzheimerdisease(AD)whiletherestwerecontrols.Wedidfourgroupsofexperimentsonthisdataset.Firstclassiedtheagegroups(Yvs.M,Mvs.O,andOvs.Y),andthenclassiedthehealthyandtheverymildtomoderateADpatients(ADvs.Control).Foreachexperiment,weuse5-foldcrossvalidationandreporttheaverageclassicationaccuracy.Theparametersofeachofthealgorithmsaresettobethose 73

PAGE 74

dataset AdaBoost LPBoost ELPBoost tBRLPBoost Yvs.M 0.960 0.980 0.973 0.985 Mvs.O 0.962 0.972 0.974 0.996 Ovs.Y 0.987 0.988 0.988 1.000 ADvs.Con. 0.923 0.967 0.968 0.977 Table5-3. ClassicationaccuracyofdifferentmethodsontheOASISdataset. maximizingtheaccuracyofthetrainingdataset.WecomparedourresultswiththosefromAdaBoost,LPBoostandELPBoost.TheresultsareshowninTable 5-3 ,fromwhichitisevidentthatourmethod(tBRLPBoost)outperformsthecompetitivemethods.AlsotBRLPBoostconvergesmuchfasterthantheotheralgorithms.AllalgorithmsarerunonalaptopwithIntel(R)Core(TM)2CPUL7500@1.6GHz,4GBmemory,GNULinuxandMATLAB(Version2010a).TheaverageCPUtimetakentoconvergeforouralgorithmis0.2778s,whileAdaBoosttakes1.8082s,LPBoosttakes2.1430sandELPBoosttakes2.1164s. 5.4.3OtherDatasetsTheEpilepsydataset[ 59 ]consistsof3Dhistograms(666)ofdisplacementvectoreldsrepresentingtheregistrationdeformationeldbetweentheleftandrighthippocampiin3D.Thegoalistodistinguishbetweenleftandrightanteriortemporallobe(L/RATL)epileptics.TheCNSdataset[ 99 ]containsthetreatmentoutcomesof60subjectsforcentralnervoussystemembryonaltumor,whichincludes21survivorsand39failures.TheColontumordataset[ 3 ]consistsof22normaland40tumorcolontissuefeatures.TheLeukemiacancerdataset[ 50 ]contains25acutemyeloidleukemia(AML)and47acutelymphoblasticleukemia(ALL)samples.ThegoalistodistinguishfromAMLtoALL.Wecomparedourmethodwiththerecentlypublishedcompetitiveclassiers,includingconicsectionclassier(CSC)[ 9 ],CSCwithmarginpursuit(CSC-M)[ 59 ],kernelFisherDiscriminants(KFD)[ 39 77 ],andkernelSVMs(SVM)[ 122 ].TheresultsareshowninTable 5-4 ,whichdepictsthatouralgorithmperformsbetterthanthecompetingclassiers. 74

PAGE 75

dataset CSC-M CSC KFD SVM tBRLPBoost Epilepsy 0.932 0.886 0.864 0.864 0.941 CNS 0.700 0.733 0.650 0.683 0.757 Colon 0.871 0.871 0.758 0.823 0.892 Leukemia 0.972 0.986 0.986 0.972 0.990 Table5-4. ClassicationaccuracyfordifferentmethodsontheEpilepsy,CNS,ColontumorandLeukemiadatasets. 5.5DiscussionsInthischapter,weproposedanewboostingalgorithmdubbedtBRLPBoost,whichistotalBregmandivergence(tBD)regularizedLPBoost.tBRLPBoostisrobusttonoiseandoutliersduetotheintrinsicrobustnesspropertyoftBD.tBRLPBoostisabletomaximizethesoftmarginforlinearlyinseparabledataset.WeshowedthattBRLPBoostrequiresonlyaconstantnumberofiterationstoconvergeandhenceisindependentofthesizeofthetrainingset,whichmakesitveryefcient.Incomparison,mostefcientboostingalgorithminthepublishedliteraturecostlogarithmicnumberofiterations.Weshowedseveralcomparisonstootherexistingboostingmethodsanddepictedbetterperformanceofouralgorithmincomparisontothestate-of-the-artmethodsinliterature.EventhoughinthischapterwefocusontKLregularizedLPBoost,wecaneasilyextendthisanduseotherclassesoftBDtoregularizetheLPBoost.Further,sincetBRLPBoostishighlyefcient,thismakesitapromisingcandidateforonlineboostingappliedtoverylargescaledatasets,dynamicenvironmentsandmultimediadatasets,whichwillbethefocusofourfuturework.Finally,tBRLPBoostcanbeveryeasilygeneralizedtomulticlassinputaswelltothecondence-relatedinput,wherethelabelisnotbinarybuttakesvaluesintheinterval[)]TJ /F8 11.955 Tf 9.3 0 Td[(1,1]. 75

PAGE 76

CHAPTER6SIMULTANEOUSSMOOTHINGANDESTIMATIONOFDTIVIAROBUSTVARIATIONALNON-LOCALMEANSRegularizeddiffusiontensorestimationisanessentialstepinDTIanalysis.Therearemanymethodsproposedinliteratureforthistaskbutmostofthemareneitherstatisticallyrobustnorfeaturepreservingdenoisingtechniquesthatcansimultaneouslyestimatesymmetricpositivedenite(SPD)diffusiontensorsfromdiffusionMRI.Oneofthemostpopulartechniquesinrecenttimesforfeaturepreservingscalar-valuedimagedenoisingisthenon-localmeanslteringmethodthathasrecentlybeengeneralizedtothecaseofdiffusionMRIdenoising.However,thesetechniquesdenoisethemulti-gradientvolumesrstandthenestimatethetensorsratherthanachievingitsimultaneouslyinauniedapproach.Moreover,someofthemdonotguaranteethepositivedenitenessoftheestimateddiffusiontensors.Inthischapter,weproposeanovelandrobustvariationalframeworkforthesimultaneoussmoothingandestimationofdiffusiontensorsfromdiffusionMRI.OurvariationalprinciplemakesuseofarecentlyintroducedtotalKullback-Leibler(tKL)divergence,whichisastatisticallyrobustsimilaritymeasurebetweendiffusiontensors,weightedbyanon-localfactoradaptedfromthetraditionalnon-localmeanslters.Forthedatadelity,weusethenonlinearleast-squarestermderivedfromtheStejskal-Tannermodel.Wepresentexperimentalresultsdepictingthepositiveperformanceofourmethodincomparisontocompetingmethodsonsyntheticandrealdataexamples. 6.1LiteratureReviewforDTIEstimationDiffusionMRIisatechniquethatusesdiffusionsensitizinggradientstonon-invasivelyimageanisotropicpropertiesoftissue.Diffusiontensorimaging(DTI)introducedbyBasseretal.[ 13 ],approximatesthediffusivityfunctionbyasymmetricpositivedenitetensorofordertwo.ThereisabundantliteratureonDTIanalysisincludingbutnotlimitedtodenoisingandtensoreldestimation[ 22 78 100 114 119 120 124 129 ],DTI 76

PAGE 77

registration,bertractographyetcandalloftheselattertaskswillbenetfromdenoisingandestimationofsmoothdiffusiontensors.Inmostoftheexistingmethods,thediffusiontensors(DTs)areestimatedusingtherawdiffusionweightedechointensityimage(DWI).Ateachvoxelofthe3Dimagelattice,thediffusionsignalintensitySisrelatedwithitsdiffusiontensorD2SPD(3)1viatheStejskal-Tannerequation[ 112 ] S=S0exp()]TJ /F5 11.955 Tf 9.3 0 Td[(bgTDg),(6)whereS0isthesignalintensitywithoutdiffusionsensitizinggradient,bisthediffusionweightingandgisthedirectionofthediffusionsensitizinggradient.EstimatingtheDTsfromDWIisachallengingproblem,sincetheDWIisnormallycorruptedwithnoise[ 100 114 119 ].Therefore,astatisticallyrobustDTIestimationmethodwhichisabletoperformfeaturepreservingdenoisingisdesired.Therearevariousmethods[ 14 37 51 94 103 114 119 129 ]thatexistintheliteraturetoachievethisgoalofestimatingDfromS.Averyearlyoneisdirecttensorestimation[ 133 ].Thoughtimeefcient,itissensitivetonoisebecauseonly7gradientdirectionsareusedtoestimateDandS0.Anothermethodistheminimumrecoveryerror(MRE)estimationorleastsquarestting[ 13 ]whichminimizestheerrorwhenrecoveringtheDTsfromtheDWI.MREisbetterthandirectestimationbutitdoesnotenforcespatialregularizationortheSPDconstraintresultinginpossibleinaccuracies.Bearingthesedecienciesinmind,researchersdevelopedvariationalframework(VF)basedestimation[ 22 120 129 ].TheseapproachestakeintoaccounttheSPDconstraintonthediffusiontensors.Thesmoothinginalltheseapproachesinvolvessomekindofweightedaveragingoverneighborhoodswhichdenethesmoothing 1SPD(3)representsthespaceof33symmetricpositivedenitematrices 77

PAGE 78

operatorsresultingfromthevariationalprinciples.Thesesmoothingoperatorsarelocallydenedanddonotcaptureglobalgeometricstructurepresentintheimage.Morerecently,somedenoisingframeworkshavebeenproposedaccordingtothestatisticalpropertiesofthenoise.TheyassumethenoisefollowstheRiciandistribution[ 30 58 ],andtheDWIisdenoisedusingmaximumlikelihoodestimation.AfterdenoisingtheDWI,onecanuseothertechniquestoestimatetheDTI.Besides,thepopularNLMbasedmethodhasbeenadaptedbymanytodenoiseDTIdatasets[ 27 134 135 ].IntheNLMbasedapproaches,onerstneedstodenoisetheDWIeldandthenestimateDTIfromthedenoisedDWI.Alternatively,thediffusiontensorsarerstestimatedandthendenoisedusingRiemannianapproach[ 20 ]ortheNLMframeworkincorporatingaLog-Euclideanmetric[ 38 ].Thedrawbackofsuchtwo-stageprocessesisthattheerrorsmightaccumulatefromonestagetotheother.Toovercometheaforementionedproblems,weproposeanovelstatisticallyrobustvariationalnon-localapproachforsimultaneoussmoothingandtensorestimationfromtherawDWIdata.ThisapproachcombinestheVF,NLMandanintrinsicallyrobustregularizeronthetensoreld.Themaincontributionsofthisapproacharethree-fold.First,weusetotalBregmandivergence(specically,thetKL)asameasuretoregularizethetensoreld.CombinedwiththeCholeskydecompositionofthediffusiontensors,thisautomaticallyensuresthepositivedenitenessoftheestimateddiffusiontensors,whichovercomesthecommonproblemformanytechniques[ 129 ]thatneedtomanuallyforcethetensortobepositivedenite.Second,itpreservesthestructureofthetensoreldwhiledenoisingviaanadaptationoftheNLMframework.Finally,itallowsforsimultaneousdenoisingandDTestimation,avoidingtheerrorpropagationofatwostageapproachdescribedearlier.Besides,thismethodcanbeeasilyextendedtohigherordertensorestimation.Wewillexplainthesepointsatlengthintherestofthechapter. 78

PAGE 79

Therestofthechapterisorganizedasfollows.InSection 6.2 ,weintroduceourproposedmethodandexploreitsproperties,followedbytheempiricalvalidationinSection 6.3 .FinallyweconcludeinSection 6.4 6.2ProposedMethodThesimultaneousdenoisingandestimationoftheDTIisachievedbyminimizingthefollowingenergyfunction: minS0,D2SPDE(S0,D)=Xx2nXi=1)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(Si(x))]TJ /F5 11.955 Tf 11.96 0 Td[(S0(x)expf)]TJ /F5 11.955 Tf 15.28 0 Td[(bgTiD(x)gig2+(1)]TJ /F3 11.955 Tf 11.96 0 Td[()Xx2Xy2V(x)w(x,y)(S0(x))]TJ /F5 11.955 Tf 11.95 0 Td[(S0(y))2+(D(x),D(y)),(6)whereisthedomainoftheimage,w(x,y)isthesimilaritybetweenvoxelsxandy,V(x)istheuserspeciedsearchwindowatx,and(D(x),D(y))isthetotalKullback-Leibler(tKL)divergenceproposedin[ 70 123 ]betweentensorsD(x)andD(y).tKLisdenedin[ 123 ]andhasbeenusedinDTIsegmentation[ 123 ]andclassication[ 69 ].WewillredeneitlaterinSection 6.2.2 forthesakeofcompleteness.Thersttermof( 6 )minimizesthenon-linearttingerror,thesecondtermenforcessmoothnessconstraintsonS0andDviaanon-localmeansregularizer.istheregularizationconstantbalancingthettingerrorandthesmoothness.NotethatSi,S0andDbydefaultrepresentthevaluesatvoxelx,unlessspeciedotherwise. 6.2.1ComputationoftheWeightw(x,y)Theweightw(x,y)isregularizesthesimilaritybetweenS0(x)andS0(y),aswellasD(x)andD(y).Usually,onerequiresthesimilaritytobeconsistentwiththesimilaritybetweentheircorrespondingdiffusionsignals.Therefore,wedenew(x,y)basedonthediffusionsignalintensitiesofthetwovoxels'neighborhoods.LetN(x)andN(y)denotetheneighborhoodsofxandyrespectively.Ify2V(x),thenw(x,y)isdenedas, w(x,y)=1 Z(x)exp)]TJ /F2 11.955 Tf 5.48 -9.68 Td[(kS(N(x)))]TJ /F5 11.955 Tf 11.96 0 Td[(S(N(y))k2=h2,(6) 79

PAGE 80

wherehistheuserspeciedlteringparameterandZisthenormalizationconstant.S(N(y))representsthesignalintensitiesoftheneighborhoodatxandkS(N(x)))]TJ /F5 11.955 Tf -425.27 -23.91 Td[(S(N(y))k2=PmjkS(j))]TJ /F5 11.955 Tf 9.3 0 Td[(S(j)k2,wherejandjarethejthvoxelsintheneighborhoodsrespectively,andmisthenumberofvoxelsineachneighborhood.From( 6 ),wecanseethatwhenw(x,y)islarge,thenS0(x)andS0(y)aswellasD(x)andD(y)respectivelyaresimilar.Inotherwords,ifthesignalintensitiesfortheneighborhoodsoftwovoxelsaresimilar,theircorrespondingDsandS0sshouldalsobesimilar.Thoughhavingverygoodaccuracy,NLMisknownforitshightimecomplexity.Toreducethetimecost,weusetwotricks.Oneistodecreasethenumberofcomputationsbyselectingonlythosevoxelswhosesignalintensityissimilartothatofthevoxelunderconsideration.Thisisspeciedbyw(x,y)=8><>:1 Z(x)exp)]TJ /F2 11.955 Tf 5.48 -9.69 Td[(kS(N(x)))]TJ /F5 11.955 Tf 11.95 0 Td[(S(N(y))k2=h2,ifkS(N(x))k2 kS(N(y))k22[1,2]0,Otherwise.Wechoose1=0.5and2=2inourexperiments.Thisprelteringprocessgreatlydecreasesthenumberofcomputations.Theothertrickisusingparallelcomputing,wherewedividethecomputationsintoseveralparts,andassignthecomputationpartstoseveralprocessors.Inourcase,wedividethevolumesinto8subvolumes,andassigneachsubvolumetooneprocessor,andadesktopwith8processorsisused.Thismulti-threadingtechniquegreatlyenhancestheefciency. 6.2.2ComputationofthetKLDivergenceMotivatedbyearlieruseofKLdivergenceasasimilaritymeasurebetweenDTsinliterature[ 129 ],weusetherecentlyintroducedtKL[ 123 ]tomeasurethesimilaritybetweentensors.tKLhasthepropertyofbeingintrinsicallyrobusttonoiseandoutliers,yieldsaclosedformformulaforcomputingthemedian(an`1-normaverage)foraset 80

PAGE 81

oftensors,andisinvarianttospeciallineargrouptransformations(denotedasSL(n))transformationsthathavedeterminantone[ 123 ].Notethatorder-2SPDtensorscanbeseenascovariancematricesofzeromeanGaussianprobabilitydensityfunctions(pdf)[ 129 ].LetP,Q2SPD(l),thentheircorrespondingpdfarep(t;P)=1 p (2)ldetPexp)]TJ /F6 11.955 Tf 10.49 8.09 Td[(tTP)]TJ /F4 7.97 Tf 6.58 0 Td[(1t 2,q(t;Q)=1 p (2)ldetQexp)]TJ /F6 11.955 Tf 10.49 8.08 Td[(tTQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1t 2,andthetKLbetweenthem(inclosedform)isgivenby, (P,Q)=Rplogp qdt q 1+R(1+logq)2qdt=log(det(P)]TJ /F4 7.97 Tf 6.59 0 Td[(1Q))+tr(Q)]TJ /F4 7.97 Tf 6.59 0 Td[(1P))]TJ /F29 10.909 Tf 10.91 0 Td[(m 2q c1+(log(detQ))2 4)]TJ /F29 10.909 Tf 10.91 0 Td[(c2log(detQ),wherec1=3l 4+l2log2 2+(llog2)2 4andc2=l(1+log2) 2.Moreover,theminimizationofthethirdtermin( 6 )minD(x)Xx2Xy2V(x)(D(x),D(y))leadstoan`1-normaveragewhichwasshowntohaveclosedformexpression[ 123 ].In[ 123 ],thiswascalledthet-centerandwasshowntobeinvarianttotransformationsfromtheSL(n)group,i.e.,(P,Q)=(AtPA,AtQA),wheredetA=1.Furthermore,givenasetofSPDtensorsfQigmi=1,itst-centerPisgivenby[ 123 ] P=argminPmXi(P,Qi),(6)andPisexplicitlyexpressedas P=(Xiai PjajQ)]TJ /F4 7.97 Tf 6.58 0 Td[(1i))]TJ /F4 7.97 Tf 6.58 0 Td[(1,ai= 2r c1+(log(detQi))2 4)]TJ /F29 10.909 Tf 10.91 0 Td[(c2log(detQi)!)]TJ /F4 7.97 Tf 6.59 0 Td[(1.(6)Thet-centerforasetofDTsistheweightedharmonicmean,whichisinclosedform.Moreover,theweightisinvarianttoSL(n)transformations,i.e.,ai(Qi)=ai(ATQiA),8A2 81

PAGE 82

SL(n).Thet-centerafterthetransformationbecomes^P=(Pai(ATQiA))]TJ /F4 7.97 Tf 6.58 0 Td[(1))]TJ /F4 7.97 Tf 6.58 0 Td[(1=ATPA.ThismeansthatiffQigmi=1aretransformedbysomememberofSL(n),thet-centerwillundergothesametransformation.Itisalsofoundthatthet-centerisstatisticallyrobusttonoiseinthattheweightissmallerifthetensorhasmorenoise[ 123 ]. 6.2.3TheSPDConstraintWenowshowhowtoguaranteethepositivedenitenessconstraintonthediffusiontensorstobeestimatedfromtheDWIdata.ItisknownthatifamatrixD2SPD,theirexistsauniquelowerdiagonalmatrixLwithitsdiagonalvaluesallpositive,andD=LLT[ 49 ].ThisistheCholeskyfactorizationtheorem.Manyresearchers[ 129 ]haveusedCholeskyfactorizationtoensurethepositivedeniteness.TheyrstcomputeL,enforcingthediagonalvaluesofLtobepositive,andconsequently,LLTwillbepositive.Unlikethistechnique,weuseCholeskydecompositionandtKLdivergencetoregularizethetensoreld,andthisautomaticallyensuresthediagonalvaluesofLtobepositive.Thepointscanbevalidatedasfollows.SubstitutingD=LLTinto( 6 ),weget (L(x),L(y))=P3i=1(logLii(y))]TJ /F26 10.909 Tf 10.91 0 Td[(logLii(x))+tr(L)]TJ /F7 7.97 Tf 6.59 0 Td[(T(y)L)]TJ /F4 7.97 Tf 6.59 0 Td[(1(y)L(x)LT(x)))]TJ /F26 10.909 Tf 10.91 0 Td[(1.5 q c1+(P3i=1logLii(y))2 4)]TJ /F29 10.909 Tf 10.91 0 Td[(c2P3i=1logLii(y).(6)Becauseofusingthelogcomputation,Eq.( 6 )automaticallyensuresLiistobepositive,thereforewedonotneedtoaddtheSPDconstraintmanually. 6.2.4NumericalSolutionInthissection,wepresentthenumericalsolutiontothevariationalprinciple( 6 ).Thepartialderivativeequationsof( 6 )withrespecttoS0andLcanbecomputed 82

PAGE 83

explicitlyandare, @E @S0(x)=)]TJ /F8 11.955 Tf 11.95 0 Td[(2nXi=1)]TJ /F5 11.955 Tf 5.48 -9.69 Td[(Si)]TJ /F5 11.955 Tf 11.96 0 Td[(S0expf)]TJ /F5 11.955 Tf 15.28 0 Td[(bgTiLLTgigexpf)]TJ /F5 11.955 Tf 15.27 0 Td[(bgTiLLTgig)]TJ /F8 11.955 Tf 11.95 0 Td[(2(1)]TJ /F3 11.955 Tf 11.96 0 Td[()Xy2V(x)w(x,y)(S0(x))]TJ /F5 11.955 Tf 11.96 0 Td[(S0(y)),@E @L(x)=4nXi=1)]TJ /F5 11.955 Tf 5.48 -9.69 Td[(Si)]TJ /F5 11.955 Tf 11.96 0 Td[(S0expf)]TJ /F5 11.955 Tf 15.28 0 Td[(bgTiLLTgigS0expf)]TJ /F5 11.955 Tf 15.28 0 Td[(bgTiLLTgigbLTgigTi)]TJ /F8 11.955 Tf 11.95 0 Td[(2(1)]TJ /F3 11.955 Tf 11.96 0 Td[()Xy2V(x)w(x,y)(L)]TJ /F4 7.97 Tf 6.58 0 Td[(1(x))]TJ /F6 11.955 Tf 11.96 0 Td[(LT(x)L)]TJ /F7 7.97 Tf 6.59 0 Td[(T(y)L)]TJ /F4 7.97 Tf 6.59 0 Td[(1(y)) q c1+(P3i=1logLii(y))2 4)]TJ /F5 11.955 Tf 11.96 0 Td[(c2P3i=1logLii(y).(6)Tosolve( 6 ),weusethelimitedmemoryquasi-Newtonmethoddescribedin[ 86 ].Thismethodisusefulforsolvinglargeproblemswithalotofvariables,asisinourcase.ThismethodmaintainssimpleandcompactapproximationsofHessianmatricesmakingthemrequire,asthenamesuggests,modeststorage,besidesyieldinglinearrateofconvergence.Specically,weuseL-BFGS[ 86 ]toconstructtheHessianapproximation. 6.3ExperimentalResultsWeevaluateourmethodonbothsyntheticdatasetswithvariouslevelsofnoise,andonrealdatasets.Wecomparedourmethodwithotherstate-of-the-arttechniquesincludingthetechniquesVF[ 120 ],NLMt[ 135 ]andNLM[ 135 ]respectively.WealsopresenttheMREmethodforcomparisonsinceseveralsoftwarepackagesinvogueusethistechniqueduetoitssimplicity.WeimplementedVFandNLMtbyourselvessincewedidnotndanyopensourceversionsontheweb.FortheNLM,weusedexistingcode2forDWIdenoisingandusedourownimplementationoftheleastsquaresttingtoestimateDTIfromthedenoisedDWI.Toensurefairness,wetunedalltheparametersofeachmethodforeveryexperiment,andchosethesetofparametersyieldingthebestresults.Thevisualandnumericalresultsshowthatourmethodyieldsbetterresultsthancompetingmethods. 2 https://www.irisa.fr/visages/benchmarks/ 83

PAGE 84

6.3.1DTIEstimationonSyntheticDatasetsThesyntheticdataisa1616tensoreldwithtwohomogeneousregionsasshowninFigure 6-1A .WeletS0=5,b=1500s=mm2,andgbe22uniformly-spaceddirectionsontheunitspherestartingfrom(1,0,0).SubstitutingtheDTs,S0,b,gintotheStejskal-Tannerequation,wegeneratea161622DWIS.OnerepresentativesliceofSisshowninFigure 6-1B .Thenfollowingthemethodproposedin[ 58 ],weaddRiciannoisetoSandget~S,usingtheformula,~S(x)=p (S(x)+nr)2+n2i,wherenrandniN(0,).Figure 6-1C showsthesliceinFigure 6-1B afteraddingnoise(SNR=10).Byvarying,wegetdifferentlevelsofnoiseandthereforeawiderangeofsignaltonoiseratio(SNR).TheestimatedDTIfromusingMRE,VF,NLMt,NLM,andtheproposedmethodareshowninFigure 6-1 .Thegureshowsthatourmethodcanestimatethetensoreldmoreaccurately.Toquantitativelyevaluatetheproposedmodel,wecomparedtheaverageoftheangledifferencebetweentheprincipledirectionsoftheestimatedtensoreldandthegroundtruthtensoreld,andthedifferenceS0betweentheestimatedandgroundtruthS0.TheresultsareshowninTable 6-1 ,fromwhichitisevidentthatourmethodoutperformsothersandthesignicanceinperformanceismoreevidentathighernoiselevels.TheaverageCPUtimetakentoconvergeforourmethodonadesktopcomputerwithIntel8Core2.8GHz,24GBofmemory,GNULinuxandMATLAB(Version2010a)is7.03s,whereas,NLMrequires10.52s(notebothmethodsareexecutedusingparallelcomputing). 6.3.2DTIEstimationonRealDatasetsWealsodidDTIestimationona12496403DratspinalcordDWI.ThedatawasacquiredusingaPGSEtechniquewithTR=1.5s,TE=28.3ms,bandwidth=35Khz,22diffusionweightedimageswithb-valueabout1014s=mm2.TheestimatedtensoreldisshowninFigure 6-2 .Wecomparedourmethodwithallaforementionedmethods,howeverduetospacelimit,weonlypresenttheresultsofMREandNLM.Fromthese 84

PAGE 85

A B C D E F G H Figure6-1. GroundtruthsyntheticDTIeld,theoriginalDWI,theRiciannoiseaffectedDWI,estimationusingMRE,VF,NLMt,NLM,andtheproposedmethod. SNR error MRE VF NLMt NLM proposed 50 20.112.1 8.410.2 8.910.7 6.010.1 5.87.3 S0 0.540.09 0.660.05 0.640.08 0.310.04 0.280.01 40 22.112.5 12.113.2 15.714.2 7.212.5 6.18.6 S0 0.750.17 0.750.31 0.940.41 0.640.30 0.530.27 30 22.312.9 19.513.9 18.314.7 7.612.7 6.89.7 S0 2.242.16 1.031.22 1.031.31 1.021.21 0.810.69 15 28.317.1 27.215.1 25.616.2 14.716.1 8.210.3 S0 3.812.24 1.912.02 1.861.87 1.851.77 1.020.87 8 43.223.4 32.925.8 28.220.6 20.218.5 8.711.0 S0 5.294.36 2.482.72 2.292.32 2.242.19 1.090.92 Table6-1. ErrorinestimatedDTIandS0,usingdifferentmethods,fromsyntheticDWIwithdifferentlevelsofnoise. gures,wecanseethatourproposedmethodcanestimateasmoothertensoreldwhichpreservesthestructuremuchbettercomparedwithothermethods.WealsodidDTIestimationona10080323DratbrainDWI.ThedatawasacquiredusingaBrukerscannerundertheSpinEchotechniquewithTR=2s,TE=28ms,52diffusionweightedimageswithab-valueof1334s=mm2.TheFAandtheprincipal 85

PAGE 86

A B C Figure6-2. TheguresaretheonesliceoftheestimatedtensoreldsusingMRE,NLM,andtheproposedmethodrespectively. eigenvectorsoftheestimatedtensoreldareshowninFigure 6-3 ,whichshowedthecomparisonofourmethodwithMREandNLM.Theresultsillustratethatourproposedmethodcanestimatethetensoreldmoreaccuratelycomparedwithothers. 6.4DiscussionsWeproposedarobustvariationalnon-localmeansapproachforsimultaneousdenoisingandDTIestimation.Theproposedmethodcombinesthevariationalframework,non-localmeansandanintrinsicallyrobustsmoothnessconstraint.Inthevariationalprinciple,weusednon-lineardiffusiontensorttingterm,alongwithacombinationofnon-localmeansandthetKLbasedsmoothnessfordenoising.TospeeduptheNLMmethod,weprelteredthevoxelsinthesearchwindowtoreducethenumberofcomputationsandmadeuseofparallelcomputingtodecreasethecomputationalload.Thisvariationalnon-localapproachwasvalidatedwithsyntheticandrealdataandshowntobemoreaccuratethancompetingmethodsintheliterature.WedidnothowevercomparewithmanymethodsinliteraturethatwerealreadycomparedtotheNLMtechniquein[ 135 ].Forfuturework,weplantodevelopaGPU-basedimplementationtobetterthecomputationtime.Wewillalsoexploreotherquantitativemeasuresofvalidationsuchasmethodnoisedenedfortensors.Aftergettingamore 86

PAGE 87

A B C D E F Figure6-3. TheFAoftheestimatedtensoreldsusingMRE,NLM,theproposedmethod,andtheprincipaleigenvectorsoftheROIs. comprehensivetensorestimationtechnique,wewillutilizeitasapreprocessingstepfortheapplicationsofbertrackingandDTIsegmentation. 87

PAGE 88

CHAPTER7CONCLUSIONSInthisthesis,wedevelopedanovelrobustdivergencedubbedthetotalBregmandivergence(TBD),whichisintrinsicallyrobusttonoiseandoutliers.Thegoalherewasnotsimplytodevelopyetanotherdivergencemeasurebuttodevelopanintrinsicallyrobustdivergence.Thiswasachievedviaalterationofthebasicnotionofdivergencewhichmeasurestheordinatedistancebetweenitsconvexgeneratoranditstangentapproximationtotheorthogonaldistancebetweenthesame.ThisbasicideaparallelstherelationshipbetweenleastsquaresandtotalleastsquaresbuttheimplicationsarefarmoresignicantintheeldofinformationgeometryanditsapplicationssincetheentireclassofthewellknownBregmandivergenceshavebeenredenedandsomeoftheirtheoreticalpropertiesstudied.Specically,wederivedanexplicitformformulaforthet-centerwhichistheTBD-basedmedian,thatisrobusttooutliers.InthecaseofSPDtensors,thet-centerwasshowntobeSL(n)transformationinvariant.TherobustnesspropertyofTBDwasdemonstratedhereviaapplicationstoSPDtensoreldsegmentation,shaperetrieval,regularizedboosting,andDTIestimation.Theresultsfavorablydemonstratethecompetitivenessofournewlydeneddivergenceincomparisontoexistingmethodsnotonlyintermsofrobustness,butalsointermsofcomputationalefciencyandaccuracyaswell.However,thestoryisnotyetcompleteandfurtherinvestigationsarecurrentlyunderway.OurfutureresearchwillfocusonapplicationsofTBDtobertrackingandberclustering. 88

PAGE 89

REFERENCES [1] Absil,P.A.,Mahony,R.,andSepulchre,R.Optimizationalgorithmsonmatrixmanifolds.PrincetonUniversityPress,2008. [2] Akgul,C.B.,Sankur,B.,Yemez,Y.,andSchmitt,F.Dmodelretrievalusingprobabilitydensity-basedshapedescriptors.IEEETransactionsonPatternAnalysisandMachineIntelligence31(2009).6. [3] Alon,U.,Barkai,N.,Notterman,D.A.,Gishdagger,K.,Ybarradagger,S.,Mackdagger,D.,andLevine,A.J.Broadpatternsofgeneexpressionrevealedbyclusteringanalysisoftumorandnormalcolontissuesprobedbyoligonucleotidearrays.ProceedingsoftheNationalAcademyofSciencesoftheUnitedStatesofAmerica(1999). [4] Amari,S.Differential-geometricalmethodsinstatistics.SpringerBerlinHeidelberg,1985. [5] .Integrationofstochasticmodelsbyminimizing-divergence.NeuralComputation19(2007).10:2780. [6] Babenko,B.,Yang,M.,andBelongie,S.Afamilyofonlineboostingalgorithms.OnlineLearningforComputerVision(OLCV).Kyoto,2009. [7] Bai,X.,Wang,B.,Wang,X.,Liu,W.,andTu,Z.Co-transductionforshaperetrieval.EuropeanConferenceonComputerVision(2010):861. [8] Bai,X.,Yang,X.,Latecki,L.J.,Liu,W.,andTu,Z.Learningcontextsensitiveshapesimilaritybygraphtransduction.IEEETransactionsonPatternAnalysisandMachineIntelligence32(2010).5:861. [9] Banerjee,A.,Kodipaka,S.,andVemuri,B.C.Aconicsectionclassieranditsapplicationtoimagedatasets.IEEEConferenceonComputerVisionandPatternRecognition(2006):103. [10] Banerjee,A.,Merugu,S.,Dhillon,I.S.,andGhosh,J.ClusteringwithBregmandivergences.JournalofMachineLearningResearch6(2005):1705. [11] Bar-Hillel,A.,Hertz,T.,andWeinshall,D.Objectclassrecognitionbyboostingapartbasedmodel.IEEEConferenceonComputerVisionandPatternRecognition(2005):702. [12] Barmpoutis,A.,Vemuri,B.C.,Shepherd,T.M.,andForder,J.R.TensorsplinesforinterpolationandapproximationofDT-MRIwithapplicationstosegmentationofisolatedrathippocampi.IEEETransactionsonMedicalImaging26(2007). [13] Basser,P.J.,Mattiello,J.,andLeBihan,D.Estimationoftheeffectiveself-diffusiontensorfromtheNMRspinecho.JournalofMagneticResonance103(1994).3:247. 89

PAGE 90

[14] Batchelor,P.G.,Moakher,M.,Atkinson,D.,Calamante,F.,andConnelly,A.Arigorousframeworkfordiffusiontensorcalculus.MagneticResonanceinMedicine53(2005).1:221. [15] Belongie,S.,Malik,J.,andPuzicha,J.Shapematchingandobjectrecognitionusingshapecontexts.IEEETransactionsonPatternAnalysisandMachineIntelligence24(2002):509. [16] Biederman,I.andJu,G.Surfacevs.edge-baseddeterminantsofvisualrecognition.CognitivePsychology20(1988):38. [17] Boyd,S.andVandenberghe,L.Convexoptimization.CambridgeUniversityPress,Cambridge,England(2004). [18] Bregman,L.M.Therelaxationmethodofndingthecommonpointsofconvexsetsanditsapplicationtothesolutionofproblemsinconvexprogramming.USSRComputationalMathematicsandMathematicalPhysics7(1967):200. [19] Carrasco,V.S.,Talavera,G.,Igea,J.,andCastresana,J.TheKtreescore:quanticationofdifferencesintherelativebranchlengthandtopologyofphylogenetictrees.BioinformaticsApplicationsNote23(2007).21:2954. [20] Castano-Moraga,C.A.,Lenglet,C.,Deriche,R.,andRuiz-Alzola,J.ARiemannianapproachtoanisotropiclteringoftensorelds.SignalProcess-ing87(2007).2:217. [21] Chan,T.F.andVese,L.A.Activecontourswithoutedges.IEEETransactionsonImageProcessing10(2001). [22] Chefd'hotel,C.,Tschumperle,D.,Deriche,R.,andFaugeras,O.Regularizingowsforconstrainedmatrixvaluedimages.JournalofMathematicalImagingandVision20(2004).1-2:147. [23] Chen,L.,Feris,R.,andTurk,M.EfcientpartialshapematchingusingSmith-Watermanalgorithm.IEEEConferenceonComputerVisionandPat-ternRecognitionWorkshop(2008):1. [24] Chui,H.andRangarajan,A.Afeatureregistrationframeworkusingmixturemodels.IEEEWorkshoponMathematicalMethodsinBiomedicalImageAnalysis(2000):190. [25] Cohen,W.W.Efcientruleinduction.InternationalConferenceonMachineLearning(1995). [26] Cohen,W.W.andSinger,Y.Asimple,fast,andeffectiverulelearner.NationalConferenceonArticialIntelligence.1999,335. 90

PAGE 91

[27] Coupe,Pierrick,Yger,Pierre,andBarillot,Christian.Fastnonlocalmeansdenoisingfor3DMRimages.InternationalConferenceonMedicalImageComputingandComputerAssistedIntervention(MICCAI)(2006):33. [28] Damski,J.C.andGero,J.S.Alogic-basedframeworkforshaperepresentation.Computer-AidedDesign28(1996).3:169. [29] Demiriz,A.,Bennett,K.P.,andShawe-Taylor,J.Linearprogrammingboostingviacolumngeneration.MachineLearning46(2002).1:225. [30] Descoteaux,M.,Wiest-Daessl,N.,Prima,S.,Barillot,C.,andDeriche,Ra.Impactofricianadaptednon-localmeanslteringonHARDI.InternationalConferenceonMedicalImageComputingandComputerAssistedIntervention(MICCAI)(2008):122. [31] Dietterich,T.G.Anexperimentalcomparisonofthreemethodsforconstructingensemblesofdecisiontrees:bagging,boosting,andrandomization.MachineLearning40(2000):139. [32] Ding,L.andBelkin,M.Componentbasedshaperetrievalusingdifferentialproles.ACMInternationalConferenceonMultimediaInformationRetrieval(2008):216. [33] Dollar,P.,Tu,Z.,Tao,H.,andBelongie,S.Featureminingforimageclassication.IEEEConferenceonComputerVisionandPatternRecognition(2007):1. [34] Felzenszwalb,P.andJ.Schwartz.Hierarchicalmatchingofdeformableshapes.IEEEConferenceonComputerVisionandPatternRecognition(2007):1. [35] Ferrari,V.,Tuytelaars,T.,andGool,L.V.Objectdetectionbycontoursegmentnetworks.EuropeanConferenceonComputerVision3(2006):14. [36] Ferrucci,V.andPaoluzzi,A.Extrusionandboundaryevaluationformultidimensionalpolyhedra.ComputerAidedDesign1(1991):40. [37] Fillard,P.,Pennec,X.,Arsigny,V.,andAyache,N.ClinicalDT-MRIestimation,smoothing,andbertrackingwithLog-Euclideanmetrics.IEEETransactionsonImageProcessing26(2007).11:1472. [38] Fillard,Pierre,Arsigny,V,Pennec,X,Thompson,PM,andAyache,N.Extrapolationofsparsetensorelds:Applicationtothemodelingofbrainvariability.InformationProcessinginMedicalImaging19(2005):27. [39] Fisher,R.Theuseofmultiplemeasurementsintaxonomicproblems.Ann.Eugenics(1936):111. 91

PAGE 92

[40] Fletcher,P.T.andJoshi,S.Principalgeodesicanalysisonsymmetricspaces:Statisticsofdiffusiontensors.EuropeanConferenceonComputerVisionComputerVisionApproachestoMedicalImageAnalysisWorkshop(2004). [41] Fletcher,P.T.,Venkatasubramanian,S.,andJoshi,S.ThegeometricmedianonRiemannianmanifoldswithapplicationtorobustatlasestimation.NeuroImage45(2008). [42] Flynn,P.J.andJain,A.K.CAD-basedcomputervision:FromCADmodelstorelationalgraphs.IEEETransactionsonPatternAnalysisandMachineIntelligence13(1991):114. [43] Frank,A.andAsuncion,A.UCImachinelearningrepository.2010.URL http://archive.ics.uci.edu/ml [44] Freund,Y.Anadaptiveversionoftheboostbymajorityalgorithm.MachineLearning43(2001):293. [45] Freund,Y.andSchapire,R.E.Largemarginclassicationusingtheperceptronalgorithm.MachineLearning(1999):277. [46] Friedman,J.,Hastie,T.,andTibshirani,R.Additivelogisticregression:Astatisticalviewofboosting.AnnalsofStatistics(2000):337. [47] Frigyik,B.A.,Srivastava,S.,andGupta,M.R.FunctionalBregmandivergence.IEEEInternationalSymposiumonInformationTheory(2008). [48] Gavrila,D.M.ABayesian,exemplar-basedapproachtohierarchicalshapematching.IEEETransactionsonPatternAnalysisandMachineIntelligence(2007):1408. [49] Golub,GeneH.andLoan,CharlesF.Van.Matrixcomputations.JohnsHopkinsUniversityPress(1996). [50] Golub,T.,Slonim,D.K.,Tamayo,P.,Huard,C.,Gaasenbeek,M.,Mesirov,J.P.,Coller,H.,Loh,M.L.,Downing,J.R.,Caligiuri,M.A.,Bloomeld,C.D.,andLander,E.S.Molecularclassicationofcancer:classdiscoveryandclasspredictionbygeneexpressionmonitoring.Science286(1999):531. [51] Hamarneh,G.andHradsky,J.BilaterallteringofdiffusiontTensormagneticresonanceimages.IEEETransactionsonImageProcessing16(2007).10:2463. [52] Helein,F.Harmonicmaps,conservationlawsandmovingframes.(2002). [53] Ho,J.,Yang,M.,Rangarajan,A.,andVemuri,B.C.Anewafneregistrationalgorithmformatching2Dpointsets.IEEEWorkshoponApplicationsofCom-puterVision89(2007):25. 92

PAGE 93

[54] Jian,B.andVemuri,B.C.Arobustalgorithmforpointsetregistrationusingmixtureofgaussians.IEEEInternationalConferenceonComputerVision2(2005):1246. [55] Joshi,S.,Davis,B.,Jomier,M.,andGerig,G.Unbiaseddiffeomorphicatlasconstructionforcomputationalanatomy.NeuroImage(2004):S151S160. [56] Joshi,S.H.,Klassen,E.,Srivastava,A.,andJermyn,I.AnovelrepresentationforriemanniananalysisofelasticcurvesinRn.IEEEConferenceonComputerVisionandPatternRecognition(2007):1. [57] Karcher,H.Riemanniancenterofmassandmolliersmoothing.Communica-tionsonPureandAppliedMathematics30(1977). [58] Koay,ChengGuanandBasser,PeterJ.AnalyticallyexactcorrectionschemeforsignalextractionfromnoisymagnitudeMRsignals.JournalofMagneticResonance179(2006).2:317. [59] Kodipaka,S.,Banerjee,A.,andVemuri,B.C.Largemarginpursuitforaconicsectionclassier.IEEEConferenceonComputerVisionandPatternRecognition(2008). [60] Kontschieder,P.,Donoser,M.,andBischof,H.Beyondpairwiseshapesimilarityanalysis.AsianConferenceonComputerVision(2009):655. [61] Kullback,S.Informationtheoryandstatistics.WileyNewYork,1959. [62] Lancaster,J.L.,Kochunov,P.V.,Fox,P.T.,andNickerson,D.k-Treemethodforhigh-speedspatialnormalization.HumanBrainMapping6(1998):358. [63] Latecki,L.J.,Lakamper,R.,andEckhardt,U.Shapedescriptorsfornon-rigidshapeswithasingleclosedcontour.IEEEConferenceonComputerVisionandPatternRecognition1(2000):424. [64] Lenglet,C.,Rousson,M.,andDeriche,R.Segmentationof3Dprobabilitydensityeldsbysurfaceevolution:ApplicationtodiffusionMRI.InternationalConferenceonMedicalImageComputingandComputerAssistedIntervention(MICCAI)3216(2004):18. [65] Leyton,M.Aprocess-grammarforshape.ArticialIntelligence34(1988):213. [66] Ling,H.andJacobs,D.Usingtheinner-distanceforclassicationofarticulatedshapes.IEEEConferenceonComputerVisionandPatternRecognition2(2005). [67] Ling,H.andOkada,K.Anefcientearthmoversdistancealgorithmforrobusthistogramcomparison.IEEETransactionsonPatternAnalysisandMachineIntelligence29(2007):840. 93

PAGE 94

[68] Liu,M.andVemuri,B.C.RBOOST:Riemanniandistancebasedtegularizedboosting.IEEEInternationalSymposiumonBiomedicalImaging(2011):1831. [69] .RobustandefcientregularizedboostingusingtotalBregmandivergence.IEEEConferenceonComputerVisionandPatternRecognition(2011):2897. [70] Liu,Meizhu,Vemuri,BabaC,Amari,Shun-Ichi,andNielsen,Frank.TotalBregmandivergenceanditsapplicationstoshaperetrieval.IEEEConferenceonComputerVisionandPatternRecognition(2010):3463. [71] Macrini,D.,Siddiqi,K.,andDickinson,S.Fromskeletonstobonegraphs:Medialabstractionforobjectrecognition.IEEEConferenceonComputerVisionandPatternRecognition(2008):1. [72] Maji,S.,Berg,A.C.,andMalik,J.Classicationusingintersectionkernelsupportvectormachinesisefcient.IEEEConferenceonComputerVisionandPatternRecognition(2008):1. [73] Malladi,R.,Sethian,J.A.,andVemuri,B.C.Shapemodelingwithfrontpropagation:Alevelsetapproach.IEEETransactionsonPatternAnalysisandMachineIntelligence17(1995).2. [74] Marcus,D.S.,H.Wang,T.,Parker,J.,Csernansky,J.G.,Morris,J.C.,andBuckner,R.L.Openaccessseriesofimagingstudies(OASIS):Cross-sectionalMRIdatainyoung,middleaged,nondemented,anddementedolderadults.JournalofCognitiveNeuroscience19(2007):1498. [75] Masip,D.,Lapedriza,A.,andVitria,J.Boostedonlinelearningforfacerecognition.IEEETransactionsonSystems,Man,andCybernetics-PartB:Cybernetics(2009):530. [76] McNeill,G.andVijayakumar,S.Hierarchicalprocrustesmatchingforshaperetrieval.IEEEConferenceonComputerVisionandPatternRecognition1(2006). [77] Mika,S.,Ratsch,G.,Weston,J.,Scholkopf,B.,andMullers,K.R.Fisherdiscriminantanalysiswithkernels.IEEENeuralNetworksforSignalProcessing(1999):41. [78] Mishra,A,Lu,Yonggang,Meng,Jingjing,Anderson,AdamW.,andDing,Zhaohua.Uniedframeworkforanisotropicinterpolationandsmoothingofdiffusiontensorimages.NeuroImage31(2006).4:1525. [79] Moakher,M.andBatchelor,P.G.Symmetricpositive-denitematrices:Fromgeometrytoapplicationsandvisualization.VisualizationandProcessingofTensorFields,Springer,2006. 94

PAGE 95

[80] Mokhtarian,F.,Abbasi,S.,andKittler,J.Efcientandrobustretrievalbyshapecontentthroughcurvaturescalespace.InternationalWorkshoponImageDatabasesandMulti-MediaSearch(1997):51. [81] Mori,G.,Belongie,S.,andMalik,J.Efcientshapematchingusingshapecontexts.IEEETransactionsonPatternAnalysisandMachineIntelligence27(2005). [82] Morra,J.H.,Tu,Z.,Apostolova,L.G.,Green,A.E.,Toga,A.W.,andThompson,P.M.ComparisonofAdaBoostandsupportvectormachinesfordetectingAlzheimersdiseasethroughautomatedhippocampalsegmentation.IEEETransactionsonMedicalImaging(2007). [83] Mumford,D.andShah,J.Optimalapproximationsbypiecewisesmoothfunctionsandassociatedvariational-problems.CommunicationsonPureandAppliedMathematics42(1989). [84] Nielsen,F.andNock,R.Onthesmallestenclosinginformationdisk.InformationProcessingLetters105(2008):93. [85] .SidedandsymmetrizedBregmancentroids.IEEETransactionsonInformationTheory(2009):1. [86] Nocedal,JorgeandWright,StephenJ.Numericaloptimization.Springer,2000. [87] Okuma,K.,Taleghani,A.,Freitas,N.,Little,J.,andLowe,D.Aboostedparticlelter:Multitargetdetectionandtracking.EuropeanConferenceonComputerVision(2004):28. [88] Opelt,A.,Pinz,A.,Fussenegger,M.,andAuer,P.Genericobjectrecognitionwithboosting.IEEETransactionsonPatternAnalysisandMachineIntelligence28(2006).3:416. [89] Osada,R.,Funkhouser,T.,Chazelle,B.,andDobkin,D.Shapedistributions.ACMTransactionsonGraphics21(2002).4:807. [90] Pajevic,S.andBasser,P.J.Parametricandnon-parametricstatisticalanalysisofDT-MRIdata.JournalofMagneticResonance161(2003). [91] Papadakis,N.G.,Xing,D.,Houston,G.C.,Smith,J.M.,Smith,M.I.,James,M.F.,Parsons,A.A.,Huang,C.L.,Hall,L.D.,andCarpenter,T.A.Astudyofrotationallyinvariantandsymmetricindicesofdiffusionanisotropy.MagneticResonanceImaging(1999). [92] Papadakis,S.P.P.,Pratikakis,I.,andTheoharis,T.Efcient3Dshapematchingandretrievalusingaconcreteradializedsphericalprojectionrepresentation.PatternRecognition40(2007).9:2437. 95

PAGE 96

[93] Pasternak,O.,Sochen,N.,andBasser,P.J.TheeffectofmetricselectionontheanalysisofdiffusiontensorMRIdata.NeuroImage(2010). [94] Pennec,X.,Fillard,P.,andAyache,N.Ariemannianframeworkfortensorcomputing.InternationalJournalofComputerVision66(2006).1:41. [95] Perrotton,X.,Sturzel,M.,andRoux,M.Implicithierarchicalboostingformulti-viewobjectdetection.IEEEConferenceonComputerVisionandPat-ternRecognition(2010). [96] Peter,A.,Rangarajan,A.,andHo,J.ShapeL'Anerouge:slidingwaveletsforindexingandretrieval.IEEEConferenceonComputerVisionandPatternRecognition(2008). [97] Pierpaoli,C.andBasser,P.J.Towardaquantitativeassessmentofdiffusionanisotropy.MagneticResonanceinMedicine36(1996). [98] Pobil,A.P.D.andSerna,M.A.Spatialrepresentationandmotionplanning.LectureNotesinComputerScience,Springer-Verlag(1995).1014:169. [99] Pomeroy,S.L.,Tamayo,P.,Gaasenbeek,M.,Sturla,L.M.,Angelo,M.,McLaughlin,M.E.,Kim,J.Y.H.,Goumnerova,L.C.,Black,P.M.,Lau,C.,Allen,J.C.,Zagzag,D.,Olson,J.M.,Curran,T.,Wetmore,C.,Biegel,J.A.,Poggio,T.,Mukherjee,S.,Rifkin,R.,Califano,A.,Stolovitzky,G.,Louis,D.N.,Mesirov,J.P.,Lander,E.S.,andGolub,T.R.Predictionofcentralnervoussystemembryonaltumouroutcomebasedongeneexpression.Nature415(2002):436. [100] Poupon,C.,Roche,A.,Dubois,J.,Mangin,J.,andPoupon,F.Real-timeMRdiffusiontensorandQ-ballimagingusingKalmanltering.MedicalImageAnalysis12(2008):527. [101] Saffari,A.,Godec,M.,Pock,T.,Leistner,C.,andBischof,H.Onlinemulti-classLPBoost.IEEEConferenceonComputerVisionandPatternRecognition(2010). [102] Saffari,A.,Leistner,C.,andBischof,H.Regularizedmulti-classsemi-supervisedboosting.IEEEConferenceonComputerVisionandPatternRecognition(2009). [103] Salvador,R.,Pena,A.,Menon,D.K.,Carpenter,T.A.,Pickard,J.D.,andBullmore,E.T.Formalcharacterizationandextensionofthelinearizeddiffusiontensormodel.HumanBrainMapping24(2005).2:144. [104] Schapire,R.andSinger,Y.Improvedboostingalgorithmsusingcondence-ratedpredictions.MachineLearning37(1999).3:297. [105] Sebastian,T.,Klein,P.,andKimia,B.Onaligningcurves.IEEETransactionsonPatternAnalysisandMachineIntelligence25(2003).1:116. 96

PAGE 97

[106] Sebastian,T.B.,Klein,P.N.,andKimia.,B.B.Recognitionofshapesbyeditingtheirshockgraphs.IEEETransactionsonPatternAnalysisandMachineIntelligence26(2004).5:550. [107] Shilane,P.,Min,P.,Kazhdan,M.,andFunkhouser,T.ThePrincetonshapebenchmark.ProceedingsofShapeModelingInternational(2004):167. [108] Shotton,J.,Blake,A.,andCipolla,R.Multiscalecategoricalobjectrecognitionusingcontourfragments.IEEETransactionsonPatternAnalysisandMachineIntelligence30(2008):1270. [109] Siddiqi,K.andPizer,S.M.Medialrepresentations:Mathematics,algorithmsandapplications.Springer,2008. [110] Soderkvist,O.J.O.Computervisionclassicationofleavesfromswedishtrees.LinkopingUniversity,Sweden(2001). [111] Srivastava,A.,Jermyn,I.,andJoshi,S.Riemanniananalysisofprobabilitydensityfunctionswithapplicationsinvision.IEEEConferenceonComputerVisionandPatternRecognition(2007):1. [112] Stejskal,E.O.andTanner,J.E.Spindiffusionmeasurements:Spinechoesinthepresenceofatimedependenteldgradient.JournalofChemicalPhysis42(1965). [113] Strehl,A.andGhosh,J.Clusterensembles-aknowledgereuseframeworkforcombiningpartitionings.JournalofMachineLearningResearch3(2002).3:583. [114] Tang,S.,Fan,Y.,Zhu,H.,Yap,P.,Gao,W.,Lin,W.,andShen,D.Regularizationofdiffusiontensoreldusingcoupledrobustanisotropicdiffusion.IEEEWork-shoponMathematicalMethodsinBiomedicalImageAnalysis(2009):52. [115] Temlyakov,A.,Munsell,B.C.,Waggoner,J.W.,andWang,S.Twoperceptuallymotivatedstrategiesforshapeclassication.IEEEConferenceonComputerVisionandPatternRecognition(2010):2289. [116] Thomas,J.A.andCover,T.M.Elementsofinformationtheory.WileyInterscience,1991. [117] Tieu,K.andViola,P.Boostingimageretrieval.InternationalJournalofComputerVision56(2004).1:17. [118] Torralba,A.,Murphy,K.P.,andFreeman,W.T.Sharingfeatures:efcientboostingproceduresformulticlassobjectdetection.IEEEConferenceonComputerVisionandPatternRecognition(2004):762. [119] Tristan-Vega,A.andAja-Fernandez,S.DWIlteringusingjointinformationforDTIandHARDI.MedicalImageAnalysis14(2010).2:205. 97

PAGE 98

[120] Tschumperle,D.andDeriche,Ra.VariationalframeworksforDT-MRIestimation,regularizationandvisualization.IEEEInternationalConferenceonComputerVision1(2003). [121] Tu,Z.andYuille,A.Shapematchingandrecognition:Usinggenerativemodelsandinformativefeatures.EuropeanConferenceonComputerVision3(2004):195. [122] Vapnik,V.Statisticallearningtheory.JohnWileyandSons,NewYork(1999). [123] Vemuri,B.C.,Liu,M.,Amari,S.,andNielsen,F.TotalBregmandivergenceanditsapplicationstoDTIanalysis.IEEETransactionsonMedicalImaging30(2011).2:475. [124] Vemuri,BabaC,Chen,Y.,Rao,M.,McGraw,T.,Wang,Z.,andMareci,T.FibertractmappingfromdiffusiontensorMRI.ProceedingsofIEEEWorkshopVariationalandLevelSetMethods(2001). [125] Viola,P.andJones,M.J.Robustreal-timefacedetection.InternationalJournalofComputerVision57(2004).2:137. [126] Vranic,D.V.Toolsfor3Dmodelretrieval.http://merkur01.inf.uni-konstanz.de/3Dtools/(2005). [127] Wang,Z.andVemuri,B.C.Tensor-eldsegmentationusingaregion-basedactivecontourmodel.EuropeanConferenceonComputerVision(2004). [128] .DTIsegmentationusinganinformationtheoretictensordissimilaritymeasure.IEEETransactionsonMedicalImaging24(2005). [129] Wang,Z.,Vemuri,B.C.,Chen,Y.,andMareci,T.H.AconstrainedvariationalprinciplefordirectestimationandsmoothingofthediffusiontensoreldfromcomplexDWI.IEEETransactionsonMedicalImaging23(2004).8:930. [130] Warmuth,M.K.,Glocer,K.A.,andRaetsch,G.Boostingalgorithmsformaximizingthesoftmargin.AdvancesinNeuralInformationProcessingSys-tems(2007). [131] Warmuth,M.K.,Glocer,K.A.,andVishwanathan,S.V.EntropyregularizedLPBoost.InternationalConferenceonAlgorithmicLearningTheory(2008):256. [132] Weinland,D.andBoyer,E.Actionrecognitionusingexemplar-basedembedding.IEEEConferenceonComputerVisionandPatternRecognition(2008):1. [133] Westin,C.F.andMaier,S.E.AdualtensorbasissolutiontotheStejskal-TannerequationsforDT-MRI.InternationalSocietyofMagneticResonanceinMedicine(ISMRM)(2002). 98

PAGE 99

[134] Wiest-Daessle,Nicolas,Prima,Sylvain,Coupe,Pierrick,Morrissey,SeanPatrick,andBarillot,Christian.Non-Localmeansvariantsfordenoisingofdiffusion-weightedanddiffusiontensorMRI.InternationalConferenceonMedicalImageComputingandComputerAssistedIntervention(MICCAI)10(2007).2:344. [135] Wiest-Daessle,Nicolas,Prima,Sylvain,Coupe,Pierrick,Morrissey,SeanPatrick,andBarillot,Christian.Riciannoiseremovalbynon-localmeanslteringforlowsignal-to-noiseratioMRI:ApplicationstoDT-MRI.InternationalConferenceonMedicalImageComputingandComputerAssistedIntervention(MICCAI)11(2008).2:171. [136] Xu,C.,Liu,J.,andTang,X.Dshapematchingbycontourexibility.IEEETransactionsonPatternAnalysisandMachineIntelligence31(2009).1:180. [137] Yang,X.,Tezel,S.K.,andLatecki,L.Locallyconstraineddiffusionprocessonlocallydensieddistancespaceswithapplicationstoshaperetrieval.IEEEConferenceonComputerVisionandPatternRecognition(2009):357. [138] Younes,L.Computableelasticdistancebetweenshapes.SIAMJournalofAppliedMathematics58(1998).2:565. [139] Zeisl,B.,Leistner,C.,Saffari,A.,andBischof,H.On-linesemi-supervisedmultiple-instanceboosting.IEEEConferenceonComputerVisionandPatternRecognition(2010):1879. [140] Zhang,J.Divergencefunction,duality,andconvexanalysis.NeuralComputation16(2004):159. [141] Zhu,J.,Rosset,S.,Zou,H.,andHastie,T.Multi-classAdaBoost.Technicalreport,StanfordUniversity(2006). 99

PAGE 100

BIOGRAPHICALSKETCH MeizhuLiuwasborninKaifeng,Henan,P.R.China.ShereceivedherBachelorofSciencedegreefromtheComputerScienceDepartmentattheUniversityofScienceandTechnologyofChina,P.R.China,in2007.SheearnedherMasterofSciencedegreefromDepartmentofComputerandInformationScienceandEngineeringattheUniversityofFlorida,Gainesville,inMay2011.ShewillreceiveherDoctorofPhilosophydegreefromDepartmentofComputerandInformationScienceandEngineeringattheUniversityofFlorida,Gainesville,inDecember2011.Herresearchinterestsincludecomputervision,machinelearning,imageprocessingandmedicalimageanalysis,informationtheory,andcomputationalgeometry. 100