<%BANNER%>

Vision Based Robust Vehicle Detection and Tracking VIA Active Learning

Permanent Link: http://ufdc.ufl.edu/UFE0045565/00001

Material Information

Title: Vision Based Robust Vehicle Detection and Tracking VIA Active Learning
Physical Description: 1 online resource (62 p.)
Language: english
Creator: Karakkat-Narayanan, Vishnu
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2013

Subjects

Subjects / Keywords: detection -- robust -- tracking -- vehicle -- vision
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
Genre: Mechanical Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: This thesis aims to introduce a novel robust real time system capable of rapidly detecting and tracking vehicles in a video stream using a monocular vision system. The framework used for this purpose is an actively learned implementation of the Haar-like feature based Viola-Jones classifier integrated with a Lucas-Kanade Optical Flow Tracker and a distance estimation algorithm.  A passively trained supervised system is initially built by using Rectangular Haar-like features. Several increasingly complex weak classifiers,(which are essentially a degenerative set of Decision Tree classifiers) are trained at the start. These weakly trained classifiers are then conjunctively cascaded based on Adaboost to form a strong classifier which results in the elimination of much of the background and works on regions of the image which are more likely to be the candidate. This leads to an increase in speed and reduction in false alarm rates. An actively learned model is then generated from the initial passive classifier by querying misclassified instances when the model is evaluated on an independent dataset. This actively trained system is then integrated with a Lucas-Kanade optical flow tracker and distance estimator algorithm to build a complete multi-vehicle detection and tracking system capable of performing in real time. The built model is then evaluated extensively on static as well as real world data and results are presented.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Vishnu Karakkat-Narayanan.
Thesis: Thesis (M.S.)--University of Florida, 2013.
Local: Adviser: Crane, Carl D, Iii.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2013
System ID: UFE0045565:00001

Permanent Link: http://ufdc.ufl.edu/UFE0045565/00001

Material Information

Title: Vision Based Robust Vehicle Detection and Tracking VIA Active Learning
Physical Description: 1 online resource (62 p.)
Language: english
Creator: Karakkat-Narayanan, Vishnu
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2013

Subjects

Subjects / Keywords: detection -- robust -- tracking -- vehicle -- vision
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
Genre: Mechanical Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: This thesis aims to introduce a novel robust real time system capable of rapidly detecting and tracking vehicles in a video stream using a monocular vision system. The framework used for this purpose is an actively learned implementation of the Haar-like feature based Viola-Jones classifier integrated with a Lucas-Kanade Optical Flow Tracker and a distance estimation algorithm.  A passively trained supervised system is initially built by using Rectangular Haar-like features. Several increasingly complex weak classifiers,(which are essentially a degenerative set of Decision Tree classifiers) are trained at the start. These weakly trained classifiers are then conjunctively cascaded based on Adaboost to form a strong classifier which results in the elimination of much of the background and works on regions of the image which are more likely to be the candidate. This leads to an increase in speed and reduction in false alarm rates. An actively learned model is then generated from the initial passive classifier by querying misclassified instances when the model is evaluated on an independent dataset. This actively trained system is then integrated with a Lucas-Kanade optical flow tracker and distance estimator algorithm to build a complete multi-vehicle detection and tracking system capable of performing in real time. The built model is then evaluated extensively on static as well as real world data and results are presented.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Vishnu Karakkat-Narayanan.
Thesis: Thesis (M.S.)--University of Florida, 2013.
Local: Adviser: Crane, Carl D, Iii.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2013
System ID: UFE0045565:00001


This item has the following downloads:


Full Text

PAGE 1

VISIONBASEDROBUSTVEHICLEDETECTIONANDTRACKINGVIAACTIVELEARNINGByVISHNUKARAKKATNARAYANANATHESISPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFMASTEROFSCIENCEUNIVERSITYOFFLORIDA2013

PAGE 2

c2013VishnuKarakkatNarayanan 2

PAGE 3

Tomyparents,P.V.NarayananandSudhaNarayanan 3

PAGE 4

ACKNOWLEDGMENTS IwouldliketoexpressmygratitudetomyadviserDr.CarlCraneforhiscontributionandguidanceduringtheprojectphaseandthroughoutmyMastersstudy.IwouldalsoliketothankmycommitteemembersDr.PrabirBarooahandDr.AntonioArroyofortheirhelpandsuggestions.Furthermore,IamextremelygratefultomylabmembersRyanChilton,ManinderSinghPandaandDarshanPatelfortheirassistance.IwouldliketoextendabigthankyoutoClintP.Georgeforhishelpinwritingthisthesis.IoffermyregardstoallmyfriendswhodirectlyorindirectlycontributedtowardsbuildingthissystemespeciallyPaulThotakkara,RahulSubramany,GokulBhatandJacobJames.Finally,IthankmyparentsSudhaNarayananandP.V.NarayananandmybrotherVarunNarayananforalltheirencouragement,everlastingloveandcarewhichwastheguidinglightthroughoutmystudiesandmylifeandwithoutwhomIwouldneverbeinthisposition. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 9 CHAPTER 1INTRODUCTION ................................... 10 1.1Organization .................................. 12 2RELATEDRESEARCH ............................... 13 2.1VisionBasedVehicleDetection ........................ 13 2.1.1MachineLearningBasedApproaches ................ 15 2.1.2ActiveLearningBasedObjectClassication ............. 17 2.2VehicleTrackingAndDistanceEstimation .................. 19 2.2.1VehicleTracking ............................. 19 2.2.2DistanceEstimationFromaMonocularCamera ........... 21 2.3Summary .................................... 22 3ACTIVELYTRAINEDVIOLA-JONESCLASSIFIER ................ 23 3.1Viola-JonesObjectDetectionFramework .................. 23 3.1.1FeatureSelectionandImageRepresentation ............ 23 3.1.1.1RectangularHaar-likefeatures ............... 23 3.1.1.2Integralimagerepresentation ................ 24 3.1.2AdaboostBasedClassier ....................... 25 3.1.3ClassierCascade ........................... 27 3.2PassivelyTrainedClassier .......................... 28 3.3ActiveLearningPrinciples ........................... 30 3.3.1QueryByCondence ......................... 31 3.3.2QueryByMisclassication ....................... 31 3.4ActiveLearningBasedClassier ....................... 32 3.5Summary .................................... 34 4VEHICLETRACKINGANDDISTANCEESTIMATION .............. 35 4.1Lucas-KanadeOpticalFlowBasedTracking ................. 35 4.1.1Method ................................. 36 4.1.2PyramidalImplementation ....................... 37 4.1.3Implementation ............................. 38 4.2DistanceEstimation .............................. 39 5

PAGE 6

4.3Summary .................................... 41 5EVALUATIONANDDISCUSSION ......................... 43 5.1Datasets ..................................... 43 5.1.1Caltech2001/1999StaticImageDatabase .............. 43 5.1.2mvBlueFOXTestData ......................... 43 5.2EvaluationParameters ............................. 45 5.3StaticDataset .................................. 46 5.4RealTimeDataset ............................... 49 5.5ImplementationinaRealTimeScenario ................... 53 5.6Summary .................................... 53 6CONCLUSIONSANDFUTUREWORK ...................... 55 REFERENCES ....................................... 57 BIOGRAPHICALSKETCH ................................ 62 6

PAGE 7

LISTOFTABLES Table page 3-1PassiveTrainingParameters ............................ 30 3-2ActiveTrainingParameters ............................. 34 5-1ResultsonStaticDataset-Active-PassiveComparision ............. 47 5-2ResultsonRealTimeDataset-Active-PassiveComparision .......... 52 7

PAGE 8

LISTOFFIGURES Figure page 3-1RectangularHaar-likeFeatures ........................... 24 3-2IntegralImageRepresentation ........................... 26 3-3SchematicofaClassierCascade ......................... 27 3-4Exampleofapositiveimage ............................. 29 3-5Exampleofanegativeimage ............................ 30 3-6SchematicoftheActiveLearningFramework ................... 33 3-7Examplesqueriedforre-training .......................... 33 4-1PyramidalImplementationofLKOpticalFlow ................... 38 4-2SchematicshowingthealgorithmicowoftheTrackingProcess ........ 39 4-3AlgorithmicowofthecompleteDetectionandTrackingProcess ........ 42 5-1Caltech2001/1999StaticImageDataExample .................. 44 5-2RealTimetestdatacapturedusingthetestrig .................. 44 5-3ExamplesofTruePositiveDetectionsinStaticDatset .............. 47 5-4Examplesoffalsepositivesandmisseddetections ................ 47 5-5TheROCcurvesforActivelyandPassivelyTrainedClassiers ......... 48 5-6ExamplesofTruePositiveDetectionsinRealTimeTestset ........... 49 5-7Examplesoffalsepositivesandmisseddetections ................ 50 5-8TestdataDetectionTrack .............................. 50 5-9TheROCcurvesforActivelyandPassivelyTrainedClassiers ......... 52 5-10ScreenshotofFullSystem1 ............................ 53 5-11ScreenshotofFullSystem2 ............................ 54 8

PAGE 9

AbstractofThesisPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofMasterofScienceVISIONBASEDROBUSTVEHICLEDETECTIONANDTRACKINGVIAACTIVELEARNINGByVishnuKarakkatNarayananMay2013Chair:CarlD.Crane,IIIMajor:MechanicalEngineeringThisthesisaimstointroduceanovelrobustrealtimesystemcapableofrapidlydetectingandtrackingvehiclesinavideostreamusingamonocularvisionsystem.TheframeworkusedforthispurposeisanactivelylearnedimplementationoftheHaar-likefeaturebasedViola-JonesclassierintegratedwithaLucas-KanadeOpticalFlowTrackerandadistanceestimationalgorithm.ApassivelytrainedsupervisedsystemisinitiallybuiltbyusingRectangularHaar-likefeatures.Severalincreasinglycomplexweakclassiers,(whichareessentiallyadegenerativesetofDecisionTreeclassiers)aretrainedatthestart.TheseweaklytrainedclassiersarethenconjunctivelycascadedbasedonAdaboosttoformastrongclassierwhichresultsintheeliminationofmuchofthebackgroundandworksonregionsoftheimagewhicharemorelikelytobethecandidate.Thisleadstoanincreaseinspeedandreductioninfalsealarmrates.Anactivelylearnedmodelisthengeneratedfromtheinitialpassiveclassierbyqueryingmisclassiedinstanceswhenthemodelisevaluatedonanindependentdataset.ThisactivelytrainedsystemisthenintegratedwithaLucas-Kanadeopticalowtrackeranddistanceestimatoralgorithmtobuildacompletemulti-vehicledetectionandtrackingsystemcapableofperforminginrealtime.Thebuiltmodelisthenevaluatedextensivelyonstaticaswellasrealworlddataandresultsarepresented. 9

PAGE 10

CHAPTER1INTRODUCTIONMorepeopleareaffectedbyautomotiveaccidentinjuriesthananyotheraccidentrelatedinjuries.Ithasbeenestimatedthateveryyear,about20-50millionpeopleareinjuredduetoautomotiveaccidents.Around1.2millionpeoplelosetheirlivesasaresult.Therearereportssuggestingthat1to3%oftheworld'sdomesticgrossproductisspentonhealthcareandothercostswhichareattributedtoautoaccidents.Consequently,overthelastdecade,therehasbeenalotofresearchpurelydevotedtothestudyanddevelopmentofintelligentautomotivesafetysystemsandsafeautonomousvehiclesamongtheIntelligentTransportationandRoboticscommunity.Amajorityofsuchresearchconductedbyacademiciansandvehiclemanufacturersinthiseldisrelatedtothedevelopmentofsystemswhicharecapableofdetectingandtrackingothervehiclesinrealtimetrafcbyusingavarietyofsensorslikecameras,LiDaRs(LaserDetectionandRangingSensor)andinfraredsensors.Oneofthemainchallengesassociatedwithdevelopingsuchsystemsisthattheyshouldbereliable,robustandsimpletoimplement[ 28 ].Currently,themajorityofsuchsystemsemployLiDaRbasedsensingwhichprovidehighlyaccurateinformationbutdoesnotpossesamechanismwhichcanbeexploitedforenhancedandintuitivedecisionmaking.Visionbasedvehicledetectionsystemshavebeenwidelycreditedandresearchedastheonethatislowcost,efcient,andhasthepotentialtodeliveraccurateinformationtothedriverregardingtrafc,pedestrians,lanefollowingandlanedepartures.Thepurposeofthisthesisistoshedsomelightontheapplicationofmachinelearningtechniquesintheareaofrobust,realtimevehicledetectionbasedonvision.Itisawell-knownfactthatvisionbasedvehicledetectionisachallengingproblem,astheenvironmentiseverchanging,dynamic,andcluttered.Thevehiclesareconstantlyinmotion,thereexistsadiversearrayofvehiclesizesandshapes,andilluminationisnotconstant.So,ageneralpatternrecognitionbasedmodelfordesigningsuchasystem 10

PAGE 11

wouldbeoflittlevalue.Therefore,thisthesismakestheuseofanactivelytrainedcascadeofboostedclassierswhicharetrainedonarealtimedatasettoefcientlydetectvehiclesinmotion.Theunderlyingframework,initiallydevelopedforfacedetectionbyViolaandJones[ 14 ],isparticularlyusefulfortheapplicationathandbecauseiteliminatestheeffectsofocclusion,scaleinvarianceandshapeconstraintswhichareinherentinclutteredsceneslikethetrafcstream.ActiveLearningisanupcomingeldofresearchinMachineLearningcircles.Theprincipleofactivelearningisthenotionthataclassier(orlearningfunction)hastheabilitytohavesomedegreeofcontroloverthedatawhichitlearns.Thistheoryhasbeensuccessfullyimplementedinareaslikedocumentmodeling,textcategorization,andnaturallanguageprocessingwithsignicantimprovementsinclassicationaccuracyandcomputationalload.ThisstudyaimstouseactivelearningtotweaktheclassierbuiltusingtheViola-Jonesframeworkmainlytoreducetheoccurrenceoffalsepositivesinon-roaddetectionalongwithimprovementinaccuracy.TheactivelytrainedsystemisthenintegratedwithatrackingalgorithmbasedonLucas-KanadeOpticalFlowandavehiclerangeestimationalgorithmtobuildacompletemulti-vehicledetectionandtrackingsystembasedonmonocularvisionwhichisreliable,robustandcapableofperforminginrealtimeusingstandardhardware.Anexhaustiveperformanceanalysisisalsopresentedbyevaluatingthedesignedsysteminstaticaswellasrealworlddatasets.Acomparisonbetweenthepassivelytrainedandtheactivelytrainedclassierisalsoexplainedandsomeavenuesforfutureworkareexplored.Themajorcontributionofthethesisisapreliminaryinvestigationintoactivelearningbasedrealtimevehicledetectionintrafcusingamonocularvisionsystem.Evaluationsonrealtimeandstaticimagedatasetsshedslightonthesystemsrecall,localizationandrobustness. 11

PAGE 12

1.1OrganizationThethesisisorganizedasfollows:Chapter2providesacomprehensivestudyontheresearchpertainingtothecurrentstudyexplainingtheevolutionofvisionbasedvehicledetection,thedevelopmentofmachinelearningbasedvisionprocessingandactivelearningbasedonroadvehicledetection.Backgroundregardingvisionbasedtrackingand3-Drangeestimationisalsogiven.Chapter3initiallydiscussestheViola-Jonesframeworkalongwithdesignofapassiveclassiertrainedonrealworlddata.ThechapterthenmovestodiscussActiveLearningprinciplesandexplainstheconstructionofanActivelytrainedclassierbyimprovinguponthepassiveclassier.Chapter4delineatestheLucas-KanadeFeaturetrackingemployedinthisworkinconjunctionwiththeactiveclassier.Adistanceestimationmethodbasedonapin-holecameramodelisalsoexplained.Further,Chapter5containsevaluationsanddiscussionsregardingtheimplementationofthemulti-vehicledetectionandtrackingsystemonrealworldandstaticdatasets.Anexhaustiveperformanceanalysisispresentedalongwithscopeforimprovement.Finally,Chapter6summarizesthewholeideaofthethesisanddiscussesthedirectionoffuturework. 12

PAGE 13

CHAPTER2RELATEDRESEARCHAnexhaustiveoverviewoftheresearchintheeldofvisionbasedvehicledetectionandtrackingispresentedinthischapter.Theliteraturepertainingtothisworkcanbebroadlydividedintotwocategories.Therstpartdealswithpapersconcerningvehicledetectionbasedonvision.Startingfromthealgorithmicapproachtothemachinelearningapproach,thebeautifulhierarchyoftheevolutionandimprovementsinthisareaispresented.Thesecondpartexplainsandcritiquespapersconcernedwithvehicletrackingalongwithworksrelativetodistanceestimationfromasinglecamera.Finally,asummaryofallthetechniquesispresented,alongwiththebriefintroductionofthenovelimprovementsthatthispresentworkaimstointroduce. 2.1VisionBasedVehicleDetectionVehicledetectionbasedonvisionhashadtremendousresearchexposureinthepasttwodecades,bothfromtheArticialIntelligenceandRoboticscommunityandtheIntelligentTransportationSystemscommunity(wherein,themajorityofresearchisonvisionbasedsystemsforvehicledetection).Therefore,aplethoraofworkrelevanttothisareacanbefoundwhichareevaluatedbothonreal-timeandstaticimages.ThetransitionfromstereovisiontomonocularvisioninthecaseofvehicledetectioncanbebestexplainedfromtheworkofBensrhairetal.[ 2 ].Theworkpresentsaco-operativeapproachtovehicledetection,wheremonocularvisionbasedandstereovisionbasedsystemsareevaluatedindividuallyandinanintegratedsystem.Themonocularsystemusedasimplemodelbasedtemplatematchingforthedetectionpurpose.Thestudyfoundoutthatstereosystemswere10timesslowerthanmonocularsystemswhenevaluatedonstaticimagesbuthadhigheraccuracyindetectingandestimatingtheactualdistance.AsimilarsystemwasdevelopedrecentlybySivaraman[ 27 ].Thissystemusedanactivelytrainedmonocularsystem(basedonHaar-like 13

PAGE 14

features)alongwithastereorigtocalculatethedepthmap.Thesystemhadveryhighaccuracybuttook50mstoprocessasingleframe.Betkeetal.[ 3 ],utilizedthecombinationofedge,color,andmotioninformationtodetectpossiblevehiclesandlanemarkings.Thiswasoneoftheearliestandpossiblyoneofthemostdeningattemptsatperformingreal-timevisionbasedvehicledetectionandtracking.Thealgorithmusedacombinationoftemplatematchingandatemporaldifferencingmethodperformedononlinecroppedimagestodetectcars.Lanemarkingsandtrackboundariesweretrackedusingarecursiveleastsquareslter.Thenalsystemwascapableofperformingataround15frames/secinreal-timebuthadmoderateaccuracy.Theamountoftimerequiredforpre-processingandalgorithmdevelopmentdoesnotjustifytheuseofthismethodinthepresentscenario.AdifferentapproachwasintroducedbyYamaguchietal.[ 34 ]wheretheego-motionofthevehicle(themotionofthevehiclecamera)wasestimatedtoinitiallyconstructa3-Drepresentationofthesceneandthendetectavehicle.Theego-motionisestimatedbythecorrespondenceofsomefeaturepointsbetweentwodifferentimagesinwhichtheobjectsarenotmoving.Afterthe3-Dreconstructionprocess,vehiclesaredetectedbytriangulation.Thissystemwascapableofperformingat10frames/secbutagainsufferedfromahighfalsepositiverate.Vehicledetectionatnighttimeisalsoanopenavenueforresearch.Theabovementionedtechniquescannotbeusedsuccessfullyinanighttimescenario.Chenetal.[ 7 ]proposedapatternanalysisbasedonataillightclusteringapproachforvehicleclassication.Thissystem(whichcanbeonlyemployedatnight)wasabletodetectcarsaswellasmotorbikeswithanaccuracyof97.5%.DetectingvehiclesintrafcfromanoverheadcameramountedinsideatunnelwasexploredbyWuetal[ 33 ].Theproposedalgorithmconstructedanautomaticlanedetectionsystembybackgroundsubtractionanddetectedvehiclesusingablockmatchingtechnique.Thismethodissuitableforoverheadtrafcdetectionbutitcannot 14

PAGE 15

bescaledtorearendsystemswherethecameraismovingconstantlyandthescenechangesateveryframe.Thus,methodsbasedonstaticblockmatchingortemplatematchingarenotsuitableforthepresentscenario.AlthoughImageProcessingtechniquesprovideasoundtheoreticalframeworkforon-roadvehicledetection,themoderateaccuracyandhighimplementationtimehasdeviatedthisresearchtowardsmachinelearningtechniquestosolvethisproblem. 2.1.1MachineLearningBasedApproachesOneofthemoreimportantaspectsofMachineLearningistheselectionofsuitablefeaturesforclassication[ 22 ].Selectingfeaturesinclutteredscenesliketrafcstreamsforobjectdetectionisnotatrivialissue.ThemajorityofworkreportedinthisareauseseitherHistogramofGradient(HOG)featuresorHaar-likefeaturescoupledwithasuitablelearningalgorithmforobjectclassication.Balconesetal.[ 1 ]proposedarearendcollisionwarningsystemwhichusedanextensivelypre-processedimagesequencetrainedonHOGfeaturesandclassiedusingaSupportVectorMachine(SVM).ThedetectedRegionofInterest(ROI)wastrackedusingaKalmanlter.Thissystemachievedveryhighdetectionratesbutsufferedfromaveryhighfalsepositiveratealso.AsimilarsystemwasderivedbySunetal.[ 21 ]wherethefeatureswereselectedbasedonHaarwavelettransformsinsteadofHOGfeatures.AnSVMbasedclassierwasusedtoverifythehypothesesgeneratedbythefeaturedetectionalgorithm.Thissystemgaveaframerateof10Hzwhenimplementedinrealtime.Buttheaccuracywasfoundtobemoderatewithhighfalsepositiverates.ApioneeringworkintheareaofMachineLearningbasedobjectdetectionwasreportedbyPapageorgiou[ 24 ]intheyear2000.ThepaperproposedabinarySupportVectorMachine(SVM)basedclassiertrainedonHaarwavlettransforms,whichareessentiallytheintensitydifferencesbetweendifferentregionsofanimage.Thismethod 15

PAGE 16

wastailormadeforobjectdetectioninclutteredscenes.Resultswerepresentedonstaticimagesforpedestriandetection,facedetection,andcardetection.In2001,MichealJonesandPaulViola[ 14 ]presentedtheirworkonRobustobjectdetectionthroughtrainingHaar-likefeaturesviaAdaboost.ThiswasineffectamajorimprovementfromthealgorithmproposedbyPapageorgiouasthesystemallowedveryrapiddetectionwhichfacilitatedtheuseofthealgorithminrealtimesystems.Thistheorywasvalidatedbyemployingthealgorithmforthepurposeoffacedetectionwithhighlysatisfactoryresults.Animplementationofthisveryframeworkisusedinthisthesisandisexplainedindetailinthefollowingchapter.Muchofthelaterworkintheareaofvisionbasedvehicledetectionusesaderivativeofthisframework.TheViola-Jonesalgorithmthushasbecomeaverypopularframeworkformonocularvisionbasedobjectdetection.Hanetal.[ 12 ]derivedasimplisticapproachtotheimplementationofthisframeworktovehicledetectionsystems.Thisalgorithmsearchestheimageforobjectswithshadowandeliminatestherestoftheimageregionasnoiseremoval.ThentheHaar-likefeaturedetectorisrunontheimagetoclassifytheROIasavehicleornon-vehicle.Thisstudyusedstaticimagesofcarsfortrainingpurposes,thustheaccuracyoftheclassierwaslimitedtoaround82%average.ArealtimesystembasedonHaar-likefeaturescombinedwithPairwiseGeometricalHistograms(PGH)wasproposedbyYongetal[ 35 ].PGHisapowerfulshapedescriptorwhichisinvarianttorotation.ThestudycomparedtheaccuracyofclassiersbuiltwithHaar-likefeaturesandthecombinationofHaar-likefeaturesandPGH.Theresultsshowedthatthecombinationclassierperformedat95.2%accuracywhencomparedto92.4%forthesingletypefeatureclassier.Thefalsepositiveratewasrelativelythesame.AdifferentmethodwasproposedbyKhalidetal[ 16 ]whereinahypothesiswasinitiallygeneratedbyacornerdetectoralgorithmandthishypothesiswasveriedbyrunningthegeneratedROIthroughanAdaboostbasedclassifer.Theclassierwas 16

PAGE 17

trainedonstaticimages,butthesystemgavegoodaccuracyasalotoffalsepositivecandidateswereeliminatedduringthecornerandedgedetectionprocess.Butthelackofagoodtrackingalgorithmhinderstheimplementationofthissysteminrealworldapplications.Choi[ 8 ]integratedanoncomingtrafcdetectorbasedon'opticalow'alongwiththetraditionalreartrafcdetectortoconstructacomprehensivevehicledetectionalgorithm.TheopticalowwasbasedoncorrespondenceoffeaturesdetectedbytheShi-Tomasicornerdetector.TherearendvehicledetectionsystemwasalsobasedonAdaboostworkingwithHaar-likefeatures.Again,theclassierwastrainedusinggenericstaticimagesofcarsandwerevalidatedonstaticimageswithanaccuracyof89%.AKalmanlterwasintegratedwiththesystemfortrackingpreviouslydetectedvehicles.AnexhaustivestudywasconductedbyTsaietal.[ 32 ]inwhichafullyrealtimecompatiblesystemwasdevelopedforvehicledetection.TheframeworkinitiallygeneratesahypothesisbasedonacornerdetectorwithSURFbaseddescriptorsandvalidatesthehypothesisbasedontheViola-Jonesalgorithm.AKalmanlerbasedtrackerwasintegratedtothesystem.Thestudyalsocomparedtheclassicationaccuracywithrespecttotheactualdistanceofthevehiclefromtheego-vehicle.Theminimumaccuracyachievedwas91.2%withathresholdmaximumdistanceof140m. 2.1.2ActiveLearningBasedObjectClassicationActiveLearning(AL),inbrief,isasystemwhichisabletoaskauser(orsomeotherinformationsource)interactivelyabouttheoutputofsomedatapoint.Itisfullyexplainedinthenextchapterwithconclusiveexamplesandtheory.Thischapterjustfocusesonthebackgroundofworkdoneinthisarea.ALisbeingincreasinglyusedinconjunctionwithobjectdetectionamongtheRoboticsresearchcircle.Amongthemanyadvantagesareimprovementofclassierperformance,reductionoffalsealarmratesandgenerationofnewdatafortraining.Thesearethemajorfactorsthatvalidatesactivelearningasavaluabletoolformachinelearningbasedobjectdetection.Oneofthemajordrawbacks 17

PAGE 18

oftheViola-Jonesalgorithmistheissueofhighfalsealarmrates.Thus,theuseofactivelearningtogetherwiththeViola-Jonesalgorithmcouldleadtoarobustclassier.AnentropybasedapproachforobjectdetectionbasedonActiveLearningwasproposedbyHolub[ 13 ].Thestudyfocusedonreductionintheamountoftrainingsamplesrequiredbyinteractivelyselectingtheimageswhichhadhigherandbetterfeaturedensity.Thiswasachievedbyqueryingsomesortofmaximizedinformationabouteachimage.Thestudyachievedupto10xreductionsinthenumberoftrainingsamplesamongthe170categoriesofimagesanalyzed.ThebackboneofActiveLearningisthetheselectionofsamplesforinteractivelabelingofanevaluateddataset.Kapooretal[ 15 ]devisedasystembasedonGaussianProcesseswithco-variancefunctionsbasedonPyramidMatchingKernelsforautonomouslyandoptimallyselectingdataforinteractivelabeling.Thisreducesthetimeconsumedbythevisionresearcherinlabelingthedataset.Themajoradvantageofthissystemisthatitcanbeemployedinverylargedatasetswithasmalltradeoffintheclassicationaccuracy.MuchoftheliteratureintheeldofusingActiveLearningforspecicallyvehicledetectionowestotheworkofSivaraman[ 30 ][ 29 ].Intwodifferentpapers,theauthorcomparedtwodifferentmethodsofgeneratingtrainingsamples,i.e.QuerybysamplingandQuerybymisclassication.TheresultsshowedthateventhoughQuerybymisclassicationwassubjecttohigherhumancapital,itleadtobetteraccuracy.Theaccuracyachievedwas95%withafalsepositiverateof6.4%.Thisprovedasignicantreductioninfalsepositiveswhencomparedtothepassivelearningmethodsmentionedabove.ThestudyalsointegratedaCondensationlertoconstructafullyautomaticrealtimevehicledetectionandtrackingsystem.AnexhaustivecomparativestudywasconductedbySivaraman[ 28 ]in2011,wheretheauthorcomparedvarioustechniquesexploredinmonocularvehicledetectionwithaparticularemphasisonBoostedclassierbasedmethods.Also,acostsensitivebased 18

PAGE 19

analysiswasconductedonthethreepopularmethodsofactivelearningandtheresultssuggestedthatQuerybyMisclassicationgavebetterresultsintermsofbetterprecisionandrecalleventhoughthecostofhumancapitalwashigher.Basedontheavailableliterature,anActiveLearningbasedsystemusingQuerybyMisclassicationwouldbeaperfectavenueforimprovingtheViola-Jonesalgorithm. 2.2VehicleTrackingAndDistanceEstimation 2.2.1VehicleTrackingTrackingrelevantpointsfromoneimageframetoanotherinavideostreamcanbeperformedbroadlyintwodifferentways;FeaturebasedandModelbased.Featurebasedtrackingisdonebymatchingcorrespondencebetweenextractedfeaturesbetweenthetwodifferentframes.Modelbasedtrackingassumessomesortofpriorinformationabouttheobjectinitiallyandupdatestheinformationandtracksitastheframesareprocessed.Thereisextensiveliteratureavailableontheapplicationofthesetwomethodsinrealtimevehicletrackingsystems.Modelbasedtrackinghasbeenextensivelystudiedbythevisionresearchcommunity,namelyKalmanlteringandParticleltering.Severalimprovementshavebeensuggestedforoptimizingtheperformanceoftheseltersforvehicledetction[ 18 ].Kobilarovetal[ 18 ]proposedaProbabilisticDataAssociationFilter(PDAF)insteadofasinglemeasurementKalmanlterforthepurposeofpeopletracking.Butthevisualmethodsufferedfromverylowaccuracyintrackingandcouldonlybeimprovedbyusingthecameraalongwithalasersensor.Particlelters(Condensationlters)havebeenusedextensivelyintheeldofobjecttracking[ 30 ][ 10 ].Recently,Bouttefroy[ 5 ]developedaprojectiveparticlelterwhichprojectstherealworldtrajectoryoftheobjectintothecameraspace.Thisprovidesabetterestimateoftheobjectposition.Thus,thevarianceofobservedtrajectoryisreducedwhichresultsinmorerobusttracking. 19

PAGE 20

Furthermore,Rezaeeetal.[ 26 ]introducedasystemwhichintegratedparticlelteringwithmultiplecuessuchascolor,edge,texture,andmotion.Thisfusedmethodhasprovedtobemuchmorerobustthanusingaconventionalparticlelter.Buteventhoughmodelbasedtrackingmethodsprovideareliableandrobustoptionfortrackingrelevantpixels,itisverycomputationallyinternsivewhichhindersitsapplicationinrealtimesystems.Thus,lookingintofeaturebasedtrackingmethodsbecameanecessaryoptionforresearchers.Withavarietyofalgorithmsbeingdesignedforcorrespondencebetweenthefeatures(forexample,Lucas-Kanade,Horn-Schunck,Farneback'smethodetc),visionresearchersincreasinglydeviatedtowardsfeaturebasedtrackingformainlyroboticapplications.Themostimportantcriteriatobuildasuccessfulsystemistheoptimalselectionoffeaturesandtheuseofasuitablematchingalgorithm.Caoetal.[ 6 ]usedtheKanade-Lucas-Tomasi(KLT)featuretrackerworkingwiththecornersofthedetectedvehiclesasfeaturestotrackvehiclesfromtheairinaUAV.ThisstandardimplementationoftheKLTfeaturetrackerisveryusefulforsceneswithoutocclusion.Dallalzadehetal.[ 9 ]proposedanovelapproachtodeterminethecorrespondencebetweenvehiclecornersforrobusttracking.Thisalgorithmworksontheextractedghostorcastshadowofthevehicleandwasabletoachieveanaccuracyof99.8%onaverage.Theonlylimitationofthisalgorithmistheextensiveprocessingpowerconsumedinpre-processingateveryframestep.OpticalFlowbasedtrackingisaformoffeaturebasedtrackinginwhichthealgorithmcalculatestheapparentvelocitiesofmovementofbrightnesspatternsinanimage.Thisgivesareallygoodestimateoftheimagemotion.Garcia[ 11 ]usedaheavilydownsampledimagefromavideostreamtotrackvehiclesonhighwaysusingtheLucas-Kanade(LK)method.Themajordrawbackofthisstudywastheuseofthefullimageregionasthefeaturetotrackwhichhinderedtheuseofthissysteminrealtime. 20

PAGE 21

HoweverthisstudyvalidatedtherobustnessoftheLKalgorithmforimagemotionestimationasitproducedanaccuracyof98%.Liuetal.[ 19 ]introducedamulti-resolutionopticalowbasedtrackingwithanimprovementtothestandardLucas-Kanade(LK)opticalowtotrackvehiclesfromtheair.ThefeaturesusedinconjunctionwiththeLKmodulewasthecornersofvehiclesdetectedbytheHarriscornerdetectoralgorithm.Thetrackingalgorithmcouldmatchfeatureswithhighrealtimeperformance.Thus,owingtotherealtimeapplicationandthehighlevelofrobustnessrequiredtheLucas-Kanade(LK)opticalowtrackerwasdeemedtobethebestsolutionforthecurrentscenario. 2.2.2DistanceEstimationFromaMonocularCameraConventionalstereobasedapproachesarenotsuitableformostrealtimeapplications.Also,estimationofobjectdistanceaccuratelyusingasinglecamerawhichismovingisachallengingtask.Therehasbeenvariousmethodsproposedtosolvethisproblem.Amovingaperturebasedsolutionwhichassumestheapertureofthelensinmotionwouldallowforanopticalowbasedalgorithmtotrackandmeasuredistance[ 31 ].Anotherapproachestimatedtheinter-vehiculardistancebyco-relatingtheextractedwidthsofthecars[ 17 ].Anotherstudysuggestedageneticalgorithmbasedoptimizationtechniquetoestimatedistancesfairlyaccurately[ 25 ].Butallthesemethodssufferfromslowexcecutiontimesorrequiretheuseofadditionalinformationsuchasanimageofascenetakenfromadifferentperspective.Thetaskofdistanceestimationfromasinglecamerabecomesfairlysimplewhenthedimensionsoftheobjectunderconsiderationareknown.UsingthefullycalibratedcameraparametersandthesizeoftheRegionofInterest(ROI),thedistanceoftheobjectfromthecamerain3-Dworldco-ordinatescanbeestimatedeasilyifthewidthandheightofthevehicleisknown[ 3 ].Thus,assumingastandardvalueforthevehicle 21

PAGE 22

dimensionseemsareasonableattempttomeasuredistancesothatthealgorithmcouldbedeployedinarealtimescenario. 2.3SummaryThischaptergaveanoverviewoftherelatedresearchintheeldofvisionbasedvehicledetectionfrommachinelearningbasedapproachestoactivelearning.Italsodiscussedrecentadvancementsintheareaofvehicletrackingandobjectdistanceestimationusingamonocularvisionsystem.ItwasfoundoutthatanactivelytrainedHaar-likefeaturedetector(basedontheViola-Jonesalgorithm)isaviablesolutionforrobustrapidobjectdetectioninclutteredscenes(liketrafcstreams).AnimplementationofthisalgorithmworkinginconjunctionwithaLucas-Kanade(LK)basedopticalowtrackingsystemintegratedwithadistancemeasurementsystemwouldbeaneffectivesolutionforarobustlightweightmodulecapableofdetectingandtrackingvehiclesinfrontoftheego-vehicle.Thissystemcouldbedeployedinanautonomousvehicleorinstandardvehiclesasadriverassistancesystem. 22

PAGE 23

CHAPTER3ACTIVELYTRAINEDVIOLA-JONESCLASSIFIERAdetailedexplanationoftheframeworkdevelopedbyMichealJonesandPaulViola[ 14 ]isprovidedinthischapterinconjunctionwiththeprincipleofActiveLearning.Themodiedactivelytrainedimplementationofthisalgorithmisthenpresentedwithrespecttotheuseofthisvehicledetectionframework.TheapproachproposedbyViolaandJoneswasinitiallydevelopedasarapidobjectdetectiontooltobeemployedinfacedetectionsystems.SincetheunderlyingfeaturesitusestoevaluatetheRegionsofInterest(ROIs)isaformofrectangularHaar-likefeatures(explainedbelow),itisparticularlysuitableforthevehicledetectionscenario[ 28 ](astheshapeofavehicleinavideostreamisdenedbyrectangularderivatives).Thefactthatthisframeworkgivesveryhighaccuracyalongwithrapiddetectionandthepropertyofscaleandrotationinvarianceprovesitsusefulnessinanimprovedimplementationofthisalgorithmonavehicledetectionframework. 3.1Viola-JonesObjectDetectionFramework 3.1.1FeatureSelectionandImageRepresentation 3.1.1.1RectangularHaar-likefeaturesThemainideaofworkingwithfeaturesisthatitismuchfasterthanapixelbasedclassicationsystemwhichisintegraltotheideaofrapiddetectioninrealtime.Theweakclassiers(explainedlaterindetail)workswithvaluesofverysimplefeatures.ThesefeaturesarederivativesofHaarbasisfunctionsusedbyPapageorgiouet.al[ 24 ]inhistrainableobjectdetectionframework.Thethreekindsoffeaturesusedinthisstudyare: TwoRectangleFeature:AsshowninFigure3-1,thevalueofatworectanglefeatureisthedifferencebetweenthesumofpixelvalueswithintworectangularregionsinaRegionofInterest(ROI).TheRegionshouldhavethesamesizeandshouldbehorizontallyorverticallyadjacent. 23

PAGE 24

ThreeRectangleFeature:Similarly,athreerectanglefeatureisthesumofthepixelsofthetwooutsiderectanglessubtractedfromthesumofpixelsofthecentertriangle. FourRectangleFeature:Afourrectanglefeatureisthedifferenceinthesumofpixelsoftwopairsofdiagonallyoppositerectangles. Figure3-1. TheRectangularfeaturesdisplayedwithrespecttothedetectionwindow.AandBrepresentsTwoRectangleFeatures,CrepresentsaThreeRectangleFeatureandDistheFourRectangleFeature. Theminimumsizeofthedetectionwindowwaschosentobe20x20basedontrialrunsandgiventhisinformation,thesetofrectangularfeaturesismuchhigherthanthenumberofpixelsinthewindow(totheorderof150,000).Thusthisrepresentationoffeaturesisovercompleteandasuitablefeatureselectionprocedurehastobeintegratedintothealgorithmtospeeduptheclassicationprocess. 3.1.1.2IntegralimagerepresentationOneofthethreemajorcontributionsoftheoriginalalgorithmistherepresentationoftheimagesintheformofanIntegralimage.Therectangularfeaturescanbecalculatedveryrapidly(andinconstanttime)usingthisintermediaterepresentationoftheimage. 24

PAGE 25

ThepixelvalueofanIntegralimageatapoint(x,y)isthesumofthepixelvaluesofthewholeregionaboveandthetoleftofthepoint(x,y)andcanbewrittenas: I(x,y)=Xax,byi(a,b)(3)whereI(x,y)istheIntegralimagerepresentationandi(a,b)istheoriginalimagerepresentation.Theabovestatecanbereachedfromthetwooperationsbelowinonepassovertheoriginalimage: s(x,y)=s(x,y)]TJ /F5 11.955 Tf 11.95 0 Td[(1)+i(x,y)(3) I(x,y)=I(x)]TJ /F5 11.955 Tf 11.95 0 Td[(1,y)+s(x,y)(3)wheres(x,y)isthecumulativerowsumands(x,-1)=0andI(-1,y)=0.Thereby,usingtheintegralimagerepresentation,anyrectangularsumcanbecalculatedinfourarrayoperations.TheFigure3-2showstheprocessofcalculatingtherectangularsumofaregionusingtheintegralimagerepresentation.Thus,usingtheintegralimagerepresentationonecancomputetheRectangularfeaturesrapidlyandinconstanttime. 3.1.2AdaboostBasedClassierOncethesetoffeaturesiscreatedandatrainingsetofpositiveandnegativeimagesisobtained,anytypeornumberofmachinelearningapproachescouldbeusedtoobtaintherequisiteclassier.TheViola-Jonesalgorithmusesavariantof'Adaboost'forfeatureselection(selectionofasmallnumberofoptimalfeatures)andtolearntheclassicationfunctiontotraintheclassier.'Adaboost'isalearningalgorithmprimarilyusedtoboosttheperformanceofaweakclassier(orasimplelearningalgorithm). 25

PAGE 26

Figure3-2. FromtheaboveFigure,thepixelvalueat1isA.Thevalueatlocation2isA+B,location3isA+Candlocation4isA+B+C+D.ThesumwithinDis4+1-(2+3) Sincethereareover150,000featuresavailableinaspecicdetectionwindow,itcanbehypothesizedthataverysmallnumberofthesefeaturescanbeselectedtoformanefcientclassier.Thus,initiallytheweaklearningalgorithmisdesignatedtoselectonefeaturewhichseparatesthepositiveandnegativetrainingsamples.Foreachfeature,thealgorithmcomputesaspecicthresholdfunctionwhichminimizesthenumberofmisclassiedsamples.Therefore,thisclassicationfunctioncanberepresentedas: hj(x)=8><>:1ifpjfj(x)
PAGE 27

accuracymaybemorethan0.2whichisunacceptableinarealtimesystem.Thus,analgorithmforcascadingasetofincreasinglycomplexclassierswasdevisedtoincreaseaccuracyandreducecomputationaltime. 3.1.3ClassierCascadeTheideabehindthecascadecreationisthefactthatsmaller(andthereforemoreefcient)boostedclassierscanbeconstructedtorejectmostofthenegativesub-windowsinanimage(becausethethresholdfunctionoftheweakclassiercanbeadjustedsothatthefalsenegativerateisclosetozero),sothatalmostallpositiveoccurrencesaredetected.Thenmorecomplexclassierscanbeinstantiatedtoprocessthesub-windowsagaintoreducethefalsepositiverates.Figure3-3showsthelayoutofagenericcascade.OnecanobservethattheoveralldetectionprocessisintheformofadegenerateDecisionTree.Apositiveresultfromtheinitialfunctionwouldcallthesecondclassierandapositiveresultfromthesecondwouldtriggerthethirdandsoon.Anegativeresultonanyofthestageswouldmeantherejectionofthesub-windowunderprocess. Figure3-3. Schematicofadetectioncascade.ThesecascadelayersworkinasimilarwaytoanANDgate.Asub-windowwillbeselectedonlyifalltheweakclassiersinthecascadereturnTrue.Furtherprocessingmaybeadditionalcascadesoranotherdetectionalgorithm Thecascadeisdevisedsothattheinitialclassiersaresimple'Adaboost'basedclassierswhichprovideveryhighpositivedetectionrates(alongwithveryhighfalsepositives).Asthesub-windowmovesalongthecascade,itisprocessedonbyincreasinglycomplexclassierswhicheliminatethefalsepositiverate(andalso 27

PAGE 28

compromisessomeofthepositives).Thenumberofcascadesisatradeoffbetweenthedesiredaccuracy,allowablefalsepositives,andthecomputationaltime(sincetheprocessingofcomplexclassiersareslower).Thus,trainingthecascaderequiresoptimallyminimizingthenumberofstagesandthefalsepositiverateandmaximizingthepositivedetectionrate.Achievingthisautonomouslyforaspecictrainingsetisaverydifcultproblem.Thusatargetfalsepositiverateisselectedandamaximumdecreaseindetectionrateshouldbeselectedtohaltthecreationofmorecascades.Inthefollowingparts,theimplementationofthisalgorithminthecaseofvehicledetectionisexplained.Theprocessofobtaininga'passively'trainedclassierusingtheaboveapproachisdelineatedandthemodicationofthisclassierbyretrainingtheclassiertogenerateanactivelytrainedclassierisexplained. 3.2PassivelyTrainedClassierPassiveLearningisatermgiventotheprocessofdevelopingstandardsupervisedlearningalgorithmsbysupplyingtheclassierwithrandomlabeleddata.InthecaseofpassivelytrainingaclassierbasedontheViola-Jonesalgorithm,thelearningfunctionhastobesuppliedwithrandompositivesamples(imageswhichcontainsvehicles)andrandomnegativesamples(imageswhichdoesnotcontainthevehicle).Themajordrawbackofthisprocessisthatthelearningfunctiondoesnothaveanycontroloverdataittriestolearn.Thisleadstoaclassierwhichmayormaynotperformwellindesiredconditions.Inthecaseofvehicledetection,sincethenumberandtypeofvehiclesarevast,theamountoftrainingdatarequiredisveryhigh.Thus,amuchimprovedclassiercanbeconstructedbyactivelytrainingthelearningfunctionwhichgivessomedegreeofcontroloverthedatasetittriestolearnandalsoautomaticallygeneratesdatafortraining.Theinitialpassiveclassierwasconstructedfrom3500positiveimagesand4660negativeimages.Thetrainingdatawasacquiredbycontinuousgrabbingframesfrom 28

PAGE 29

astandardofftheshelfindustrialcamera(MatrixVisionmvBlueFOX120a)whichwasmountedonthedashboardofacarwhiledrivingdownbusyandemptyroadsindaytime.Thecamerawasconnectedtoalaptopwhichprocessedtheframesataresolutionof640x480andsaveditfortraining.Figures3-4and3-5showpositiveandnegativeimageexamples.ThetestrigisexplainedinChapter5. Figure3-4. Exampleofapositiveimage-Notethattherearetwopositivesamplesintheimage Thepositivetrainingsampleswerecreatedbyhandlabelingthepositiveimagesforeachpositiveinstance.Thusatotalof6417positivesampleswerecreated.Thesampledetectionsizewasselectedas20x20.ThisisthesmallestROIthedetectorworkson.Thissizewaschosentomaximizethedistanceofdetectionandalsominimizethetotalcomputationaltime.Thecascadewastrainedbyincluding23stagesandthetrainingprocesswasstopped(furtheradditionofcascadeswashalted)whenthepositivedetectionratehadreachedthelowestlimitof0.92andthefalsedetectionratehadreachedthelowestlimitof5x10e-4.Although,thisperformanceisalmostimpossibletoachieveinindependentrealworldevaluations,thistrade-offwasselectedtoensureaveryrobustperformancewhentheactivelytrainedclassierhadtobeconstructed.Table3-1listsouttheimportanttrainingparametersselectedforpassivetraining. 29

PAGE 30

Figure3-5. Exampleofanegativeimage Table3-1. PassiveTrainingParameters PositiveSamples 6417NegativeSamples 4660SampleSize 20x20Numberoftreesplits 1MinHitrate 0.92Maxfalserate 5x10e-4NumberofStages 23 3.3ActiveLearningPrinciplesActiveLearningistheatheoreticalconceptwherethelearnerorthelearningfunctionhassomedegreeofinuenceoverthedataitistryingtolearn[ 23 ].Whenappliedtotheworldofmachinelearning,thisveryconcepthastremendousimplicationsinareaswherealargeamountoftrainingdataisrequired(foreffectiveandrobustlearning)andthehumancapitaltogenerateitislimited.Thedegreeofcontrolisusedbythelearningfunctiontoaskahigherauthority(likeahumanwhohasexhaustiveknowledgeaboutthetaskathand)aboutprovidinglabelsfordataandretrainitselftogenerateamuchmoreimprovedclassieralongwithsomenewlylabeleddata.ActivelearninghasbeenextensivelyappliedintheeldofNaturallanguageunderstanding,webinformationretrievalandtextcategorization[ 23 ].Thecaseofvehicledetectionposesaproblemsimilartorandominformationunderstanding.Theegovehicleandothervehiclesareinconstantmotionandtheamountofvariabilityinclass, 30

PAGE 31

size,andcolorinvehiclesisveryhigh.Theremightalsobevibrationswhichresultinchangesinyawandpitchangleandcastshadowswhichmighttriggerfalsepositives.Thus,aclassierrequiresaverylargeanddiversedatasettolearnarobustfunctionandthisiswheretheActiveLearningconceptbecomesuseful.SomeapproachesforActiveLearningarediscussedbelow. 3.3.1QueryByCondenceQuerybycondenceworksontheprinciplethatthemostusefulandinformativebutuncertainsampleslienearthedecisionboundary(thefunctionwhichdifferentiatesthepositiveandnegativeinstances)[ 28 ].Theseexamplescouldbequeriedusinganuncertaintyorcondencemeasure(thus,thisapproachcanalsobecalledQuerybyUncertainty).Thisselectivesamplingmethodcouldbeperformedintwoways.QueryofUnlabeleddataistheprocessinwhichahumanannotatorselectivelylabelsdataevaluatedonanindependentdatasetwhichinthehumanoracle'sdiscretionfallsneartheboundary.Thismethodassumesnoknowledgeofstructureandtheclassierqualityisentirelybasedonthehumancapabilitytojudgethedatasetnearesttotheboundary.QueryoflabeleddataistheprocessofautonomouslygeneratingaCondencefunctionwhichcalculatesthecondenceofeachsamplebyretainingtheclassierfromasetofalreadylabeledexamples.Thiscondencefunctioncouldbeusedincreatinganewdatasetfromanindependentunlabeleddata. 3.3.2QueryByMisclassicationThismethodofgeneratingthedatasetforretrainingismainlyusedinareaswheretheapplicationoftheclassierisvastlydifferentfromthetrainingscenarioandalsoincaseswherethevariabilityinpositivesamplesisveryhigh.Themethodconsistsofahumanoraclemanuallymarkingdetectedinstancesasrealpositivesorfalsepositiveswhentheinitiallearningfunctionisevaluatedonanindependenthighlyvariabledataset.Themainobjectiveofthismethodistheeliminationofthefalsepositiveratebyincluding 31

PAGE 32

thedetectedfalsepositivesinthenegativessample.Thedetectedrealpositivescanbeaccommodatedintheretrainingpositivesamplevectortomaintainasupersetandavoidovertting.Althoughselectivesamplingbasedquerymethodsarefasterintermsofhumancapital,acomparativestudybySivaraman[ 28 ]ontheapprochesofActiveLearningfoundoutthatQuerybyMisclassicationoutperformedtheaboveapproachintermsofprecisionandrecalleventhoughthisperformancecameatapriceofhumancapital.Also,itwasfoundthatboththeapproachesfaroutperformedtheinitialpassivelearningfunction.ThenextsectiondealswiththeconstructionoftheActivelyLearnedclassierafterthepassiveclassierwasevaluatedonanindependentdatasetusingQuerybyMisclassication. 3.4ActiveLearningBasedClassierTheprocessofActiveLearningconsistsoftwomainstages,aninitializationstageandasamplequerywithretrainingstage.Theinitializationstageisthesameprocessascreatingapassivelearningfunction.TheQueryandretrainingstageconsistsofinitiallyobtainingnewdatabyrunningtheclassieronanunlabeled,independentdatasetandagroundtruthmechanism(ahuman)isassignedtolabelthenewlyobtaineddata.Thisdataisthenusedtoretraintheclassiertogenerateanewlearningfunction(QuerybyMisclassication).AbroadschematicoftheActiveLearningprocessisprovidedinFigure3-6:Thepassivelytrainedclassierobtainedinitiallywasevaluatedonanindependentdatasetbyusingthetestrigonabusyhighwayduringalowlightingperiod.Thisrunproducedveryhighclassicationaccuracybutmissedsometruepositivesandproducedfalsepositives.Theseinstanceswerequeriedandlabeledbyahumanoracleandincludedforretraining.Thus,theretrainingprocessconsistedof7367positivesampleswhichincludedinitialtrainingsamplesalongwithmissedtruedetectioninstancesfromthe 32

PAGE 33

Figure3-6. SchematicoftheActiveLearningFramework;Aninitialdetectorisbuiltandisthenretrainedfromqueriedsamplesevaluatedonanindependentdataset independentdataset.The4898negativesamplesconsistedentirelyoffalsepositivesfromthequeryprocess.Acascadeof25stageswascreatedfortheretrainedclassierwithsimilarparametersastheinitialpassiveclassier.Figure3-7showsthequeriedsamplesusedforretraining. Figure3-7. Figureshowingexamplesqueriedforretraining.RowAshowsthemissedtruepositivesandRowBcorrespondtofalsepositives Theclassierwastrainedwithaminimumhitrateof0.96andafalsehitrateof5x10e-5.Table3-2showstheparametersselectedforActiveLearningisexplainedbelow. 33

PAGE 34

Table3-2. ActiveTrainingParameters PositiveSamples 7367NegativeSamples 4898SampleSize 20x20Numberoftreesplits 1MinHitrate 0.96Maxfalserate 5x10e-5NumberofStages 25 3.5SummaryThischapterprovidedanexhaustiveexplanationofthealgorithmdevelopedbyJonesandViolaforrobustrapidobjectdetection.Theapplicabilityofthisframeworktotherealtimevehicledetectionscenarioispresentedeventhoughtheinitialalgorithmwasdevelopedinthecontextoffacedetection.Theprocessofgeneratingapassivelearningfunctionusingthisalgorithmisexplainedwithtrainingdatacollectedbygrabbingsamplesfromarealtimedrivingscenario.Afterevaluatingthepassiveclassieronanindependentunlabeleddataset(inrealtimeconditions),themethodofactivelyretrainingtheclassierthroughQuerybyMisclassicationisalsoexplained.Thetrainingparametersandschematicsarealsogiven. 34

PAGE 35

CHAPTER4VEHICLETRACKINGANDDISTANCEESTIMATIONTheactivelytrainedsystemcapableofclassifyingasingleframeintovehicleregionandnon-vehicleregionisnotentirelyfeasibletobedeployedinrealtimesystemslikeautonomousvehicles.Theframeworkshouldbeabletoeffectivelydecoderelevantinformationfromitlikethenumberofcars,thedistanceofeachcarfromtheegovehicleandshouldhavethecapabilityofstoringatrajectoryofthedetectedvehiclesforenhanceddecisionmaking.Thus,anOpticalFlowbasedFeaturetrackerwithadistanceestimationalgorithmisintegratedwiththeclassier.Thissystemisthentransformedtoarobustmulti-vehicledetectionandtrackingmodulecapableofperforminginrealtimewithverylowcomputationalpower. 4.1Lucas-KanadeOpticalFlowBasedTrackingThemaindrawbackofamachinelearningbasedapproachtovehicledetectionisthefactthateachframeisevaluatedindividually.Thereisnoco-relationbetweensuccessiveframes.Although,theclassierisabletoachieveveryhighaccuracy,theremightbeinstanceswhereavehicledetectedinapreviousframemaybemissedinthecurrentframe.Therefore,itisnecessarytointegrateatrackingfeatureintothealgorithmwhichworksinconjunctionwiththeclassier.OpticalFlowisthemethodofestimatingtheapparentmotionofobjectsbetweensuccessiveframes.Theassumptionthatpointsontheobject(whichisbeingtracked)havethesamebrightnessovertimeisthebasisofthemethod.Thisassumptionisvalidinthevehicledetectioncontext[ 20 ].BruceLucas,in1981developedamethodforestimatingtheopticalowbyintroducinganotherassumptionthatthepixelsunderconsiderationhavealmostconstantowrate.Thealgorithmsolvestheopticalowequationforallpixelsinthelocalneighborhoodusingtheleastsquarescriterion. 35

PAGE 36

4.1.1MethodThemethodassociatesavelocity/movementvector(u,v)toeachpixelunderconsiderationandisobtainedbycomparingtwoconsecutiveimagesbasedontheassumptionsthat: Themovementofapixelwithinthetwoconsecutiveframesisnotverysignicant. Theimagedepictsanaturalscenewheretheobjectsareinsomegreyshadewhichchangessmoothly.(i.e.themethodworksproperlyforonlygrayscaleimages)LetI(qi)betheimageintensityofageneralpixelqiintheimage.TheLucas-Kanademethodassertsthatthefollowingequationmustbesatisedbythevelocityvector(u,v). Ix(qi)u+Iy(qi)v=It(qi)(4)Where,Ix(qi),Iy(qi),andIt(qi)representsthepartialderivativesoftheintensityofthepixelunderconsiderationwithrespecttox,yandt.But,theequationhastwounknownswhichmakethesystemunderdetermined.So,thealgorithmcomputestheintensityofsomeneighborhoodpixelstoobtainmoreequations.Theseequationscanbewrittenintheformofasystemofequations(matrix)foreachofthepixelsunderconsiderationasAV=B,where A=2666666666664Ix(q1)Iy(q1)Ix(q2)Iy(q2)......Ix(qn)Iy(qn)3777777777775,V=2664uv3775,andB=2666666666664)]TJ /F4 11.955 Tf 9.29 0 Td[(It(q1))]TJ /F4 11.955 Tf 9.29 0 Td[(It(q2)...)]TJ /F4 11.955 Tf 9.3 0 Td[(It(qn)3777777777775(4) 36

PAGE 37

Thissystemhasmanymoreequationsthanunknowns(namelyuandv)andthereforeitisalwaysoverdetermined.Aleastsquaresapproachisusedtoobtainacompromisesolution.Thesystemofequationsisthusreducedto: V=(ATA))]TJ /F9 7.97 Tf 6.58 0 Td[(1ATB(4)Therefore,thevelocityvectorofthespecicpixel(andofitsneighborhood)canbeexplicitlywrittenas: 2664VxVy3775=2664PiIx(qi)2PiIx(qi)Iy(qi)PiIx(qi)Iy(qi)PiIy(qi)23775)]TJ /F9 7.97 Tf 6.58 0 Td[(12664)]TJ /F6 11.955 Tf 11.29 8.96 Td[(PiIx(qi)It(qi))]TJ /F6 11.955 Tf 11.29 8.96 Td[(PiIy(qi)It(qi)3775(4)Theresultofthealgorithmisasetofvectors(opticalow)whicharedistributedallovertheregionofinterestwhichgivesanideaofapparentmotionofanobjectintheimage.Thisvectorcanbeusedtotrackthedetectedvehiclefromoneframetoanother.Theaboveexplainedmethodhasbeenimplementedandimproveduponinmanyways.OneofthemostsuccessfulimprovementswasdonebyBouguetin2001[ 4 ]inwhichapyramidalimplementationofthetrackerwasproposed. 4.1.2PyramidalImplementationAnimagepyramidisamulti-scalerepresentationofaimagewheretheimageissubjectedtosmoothingandsub-sampling(usuallybyafactoroftwo)repeatedlytogeneratesmallerandsmallerimageswhichcanbeco-relatedtotheoriginalimage.Thegraphicalrepresentationofthisstructurelookslikeapyramid(fromwherethenameoriginated)andisusedtomulti-resolutionanalysisandforscale-spacerepresentations.ThealgorithmdevelopedbyBouguet(animplementationofwhichthecurrentstudyuses)runsaniterativeversionoftheLucas-Kanadealgorithmwhichproceedsasfollows: 37

PAGE 38

Figure4-1. Multi-ResolutioncoarsetoneOpticalFlowEstimationusingapyramidalrepresentationoftheimage Estimatethemovementvectorforeachofthepixelsunderconsideration. InterpolateI(t)toI(t+1)usingtheestimatedoweld Repeat1and2untilconvergenceTheiterativeLucas-Kanadealgorithmisinitiallyappliedtothedeepestlevelinthepyramid(topmostlayer)andtheresultisthenpropagatedthroughthenextlayerasaninitialguessforthepixeldisplacementandthennallythepixeldisplacementattheoriginalimageisreached.Thismethod(showngraphicallyinFigure4-1)increasestheaccuracyofestimation[ 4 ]anddoessoinaverycomputationallyefcientway.Animplementationofthisalgorithmisusedinthisstudy. 4.1.3ImplementationSincethisalgorithmworksonlyongrayscaleimages,thewholesetupisconvertedtoagreayscalesystem.Initialtestsonrunningtheclassierongrayscaleimagesproducedverypositiveresults.Initially,theclassierevaluatedthevideoframe.Ifvehiclesaredetected,theROIissavedtobeproducedasoutput.Ifnovehiclesaredetected,thealgorithmmovestothetrackingphasewheretheLucas-Kanadealgorithmestimatesthepositionofthevehicle 38

PAGE 39

basedonthedetectionsinthepreviousframe.TheLucas-Kanadealgorithmrequiresaninitialestimateoftheobjectposition(i.einitialpixelsofinterest)forestimatingthemovementoftheobject.Thisinitialestimateisobtainedbysettingthecornerpointsoftherectanglewhichboundsthevehicleregion.AschematicoftheprocessisshowninFigure4-2: Figure4-2. SchematicshowingthealgorithmicowoftheTrackingProcess 4.2DistanceEstimationUsually,theprocessofdeterminingthedistanceofanobjectusingasingle(monocular)visionsystemispurelyanestimationprocess.Inordertoascertainaccuratemeasurementsofrangeinformationpertainingtoaparticularsceneastereorig(asystemoftwocamerasviewingthescenefromdifferentpointsofview)isrequired.Butamajordrawbackofusingstereobasedmethodsisthattheyarecomputationallytaxingandthereforenotsuitableforarealtimesystem.Thus,methodsforeffectivelyestimatingrangeinformationfromasinglecamerahavebeenresearchedwidely.Mostofthesemethodsusesomesortofassumption,eitherfromascenepointofview(i.e.thecameraisstationary)orfromtheobjectpointofview(i.e.assumingsomeknowninformationabouttheobject).Anothermajor 39

PAGE 40

assumptionisthatthetransformationequationsuseasocalledpin-holecameramodel.Thismodelisrepresentedbythefollowingequation. 266664uv1377775=266664fxs;0;cx0;fys;cy0;0;1377775266664r11;r12;r13;t1r21;r22;r23;t2r31;r32;r33;t3377775266666664XYZ1377777775(4)whereuandvrepresentapointin2-Dcameraspace.fxandfyarethefocallengths(expressedinpixelrelatedunits).(cx,cy)representaprincipalpoint,generallytheimagecenter.Thesecond4x4matrixistherotationmatrixwhichindicatestherotationofthecameracenterwithrespecttotheobjectcenter.Thefourthcolumnofthematrixrepresentthetranslationalelementwhichrepresenttherelativemotionbetweenthecameraandtheobject.Theconversionfactorsisthefactorthattransformspixelrelatedunitstomillimeters(realworldunits).Thematrixcontainingthefocallengthsandtheprincipalpointiscalledtheintrinsicmatrix.Thismatrixisconstantforeverycamera(whichworksonnormalzoom)anddoesnotvarywiththetypeofsceneviewed.Thesecond4x4matrixiscalledtheextrinsicmatrix.Thismatrixgivesinformationaboutthecameracenterwithrespecttotheworldcoordinatesystemandtheheadingofthecamera(i.ethetranslation).Errorsresultingfromdistortionandmisalignedlensesarenotconsideredinthisstudy.Theperspectivepin-holecameramodeldescribedaboveisusedtoestimatethedistanceofavehicledetectedinthevideoframe.Sincethevideoisprocessedframebyframe,thedistanceisestimatedateverytimestep.Thefollowingassumptionsaremadeincontextofthecurrentscenariofordistanceestimation. Theworldcoordinatecenterisxedatthecameracenter.Thismeansthattheextrinsicparameterswillnothaveaneffectonthetransformationfrom3-Dpointsto2-Dpixels. 40

PAGE 41

TheZaxisofthecameraisalongthelineoftheroad.Thereisnoyaworrollanglebetweentheroadandthecamera. Highwaylaneshavenegligibleslopes.Therefore,thepitchangleisalsozero. Estimatesforthewidthandheightofatypicalcarareknownandallvehicleshavecomparablesizes.Thus,giventhetypicalwidthandheightofastandardcarWandH,andthewidthandheightofthedetectedRegionofinterest(ROI)inpixelrelatedcoordinateswandh,onecanestimatethedistanceofthecarbythefollowingtwoequations. Z1=sfW wZ2=sfH h(4)wherefistheaverageoffxandfy.Thus,thedistanceestimateofadetectedvehicle,updatedateverytimestepforeverydetectedvehicleistheaverageofZ1andZ2.Therefore,acompletemulti-vehicledetctionandtrackingsystemwithrangeestimationusingamonocularvisionsystemcanbeimplementedasshowninFigure4-3. 4.3SummaryThischapterdelineatedthetheorybehindtheLucas-KanadeOpticalowbasedtrackingalgorithmalongwithDistanceestimationusingapin-holecameramodel.TheintegrationofapyramidalimplementationoftheLucas-Kanadealgorithmtotheactivelytrainedclassierforenhancedrealtimevehicletrackingisalsoexplained.Furthermore,afullyrealtimesystemabletorobustlydetectandtrackvehiclesandalsoestimate,fairlyaccurately,therangeinformationofdetectedvehiclesisoutlinedusingaschematic.Thissystemcanbeimplementedinastandardvehicleforenhanceddriversafetyorinautonomousvehiclesasaresourceforclassifyingvehiclesandasacognitiveinformationtoolforsensingandperception. 41

PAGE 42

Figure4-3. AlgorithmicowofthecompleteDetectionandTrackingProcess 42

PAGE 43

CHAPTER5EVALUATIONANDDISCUSSIONTheprocessofevaluatingthedesignedsysteminvolvesrunningtheactivelytrainedclassieronspecic(StaticandRealTime)datasetsandjudgingitsperformancebasedonTruePositiveDetectionsandFalsePositiveDetections.FalseNegativesarenotofmuchimportancetothisstudybecausetheTruePositivesgiveanempiricalmeasureofFalseNegativeDetections.Theresultsoftheevaluationiscomprehensivelydiscussedbasedonspecicallydenedevaluationparameterswhichpresentthesystem'srobustness,sensitivity,recall,scalabilityandspecicity.Acomparisonbetweentheactivelytrainedandpassivelytrainedclassierisalsogiven.Theprocessofimplementingthefullmulti-vehicledetectionandtrackingsysteminarealtimescenarioispresented.Finally,theresultsaresummarizedandthescopeforfutureworkisoutlined. 5.1Datasets 5.1.1Caltech2001/1999StaticImageDatabaseThisdatasetconsistsof1155differentimagesoftherearviewofcarsandwascompiledinitiallyin1999andthenmodiedin2001.Mostoftheimageshaveviewpointswhichareveryclosetothecamera(unlikethecaseformostofthetrainingexamples)andalsoconsistsofsomemodelsofcarswhichareold.Thisdatasethasbeenwidelyusedinconjunctionwithtestingavarietyofvisionbaseddetectionalgorithmsandserveasabenchmarkinstaticimagetesting.AnexampleimageisshowninFigure5-1.ThisdatasetcanbepubliclyaccessedattheComputationalVisionpageatwww.vision.caltech.edu. 5.1.2mvBlueFOXTestDataThisdatasetconsistsof362framesofvideocollectedusingthemvBlueFOXcamera(whichisusedinthisspecicapplication)usingthetestrig.Theframesconsistsofacombinationofframescollectedatdaytimeandatlowlightningconditions.Therst 43

PAGE 44

Figure5-1. Caltech2001/1999StaticImageDataExample 350framesconsistofconsecutiveframescollectedatdaytimewithonevehicleofinterestineachframeandthenext12framesconsistofconsecutiveframescollectedduringsunsetwithtwovehiclesofinterestineachframe.Alltheframeswereannotatedfortruepositivesandsetasidefortesting.AscreenshotoftheTestDataisshowninFigure5-2: Figure5-2. RealTimetestdatacapturedusingthetestrig.Thisframebelongstothedaytimeset Thetestrigconsistedofaninitiallycalibratedcamera(MatrixVisionmvBlueFOX-120aforwhichtheintrinsicparametersareknown)rigidlyattachedtothetopofthedashboardofavehicleabovethecenterconsole.Theaxis(z-axis)ofthe 44

PAGE 45

camerawaskeptparalleltotheroadandthelenswasadjustedatnormalzoom.ThecamerawasattachedtoalaptopviaUSBandwasoperatedbyahumanforspecictasks(DataCollection,CaptureandTesting). 5.2EvaluationParametersThedesiredoutcomeoftestingtheclassieristhequanticationoftheclassier'srobustness,sensitivity,specicity,scalability,andrecallintermsofnumericalvalues.Moststudiesonvehicledetectionusingvisionquantifytheirresultsbyinitiallycroppingtheimageandnormalizingthemtoaspecicviewandrunningtheclassieronthesepre-processedimages.Thismethod,eventhough,hashigheraccuracyandreportlowerfalsepositives,doesnotaccountforrobustnessandscalability.Although,morerecentstudieshavecomeupwithparameterstoevaluateaccuracyandrecallinrealtimesystems,theydonotoffernumericalvaluesforquantifyingprecision,robustness,andspecicity.Thefollowingparametershavebeenusedinthisstudytoevaluatetheclassiersintermsofaccuracy,robustness,recall,sensitivity,andscalability. TrueDetectionRate(TDR)ismeasuredbydividingthenumberoftrulydetectedvehiclesbythetotalnumberofvehicles.Thisparametergivesameasureofaccuracyandrecall. TDR=TruePositives(No.ofdetectedvehicles) ActualNo.ofvehicles(5) FalsePositiveRate(FPR)isobtainedbydividingthenumberoffalsedetections(falsepositives)bythetotalnumberofdetectedvehicles(TrueandFalse).Thisparameterisanestimateofthesystem'sprecisionandrobustness. FPR=No.offalsedetections ActualNo.ofvehicles+FalseDetections(5) AverageTrueDetectionperFrame(ATF)isthenumberoftruepositives(detected)dividedbythetotalnumberofframesprocessed.Thisisameasureofsensitivityofthesystem. ATF=TruePositives Totalnumberofframes(5) 45

PAGE 46

AverageFalseDetectionperFrame(AFF)ismeasuredbydividingthenumberoffalsepositivesbytotalnumberofframesprocessed.Thisquantitygivesanumericalmeasureforrobustnessandspecicity. AFF=FalsePositives Totalnumberofframes(5) AverageFalseDetectionperVehicle(AFV)isthetotalnumberoffalsedetctionsdividedbythenumberofvehiclesontheroad.Itindicatesrobustnessandprecision. AFV=FalsePositives TotalnumberofVehicles(5)Theoverallperformanceofthesystemintermsoftheabovementionedparametersgivesusanestimateofthescalabilityofthesystem.Thisisameasureofhowtheframeworkcanbeadaptedtobeusedinmoreadvancedandrealtimescenarios(likeinautonomousvehicles). 5.3StaticDatasetTheactivelytrainedclassierwasevaluatedon1155staticimagespubliclyavailableandtheperformancecharacteristicsdescribedaboveareevaluated.AgoodclassierperformanceonthisdatasetwouldjustifyitsrobustnessandscalabilityTheclassierproducedanaccuracyof92.98%withaFalseDetectionRateof0.124.SomeofthedetectionresultsareconsolidatedinFigures5-3and5-4.Incomparisonthepassivelytrainedclassierreturnedanaccuracyof93.5%butwithafalsepositiverateof0.35.ThisisthemajortradeoffwhichhastobeaddressedwhenchoosinganActivelytrainedorPassivelyTrainedclassier.Thepassivelytrainedclassier,asshowninthecaseofstaticdatasets,performedmarginallywellbutthepriceintermsofFalsePositivesisveryhigh.ThisjustiestheuseofanActivelyTrainedclassierinrealtimescenarios,asitperformsmuchbetterintermsofprecisionandrobustnessasexplainedbelow.AcomparisonofalltheevaluationparametersforthetwoclassiersispresentedinTable5-1. 46

PAGE 47

Figure5-3. ExamplesofTruePositiveDetectionsinStaticDatset Figure5-4. Examplesoffalsepositivesandmisseddetections-Wecanobservethatmostofthemissedsamplesareduetothefactthatthetrainingdatacontainedexamplesexclusivelyfromanrealtimedataset.Thetrainingdatamodeldidnotcontainmanyexamplesofcarsveryclosetothecamera. Table5-1. ResultsonStaticDataset-Active-PassiveComparision Parameter PassiveClassier ActiveClassier TrueDetectionRate 93.5% 92.8% FalsePositiveRate 0.35 0.124 AverageTrueperFrame 0.985 0.98 AverageFalseperFrame 0.48 0.16 AverageFalseperVehicle 0.45 0.138 47

PAGE 48

Although,theaccuracyofthePassiveclassierishigher,thefalsepositiverateisalmostthreetimesasmuchastheActiveclassier.Also,averagefalsedetectionsperframeandpervehicleisfourtimeshigherwhencomparedtotheActiveclassier.Thus,onecancondentlyarguethattheActiveclassierperformsmuchbetterintermsofprecisioninstatictests.Further,inFigure5-5aReceiverOperatingCurve(ROC)isplottedbetweentheTrueDetectionsandFalsePositivesRate.Thisgraphgivesanideaoftheclassierssensitivity(accuracy)withrespecttospecicity(themeasureofhowtruetheclassicationsare). Figure5-5. TheROCcurvesforActivelyandPassivelyTrainedClassiers IntermsofanROCcurve,abetterclassieristheonewhichismorejustiedtowardstheleftofthelinewhichdividesthegraphat45degrees.Fromtheabovecurve,wecaneasilyinferthattheActivelyTrainedclassierismuchstrongerinperformanceintermsofspecicitythanthePassiveclassier.Therefore,instatictestsonecanconcludethattheActiveclassierperformsmuchbetterintermsofprecisionandrobustnessalthoughthereisasmalltradeoffintermsofaccuracy. 48

PAGE 49

5.4RealTimeDatasetTheActivelyTrainedclassierwasevaluatedon362framesofrealtimetestdata.Mostofthedatawascomprisedofconsecutiveframesofvideostreamtakenduringgoodlightningconditions(sunny).Someframeswerealsoobtainedatlowlightingconditions(halfanhourbeforesunset)toevaluatetheperformanceoftheclassierinsuchconditions.Itwasfoundthattheclassierreturnedanaccuracyof98.8%(i.e.372hitsoutof376vehicles).Thefalsepositiveratewas0.124.Anaverageframerateof15.6framespersecondwasachievedintestingonanInteli5Processor(2.3GhzQuadcore)with4GBRAM.Theprocessutilizednomulti-threadingorGPUoptimization.PreviousstudiessupportthefactthatusingGPUoptimizationofvisionbasedalgorithmscouldspeedupprocessesbymorethan10times.SomeexamplesofdetectedresultsareshowninFigures5-6and5-7. Figure5-6. ExamplesofTruePositiveDetectionsinRealTimeTestset.Therstcolumnshowstwoframesinverylowlightingconditionswhereboththecarsweredetectedaccurately.Thesecondandthirdcolumnsshowstwoconsecutiveframeseachwherethecarwasdetectedinboththecases.Therealtimesetperformedbetterthanthestaticdatasetsincethetrainingdatawascapturedusingthesametestrig AgraphicillustrationofthenumberofvehiclesdetectedateachframecomparedtotheactualnumberofvehiclesispresentedinFigure5-8.Onecanobservethatthe 49

PAGE 50

Figure5-7. Examplesoffalsepositivesandmisseddetections.Wecanobservethatinthersttwopictures,falsepositiveareduetoothervehiclesandincomingvehicles.Inthethirdcase,lowlightingpreventeddetectionofonecarandinthenalcasethereweretwodetectionsonthesamecar Figure5-8. Graphillustratingthenumberofvehiclesdetectedateachframeandfalsepositives.Thisiscomparedtotheactualnumberofvehiclesinthescene 50

PAGE 51

classiermaintainsthetrackofdetectedvehicleswithconsiderableefciency.But,insomeframes,thetrackofthevehicleiscompletelylost(i.eamisseddetection).Thus,integratingafeaturebasedtrackerwouldfullyeliminatetheproblemofmisseddetections(eventhoughtheyoccurveryrarely).Wecanalsoobservethatthefalsedetectionrateisnotcontinuousforeverytwoframes.Thereisnofalsepositivewhichwasdetectedovertwoframes.Thisgivesanincentivetomodifythefeaturebasedtrackertoonlytrackregionswhichhavebeendetectedovertwoframes.Thus,thewholeframeworkwasmodiedtoaddressthisissue.Anotherobservationmadewasthat,fromFigure5-7,onecaninferthatsomeofthefalsepositivesareactuallydetectionoverthesamewindowbutwithalargerarea.Thesedetectionscanbecheckedforanddoubledetectionsinthesamevicinitycanbeaccountedforbytweakingthealgorithmtoskipoverpreviouslydetectedregionsasthedetectionwindowismadelarger.Finally,thehighlightofthetestwasthefactthattheAverageFalsepositivesperFramefortherecognizerwasfoundouttobe0.12.Thismeansthatwhentheclassierisintegratedwithatracker,thisvalueislowenoughsothatitisnotconsistenttocreatewrongtracks.Thishighlightsthescalabilityoftheclassiertomoresophisticatedframeworks.Thepassivelytrainedclassierwasalsoevaluatedusingtherealtimedataset.Incontrasttothestaticdataset,theclassierinthiscaseproducedalower(almostequal)accuracyof98.6%whencomparedtotheActivelytrainedclassier.Therateoffalsedetectionswas0.24.ThiswaslowerthanthestaticfalsepositiveratebutalmosttwiceasmuchwhencomparedtotheActivelyTrainedclassier.TheevaluationparametersareshowninTable5-2:OnecanobservethatthePassiveclassierreturnedanalmostcomparableaccuracytotheActiveclassier.Also,theaveragetruedetectionsperframeforboththeclassiersarethesame.ThisvalidatestheuseoftheHaar-likefeaturebasedViola-Jonesalgorithmforvisionbasedvehicledetection.But,thefalsepositiverate 51

PAGE 52

Table5-2. ResultsonRealTimeDataset-Active-PassiveComparision Parameter PassiveClassier ActiveClassier TrueDetectionRate 98.6% 98.8% FalsePositiveRate 0.24 0.11 AverageTrueperFrame 1.02 1.03 AverageFalseperFrame 0.32 0.12 AverageFalseperVehicle 0.311 0.12 FrameRate 15.6 15.4 Figure5-9. TheROCcurvesforActivelyandPassivelyTrainedClassiers obtainedwasaround0.24.Thismeansthattheaveragefalsedetectionsperframewasaround0.32.ThisvaluehinderstheuseofthePassiveclassierinmoreadvancedandsophisticatedframeworks.TheActiveclassierreturnedalmostonethirdlowervaluesinaveragefalseperframeandaveragefalsepervehicle.ThisprovesthatanActiveLearningbasedalgorithmhasamuchstrongerperformanceintermsofrobustness,scalability,andprecision.TheReceiverOperatingCurve(ROC)depictingTrueDetectionswithrespecttoFalsePositivesisplottedinFigure5-9:OnecannotethattheActiveClassierperformsmuchbetterintermsofspecicity(Falsedetections)thanthePassiveclassier,particularlyasthesampleincludesmore 52

PAGE 53

exampleswithhighertruedetections.Thus,itcanbearguedthattheActiveclassierismuchmorerobustandsensitivetonoisethanthePassiveclassiertherebyvalidatingitsuseinrealtimeplatformslikeinIntelligentVehicles. 5.5ImplementationinaRealTimeScenarioThecompletemulti-vehicledetectionandtrackingsystemwithintegratedLucas-KanadetrackerandDistanceestimation,outlinedinFigure4-10,wasimplementedinarealtimescenariousingthetestrigexplainedearlier.Thetestingconditionsweresunnywithlighttrafc.ThealgorithmcouldonlyruningrayscaleastheTrackingmoduleonlyworkedwithgrayscaleimages.SomeframesfromtherunareshownandexplainedinFigures5-10and5-11: Figure5-10. Aframefromthefullsystemimplementationshowingadetectedcarwithitsdistanceestimate 5.6SummaryThischapterprovidedtheproofofconceptimplementaionoftheActivelytrainedViola-Jonesclassierforthepurposeofrealtimevehicledetection.Theclassierwasevaluatedonpubliclyavailablestaticimagedatasetsaswellasonrealtimevideo 53

PAGE 54

Figure5-11. Thenextframewherethecarwasnotdetectedbutwastrackedfromthepreviousframe.Thusthedistanceestimateyieldsthesamevalue.Thedistanceismeasuredas498.67incheswhichscalestoaround12.5meters. streamcollectedusingthetestrig.TheclassierwasthencomparedtothePassivelytrainedclassierbasedonsomespecicevaluationcriteriaandresultswerepresented.Thefullimplementationofthemulti-vehicledetectionandtrackingsystemcompletewithafeaturetrackeranddistanceestimationonarealtimescenarioisalsopresented. 54

PAGE 55

CHAPTER6CONCLUSIONSANDFUTUREWORKTheideaofthisthesiswastopresentavisionbasedvehicledetectionandtrackingsystemwhichcanbeimplementedinrealtimesystemssuchasinintelligentvehiclesandautonomouscars.Themainchallengeinaddressingthisissuewastocreatearobust,reliablesystemwhichwassimpletoimplement.AmachinelearningbasedapproachwasdevisedtosolvethisproblemwhereacascadeofclassiersweretrainedbasedonAdaboost(workingonRectangularHaar-likefeaturesinanimage)torapidlydetectRegionsofInterest(ROIs)correspondingtocarsinavideoframe.ThisclassierwasthenretrainedusingQuerybyMisclassicationtoproduceanActiveclassierwhichwasmuchmoresensitivetonoise.ThisclassierwasthenintegratedwithaFeaturebased(Lucas-Kanade)trackeralongwithadistanceestimationalgorithm(monocularvisionbased)tobuildacompletemulti-vehicledetectionandtrackingsystem.Thisframeworkwasevaluatedonstaticandrealtimedataandtheperformancewasfoundtobehighlysatisfactorywith98%detectionratesat16framespersecond.Theperformancecharacteristicsofthissystemenablesittobedeployedinmoresophisticatedandadvancedframeworks.Goingfurther,avarietyofprinciplescouldbeimprovedupon.Asignicantimprovementmightarisefromusingaverylargetrainingdatasetrepresentingalllightingconditionsandvehicletypes.Theonlydrawbackofusingalargedatasetistheamountofhumancapitalinvolvedinannotatingthegroundtruth.Therfore,thenextstepwouldbetodeviseamechanismwhichcouldautomaticallyannotatetheRegionofInterestbyusingsomeclassicalapproaches(whichareveryrobust,butdonotperforminrealtime)toinitiallyclassifyvehicles.ThismethodcouldalsobeusedtoselectivelysampletheindependentdatasetusedforActiveLearningandqueryitforretraining.Integrationoflanedetection,trajectorylearningandpedestriandetectionaresomeotherkey 55

PAGE 56

advanceswhichcouldbeintegratedintothecurrentsystemfortransformingthesystemintoacompleteenhancedActivesafetysystem. 56

PAGE 57

REFERENCES [1] Balcones,D,Llorca,D,andSotelo,M.Real-timevision-basedvehicledetectionforrear-endcollisionmitigationsystems.ComputerAidedSystemsTheory(2009):320.URL http://www.springerlink.com/index/UU7VU34310825343.pdf [2] Bensrhair,AandBertozzi,M.Acooperativeapproachtovision-basedvehicledetection.IntelligentTransportationSystemsConferenceProceedings(2001):209.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=948657 [3] Betke,Margrit,Haritaoglu,Esin,andDavis,LarryS.Real-timemultiplevehicledetectionandtrackingfromamovingvehicle.MachineVisionandApplications12(2000).2:69.URL http://www.springerlink.com/openurl.asp?genre=article\&id=doi:10.1007/s001380050126 [4] Bouguet,Jean-yves.PyramidalImplementationoftheLucasKanadeFeatureTrackerDescriptionofthealgorithm.Tech.Rep.2,2001. [5] Bouttefroy,P.L.M.,Bouzerdoum,A.,Phung,S.L.,andBeghdadi,A.VehicleTrackingUsingProjectiveParticleFilter.2009SixthIEEEInternationalConferenceonAdvancedVideoandSignalBasedSurveillance.IEEE,2009,7.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5279471'escapeXml='false'/> [6] Cao,Xianbin,Lan,Jinhe,Yan,Pingkun,andLi,Xuelong.KLTFeatureBasedVehicleDetectionandTrackinginAirborneVideos.2011SixthInternationalConferenceonImageandGraphics.IEEE,2011,673.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6005950'escapeXml='false'/> [7] Chen,Yen-Lin,Wu,Bing-Fei,andFan,Chung-Jui.Real-timevision-basedmultiplevehicledetectionandtrackingfornighttimetrafcsurveillance.2009IEEEInternationalConferenceonSystems,ManandCybernetics.IEEE,2009,3352.URL http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true\&arnumber=5346191\&contentType=Conference+Publications [8] Choi,Jaesik.RealtimeOn-RoadVehicleDetectionwithOpticalFlowsandHaar-LikeFeatureDetectors.(2012). 57

PAGE 58

URL https://www.ideals.illinois.edu/handle/2142/30794 [9] Dallalzadeh,ElhamandGuru,D.S.Feature-basedtrackingapproachfordetectionofmovingvehicleintrafcvideos.ProceedingsoftheFirstInternationalConferenceonIntelligentInteractiveTechnologiesandMultimedia-IITM'10.NewYork,NewYork,USA:ACMPress,2010,254.URL http://dl.acm.org/citation.cfm?id=1963564.1963609 [10] Firouzi,HandNajjaran,H.Real-timemonocularvision-basedobjecttrackingwithobjectdistanceandmotionestimation.IEEEASMEInternationalConferenceonAdvancedIntelligentMechatronics(AIM)...(2010).URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5695936 [11] Garca,RIRandShu,D.VisionbasedVehicleTrackingusingahighanglecamera.Tech.Rep.gure1,????URL http://www.ces.clemson.edu/stb/ece847/projects/VISION_BASED_VEHICLE_TRACKING.pdf [12] Han,S,Han,Youngjoon,andHahn,Hernsoo.VehicleDetectionMethodUsingHaar-LikeFratureonRealTimeSystem.WorldAcademyofScience,EngineeringandTechnology(2009):455.URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.192.8220\&rep=rep1\&type=pdf [13] Holub,Alex,Perona,Pietro,andBurl,MichaelC.Entropy-basedactivelearningforobjectrecognition.2008IEEEComputerSocietyConferenceonComputerVisionandPatternRecognitionWorkshops(2008):1.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4563068 [14] Jones,MJandViola,P.Robustreal-timeobjectdetection.WorkshoponStatisticalandComputationalTheoriesofVision(2001).February.URL https://www.hpl.hp.com/techreports/Compaq-DEC/CRL-2001-1.html [15] Kapoor,Ashish,Grauman,Kristen,Urtasun,Raquel,andDarrell,Trevor.ActiveLearningwithGaussianProcessesforObjectCategorization.2007IEEE11thInternationalConferenceonComputerVision(2007):1.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4408844 58

PAGE 59

[16] Khalid,Zebbara,Mazoul,Abdenbi,andAnsari,MohamedEl.Anewvehicledetectionmethod.InternationalJournalofAdvancedComputerScienceandApplications(2011):72.URL https://www.thesai.org/Downloads/SpecialIssueNo3/Paper12-Anewvehicledetectionmethod.pdf [17] Kim,GiseokandCho,Jae-Soo.Vision-basedvehicledetectionandinter-vehicledistanceestimationfordriveralarmsystem.OpticalReview19(2012).6:388.URL http://www.springerlink.com/index/10.1007/s10043-012-0063-1 [18] Kobilarov,M.,Sukhatme,G.,Hyams,J.,andBatavia,P.Peopletrackingandfollowingwithmobilerobotusinganomnidirectionalcameraandalaser.Proceedings2006IEEEInternationalConferenceonRoboticsandAutomation,2006.ICRA2006.(2006).May:557.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1641769 [19] Liu,Chendong;ZhangYunzhou,Meng;Wu.Motionvehicletrackingbasedonmulti-resolutionopticalowandmulti-scaleHarriscornerdetection.2007IEEEInternationalConferenceonRoboticsandBiomimetics(ROBIO).IEEE,2007,2032.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4522480'escapeXml='false'/> [20] Lucas,BruceD.GeneralizedImageMatchingbyamethododdifferences.Ph.D.thesis,1981. [21] Miller,R.,Bebis,G.,andDiMeo,D.Areal-timeprecrashvehicledetectionsystem.SixthIEEEWorkshoponApplicationsofComputerVision,2002.(WACV2002).Proceedings.(????):171.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1182177 [22] Nilsson,NilsJ.Introductiontomachinelearning.2004.URL http://scholar.google.com/scholar?hl=en\&btnG=Search\&q=intitle:INTRODUCTION+TO+MACHINE+LEARNING+AN+EARLY+DRAFT+OF+A+PROPOSED+TEXTBOOK+Department+of+Computer+Science#0http://books.google.com/books?hl=en\&lr=\&id=1k0_-WroiqEC\&oi=fnd\&pg=PR13\&dq=INTRODUCTION+TO+MACHINE+LEARNING\&ots=p90K-RcLuL\&sig=nQgzbYlsUF8gX9djK_-MEWk_2jg 59

PAGE 60

[23] Olsson,Fredrik.Aliteraturesurveyofactivemachinelearninginthecontextofnaturallanguageprocessing.SwedishInstituteofComputerScience(2009):134. [24] Papageorgiou,ConstantineandPoggio,Tomaso.Atrainablesystemforobjectdetection.InternationalJournalofComputerVision38(2000).1:15.URL http://www.springerlink.com/index/WW5UT522540L672R.pdf [25] Parekh,KaytonB.AComputerVisionApplicationtoaccuratelyestimateobjectdistance.Ph.D.thesis,????URL http://digitalcommons.macalester.edu/cgi/viewcontent.cgi?article=1018\&context=mathcs_honors [26] Rezaee,Hamideh,Aghagolzadeh,Ali,andSeyedarabi,Hadi.Vehicletrackingbyfusingmultiplecuesinstructuredenvironmentsusingparticlelter.2010IEEEAsiaPacicConferenceonCircuitsandSystems.IEEE,2010,1001.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5775069'escapeXml='false'/> [27] SayananSivaraman,MohanM.Trivedi.CombiningMonocularandStereo-VisionforReal-TimeVehicleRangingandTrackingonMultilaneHighways.14thInternationalIEEEConferenceonIntelligentTransportationSystems.2011,1249.URL http://130.203.133.150/viewdoc/summary;jsessionid=FF2E2F450D2498414806725AE886A282?doi=10.1.1.228.3732 [28] Sivaraman,SayananandTrivedi,MohanM.Activelearningforon-roadvehicledetection:acomparativestudy.MachineVisionandApplications(2011).URL http://www.springerlink.com/index/10.1007/s00138-011-0388-y [29] Sivaraman,SayananandTrivedi,MohanManubhai.Activelearningbasedrobustmonocularvehicledetectionforon-roadsafetysystems.2009IEEEIntelligentVehiclesSymposium.IEEE,2009,399.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5164311 [30] .AGeneralActive-LearningFrameworkforOn-RoadVehicleRecognitionandTracking.IEEETransactionsonIntelligentTransportationSystems11(2010).2:267.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5411825 60

PAGE 61

[31] Subramanian,Anbumani.Imagesegmentationandrangeestimationusingamoving-aperturelens.Ph.D.thesis,2001.URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.5.9656\&rep=rep1\&type=pdf [32] Tsai,Yi-Min,Tsai,Chih-Chung,Huang,Keng-Yen,andChen,Liang-Gee.Anintelligentvision-basedvehicledetectionandtrackingsystemforautomotiveapplications.2011IEEEInternationalConferenceonConsumerElectronics(ICCE).IEEE,2011,113.URL http://www.researchgate.net/publication/224222737_An_intelligent_vision-based_vehicle_detection_and_tracking_system_for_automotive_applications [33] Wu,Bing-fei,Kao,Chih-Chung,Liu,Chih-Chun,Fan,Chung-Jui,andChen,Chao-Jung.Thevision-basedvehicledetectionandincidentdetectionsysteminHsueh-Shantunnel.2008IEEEInternationalSymposiumonIndustrialElectronics.IEEE,2008,1394.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4677196 [34] Yamaguchi,K.,Kato,T.,andNinomiya,Y.VehicleEgo-MotionEstimationandMovingObjectDetectionusingaMonocularCamera.18thInternationalConferenceonPatternRecognition(ICPR'06)(2006):610.URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1699915 [35] Yong,Xi,Zhang,L,andSong,Zhangjun.Real-timevehicledetectionbasedonHaarfeaturesandPairwiseGeometricalHistograms.IEEEInternationalConferenceonInformationandAutomation(2011).June:390.URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5949023 61

PAGE 62

BIOGRAPHICALSKETCH VishnuKarakkatNarayananwasborninKeralainSouthIndia.HereceivedhisM.S.inmechanicalengineeringfromtheUniversityofFlorida,GainesvilleinMay2013withaminorinelectricalandcomputerengineering.HehadreceivedhisBTech.inmechanicalengineeringfromAmritaUniversity,Coimbatore,IndiainMay2011.HisresearchisfocusedofapplicationsofComputerVisionandMachineLearningtorealtimesystems.HedevelopedaninterestandpassionforMachineLearningandComputerVision,duringhisnalyearofundergraduatestudywhileworkingonhisBachelorsthesisonMachineLearningbasedManufacturingOptimization.Heplanstopursueadoctoratedegreeinthesameeldandmoveontoacademia. 62