<%BANNER%>

Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-12-31.

Permanent Link: http://ufdc.ufl.edu/UFE0042621/00001

Material Information

Title: Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-12-31.
Physical Description: Book
Language: english
Creator: Subbiah, Sundar
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: Agricultural and Biological Engineering -- Dissertations, Academic -- UF
Genre: Agricultural and Biological Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Statement of Responsibility: by Sundar Subbiah.
Thesis: Thesis (M.S.)--University of Florida, 2010.
Local: Adviser: Burks, Thomas F.
Electronic Access: INACCESSIBLE UNTIL 2012-12-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042621:00001

Permanent Link: http://ufdc.ufl.edu/UFE0042621/00001

Material Information

Title: Record for a UF thesis. Title & abstract won't display until thesis is accessible after 2012-12-31.
Physical Description: Book
Language: english
Creator: Subbiah, Sundar
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: Agricultural and Biological Engineering -- Dissertations, Academic -- UF
Genre: Agricultural and Biological Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Statement of Responsibility: by Sundar Subbiah.
Thesis: Thesis (M.S.)--University of Florida, 2010.
Local: Adviser: Burks, Thomas F.
Electronic Access: INACCESSIBLE UNTIL 2012-12-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042621:00001


This item has the following downloads:


Full Text

PAGE 1

ROBUSTAUTONOMOUSGUIDANCEFORCITRUSGROVES By SUNDARSUBBIAH ATHESISPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF MASTEROFSCIENCE UNIVERSITYOFFLORIDA 2010

PAGE 2

c r 2010SundarSubbiah 2

PAGE 3

Tomyparents,whoweremyforemostteachers,withoutwhoseu nwaveringsupportand encouragement,thiswouldnothavebeenpossible 3

PAGE 4

ACKNOWLEDGMENTS IsincerelythankDr.ThomasF.Burksforbeingagreatadviso r.Hisacademicand moralguidanceoverthecourseofthreeyearshashelpedmesh apemylifeandcareer ininnumerableways.Myspecialthanksforhispatientadvic eandencouragement throughthecourseofmyresearch. IwouldliketothankGregoryPughandMikeZingaro,forhelpi ngmeoutonall sortsoftechnicalproblemsthataroseduringresearch.Ial sothankthemforofferingthe pleasureoftheircompanyinthelab. IthankDr.DukeBulanonandfellowgraduatestudents—Venka t,Anirudh,Vijay andIan—forcomingupwithhelpfulideasandsuggestionsdur ingthecourseofmy research. IwouldliketoacknowledgetheAgriculturalandBiological Engineeringdepartment andstaffmembersfortheirhelpandsupport.Finally,speci althankstoallthefree-software tools,(especiallyQt,L A T E X2 ,EmacsandInkScape)thatprovedimmenselyusefulfor researchandthesiswriting. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ..................................4 LISTOFTABLES ......................................8 LISTOFFIGURES .....................................9 ABSTRACT .........................................11 CHAPTER 1INTRODUCTION ...................................13 1.1CitrusHarvesting ................................13 1.2MechanicalHarvesting .............................14 1.3AutonomousGuidance .............................15 1.4SoftwareSystems ...............................16 2BACKGROUND ...................................18 2.1ScopeofPreviousWork ............................18 2.2HardwareConguration ............................18 2.2.1EGatorUtilityVehicle ..........................19 2.2.2LADAR .................................20 2.2.3Camera .................................20 2.2.4OrientationandLocalizationSensors .................21 2.2.5AdditionalComputationalUnits ....................21 2.3SoftwareConguration .............................22 2.3.1ProgrammingEnvironment ......................22 2.3.2SoftwareArchitecture .........................23 2.3.3Algorithms ................................24 2.3.3.1MachinevisionandLADARbasedautonomousguidance 24 2.3.3.2Headlandturning .......................25 3LITERATUREREVIEW ...............................27 3.1AgriculturalAutonomousGuidance ......................27 3.1.1ContactMethods ............................27 3.1.2Non-ContactMethods .........................28 3.1.2.1Ultrasonicmethods .....................28 3.1.2.2Laserbasedtriangulation ..................29 3.1.2.3Deadreckoning .......................29 3.1.2.4GPSbasedmethods ....................31 3.1.2.5Visionbasedmethods ....................31 3.1.3HeadlandTurning ............................33 3.2SoftwareArchitecture .............................35 5

PAGE 6

4SOFTWAREARCHITECTURE ...........................38 4.1SoftwareinAgriculture .............................38 4.2FieldAutomationSoftware ...........................40 4.2.1Requirements ..............................40 4.2.2Design ..................................41 4.2.3DesignChoices .............................44 4.3The DashBoard System ............................45 4.3.1TheDevice ...............................46 4.3.2TheController ..............................46 4.3.3TheDashboard .............................47 4.4Implementation .................................49 4.4.1GUI ...................................49 4.4.2OtherFeatures .............................51 4.5Discussion ...................................51 4.5.1Advantages ...............................51 4.5.2Limitations ................................52 5ODOMETRY .....................................55 5.1Methods .....................................56 5.2VisualOdometry ................................59 5.2.1TechniqueOverview ..........................60 5.2.2CoordinateSystems ..........................61 5.2.2.1Cameramodel ........................61 5.2.2.2Globalcoordinates ......................63 5.2.3FeaturePointTracking .........................64 5.2.3.1Featuredetection ......................64 5.2.3.2Tracking ............................65 5.2.3.3Displacementestimation ..................66 5.2.3.4Orientationestimation ....................67 5.2.3.5Visionsystemcalibration ..................68 5.2.4ExperimentsandDiscussion .....................69 5.2.5Limitations ................................71 5.3KalmanFiltering ................................71 5.3.1KalmanFilterTheory ..........................72 5.3.2EnhancedVisualOdometry ......................73 5.3.2.1Systemmodel ........................73 5.3.2.2Determinationofconstants .................75 5.3.2.3Kalmanlterdesign .....................77 5.4Experiments ..................................79 5.4.1DistanceMeasurement .........................79 5.4.2HeadingMeasurement .........................80 6

PAGE 7

6CITRUSGROVENAVIGATION ...........................81 6.1SoftwareArchitecture .............................81 6.2NavigationFunctions ..............................82 6.2.1AlleywayNavigation ..........................82 6.2.2End-of-RowTurnControl ........................83 6.2.2.1Point-to-pointdrivingcontroller ...............84 6.2.2.2Steeringandspeedcontrol .................86 6.3Results .....................................87 6.3.1EvaluationofPoint-to-pointDrivingController ............87 6.3.2EvaluationofLoop-turnBehavior ...................88 7CONCLUSION ....................................91 7.1Limitations ...................................92 7.1.1SoftwareArchitecture .........................92 7.1.2Odometry ................................93 7.2FutureDirections ................................93 APPENDIX AUSINGTHEDASHBOARD .............................95 BUMLDIAGRAMS ...................................111 REFERENCES .......................................113 BIOGRAPHICALSKETCH ................................119 7

PAGE 8

LISTOFTABLES Table page 2-1HardwareSpecications ...............................22 5-1Effectoffeaturecountonaccuracyandcomputationalef ciency ........70 5-2Effectofterrainonaccuracy:Errorinthedistancetrav eled ...........71 5-3Distancemeasurementperformance ........................79 5-4Distancemeasurementperformanceonfeaturelesssurfa ce(Road) .......79 5-5Headingmeasurementperformance ........................80 6-1Straightlinetrackingperformance .........................88 6-2Loopturntrackingperformance ...........................89 8

PAGE 9

LISTOFFIGURES Figure page 2-1CitrusGroveIllustration ...............................18 2-2Schematicshowingsystemconguration .....................19 2-3Softwarearchitecturedescribedin[46] .......................24 4-1Ahierarchicalcontrolarchitecture .........................40 4-2UMLRepresentationoftheDashboard .......................45 4-3DashboardCallSequence ..............................48 4-4DashboardImplementationSchematic .......................49 4-5Deviceselection ...................................50 4-6MainWindow .....................................50 5-1VisualOdometryin[59] ...............................60 5-2Visualodometryoverview ..............................61 5-3Co-ordinatesystems .................................62 5-4Visualodometry—controlow ...........................64 5-5FeaturedetectionROIandfeaturesinatypicalFOV ...............65 5-6Groundplanemotionvector .............................67 5-7Sampleimagesofroadandturfused .......................70 5-8IllustrationofK sdeterminationexperiment .....................76 5-9Kalmanlteredvisualodometry—controlow ..................77 6-1SoftwareImplementation ..............................82 6-2Alley-wayNavigation—ControlFlow ........................83 6-3End-of-rowTurningillustration ............................84 6-4Point-to-Pointdrivingcontrollerblockdiagram ...................85 6-5Reachingvia-pointswiththePoint-to-Pointdrivingco ntroller ...........86 6-6Pathtrackingaccuracyforastraightlinepath ...................88 6-7Pathtrackingaccuracyforaloopturn .......................89 9

PAGE 10

B-1UMLRepresentationoftheGuidanceapplicationUsingDa shboard .......111 B-2UMLRepresentationofDashBoardcomponents .................112 10

PAGE 11

AbstractofThesisPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentofthe RequirementsfortheDegreeofMasterofScience ROBUSTAUTONOMOUSGUIDANCEFORCITRUSGROVES By SundarSubbiah December2010 Chair:ThomasFBurksMajor:AgriculturalandBiologicalEngineering Themainfocusofthisthesisistheconstructionofrobustag riculturalguidance systems.Thisworkbuildsuponpreviousresearchonbuildin gagriculturalguidance incitrusgrovesandtriestoimproveonitsrobustness.Ther obustnessofthesystem wasimprovedonfromtwodifferentfronts-controlsoftware architectureandheadland turningmaneuvers. Theprimelimitingfactorsintheexistingcontrolsoftware werestudiedand comparedwithsoftwareapproachestakeninothercontempor arynon-agricultural systems.Asaresult,arobustsoftwareframeworkcalledthe DashBoard hasbeen arrivedupon.Thisnewsoftwareframework,improvesrobust nessbylayingafoundation thatadequatelyisolatessoftwarecomponentsandchanneli zesthecommunication betweenthem.Thisisolationiskeytoabstractingthecompl exityofsoftwareand ensuringthattheassumptionsofonecomponentdonotadvers elyaffecttheothers. Thus,theresultingcontrolapplicationsarerobustandpre dictable.Apartfrom robustness;thecomplexity,scalabilityandre-usability characteristicsoftheframework areanalyzed. Headlandturningbehavioroftheexistingsystem,underpra cticalgroveconditions wasstudiedanduncertaintieshamperingrobustturningwer eidentied.Anewmethod, whichtakesaclosed-loopapproachtowardturning,usingKa lmanlteredvisual odometrytolocalizethevehicleina2-Dspace,hasbeendeve loped.Thiseliminates 11

PAGE 12

theneedforaGPS,andhenceallowsthesystemtoberobustund erpoorGPSsignal conditions.Asimpleguidancealgorithm,formaneuveringi ntheheadland,basedon via-pointshasbeendeveoped.Thiscouldfacilitatethesys temdealwithissueslike insufcientturningspace,roughterrainandirregulariti esinthetreerow.Theapproach wastestedundertypicaleldconditionsandtheresultsare presented. 12

PAGE 13

CHAPTER1 INTRODUCTION Citrusfruitsarebeingproducedandconsumedincountriesa roundtheworld. Presently,80%ofworld'scitrusproductionisgeographica llyconcentratedinFlorida, UnitedStatesandS aoPaulo,Brazil[ 44 ].Thankstoglobaltradeliberalizationand improvementsincitrustransportationandpackagingtechn ologies;citrusconsumption andthusproductionhasbeenexperiencingstronggrowthsin cethe1980s.While consumptionisontherise,citrusproductionfacilitiesar oundtheworldcompetewith eachotherinkeepingupqualityandbringingdownproductio ncost.TheFloridacitrus industryisthelargestintheUnitedStatesandisamajoreco nomicforceinthestate. 1.1CitrusHarvesting Risinglandvalues,climaticvariations,diseasesandlabo ravailabilityarethemajor challengesfacedbytheFloridacitrusindustryinrecentye ars.Thoughlaboravailability doesnotcontributetoproductionlossesasmuchasdiseases andtherecentfreezes, ittendstoplacetheindustryinacompetitivedisadvantage withotherproducerslike Brazil.Citrusharvestinghasbeenhistoricallychallengi ngbothintermsofcostandlabor availability.Forthe2003-04season,harvestingexpendit ureaccountedforanaverage of41%ofthetotalproductioncostinfarmsacrossFlorida[ 44 ].Harvestingiscomprised ofthreeactivities-Picking,Road-sidingandHauling.Pic kingistheremovaloffruit fromthetree,road-sidingisthetransportoffruitfromthe treestothetrucknear-by andhaulingisthemasstransportoffruitfromthefarmstoth epackerorprocessor. Thersttwooperationsarethemostlaborintensiveandthey representthemajor costcomponent.Fruitpickingandroad-sidingalone,costs toabout$1.60perboxof harvestedfruitinFloridaagainst$0.384inS aoPaulo,Brazil[ 27 ].Addingtothelabor costsisthefactthatthislaborrequirementisseasonal.Th eindustrytypicallyemploys around5000workersinyearroundfarmmaintenanceandforth eharvestingseason,up 13

PAGE 14

to20,000moreworkersareneeded.Findingtemporarylabora tsuchhugevolumesis difcultandtendstobecomeexpensive. Novelharvestingpractices,thateitherreducethelaborre quirementorimprove harvestingproductivity,canbeexpectedtohaveasignica ntimpactontheproduction costandthushelpcountermarketforces.The$9.3billion 1 Floridacitrusindustry isdominatedbytechnologysavvy,agglomeratedcorporatio nsequippedformass production.Overthepastdecade,theindustryalsosawgrow thofthirdpartyservice providerswho,forexample,provideonlyharvestingservic estothefarmowners.Given thescaleandglobalimpactofthisindustry,itisreasonabl etoexpectthattheindustry willwelcomeintegrationoftoday'scuttingedgetechnolog iesintoharvesting.Over theyears,numerousresearchershaveventuredintoimprovi ngfarmyields,disease identication,useofabscissionchemicalsandmechanizat ionofharvesting,withvarying degreesofsuccess. 1.2MechanicalHarvesting TheFloridaindustrystartedshowinginterest,duringthel ate1960sbecauseof laborshortage.Importinglaborwasrestrictedandlaborde mandsfromthegrowthof highprot,non-agriculturalsectorschokedlaborsupply[ 57 ].Respondingtocallsfor improvementincitrusharvestingpractices,variousmecha nicalharvestingtechnologies havebeendeveloped.Mechanicalharvestingiseffectivebe causeithelpsreduce harvestingcostsandatthesametimehelpfarmownersdepend onfewerpeopleinthe harvestingseason.Techniqueslikecanopyandtrunkshakin ghavebeensuccessfully adoptedinthecitrusfarmsofFlorida.Asaresult,theuseof mechanicalharvestershas seenasteadyincreasesince1999.Mechanicalharvestingha sbeenprovedtohave savedanestimated10to20%inharvestingcosts[ 37 ]inthecurrentstateoftheart. 1 2003-04estimates 14

PAGE 15

Roboticcitrusharvesting,asanemergingtechnologyinthi sareaholdspromises forimprovedharvestingefciencyandreducedtreedamage. Multidisciplinaryresearch effortsininvestigatingroboticharvestingproblemsarei nprogress.Machinevisionfor fruitidenticationanddiseasedetection,advancedrobot icmanipulationtechniquesfor harvestingandautonomousvehicleguidanceareamongthech iefresearchavenues[ 8 ]. Onedifferencebetweendesignofindustrialandagricultur almachineryisthat, agriculturalmachineryshouldbedesignedtoworkwithnatu re;andthatintroduces awiderangeofuncertainties.Inthecaseofcitrusharvesti ng,harvestingmachinery isrequiredtobeablehandlevariousfruitvarietiesandwor kingroveswithvarying dimensions.Differentcitrusfruitvarietieshavemajordi fferencesintermsofcolor, sizeandweight.Thesedifferences,forinstance,arethema inchallengesindesign ofautomatedvisionbasedfruitidenticationsystemsforh arvesting.Likewise,grove dimensionsandtreeheightsofexistinggrovesdictatethes izeandbehaviorofdesigned machinery.Whileindustrialmaterialhandlingrobotsenjo yatleastasemi-structured workingenvironment,agriculturalrobotsmustndtheirwa ythroughill-structuredtree branches. 1.3AutonomousGuidance Asvariousharvestingtechnologieshavebeenindevelopmen toverthepast30 years,theideathatguidingtheharvestingvehiclecanbeau tomatedhasstartedto emerge[ 51 ].Autonomousguidancecanbeviewedascompletelyreplacin gthehuman operator,orassistinganoperatorindriving.Theadvantag eofhavingsuchasystem isevident-theoperatorcannowconcentrateonothertasksw hilethevehiclemoves. Giventhepresentstateoftechnology,autonomousdrivingc apabilitieshavenotfound enoughgroundinroadtransport,primarilybecauseofsafet yconcerns.Incaseof agriculturalapplications,thechallengesareentirelydi fferent.Forapplicationslike spraying,planting,harvesting,etc.,thevehiclehastomo veataveryslowandconstant pace.Sometimesthevehiclesarebigandrequiremoreeffort foradrivertooperate. 15

PAGE 16

Withlonghoursofslowandconstantdriving,itbecomesamen ialtask,whichthe humanoperatorwouldliketoavoid.Thismakesguidanceanat tractivecandidatefor automation. Theguidanceproblemcanbephrasedasthecontinuouspositi oningofamoving vehicleorimplement,throughadesiredpath,whilemaintai ningasaferelationship withsurroundingobjects.Thereexistsadesiredstate,ana ctualstate,andanerrorat anypointoftime.Autonomousguidancecanthusbeviewedasa trackingproblem, withpositionasthecontrolledparameter.Forclosedloopc ontrol,wewouldrequire theautomaticcontrolsystemtoknowtheprecisestateofthe vehicleinreal-time. Assumingatwo-dimensionalworld,thestatecanbeminimall yexpressedwithcartesian co-ordinatesandtheheadingangle.Nowthecontrollershou ldbeabletodetermine theactionnecessarytominimizetheerrorinpositionandor ientation.Atrivialcontrol strategywouldbetocomeupwiththeerroranduseittodrivet heactuator,butfor purposesofoptimalsteering,thesystemmighthavetoconsi derthepathlyingaheadin makingcontroldecisions.Also,decisionslikewhentostar tturns,movingtothenextrow andhowtoturn,complicatethecontrol. Inadditiontothechallengesdiscussedintheprevioussect ions,absenceof road-likemarkers,unevenandmarshyterrainwithsporadic GPSreceptioncanpose signicantdifculties.Giventhecomplexityofcontrolin volvedandtheuncertainties faced,itsnothardtofathomthatthesolutionwouldrequire asignicantamountof software.Softwareforsuchcontrolsystemsisinherentlyc oncurrentandmulti-threaded innature;thusrequiringasignicantamountofattention. 1.4SoftwareSystems Complexautonomousguidancesystems,willrequiremultipl econtrolloopswhich arerobusttonoiseandappropriatelyresponsive.Insoftwa retermsthistranslatesto anapplicationthatismulti-threadedandreal-timeinnatu re.Whenmultipleconcurrent threadsofexecutionareused,itisoftenrequiredtosynchr onizeandensureconsistency 16

PAGE 17

oftheshareddata.Also,quiteoftentheneedforusingmulti plecomputingunitsfor solvingaproblem,arises.Thiscouldbeduetocomputationa lrequirementsordue todataavailabilitytoindividualcomputers.Insuchcases ,itisbenecialtohavea distributedsoftwaresystemthatiscapableofusingthenet workofdataexchange. Unfortunately,softwareinvolvingsuchmechanismsbecome sfragileandvulnerable touncertainties.Whileconstructionof100%reliablesoft wareremainsunrealizablein today'ssystems,engineeringpracticescanhelpboosttheo verallreliabilityofthesystem considerably.Constructionofsuchasystemforautonomous guidanceisoneofthe themesofthisresearch. Themainobjectiveofthisresearcheffortisimprovingther obustnessofanexisting guidancesystemdevelopedin[ 46 ].Inlookingforwaystoenhancetherobustnessofthe system,itwasidentiedthataexiblesoftwarefoundation isnecessary.Thisfoundation wasrequiredtoenablethecontrolsystemdesignerbuildrob ustsystemsontopofit. Therequirementsofsuchafoundationwereinitiallyidenti edas, 1.Supportconcurrentlyrunningsoftwarecomponents2.Enableinterfacesforcommunicationbetweencomponents3.AllowdistributionofdataacrossanetworkWithsuchafoundationinplace,thesoftwarecancontinueto growinacontrolled framework,thusmakingtheoverallsystemmorereliable. Anothermainobjectiveofthisresearchistoenhancethetur ningperformance oftheexistingguidancesystem.Anearlierapproachforthe autonomousheadland turningmaneuverby[ 49 ]wassuccessfullyimplementedonatestvehicleplatformun der certaingroveconditions.Thisapproachwasreimplemented toproducerobustresults undervaryinggroveconditions.Thisnewapproachenjoysro bustnessmainlydueto thesoftwareframework.Theframeworkenableddevelopment ofmultiple,concurrent, controllingagentsthatcompetetocontroltheturnbehavio r.Eachagentisuniquein termsofaccuracyandrelevance.Afusionalgorithmthenfus esthecontroleffortsof theseagents,toproducethemostrelevantcontroleffortfo rthegivencontext. 17

PAGE 18

CHAPTER2 BACKGROUND Sincethisworkisbuiltuponapreviouswork,establishment ofboundarybetween thecurrentandpreviousresearchbecomesnecessary.Inthi schapter,abriefaccount ontheworkof[ 46 ]discussingtheoverallscope,hardwareandsoftwarecong urations andtheresultsobtained,arepresentedinthefollowingsec tions. 2.1ScopeofPreviousWork Themainobjectiveof[ 46 ]wastobuildabasicautonomousguidancesystemfor citrusgroveapplications.Thedevelopmentstartedonareg ularJohnDeeretractor, afteranelectro-hydraulicretrotforsteeringcontrol.T hesteeringwascontrolledbyan on-boardcomputer.Thetractorwasfoundtofollowalleyway swithanerroroflessthan 1%atamaximumspeedof3.1 m=s. Figure2-1.CitrusGroveIllustration:Alleyways,headlan dandturnprocedures 2.2HardwareConguration Forfurtherdevelopment,anelectrically-poweredvehicle platformhasbeenused. Thevehiclewasttedwiththerequiredsensorsandcomputat ionalunits.Withthis platformset,thedevelopmentfocusedmainlyontwoaspects ofguidance-Alleyway navigationandheadlandturns.Alley-wayisthespaceinbet weentreerows,andis primarilyusedformaintainingthetreesthemselves.Headl andisawiderstripoflandat theendoftreerows.Theheadlandisusedforhaulingandmove mentofotherheavy equipment.Thedimensionsofalleywaysandheadlandsdiffe rindifferentgroves.The 18

PAGE 19

vehiclewasprogrammedtomovethroughrelativelystraight alleywaysandattheendof alley,turnintothenextalley.U-turnsandswitch-backtur nswereprogrammedintothe turningalgorithm. Thehardwarecomponentsthatconstitutethevehicleplatfo rmarebrieydescribed inthefollowingsections.Ineachofthesections,theworki ngprinciple,basicspecications andtheutilityofthecomponentisexplained.Atableofsele ctedspecicationsrelevant tothisresearcharepresentedattheend. Figure2-2.Schematicshowingsystemconguration2.2.1EGatorUtilityVehicle The EGator isanelectricallypoweredagriculturalutilityvehicle,c apableofcarrying lightloads.Itispoweredbyeight6Vbatteriesconnectedin serieswitha48VDC motor.Themotoriscontrolledbyanembeddedmotorcontroll er[ 21 ],whichgets positionfeedbackthroughanencoderattachedtothemotors haft.Themotordrives therearwheelsthroughadrivetrain.Thefrontwheelsfreer ollandtheyareusedfor steering.Anelectronicsteeringcontrollercontrolsthef rontwheelanglethrougha servomechanism.Thesteeringcontrollerallowsforelectr onicsettingofwheelangleby meansofaCANinterface.Forautonomouscontrolofsteering ,anon-board,PC/104 [ 12 ]compliantindustrialgrade,generalpurposeembeddedcom puterisused.The computercommandsthesteeringcontrollerviatheCANandis alsointerfacedtothe 19

PAGE 20

tractionmotorcontroller.Itisfullyprogrammableandthu sallowscompletecontrolof thevehicle'smotionthroughsoftware.However,itisauniprocessormachineandhas limitedcomputationalpower.ApowerfulwirelessLANacces spointonthevehicleallows networkingtheon-boardcomputerwithotherstationarysys tems.Thevehicledashboard controlsallowswitchingbetweenmanualorautonomousmode s.Inmanualmode,the vehicleallowsdrivingwithasteeringwheelandpedal.2.2.2LADAR ALADARorLaserRADARisanopticaldistancemeasurementdev ice.The standardSICKLMS200typeLADARusedinthisworkhasamaximu mscanwidth of180degreesandmaximumrangeof80meters.Alaseremitter -receiverassembly shootsshortpulsesoflighttotheobjectinfront.Thereec tedpulseisthenreceived andthetimeofightisusedtoestimatethedistanceoftheob ject.Arotatingmirror housedinsidethedevicedirectsthepulsetodifferentdire ctionsfrom0to180degrees inincrementsof0.5degrees.Thusthedevicecollectsdista ncesat361different directions,producingatwodimensionalmap.Anarrayof16b itdistancemeasures, representingthemapisavailablethroughaserialinterfac e.TheLADARwasmounted onaswivelmountandattachedtothefrontofthevehicle.Thi smountallowsthedevice topitchupanddownby 30degfromthegroundplane.Since,everyscanfromthe devicecanyieldatwodimensionalmapofobjectslyingaroun d,itispossibletoobtain athree-dimensionalmapofthesurroundingbyswiveling.As tandardRS-232serial interfacecantransferamaximumofonlyuptosevenframespe rsecond.Consequently, aspecialhigh-speedserialinterface-RS-422,withabaudr ateof500Kilobaud,isused tocommunicateatframeratesofaround35Hz.2.2.3Camera ASonyindustrialmachinevisioncamera,enclosedinmetalc asingwasmountedto thefront,togetafullviewofthepathahead.UnliketheLADA R,thecamera'sangleis xed.Standard640480NTSCimageswith24-bitpixelsareavailablefromthecamera 20

PAGE 21

throughaspecialframe-grabberatamaximumframerateof38 Hz.Theframegrabber isaPCIcardwithcommunicationhardwaretoassisttransfer ofvideodatafromthe cameratothesystem'sphysicalmemory,bydirectmemoryacc ess(DMA).Thusimages couldbeacquiredinreal-timewithoutslowingdowntheCPU.2.2.4OrientationandLocalizationSensors Tomeasurethevehicleorientation,aMicrostrain3DMorien tationsensor,capable oftri-axialinclinationandaccelerationmeasurementisu sed.Thedigitalorientation sensormeasurestheabsoluteorientationofthevehicleint hreeaxes,withrespectto theearth'smagneticaxis.Itisalsocapableofmeasuringro tationsintermsofrotation matrices.Thesensorhasitsownembeddedmicro-controller andcandobasicltering. CommunicationwiththesensorisenabledthrougharegularR S-232serialinterface. AJohnDeereDifferentialGPS(DGPS)receiverisavailablet ogettheabsolute positionofthevehicleinthreedimensions,whennecessary .However,thisisonlyused asabackupdeviceduringnavigation.ADGPSispreciseuptoa meterdistancein twoco-ordinates.AGPSworksbyreceivingtimesignalsfrom 4differentsatellitesand triangulatingthepositionofthereceiver.ADGPSinadditi ontotriangulating,reduces theerrorinpositionbycomparingitspositionwithaterres trialreferencestation.The DGPScanmeasurelatitude,longitude,altitudeandaverage velocity,providedthatthere areatleast4satellitesinrange.2.2.5AdditionalComputationalUnits Theon-boardcomputerwasfoundtobeinadequateforhandlin gsensoryinputs fromalltheabovesensors.Thusanadditional,generalpurp ose,industrialform-factor computerwasaddedtothesystem.Allthesensorswereconnec tedtothiscomputer, asitwasusedformakingdecisionsonheading.[ 46 ]callsthisthe“Shoebox”,dueto thesmallform-factor.Serialcommunicationwassetupbetw eenthiscomputerandthe on-boardcomputerfortransmissionofheadingvalues. 21

PAGE 22

Table2-1.HardwareSpecications ComponentSpecicationValue CameraMax.Framerate24 HzResolution400 x 300 pixOutputRGBLADARMax.Framerate17 HzResolution0.5 ScanningRange180 Range80 mIMUAngularResolution0.25 PitchRange180 YawRange360 RollRange360 UpdateFrequency100 HzGPSResolution1 mUpdateFrequency5 HzWheelEncoderResolution3.5 clicks=mmUpdateFrequency40 HzSteeringControllerAngularResolution1 100 Range 30 UpdateFrequency40 HzInterfaceCANSpeedControllerResolution1 mm=sRange02000 mm=sUpdateFrequency40 HzInterfaceCAN 2.3SoftwareConguration 2.3.1ProgrammingEnvironment Thesystemconsistsoftwocomputingunits-thePC-104andth eShoebox,both generalpurposeandprogrammable.ThePC-104wasembeddedi ntothevehicle platformanddirectlycommunicateswithactuatorsthatcon trolthevehicle'sspeedand direction.Thisembeddedcomputerwaspre-installedwitht he Linux operatingsystem (SUSELinux10).Forprogramming,theC++programminglangu agewasused,for 22

PAGE 23

reasonsofperformanceandeaseofprogramming.Thestandar dfreedistributionofthe GNUC++Compilerwasused. Ontheotherhand,theShoeboxranontheWindowsXPoperating system. ProgrammingwasdonewiththeC++programminglanguagebut, usingthe Microsoft VisualStudio2003 environmentandthe MicrosoftC++ compiler.Thisconguration allowsforprogrammingGUIapplicationsforMicrosoftWind owsoperatingsystems.It shouldbenotedthattheabovecongurationdoesnotguarant eereal-timeexecution. Sincethesystemisconstantlyunderdevelopment,theneedf ordeterministicexecution waslesspronouncedthantheneedforeaseofprogramming.2.3.2SoftwareArchitecture Theproblembeingsolved,i.e.,autonomousguidancehasbee nbrokenintotwo parts.Theyare, Drivingthevehicleactuatorsatasetspeedanddirection Processingsensoryinputtocalculateheading TherstpartoftheproblemhasbeensolvedinthePC-104,bym eansofaloop thatcontinuouslycommandsthevehiclehardwareinrespons etospeedandheading commandsreceivedinoneofitsserialports.Thedevicesint hiscasearespeed controller,steeringcontrollerandserialinputfromtheS hoebox.Thesedevicesare abstractlyrepresentedbyaC++class,whichcontainsseria lcommunicationprimitives. Themainloopmakesuseoftheseclassestoreadandwritetoth erespectivedevices. TheheadinginputreceivedfromtheShoeboxisconvertedtot heactuator'sacceptable rangeandislteredwithamovingaveragelter. TheShoeboxrunsthesecondpartofthecomputation.Itcolle ctsdatafromthe sensors-Camera,LADAR,GPSandOrientationsensor,anddet erminestheheading angle.Theexactalgorithmsusedaredescribedinthenextse ction.Sinceallsensors (exceptthecamera)communicatethroughaserialinterface ,agenericserialdevice classencapsulatesalltheserialportfunctionality.Thea lgorithmsusedeviceclasses 23

PAGE 24

toreadandwritedatatoports.UnlikethePC-104,herethesy steminteractswitha user;henceaGUIthatshowsthesensorreadingsandallowsus ercontrolofdevices hasbeencreated.Aguidancethreadcontinuouslyreadsthes ensorsandproducesa headingangle.Aheadlanddetectionthreadcontinuouslymo nitorstheenvironmentfor approachingheadlandandtriggersaturnsequencewhentheh eadlandisreached. Figure2-3.Softwarearchitecturedescribedin[ 46 ] 2.3.3Algorithms Asdiscussedpreviously,themainobjectivesofthesystema reguidanceinalley waysandheadlandturning.Alleywayguidanceisthecontinu ouspositioningofthe vehicleinbetweennearly-straightrowsoftrees.Thewidth ofalleywayscanvaryfrom grovetogrove.Headlandturningisaspecialguidancerequi rementattheendofrow; thevehiclehastobeguidedintothenextalleywaybymeansof a180degturn.The algorithmsthataccomplishthesetwoobjectivesarebriey explainedinthissection. 2.3.3.1MachinevisionandLADARbasedautonomousguidance Machinevisioninanalleywayworksbydifferentiatingbetw eentreerowsand pathbymeansofcolorsegmentation.Undernormaldaylightc onditions,thetreerow, thehorizonandthegroundhavedistinctcolors.Thus,using axedcamera,thetree rowsoneithersidecanbeidentiedandapproximatetree-ro w-linescanbettedon theimage.In[ 47 ],amedianlineinbetweentheleftandrighttree-lineswasc alculated. 24

PAGE 25

Usingthislineasareference,aPIDcontrollerwasusedforg uidingthevehiclein betweenthetree-lines. WhenusingtheLADAR,thesamebasicprincipleasthevisioni sused.TheLADAR wastiltedtoscanaxedlook-aheadpointinthealleyway.Th eLADARrecordedthe treesoneithersides,andamedianpointinbetweentheobsta cleswasobtained.This medianwasusedasareferencetoguidethevehicle.Itwascon cludedthattheLADAR basedguidancewasmoreaccuratethanvisionbasedguidance forbothstraightand curvedpaths.However,boththesystemswerefoundtobeaccu rateenoughtobe comparedwithhumandriving. SinceLADARguidanceisstrictlybasedonasinglelook-ahea dpoint,missing treerowsanduneventreeshapesposedsignicantdifculty infollowingstraightlines. Ontheotherhand,thevisionwaslessaccurateandhadachanc eofbeingrendered uselessbyobstructingbranchesintheeldofview.In[ 48 ],anovelsensorfusionbased approachthatcombinestheadvantagesofbothLADARandvisi onwasdeveloped.This approachusedaFuzzyLogicEnhancedKalmanFiltertofuseth esensorreadingsand modelarobustsensor.2.3.3.2Headlandturning Tonegotiateaheadlandturn,thevehicleisexpectedtomake acontinuous180 degreeturn,oncetheendofrowisreached.In[ 49 ],theturningproblemhasbeen brokenintotwosub-problems-headlanddetectionandturni ng.Thedetectionofendof rowisdonebyconstantlymonitoringthevisualeldofviewf orrecedingtreerows.The LADARaloneisnotsufcientbecauseitwassusceptibletofa lsepositivesrenderedby missingtreesinthealleyway.Thisisduetothefactthatthe LADARguidanceisbased onasinglelookaheadpoint,whereasthevisualguidancetak eitsentireeldofview intoconsideration.However,tomakesurethatthevehiclei nitiatestheturnpreciselyat theendofrow,theLADARwasusedonlytowardtheendofrow.Al so,togetabetter viewoftheheadland,theLADARwasswepttoanangleof15dega bovehorizontal 25

PAGE 26

and30degbelowhorizontal.Thishelpedestablishtheendof rowpreciselyusingthe three-dimensionalviewobtained. Tostartturning,theIMUisconstantlymonitoredtilltheve hicle'sorientationchanges to150deg.Thesteeringwasmaintainedatafullyturnedposi tioninthedirectionof interest.Aftertherequiredorientation,thevehicleswit chesbacktoalleywaymodeto enterintothenextalleyway.Tobeabletogetasmuchspaceas possibleforturning,a slightturnintheoppositedirectionwasrequiredbeforeth eU-turn.Whileexecutingthe turn,theLADARconstantlymonitorstheeldforanyobstacl es;ineventofobstacles intheturnpath,thevehiclecomestoastop.Thevehiclewasa bletomakealoop-turn andathree-pointorswitch-backturninthisfashion.Incas eoftheswitch-backturn,the vehicledrovetothereverseforaxeddistanceatanorienta tionof90deg,andthen proceededtoturnintothenextalleyway. 26

PAGE 27

CHAPTER3 LITERATUREREVIEW Adetailedaccountofdevelopmentsintheeldofautonomous guidanceare presentedinthischapter.Theabilityofanautonomoussyst emtolocalizeitselfiskeyto navigation;however,foragriculturalnavigation,themai nfocusisoftenfollowingacrop rowornavigatingin-betweencroprows.Thus,agloballyref erencedlocalizationsystem isunnecessaryinmostcases.Researchershavetriedtouseb othGPSbasedandlocal featurebasedguidancesystems.Localizationmethodsandc rop-rowsensingmethods adoptedforguidance,throughtheyearshasbeenpresenteda ndasurveyofhowthese methodshavebeenusedforin-rowandheadlandguidanceispr esented.Following guidance,adetailedaccountonsoftwarearchitecturesuse dforautonomoussystemsin generalispresented.Somerecentdevelopmentsinadoption ofsoftwareframeworksfor agriculturalautomationhasbeenincludedattheend. 3.1AgriculturalAutonomousGuidance Automationhashadaconsiderableimpactontheagricultura lindustryinthepast decades.Theagriculturalengineeringcommunityhasbeenc onstantlyabsorbing newertechnologiesfromotherelds.Emergenceofthelates tautonomousagricultural guidancesystemslikethe JohnDeereAutoTrac TM and Case-IHAccuGuide TM standas evidencetothisdevelopment.Theideaofautomaticguidanc eisfairlyold.Evidence suggestsanautomaticallyguidedtractorbeingpatentedas earlyas1929[ 20 ]. 3.1.1ContactMethods Allearlyeffortsofguidanceusedsomekindofcontactmetho dtosensethe path.Inoneapproach,amechanicalfeelerslidesoverthecr oproworridgewhilethe vehiclemoves.Thefeelerinputisusedforcontrollingtheh ydraulicsteeringsystem. Thisguidancemechanismwasusuallyavailableasaretrotf orexistingvehicles.The disadvantagetothismethodbeing;misguidanceduetolocal deformitiesinanotherwise straightrow.Alsothemachinewasfoundtohaveworkedlesse ffectivelywithcurved 27

PAGE 28

rows[ 36 ].Inanotherapproach[ 40 ],anoverheadtrackguidedasmallsprayingvehicle. Acableconnectingtheoverheadguideandthevehicle,acted asafeeler.Theangle producedbythecablewassensedtoestimatethelateralposi tionofthevehiclewith respecttotheguide.Thesystemwasshowntohaveworkedwith amaximumerrorof12 cmforspeedsunder0.5m/s. Themaindisadvantageofcontactmethodsisthat,themechan icalnatureofthe sensingmechanismlimitsthespeedofoperation.Inmostcas es,itrequirescropstobe plantedinacompliantfashionoranelaboraterail-likeext ernalarrangementstoguide. 3.1.2Non-ContactMethods Guidancemethodsthatdonotrequiremechanicalcontactoft hemachinewith thecrop,forthepurposeofsensingalone,areclassiedund ernon-contactmethods. Almostallofthetoday'ssuccessfulcommercialmethodsfal lintothiscategory.One primitivenon-contactmethodwasusingelectricalinducta nce.Anelectricalcable buriedunderground,carryinganACsignalwascontinuously sensedbyanon-board electro-magneticcoil.Sincethestrengthoftheelectroma gneticsignaldependsonthe positionofthereceiver,thevehiclecouldestimatetheerr orinthepath.Thismethod ispopularinindustrialmaterialhandlingapplications,b utmighthinderagricultural operationsthatrequirediggingtheground.3.1.2.1Ultrasonicmethods Ultrasonicsensorsmeasuredistancebybouncinganultraso nicpulseonobstacles. Thedistanceisestimatedbytimeofightmeasurementsmade onthepulse.Singhet al.[ 43 ]developedasix-wheel,differentialsteering,greenhous esprayerplatform,for sprayingpotentiallyharmfulchemicalsingreenhouses.Th evehicleplatformusedsix ultrasonicsensorsoptimallyplacedtogetmaximumrangein formationpossible.The vehiclecouldsuccessfullynavigatetheintendedenvironm entwithamaximumspeed of1.8 m=swithanaccuracyofupto3cm.[ 43 ]hasnotedthatthetechnologymaynot 28

PAGE 29

beentirelysuitedforhighlyunstructuredenvironments,s incetheultrasonicsensors requiredthetargetsurfaceperpendiculartotheultrasoni cbeam,forthebeamtoreturn. 3.1.2.2Laserbasedtriangulation Holmqvist[ 18 ]usedalaserbasedtriangulationmethodfordetermination ofposition andorientation.Arotatinglaseremitterwasattachedtoth evehicle,onanelevated support.Fixedreectorswereplacedaroundtheeld,sucht hatthelaserlightemitted fromthevehiclecouldbereectedbacktoit.Fromthedistan ceandangleofreected light,thepositionandorientationofthevehiclewasdeter mined.Usingthismethod,it hasbeenreportedthatanerrorof5cmineachoftheaxesand1m illiradianalongeach oftheorientationaxes,wasobserved.Positioningthevehi cleenabledpreciseguidance andturningwithintheeld.However,thismethodrequiredi nstallationofreectors within300mofthevehicle,andmaybelimitedforcropsthata retallenoughtoblockthe vehicle'slineofsight. Severalothersincluding[ 51 ]usedLADARasaprincipalsourceofdistance measurement.TheLADARhasprovedtobeveryusefulforaccur atedetermination ofcroprowsorobstacles.Theabsenceofanyotherusefulinf ormationotherthan distance,hasdrivenresearcherstousetheLADARinconjunc tionwithothertechniques likevisiontoproducemorereliableguidancemethods.3.1.2.3Deadreckoning Themethodofdeterminingcurrentpositionbyintegrationo fvelocityandheading overtimeiscalleddeadreckoning.Deadreckoninghastradi tionallybeenusedin navigation,especiallyinmarinenavigationduetothefact thatthemethodworks withoutusinglandmarks.Itdemandsaccuratemeasuremento fvelocityandheading becauseerrorsinmeasurementstendtoaccumulate.Deadrec koninghasbeenused bymanyagriculturalnavigationsystemsinthepastandpres ent[ 36 ].Severalsensing mechanismsavailableforaccuratemeasurementarediscuss edbelow. 29

PAGE 30

RotaryEncoders .Rotaryencodersareubiquitousamongautonomousnavigati on equipmentoranyformofpositionmeasurement.Theyarechea perthanothermethods ofmeasurementandareconsiderablyreliable.Howeverinca seofagricultural applications,wherewheelslippage,wetsoilanduneventer rainarecommon,wheel encodersmaynotbeaccurateenough.Sincepositionmeasure mentiscumulative, smallerrorsrapidlyreducepositionaccuracy. GeomagneticDirectionSensor(GDS) .GDSusesmagnetometersforsensingthe earthsmagneticeldandthusestablishtheorientationoft hesensorwithrespecttothe earth.Bensonetal.[ 3 ]evaluatedtheeffectivenessofaux-gatemagnetometerba sed GDSforagriculturalapplicationsandfoundthatGDScouldg uidethroughstraightlines withaverageerrorsoflessthan1cm.Themagnetometerwasfo undtobesusceptible todisturbancefromthevehicle'smagneticinterferencean dhadtobemountedonan specialaluminummount,holdingthesensorunitawayfromth evehicle'sbody. InertialMeasurement .Aninertialmeasurementunit(IMU)usuallyconsistsof tri-axiallinearaccelerometerandatri-axialangularrat esensor(gyro).Acombination ofmeasurementsfromthesesensorsisusedtocalculatethep itch,yawandrollof thevehicle.Linearaccelerationsinthethreeaxescouldbe usedtoestimatethe velocityofthevehicle.Inertialmeasurementiswidelyuse dintheaviationindustry fordeadreckoning.[ 26 ]usedacombinationofGDSandgyroscopestoobtainmean positionerrorsof5cm.Thegyroscopemeasurementshadtobe improvedusingsignal processingtechniquesforisolationofnoise. Akiraetal.[ 1 ]developedalow-costsensormodulecomposedofthreevibra tory gyroscopesandtwoinclinometersforprecisedeterminatio nofheadingangles.This sensorpackagewasusedinconjunctionwithaDGPStoprovide autonomousguidance. ThesystemtestedagainstagroundtruthIMUsensorproduced r.m.serrorof1.59 degintheheadingangle.Itwasestablishedthatthedevelop edunitwaspractically applicablefornavigationpurposesasmuchasanRTKGPSandp reciseIMUsystem. 30

PAGE 31

3.1.2.4GPSbasedmethods WiththeavailabilityofGPSforcivilianusersby1996,itha sprovedtobean accurateandcheapreplacementforallnavigationalneeds. OConnoretal.[ 32 ]of StanfordUniversitydemonstratedtheuseofGPSonfarmvehi cles.Inaclosed-loop trackingcontrolarrangement,usingaGPS,microprocessor andhydraulicsteering controller,thesystemtrackedpathswithmaximumlaterale rrorof10cm.This1996 demonstrationreplacedtheuseofmultipleheadingandvelo citymeasurementsensors withasingle,cheapandreliableGPS.Howevertheguidancet hroughcurvedpathswas notaddressedinthispaper.Also,someoftheearlyGPSrecei vershadlatenciesupto1 s.Thispresentedlimitsonthespeedatwhichvehiclescould travel. Variousresearchers[ 35 ]thenfoundtheRTK-GPSwithhighupdateratetobe accurateandfastenoughforagriculturalapplications.St ombaughetal.[ 45 ]useda 5HzRTK-GPSforguidanceofa2WDCase7720tractorforaspeed of4.5m/sand amaximumpositionerrorof16cm.Noguchietal.[ 30 ]developedanRTK-GPSand IMUguidedrobotictractorsystem,withanr.m.serrorofwit hin5cm.Thetestswere conductedfordiverseagriculturaloperationslikesprayi ng,tilling,plantingetc.,inhuman operatedspeeds. GPSremainsthepreferredsensoramongmostresearcheffort sintheagricultural navigationarea[ 36 ].However,GPSreceiverscouldnotbeentirelyreliabledue to factorslikemulti-patheffectsandsignalreceptionerror s.Thiscouldposesignicant problemsincitrusgroveswhere,treesmightoftenobstruct GPSreceivers[ 51 ]. 3.1.2.5Visionbasedmethods Visionbasedmethodsrelyononeormoreimagingdevicesonth evehicletosense theenvironment.Machinevisioncapabilitiescanprovidel ocalinformationrelating tocroprowsorobstacles.In[ 14 ],developmentofavisionbasedguidancesystem thatfollowedrowcropsisdiscussed.Abatterypoweredgolf cartwastestedoncorn eldsatspeedsofabout4km/handwasfoundtofollowcroprow swithmaximum 31

PAGE 32

errorof5cm.Subsequently,in[ 15 ],aJ.I.Case-IH7110tractorwasttedwitha colorvideocameraandaIntel80286computertoprocessthei mages.Thesystem segmentedthecroprowfromthebackgroundbasedonusersele ctedtargetcolor.The systemwasfoundtobeas'skillful'asahumanoperatoruptos peedsof4.8km/h.The accuracyofvisionsystemsweretypicallyaffectedbydiffe rencesinambientlighting, shadowsandenvironmentaleffectslikedust,fog,etc.,Ben sonetal.[ 2 ]usedarticial lightinginanattempttoavoidambientlightingproblems,b utencounteredproblems withshadows.Ascomputingbecamecheaper,itbecamepossib letoruncomplex imageprocessingalgorithms.Methodslikehoughtransform ,stereoimageprocessing, principalcomponentanalysisandk-meansclusteringexist incontemporarysystems, forrobustclassicationoffeatures[ 35 ].[ 33 ]describesaunsupervisedclassierfor segmentingcutanduncutregionsofalfalfacrop.Thealgori thmwasusedtoguidea harvestertospeedsofupto4mph.Takahashietal.[ 52 ]usedstereovisiontoobtain 3Ddistancesfromthecamerastotargetobjects.Itwasobser vedthatstereovision requiresthinandsharpfeaturesfordistinctionintheleft andrightimages.Anerrorof 10%to5%forobjectswithinarangeof20mwasreported.Theerrorswe refoundto bedependentonthewidthofafeatureinpixels. Themainproblemwithvisionbasedcontrolisthat,thealgor ithmsinvolvedare verycomplexandanumberofinputconditionsaffectthesyst em'scontroldecision. Underuncontrolledoutdoorconditions,itbecomesimpossi bletobuildavisionsystem thatmakesthecorrectdecisionsforallpossibleinputs.Th esameistruewithother crop-rowsensingmechanismslikeLADAR.Giventhattheagri culturalenvironment ishighlyunstructured,decisionsbasedonlyonasinglesen singmechanismisnot satisfactory[ 35 ].Withsensorfusion,itbecomespossibletobuildarobusts ystem withasetofunreliablesensors.Zhangetal.[ 61 ]testedafusionsystemusingtwo ExtendedKalmanFilters(EKF)tofusetheoutputsofthreese nsors-RTK-GPS,GDS andmachinevision.Itwasfoundthatthefusedsystemperfor medbetterthanusingthe 32

PAGE 33

RTK-GPSalone.Subramanian&Burks[ 48 ]usedfuzzyenhancedkalmanlterfusing datafromvision,LADAR,IMUandencoder,toguideatractori ncurvedtracks.Using vision-onlyguidancesometimesfailswhenthecameraisacc identallyobstructedby vegetation.Ontheotherhand,theLADARmeasurementfailsw hentreesrowsmissone ortwotrees.So,usingcontinuous-valuednumericreliabil ityfunctionsforthesesensors wasnotpractical.Afuzzylogicsupervisorwasusedtodecid eonthereliabilityfactorsof sensorsindifferentcontexts.Itwasfoundthatwiththefuz zy-EKFschemetheaverage errorreducedto1.9cmfrom2.5cmforLADARonlyguidance,fo raspeedof3.1m/s. 3.1.3HeadlandTurning Thereisawealthofliteratureavailableonguidancealongc roprows.Thisrequires theguidancealgorithmtorecognizethecroprowandguideth evehiclebyfollowingit. IncaseofGPSguidedsystems,localadjustmentsaremadeong loballyplannedpaths toslightlyalteranotherwisestraightlinepath.Forcitru sgroves,alleywayguidanceis achievedbydetectingtreerowsandguidingthevehicleinbe tweenthem.Completely autonomousguidancewouldrequirethecapabilitytomaketu rnsintheheadlandand enterthenextalleyway.Croprowsoranykindofmarkersaren otavailableforguidance intheheadland,andthusthereisaneedforthevehicletodel iberatelyplanapath. Severalfactorslikespaceavailability,vehiclelimitati ons,obstructions,timetakentoturn havetobetakenintoaccountwhenplanningaturn. Noguchietal.[ 31 ]usedathirdordersplinefunctiontoplanacurvedtrajecto ry andguidedthevehiclealongthepath.RTK-GPSandFiberopti cgyroscope(FOG) wereusedtolocalizethevehicleduringguidance.Thevehic lewasabletogenerate aturninordertoreachaspeciedpointwithanorientationo f180degfrominitial. Theerrormeasuredbetweenthecommandedtargetpointandth eactualtargetpoint wasmeasuredtobeamaximumof5cm.Alsothealgorithmwaspro grammedto generatebothu-turnsandswitch-backturns.[ 55 ]proposesapoint-to-pointnavigation algorithmfornavigationofturnsratherthanplanninginte rmsofgeometricprimitives. 33

PAGE 34

Theproposedalgorithmplanspathsbetweenanytwogivenpoi nts,consideringthe locationofobstacles.Thegeneratedpathisthenoptimized bymeansofnumerical optimization,subjecttoconstraintsofvehicle'sturning capabilities,timeandspace required.Smoothtrajectoriesresultinginsmoothvelocit yandsteeringproleswere generatedandthealgorithmwastestedonanautonomousweed er.Itwasnotedthat thecomputingrequirementsofthealgorithmwasprohibitiv eforlongdistances,because thenumberofpossiblepathscouldbehuge.[ 10 ]recommendsmodelingpathswith geometricprimitives(linesandcirculararcsconnectedwi thclothoids 1 ),resultsin curvaturecontinuityanddemandlesscomputationalcostco mparedtopoint-to-point trajectorygeneratorslikethatsuggestedby[ 55 ].Asteeringcontrollerusingaprecise kinematicmodelextendedwithslidingparameters,isusedt oguidethevehiclethrough thegeneratedpath.Also,unlikeotherexperimentsthat,ar emostlybasedonaconstant velocityassumption,amodel-predictivevelocitycontrol isconsidered.Experimental resultsshowedthatthegeneratedpathsforaswitch-backtu rnweretrackedwithan accuracyof 5cm. Itshouldalsobenotedthatsuchaccurateturnsfromallthep resentedworkcould beachievedpartlybecauseofaccuratefeedbackfromGPSuse d.Consideringthe absenceofGPS,[ 50 ]proposesasimplestrategybasedonthepure-pursuitalgor ithm. Axedtargetpositionisfedtothealgorithmandthealgorit hmdrivesthevehicleby continuouslytryingtominimizethepositionerrorbetween currentandtargetpositions. U-turnsandswitch-backturnsweremadeusingdead-reckoni nginsteadofGPS.The detailsoftheturningbehaviorwerediscussedintheprevio uschapter. 1 Aclothoidisacontinuous-valuedfunctionthatrepresents thetransitioncurvebetweena straightlineandacircularpath.Itisusedinhighwayengin eeringfordesignoftransitioncurves betweenstraightandcurvedroadsegments. 34

PAGE 35

3.2SoftwareArchitecture Theneedforsoftwarearchitecturesforuseincontrolsyste mswasfeltwhenearly robotdesignersfoundcontrolalgorithmsgrowincreasingl ycomplex.Theearlycontrol architectureswereclassiedasrobotcontrolarchitectur esandwereusuallydealtas asubjectofArticialIntelligence.Mostearlycontrolfra meworkswerecoupledwith autonomy,andusuallyenforcedacertainhierarchyordepen dencybetweenmodules. Withimprovementsincontrolsystems,controlframeworkss tartedtobeusedfor automaticcontrolapplicationsotherthanrobotics.Asare sult,controlarchitectures startedtodroptheimpliedautonomyandbecamemoregeneric .Improvementsin softwaretechnologylikeintroductionofnewlanguagesand programmingparadigmshas alsohadsignicantimpactontrendsincontrolarchitectur es.Thissectionisdevotedto thehistoricaldevelopmentsstartingfromearlyrobotcont rolarchitecturestothemore recentgenericcontrolsystemarchitectures. Ascontrolsystemsgrewincomplexity,theneedfororganize ddevelopmentofits softwarewasmorepronounced.Itistheveryrequirementsof anautonomoussystem thatmakesthesoftwarecomplex.Brooks[ 7 ]triedtoestablishthebasicqualitiesofan idealrobotcontrolsystemas, Abletohandlemultiple,conictinggoals Abletofusedatafrommultiplesensors Robustnessandtolerancetowardfailures Extensibility Inhispaper[ 7 ],Brookssuggestsarobustcontrolarchitecturecalledsub sumption.The mainaimofthisarchitectureistomodelacontrolframework thathasrobustnessbuilt-in. Incaseofsubsumption,behaviorsaredenedasindividualc ontrolloops,andrange fromprimitivetoadvanced.Advancedbehaviorssubsumepri mitivebehaviors,andthus layersofcontrolloopsarebuiltontopofoneanothertomode lacomplexbehavior. Robustnessisintroducedinthesystembythefactthat,apar tialfailureinoneofthe controlloopsstillrenderedthesystemfunctionalwithpri mitivebehaviors.In[ 42 ],the 35

PAGE 36

taskcontrolarchitectureforamobilerobotisconsidered. Thecentralcharacteristicof thisworkistaskdecomposition.Acentralcontroller,rece ivedtasksfromtheuseror othersourcesanddecomposedintoatasktreewithprimitive tasks,anddispatched thetaskstothecorrespondingcomponents.Thecomponentsw ereinturn,software modulesprogrammedtoreceiveandexecutetasks.Thearchit ectureallowedenforcing sequentialtaskexecution,resourcesharing,exceptionha ndlingandconcurrentplanning andexecution. AnotherarchitecturecalledORCCADdevelopedin[ 4 ]simpliestheprocessof controlsoftwaredevelopmentbyspecialprogramminglangu age.UnliketheTCA, ORCCADhasonlytwolayersoftaskdecomposition;anytaskca nbedecomposedto primitiveROBOT-TASKsandindividualROBOT-TASKsareden edusingROBOT-PROCEDUREs. AROBOT-TASKcanbespeciedusingblocksofROBOT-PROCEDUR EsinaGUI basededitor,andindividualROBOT-PROCEDUREshavetobesp eciedusingthe Maestroprogramminglanguage.TheORCCADalsomarksthedil utionofautonomyand hierarchybeingenforcedinthecontrolarchitecture. CLARATyisthecontrolarchitectureusedbyNASAdevelopedf orreuseof algorithmsonmultipleroboticplatforms.In[ 28 ],themainfeaturesofthearchitecture areexplained.CLARATydenestwolayersofsoftware;funct ionalanddecisionlayers. Thefunctionallayercontainsfunctionalabstractionsofl ow-levelvehiclefunctionslike navigation,objectavoidance,vision,etc.,andthedecisi onlayerisresponsiblefor thedeliberativeactionslikeplanning,taskdecompositio n,etc.,Thefunctionallayer abstractstheentireunderlyingrobotplatformandallplat formspecicsoftwareis containedwithinthefunctionallayer.Whenusingthesamep lanningalgorithmsin structurallydistinctrobots,onlythefunctionallayerch anges.Thedecisionlayercan accessthefunctionallayeratvariouslevelsofgranularit ytogettheplansexecuted.The objectorientednatureofthefunctionallayerallowsreuse andportabilityamongdifferent platformswithoutrequiringchangesinthedecisionlayera lgorithms. 36

PAGE 37

TheComponentBasedSoftwareEngineering(CBSE)paradigmh asfound acceptance,inthedevelopmentoflargesoftwaresystems,o verthepastdecade.CBSE isthecompartmentalizationofsoftwaremodulesbydevelop mentofanapplicationinto separatebinariesandenablingwell-denedinterfacesfor communicationbetween thecomponentbinaries.Thisisusefulfordevelopmentofla rgesystemsbecause,the isolationhelpscollaborationofexpertswithouttheassum ptionsofoneaffectingthe other.ManycontrolarchitecturesembracingtheCBSEappro achhaveemergedthrough theyears[ 22 54 ].InatypicalCBSEbasedcontrolarchitecture,individual sensors andcontrolalgorithmsarewrappedintoseparatecomponent s.Eachcomponentis thenallowedtoexposeonlytherequiredparametersandcont rolfunctions.Itcanbe seenthatthisclassofarchitecturesdoesnotenforceanypa rticularhierarchy;the controldesignerisfreetoarrangethecomponentstosuitth erequirement.Inaddition tospecifyingcomponents,theJointArchitectureforUnman nedSystems(JAUS) specicationbytheDepartmentofDefense,speciesmessag esforcommunication betweencomponents.ThussystemsusingtheJAUSframeworkm usthavethe componentsdeliberatelymessageeachotherforexchangeof informationandcontrol. In[ 62 ],aproposalforusageofJAUSforautonomousagriculturale quipmenthasbeen presented,showinginterestforadoptionofsoftwareframe worksintheagricultural community.In[ 17 ],aC#based,modular,object-orientedsystemispresented .The frameworkwasprimarilydevelopedforagriculturalnaviga tionandhasaGUIpertaining tonavigation. 37

PAGE 38

CHAPTER4 SOFTWAREARCHITECTURE Computershavebeenplayinganimportantroleinagricultur aloperationsand managementoverthepastdecade.Theuseofcomputershasrev olutionizedthe followingdomainsofagriculturalproduction-Datacaptur e,dataanalysis,automatic controlanddecisionsupport.Inthefollowingsection,abr iefaccountontheroleof softwareinagriculturalengineeringispresented,inorde rtoacquaintthereaderwith theveryrangeofapplicationscomputersandsoftwarendhe re.Inthesubsequent sections,adetailedaccountonthesoftwareusedinthisres earcheffortispresented. 4.1SoftwareinAgriculture Insituationswherethedataacquiredfromasensororagroup ofsensorsmaynot bereadilyusable,acomputercanactasanintermediateproc essortoextractuseful informationfromthedata.Arangeofdataprocessingtechni quesrangingfromsimple ltering,tomachinevisionalgorithms,areinusetoday.Al socomputationbringsthe abilitytoextractinformationoutofnoisysensors.Forins tance,multi-sensorfusion algorithmsfusedatafrommultiplenoisysensorstocompute anaccurateestimateof themeasuredquantity,thusavoidingtheneedforusingasin gleexpensivesensor.In agriculturalapplicationswhereuncertaintiesareatlarg e,suchdatacapturetechniques areenablersofautomation.Inaddition,ascheapcomputati onalpowerisbecoming commonplace;thishelpsreducetheoverallcostofasystem. Novelcomputernetworkingtechnologieslikewirelesssens ornetworksarebeing widelyresearchedintheagriculturalcommunity.Theinher entabilitiesofawireless sensornetwork,enablesclosemonitoringofwidegeographi calareas.Asetof wirelesssensorsinagriculturalfarmlandscouldcollecta varietyofinformationfrom, soilconditions,atmosphericconditions,pestsetc.Thema inchallengehereispower consumptioninsensornodesaswirelesstransmissionitsel fispowerconsuming. 38

PAGE 39

Specialnetworkmanagementsoftwareisrequiredtomaintai nthebalancebetween powerconsumptionandfrequencyofwirelesstransmissions [ 63 ]. Theagriculturalindustryisalsobeginningtoseethebene tsofmanagement informationsystems,throughintegratedfarmmanagements oftwareproductsofferedby majoragriculturalequipmentcompanieslikeJohnDeere,In candCaseNewHolland. Farmmanagementsystems,withthehelpofGlobalPositionin gSystem(GPS),can mapvariouscropvariableslikesoilcondition,yield,dise asespread,pests-both geographicallyandtemporally.Bybeingabletoseethisinf ormationatdifferentlevels, thefarmercanmakeinformeddecisionsonfertilizerandpes ticideusetomaximize efciency.Thecoretosuchinformationsystemsisacentral databasemanagement system,andanetworkofservers,thatfacilitatesstoragea ndretrievalofvastamountsof information. Thevarietyofcropsandspecializedexpertiserequiredint hecultivationofeach cropvarietyhasbeenperfectapplicationsforexpertsyste ms.Expertssystemsareused fordiseasediagnosisandtreatmentofdifferentcrops.Exp ertsystemscanalsomake decisionsonirrigationandfertilizerrequirementoverva ryingclimaticconditions[ 19 ]. Fieldautomationisanotherrichareawherecomputersndni cheapplications. Automationofagriculturalequipmentreducestheneedform anpowerasdiscussedin thepreviouschapters.Theunstructuredoutdoorenvironme ntofferedbyanagricultural settingpresentsthemainchallengeinautomation.Unlikef actoryautomation,eld automationdemandsmoreintelligencefromthesystem.This chapterisdedicatedtothe descriptionofcharacteristics,designchoicesandprinci plesgoverningtheconstruction ofsuchsoftware. 39

PAGE 40

4.2FieldAutomationSoftware 4.2.1Requirements Thedesignrequirementsofautomationsoftwareisprimaril ydrivenbycertain commoncharacteristicsofautomationproblems.Themainre quirementsforthegiven applicationareidentiedasfollows, Simultaneity Real-timeresponse Hierarchy Componentcommunication Automationproblemsareinherentlysynchronouswithsimul taneousevents occurringinthephysicalworld.Theyareoftenrequiredtor espondtoeventsina timelyandreliablemannerforextendedperiodsoftime.The primaryrequirementof thisresearcheffortwastobuildasoftwarecontrolsystemt hatwillenablethevehicle toguideitselfunderanitesetofconditions.Guidanceini tselfisacompoundproblem andcanbedividedintosubproblemslike-obstacleavoidanc e,path-planning,path behavioretc.,dependingonthelevelofsophistication.Fo rcitrusgroveguidance,the vehicleisrequiredtohavetwobasicbehaviors-alleywaygu idanceandend-of-row turning.Assumingabasicguidancesystem,atanypoint,the vehicleisinanyoneof thetwomodes.Inadditiontocarryingoutbehaviors,ahighe rlevelbehaviorarbitrator isrequiredtoswitchbehaviors.Thearbitratorconstantly monitorstheenvironmentand selectsthemostappropriatebehavior.Thisapparentlyres ultsinahierarchicalcontrol architecture,wherealllevelsofcontrolaresimultaneous lyactive. nnrnn Figure4-1.HierarchicalcontrolArchitecture 40

PAGE 41

Thevehiclehasasuiteofdifferentsensorsasdiscussedinc hapter2.The controllersshouldhavesimultaneousaccesstoallthedevi cesinthesystem.The dataproducedfromthesensorsvaryfromasingle32bitinteg ertoanRGBimage. Thedevicescannotbeexpectedtohaveanylevelofuniformit yexceptforthefactthat theyproducesomekindofdata.Thecommunicationprotocols involvedinfetching datafromthedevicesareuniquetoeachdeviceandhencewoul drequireseparate driverprogramsforeachsensorandactuatorinthesystem.F orinstance,thecamera communicatesthroughauniquePCIinterface,thatfacilita tesDMA.UnliketheGPS thatcontinuouslyproducesdatavaluesinitsoutputport,t hecamerainterfacecardis anactivedevicethatwritesthesystem'sphysicalmemorydi rectly.So,amanufacturer supplied,customdriverpackagehastobeusedtointerfacet hecamera.Device driversaside,theusersofthedevices-thecontrollershav etoprogrammedtorunina continuousloop.Thecontrollersmighthavetointeractwit heachotherorevencontrol eachother.Astoday'scomputersareinherentlysequential machines,thesimultaneous controllerswoulddemandconcurrencyandsynchronization fromtheprogramming environment.Inaddition,theuserhastobeabletomonitora ndcontrolthebehavior usingaGraphicalUserInterface(GUI). Inadditiontotheaboveapplicationspecicrequirements, softwareengineering requirementslikescalability,re-usability,portabilit y,adaptability,interoperabilityand ease-of-usealsoapply.4.2.2Design Beforeattemptingtorealizetheblocksingure 4-1 ,itisnecessarytohavea softwarecontrolframeworkthathelpsbuildthishierarchy ofcomponents-Aframework thatwillprovideahostingenvironmentforcomponentstoop erateandexchange information.Thisisessentialbecauseitwouldbeimpracti calandundesirabletoinclude communicationfunctionsasapartofeveryindividualcompo nent.Acentrallumpof 41

PAGE 42

functionsthatthecomponentscoulduseformaintenanceand communicationwouldbe helpful.Therequirementsofasuchgenericframeworkareli stedbelow, Supportsmultipleconcurrentcomponents Supportscommunicationbetweencomponents Limitedrun-timeoverhead Flexibility Extensibility Supportsdistributionofcomponents Quickandeasytouse Theframeworkmustconsiderconcurrencyatalllevelsofcon trol.Everysensor andactuatorshouldconsistentlyhandlerequestsfrommult iplecomponents,i.e.,the behaviorofacomponentshouldbeexactlythesamewhenrunni ngindependentlyor withothercomponentsinthesystem.Concurrencycanbeanis suewhencomponents arenotsufcientlyisolated.Thisisespeciallytruewhenm ultiplecomponentswork onasinglepieceofsharedmemory.Forinstance,multipleim ageprocessingroutines readingfromthesamecopyofanimage,whiletheimageacquis itioncomponentisin theprocessofrefreshingtheimage,mightgetdifferentima ges. Componentsshouldbeabletocommunicateallkindsofdata,a ndshouldnot belimitedtojustnumericalvalues.Also,anycomponentint hesystemshouldbe accessibletoallothercomponentsinthesystem.Whenallco mponentsarerunningina singleaddressspace,communicationcanbeenabledbymeans ofsharedmemory. Aproducercomponentproducesavalueandmultipleconsumer sreadfromthe samememorylocation.Communicationcanbecomeachallenge whendistributing componentsacrossthenetwork.Multiplecopiesofasingled atavaluehavetobemade inordertoserverequestsofconsumers.Thisisaccomplishe dbymeansofpassing datamessagesacrossthenetwork.Thetwomaincommunicatio nparadigmsinusein variouscomponentsare,request-replyandpublish-subscr ibe.Inarequest-replytype architecture,theconsumerhastorequesttheproducerford ataeachtimeavalueis required.Thismightrequirealotofnetworkbandwidth,but isverysimpletoimplement 42

PAGE 43

andmanagefailures.Inapublish-subscribetypearchitect ure,theconsumerssubscribe fordataonlyonce,orwithapredeterminedperiodicity.The producersmaintainalistof subscribersandkeepsendingupdatemessageswheneveraval uechanges.TheJAUS architectureisdesignedtobeapublish-subscribetypearc hitecture[ 53 ]. Therun-timeoverheadoftheframeworkshouldnotrisewitht henumberof components.Frameworksmighthavetousespecializedarbit rationalgorithmstoroute callsbetweencomponents.Forreal-timeapplications,the runtimeoftheframeworkhas tobedeterministicunderallconditions.Forusewithsafet ycriticaldistributedsystems, theCommonObjectRequestBrokerArchitecture(CORBA)hasb eenextendedwith real-timeguaranteesin[ 25 ]. Theframeworkshouldnotlimitthecontrolsystemdesignedi nusingapreferred orderorhierarchy.Usinganuniforminterfaceforallcompo nentswillhelparrangingthe componentsinarbitraryorder.Limitationscansurfacewhe naparticularcomponent isgivenpriorityoraccessrightsovertheothers.Ontheoth erhandlimitingtheroles ofacomponentcanalsohelpinimprovingperformance.Forex ample,differentiating componentsonthebasisofdataowdirectioncanhelpmanage concurrency. Anopenframeworkthatsupportseffortlessadditionofcomp onentswillhelp buildthesysteminsmallincrements.Extensibility,canal someaninclusionofexisting softwareintothenewsystemwithminimaleffort.In[ 6 ],aComponentBasedSoftware Engineering(CBSE)approachtowardsbuildingrobotsisdes cribed.TheCBSE paradigmiswidelyusedinlargesoftwaresystems,foritssc alability.CBSEadvocates buildingoflargesoftwaresystemsfromsimplere-usable,p re-producedsoftware componentsasinelectroniccircuits.Eachcomponentisaco mpiledbinarybyitselfand theframeworkhasnomeanstoaccesstheinternaldatastruct uresofacomponent, exceptfortheones,thecomponentexposes.Thus,CBSEallow sforcomponentstobe immunefrominternalassumptionsofothercomponentsandat thesametimeallowing safecollaborationbetweenexperts. 43

PAGE 44

Distributionofcomponentsacrossthenetworkisessential intoday'scontrol systems,asproblemsoftengettoobigforasinglecomputert ohandle.Distribution means,componentsareallowedtotalkthroughanetworkandh avetohaveadditional codetoaccomplishthis.However,iftheprogrammerhadtopr ogramthenetwork interactionswhenprogrammingeachcomponent,itwoulddef eatthepurposeofhaving aframework.Distributedframeworksavoidthisbylettingt heframeworkabstractthe network.Soindividualcomponentsareunawareofwherether esourcestheyaccessare located.Thismeans,theprogrammingsemanticsforrequest ingvaluesfromdifferent componentsarethesameregardlessofthelocationofthecom ponent.Thisreduces complexityonthepartofthecomponentprogrammeraswellas enablesportabilityof componentsinthenetwork. Eventhough,theconstructionofframeworksisinthedomain ofsoftwareengineering, theusersofsuchframeworksaretypicallymechanicalorele ctricalengineers. Therefore,itisdesirabletohidedetailsofcomponentcomm unication,concurrency anddistribution.Betterabstractionsleadtoease-of-use andscalabilityoftheframework. 4.2.3DesignChoices Manysuchcontrolframeworkshavebeendiscussedintheprev iouschapter, startingfromtheearlyoneslikesubsumption[ 7 ],TaskControlArchitecture[ 42 ]to thelatestframeworkslikeCORBA[ 25 ]andJAUS[ 53 ].Someoftheoldercontrol architectureslikesubsumptionandTCAwerebuiltforauton omouscontrolofrobots andhencedidnotdifferentiateAIfromcontrolarchitectur e.Thesedesignstypically enforcedacertainhierarchy,ratherthanthesoftwareengi neeringaspectsofbuilding autonomoussystems.Thoughitisdesirabletouseoneofthes eframeworks,most frameworksareaimedatcontrolsystemsthatspanacrosshun dredsofnodes.The veryscaleandgeneralityofsomeoftheseframeworksmaybei rrelevantforsimple systems.Withhigh-level,objectorientedprogramminglan guageslikeC#andJava, buildingofsmallandmediumsizedsystemsfromscratchhave becomerelatively 44

PAGE 45

easy.Manysmallsystemdesignersoptforcustomframeworks ratherthangothrough learningandimplementingcomponentsofsomestandardfram ework[ 13 17 ].Inour problem,thesystemwasverysimplewithlessthan10compone ntsthatrunintwo differentcomputers.Thus,acustomframeworkbasedonobje ct-orientedprincipleswas chosen.ThemostpopularlanguageoptionsavailableareC++ ,C#andJava.C++was selectedoverJavaandC#becauseofrun-time,futurereal-t imerequirements,backward compatibilityandportability. 4.3The DashBoard System Tomeetthedemandsoftheagriculturalguidancesystem,wed erivedanobject orientedframeworkcalledtheDashboard.TheDashboardisa objectcapableofhosting anumberofheterogeneouscomponentobjects.Theprimaryre sponsibilitiesofthe Dashboardinclude-initializingcomponentobjects,manag ingcommunicationbetween theobjectsandobjectlifetimemanagement. Figure4-2.UMLRepresentationoftheDashboard 45

PAGE 46

4.3.1TheDevice ADevice,incontextoftheDashboard,isanobjectthatisuse dtomodelasensor oractuator.Asensorcontinuouslyproducesmeasurementsa ndanactuatorprovides controlfunctions.Adevice,beingthegeneralizationofth etwo,canproduceanynumber ofmeasurementsorhaveanynumberofcontrolfunctions.The CDevicePlugin class providesageneralizedabstractionofanysensororactuato rinthesystem.Thisclassis justamodelanditdoesn'tnecessarilymodelanactualhardw aredevice;evenvirtual, software-onlydevicescanbeusedforthepurposeofmodular ity.Inthereal-world, deviceseitherareselfupdating,i.e.,continuouslyprodu cemeasurementsorneedan explicitrequestfromthereader.Usually,measurementsth ataresparinglyusedwould requireanexplicitrequest,tosavethedevicefromcontinu ouslyworkingwhenitisnot beingread.Thisconceptisinternallymodeledintoadevice objectintheformofactive andpassivemeasurements. ForthedevicetobeapartoftheDashboardsystem,theprogra mmerhastowrap thedevicedriverimplementationintothethe CDevicePlugin classandspecifywhich membervariablesandfunctionstoexportfromtheclass.The deviceclasscanbeeither programmedtocontinuouslymeasureandupdateaparametero rtomeasureonlyupon arequestfromtheuser. CDevicePlugin offerstheoptionofdeclaringaparameteras passiveoractive.Thisdifferentiationbetweenactiveand passivemeasurementsisdone onlyinsidetheboundariesofthedeviceandtheusersneedno tbeawareofthetype ofmeasurements.Abstractingsuchdetailsawayfromtheuse rmakestheDashboard interfacesimpleandeasytolearn.Thedetailsofthesemant icsaregiveninAppendixA. Also,whilespecifyingtheexportableresources,theprogr ammerhastoassociateeach resourcewitharesourceID,foridenticationbyusers.4.3.2TheController The CControllerPlugin classprovidesanabstractionforcontrollers.Controller s canalsoexporttheirinternalparametersandfunctions.Ev erycontrollerobjecthasto 46

PAGE 47

implement controllerInit(),controllerMain(),controllerDeinit( ) functions. The controllerMain() functionrepresentsthemaincontrolloopandwillbecalled repeatedlyatafrequencysetbytheuser.The CControllerPlugin classalsoderives allpropertiesfromthe CDevicePlugin class.Thus,acontrollercanalsobetreatedasa devicethatiscapableofproducingmeasurementsandbeingc ontrolled.Thisproperty enablesthecontrollerstobereadandcontrolledbyotherco ntrollers.Controllers canaccessparametersandfunctionsofeachotherandthatof otherdevices.For instance,considerasimpleclimatecontrolsystemwithtwo controlledvariables -temperatureandhumidity.UsingtheDashboardsystem,the temperatureand humiditysensorsandcorrespondingactuatorscanbemodele das CDevicePlugin objectsandtheindependenttemperatureandhumiditycontr ollercanbemodeled as CControllerPlugin objects.Thesensorswouldexporttemperatureandhumidity valuesandtheactuatorwouldexposecontrolfunctionsthat setfanspeedandwater ow.Thecontrolleralsoexportcontrolfunctionsthatallo wtheusertosetthecontrolled parametersandothercongurationparameters.Thisalsoen ablestheindividual controllerstointeractorshareinformationwitheachothe r.ThustheDashboardtogether withdeviceandcontrollerplug-inscreatesanobjectorien tedplatformforprogramming multiplesimultaneouscontrolloops.4.3.3TheDashboard Instantiation .TheuserhastocreatetheDashboardandtheDashboardintur n, instantiatesallthedevicesandcontrollersattached.Dur inginstantiation,theDashboard querieseachdeviceandcontrollerforexportedresourcesa ndmaintainsadatabaseof exportedresourceswiththeirIDs.AtthisstagetheDashboa rdmaintainsthehandlesto allthedevicesandcontrollersinthesystem.Throughselec tiveinstantiationfunctions attach() and detach() theusercanchoosetokeepordropindividualdevices.This featurehelpsinrunningthesystemwithfewerdevicesineve ntofpartialfailures. 47

PAGE 48

Figure4-3.DashboardCallSequence Resolution .Whenauserorcontrollerrequiresaparticularresourceth euser callsthe get() or set() functionsoftheDashboard.The Dashboardinternallymaintainspointerstotheseresource sandresolvesthecallduring runtimeusingtheresourceID.Oncecalled,theDashboardch eckswhethertheresource isavailable;ifavailable,therequestedresourceisretur ned.Incaseofsensors,the requestedresourceisameasuredparameterandincaseofact uators,therequested resourceisacontrolfunction.Also,onlytheresources,i. e.,themeasurementsand controlfunctionsandnotthedeviceitselfisexposedtothe user.Thustheuserisnot requiredtoaddressresourcesusingaspecichardwaredevi ce.Thiscouldbeusefulin ascenariowhenmultiplesensorsprovideinformationonthe samemeasurement.Atthe 48

PAGE 49

instantiationstage,theusercanselectwhichdevicetoins tantiateandcanusethesame functioncallstoachieveadifferenteffect.Thisabstract ionofdevicesalsorevokesthe needtochangedeviceusercodewithchangesintheunderlyin gdevice. 4.4Implementation Withcontrolleranddeviceabstractions,theDashboardsys temisaframeworkfor modelingofcontrolsystems.Fortheautonomousguidancepr oblem,Subramanian[ 46 ] haddevelopeddriversforalltheI/Odevicesandalgorithms foralleywaynavigationand basicheadlandturning.Thedeviceprogramswereappropria telywrappedintoDevice plug-insandthealleywaynavigationalgorithmwasmodeled asacontroller.AGUI capableofdisplayingsensoroutputswasalsocreated. Figure4-4.DashboardImplementationSchematic4.4.1GUI ThedashboardGUIwasimplementedusingtheopen-sourceQtG UItoolkit. TheQttoolkitprovidesandeasyobject-orientedinterface toGUIanditisplatform 49

PAGE 50

independent.Sinceplatformindependencehasbeenoneofth egoalsofthesystem, usingQtisessential.TheGUIallowstheusertoselectively addandremovedevices frominstantiationlist.Thisfeatureprovedhelpfuliniso latinghardwareissues,andthe system'sresilienceagainstpartialfailures.TheGUIshow sreadingsfromalldevices andcontrollersthatareactiveatanypoint.Inadditiontom onitoring,theusercanalso manuallystartandstopcontrollers. Figure4-5.DashboardDeviceSelection Figure4-6.MainWindow 50

PAGE 51

4.4.2OtherFeatures Otherfeaturesofthesystemincludearecordandplaysystem capableofrecording thevehicle'senvironmentincludingimagesandLADARreadi ngsandplayingback.The recordandplayfunctionsareimplementedascontrollercap ableofproducingadata lewhentheuserselectsrecord.Thelecanlaterbeusedtov iewandtestvarious controllers.Thisfeaturewasmainlyusedindevelopmentof controlalgorithmsdescribed inlaterchapters. Theauto-congfeatureallowstheusertosetsystemcongur ationparameters andchoosewhichcontrollerstostartatrun-time.Detailed congurationinformation liketheturningradius,actuatorlimits,dimensionsofthe vehicle,positionofthecamera andothersystemvariablesarefedintheformofatextledur ingprogramstartup.The programthenparsesthiscongurationinformationandsets theappropriateparameters. Thisfeaturewasparticularlyusefuliniterativelytestin gdifferentsystemcongurations. Anexternalscriptcouldchangethecongurationleforeac hiterationandrunthe programandrecordresults.AnintegratedGUIbasedlogging system,providesavisual displayofvitalerrors,warningsanddebuginformationdur ingruntimeandrecordsthem todiskfordiagnostics. 4.5Discussion 4.5.1Advantages TosumuptheadvantagesofhavingtheDashboardsystem, Itprovidesanobject-orientedframeworkforincrementald evelopment,and improvedscalability. UsingC++,asopposedtootherhigh-levellanguages,provid esmaximum performanceandportability Since,allthecomponentsaredesignedtorunwithinthesame addressspace, memorycanbesharedandthusonlyminimalreplicationofdat aisrequired. Thesoftwareframeworkdevelopedprovidesconstructionof concurrentlyrunning controllersandalsoallowsthemtohierarchicallyarrange themastheuserrequires 51

PAGE 52

themtobe,withoutimposinganyspecichierarchy.Theabst ractionsprovidedin theframeworkwillhelptheusertomodelanyarbitrarydevic eorcontrollerandthe objectwillbereadytointer-operatewiththeexistingsyst emwithminimalcodefor communication.Also,sincetheresourceaccesshasbeensta ndardizedbymeansofthe DashBoard ,codechangestoacertaincomponentdonothavearippleeffe ctacrossthe system.Astheobject-orientedmodelsufcientlyabstract sthecomponents,asystem designercouldbuildthesystemfromitscomponentswithout requiringtounderstandthe internalsofeachcomponent.Thesequalitiesaredesirable forincrementaldevelopment andmaintenance. Inadditiontoenablingcommunication,theframeworkalsot akescareofthe life-timesoftheindividualcomponents.Oncetheprogramm erhasspeciedan individualcomponent,creationofthecomponentobject,se lectiveinstantiationof thecomponentandnallydestructionaretakencareofbythe framework.Alsothe frameworkallowsisolationofcomponents,i.e.,acomponen tcouldbestoppedand restartedagainwithoutinterruptingtheotherindependen tcomponents.Thisfeature greatlyhelpsindealingwithhardwarefailuresinthesyste m. 4.5.2Limitations Resourceaccessoverhead .Though,theframeworkisdesignedformaximum performance,everyaccesstoameasuredvalueorcontrolfun ctiongeneratessome overhead.Inthepresentsystem,thisoverheadisafunction ofthenumberofactive resourcescurrentlyinthesystem(O ( n )).Thiscouldbeascalabilitybottleneckwhenthe systemisextendedtouseresourcesintheorderofseveralhu ndreds.Usingefcient searchingalgorithmsforarbitrationofresourceaccessca lls,thiscouldbebroughtdown tobeindependentofnumberofresources(O (1)). Synchronization .TheDashboardsystemmodelseachcontroller(controlloop )to beaseparatethreadofexecution.Whenmultiplecontroller saccessasinglechunkof memorysimultaneously,inconsistenciescouldoccur.Fori nstance,thecameraupdates 52

PAGE 53

achunkofmemoryofsize3004003bytes,every1 20thofasecond.Whenother componentsrequestfortheimage,theDashboardsystemsimp lyreturnsapointerto theimagememory,forreading.As,readingandwritinghappe nsinparallel,itispossible thatthereaderthreadsgetinconsistentdata.Theframewor kintroducesspecialsetof functionsthatserializethereadandwriteofdata.Synchro nizationalsocomeswithan increasedruntimeoverhead,becauseofthewaitingtime. Also,itispossiblethatasingleactuatormightreceivecon ictingcommandsfrom twodifferentcontrollers.Thiscouldbeaseriousissuewhe nitcomestomechanical parts.Suchanerrorcancausewearofmechanicalpartsorpos esafetyconcerns.Itis possibletoassociateor'lock-in'actuatorswithaparticu larcontrolleranddenyaccess toothers.Unfortunately,itisdifculttogeneralizethis ideaacrossallcomponents. Sometimestheapplicationitselfmightneedtoassociatemu ltiplecontrollers.For example,considervehiclespeedcontrol-Asoftwarespeedc ontrollermightbe associatedwiththeactuators.Anothercontroller,possib lyanobstacledetectormight requireanemergencystop.Denyingaccesstotheemergencys topmechanismcould bedisastrous.Further,associatingasetofactuatorswith acontroller,translatesto rememberingthesetassociationsforacertainperiodoftim e.So,specialfunctions forassociatinganddissociatingcontrolhavetobeintrodu ced.Thiswillconsiderably complicatetheuserinterfaceoftheframework.Aneasysolu tiontothisissueisto allowtheuserbuildspecialcontrolarbitrationcomponent s.Forexample,inthevehicle speedcontrolproblemposedabove,theuserhastocreateasp ecialcontrolarbitration componentthatreceivesinputsfromboththespeedcontroll erandobstacledetector andinturncommandstheactuator. Typesafety .Theframework'sroleislimitedtoresolvingdataaccesses anddoes notdoanyprocessingonthedataitself.Fromtheframework' spointofview,aproducer producesachunkofbitsandaconsumerrequiresit;noinform ationaboutthedata typeisinvolvedintheexchange.Thus,whenreadingthedata ,theconsumerisnot 53

PAGE 54

awareofthedatatypeandhastoassumethattheproducerprod ucedinaccordance withasetstandard.Thus,datatypechangesintheproducerw ouldrequirechanges intheconsumercode.Storingtypeinformationalongwithth edatawouldhelp,butwill resultinincreasedcomplexityandruntimeoverhead.Thetr adeoffhereisincreased dependencebetweencomponents,whichleadstoanavalanche ofchangeswhena singlecomponentischanged. Distributionissues .Sincethesystemisdesignedtotakeadvantageofshared memorytoexportandimportdata,distributionofcomponent sacrossanetworkwill requireasignicantamountofeffort.Whencomponentsfrom othercomputersrequest forinformation,theframeworkhastobeabletoreplicateth edataandpassalongthe network.Also,whenmultiplenodesareinvolved,routingin formationforeachnode hastobestoredandmadeavailabletoeveryothernode.Netwo rkmanagement,i.e., maintainingtheroutinginformationforparticipatingnod esandhandlingjoining/leaving nodesbecomesaresponsibilityoftheframework.Sincethes ystemenvisionedfor agriculturalguidancedidnotrequireextensivedistribut ion,atleastintheimmediate future,inrealizationofshorttermgoals,thedashboardsy stemwasdesignedinherently stand-alone.Nevertheless,futuredistributionextensio nsarepossibleontopofthe existingframework. 54

PAGE 55

CHAPTER5 ODOMETRY Odometryistheestimationofpositionbyaccumulatingdisp lacementand directionofdisplacementovertime.Beforetheintroducti onofGPSforcivilianuse, odometrymethodswerewidelyusedfornavigation.Inertial navigationsystems,using accelerometersandorientationsensors,basedonodometry areusedforposition estimationinaircraftsandships.Duetothemechanicalnat ureofsuchsystemsand highercosts,simpleandlessaccuratemethodsarepopulari nshortdistanceand indoornavigation.TheeffectivenessofGPSinagricultura lapplicationsisdependent onsignalreceptionintheeld[ 51 ],whichinturnisdependentonlandscapeand surroundingvegetation.Odometrybasedonwheelencodersa nddirectionsensorshave beenapopularmethodamongrobotbuilders.Bysimplyintegr atingalldisplacements andheadinganglesoveraperiodoftime,thecurrentpositio nofthevehiclecanbe estimated.Duetotheaccumulativenatureofthecalculatio n,errorsproducedinthe measurementtendtoaccumulate.Thusdeadreckoningdemand stheuseofprecise sensors. In[ 46 ],thevehiclewasguidedbyobservingthetreelineatthepat hboundaries andtryingtosticktothepath-median.Headlandturningman euverwasbasedonyaw feedbackgivenbytheIMU;withoutfeedbackonthepositiono fthevehicle,thevehicle wascommandedtoturn-around.Inbothcases,apreciseodome trytechniquewas notrequired.However,ifprecisepositionfeedbackwasmad eavailabletothesystem, robustandefcientnavigationalgorithmsmightbeabletot akeadvantageofit.For example,theoldsystemwasguidedbasedontreepositionsat axedlook-aheadpoint onthepathway.Thusthepathgeneratedbytheguidancesyste misasjaggedasthe treeline.Ifthesystemconsideredasectionofpathahead,w ithpositionfeedback,a smoothpathcouldbegenerated.Inthecaseofturning,withp ositionfeedbackin( x y )co-ordinatesandacontrolalgorithm,asmoothturnpathcan begenerated,allowing 55

PAGE 56

foruncertaintieslikeunevenrowendingsandunexpectedar tifactsattheend-or-row. Thuspositionfeedbackmakesastrongcasefortheoverallro bustnessoftheguidance system.Thescopeofthisresearcheffortcoversthedevelop mentofarobustposition feedbacksystemanddevelopmentofaturningalgorithmthat takesadvantageofit. Thisrobustestimationmethodcouldopenadoorofpossibili tiesforadvancedturning behaviorssuchasobstacleavoidance,optimalpathplannin getc. Afterndingtheuseofmagnetometerandwheelencoderbased trackingunsuitable forrobustguidance,avisualodometrybasedmethodoftrack ing,developedin[ 59 ], wasimplemented.Forthepurposeofrough-terrain,open-e ldnavigation,thismethod wasfurtherrenedbyusingaKalmanlter.Inthischapter,a detailedaccountonvisual odometryandhowitcanbeusedforrobustvehicleguidancewi llbepresentedwith supportingexperimentalresults. 5.1Methods Traditionally,agriculturalguidancewasenabledbyfollo wingcroprowsbymechanical oropticalmeans.Mechanicalcontactmethodsusingspecial crop-feelerattachments wereusedtosensethecroprowforguidance[ 23 40 58 ].Mechanicalguidance methodslimitedtheoperationalspeedandaccuracyofthesy stem.Non-contact methodsbasedonlaserranging[ 18 ],ultrasound[ 43 ]anddead-reckoning[ 1 ]were developedtoimproveaccuracyandspeed.Withtheintroduct ionofGPSinthe90s, OConnoretal.[ 32 ]demonstratedtheuseofGPSforagriculturalpurposes.Sev eral researchershavefoundtheinertialnavigationsystemusef ulinconjunctionwithaGPS. [ 24 30 ]usedFiberopticGyroscope(FOG)inadditiontoanRTK-GPSf oraccuraciesof upto5cmincurvedpaths.Later,Guo&Feng[ 16 ]introducedalow-costsensorsystem basedonaGPS(withpositionaccuracyof3mandupdaterate1H z)andsolid-state inertialmeasurementunit.Asensorfusionalgorithmfused thesensoroutputsto achieveanaccuracyof0.3mata50Hzupdaterate.Also,thesy stemwasshowed totolerateGPSsignaloutageslastingupto30seconds,with amaximumpositioning 56

PAGE 57

errorof0.5m.ShowingthattheGPSmaybeunsuitableinagric ulturalenvironments duetopoorsatellitereception,[ 9 ]tookadvantageofcomputationalpower,tobuild visionandLADARbasedguidancesystemsforgreenhouseandc itrusgroveguidance applications[ 51 ].Since,theroleofGPSislimitedtolocalization,arobust autonomous guidanceplatformwouldneedsomekindofenvironmentsensi ngadditiontomakelocal adjustmentstoaglobalpath,suchasmaneuveringaroundobs tacles. Dead-reckoningcombinedwithvisionandLADARbasedguidan cesystemcould producemorerobustresults.However,sincedead-reckonin gproducesonlyaposition estimate,theeffectivenessofthesystemwilldependonthe accuracyoftheestimate. Subramainian[ 46 ]usedawheelencoderanda3-axisorientationsensor.Wheel slip causestheencodertorecordexcessmovement,thantheactua ldistancecovered.In agriculturalenvironments,wheelslipduetouneven,marsh yterrainisverycommon.On theotherhand,theorientationsensorusesamagnetometert odetectearth'smagnetic eld,andiseasilyinterferedbyexternalmagneticelds.B ensonetal.[ 3 ]successfully usedmagnetometers,inconjunctionwithGPS,forodometry; buthadtomountthe magnetometerinaspecialaluminummount,holdingthesenso rawayfromthevehicle's body.Thesensorrequirescalibrationwheneveritcomesinc ontactwithametallic surface,sincetheresidualmagneticelds 1 couldaffectthesensor'saccuracy.Also, testsconductedintypicaloutdoorenvironmentsrevealedt hatthesensorisdisturbedby othervehiclesandlargemechanicalstructuresinthevicin ity. Analternativetousingtheorientationsensoristousethev ehicle'ssteering angletoestimatetheorientationanalytically.Giventhes teeringangle r radians,the 1 Magneticeldsleftonmetallicobjectsduetosustainedcon tactwithmagnets,typicallyfrom manufacturingprocesses 57

PAGE 58

orientation ofthevehicleafteratimeintervaltisgivenby, = d rr (5–1) Where,disthedistancecoveredintimet. and,rr istheradiusofcurvaturepossiblewith r Itcanbeseenthat,theheadingestimate intheabovemethod,isagaindependent onthedistanceestimatefromtheencoder,makingitonlyasa ccurateastheencoder readings.However,itcanstillbeusedtoproduceavalidest imateofthevehicleposition, andwillbeusedlater. Machinevisionhasbeenamajorareaofexplorationfortheag riculturalguidance community.Overthepastfewdecades,researchersmainlyex ploitedthevisual demarcationbetweencroprowsandsoil(cut-uncutcroporti lled-untiledsoil),asa cueforguidance.Typically,acameramountedinfrontofthe vehicle,observesthe eldahead.Theimagesaresegmentedintermsofvegetationa ndsoilusingimage processingtechniques.Astraightlinerepresentingthecr oprowisttedusingthe cropsegments,andisusedforguidance.Searcy[ 38 ]usedtheinfraredspectrumto differentiatebetweensoilandvegetation.ABayesianclas sierwasusedtosegmentthe imageintermsofcroprowsandsoil.Gerrishetal.[ 15 ]controlledatractoralongnear straightlinepaths,usingcomputervisionalone.Thesyste mwasdevelopedtobeable trackanycroprowsbyallowingtheusertoinitiallyselectt herowcolorfromtheimage; thesystemwouldsegmenttherowsthenon. Thevisionmethodsdiscussedaboveusedvisionasameansofi dentifyingcrop rowposition,anddidnottrackthemovementofthevehicleit self.Pinto&Reid[ 34 ], usingasetofparallelcroprowsintheeldofview(FOV),the alignmentofthecroprows withrespecttothevehicle's(observer's)headingwasextr acted.Itisalsopossibleto detectandtrackthemotionoftheobserver,usingvisualodo metrytechniques.Wang etal.[ 56 ]usesastereovisionbasedheadingestimationsystemfortr ackingamoving 58

PAGE 59

vehicleinanopeneldenvironment.Inthismethod,apairof imageswereacquiredand stereo-matchedtoobtainthe3-Dlocationsofthepointsins pace.Usingsubsequent imagesfromthemovingobserver,themovementofthefeature sweretrackedin3-D. Themotionvectorswerethenprocessedtodeterminethemagn itudeanddirection ofobservermotion.In[ 59 ],visualodometrywasusedtoguideanindoorvehicle platform,proposedforgreenhouses.Assuminganeventerra inaheadofthevehicle,this methodsuccessfullytracksthevehicleusingasinglecamer a.Visualodometryseems promisingforvehicletrackingbecauseitisnotsusceptibl etointerferenceasincaseof magnetometersandprovidesareasonablyaccuratemeasureo fdirectionandheading. Inthefollowingsections,detailsoftheworkdonein[ 59 ]anditsadoptiontocitrusgrove guidanceisdiscussed. 5.2VisualOdometry Theadvantagesthatvisualodometryoffersagainsttheothe rodometrytechniques discussedaboveare, Measurementisbasedonthemovementofobjectsaroundtheve hicle,andhence isamoredirectformofmeasurementthanmeasuringwheelrot ations. Notaffectedbyslippageandinterferencefromothersource s Theonlysensorneededisacamera;andinmostguidancesyste ms,camerais alreadyapartofthesystem. Measuresmotionintermsof3-Dvectors. Visualodometryisverypopularamongroboticscommunityan dhasbeenwidely usedinlandvehiclenavigation[ 29 ].NASA'smarsroverusedthistechniquetotrackits waythroughthesandymartianterrain[ 11 ].Inmostnavigationapplications,astereo visionsystemisusedtoextractfeaturesinthreedimension s,andbytrackingthose featuresinthesubsequentframes,3Dmotionvectorswerede rived.Thetechnique discussedin[ 59 ]simpliestheproblemofndingthe3Dlocationoffeatures by 59

PAGE 60

assumingthatthefeaturepointisonthegroundplane.Thene edforstereovision andtheassociatedcomputationisavoidedbymakingthissim plifyingassumption. ASprayervehicle BGroundplaneassumption Figure5-1.VisualOdometryin[ 59 ] 5.2.1TechniqueOverview Thedeterminationofdistancetraveledbytheobserverinvo lvesfourmainsteps -Featuredetection,featuretracking,groundplaneprojec tionandmotiondetection.A setofprominentfeaturesfromtheimagearerstdetectedus ingafeaturedetection algorithmsuchasHarriscornerdetection.Inthenextframe ,ifthecamerahasmoved, thefeatureswillbedisplaced.Afeaturetrackingalgorith msuchastheKLTtracker, conductsasearcharoundthepreviousfeaturepositions.Th edisplacementofthe featurepointsfromthepreviousframetothecurrentframea rereturnedasmotion vectors.Thesemotionvectorsareintheimageco-ordinates andhavetogothrough asetofcoordinatetransformationstodeterminehowmuchth efeatureshavemoved intherealworld.Thereal-worldmotionvectorsofthesetof featuresarelteredfor outliersandaveragedtoproduceasinglevector,therevers eofwhichrepresentsthe estimatedmotionoftheobserver.Thetrackedfeaturesarer etainedinmemoryand featuretracking,transformation,etcarerepeatedforthe subsequentframes.Asfeatures continuouslymoveoutoftheFOVandarelost.Whenthenumber offeaturesfallsbelow 60

PAGE 61

asetthreshold,newfeaturesareaddedtothetrackinglistb yinvokingthedetection algorithm. Theexistingguidancesystemalreadyhadacameraforthetre erowdetection;the samesetupcanbeusedforvisualodometrywiththeadditiono fsoftware. Images Images Images Feature Tracker Co-ordinate Transformation Ground plane projection Motion estimation Displacement &Heading Figure5-2.Visualodometryoverview5.2.2CoordinateSystems Therearethreeco-ordinatesystemsinvolvedingoingfromf eaturepointsina 2-Dimagetoreal-worldco-ordinates.Asshowninthegure 5-3 ,thecameramount's geometrygivesrisetothreeco-ordinatesystems-thecamer aco-ordinatesystem( C ) withoriginO C,vehicleco-ordinatesystem( V )withoriginO Vandtheglobalsystem( G ) withoriginO G.Inthefollowingsections,theco-ordinatesystemswillbe denedandthe transformationsbetweenthemderived.5.2.2.1Cameramodel Theintrinsic33cameracalibrationmatrixCprovidestransformationbetween apointintheimage,toavectorinEuclidean3-Dspace.Cisconstitutedbycamera specicparameterslikeprincipalpoint,focallength,ske wanddistortion. 61

PAGE 62

Figure5-3.Coordinatesystems 266664u v 1377775= C 266664x r y r f377775 (5–2) where,C =266664 u 0 u 0 0v v o 0 0 1377775 (5–3) and 266664x r y r f377775 isthevectorpassingthroughthedirectionofthepointofin terestin Euclidean3-Dspace. u ,v u 0 v 0arecalledtheintrinsicparametersofthecamera,andcanbe determinedbycameracalibration.TheMATLABcameracalibr ationtoolboxwas usedtodeterminetheintrinsicparametersofthevehicle's camera. Theextrinsiccameracalibrationmatrix(X)isa34rotation-translationmatrix thattranslatespointsfromcameraco-ordinatestovehicle co-ordinates.Thevehicle co-ordinatesystemisbasedonthevehicle'scenter,andthe cameraco-ordinatesystem 62

PAGE 63

isbasedonthecameraorigin.Xisdeterminedbyphysicalmeasurementstakenfrom theset-up,likecameraelevation,cameraorientationetc. 266666664x v y v z v 1377777775= X 266666664x c y c z c 1377777775 (5–4) where,X = [ RjT ] =2666666641 0 0 L x 0 cos sinL y 0 sincosL z 0 0 0 1377777775 (5–5)[ x v y v z v ] Trepresentsapointinvehicleco-ordinatesystemand[ x c y c z c ] Trepresentsa pointincameraco-ordinatesystem. Thetranslationparameters–L x L yandL zrepresentthetranslationofcamera's originw.r.tthevehicleco-ordinatesand isthecamera'sorientation.Thetransformation matrix,V = XC1(5–6) converts2-Dpointsfromtheimageto3-Dpointswithrespect tothevehicle'scenter. Allfeaturepointsfromtheimagesneedtoundergothistrans formationtondtheactual distancetraveledinvehicleco-ordinates.5.2.2.2Globalcoordinates Inadditiontotheabovetwoco-ordinatesystems,aglobalco -ordinatesystem, basedonthevehicle'sstartingposition,isrequiredtotra ckthevehicle'sposition. Initiallytheglobalaxesandthevehicle'sco-ordinateaxe sintersect,andwhenthe vehiclemoves,theybegintoseparate.Thoughthistransfor mationisnotrequiredfor visualodometryitwillbeusedformappingofeldfeaturess uchastreerowsand 63

PAGE 64

planningturns.Thistransformationmatrixisincremental lyupdatedbynewpositionand orientationofthevehicle,asthevehiclemoves.5.2.3FeaturePointTracking Read frame (hasfeatures) !(hasfeatures) Detect features Track features Ground Projection Motion Estimate Time increment Drop frame (invalid) Update vehiclepos (valid) Figure5-4.Visualodometry—controlow5.2.3.1Featuredetection Afeaturepointisdenedasanisolatedlocalareaofmaximum orminimum intensityinaimage.Itisdesirabletohaveafeaturepointt hatisstableunderavariety ofgeometricdeformations,fortrackingthemintofuturefr ames.[ 39 ]proposesafeature selectionmethodthatselectsfeaturesthatareoptimalfor tracking,basedontexture andfeaturedissimilarity.The goodFeaturesToTrack algorithmasdescribedin[ 39 ]is availableasamethodinthe OpenCV library.Thisalgorithmisusedforinitializingthe visualodometerwithaninitialsetoffeatures(N max). goodFeaturesToTrack canbe conguredwithvariousparameterssuchas,numberoffeatur es(N max),minimumfeature 64

PAGE 65

quality(Q min),distancebetweenfeatures(d f)andfeaturesize(s f).Theseparameters wereselectedbasedonexperimentsconductedwiththevisua lodometerandwillbe discussedtowardtheend.Since,featurepointontheground planearetheonlyfeatures ofinterest,werestrictoursearchareatoarectangularreg ionofinterest(ROI)inthe image,thatrepresentsatrapezoidalpieceofground,ahead ofthevehicle.TheROI isempiricallyselectedtohaveitstopleftvertexat( h 2 w 4 )andbottomrightvertexat( 4 h 5 3 w 4 ),wherehandwaretheheightandwidthoftheimage.NotethattheROIis selectedforaxedcameraangle(12.2 )andmighthavetochangewithchangesinthe cameraangle. Figure5-5.FeaturedetectionROIandfeaturesinatypicalF OV 5.2.3.2Tracking TheLucas-Kanademethodforopticalowestimationisusedf ortrackingin[ 59 ].A pyramidalimplementationofthesamealgorithm,thatoptim izestheruntimehasbeen implementedin[ 5 ].Thepyramidalsearchingalgorithm,rstcomputespixelm otionina lowresolutionimage,andusesitasamotionestimate.Theac curacyoftheestimateis improvedbyrepeatedlysearchingonhigherresolutionimag es.Thishelpstrackinglarge 65

PAGE 66

pixelmotionsinfewersearchsteps.Asetoffeaturesfromaf rameF tfromtimetand theirpositionsP taregiventothealgorithm,andthealgorithmcomputesnewpo sitionsP t + usingF t + .Duetoocclusionsandniteimagesize,thealgorithmdecla ressome featuresas'lost',usingafeaturestatusindicatorforeac hfeature.Also,atrackingquality gure(Q t)basedonthedissimilaritybetweenthefeaturepointsinth etwoframes isproducedattheendofthetrackingcomputation.Q tcanbeusedasametricfor prioritizingthemotionestimateswhencalculatingtheres ultantmotionoftheobserver. WhilefeaturedetectionislimitedtotheROI,trackingisno tlimitedandisallowedto proceedtilltheedgeoftheimage.5.2.3.3Displacementestimation Therelativedisplacementofthevehiclefromtimeinstantn1toncanbe determinedbyndingtherelativedisplacementofthefeatu repointsontheground plane. UsingthecameramatrixC1fromequation 5–6 ,avectorpointinginthedirectionof thefeaturepointin C canbefound.ByfurtherapplyingthetransformationV,thevector canberepresentedintermsofco-ordinatesystem V Thus,thevectorw.r.t V is 266666664x r y r z r 1377777775= XC1 266666664u i v i 1 1377777775 Theintersectionofthevectorintheaboveequation,withth egroundplane,givenbyy v = 0representsthecorrespondingpointonthegroundplane. Solvingtheequationofthelineandtheplane,thepointofgr oundplaneis, 66

PAGE 67

266664x p y p z p377775=266664L xx rL y y r 0 L zz rL y y r377775 (5–7) Thegroundprojections(g n i)offeaturepointsfromframeF n1andF narethus determined. O V X V Z V + + g n1 i g n i v n i rn iFigure5-6.Groundplanemotionvector Thedifferencevector,v n i = g n1 ig n i,representsthemotionvectorsofeachfeature pointi,onthegroundplane.Aweightednumericalaveragebasedont hequalitymetricQ ifromthetrackingalgorithm,isappliedonthevectorstoarr iveatasingleresultant vector v n,thatdescribesthedisplacementoftheobserverduringtim einstantn. 5.2.3.4Orientationestimation Themotionvectorobtainedintheprevioussectiondescribe sthemotioninboththeXandZdirectionsw.r.t G .Theanglebetweenthelinesconnectingthepointsg n1 iandg n iwiththevehicle'soriginO vcanbeusedasareasonableestimateofthechangein vehicleheading. Thechangeinheading ricanbefoundusingthevectordotproduct, rn i = cos1g n1 ig n i jg n1 ijjg n ij 67

PAGE 68

Aweightedaverageoverallthefeaturepointsproducesthe nalheadingchange rnatinstantn. 5.2.3.5Visionsystemcalibration Tovalidatethevisionsystemdeveloped,thefollowingexpe rimentwasperformed. Thegroundaheadofthevehiclewasgraduatedinincrementso f1 m,toforma2-D gridofpoints Animageofthegraduatedgroundplanewascapturedusingthe visioncamera Thesemarkingsonthegroundplane,withknown3-Dlocations —( x ti y ti 0),were locatedintheimageaspixelvalues–( u i v i ) Theestimated3-Dpositions,generatedbythecameramatrix ,werecomparedwith groundtruthvalues,andtheerrorwascharacterized. Uponcomparison,itwasfoundthattheerrorintheestimatio nshowedan approximatelyquadratictrend,withincreaseinlineardis tancefromthecamera.The reasonsfortheerrorcouldbebecauseoftheaccuracyofthet ransformationmatrices. Since,someofthematrixelementsarephysicalmeasurement s,theaccuracyof suchmeasurementsdependsonthemeasurementmethod.Event houghamillimeter accuratelaserrangenderwasusedinthemeasurements,due tophysicallimitations accuracycouldbelimited.Forinstance,thecameraisseale dinametalcasing,and thereisnoaccuratemeansoflocatingthecamera'soriginor theexactorientationofthe camera.Thusanalcalibrationstepbecomesnecessarytoco rrectforsucherrors. TheerrorintheestimationwasthenmodeledusingtheMATLAB curve-ttingtool, andarelationbetweentheestimationerrorVs.thelineardi stanceofthepointwas determined.Thisrelationwasusedtoproduceanaccurate3Dpositionestimateofan arbitrarypointofinterest.Thecalibrationprocedureimp rovedtheaccuracyofthevision systemonatterrain. Thoughthe3-Dpositioningoffeaturesisaccurate,somefea turesareabruptlylost insubsequentframes.Whenthetrackingalgorithmlosestra ckofafeature,thefeature isdeclared'lost'.Butincaseofrelativelyat,featurele ssorne-texturedterrainlikea 68

PAGE 69

concreteoor,thedisparitybetweenfeaturepatternsisve rysmallandlostfeaturesare oftenmistakenfornearbysimilarpatterns.Whilesucherro rsarehardtoeliminate,such errorscanbeminimizedbyusingaweightedaverageoftracki ngquality. Sometimeslossoffeaturescouldbeduetojerkymotionorocc lusion.Tominimize trackingerrorinsuchscenarios,thetrackingalgorithmis allowedtodropcertainframes thatareconsidereduntraceable.Whenaframeisdropped,th emotionvectorestimated fortheprevioustimeinstantisconsideredforthecurrentt imeinstant.Afteradropevent hasoccurred,thetrackerisresetwithanewsetoffeatures.5.2.4ExperimentsandDiscussion Thegreaterthenumberoffeaturepointstrackedbythesyste m,greateristhe robustnessofthesystem.Withmorepointstotrack,thecomp utationalloadonthe systemisalsohigh.Fromthetrackingalgorithm,itcanbese enthatthetrackerneeds acertainminimumnumberoffeaturestobehavereliably.The systeminitiallystarts withamaximumnumberoffeaturesandstartslosingfeatures asthevehiclemoves. Thetrackercallsthefeaturedetectorwhenthenumberoffea turesfallsbelowaset minimum.Themoreoftenthefeaturedetectoriscalled,theh igherthecomputational cost.Assumingthatthefeaturesarelostataconstantrate, allowingthetrackerto operateatawiderbandoffeaturecount,shouldhelpminimiz ethecomputationalcosts. Ontheotherhandoperatingonalessernumberoffeaturescan affecttheaccuracyof measurement.Tondtheeffectoftheoperatingbandontheco mputationalcostand accuracy,trialswithdifferentoperatingbandsonthesame terrainwereconducted.The vehiclewasdrivenonastraightlinepathof75ft(22.8m) 2 andthetrackingresults werenoteddown. 2 Thetypicalturncircumferenceisabout15m–20m 69

PAGE 70

Table5-1.Effectoffeaturecountonaccuracyandcomputati onalefciency N max N minError%framesdroppedProcessingtime(ms/frame) 5213.77532.43 102 -1.71333.30 1055.93549.6910721.40534.641521.57547.021556.41546.561576.90637.4415109.67551.2915156.22535.54 Fromtheresultsintable 5-1 ,itcanbeseenthatwhenthealgorithmisallowedto operatewithinawiderbandoffeaturecounts,theaccuracyi shigher.Atthesametime, morefeaturescontributetoahigherprocessingtime. Effectofterrain .Theaccuracyofvisualodometrygreatlydependsonthequal ity offeaturesandthequalityoftracking.Insurfaceswhereth edisparitybetweenfeatures isrich,likepatchyandgrassylandscape,thetechniquecou ldbeexpectedtoperform betterthanonne-texturedsurfaces.Theresultsshownint able 5-2 demonstratethe relativeapplicabilityofthistechniqueonterrains. ARoad BTurf Figure5-7.Sampleimagesofroadandturfused 70

PAGE 71

Table5-2.Effectofterrainonaccuracy:Errorinthedistan cetraveled TerrainAverageError(%)MaxError(%)MinError(%) Road35.5041.1429.56Grass10.4214.405.04 5.2.5Limitations Apartfromtheabovefactors,otherfactorslikeoperatings peed,cameramount angleandterrainroughnessaffecttheaccuracy.Forexperi mentalpurposes,the operatingspeedofthevehiclewasmaintainedaroundanaver ageof1.2 m=s. Theerrorsobservedintable 5-2 ,whileseeminglyacceptableforthepurposesof controllingthevehicleovershortturns,requiresarelati velysmoothandfeature-rich terrain.Whentheterrainisrough,orifthevehicleexperie ncessubstantialvibrationdue toacceleration,accuracyislostconsiderably.Byobservi ngthebehaviorofthevisual odometeroneldrecordedsequences,theeffectofvibratio nonfeaturetrackingwas observed.Asfeaturetrackingisdisruptedduetoajerkinth esequence,theodometer isforcedtodropthecurrentframeandsearchfornewfeature sinthenextframe.Thus trackingisdisruptedfortwoconsecutiveframes.Withever yjerkinthemotion,accuracy iscompromisedandtheerrorsaccumulateinthenalestimat e. 5.3KalmanFiltering Unlikecommondigitalltersthatuseonlyarangeofmeasure mentstoestimate theaccuratevalueofavariable,theKalmanltertakestheu nderlyingprocess,that producesthemeasurements,intoconsiderationtoproducea nestimate.Thusitis possibletohavenoisysensormeasurementsandstillhavear easonablyaccurateidea oftheenvironment,usingaKalmanFilter.Bensonetal.[ 3 ]usedthesametofuseGPS andINSmeasurements,thuscombiningtheaccuracyofGPSwit hthehighupdaterate oftheINS.Withcomputationalcostbecomingcheapereveryd ay,thelterallowsforthe useofcheap,imprecisesensorswithaddedcomputationalef fortinsteadofexpensive andaccuratesensors.Inourapplication,theKalmanlterw asusedtorenevisual odometrytobecomeaviabletechniqueforguidance. 71

PAGE 72

5.3.1KalmanFilterTheory Kalmanlterfusesinformationfromaninternalsystemmode landtheactual measurement.Byrecursivelycorrectingtheinternalmodel withexternalmeasurements, keepsthemodel'in-track'withtheenvironment.Forthisre ason,thelterneedsa mathematicalmodelofthephysicalsystemthatproducesthe measurement. Considerthefollowingnoisylinearsystem,x k+1 = Ax k + Bu k + w k(Stateequation)y k = Cx k + z k(Outputequation)(5–8) Where,x kisthestatevector,u kistheinputvector,w kistheprocessnoisevector andz kisthemeasurementnoise.A BandCarematricesthatformthestate-space descriptionofthesystem. Noiseisintroducedinthesystemintwoforms—duringthetra nsitionofstates fromktok + 1andduringmeasurement.Assumingthatw kandz kareindependent randomvariablesrepresentingGaussianwhitenoise,theco variancesQandRcanbe expressedas,Q = E [ w k w T k ] R = E [ z k z T k ](5–9) WheretheoperatorE []denotestheexpectedvalue. TheKalmanlterrecursivelyoperatesintwodistinctsteps —PredictandUpdate. First,theltermakesanefforttoaccuratelydeterminethe estimateofx k +1using thestatemodel.Alongwiththestateestimate,aprocesscov arianceestimateP kis alsoproduced.Intheupdatestep,thestateestimateproduc edisrenedusingthe measurementobtainedattimek.Theamountofimpactthemeasurementy kwill haveonnalestimateofthestateisdependentontheprocess errorcovarianceand themeasurementerrorcovariance.A'weighting'matrixcal ledtheKalmanGainKis producedintheupdatestatethathelpsblendthepredicteds tateandthemeasurement. 72

PAGE 73

PredictSteps^ xk = A ^ x k1 + Bu k1 P k= AP k1 A T + Q(5–10) UpdateStepsK k = P 1 k C T ( CPk C T + R )1 ^ x k = ^ xk + K k ( z kC ^ xk ) P k = ( IK k C ) Pk(5–11) Where,^ x k-KalmanlterestimateofstateP k-Kalmanestimationerrorcovariance^ xk Pkapriori estimatesoftheaboveK k-KalmanGainQ-ProcessnoisecovarianceR-MeasurementnoisecovarianceI-Identitymatrix ThoughQandRareshownasconstantsintheaboveequations,theycanvary withtime.Inequations 5–11 ,K kisusedtocorrectthe apriori stateestimateusinga blendofQandR.Thusbytuningthesecovariancematricestheimpactofmeas urement orinternalestimateonthenalstateestimatecanbecontro lled.Theupdatedstate estimate^ x k,calledthe aposteriori estimate,isproducedwithaminimizederror covariance,andisusedtocalculatethe apriori estimateofthenexttimestep[ 60 ]. 5.3.2EnhancedVisualOdometry5.3.2.1Systemmodel ToapplyKalmanlteringtoanyproblem,thephysicalproces sthatproducesthe measurementhastobemodeledrst.Inodometry,thestateva riablesthatuniquely 73

PAGE 74

denethestateofthesystematanytimeinstantaretheposit ionandheading.Thus,the stateofthesystemcanberepresentedas,x k =266664x k y kk377775 Where,x k y k-VehiclePositioninglobalco-ordinates k-Vehicleheadinginglobalco-ordinates Theinputstothevehicleareaccelerationandsteering.Whe elrotationsare measurablethroughtheencoderconnectedtothevehicle'sd rive.Thespeedcommand giventothevehicle,throughthemanualpedalorthroughthe drivingcomputer,canbe directlymeasuredasdisplacementcommandusingthisencod er.Thustheinputvectoru kcanbewrittenas,u k =264n k n kk375 Where,n k-Wheelrotations k-Steeringcommandn kk-WrenchEffort ThetransitionmatrixAfrom 5–8 ,modelsthedynamicsinthesystemandisalsothe unforcedresponseofthesystem.Bymakingasimplifyingass umptionthatthesystem doesnotexhibitdynamics,Acanbeassumedtobea33identitymatrix.Thismeans thatthesystemislosslessandcontinuestobeinmotiononce setinmotion.Also,there isnointeractionbetweentheindividualstatevariables.T hecontrolmatrixBmapsthe inputstostatevariables,andinvolvesconstanttransform ations.Measurementfromthe sensor—inthiscasevisualodometer—isdirectlyintheform ofthestatevariablesand 74

PAGE 75

thusthemeasurementmatrixcanbeassumedtobea33identitymatrix.Thus,a simplemodelforthephysicalsystemasbelowhasbeenarrive dat. 266664x k +1 y k +1k +1377775=2666641 0 0 0 1 0 0 0 1377775 266664x k y kk377775+266664K w cosk 0 K w sink 0 0 K s377775 264n k n kk375y k =2666641 0 0 0 1 0 0 0 1377775 266664x k y kk377775 (5–12) Where,K w-WheelconstantK s-Steeringconstant 5.3.2.2DeterminationofconstantsK wrelatesmeasuredwheelrotationstodistancecoveredinglo balcoordinates. DerivationofK wisdonebyexperimentalmeans;thevehiclewasdriventosetd istances andtheencoderreadingsmeasured.Theconstantwasdetermi nedbythefollowing relation,K w =Distancecovered n kTheexperimentswererepeatedoverdifferentterrainandpa thtrajectoriesandan averageK wwasdetermined.K srelatesthedisplacement,steeringangleandheadingangle .Let bethe steeringanglerequiredtodescribeacircleofradiusr.Then, kr = K(5–13) 75

PAGE 76

Givenacirculartrajectoryofradiusr,theheadingangle( k) 3 whenintrajectory canbefoundfromthedistancethevehiclemovedalongthecir cumferenceofthecircle. r K w n kkFigure5-8.IllustrationofK sdeterminationexperiment k 2= K w n k 2rFromequation 5–13 k = K w n kk K = K s n kkWhere,K s = K w K. TheconstantK swasdeterminedbyexperimentation.Thevehiclewasdriveni n circlesofvaryingradii,andthesteeringanglerequiredin eachcasewasnoteddown. 3 Notethat kand kareinradians 76

PAGE 77

Withenoughnumberofreadings,theconstantK sineachcasewasdeterminedand averaged.5.3.2.3Kalmanlterdesign Read frame (hasfeatures) !(hasfeatures) Detect features Track features Ground Projection Motion Estimate Read Inputsu k Kalman Predict Time increment SetR =1 (invalid) Update vehiclepos Kalman Update SetR/V n p(valid) Figure5-9.Kalmanlteredvisualodometry—controlow InordertocompletelyimplementtheKalmanlterdescribed inequations 5–10 and 5–11 ,theerrorcovariancesfortheprocessandmeasurementhave tobelledinthe Kalmanlter.Accordingtotheapplicationrequirements,w ehaveameasurement 77

PAGE 78

system,whoseaccuracyisdependentonthequalityofthesur face,andcertain discontinuouseventslikeroughterrainandsuddenstartor stop.Itisdesirableto havearobustlterthatisresponsivetosuchdiscontinuous eventsintheenvironment. Incaseofsurfacequality,itispossibletomodelthemeasur ementerrorcovarianceasa continuousfunctionofsurfacequality.However,fordisco ntinuousevents,acontinuous functionmaynothelp. Themeasurementerrorcovarianceof 5–11 ,Risa33matrix.Thevisual odometerprovidesthetrackingerror(uncertaintyintrack ing)asaqualitymeasure foreverysuccessfullytrackedfeature.Whenthegroundsur faceisfeaturerichthis errorisminimalandviceversa.Thus,thevarianceoftracki ngerror(V f)canbethought ofasadirectmeasureofvisualodometer'sreliability.How ever,thetrackingqualityis one-dimensionalunlikethestatethatisthree-dimensiona l.Assumingthatthevariables arenotcorrelated,andapplyingthesamevariancetoallthe statevariables,weget,R = V fI 3Where,V f =Pn f =1 e 2 fande fisthetrackingerrorforanindividualfeaturef. However,infaceofuncertainties,likebumpsintheterrain ,thetrackerlosesall trackedfeaturesandhastodroptheframe.Insuchanevent,t herearenofeaturesto considerandtherelationshipdescribedabovebreaks.Thea lgorithmhandlesthiscase byfeedinginnity(highnumericalvalue)forR.Consequently,theKalmanlter,inthe nextstateestimationstep,justignoresthevisualodomete r'sstateestimate. Theprocessnoisecovarianceisinsignicantinthiscase,a stheprocessmodelis relativelysimple.Thusaconstantempiricalvalueisfedin totheQmatrixoftheupdate equation 5–11 .Q = CI 378

PAGE 79

5.4Experiments TondouthoweffectivetheKalmanlteredodometerwasaset ofexperiments weredoneondifferentconditions,andtheperformanceofvi sualodometerwithand withoutKalmanlteringwasmeasuredandcompared.5.4.1DistanceMeasurement Thevehiclewasrunforasetdistance(17.7m)ongrassyterra in,andthedistance recordedbythevisualodometerwithandwithoutKalmanlte rwasrecorded.Themain challengeingrassyterrainisdroppedframesduetojerkymo tion. Table5-3.Distancemeasurementperformance TrialUnltered(mm)Error%Kalmanltered(mm)Error% 11574112.445170563.63821552014.046174091.6443168505.044174471.42941547114.407172502.5425166696.185170633.598 Theaboveexperimentwasrepeatedonafairlyfeaturelessro adsurfacefora distanceof22m.Themainchallengehereisthequalityoffea tures. Table5-4.Distancemeasurementperformanceonfeatureles ssurface(Road) TrialUnltered(mm)Error%Kalmanltered(mm)Error% 11600237.482219290.32221558741.1431746420.61831608536.773210334.39541698029.5641923912.55051659832.5461904813.418 Whenthevisualodometerisrunonane-texturedsurface,th efeaturetracking algorithmwasmisleadduetolessdissimilaritybetweenadj acentfeatures.Thereliability guresreportedbythevisualodometryindicatedreliablet racking,whereasinreality, featuresweremis-tracked.ThisagainmisleadtheKalmanl terinacceptingthevisual odometer'smeasurements.However,weobserveamarkedimpr ovementintheturf, wheretheKalmanltersuccessfullyrejecteddroppedframe s. 79

PAGE 80

5.4.2HeadingMeasurement Tovalidatetheorientationmeasurementmechanism,asetof testswereconducted onthemachine.Withaconstantsteeringcommand,thevehicl ewasmadetodescribe circlesandwasmadetoturntillaxedangle.Beforestartin gthetest,thevehicles currentorientationwasmarkedontheground.Thenwithacon stantsteeringangle of30 deg,thevehiclewasdriveninacircleuntilthestartpositionw asreachedagain. Theactualangledescribedbythecircularmotion(2rad),wascomparedagainstthe measuredangles.Table5-5.Headingmeasurementperformance TrialActualAngleUnlteredErrorKalmanFilteredError (rad)(rad)(%)(rad)(%) 16.286.50-3.506.112.7126.286.29-0.156.073.3436.285.994.626.024.1446.286.191.436.112.7156.286.250.486.083.18 TheresultsaboveshowthattheKalmanlterdidnotsignica ntlyimproveheading measurement,exceptforthefactthattherangeoferrorvalu eshavereducedfrom8.12%to1.43%.Thelter'sinaccuracycanbeattributedtoinadequateveh iclemodel inequation 5–12 .Theinteractionsbetweentheindiviualwheelsandthebody ofthe vehicle,andthedynamicsinvolvedinsteeringamovingvehi clecouldimprovethe kalmanlter'sestimateofposition. 80

PAGE 81

CHAPTER6 CITRUSGROVENAVIGATION Inthepreviouschaptersasoftwarearchitecturethatiscap ableofhostingdiscrete controllercomponentsandakalmanlteredvisualodometry techniqueweredeveloped. Usingthesoftwarearchitectureofchapter 4 ,itispossibletoconstructacontrolsystem withmultiplesimultaneouscontrollersexchangingdataam ongthemselves.Usingthe odometrytechniquediscussedinchapter 5 asafeedback,itispossibletodevelop advancednavigationcontrollers.Inthischapter,asimple navigationcontrolsystem thatdemonstratestheusabilityofthetwoconceptsaboveha sbeenpresented.The controlsystemdevelopedwillthenbeemployedtoguidethev ehicletocitrusgrove requirements—alleywayandend-of-rowguidance. 6.1SoftwareArchitecture The DashBoard frameworkdevelopedinchapter 4 wasusedasacontainerof allthesoftwarecontrollercomponents.TheTurncontrolle r,Alley-waycontrollerand Odometerweredevelopedindependentlyandintegratedinto theframework.The frameworktakescareofcreatingandmaintainingthecommun icationbetweenthese components.Figure 6-1 showsallthesoftwarecomponentsthatconstitutethenal navigationapplication.TheAlley-waycontrollercompone ntcontainssoftwarethatwas developedpreviouslyandwaswrappedwiththeDashBoard-sp ecicwrappercode tointegrateintothispresentsoftwarearchitecture.From thegure,itcanbeseen thatthevehiclessteeringandspeedarecontrolledbythree differentcomponents —UserInterface(GUI),TurncontrolandAlley-waycontrol. TheAlley-waycontrol utilizesvisionandLarainformationtosegmentthepathand issuessteeringand speedcommands.Ontheotherhand,theTurncontrollerutili zesthenewlydeveloped Odometertomeasurepositionandissuesspeedandsteeringc ommands.Thevehicle isalsodirectlycontrollablebytheuserthroughtheGUI.(R eferchapter 4 fordetailson concurrencyandcommunicationissues). 81

PAGE 82

Figure6-1.SoftwareImplementation 6.2NavigationFunctions 6.2.1AlleywayNavigation Thesoftwarecontrollersforalleywaynavigationdevelope din[ 46 ]wasre-used intheapplication.Acolor-imagesegmentationalgorithms egmentsthepathfrom thevegetationandproduceslinesrepresentingthepathbou ndary.Forreliablepath segmentation,LADARmeasurementsfromthepathindicating thelocationofvegetation isalsoused.Informationfromthetwosourcesisfusedusing aFuzzy-Kalmanlterand isusedforsteeringthevehiclethroughthealleyway.Notet hatthiswayofguidanceis onlypossiblewhilethevehicleisinthealleyway.Thus,ane nd-of-rowdetectioncheck isperformedinthecontrollerasapartofvisionprocessing .Whenthevehicledetects 82

PAGE 83

anend-of-row,alleywaycontrolistemporarilysuspendeda ndtheturncontrolleris activated. Pathfrom vision Pathfrom LADAR Fuse Paths Command steering (!EoR) (EoRreached) (TurnComplete) EoR Turning (TurnIncomplete) Figure6-2.Alley-wayNavigation—ControlFlow6.2.2End-of-RowTurnControl In[ 46 ],asimplepoint-to-pointdrivingcontrollerusingtheGPS forpositionfeedback wasimplementedandtested.Theresultingsystemwasableto reachadiscreteset ofinputpointswithapositionaccuracyof1m[ 46 ].Thedistancebetweenthecurrent positionandtherequiredpositionwascalculatedandtheve hiclewasdriveninthe directionofthetargetpointuntilthetargetdistanceredu cedtozero.Thissameideawas extendedinthecurrentresearch,exceptthatthepositionf eedbackusedhereisVisual Odometry. APIDcontrollerwasimplementedtoregulatethespeedtoase tcruisespeed.With controlledspeed,thevehiclewasdriveninthedirectionof thetargetuntilthetarget wasreached.Sincethevehicleplatformusedisnon-holonom ic,thevehiclewillnot necessarilybeabletoreachtargetsinstraight-linepaths .Alsothesteeringmechanism ismechanicallylimitedtoacertainmaximumangle,sodirec tingthevehicleinthe target'sdirectionisnotpossibleinasinglestep.Withcon tinuouspositionandheading feedbackfromtheodometer,thesystemcontinuouslyadjust sthesteeringuntilthe 83

PAGE 84

targetisachieved.Ineachstep,thesystemincrementallyg etsclosetothetargetand nallyreachesthetargetpoint. Inthenextstep,adiscretesetofpointsrepresentingaloop -turnwasgiventothe drivingcontrollerandthecontrollerguidedthevehicleto thepoints,oneaftertheother thusdescribingaloopturn.Theway-pointsrepresentingth eturnwererecordedfrom amanualloopturndoneintheheadland.Duringthetrials,th evehiclewasfoundto closelyfollowthemanualturns.Figure 6-3 isanillustrationofthecommandedpointsfor theloop-turnandvehicletrajectoryattheend-of-row. Figure6-3.End-of-rowTurningillustration6.2.2.1Point-to-pointdrivingcontroller Assumingthattheusersuppliesthecontrolpointsforthere quiredmaneuver,the systemhastobecapableofsteeringitselftothenextvia-po int.APoint-to-Pointdriving controllerwasbuiltforthispurpose.Thecontrollercontr olsboththesteeringandspeed ofthevehiclein-ordertoachievethevia-pointsintheorde rtheyweresuppliedtothe controller.Internally,aFIFOqueueofvia-pointsismaint ainedinthecontroller'smemory, whichtheuserandothercontrollerscanaccessthroughthe DashBoard .Thevia-point 84

PAGE 85

queueisappendedwithpointsattheend,andthecontrollerc onstantlytriestoreach thepointatthefrontofthequeue.Whentheoldestvia-point isreached,itisdequeued, andthecontrollerpursuesthenextinqueue.Todeterminere quiredsteeringandspeed, thedrivingcontrollerinternallyreliesontwodistinctco ntrollers.Theinterconnectionof variouscomponentcontrollersisshowningure 6-4 Figure6-4.Point-to-Pointdrivingcontrollerblockdiagr am Asimpleon-offcontrolstrategyhasbeenadoptedforpointto-pointdriving.The controllergivesoutsteeringandspeedsignalstothevehic le,untilthetimeavia-pointis reached.'Reaching'apointherecanbedenedasplacingthe vehiclewithin cmfrom thetargetpoint.Where cisthecontrollerdistancetolerance.Thissimple,on-offc ontrol strategyworkswellwhenthevia-pointsaremoreorlessalig nedwiththevehicles currentheading.Asthevehicleplatformismechanicallyli mitedtoamaximumsteering angle,targetsthatrequiresharpturninginashortdistanc emightbeunabletoreach. Withthissimplestrategy,thevehicleresortstomakingcir clesaroundthetargetin-order toreachthetarget.Toenabletocontrollertorecognizewhe ntargetsaremissed,the targetangle tisconstantlymonitoredasthetargetapproaches.As tdivergestomore 85

PAGE 86

AReachingtarget BSpiralingmissedtarget CDynamicscalingof ctoavertspiraling Figure6-5.Reachingvia-pointswiththePoint-to-Pointdr ivingcontroller than 90 deg,thetolerance Ciscompromisedbyaconstantfactor.Thishelpsprevent thevehiclemovearoundthetargetpointsandstrayingfromi ntendedtrajectory. 6.2.2.2Steeringandspeedcontrol Usingpositionandheadingfeedbackfromthevisualodomete rthesteering controllercalculatestheheadinganglerequiredtoreacht hetargetusingthefollowing relation. req = tan1x cx t y cy t 86

PAGE 87

Where,x candx trepresentthecurrentvehiclepositionandtargetposition Quiteoftentherequiredsteeringangleismorethatwhatism echanicallypossible bythevehicle.Thissteeringvalueislimitedbeforesendin gtotheactuators. Bydifferentiatingpositioninformationovertime,theins tantaneousvelocityof vehicleisderivedandisusedforspeedcontrol.Since,theV isualOdometerisa relativelyslowcomponentandspeedcontrolrequirestober unatamuchhigher frequencytoachieveprecisecontrol-positioninformatio nfromtheencoderwas preferredratherthanthevisualodometer.APIDcontroller wasusedtoarriveatthe requiredvelocity.ThePIDcontrollerwasthentunedforsmo othperformance,onplain anduneventerrain.Tolimitthenoiseinthecontrolloopfro menteringthecontrollerand henceamplied,anaveraginglterwasemployedthespeedco ntrollerfeedbackpath. 6.3Results 6.3.1EvaluationofPoint-to-pointDrivingController Inordertoevaluatetheperformanceoftheturncontroller, theaccuracywithwhich thevehiclecanreachapointinspaceisevaluated.Commandp ointsonspaceat differentdistancesanddirectionsweregiventothevehicl eandthevehiclewasletto navigatetothecommandedpoint.Uponstopping,thepathoft hevehiclewastraced anddifferencebetweenthecommandedpositionandtheactua lnalpositionofthe vehicleweremeasuredandplotted. A30cmaccurateDGPSwasusedforprovidinggroundtruthesti matesforthe experiment.Whenavia-point,9minfrontofthevehicleisis suedtothecontroller, thevehicleusedthevisualodometerfeedbacktoreachthevi a-point.Thegure 6-6 ,showsaplotofactualpositionsrecordedbytheGPSandesti matedpositions trackedbythevisualodometer.Itcanbeobservedthat,thep ositionerrorinthe estimationaccumulatesovertimeandhenceresultsindiver gencefromactualpath. Theexperimentwasconductedonfairlysmoothterrainandth evelocitywassetat0.5 m/s. 87

PAGE 88

-5 0 5 0 1 2 3 4 5 6 7 8 X(m)Y(m) Position Estimate GPS Positioning Figure6-6.Pathtrackingaccuracyforastraightlinepath SincetheVisualodometerrunsatalowerfrequencythantheG PS,thereadings obtainedhadtobeinterpolatedforcomparison.Thefollowi ngstatisticsresultedfromthe comparison.Table6-1.Straightlinetrackingperformance MeanError(m)0.746Max.Error(m)1.629RMSError(m)0.456 Theerrorstatisticsfordifferenttrialvariedbydifferen tamountsindicatingrandomness inthetrackingprocess.Thisrandomnesscouldbearesultof visionasdiscussed previouslyinthedevelopmentofthevisualodometer.Alsob ettervehiclemodelingcould helpreducedivergenceovertime.6.3.2EvaluationofLoop-turnBehavior Asexplainedintheprevioussections,asetofcommandpoint srepresentinga loop-turnaregiventothevehicleandtheturningperforman ceduringeachturnwas 88

PAGE 89

measuredintermsofpathtrackingaccuracy.Thecommandedt rajectoryandtheactual trajectorywererecordedandcompared. Thevehiclewasrstcommandedtoturnintothenextrowusing aloop-turnand theturnpathtakenwasrecordedusingaDGPS.Asintheprevio usexperiment,the intendedpathandtheactualpathdiverge,resultingingrow ingpositionerror. -3 -2 -1 0 1 2 3 4 5 6 0 1 2 3 4 5 6 X(m)Y(m) Position Estimate GPS Positioning Figure6-7.PathtrackingaccuracyforaloopturnTable6-2.Loopturntrackingperformance MeanError(m)0.774Max.Error(m)1.532RMSError(m)0.419End-ptError(m)0.506 Therepeatabilityoftheaboveresultssufferbadlyduetora ndomnessofenvironmental factors.Thereisnoguaranteethatthesamefeatureswillbe selectedineachtrial. Noiseintheodometerreadingsprimarilyinducedduetothev ibrationofcameramount, amountstorandomnessinbehavior.Anothersystemicerrort hatisevidentingure 6-7 isthattheGPSusedwasmountedonthevehicle'sfront,where astheodometertracked 89

PAGE 90

thevehicle'scenterpoint.Thedistancebetweenthetwoisa tleast0.5m,andthishas amountedtoaconstantoffsetintheturn. Anothermajorfactorthataffectsvisualodometryisthefac tthatthismethoduses monocularvision.ThefeaturescollectedfromtheROI,area ssumedtolieonaat groundplane.Inreality,theterrainisunevenandvegetati onattheend-of-rowcould contributetoestimationerror. However,evenwithpathtrackingerror,thisapproachisabl etomakeanend-or-row turninanunsupervisedsettingwhichisanimprovementover thepreviousopen-loop techniqueemployedin[ 46 ].Withend-pointerrorof0.5m,themethodwarrants placementofthevehicleinthenextrow. Modelingerrors .Thekalmanlterdevelopedinthepreviouschapterisbased ona simplemodel.Modelsforfourwheeled,car-likevehiclesha vebeenwidelystudiedand usedintheresearchcommunity[ 41 ].Also,thevehiclesteeringisnotinstantaneousin nature;thevehicletakesacycloidalpathwhenturningrath erthantheassumedcircular path.Thesemodelingimprovementscouldhelpthekalmanlt erestimatethestateeven moreaccurately.Infuture,theaccuracyofthepresentsyst emcouldbeimprovedby takingtheabovefactorsintoconsideration. 90

PAGE 91

CHAPTER7 CONCLUSION Agenericsoftwarecontrolarchitectureandanautonomousg uidancesystem basedonthesoftwareframework,weredevelopedduringthec ourseofthisresearch. Agriculturalguidancesystemshavebeendevelopedbyresea rchersaroundtheworld overthedecades.Forthesesystemstobescalableandinteroperablewithother systems,acommonsoftwareframeworkisnecessary.The DashBoard frameworkisa rststeptowardsuchorganizationofsoftware.Aseffectiv enessmetricsforthesoftware systemareimpossibletoarriveat,inthecurrentstateofso ftwaretechnology,bymeans ofrsthandexperienceindevelopingtheguidancesystemba sedontheframework provedeasy.Thecorecomponentsandusabilityoftheprogra mminginterfaceswillbe puttotestbyfuturedeveloperstryingtoscalethesystem. Theodometrytechniquedevelopedisbasedontheworkdoneby [ 59 ]forindoor robotguidance.Whenusingvisualodometryinoutdoorenvir onments,vibrationplayed adisruptingroleinthetracking.Bytryingtoselectivelyi ncludethedatafromalternative sensors,theeffectofvibrationswerereduced.Thisadapta tionwasthensuccessfully usedfortrackingthevehiclewhileguidingthroughend-ofrowturns. Thepreviousguidancesystemsufferedfromuncertaintiesi nthenaturalenvironment. Developmentofintelligentguidancealgorithms,wereimpe dedduetolimitedtracking abilityinthevehicle.Also,expansionofcapabilitiesmea ntsignicantreworkofsoftware inthesystem.Theverycomplexityoftheinitialarchitectu re,presentedahurdlefor furtherdevelopmentandmaintenanceofthesystem.Aneffor ttoaddressboththese issueshasbeenmadethroughthisresearch. The DashBoard systemcanbeexpectedtoplayamajorroleinscalabilityoft his system.Expansionofthesystem'scapabilitieshasbeenorg anizedbymeansofdevices andcontrollers.Also,astheexistingsoftwarehasbeenorg anizedthesameway, 91

PAGE 92

maintenancebecomeseasier.Asaresult,thefoundationfor arobustguidancesystem hasbeenarrivedat. Anodometercomponentthatpresentsopeninterfacesfordev elopmentofintelligent algorithmshasbeenmadeavailableforfuturedevelopment. Thisnewtrackingability mightprovecrucialforimprovingrobustnessandefciency ofthesystem.Withno specialsensorsneeded,visualodometryisalsoacost-effe ctivetechnique. 7.1Limitations 7.1.1SoftwareArchitecture Customframeworkslikethedashboardandtheonedevelopedf oragricultural navigationin[ 17 ].couldbeusefulforsimplestandalonesystems.Butwhenth ese systemsscale,orbecomeapartofabiggercontrolsystem,pr opertieslikedistribution, protectionandCBSEbecomeimportant.Largesystemstendto behighlydistributed andbedevelopedandmaintainedbyagroupofpeople. Currentlythedashboardsystemdoesnothaveadistributede xtension.Thus, anyinterfacetoanothercomputershouldbeprogrammedbyth euser.Eveninthe navigationcontrolsystemdeveloped,thesensoryprocesso r(PentiumIII/Windows XP)communicateswiththevehiclecontroller(PentiumII/L inux)viaserialinterface. Themainsensoryprocessorseesthevehiclecontrollerasa' device'andthedevice classincludescodeforserialcommunication.Asimplerequ est-replyprotocolhasbeen developedforenablingcommunicationbetweenthesystems. Scalingproblemsmight ariseintwoscenarios—additionofcomponents,additionof nodes. Thedrivingcontrollercontrolsthevehicleactuatorsandr eadstheencoder.Ifwe supposearequirementthatrequiresanadditionalparamete rtobecommunicated betweenthetwonodesarises,thecommunicationprotocolha stobeextendedorat least,programmingeffortinboththesender'ssideandther eceiver'ssidehastobeput in.Whenanewnode,forinstance,ahand-heldmonitoringnod ehastobeconnectedto 92

PAGE 93

thesystem,againthecommunicationbetweenthenewnodeand theoldnodeshasto beprogrammed.Thus,networkingmightbecomeabottleneckf orscalability. Giventhatthedashboardframeworkisobject-oriented,the existingframework couldbeextendedtosupportdistributionofcomponentcont rollers.Onemajorstep towarddistributionwillbetogureatransparentandefci entmethodtotransfer datafromfromonecomponenttoanother.Bybeingtransparen t,theindividual componentsofferauniformprogramminginterface,indepen dentofwhichnodethey areon.Currently,communicationbetweencomponentsisena bledbyexposingpointers toasinglememorylocation,wheredatafromsensorsisconti nuouslylledin.This shared-memoryapproachofferssimplicityofprogramminga ndrun-timeefciency,but doesnotextendtoadistributedenvironment.Whencomponen tsaredistributedona networktheyarenomorepartofthesameaddressspace.7.1.2Odometry TheKalmanlteredvisualodometrytechniquedevelopedhas beenshownto beeffectivefordistanceslessthan10m,suchastheend-ofrowturning.Fromthe resultsintable 5-5 ,itcanbeseenthatthetechniquecanproduceerrorsupto20% onfeaturelesssurfaces.Also,thesametechniquecannotbe expectedtomeasure accuratelyoverlongerdistances,duetotheaccumulativen atureoferror. 7.2FutureDirections Pursuingthevisionofdevelopingarobustguidanceplatfor mforcitrusgroves, furtherresearchonthesubjectcantakemultipledirection s.Someofthemarelisted below: Distributedsoftwareframework AdvancedNavigation AdvancedMotioncontrol Theutilityofdistributionhasbeendiscussedenoughinthe previoussections. Distributionwillallowtheguidanceplatformtointer-ope ratewithimplementsandother 93

PAGE 94

intelligentcomponentsinthesystem.Variousdistributed frameworksforroboticsand automationareavailableasoftoday.Eachframeworkcanbeu niqueintermsofcore architecture,usability,portabilityandefciency.Thef rameworkthatismostsuitedfor theapplicationhastobeselectedafterstudyingtheadvant agesanddisadvantagesof individualframeworks.Systemdevelopersalsoresorttode velopingcustomframeworks, duetoreasonsofusabilityandreliability. Beingabletoplanpathsaroundobstaclesandforsituations otherthanjustthe alleywayandend-of-row,mighthelpextendtheguidancepla tformforawidevarietyof applications.Suchrobustnavigationmethodscanalsoredu cetheamountofhuman interventionneededinguidance. ThecurrentmotioncontrollerisasimplePIDcontrollertha tdoesn'tconsiderloop delayandsaturationinthecontrolloop.Asaresult,theact ualpathfolloweddeviates fromthegeneratedpath.Thiscanbeimprovedbyusingmorefa ithfulvehiclemodels, thatincludecharacteristicslikedelayandsaturation.In [ 10 ],thevehicles,lateraland longitudinalslipisconsideredforthevehicle'smotionmo del.Thesefactorscouldcome intoplaywhenextendingautonomousguidancetoheavyoff-r oadequipment. 94

PAGE 95

APPENDIXA USINGTHEDASHBOARD Thisappendixisdevotedtodocumentingthesemanticdetail softhe DashBoard frameworkdescribedinChapter 4 .Therequirements,designandapplicabilityofthe frameworkwerediscussedindetailinChapter 4 .Thissectionofappendix,canbe treatedasausermanualforconstructingcomponentsinthef ramework. The DashBoard isanabstractinterfaceseparatingthecontrollogicfromt he underlyingimplementation.Itcanbeviewedasaswitchingd evicesimilartoatelephone switchingnetworkanddoesnotmanipulatethedatathatitha ndles. Whentheusingthe DashBoard ,theuserhastodenethefunctionsanddata membersofacomponent(oradevice)andspecifywhatresourc esareaccessiblefor othercomponents.The DashBoard willexportallresources(dataandcontrolfunctions) thataremarkedexportable,andwillmakethemaccessibleto othercomponentsinthe system.Theuser'sroleislimitedtocreatingadeviceandma rkingwhichresources areexportable,ratherthantheusercreatingagroupofglob allyaccessibledevicesand presentingauniqueinterfaceforeachdeviceforthecontro llogictouse.Thiscreates twodistinctproblems,1.Sincedevicesarefreetohavetheirownuniqueinterface, thedevelopment ofcontrolfunctionsusingthesedeviceswillneedknowledg eofallthedevice interfaces. 2.Sinceeachdeviceisstaticallyconnectedtothecontroll ogic,simpletaskslike enabling/disablingaparticulardeviceatruntime,orswit chingonedevicefor anotherrequireconsiderablyeffort. The DashBoard frameworkhandlestherstissuebyabstractingallthedevi ces availableinthesystemandexposinganuniforminterfacefo ralldevices.i.e.,thedevice programmersareallowedtoexportdataandservicesfromade viceandthe DashBoard takescareofindexingtheseresourceswithuniqueuserden edIDs.Thusaccessing adeviceisamatterofreadingorwritingtoaresourceID,and alltheIOoperations couldbesimpliedtojustareadorwriteinthecontrollogic .Theuserdescribingthe 95

PAGE 96

controllogichastoeitherreadorwritearesourceusingits ID,insteadofhavingto haveuniqueaccessfunctionsforeachtypeofdevice.The DashBoard takescareof routingandtranslatingtheread/writecallstotheappropr iatedevices.Thecontrollogic simplyaccessesresourcesandallotherIO-speciccodelik ereadinghardwareports, waiting,synchronizationetc.,areincludedinthedevicec lass.Thisisolationbetween thecontrollogicandtheunderlyingimplementationmakest hecontrollogicportableand maintainable. Withthe DashBoard controllogicisnotstaticallyconnectedtoresourcesinde vices. Everyrequestfromthecontrollogicisassociatedwithares ource-IDandisresolved bythe DashBoard duringruntime.Thus,itispossibletodynamicallyenable/ disable andevenswitchresourcesbyre-associatingthedevicesund erthe DashBoard ,withno changestothecontrollogic. Insteadofallowingtheusertoplacecodeinanyfashion,the frameworkregulates theplacementofdevicespeciccodeandcontrollogic.Theu serisrstexpectedto programallthedevicesandlinkthemtothedashBoardobject .Theexportedresources fromthesedeviceswillbeaccessibleforreading/writingt hroughthedashboard'sget/set interface.Thecontrollogiccodehastobeplacedinsidespe cializedobjectscalled controllersandhastobelinkedtothedashboard.Thecontro llerobjectscreatedthis wayalsogaintheabilitytoaccessallresourcesunderthefr ameworkusingthesame get/setinterface.Theinteractionbetweenauserwhointen dstocreateamulti-controller softwarecontrolsystem,andtheframeworkisreducedtothe followingthreesituations. AddI/Odevices Addcontrollers Accessresources Usageinformationoneachoftheabovefunctionsaredetaile dinthesectionsbelow. 96

PAGE 97

AddinganI/Odevice Procedure Thissectionassumesthattheuseralreadyhasthesoftwared riverrequiredfor communicatingwiththehardwaredevice.Forinstance,saya GPShastobeintegrated intothesystem.Theuserisresponsibleforprogrammingthe communicationprimitives forthetheGPS.Forthepurposeofthisillustration,letusa ssumethatacertainGPS offersthefollowingthreereadings—latitude,longitude, velocityandoffersaservicefor settingthedevicecongurationags.Letthefunctionsbew rappedintoasimpleC++ object CGPS thatmodelstheactualGPSdevice. GPSClassDeclaration : classCGPS{public://GPSservices doublelat;doublelon;doublevelocity;intsetConfig(longflags); private://DeviceInternals doublegetLatitude();doublegetLongitude(); };Themembervariables lat,lon alwayscontainthelatestGPSreadingsandthe setConfig() functionallowstheusertocongurethehardwaredevice,by sendingin settingbits.Thesoftwarereferstodatamembersofadevice asParametersandthe servicesasControlfunctions. Tointegratetheaboveclassintothe DashBoard systemthefollowingstepshaveto befollowed.Step1 Thedeviceclass( CGPS )hastoinheritthe CDbfDevicePlugin baseclass. Step2 Eachinputfunctionthatperformsacertaincontrolactiono nthedevicemustbe declaredusingthe DECLARE_CONTROL_FN declarationmacroinsidetheclass'scope. 97

PAGE 98

Step3 Intheclass'constructor,themember-variablesthatholdt hereadingshavetobe registeredforexportusinganyofthefollowingfunctionspassiveParamExport() or activeParamExport() .Toassociateamemberfunctionasanupdaterfunction foracertainactiveparameter,theupdaterfunctionmustbe apublicmemberofthe sameclass,andhastobedeclaredusingthe DECLARE_UPDATER_FN macro. Step4 The CDbfDevicePlugin denesvirtualfunctionsfordeviceinitializationand de-initialization.Thesub-classcanoptionallyimplemen tthesefunctions.These functionswillbecalledwhentheuserexplicitlyrequestst hedashboardtoattachor detachthedevice.So,itisdesirabletoaddonetimedevicei nitializationoperations likeportsettings,open/closecommportsetc.Placingthis codeintheconstructor means,theabilitytoinitializesystemhardwareonrequest islost,sincealldevice constructorswillbecalledirrespectiveofwhetherthedev iceisattachedornot. Step5 Inanalplugging-instep,theclasshastobeinstantiatedi ntothe DashBoard systeminthe DbfPluginHolder.cpp le,usingthe PLUG_IN macro.Thismacro containscodeforcreationofthedeviceclassandexportand listingofmember dataandfunctionsintheDashBoard. Syntax1. DECLARE_CONTROL_FN(class,function,returntype,argtyp e) :Declarea controlfunction. classNameoftheclassinwhichthefunctionisdeclaredin. functionNameofthecontrolfunctionthatsisbeingdeclare d. returntypeControlfunctionreturntype. argtypeControlfunctionargumenttype[Thenumberofargum entspassabletoa controlfunctionislimitedtoone]. 2. voidpassiveParamExport(DbfParamIDID,stringname,stri ngdesc, void*ptr):Allowsexportofpassivememberdata.Passivemembersarem embervariables thatdonotrequireexplicitupdatecallsfromthe DashBoard 98

PAGE 99

IDAuniqueparameterIDusingwhich,clientscanaddressthi sparameter. [ParameterIDshavetobecreatedin DBFParams.h ,foreachnewparameter exported] nameAtextnamerepresentingtheparameter,fordisplaypur poses descAtextdescriptionoftheparameterlikepurpose,units etc.,fordisplay purposes ptrPointertothememberdatathatisexported 3. voidactiveParamExport(DbfParamIDID,stringname,strin gdesc, void*ptr,updaterfn):Allowsexportofactivememberdata.Activemembersaremem bervariablesthat requireupdatecallsfromthe DashBoard .Theupdatecallistriggeredbeforeeach timetheparameterisaccessed. IDAuniqueparameterIDusingwhich,clientscanaddressthi sparameter. [ParameterIDshavetobecreatedin DBFParams.h ,foreachnewparameter exported] nameAtextnamerepresentingtheparameter,fordisplaypur poses descAtextdescriptionoftheparameterlikepurpose,units etc.,fordisplay purposes ptrPointertothememberdatathatisexported fnAnupdaterfunctionthathastobecalledbeforereadingth eparameter 4. DECLARE_UPDATER_FN(class,function,returntype,argtyp e) :Declarea controlfunction. classNameoftheclassinwhichthefunctionisdeclaredin. functionNameofthecontrolfunctionthatsisbeingdeclare d. returntypeControlfunctionreturntype. argtypeControlfunctionargumenttype[Thenumberofargum entspassabletoa controlfunctionislimitedtoone]. Example The CGPS classdeclaredasasimpleclasscontainingdeviceIOfuncti onsinthe previouscodesnippet,tobeintegratedintotheDashBoards ystemhastohavefollowing codechanges.Letsassumethatthemembers lat and lon areupdatedbycallingthe appropriateupdatefunctionsandthemember velocity isbeingperiodicallyupdatedby 99

PAGE 100

aninterruptmechanismwithintheobjectCGPS.Thusin DashBoard terms,latandlon are'active'parametersandvelocityis'passive'.GPSIODeviceclassdeclaration : \\DeclarationofCGPSclass#include"DbfDevicePlugin.h"classCGPS::publicCDbfDevicePlugin{public://GPSservices doublelat;doublelon;doublevelocity;DECLARE_CONTROL_FN(CGPS,setConfig,int,long);DECLARE_UPDATER_FN(CGPS,getLatitude,double,dbfVoidP tr); DECLARE_UPDATER_FN(CGPS,getLongitude,double,dbfVoid Ptr); //Updaterfunctionscannottake'void'foranargument private://DeviceInternals};Constructordenition: CGPS::CGPS(void){ ..//ExportparametersactiveParamExport(DBFPID_GPS_LATITUDE,"Latitude","LatitudefromGPS(deg)",&lat,getLatitude);activeParamExport(DBFPID_GPS_LONGITUDE,"Longitude","LongitudefromGPS(deg)",&lon,getLongitude);passiveParamExport(DBFPID_GPS_VELOCITY,"Velocity","GroundvelocityfromGPS(m/s)",&velocity);//ExportcontrolcontrolExport(DBFCID_GPS_CONFIG,"Configbits","SetGPSdeviceconfiguration",setConfig); 100

PAGE 101

//Setdevicenameanddescriptionstrings//Thiswillbeusedfordisplaypurposesm_DeviceName="SampleGPS";m_DeviceDescription="Measuresthecurrentpositionin\ Lat(deg)andLon(deg)"; .. }dbfErrCGPS::initialize(void){ //OpenGPSport..//Initialsettings.. }dbfErrCGPS::deinitialize(void){ //CloseGPSport..//Resethardwaretooriginalstate.. } Theuseralsohastoprovidedenitionsforalltheupdateran dcontrollerfunctions declaredintheclass.Also,theuserhastomakesurethepara meterandcontrol IDs,usedintheexportfunctionsaredenedinthe "DBFParams.h" header,underthe appropriateenumsforparametersandcontrolfunctions.Fi nally,toattachthisclassto theDashboardsystem,theclasshastobeinstantiatedusing the PLUG_IN() macrointhe DbfPluginHolder.cpp placeholderle.FortheGPS,adding PLUG_IN(CGPS); intheappropriatesectionforIOdevicesisenoughforattac hingtothesystem.Upon 101

PAGE 102

invokingtheDashBoard,allclassesthusattachedtothedas hboardwillbeinstantiated andthentheappropriatedevicescouldbeselectivelyiniti alizeduponrequest. AddingaController Inthe DashBoard system,acontrollerisasoftwarecomponent,thathasthe followingproperties. Loopsinnitelyatasetfrequency AccessesIOdevicesperiodically Canexportdataandcontrolfunctionsthemselves Procedure Anycontrollerclassmustbeasub-classof CDbfControllerPlugin class.Usingthis class,theuserhastowritethemaincontrolloopcodeundera specicfunctionandthe frameworkmakessurethatthecodeiscalledrepeatedlywith adesiredfrequency,for thecontroller'slifetime.Thebase-classdenesthefollo wingsetofpure-virtualfunctions whicheverycontrollerisrequiredtoimplement.1. dbfErrcontrollerMain(dbfVoidPtr) 2. dbfErrcontrollerInit(dbfVoidPtr) 3. dbfErrcontrollerDeInit(dbfVoidPtr) The controllerMain functionwillbecalledrepeatedlybytheDashBoardsystem, thustheuserhastoimplementthemaincontrollerloopunder thisfunction.The controllerInit and controllerDeInit functionsarecalledonlyonceduringthe startandendofthecontroller'slife-time.Thesefunction scouldbeimplementedfor settingupthecontroller,initializationofvariablesetc Inthecontroller'smemberfunctions,allparametersandco ntrolfunctionsexported IOdevicespluggedintothesystem,areaccessible.Thecont rollerprogrammerhasto usesimplefunctioncallstoaccesstheexportsbytheirresp ectiveIDs.Thesemanticsof theresourceaccesscallswithbediscussedinthenextsecti on. 102

PAGE 103

Sincecontrollersmightalsohavetobeabletoexportintern almeasurementsand supportcontrolfunctions,the CDbfControllerPlugin classextendsallfunctionality fromthe CDbfDevicePlugin classdiscussedintheprevioussection.Thisallowsthe controllertobetreatedlikeanIOdeviceaswell.Thusallth edataexportandcontrol exportsemanticsdescribedaboveisapplicabletothecontr ollerstoo.i.e.,controllers canproducereadingsandbecontrolledbyacontrolleratahi gher-level.Thisproperty warrantstheabilitytobuildmulti-levelcontrollersusin gthe DashBoard Example ConsiderasimpleexampleofaPIDvelocitycontroller,usin gvelocityreadingsfrom theGPSclasscreatedaboveasainputandcontrollingtheveh icle'sspeed. ControllerDeclaration: #include"DbfControllerPlugin.h"classCruiseController:publicCDbfControllerPlugin{public: CruiseController();~CruiseController();DECLARE_CONTROL_FN(CruiseController,setCruiseSpeed, dbfErr,dbfVoidPtr); dbfErrcontrollerMain(dbfVoidPtr);dbfErrcontrollerInit(dbfVoidPtr);dbfErrcontrollerDeInit(dbfVoidPtr); private: doubleKp;doubleKi;doubleKd;CPIDControllerpid; };ControllerDenition: CruiseController::CruiseController(void){ 103

PAGE 104

controlExport(DBFCID_CRUISE_SPEED,"CruiseSpeed", "SetCruisecontrollerset-point",setCruiseSpeed); }CruiseController::~CruiseController(void){}dbfErrCruiseController::setCruiseSpeed(dbfVoidPtrsp eed) { pid.setPoint(double(*speed)); }dbfErrCruiseController::controllerInit(dbfVoidPtrp){ pid.setConsts(Kp,Ki,Kd);pid.setPoint(0);returnDBFERR_SUCCESS; }dbfErrCruiseController::controllerDeInit(dbfVoidPtr p) { returnDBFERR_SUCCESS; }dbfErrCruiseController::controllerMain(dbfVoidPtrp){ doublemeasurement;doublecommand;dbfErrerr;//readvelocityfromGPSmeasurement=getDouble(DBFPID_GPS_VELOCITY);command=pid.getCommand(measurement);//Assumeahypotheticalvehiclespeedcontrollerwith//IDDBFCID_VEHICLE_SPEEDerr=set(DBFCID_VEHICLE_SPEED,command);returnerr; 104

PAGE 105

} Asanalstepthecruisecontrollerclasshastobeinstantia tedusingthemacro PLUG_IN_CLR withinthe DbfPluginHolder.cpp le.Thesyntaxforthemacroisasbelow: PLUG_IN_CLR(CNAME,CLRID,FREQ) -Pluginacontrollerobjectintothe DashBoard system. CNAME :Nameofthecontrollerclass CLRID :AuniqueIDwhichthedashboarduserscanusetoaddressthec ontroller FREQ :Thefrequencywithwhichthecontroller's controllerMain functionwillbe called,inHz Usingtheabovesemantic,theinstructionforinstantiatio nforthe CruiseController examplewouldbe: PLUG_IN_CLR(CruiseController,DBFCLRID_CRUISECTRL,20 ) Thisstatementmakessurethatacruisecontrollerobject,d enedinthepreviouscode segments,willbeinstantiatedandwillberunatafrequency of20Hz. ResourceAccess Duringruntimeasingle CDashBoard objectmanagesallthedevices,controllersand theirassociatedresources.Userswhocreatetheobjectcan useitsservicestoaccess theunderlyingcontrollersanddevices.Thefollowingprim aryservicesareofferedbythe object Attachdetachdevicesandcontrollers GetandSetparametersandcontrolfunctions Syntax Thedetailedlistofservicesofferedbythe CDashBoard classareasbelow: 1. dbfErrgetParamPtr(DbfParamIDparamID,DbfParamPtr*par amPtr) -Getan importedparameter.Dashboarduserscancallthisfunctiontoaccessanysensore xportedparameter. Thisfunctiondoesn'tcallupdaters,asitisintendedtobep lain. paramID -TheparameterID(thekeyforsearching) paramPtr -Pointertoalocationwherepointertotherequestedresour ceisstored return-Errorsincurredwhilefetchingtherequestedparam eter 105

PAGE 106

2. dbfSInt32getInt(DbfParamIDparamID,dbfVoidPtrupdater Data=NULL) GetanimportedintegerparameterDashboarduserscancallthisfunctiontoaccessanysensore xportedparameter. Thecalliscascadedtoanupdaterfunctionifdenedbythedr iver paramID -TheparameterID(thekeyforsearching) updaterData -Ameansofpassingdatatotheupdaterfunction return-Integerparametervalue 3. dbfFloatgetFloat(DbfParamIDparamID,dbfVoidPtrupdate rData=NULL) GetanimportedoatingpointparameterDashboarduserscancallthisfunctiontoaccessanysensore xportedparameter. Thecalliscascadedtoanupdaterfunctionifdenedbythedr iver paramID -TheparameterID(thekeyforsearching) updaterData -Ameansofpassingdatatotheupdaterfunction return-Floatingpointparametervalue 4. dbfDoublegetDouble(DbfParamIDparamID,dbfVoidPtrupda terData=NULL) GetanimportedoatingpointparameterDashboarduserscancallthisfunctiontoaccessanysensore xportedparameter. Thecalliscascadedtoanupdaterfunctionifdenedbythedr iver paramID -TheparameterID(thekeyforsearching) updaterData -Ameansofpassingdatatotheupdaterfunction return-Floatingpointparametervalue 5. dbfErrupdateParam(DbfParamIDparamID,dbfVoidPtrupdat erData=NULL) Updateaparticularparameter(doesn'treturnparameter)Updatesalistedparameterbycallingthedened'updater'f n. paramID -TheparameterID(thekeyforsearching) updaterData -Ameansofpassingdatatotheupdaterfunction return-Errorsincurredwhilefetchingtherequestedparam eter 6. dbfErrupdateParam(DbfParamIDparamID) -[Overloaded]Updateaparticular parameter(doesn'treturnparameter)Updatesalistedparameterbyca llingthedened 'updater'fn.paramID -TheparameterID(thekeyforsearching) return-Errorsincurredwhilefetchingtherequestedparam eter 7. dbfErrset(DbfControlIDcontrolID,dbfVoidPtrcontrolAr g=NULL) ExecuteacontrolfunctionDashboarduserscancallthisfunctiontoaccessanyactuato r-exportedcontrol function.controlID -ThecontrolID(thekeyforsearching) 106

PAGE 107

controlArg -Pointertoastructurethatcanbepassedtothecontrolfunc tion(the dbfVoidPtrargumentisameansofpassingparameterstoavar ietyoffunctions withauniedinterface)return-Errorsincurredwhileexecutingarequestedcontro lfunction 8. dbfErrset(DbfControlIDcontrolID,dbfSInt32integerArg ) -Executea controlfunction[integerargoverload]Dashboarduserscancallthisfunctiontoaccessanyactuato r-exportedcontrol function.controlID -ThecontrolID(thekeyforsearching) integerArg -Pointertoastructurethatcanbepassedtothecontrolfunc tion [makesuserprogramslookbetter]return-Errorsincurredwhileexecutingarequestedcontro lfunction 9. dbfErrset(DbfControlIDcontrolID,dbfFloatfloatArg) -Executeacontrol function[oatargoverload]Dashboarduserscancallthisfunctiontoaccessanyactuato r-exportedcontrol function.controlID -ThecontrolID(thekeyforsearching) floatArg -Pointertoastructurethatcanbepassedtothecontrolfunc tion[makes userprogramslookbetter]return-Errorsincurredwhileexecutingarequestedcontro lfunction 10. dbfErrstart(DbfControllerIDcontrollerID) -startanautomaticcontroller Dashboarduserscancallthisfunctiontostartanypluggedincontrolfunction. controllerID -ThecontrollerID(thekeyforsearching) return-Errorsincurredwhileexecutingarequestedcontro lfunction 11. dbfErrstop(DbfControllerIDcontrollerID) -stoparunningautomatic controllerDashboarduserscancallthisfunctiontostopanyplugged-i ncontrolfunction currentlyrunningcontrollerID -ThecontrollerID(thekeyforsearching) return-Errorsincurredwhileexecutingarequestedcontro lfunction 12. dbfUInt32getFreq(DbfControllerIDcontrollerID) -ndthefrequencywith whichacontrollerisrunningReturnsthefrequencywithwhichacontrollerissettorun.controllerID -ThecontrollerID(thekeyforsearching) return-Frequency 13. dbfBoolisRunning(DbfControllerIDcontrollerID) -ndcontrollerstatus Checkswhetheracontrolleriscurrentlyrunningornot.controllerID -ThecontrollerID(thekeyforsearching) return-trueifrunning,falseifstopped 107

PAGE 108

14. conststd::vector*getInputCapabili ties(void) 15. conststd::vector*getOutputCapabi lities(void) 16. conststd::vector*getControlCapa bilities(void) -Checktheresourcesavailablefromthedashboard.Eachfun ctionabovereturns anarrayof'capability'structures.Thisfunctionishelpf ulfordashboard'sclientsfor listingdetailsofresourcesavailableinthesystem.Notet hatdeviceresourcesare listednomattertheyareattachedornot.CapabilityDenition: //MeasurementCapabilitytypedefstructIcapabilities_struct{ DbfParamIDparamID;stringparamName;stringparamDesc;stringdeviceName;dbfDeviceInstIDdeviceInstID; }InputCapability;//ControlcapabilitytypedefstructOcapabilities_struct{ DbfControlIDcontrolID;stringcontrolName;stringcontrolDesc;stringdeviceName;dbfDeviceInstIDdeviceInstID; }OutputCapability;//ControllercapabilitytypedefstructCcapabilities_struct{ DbfControllerIDcontrollerID;stringdeviceName;dbfDeviceInstIDdeviceInstID; }ControlCapability;return-Listofallcapabilitiesoftherequestedcategory 17. dbfErrattach(dbfDeviceInstIDdeviceID) -Attachadevicetothedashboard. 108

PAGE 109

Attachingadevicemeansthedevice's initialize functionwillbecalledandthe resources(parametersandcontrolfunctions)containedwi thinthedevicewillbe availableforaccessuponsuccessfulcompletionofattach.deviceID -uniqueIDofthedevicethathastobeattached. return-Errorsencounteredduringdeviceinitialization. 18. dbfErrdetach(dbfDeviceInstIDdeviceID) -Detachadevicefromthe dashboardDetachingadevicemeans,thedevice's deinitialize functionwillbecalledand theresourcesexportedfromthedevicewillbeunavailable. Example Theservicesofferedbythedashboardhavebeendesignedfor constructing applicationsontopofit.Inthisexample,avelocitycontro lapplicationisdevelopedon topofthe Dashboard .Thecomponentsofthe DashBoard — CGPS and CCruiseController havealreadybeendevelopedintheprevioussections.Thoug h,theapplicationwillnot havetoknowaboutthesecomponentsclasses,theresourceID sdeclaredinthe previousexampleswillbeusedinthemainapplication.ApplicationCode: #include"Dashboard.h"#include#includenamespacestd;intmain(void){ CDashboarddb;conststd::vector*pInputsList;conststd::vector*pOutputsList;//PopulatingthecapabilitieslistpInputsList=db.getInputCapabilities();pOutputsList=db.getOutputCapabilities(); 109

PAGE 110

inputCapsReaderinReader=pInputsList->begin();outputCapsReaderopReader=pOutputsList->begin();//Listingandattachingtheresourcesonebyone.//Thedevicescanbeattachedselectivelytoo.cout<<"ListofMeasuredparams:";for(;inReader!=pInputsList->end();inReader++) { cout<<(*inReader)->paramName<<"["<<(*inReader)->par amID<<"],"; db.attach((*inReader)->deviceInstID); } cout<<"ListofControls:";for(;opReader!=pOutputsList->end();opReader++) { cout<<(*opReader)->controlName<<"["<<(*opReader)->c ontrolID<<"],"; db.attach((*opReader)->deviceInstID); } //StartthethePIDspeedcontrollerdb.start(DBFCLRID_CRUISECTRL);cout<<"InitialVelocity="<
University of Florida Home Page