Collaborative Coordination and Control for an Implemented Heterogeneous Swarm of UAVs and UGVs

MISSING IMAGE

Material Information

Title:
Collaborative Coordination and Control for an Implemented Heterogeneous Swarm of UAVs and UGVs
Physical Description:
1 online resource (6 p.)
Language:
english
Creator:
Weaver, Joshua N
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Electrical and Computer Engineering
Committee Chair:
ARROYO,AMAURI ANTONIO
Committee Co-Chair:
LAM,HERMAN
Committee Members:
LATCHMAN,HANIPH A
CRANE,CARL D,III
SCHWARTZ,ERIC M

Subjects

Subjects / Keywords:
autonomous -- collaborative -- heterogeneous -- robotics -- ros -- swarms
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre:
Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Over the last few years, cooperative autonomous systems have become a popular solution for accomplishing tasks that are otherwise performed by human operators. Several strides have been made with homogeneous systems of vehicles in areas of localization, formation behaviors, path planning, task allocation, and vehicle controls. This literature will describe work performed in the development of a heterogeneous system and architecture of platforms consisting of unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) that cooperatively work together to accomplish tasks such as search and rescue in an outdoor environment. The Robot Operating System (ROS) is used between and within each vehicle to handle cooperative high level planning and task allocation, as well as control of each vehicles individual capabilities specializing in identification, classification, and tracking. A description of the system of UAVs, UGVs, and control station, as well as developed vehicle capabilities is given. These different capabilities include locating and characterizing obstacles using LIDAR, or using vision to locate objects through standard color recognition techniques or specialized fiducial tags. Detailed characterizations of a specialized communication protocol, with emphasis on error control, data size management, optimization, as well as the method used for communicating to various vehicles in the field will be provided. The context and use of this protocol is built within a special package that works directly with the ROS communication system, converting data into smaller, more manageable components within a multi-master system. Mission control between vehicles includes heterogeneous path planning, task allocation, and vehicle control through a custom swarm architecture. This architecture is presented with details on procedure for use, while also providing information on the modular capabilities in generating new missions and vehicle control plugins. Also illustrated is a specialized plugin used within mission planning, the goal planner, which is capable of defining various lawnmower and spiral search paths for many vehicles in a multi-point convex polygon given the varying capabilities of the vehicle. Finally, real environment testing results are given that will characterize the benefits of the heterogeneous architecture and simultaneously show the functionality and application of the system.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Joshua N Weaver.
Thesis:
Thesis (Ph.D.)--University of Florida, 2014.
Local:
Adviser: ARROYO,AMAURI ANTONIO.
Local:
Co-adviser: LAM,HERMAN.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2014
System ID:
UFE0046677:00001


This item is only available as the following downloads:


Full Text

PAGE 1

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems October 7-12, 2012. Vilamoura, Algarve, Portugal 978-1-4673-1736-8/12/S31.00 2012 Crown 870

PAGE 2

871

PAGE 3

872

PAGE 4

873

PAGE 5

874

PAGE 6

875



PAGE 1

COLLABORATIVECOORDINATIONANDCONTROLFORANIMPLEMENTEDHETEROGENEOUSSWARMOFUAVSANDUGVSByJOSHUAN.WEAVERADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2014

PAGE 2

c2014JoshuaN.Weaver 2

PAGE 3

Tomyawe-inspiringwifeToniandmyamazingchildrenZoeandKaiden 3

PAGE 4

ACKNOWLEDGMENTS First,Iwouldliketostartbythankingmyextraordinarywifeforhercontinuedlove,patience,andsupportduringthepursuitofmydegrees.Shenotonlybelievedinmypotential,butmotivatedandcheeredmeonduringthetimesthatIdidnottrustinmyself.Withoutherbymyside,Iwouldhavebeenunabletoaccomplishedmygoals.NextIwanttothankmywonderfulchildrenforprovidinghopeandreasontocontinuemywork.IwouldalsoliketothankmyadvisorsDr.A.AntonioArroyoandDr.EricM.Schwartzforguidance,support,andunderstandingovertheyears.ItwaswiththeirassistancethatprovidedmetheopportunitytocontinuemyeducationintheenvironmentofroboticsthatIenjoy.Theyprovidedmewiththefreedomtofollowmygoalsinresearch,whilealsoinstructingmeinmymanyotherinterestsofeducation.Further,IwouldliketothanktherestofthemembersoftheMachineIntelligenceLab,withaspecialthankstomyteam,CongreGators.YourfriendshipandcollaborationduringmytimeattheUniversityofFloridamadeitmuchmoreenjoyable.IamalsothankfulforthesupportIreceivedfromtheDepartmentofDefenseS.M.A.R.T.FellowshipProgram.ItwasthroughthisprogramthatIreceivedthefundingformygraduatedegreeandtheopportunitytofurthermyeducation. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 8 LISTOFFIGURES ..................................... 9 LISTOFABBREVIATIONS ................................ 13 CHAPTER ABSTRACT ......................................... 15 1INTRODUCTION ................................... 17 1.1Motivation .................................... 17 1.2Organization .................................. 18 2BACKGROUND ................................... 20 2.1FullyDevelopedCooperativeSystems .................... 20 2.1.1UniversityofPennsylvania ....................... 20 2.1.2EidgenossischeTechnischeHochschuleZurich ........... 23 2.1.3StanfordUniversity ........................... 24 2.1.4SmallerCollaborativeProjects ..................... 24 2.2SystemsDevelopedforGroupsofUAVsorUGVs .............. 26 2.3Conclusions ................................... 28 3EXPLORATORYHOMOGENEOUSSWARMDEVELOPMENT ......... 29 3.1HomogeneousSwarmofUGVsinGoogleCloud .............. 29 3.1.1DevelopmentofInheritedSystem ................... 29 3.1.2FurtherAdvancementsforHomogeneousSwarmofUGVs ..... 32 3.2AutonomousUAVinROSEnvironment .................... 33 3.2.1BackgroundinROSEnvironment ................... 33 3.2.2UAVSystem ............................... 35 3.2.3ROSUAVControlSystem ....................... 36 3.2.4ROSVisionControlSystem ...................... 37 3.2.5AndroidApplicationforMissionControl ................ 37 3.2.6Testing .................................. 39 3.3UAVPerformingAutonomousLandingonUSV ............... 39 3.3.1USVandUAVsystem ......................... 39 3.3.2ROSUSVandUAVControlSystem ................. 40 3.3.3VisualProcessingandControl ..................... 43 3.3.4Testing .................................. 46 5

PAGE 6

4HETEROGENEOUSSWARMARCHITECTURE ................. 47 4.1CooperativeSystemOverview ........................ 47 4.1.1UnmannedGroundVehicleDesign .................. 47 4.1.2UnmannedAerialVehicleDesign ................... 49 4.1.3BaseStationLayout .......................... 51 4.2GraphicalUserInterfaces ........................... 51 4.2.1PrimaryROSQtGraphicalUserInterface .............. 52 4.2.1.1WaypointPlugin ....................... 53 4.2.1.2CommandPlugin ...................... 54 4.2.1.3StatusPlugin ......................... 54 4.2.2SatelliteAndroidTabletInterface ................... 55 4.3XBeeBridgeCommunications ........................ 56 4.3.1NodeInitialization ............................ 59 4.3.2ManagementofROSMessages ................... 64 4.3.3ManagementofROSServices .................... 68 4.4SwarmCore .................................. 73 4.4.1RollCall ................................. 74 4.4.2MissionControl ............................. 74 4.4.2.1NodeInitialization ...................... 76 4.4.2.2BuildingListofKnownAgents ............... 77 4.4.2.3MissionPreparationandTransmission ........... 77 4.4.2.4MissionCommands ..................... 80 4.4.3AgentControl .............................. 80 4.4.3.1NodeInitialization ...................... 83 4.4.3.2FlowofAgentStateMachine ................ 83 4.4.4SwarmBase .............................. 91 4.4.5BaseGoalPlaner ............................ 95 4.4.5.1CongurationofGoalPlanner ............... 97 4.4.5.2CalculatingDifferentGoalPathTypes ........... 99 4.4.6BaseVehicleControl .......................... 115 4.5AgentCapabilities ............................... 115 4.5.1LIDAR .................................. 116 4.5.1.1ObstacleDetection ..................... 116 4.5.1.2ObstacleAvoidance ..................... 117 4.5.2Vision .................................. 117 4.5.2.1FiducialMarkerTracking .................. 117 4.5.2.2ObstacleDetection ..................... 118 5FIELDTESTINGRESULTS,CONCLUSIONS,ANDFUTUREWORK ..... 119 5.1TestingandResults .............................. 119 5.1.1Assumptions .............................. 119 5.1.2SystemsTesting ............................ 119 5.1.3FieldTesting ............................... 124 5.1.4InitialArchitectureTesting ....................... 125 6

PAGE 7

5.1.5CompleteSystemTest ......................... 128 5.1.6SeparateGroupTesting ........................ 131 5.2Conclusions ................................... 132 5.3FutureWork ................................... 133 APPENDIX ADYNAMICSOFAQUADROTOR .......................... 136 BCODESAMPLES .................................. 144 REFERENCES ....................................... 152 BIOGRAPHICALSKETCH ................................ 156 7

PAGE 8

LISTOFTABLES Table page 4-1RolesforUGVsPerformingMissions ........................ 49 4-2RolesforUAVsPerformingMissions ........................ 51 4-3PossibleIDsforwhowillreceivethemessage ................... 58 4-4PossibleIDsforwhomaysendthemessage ................... 58 4-5XBeemessageelds ................................ 59 4-6TaskTypesforAgentsPerformingMissions .................... 85 4-7DescriptionofstagesforalawnmowerpathfromFigure 4-47 .......... 106 8

PAGE 9

LISTOFFIGURES Figure page 2-1UAVBlimpandUGVvehiclesusedinearlierUPennresearch .......... 21 2-2CollectionofvehiclesusedinMulti-RobotteaminGRASPLab ......... 22 2-3GRASPLabUPennAscTecquadrotorswarm ................... 23 2-4ObjectAvoidanceusingUAVandUGV ....................... 27 3-1VehiclesusedindevelopingahomogeneousswarmofautonomousUGVrobots:A)LegoMinestormNXTPlatformB)XTMRailRCVehicle ............ 30 3-2XTMRailPrototypesusedinSwarmSystem:A)XTMRailPrototype1B)XTMRailPrototype2 ................................... 32 3-3TopandSideViewsoftheAscTecPelicanUsedForProject ........... 35 3-4AndroidApplicationforMissionControl ...................... 38 3-5USVCadModelandFlamewheelF450QuadrotorforRoboBoatCompetition:A)USVCadModelB)FlamewheelF450 ..................... 40 3-6ROS BridgeandAgent ControlDataFlow:A)ROS BridgecommunicationbetweenvehiclesB)DataowwithAgent Control ................ 42 3-7ProcessedImageofLandingPad .......................... 44 3-8ROS BridgeandAgent ControlDataFlow:A)FilteredandExtractedImageofLandingDockandPuckB)ProcessedImageShowingLandingDockBorderandPuck ....................................... 45 4-1XTMRailFinalDesignusedinHeterogeneousSwarmArchitecture:A)XTMRailFinalDesign(external)B)XTMRailFinalDesign(internal) ......... 48 4-2FlamewheelModelsusedinHeterogeneousSwarmArchitecture:A)F450FlamewheelQuadrotorB)F550FlamewheelHexarotor ............. 50 4-3CustomModelsusedinHeterogeneousSwarmArchitecture:A)CustomHexarotorDesignB)CustomOctorotorX-8Design ...................... 50 4-4TheCongreGatorsGUI ............................... 52 4-5ROSMessageandServiceowforwaypoint plugin ............... 53 4-6ROSMessageandServiceowforcommand plugin ............... 54 4-7ROSMessageandServiceowforstatus plugin ................. 55 4-8DataowbetweentwoXBeeBridgecongureddevices ............. 56 9

PAGE 10

4-9MessagestructureformessagesinXBeeBridge ................. 57 4-10MessagestructureforservicesinXBeeBridge .................. 57 4-11NodeInitializationforxbee bridge. ......................... 61 4-12Expandtopicparametersforxbee bridge. ..................... 62 4-13ReadloopofXBeeinATmodeforxbee bridge. .................. 63 4-14FindNewSubscriber/ServiceLoopforxbee bridge. ............... 64 4-15ROSmessagedataowbetweentwoXBeeBridgecongureddevices ..... 65 4-16Standardsubscribercallbackforxbee bridge. ................... 65 4-17Transmitdataforxbee bridge. ........................... 67 4-18Buildmessagestringforxbee bridge. ....................... 68 4-19ParseStringforxbee bridge. ............................ 69 4-20Standardpublisherforxbee bridge. ........................ 69 4-21BuildROSmessageforxbee bridge. ........................ 70 4-22ROSservicedataowbetweentwoXBeeBridgecongureddevices ...... 70 4-23Standardservicecallbackforxbee bridge. ..................... 71 4-24Standardclienttransmitforxbee bridge. ...................... 72 4-25Standardclientresponseforxbee bridge. ..................... 73 4-26ROSdataowbetweenagent role serverandagent list publisher. ....... 75 4-27ROSdataowformission control ......................... 76 4-28Nodeinitializationformission control ........................ 77 4-29Agentlistcallbackformission control ....................... 78 4-30Sendmissionhandlerformission control ..................... 79 4-31Sendcommandhandlerformission control .................... 81 4-32ROSdataowforagent control ........................... 82 4-33NodeInitializationforagent control ......................... 86 4-34Flowofthestatetimerforagent control ...................... 87 4-35Agentlistsubscriberforagent control ....................... 92 10

PAGE 11

4-36Localagentstatussubscriberforagent control .................. 92 4-37Heartbeatsubscriberforagent control ....................... 93 4-38SetMissionserviceforagent control ........................ 93 4-39ControlMissionserviceforagent control ...................... 94 4-40Groupagentstatussubscriberforagent control .................. 94 4-41Goalpathsfoundfromthebase goal planner:A)CrossinglawnmowerpathformultipleagentsB)Spiralpathforsingleagent ................. 96 4-42InitializationandPlanningforpluginlibrarygoal planner ............. 97 4-43PlanPathsfunctionforgoal planner ........................ 98 4-44Containertocollectpointsofacrossinglawnmowerpathingoal planner .... 100 4-45Methodforcalculatingacrossinglawnmowerpointingoal planner ....... 102 4-46AnalyticGeometrymodelusedforplanningpaths ................ 105 4-47Seriesofstagesthatleadtondafullsequenceforalawnmowerpath ..... 105 4-48Methodforcalculatinganup/downlawnmowerpointingoal planner ...... 107 4-49Containertocollectpointsofasplitlawnmowerpathingoal planner ...... 110 4-50Containertocollectpointsofaspiralpathingoal planner ............ 111 4-51Methodforcalculatingarstspiralpointingoal planner ............. 112 4-52Methodforcalculatingaspiralpointingoal planner ............... 113 4-53Seriesofstagesthatleadtondafullsequenceforaspiralpath ........ 115 4-54FiducialMarkerImageandVisualizationWithintheROSEnvironment:A)FiducialARMarkerB)ROSVisualizationofFiducialMarkerPositionandOrientation 117 5-1SingleVehicleLawnmowerPlanandCoverageRange:A)SingleVehicleLawnmowerPlanB)CoverageRangeofaSingleVehicleLawnmowerPlan ......... 120 5-2PlannedCrossingLawnmowerSearchPatternforThreeAgentsinaMulti-sidedPolygon ........................................ 121 5-3PlannedSplitLawnmowerSearchPatternforThreeAgentsinaSimplePolygon 122 5-4PlannedSplitSpiralSearchPatternforThreeAgentsinaSimplePolygon ... 123 5-5PlannedCrossingLawnmowerPatternforTwoUGVsandOneUAVwithintheSwarmArchitectureGUI ............................... 123 11

PAGE 12

5-6OneUAVandTwoUGVsBeingUsedinFieldTesting .............. 126 5-7MissionStartShowingROIandPlannedSearchPathsforEachAgent ..... 126 5-8MissionAfterCompletingOneLegofPlannedSearchPaths .......... 127 5-9MissionCompleteShowingAgentsatFinalGoalPointinSearchPattern .... 127 5-10InitialSetupforFieldandVehiclesDuringCompleteSystemTests:A)FieldSetupB)VehicleSetup ............................... 129 5-11InitialMissionSetupforCompleteSystemTests ................. 130 5-12MissionResultforCompleteSystemTestsDetectingAllobjects ......... 130 A-1Dynamicsystemmodelandreferenceframe. ................... 136 12

PAGE 13

LISTOFABBREVIATIONS ALVARALibraryforVirtualandAugmentedRealityAPMArduPilot-MegaARAugmentedRealityCOTSCommercialOff-the-ShelfGPSGlobalPositionSystemGRASPGeneralRobotics,Automation,Sensing,&PerceptionGUIGraphicalUserInterfaceIMUInertialMeasurementUnitLIDARLightDetectionandRangingMANETMobileAd-HocNETWorksMAVLinkMicroAirVehicleLinkMILMachineIntelligenceLabMSGMessageOpenCVOpenSourceComputerVisionPIDProportionalIntegralDerivativeRCRemoteControlROIRegionofInterestROSRobotOperatingSystemRPCRemoteProcedureCallRQTROSQtSLAMSimultaneousLocalizationAndMapping 13

PAGE 14

SRVServiceUAVUnmannedAerialVehicleUGVUnmannedGroundVehicleUSVUnmannedSurfaceVehicleUTMUniversalTransverseMercatorUUVUnmannedUnderwaterVehicle 14

PAGE 15

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyCOLLABORATIVECOORDINATIONANDCONTROLFORANIMPLEMENTEDHETEROGENEOUSSWARMOFUAVSANDUGVSByJoshuaN.WeaverMay2014Chair:A.AntonioArroyoMajor:ElectricalandComputerEngineeringOverthelastfewyears,cooperativeautonomoussystemshavebecomeapopularsolutionforaccomplishingtasksthatareotherwiseperformedbyhumanoperators.Severalstrideshavebeenmadewithhomogeneoussystemsofvehiclesinareasoflocalization,formationbehaviors,pathplanning,taskallocation,andvehiclecontrols.Thisliteraturewilldescribeworkperformedinthedevelopmentofaheterogeneoussystemandarchitectureofplatformsconsistingofunmannedgroundvehicles(UGVs)andunmannedaerialvehicles(UAVs)thatcooperativelyworktogethertoaccomplishtaskssuchassearchandrescueinanoutdoorenvironment.TheRobotOperatingSystem(ROS)isusedbetweenandwithineachvehicletohandlecooperativehighlevelplanningandtaskallocation,aswellascontrolofeachvehicle'sindividualcapabilitiesspecializinginidentication,classication,andtracking.AdescriptionofthesystemofUAVs,UGVs,andcontrolstation,aswellasdevelopedvehiclecapabilitiesisgiven.ThesedifferentcapabilitiesincludelocatingandcharacterizingobstaclesusingLIDAR,orusingvisiontolocateobjectsthroughstandardcolorrecognitiontechniquesorspecializedducialtags.Detailedcharacterizationsofaspecializedcommunicationprotocol,withemphasisonerrorcontrol,datasizemanagement,optimization,aswellasthemethodusedforcommunicatingtovariousvehiclesintheeldwillbeprovided.Thecontextanduseofthisprotocolisbuiltwithin 15

PAGE 16

aspecialpackagethatworksdirectlywiththeROScommunicationsystem,convertingdataintosmaller,moremanageablecomponentswithinamulti-mastersystem.Missioncontrolbetweenvehiclesincludesheterogeneouspathplanning,taskallocation,andvehiclecontrolthroughacustomswarmarchitecture.Thisarchitectureispresentedwithdetailsonprocedureforuse,whilealsoprovidinginformationonthemodularcapabilitiesingeneratingnewmissionsandvehiclecontrolplugins.Alsoillustratedisaspecializedpluginusedwithinmissionplanning,thegoalplanner,whichiscapableofdeningvariouslawnmowerandspiralsearchpathsformanyvehiclesinamulti-pointconvexpolygongiventhevaryingcapabilitiesofthevehicle.Finally,realenvironmenttestingresultsaregiventhatwillcharacterizethebenetsoftheheterogeneousarchitectureandsimultaneouslyshowthefunctionalityandapplicationofthesystem. 16

PAGE 17

CHAPTER1INTRODUCTIONRecently,unmannedsystemshavecometotheforefrontoftechnologybeingdevelopedformilitary,civilian,andresearchpurposes.Thisisinpartduetotheirgrowingeconomicfeasibility,easeofuse,andtheabilitytotraveltolocationshumansareunabletoreach.Thesesystemsarealsoabletosafelyandeffectivelyhandledifcultsituationswithoutputtinghumanslivesatrisk.Thisabilitytocreateasaferenvironmentindangeroussituationshascreatedapushinboththeprivateandpublicsectortomakethesetoolsmoreaccessible.Onemainfocusistoreplacemannedaerialandgroundcapabilitieswithunmannedaerial(UAV),ground(UGV),underwater(UUV),andsurface(USV).Afocushasalsobeenplacedontheincreasedautonomyofthesesystemstomakedecisionsandaccomplishtaskswithminimaluserinput,freeingupmanpowerforotheruses.Thisishighlyusefulinasystemwheremorethanasinglevehicleisusedtosearchandgatherinformation.Forinstance,inasearchandrescuescenario,wheresafelycoveringasmuchgroundaspossibleinaveryshortamountoftimeiskey,vehiclesthatcanperformtheirtaskwithouttakingupmanpowerisideal.Beyondsimplybuildingautonomousunmannedvehicles,researchgoalshavelookedintothebenetsofusinghomogeneous,i.e.similarnatureandtypevehicles,orheterogeneous,whichusesvaryingtypesofvehicles,possiblyindifferentplanesofoperation. 1.1MotivationThisresearchdescribesaswarmarchitectureusedwithinasystemofmultipleautonomousunmannedgroundandaerialvehiclesformingaheterogeneouscooperativeswarmtoaccomplishgoalsandmissionssuchassearchandrescueinanoutdoorenvironment.Thoughsomeexamplesoffullyheterogeneoussystemsexist,thisisthersttime,asfarasweknow,thatasystemonthislevelhasbeencreated.AcoalitionofUGVsandUAVshavebeencreatedtoworkindependentlyofeachother; 17

PAGE 18

witheachvehiclehavingthecapacityofbeingcompletelyautonomoususingeitherwaypointnavigationoramissionarbitratorthatexistswithintheRobotOperatingSystem(ROS)environment.Thougheachvehicleisindependenttheyhavethecapabilitytocooperativelyplantheirdesiredtaskstocompletethemission.Vehiclesareinitiallyunknowntoeachotherastheyjointheswarmarchitecture.Duringtheprocessesofconnectingtotheswarm,eachvehiclegoesthroughaprocessofrollcallhandshakingtoacknowledgethebasecontrolleroftheswarmwhileallowingallothervehiclesalreadyintheswarmtobeupdatedofit'spresence.Thearchitectureallowsadenedmissiontobeprocessedandsenttovehiclesintheswarm,whilealsoupdatingnewvehiclesastheycomeonline.Thearchitectureallowsmultiplepluginmissiontypesaswellasbehaviorsforvehiclecontrolspermissiontype.Onesuchmissionmaybedenedasasearchandrescuemissionwherevehiclessplittheeldofoperationintosectionsbasedonthenumberofvehiclesinthegroup.UAVsmaybeusedtoeithersearchtheeldaheadofUGVstomarklow-condencetargetsofinterestaswellasobstaclesthatshouldbeavoided,ortosearchwithUGVs,stayingabovethemastheytravelthroughtheenvironment.WhetherwithorfollowingbehindUAVs,UGVsaretypicallyusedtobetteridentifytargetswithahigherdegreeofcondence.Whentargetsofinterestsaredetermined,aclassicationphaseisstartedinwhichaspecializedgroupofUGVswilltraveltoobjectsofinteresttoattempttoclassifythetargetforanyhumanoperators. 1.2OrganizationThisdissertationisorganizedasfollows:InChapter 2 ,previousworkfromexternalresearchinhomogeneousandheterogeneouscooperativesystemsaswellastheirdevelopmentsintoadvancementsthathelpthesesystemsaredetailed.Cooperativesystemsthathavebeenfullydevelopedbyuniversityandsmallprojectresearchisrstdescribedfollowedbythesmalladvancementsinsystemsthatarenotfullytestedbutstillmaybeusedtoimprovecooperativesystems. 18

PAGE 19

InChapter 3 ,relevantworkcontributingtotheprimaryresearchishighlighted.AhomogeneousswarmofUGVsthatinteractthroughacloudenvironmentispresented,detailingthesystemandwhathasbeenaccomplished.AnautonomousUAVthatiscontrolledthroughtheRobotOperatingSystem(ROS)environmentandiscapableoftakingpre-plannedmissionsthroughanAndroidTabletisdescribedintermsofvehiclecontrol,sensorcontrol,andAndroidapplicationdetail.FurtherdetailsofexpandingtheautonomousUAVsysteminROStomultiplevehiclesaregiven.FinallyavisionsystemmadeforallowinganUAVtolaunchandlandfromanautonomousboatisdescribed.InChapter 4 ,theprimaryresearchoftheheterogeneouscollaborativesystemaredetailed.Informationinregardstounmannedgroundandaerialvehiclesusedinthecooperativesystemisgiven,followingabreakdownofthesysteminformationsharingprocess.Theswarmarchitectureusedtoallowthevariousvehiclestocommunicateandworktogetherisdescribed.Thehighlevelgoalplannerandlowlevelvehiclecontrollersystemsareexplainedaswellashowtheymaybemodiedforcustomdesigns.Finally,theLIDARandvisionbasedobstacledetectionsystemisdetailedaswellashowinformationissharedbetweenagents.InChapter 5 ,themethodfortestingandgeneralassumptionsareoutlined.Resultsfromthevariousstagesoftestingaregiven.Experimentationresultswithmethodsofcentralizedanddecentralizedcontrolareprovided.Conclusionsandcontributionsofthisresearchareexamined.Furthermore,plansandideasforfutureworkwithinthisresearcharedescribed. 19

PAGE 20

CHAPTER2BACKGROUNDTherearevariousfocusescurrentlybeingresearchedintheareaofcooperativehomogeneousandheterogeneoussystems.Someofthesesystemsfocusontheconceptofhomogeneousvehiclesswarmingandformingacollectivebehavior,whileothersfocusmoreondevelopingadditionalcapabilitiesofusingaheterogeneoussystemineitherasingleenvironmentormultipleenvironments,suchassea,ground,orair. 2.1FullyDevelopedCooperativeSystemsOverthepastfewyearsthestateoftheartforhomogeneousandheterogeneousgroupsofvehicleshavebeenresearched.DuanandLiufoundvariousprojectsthathaveadvancedtechniquesinocking,formation,andnetworkcontrol,whichwereusedinvariousmissiontypesfordemonstration[ 1 ].Specicallytheyfoundgroups,suchastheUniversityofPennsylvania,lookedintosystemsofUGVsandUAVsthatworkcooperativelytoaccomplishtasksbycreatingformations.TheyalsofoundastrongpushtocreatesystemsutilizingtheMobileAd-HocNetworks(MANET)featurewhichturnsUAVsintonetworkrepeatersforUGVs.Theyalsodiscussthemissiontypesthesesystemsarebeingbuilttoworkwithin,suchasSearchingandLocalization,orTrackingandPursuit-Evasionsystems.Amorein-depthreviewofthethesendingsaswellaswellasotherprojectsandtheirdevelopmentshasbeengivenbelow. 2.1.1UniversityofPennsylvaniaTheUniversityofPennsylvaniaGeneralRobotics,Automation,Sensing,andPerception(GRASP)Labhasbeenworkingonthedevelopmentofcollaborativesystemsoverthepastdecade.Duringearlydevelopment,theGRASPLabfocusedonformationcontrolofgroundvehiclesusingasystemofUGVsthatcoalescingintoformationsaroundashepherdingUAV,orblimpasshowninFigure 2-1 [ 2 4 ].Focus 20

PAGE 21

wasthenplacedonusingExpectedMaximizationstatisticalandpatternrecognitionmethodsforgeneratingGaussiandistributionsofvehiclesontheground,thenusingthesedistributionsinordertosplitthegroupsintoformationsperUAV.TheoverallgoalofthissystemwastoshowhowmultipleUAVscouldbeusedtocollectinformationonagroupofUGVs,thenusingthisinformationtheycommandgroupsofUGVstorstformaformation,andthenguidingthegroupfromwaypointtowaypoint. Figure2-1. UAVBlimpandUGVvehiclesusedinearlierUPennresearch UPennalsodevelopedasystemforperformingreconnaissanceandtrackinginanurbanenvironmentusingUAVsandUGVs[ 5 7 ].Inthissystem,showninFigure 2-2 ,UAVplanesoutttedwithanInertialMeasurementUnit(IMU),controlpackage,andcamerapackagearetaskedwithyingintoanurbanareaandplanningtheirownpathtosearchandcollectgroundfeaturedata.Oncecoverageiscomplete,theUAVstravelbacktoabasewherethedataisoff-loaded,processed,andusedtocreateanapriorimapoffeaturesforUGVstouse.Oncegenerated,thismapissenttoagroupofUGVs,whicharethencommandedtoentertheeldofoperationandperformsearchandtrackingbehaviorsaccordingtotargetsofinterestonthemap.OnesuchdemonstrationshowstheUGVslocatingabuildingthatholdsatargetofinterestandthentracking/followingthattargetoutsidetheeldofoperation.Amajorfocusforthissystemwashowtouseaprioriinformationtolocalizetargetsintheareasothatgroundvehiclesareabletoefcientlycollectdata. 21

PAGE 22

A BFigure2-2. CollectionofvehiclesusedinMulti-RobotteaminGRASPLab:A)UAVPlanesusedbyGraspLabB)CollectionofUGVsusedbyGraspLab Morerecently,UPennhasbecomefamousfortheirvariousvideosofcooperationbetweenUAVquadrotorsperformingformationights,aerialdances,graspingandmovingobjects,buildingstructures,andvariousothercooperativetasks.KumarandMichaeldescribedhowmicroUAVs,specicallyquadrotors,offeranewareaofopportunityinresearchandapplicationforsystemssuchasUPenn'spreviouswork[ 8 ].OneoftherstprojectsappliedtothistheorywastheuseofagroupofAscTecHummingbirdQuadrotorsUAVscollectivelymovingfrompointtopointinatightformation(showninFigure 2-3 ).Givenpreciseposeinformation,Turpinetal.describedhowaformationiscreatedusingparametersgiventhroughashapematrix[ 9 ].EachUAVplansitsowntrajectory,usingboththegivenparametersandamotionsensingcameratoestimatethestateofotherUAVsandplanaccordingly.Mellingeretal.developedasystemofquadrotorsthatcooperativelygrabandmovewoodblocksofvaryingsizethroughoutathree-dimensionalenvironment[ 10 ].Mostofthefocusofthesesystemshasbeenplacedonthenon-linearcontrolandtrajectoryplanningforeachvehicleinacontrolledenvironment.AlloftheaboveworkisperformedinacontrolledenvironmentwithmultiplemotioncapturecamerasbyVicon.Althoughvehicleswithin 22

PAGE 23

thesesystemsknowtheirexactposeintheenvironment,alimitationduetotheuseofthesecameraswillnotallowoperationoutsideofastrictlycontrolledenvironment. Figure2-3. GRASPLabUPennAscTecquadrotorswarm 2.1.2EidgenossischeTechnischeHochschuleZurichETHZurichisanotheruniversitygainingnotorietyfortheiruseofUAVmultirotorvehicles.ThrougharesearchpartnershipwithAscTec,theyuseHummingbird,PelicanandFireyUAVstocooperativelyaccomplishtasks.OnesuchprojectincorporatesaSwarmofmicroFLYingrobots(sFLY)foruseinsearchandrescuemissionwhereUAVsattempttondvictimsinaGPS-Deniedenvironment[ 11 ].ETHZurichdetailsasystemwheremultipleUAVsareusedtocollectimageryviaasinglecamera,whichisthenfusedwithvehiclestateinformationtoformaglobalmapwithhighlightedareasofinterest[ 12 13 ].Thefocusofthisworkistoperformlocalizationbycreatingamapthatisusedtoextractlocationsofobjectsintheareatodevelopapose,andthenlocatingobjectsofinterestinrelationtothevehiclesownunderstandingoftheenvironment.Thoughinitialtestingwasperformedinacontrolledenvironment,theyhavesincemovedtooutsideenvironmentsandshownthattheirmethodisstilleffective.Asanexampleoftheirefcientcontrolofquadrotors,ETHZurich'sFlyingMachineArenalabdevelopedalgorithmsthatallowquadrotorstocooperateinplayingacrobaticgames.Onesuchalgorithm,asdenedbyRitzetal.,allowsaeetofquadrotorstoplay 23

PAGE 24

catchwithaball[ 14 ].Thisworkhasadvancedintoasystemwherequadrotorsbalancepolesonthetopoftheirframeandthenhavetheabilitytotossthepolebetweenvehicles.ThisworkfollowscloselytotheUniversityofPennsylvaniawiththefocusoncontrolsandtrajectorytrackingofUAVsolelyoperatesbyuseofamotioncapturecamerasystem. 2.1.3StanfordUniversityIn2001,StanfordUniversitycreatedaprojectknownastheStanfordTestbedofAutonomousRotorcraftforMultiAgentControlorSTARMAC[ 15 ].Usingbothcentralizedanddecentralizedcontrol,theirresearchfocusedontheuseofsmallUAVquadrotorsusingwaypointnavigation,withagoalofdevelopingatestbedsysteminoutdoorenvironmentsinordertoshowthevalidityofusingmulti-agentalgorithmsinareal-worldsituation.LaterprojectsdevelopedatrajectorytrackingcontrolsystemthatcreateapathfromgivenwaypointsandvelocitiesallowingaUAVquadrotortonavigatethroughaclutteredenvironment[ 16 ].ThesystemwasbuiltforasingularUAVandtestedinanindoorenvironment,witheventualgoalsofplacingthesystemtechniqueintoagroupofUAVsforimprovedcontrol. 2.1.4SmallerCollaborativeProjectsSeveralothergroupshavealsocreatedsystemsforcooperativevehicles,andthoughonasmallerscalethanprojectspreviousmentioned,signicantadvancementshavebeenmade.Forinstance,Vandapeletal.atCarnegieMellonUniversityworkedonthedevelopmentofasystemusingasingleUAVandmultipleUGVs[ 17 ].Thoughthesystemdidnotcooperativelyworkwitheachotherinrealtime,asystemwascreatedinwhichaUAVcouldbeusedtotravelintoaeldofoperationandcollectimagerylocalizationdata.OnceprocessedthisdatawouldbeusedbytheUGVstounderstandanenvironmentbeforedrivingintoit.AnotherexampleofworkfocusingonquadrotorcontrolsandtrajectorytrackinginformationhasbeenindevelopmentattheMassachusettsInstituteofTechnology.Called 24

PAGE 25

RAVEN(Real-timeindoorAutonomoustestENvironment),thissystem,whichusedViconmotioncapturecamerasystems,performedrobustcoordinationalgorithmsforagroupofquadrotorstofurtherdevelopvision-basedsensingsystemsthatbothdetectedandguidedthevehicles.Howetal.describesthesystemspurposeasawaytosimplifyandhandlemulti-vehiclecoordinationandcontrolatalowerlevel,thereforeallowinghigh-levelbehaviorstobeimplementedquickly[ 18 ].TheUniversityofTexasinArlingtonhasalsoworkedwithcollaborativesystemsatdifferentlevels.ZenginandDoganfocusedonhowagroupofUAVsfollowingacooperativetechniquecantrackatargetthroughanadversarialenvironment[ 19 ].Theydevelopedcostfunctionsbaseoffofinformationgivensuchaszonesthataredeclaredunsafeintermsofthreatlevelexposure,obstacleswithintheenvironment,thedesiredformationthattheUAVsshouldkeepwhiletracking,andnallythedistancetothetarget.ThiscostfunctionisthenusedtogenerateprobabilitydensityfunctionsthatareusedtosteertheUAVstowardsorawayfromobjectswithintheenvironmentwhilestilltrackingtheprimarytarget.Amorecontrolled,butlargerheterogeneousswarmprojectwasdevelopedbetweenvehiclescalledeye-botsandfoot-bots.Ducatelleetal.andMathewsetal.bothdescribeasystemmadeofsmalltrack/wheelgroundvehicles(foot-bots)andvethrusterairvehicles(eye-bots)thatsolvetaskswithinacontrolledindoorenvironment.Eye-botsaresetuptoythroughtheenvironmentandattachthemselvesmagneticallytoaceiling.Thougheye-botsaredescribedascapableofight,alltestsexplainthateachismanuallyplacedinagridpatterntoassistfoot-botsinnavigatingbetweentwotargetsinaforagingtasksystemthatcloselymimicsantcolonymovement[ 20 21 ].Otherfocusesoftheresearchshowhowaeye-botsassistfoot-botsintraversingagapclearingenvironment[ 22 ],usingasingleeye-bottoassistfoot-botsintravelingoverahillintermsofanglesteepnessandcalculatinggroupformationinordertotraverseanangletoosteepforasinglevehicletoclimb[ 23 ]. 25

PAGE 26

AnotherexampleoffocusinlocalizationisgivenbyDeSilvaetal.whodescribesaheterogeneoussystem,inwhichUAVsuseacombinationofvisualandacousticdatatolocalizepositionwithagroundvehiclewithinaclosedenvironment[ 24 ].AbearingsensorusesimagerydatafromapinholecameraontheUAVtolockontoagroundvehicleandcalculatethegeneratedheading.Arangingsensor,orTimeofFlight(TOF)sensor,usesanacousticsourceandreceiverbetweentheUAVandUGV.Theamountoftimeforthesourcetotravelthroughairgivesthedistancebetweenthetwotargets.Acombinationofthetwosensorsgivesastatisticallyaccuraterelativelocationbetweenthetwovehicles.MorerecentlyGarzonetal.developedaheterogeneouscooperativesystembetweenasingleUAVandUGV(Figure 2-4 )[ 25 ].AUAVwasmanuallyownaboveanautonomousUGVwiththerequirementthattheUGVstayintheeldofviewduringtheentireight.AerialimagerywascollectedandprocessedtosupplytheUGVwithobstacledataintheforwarddirectionoftheUGV.Usingthisinformation,aUGVcouldthentraverseapathinanoutsideenvironmentsafelygiventhattheUAVaccuratelyportrayedtheenvironment. 2.2SystemsDevelopedforGroupsofUAVsorUGVsThereareafewresearchprojectsthatplacefocusonindividualcomponentsofacooperativesystemsinacontrolledorsynthesizedenvironment.Onesuchproject,explainsTannerandChristodoulakisofUniversityofNewMexico,focusesonthecoordinationbetweentwoheterogeneousgroupsofUAVsandUGVs[ 26 ].Intheirsystem,groundvehiclesinteractonanearestneighborsystem,synchronizingvelocitiestomaintainformationduringtravel.Groundvehicles,meanwhile,estimatethecentroidoftheformationandtransmittheinformationtoanaerialvehiclewhichtravelsalongwiththeformation.Othersystemsfocusonhowtohandleleaderandfollowerdevelopmentbetweenrobotswarms.Leeetal.usesagroupconsensusalgorithmtoassistintheselection 26

PAGE 27

Figure2-4. ObjectAvoidanceusingUAVandUGV ofaleaderforagroupofundenedrobotswarms[ 27 ].Thealgorithmallowsforself-adjustment,givingaleadertheabilitytoeitherrecruitmoremembersastheswarmgrows,orasataskrequiresmoreagents.Thissystemofvehiclescommunicatesbackandforthusinglocalprobabilisticmethodstodenewhichvehicleisbesttoleadandwhichvehicleswillattempttoaccomplishagiventask,therefore,inherentlysplittingthegroup.Visionisalwaysamajorcomponentofcooperativesystems,andinsomecases,isusedfordockingofaUAVonaUGV.Inordertocombattherelativelylimitedrun-timeofaUAV,onesuchsystemutilizingbothmarker-basedandoptical-owbasedcomputervisiontechniqueswasattemptedbyLietal.,givinganAscTecHummingbirdtheabilitytotrackpositionandvelocityofaPioneer3-DXUGVvehicleandthentoaligntotheUGVfordocking[ 28 ].Marker-basedvisiontechniquesareusedtotracktwoledsontheUGVtoassistintheHummingbirdstrackingofthegroundvehicle.OncetheHummingbird 27

PAGE 28

istoocloseformarker-basedvisiontowork,itswitchestoanoptical-owtechniquetonishlandingthevehicle. 2.3ConclusionsThroughallofthepapersreviewed,itcanbeseenthatamajortrendtowardscooperativesystemsutilizingUAVshasbeenestablished.Mostofthesystemsfocusoncontrolandformationbehaviors.Varioussystemshavethebenetofgivingamoreaccurateposeforeachvehiclegiventheuseofamyriadofmotioncapturecameras,allowingresearchtofocusmoreonthehigh-levelcontrolbehaviorsandlessonhowtogetaccurateinformationonvehiclelocation.Asdiscussed,theseplatformsallowforhigherfunctioning,howevertheycannotfunctioninarealworldtestenvironmentduetotheirdependencyoncamerasthatrequireacontrolledenvironment.Othersystems,suchasETHZurichssFlyproject,focusonhowtousesystemsinanoutdoorenvironment.ThesesystemshavecreatednewtechniquesinLocalization,CollectiveNetworking,FormationsandFlocking,ObstacleAvoidance,andPathPlanningtoallowacooperativesystemtobeutilizedinarealworldtestenvironment.Thoughthesesystemsareadvancingthecapabilitiesofacooperativehomogeneousgroupofvehicles,veryfewfocusonthebenetsofusingheterogeneousgroups. 28

PAGE 29

CHAPTER3EXPLORATORYHOMOGENEOUSSWARMDEVELOPMENTInitially,abetterunderstandinginhowtoworkwithacollaborativesystemandtodevelopabasefortheproposedheterogeneoussystemwaspursued,usinganexternalprojectwithahomogeneousswarmofUGVsthatwasinheritedandfurtherdeveloped.Afterward,aprojectwasinitiatedtorstdevelopasingleUAVbeingcontrolledbyanAndroidtabletwithintheRobotOperatingSystem(ROS)environmentwasestablished.AfollowupprojectwascreatedtoexpandthecapabilitiesoftheUAVusingvisiontolocateandlandataspeciedtarget. 3.1HomogeneousSwarmofUGVsinGoogleCloud 3.1.1DevelopmentofInheritedSystemInitially,anexternalresearchgrouprstsetouttodevelopanautonomousgroupofUGVrobotswiththehopesofforminganemergentcollectivebehaviorbetweenallvehicleswhilecompletingatask.Wrightetal.describesthegoaloftheworktobeasystemthatusessimpleandcostefcientvehicleswithbasicsensingcapabilitiestoformamorecapablegroupofvehicleswiththeabilitytoaccomplishcomplextasks[ 29 ].SoftwaredevelopmentfortheswarmofvehiclesrstbeganonLegoMinestormsNXTplatforms(Figure 3-1A )toallowforexible,butapplied,testingoftheirsystem.TheprojectwasthenmovedtoamorecapableRCgroundvehicle,theXTMRail(Figure 3-1B ),whichwasmodiedwithasmallARMbasedcomputer,communicationcomponents,controlelectronics,andsonarsensing.TheXTMRailwasusedforexperimentaltestingandexercisesofmissionssuchasAreaSurveyandAreaSetupinanoutdoortestingenvironment.Theswarmarchitectureconsistedofthreemainlevels:acloudenvironment,abasestation,andindividualUGVswarmagents.ThedefaultsetupofthesystemwasforeachindividualUGVtohavetwo-waycommunicationwithabasestationthatwaslocatedintheeldofoperation.Thebasestationhadtheabilitytosendmissionparameters 29

PAGE 30

A BFigure3-1. VehiclesusedindevelopingahomogeneousswarmofautonomousUGVrobots:A)LegoMinestormNXTPlatformB)XTMRailRCVehicle toeachindividualagentand,uponconrmationthemissionwasreceived,thebasestationcouldthensendthecommandtoexecutethemission.Whilethemissionwasunderway,thebasestationthenentereda10Hzcommandloopwhereitreceiveddesiredinformationforeachagentexecutingthemission.Theindividualagentshadtheirownexecutionloopthatwouldreceiveamissionandonceallparameterswerereceived,wouldthensendanacknowledgmentcommand.Duringtravel,theseagentshadalowlevelcontrolloopandselfpreservationbehaviorstohandlemovementintheeldofoperation,whilealsohavingacommunicationloopthatwouldcontinuouslytransmitpositiondatatothebasestation.Asobjectsofinterestweresensedintheenvironment,therecognitionoftheobjectwasalsosenttothebasestation.Alongwiththevehicles,acloudenvironmentexistingwithinGoogleCloudwasusedwiththeabilitytocontrolorviewthroughaGoogleCloudApp.Thecloudenvironmentexistedfortwopurposes.FirstlytoshareinformationthattheswarmhadcollectedwithanexternalsystemrunningaGoogleCloudAppprogramdisplayingtheinformationontoaGoogleMapsrepresentation,thusallowinghumanobserverseldsituational 30

PAGE 31

awareness.Secondlythissystemallowedmissionparameterstobedecideduponandthensenttotheswarm.Alldatacollectedbyagentsintheswarmwastransmittedtothebasestationandsubsequentlyrelayedtothecloudenvironment.Allclouddatawasthendisplayedinrealtimethroughahumaninterfaceusinga2Dmodeloftheeldofoperation.Thehumaninterfacealloweduserstocontroltheswarmasdesiredbychangingtheindividualparameters,authorizing,oraltogethercancelingthemissions.Alongwithcreatingthephysicalswarmofvehicles,workwasperformedindevelopingameshnetworktopologyinordertohandlecommunicationswithindividualUGVsandthebasestation.Scenarioswerereviewedintermsofhowloworlostnetworksignalsituationscouldaffecttheswarmasawhole.Onemeshhealingtechniqueusedtorepairthestrengthofcommunicationinvolvedhaltingavehicle'scurrentmissionprogressandreturningtothelastknowncentroidoftheswarm.Insituationswherethevehiclelostsignalcompletely,thevehiclewouldfollowthepreviousdescribedmeshhealingmethodwhiletheentireswarmwouldstoptheirmissionprogressandtraveltothelastknownpositionofthevehiclewiththelostsignal.Giventheinefciencyofthismethod,analmethod,LeaveNoManBehind,wasexplored.Inthismethod,thevehiclewiththelostsignalwouldremaininplaceandasmallgroupofvehiclesweredispatchedfromtheswarmtondthelostvehicle.Thougheachmethodisdiscussedindetail,theleastefcientmethodofhandlinglostconnectionsinwhichtheagentwiththelostsignalreturnstothelastknownpositionofthebasestationwasused.Duringexperimentation,amissionwassenttothecloudenvironmentandthebasestationwouldreceivethedetails.Oncetheagentsoftheswarmreceivedmissionparametersandbeganthemission,theywouldattempttorandomlytravelaroundtheenvironmenttocompletethemission.Theyself-formedagridpatternwhilebouncingawayfromeachotheraccordingtoavoidancealgorithms. 31

PAGE 32

3.1.2FurtherAdvancementsforHomogeneousSwarmofUGVsThesystemwasdevelopedfurtheronceitwasfullyinherited.Timewasinitiallyspentmodifyingthecodebaseandtherstprototype,asseeninFigure 3-2A ,oftheagentswasslightlyoverhauledtocleanupsomeoftheissuesfoundduringprevioustesting,whileplansweremadeastowhatthenewphaseofthesystemwouldbe[ 30 31 ].VehiclecontrolwaschangedfromasimpleGPSheadingsystemwhichresultedinaserpentinedrivingpatterntoanInertialMeasurementUnit(IMU)basedsystemwithProportionalIntegralDerivative(PID)controltobetterestimatepositionandorientationbetweenGPSreadingsandresultinginbettercontrolledsteering. A BFigure3-2. XTMRailPrototypesusedinSwarmSystem:A)XTMRailPrototype1B)XTMRailPrototype2 Duetocapabilitylimitations,somehardwaremodicationsweremadetothevehicles.AfasterCPUcapableofhandlingsensorandvehiclestatemoreefcientlywasadded.Allequipmentonthevehiclewasmovedfromatopheavycontainertowithintherollcageofthevehicle.Wi-FiwasalsoaddedtothevehiclestoallowforaMobileAdHocNetwork(MANET)wheninrangeofotheragentsorthebasestation,thusallowingforabetterbandwidthversusthepreviousserialconnectionsmadeviaXBeeRFTransmitters. 32

PAGE 33

Testswerebeenperformedwithboththenewprototype(Figure 3-2B )andoldprototypeformingtheswarm.Thesameresultingbehaviorfromprevioustestswasobserved,however,thecontrolandcapabilityofthenewerprototypewasproventobemoreefcientataccomplishingtasksandvehicleswouldaccomplishdesiredtrajectoriesmoreseamlessly. 3.2AutonomousUAVinROSEnvironmentUponcompletionoftheUGVbasedhomogeneousswarmproject,anotherprojectwasstartedwiththefocusofdevelopingasystemwhereaUAVcouldbecontrolledviacommandsgivenbyanAndroidtabletina3GMobilenetwork.Thereweretwomainfocusesinthedevelopmentofthissystem:FirstlytodevelopabetterunderstandingofROSintermsofcommunication,control,andcomputervision.SecondlytodevelopaUAVthatcouldbegivenamissionintermsofwaypoints,thentraveltothosepointswhilereceivingimagerydata.Theprojectwassplitintofourmajordevelopments:PreparationoftheUAVsystem,developingtheROScontrolsystemfortheUAV,theROScomponentscontrollingvision,andtheAndroidApplicationforsendingandcontrollingmissions. 3.2.1BackgroundinROSEnvironmentROSisanopen-sourceframeworkforcopiousrobotplatformscontainingvarioustools,libraries,andconventionstoassistinrapiddevelopmentofvariouscomplexbehaviorswhileprovidinganoperatingsystem-likeenvironment.OriginallydevelopedbyStanfordArticialIntelligenceLaboratoryandcurrentlyunderWillowGarage,alargecommunityofcompaniesandhobbyistsnowpulltogethertocreateopen-sourcesoftwarewithinROStoassistincommunicatingwithdevices,controllingvehicles,visionandmapping,andmanyotherareasofinterest.Firstandforemost,ROSprovidesamessagepassingframeworktoallowmultipleprocesseswithinasystemtocommunicate.ByforcinguserstoimplementclearinterfacesdenedinthemessageInterfaceDescriptionLanguageemployedwithin 33

PAGE 34

ROS,specicparametersandtypesareusedtodeneasimpleMessagedatastructure.Tohandlecommunicationbetweenprocesses,TopicsaredenedasanonymousbuseswhichpassmessagedatastructuresbetweenprocessesrunningunderaprimaryROSMasterdatabase.Beinganonymous,topicsareindependentofwhocreatesandwhoconsumestheinformationtransmitted.Aprocessmaypublishatopicusingthepublisherinterface,whileanotherprocessmaysubscribetothetopicusingasubscriberinterface.Anothermethodofcommunicationbetweenprocessesfollowsarequestandresponseparadigm.Oppositetomessage'smany-to-manyone-waytransport,ROSallowsaRemoteProcedureCall(RPC)structurethroughthedistributedprocessenvironmentwiththeuseofServices,whicharesplitintotwomessagesthatdenetherequestandreplycomponents.RunningwithinROS,aServiceprocesswillpresentaserviceunderaspeciedstringnameusingthereplymessagetype,whileanotherClientprocesscallstheservicebycallingtheservicenameusingarequestmessage.TheService-Clientcommunicationistwo-way,sothatoncetheservicehascompletedthedesiredtask,aresponseissentbacktothecallingclient.ProcessesinROSaretypicallydenedasNodes.Eachnode,whenstarted,connectstotheROSMasterwhichprovidesnamingandregistrationusinganodehandler.ROSMastercombinesnodesintoagraphstructure,assistingnodeswithndingeachotheraswellasstreamingtopics,RPCservices,andaninternalparameterserver.Nodesareintendedtocontrolaspecicprocess,forexample,adriverforalaserrangender,abehaviortogenerateanobstacleavoidancepath,oradrivertocontrolmotorsdependentonaspecicinput.AllsoftwareinROSisorganizedintopackages.Packagesaretypicallycategorizedtohandleaspeciccomponentorbehaviorwhilebeinglightweightandeasytoreuseundervariousprojects.APackagemaycontainnodes,messages,services,libraries, 34

PAGE 35

congurationles,oranythingelsethatmaybeneededtocreateaspeciccomponentofadesign.ROSalsoprovidestheabilitytogroupUnderROS,allnodesmustconnecttoasingleROSMaster.Inthemajorityofmultiplevehiclesystems,aROSMaster/Slavecongurationisused.Inthiscase,asingleROSMasteriscreatedandallothersystemsaretreatedasslavesthatareconguredtoconnecttothesingleROSMaster.InthesituationwheretheROSMasterislost,thenodesrunningontheROSSlavecongurationceasetofunction. 3.2.2UAVSystem A BFigure3-3. TopandSideViewsoftheAscTecPelicanUsedForProject:A)AscTecPelicanSideViewB)AscTecPelicanTopView TheUAVusedfortheproject,anAscTecPelican,canbeseeninFigure 3-3 .Figure 3-3A showsasideproleoftheAscTecPelican.Thetopsectiondesignatedbyanorangeboxshowsthein-houseAscTecAutopilotsystemcreatedbyAscTecthathandleslow-levelcontrolsuchasattitude,altitude,andloitercontrol.Commandprotocolsare 35

PAGE 36

senttotheAscTecPelicanviaaseriallinkbetweentheautopilotsystemandAscTecAtomboardhousedwithintheUAVbody,designatedbyaredboxinFigure 3-3A .UbuntuandROSwereinstalledontotheAtomboardtohandlehigh-levelbehaviors.Thenalcomponentofthesideview,shownintheblueboxinFigure 3-3A ,istheFireFlyMVMissionVisionCamerasystembyPointGrey.Figure 3-3B showsatopproleoftheAscTecPelican.Allthreedevicesdesignatedbythegreen,red,andblueboxes(GPS,magnetometer,andElectronicSpeedControllerrespectively),aredirectlyconnectedtotheAscTecAutopilot.TheGPSandmagnetometersupplytheAutopilotwithsensorreadingsforLoiterandAttitudecontrol,whiletheElectronicSpeedControllersareusedtocontroleachmotor. 3.2.3ROSUAVControlSystemCommunication,control,anddatacollectiononthehigh-levelAtomboardishandledwithintheROSenvironmentthroughmultiplenodes.ThemainnodefortheUAV,AscTec Drivers,wasinitiallycreatedbyCommunityCollegeofNewYorkfortheirresearchin3Dindoormappingwithmicro-UAVs[ 32 33 ].Thenodeoriginallyprovidedmanualcontrolandbasicstatusmessages,however,itwaslatermodiedtoincludewaypointcontrolandvariousstatusmessagesneededfortheproject.TheupdateimplementedallowedasinglewaypointtobereceivedthroughaROSmessagethatisthensenttothePelicanasthecurrentdesiredwaypoint.Thenodealsoreceivedlaunch,land,andgotowaypointcommandsthatsentthePelicanonitswaytoeachdesiredwaypointduringthemission.TheVehicle Controlnodewascreatedasabridgebetweenanexternalcontrolprogramsuchas,butnotlimitedto,anAndroidDeviceandtheAscTecPelican.ThenodehadacomponentthatreceivedamissionintheformofawaypointlistthroughROSfromtheexternalcontrolprogram.Commandsintermsofstarting,pausing,modifying,orstoppingthemissioncanalsobesenttothisnode.Theothercomponentsincludedinthenodewereaninformationlooplinkedtoanexternalcontrolprogram 36

PAGE 37

andahigh-levelarbitratorthatwasusedtodecidewhetherthecurrentwaypointhadbeenreachedandanewoneestablished.AnalcomponentofthenodereceivedandtransmittedsensorandvehiclestateinformationfromtheUAVtotheexternalcontrolprogramforsituationalawarenessofthevehicleandmissionprogress.TheVehicle Controlnodewasmadespecicallytobeasneutralaspossible,thusallowingittoworkwithanyUAVDrivernodethatwouldreceivebasiccommandsandawaypointfornavigation.ItisaccuratetonotethatthestabilityofthevehiclewhenyingautonomouslyisowedmostlytothecapabilityoftheAscTecAutopilotwhiletheefciencyofthemissioncontrolisowedtotheVehicle Controlnode.Stabilityofothervehiclesthatusethesoftwarewillrelydirectlyonthatvehicle'sAutopilot. 3.2.4ROSVisionControlSystemVisioncontrolwasaseparatecomponentoftheUAVandwastaskedtocollectmissionimageryfromaFireFlyMVMissionVisionCamerasystemusingarewirebased1394driverpackagewithinROStodrivethecamera.TheMission Loggernodewasthencreatedtologightdataandstoreimagery.Flightdatawasloggedbyreceivingstatusinformationofthevehicleatacongurablerate.StatusinformationsuchasvehiclestatewasloggedintoaCSVleforplaybackandmissionreview.Thesecondpurposeofthesystemwastotimeandgpsstampimageswhilesavingthem,alsoatacongurablerate.Bothofthesecomponentsstoredinformationtoablackboxthatcouldberemovedandreplacedattheendofthemission.Alldatacouldthenbeplacedintotheexternalcontrolprogramandbeusedtoplaybackthemissionandreviewallcollectedinformation,orbeprocessedintocomputervisionalgorithmstoextractdatasuchas3Dinformationontheeldofoperation. 3.2.5AndroidApplicationforMissionControlTheinteractivecomponentofthesystemwascontainedwithintheexternalcontrolprogram.AnAndroidbasedapplicationwasmadebycombiningGoogle 37

PAGE 38

Maps,ROS-Java,andadatabaseandcontrolsystemtoholdmissionplans.Theapplication,showninFigure 3-4 ,allowedausertoplanamissionviaawaypointlistanddesiredmovementsorsettingstomakeatthosewaypoints.TheleftsideoftheapplicationprovidedtheuseraGoogleMapsinterfacetoenteradefaultwaypointbysimplytouchingthatpointontheinterface.Bymanipulatingthewaypointsastheywerecreated,acompletereconnaissancemissionwithdirectionalcontrolcouldbecreated.TheusercouldthenusetheAndroidapplicationtoprovidemissioncontrolovertheconnectedvehicle,inthiscasethePelicanUAV. Figure3-4. AndroidApplicationforMissionControl TheROS-JAVAenvironmentontheAndroidtabletandtheROSenvironmentontheAtomboardinitializedcommunicationprotocolsrunninginbackgroundthreads,whiletheVehicle Controlnodehandledmissioncontrolandreceiptofcommands.Onceamissionwasunderway,thetoprightwindowoftheapplicationcouldbescrolledinordertoviewmoreaccurateinformationintermsofmissionstatus.Thebottomrightwindowoftheapplicationstreamedeithercapturedimageswhenconnectedin3Gmode,orvideowhenonahigherbandwidthWi-Ficonnection. 38

PAGE 39

3.2.6TestingMultipletestsofthesystemwereperformedtomeasuretheabilityofthesystemwhenhandlingacompletemission.Missioncommandssuchasloiter,user-denedlawnmowersearchpatterns,buildingreconnaissance,andrandompatternsweregeneratedtodemonstratethevehiclescapabilitytofollowapre-plannedmission.Vehicledataandimagerywascollectedthroughoutallmissions,andonceamissionwascompleted,thevehiclewouldland,theblackboxwasswitchedoutforanother,andanewmissionwasstarted.Whilethenewmissionwasunderway,thepreviouslyusedblackboxwasthenusedtoplaythepreviousmissionontheAndroidApplicationwhileallowingtheabilitytoswitchbetweenmissionplaybackandcurrentmissionview. 3.3UAVPerformingAutonomousLandingonUSVAfollowupprojectandajointexercisewiththeautonomousUSVwithinMIL,PropaGator,wastousetheautonomousUAVtoassisttheUSVintheRoboBoatcompetition[ 34 ].InpreparingfortheRoboBoatcompetitioninwhichvarioustasksmustbecompletedbytheUSV,onesuchtaskrequiredtheuseofanexternalplatformtoleavetheUSVandretrieveapuck.TheprojectinvolvedthedevelopmentofacheaperUAVwithrespecttotheAscTecPelicanaswellasfurtheradvancementsinvisiontoassistincompletingthetask. 3.3.1USVandUAVsystemAmodelofthePropaGatorUSVusedfortheRoboBoatcompetitionisshowninFigure 3-5A .TheUAVusedfortheprojectwasaquadrotorusingacommerciallyavailableDJIFlameWheelF450frame.Off-the-shelfpartswereselectedfortheUAVtominimizethecostoftheoverallsystem,aswellastopromoteopens-sourcecollaboration.TheABSlandinggearoftheUAVwasdesignedsothatintheeventtheUAVcrashesoverland,thelandinggearwillabsorbmostoftheimpactandbreakoff,leavingtherestoftheframewithminimumdamage.Figure 3-5B showsthephysicalUAVusedinthisstudy. 39

PAGE 40

A BFigure3-5. USVCadModelandFlamewheelF450QuadrotorforRoboBoatCompetition:A)USVCadModelB)FlamewheelF450 BasicstabilizationandnavigationwashandledwithanArduPilotMega(APM)2.5+autopilotcontrolboard.ControlandsensorprocessingabilitieswereprovidedtotheUAVviaaquad-coreARMODROID-U2installedwithLinux-UbuntuandROS.TheODROIDretrievedinformationsuchascurrentorientationandGPSlocationfromtheAPM.TheODROIDwasalsocapableofsendingtheAPMinformationsuchasadesiredattitudeorGPSwaypoint,similartotheearlierprojectwiththeAscTecPelican.ThemainsensoroftheUAVwasaLogitechHDProWebcamC920camerawhichwasusedforcomputervisionaswellasobstacleandtargetidentication.CommunicationbetweentheUAVandUSVwashandledbyXBeeRFdeviceswhichhavearangeofapproximatelyof300feet. 3.3.2ROSUSVandUAVControlSystemBoththeUSVandUAVmakeuseoftheROSenvironmentforcontrol.SinceallnodescreatedarestartedwithandmusthaveaccesstoaROSMasterCore,alossofconnectiontosuchROSMasterwouldresultinallnodesandcontrolsshuttingdown,effectivelystoppingthevehiclefromfunctioning.EachvehicleisthereforesetupasaROSMastertoavoidthisfailure.Thougheachvehicleisindependentandisthemaster 40

PAGE 41

ofitsowninstanceofROS,vehiclesareabletocommunicateviaRFsothattheycancooperativelyplantheirdesiredtaskstocompletethemission.Communication,control,anddataisallhandledwithintheROSenvironmentthroughmultiplenodes.ROS BridgenoderunsseparatelyoneachROSMaster,handlingcommunicationbetweentheUSVandUAVwithintheoverallROSenvironment(Figure 3-6A ).ROS BridgeisconguredusingspecicparameterswhichdenedesiredtopicsandservicestobesharedbetweenmultipleROSMasters.OnceatopicorserviceismodiedwithnewdatawithinaROSMaster,ROS Bridgeacknowledgesthechange,andbroadcaststheresultingdataviaRFcommunications.OtherRFdevicesthenreceivethisdata,andsubsequentlyupdatetheirownROSMasters.Withinthisresearch,ROS BridgeprimarilyhandlesmissionobjectsintheformofdesiredGPSpointsaswellasmissionstatusinformation.DirectUAVcontrolandstabilizationislefttotheAPMautopilot.However,missionnavigation,aswellasdirectcontrolforvisualservoing,ishandledviatheAPM Drivernode.TheAPMallowsexternaldevicestocommunicatewiththeonboardautopilotsoftwareviaastandardizedMicroAirVehicleCommunicationProtocolcalledMAVLink.Specically,theAPM DriverusesMAVLinktoreceiveattitudeandstatusinformationfromtheAPMwhichisthenpublishedtoROS.OthernodesmayspecifymissionobjectivesintheformofGPSwaypointswhicharesenttotheAPMforwaypointnavigation.CommandsmayalsobesentthatresultintheAPMarming,disarming,launching,landing,ormodifyingstabilizationmodes.Duringvisualservoing,theAPM Drivernodeisusedtosendroll,pitch,yaw,andaltitudeadjustmentstothecontroller.Missionplanning,control,andcommunicationwithadeviceismanagedthroughtheAgent Controlnode,thesuccessortoVehicle ControlcreatedfortheAscTecPelican.Agent Controlactsasaninterpreterandmissioncontrollerforvehicledrivernodes(Figure 3-6B ).SensordatasuchasIMU,batteryvoltage,andGPSinformation 41

PAGE 42

areretrievedfromadriver,suchasAPM Driver.ThedatamaybemodiedandthenpublishedbacktoROSasdesired.Subsequently,thenodeperformsasthemissionplannerandcontrolsystem. A BFigure3-6. ROS BridgeandAgent ControlDataFlow:A)ROS BridgecommunicationbetweenvehiclesB)DataowwithAgent Control ThemissionstatesystemusesdataretrievedfromtheAPM,externalsourcessuchastheUSV,waypointlists,andtrackingdatafromvision.Basedonthestateofthemissionplanner,waypointsandcommandsmaybesenttotheAPM Driver.ThroughacombinationoftheAPM Driver,Agent Control,andROS,manytypesofmultirotoraerialvehiclesmaybecontrolled.VisualprocessingishandledwithintheVision Drivernode.Imagesarecollectedfromthecamerathroughanothernode,CAM Driver,andarethenpublishedtoROSusingstandardimagemessages.ImagesaretransferredtoaformatusableinOpenCVbyCVBridge,aROSlibrarywhichactsasaninterfacebetweenROSandOpenCV.TheVision Drivernodeprocessestheincomingimage,extractstherequireddata,andthenpublishestargetdataoftheitembeingtrackedtoROS.Therearevariousothersmallernodesthatareusedtoassistinthecontrolorsharingofinformationasneeded. 42

PAGE 43

Aspreviouslymentioned,theUSValsousesROSforcontrol,however,forthepurposesofthisresearch,onlytwocomponentswereinprimaryusewhencommunicatingwiththeUAV:Mission PlannerandROS Bridge.DuringthelaunchandrecoveryoftheUAV,theUSVloiteredinageneralposition,usingtheMission PlannernodetostarttheUAVmissionaswellassendandreceivestatusupdatesduringthemission.WhenthemissionwascompleteandtheUAVwasreturningforrecovery,GPSpositionsofdesiredtargetsweresenttotheUAVaswellastheUSVslocation.Mission PlannersendsthisinformationfromtheROSMasterontheUSVtotheROSMasterontheUAVthroughROS Bridge. 3.3.3VisualProcessingandControlAllofthevisionprocessingperformedbytheUAViscomputedon-boardbytheODROID.TheUAVisabletotellwhenithasfounddesiredtargets,suchasalandingpad,byrunninganapplicationthatusesadaptivethresholdingandblobdetectionalgorithms.TheapplicationiswrittenusingOpenCV,anopen-sourcelibraryofprogrammingfunctionsforrealtimecomputervision.Aseachimageisreceived,aninitialGaussianblurlterisperformedtoreduceimagenoise.TheresultingimageissplitintoindividualR,G,andBchannelsaswellasconvertedtoHSVandLabcolorspaces.Adaptivethresholdingandmodicationofeachcolorspacechannelsisusedtocreatevariouscolorlterstaskedtodetectcolorsofthedesiredtargetstobetracked.Dilationanderosionmorphologicalfunctionsareusedtofurtherreducenoiseandllingapsofthethresholdedimage.Theresultinglteredimageisprocessedfurthertodetectblobsandcontoursintheimage.Eachblobfoundresultsinacalculatedarea,perimeter,centroid,andshapedeclaration.Blobsarereviewedtomeetcertaincriteriaintermsofminimumandmaximumsizesaswellasclosenesstosquareorcircularobjects.Dependingonthedesiredtarget,certainaspectsoftheimageareextracted. 43

PAGE 44

Toallowforrobustness,aKalmanFilterisusedonthenalimagetoallowforaccuratetrackingincaseofobstructionsorshortlossofthedesiredtarget.ThelterallowedtheUAVtoreturntoanexpectedareaifitpotentiallyovershotthedesiredtarget.ThreeredsquaresforminganisoscelestriangleareusedtodesignatethedesiredlandingplaceontheUSV.Thecentroidofeachsquareislocatedusingvision.Ifonlyoneortwosquareswerelocated,thecentroidofeitheranindividualsquareorbothsquarescombinedwereusedasthedesiredtargetfortheUAV.Onceallthreesquareswerelocatedwithintheimage,thelengthbetweeneachsquareiscalculatedtounderstandtheshapeofthetriangle.Thecentroidofallthreesquarescombinedisusedasthedesiredtargetlocation,whiletheshapeofthetriangleisusedtodesignatethedesiredheadingfortheUAVtoland.Figure 3-7 showsanexampleofthisprocedure. Figure3-7. ProcessedImageofLandingPad Anothertypeoftarget,intheformofalandingdock,holdsasmallcircularobject,orhockeypuck.Thelandingdockisalsodesignatedbyacoloredpinkborderaroundthedock.Whenattemptingtolocatethelandingdock,thecoloredborderisrstrecognizedandusedtoextractaregionofinterest(ROI)fromtheimage,asshowninFigure 3-8A 44

PAGE 45

Itisalsopossibletoignorethecoloredborderandextracttheshapeofthedockagainsttheforegroundofthewater.ThenewROIisprocessedtolookforthesmallobject,showninFigure 3-8B ,inwhichthecentroidisusedasthedesiredtargetlocation.Oncethecentroidoftheobjectislocated,anearestneighborbetweenthecentroidandtheedgeofthedockisfound,resultinginadesiredheadingfortheUAVtolandandavoidfallingintothewater. A BFigure3-8. ROS BridgeandAgent ControlDataFlow:A)FilteredandExtractedImageofLandingDockandPuckB)ProcessedImageShowingLandingDockBorderandPuck OnceatargetislocatedthroughvisionprocessingandtheresultingpositionofthetargetispublishedinROS,theAgent Controlnodehandlesvisualservoingtoalignandlandonthetarget.ThisishandledbyatwostagePIDcontrollerthattakesintargetdataandattitudeinformationandresultsinadesiredroll,pitch,andyawadjustments.OnestageofthePIDcontrollerhandlesalignmentofheading.Onceadesiredheading,dependingonthetypeoftarget,ispublishedtoROS,theyawcontrollerhandlesaligningthevehicletothedesiredtarget.Thisistohandleoffsetsincameraplacementoraligningthevehicletoadesiredlandinglocationonthelandingpad. 45

PAGE 46

IfadesiredheadingisnotpublishedtoROSoriftheyawcontrollerhasaccomplisheditscurrentgoal,stagetwoofthePIDcontrollerhandlesrollandpitchcontrol.Adirectionalvectorischosenbetweenthecurrentvehiclespositionandheadinginreferencetothedesiredtargetsposition.ThevectorisbrokendownintorollandpitcherrorswhichareusedwithinthestagetwoPIDcontrollertoadjusttheUAVposition.OncetheUAVisabovethedesiredtargetitbeginstodecreaseinaltitude.Adjustmentsmaybemadeinroll,pitch,andyawduringthedescent. 3.3.4TestingThemissionplannerontheUSVinitiatedthelaunchandrecoverymissionfortheUAV.TheUSVsentacommandtotheUAVtolaunchviaROS Bridge.Onceairborne,theUAVbeganitsownmissionbyfollowingaseriesofwaypointsuntilitlocatedthelandingdockwithitscamera.TheUAVthenbeganvisualservoingsothatitwasproperlyalignedoverthecircularVelcro-coveredobjecttoberecovered.TheUAVlandedontheobjectandpickeditupusingaVelcronet.TheUAVthenlaunchedwiththeobjectfromthelandingdockandbeganmakingitswaybacktotheUSVusingGPSwaypoints.Onceitwascloseenoughforthecameratolocatethelandingpadontheboat,theUAVsvisionprocessingbeganvisualservoingtoorientandpositionitselfbeforeitbeganitsdescent.Oncetheboatsuccessfullylanded,theUAVsendsconrmationofmissionaccomplishedtotheUSV.TheUSVisthenabletocontinuetherestofitsownplannedmissions. 46

PAGE 47

CHAPTER4HETEROGENEOUSSWARMARCHITECTUREInthischapter,animplementationofaheterogeneouscollaborativearchitectureofUAVsandUGVsworkingtogetherinunisoninordertocompleteataskinarealworldout-doorenvironmentisgiven. 4.1CooperativeSystemOverviewDescriptionsofeachagentthatcomposethegroundandaircomponentsofthesystemaswellasabasicdescriptionofthebasestationaregivenbelow. 4.1.1UnmannedGroundVehicleDesignTheUnmannedGroundVehicles(UGV)thatconstitutegroundagentsinthecooperativesystemarearevisedversionofthosedescribedinChapter 3 .EachagentisamodiedRCracingvehiclewithacustomcarbonberframedesignedandmanufacturedtohelpprotecttheinternally-housedelectroniccomponents.EachagenthouseseitheranODROID-U2quad-core1.7GHzExynosARMsingleboardcomputer,or,duetoupdatesduringdevelopment,someagentsareoutttedwithanupdatedODROID-U3,whichisafunctionallyequivalentboardtotheODROID-U2.AllhighlevelfunctionsfortheagentareperformedontheODROID,including,butnotlimitedto,planning,directPIDsteeringandcontrol,andsensorinformationprocessing.AgentnavigationandcontrolispartiallyhandledbyanArduPilot-Mega(APM)2.6.TheAPMsupportsvariousopensourceapplicationscommittedtothecontrolofxedwing,multi-rotor,androvertypeagents.IthousesanInertialMeasurementUnit(IMU)withaccelerometerandgyroscopesensorsforpositionalinformation,andabarometerforaltitudeinformation.Anoff-boardGPSandmagnetometerareusedtoimprovepositionalandorientationinformation.ThesoftwaresuiteloadedontheAPMisamodiedversionoftheroversoftwareintendedtocommandanAckermannbasedsteeringvehiclesuchastheXTMRail.TheODROIDdirectlyinterfaceswiththe 47

PAGE 48

APMtoallowsensorinformationtobeacquired,aswellasprovidesinglewaypointorsteering/throttlecontrolcommands.InadditiontotheAPM,thesensorsuiteconsistsofsonarforobstacleavoidance,aHokuyoURG-04LX-UG01LIDARforobstacleandtargetidenticationaswellasavoidance,andaLogitechC920webcameraforcomputervisionclassication.Onlysomeofthegroundagentsareequippedwithcamerastoallowforalocalheterogeneousgroupofagentswithdifferentcapabilities.CommunicationishandledbybothXBee900HPDigiMeshenabledRFmodulesaswellasWireless-NWi-Fi.Wi-Fiisusedwheninrangeofthebasestationtotransmithighbandwidthimageryandotherdatathatmaybeusefultothebasestationforcontroldecisionsorhumanobservation.Wi-FiisalsousedbetweenagentsthroughasharedAD-HOCnetworkthatallowsdistributionoflocalizationandenvironmentdatawithintheROSenvironment.RFcommunicationisusedtosendmissionsandcommandstoagentsaswellallowsharingofstatusandobstacleinformation. A BFigure4-1. XTMRailFinalDesignusedinHeterogeneousSwarmArchitecture:A)XTMRailFinalDesign(external)B)XTMRailFinalDesign(internal) Figure 4-1 showsanexternalandinternalviewoftheUGVagent.Table 4-1 describesthethreepossiblerolesthataUGVwillserve.UGVswiththeSearchrolecalculatealistofgoalstocoveranROIinconjunctionwithothersearch 48

PAGE 49

Table4-1. RolesforUGVsPerformingMissions TaskTypeDescription SearchUsingadeterminedplan,searchanareaforobstaclesandtargetswithotheragents.IdentifyUsingfoundobstacleinformation,navigatetoeachtargetandattempttoidentify.PathFollowingUsingasimplepredeterminedpath,navigateandcollectobstacleandtargetdata. agents.Oncealistofgoalsischosen,theagentthennavigatestoeachgoalwhileperforminganobstacleandtargetsearch.Agloballistsofobstaclesandtargetsisupdatedforotheragentstouse.IdentifybasedUGVsuselocatedobstacleandtargetinformationtoplanapaththroughtheenvironmenttoeachpotentialtarget.OncetheIdentifyUGVhastraveledtothepotentialtarget,itattemptsidenticationofthetargetaccordingtosetparameters.SimilartotheSearchrole,PathFollowingUGVstravelalongapathandattempttolocateobstaclesandtargets,however,thepathispredeterminedandprovidedbyaoperator. 4.1.2UnmannedAerialVehicleDesignTheUnmannedAerialVehicles(UAV)thatcomposetheairagentswithintheswarmuseavarietyofCommercialof-the-shelf(COTS)framesincludingtheDJIF450quadrotorandDJIF550hexarotorFlamewheelmodels.TheAscTecpelicandescribedintheUAVprojectfromChapter 3 isalsoutilizedwithintheswarm.Formoreaddedaerialdiversity,twocollapsiblehexarotorandoctorotormodelswerecustomdesignedandbuilt.AllUAVsusebrushlessmotorspoweredbythreeorfourcellLiPobatteries.SimilartotheUGVs,anAPMisusedtohandlebasicstabilizationofeachagent.AsUAVsareusedasaneyeinthesky,theprimarysensorforeachisaLogitechC920webcamera,whichisusedforcomputervisionperformingobstacleandtargetidentication.Allhighlevelfunction,decisionmaking,visionprocessing,andPIDagentcontrolishandledonboardusingthesamequad-coreARMcomputerastheUGVs,the 49

PAGE 50

ODROID-U2.AlsosimilartotheUGVs,communicationsforUAVsisimplementedwiththeXBee900HPDigiMeshRFmoduleandWireless-NWi-Fi.Figure 4-2 showseachFlamewheelmodel. A BFigure4-2. FlamewheelModelsusedinHeterogeneousSwarmArchitecture:A)F450FlamewheelQuadrotorB)F550FlamewheelHexarotor Figure 4-3 showseachcustommodel. A BFigure4-3. CustomModelsusedinHeterogeneousSwarmArchitecture:A)CustomHexarotorDesignB)CustomOctorotorX-8Design Table 4-2 describesthethreepossiblerolesthataUAVwillserve.ForUAVagents,theSearchandPathFollowingrolesaresimilartothemethodemployedonUGVagents,exceptsearchpathsareonlycalculatedusingotherUAVagents.UAVs 50

PAGE 51

Table4-2. RolesforUAVsPerformingMissions TaskTypeDescription SearchUsingadeterminedplan,searchanareaforobstaclesandtargetswithotheragents.TrackUsingpositioninformationforanagent,navigatetoagent,locatedtrackingtarget,thenfollowagent.PathFollowingUsingasimplepredeterminedpath,navigateandcollectobstacleandtargetdata. employingtheTrackroleareusedtofollowanotheragent,typicallyaUGV,andactasanextrasensor.Initially,theUAVwillnavigatetoadesiredagent'spositionandthenlocateatargetontheagentusingvision.TheTrackUAVwillthenfollowtheagentthroughouttheenvironmentwhilefeedingitinformationthatmaybeusedanywaynecessary. 4.1.3BaseStationLayoutAbasestationisusedtoprovideacentralhubforalltheinformationonagentswithinthecooperativesystemaswellasamethodforcontrolofthesystem.Thebasestationiscomprisedofacomputer,suchasadesktoportablet,withcommunicationcapabilitiesthatmimictheUGVsandUAVs.Ifthecollaborativesystemissetupinacentralizedstructure,missionsandcommandsaregeneratedonthebasestationandaresenttoeachagent.Positionaldata,missionstatus,andtargetorobstaclelistsmayberepresentedonthebasestation.Otherwise,ifthesystemisinadecentralizedstructure,thebasestationisusedtosimplyrelayinformationaboutthemissiontoaoperator. 4.2GraphicalUserInterfacesGraphicalUserInterfaces(GUIs)areusedwithintheswarmarchitecturetobetterprovideoperatorswithinformationandcontroloftheagentsperformingamission.ThebasestationprovidestheprimaryGUI,however,satellitetabletbasestationsrunningonAndroidtabletsmaybeusedtodirectorcommandtheswarm,asdescribedin[ 35 ]. 51

PAGE 52

4.2.1PrimaryROSQtGraphicalUserInterfaceWithinROS,thereisapackagecalledROSQt(RQT).ThispackageallowsvariousQTFrameworkbasedpluginstobecreatedandimplementedwithinasinglegraphicalinterface.EachpluginexistswithintheROSenvironment,allowingthemtouseROSbasedfunctionsasneeded.ManyofthetoolswithinROSmayalsobedockedinthegraphicalframeworkofRQT,allowingforaGUIthatbenetsfromROSvisualizationtools. Figure4-4. TheCongreGatorsGUI TheCongreGatorsGUIistheprimaryinterfacebetweenanoperatoratthebasestationandtheagentsusedintheswarm.TheGUIisusedtoprepare,manage,andcommandmissionswhilemonitoringagentandmissionstatusinformation.SpecicallymadetoworkwithinaROSenvironmentusingRQT,theGUIpackageiscalledcongregators rqt,followingtheframeworkdenedbyRQT,issplitintovarioussmallerpluginsthatmaybeswappedinoroutasdesired.Eachpluginisshownin 52

PAGE 53

Figure 4-4 ,wherethewaypoint,command,andstatuspluginsarerepresentedbytheblue,green,andredboxesrespectively. 4.2.1.1WaypointPlugin Figure4-5. ROSMessageandServiceowforwaypoint plugin Thewaypoint pluginconsistsofaQTwebviewtoholdtheGoogleMapsJavaScriptAPIaswellasafewbuttonsforoperatorinput.Initially,thepluginisusedtodenedirectwaypoint-to-waypointpathsortodeneaRegionofInterest(ROI)thatwillbeusedinmissionplanning.Figure 4-5 describesthedataowinandoutofthewaypoint pluginofcongregators rqt.DashedLinesrepresentdatathatistransmittedtoorfromanexternalROSMaster,whiledashed-dottedlinesrepresentdatathatistransmittedtoorfromeitheraninternalorexternalROSMaster.Waypointsaredenedbyoperatorinteractionwiththeplugininterface.Tomodifythewaypoints,variousoptionsaregiventotheoperatorthroughcommandbuttons:Add,Delete,Modify,New,andClear.Add,Delete,andModifyaretogglebuttonsandmustbeactivatedbeforeanychangemaybemadethroughthewaypoint plugin.TheNewandClearbuttonswillalloweitheranewwaypointtobecreatedusingapop-updialog,orforallcurrentwaypointstobecleared.Waypointlistsmaybeintendedforindividualagentsorallagents,therefore,adropdownboxisprovidedtoassistinchoosingwhattheintendedtargetis.Astheintendedtargetischanged,allwaypointsarerefreshedwiththosefromthenewintendedtargetswaypointlist.Anytimeawaypointismodied,aROSmessageintheformofwaypoint listisgenerated,whereisreplacedwiththeagentnameortheALLreference. 53

PAGE 54

Thewaypoint pluginisalsousedtotrackandreportpositionalstatusinformationofanyagentorobstacleinformationwhichhasbeenreportedtothebasestation.Agentstatusinformationisreceivedthroughtheagent statusmessagewhileobstacleinformationiscollectedthroughtheobstacle listmessage.Giventhataprioriknowledgeofagentsisnotknown,theagent listmessageisusedtoprovidenamesofallagentsthathavereportedinfortheswarm. 4.2.1.2CommandPlugin Figure4-6. ROSMessageandServiceowforcommand plugin Figure 4-6 describesthedataowinandoutofthewaypoint pluginofcongregators rqt.DatamaybetransmittedtoorfromaninternalorexternalROSMaster.Thecommand pluginprovidestheoperatorwithdirectcontroloverthemission.Similartothewaypoint plugin,adropdownboxisprovidedtodecidewhotheintendedoperativetoreceivecommandsis.TheSettingsbuttonopensupapop-updialogfortheoperatortochangeandsetspecicsettingspertainingtoamissiontype.Eachtimethesettingsareupdated,themission settingsmessageistransmitted.TheSendMissionbuttontriggersaservice,send mission,thatmaybeusedbyalowerlevelpackage(suchasmission control)totriggeracombinationofallcurrentrelevantmissiondata.TheStart,Pause,Abort,andReturntoBasebuttonsalltriggerthesend commandservicethatmaybeusedbyalowerlevelpackagetomodifythestateofthemission. 4.2.1.3StatusPluginThestatus pluginprovidesaverysimpleinterfacetoshowanagentsinformationonceithasbeenreceived.Multiplestatustabbedwindowswillbepopulatedasagentscomeonlineandbeginsharinginformation.Figure 4-7 describesthedataowinand 54

PAGE 55

Figure4-7. ROSMessageandServiceowforstatus plugin outofthewaypoint pluginofcongregators rqt.DatamaybetransmittedtoorfromaninternalorexternalROSMaster. 4.2.2SatelliteAndroidTabletInterfaceToexpandcontrollabilityoftheswarm,Androidtabletsmaybeusedassatellitecontrolstationswithintheeldofoperation.AndroidtabletsprovideanoperatorwiththesamecapabilitythattheCongreGatorsGUIprovidesintermsofeldcontrolandobservation.Whileintheeld,anoperatormaychangemissionplansortakecontrolofaspecicvehicle.TheAndroidtabletsrunamodiedversionoftheapplicationintroducedfortheUAVswarmprojectdescribedinChapter 3 .Theapplication'swaypointfragmenthasbeenfullyrewrittenusingGoogleMapsAndroidAPIv2,whichallowsforamoreseamlessuseofmapswithinfragmentactivities.Additionally,insteadofconnectingtotheROSnetworkasaROSslavedevice,communicationishandledthroughRFcommunicationstoallowforthesamerangeandnetworkthateachagentrunson,effectivelyturningthetabletintoanagentoftheswarm.RFcommunicationisprovidedbyasmallphysicalpackagethatisattachedtothetablet.ThepackagehousesanIOIO-OTGdevelopmentboard,anXBee900HPDigiMeshenabledRFmodule,andbattery.Thenewandroidapplication'scommunicationstructurehasbeenrewrittentocommunicatewiththecommunicationspackagewiththecapabilitytosendandparsedatabetweenagentsintheswarm.ThemethodemployedissimilartothesamemethodthatRFcommunicationsarehandledoneachUGVorUAV. 55

PAGE 56

TheinputandoutputtypedataisequivalenttowhatistransmittedtoandfromtheCongreGatorsGUI. 4.3XBeeBridgeCommunicationsAmajorfocusintheswarmarchitecturedesigniscommunication.WithinaROSmaster,dataistransmittedbetweennodesusingtheROSinfrastructureofmessagesandservices,however,disseminatingthistypeofdatabetweenmultipleROSbaseddevicescanbecomedifcult.NodesmusthaveaccesstoaROSmastertohandlenamingandregistration,aswellastrackingandadministeringtopicsofdata.Itispossibletomakeonedeviceintheenvironmentthemaster,andeveryotherdevicetheslave,aswasperformedinhomogeneousswarmtestingdiscussedinChapter 3 .However,ifthemasteriseverlost,allslavedeviceswillceasetofunction.Consequently,intheswarmenvironment,eachvehicleissetupasamaster.SharinginformationbetweenmultiplemastersisadevelopingsituationwithinROS.Twosolutionsemployedformulti-mastercommunicationwithintheswarmarchitectureincludetheROSprovidedmulti-masterpackageandthemostcommonlyusedcustomXBeeBridgepackagecreatedforthisresearch. Figure4-8. DataowbetweentwoXBeeBridgecongureddevices TheXBeeBridgepackageisamulti-mastersolutionusedwithinthedenedswarmarchitectureandservesasasignicantlymoreadvancedversionoftheROS BridgenodedescribedinChapter 3 .MovingawayfromthelimitationsofWi-Fi,longrangeRFXBeedevicesareusedtoshareinformationbetweenagents.TheXBeeBridgepackageisstructuredtolistentospecicmessagesorservicesonaROSMasterand 56

PAGE 57

thentransmitthemexternallytootheragents,alsorunningaROSMaster.XBeeBridgesuppliesthexbee bridgeROSPythonnode.Figure 4-8 summarizesthedatapathwaysforROSmessagesandservices.IfanagentispublishingaROSmessagethatisconguredwithinXBeeBridgetobetransmittedoverRFtoanotheragent,itistreatedasasingledirectionmessage.XBeeBridgeisconguredtocreateasubscribertolistenforthedesiredROSmessage.WhenevertheROSmessageispublishedontheagent,XBeeBridgepackagesthemessage,transmitsitoveranXBeedevice,andthenadesiredreceivingXBeedeviceistaskedwithrepackagingthemessageintoaROSmessagetobepublishedonthenewROSdatabase.IfanagentisstartingaROSclientrequestthatisconguredwithinXBeeBridge,itistreatedasasendandrespondmessage.XBeeBridgeisdesignedtocreateamatchingserviceinplaceofaconstructedclient.Whenthatclientsendsarequest,itispackagedandtransmittedoveranXBeedevice.AdesiredreceivingXBeeattemptstorepackagethemessageintoaclient,contacttheserviceonthenewROSdatabase,processesaresponse,andthentransmitstheresponsebacktotheoriginalcallingagent.Atanypointthroughouttheprocessafailureoccurs,theoriginalROSclientreceivesafailedresponse. Figure4-9. MessagestructureformessagesinXBeeBridge Figure4-10. MessagestructureforservicesinXBeeBridge StandardROSmessagesaretransmittedintheformatshowninFigure 4-9 ,whileFigure 4-10 demonstrateshowservicesaretransmitted.Whensendingdatabetween 57

PAGE 58

Table4-3. PossibleIDsforwhowillreceivethemessage ToIDDescription ALLDAlldevicesALLAAllagentsBASEBasestationTABxTabletwithxnumberAGxxGroundagentwithxxnumberAAxxAiragentwithxxnumber Table4-4. PossibleIDsforwhomaysendthemessage FromIDDescription BASEBasestationTABxTabletwithxnumberAGxxGroundagentwithxxnumberAAxxAiragentwithxxnumber systemsintheswarm,anIDisgivenforthereceiverandthetransmitter.ExamplesofpossibleIDstobeusedforthereceiver,ortheToeld,aregiveninTable 4-3 ,whileexamplesofpossibleIDsforthetransmitter,orFromeld,aregiveninTable 4-4 .ThePacket#andPacketSizeeldsareusedtospecifywhethertheXBeemessageispartofapacketwithagivensize,andwhichnumberinthepacketthecurrentmessagerepresents.TheTageldisathreecharacterstringthatrepresentsthetypeofdatathatthemessageholds.ThiscomponentisoperatordenedpermessagetypeasdenedlaterduringcongurationoftheXBeeBridgepackage.Forservices,theT/RFlageldrepresentswhetherthemessageisaROSservicebeingtransmitted(requested)oraROSclientresponding.Finally,theDataeldisacommadelimitedlistofallvariablesthatmakeuptheXBeemessageandistobeparsedintothedesiredROSmessageorservice.EacheldissummarizedinTable 4-5 .ErrorcontrolforthecommunicationsystemishandledpartiallybytheXBeeprotocolandthecustomprotocoldescribedhere.Asamessageiscreated,achecksumisgeneratedfortheentiremessageandissubsequentlycheckedonthereceivingend.Aspecialcharacterisalsousedtohelprepresentthebeginningandendofamessage.IfstandardROSmessagesarereceivedincorrectly,arequestmaybesenttothe 58

PAGE 59

transmittingdevicetorequestthemessagebesentagain.WhentransmittingROSservices,informationontherequestbeingsentisstoredinamappedbufferthatholdsdataformultiplevehiclesandrequestmessages.Asthecorrectresponseisreceived,itischeckedagainstthebuffertoconrmthatthemessageisbeingheldfor. Table4-5. XBeemessageelds FieldDescription ToFourcharacterstringrepresentingwhomessageistransmittedto.FromFourcharacterstringrepresentingwhomessageistransmittedfrom.Packet#Numberrepresentingwhichpacketthismessageisinasingleorgroupofpackets.PacketSizeMaximumnumberofpacketsintherelatedmessage.TagThreecharacterstringrepresentingthetypeofdatathemessagebelongsto.T/RFlag#Onecharacteragrepresentingwhethermessageisarequestorresponse(OnlyforROSservices).DataAcommadelimited,variablysized,characterstringrepresentingdataofmessageofTagtype.ChecksumTwocharacterchecksumresultoftheentiremessage(OnlyinXBeeATmode). 4.3.1NodeInitializationXBeeBridgeissetupasadynamicpackagetoexpeditesetupandremovetheneedforreprogrammingeachtimenewmessagesorservicesaretobeimplemented.Throughtheuseofacongurationle,thecommunicationissetuptolistentoanytopiconROSaswellastodenewhichvariableswithinthedatatopicwillbetransmitted.Thisallowsalevelofexpandabilitythatcurrentmulti-masterpackagesdonotallow.TopicsmayincludeROSmessagesorservicesandaredenedinthesetupleas: topics:topic1:tag_id:default_name:type:msgpackage:module: 59

PAGE 60

var:---<[LIST]>topic2:tag_id:default_name:type:srvpackage:module:req:---<[LIST]>res:---<[LIST]>Topicsinthecongurationleholdvariousparameters.Thetag idparameterprovidesathreecharacterstringthatistiedtoallmessagesofthistopictype.Thistag idisaddedtoeachmessagetocontrolhowthedataispackagedfortransmissionorhowitisparsedintoaROStopic.Thedefault nameparameterholdsasimplekeytiedtothetopicthatisusedfortheinternaldatastructuresofXBeeBridge.Thetypeparameterspeciesthetypeofpackagebeingdenedbythetopic,whethermsgorsrv.ThepackageandmoduleparametersareusedtospecifythenameoftheROSpackageaswellasthenameoftheROSmessageorROSservicethatatopicbelongsto.Thevar,req,andresparametersareusedtospecifythewhichoftheexactvariablesbelongingtothetopicwillbetransmitted.VariablesandClassesaresimplydenedbyadash,whilelistsaredenedbyadashwithacommadelimitedlistsurroundedbybrackets. 60

PAGE 61

Figure4-11. NodeInitializationforxbee bridge. 61

PAGE 62

Thexbee bridgenodeprovidesthedesiredfunctionalityoftheXBeeBridgepackage.Beforestarting,xbee bridgeexpectsthatthecongurationledescribedpreviouslyhasbeenloadedthroughtheuseoftheROSparameterserver.Figure 4-11 showsthenodeinitializationofxbee bridge.Atrst,theXBeedeviceisconguredaccordingtowhichmodeitshouldcommunicatein.Aparameterofxbee bridgespecieswhethertheXBeeisinATorAPImode,allowingforeitherserialorspecialAPIpacketcommunicationbetweenXBeedevices.IfinATmode,asimpleserialconnectionismadefortheXBeeport,however,ifinAPImode,animportedlibraryisusedtocreateaconnectionandvariableforusewiththeXBee. Figure4-12. Expandtopicparametersforxbee bridge. Next,thetopics,subscribers,andservicesparametersareloadedfromtheROSparameterserverintopythondatastructures.Eachkeyinthelistoftopicsischeckedastowhetheritisaserviceoramessagetopic.Ifthetopicisaservice,thebase,request,andresponsemodulesareeachloaded.Ifthetopicisamessage,themessagemoduleforthetopicisloaded.Nexteachmoduleisparsed,asshowninFigure 4-12 tondinformationoneachvariable.Theexpand topic infofunctionloopsthroughallvariables 62

PAGE 63

thatmakeupamodule.Ifthevariableisoftypelist,thenameofthevariableisaddedtoalistlists.However,ifthevariableisoftypeclass,thenameofthevariableisaddedtoalistofclasses,andthenthevariableistreatedasanestedmoduleandrecursivelycalledbyexpand topic infotobefurtherparsed.expand topic inforeturnsthecollectionoflistsandclasseswhicharethenstoredinthetopicsdatastructureforlateruse.Oncetherequiredinformationonatopichasbeenextracted,subscribersandservicesarecreatedforeachvariablespeciedinthesubscribersandservicesparameters.Withinitializationcomplete,thenodecreatesacallbackfunctionspeciedbythetypeofmodetheXBeeemploysforprocessingtheincomingdata.IfinAPImode,theimportedpython-xbeelibraryhandlesreceiptofamessageandconrmationthatitisaccurate.Acallbackfunctionwithinthelibraryisthenutilizedtocalltheparse stringfunction. Figure4-13. ReadloopofXBeeinATmodeforxbee bridge. IfinATmode,aROSspinloopiscreatedthathandlestheserialdatareceivedfromtheXBee.AsshowninFigure 4-13 ,ablockingcalltotheserialportisstarted.InATmode,thereisnobuiltinconrmationthattheserialmessagereceivedisaccurate.Toestablishthereceivedmessageasaccurate,itisrstcheckedforleftandrightbrackets 63

PAGE 64

torepresentacompletemessage.Then,achecksumisthencalculatedandcomparedtothelasttwocharactersoftheserialstringtoconrmthatalleldsoftheserialstringwasreceivedcorrectly.Afterconrmationthatthemessageisaccurate,parse stringfunctioniscalled. Figure4-14. FindNewSubscriber/ServiceLoopforxbee bridge. Afterthereadcallbackhasbeencongured,themainfunctionentersacongurableloopthatmonitorsthestateofsubscribersandservicesontheROSMaster,asshowninFigure 4-14 .Asnewsubscribersandservicesarecreated,theirnamesarecomparedtothoseloadedintotheparameterlistforxbee bridge.Ifanamehasapartialmatch,itisaddedtothelistofsubscribersorservicesthatxbee bridgeisconguredtolistento.Anycomponentofthetopicorservicenamethatisnotregisteredisstoredtobetransmittedwiththemessage. 4.3.2ManagementofROSMessagesOnceinitialized,xbee bridgeistaskedwithhandlingROSmessagesandservices.Figure 4-15 showstheprimarypathfordataowofROSMessagesbetweentwoxbee bridgecongureddevices.Duringinitialization,subscribersarecreatedfor 64

PAGE 65

Figure4-15. ROSmessagedataowbetweentwoXBeeBridgecongureddevices Figure4-16. Standardsubscribercallbackforxbee bridge. desiredtopics.WheneverthesubscribedmessageispublishedtotheROSdatabase,xbee bridgetriggersthestandard subscriber callback,showninFigure 4-16 .Instandard subscriber callback,thetypeofROSmessageandthedesiredtargetareusedtoformthebeginningpartsofthemessagetobesent.Next,theROSmessagedatastructureissenttothetransmit datafunctionforformatting.Figure 4-17 showsthetransmit datafunction,whichistaskedwithloopingovertheROSmessagedatastructuretoformaXBeemessage.TheXBeemessageisformedusingthebuild message stringfunction,showninFigure 4-18 .AsindividualcomponentsoftheROSmessagearesenttobuild message string,thetypeofcomponentisreviewed.Foreachcomponentreviewed,aresultingstringrepresentingthevalueofthecomponentisreturned.Ifthecomponentisasimplevariable,thevalueisconvertedintoastringandreturned.Ifthecomponentisalist,itisassumedthattherearemultiplevariablesinsidethelist,therefore,theyarerecursivelypassedintothebuild message stringfunctionuntilthelistisempty.Allreturnedvaluesoftherecursivecallsarecombinedintoasinglestringandthenreturned.Finally,if 65

PAGE 66

thecomponentisaclass,allcomponentsoftheclassarerecursivelypassedintothebuild message stringsimilarlytohowlistsarehandled.Allreturnedvaluesfromthebuild message stringfunctioncallwithintransmit dataarecombinedintoasinglestring.Nowthatamessagestringiscollected,itisformattedaccordingtothemodeforcommunicationthattheXBeeisconguredfor.IftheXBeeisinATmode,achecksumiscalculatedovertheentiremessageandisaddedtotheendofthemessagestring.Themessageischeckedaccordingtosizeandispossiblysplitintomultiplesmallerpacketsifneeded.Finally,eachmessageissurroundedbyleftandrightbracketsbeforebeingsentseriallytotheXBeedevicefortransmission.IftheXBeeisinAPImode,themessageisstructuredusingtheAPIXBeevariablesetupduringinitialization.Themessageischeckedaccordingtosizeandispossiblysplitintomultiplesmallerpacketsifneeded.EachmessageisconguredintheAPIpacketstructurefortheXBeeandthentransmitted.Onthereceivingend,xbee bridgehasalreadybeenconguredtoconrmamessageasaccurate,andthentopassthedatatotheparse stringfunction.AsshowninFigure 4-19 ,theparse stringfunctionexecutesparsingserialmessagesintoROSmessagesorservices.First,amessageissplitintocomponents:toStr,fromStr,currPacketNum,packetSize,anddata.ThedatacomponentisfurthersplitintothetagStrandthedataStr.Ifdataispartofapacket,thenitisstoredinamappedvariabletiedspecicallytoakeythatconsistsofwhothemessagewasfromandthegiventagforthemessage.Ifthedataisthelastpartofapacketorisasinglepacket,itisparsedaccordingtothetypeofmessage.IfthemessageisaROSmessage,aninitialcheckisperformedtoseeifapublisherforthetopichasbeencreatedyet,andthenthemessagedataandROSmessagevariabledatastructureissenttothestandard publisherfunction.AsshowninFigure 4-20 ,thestandard publisherfunctionsimplysplitsthereceivedmessagestringintoseparatecomponents.Next,thetagStrcomponentisusedtoextract 66

PAGE 67

Figure4-17. Transmitdataforxbee bridge. adatastructurefortheROSmessagefromthetopicdatastructurethatwasloadedduringinitialization.Thisdatastructure,knownasvar list,includesalistofallvariablesthattheparsedmessagestringshouldhave,aswellastheorderthattheyshouldbein.Alongwithvar list,relevantclassesandlistsareloadedfromthosefoundintheexpand topic infofunction.Finally,aROSmessagevariable,overheadmessage,iscreatedforthedesiredROSmessagetype.Eachofthesecomponentsaresenttothebuild ros messagefunction.Thebuild ros messagefunction,showninFigure 4-21 ,issimilarinoperationtothebuild message stringfunction.Initially,theROSbuild ros message,checkstherstcomponentofthevar listdatastructureasakey.Ifthekeyisasimplevariable,therstvalueofthemessagestringistypecastintothecorrecttypedesignatedbythekey,andthensetastheattributeoftheoverheadmessagespeciedbykey.Ifthecomponentisalist,itisassumedthat 67

PAGE 68

Figure4-18. Buildmessagestringforxbee bridge. therearemultipleitemsinsidethelistthatmaybeavariable,nestedlist,ornestedclass.Therefore,theyarerecursivelypassedintothebuild ros messagefunctionuntilthelistisempty.Atemporaryclassdenedbythetypeofthelistiscreatedtoholdvaluesfoundfromtherecursivereturnofbuild ros message.Aseachvalueisreturned,theyareappendedtothetemporaryclass,formingalist.Oncecomplete,thetemporaryclassissetastheattributedoftheoverheadmessage,justlikeindividualvariablesare.Finally,ifthekeyisaclass,allcomponentsoftheclassarerecursivelypassedintothebuild message stringsimilarlytohowlistsarehandled.OncethenewlyformedROSmessageisreturned,itispublishedtotheROSdatabase. 4.3.3ManagementofROSServicesxbee bridgeisalsotaskedwithhandlingROSservices.ShowninFigure 4-22 ,ROSservicetransmissionissimilartoROSmessages,however,thereisanaddedresponsepaththatoccurs.Duringinitialization,servicesarecreatedfordesiredtopics.Whenever 68

PAGE 69

Figure4-19. ParseStringforxbee bridge. Figure4-20. Standardpublisherforxbee bridge. 69

PAGE 70

Figure4-21. BuildROSmessageforxbee bridge. Figure4-22. ROSservicedataowbetweentwoXBeeBridgecongureddevices aROSclienttriggersarequesttoaregisteredXBeeservice,xbee bridgetriggersthestandard service callback,showninFigure 4-23 .Instandard service callback,thetypeofROSservicerequestandthedesiredtargetareusedtoformthebeginningportionsofthemessagetobesent.Next,theROSservicerequestdatastructureissenttothetransmit datafunctionforformatting.SimilartohowROSmessagesaredeveloped,transmit datastructurestheROSservicerequest,exceptwithanadditionalcharacter,thetrFlag,thatspeciesthemessageasatransmittedrequest. 70

PAGE 71

Aftertransmittingthedata,thestandard service callbackholdsforaresponse.Thecallbackaddsitselftoadatastructureofothercallbacksthatareawaitingresponsesfromclientrequestsdescribedlater.Ifarequestisnotreceivedinacertaintimeoutperiod,afailedresponseisreturned. Figure4-23. Standardservicecallbackforxbee bridge. Onthereceivingend,xbee bridgeagainbehavessimilartohowROSmessagesarereceived.Theparse stringfunctioniscalledtosplitthemessageanddecidewhattypeofmessagewasreceived.WhenthemessageisaROSservicespeciedasarequestmessagebythetrFlag,thestandard client transmitfunctioniscalled.Figure 4-24 showsthestandard client transmitfunctionbeginsinthesamewayasstandard publisher.AmessageisrststructuredintoaROSmessageusingthebuild ros messagefunction,butisnotpublished.Thepurposeofstandard client transmitistomanagetransmittingaROSclientrequestmessagetoaROSservice.Oncethereceivedclientrequesthasbeenstructured,ablockingholdcheckstoseeifthedesired 71

PAGE 72

servicehasbeenstarted.Iftheservicehasnotbeenstarted,atimeoutoccursandafailedresponseissentbacktotheoriginalcallingROSclient.Otherwise,thedataissenttotheROSservice,processed,andaresponseisreceived.Thisresponse,inturn,issenttotheoriginalrequestingXBeedevicethroughthetransmit datafunction.InthecaseofaROSserviceresponse,transmit datastructurestheROSserviceresponsemessagewiththetrFlagsetforatransmittedresponse. Figure4-24. Standardclienttransmitforxbee bridge. Duringthenallegfortransmission,theROSserviceresponseisreceivedbytheoriginaltransmittingXBee.Themessageisagainpassedtotheparse stringfunctionwhichusesthetrFlagtorecognizethemessagesasaROSserviceresponseandthencallsthestandard client responsefunction,whichisshowninFigure 4-25 .ThemessageisstructuredintoaROSmessageusingthebuild ros messagefunction.Thetypeofmessageandsourceisthencomparedtoadatastructureholdingalistofservicesthathavebeensentoutandareawaitingresponses.Ifthereceivedmessageispartofthedatastructure,itthentriggersaresponseinthestandard service callback 72

PAGE 73

Figure4-25. Standardclientresponseforxbee bridge. functionshowninFigure 4-23 .TheresponseispassedtotheoriginalROSclientandisremovedfromxbee bridge. 4.4SwarmCoreTheSwarmCore(swarm core)meta-packageservesasthecorecomponentinthearchitecturefordirectingmissionsamongthevariousagentsthatmakeuptheswarm.Themeta-packageismadeofmultiplepackagesthatdirectseveralstagesofamission.SwarmCoreismadetobecustomizable,allowingadiverseselectionofmissiontypes,planners,orvehiclecontrolapplicationstobeimplemented.Toprovidethiscustomization,themeta-packagetakesadvantageofROSscommunicationinfrastructureandpluginlibrarysystemtogiveausermodularityofwhichmethodsareutilizedtocompleteamission.SwarmCoreissetupusingahierarchicalstructure.TheRollCallpackageassistsinregistrationofnewvehiclestotheswarm.TheMissionControlandAgentControlpackagesareusedtoadministermissioncommunicationbetweenthebasestationandagentsoftheswarm.TwopluginlibrariesexistandareusedwithinAgentControltoassistingoalplanningaccordingtomissionsettingsandlowlevelvehiclecontrolper 73

PAGE 74

missionsettingsandgoals.ItispresumedthattheSwarmCoreframeworkisconguredthroughoutallagentsandatleastasinglebasestation.ThepurposeofthebasestationistoeitherassistauserinchoosingandcontrollingthemissionwiththeaidofMissionControl,ortosimplygiveauserinformationontheprogressionofthemission. 4.4.1RollCallSinceafunctionofSwarmCoreisexpandability,knowingallagentsandtheirpurposeaprioritomissionstartisnotdesired.Asnewagentscomeonline,itispreferredthattheyreporttheirpresenceandroletoallvehiclesintheswarm.RollCallisasimplepackagewithtwonodes,agent role serverandagent list publisher,taskedwithachievingthisgoal.agent role serverisintendedtorunontheagent,whileagent list publisherrunsonthebasestation.Asagentsareactivated,theyinitiatetheagent role callservicemessage,resultinginagent role serversendingtheagent rolemessagetothebasestationcontainingtheagent'sname,role,andparametersusedtodeneits'capabilities.Thisinformationisreceivedonthebasestationbytheagent list publisher,whichistaskedwithrespondingtoagent role serverviatherole acknowledgemessage,andthentopackagetheagentsinformationintoalist,agent list,whichispublishedforallagents.Thislistallowsnewagentstogetinformationonagentsthathavealreadybeenincorporatedintotheswarm,whilealsoallowingotheragentstogetinformationofthenewlyintroducedagent.Figure 4-26 showshowmessagesandtransmittedbetweennodesoftheRollCallpackage. 4.4.2MissionControlTheMissionControlpackagemanagesdisseminationofmissionsandparameterstoagents,whilehandlingoverheadcontroloftheprogressofamissionforasingleagentorallagentsatanygiventime.AllprocessingforMissionControlishandledthroughthemission controlnode.AmissionispreparedexternallyeitherthroughthebasestationGUIorthroughasimplepackagereadingacongurationlewithmissiondetails.Missionsaredenedas 74

PAGE 75

Figure4-26. ROSdataowbetweenagent role serverandagent list publisher. asetofmissionsettings,includingthetypeofmission,aswellasagivenwaypointlistorregionofinterestinwhichtowork.Missionsmaybedenedforasingleagentorfortheentiregroup.Onceamissionhasbeenreceivedandispackaged,mission controlattemptstosendittotheintendedtarget,waitingasetperiodforfeedbackfromeachtarget.Ifthetargetforthemissionistheentiregroupofagents,themissionisstoredforlateruse.Missionsincludeaminimumnumberofagentsthatmustrespondbeforeacknowledgingsuccessfultransmissionofthemission.Oncetheminimumnumberofagentshavereceivedthemission,mission controlpreparestoreceiveeithernewmissionupdatesorcommandsfromanexternaluserintheformofStart,Abort,Pause,orReturntoBase.Similartohowmissionsaretransmitted,onceacommandisreceived,thecommandispackagedandsenttoeachintendedtargetandthenawaitsaresponse.Ifsenttothegroup,thecommandisstored.Amajorfocusofmission controlistheeffortforhandlingnewagentsaddedtotheswarmafteragroupmissionorcommandissent.Whenanymissionorcommandissenttoasingleagent,itisassumedthattheinformationwasneverintendedforthegroup,soonlythatvehiclewillbeaffected.Whengroupmissionsorcommandsaresent,theyarestored.Asanewagentisaddedtothegroup,mission controlistaskedwithsendingthenewagentthelateststoredmissionandcommandsotheycanjointheongoingmission. 75

PAGE 76

Animportantfactorofmission controlismodularity.Thoughdescribedasexistingonthebasestation,mission controlcanrunonanyagentintheswarm,furtherdecentralizingthesystem.Itisintendedthatonlyasinglemission controlnodeisrunningintheentireswarmarchitecture,soaleaderexplorerpackageisusedtodenetheidentityoftheleaderoftheswarm.Oncetheleaderisdiscovered,itwillbecomethecentralpointofmissioncontrolunlessitfailsandanotherleadermustbechosen.ThedataowofROSmessagesusedinmission controlareshowninFigure 4-27 .Thefollowingsectionsdescribethefunctionalowofthemission controlnode. Figure4-27. ROSdataowformission control 4.4.2.1NodeInitializationTheinitializationofmission control,showninFigure 4-28 ,istaskedwithloadingparametersandsettingupROScommunications.First,allcommunicationsformissionparameters,themission settingsandALL/mission waypoint listROSmessages,areconguredwiththeupdateMissionCallbackandupdateWaypointsCallbackfunctions.Next,ROSservicesforsend missionandsend commandaregiventhesendMissionHandlerandthesendCommandHandlerfunctionsrespectively.Thesend missionservicetriggerspreparingmissionstobesenttoagents,whilethesend commandtriggerstheStart,Abort,Pause,orReturntoBasecommands 76

PAGE 77

tobesenttoagents.Finally,theagent listmessagefromtheRollCallpackageissubscribedwiththegroupAgentListCallback. Figure4-28. Nodeinitializationformission control 4.4.2.2BuildingListofKnownAgentsAsnewagentsareaddedtotheswarm,theyregisterthroughtheRollCallpackageandtheagent listmessageisupdatedwiththeagent'scapabilities.Themission controlnodelistensforthismessageandtriggersthegroupAgentListCallbackfunction,showninFigure 4-29 .Initially,thisfunctionhandlesupdatingalocallistofallagents.Theagent listmessageisstoredforusethroughoutthenodeandischeckedforanychangesinavailableagents.Ifanagentisnew,anewsubscriberissetuptolistenformissionwaypointliststhatpertaintothenewagentusingthemessage/mission waypoint list.Insituationswhereamissionisalreadyinprogress,thegroupAgentListCallbackfunctionistaskedwithsendinganyrelevantmissionandcommandinformationtothenewagent.ThisisperformedinthesamemannerthatthesendMissionHandlerandthesendCommandHandlerfunctionshandlemissionandcommandtransmission. 4.4.2.3MissionPreparationandTransmissionAprimaryfocusofmission controlistoformmissionsthatwillbeusedbytheswarm.Thisisperformedthroughtheassistanceofanexternalpackage.Missionsarersttransmittedthroughthemission settingsmessagetriggeringtheupdateMissionCallbackfunctionwhichsimplystoresthesettingsforlateruse.MissionsalsorequiremissionwaypointlistthatdeneseitherapathorROItobeusedforthemission.Mission 77

PAGE 78

Figure4-29. Agentlistcallbackformission control waypointlistsarereceivedforeithertheentiregroupofagentsorforindividualagentsthroughtheALL/mission waypoint listor/mission waypoint listrespectively.Wheneachlistispublishedbytheexternalpackage,theupdateWaypointsCallbackfunctioniscalledtosimplystorethewaypointlistinadatastructure.Missionsaretriggeredforpackagingandtransmissionwhenthesend missionservicetriggersthesendMissionHandlerfunction.ShowninFigure 4-30 ,sendMissionHandlerstartsbyconrmingthatatleastoneagenthasjoinedtheswarm.Next,thedesiredoperativeofthesend missionserviceischeckedagainstbeingALLagentsorbeingaspecicagent.Ifthemissionisforaspecicagent,itisconrmedthatamissionwaypointlistfortheagenthasbeenstoredbeforeamissioncanbesent.Ifthelistexists,amissionispackagedaccordingtotheparameterssetforthemissionandisthensenttotheagentthroughthe/set missionservice.Iftheagentreturnsanacknowledge,thenmissionsendisasuccessandisstoredforfutureuse,otherwiseitisreportedasafailure.IfthemissionisforALLagents,itisconrmedthatthemissionwaypointlistgivenbyALL/mission waypoint listhasbeenstored.Ifthelistexists,amissionis 78

PAGE 79

Figure4-30. Sendmissionhandlerformission control 79

PAGE 80

packagedaccordingtotheparameterssetforthemission.Next,themissionissenttoallagentswhohavecheckedinthroughRollCall.Missionsaretransmittedthroughthe/set missionservice,makingnoteofwhichagentsacknowledgereceiptornot.Onceallagentsacknowledgereceipt,missionsendisconsideredasuccessandisstoredforfutureuse. 4.4.2.4MissionCommandsAnotherfocusofmission controlismanagingthecommandstateoftheswarm.Commandsforthemissionaresendtomission controlfromanexternalpackagethroughthesend commandservicewhichtriggersthesendCommandHandlerfunction.ShowninFigure 4-31 ,sendCommandHandlerfunctionssimilarlytosendMissionHandler.First,itisconrmedthatatleastoneagenthasjoinedtheswarm.Nextthedesiredoperativeofthesend commandserviceischeckedagainstbeingALLagentsorbeingaspecicagent.Ifthecommandisforaspecicagent,itisrstconrmedthatamissionhasbeensentandreceivedbytheagent.Iftheagenthasacknowledgedamission,acommandissenttotheagentthroughthe/control missionservice.Iftheagentreturnsanacknowledgeforthecommandtransmission,thencommandsendisasuccess,otherwiseitisreportedasafailure.IfthecommandisforALLagents,itisrstconrmedthatamissionhasbeensenttoALLagents.IfALLagentshaveacknowledgedreceiptofamission,acommandistransmittedtoeachagentthroughthe/control missionservice.Ifeachagentacknowledgesthatthecommandwasreceived,thecommandsendisasuccessandthecurrentcommandisstoredforfutureuse. 4.4.3AgentControlContrastingtotheMissionControlpackage,theAgentControlpackagehasthepurposeofreceivingmissionsandcommandsthataretransmittedfromMissionControl.Thepackagemanagesparsingamissionintospecicsrequiredtodevelopaplanfor 80

PAGE 81

Figure4-31. Sendcommandhandlerformission control 81

PAGE 82

theagent,whilecategorizingcommandstomodifythemissionstateofthevehicleforwhenaplanhasbeenfound.AgentControlalsohandleshighlevelmissionrelatedcommunicationwithagentsthatjointhegroupandaredescribedonagentlist.AllprocessingfortheAgentControlpackageishandledwithintheagent controlnode.Asagent controlistaskedtohandlemissiongoalplanningaswellasdirectvehiclecontrol,separatelibrariesarecreatedtohandleeachcomponent.Itisintendedthatagent controlbecustomizable,therefore,insteadofusinglibrariesdirectly,theROSpluginlibpackageisutilizedtodynamicallyloadcustomdenedlibrariesthathandletherequiredtasks.TheBaseGoalPlannerandBaseVehicleControlpackagesincludedwithintheSwarmCoremeta-packageprovidestandardlibrariesforusewithagent control.C++interfacesthatdenewhatthebase goal plannerlibraryandthebase vehicle controllibrarymustdoaredenedintheSwarmCorepackage.Customlibrariesmaybemadeaslongastheseinterfacesarefollowed.AlsodenedintheSwarmCorepackageistheAgentclass,whichisusedtoholdparametersanddeneinformationaboutaspecicAgent.Thisclassisusedthroughoutthecustomlibrariestoholdtherequiredinformationaboutallagents. Figure4-32. ROSdataowforagent control 82

PAGE 83

Figure 4-32 showstheROSdataowfortheagent controlnode.Thefollowingparagraphsdiscussthefunctionalprogressionoftheagent controlnodeformissionmanagement. 4.4.3.1NodeInitializationAtstartup,theagent controlnodeusestheROSparametersystemtoloadvariousparametersneededfordirectvehiclecontrolandplanning.ROSSubscribersareinitializedtocommunicatewithlocalagent status,agent list,andtheglobalheartbeatmessages.Twoservicesarestartedtolistenfortheset missionandcontrol missionservicecommands.Asagent controlistaskedtohandlemissiongoalplanningaswellasdirectvehiclecontrol,separatelibrariesarecreatedtohandleeachcomponent.Itisintendedthatagent controlbecustomizable,therefore,insteadofusinglibrariesdirectly,theROSLibraryPluginpackageisutilizedtoloadcustomdenedlibrariesthathandletherequiredtasks.Todenewhichlibrarytouse,thebase goal plannerandbase vehicle controlstringparametersareloadedwiththelibrariesnameonstartup.ThenameiscomparedtoallsharedlibrarieswithinthenamespaceoftheSwarmCorepackage.Ifthenameresultsinanactiveclass,itisloadedtoaclassvariableforlateruse. 4.4.3.2FlowofAgentStateMachineAftertheinitialstart-upofthepackage,agent controlentersthemissionstatemachine.IntheInitializationstate,agent controlistaskedtotransmititsroleandcapabilitiestotheswarm.Toperformthisassignment,theagent roll callservicecommandissenttotriggertheagent role servernode.Ifagent role serverrespondswithanacknowledgeresponse,agent controlmovestotheMissionWaitstate,however,afailedtransmissionwillresultinmultiplerepeatattempts.Oneofthepreloadedparametersdenesatimeoutthatwillbeusedtoendrepeatattempts.Ifagent controlisunabletotransmit 83

PAGE 84

therequiredinformation,thestatemachineissettotheMissionFailedstatesinceinitialcommunicationwiththeswarmisveryimportant.WhileintheMissionWaitstate,agent controlholdsuntiltheset missionservicecommandisreceived.Duringreceiptofthemessage,themissiondataisstoredandthestateischangedtoPlanMission.BeforethePlanMissionstatewillprogress,theagent listmessagemusthavebeenreceivedandparsedatleastonce.Thismessageshouldbereceivedfromthesubscribersetupduringstartup.Eachtimeanewagentisaddedtothegroup,thismessageiscalledtoparsetheagent listandsplitagentsintomissiongroupsstructuredintheAgentclass.Amissiongroupisdenedasagroupofagentstaskedwiththesamerole(e.g.,search,identify,etc.).Oncedatahasbeenreceivedabouttheswarm,thePlanMissionstateisfurthertaskedtohandlediscoveryofagoalplanfortheagent.Goalsareonlydenedwhenanewmissionisreceivedoranewagenthasbeenaddedtothemissiongroup.Calculatingthedesiredplanforanagentisperformedaccordingtothemissionchosen,themissionparametersgiven,andtheroleoftheagent(whichmissiongroupitbelongsto).Therearethreetypesofdenedmissionsaswellasthreetypesofmissiongroupsforagent control.Searchmissionsassumethattheoutcomeforvehiclesistoperformasearchoveraregionofinterestwhilemaximizingcoverage.TasksformissiongroupsperformingmissionsaredenedinTable 4-6 .PathmissionsaremuchsimplerthanSearchmissions.APathmissionistypicallysenttoasinglevehicleandistreatedasamethodofoverridinganycurrentmissionandgivingavehicleadirectpathtotravel.Thismaybeperformedduringtestingorduringmissionsifaoperatordesiresadirectsearchpath.RandommissionsaresimilartoSearchmissionsinregardstothedesireforsearchanarea.TheprimarydifferenceissearchvehiclesinaRandommissionarenotgivenagoallistasitisassumedtheywilltraveltheenvironmentrandomly.TrackandIdentifyvehiclesfollowthesameplanningbehaviorperformedinSearchmissions.Ifplanningfortheagentisunsuccessful,the 84

PAGE 85

Table4-6. TaskTypesforAgentsPerformingMissions TaskTypeDescription SearchEveryagentsinformationandtheregionofinterestforthesearcharesenttothebase goal plannerclassasdenedintheinterface.TheregionsofinterestisassumedtobedenedasanareaofGPSlocationsgiveninthemissionsettings.Itisexpectedthatthebase goal plannerwillreturnaseriesofGPSpointstobeusedasthegoallisttotraveltheenvironmentandcompletecoverage.Ifforanyreasonthatthegoalplannercouldnotgeneratealist,itisassumedthatthemissionfailsandthestateissettoMissionFail.IftheGoallistisfound,sendittoallagentsinthegrouptoreachaconsensus.TrackAsimplemethodofdetectingwhichagentshouldbetrackedisusedtogettheagentsGPSlocation.Thesinglelocationissentasthegoallist.IdentifyAnemptygoallistissent.Identifyvehiclesaregivengoalsdynamicallyasthemissionprogresses. stateissettoMissionFail.Ifplanningissuccessful,thebase vehicle controlclassisinitializedwiththeplan,missiondetails,andagentdetailsbeforemovingtotheCommandWaitstate.WhileintheCommandWaitstate,agent controlholdsuntilthecontrol missionservicecommandisreceived.TheonlycommandacceptedatthismomentistheRuncommand,whichwillresultinthecurrentmissionplansettothebase vehicle controlclassbeingstarted.Oncethemissionhasbegun,agent controlwillentertheMissionRunningstate.Whilethemissionisrunning,agent controlissettolistentotheresultsofthemissioninprogressinthebase vehicle controlclass.Ifthemissioniscomplete,agent controlmovestotheMissionCompletestate,however,ifthereareanyerrorsorfailures,agent controlmovestotheMissionFailstate.Duringthecurrentstate,othercommandssentthroughthecontrol missionservicecommandarealsowatched.IfthePause,Abort,orReturntoBasecommandsareeverreceived,thenagent controlwillmovetotherespectivestates. 85

PAGE 86

Whenamissionisnishedandagent controlenterstheMissionCompletestate,thesystemrstcheckstoseeifanyfollow-upmissionsarerequired.ThisspecicallyhappensifanewplanisrequiredforthegivenroleoftheAgenttocompletetheoverallmission.Ifaplanisrequired,allsettingsarecollectedandthestateissettothePlanMissionstate.Ifnoplanisrequired,thestateissettoReturntoBase,wherethebase vehicle controlclassissenttheReturntoBasecommandandisassumedtobetravelingbacktothebasestation.Previously,theMissionPauseandMissionAbortstateswerementioned.WhileintheMissionPausestate,thebase vehicle controlclassissentthePausecommand.Typically,pauseshouldresultinthevehicletakingprecautionstondastationarypositionandwaitforneworders.WhileintheMissionAbortstate,base vehicle controlclassissenttheAbortcommand,whichisassumedtoforcethevehicletostopimmediateandsafely.Abortisseenasahardfailsafetoendthemissionincaseaoperatorseesanissue. Figure4-33. NodeInitializationforagent control 86

PAGE 87

Figure4-34. Flowofthestatetimerforagent control 87

PAGE 88

Figure4-34. Flowofthestatetimerforagent control(continued) 88

PAGE 89

Figure4-34. Flowofthestatetimerforagent control(continued) 89

PAGE 90

Figure4-34. Flowofthestatetimerforagent control(continued) 90

PAGE 91

Figure4-34. Flowofthestatetimerforagent control(continued) 4.4.4SwarmBaseWithintheSwarmBasepackage,twoC++interfaceswillbefoundthatdenetwopluginlibrariesthatshouldbeusedwithintheAgentControlPackage.Therstinterface,base goal planner,denesfunctionsneededforthepluginlibrarytaskedwithcalculatingthegoalsneededforanagenttocompleteamission.Theinterfacedescribesafunction,makePlan,thattakesargumentsfortheknownlistofagents,the 91

PAGE 92

Figure4-35. Agentlistsubscriberforagent control Figure4-36. Localagentstatussubscriberforagent control 92

PAGE 93

Figure4-37. Heartbeatsubscriberforagent control Figure4-38. SetMissionserviceforagent control 93

PAGE 94

Figure4-39. ControlMissionserviceforagent control Figure4-40. Groupagentstatussubscriberforagent control 94

PAGE 95

regionofinterestpolygonthatdenetheoutsideboundaryofamissionaccordingtomissionsettings,andnallyareferencetothelistofpointstobereturnedasthegoal.Anotherfunctiondescribedinthefunction,replan,usesthepreviousdenedboundary,buttakesargumentsforthenewcurrentlocationofagentsandareferencetoanewlistofpointstobereturnedasthegoal.Thesecondinterface,base vehicle control,denesfunctionsneededforthepluginlibrarytaskedwithcontroloveraspecicagent.Thislibraryisspecialized,requiringdirectknowledgeofthetypeofvehicle,thecontrollerthatisemployed,andthetypeofstatusmessagesitsends.TherstfunctiondescribedintheinterfaceisthesetPlanfunction,whichtakesalistofpointsdeninggoalsforthevehiclesmissionaswellasalistofknownagentsintheswarm.Anotherfunction,isMissionComplete,isarequesttoseeifallgoalsofthecurrentagentsmissionhavebeenmet,resultinginaBooleanresponse.ThenalseriesoffunctionssimplypassexpectedcommandtypestothepluginlibraryintheformofStart,Pause,Abort,andReturntoBase.Itisuptotheimplementeroftheclasstodecidewhetherthispluginshouldcontrolthevehicledirectly,orjustusetheinterfacetoinstigatealowerlevelcontrollersuchasthatfoundintheNavigationpackage.AnalcomponentoftheSwarmBasepackageistheAgentclass.Thisisasimple,centraldenitionofwhatagentsare.TheclassdenescharacteristicsofagentsandservesasalocationtostorerelevantdataaboutagentsthatmaybeneededthroughouttheSwarmNavigationmeta-package.Exampleofsomedataincludecoveragerate,fuelstatus,andfuelcost,whichareusedforplanninggoalsfortheagent. 4.4.5BaseGoalPlanerAsdiscussedintheAgentControlpackagesection,theagent controlnodeexpectsthattwopluginlibrariesarecreatedusingtheinterfacesdenedintheSwarmBasepackage.TheBaseGoalPlannerdenesonesuchpluginlibrarythatisusedwithintheagent controlnode.Thispluginlibraryistaskedwiththecreationofasearch 95

PAGE 96

planaccordingtoagivensetofagentsandaknownregionofinterest(ROI)thatdenesanoutersearchboundary.TheBaseGoalPlannerpackagegeneratesthebase goal plannerlibrary.AlistofagentsisdenedusingtheAgentclassdescribedintheSwarmBasepackagesection.Informationsuchascoveragerate,fuelstatus,fuelcost,andstartpositionaresenttothebase goal planneralongwiththesearchboundary.Usingthesearguments,asearchpatternisfoundforeachagent,includingthelocalagentinwhichtheagent controlnodeiscurrentlyrunningon.ThedesiredsearchpatterntypeisdenedusingROSparameters.Thebase goal planneriscapableofgeneratingacrossinglawnmower(agentsoverlap),separatelawnmowerpathsperagent(nooverlap),oraspiralsearchpatternperagent.ExamplesmaybefoundinFigure 4-41 A BFigure4-41. Goalpathsfoundfromthebase goal planner:A)CrossinglawnmowerpathformultipleagentsB)Spiralpathforsingleagent Itisassumedthatbase goal plannerdesiresapathwhichresultsintheagentmakingitbacktothestartposition.Duringcalculationofthepaths,fuelusedismonitoredsoapointofnoreturnmaybefoundandusedforendingthemission.Itisalsopossibletoforceare-planofthemissionifdesiredfromagent control. 96

PAGE 97

Itisimportanttonotethateachagentwillplantheirownpathandanexpectedpathforeachagent.Thefollowingsectiondetailsthemethodologyanagentusestocalculatethedesiredpaths. 4.4.5.1CongurationofGoalPlanner Figure4-42. InitializationandPlanningforpluginlibrarygoal planner Thebase goal plannerisrstinitializedduringitscreationintheagent controlnode.AsshowninFigure 4-42 ,initializationrstconguresdesiredsettingsaswellasthetypeofpathgenerationtobeusedforthemission.Afterinitialization,thegoalplannerisstartedwhenplanningmissionsthatrequirealistofgoalsforpreparingasearch.AseriesofGPSpointsintheshapeofaconvexpolygonissentasaROI,aswellasalistofAgents,theircapabilities,andalistofmissionparameters.AllGPSpoints,includingtheinitialpositionsofeachagentisconvertedfromGPSLatitudeandLongitudevaluestoUniversalTransverseMercator(UTM)xandypoints.Afterthe 97

PAGE 98

conversion,thenearestsideandcorneroftheROIpolygontoallagentsischosenasthestartingpointsforcalculatingthesearchpath. Figure4-43. PlanPathsfunctionforgoal planner Witheachagentsinformation,theconvertedROIpolygon,andthestartingsideofthepolygon,theplanPathsfunctioniscalled.ShowninFigure 4-43 ,initiallythelistofagentsissortedinorderofwhichagentisclosestothestartingcornerofthepolygon.Next,thecollectivecoverageareaforeachagentiscalculated,aswellastheareaof 98

PAGE 99

coveragebeforetheagentthatisplanningthepath.Finally,thetypeofplantobefoundischeckedandperformed.IfaCrossingLawnmowerpatternischosen,allinformationregardingthemissionandagentsispasseddirectiontothemethodforgeneratingthepattern.IfaSplitLawnmowerorSpiralpatternischosen,theROIpolygonisrstsplitintosectionspervehiclebasedontheirfuelandsearchcapabilities.Afterbeingsplit,themethodloopsovereachsectionwhileplanningthedesiredpatternperagentinorderofclosesttothestartingcornerpoint.Onceacompletegoalplanhasbeencalculated,itisreturnedtothecallingfunctioninagent control. 4.4.5.2CalculatingDifferentGoalPathTypesTherstpossiblepatternisthecrossinglawnmowerpath,describedinFigure 4-44 .Thispatternfocusesoncreatingapathforeachagent,assumingthattheyeachstartneareachotherandwilltakethenextpathinorderfromclosesttothestartingcorneroftheROIpolygon.Initially,apathisdesiredtostartfromthenearestcornerofallagentsandforeachagenttoentertheROIthroughthenearestsidethateachagentisalong.Fromhere,thedesiredpathwillmovealongthenextclosestsideoftheROI.Thisorientationisdenedasmovingfromthebottomsegment,alongthesidesegment,andthentothetopsegment.ThebottomsegmentwillalwaysbetheROIsidethatthepathismovingawayfrom,whilethetopsegmentwillalwaysbetheROIsidethepathismovingtowards.ThesidesegmentisdenedinitiallyastheROIsidethepathmovesalong,butislatershownaspathsthathavebeengeneratedandarenowusedasreferenceswhencreatingnewpaths.Calculatingapathissplitintotwotypesofpoints:thecrossoverandupward/downwardpoint.Thecrossoverpointisusedtogenerateapathasitcrossesfromtheendofonepathtothebeginningofanotherpath.Theupward/downwardpointdenestheendofapaththatbeganatacrossoverpoint.Tobegintheprocessofcalculatingthedesiredpoints,therststepistocalculatethedirectionofthestartingindiceswhichisusedto 99

PAGE 100

Figure4-44. Containertocollectpointsofacrossinglawnmowerpathingoal planner 100

PAGE 101

Figure4-44. Containertocollectpointsofacrossinglawnmowerpathingoal planner(continued) 101

PAGE 102

knowwhatdirectionwithintheROIthepathwilltravel.Thisisusedtocalculatetheinitialbottom,top,andsidesegments. Figure4-45. Methodforcalculatingacrossinglawnmowerpointingoal planner 102

PAGE 103

Figure4-45. Methodforcalculatingacrossinglawnmowerpointingoal planner(continued) Onceallneededdataiscollected,theinitialpathgoaliscalculatedgivenasetdistanceawayfromthestartingcorneralongthebottomsegment.Sincethisistherstpoint,itistreatedasacrossoverpointfromthesidesegmentandisfoundusingthemethoddescribedinFigure 4-45 .Themethodisusedforcalculatinganycrossoverpointsneededtogeneratethepath.Thismethodrstconrmsthattheindicesdonotoverlaporgenerateapaththatwouldgonowhere.Nextapointiscalculatedalongthebottomsegmentaway 103

PAGE 104

fromthesidesegment.Thiscrossoverpoint,aswellasallothercrossoverpointsandupward/downwardpointsfoundalongatoporbottomsegment,isfoundthroughthesueofanalytic-geometry.First,thesidesegmentisdenedasvectorUwhilethetoporbottomsegmentisdenedasvectorV.VectorUisgivenastheequationofalineax+by+c=0 (4)(y1)]TJ /F8 11.955 Tf 11.96 0 Td[(y2)x+(x2)]TJ /F8 11.955 Tf 11.95 0 Td[(x1)y+(x1y2)]TJ /F8 11.955 Tf 11.96 0 Td[(x2y1)=0 (4)where(x1,y1)and(x2,y2)aredenedbytheendpointsofthesidesegment.Visdenedinparametrictermsx(t)=mt+x0 (4)y(t)=nt+y0 (4)wherepointP0isdenedattheintersectionofvectorsUandV,or(x0,y0),andisdenedasavectorparalleltoit.isthenormalizedresultoftheendpointandstartpointofVsubtracted.Thiscreatesthedesireddistancevectorthatwilllaterbeused.Givenadesireddistance,orcoveragecapability,betweenapointP=(x,y)andU,Equation 4 isdened,wheretrepresentsthedistancealongVtoachievethedesireddistanced. jax(t)+by(t)+cj p a2+b2=d(4)Withthegivenassumptions,theequationissimpliedtoEquation 4 104

PAGE 105

j(am+bn)tj p a2+b2=d(4)andsolvedfortinEquation 4 t=dp a2+b2 am+bn(4) Figure4-46. AnalyticGeometrymodelusedforplanningpaths Figure4-47. Seriesofstagesthatleadtondafullsequenceforalawnmowerpath 105

PAGE 106

Onlythepositiveresultoftisneeded.Oncetisfound,itismultipliedbythedistancevectortondthepointP=(x,y)alongV.Inthemethodbeingdescribed,thispointisdenedasthecrossoverpoint.AreferencefortheaboveequationsisgiveninFigure 4-46 .Figure 4-47 providesanexampleoftherstfewstageswhenndingalawnmowerpath,whileTable 4-7 providesinformationonhowUandP0isdened. Table4-7. DescriptionofstagesforalawnmowerpathfromFigure 4-47 StageVectorUPointP0 I(start)-(start-1)startII(start-1)-(start-2)start-2III(start)-(start-1)startIV(start-1)-(start-2)start-2 ThenextstepinthemethodistoconrmthatthepointliesalongthesegmentdenedbyV(toporbottomsegment).Ifthepointisalongthesegment,itispushedtothepathlistandplanningiscontinuestolookforthenextpoint.Ifthepointisnotonthesegment,thedistancebetweenthesidesegmentandthenextsegmentalongtheROIiscalculated.Ifthedistanceisgreaterthantheagentscoveragerange,thecornerpointisaddedtothegoallist.ThesegmentbeingtestedisshiftedtothenextsegmentinorderalongtheROIandisconrmedtobeanewsegment.Ifitis,newboundarypointsarecollectedandplanningcontinues,otherwise,planningisnished.WhencheckingthedistancebetweenthesidesegmentandthenextsegmentalongtheROI,ifthedistanceislessthanthecoveragearea,thelinesegmentshouldbeshiftedtothenextsegment.Again,thenewsegmentshouldbeconrmedtobeanewsegmentandthenewboundarypointscalculated.Finally,ifthenewcornerisnotalreadyonthelist,additandcontinueplanning.Ifthenewcornerisonthelist,runthemethodagainandattempttondacrossoverpointalongthenewsegment.Withtherstpointfound,themethodreturnstondtherestofthecrossinglawnmowerpoints,againshowninFigure 4-44 .Thesecondpointissimilartotherstasitistreatedasaninitialcrossoverpointfromthesidesegment.Duetothis,itisfound 106

PAGE 107

usingthesamecrossoverlawnmowerpointfunction.Thismethodisrepeatedoverthenumberofagentsincludedinthesearchmission,eachusingthepreviousagentspathasthenewsidesegment.ThisprocessgenerateseachagentsinitialupwardpaththroughtheROI.Thesmallchangeisnowthatabeginningpathhasbeenfound,eachnewagentpathusesthemethodforcalculatinganupward/downwardlawnmowerpoint. Figure4-48. Methodforcalculatinganup/downlawnmowerpointingoal planner Therearetwoprimarydifferencesbetweenthecrossoverlawnmowerpointmethodandtheupward/downwardlawnmowerpointmethodshowninFigure 4-48 .Firstly, 107

PAGE 108

Figure4-48. Methodforcalculatinganup/downlawnmowerpointingoal planner(continued) 108

PAGE 109

whencalculatingthenewpoint,thesidesegmentchosenisthesameasthatusedforcalculatingthepreviouscrossoverpoint,however,thedirectionofthesidesegmentisreversed.Thenextdifferenceiswhetheranoptionforthepointtobecalculatedattheedgeorbeoffset.Ifoffset,atemporarydirectionvectoralongthepathbeingtestediscalculated.Thenewpointisshiftedbackwardsalongthedirectionvectoratthegivenoffsetdistance.Thispointorthepointwithouttheoffsetifnotrequestedischeckedinthesamemannerasthecrossoverlawnmowerpoint.Afteraninitialupwardpathhasbeenfoundforeachagent,aloopisperformed,lookingforthecrossoveranddownwardpathforeachagent,followedbythecrossoverandupwardpathforeachagent.ThisprocessiscontinueduntilthelastpossiblepathfortheROIisfound,whichthentriggersallpathstobereturned.Alongwiththecrossinglawnmowerpattern,thesplitlawnmowerpathattemptstondalawnmowerpatternforeachagent.Theprimarydifferenceisthatthepolygonissplitintosmallerpolygonsbeforehandandthelawnmowerpatternbeingfoundisforasingleagent.ThissingularapproachisshowninFigure 4-49 .Thenalpatternthatmaybefoundisthespiralpath.Thismethodonlygeneratesspiralpathsperagentusingsmallerpolygonregions.ThemethodforndingaspiralpathisdescribedinFigure 4-50 .Similartothelawnmowerpattern,therststepistocalculatethedirectionofthestartingindiceswhichisusedtoknowwhatdirectionwithintheROIthepathwilltravel.Thisisusedtocalculatetheinitialbottom,top,andsidesegments.Next,theROIpolygonischeckedtoseeifthepointsaredenedinaclockwiseorcounter-clockwisefashion.Afterward,indicesfortheROIareorderedinthedirectionthatthespiralwillbeformed.Nowthatallrequiredinformationisfound,theinitialspiralpointisfoundalongtherstsidesegmentusingthemethodshowninFigure 4-51 .First,theboundarypointsforthesideandtopsegmentsisfound.Next,calculatingthespiralpointusesthesame 109

PAGE 110

Figure4-49. Containertocollectpointsofasplitlawnmowerpathingoal planner 110

PAGE 111

Figure4-50. Containertocollectpointsofaspiralpathingoal planner 111

PAGE 112

Figure4-51. Methodforcalculatingarstspiralpointingoal planner analyticgeometrymethoddescribedforthelawnmowerpathinEquations 4 through 4 .Thepointisconrmedbeinthedesiredboundariesandisaddedtothepath.Aftertheinitialspiralpointisfound,anotherpointisfoundtonishouttherstpath.ThispointusesthemethoddescribedinFigure 4-52 .Initially,itisconrmedthattheoveralllengthofthetopsegmentisgreaterthanthecoveragearea,otherwiseplanningcontinueswiththenextsegment.Anewpointiscalculatedandconrmedtobeintheboundaryofthetopsegment.Ifwithintheboundary,adirectionvectoriscalculatedforthecurrentpathsegmentbeingprocessed.Then,anewpointisfoundbytravelingthedistanceoftheagentscoverageratebackwardsalongthepathawayfromthesegment.Iftheshiftedpointisonthecorrectsideofthebottomsegment,itisaddedtothepathandplanningcontinues.Ifthepointbeingcheckedwasnotoriginallywithintheboundaryspace,thedistancebetweenthesidesegmentandtopsegmentiscalculated.Ifthereisroom,anewpoint 112

PAGE 113

Figure4-52. Methodforcalculatingaspiralpointingoal planner 113

PAGE 114

Figure4-52. Methodforcalculatingaspiralpointingoal planner(continued) basedonthecorneroftheROIischosenandconrmedtobeacceptable.Ifthepointisnotacceptable,planningisnished.Ifthereisnodistancebetweenthesidesegmentandthetopsegment,planningisalsonished.Aftertherstpathforthespiralpatternisfound,theseriesofindicesisrotatedtolookatthenextsequenceofsegmentsandcalculateanewpathbesidetheROI.Thiscontinuesuntilthelastsegment.Fromthispoint,theseriesofindicesisrotated,however,insteadoflookingattheROI,thenewindicespointtotheinnermostpathused 114

PAGE 115

togeneratethespiral.Inthismanner,thespiralcontinuouslyclosesinonthecentroidoftheROI.SeeFigure 4-53 foranexampleoftherststagesincreatingaspiralpattern. Figure4-53. Seriesofstagesthatleadtondafullsequenceforaspiralpath 4.4.6BaseVehicleControlAsdiscussedintheAgentControlpackagesection,theagent controlnodeexpectsthattwopluginlibrariesarecreatedusingtheinterfacesdenedintheSwarmBasepackage.TheVehicleControlpackagedenesonesuchpluginlibrarythatisusedwithintheagent controlnode.Thispluginlibraryistaskedwiththecontroloftheagentduringamissionusingdirectvehicleinteraction.TheprimaryvehicletypeusedinthedenedswarmarevehiclesthatusetheAPMautopilotcontroller,therefore,thedescribedpluginlibraryisdedicatedtonodesmeantspecicallyfortheAPM.TheVehicleControlPackagecreatesmultipleROSnodesthatareusedtocontrolbothairandgroundvehiclestaskedwithvariousmissiontypes.ThesenodeswilleitherdenecommandstobedirectlyprocessedbytheVehicleDriverpackage,orwilldenegoalstobeusedandperformedwithinanothervehicleplanningpackagesuchastheNavigationpackage. 4.5AgentCapabilitiesToallowtheagentsaseriesofcapabilitiesthroughouttheswarmarchitecture,variousbehaviorswerewrittenusingfunctionsinvolvingLIDARandVision. 115

PAGE 116

4.5.1LIDARGroundagentsusetheHokuyoURG-04LX-UG01LIDARfordetectingobstacleswithintheenvironment.Obstacledataiscollectedeithertobetransmittedintotheswarmforothervehiclestouse,orforobstacleavoidancewhiletravelingthroughtheenvironment. 4.5.1.1ObstacleDetectionObstacledetectionisperformedinmultiplestages.First,dataisreceivedfromtheLIDARandtransformedtoaPointCloud.ThetransformationtakesintoaccounttheorientationtransformbetweenthevehicleandtheanglethattheLIDARislookingoutintotheenvironment.Nextthedataissearchedtoextractblocksofpointsthatrepresentanobstacle.Toreducetheamountofinformationrequiredtorepresentanobject,eachcollectionofpointsrepresentingablockispassedthroughaloopinglterthatusesEquation 4 tocomparepointsandseeiftheyarecollinear.Agiventhresholdmaybemodiedtoallowbetterdetectioninanosierenvironment.Obstaclesarereducedtoendpointsthatrepresentthelinesthatmakeuptheobject. j(y1)]TJ /F8 11.955 Tf 11.96 0 Td[(y2)(x1)]TJ /F8 11.955 Tf 11.96 0 Td[(x3))]TJ /F9 11.955 Tf 11.95 0 Td[((y1)]TJ /F8 11.955 Tf 11.96 0 Td[(y3)(x1)]TJ /F8 11.955 Tf 11.95 0 Td[(x2)j<=threshold(4)Afterdetectingobjects,thecurrentorientationofallendpointsistransformedinrelationtothevehiclesheadingrelativetothecardinalcoordinatesystem.TheGPSpositionofthevehicleisconvertedtoUTMandthevehicle'spositionisaddedtoeachendpointtocreateanaccurateUTMpositionforeachendpoint.Endpointsarethenstoredtobereviewedonsubsequentscans.Whenanewscanisreceived,allpointsareconvertedagain,andthencomparedtopreviouscollectionsofpoints.Segmentsarecomparedtoprevioussegmentsusingfourcombinationstoseeiftheyarecollinear.Thecombinationsaredenedfortwosegments(p1,q1)and(p2,q2)intermsofcheckingcollinearitybetween(p1,q1,p2),(p1,q1,q2),(p2,q2,p1),and(p2,q2,q1). 116

PAGE 117

Asobstaclesaredenedandnolongerseen,theirendpointsareconvertedfromUTMtoGPSpositionsandtransmittedtovehiclestouse. 4.5.1.2ObstacleAvoidanceDuringtheprocessofdetectingobstacles,aLIDARscanisusedtoalsocheckforobjectswithinathreshold.Asobjectsaredetected,aPIDcontrollerisusedtonavigatearoundtheobstacle. 4.5.2VisionVisionisusedonbothgroundandairvehiclestolocatepossibletargetsandidentifythem.AnumberofUAVsusevisiontoidentifyaUGVandattempttofollowitusingducialmarkertracking.Duringthisprocess,aUAVmayattempttolocatepossibletargetstobeavoidedoridentied.UGVsusevisionwhenclassifyinganobjectassomethingthatmustberescuedorreviewedbyauser. 4.5.2.1FiducialMarkerTracking A BFigure4-54. FiducialMarkerImageandVisualizationWithintheROSEnvironment:A)FiducialARMarkerB)ROSVisualizationofFiducialMarkerPositionandOrientation UAVstrackUGVsintheenvironmentwiththeaidofaducialmarkertrackingsystem.WiththeassistanceofALVAR,anopensourceaugmentedreality(AR)tag 117

PAGE 118

trackinglibrary,aducialmarker,asshowninFigure 4-54A ,iscreatedwithaspecicnumberrepresentation.EachUGVisthengiventhenumberedARtag.Beforeprocessingmaybecompletedon-boardanUAV,camerasarecalibratedusingaclassicalblack-whitecheckerboardpatterntodetectthedistortionmatrixandcameramatrix.Thedistortionmatrixisusedfortherecticationprocessoftakingthecamera`andoutputtingaclearerimage.Thecameramatrixisusedtoholdthefocallengthsandopticalcenterofthecamerathatmaybeusedincalculatingpositionofobjectsinanimage.WhenanUAVisstarted,itwilllinkupandtraveltoaknownpositionofaspecicUGV.Oncethelinkismade,notothervehiclewilllinkupbetweenthetwothatmadetheoriginallink.WhentheUAVhasreachedthedesiredlocation,thecameraimage,alongwiththedistortionandcameramatrices,arepassedtotheARtaglibrarytolocateandextracttheARtag.3DpositionandorientationforthetagislocatedandtransmittedbackthetheUAV.Usingthisinformation,theUAVthenattemptstofollowthevehicleintheenvironmentusingasimplePIDcontrollerthatcontrolsrollandpitch.ForARtagtracking,couplingheadingtotheUGVisnotrequiredasthecameraisdownward-facing.Theknownwidthandcalculatedlocationoftheobjectisusedasareferenceduringobstacledetection. 4.5.2.2ObstacleDetectionWhileUAVsaretravelingthroughtheenvironment,theyareconstantlyattemptingtolocateobstaclesortargetswithintheenvironment.Asthesetargetsarelocated,theirestimatedpositioniscalculatedwithrespecttotheARtag'slocationandcurrentsizeonthescreen.SincetheARtag'sphysicalsizeisaknownparameterduringthemission,thenumberofpixelsthatmakeupthetagareusedtoestimatethedistancebetweenobjectsgiventhenumberofpixelsandangletotheobject. 118

PAGE 119

CHAPTER5FIELDTESTINGRESULTS,CONCLUSIONS,ANDFUTUREWORK 5.1TestingandResultsThefollowingsectiondetailsvariousassumptionsmadefortestingaswellastheresultsoftesting. 5.1.1AssumptionsTherewereafewassumptionsmadeforthecooperativesystemtofunctionasintended.First,theoutdoorenvironmentisassumedtobeanopeneldwithnohighstructuresthatwillcausecollisionissueswiththeUAVs.TestsareperformedinacompletelyopeneldastofocusmoreongroupbehaviorsversusUAVobstacleavoidance,whichisanissuebeingsolvedbyotherworksandisnotafocusofthisresearch.Furthertestingisdesiredinamoreclutteredenvironment,inwhichtreesandotherobstaclesmayobstructaUAVsviewoftheground,butwillbeperformedinfollowupworks.Anotherassumptionisthatobstaclesintheeldofoperationwillbecontrolled.Thevisionusedintestingdoesnotfollowcomputervisionorpatternrecognitiontechniquestoperceivecomplexobjects,butinsteadisusedtodetectobjectsofspecicshapesandcolors.Thesedifferentobjectswillpertaintoeitherobstaclesortargetsofinterestdependingonthetechniqueused.Alongwiththeobstaclesintheeld,specialpatternsandcolorsknownasARtagsareusedfortrackingtheagentstoassistinlocalization. 5.1.2SystemsTestingBeforeenvironmenttesting,variouscapabilitytestswereperformedonthecomponentsofthearchitecture.Communicationsinvolvedtheabilityofthexbee bridgetohandleanytypeoftopicorservicethatROScouldgeneratewithinthearchitecture.Testinginvolvedgeneratingmultiplemessagesinternallyandexternallytobetransmittedinasingledirectionbetweenvehicleswhilereviewinglossofdata.Next,variousserviceswerealsocreatedtotestthesystemsabilitytosendaservicerequestfromone 119

PAGE 120

vehicletoanothervehiclethatwouldhandletherequest,processit,andtransmittheresponsebacktothevehicle. A BFigure5-1. SingleVehicleLawnmowerPlanandCoverageRange:A)SingleVehicleLawnmowerPlanB)CoverageRangeofaSingleVehicleLawnmowerPlan 120

PAGE 121

AsstatedinChapter 4 ,thexbee bridgehandlesconguringcommunicationsthroughtheuseofacongurationle.Testingoriginallyfocusedonwhetherthesemessagesorservicesweregeneratedautomaticallyandcorrectlyifthecongurationlewasproperlycongured.Testsalsoconrmedtherobustnessandspeedatwhichmessageswereconguredandtransmitted,recognizingthespeedatwhichtheyaregeneratedwithinROSandthentransmittedandreceivedcorrectly.Eachofthesetestsshowedthatthexbee bridgebehavedasintendedandhandledallcommunicationsasrequired. Figure5-2. PlannedCrossingLawnmowerSearchPatternforThreeAgentsinaMulti-sidedPolygon SystemtestingalsoconrmedthattheBaseGoalPlannerpackagecreatedthevariouslawnmowerandspiralpatternsasdesired.Variousvehicleswereaddedtotheplannerwithvaryingsearchrangesandcapabilitiestotesthowaplanwasgeneratedbetweensingleandmultiplevehicles.Initially,theabilitytogenerateaplanandhavecompleteornearcompletecoveragewasconrmed.Figure 5-1A demonstratesasimpledesiredlawnmowerplanforasinglevehicle,whileFigure 5-1B providestheexpectedcoveragerateofthevehiclewhileperformingthemission. 121

PAGE 122

Multiplevehiclesweregivensimulatedpositionsfromarealenvironmentwithactualsearchrangeandcapabilitiestogeneratethepossibletypesofpathsthatmaybegenerated.Figure 5-2 showsthreevehiclesthatstartnearthetopleftcornerofamulti-sidedpolygonandpreparetoperformacrossinglawnmowersearch.Eachvehiclehasadifferentsearchcapability;fromshortesttolargestrange,theorderagent1,agent3,andagent2.Itcanbeseenasgoalsaregeneratedthatvariouspathsbetweenvehiclesmaybeveryclosetogetherorfurtherapartduetothecapabilityoftheagent.Goalscrossovereachotherasthevehicleisexpectedtotraversetheenvironment,whilepayingspecialattentiontooccasionallydoublecoveracornertoattemptonehundredpercentcoverage. Figure5-3. PlannedSplitLawnmowerSearchPatternforThreeAgentsinaSimplePolygon Twoothersearchpathsdemonstratethesplitlawnmowerandspiralpatterns,asshowninFigure 5-3 andFigure 5-4 respectively.Eachpatternisextractedfromasinglepolygonthatissplitintothreesmallerpolygonsbasedonaratiodenedbyeachagentscapability.Agentsareshowntohavevaryingrangesofsearch,inorderfromhighesttolowest,agent1,agent2,andagent3.Allthreevehiclesstartatthetopleftofthesearch 122

PAGE 123

Figure5-4. PlannedSplitSpiralSearchPatternforThreeAgentsinaSimplePolygon area,and,therefore,beginthereplannedsearchatthetopleftoftheirdesignatedsub-region. Figure5-5. PlannedCrossingLawnmowerPatternforTwoUGVsandOneUAVwithintheSwarmArchitectureGUI 123

PAGE 124

AnexampleofamissionplancreatedwiththeswarmarchitectureGUIisshowninFigure 5-5 .Theimageshowsaplannedmissionusingtwogroundvehicles(bluedashedlines)andoneairvehicle(greenline)preparingtoperformacrossinglawnmowerpattern.ThenalsystemstestincludedmultipleiterationsoftestingLIDARandvisioncomponentsinvaryingsituationswithchangingstructuresincontrolledanduncontrolledlightingsituations.Occasionaltestingwasperformedwithinamulti-vehiclesimulationwithinacustomconguredPlayer-StagesimulationandamasterslaveROSsetup.EachtestconrmedthesameresultsasdiscussedinChapter 4 inwhicheachsystemwasabletodetectthedesiredresultsasneeded.Beyondsimplylocatingobjectswithvision,thevisiontrackingsystemwastestedbetweenanUAVtrackinganUGV.TheUAVvehiclewassetupwiththetrackbehaviorandtheUGVwassetupwiththesearchbehavior.Atmissionstart,theUGVwouldbegivenanROIandplanasearchmission.Whenthemissionwasstarted,theUGVwouldholdpositionuntilthelinkedUAVsendsacommandforittobeginthedesiredsearch.TheUAVwouldholdpositionuntiltheGPSlocationoftheUGVwasreceived,triggeringtheUAVtolaunchandtraveltotheUGV'slocation.Whenatthedesiredlocation,theUAVwouldusevisiontolocatetheARtagontopoftheUGVwhiletellingtheUGVtostartit'smission.Thetwovehiclesnavigatedthroughtheenvironment,theUAVfollowingtheUGV. 5.1.3FieldTestingAftertheprimarycomponentsofthearchitecturewerefullytestedinacontrolledmanner,multipleeldtestswereperformedtovieweachbehaviorinarealenvironment.Earlytestinginvolvedvariousissuesinregardstocontrollabilityofvehicleswhenperformingrealbehaviors.ForUGVs,theworstcasefailuresinvolvedagroundvehicleattemptingtoleavethesearchareaathighspeeds,forUAVs,however,worstcasefailurestypicallyincludedincreasingaltitudedrasticallyuntiltakenunderremotecontrol 124

PAGE 125

(RC)byasafetyyer,ordecreasingaltitudequicklyuntilthevehiclecrashed.Afterchangingsomecongurationswiththecontrolofthevehicles,aswellasenablingandturningthethresholdsoffail-safesthatwerecreatedatthebeginningofthedesignforthearchitecture,safetyofthevehicleswasincreased.Testinginvolvedasinglelaptopactingasthebasestation.TheswarmarchitectureGUIwasrunningonthelaptopwhichalsoranthemission control,roll call,xbee bridgenodes.CommunicationswerehandledviaXBeedeviceandantenna.BothUAVsandUGVsranthevariousvehiclecontrollers,agent control,roll call,xbee bridge,vision,andlidarnodes.Withinagentcontrol,therequiredmissionplannerandbehaviorpluginswereimplementedtohandlethevehiclesvariousrolesbasedonthedesiredmission.EarliertestingdidoccuronWi-Finetworkswherethebasestationwasthemasterandthevehiclesweretheslave,however,forprimarytesting,theXBee'sandmulti-mastersystemwereused.AlltestingoccurredattheUniversityofFloridainanopeneldnamedFlavetField,GPSpositionlatitude:29.6464592,longitude:-82.3543872. 5.1.4InitialArchitectureTestingDuringtheinitialtestsUGVsandUAVsseparatelytogetabaselineforcontrol.ThreeUGVsweregivensearchpatternstonavigatethrougharegionofinterest.Thevariousfail-safesandcommandsoftheGUIweretestedtoconrmthattheUGVsrespondedasexpectedwhilethemissionwasunderway,includingre-planningwhenrequired.AsimilarmissionwasperformedwithtwoUAVsthatemployed3Dnavigationandalargercapablesearcharea.Afollowuptestcombinedbothgroundandairvehicles.TwoUGVsandoneUAVwerecombinedtocreatetheswarm,eachsetwithsearchroles.UGVswereconguredwithsearchrangesofvemeterswhileUAVswereconguredwithsearchrangesoftenmeters;eachrangedictatedbytheabilitiesoftheLIDARandvisioncamerasystemsrespectively.Figure 5-6 showsapictureoftwoUGVsandoneUAVbeforethetestbegins.Figures 5-7 5-8 ,and 5-9 showthreeseparatephasesofthemission.Therst 125

PAGE 126

Figure5-6. OneUAVandTwoUGVsBeingUsedinFieldTesting Figure5-7. MissionStartShowingROIandPlannedSearchPathsforEachAgent 126

PAGE 127

Figure5-8. MissionAfterCompletingOneLegofPlannedSearchPaths Figure5-9. MissionCompleteShowingAgentsatFinalGoalPointinSearchPattern 127

PAGE 128

gurefocusesonthedisplayingtheROIandtheplannedgoalpathforthetwoUGVs(bluedashedline)andUAV(greendashedline).Thesecondgureistakenofmissionprogressonthebasestationaftereachvehiclehascompletedtherstlegofthesearchandaretravelingtothenextleg.Thenalguredisplayedthatresultsofthevehiclesfollowingtheplannedpathandreachingthenalgoalontheirlist.Afterthistestwasrunmultipletimestoconrmconsistency,thecentralizedstructurewastradedforadecentralizedsetup.AsdescribedinChapter4,thecodewaswritteninamethodwheremissioncontrolandthebasestationisirrelevantofwhereitisrun.ThemainmissioncontrolandbasestationcodewasmovedtoanUAV,whiletheoriginallaptopactingasabasestationwasonlyusedasasatellitestationwherethemissionROIwasstillchosenandresultsweredisplayed.Thismissionwastestedandconcludedwiththesameresultsasallprevioustests.Theprimarydifferencesnoticedwereaslightlyslowerruntimeandasmalllossofdataaround10%;eachexpectedduetoallXBeecommunicationsrunningthroughasinglevehiclecomputerwhichislesscapablethanthelaptopcomputer. 5.1.5CompleteSystemTestThecompleteswarmarchitecturetestinvolvedacombinationofthreeUGVsandtwoUAVs.AROIwaschosenatthetestinggrounds(FlavetField)usingtheswarmarchitectureGUI.TwoUGVsandasingleUAVperformedsearchbehaviors,dividingtheROIintomultiplepathsbasedoneachvehiclessearchcapabilities.ThesecondUAVwasgivenatrackbehaviorandconguredtofollowandassisttheclosestUGV.ThenalUGVwasgiventheidentifybehavior,whichwastaskedwithtravelingtotargetslocatedbythevehiclesduringtheirsearchmissions.ObstaclesweredenedasorangecircularbucketsintheROI,whiletargetsweredenedaspinkboxes.Figure 5-10 showstheinitialsetupoftheeldandthevehicles.AsshowninFigure 5-11 ,themissionwasconguredwithaspecicROIfortheeld.Theplannedpathsforthesearchvehicleswerefoundanddisplayedwithinthe 128

PAGE 129

A BFigure5-10. InitialSetupforFieldandVehiclesDuringCompleteSystemTests:A)FieldSetupB)VehicleSetup 129

PAGE 130

Figure5-11. InitialMissionSetupforCompleteSystemTests Figure5-12. MissionResultforCompleteSystemTestsDetectingAllobjects 130

PAGE 131

GUI.Themissionwastestedvarioustimeswithdifferentsettings.Duringmissions,eachsearchvehiclebegantravelingthroughtheenvironment.BoththeUGVsandUAVsfollowedthecalculatedpathascloselyaspossible,however,duetoenvironmentalissuesincludinghighwindsandheavycloudcovercausinglowGPSsignals,theUAVoccasionallystrayedfromthesearchpath.Duringtherstexperiments,UGVsandUAVsweretaskedwithlocatingpinkboxes.UGVsusedLIDARtolocateboxesbyextractingthelinesthatmakeupthebox.UAVsusedvisiontolocatethepinktopsoftheboxeswhileyingoverthem.Asthepinktargetswerelocated,theUGVwiththeidentifybehaviorwastaskedtotraveltothetargetandattempttoidentifyitwithvision.Aftertheseexperimentswerecompleted,detectingtheorangeobstacleswasaddedtotherequirementstodistinguishdetectionofthetwotypesofobjects.UGVsdidhaveissuesoccasionallydetectingthecircularbuckets,whiletheUAVsdetectedtheobjectsandprovidedallvehicleswiththeobstacledata.UGVswereabletousetheobstacleinformationtonavigatearoundtheobstaclesintheenvironment.ThenalresultsofthesearchmissionforallobstaclesisshowninFigure 5-12 5.1.6SeparateGroupTestingDuetoarelatedproject,anothergroupattheUniversityofFloridatookadvantageoftheworkdescribedinthisresearch.Theprojectcalledfortwoquadrotorstoworktogethertocompleteatask.Therstquadrotor,actingasasearchvehicle,wouldtravelthroughanenvironmentlookingforamarkerrepresentingatarget,inthiscasealargepinkball.ThetestwasinitiatedbychoosingaROIforthesearchareawithintheswarmarchitectureGUI.Fromhere,aspiralsearchpatternwaschosenusingtheBaseGoalPlannerpackageandgiventothequadrotor.Duringthesearch,assoonasthemarkerwaslocated,theGPSlocationoftheballwascalculatedandtransmittedtoanotherquadrotoractingasanidentifyvehicle.Withthelocationofthemarkerfound,thesecondquadrotorlaunchedandtraveledtowards 131

PAGE 132

themarkerastherstquadrotorreturnedtoitslaunchposition.Thesecondquadrotorlocatedthemarker,droppedapayloadatthelocation,andthenreturnedbacktoitslaunchposition.Thoughtheprojectdidnotspecicallydemonstratetheheterogeneouscapabilityoftheswarmarchitecture,itdoesshowtheoverallcapability.Fortheirproject,allthatwassimplyneededwasavisionfunction,twoslightlymodiedbehaviorsfortheirvehicles,andanextracongurationlineforthexbee bridge.TheprojectcouldhavealsoeasilytakenadvantageofanUGVtotraveltothemarkerpositioninsteadofanUAV,orforbothtoworktogetherintandem. 5.2ConclusionsTheworkintroducedinthisdissertationistherstofitskindintermsofdevelopmentforanarchitectureforacooperativeheterogeneoussystemofvehicles.ThesystemdescribeddenesaheterogeneoussetofplatformstobeusedinfurtherresearchfortheMachineIntelligenceLab(MIL).Variouscontributionshavebeenmadeintheareaofcontrolandnavigationofmultiplegroundandairvehiclesusingcentralizedanddecentralizedcontrolschemesthroughtheuseofacentralarchitecture.Theresearchenablesuserstoquicklyandefcientlycreategoalplannersandvehiclebehaviorsthatmaybeusedbetweenvarioustypesofvehiclesforcontrolandcompletionofmissions.ThebasesystemforthearchitectureiscapableofrunningonabasestationsimilartoacomputerusingtheprovidedGUI,thereforecreatingacentralizedcontrolsituation,orthebasesystemmayrunonavehicleintheswarm,allowingtheentirecontrolsystemtobedecentralized.AnothercontributionincludesaROSbasedmulti-mastercommunicationsystemusingRFcommunicationsbetweenagentsoftheswarm.Thesystemdescribedincludedacustomcommunicationprotocolthatwasusedtodecreasetheamountofdatasent,handleerrorsindatatransmitted,andhandledirectionoftransmissiontoallorspecicvehiclesintheswarm.Thecodebehindthecommunicationsystemwas 132

PAGE 133

optimizedtoquicklytakemessagesorservicesfromwithinROSandeitherconvertthemorrecreatethemrecursively.Thecodewasalsodynamic,allowinguserstoquicklychangeoraddnewmessagestructurestocommunicationsthroughtheuseofasimplecongurationle,removingtheneedtorecodethepackage.Furthercontributionsadvancedmethodsintaskallocationandpathplanningformultiplevehicleswhenprovidedaconvexdenedpolygonsearcharea.Usingvehiclecapabilitiesandmissionparameters,searchpathsforeachvehiclemaybegenerated.Finally,amethodforallowingairvehiclestofollowandtrackgroundvehiclesusingducialimagemethodsaswellaslocatingobjectsintheenvironmentisprovided.Eachoftheseadvancementshasbeendescribedwithinthegivenarchitectureandshowntofunctionthroughvariouseldtestsperformedonmultipleplatformvehicles.EachoftheabovecontributionshavebeenproducedwithastandardBerkeleySoftwareDistribution(BSD)licenseandareprovidedonlinewiththeopen-sourceROScommunityforfurtheruseanddevelopments. 5.3FutureWorkTheworkpresentedinthispaperdemonstratesthegroundworkfordevelopingafullycapableswarmenvironmentbetweenvaryingtypesofgroundandairvehiclesoperatinginarealenvironment.Theworkalsoprovidesthebeginningsofvariousbehaviorsthatwillbeveryusefulinaddingfurthercapabilitiestoeachvehicle,however,therearevariousideasthatwereconceivedbeforetheresearchorduring.First,furtheradvancementsmaybemadeinthecapabilitiesoftheswarminregardstonavigationandcontrolofthevehicles.Duringtheresearchdenedinthispaper,someworkwasstartedonintroducingsquadbasedcontrolalongwithlocalWi-Fimulti-masternetworkswhileusingRFforlongdistance.Iftheswarmstartedwithgreaternumbersitwouldinitiallybesplitintosmallergroupswithaleaderpergroup.Planningfortheswarmwouldthensplittheeldintoseparatepartspersquad.InternalcommunicationswouldbehandledbetweeneachotherwithinaMobileAdHocNetwork 133

PAGE 134

(MANET).Communicationstoothersquadsorabasestationwouldbecollatedandferriedbetweensquadleaders.Muchofthisinfrastructurewasdevelopedbetweenthesmallerhomogeneousswarmworkbutwasnotincludedintheheterogeneousswarmarchitecture.Next,workwasperformedongraphtheorybasednavigationandSimultaneousLocalizationandMapping(SLAM)forgroundvehiclesbasedonAckermannsteering.Startingwithanunknownenvironment,UGVsusedLIDARtomapobstaclesortargetsastheytravelthroughtheenvironment.Asothervehiclesmaytravelthroughthisenvironment,theymayuseaglobalplannerwithgainedknowledgeoftheenvironmenttoplanapathusinggraphtheorybasednavigation.ThenalocalplannerwouldbeusedtoassistinnavigatingtheenvironmentthroughtheaidofLIDARbasedobstacleavoidance.MethodsreviewingthebenetsofaA*orDijkstra'salgorithmsforgeneratingbetterpathsthroughanenvironmentorplanninggoalswasreviewedduringresearch.Thisworkmaybecontinuedtofurtherdevelopthecapabilitiesofthevehiclesused.OneofthemaincomponentsthatwasnotabletobeaddedinvolvedplanningforvehicleswiththeBaseGoalPlannerpackageandcouldbeafocusforfuturework.Planningcurrentlyisonlypossiblewithinaconvexdenedpolygon,however,itwouldbeveryusefultoallowtheusertodeneaplanasaconcavepolygon.Anotherfocuscouldbeplacedonthetypeofmissionstobeperformed.TimecouldbespentonfurtherimplementingtheAreaSurveymissionthatwasusedontheoriginalhomogeneousswarmworkdescribedinChapter 3 .AperimetermaintenancemissionfrompreviousworkinvolvingmultipleUGVsandUAVscouldbeusedtoformanequidistantperimeteraroundadesiredcentroid,providingtheentireouterperimeterwithcoverage.Asthecentroidoftheperimetertraveled,thevehicleswouldfollow.Theseideascaneasilybeimplementedthroughaddedbehaviorsthatworkwithintheswarmarchitecture. 134

PAGE 135

Analfocusoffutureworkcouldfocusontheindividualvehiclecapabilitiesusedinthiswork.VisionwasusedontheUAVstofollowtheUGVsthroughouttheenvironmentwhilealsoattemptingtodetectobstaclesintheenvironment.Moreadvancedmethodsmaybeusedtomoreaccuratelyestimatethepositionofobjectsintheenvironment.Insteadofidentifyingtargetsthataresimpleshapesandcolors,bettermethodsinpatternrecognitionmaybeusedtodetecthumanshapedobjectsforabettersearchandrescuesystem. 135

PAGE 136

APPENDIXADYNAMICSOFAQUADROTOR FigureA-1. Dynamicsystemmodelandreferenceframe. Thoughthemultirotorsusedinthisresearchalreadyincludeanautopilotthathandledthecontrolsforstabilization,thedynamicsofaquadrotorwasexploredforanimprovedunderstandingbeforedevelopingPIDcontrollersusedforhighlevelnavigation.Thedynamicmodelforaquadrotormaybedenedintworeferenceframes.TherstframeisintermsoftheEarthFixedFrame(inertialframe)withENU(East-North-Up)coordinatesystem.ThesecondframeisaBodyFixedFrame(bodyframe)withNED(North-East-Down)coordinatesystem.Thebodyframematchesthexedframe'soriginonthebodyofthequadcopteratthesamepointasthecenterofmassforthevehicle.AnexampleoftheframesetupcanalsobeseeninFigure A-1 .Todescribethedynamicsofthe6degreeoffreedomvehicle,variousstatesofthesystemmaybedescribed.Fortheabilitytotrackandknowthelocationofthevehicleinrelationshiptosomeoriginpoint,thepositionofthecenterofmassofthebodyframeintheinertialframeri,orthe(x,y,z)point,andthelinearvelocitycomponentsinthebodyframevb,or(u,v,w),mustbedened.Fortheabilitytorealizethevehiclesattitudeandorientationaboutallpointsofrotationatagivenpoint,theorientationofthebodyintheinertialframe,orEuleranglesroll(),pitch(),andyaw( ),alongwiththeangular 136

PAGE 137

rotationsinthebodyframewb,or(p,q,r)mustbedened.Thecomponentsandstatesarepresentedas ri=8>>>><>>>>:xiyizi9>>>>=>>>>;andvb=8>>>><>>>>:uvw9>>>>=>>>>;=8>>>><>>>>:_xb_yb_zb9>>>>=>>>>;=8>>>><>>>>: 9>>>>=>>>>;andwb=8>>>><>>>>:pqr9>>>>=>>>>;=8>>>><>>>>:___ 9>>>>=>>>>;x=rivbwbx=xyz uvwpqrT (A)Thecontrolsforthesystemmaybedenedbyfourseparateinputs(T1,T2,T3,T4)whichmaybedescribedinfourdifferentcombinations. Collectivethrustforthesystem. u1=(T1+T2+T3+T4)(A) Rollorthemomentaboutthexaxisofthesystemdependentonthespeedsofthesecondandfourthprop(disdistancefromproptocenterofmass). u2=d(T4)]TJ /F8 11.955 Tf 11.96 0 Td[(T2)(A) Pitchorthemomentabouttheyaxisofthesystemdependentonthespeedsoftherstandthirdprop(disdistancefromproptocenterofmass). u3=d(T1)]TJ /F8 11.955 Tf 11.96 0 Td[(T3)(A) Yaworthemomentabouttheyaxisofthesystemdependentonthespeedsofallprops. u4=ktm(T1)]TJ /F8 11.955 Tf 11.95 0 Td[(T2+T3)]TJ /F8 11.955 Tf 11.95 0 Td[(T4)(A)Thefollowingequationswerecreatedwithreferenceto[ 36 ]and[ 37 ].Kinematics 137

PAGE 138

ThreerotationsbasedonthegeometricallocationsforshowninFigure A-1 canbeusedtomovecomponentsfromtheinertialframetothebodyframebygeneratingacoordinatetransformationmatrix. R()=2666641000cs0)]TJ /F8 11.955 Tf 9.3 0 Td[(sc377775R()=266664c0)]TJ /F8 11.955 Tf 9.29 0 Td[(s010s0c377775R( )=266664c s 0)]TJ /F8 11.955 Tf 9.3 0 Td[(s c 0001377775S=R()R()R( )S=266664cc cs )]TJ /F8 11.955 Tf 9.29 0 Td[(sssc )]TJ /F8 11.955 Tf 11.95 0 Td[(cs cc +sss sccsc +ss css )]TJ /F8 11.955 Tf 11.95 0 Td[(sc cc377775 (A)ThecomponentsofthebodyframemaybecharacterizedintheinertialframeusingEquation A ,orviceversausingatransposeofEquation A .8>>>><>>>>:xbybzb9>>>>=>>>>;=S8>>>><>>>>:xiyizi9>>>>=>>>>;ri=8>>>><>>>>:xiyizi9>>>>=>>>>;=ST8>>>><>>>>:xbybzb9>>>>=>>>>; (A) 138

PAGE 139

Equation A alsodemonstratesthatifvelocitycomponentsinthebodyframeVbareknown,thenvelocitycomponentsintheinertialframemayalsobefound.8>>>><>>>>:_xi_yi_zi9>>>>=>>>>;=ST8>>>><>>>>:uvw9>>>>=>>>>;8>>>><>>>>:_xi_yi_zi9>>>>=>>>>;=266664cc (ssc )]TJ /F8 11.955 Tf 11.95 0 Td[(cs )(csc +ss )cs (sss +cc )(css )]TJ /F8 11.955 Tf 11.95 0 Td[(sc ))]TJ /F8 11.955 Tf 9.29 0 Td[(ssccc377775266664uvw377775 (A)EulerAnglesEquation A maybeintegratedtondthepositionofthebodyframeininertialspace,however,valuesfor,,and mustbeknown.ThesevalueschangeovertimeanddenetheEulerrates_,_,and_ whichdependontheangularbodyratesp,q,andr.Thedifferentratesarerelatedthroughrotationsshownas8>>>><>>>>:pqr9>>>>=>>>>;=R()266664_00377775+R()R()2666640_0377775+R()R()R( )26666400_ 3777758>>>><>>>>:pqr9>>>>=>>>>;=26666410)]TJ /F8 11.955 Tf 9.3 0 Td[(s0ccs0)]TJ /F8 11.955 Tf 9.3 0 Td[(scc3777758>>>><>>>>:___ 9>>>>=>>>>;8>>>><>>>>:___ 9>>>>=>>>>;=2666641stanctan0c)]TJ /F8 11.955 Tf 9.29 0 Td[(s0sseccsec3777758>>>><>>>>:pqr9>>>>=>>>>; (A)LinearandAngularMomentum 139

PAGE 140

ForceandmomentsappliedtothevehiclerevealthefollowingXF=mvb dt+m(wbvb) (A)XM=Iwb dt+(wbIvb) (A)NotingthatPF=Fg+Fprop,FgandFpropmustbefound.Giventheorientationofthevehicle,gravityisdenedasbeinginthezdirection,thereforegravitationalforces(Fg)maybetranslatedasFg=mS00)]TJ /F8 11.955 Tf 9.3 0 Td[(gTFg=mg)]TJ /F8 11.955 Tf 9.3 0 Td[(sscccT (A)ForFpropthevehicleisassumedtobeinahoveringstateandthereforeresultsinforcesonlyactinginthezdirectionofthevehicle.Thisforceiscalculatedfromthethrustforcesandresultsin Fprop=00)]TJ /F8 11.955 Tf 9.29 0 Td[(u1T(A)GivenacombinationofEquations A and A intoEquation A ,alongwithareorderingofterms,theresultingvectoroflinearaccelerationsmaybefoundasvb dt=1 mXF)]TJ /F9 11.955 Tf 11.95 0 Td[((wbvb)8>>>><>>>>:_u_v_w9>>>>=>>>>;=g266664)]TJ /F8 11.955 Tf 9.3 0 Td[(ssccc377775+1 m26666400)]TJ /F8 11.955 Tf 9.3 0 Td[(u1377775)]TJ /F12 11.955 Tf 11.95 38.38 Td[(266664qw)]TJ /F8 11.955 Tf 11.95 0 Td[(rvru)]TJ /F8 11.955 Tf 11.96 0 Td[(pwpv)]TJ /F8 11.955 Tf 11.95 0 Td[(qu377775 (A)wherethevariablesT1,T2,T3,andT4representthethrustforcesgeneratedfromthepropsofthevehicle,andthe_u,_v,_wrepresenttheaccelerationcomponentsforthebody.Lookingatthemomentofinertiaforasymmetric,rigidbodywhichisalignedwiththecenterofmass,thefollowingInertiamatrixisfound 140

PAGE 141

I=266664I11000I22000I33377775(A)RevisitingEquation A withtheinertiamatrixfromEquation A showsXM=Iwb dt+(wbIvb)XM=266664I11_pI22_qI33_r377775+266664pqr377775266664I11pI22qI33r377775XM=266664I11_pI22_qI33_r377775+266664qI33r)]TJ /F8 11.955 Tf 11.95 0 Td[(rI22qrI11p)]TJ /F8 11.955 Tf 11.95 0 Td[(pI33rpI22q)]TJ /F8 11.955 Tf 11.96 0 Td[(qI11p377775 (A)SolvingEquation A for_p,_q,and_rresultsin 266664_p_q_r377775=266664Mx I11My I22Mz I33377775)]TJ /F12 11.955 Tf 11.95 38.38 Td[(266664qr(I33)]TJ /F11 7.97 Tf 6.58 0 Td[(I22) I11rp(I11)]TJ /F11 7.97 Tf 6.59 0 Td[(I33) I22pq(I22)]TJ /F11 7.97 Tf 6.59 0 Td[(I11) I33377775(A)TondthemomentsofthebodyframedenedinFigure A-1 ,thesumofallmomentsaboutroll,pitch,andyawmustdetermined.ThesemomentsweredenedastheinputstooursystemandaredescribedinEquations A A ,and A .Mx=u2=d(T4)]TJ /F8 11.955 Tf 11.96 0 Td[(T2) (A)My=u3=d(T3)]TJ /F8 11.955 Tf 11.96 0 Td[(T1) (A)Mz=u4=ktm(T1)]TJ /F8 11.955 Tf 11.96 0 Td[(T2+T3)]TJ /F8 11.955 Tf 11.95 0 Td[(T4) (A) 141

PAGE 142

RevisitingEquation A withthevariousmomentequations,thevaluesfor_p,_q,and_rmaybefound.CalculationofInertialEquationsThemomentofinertiaequations[ 36 ]maybeusedtocalculatethetermsfromEquation A .I12=1 12mm(l2y+l2z)I13=1 12mm(l2y+l2z)+mmd2I11=2I12+2I13 (A)I21=1 12mm(l2x+l2z)I23=1 12mm(l2x+l2z)+mmd2I22=2I21+2I23 (A)I31=1 12mm(l2x+l2y)+mmd2I32=1 12mm(l2x+l2y)+mmd2I33=2I31+2I32 (A)StateSpaceModelusingEquationsofMotionGiventhestatevectordescribedinEquation A andtheinputvector u=T1T2T3T4Tthefollowinglinearsystemmaybedetermined_x(t)=Ax(t)+Bu(t)y(t)=Cx(t)+Du(t) 142

PAGE 143

Where_x=_x_y_z___ _u_v_w_p_q_rT_x=uvwpqr_u_v_w_p_q_rT (A) (A)TheequationsofmotiondenedfromEquations A A A A providethefollowingequationsforthesystem(wherecandsrepresentcosandsin) f(x,u)=8>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:_x=(cc )u+(ssc )]TJ /F8 11.955 Tf 11.96 0 Td[(cs )v+(csc +ss )w_y=(cs )u+(sss +cc )v+(css )]TJ /F8 11.955 Tf 11.96 0 Td[(sc )w_z=()]TJ /F8 11.955 Tf 9.3 0 Td[(s)u+(sc)v+(cc)w_=p+(stan)q+(ctan)r_=(c)q)]TJ /F9 11.955 Tf 11.95 0 Td[((s)r_ =(ssec)q+(csec)r_u=)]TJ /F8 11.955 Tf 9.3 0 Td[(gs)]TJ /F9 11.955 Tf 11.96 0 Td[((qw)]TJ /F8 11.955 Tf 11.95 0 Td[(rv)_v=g(sc))]TJ /F9 11.955 Tf 11.96 0 Td[((ru)]TJ /F8 11.955 Tf 11.96 0 Td[(pw)_w=g(cc))]TJ /F10 7.97 Tf 14.45 4.7 Td[(1 m(u1))]TJ /F9 11.955 Tf 11.96 0 Td[((pv)]TJ /F8 11.955 Tf 11.95 0 Td[(qu)_p=)]TJ /F8 11.955 Tf 10.49 8.09 Td[(qr(I33)]TJ /F8 11.955 Tf 11.95 0 Td[(I22) I11+u2 I11_q=)]TJ /F8 11.955 Tf 10.49 8.09 Td[(rp(I11)]TJ /F8 11.955 Tf 11.96 0 Td[(I33) I22+u3 I22_r=)]TJ /F8 11.955 Tf 10.49 8.08 Td[(pq(I22)]TJ /F8 11.955 Tf 11.96 0 Td[(I11) I33+u4 I33(A) 143

PAGE 144

APPENDIXBCODESAMPLESPythonCodeSamplefromxbee bridgenode. def expand msg(msg,classes,lists): slots=msg. slots types=msg. slot types for slot,typ in itertools.izip(slots,types): value= getattr (msg,slot) split=typ.split( / ) if isinstance (value, list ): lists.append(slot) if len (split)>1: # See if class is a list if split[1].find( [] )!=)]TJ /F4 9.963 Tf 8.12 0 Td[(1: split[1]=split[1][:)]TJ /F4 9.963 Tf 9.27 0 Td[(2] lists.append(slot) module= getattr (import module(split[0]+ + msg ),split[1]) classes[slot]=module tempclasses,templists=expand msg(module, dict (),[]) classes= dict (classes,tempclasses) lists+=templists return (classes,lists) C++CodeSamplefromgoal planner int calculateVerticalLawnmowerPoint( double myCoverage, double otherCoverage, const Eigen::Vector2d&sidePoint1, 144

PAGE 145

const Eigen::Vector2d&sidePoint2, const Point2DVector&coordinates, Eigen::Vector4d&boundary, Eigen::Vector2i¤tIndex, Eigen::Vector2iotherIndex, Eigen::Vector2d&point, Point2DVector&pointList, int direction,)f Eigen::Vector2dtestPoint; Eigen::Vector2dnextPoint; Eigen::Vector2ddirectionVector(0,0); Eigen::Vector2dcurrentDirectionVector(0,0); double directionDistance=0; // // Attempt to calculate next point for lawnmower path loop if // needed // while ( true )f // If indices overlap end plan if ((sidePoint1==coordinates[currentIndex[0]])&& (sidePoint2==coordinates[currentIndex[1]]))f return false ; g calculateDirectionVectorAndDistance(sidePoint1, sidePoint2,coordinates[currentIndex[START]], coordinates[currentIndex[END]],directionVector, myCoverage+otherCoverage,directionDistance); // // Calculate next point to be tested // x ( t ) = m t + x0 145

PAGE 146

// y ( t ) = n t + y0 // testPoint.x()=directionVector[START]directionDistance+ sidePoint1.x(); testPoint.y()=directionVector[END]directionDistance+ sidePoint1.y(); // Offset from border if (!edgeSearch)f currentDirectionVector=nextPoint)]TJ /F4 9.963 Tf 14.81 0 Td[(pointList[pointList.size())]TJ /F4 9.963 Tf 8.6 0 Td[(1]; currentDirectionVector=currentDirectionVector/ currentDirectionVector.norm() ; nextPoint.x()=currentDirectionVector[START])]TJ /F4 9.963 Tf 18.69 0 Td[(borderOffset+ nextPoint.x(); nextPoint.y()=currentDirectionVector[START])]TJ /F4 9.963 Tf 18.69 0 Td[(borderOffset+ nextPoint.y(); g else f nextPoint=testPoint; g ROS INFO( Point to check (% f % f ) ,nextPoint.x(),nextPoint.y()); // // Check point to see if it is valid : // If the calculated point is within the final point on the line push // to list // else if the small chance that the final point == the last point on // line just end // if (((almostEqualUlps(boundary[LEFT],boundary[RIGHT],maxUlps)&& almostEqualUlps(boundary[LEFT],nextPoint.x(),maxUlps))jj 146

PAGE 147

(nextPoint.x()>=boundary[LEFT]&& nextPoint.x()<=boundary[RIGHT]))&& ((almostEqualUlps(boundary[BOTTOM],boundary[TOP],maxUlps)&& almostEqualUlps(boundary[TOP],nextPoint.y(),maxUlps))jj (nextPoint.y()>=boundary[BOTTOM]&& nextPoint.y()<=boundary[TOP]))) f // // Push point to list and return // ROS INFO( Point is within line segment ); point=nextPoint; pointList.push back(nextPoint); return true ; g else if ((nextPoint.x()boundary[RIGHT])jj (nextPoint.y()boundary[TOP]))f // TODO Review and see if it is best to only add the final point if // coverage has not happened // hint : this should be done when taking angle of entry into account ROS INFO( Point is outside line segment ); ROS INFO( Start Point (% f % f ) End Point (% f % f ) sidePoint1.x(),sidePoint1.y(), coordinates[currentIndex[END]].x(), coordinates[currentIndex[END]].y()); ROS INFO( Distance between is % f ,(sidePoint1)]TJ ET 1 G 1 g 0 G 0 g 1 G 1 g q 0 -593.57 468 19.79 re f Q 0 G 0 g 1 G 1 g 0 G 0 g BT /F4 9.963 Tf 90.85 -587.64 Td[(coordinates[currentIndex[END]]).norm()); ROS INFO( My Range is % f ,myCoverage); ROS INFO( Previous Range is % f ,otherCoverage); 147

PAGE 148

// // Since point is not on segment check distance to next segment // If less than horizontal Coverage )]TJ ET 0 G 0 g 0 0.6 0 RG 0 0.6 0 rg 0 G 0 g 0 0.6 0 RG 0 0.6 0 rg BT /F4 9.963 Tf 263.84 -53.42 Td[(agent coverage end point // should be added for this agent to cover final hole // else begin checking next line segment // if (((sidePoint1)]TJ /F4 9.963 Tf 14.32 0 Td[(coordinates[currentIndex[END]]).norm())> (otherCoverage))f ROS INFO( Agent should still handle searching this area ); // // Push end point to list // point=coordinates[currentIndex[END]]; pointList.push back(coordinates[currentIndex[END]]); // // Shift to next line segment // // Shift to next indices ROS INFO( Find New Indices ); ROS INFO( Before Top (% d % d ) Bottom (% d % d ) ,currentIndex[START], currentIndex[END],otherIndex[START],otherIndex[END]); calculateNextIndices(currentIndex,direction,coordinates.size())]TJ /F4 9.963 Tf 8.01 0 Td[(1); ROS INFO( Before Top (% d % d ) Bottom (% d % d ) ,currentIndex[START], currentIndex[END],otherIndex[START],otherIndex[END]); // // Check if new points segments are valid for planning // If line segments overlap end planning // Else if Corner End point endices match and either corner point // of new line segment and distance to the parallel point // already on the list is less than my desired coverage 148

PAGE 149

// end planning ( Distance long entire segment is worth // searching ) // Else calculate a new direction vector and check if the corner // point is already on the list // if ((currentIndex[0]==otherIndex[1])&& (currentIndex[1]==otherIndex[0]))f ROS INFO( Line Segments Overlap ); return false ; g else if ((currentIndex[1]==otherIndex[1])&& ((sidePoint1)]TJ /F4 9.963 Tf 14.32 0 Td[(coordinates[currentIndex[0]]).norm()<= (myCoverage+otherCoverage))&& ((sidePoint2)]TJ /F4 9.963 Tf 14.32 0 Td[(coordinates[currentIndex[1]]).norm()<= (myCoverage+otherCoverage)))f ROS INFO( New Point Matches previous corner point ); return false ; g // Redefine Bounding Box for new line segment getBoundaryPoints(coordinates[currentIndex[START]], coordinates[currentIndex[END]],boundary); return true ; g else f ROS INFO( No Point shift to next segment ); // // Shift to next line segment // // Shift to next indices ROS INFO( Find New Indices ); ROS INFO( Before Top (% d % d ) Bottom (% d % d ) ,currentIndex[START], currentIndex[END],otherIndex[START],otherIndex[END]); calculateNextIndices(currentIndex,direction,coordinates.size())]TJ /F4 9.963 Tf 8.01 0 Td[(1); 149

PAGE 150

ROS INFO( Before Top (% d % d ) Bottom (% d % d ) ,currentIndex[START], currentIndex[END],otherIndex[START],otherIndex[END]); // // Check if new points segments are valid for planning // If line segments overlap end planning // Else if Corner End point endices match and either corner point // of new line segment and distance to the parallel point // already on the list is less than my desired coverage // end planning ( Distance long entire segment is worth // searching ) // Else calculate a new direction vector and check if the corner // point is already on the list // if ((currentIndex[0]==otherIndex[1])&& (currentIndex[1]==otherIndex[0]))f ROS INFO( Line Segments Overlap ); return false ; g else if ((currentIndex[1]==otherIndex[1])&& ((sidePoint1)]TJ /F4 9.963 Tf 14.32 0 Td[(coordinates[currentIndex[0]]).norm()<= (myCoverage+otherCoverage))&& ((sidePoint2)]TJ /F4 9.963 Tf 14.32 0 Td[(coordinates[currentIndex[1]]).norm()<= (myCoverage+otherCoverage)))f ROS INFO( New Point Matches previous corner point ); return false ; g // Redefine Bounding Box for new line segment getBoundaryPoints(coordinates[currentIndex[START]], coordinates[currentIndex[END]],boundary); // If the corner point is already on the list continue current // search and find next point 150

PAGE 151

// else push the corner point to list if (coordinates[currentIndex[START]]==sidePoint1)f ROS INFO( Continue and find point along next segment ); continue ; g else f ROS INFO( Put First coordinate of next line on list ); point=coordinates[currentIndex[START]]; pointList.push back(coordinates[currentIndex[START]]); return true ; g g g g g 151

PAGE 152

REFERENCES [1] H.DuanandS.Liu,Unmannedair/groundvehiclesheterogeneouscooperativetechniques:Currentstatusandprospects,ScienceChinaTechnologicalSciences,vol.53,pp.1349,2010. [2] L.Chaimowicz,B.Grocholsky,J.F.Keller,R.V.Kumar,andC.J.Taylor,Experimentsinmultirobotair-groundcoordination,inRoboticsandAutoma-tion,2004.Proceedings.ICRA'04.2004IEEEInternationalConferenceon,2004,vol.4,pp.4053. [3] L.ChaimowiczandV.Kumar,Aerialshepherds:Coordinationamonguavsandswarmsofrobots,inInProc.ofDARS04,2004,pp.243. [4] N.Michael,C.Belta,andV.KumarKumar,Controllingthreedimensionalswarmsofrobots,inRoboticsandAutomation,2006.ICRA2006.Proceedings2006IEEEInternationalConferenceon,may2006,pp.964. [5] B.Grocholsky,J.F.Keller,V.Kumar,andG.Pappas,Cooperativeairandgroundsurveillance,RoboticsAutomationMagazine,IEEE,vol.13,no.3,pp.16,sept.2006. [6] N.Michael,J.Fink,andV.Kumar,Controllingateamofgroundrobotsviaanaerialrobot,inIntelligentRobotsandSystems,2007.IROS2007.IEEE/RSJInternationalConferenceon,292007-nov.22007,pp.965. [7] M.A.Hsieh,A.Cowley,J.F.Keller,L.Chaimowicz,B.Grocholsky,V.Kumar,C.J.Taylor,Y.Endo,R.C.Arkin,B.Jung,D.F.Wolf,G.S.Sukhatme,andD.C.MacKenzie,Adaptiveteamsofautonomousaerialandgroundrobotsforsituationalawareness:Fieldreports,JournalofFieldRobotics,vol.24,no.11-12,pp.991,Nov.2007. [8] V.KumarandN.Michael,Opportunitiesandchallengeswithautonomousmicroaerialvehicles,Int.J.Rob.Res.,vol.31,no.11,pp.1279,Sept.2012. [9] M.Turpin,N.Michael,andV.Kumar,Trajectorydesignandcontrolforaggressiveformationightwithquadrotors,AutonomousRobots,vol.33,pp.143,2012. [10] D.Mellinger,M.Shomin,N.Michael,andV.Kumar,Cooperativegraspingandtransportusingmultiplequadrotors,inDistributedAutonomousRoboticSystems,A.Martinoli,F.Mondada,N.Correll,G.Mermoud,M.Egerstedt,M.A.Hsieh,L.E.Parker,andK.Sty,Eds.,vol.83ofSpringerTractsinAdvancedRobotics,pp.545.SpringerBerlinHeidelberg,2013. [11] M.Achtelik,M.Achtelik,Y.Brunet,M.Chli,S.Chatzichristos,J.-D.Decotignie,K.-M.Doth,F.Fraundorfer,L.Kneip,D.Gurdan,L.Heng,E.Kosmatopoulos,L.Doitsidis,G.H.Lee,S.Lynen,A.Martinelli,L.Meier,M.Pollefeys,D.Piguet,A.Renzaglia,D.Scaramuzza,R.Siegwart,J.Stumpf,P.Tanskanen,C.Troiani,and 152

PAGE 153

S.Weiss,Sy:Swarmofmicroyingrobots,inIntelligentRobotsandSystems(IROS),2012IEEE/RSJInternationalConferenceon,Oct.2012,pp.2649. [12] S.Weiss,D.Scaramuzza,andR.Siegwart,Monocular-slambasednavigationforautonomousmicrohelicoptersingps-deniedenvironments,JournalofFieldRobotics,vol.28,no.6,pp.854,2011. [13] L.Doitsidis,S.Weiss,A.Renzaglia,M.Achtelik,E.Kosmatopoulos,R.Siegwart,andD.Scaramuzza,Optimalsurveillancecoverageforteamsofmicroaerialvehiclesingps-deniedenvironmentsusingonboardvision,AutonomousRobots,vol.33,pp.173,2012. [14] R.Ritz,M.W.Muller,M.Hehn,andR.D'Andrea,Cooperativequadrocopterballthrowingandcatching,inIntelligentRobotsandSystems(IROS),2012IEEE/RSJInternationalConferenceon,Oct.2012,pp.4972. [15] G.Hoffmann,D.G.Rajnarayan,S.L.Waslander,D.Dostal,J.S.Jang,andC.J.Tomlin,Thestanfordtestbedofautonomousrotorcraftformultiagentcontrol(starmac),ininProceedingsofthe23rdDigitalAvionicsSystemsConference,SaltLakeCity,Utah,November2004,pp.12.E.410. [16] G.M.Hoffmann,S.L.Waslander,,andC.J.Tomlin,Quadrotorhelicoptertrajectorytrackingcontrol,inIntheProceedingsoftheAIAAGuidance,Navigation,andControlConference,Honolulu,HI,August2008,pp.251. [17] N.Vandapel,R.R.Donamukkala,andM.Hebert,Unmannedgroundvehiclenavigationusingaerialladardata,TheInternationalJournalofRoboticsResearch,vol.25,no.1,pp.31,January2006. [18] J.P.How,B.Bethke,A.Frank,D.Dale,andJ.Vian,Real-timeindoorautonomousvehicletestenvironment,ControlSystems,IEEE,vol.28,no.2,pp.51,April2008. [19] U.ZenginandA.Dogan,Cooperativetargetpursuitbymultipleuavsinanadversarialenvironment,RoboticsandAutonomousSystems,vol.59,no.12,pp.10491059,2011. [20] F.Ducatelle,G.Caro,andL.Gambardella,Cooperativestigmergicnavigationinaheterogeneousroboticswarm,inFromAnimalstoAnimats11,S.Doncieux,B.Girard,A.Guillot,J.Hallam,J.-A.Meyer,andJ.-B.Mouret,Eds.,vol.6226ofLectureNotesinComputerScience,pp.607.SpringerBerlinHeidelberg,2010. [21] F.Ducatelle,G.A.DiCaro,andL.M.Gambardella,Cooperativeself-organizationinaheterogeneousswarmroboticsystem,inProceedingsofthe12thannualconferenceonGeneticandevolutionarycomputation,NewYork,NY,USA,2010,GECCO'10,pp.87,ACM. 153

PAGE 154

[22] N.Mathews,A.Christensen,R.OGrady,andM.Dorigo,Cooperationinaheterogeneousrobotswarmthroughspatiallytargetedcommunication,inSwarmIntelligence,M.Dorigo,M.Birattari,G.Caro,R.Doursat,A.Engelbrecht,D.Floreano,L.Gambardella,R.Gro,E.?ahin,H.Sayama,andT.Sttzle,Eds.,vol.6234ofLectureNotesinComputerScience,pp.400.SpringerBerlinHeidelberg,2010. [23] N.Mathews,A.Stranieri,A.Scheidler,andM.Dorigo,Supervisedmorphogenesis:morphologycontrolofground-basedself-assemblingrobotsbyaerialrobots,inProceedingsofthe11thInternationalConferenceonAutonomousAgentsandMultiagentSystems-Volume1,Richland,SC,2012,AAMAS'12,pp.97,InternationalFoundationforAutonomousAgentsandMultiagentSystems. [24] O.D.Silva,G.K.I.Mann,andR.G.Gosine,Developmentofarelativelocalizationschemeforground-aerialmulti-robotsystems,inIntelligentRobotsandSystems(IROS),2012IEEE/RSJInternationalConferenceon,Oct2012,pp.870. [25] M.Garzn,J.Valente,D.Zapata,andA.Barrientos,Anaerial-groundroboticsystemfornavigationandobstaclemappinginlargeoutdoorareas,Sensors,vol.13,no.1,pp.1247,2013. [26] H.G.TannerandD.K.Christodoulakis,Decentralizedcooperativecontrolofheterogeneousvehiclegroups,RoboticsandAutonomousSystems,vol.55,no.11,pp.811823,2007. [27] G.Lee,Y.Sato,andN.Y.Chong,Decentralizedtask-orientedlocalgroupgenerationforrobotswarms,inUbiquitousRobotsandAmbientIntelligence(URAI),20118thInternationalConferenceon,nov.2011,pp.174. [28] W.Li,T.Zhang,andK.Kuhnlenz,Avision-guidedautonomousquadrotorinanair-groundmulti-robotsystem,inRoboticsandAutomation(ICRA),2011IEEEInternationalConferenceon,2011,pp.2980. [29] S.Wright,D.A.A.Arroyo,I.A.Arroyo,J.W.Simpkins,B.Wright,andD.L.M.Zamstein,Emergentcollectivebehaviorinautonomoussynergisticswarmrobots,inFloridaConferenceonRecentAdvancesinRobotics,Gainesville,FL,May4-52011. [30] G.Dash,J.Weaver,D.A.A.Arroyo,S.Wright,andD.E.M.Schwartz,Orchestrationofintelligentgroundvehiclesintoahomogeneousswarmviathegooglecloud,inFloridaConferenceonRecentAdvancesinRobotics,BocaRaton,FL,May10-112012. [31] J.Weaver,G.Dash,D.A.A.Arroyo,andD.E.M.Schwartz,Heterogeneouscollaborativeswarmsofautonomousagentswithvaryingcapabilities,inFloridaConferenceonRecentAdvancesinRobotics,BocaRaton,FL,May10-112012. 154

PAGE 155

[32] W.Morris,I.Dryanovski,andJ.Xiao,dindoormappingformicro-uavsusinghybridrangendersandmulti-volumeoccupancygrids,intheworkshoponRGB-D:AdvancedReasoningwithDepthCameras,onJune27,2010,inconjunctionwith2010Robotics:ScienceandSystemsConference(RSS2010),UniversidaddeZaragoza,Spain,June27-302010. [33] I.Dryanovski,W.Morris,andJ.Xiao,Multi-volumeoccupancygrids:anefcientprobabilistic3dmappingmodelformicroaerialvehicles,inIEEE/RSJInt.Conf.onIntelligentRobotsandSystems(IROS2010),Oct2010,pp.15531559. [34] J.Weaver,D.Frank,D.A.A.Arroyo,andD.E.M.Schwartz,Uavperformingautonomouslandingonusvutilizingtherobotoperatingsystem,inASMEEarlyCareerTechnicalConference2013,Birmingham,AL,November2-32013,vol.12,pp.119. [35] J.Weaver,G.Dash,M.Thompson,D.A.A.Arroyo,andD.E.M.Schwartz,Controlofuavthroughtherobotoperatingsystemandandroid,inFloridaConferenceonRecentAdvancesinRobotics,Tallahassee,FL,May16-172013. [36] L.V.Schmidt,Introductiontoaircraftightdynamics,AmerInstofAeronautics,1998. [37] S.L.Waslander,G.M.Hoffmann,J.S.Jang,andC.J.Tomlin,Multi-agentquadrotortestbedcontroldesign:Integralslidingmodevs.reinforcementlearning,inIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems,2005,pp.3712. 155

PAGE 156

BIOGRAPHICALSKETCH JoshuaNathanielWeaverwasborninTacoma,Washington,November29,1983.Ashegrewuphehadalwaysbeeninterestedinscienceandtechnology,takingeverythinginhiswayaparttolearnhowitworked,andlaterattemptingtoputitbacktogether.Throughmiddleschoolandhighschoolhealwaysfoundhimselfveryinterestedintheconceptsthatweretaughtandtheadvancementsthatweremadeeveryday,whiledreamingthathewouldbealeaderinhiseldsomeday.AftergraduatingintheTopTenofhisclassatMosleyHighSchoolheattendedGulfCoastCommunityCollegewherehecontinuehiseducationwhilealsoresearchandlearningonhisown.WhileatGulfCoastCommunityCollegehewasapartofthehonorsprogramandthepresidentofPhiThetaKappa.Mr.WeaverlaterattendedFloridaStateUniversitywherehecontinuedhisfocusonelectricalandcomputerengineeringwhilealwayslookingfurtherintorobotics.AftergraduatingatthetopofhisclasswithhisBachelorsofScienceinelectricalengineeringin2007.NexthetooktimeworkingforaprivatecompanybeforemovingtoworkfortheUSNavywherehewasabletocontinuehisinterestinrobotics.AfterbeingacceptedintotheSMARTFellowshipprogram,heattendedtheUniversityofFloridawhereheworkedintheMachineIntelligenceLaboratory(MIL)undertheadvisementandmentorshipofDrA.AntonioArroyoandDr.EricSchwartz.Withthisnewfocus,Mr.Weaverwasabletofurtherhisunderstandinginsoftwaredevelopmentforautonomousroboticsforgroundandaerialvehicles.HerstcompletedhisMastersofScienceinlate2011whilelatercompletinghisDoctorofPhilosophyinelectricalandcomputerengineeringin2014.HisPhDresearchfocusedontheadvancementanddevelopmentofaswarmarchitectureformultipleunmannedgroundandaerialvehicles. 156