Mobile Mapping Beneath Forest Canopy

MISSING IMAGE

Material Information

Title:
Mobile Mapping Beneath Forest Canopy System Development and Application
Physical Description:
1 online resource (181 p.)
Language:
english
Creator:
Benjamin,Adam R
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Master's ( M.S.)
Degree Grantor:
University of Florida
Degree Disciplines:
Forest Resources and Conservation
Committee Chair:
Mohamed, Ahmed Hassan
Committee Members:
Dewitt, Bon A
Yarmola, Valeriy

Subjects

Subjects / Keywords:
canopy -- forest -- geomatics -- heading -- inertial -- mapping -- mobile -- navigation -- photogrammetry -- visionaiding
Forest Resources and Conservation -- Dissertations, Academic -- UF
Genre:
Forest Resources and Conservation thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Terrestrial Mobile Mapping Systems (MMS) can provide under canopy geospatial information to forest managers that is inaccessible from aerial vehicles and too time consuming for traditional land-based surveying methods. Methodology for designing, testing, and calibrating the georeferencing (positioning/orientation) and imaging MMS subsystems of the GatorMMS, a terrestrial forest MMS mounted on an All-Terrain Vehicle (ATV), is discussed. The principal components of these subsystems are the GPS-aided inertial navigation system and the digital single lens reflex (DSLR) camera. Due to lack of reliable GPS positioning under forest canopy, both GatorMMS applications explored herein focus on navigation trajectory precision. Using image thresholding to determine canopy density along a forested test track, canopy density is significantly correlated with the number of available satellites, PDOP, and navigation trajectory precision. Interestingly, time elapsed from static observation had the most significant correlation with navigation trajectory precision. Through vision-aiding (VA) using photogrammetric bundle adjustments, the navigation trajectory precision improves significantly including a one order of magnitude improvement by the heading orientation parameter precision. This major breakthrough for the heading orientation parameter is essential because angular error propagates over the distance the object is from the sensor. A methodology to improve near real-time direct georeferencing is proposed via implementation of a sequential VA algorithm. Through simulation and field collected data sets, the sequential VA algorithm showed substantial gains in angular precision and angular accuracy.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Adam R Benjamin.
Thesis:
Thesis (M.S.)--University of Florida, 2011.
Local:
Adviser: Mohamed, Ahmed Hassan.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043399:00001


This item is only available as the following downloads:


Full Text

PAGE 1

1 MOBILE MAPPING BENEATH FOREST CANOPY: SYSTEM DEVELOPMENT AND APPLICATION By ADAM R. BENJAMIN A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIE NCE UNIVERSITY OF FLORIDA 2011

PAGE 2

2 2011 Adam R. Benjamin

PAGE 3

3 To Mom, Dad, Erica, & Kai

PAGE 4

4 ACKNOWLEDGMENTS I cannot thank Dr. Ahmed Mohamed enough for extending me the opportunity to study with him at the University of Florida. His passion for geomatics and navigation is evident in the amount of time and effort he has put in helping me throughout this project and my course of study. Thanks to Dr. Bon Dewitt for introducing me to the field of photogrammetry, a focal topic of this thesis and being a mentor throughout my graduate education I thank Dr. Valeriy Yarmola for his support of my thesis project and collaboration on the G3 research projects. Recognition is necessary for all of the NaGeM research team members that have helpe d me with this thesis project. I thank Ben Wilkinson for his help in programming complex algorithms and being a sounding board for all my ideas that are "never going to work." Thanks to Kuei tsung (Philips) Shih for his tireless effort in making sure my Na GeM research project was successful through software development and electronics fabrication. Thanks to Apostolos (Tolee) Mamatas for collaborating with me on this vision aiding concept including hardware fabrication and also partaking in many enlightening navigation related discussions. Thanks to Nicholas DiGruttolo, Zoltan Szantoi, and Sowmya Selvarajan for your guidance and support throughout our graduate experience. Thank you to Dan Schultz and Gary St. John at Austin Cary Memorial Forest. Between prov iding access to the ATV and assistance in mounting the GatorMMS components to the ATV, your help is very much appreciated. Also, thanks to Dr. Amr Abd Elrahman for providing access to the PhotoModeler camera calibration software.

PAGE 5

5 Additionally, I thank the UF College of Agriculture and Life Sciences (CALS), the UF School of Forest Resources and Conservation (SFRC), and RIEGL USA http://www.rieglusa.com/ for funding for my graduate assistantship. All of what I have done would not have been possible without my family. I thank my parents, Muriel and Bobby and my brother, Zack, for their constant encouragement and support. Lastly, I thank my "dp", Erica for keeping our home in order and providing Kai with so much care tha t I could accomplish as much as I have over the past two years.

PAGE 6

6 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 9 LIST OF FIGURES ................................ ................................ ................................ ........ 10 LIST OF ABBREVIATIONS ................................ ................................ ........................... 14 ABSTRACT ................................ ................................ ................................ ................... 18 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 20 1.1 Problem Background ................................ ................................ ........................ 20 1.2 Motivation ................................ ................................ ................................ ......... 22 1.3 Objectives ................................ ................................ ................................ ......... 22 1.4 Research Questions ................................ ................................ ......................... 22 1.5 Hypothesis ................................ ................................ ................................ ........ 22 1.6 Report Structure ................................ ................................ ............................... 23 2 LITERATURE REVIEW ................................ ................................ .......................... 24 2.1 Mobile Mapping Systems ................................ ................................ .................. 24 2.2 Vision Aiding ................................ ................................ ................................ ..... 27 2.3 Forest Mapping ................................ ................................ ................................ 30 2.3.1 GPS Positioning in the Forest ................................ ................................ 31 2.3.2 Forest Canopy Density Affecting GPS Positioning ................................ .. 32 2.3.3 Mobile Mapping Systems Under Forest Canopy ................................ ..... 35 3 GATORMMS COMPONENTS ................................ ................................ ................ 39 3.1 GatorMMS The Mobile Mapping System for t he Forest ................................ 39 3.2 Georeferencing System ................................ ................................ .................... 40 3.2.1 GPS ................................ ................................ ................................ ......... 40 3.2.2 GPS Aided Inertial Navigation Systems ................................ .................. 41 3.3 Imaging System ................................ ................................ ................................ 43 3.3.1 Camera ................................ ................................ ................................ .... 44 3.3.2 Lenses ................................ ................................ ................................ ..... 45 3.4 MMS Coordinate Systems ................................ ................................ ................ 46 4 GEOREFERENCING SYSTEM TESTING ................................ .............................. 53 4.1 Study Site Austin Cary Memorial Forest ................................ ........................ 53

PAGE 7

7 4.2 Data Acquisition ................................ ................................ ................................ 53 4.3 GPS/GNSS Processing in the Forest ................................ ................................ 55 4.4 INS Free Navigation ................................ ................................ ......................... 56 4.5 GPS Aided INS Navigation ................................ ................................ ............... 58 4.5.1 Loosely Coupled GPS/INS Integration ................................ .................... 59 4.5.2 Loosely Coupled Integration and Tightly Coupled Integration Comparison ................................ ................................ ................................ ... 60 4.5.3 GPS Aided Inertial Navigation Trajectory Absolute Accuracy Analysis ................................ ................................ ................................ ......... 61 4.6 Georeferencing System Testing Lessons Learned ................................ ........... 63 5 IMAGING SYSTEM TESTING ................................ ................................ ................ 79 5.1 Self Calibrating Bundle Adjustment ................................ ................................ .. 79 5.2 Camera Calibration Interior Orientation ................................ ......................... 80 5.2.1 Methods ................................ ................................ ................................ ... 81 5.2.2 Results ................................ ................................ ................................ .... 82 5.3 Boresight Lever Arm Calib ration ................................ ................................ ....... 83 5.3.1 Reed Lab Roof BSLA Calibration Field ................................ ................... 84 5.3.2 BSLA Calibration Methods ................................ ................................ ...... 85 5.3.3 BSLA Calibration Results ................................ ................................ ........ 86 5.4 Imaging System Testing Lessons Learned ................................ ................... 88 6 GATORMMS: VERTICAL O RIENTATION FOR CANOPY DENISTY ANALYSIS .. 9 8 6.1 GatorMMSv2.0 Applications Canopy Density Analysis ................................ .. 98 6.2 Methods ................................ ................................ ................................ ............ 98 6.3 Preliminary Image Processing Results ................................ ........................... 100 6.4 GatorMMS Testing Results ................................ ................................ ............. 101 6.5 GatorMMS Canopy Density Analysis Lessons Learned ................................ .. 104 7 GATORMMS: HORIZONTAL ORIENTATION FOR VISION AIDING ................... 114 7.1 GatorMMS v2.0 Application: Vision Aiding ................................ ...................... 114 7.2 ACMF VA Methods ................................ ................................ ......................... 115 7.2.1 Orientation ................................ ................................ ............................. 115 7.2.2 ACMF VA Test Site ................................ ................................ ............... 115 7.2.3 ACMF VA Procedure ................................ ................................ ....... 117 7.3 GatorMMS VA Results ................................ ................................ .................... 117 7.3.1 ACMF VA Non ZUPT Trials ................................ ................................ ... 118 7.3.2 ACMF VA ZUPT Trial ................................ ................................ ............ 120 7.3.3 ACMF VA Trials: Low A ccuracy H3 IMU ................................ ............... 121 7.4 GatorMMS VA Lessons Learned ................................ ................................ .... 122 8 SEQUENTIAL VISION AIDING ................................ ................................ ............ 135 8.1 Theoretical Framework ................................ ................................ ................... 135 8.1.1 Kalman Filtering ................................ ................................ .................... 135

PAGE 8

8 8.1.2 Coplanarity Condition ................................ ................................ ............ 136 8.2 Methods ................................ ................................ ................................ .......... 138 8.2.1 Dynamics Model ................................ ................................ .................... 138 8.2.2 Observation Model ................................ ................................ ................ 139 8.2.3 Optimal Smoothing ................................ ................................ ................ 139 8.3 Experimental Results ................................ ................................ ...................... 141 8.3.1 Simulation Mo del ................................ ................................ ................... 141 8.3.2 UAV Flight Data Archer Field ................................ ............................. 145 8.4 Summary & Conclusion ................................ ................................ .................. 147 9 CONCLUSION ................................ ................................ ................................ ...... 155 9.1 Summary & Conclusions ................................ ................................ ................. 155 9.2 Recommendations ................................ ................................ .......................... 157 APPENDIX A ................................ ...................... 159 B KALMAN FILTER DETAILS ................................ ................................ .................. 161 C COPLANARITY CONDITION DETAILS ................................ ................................ 166 C.1 Non linear Coplanarity Condition ................................ ................................ .... 166 C.2 Linearization of the Coplanarity Condition ................................ ...................... 168 LIST OF REFERENCES ................................ ................................ ............................. 174 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 180

PAGE 9

9 LIST OF TABLES Table page 3 1 GPS receiver classification ................................ ................................ ................. 48 3 2 Inertial navigation gyroscope classification ................................ ......................... 48 4 1 Zero velocity upd ates and coordinate updates for inertial navigation processing ................................ ................................ ................................ .......... 64 4 2 Tightly coupled & loosely coupled GPS aided inertial navigation solution comparison ................................ ................................ ................................ ......... 64 4 3 Absolute accuracy investigation of loosely coupled integration .......................... 65 4 4 Absolute accuracy investigation of tightly coupled integration ............................ 65 5 1 Differences in camera calibration software programs. ................................ ........ 89 5 2 IOP camera calibration results from PhotoModeler 6 ................................ ....... 89 5 3 IOP camera calibration results from PhotoModeler 6 converted to pixels from mm. ................................ ................................ ................................ ............ 90 5 4 IOP camera calibration verification results ................................ ......................... 91 5 5 RLA BSLA calibration field coordinates ................................ .............................. 92 5 6 GatorMMSv2.0 lever arm calibration results. ................................ .................... 93 5 7 GatorMMSv2.0 boresight angle calibration results. ................................ ............ 93 5 8 GatorMMSv2.0 final BSLA calibration results. ................................ .................... 94 6 1 H ypothesis testing of CDI correlation for statistical significance of rel ationships ................................ ................................ ................................ ..... 106 7 1 VA test area ground control point coordinates in UTM 17N WGS84 ................ 125 7 2 VA test area target coordinates in UTM 17N WGS84 ................................ ....... 125 7 3 Hypothesis testing of population variances for VA trial 1 ................................ .. 126 7 4 Hypothesis testing of population variances for VA trial 2. ................................ 126 7 5 Hypothesis testing of population variances for VA trial 3 ................................ 126

PAGE 10

10 LIST OF FIGURES Figure page 2 1 Characterization of forest canopy: canopy density versus canopy cover ........... 38 3 1 GatorMMS plat form configurations ................................ ................................ ..... 48 3 2 Indirect georeferencing Collinearity condi tion for an aerial stereo model ......... 49 3 3 Loosely coupled GPS/IN S integration flow chart ................................ ............... 49 3 4 Tightly coupled GPS/INS integr ation flow chart ................................ .................. 50 3 5 Georeferencing system components for GatorMMS v2.0 ................................ .... 50 3 6 Imaging system lenses for GatorMMSv2.0. ................................ ........................ 51 3 7 Lens geometry comparison: wide angle vs. fisheye ................................ .......... 51 3 8 Physical relationship between GatorMMS sensors including the IMU, the GPS ARP, and the D200 camera ................................ ................................ ....... 52 4 1 Vicinity map of Austin Cary Memorial Fores t ................................ ...................... 66 4 2 Static GPS observation of ACMF ground control points ................................ ..... 66 4 3 GatorMMSv1.0 georeferencing system test tracks in Austin Cary M emorial Forest ................................ ................................ ................................ ................. 67 4 4 GPS satellite status ................................ ................................ ............................ 67 4 5 GPS baseline status KAR processing ................................ ............................. 68 4 6 GPS solution plot KAR processing. ................................ ................................ 68 4 7 GPS solution ARTK processing ................................ ................................ ........ 69 4 8 GPS baseline status ARTK processing ................................ ............................ 69 4 9 IMU only navigation trajectory ................................ ................................ ............ 70 4 10 INS free navigation velocity accuracy plot ................................ ........................ 70 4 11 INS free navigation position accuracy plot ................................ ....................... 71 4 12 INS free navigation ZUPTs and CUPTS trajectory ................................ .......... 71 4 13 INS free navigation ZUPTs and CUPTS velocity accuracy ........................... 72

PAGE 11

11 4 14 INS free navigation ZUPTs and CUPTS positional accuracy ........................ 72 4 15 GPS aided INS navigation forward only solution ................................ ............. 73 4 16 GPS aided INS navigation combined filtered solution ................................ ..... 73 4 17 GPS aided INS navigation forward solution positional accuracy .................. 74 4 18 GPS aided INS navigation combined filtered solution positional accuracy ... 74 4 19 GPS aided INS navigation forward solution velocity accuracy ..................... 75 4 20 GPS aided INS navigation combined filtered solution velocity accuracy ...... 75 4 21 GPS aided INS navigation loosely coupled combined and smoothed navigation trajectory ................................ ................................ ........................... 76 4 22 GPS aided INS navigation tightly coupled co mbined and smoothed navigation trajectory ................................ ................................ ........................... 76 4 23 GPS aided INS navigation loosely coupled combined and smoothed positional accuracy ................................ ................................ ............................. 77 4 24 GPS aided INS navigation tightly coupled combined and smoothed positional accuracy ................................ ................................ ............................. 77 4 25 Tightly coupled & loosely coupled horizontal GPS aided inertial navigation trajectory comp arison ................................ ................................ ......................... 78 5 1 PhotoModeler camera calibration grid ................................ ............................. 95 5 2 Radial lens distortion curve. ................................ ................................ ................ 95 5 3 Vicinity map of Reed Lab BSLA calibration field ................................ ................. 96 5 4 Roof view of RLA BSLA calibration field GCPs ................................ .................. 96 5 5 GatorMMSv2.0 camera & IMU coordinate frames ................................ .............. 97 5 6 Estimated angular orientation precision from BSLA calibration .......................... 97 6 1 Fisheye fie ld of view comparison ................................ ................................ ...... 106 6 2 Image thresholding implementation example for grayscale images ................. 107 6 3 Image thresholding implementati on example for blue channel only grayscale images ................................ ................................ ................................ .............. 107 6 4 Map of ACMF CDI analysis test tracks ................................ ............................. 108

PAGE 12

12 6 5 Typical image from ACMF CDI a nalysis on January 27, 2011 .......................... 108 6 6 Plot of CDI results from ACMF test track 2 ................................ ....................... 10 9 6 7 Typical image from ACMF CDI analysis on March 3, 2011 .............................. 109 6 8 Plot of CDI results from ACMF test track 1 ................................ ....................... 110 6 9 Plot of CDI results relative to tightly coupled position solutio n from ACMF test track 1. ................................ ................................ ................................ .............. 111 6 10 Plot of CDI results relative to GNSS only position solution from ACMF test track 1 ................................ ................................ ................................ ............... 112 6 11 Plot of CDI results with tightly coupled positional solution precision from ACMF test track 1 ................................ ................................ ............................. 113 6 12 Plot of relationship between horizontal precision and time elapsed from static observation. ................................ ................................ ................................ ...... 113 7 1 ACMF VA test track. ................................ ................................ ......................... 127 7 2 ACMF VA field setup. ................................ ................................ ....................... 127 7 3 ACMF VA t est area. ................................ ................................ .......................... 128 7 4 Typical image of ACMF VA test area ................................ ................................ 128 7 5 Estimated orientation angle precision for trial 1 ................................ ................ 129 7 6 Comparison of pre SCBA & post SCBA orientation angle precision for trial 1 129 7 7 Estimated position precision for ACMF VA trial 1. ................................ ............ 130 7 8 Comparison of pre SCBA & post SCBA position precision for ACMF VA trial 1 ................................ ................................ ................................ ....................... 130 7 9 Comparison of pre SCBA & post SCBA orientati on angle p recision for trial 2 131 7 10 Comparison of pre SCBA & post SCBA position precision for ACMF VA trial 2 ................................ ................................ ................................ ....................... 131 7 11 Estimated orientati on angle precision for ACMF VA trial 3 ............................... 132 7 12 Estimated position precision for ACMF VA trial 3. ................................ ............ 132 7 13 Comparison of pre SCBA & post SCBA orientation precision for ACMF VA trial 3 ................................ ................................ ................................ ................. 133

PAGE 13

13 7 14 Comparison of pre SCBA & post SCBA position precision for ACMF VA trial 3. ................................ ................................ ................................ ...................... 133 7 15 Low accuracy H3 IMU inertial trajectory from ACMF VA trial 1 ........................ 134 7 16 Low accuracy H3 IMU inertial trajectory from ACMF VA trial 3 ........................ 134 8 1 Geometry for the coplanarity condition ................................ ............................. 149 8 2 Improved orientation precision due to improved position precision with forward Kalman filtering ................................ ................................ .................... 149 8 3 Improved orientation precision due to improved position precision with backward optimal smoothing ................................ ................................ ............ 150 8 4 Orientation accuracy of backward optimal smoothin g over entire image sequence. ................................ ................................ ................................ ......... 151 8 5 RMSE for different position precisions with forward Kalman filtering and backward optimal smoothing. ................................ ................................ ........... 151 8 6 The effect of the number of tie points on average angular orientation precision with forward Kalman filtering and backward optimal smoothing. ....... 152 8 7 The effect of the number of t ie points on average angular orientation accuracy with forward Kalman filtering and backward optimal smoothing. ....... 152 8 8 UAV aerial image collected over Archer Field ................................ .................. 153 8 9 Detail of improved orientation precision due to forward Kalman filtering and backward optimal smoothing ................................ ................................ ............ 153 8 10 Orientation accuracy of backward optimal smoothing over entire image sequence ................................ ................................ ................................ .......... 154 8 11 RMSE for different position precisions with forward Kalman filtering and backward optimal smoothing ................................ ................................ ............ 154 B 1 Kalman filter operation block diagram ................................ .............................. 165

PAGE 14

14 LIST OF ABBREVIATION S Standard Deviation 2D Two Dimensional ACMF Austin Cary Memorial Forest ALSM Airborne Laser Swath Mapping ANOVA Analysis of Variance A R Ambiguity Resolution ARP Antenna Reference Point ARTK Advance Real T ime Kinematic AT Aerotriangulation ATV All Terrain Vehicle BA Bundle Adjustment BS Backsight BSLA Boresight Lever Arm CCD Charge Coupled Device CF Coordinate Frame cm Centimeter CDI C anopy Density Index CORS Continuously Operating Reference Station COTS Consumer Off The Shelf CUPT Coordinate Update D Delta (difference) (i.e. DE Delta East) DBH Diameter at Breast Height DG Direct Georeferencing DOP Dilution of Precision

PAGE 15

15 DSLR Digital S ingle Lens Reflex E East EKF Extended Kalman Filter EOP Exterior Orientation Parameters FE Fisheye (lens) FLM Focal Length Multiplier FOV Field of View GAIN GPS A ided Inertial Navigation GCP Ground Control Point GI S Geographic Information System GNSS Glo bal Navigation Satellite System GNVL Gainesville Airport CORS GPS Global Positioning System h Ellipsoid Height/Geodetic Height Height A bove Ellipsoid H Orthome tric Height Height A bove Geoid Hor Horizontal hr Hour Hz Hertz i n Inch IMU Inertial Measur e ment Unit INS Inertial Navigation System IOP Interior Orientation Parameters KAR Kinematic Ambiguity Resolution KF Kalman Filter km Kilometer

PAGE 16

16 LC Loosely Coupled ( GPS/INS Integration) LiDAR Light Detection and Ranging LSM Least Squares Matching m Meter m m Millimeter m/s Meters per Second MEMS Micro Electro Mechanical Systems MP Megapixel MMS Mobile Mapping System N North NGS National Geodetic Survey OPK Omega Phi Kappa PDOP Positional Dilution of Precision Positioning and Orientation Systems PM PhotoModeler QA Quality Assurance QC Quality Control RGB Red Green Blue RLA Reed Lab Home of University of Florida Geomatics Department (Gainesville, FL) RLAB Reed Lab CORS Operated by Florida Department of Tr ansportation RLG Ring Laser Gyroscopes RMSE Root Mean Squared Error RPY Roll Pitch Yaw RTS Rauch Tung Striebel RTK Real Time Kinematic

PAGE 17

17 s Second SA Selective Availability SCBA Self Calibrating Bundle Adjustment S D Standard Deviation SIFT Scale Invariant Feature Transform SLAM Simultaneous Localization and Mapping SNR Signal to Noise Ratio SPAN Synchronized Position Attitude Navigation SV Satellite Vehicle TC Tightly Coupled ( GPS/INS Integration) TS Total Station TT Topcon Tools 7.2 U Up (Vertical Component) UAV Unmanned Aerial Vehicle UTM Universal Transverse Mercator Ver V ertical VA Vision Aiding VS Versus WA Wide Angle (lens) WGS84 World Geodetic System 1984 WIE Waypoint Inertial Explorer ZUPT Zero V elocity Update

PAGE 18

18 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Science MOBILE MAPPING BENEATH FOREST CANOPY: SYSTEM DEVELOPMENT AND APPLICATION By Adam R. Benjamin August 2011 Chair: Ahmed Mohamed Major: Forest Resources and Conservation Terrestrial Mobile Mapping Syst ems (MMS) can provide under canopy geospatial information to forest managers that is inaccessible from aerial vehicles and too time consuming for traditional land based surveying methods. Methodology for d esign ing testing, and calibrat ing the georeferenci ng (positioning/orientation) and imaging MMS subsystems of the GatorMMS, a terrestrial forest MMS mounted on an All Terr ain Vehicle (ATV), is discussed The principal components of these subsystems are the GPS aided inertial navigation system and the digit al single lens reflex (DSLR) camera. Due to lack of reliable GPS positioning under forest canopy, both GatorMMS applications explored herein focus on navigation trajectory precision. Using image thresholding to determine canopy density along a forested tes t track, canopy density is significantly correlated with the number of available satellites, PDOP, and navigation trajectory precision. Interestingly, time elapsed from static observation had the most significant correlation with navigation trajectory prec ision Through vision aiding (VA) using photogrammetric bundle adjustments, the navigation trajectory precision improves significantly including a one order of magnitude improvement by the h eading orientation

PAGE 19

19 parameter precision. This major breakthrough fo r the heading orientation parameter is essential because angular error propagates over the distance the object is from the sensor. A methodology to improve near real time direct georeferencing is proposed via implement ation of a sequential VA algorithm Th rough simulation and field collected data sets, the sequential VA algorithm showed substantial gains in angular precision and angular accuracy.

PAGE 20

20 CHAPTER 1 INTRODUCTION 1.1 Problem Background From the family plan ning its next trip using Google Earth to the transportation engineer using LiDAR point cloud processing to perform preliminary highway overpass inspections, geospatial data sets are a common denominator that makes these inquiries all possible. The desire for geospatial information from gover nment agencies, corporations, academic institutions, and consumers continues with no prospect of relenting. Mobile Mapping Systems (MMS) technology has enabled Geographic Information System (GIS) and geomatics professionals the ability to acquire more geos patial data in less time than traditional surveying methods and often at reduced costs ( Li 1997 ) Unmanned A erial Vehicles (UAV ) and land based MMS platforms are two data acquisition tools that have emerged to meet the demand for geospatial data. Mobile Ma pping Systems (MMS) have great potential to meet the needs of those who desire vast amounts of geospatial data without the constraint of a fixed location. However, there is inherent difficulty that comes with georeferencing from a mobile platform instead o f a static one. G eoreferencing is the process of determin ing the time, position/location, and attitude/orientation of an event in space ( Skaloud 1999 ) Direct georeferencing (DG) incorporates the use of onboard inertial navigation system (INS) sensors to d etermine these parameters. When direct georeferencing from a mobile platform, kinematic positioning and dynamic orientation are additional unknown parameters that need to be resolved relative to static direct georeferencing. These mobile navigation traject ory computations are further compounded by gaps in the GPS position trajectory.

PAGE 21

21 GPS outage prone areas such as forests and urban environments pose a significant problem for geomatics professionals using MMS. Due to the value of natural resources and infras tructure in these environments, development of solutions to the direct georeferencing problem is necessary. Different sensors and techniques have been utilized to supplement the inertial navigation trajectory including vision aiding (VA) methods. Due to th e enormous file sizes and complex calculations, using the most efficient VA method is essential. Simultaneous batch post processing of the imagery and navigation data is the most common VA method due to minimization of error ( Wolf and Dewitt 2000 ) However simultaneous VA does not afford the user the ability to perform real time or near real time mapping operations because all data must be collected prior to performing the batch operation. A couple examples of near real time applications include machine co ntrol for forestry applications or disaster response in GPS signal obscured environments. Sequential VA has the potential for being a more efficient method for real time applications; however, this method suffers from error propagation. Thus, the developme nt of a sequential VA technique that limits error propagation would be beneficial to the geospatial community for near real time applications. Furthermore, many geodetic grade GPS receivers and tactical grade inertial navigation systems cost tens of thous ands of dollars. Low cost, low accuracy GPS/INS cost hundreds of dollars. If the low cost systems can be supplemented by vision aiding to provide a similar navigation trajectory to geodetic/ tactical grade systems, more people would have access to this tech nology. Greater access could lead to more MMS improvements and broader applications.

PAGE 22

22 1.2 Motivation Contemporary forest managers are challenged to make efficient decisions in a short amount of time and based on as much information as possible. Aerial geosp atial platforms provide rapid acquisition of forest data but they often lack information on under canopy structure, diameter at breast height (DBH) of trees, and ladder fuel data for forest fire risk assessment. Terrestrial MMS beneath forest canopy can fi ll the data gap at reasonable cost and fast turn around time. 1. 3 Objectives The objectives of this thesis research are to (1) d evelop a terrestrial remote sensing mobile mapping system for use in forested areas GatorMMS ; (2) i nvestigate GPS/INS processi ng techniques and operational methods for improving the navigation solution beneath forest canopy ; (3) e xamine the relationship between forest canopy and navigation solution accuracy for MMS platforms in forests ; and (4) i nvestigate vision aiding as a tech nique to improv e direct georeferencing for near real time applications 1.4 Research Questions The following three research questions will be addressed through the course of this thesis. What quantitative impact does the tree canopy density (CDI) have on t he navigation solution accuracy of terrestrial MMS? C an VA improve the georeferencing solution in high CDI areas where signal outages are most prevalent? Can a sequential VA technique be developed to improve the direct georeferencing solution of aerial MMS for future application to terrestrial MMS ? 1.5 Hypothesis Due to the importance of GPS derived positions in the GPS aided inertial navigation solution, expectations are that canopy density will be highly correlated with

PAGE 23

23 navigation trajectory accuracy. As a result, it is hypothesized that terrestrial MMS direct georeferencing will benefit significantly from VA in highly signal degraded environments (high CDI areas). During this study, a proof of concept will be developed. The eventual real time application will be a future development. The goal is that MMS developers will have a method for quantifying the forest canopy for use as an input in inertial navigation processing. This input will trigger the use of sequential VA for real time applications. 1. 6 Rep ort Structure This thesis project covers information on technology related to mobile mapping systems including INS, GPS, photogrammetry, and remote sensing. Chapter 2 is a review of previous literature regarding these technologies in both forested and non fore sted environments. Chapter 3 covers the components of the GatorMMS including a discussion of the georeferencing system, imaging system, and coordinate systems The georeferencing system testing for GatorMMS v1.0 is co vered in Chapter 4. In C hapter 5 c amera calibration and boresight/lever arm calibration is discussed as part of the imaging system testing. Chapter 6 is a discussion of the vertically oriented camera system for analysis of the relationship between canopy density and navigation trajectory a ccuracy. In Chapter 7, the concept of VA from a terrestrial platform beneath forest canopy is explored. Chapter 8 discusses the implementation of a sequential vision aiding algorithm from an aerial platform with the goal of improving the angular orientatio n of the platform. Conclusions and recommendations for future studies are covered in Chapter 9

PAGE 24

24 CHAPTER 2 LITERATURE REVIEW 2.1 Mobile Mapping Systems Mobile Mapping Systems (MMS) are remote sensing platforms used to capture data for mapping of its surrou nding environment. MMS has been defined as the "product that integrates concepts of kinematic geodesy, aerospace engineering, automatic control, remote sensing, and digital photogrammetry to acquire, store, and process measurable quantities that sufficient ly describe spatial and/or physical characteristics of a part of the Earth's surface" ( Hassan et al. 2006 ) Original developments in the MMS field were restricted to applications where the exterior orientation of the platform was derived from existing cont rol points; however, drastic technological improvement in satellite and inertial navigation made geomatics and engineering professionals reevaluate the development of MMS ( Schwarz and El Sheimy 2004 ) Both land based and aerial MMS have undergone drastic c hanges in the last two decades. Airplanes were the original modern MMS platform for aerial photogrammetry. As defined in ( Wolf and Dewitt 2000 ) photogrammetry is the "art, science, and technology of obtaining reliable information about physical objects a nd the environment through processes of recording, measuring, and interpreting photographic images and patterns of recorded radiant electromagnetic energy and other phenomena." This broad definition includes two clear areas of photogrammetry: interpretativ e and metric. Interpretative photogrammetry involves object recognition, identification, and judgment of object significance through systematic analysis ( ibid ) Metric photogrammetry involves precise measurement of photographs to determine relative locatio n of points in

PAGE 25

25 the images for the purpose of distance, area, elevation, angle, and elevation calculations ( ibid ) Aerial and land based MMS have been used extensively in both of those photogrammetric areas. For mapping the precise location of objects, met ric photogrammetry is the only method. To obtain the desired object locations, the orientation and position of the camera/sensor is a necessity. For years, the only practical solution for camera position and orientation determination was knowledge of the a bsolute position of objects on the ground. The relationship between the absolute object coordinates, the corresponding image coordinates, and the camera calibration parameters would reveal the exterior orientation parameters of the camera through rigorous computations. Establishing extensive networks of photo identifiable ground control points is both costly and time consuming. Thus, development of positioning and orientation sensors onboard MMS to supplement ground control points was celebrated throughout the photogrammetric community ( Schwarz and El Sheimy 2004 ) Typically consisting of a Global Positioning System (GPS) antenna/receiver and an Inertial Measurement Unit (IMU), these positioning and orientation sensors are collectively referred to as a GPS a ided Inertial Navigation System (INS). Unmanned Autonomous Vehicle (UAV) and land based MMS platform development have grown exponentially with the technological advances in INS. The small size and versatility of UAVs provides users a cost friendly and eff icient approach to small aerial mapping projects. The diverse research areas investigated with UAVs include wildlife and ecological monitoring (Perry 2009; Wilkinson et al. 2009), tidal zone mapping ( DiGruttolo 2010), and automatic detection of forest fire s (Merino et al. 2006).

PAGE 26

26 Meanwhile, various research groups and corporations saw the niche that terrestrial MMS provided people looking to monitor infrastructure from street view. Many examples of recent developm ents in MMS include the use of vans and truck s as the mobile platform ( Talaya et al. 2004; Schwarz and El Sheimy 2004; Alshawa et al. 2007; Grafe 2007; Barber et al. 2008; Haala et al. 2008; Glennie 2009 ) Predominantly, these vehicles were operating with LiDAR (Light Detection and Ranging/laser scan ners) and cameras as remote sensors for monitoring infrastructure in the built environment. Furthermore, boat based LiDAR has recently become viable as a supplement to the underwater usage of side scan sonar employed by hydrographic surveying vessels ( Moha med 2007; Alho et al. 2009 ) Unmanned Autonomous Vehicles and land based MMS would not be viable for mapping without the underlying navigation trajectory solution: the position and orientation of the sensor. Further information regarding the inertial navig ation and global positioning sensors will be discussed in Chapter 3. While GPS and IMU data are the dominant navigation trajectory updates, other sensors can also provide valuable update information. Distance measurement instruments (DMI) can provide wheel revolution updates for the GPS aided inertial navigation trajectory ( NovAtel 2006 ) These velocity aiding odometer measurements have been successfully integrated to improve positional accuracy ( Hassan et al. 2006 ) Furthermore, image matching algorithms c an be used to detect targets and provide positional updates ( Tao et al. 2001; Hassan et al. 2006 ) Image matching for navigation updates and mapping features from a mobile platform requires sensors able to capture data regarding the physical characteristic s it passes. Digital cameras, digital video cameras, and two dimensional laser scanners are

PAGE 27

27 the most frequently used data acquisition devices ( Petrie 2010 ) Typical specifications for MMS digital frame cameras include small format sizes (1 to 2 megapixels) high framing rates (7 to 15 Hz), and very short exposure times to eliminate blurring ( ibid ) Laser scanners capture data by measuring angles and distance to objects the MMS passes. Typically, these scanners are operated in 2D mode which means the scan an gle does not change; thus, the scanner continuously scans in a vertical plane ( ibid ) As the MMS moves, the third dimension of the output point cloud is created. 2.2 Vision Aiding Vision aiding of the navigation solution using imagery has become an integ ral component of low cost IMU/GPS sub systems providing direct georeferencing to remote sensing systems. The data workflow to recover the orientation parameters rigorously requires the simultaneous handling of large amounts of imagery and navigation data. In some situations, with small UAVs for example, a flight block of thousands of images is the norm. The normal matrix of the blended imagery and navigation data can be very large in size for regular computers to handle efficiently. A Kalman filtering appro ach to sequentially process the blended navigation and imagery data can be used. Georeferencing parameters are then computed for every exposure station. To properly direct georeference imagery, the position and orientation of the sensor at exposure times must be known. These elements are commonly referred to as exterior orientation parameters (EOP). In indirect georeferencing, aerial triangulation is used to obtain the exterior orientation parameters of an individual exposure station using known ground con trol point coordinates and its counterparts in the images. The other interest of aerial t riangulation (AT) is obtaining the object space coordinates of other points imaged in the aerial images.

PAGE 28

28 Bundle Adjustment (BA) is a technique used in analytical AT to obtain EOPs. BA uses a least squares approach to minimize the errors of a bundle of rays connecting the photo coordinate measurements with the control coordinates on the ground. This is usually done by incorporating either the collinearity equations to determine the EOPs through determination of the object space coordinates of the imaged points or directly through the coplanarity condition ( Kersten and Baltsavias 1994; Tommaselli and Tozzi 1996; Haala et al. 1998; Wolf and Dewitt 2000; Mikhail et al. 200 1; Wang and Clarke 2001 ) Collinearity is the condition that the incident nodal point of the lens (exposure station), any object point, and its corresponding image point all lie on the same line in three dimensional space ( Wolf and Dewitt 2000 ) Coplanar ity, on the other hand, is the condition that two exposure stations of a stereo pair, any object point, and the corresponding image points of the two photos all lie in the same plane ( ibid ) Since no object coordinates are involved in the coplanarity condi tion, initial approximations of the object space coordinates are not necessary. This proves to be a valuable asset for using the coplanarity equation. Bundle adjustments can be performed either sequentially or simultaneously. Each method has three main com ponents: relative orientation of each stereo model, connection of adjacent models to form continuous strips and/or blocks, and simultaneous adjustment of the photos from the strip/block to ground control. Relative orientation is used to determine the relat ive angular attitude and positional difference between two photographs when the images were captured ( ibid ) Absolute orientation takes the relatively oriented stereo models and transforms them to the ground using

PAGE 29

29 three dimensional conformal coordinate tra nsformations. The unknown quantities of BA in either method are the object space coordinates (XYZ) of object points and the exterior orientation parameters (georeferencing parameters) of each photograph. Both sequential and simultaneous bundle adjustments have certain advantages and disadvantages. The greatest disadvantage for sequential bundle adjustments is the nonlinear accumulation of random error along an image strip as more stereo models are added to the adjustment ( ibid ) Simultaneous BA avoids this error accumulation by processing all measurements at once This provides a more robust method for determining the optimal solution. Simultaneous BA, however, comes with a computational burden in the form of huge matrix operations imposed by the large amou nts of imagery in the strips/blocks, specifically with small format imagery. If a method existed for reducing error accumulation in sequential BA, the computational time incurred by doing the adjustment sequentially as opposed to simultaneously could be a great advantage especially in near real time applications. Consequently, many experts especially in the field of navigation are immensely interested in sequential estimation as a pressing research topic. A number of approaches to sequential estimation thro ugh bundle adjustments are used in the field of robot vision or vision metrology ( Kersten and Baltsavias 1994; Edmundson and Fraser 1998; Di et al. 2008 ) To accurately obtain the orientation and position of the robot, a simultaneous bundle adjustment of a ll the previous geospatial data would not be feasible as the robot/UAV needs the geospatial information in near real time to continue navigation. Thus, sequential estimation theory is used. Sequential estimation approaches using collinearity equations with image matching techniques are

PAGE 30

30 common ( Kersten and Baltsavias 1994; Tommaselli and Tozzi 1996; Haala et al. 1998; Wang and Clarke 2001 ) The algorithm developed in ( Webb 2007 ) applies coplanarity as the observation model for Kalman filtering to the sequent ial aerotriangulation problem with success. The premise of that research was navigation, not georeferencing the acquired imagery. Thus, research of sequential BA using Kalman filtering and optimal smoothing as a method for reducing the sequential accumulat ion of error normally associated with AT to provide accurate and precise georeferencing parameters is lacking. 2.3 Forest Mapping In 2002, forests covered approximately 749 million of the 2.3 billion acres of land mass in the United States (Smith et al. 2 004). This one third proportion of forest land to non forest land has remained relatively stable for the past 100 years; however, many forests are quite dynamic as the lands are logged and managed for their timber resources. Of the 749 million acres, only 10% (77 million acres) is reserved from commercial timber harvesting (ibid). Furthermore, timberland covers more than 504 million acres of forest land (ibid). Timberland is forest land able to produce more than 20 cubic feet per acre per year without being legally withdrawn from timber production. The volume and value of timberland resources necessitates efficient management techniques which includes having an accurate inventory. Accurately positioning oneself in a forested environment is critical to many real time and near real time applications such as mapping forest inventories ( Zengin and Yesil 2006 ) fighting forest fires ( Xiaopeng et al. 2008 ) and harvesting trees with machine control ( Rossmann et al. 2009 ) Since GPS positioning requires accurate ti ming and ranging between earth orbiting satellites and the GPS receiver, sky

PAGE 31

31 obstructions pose an inherent problem to acquiring GPS signals under a forest canopy. Thus, many studies have focused on different aspects of determining GPS position in a forest. 2.3.1 GPS Positioning in the Forest After Selective Availability (SA) was turned off on 1 May 2000, a Japanese research group studied the Signal to Noise Ratio (SNR) of GPS signals impacted by forest conditions including natural forest, plantation forest forest road, and forest clearings ( Gandaseca et al. 2001 ) A higher SNR indicates a better signal. Logically, the clearing area had the highest SNR and the plantation area had the lowest SNR due to dense tree canopy. Gandaseca et al. (2001) also showed t hat using a GPS antenna height of 4.2m significantly reduced position errors relative to 1.0m antenna height. Unfortunately, an antenna height of 4.2m is unrealistically high for most practical survey applications. A performance study of Real Time Kinemati c (RTK) GPS using two geodetic grade GPS receivers and a navigation grade handheld GPS receiver under forest cover was conducted to compare point positions from the different techniques ( Zengin and Yesil 2006 ) Nine ground control points were surveyed mult iple times in nine separate stands. Each stand had a unique dominant tree species. Absolute accuracy could not be assessed since the study areas lacked absolute ground control point coordinates; thus, the two position solutions were compared relative to ea ch other. Through numerous trials, the RTK measurements understandably were more precise (i.e. greater repeatability) than the handheld receiver measurements Deciduous forest stands had more consistent positional precision results than coniferous stands. This difference between pine and deciduous stands has been well documented in previous GPS

PAGE 32

32 studies analyzing forest cover type during leaf on/off and seasonal changes ( Deckert and Bolstad 1996; Sigrist et al. 1999, Piedallu and Gegout 2005 ) An extensive s tudy in the forest of northeast France investigated the influence of four factors on GPS accuracy: receiver type, forest cover (open cover, coppice regeneration, and deciduous high forest), GPS survey components, and season (winter or summer) ( Piedallu and Gegout 2005 ) The number of recordings, Positional Dilution of Precision (PDOP) threshold, time interval between recordings, and differential correction availability were modified as part of the GPS survey component analysis for four GPS receivers ranging from navigation grade to mapping grade. Using one grou nd control point with decimeter level accuracy in each forest cover class, the only insignificant factor on GPS accuracy was the season. Receiver type and forest cover each modified the positional accu racy by a factor of 2 to 3. Increasing the number of recordings, increasing the time interval, or decreasing the PDOP threshold significantly improved precision of the positional solution. 2.3.2 Forest Canopy Density A ffecting GPS Positioning While forest canopy can significantly alter GPS performance, one omission in these studies was the lack of quantification of forest canopy characteristics such as crown cover and canopy closure. The "proportion of the sky hemisphere obscured by vegetation when viewed f rom a single point" is canopy closure (canopy density); meanwhile, canopy cover describes the "proportion of the forest floor covered by the vertical projection of the tree crowns" ( Jennings et al. 1999 ) F igure 2 1 modified after Jennings et al. (1999) de picts the difference between these two terms. With respect to GPS positioning and receiving satellite signals at one moment in time, forest canopy density is the preferred metric. Thus, skyward looking hemispherical photography

PAGE 33

33 which captures a wide angle view of the forest canopy from a single point is used extensively in GPS studies relating canopy density to GPS positional performance ( Jennings et al. 1999; Sigrist et al. 1999; Frazer et al. 2001; Holden et al. 2001; Zheng et al. 2005; Hu et al. 2009 ) Holden et al. (2001) investigated a method for relating GPS performance to forest canopy using this skyward looking hemispherical photography above the ground control points. Precision of the differential GPS solution was strongly correlated with total s ky obstruction, size of the largest hole in the canopy, and fragmentation of the sky view. The authors found there was no significant difference in positional precision between open sky view and 20% closed canopy H owever, positional precision degraded si gnificantly after 20% closed canopy by a factor of 5 to 7 times for points under heavy closed canopy. Sigrist et al. (1999) analyzed the impact of quantified canopy characteristics on PDOP and absolute accuracy of the GPS positional solutions. Interestingl y, Sigrist et al. (1999) reported that absolute accuracy, as measured by root mean square error, decreased by fourfold from open sky to 20% canopy cover. Absolute accuracy degradation slowed considerably with only a 50% increase in RMSE from 20% canopy cov er to almost 100% closed canopy. Zheng et al. (2005) confirmed the Holden et al. (2001) study conclusions by evaluating GPS positional accuracy along a forested trail of 26 ground c ontrol points Using an analysis of variance (ANOVA) method, the real time GPS positional accuracy was significantly degraded by increases in the forest canopy density. From Holden et al. (2001) and Sigrist et al. (1999) the contrast between accuracy and precision is apparent. Having a precise solution does not mean that the GPS positional solution is accurate. Satellite obstruction of a minimal

PAGE 34

34 portion of the sky (>20%) can alter the geometry of the satellite constellation such that the positional fixes are precise (high repeatability) but inaccurate. Likewise, the deterioration of precision may only minimally impact the already poor GPS positional accuracies. Both Sigrist et al. (1999) and Holden et al. (2001) described the conversion of gray scale images to black and white images. However, the thresholding methodology used to determine percentage of sky obstruction pixels was omitted. In contrast, Zheng et al. (2005) implemented the image thresholding algorithm called Otsu's method He described in detail how he used this method t o differentiate sky pixels from canopy pixels in the Olympus C 3040 digital images. From Otsu's method which minimizes the intra group variance between sky and canopy pixels, the canopy density index (CDI) metric was determined. The Euclidean distance approximation transformation used to quantify the ob struction pattern (largest canopy hole) and fragmentation of sky view metrics was explained in the Holden et al. paper. Comparing digital and film cameras using hemispherical photography for forest canopy metric determination, Frazier et al. (2001) found t hat the Nikon Coolpix 950 produced canopy openness estimates that were 1.4 times greater than conventional Nikon F film camera estimates. The authors advised a cautious approach to using this digital camera model when making forest canopy measurements. Significant advances in digital camera techn ology have been made since 2001. H owever, chromatic aberration common with consumer level digital camera optics might still cause a problem. Thus, the impact color blur can have on the detection of vegetation edg es and canopy gaps should be monitored in this research study.

PAGE 35

35 Airborne laser swath mapping (ALSM) is another remote sensing technique that has been used to quantify the forest structure and canopy density in the development of a method to predict reliabil ity of a GPS receiver under forest canopy ( Wright 2008 ) Through LiDAR post processing and GPS signal to noise ratio (SNR) analysis, GPS signal loss was strongly correlated with the complexity of local canopy structure and density. Encouragingly, the devel opment of a model for prediction of GPS SNR based on canopy structure showed promise for use in future 2D signal attenuation maps. The remote sensing techniques developed in these quantification studies will be valuable for characterizing the forest canop y density of the trails used in this study. The methodology set forth in the image thresholding algorithms will enable the author to accurately characterize the canopy closure during the inertial trajectories. This characterization will be useful in addres sing a primary research question regarding the implementation of vision aiding in high canopy density areas. 2.3.3 Mobile Mapping Systems Under Forest Canopy Through this review of GPS performance studies in forests, signal degradation and outages are obst acles that need to be overcome in the design of MMS for forested environments. Fortunately, development of MMS in signal degraded environments is an ongoing academic and industrial pursuit. Due to the high initial cost of the systems, much of the MMS resea rch involving GPS outage prone areas centers on the difficulty of obtaining an accurate navigation trajectory in urban areas for commercial applications ( Bayoud 2005; Kennedy et al. 2006; Nassar et al. 2007; Kukko et al. 2007; Haala et al. 2008 ) Signal bl ockages from buildings, overpasses, and/or trees were encountered in each study. Significant improvements in the navigation trajectory accuracy and positioning availability were seen with GPS aided inertial navigation relative to a GPS

PAGE 36

36 only solution in eac h case. Kennedy et al. saw greater improvement during signal outages when a distance measurement instrument (DMI) update was introduced into the filtering solution acquired with Novatel 's tactical grade Synchronized Position Attitude Navigation (SPAN ) sy stem. Both loosely coupled and tightly coupled GPS/INS integration techniques were tested. With minute long GPS outages, the DMI updates improved relative trajectory solution accuracy by 55% relative to the baseline navigation grade IMU trajectory ( Kennedy et al. 2006 ) For surveying under heavy forest conditions, an Applanix POS LS backpack mounted inertial land positioning/navigation system was developed and tested ( Gillet et al. 2001; Reutebuch et al. 2003 ) The authors emphasized the importance of using frequent zero velocity updates (ZUPTS) to dampen the 0.01 degrees/ho ur drift associated with the ring laser gyros (RLG) in the twenty year old navigation grade IMU. ZUPTS occur when the MMS remains stationary for a set period of time. Initial results showed the growth rate of both the horizontal and vertical real time erro rs to be approximately 2 meters per kilometer ( Gillet et al. 2001 ) Additional field testing of the Applanix POS LV backpack system evaluated positional accuracy and terrain profile development potential ( Reutebuch et al. 2003 ) To test the ability of the POS LV as an inertial only system, this testing did not utilize GPS updates. The MMS was initialized on a point of known coordinates at the beginning and end of each run. Using ZUPTs on average every 40 seconds (s) the system had an ave rage positional accuracy of 2.3ft in real time and 1.4 ft in post processing. Comparing the terrain profile generated by the backpack INS with a previously generated LiDAR digital terrain model (DTM), average post processed elevation differe nces along the profiles was 0.7 ft. While

PAGE 37

37 less frequent ZUPTs are preferred, this study is promising for a backpack INS system under heavy fo rest cover. Nassar et al. (2007) analyz ed different filtering and smoothing approaches to deal with GPS signal outages in computation of navigation trajectories. The non photogrammetric bridging methods utilized showed drastic improvement in position error s regardless of which Kalman filtering technique was implemented. This study was focused primarily on improv ing the post processed solution. T hus, real time applications were not covered with this approach. Machine control for forest harvesting equipment h as encouraged research in real time forest navigation due to minimal satellite coverage under forest canopy. Approaches involving laser scanners and other non photogrammetric sensors were the most common in forest navigation applications ( Rossman et al. 20 09; Morales et al. 2010 ) These techniques utilized Simultaneous Localization and Mapping (SLAM) algorithms. Various photogrammetric techniques are in development for aiding UAV navigation in real time ( Webb 2007; Samadzadegan et al. 2007; Taylor 2009 ) Wh ile land based robots using vision aiding has been researched ( Kersten and Baltsavias 1994; Edmundson and Fraser 1998; Bayoud 2005; Di et al. 2008 ) studies of terrestrial based applications of image aiding in the forest are lacking. Thus, image aiding of the navigation solution in forested areas is a research area in need of further study.

PAGE 38

38 Figure 2 1. Characterization of forest canopy: canopy density versus canopy cover (modified after Jennings, S., N. Brown, and D. Sheil. 1999. Assessing forest canop ies and understory illumination: canopy closure, canopy cover and other measures. Forestry. 72(1): 59 74. )

PAGE 39

39 CHAPTER 3 GATORMMS C OMPONENTS 3.1 GatorMMS The Mobile Mapping System for the Forest Before designing a mobile mapping system, consideration mus t be paid to the platform from which the direct georeferencing will occur. The subject environment plays a large role in platform selection. When operating in a forest, roads are often unpaved and bumpy. Additionally, the platform should be capable of goin g off the trails. Aerial vehicles would be ideal; however, trees would obscure imagery if flown above the canopy Flying amongst the canopy would be dangerous and unreliable due to line of sight issues. Thus, the availability of an all terrain vehicle for deployment of the GatorMMS provided a logical solution to any rugged terrain encountered Two primary versions of the ATV mounted GatorMMS have been deployed on board the 2006 Arctic Cat 400 4x4 ATV GatorMMS v1.0 was purely a navigation and georeferencing system testing platform. Figure 3 1 shows the v1.0 components including Novatel GPS/INS system and a c arbon fiber antenna frame mount. All components were mounted to the rear ATV speed rack with tie straps. Results of the testing will be discusse d in Chapter 4. Design limitations of GatorMMS v1.0 include flexing of the antenna frame mount during mobile observations, low profile of the GPS antenna relative to the ATV driver, prolonged setup time to secure all components to the ATV, and potential fo r shifting components due to ATV vibration and the loosening of tie straps. As shown in Figure 3 1 GatorMMSv2.0 has both primary mobile mapping sub systems: the georeferencing system and the imaging system. V2.0 addressed the limitations of v1.0 through th e use of a rigid aluminum frame for securing all subsystem components. Mounting this frame to a motorcycle stand bolted to the ATV

PAGE 40

40 elevated the system 16in above the ATV spe ed rack providing a better view for the imaging system and substantially decreasing the signal obstructions from the ATV driver. Details of the components of these MMS subsystems are given in subsequent Chapter 3 sections. 3.2 Georeferencing System Recall the georeferencing system is responsible for determin ing the time, position/locati on, and attitude/orientation of an event in space Often this event is the moment the shutter closes for an aerial or terrestrial photograph. T he exterior orientation parameters for the image are the georeferencing solution Prior to on board inertial navig ation sensors georeferencing a strip of aerial photos was only possible through a bundle adjustment utilizing a ne twork of ground control points (GCPs). With known coordinates for the GCPs, the bundle adjustment output include s the exterior orientation pa rameters for the attitude and position of each image. Figure 3 2 depicts indirect georeferencing for a stereo pair of aerial images from GCPs Direct georeferencing provided by o nboard INS sensors reduce s the need for dense networks of GCPs. Instead, GCPs can serve as check points or additional control redundancy in the bundle adjustment. Direct georeferencing has been a rapidly developing area of concern for geospatial experts including navigation, remote sensing, and photogrammetry specialists. The remain der of the section discusses the direct georeferencing components: GPS and INS. 3.2.1 GPS GPS position is the primary position input for the navigation component of MMS. GPS antenna s acquire signals from the GPS satellite constellation orbiting Earth. Thes e signals are transmitted to the GPS receiver for signal processing. After correcting for

PAGE 41

41 many error sources including timing and atmospheric effects, GPS coordinates are calculated from the intersection of satellite ranges. To obtain a position fix, signa ls from four satellites are necessary to solve for time and the position in three dimensions: latitude, longitude and ellipsoidal height ( Zengin and Yesil 2006 ) A ll GPS receivers are not created equal. Table 3 1 outlines the three classes of GPS receiver s: survey grade mapping grade, and consumer grade. For the most accurate direct georeferencing solution, a survey grade GPS antenna and receiver should be used. Alternatively, a mapping grade receiver can be used as long as GPS post processing techniques are capable of deriving survey grade accuracies from the mapping grade data. Recent research studies have shown that these GNSS processing algorithms with mapping grade GPS receivers are efficient in obtaining survey grade accuracies ( DiGruttolo 2010). 3.2 .2 GPS Aided Inertial Navigation Systems The INS forms the trajectory subsystem of the MMS. The IMU, the backbone of the INS, has two orthogonal triads of sensors. One triad consists of accelerometers to measure acceleration along three axes. Three orthogo nal gyroscopes form the triad used to measure rotational velocity along the three axes. The orthogonal property means the three axes (XYZ or ENU) form right angles to each other. The orientation accuracy of an IMU is mostly determined by the gyro drift rat e of the gyroscope sensor ( Schwarz and El Sheimy 2004 ) Thus, IMUs are classified according to the characteristics of the gyroscope drift rate. Table 3 2 classifi es the different grade gyroscopes based on these drift rates. Strategic grade and navigation grade IMUs are extremely expensive. Thus, their use in forest MMS research for future applications is not pr actical. Furthermore,

PAGE 42

42 strategic grade IMUs do not require GPS positional updates for accurate georeferencing after initialization for months at a ti me. Th eir usage would render the investigation of minute long GPS outages due to forest canopy obstruction moot. Tactical grade and low accuracy grade IMUs are more likely to be used in a terrestrial or inexpensive MMS. H ence, these grade inertial measurem ent units are investigated in this study. A ccelerometers form t he other sensor triad in an IMU. These sensors measure the acceleration of the body along the three orthogonal axes T h e accelerometer data set is integrated with the attitude (orientation) mea surements for an inertial trajectory determination. Typically, update rates for the INS system are 100 hertz (Hz) or times per second The typical position/velocity update rate from a GPS re ceiver for a GPS aided INS is 1 Hz. Without GPS positional updates the drift associated with the lower grade IMU sensors would result in a n un usable navigation trajectory. GPS and INS are complimentary because the errors are highly non correlated. The dominant INS error is long term drift error (error propagates over ti me); the dominant GPS error is short term white noise (errors for each moment of measurement are independent) ( Schwarz and Wei 2000 ) Thus, the INS system is best suited for filling in short gaps in the GPS determined navigation trajectory. There are two primary GPS/INS integration techniques. The simplest closed loop GPS/INS integration approach is loosely coupled (LC) integration ( Figure 3 3 ). The GPS position and velocity are computed in a separate filter before being integrated with the INS measurement s. A common but more complex integration technique is the tightly coupled (TC) integration approach ( Figure 3 4 ). Instead of integrating a previously

PAGE 43

43 computed GPS position and velocity into the GPS/INS integration filter, satellite phase ranges and range r ates a re integrated. The advantage of TC processing is the ability of the TC filter to use less than the previously required four satellite ranges. TC integration will be extremely important under heavy forest canopy where acquisition of four satellite ran ges at one epoch is unlikely. U nless four satellites were in view to calculate a GPS position the GPS ranges that were collected would not be used i n a LC integration approach Thus, use of all measurements in the TC integration approach should improve th e overall accuracy of the TC navigation trajectory. Both versions of GatorMMS (Figure 3 1) employed Novatel SPAN (Synchronous Position, Attitude and Navigation) system as the primary georeferencing system. The system consists of a tactical grade Honey well AG58 IMU, a survey grade Nova tel DL 4plus GPS receiver and a survey grade Novatel 702 GG antenna. The experimental low cost inertial navigation system consists of a Magellan AC12 L1 phase only GPS receiver w ith a low accuracy grade M EMS ense H3 IM U. GPS signals ar e split to each GPS receiver from the geodetic grade Novatel GPS 702 GG antenna. Figure 3 5 shows the relative size discrepancy in the housing between the 1/hr tactical grade AG58 IMU and the 1 00/hr low accuracy H3 IMU. 3.3 Imaging Syst em The imaging system comprises the second subsystem of a MMS. Historically, film cameras were the predominant imaging sensors. However digital cameras have superseded film cameras as the non LiDAR imaging sensor used in virtually all terrestrial based MM S. Instead of imaging to film, the light photons from the lens are converted to electrical charge by individual diodes arranged on a charge coupled device (CCD) array (Wolf and Dewitt 2000). Each diode corresponds with a pixel in the

PAGE 44

44 resultant image. While the CCD array eliminates the need for film distortion corrections, there are imperfections in the array from the manufacturing process that can be resolved in the camera calibration. 3.3.1 Camera The GatorMMSv2.0 employs a digital single lens reflex (DSLR ) Nikon D200 camera. This 10 megapixel (MP) camera is equipped with a flash sync terminal and a timer function capable of capturing 999 images at a minimum interval of 1Hz. The flash sync terminal was ess ential for providing an output signal that could be relayed to the DL 4plus receiver. After configuration, the Novatel receiver marked each time the shutter closed in the raw navigation data file. Without this functionality, the timing differences between the internal camera clock timer and the receiver c lock would have yielded imprecise results. The receiver clock is updated via the GPS signals. When configuring the D200 for a MMS test, attention was given to the DSLR settings that could sever e ly impact the quality of the terrestrial im agery. The most imp ortant settings were the aperture and the shutter speed. Aperture is the size of the opening in the lens which allows light onto the CCD array. Often times, the aperture is given a s an f stop number; a smaller f stop means a larger aperture. When using a l arger aperture, a greater proportion of the lens is being used. S ubsequently this increases the amount of distortion present in the resultant image. The shutter speed is the time between the mechanical shutter open ing and clos ing Thus, the shutter speed also affects the amount of light reaching the CCD array. When imaging static objects from a moving platform or moving objects from a static platform, the goal is to use the quickest shutter speed to capture the objects without blur. The basic concept for M MS operation is to find a balance between using the smallest aperture to reduce distortion

PAGE 45

45 and the quickest shutter speed that still allows in enough light to properly image objects on the CCD array. The wide variation of shadowed areas in the forest along the MMS test track further complicated the lighting for determination of the proper settings. The D200 lacks a full frame CCD array. This means images captured with a 50mm lens will have a smaller FOV on the D200 than images captured with a 50mm lens on a film or full frame DSLR camera. To determine the equivalent focal length for a full frame camera, a focal length multiplier (FLM) needs to be used with the focal length for the cropped sensor. The D200 has a 1.5 FLM. 3.3.2 Lens es To properly implement vi sion aiding through the use of stereo models, overlap between successive images is essential. Since the minimum timing interval for the D200 is 1Hz and the D200 lacks a full frame two methods existed to get the n ecessary overlap: increase the field of vie w (FOV) or travel at a slower speed. For future applications, increasing the FOV was the most logical approach. A fixed focal length 24mm f/2.8D AF Nikkor wide angl e (WA) lens and a n 8mm f/3.5 Manual Focus Pro Optic fish eye (FE) lens were explored as two lens es that could increase FOV for the mobile mapping operation (Figure 3 6) The e quivalent focal length for these lenses is 36mm and 12mm, respectively. The WA lens has central perspective geometry found in the common pinhole camera model Figure 3 7 il lustrates the difference between WA central perspective geometry and FE projection geometry. T he FE lens captures a much greater FOV; however, the tradeoff of this expanded FOV is a substantial increase in radial distortion. Thus, t he calibration model for a fisheye lens is more complex than the standard pinhole

PAGE 46

46 camera (central perspective geometry) calibration model. Camera calibration is discussed in detail in Chapter 5. The D 200 DSLR camera is capable of being mounted either horizontally or vertically on the GatorMMSv2.0 aluminum frame (Figure 3 6) The horizontal orientation is for the VA component of the research. The vertical orientation is for capturing images of the forest canopy for canopy density determination. 3. 4 MMS Coordinate Systems The Gator MMS georeferencing and imaging system s have been discuss ed as separate entities to this point Vision aiding relies on the integration of both subsystem s i nto one MMS To discuss sensor int egration, knowledge of the sensor coordinate frame s (CF ) and sensor offsets is necessary. The discussion of these relationships that follows is derived from Ellum and El Sheimy ( 2002 ) Figure 3 8 depicts the physical relationship between the origin of the IMU orthogonal axes the GPS antenna reference point (ARP), and the incident nodal point of the camera lens. For simplicity, the three sensors are referred to as IMU, GPS, and camera. The mathematical relationship between the coordinates of a point in the mapping CF ( ) and the same point in the camera CF ( ) is expressed in Equation 3 1 as : (3 1) where C oordinates of the GPS ARP in the mapping C F R otation ma trix from the IMU C F to the mapping C F R otation matrix from the camera C F to the IMU C F

PAGE 47

47 V ector from the camera to the GPS ARP in the camera C F Sc ale between the camera C F and mapping C F Determining and is part of the boresight and lever arm calibration discussed in Chapter 5. Using the mathema tical relationship between the imagi ng and georeferencing subsystems makes the vision aiding component of MMS operation feasible

PAGE 48

48 Table 3 1 GPS receiver classification. Note that accuracy is dependent on many factors. This is expected accuracy in optimal conditions. GPS Receiver Class/G rade Expected Accuracy (m) Survey 0.01 Mapping 1 Consumer/Recreational/Navigation 5 10 Table 3 2 Inertial navigation gyroscope c lassification ( Schwarz and El Sheimy 2004 ) Gyroscope Class/Grade Constant Drift Rate (deg/h) Strategic 0.0005 0.001 Navigation 0.002 0.01 Tactical 1 10 Low accuracy 100 10,000 A B Figure 3 1. GatorMMS platform configurations A) v1.0 Georeferencing subsystem only, B) v2.0 Georeferencing & imaging subsystems (images courtesy of author)

PAGE 49

49 Figure 3 2 Indirect georeferencing Collinearity condition for an aerial stereo model with EOP1 and EOP2 derived from the known coordinates of GCP D, GCP E, and GCP F Figure 3 3 Loosely coupled GPS/INS integration flow c hart (revised with permission from Mohamed, A.H. 2010. SUR 6535 GPS/INS Integration Course Notes. University of Florida: Gainesville, Florida. )

PAGE 50

50 Figure 3 4 Tightly coupled GPS/INS integration flow c hart (revised with permission from Mohamed, A.H. 2010. SUR 6535 GP S/INS Integration Course Notes. University of Florida: Gainesville, Florida. ) Figure 3 5 Georeferencing system components for GatorMMSv2.0 (image courtesy of author)

PAGE 51

51 Figure 3 6 Imaging system lenses for GatorMMSv2.0. The hose clamps affixed to e ach lens set the focus at infinity. This was important for maintaining consistency in the data from calibration through data acquisition. (images courtesy of author) Figure 3 7 Lens geometry comparison: wide angle vs. fisheye ( modified after Schneider D., E. Schwalbe, and H. G Maas. 2009. Validation of geome tric models for fisheye lenses. ISPRS Journal of Photogrammetry and Remote Sensing 64 (3): 259 266 )

PAGE 52

52 Figure 3 8 Physical relatio nship between GatorMMS sensors i ncluding the IMU, the GPS ARP, a nd the D200 camera ( modified after Ellum, C.M. and N. El Sheimy. 2002. The Calibration of Image Based Mobile Mapping Systems. In Proceedings of the 2nd Symposium on Geodesy for Geotechnical and Structural Engineering Berlin, Germany: The International Ass ociation of Geodesy (IAG) ).

PAGE 53

53 CHAPTER 4 GEOREFERENCING SYSTE M TESTING 4.1 Study Site Austin Cary Memorial Forest Austin Cary Memorial Forest (ACMF) is a 2043 acre southern pine flatwoods forest administered under the direction of the School of Forest Reso urces and Conservation at the University of Florida ( Muller and Maehr 2000; Powell et al. 2005 ) The forest serves t hree primary purposes: resident/ community /student education, extension and demonstration, and research. This managed forest has a multitude of roads and fire trails which served as an excellent location for testing the georeferencing system, GatorMMS v1.0. Figure 4 1 shows a map of the location of the forest in relation to downtown Gainesville in North Central Florida. The goal of the GatorMMS v1.0 testing was to gain experience in the field with different GPS/INS acquisition methods and investigate various processing schemes with Waypoint Inertial Explorer 8.10 (WIE) The subsequent analysis provided a template for the processing and data collection schemes with GatorMMSv2.0. 4.2 Data Acquisition On March 26, 2010, the SUR 6535 GPS/INS Integration class collect ed inertial navigation data using GatorMMSv1.0 at ACMF After strapping the Novatel GPS aided INS to the ATV, the class members traverse d through the forest on primary and secondary roads that may have been inaccessible via conventional automobile. In the summer of 2009, ground control points ( GCPs ) were installed along these roadways (Figure 4 2). Static GPS sessions were performed to establish coordinates for these GCPs through a GPS network adjustment using the Gainesville (GNVL) Continuously

PAGE 54

54 Operating Reference Station (CORS) as the master control point. GNVL is located at the Gainesville Regional Airport approximately 8km from ACMF. During the test runs the ATV was stopped on a number of the GCPs to perform a static observation. The ATV was backed down over the point by centering the GPS antenna as close as possible to the GCP. The static sessions served a two fold purpose for the post processing of the data. First, Zero Velocity Updates (ZUPTs) would be used to bound the propagation of error in position, velocity and attitude. Furthermore, Coord inate Updates (CUPTs) could be introduced into the processing filter to aid in the navigation solution computation This CUPT trajectory serve d as a check for the navigation solution versus the non CUPT trajectory F ifteen static observations were made on fourteen o f the twenty GCPs (one GCP was observed twice). Before traversing the forest, INS initialization was necessary. In strap down systems, this process is called analytical gyrocompassing. While the vehicle is stationary, sensor measurements are rec orded so the initial attitude of the system can be resolved. It is advisable that stationary initialization be performed at the beginning and end of each session for two reasons. With good satellite visibility, ambiguity resolution (AR) of stationary GPS d ata is easier than AR on kinematic GPS data. Second, the end of session stationary period enables the processing software to perform a backwards filtering operation with greater precision since the end attitude alignment is more easily resolved. Two trave rse loops were performed by the class members: Loop West and Loop East ( Figure 4 3 ). The origin for both loops was in the vicinity of GCP 1 near the ACMF offices. Before processing the inertial navigation data, the GPS only solution needed to

PAGE 55

55 be computed u sing GNVL as the master control station This Global Navigation Satellite Systems (GNSS) solution provide s the initial position for the INS free navigation and the position updates for the GPS aided INS navigation. 4.3 GPS/GNSS Processing in the Forest F or the initial GPS processing, the Kinematic Ambiguity Resolution (KAR) method was implemented using the WIE software. T he WIE H elp M anual recommends using KAR f or periods of extremely poor geometry or loss of lock (Inertial Explorer 2008) B oth are scena rios encountered in a forest environment with a multitude of satellite obstructions from the tree canopy. Figure 4 4 is a plot of the available GPS satellites during the observation period. The solid lines indicate satellite availability for the given geog raphic position where colors correspond to different elevations and the red bars indicate a loss of lock or no observability by the Novatel GPS receiver. T he forest canopy provided plenty of GPS signal outages. Figure 4 5 is a plot showing the ambiguity resolution status of the KAR processing. At most, one baseline solution was fixed with a high number of outage s or float solutions. Redundant fixed ambiguity baselines are imperative for optimal determination of observation location. Figure 4 6 s hows the resultant KAR processing solution using WIE's defaults (automatic forward and reverse processing, loose DOP standards, etc.). The bright green areas have the highest quality solutions (Q1) while the bright red areas have the lowest quality solution (Q6). U nfortunately, areas with solutions are few and far between considering the length of the entire trajectory. Thus before proceeding with the inertial data processing, a better GPS solution was sought The Advanced Real Time Kinematic (ARTK) WIE GNSS process ing profile was used next. Given the results in Figure 4 7 and Figure 4 8 the ARTK profile provided a

PAGE 56

56 much better position solution (more solutions along trajectory including more Q1 solutions) due to a higher number of fixed ambiguity baselines ( teal bar s in Figure 4 8 ). Having this improved GPS solution trajectory means more frequent and more reliable position updates in the inertial processing for the GPS aided inertial navigation. Also, greater confidence can now be placed on the initial posit ion coord inates used in the INS free n avigation processing. 4.4 INS Free Navigation When performing IMU only processing, the default for WIE is to choose the first epoch for initial position. Unfortunately, this means that the estimated initial velocity and positio n standard deviations will be quite large. Since the initial error propagates throughout the entire trajectory bounded only by the Schuler oscillation in the horizontal channels and unbounded in the vertical channel, the optimal initial value will be deriv ed from an initialization epoch with the highest precision (smallest standard deviations). For this field work, the original default epoch was the 400820 th second of the GPS Week while the optimal epoch was the 400980 th second of the GPS Week. The optimal epoch had standard deviations about one half of the original epoch standard deviations for velocity in the horizontal and vertical channels. It is apparent from Figure 4 9 that even with the optimal initial position, the gyroscope drift and misalignment ma kes the navigation trajectory completely unreliable. The amount by which the position drifts is remarkable in the two hour data acquisition sessio n ( N ote the scale: each box is 100 km by 1 00 km). Furthermore, the WIE IMU only processing determines ZUPTs aut omatically. Thus, the bounding of the errors in the horizontal channels is not seen in Figure 4 10 When updates occur, the Schuler oscillation is broken. Furthermore, the estimated

PAGE 57

57 precision of the position in Figure 4 11 confirms what the reader already intuitivel y knows: the INS free n avigation trajectory is not reliable. While position updates from the previously determined GNSS solution create a drastic improvement in the trajectory, the author tested an IMU only solution that included both ZUPTs and C UPTs. Table 4 1 has a complete list of the ZUPTs and CUPTS used. The start and end times for the ZUPTS were determined by using the field notes to estimate the time at each point and interpolating the beginning and end times from the velocity profile of th e INS free navigation solution. The GCP coordinates for the CUPTS were provided by the instructor from previous processing of the GCP static observation sessions. The projected coordinates are in UTM 17N WGS84. While the solution as seen in Figure 4 12 is not usable as a navigation trajectory, analysis of the errors in Figure 4 13 and Figure 4 14 show s the effect that the updates have on the propagation of errors in the navigation solution. The velocity accuracy plot show the errors propagating until a ZUPT is processed. At which point, the error 'resets' and drifts again until the next ZUPT. Likewise, the CUPTs perform the same function in the positional accuracy plot. The CUPTs contribute to the spider web like effect of the navigation trajectory plot as s een in Figure 4 12 Even with a tactical grade IMU such as the SPAN system (1/hr), the inertial data and infrequent updates are not sufficient enough to provide a usable navigation trajectory. Thus, the development of GPS aided inertial navigation is nec essary at this point. This complimentary pair of navigation systems works so well because the errors are highly non correlated. The dominant GPS error is short term white noise (errors occur on an epoch by epoch basis) while the dominant INS error is long term drift error

PAGE 58

58 (as already seen in IMU only navigation). Thus, the extremely precise INS system is best suited for filling in short gaps in the GPS determined navigation trajectory. Through the following analysis of GPS aided inertial navigation, improve ments in the navigation trajectory are quite prevalent. 4.5 GPS A ided INS Navigation There were three primary objectives in exploration of GPS aided inertial navigation (GAIN). First a brief comparison of forward only processing versus combined backward a nd forward processing was sought. The second objective was comparison of the GAIN solution of the GCPs with the known GCP coordinates. The GAIN trajectory positions from the ZUPT intervals were averaged and compared to the known coordinates. Third, the aut hor sought to compare GPS/INS integration techniques to compare their respective trajectories. Specifically, loosely coupled and tightly coupled integration approaches were compared. As discussed in Chapter 3, the simplest closed loop GPS/INS integration a pproach is loosely coupled (LC) integration. With LC integration, the GPS position and velocity are computed in a separate filter before being integrated with the INS navigation states. A common but more complex integration technique is the tightly coupled (TC) integration approach Instead of integrating a previously computed GPS position and velocity into the GPS/INS integration filter, satellite phase ranges and range rates are integrated. The advantage to this is the ability of the filter to use less th an the LC required four satellite ranges. Thus, all measurements can be used in the TC integration approach which should help the overall accuracy of the TC navigation trajectory. During WIE processing, the user has the choice of using a forward only a ba ckward only or a combined (backward & forward) solution. This means the

PAGE 59

59 integration techniques described above are run using the initial epoch (forward) or alternatively, the final epoch in the data set becomes the first epoch (backward). The combined so lution uses a filtering technique (Kalman Filter, EKF, etc) to combine the forward and backward solution. If further filtering of the navigation trajectory is desired, then a Rauch Tung Striebel (RTS) smoothing algorithm can be implemented. After all WIE p rocessing, the RTS combined solution will be considered the final GAIN trajectory for each integration technique. 4.5.1 Loosely Coupled GPS/INS Integration When performing filtering in one direction (i.e. forward only) after long periods of GPS position ou tages, the GAIN trajectory will drift. Recall the central problem with INS navigation is the long term drift error when no GPS position updates are available. In the Figure 4 15 trajectory drift in one directional processing is visible along the eastern e dge of the eastern loop as the loop merges with the western loop and where the western loop returns to the origin. Recall the red portions of the trajectory are the Q6 positions. Thus, the longer the ATV went without receiving adequate GPS signals from at least four satellites, the INS error propagated. When the backwards LC integration technique was implemented, the drift was visible on the opposite ends of the GPS position/velocity update outages. When the two solutions were combined via filtering, the re sultant GAIN trajectory in Figure 4 16 h ad no significant drifting. Thus, it is imperative that a combined approach be taken with GAIN trajectory determination. For further emphasis regarding the gains in a combined solution, Figure 4 17 and Figure 4 18 sh ow the positional accuracy for the forward only processing solution was approximately three times worse than the combined solution. Similarly, the velocity

PAGE 60

60 accuracy for the forward only processing solution in Figure 4 19 was approximately two times worse t han the combined solution in Figure 4 20 This discussion focused on a comparison of intermediate steps in the LC integration approach. The purpose was show ing the increases in accuracy that are made in processing beyond the initial forward only processing Significant increases in accuracy we re also made between the combined filtering and smoothing steps These comparisons will be discussed in the next integration technique comparison section 4.5.2 Loosely Coupled Integration and Tightly Coupled Integrati on Comparison Using the GatorMMSv1.0 data, a comparison of GPS/INS integration techniques was undertaken. Figure 4 21 and Figure 4 22 show the resultan t combined and smoothed LC and TC trajectories respectively The solutions seem fairly similar with subt le differences in the quality of the rendered trajectories (i.e. the hook off the eastern loop, the northeast straightaway on the eastern loop, etc.). Furthermore, the positional accuracy plots as see in Figure 4 23 and Figure 4 24 are almost identical. Th e maximum peaks on the TC positional accuracy plot had peaks that were slightly lower than the maximum peaks on the LC positional accuracy plot Intuitively, this makes sense as the receiver in these poor positional accuracy areas most likely was receiving signals from less than 4 satellites. The satellite ranges were used in the computation of the GPS aided INS positions but the estimated precision of these positions was only slightly better than having no complete (at least 4 satellites) position in the L C integration model. Similar results between the 2 techniques were found in the velocity error plo ts which have not been included A more empirical navigation trajectory comparison was necessary to determine the similarity between the two GPS/INS navigatio n trajectories. Both trajectories were

PAGE 61

61 output from WIE in geographic coordinates (latitude/longitude WGS84) and projected grid coordinates (UTM, Zone 17N WGS84). T he difference between the TC integration approach and the LC integration approach formed the basis of the quantitative analysis using root mean square error (RMSE) calculations In this analysis, the 'error' in RMSE refers to the discrepancy between the two approaches. Since coordinate differences are easier to grasp in meters rather than arcs econds, the UTM projected coordinates were used as the basis for comparison. From Table 4 2 the RMSE calculations of the 7964 shared observations show the discrepancy in the northern coordinates was 50% greater than the discrepancy in the eastern coordina tes. The RMSE in the vertical channel was slightly greater than the northern coordinate RMSE. Horizontally, the t wo trajectories were within 1f t (~0.3m) of each other. Three dimensionally, the discrepancy was less than a foot and a half (~0.4m). From Figur e 4 25 and Figure 4 26 the coordinate differences between the two GAIN trajecto ries were never larger than 2.5 m in any channel. Thus, it is reasonable to conclude that these two GPS/INS integration approaches achieved consistently similar results with the implementation of filtering and smoothing algorithms 4.5.3 GPS Aided Inertial Navigation Trajectory Absolute Accuracy Analysis After establishing relative accuracy between the two integration approaches, an analysis of absolute accuracy was necessary. This process was accomplished through averaging position coordinates during the ZUPT intervals and comparing these to the known GCP coordinates. The GCP coordinates are from previous static observation of the ground control. The author was unable to obtai n the original observation data for verification so he relied on the given coordinates. The heights were given in orthometric height. Without knowledge of the geoid used, the author applied a generic geoid

PAGE 62

62 undulation correction to the entire area. The corr ection of 27.868 m was obtained from the National Geodetic Survey's GEOID03 Geodetic Toolkit for the given latitude and longitude of GCP1 In the future, it is suggested that ellipsoidal heights be used for all trajectory comparisons. From Table 4 3 and Ta ble 4 4 there are four blunders in identifying the ZUPTs (error in the author's interpolation of the velocity profile). More importantly, a systematic error in the data is quite prevalent for the other 11 points. One can call this a bias due to the magnit ude of the standard deviations (mm to cm) for the ZUPT session average s relative to the magnitude of the bias (m). The 3 m bias would not be caused by the random centering and misleveling errors caused by inaccurate setup positions. Given the relative accu racy of the LC and TC integration solutions, the most likely cause of the systematic error is a datum shift. After extensive exploration of the given GCP coordinates and ArcGIS files associated with previous work at ACMF, the author was unable to ascertain the source of the datum shift. The author can assert that the coordinate trajectories provided in this analysis we re all properly output from WIE in UTM projection 17N WGS84. Without being able to locate the original observation files used in the proces sing of the static GCP baseline solutions, the absolute accuracy cannot be definitively answered. Lacking confidence in the known GCP coordinates, further processing with CUPTs was not investigated at this time. However, once the original files are located a comparison of geographic coordinates is suggested to avoid the software programs (WIE vs. ArcGIS) using different NAD83 to WGS84 datum transformations as well as eliminating potential discrepancies in projections.

PAGE 63

63 4.6 Georeferencing System Testing Les sons Learned From the georeferencing system testing with GatorMMSv1.0, the following lessons can be used in future GPS aided inertial navigation analysis with subsequent GatorMMS generations. In WIE, the ARTK profile is better at resolving kinematic intege r ambiguity than the default KAR profile for terrestrial based mobile mapping in the forest. Thus, AR TK should be used for future GatorMMS data sets. Furthermore, whether implementing loosely coupled or tightly coupled integration in WIE, output trajectori es should always be combined and smoothed for optimal results For the current data set, t he given GCP coordinates are unreliable due to lack of raw observation files and datum conversion uncertainties. I f short baseline observations while mobile mapping a re desired a new GCP network should be establ ished in the project study area. Additionally, the ellipsoidal height should be used for MMS height analysis. This will avoid the need for geoids and orthometric heights. Numerous issues encountered with the us e of zero velocity and coordinate updates can be used as learning tools moving forward. First, GPS free inertial navigation using ZUPTs is not accurate enough for mobile mapping with a tactical grade IMU. Second, c oordinate updates are difficult to incorpo rate into the processing accurately due to orientation and misalignment error when driving over the GCPs. Lastly, i f zero velocity updates are to be integrated with GPS observations accurate field records must maintained for ZUPT intervals to minimize int erpolation errors

PAGE 64

64 Table 4 1. Zero velocity updates and coordinate u pdates f or inertial n avigation p rocessing Column 1 shows the ground control point for each CUPT/ZUPT. Columns 2 4 are the coordinates for each CUPT. Column 5 is the estimated GPS time f rom the field notes. Columns 6 7 are the interpolated time intervals from the inertial navigation trajectory. GCP Easting (m) Northing (m) ell. h (m) GPS Time (s) GPS Start (s) GPS End (s) 2 382211.505 3291945.804 21.426 403200 403230 403430 20 381841.110 3291290.946 20.593 403680 403680 403745 5 382071.696 3289614.479 15.142 403860 403820 403880 15 381963.261 3289976.695 19.566 404100 404030 404080 14 382048.035 3289918.497 18.306 404220 404150 404180 13 382216.838 3289900.409 17.987 4043 40 404250 404305 12 382355.906 3289946.823 17.078 404400 404365 404435 11 382389.884 3290097.208 17.884 404580 404505 404575 10 382400.410 3290285.061 19.029 404700 404660 404695 1 382476.296 3291663.698 21.875 404880 404905 405305 18 382991.968 32920 85.940 22.455 405600 405590 405660 17 383910.385 3292751.706 21.543 405840 405850 405920 16 383870.391 3291761.493 19.395 406080 406135 406255 3 383840.367 3291325.734 17.462 406500 406425 406470 1 382476.296 3291663.698 21.875 407400 407380 407730 T able 4 2. Tightly coupled & loosely coupled GPS aided inertial navigation solution comparison DE DN D Hor DE 2 DN 2 DZ DZ 2 (m) (m) (m) (m 2 ) (m 2 ) (m) (m 2 ) Average 0.027 0.009 0.107 0.010 SD 0.161 0.249 0.278 0.270 RMSE 0.163 0.249 0.270 RMSE_Hor 0.298 RMSE_All 0.402

PAGE 65

65 Table 4 3. Absolute accuracy investigation of loosely coupled integration through comparison of the loosely coupled ZUPT interval average with the CUPT GCP D E D N D h ErrorDist Ver 2 0.699 2.480 1.499 2.981 0.093 0.025 20 1.130 2.065 1.315 2.696 0.027 0.009 5 294.262 1322.827 6.420 1355.177 0.062 0.013 15 1.203 2.658 1.540 3.299 0.005 0.010 14 0.912 2.457 1.369 2.957 0.014 0.019 13 1.511 2.338 2.934 4.044 0.174 0.069 12 0.839 2.194 1.558 2.819 0.015 0.025 11 0.532 2.323 1 .589 2.864 0.011 0.012 10 0.328 2.332 1.603 2.849 0.131 0.034 1 0.917 2.647 1.460 3.159 0.011 0.024 18 0.852 2.666 1.424 3.140 0.009 0.020 17 0.4 70 3.463 1.111 3.667 0.012 0.006 16 1395.009 95.153 3.930 1398.256 0.007 0.011 3 1622.682 397.257 4.496 1670.608 0.008 0.015 1 1393.856 193.161 2.512 1407.178 201.459 1.022 Table 4 4. Absolute accuracy investigation of tightly coupled integration through comparison of the tightly coupled ZUPT interval average with the CUPT GCP D E D N D h ErrorDist 2 0.698 2.479 1.496 2.979 0.093 0.021 20 1.130 2.064 1.312 2.694 0.009 0.021 5 294.262 1322.827 6.421 1355.177 0.061 0.013 15 1.216 2.650 1.561 3.308 0.004 0.011 14 0.950 2.472 1.467 3.027 0.054 0.122 13 1.313 2.176 3.508 4.332 0.077 0.065 12 0.856 2.107 1.882 2.952 0.067 0.290 11 0.460 2.312 1.672 2.890 0.100 0.116 10 0.265 2.590 1.059 2.810 0 .350 0.342 1 0.917 2.646 1.460 3.159 0.011 0.025 18 0.851 2.666 1.424 3.140 0.010 0.022 17 0.470 3.460 1.115 3.665 0.012 0.005 16 1395.010 95.154 3.930 139 8.257 0.007 0.011 3 1364.997 340.625 5.888 1406.868 0.008 0.016 1 1393.833 193.161 2.530 1407.156 201.453 1.042

PAGE 66

66 Figure 4 1. Vicinity map of Austin Cary Memorial Forest Figure 4 2. Static GPS observation of ACMF ground control points. Each GCP is monumented with a 36" rebar reinforcing rod, held in place by an anchor bolt, and flagged with a rigid plastic numbered witness. (image courtesy of author)

PAGE 67

67 Figure 4 3 GatorMMSv1 .0 georeferencing system test tracks in Austin Cary Memorial Forest Figure 4 4. GPS satellite status

PAGE 68

68 Figure 4 5. GPS baseline status (float vs. fixed ambiguity) KAR processing Figure 4 6. GPS solution plot KAR processing. Note that green is the best quality solution (Q1) and red is the worst quality solution (Q6).

PAGE 69

69 Figure 4 7. GPS solution ARTK processing Figure 4 8. GPS baseline status (float vs. fixed ambiguity) ARTK processing

PAGE 70

70 Figure 4 9. IMU only navigation trajectory Figure 4 10. INS free navigation velocity accu racy plot

PAGE 71

71 Figure 4 11. INS free navigation position accuracy plot Figure 4 12. INS free navigation ZUPTs and CUPTS trajectory

PAGE 72

72 Figure 4 13. INS free navigation ZUPTs and CUPTS velocity accuracy Figure 4 14. INS free navigation ZUPTs and CUPTS positional accuracy

PAGE 73

73 Figure 4 15. GPS aided INS navigation forward only solution Figure 4 16. GPS aided INS navigation combined filtered solution

PAGE 74

74 Figure 4 17. GPS aided INS navigation forward solution positional accuracy Figu re 4 1 8. GPS aided INS n avigation combined filtered solution positional a ccuracy

PAGE 75

75 Figure 4 1 9. GPS aided INS navigation forward s olution velocity a ccuracy Figure 4 20 GPS aided INS n avigation combined filtered s olution velocity a ccuracy

PAGE 76

76 Figure 4 21. GPS aided INS navigation loosely coupled combined and smoothed navigation trajectory Figure 4 22. GPS aided INS navigation tightly coupled combined and smoothed navigation trajectory

PAGE 77

77 Figure 4 23. GPS aided INS navigation loose ly coupled combined and smoothed positional accuracy Figure 4 24. GPS aided INS n avigation tightly coupled combined and smoothed positional accuracy

PAGE 78

78 Figure 4 25 Tightly coupled & loosely coupled horizontal GPS aided inertial navigation trajector y comparison Figure 4 26 Tightly coupled & loosely coupled vertical GPS aided inertial navigation trajectory comparison

PAGE 79

79 CHAPTER 5 IMAGING SYSTEM TESTI NG 5.1 Self Calibrating Bundle Adjustment Obtaining accurate and precise results from the georefere ncing system is only valuable for MMS operation if the imaging system is calibrated Knowledge of the intrinsic/ interior orientation parameters (IOPs) and extrinsic/exterior orientation parameters (EOPs) is imperative for vision aiding implementation. Resu lts from a self calibrating bundle adjustment (SCBA) include both the IOPs and EOPs. The computational process of analytic SCBA is a common photogrammetric process that incorporates the use of augmented collinearity equations. The augmentation is d ue to th e incl usion of the additional IOP terms including principal point offsets, calibrated focal length, radial lens distortion, and decentering lens distortion (Wolf and Dewitt 2000). The SCBA mathematical model presented in Equations 5 1 through Equation 5 8 is from Wolf and Dewitt (2000) where x a y a are measured photo coordinates, x o y o are principal point coordinates, k 1 k 2 k 3 are symmetric radial lens distortion coefficients, p 1 p 2 p 3 are decentering distortion coefficients, and f is calibrated focal length. (5 1) (5 2) (5 3)

PAGE 80

80 (5 4) (5 5) The terms ( q r s ) are from the linearization of the collinearity equations where m XX are elements of the 3x3 Omega Phi Kappa (OPK) rotation matrix, X A Y A Z A are object point coordinates, and X L Y L Z L are exposure station coordinates. (5 6) (5 7) (5 8) If the IOPs were omitted, the focal length and principal point offsets would need to be known from a previous calibration to perform a bundle adjustment with the non augmented collinearity e quations. The EOPs would be the only parameters solved for in this case. 5.2 Camera Calibration Interior Orientation Recall from Chapter 3 (Figure 3 8) that the offsets between the camera and GPS ARP are needed to determine the lever arm. Likewise, the r otation matrix between the camera coordinate frame (CF) and the IMU CF is necessary to rotate points captured by the imaging system into the mapping CF for vision aiding This rotation matrix is called the boresight calibration. The goal of performing a SC BA for determining the IOPs is to eventually fix the se intrinsic parameters when perform ing the boresight angle lever arm (BSLA) calibration.

PAGE 81

81 Using a consumer off the shelf (COTS) DSLR camera has its potential disadvantages relative to using a traditional aerial photogrammetric sensor. Traditional aer ial photogrammetric sensors have extremely stable interior orientation parameters. After calibration, the IOPs of these sensors are considered known parameters. Fewer unknowns in the bundle adjustment can resul t in higher confidence in the EOP results. Accurate and precise EOPs are essential for the BSLA calibration. As COTS DSLR cameras have been us ed increasingly in close range terrestrial applications, t he IOPs for these cameras ha ve been routinely studied t o better understand the geometric consistency and stability of the intrinsic parameters. ( Wackrow et al. 2007) studied the COTS Nikon Coolpix 5400 camera During the investigation of 7 identical camera/lens combinations, the authors determined that the l ens distortion curves were stable ov er the 1 year testing interval Th e IOP stability from the Wackrow et al. (2007) study was encouraging that the IOPs for the 24mm Nikkor lens ( focus fixed at infinity and aperture fixed at f/22 ) would also have geometri c st ability through BSLA calibration and VA field testing. 5.2.1 Methods Two software programs were used for camera calibration, PhotoModeler 6 (PM) by Eos Systems Inc. and SCBUN by Bon Dewitt, PhD. SCBUN implements the SCBA mathematical model discussed in Section 5.1. Without explicitly indicating in the program documentation which SCBA model PM implements, the PM camera calibration module outputs the same interior orientation parameters as SCBUN. U sing PM is advantageous for automark ing hundreds of targ ets with sub pixel accuracy After performing camera calibration with PM, the results were verified against SCBUN. Before

PAGE 82

82 comparing IOP results from the SCBAs Table 5 1 outlines the primary differences in the programs that needed to be accounted for For the camera calibration process, a 3ft by 3ft calibration grid of 144 targets was used (Figure 5 1 ). To orient the camera in object space, f our of the grid targets were auto coded fiducial targets Sub pixel automarking of the targets was achieved through l east squares matching or centroid determination. Since the object space distances between each target in the grid were known, targets which did not fit the object space coordinates with 95% certainty were rejected prior to the SCBA. Figure 5 1 also depicts a typical image after the automarking process. Due to the use of a flat sheet calibration field, t he main drawback to using PhotoModeler 6 for the camera calibration is the lack of depth of field. This property makes the focal length a difficult paramete r to resolve. A subsequent version of PM PhotoModeler 2011, has remedied this issue through t he use of multi sheet calibrations with autocoded targets. The sheets can be placed at varying heights giving proper depth of field. The author did not have acce ss to the newer version of PM but suggest s that others use that newest PM version for intrinsic camera calibration. 5.2.2 Results Two intrinsic camera calibrations were run using PM to ensure calibration consistency The data sets were collected on differe nt days after attaching and detaching the 24mm lens. PM requires at least 6 photos of the calibration grid from a variety of angles and positions. When capturing images, t he main premise is to fill the image with grid targets to properly model the lens dis tortion at the periphery of the image. The decision to use the smallest aperture setting available (f/22) was based on

PAGE 83

83 reducing distortion for MMS operation Table 5 2 outlines the IOP result s for both 24mm Nikkor Nikon D200 calibration tests Since the intrinsic parameters directly affect the image coordinates (measured in pixels) it makes sense to convert these IOPs from mm to pixels. Furthermore, the mm pixel conversion needs to be completed to verify the results with SCBUN. The dimensions of the D200 CCD array are 3872 x 2592 pixels. Using the format width and format height from each SCBA, the number of pixels per mm in height and width can be determined. The average of pix/mm(h) and pix/mm(w) is the conversion factor used. Table 5 3 shows the PM cali bration results in an easier format to understand relative to the image points being measured. As previously stated, the focal length is the most difficult parameter to resolve due to lack of depth of field. Thus, the geometric consistency between the two solutions especially with the radial distortion modeling is excellent. One additional check was performed to verify the solution uses the correct SCBA mathematical model The image coordinates, control coordinates, and exterior orientation parameters were input into SCBUN. The interior orientation parameters were allowed to adjust without constraint T he results in Table 5 4 show that the SCBA model used by PM is consistent with SCBUN. The radial distortion curve produced from the SCBUN IOP results is shown in Figure 5 2. This figure shows the polynomial increase in distortion relative to the increase in the radial offset distance an image object is from the principal point. The SCBUN results were held for the BSLA calibration. 5.3 Boresight Lever Arm Calibr ation When designing GatorMMSv2.0, it was important that detaching and reattaching the D200 camera could be done with repeatable results. Thus, a goal of the design was

PAGE 84

84 a fixed BSLA calibration. The bottom of the D200 has two sockets: a threaded tripod soc ket and a smaller locking socket. By fixing the camera to the aluminum frame with a 20 tripo d screw, the orientation was repeatable by inserting the head of a fixed machine screw into the locking socket. The u s e of two screws provided repeatable B SLA results for both the horizontal and vertical D200 orientations 5.3.1 Reed Lab Roof BSLA Calibration Field To perform a BSLA calibration, establishment of a network of ground control points is necessary for accurate EOP determination from the SCBA. Thu s, the GCPs needed mapping CF coordinates established with geodetic control. GPS observations and total station surveying were used to establish these mapping CF coordinates. As the home to the University of Florida Geomatics department, the roof of Reed L ab (RLA) shown in Figure 5 3 provided a convenient location to perform the BSLA calibration. Three observation pillars are established on the RLA roof for previous and ongoing geodetic observation/testing. One of the pillars is used for the Florida Departm ent of Transportation CORS station, RLAB. The pillar on the southwest corner of the RLA roof (RLABsw) was the occupation station for the reflectorless total station survey of the GCPs. A backsight (BS) for the TS survey was established 57 m away on the g r ound near Rhines Hall. Each GCP target was observed with four sets of forward and reverse face reflectorless TS shots for maximum redundancy. Figure 5 4 shows a majority of the GCP targets viewed from the BSLA calibration area The GCP targets were primari ly retro reflective targets with a 2mm diam e ter center hole used for sighting purposes To establish geodetic control, two hour plus static GPS observations were undertaken simultaneously at RLABsw, BS, RLAB and GNVL on four separate days in

PAGE 85

85 September 2010 GrafNet software package using GNVL and RLAB as master control stations. While GrafNet is purely a geodetic processing software package, the GNSS processing engine s are the same GNSS engine s used when processing the inertial trajectories. This ensures consistency in the GPS /GNSS processing solutions. T he results of the network adjustment were imported into Topcon Corpora Tools 7.2 software, a package capable of processing both total station and GPS observations. When adjusting the total station observations, t he WGS84 GNSS solution from GrafNet was held as control The sub cm precision mapping CF control point coordinates found in Table 5 5 are given in WGS84 UTM 17N 5.3.2 B SLA Calibration M ethods From Chapter 3 and (Ellum and El Sheimy 200 2 ) the two variables being solved for in the BSLA calibration are the lever arm from the GPS ARP to the camera ( ) and the boresight angles between the IMU CF and the camera CF ( ). The lever arm between the GPS ARP and the IMU center ( ) is straightforward using conventional measuring techniques and mechanical drawings. This lever arm position vector (DX, DY, DZ) in the IMU CF for GatorMMSv2.0 is (0.000m, 0.008m, 0.242m). The default navigation trajectory output from Waypoint Inertial Explorer is the mapping CF coordinates of the IMU CF origin. Thus, the lever arm between the ARP and the camera has been reduced to the lever arm between the IMU center and the incident nodal point of the lens ( ). The incident n odal point of the lens can only be estimated as its physical location is difficult to measure. Using EOPs derived from the

PAGE 86

86 SCBA of the BSLA calibration field ( and ) and the position fro m the navigation trajectory solution ( ), the lever arm is easily derived in Equation 5 9 (5 9) To determine the boresight angles, the angular rotations in the OPK convention are output from WIE a s the orientation component of the navigation trajectory The OPK convention is used to describe the angular rotation matrix from the mapping CF (ground) to the camera ( ) and the IMU ( ). To utilize the boresight angles in WIE after calibration, the rotation matrix from the IMU CF to the camera CF ( ) is desired as shown in Equation 5 10 (5 10) To perform the SCBA in SCBUN initial approximations for the EOP s were necessary. The simplest method for obtaining the angular approximations was estimating the initial boresight angle rotation matrix ( ) to be within the export wizard of WIE Figure 5 5 which depicts the camera and IMU CFs, is a visual aid for approximating the orientation of the IMU CF with respect to the camera CF. 5.3.3 BSLA Calibration Results The BSLA calibration was performed on February 8, 2011. Thirty images captured at various EOP ang les were acquired over a 5 m path on the RLA roof while maintaining a set distance from the calibration field. Thirteen images from the 30 second sample were used in the calibration. After quality checking the GNSS a nd inertial navigation solutions one na vigation trajector y w as output from WIE This

PAGE 87

87 trajectory included the position of the IMU center with respect to the mapping CF and OPK expressed in the orientation of the camera coordinate frame as estimated with the (90 90 0 ) boresight angles. The im age points were marked by hand using an ImagePick script written for MATLAB The magnification tools with this script enabled the estimated precision of the image coordinates to be 0.5 pix. Also, recall that the IOPs from the camera calibration testing we re fixed. This means the inputs subject to greatest adjustment were the EOP initial approximations. The difference between the initial EOP approximation s and the EOP results from the SCBA provided the data for the BSLA calibration derivation. The lever ar m offsets for of GatorMMSv2.0 expressed along the x, y, and z axes of the IMU coordinate frame are in Table 5 6. Likewise, the boresight angle updates ( ) for GatorMMSv2.0 expressed in OP K are in Table 5 7. Figure 5 6 shows the estimated angular precision of the navigation trajectory. The orientation results from the BSLA calibration with mm level precision for the lever arm and milliradian precision for the boresight angles are excellent relative to the input standard deviations. The original boresight angle estimation ( ) and the boresight angle correction ( ) are combined in Equation 5 11 to determine the final set of b oresight angles. (5 11) The final boresight angles were input into WIE. A navigation trajectory with the final boresight angles was checked against the EOP results from the SCBA. The discrepancies between these sets of orientation angles for each image were all within the acceptable tolerance level of milliradian precision noted in Table 5 7. A ngular

PAGE 88

88 precision is essential because angular error propagates over the distance the object is from the sensor. The BSLA orientation angle p recisions are approximately 0.05 degrees which is one milliradian apiece. This means for every meter an object is from the sensor the corresponding error grows by approximately 1mm. These satisfactory final BSLA calibration results to be used in the vision aiding testing are in Table 5 8. 5.4 Imaging System Testing Lessons Learned Multiple lessons can be taken from the GatorMMSv2.0 imaging system testing for appli cation to subsequent GatorMMS generations. For camera calibration, it is essential to maximize the depth of field for determining the IOPs of a camera especially focal length. Thus, a three dimensional calibration grid as opposed to the two dimensional calibration plane used herein is ideal. Furthermore, a commercial software package that automarks po ints using coded targets is excellent for efficiency. However, the automarking method must contain quality control measures to ensure accuracy Also, initial difficulty was encountered with the transfer and comparison of camera calibration parameters betwe en software packages. Thus, k nowing the camera coordinate system orientation for each software package is essential For future testing, BSLA calibration could be performed for each canopy density or vision aiding test To do so a routine should be establ ished to minimize systematic errors. For instance, a calibration grid could be brought to the field for IOP determination both before and after data collection for geometric stability of the COTS DSLR lens/camera combinations.

PAGE 89

89 Table 5 1 Differences in camera calibration software programs. Property PhotoModeler 6 SCBUN Origin of Image Plane Top Left Bottom Left Direction of +Z Axis Out from Camera Body Into Camera Body IOP Output Units mm pixels Table 5 2 IOP camera calibration results from Pho toModeler 6. The parameters are consistent between the two data sets. Note that the decentering distortion is negligible for this lens/camera combination. Test 1 Test 2 SCBA Images 9 7 Focal Length (mm) 25.173516 25.197745 FL (mm) 0.008 0.017 Xp principal point x (mm) 11.903442 11.918081 Xp (mm) 0.001 0.003 Yp principal point y (mm) 8.145751 8.126415 Yp (mm) 0.002 0.005 Fw format width (mm) 23.999625 23.999150 Fw (mm) 3.5e 004 5.9e 0 04 Fh format height (mm) 16.066116 16.066116 K1 radial distortion 1 1.58E 04 1.60E 04 K1 1.80E 06 2.40E 06 K2 radial distortion 2 1.97E 07 2.12E 07 K2 8.90E 09 1.10E 08 K3 radial distortion 3 0.00E+00 0.00E+00 P1 decentering di stortion 1 0.00E+00 0.00E+00 P2 decentering distortion 2 0.00E+00 0.00E+00

PAGE 90

90 Table 5 3 IOP camera calibration results from PhotoModeler 6 converted to pixels from mm. The largest discrepancies between the two data sets are the principa l point offsets and focal length. Test 1 Test 2 SCBA Images 9 7 Fw format width (mm) 23.999625 23.99915 Fh format height (mm) 16.066116 16.066116 Fw pix 3872 3872 Fh pix 2592 2592 pix/mm (w) 161.336 161.339 pix/mm (h) 161.333 161.333 avg pix/mm 161.335 161.336 Focal Length (pix) 4061.359 4065.308 FL (pix) 1.291 2.743 Xp principal point x (pix) 1920.437 1922.818 Xp (pix) 0.161 0.484 Yp principal point y (pix) 1314.191 1311.085 Yp (pix) 0.323 0.807 K1 radial distort ion 1 6.05E 09 6.13E 09 K1 6.92E 11 9.22E 11 K2 radial distortion 2 2.91E 16 3.13E 16 K2 1.31E 17 1.62E 17 K3 radial distortion 3 0.00E+00 0.00E+00 P1 decentering distortion 1 0.00E+00 0.00E+00 P2 decentering distortion 2 0.00E+00 0 .00E+00

PAGE 91

91 Table 5 4 IOP camera calibration verification results. The differences between the two camera calibration software packages were negligible. This ensures the correct implementation of the SCBA mathematical model in PM. Note the pri ncipal point offset for y of the SCBUN solution was converted from the SCBUN camera CF orientation to the PM camera CF orientation. Test 2 PM Test 2 SCBUN SCBA Images 7 7 Focal Length (pix) 4065.308 4065.872 FL (pix) 2.743 2.822 Xp principal point x (pix) 1922.818 1922.972 Xp (pix) 0.484 0.844 Yp principal point y (pix) 1311.085 1310.453 Yp (pix) 0.807 1.029 K1 radial distortion 1 6.12769E 09 6.35E 09 K1 9.22036E 11 1.75E 10 K2 r adial distortion 2 3.13197E 16 4.33E 16 K3 1.62355E 17 8.65E 17 K3 radial distortion 3 0 1.96E 23 K2 1.45E 23 P1 decentering distortion 1 0 8.47E 10 P1 1.65E 08 P2 decentering distortion 2 0 4.07E 08 P2 1.76E 08

PAGE 92

92 Table 5 5 RLA BSLA calibration field coordinates. These coordinates will be used for the EOP determination in the SCBA. Note that the standard deviations for RLAB and GNVL are 0 because these are published coordinates transformed into the W GS84 UTM 17N mapping CF. Name Easting (m) Northing (m) h (m) Control h (m) Hor (m) G1 369508.558 3280549.591 23.886 None 0.005 0.006 G2 369510.449 3280549.570 23.886 None 0.005 0.006 G3 369512.322 3280549.546 23.889 None 0.005 0.006 G4 369520.075 3280543.551 23.906 None 0.005 0.006 G5 369520.026 3280538.822 23.904 No ne 0.005 0.006 OB1 369511.790 3280550.624 24.202 None 0.005 0.006 OB2 369512.275 3280550.622 25.822 None 0.005 0.006 OB3 369512.696 3280550.616 25.423 None 0.005 0.006 OB4 369512.905 3280550.613 24.202 None 0.005 0.006 OB5 369514.525 3280550.596 25.03 5 None 0.005 0.006 OB6 369515.211 3280550.587 25.634 None 0.005 0.006 GB1 369516.596 3280550.570 25.506 None 0.005 0.006 GB2 369516.897 3280550.552 26.242 None 0.005 0.006 GB3 369517.362 3280550.552 25.947 None 0.005 0.006 BS1 369454.111 3280530.302 1 2.343 Both 0.004 0.005 GNVL 376457.776 3284735.156 22.421 Both 0 0 RLABsw 369509.654 3280523.723 24.950 Both 0.004 0.005 RLAB 369518.291 3280523.869 25.130 Both 0 0

PAGE 93

93 Table 5 6 GatorMMSv2.0 lever arm calibration results. The results a re expressed along the x, y, and z axes for the IMU CF. The x and z lever arm offsets are quite stable. Meanwhile, the y lever arm offset is not as stable. Image x y z AB5_0042 0.262 0.030 0.020 AB5_0044 0.253 0.030 0.013 AB5_0046 0.254 0.037 0.019 AB5_0048 0.252 0.035 0.011 AB5_0050 0.247 0.037 0.009 AB5_0051 0.249 0.029 0.010 AB5_0053 0.265 0.024 0.026 AB5_0059 0.267 0.023 0.018 AB5_0062 0.252 0.006 0.020 AB5_0063 0.264 0.012 0.009 AB5_0066 0.258 0.012 0.030 A B5_0067 0.258 0.008 0.021 AB5_0068 0.247 0.018 0.018 Mean 0.256 0.022 0.017 0.007 0.014 0.007 Table 5 7 GatorMMSv2.0 boresight angle calibration results. These angles reflect the difference from the initial (90 90 0 ) approximation. The precision of all three boresight angles is less than 5 arcminutes each. Image AB5_0042 0.7319 4.0082 0.2174 AB5_0044 0.7360 4.0221 0.2692 AB5_0046 0.7690 3.7949 0.2430 AB5_0048 0.7397 4.0195 0.2012 AB5_0050 0.7558 4.0123 0.3068 AB5_0051 0.7077 3.9893 0.2786 AB5_0053 0.6527 3.9329 0.2294 AB5_0059 0.704 7 3.8797 0.2493 AB5_0062 0.6031 3.9582 0.2218 AB5_0063 0.6532 4.0442 0.2630 AB5_0066 0.6607 3.9097 0.2485 AB5_0067 0.5094 3.9495 0.3053 AB5_0068 0.7427 4.0244 0.2568 Mean 0.6898 3.9650 0.2531 deg 0.0730 0.0715 0.0320 arcmin 4.4 4.3 1.9

PAGE 94

94 Table 5 8 GatorMMSv2.0 final BSLA calibration results. The lever arm is the position of the camera with respect to the center of the IMU in the IMU CF. The boresight angles are the rotations from the center of the IMU to the incident nodal point of the camera lens about the x, y, and z axes of the IMU CF. Boresight Angles Lever Arm Offsets Omega Phi Kappa x y z 80.1471 1 85.975 55 9.623 66 0.256 m 0.022 m 0.017 m

PAGE 95

95 Figure 5 1 PhotoModeler camera calibration grid. The fiducial targets labeled in the left photo are automatically identified by the number of dashes around the target. The white sub pixel automarks in the right photo passed all quality control checks for inclusion in the SCBA. (images courtesy of author) Figure 5 2 Radial lens distortion curve for the Nikon D200 Radial distortion increases polynomially the further away from the principal point. Without correcting for lens distortion in the SCBA model, a BA utilizing the collinearity equations would not provide usable results.

PAGE 96

96 Figure 5 3 Vicinity map of Reed Lab BSLA calibration field. The field was established with nine targets on the wall of the staircase to the roof (southern face of the white rectangle lying to the north of the RLA roof) and five targets along the northern portion of the RLA roof gutter. (image from Google Earth ) Figure 5 4 Roof view of RLA BSLA calibration field GCPs. The GCP targets were primarily retro reflective targets with an approximately 2mm diameter center hole for sighting purposes. Target GB2 was a PK nail surrounded by alternating white and blac k concentric circular washers for maximum contrast in the imagery. (image courtesy of author)

PAGE 97

97 Figure 5 5 GatorMMSv2.0 camera & IMU coordinate frames. Note that the angular rotations are positive counterclockwise when looking down the arrow towards the origin of the CF. Omega ( ) is about the x axis. Phi ( ) is about the once rotated y axis. Kappa ( ) is about the twice rotated z axis. Figure 5 6 Estimated angular orientation precision from BSLA calibration. As expected, the orientation precision is more stable in the horiz ontal channels than the vertical channel.

PAGE 98

98 CHAPTER 6 GATORMMS: VERTICAL O RIENTATION FOR CANOP Y DENISTY ANALYSIS 6 .1 GatorMMSv2.0 Applications Canopy Density Analysis GatorMMS development has been the focal point of discussion thus far. Discussion of the application of this technology for use as a mobile terrestrial remote sensing system has been minimal. This chapter investigates one remote sensing application of the GatorMMSv2.0, canopy density analysis. Canopy density is the proportion of the sky obscu red by vegetation when viewed from a single point near the ground (Figure 2 1) includes overhead vegetation, tree trunks, and lower tree branches. Previously mentioned in the forest mapping background, skyward looking hemi spherical photography used to capture a wide angle view of the forest canopy from a single point is implemente d extensively in GPS studies relating canopy density to GPS positiona l performance ( Jennings et al. 1999; Sigrist et al. 1999; Frazer et al. 2001; Holden et al. 2001; Zh eng et al. 2005; Hu et al. 2009) To determine canopy density using a digital image, sky pixels need to be differentiated from canopy pixels. Image thresholding is the simplest method for segmenting images. The canopy density index ( CDI) is the proportion of canopy pixels to total pixels in a segmented image. 6 .2 Methods To capture the nece ssary canopy imagery, the vertically oriented D200 is equipped with the 8mm f/3.5 Manual Focus Pro Optic fisheye (FE) lens (Figure 3 6). The Pro Optic FE lens and D200 camera produce a diagonal 180 fisheye image. Figure 6 1 modified after Schneider et al. depicts the effective field of view (FOV) of this lens camera combination relative to the FOV for a circular 180 fisheye lens camera

PAGE 99

99 combinat ion. The primary drawback of using the diagonal FE format is cropping large areas of the canopy in front of and behind the GatorMMS. An advantage of the diagonal FE format is us ing the entire sensor means greater detail of objects captured in the image. Af ter acquir ing jpeg images of the canopy, image processing is necessary to determine the CDI. Conversion of the full color jpegs to grayscale images is a simple process whereby only the intensity channel is preserved. The grayscale images are the input for the image thresholding algorithm. The goal of the algorithm is to convert these grayscale images into binary images with the two binary classes representing forest canopy and sky. Zheng et al. (2005) had success using Otsu's image thresholding method for s egmenting canopy pixels from sky pixels. Otsu's image thresholding method analyzes the intensity pixel data for a bimodal distribution. Briefly, a mean is calculated for each mode. From the mean s variances for each of the subsets are computed. The varianc e is a common measure for subset homogeneity. Thus, a low homogeneity subset will have high variance (Zheng et al 2005). The goal of Otsu's image thresholding is minimizing the intra group variance between the two subsets. For more information on this me thod, consult Zheng et al. ( 2005 ) Otsu ( 1979 ) or Appendix A for the derivation For algorithm success, it is important that homogenous lighting is available throughout the image. On a sunny day, the area where the sun shi nes through the canopy on one part of the image make s the intensity values for those sky pixels vastly

PAGE 100

100 different than sky pixel intensity value s in another portion of the image. This phenomenon skew s the bimodal distribution. Thus, a cloudy day is ideal. Us ing MATLAB a batch processing algorithm was developed to handle all of the images al ong the navigation trajectory. MATLAB has a built in function called Quality control and quality assurance (QC \ QA) of the output binary images i s necessary. Visual comparison of original images with binary images is sufficient QC \ QA for this analysis. A simple ratio of canopy pixels to total pixels is satisfactory for computing the canopy density index (CDI). 6. 3 Preliminary Image Processing Results Preliminary testing of the image processing algorithm was necessary prior to implementation on the forest navigation data sets. On January 6, 2011, a set of vertically oriented images was collected al on g a walkway adjacent to the North Lawn on the University of Florida Gainesville campus. T he partly sunny weather conditions were not ideal for the bimodal distribution Figure 6 2 is a sample image from the set of 26 images with the subsequent output grayscale and binary images. The use of the intensity channel for the grayscale image was not reliable due to a large portion of the sky mis classified as canopy. Previous studies indicated using the intensity from only the blue channel of a red green blue (RGB) jpeg image was the best w ay to separate sky from canopy (Frazer et al. 2001, Nobis and Hunziker 2005). After implementing this change in the MATLAB algorithm, the results were drastically improved (Figure 6 3). Further preliminary testing under cloudy conditions confirmed that us ing the blue channel intensity was the optimal image component for thresholding.

PAGE 101

101 6.4 GatorMMS Testing Results The premise of this remote sensing application is to analyze CDI relative to the navigation trajectory solution. GatorMMS testing of the vertical ly oriented D200 FE lens combination was conducted in Austin Cary Memorial Forest on January 27, 2011 and March 3, 2011 (Figure 6 4) The cloudy conditions near dusk in January were optimal for canopy density analysis The MATLAB script using blue channe l intensity was implemented with excellent results. Figure 6 5 shows typical results of the image processing. Unfortunately, the inertial navigation data acquired with the images was not capable of being processed in Waypoint Inertial Explorer Short ini tial static alignments with poor GPS satellite coverage are the likely culprits for lack of an inertial navigation trajectory solution This processing issue was resolved in March. With a lack of GPS positions along the trajectory in January, georeferencin g the images without the inertial data was not an option. Th us, the primary relationship sought in th e CDI versus navigation trajectory analysis could not be pursued from the January data sets. Interestingly, analysis of the CDI s pikes in Figure 6 6 can he lp with interpolation of the GatorMMS location along test track 2 For instance, the GatorMMS was initialized in an open area lacking substantial forest canopy at the southern end of test track 2. Shortly after heading northward, the canopy became denser i n image 25 before opening up again near a newly planted pine stand in image 40. O n March 3, 2011 conditions for acquiring quality canopy imagery were suboptimal due to bright sun visible in most photos. While t he acquisition time of day was approximately the same as during the January data collection the day light hours were considerably longer in March leading to a higher sun elevation. Furthermore, few

PAGE 102

102 clouds meant the sun would rarely be obscured from the FOV of the lens. These discouraging factors mea nt only one trial around test track1 of vertically oriented imagery was collected. Given the obstacles encountered in the field, Figure 6 7 is a typical example of the surprisingly decent CDI results from the March testing. The impact of the sun directly s hining through portions of the canopy w as expected to have a more adverse impact on the thresholding algorithm than the final March results reveal Post mission analysis of the CDI levels over the course of the test trail show there is considerably less va riability in the canopy density of test track 1 (Figure 6 8) than in test track 2 (Figure 6 6). T here were no issues preventing the raw inertial navigation data from being processed in WIE However the lack of canopy variability severely hampered the ana lysis of a relationship between the navigation trajectory position solution and CDI. Precision statistics regarding the inertial trajectory solution wer e output from Inertial Explorer Unfortunately important data regarding signal to noise ratio (SN R) wa s un available. Analysis of the independent variable CDI against the available positional precision metrics was undertaken through graphical analysis and then more rigorous hypothesis testing. CDI significantly impacts the horizontal and vertical positional precision of the tightly coupled position solution (Figure 6 9) or the GNSS only position solution (Figure 6 10). Furthermore the positional precision is most significantly impacted by the long static initialization periods at the beginning and end of th e data sets (Figure 6 11). From visual analysis of Figure 6 11 f urther investigation into the relationship between time elapsed from static observation and horizontal precision was necessary

PAGE 103

103 A correlation coefficient o f nearly 1 seen in Figure 6 12 indi cated that there is a strong linear relationship between elapsed time from static observation and horizontal precision. To test the statistical significance of the correlation coefficients discussed in Figure 6 9 through Figure 6 12 correlation hypothesi s testing was undertaken. Hypothesis testing determined if the sample correlation coefficient (R value) was statistically significance. Table 6 1 shows the only statisti cally insignificant linear or exponential relationship between two variables at a 99.9% confidence level was the linear relationship between CDI and PDOP. (6 1) (6 2) (6 3) (Two Tailed Test) (6 4) Test statistic: (6 5) Critical value from table: (6 6) False (6 7) For all variable pairs except the linear relationship between CDI and PDOP, t he null hypothesis can be rejected with a 0.1% probab ility of committing a Type 1 error

PAGE 104

104 (false positive). Therefore, it can be said that the population correlation coefficient between each remaining variable and CDI is both nonzero and significant. From Table 6 1, the correlation between time elapsed and hor izontal precision was the most statistically significant relationship. 6.5 GatorMMS Canopy Density Analysis Lessons Learned Many difficulties were encountered with t he analysis of canopy density relative to the navigation tra jectory solution. These issues incl uded imagery acquisition conditions, inertial processing aspects, and data analysis conclusions. Multiple lessons learned can be applied to future canopy density studies. While cloud cover and an acquisition time near dusk are optimal, excellent results ca n still be achieved when the sun is visible by using the blue channel of RGB images to separate sky from forest canopy. Unfortunately, using these thresholding results with inertial navigation data is often not feasible without l ong static initializations at the beginning and end of each test The long initializations reduce the chances that inertial navigation data sets will be unusable. The ideal test track should have large variability in canopy density with multiple iterations of traversing between open field and thick woods This will help break the correlation between initialization and positional precision. Furthermore, multiple passes of the same test track over different days will create a solid foundation for the relationship between CDI (which sho uld be fairly static) and position precision (which should fluctuate depending upon the satellite configuration). Investigation of a dditional parameters could make the comparison of canopy characteristics and inertial navigation parameters more robust. Us ing software that is capable of extracting the signal to noise ratio (SNR) would provide another metric to

PAGE 105

105 evaluate against CDI. Also, canopy gap (structure) is another canopy metric that could be evaluated against positional precision and SNR

PAGE 106

106 Table 6 1 Hypothesis testing of CDI correlation for statistical significance of relationships. All but one relationship is statistically significant at a 99.9% level of confidence. Variable Relationship r 2 t ( Test Statistic) Time vs. Hor Lin ear 0.9884 129.888 CDI v s. Satellites Linear 0.1231 5.272 CDI vs. Satellites Exp onential 0.1133 5.030 CDI vs. PDOP L in ear 0.0434 2.997 CDI v s. PDOP E xp onential 0.0738 3.972 CDI v TC Lin ear 0.5570 15.778 CDI v TC Exp onential 0.7079 21.905 CDI v GNSS Lin ear 0.2171 7.410 CDI v GNSS Exp onential 0.5743 16.344 CDI v s. Ver TC Lin ear 0.4926 13.865 CDI v s. Ver TC Exp onential 0.7219 22.671 CDI v s. Ver GNSS Lin ear 0.2142 7.347 CDI v s. Ver GNSS Exp onential 0.5528 15.645 Figure 6 1 Fisheye field of view comparison. Depending on the application, a diagonal or circular FE could be more beneficial. (i.e. greater detail vs. greater coverage area) ( Modified after Schneider, D., E. Schwalbe, and H. G Maas. 2009. Validation of geome tric models for fisheye lenses. ISPRS Journal of Photogrammetry and Remote Sensing 64 (3): 259 266 )

PAGE 107

107 Figure 6 2 Image thresholding implementation example for grayscale images. Less than ideal sky conditions led to poor final canopy density index results. (images courtesy of author) Figure 6 3 Image thresholding implementation example for blue channel only grayscale images. Less than ideal sky conditions did not corrupt the canopy density index results. The binary image passes the visual QC/QA. (images courtesy of author)

PAGE 108

108 Figure 6 4 Map of ACMF CDI analysis test tracks. On January 27, 2011, the northern portion of track 2 and track 1 was run as one combined test trial. Also in January, track 2 was run once from south to north. On March 3, 2011, track 1 was run once. Figure 6 5 Typical image from ACMF CDI analysis on January 27, 2011. With dusk approaching and consistent cloud cover, weather conditions were optimal. Excellent results such as these from the CDI algorithm implementation passed the visual QC/QA. (images courtesy of author)

PAGE 109

109 Figure 6 6 Plot of CDI results from ACMF test track 2. The sharp spikes in the CDI trajectory indicate leaving/entering open canopy areas. For example the GatorMMS was initialized in an open area lacking substantial forest canopy at the southern end of test track 2. Shortly after heading northward, the canopy became denser in image 25 before opening up again near a newly planted pine stand in image 40. Figure 6 7 Typical image from ACMF CDI analysis on March 3, 2011. The sun is visible in the lower left hand corner of the RGB image. This causes a portion of the final binary image to be washed out (Note the missing tree trunk in the lower left hand portion of the binary image). The washout causes the reported CDI level to be slightly lower than the true CDI. However, the CDI value is still representative of the canopy density. (images courtesy of author)

PAGE 110

110 Figure 6 8 Plot of CDI results from ACMF test track 1. GatorMMS initialization was in the area with the least amount of canopy cover along test track 1. However, the canopy density at initialization was still substantial. This led to little variability along the test track.

PAGE 111

111 Figure 6 9 Plot of CDI results relative to tightly coupled position solution from ACMF test track 1. The exponential trend lines and corresponding R squared values indicate the horizontal and vertical precisions are correlated with the CDI.

PAGE 112

11 2 Figure 6 10 Plot of CDI results relative to GNSS only position solution from ACMF test track 1. The linear trend lines indicate the horizontal and vertical precisions have minimal correlation with the CDI. Meanwhile the exponential trend lines between the CDI and positional prec isions have some correlation.

PAGE 113

113 Figure 6 11 Plot of CDI results with tightly coupled positional solution precision from ACMF test track 1.The greatest precision was at the beginning and end of the test run. These small standard deviations can be attrib uted primarily to the long static initializations not the small relative decrease in canopy density. Figure 6 12 Plot of relationship between horizontal precision and time elapsed from static observation. The strong linear relationship between these two variables is substantiated by the correlation coefficient of nearly 1.

PAGE 114

114 CHAPTER 7 GATORMMS: HO RIZONTAL ORIENTATION FOR VISION AIDING 7.1 GatorMMSv2.0 Application: Vision Aiding The previous chapter discussed canopy density analysis as the first appli cation of GatorMMS technology. This chapter explores another GatorMMS application implemented in the Austin Cary Memorial Forest: v ision aiding (VA).VA is improving the georeferencing parameters of the navigation trajectory through imagery derived exterior orientation parameter (EOP) updates Recall georeferencing is the process of determin ing the time, position/location, and attitude/orientation of an event in space ( Skaloud 1999 ) Thus, improving the precision and accuracy of the orientation parameters (O PK) and position parameters (XYZ of the incident nodal point of the lens) will help the final goal of accurately mapping features captured in the imagery. Recall from Chapter 2 the discussion of VA and bundle adjustments (BA). The progression of any bundle adjustment is determining the interior orientation first, the relative orientation between two images in a stereo model second, and the absolute orientation of the stereo model in a mapping coordinate frame last. Using relative orientation between two ima ges, the noise from the inertial navigation sensors can be reduced significantly through BA implementation. This noise reduction is seen through increased precision in the georeferencing orientation angles post BA. Common points between overlapping images are necessary for the creation of stereo models the basis of photogrammetric bundle adjustments. These common points are also referred to as tie points or pass points. For VA implementation, accurate image space coordinates are necessary to ensure that t he correct corresponding pass point feature s are used in the relative/absolute orientation A multitude of least squares

PAGE 115

115 algorithms exist for minimizing errors in determining and matching points between images. Recall that PhotoModeler 6 used centroid det ermination and least squares matching (LSM) on the camera calibration grid images. The Scale Invariant Feature Transform (SIFT) algorithm developed by Lowe has been used extensively in computer vision for automatically creating tie points between images (L owe 1999) P ass point generation from aerial video image ry is another exa mple of the growing research interest in automating these functions (Wilkinson et al. 2009). The purpose of this GatorMMSv2.0 VA research is to develop a proof of concept in the fores t. Thus, a manual approach to pass point generation is undertaken. Automatic pass point generation is a research opportunity to be explored in future studies. 7.2 ACMF VA Methods 7.2.1 Orientation VA imagery was captured with the 24mm f/2.8D AF Nikkor wi de angle (WA) lens mounted on the horizontally oriented D200 Nikon camera (Figure 3 6). The horizontal orientation means the widest portion of the camera sensor is aligned with the direction of travel. This feature is important for maximizing overlap betw een stereo images. Relative to the 180 diagonal field of view (FOV) for the 8mm Pro Optic FE lens D200 camera combination, the horizontal FOV and vertical FOV for this VA camera lens combination are approximately 51 and 35, respectively. Between orient ing the sensor horizontally reducing vehicle speed, or increasing camera acquisition rates, maximum image overlap is crucial for creating usable stereo models. 7.2.2 ACMF VA Test Site The test track for the GatorMMSv2.0 VA implementation in Figure 7 1 was chosen based on the dense canopy cover and existing network of ground control points (GCPs)

PAGE 116

116 (Figure 4 2) From the Chapter 4 georeferencing system testing analysis, resurveying of the existing control was suggested due to discrepancies between the navigat ion trajectory and potential datum transformation errors in the original GPS network adjustment processing. On March 3, 2011, six hour simultaneous static GPS observation s of all six GCPs w ere conducted. The occupations were split into one hour observatio ns and processed in Waypoin t GrafNet software usin g the Gainesville airport CORS (GNVL) as the master control station. The UTM 17N WGS84 coordinates from the network adjustment are in Table 7 1. To develop the VA proof of concept, high contrast targets w ere mounted to trees along test track 1. These VA targets featured in Figure 7 2 made manual target detection easier and represented a best case scenario for VA implementation. Furthermore, accurate and precise surveying of these targets could be accomplis hed with the well defined cross from a Philips head screw as the center point. To concentrate VA implementation efforts, the southwestern portion of the approximately 550m test track 1 served as the focal VA testing area (Figure 7 3). Typical imagery acqu ired by the GatorMMSv2.0 in the vicinity of the VA test area is shown in Figure 7 4. A survey of this area was conducted with a reflectorless total station (TS) on February 25, 2010. Each VA target was surveyed with two sets of forward and reverse face rea dings from both GCP10 and CP50. Using Topcon Tools 7.2 (TT), the GCP coordinates from Table 7 1 were imported and held as control relative to the TS measurements. From the TT network adjustment, the UTM 17N WGS84 coordinates for the VA targets are in Table 7 2.

PAGE 117

117 7.2.3 ACMF VA Procedure To test if VA impl ementation can improve the reliability of the navigation trajectory for mapping purposes, the EOPs derived from self calibrating bundle adjustments (SCBA) of the VA test area imagery are analyzed. The boresight lever arm (BSLA) calibration parameters from Table 5 8 are input into WIE to determine the initial estimated image EOP s for the SCBA Likewise, the target coordinates from Table 7 2 are SCBA inputs for the object space coordinates. These target coordinates are used as tie points not control points. T he pre adjustment EOP standard deviations generated by WIE and the post adjustment EOP standard deviations g ener ated by SCBUN are compared in an analysis of precision Both ZUPT and non ZUPT trials are used. The primary limitation of this VA methodology is the need for approximations of the VA target object space coord inates Knowing these coordinates is the primary drawback of using the collinearity equations in the SCBA for VA purposes. In Chapter 8, exploration of the coplanarity condition as th e observa tion model for the BA is discussed. I nitial object space coordinate approximations are not necessary for coplanarity condition implementation Additional limitations of the ACMF VA target methodology are discussed in the results. 7. 3 GatorMMS VA Results On March 3, 2011, three trial runs of VA test track 1 commencing near GCP7 were conducted. During each run, the GatorMMSv2.0 ATV travelled between 5 and 10 miles per hour. Trial 1 and t rial 2 were continuous non ZUPT traject ories. Trial 3 was a ZUPT trial wi th zero velocity updates at each corner of the loop (SW near GCP10; SE near GCP9; NE near GCP8).

PAGE 118

118 7.3.1 ACMF VA Non ZUPT Trial s Twenty three images of the VA test area along test track 1 were used in the SCBA for t rial 1. Twelve VA targets were seen in at l east 2 of the 23 images. Furthermore, each image had between 3 and 8 VA targets. Two additional images in the test area were unused due to image blur. The trial 1 shutter speed of 1/125s was much slower than the ideal speed of close to 1/1000s. Unfortunate ly dark shadows under the forest canopy and a fixed aperture setting meant the only option for letting in enough light to the camera sensor was slowing the shutter speed. After processing the GPS and inertial data as outlined in the georeferencing system testing recommendations from Chapter 4, the precision of the orientation parameters for the tightly coupled (TC) and smoothed inertial navigation trajectory of t rial 1 is shown in Figure 7 5. The heading is the least precise orientatio n angle. When compar ing the pre adjustment and post adjustment EOPs, the SCBA significantly improves the orientation precision in Figure 7 6. Recall from Figure 5 5 that heading/azimuth is related to the angle kappa about the twice rotated z axis of the IMU This axis corresp onds to the angle phi about the once rotated y axis of the camera coordinate frame (CF). Thus, seeing the greatest improvement from VA implementation in the angle phi is expected. VA implementation had similar positive results on t he position precision of the smoothed TC navigation trajectory Figure 7 7 shows the Easting Northing height (ENh) coordinate precision for the estimated nodal point of the camera lens along the entire trajectory. The half meter to meter precision is not suitable for many mapping operations. Thus, significant gains in position precision by almost two orders of

PAGE 119

119 magnitude ( m level to mm level precision) shown in Figure 7 8 were definitely encouraging. To test the statistical significance of the improvement in the h orizontal precision between pre BA and post BA hypothesis testing of population variances was undertaken using the F distribution Hypothesis testing determined if the difference in the population variances of the two sample sets was statistically significant at a 99.9% lev el of significance. The results shown in Table 7 3 indicate the improvement in position and orientation precision is significant for all exterior orientation parameters. (7 1) (7 2) (7 3) (One Tailed Test) (7 4) Test statistic: (7 5) Critical value from table: (7 6) False (7 7) T he null hypothesis can be rejected with a 0.1% probability of committing a Type 1 error (false positive) for all 6 parameters Therefore, it can be said that the gains in precision from SCBA are both nonzero and significant.

PAGE 120

120 After brightness analysis of the trial 1 imagery, the shutter speed was decreased to 1/200sec for trial 2 The goal was to minimize image blur o f VA targets for accurate determination of image space coordinates In t rial 2, the same 12 VA targets were seen in at least 2 images. Likewise, 23 images were used in the SCBA. No images were omitted due to image blur The trial 2 input EOP orientation pr ecision was similar to trial 1. Meanwhile, trial 2 position precision was slightly better than trial 1. Figure 7 5 and Figure 7 7 are representative of the WIE smoothed TC navigation trajectory results for trial 2. Furthermore, Figure 7 9 and Figure 7 10 r einforce the approximately one order of magnitude position and orientation precision gains made during non ZUPT trials. Similar to trial 1, variance hypothesis testing showed the improvement in precision for each exterior orientation parameter was statisti cally significant. Table 7 4 summarizes these findings. 7.3.2 ACMF VA ZUPT Trial Zero velocity updates (ZUPT) have been used in GPS outage prone areas to bound position and attitude errors in the inertial navigation trajectory ZUPTs serve as a type of sta tic reinitialization or alignment procedure. ACMF VA trial 3 had 3 ZUPTs during the course of the test track 1 loop. P rior to reaching CP50 in the southwestern VA test area (Figure 7 3), a 65 s econd ZUPT was implemented. The goal was to determine if gains in position and orientation precision from SCBA would still be significant relative to the more precise ZUPT navigation trajectory. Analysis of the entire WIE smoothed TC navigation trajectory shows excellent position precision for each of the 3 ZUPTS. Fur thermore, Figure 7 11 and Figure 7 12 show the maximum position and orientation standard deviation s during trial 3 ATV

PAGE 121

121 navigation are less tha n the maximum kinematic trial 1 or trial 2 EOP standard deviations. Only kinematic images were used in the trial 3 SCBA. There were 9 kinematic VA test area images pre ZUPT and 15 kinematic images post ZUPT The 66 static images during the ZUPT were omitted. Interestingly, 15 VA targets were visible in at least 2 images with 3 to 9 targets visible per image. As expec ted, Figure 7 13 shows similar heading precision gains for the SCBA results. Furthermore, Figure 7 14 shows that gains in input position precision are possible even right before and right after a ZUPT. Using a critical F statistic of 4.29, hypothesis test ing on the pre and post SCBA EOP precisions in Table 7 5 confirmed the gains in EOP precision were statistically significant. This is similar to the results from the non ZUPT ACMF VA Trials. From this ACMF VA analysis, photogrammetric adjustments through V A are the most crucial tactical grade inertial navigation supplement for aiding heading orientation angles both in the ZUPT and non ZUPT trials. 7.3.3 ACMF VA Trial s : Low A ccuracy H3 IMU Another component of th e ACMF VA analysis was comparing a low accura cy M EMS ense H3 IMU to the tactical grade SPAN system (Figure 3 5) If the H3 could be used with results nearly equivalent to Novatel system, then consumer costs would drop dramatically. This could lead to a subsequent rise in availability of this GatorMMS mobile mapping technology for fores t managers. Unfortunately, the implementation of VA with the H3 inertial navigation trajectory was not possible in these non ZUPT or ZUPT trials. After overcoming the difficulty of processing the raw H3 inertial data in WIE, additional problems arose. WIE was incapable of aligning the H3 without transferring the alignment. The mean and standard

PAGE 122

122 deviations for roll/pitch/yaw (RPY) during the long static initializations at the beginning and end of each trajectory were transferred as the forward and reverse al ignment. When processing the inertial data, the model that WIE uses to weight the inertial and GPS observations was not sufficient at handling the drift associated with the low accuracy H3 IMU. Figure 7 15 shows the erroneous trial 1 non ZUPT smoothed LC p osition trajectory The trajectory not only lacks the shape of test track 1 (Figure 7 1) but interpolation of the ATV speed from the trajectory at approximately 10m/s is approximately four times faster than the true speed. As a result, the precision and ac curacy of the EOPs from the non ZUPT trajectory were not reliable for VA without consistent position updates. Initial analysis of the ACMF VA ZUPT trial 3 was encouraging. The relative position of all static time intervals resembled the location of the tes t track 1 loop (Figure 7 16) Since the ZUPT in the southwest corner was at the VA test area, there was potential that VA implementation using SCBA was feasible. Upon looking at the smoothed LC EOP trajectory for t rial 3, it was apparent that the rapid det erioration of the coordinate quality and wild fluctuations in the OPK angles was not suitable for SCBA. The OPK angles seemed to suggest the dynamics of the ATV were similar to that of a fighter jet. Upon SCBA implementation, the adjustment failed. 7. 4 Gat orMMS VA Lessons Learned Vision aiding using photogrammetric adjustments can be implemented in the forest given certain conditions. Thus far, VA implementation conditions include preprocessing of the inertial navigation trajectory, knowledge of control poi nt/target coordinates use of artificial VA targets, access to a tactical grade IMU, and robust proprietary software for inertial processing To implement vision aiding for near real time

PAGE 123

123 applications, the system must be capable of integrating GPS, inertia l, and image updates sequentially on the fly. Thus, knowledge of absolute control point targets must not be a necessity. The development of a vision aiding algorithm from an aerial platform that satisfies these requirements is discussed further in Chapter 8. For the GatorMMSv2.0, issues exist ed that provided valuable lessons for future forest GatorMMS VA implementation including guidance on algorithm development First, k nowing absolute control point coordinates for mapping operations is not feasible on a large scale whether in a forest or urban landscape Thus, direct georeferencing must provide the position and orientation updates. Tracking targets in shadowed areas without distinct features from the built environment is a difficult and complex task. Test ing the reliability of existing pass point generating algorithms under forest canopy is a research area in need of future study. Additionally, deciding the best shutter speed at the beginning of a mobile mapping operation under forest canopy is difficult due to shifting shadows and highly variable lighting along the trail. Decisions should be based on the canopy density (i.e. shadows) of the most important areas. For future terrestrial algorithm development consideration must be paid to the difference in base to height ratio between aerial photogrammetry and close range terrestrial photogrammetry. Objects in the background will pass behind foreground objects throughout a strip of photos. This property could complicate the automatic pass point generating so ftware. The use of a camera with a frame rat e greater than 1Hz could limit large geometric differences between sequential images.

PAGE 124

124 A dditional iner tial sensors could help the low accuracy H3 IMU navigation trajectory. A magnetic compass or a dual antenna GPS receiver for heading updates could help resolve the lack of azimuthal precision. Odometer updates could also help with velocity updates and dampen errors associated with gyro drift. Lastly, a ccurately calibrating imagery for the radial distortion in FE le nses could aid the difficultly often encountered with image feature overlap in terrestrial MMS operation from typical wide angle lens The FE FOV captures a broader view of the area leading to greater redundancy of common pass points. This is assuming that the coordinates of each pass point can be accurately captured in spite of the reduced resolution of each feature in the image.

PAGE 125

125 Table 7 1 VA test area ground control point coordinates in UTM 17N WGS84. The coordinates for GNVL are published by the Nati onal Geodetic Survey (NGS). Thus, the default coordinate precision horizontally and vertically is 0.000m. Station Easting (m) Northing (m) H Ell (m) Hor (m) Ver (m) GNVL 376457.773 3284735.156 22.425 0.000 0.000 CP50 382414.526 3290286.258 16.108 0.007 0.011 GCP10 382399.897 3290285.729 15.997 0.007 0.011 GCP4 382380.698 3290441.226 17.311 0.007 0.011 GCP7 382357.859 3290442.814 17.357 0.0 07 0.010 GCP8 382545.217 3290376.449 16.886 0.007 0.011 GCP9 382538.835 3290287.678 16.090 0.007 0.011 Table 7 2. VA test area target coordinates in UTM 17N WGS84. The horizontal and vertical precision of the coordinates reflect the precision of the T S measurements. The precision of the GPS coordinates was not propagated through to the VA targets due to limitations within the TT software. Station Easting (m) Northing (m) H Ell (m) Hor (m) Ver (m) 5 382426.871 3290275.157 18.12 0 0.007 0.002 6 382432.339 3290283.814 18.787 0.007 0.003 7 382427.22 0 3290291.59 0 17.888 0.005 0.002 8 382433.75 0 3290295.504 18.092 0.005 0.002 9 382432.572 3290304.57 0 18.383 0.005 0.002 10 382 434.883 3290313.943 18.597 0.005 0.002 11 382426.823 3290311.884 18.872 0.005 0.002 12 382420.172 3290319.26 0 18.776 0.005 0.002 13 382415.302 3290315.752 18.844 0.005 0.002 14 382408.753 3290315.497 18.844 0.005 0.002 15 382405.451 3290318.787 20.665 0.005 0.002 16 382410.478 3290310.438 18.61 0 0.004 0.002 17 382414.51 0 3290304.015 18.94 0 0.004 0.002 18 382413.691 3290300.875 20.423 0.004 0.002 19 382419.347 3290297.611 18.318 0.004 0.002 20 382424.071 3290298.848 18.352 0.005 0.002 21 382419.07 9 3290290.537 17.737 0.004 0.002

PAGE 126

126 Table 7 3. Hypothesis testing of population variances for VA t rial 1. All exterior orientation parameters had statistically significant improvements in precision through the implementation of SCBA. Exterior Orientat ion Parameter F (Test S tatistic) 16.49 1541.42 6.49 31.22 51.29 59.96 Table 7 4 Hypothesis testing of population variances for VA t rial 2 With a critical F test statistic, a ll exterior orientat ion parameters had statistically significant improvements in precision through the implementation of SCBA with a 99.9% level of confidence. Exterior Orientation Parameter F ( Test S tatistic) 12.01 1080.22 6.37 Easti 24.76 42.92 46.33 Table 7 5 Hypothesis testing of population variances for VA t rial 3 All exterior orientation parameters had statistically significant improvements in precision through the implementation o f SCBA with a 99.9% level of confidence. Exterior Orientation Parameter F (test statistic) 25 73 2467.48 17.71 43.31 112.73 133.47

PAGE 127

127 Figure 7 1. ACMF VA test track. Te st track 1 from Figure 6 4 is the subject area for testing the VA implementation of the GatorMMSv2.0. All VA targets were surveyed from GCP10 and CP50 using GCP9 as a backsight. Figure 7 2. ACMF VA field setup. The left image is a detail of the high co ntrast VA targets. Each VA target consists of a white Phillips head screw inside black and white concentric washers mounted to a 4in by 4in square of black aluminum flashing. The original silver materials are spray painted black and white for maximum contr ast. The right image is the typical setup for fixing the targets to the mature pine trees in the VA test area. (images courtesy of author)

PAGE 128

128 Figure 7 3 ACMF VA test area. The southwestern portion of test track 1 shows the excellent VA test area depth of field. VA target 10 was the most difficult target to locate due to consistent shadows under the dense forest canopy. Figure 7 4 Typical image of ACMF VA test area. This example of a typical VA image represents that the entire field of targets was neve r captured in one image. Also, target features in the shadowed background would be difficult to track without the high contrast targets. (image courtesy of author)

PAGE 129

129 Figure 7 5 Estimated orientation angle precision for trial 1. After smoothing the TC tr ajectory, heading is the orientation angle which needs the greatest improvement. Figure 7 6 Comparison of pre SCBA & post SCBA orientation angle precision for trial 1. Heading, which corresponds with the angle phi in the camera coordinate frame, impr oved by one order of magnitude due to VA implementation.

PAGE 130

130 Figure 7 7 Estimated position precision for ACMF VA trial 1. The 1m positional coordinate standard deviations are not suitable for survey grade mapping. Figure 7 8 Comparison of pre SCBA & post SCBA position precision for ACMF VA trial 1. The original 1m positional coordinate standard deviations were improved drastically by approximately one order of magnitude.

PAGE 131

131 Figure 7 9. Comparison of pre SCBA & post SCBA orientation a ngl e precision for t rial 2. Again, SCBA successfully resolved the lack of precision for heading with a one order of magnitude precision increase in phi. The increase in EOP standard deviations at either end of the bundle adjustment strip is common for BA. Beyond the begi nning and end of a subject area, images are often captured for the adjustment but not used for mapping purposes due to lack of precision at the edges. Figure 7 10. Comparison of pre SCBA & post SCBA position precision for ACMF VA t rial 2. Again, the o riginal 0.5 m positional coordinate standard deviations were improved drastically by approximately one half order of magnitude.

PAGE 132

132 Figure 7 11 Estimated orientation angle precision for ACMF VA trial 3. Heading from the smoothed TC navigation trajectory is still the least precise orientation angle. However, the average heading precision for the kinematic period is less than a similar average for the non ZUPT trials. Figure 7 12 Estimated position precision for ACMF VA trial 3. The ZUPTs had the greates t impact on the positional coordinate standard deviations through bounding positional errors. The ZUPT intervals are shown above as troughs.

PAGE 133

133 Figure 7 13 Comparison of pre SCBA & post SCBA orientation precision for ACMF VA trial 3. The gains in orient ation precision, especially heading, are similar to the non ZUPT trials. Figure 7 14 Comparison of pre SCBA & post SCBA position precision for ACMF VA trial 3. Images 1 through 9 are pre ZUPT and images 10 24 are post ZUPT. While the initial cm level and low dm level precision was already suitable for some mapping applications, further refinement of the position precision is possible through photogrammetric adjustments for survey grade mapping.

PAGE 134

134 Figure 7 15. Low accuracy H3 IMU inertial trajectory f rom ACMF VA trial 1. The trajectory does not resemble the test track 1 loop except at the northwestern corner (the point of static alignment). Each square represents 30m x 30m of horizontal ground distance. Through interpolation of three camera events per square, the trajectory shows the vehicle was traveling 4 times faster than the true 2.5m/s vehicle speed. Furthermore, the position precision from this trajectory was as high as 200m during kinematic operation under the forest canopy (Figure not shown). Figure 7 16. Low accuracy H3 IMU inertial trajectory from ACMF VA trial 3. The four light blue areas represent the increased position quality at the static ZUPT interval locations. The lack of quality GPS positions under the tree canopy made WIE rely on the low accuracy IMU for inertial navigation position updates. As a result, the position precision was quite poor.

PAGE 135

135 CHAPTER 8 SEQUENTIAL VISION AIDING 8.1 Theoretical Framework This research analyzes K al man filtering with an optimal smoother in sequentia l bundle adjustment as a method for reducing the sequential accumulation of error normally associated with aerial triangulation in order to provide accurate and precise georeferencing parameters. The goal is to apply the sequential bundle adjustment algori thm developed herein to the terrestrial GatorMMS forest application in the f u t ure 8.1.1 Kalman Filtering The Kalman filt er is an algorithm that implements a predictor corrector estimator to minimize the estimated error covariance of the state (Gelb 1974). The filter achieves that by utilizing knowledge of system and measurement dynamics, assumed statistics noises and measurement errors, and initial condition information (Gelb 1974). Kalman filtering is the most common technique for estimating the state of a linear system and is widely used in many applications such as navigation with INS GPS systems (Nassar et al. 2007; Webb 2007), satellite orbit prediction (Xiong et al. 2009), and in many other fields. The objective of the Kalman filter is to obtain the s ystem state estimate ( a posteriori state estimate) as a linear combination of a predicted estimate ( a priori estimate) and a weighted difference between an observation and a measurement prediction ( ) In equation form, (8 1)

PAGE 136

136 where K is the Kalman gain that minimizes the a posteriori error covariance, ( ) is the measurement innovation or resid ual, and is the coefficient matrix that relates the state estimate to the observation Interested readers are referred to Appendix B or K al man filtering texts (i.e., K alman 1960; Rau ch et al. 1965; Gelb 1974; Welch and Bishop 2001) for details of the algorithm. 8.1.2 Coplanarity Condition The coplanarity condition is used as the o bservation model in the Kalman f ilter algorithm. If two photographs are relatively oriented wit h respect to each other, then the object space rays defined by a pair of conjugate image points and their respective exposure stations will intersect at exactly one point (Wolf and Dewitt 2000; Mikhail et al. 2001). The object space position of that point occurs at the intersection. The two object space rays in combination with the position vector connecting the two exposure stations form the three sides of a triangle. This triangle defines the plane satisfied by the condition illustrated in Figure 8 1 modi fied after Mikhail et al. ( 2001). The coplanarity condition is based on the fact that the volume ( ) of the parallelepiped (a polyhedron consisting of all parallelogram faces) of three coplanar vectors will be 0 as shown in E quation 8 2 (8 2) As illustrated in Figure 8 1 the base vector between the two exposure stations is Vectors, ( and ( are the object space lays originating from the exposure

PAGE 137

137 stations through their respective conjugate im age points to the common object space point. Each exposure station is represented by its object space coordinates ( ) In order to further develop the observation model, the coplanarity condition takes the observation equation form in Equation 8 3 (Mikhail et al. 2001) (8 3) where is the equivalent observation, and is the fundamental matrix that contains the exterior orientation parameters (EOPs) and the interio r orientation parameters (IOPs) of both images in the stereo model, and expressed as: (8 4) where is the calibration matrix of the interior orientation parameters: (8 5) is the skew symmetric matrix of the base vector information between the two exposure stations: (8 6) and is the rotation matrix of image in the stereo pair

PAGE 138

138 (8 7) The components of these matrices, as well as the details of t he coplanarity equations, are discussed in photogrammetry books (i.e., Wolf and Dewitt 2000; Mikhail et al. 2001). Appendix C describes the coplanarity condition in further detail including the linearization of the coplanarity condition necessary for imp lementing this observation model. 8.2 Methods 8.2.1 Dynamics Model Kalman filtering requires a dynamics model to transition between epochs. Simplification of the dynamics model was desired to isolate the impact that the observation model had. Thus, certain assumptions were made that affected the time update equations in the Kalman filter algorithm. First, it was assumed that the a priori state estimate ( ) and its covariance matrix ( ) are equal to the a posterior i state estimate from the previous epoch ( ) and its covariance matrix ( ) Thus, the original linear discrete time controlled process Kalman filter equation mentioned earlier is simplifie d from: (8 8) I n a similar fashion, (8 9)

PAGE 139

139 8.2.2 Observation Model The objective of this research is to test the coplanarity condition as the observation model for a Kalman filtering approach to solving for the georeferencing parameters (EOPs). Given the dynamics model above, the measurement update equation for the Kalman gain ( ) is written in Equation 8 10 as (8 10) where is the equivalent measurement noise covariance matrix formed by the m easurement noise covariance matrix and the Jacobian matrix of the coplanarity equation with respect to the image point observations (8 11) Likewise, the observation is equivalent to from the linearized form of the coplanarity equation. Substituting into the second discrete Kalman filter measurement update equation yields the a posteriori state estimate in Equatio n 8 12 : (8 12) By integrating these assumptions with the observation model modifications, the process of Kalman filtering can be investigated via simulation. In order to make the exterior orientation parameter solution more robust than a simple forward filtering process, an optimal smoothing technique is implemented. 8.2.3 Optimal Smoothing Optimal smoothing does not require an observation model T hus, reimplementation of the coplanarity condition observation equation is not necessary.

PAGE 140

140 Smoothing is a fun ction of stochastic weighting only. It is a post mission processing scheme that uses all measurements from the initial epoch to a time to estimate a system state at epoch such that Note that the a priori and a posteriori system state estimates as well as the a priori and a posteriori covariance estimates from the forward Kalman filtering process are stored for each e poch between 0 and These estimates then form the basis of the backward smoothing operation. In theory, backward optimal smoothing accounts for the shortcoming of the forward filtering algorithm by achieving an optimal s olution equivalent to the simultaneous batch processing of all the data. While near real time processing is desired, the initial implementation of this algorithm is to simplify the model as much as possible without sacrificing optimization. There are three main types of optimal smoothers: fixed interval, fixed point, and fixed lag. This model implements a fixed interval smoothing algorithm called the Rauch Tung Striebel (RTS) backward smoother. The RTS smoother is the least complex fixed interval optimal ba ckward smoother (Gelb 1974; Nassar et al. 2007). The backward sweep of the RTS commences at the culmination of the forward Kalman filter sweep. At this point, the initial smoothed system state estimate is equal to the a posteriori system state estimate Likewise, the initial smoothed covariance estimate is equal to the a posteriori system state estimate The smoothed system state estimate at time in the RTS algorithm (Rauch et al 1965) is:

PAGE 141

141 (8 13) where is the smoothing gain matrix (similar to the Kalman gain matrix from th e forward filtering process), (8 14) Note that the covariance matrix of the smoothed estimates in Equation 8 15 is not necessary for computing the smoothed system state estimate in our case However, analysis of that c ovariance matrix was deemed necessary for determining the relative precis ion of the algorithm: (8 15) 8.3 Experimental Results 8.3.1 Simulation Model In order to test the functionality of this algorithm, a simulation model was created to mimic imagery a cquisition over a predetermined flight path. While the end goal is the utilization of this algorithm with thousands of images per flight, the initial simulation model was a strip of 500 photographs. We assume the photographs have been previously calibrated and corrected for all distortions. Over this minimal strip of images, both the forward filter and the backward optimal smoother could be seen converging on a steady state solution. Thus, enlarging the test strip at this point would only contribute additio nal redundancy to the steady state solution. Standardization of the simulations was necessary for a comparison across trials. To do this, a seeded random number generator was utilized to perturb the original inputs by altering their respective observation standard deviations (precision). A simple

PAGE 142

142 structure was designed to simulate and standardize an image matching algorithm based upon the desired number of tie points between the overlapping images. Recall that tie points are crucial for the proper implement ation of the coplanarity condition as the observation model. The focal point of this research is to determine how well this Kalman filtering and optimal smoothing algorithm can handle different parameters that affect the accuracy and precision of georefere ncing parameters for simulated UAV imagery. Numerous trials were run altering the number of tie points and the initial GPS positional precision. The trials standardized the input standard deviation of the exterior orientation angle s to 1. Most inertial me asurement units (IMU) found on a UAV are micro electromechanical systems (MEMS) capable of obtaining an RMS E of less than 0.5. Thus, the input precision is an overly pessimistic estimate. The following is a discussion of the initial findings from implemen ting this filtering and smoothing algorithm to sequential aerial triangulation. The flying height was set t o 200 m for the simulations and sixteen conjugate tie point pairs were used to satisfy the coplanarity condition in each stereo pair. The positional p recision was evaluated with horizontal precision twice as good as vertical precision. For example, the trial wi th a horizontal precision of 10 m was given an i nitial vertical precision of 20 m. Five separate trials were investigated with these input requirem ents. This was done to determine the effect input precision of the GPS position had on the output precision of the orientation angles. The following graph shows the convergence of the forward filter to a steady state solution for the precision of the exter ior orientation angles; similar results are obtained for the remaining angles.

PAGE 143

143 From Figure 8 2, the filter is increasing the precision of the orientation angles as the sequential orientation proceeds down the strip of photos A lthough the precision of the GPS position has improved by a factor of 10 for four of the five trials, the improvement in the orientation precision does not improve by a factor of 10. These findings are typical across the various tie point trials. The implication is that improving GPS positional precision from consumer grade ( 10 ) to survey grade ( 0.1 m or 0.01 m ) results in substantial gains in orientation precision. The same set of data (16 tie points with a 200 m flying height) was input in the b ackward smoothing algorithm ( Figure 8 3). The orientation precision results ind icate that the smoother can further refine the forward filter output. This marked improvement in orientation precision from 1 to 0.10 offers encouraging results for the implementation of this algorithm. In the simulation model, the actual or true v alues are known. Using d ata from previous trials, the authors analyzed the algorithm accuracy by comparing the true values with the filtered/smoothed results. The term "residual" will be used to define the difference between the true value and the estimate d (filtered/smoothed) value, while the term "error" will be used to define the injected simulation error. The errors in the orientation angles are illustrated in Figure 8 4. Theoretically, the residuals should be converging to 0, with oscillations of 0.2 based on algorithm design and the precision results. The gentle oscillations in the RTS smoothing curve over the 500 image sequence about 0 in this simulation offer stro ng support for this theory ( Figure 8 4). The similarities between the weighted avera ge and RTS smoothing are typical across the trials for all angles and tie point combinations.

PAGE 144

144 The root mean square error (RMSE) across the different input position precisions is shown in Figure 8 5. Recall that through the design of the simulation model th e original unfiltered RMSE is 1. The angular accuracy as measured by RM SE improves by approximately 75% from the consumer grade GPS position precision of 10 m to the mapping grade GPS position precision of 1m ( Figure 8 5 ). The improvement from mappin g grade to survey grade precision (l ess than or equal to 0.1 m ) is less substantial These findings suggest that advanced geodetic grade GPS receivers may not provide a substantial enough improvement in accuracy to create a positive return on investment. Furthermore, a less expensive mapping grade GPS unit may be just as suitable for georeferencing parame ter accuracy and precision as a t op of the line geodetic grade GPS unit when this algorithm is implemented. The simulation model analysis makes it appare nt that implementation of the sequential orientation algorithm can provide the user with substantial gains in both the precision and accuracy of the EOPs Thus far, no thorough investigation of the optimal number of conjugate tie point pairs was made. Furt her simulation trials were run to analyze the impact of increasing/decreasing the number of t ie points between the images. Fewer tie point pairs means increased computational efficiency. However, utilization of more tie points lead s to improv ed strength of the geometric stereo model. Furthermore, using the Kalman filter already increases computational efficiency relative to a bundle adjustment. Thus, the expectation is that greater gains in angular orientation precision can be expected with an increase in t he number of tie points used. The relationship between an average angular precision and the number of tie points using

PAGE 145

145 an input positional precision of 0.1 m horizontal and 0.2 m vertical over the 500 image strip is illustrated in Figure 8 6. The flyi ng he ight was maintained at 200 m for the simulation. The results in Figure 8 6 show an increase in the number of tie points has a direct linear relationship with an increase in precision of angular orientation (decrease in ) Relative to the precision of the original orientation angle ( 1), any subsequent gains in precision after the initial trial with four tie points are only nominal improvements due to the computational efficiency of the algorithm. The relationship betwe en accuracy (RMSE) and the number of tie points in each of the five trials is shown in Figure 8 7. As shown in Figure 8 7, the 50% improvement in orientation accuracy gained by using nine tie points instead of four is considerable However, all subsequent gains from using more tie points did not result in noteworthy accuracy improvement. These results were typical when comparing the previously used five input positional pr ecisions over the different tie point number trials. When implementing this algorithm, the minimum optimal number of tie points per image pair is nine. Any increase in the number of tie points used will further streng then the geometry of the stereo model but will lead to only nominal gains in angular precision and accuracy. 8.3.2 UAV Flight Data Archer Field Having shown significant gains in precision and accuracy for this sequential orientation algorithm in a controlled simulation environment, the algorithm's effectiveness was tested on actual UAV flight data. The data set used for the te st was captured in 2002 using a video camera mounted on a University of Florida Geomatics Program designed UAV over Archer Field. Still aerial photography ( Figure 8 8 ) derived from the video footage was evaluat ed by th is algorithm. The Archer Field site wa s

PAGE 146

146 chosen because of the numerous ground control points (GCPs) available to the researchers. The navigation data (angular orientation and position of the nodal point of the camera lens) captured during flight was deemed unreliable for use in evaluating accu racy. A simultaneous aer ial triangulation bundle adjustment was performed to determine the EOP navigation baseline from which the accuracy of this algorithm implementation could be evaluated. The flight strip consisted of 61 images with almost 80% overlap. Each image pair had about 25 30 tie point pairs. It is important to note that the tie point geometry is not nearly as strong as the simulation model tie point geometry. Thus, expectations were somewhat lower for this application. In order to show the effe ct of the forward Kalman filter and backward optimal smoother, the AT derived EOPs were perturbed in position and attitude. Gaussian noise with a standard deviation of 5 was added to the AT derived orientation angles. Bundle adjustment results suggest a h orizontal and vertical positional precision of approximately 0.1 m for the nodal point of the camera lens and an attitude precision of approximately 0.1. Thus, the true position and attitude was already perturbed. Gaussian noise added via the input posit ional precision and input orientation precision further perturbed the exterior orientation parameters. The filtered and smoothed precision results from the Archer UAV imagery exhibit significant gains in orientation precision. The appr oximately 60% improve ment in orientation precision from the Archer imagery for an input position is shown in Figure 8 9 The orientation precision results illustrate that the smoother further refines the forward filter output. This marked improvement in orientation precision f rom 5 to 1.5 offers encouraging results for the continued implementation of this algorithm.

PAGE 147

147 As mentioned previously, the authors performed a simultaneous bundle adjustment to serve as the EOP angular attitude baseline; the errors in the orientation angles are shown in Figure 8 4. Theoretically, the residuals should be converging to 0 with oscillations of 1.5 based on algorithm design and the precision results. The oscillations in the RTS smoothing curve over the 61 image sequence (about 0 in thi s model) would offer even stronger support with a longer strip of imagery ( Figure 8 10 ). Similar results were found for RTS smoothing across the trials for all angles and input position precisions. The root mean square error (RMSE) across the different inp ut position precisions for the Archer UAV imagery is graphically illustrated in Figure 8 1 1 Given the RMSE for the original input orientation parameters is 5, the angular accuracy as measured by R MSE significantly improves ( Figure 8 10). Approximately 5 0% gain can be realized from the original navigation positional precision to survey grade positional precision ( 0.1 m ). These results suggest that with weaker tie point geometry, advanced geodetic grade GPS receivers may provide a substantial enough improvement in accuracy to create a positive return on investment if GPS signal processing of mapping grade receivers cann ot produce decimeter level precision. A flight strip with more images may further improve the convergence of the oscillations around 0 for the angular orientation accuracy. 8.4 S ummary & Conclusion A method to process aerial imagery sequentially using an algorithm based on forward Kalman filtering and backward RTS optimal smoothing has been presented Th e increases of approximately 90% in angular precision and approximately 80% in orientation accuracy achieved with this algorithm in simulation relative to the original input data are a testament to the gains that can be made utilizing this algorithm for

PAGE 148

148 extrapolating quality georeferencing parameters from sequential aerial triangulation. Furthermore, initial findings using actual UAV flight data show signifi cant improvements in attitude precision of 60 % and attitude accuracy of 50% When utilizing this algorithm, it is suggested that imagery is acquired prior to reaching the target area. This premature imagery acquisition allows the system to initialize and r each steady state. Likewise, obtaining a few images beyond the target area is advised to avoid end of the strip errors from affecting the beginning of the smoothing algorithm. Further analysis of the strip model with different UAV data sets will be conduct ed. The analysis of this algorithm with block sequential aerial triangulation will be a focus for further research both in simulation and with UAV data sets This block sequential AT is a natural progression because imagery acquisition is usually done in o verlapping strips that form blocks of aerial photographs. Likewise, the application of this algorithm to a terrestrial platform is an area of much interest to the future development of GatorMMS VA implementation.

PAGE 149

149 Figure 8 1 Geometry for the coplanari ty condition (modified after Mikhail, E.M., J.S. Bethel, and J.C. McGlone. 2001. Introduction to modern photogrammetry New York: Wiley. Available at: http://www.loc.gov/catdir/toc/wiley02 1/2001281274.html ) Figure 8 2. Improved orientation precision due to improved position precision with forward Kalman filtering

PAGE 150

150 Figure 8 3 Improved orientation precision due to improved position precision with backward optimal smoothing. a: Det ail of improved orientation precision due to forward Kalman filtering and backward optimal smoothing

PAGE 151

151 Figure 8 4 Orientation accuracy of backward optimal smoothing over entire image sequence. Figure 8 5 RMSE for different position precisions with f orward Kalman filtering and backward optimal smoothing.

PAGE 152

152 Figure 8 6 The effect of the number of tie points on average angular orientation precision with forward Kalman filtering and backward optimal smoothing. Figure 8 7 The effect of the number of tie points on average angular orientation accuracy with forward Kalman filtering and backward optimal smoothing.

PAGE 153

153 Figure 8 8 UAV aerial image collected over Archer Field. The white circles in the image are the ground control points. (image courtesy o f Benjamin Wilkinson) Figure 8 9 Detail of improved orientation precision due to forward Kalman filtering and backward optimal smoothing for Archer UAV data

PAGE 154

154 Figure 8 10 Orientation accuracy of backward optimal smoothing o ver entire image seq uence through evaluation of RTS smoothing residuals for Archer UAV data Figure 8 1 1 RMSE of angular orientation for different position precisions with forward Kalman filtering and backward optimal smoothing for Archer UAV data

PAGE 155

155 CHAPTER 9 CONCLUSION 9 .1 Sum mary & Conclusion s E xtensive literature review of mobile mapping systems, vision aiding, and forest mapping reveal ed many research studies on each separate topic However, a lack of literature existed on the integration of these three topics in the use of vision aiding by a terrestrial MMS for navigation and mapping in a forested environment. The thesis addressed this void through presentation of a method for developing and testing a terrestrial remote sensing mobile mapping system the GatorMMS, beneath fo rest canopy. A thorough discussion of GatorMMS components focused on the two primary GatorMMS subsystems, the georeferencing system and the imaging system, as separate entities. The time, position, and orientation of an event in space (i.e. moment of came ra exposure) we re determined by the GPS aided inertial navigation georefe rencing system. GPS/INS processing techniques and operational methods for improving the navigation solution beneath forest canopy were investigated in the georeferencing system testin g. A digital single lens reflex camera wa s the principal component of the GatorMMS imaging subsystem. The stability of the DSLR camera interior orientation parameters were analyzed through the use of two separate calibration programs. After calibrati ng and testing each subsystem separately, the boresight lever arm (BSLA) calibration united these subsystems as one entity, the GatorMMS. The resultant BSLA calibration results we re mm level lever a r m precision and milliradian level boresight angle precision. Th ese calibration precision results,

PAGE 156

156 especially the stable orientation angles, ma d e vision aiding implementation feasible for the GatorMMS. To determine where forest cano p y conditions necessitated vision aiding implement ation a GatorMMS canopy density (CDI ) analysis operation was undertaken. A vertically oriented camera with a fish eye lens provided a wide angle view of the forest canopy along two test tracks. Even with a lack of canopy variability along the A ustin C ary M emorial F orest (ACMF) test track s s ignificant correlation existed between the canopy density and a few variables including number of satellites, PDOP, and horizontal/vertical precision for different GPS/INS integration techniques. Interestingly, the relationship between horizontal precision of the navigation solution and the time elapsed from static observation at these test tracks had the most significant correlation A high CDI test area along the ACMF test tracks was chosen for the GatorMMS vision aiding (VA) application. The DLSR camera with a fixed focal length wide angle lens was operated in horizontal orientation for the VA application. U sing photogrammetric adjustments to test VA in the forest, all exterior orientation parameter (EOP) precisions were significantly improved through VA implementation T he EOP precision that gained the most from VA heading, significantly improved by one order of magnitude H eading is the least precise EOP ; thus, improvement of this magnitude is emphasized as a major breakthrough Improvements in angular precision are essential because angular error propagates over the distance the object is from the sensor. The thesis research concluded with the development of a n orientation algorithm to sequentially handle image, position, and orientation updates. The s equential algorithm development was a product of making the vision aiding method a near real time

PAGE 157

157 approach as opposed to the batch post processing methods used in the ACMF VA test area. The direct georeferencing results from th e sequential algorithm showed quantitative improvements in angular precision and angular accuracy relative to vision aiding free navigation. 9 .2 Recommendations The most pressing need for future study is the application of the sequential orientation algorithm to a terrestrial platfor m. The direct georeferencing algorithm was not tested with loss of GPS position. Thus, implementation in an open area with GPS updates is important before implement ing under forest canopy with significant satellite obstructions. Future r esearch and applic ation of computer vision pass point generation algorithms for feature extraction in forested areas is necessary This would eliminate the manually intensive methods developed in Chapter 7 covering the ACMF VA implementation. Fisheye lenses were only used in the canopy density application. Using these lenses for horizontal MMS deployment would lead to an increased field of view and greater stereo model overlap between consecutive images. Two main obstacles exist with the use of fisheye lenses for this purpo se. Before field implementation, the interior orientation parameters must be accurately calibrated to model the geometry and distortions of the different lens structure. Previous attempts of modeling fisheye projection geometry with c onventional pinhole ca mera calibration software w ere unsuccessful. Thus, new calibration software must be developed that incorporates the lens geometry outlined in Figure 3 7. In addition, the increased field of view means objects will appear smaller and be imaged in fewer pixe ls relative to conventional

PAGE 158

158 cameras. Thus, algorithms must be efficient at matching smaller tie points between consecutive images. After improving the navigation trajectory in GPS outage prone areas, feature mapping from VA stereo models is a natural resea rch progression for f orest managers that should be explored. The extracted features w ould contain the geospatial information forest managers need for input into a forest GIS to make responsible management decisions.

PAGE 159

159 APPENDIX A LDING M ETHOD Appendix A are from Zheng et al. ( 2005 ) and Otsu ( 1979 ) Please consult these sources for f urther background information and additional derivation information The followin sky versus canopy classification. (A 1) where is the gray level is the number of gray levels is the normalized frequency of is the threshold is the normalized fraction of pixels classified as canopy and is the normalized fraction of pixels classified as sky The mean graysca le values for the canopy pixels, and the sky pixels, are: (A 2) (A 3) The variance for the canopy pi xels, is: (A 4)

PAGE 160

160 Likewise, the variance for the sky pixels, is: (A 5) The intra group variance, is: (A 6) By testing each grayscale value from 0 at the beginning of the histogram to at the end of the histogram, the threshold will be the value that minimizes

PAGE 161

161 APPENDIX B KALMAN FILTER DETAIL S The Kalman filter is a set of mathematical equations the implement a predictor corrector type estimator that is considered optimal because it minimizes the e stimated error covariance. Per Gelb ( thm that processes measurements to deduce a minimum error estimate of the state of a system by utilizing: knowledge of system and measurement dynamics, assumed Kalman fil ter is the most common technique for estimating the state of a linear system. Modified after Gelb ( 1974), Figure B 1 is a block diagram depict ing the linear flow of the Kalman filter operation. The Kalman filter predicts variables of the next state from The Kalman filter equations as follows are derived and fou nd in numerous texts including Kalman ( 1960 ), Gelb ( 1974 ), and Welch and Bishop ( 2001). Expressing this linear discrete time controlled proce ss diagram in equation form, (B 1) with the measurement equation expressed (B 2) where is the process noise (system error) is the measurement noise (measurement error) is the optional control input is the matrix that relates the state at the previous time step to the current time step is the matrix that relates the opti onal control input to the system state and is the matrix that relates the state to the observation.

PAGE 162

162 The system error and the measurement error are assumed to be independent of each other, white, and with normal probabil ity distributions. Put another way, the noise signal can be thought of as being completely uncorrelated from itself except at a precise epoch at which point the signal is completely correlated with itself. This leads individuals to refer to these signals a s independent since any sample of the signal at one time is independent (uncorrelated) from a signal sample at any other time. Again, the objective of the Kalman filter is to obtain the system state estimate ( a posterior i state estimate) as a linear combination of a predicted estimate ( a priori estimate) and a weighted difference between an observation and a measurement prediction In equation form, (B 3) where is the Kalman gain that minimizes the a posteriori error covariance and ( ) is the measurement innovation or residual. The residual is a reflection of the difference between the predicted measurement and the observation (actual measurement). A popular form of the Kalman gain matrix equation as derived in (Welch and Bishop 2001) is shown below. (B 4) where is the a priori error covariance is the transpose of the matrix which relates the state to the observation and is the measurement noise covariance matrix The final measurement up date equation used in the discrete Kalman filter algorithm is the a posteriori estimate error covariance equation. This update is used to

PAGE 163

163 formulate the a priori estimate error covariance for the next epoch, ( ) The covari ance measurement equation update is shown below. (B 5) where is the a posteriori error covariance matrix and is the identity matrix. The discrete Kalman filter algorithm is a recurs ive filter as it constantly predicts and subsequently corrects the predictions. Thus, time update equations must be considered in addition to the previously discussed measurement update equations. The following equation forms the a priori state estimate by substituting the initial estimate the a posteriori state estimate from the previous epoch, into the original linear discrete time controlled process Kalman filter equation mentioned earlier. (B 6) Likewise, it is necessary that the error covariance is projected ahead from the previous epoch. The a priori covariance estimate is computed by incorporating the initial estimate the a posteriori covariance est imate from the previous epoch, with the process noise covariance matrix (B 7) The three measurement update equations serve as corrections to the two time update equations in a cyclical manner as more measurements are added to the algorithm. As a result, the Kalman filter is being conditioned by all of the previous measurements. By sequentially handling the observation data as it comes in and incorporating this data into the filtering process, the storage requirements are much less

PAGE 164

164 demanding than a batch processing algorithm. To implement the discrete Kalman filter algorithm, an observation model must be developed.

PAGE 165

165 Figure B 1 Kalman filter operation block diagram (modif ied after Gelb, A. 1974. Applied Optimal Estimation Cambridge, MA: MIT Press. )

PAGE 166

166 APPENDIX C COPLANARITY CONDITIO N DETAIL S C.1 Non linear Coplanarity Condition The basic coplanarity eq uations have been adopted from Mikhail et al ( 2001). Where gaps in the derivation existed, further explanation and equation derivation has been provided to offer a more comprehensive overview of the coplanarity condition. As illustrated in Figure 8 1 the base vector between the two exposure stations is Vectors, and are the object space rays from the exposure stations through their respective conjugate image points to the common object space point. The components of the se three vectors are shown in Equation C 1, C 2, and C 3. (C 1) (C 2) (C 3) where are principal point coordinates are the image coordinates and is the focal length of the camera (C 4) Through matrix multiplication, expansion of the object space ray components follows:

PAGE 167

167 (C 5 ) Expressing these components in determinant form, (C 6) To further develop the observation model, the coplana rity condition can also take the form (C 7) where is the equivalent observation and is the fundamental matrix that contains the exterior orientation parameters (EOPs) and the interi or orientation parameters (IOPs) of both images in the stereo model (C 8) where is the calibration matrix that contains the interior orientation parameters as shown in Equation C 9, is the skew symmetric matrix that contains the base vector information between the two exposure stations, and is the rotation matrix of image 1 in the stereopair. (C 9)

PAGE 168

168 (C 10) Further derivation of the skew symmet ric coplanarity model can be found in (Mikhail et al 2001). The skew symmetric matrix form of the coplanarity constraint is implemented in (Webb 2007) as the observation model for an extended Kalman filter. Webb uses the observation model for vision based state estimation. The focus of the dissertation lies more in the use of the coplanarity condition for UAV navigation than for georeferencing aerial imagery. For implementation of the algorithm developed here, the premise is to obtain an optimal solution f or the georeferencing exterior orientation parameters that satisfies the coplanarity equation for a number of tie points between two images. The coplanarity condition is nonlinear as expressed above in either determinant form or skew symmetric form. C.2 L inearization of the Coplanarity Condition This algorithm implements the linearized form of the coplanarity equations for use with Kalman filtering. To linearize the determinant form of the coplanarity equation, partial derivatives of the determinant with respect to exterior orientation parameters and image point measurements are necessary. Recall that the derivative of a determinant of order 3 with respect to a parameter is equal to the sum of 3 determinants. If , and are the three rows of a determinant then (C 11)

PAGE 169

169 The linearized form of the coplanarity equation is written in Equation C 12 as an expression for one pair of conjugate image points, tie points. (C 12) w here is the vector of four observational residuals and are the corrections to the ext erior orientation parameters. (C 13) (C 14) (C 15) The matrices, and are known as Jacobian matrices. In vector calculus, a Jacobian matrix is the matrix containing the coeffi cients of the linearized observation equations (Ghilani and Wolf 2006). is the Jacobian matrix of the coplanarity equation with respect to the image point observations. is the Jacobian matrix of the coplanarity equation with respect to the exterior orientation parameters. The Jacobian represents the best linear approximation to a differentiable function at a given point. Thus, the Jacobian matrix consisting of all the tie points can be thought of as the effect geometry has on the determination of the exterior orientation parameters. Due to each image coordinate appearing in only one row, each of the elements of comprises only one determinant. The partial derivati ves with respect to the four image point observations are shown in Equations C 16 through C 19

PAGE 170

170 where (C 16) where (C 17) where (C 18) where (C 19) T he partial derivatives of the coplanarity equation with respect to the exterior orientation parameters for the first image can be found below. The partial derivatives for

PAGE 171

171 the second image are derived in a similar manner and are not shown to avoid redundanc y. Recall the Jacobian of is: (C 20) To simplify, let equal ( ) and equal ( ) (C 21) where (C 22) (C 23) where (C 24)

PAGE 172

172 (C 2 5 ) where (C 2 6 ) (C 27) (C 28) (C 29) Components , and are derived for the second image in the same manner as , and respectively were derived for the first image. (C 30)

PAGE 173

173 (C 31) (C 32) This comprehensive explora tion of the coplanarity condition was done to emphasize the implementation of this atypical observation model. Having created the Jacobian matrices, one must revisit the Kalman filter algorithm to discuss the incorporation of this measurement data into the linear filter.

PAGE 174

174 LIST OF REFERENCES Alho, P., A. Kukko, H Hyypp, H. Kaartinen, J Hyypp, and A. Jaakkola. 2009. Application of boat based laser scanning for river survey. Earth Surface Processes and Landforms. 34(13): 1831 1838. Alshawa, M., E. Smigiel, P. Grussenmeyer, and T. Landes. 2007. Integration of a Terrestrial LiDAR on a Mobile Mapping Platform: first experiences. In 5th International Symposium on Mobile Mapping Technology MMT Padua, Italy. Barber, D., J. Mills, and S. Smith Voysey. 2008. Geomet ric validation of a ground based mobile laser scanning system. ISPRS Journal of Photogrammetry and Remote Sensing. 63(1): 128 141. Bayoud, F.A. 2006. Development of a robotic mobile mapping system by vision aided inertial navigation: a geomatics approach PhD Thesis. Lausanne, Switzerland: cole polytechnique fdrale de Lausanne. Available at: http://biblion.epfl.ch/EPFL/theses/2005/3440/EPFL_TH3440.pdf Benj amin, A., A. Mohamed, and B. Wilkinson. 2010. Sequential Bundle Adjustment Using Kalman Filtering and Optimal Smoothing. Surveying and Land Information Science. 70(3): 161 169. Deckert, C. and P.V. Bolstad. 1996. Forest canopy, terrain, and distance effect s on global positioning system point accuracy. Photogrammetric Engineering and Remote Sensing. 62(3): 317 321. Di, K.C., F. L. Xu, J. Wang, S. Agarwal, E. Brodyagina, R X Li, and L. Matthies. 2008. Photogrammetric processing of rover imagery of the 2003 Ma rs Exploration Rover mission. ISPRS Journal of Photogrammetry and Remote Sensing. 63(2): 181 201. DiGruttolo, N. 2010. Marrying topography and tides; a high resolution tidal datum and intertidal zone elevation model for improved determination of short term sea level rise impacts in Florida bays Thesis (M.S.). Gainesville, FL: University of Florida. Available at: http://etd.fcla.edu/UF/UFE0042603/digruttolo_n.pdf Edmundson, K.L. and C.S. Fras er. 1998. A practical evaluation of sequential estimation for vision metrology. ISPRS Journal of Photogrammetry and Remote Sensing. 53(5): 272 285. Ellum, C.M. and N. El Sheimy. 2002. The Calibration of Image Based Mobile Mapping Systems. In Proceedings of the 2nd Symposium on Geodesy for Geotechnical and Structural Engineering Berlin, Germany: The International Association of Geodesy (IAG). Frazer, G.W., R. A. Fournier, J. A. Trofymow, and R. J. Hall. 2001. A comparison of digital and film fisheye photogr aphy for analysis of forest canopy structure and gap light transmission. Agricultural and Forest Meteorology. 109(4): 249 263.

PAGE 175

175 Gandaseca, S., T. Yoshimura, and H. Hasegawa. 2001. Evaluating the Positioning Performance of GPS Surveying Under Different Fores t Conditions in Japan. In Proceedings of the First International Precision Forestry Cooperative Symposium Precision Forestry. Seattle, WA: Institute of Forest Resources University of Washington. Gelb, A. 1974. Applied Optimal Estimation Cambridge, MA: MIT Press. Gillet, J., B. Scherzinger, and E. Lithopoulos. 2001. Tightly Coupled Inertial/GPS System for Precision Forestry Surveys Under Canopy: Test Results. In Proceedings of the First International Precision Forestry Cooperative Symposium Precision Fo restry. Seattle, WA: Institute of Forest Resources University of Washington. Glennie, C. 2009. Kinematic Terrestrial Light Detection and Ranging System for Scanning. Transportation Research Record. (2105): 135 141. Grfe, G. 2007. Quality management in k inematic laser scanning applications. International Archives of Photogrammetry and Remote Sensing XXXVI (5/C55). Available at: http://www.cirgeo.unipd.it/ cirgeo/convegni/mmt2007/proceedings/papers/graefe_ gunnar.pdf Haala, N., M. Peter, J. Kremer, and G. Hunter. 2008. Mobile LiDAR mapping for 3D point cloud collection in urban areas: a performance test. International Archives of Photogrammetry and Remote S ensing. 37(B5): 1119 1124. Haala, N., D. Stallmann, and M. Cramer, 1998. Calibration of Directly Measured Position and Attitude by Aerotriangulation of Three Line Airborne Imagery. International Archives of Photogrammetry and Remote Sensing. 32(Part 3/1): 23 30. Hassan, T., C. Ellum, and N. El Sheimy. 2006. Bridging Land based Mobile Mapping Using Photogrammetric Adjustments. In Proceedings of the ISPRS Commission I Symposium From Sensors to Imagery. Marne la Vallee, France: ISPRS: 128 139. Available at: http://www.isprs.org/proceedings/XXXVI/part1/Papers/T10 44.pdf Holden, N. M., AA Martin, P. M. O. Owende, and S. M. Ward. 2001. A method for relating GPS performance to forest can opy. International Journal of Forest Engineering. 12(2): 51 56. Hu, L., Z. Gong, J. Li, and J. Zhu. 2009. Estimation of canopy gap size and gap shape using a hemispherical photograph. Trees. 23(5): 1101 1108. Inertial Explorer 2008. Inertial Explorer Help Manual Waypoint Products Group Novatel

PAGE 176

176 Jennings, S., N. Brown, and D. Sheil. 1999. Assessing forest canopies and understory illumination: canopy closure, canopy cover and other measures. Forestry. 72(1): 59 74. Kalm an, R.E. 1960. A New Approach to Linear Filtering and Prediction Problems. Transactions of the ASME Journal of Basic Engineering. 82D: 35 45. Kennedy, S., J. Hamilton, and H. Martell. 2006. Architecture and system performance of SPAN Novatel solution. In Proceedings of IEEE/ION PLANS 2006 San Diego, CA. Available at: http://webone.novatel.ca/assets/Documents/Papers/SPAN_PLANS2006.pdf Kersten, T.P. and E.P. Balt savias. 1994. Sequential Estimation of Sensor Orientation for Stereo Images Sequences. International Archives of Photogrammetry and Remote Sensing 30(5): 206 213. Kukko, A., C. O Andrei, V. M Salminen, H. Kaartinen, Y Chen, P. Rnnholm, H Hyypp, J Hyypp R. Chen, and H. Haggrn, others. 2007. Road Environment Mapping System of the Finnish Geodetic Institute FGI Roamer. In Laser Scanning 2007 and SilviLaser 2007 3/W52. ISPRS Workshop 2007. Espoo, Finland: ISPRS, 241 247. Available at: http://www.isprs.org/proceedings/XXXVI/3 W52/final_papers/Kukko_2007b.pdf Li, R.X. 1997. Mobile mapping: An emerging technology for spatial data acquisition. Photogrammetric Engineering and Remote Sensing. 63(9): 1085 1092. Lowe, D.G. 1999. Object recognition from local scale invariant features. In Seventh IEEE International Conference on Computer Vision. Corfu, Greece, 1150 1157. Mikhail, E.M., J.S. Bethel, and J.C. McGlone. 2001. Introd uction to modern photogrammetry New York: Wiley. Available at: http://www.loc.gov/catdir/toc/wiley021/2001281274.html Mohamed, A.H. 2010. SUR 6535 GPS/INS Integration Course Notes. Un iversity of Florida: Gainesville, Florida. shoreline. Point of Beginning. (November). Morales, Y., T. Tsubouchi, and S. Yuta. 2010. Vehicle Localization in Outdoor Mountaino us Forested Paths and Extension of Two Dimensional Road Centerline Maps to Three Dimensional Maps. Advanced Robotics. 24(4): 489 513. Muller, R.N. and D.S. Maehr. 2000. Are Universities Leaders in the Stewardship of Conservation Lands? BioScience. 50(8): 7 07 712.

PAGE 177

177 Nassar, S., X. J. Niu, and N. El Sheimy. 2007. Land vehicle INS/GPS accurate positioning during GPS signal blockage periods. Journal of Surveying Engineering ASCE. 133(3): 134 143. Nobis, M. and U. Hunziker. 2005. Automatic thresholding for hemisp herical canopy photographs based on edge detection. Agricultural and Forest Meteorology. 128(3 4): 243 250. NovAtel Inc. 2006. GNSS/INS and DMI Processing of SPAN data with Inertial Explorer 7.61. Available at: http://webone.novatel.ca/assets/Documents/Waypoint/Reports/IE_DMI_ResultsV 2.pdf Otsu, N. 1979. A threshold sel ection method from gray level histograms. IEEE Transactions on Systems, Man, and Cybernetics. 9(1): 62 66. Perry, J. 2009. A synthesized directly georeferenced remote sensing technique for small unmanned aerial vehicles M.S. Gainesville, FL: University of Florida. Available at: http://etd.fcla.edu/UF/UFE0025070/perry_j.pdf Petrie, G. 2010. An Introduction to the Technology Mobile Mapping Systems. GEOInformatics. 32 43. Piedallu, C. and J. Ggout. 2005. Effects of forest environment and survey protocol on GPS accuracy. Photogrammetric Engineering and Remote Sensing. 71(9): 1071 1078. Powell, T. L, G. Starr, K. L Clark, TA Martin, and H. L Gholz. 2005. Ecosystem and understory water and energy excha nge for a mature, naturally regenerated pine flatwoods forest in north Florida. Canadian Journal of Forest Research. 35(7): 1568 1580. Rauch, H. E., F. Tung, and C. T. Striebel. 1965. Maximum likelihood estimates of dynamic systems. AIAA Journal. 3(8): 144 5 1450. Reutebuch, S. E, W. W. Carson, and K. M. Ahmed. 2003. A Test of the Applanix POS LS Inertial Positioning System for the Collection of Terrestrial Coordinates Under a Heavy Forest Canopy. In Proceedings of the Second International Precision Forest ry Symposium Seattle, WA, 21 27. Available at: http://www.cfr.washington.edu/research.pfc/publications/Reutebuch_Applanix_pa per6_9_03.pdf Rossmann, I J, D.I.P. Krahwinkler, and D.I.C. Schlette. 2009. Navigation of Mobile Robots in Natural Environments: Using Sensor Fusion in Forestry. In 7th International Conference on Computing, Communications and Control Technologies (CCCT), July 10th 13th Orlando, Florida.

PAGE 178

178 Samadzadegan, F., M. Hahn, and S. Saeedi. 2007. Position Estimation of Aerial Vehicle Based on a Vision Aided Navigation System. In Visualization and Exploration of Geospatial Data 4/W45. 2007 ISPRS Joint Workshop. Stuttgart, Germany: ISPRS. Av ailable at: http://www.geovisualisierung.net/isprs2007/docs/41_Samadzadegan.pdf Schneider, D., E. Schwalbe, and H. G Maas. 2009. Validation of geome tric models for fisheye l enses. ISPRS Journal of Photogrammetry and Remote Sensing 64 (3): 259 266. Schwarz, K. P, and N. El Sheimy. 2004. Mobile Mapping Systems State of the Art and Future Trends. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. 35(B5): 759 768. Schwarz, K.P. & Wei, M., 2000. INS/GPS Integration for Geodetic Applications Lecture Notes. Dept of Geomatics Engineering: University of Calgary. Sigrist, P., M. Coppin, and P. Hermy. 1999. Impact of forest canopy on quality a nd accuracy of GPS measurements. International Journal of Remote Sensing. 20(18): 3595 3610. Optimizing georeferencing of airborne survey systems by INS/DGPS PhD Thesis. Calgary, Alberta, CAN: University of Calgary. Available at: http://www.ucalgary.ca/engo_webdocs/KPS/99.20126.JSkaloud.pdf Smith, W. Brad, P. Miles, J. S. Vissage, and S. A. Pugh. Forest Resources of the United States, 2002. Available at: http://www.ncrs.fs.fed.us/ Talaya, J., R. Alamus, E. Bosch, A. Serra, W. Kornus, and A. Baron. 2004. Integration of a terrestrial laser scanner with GPS/IMU orientation sensors. In Proceedings of the Congress of the Intern ational Society for Photogrammetry and Remote Sensing (ISPRS) XX ISPRS Congress, Commission 5. Istanbul, Turkey. Tao, C.V. 2000. Semi Automated Object Measurement Using Multiple Image Matching from Mobile Mapping Image Sequences. Photogrammetric Engineeri ng and Remote Sensing. 66(12): 1477 1485. Taylor, C.N. 2009. Enabling Navigation of MAVs through Inertial, Vision, and Air Pressure Sensor Fusion. Multisensor Fusion and Integration for Intelligent Systems. 143 158. Tommaselli, A.M.G. and C.L. Tozzi. 1996. A recursive approach to space resection using straight lines. Photogrammetric Engineering and Remote Sensing 62(1): 57 66. Wackrow, R., J.H. Chandler, and P. Bryan. 2007. Geometric consistency and stability of consumer grade digital cameras for accurate spatial measurement. The Photogrammetric Record. 22(118): 121 134.

PAGE 179

179 Wang, X. and T.A. Clarke. 2001. Separate adjustment in close range photogrammetry. ISPRS Journal of Photogrammetry and Remote Sensing. 55(5 6): 289 298. Webb, T.P. 2007. Vision based state estimation for uninhabited aerial vehicles using the coplanarity constraint PhD Dissertation. Gainesville, FL: University of Florida. Available at: http://purl.fcla.edu/fcla/etd/UFE0019262 Welch, G. and G. Bishop. 2001. An Introduction to the Kalman Filter. In SIGGRAPH 2001 Chapel Hill, NC: University of North Carolina at Chapel Hill, 45. Available at: http://www.cs. unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_08.pdf Wilkinson, B.E., B.A. Dewitt, A.C. Watts, A.H. Mohamed, and M.A. Burgess. 2009. A New Approach for Pass point Generation from Aerial Video Imagery. Photogrammetric Engineering and Remote Sensing. 75(12): 1415 1423. Wolf, P.R. and B.A. Dewitt. 2000. Elements of Photogrammetry: with Applications in GIS Boston: McGraw Hill. Wright, W.C. 2008. Quantifying global position system signal attenuation as a function of three dimensional forest canopy struct ure Thesis (M.S.). Gainesville, FL: University of Florida. Available at: http://etd.fcla.edu/UF/UFE0021906/wright_w.pdf Xiaopeng, W., K. Gaofeng, and H. Xiaobin. 2008. Forest fire fighting comm and and decision making system study based on 3s technology take Jiaohe city in Jilin province as an example. In Spatial Data Integration for Emergency Services ISPRS Congress Beijing 2008. Beijing, China: ISPRS, 1113 1116. Available at: http://www.isprs.org/proceedings/XXXVII/congress/4_pdf/194.pdf Xiong, K., L. D. Liu, and H. Y. Zhang. 2009. Modified unscented Kalman filtering and its application in autonomous satellite navig ation. Aerospace Science and Technology. 13(4 5): 238 246. Zengin, H. and A. Yesil. 2006. Comparing the performances of real time kinematic GPS and a handheld GPS receiver under forest cover. Turkish Journal of Agriculture and Forestry. 30: 101 110. Zheng, J., Y. Wang, and N.L. Nihan. 2005. Quantitative evaluation of GPS performance under forest canopies. In Networking, Sensing and Control, 2005. Proceedings. 2005 IEEE. 777 782.

PAGE 180

180 BIOGRAPHICAL SKETCH Adam R Benjamin was born in South Kingstown, Rhode Islan d to his parents, Robert and Muriel Benjamin in April 1 982. Adam has a young son, Kai Benjamin, with his wife Erica Sweitzer. He has one younger brother, Zachary Benjamin. Adam was raised in the small coastal town of Narragansett, RI. While growing up, h e was active in many outdoor activities including basketball, golf, soccer, baseball, hiking, camping, and the beach. He attended The Prout School in Wakefield, RI where he obtained an International Baccalaureate Diploma in 2000. Upon graduation, he enroll ed in Elon College (now Elon University) as an Honors Fellow. Adam graduated cum laude in May 2004 with a BS degree in Mathematics and extensive studies in Economics. Adam successfully pursued real estate licensure in Rhode Island and Florida. He worked i n the real estate and mortgage industry until 2006. Adam then switched careers by working at a small surveying and engineering firm in Charlestown, RI. H e enjoyed the mix of applied mathematics and science in an outdoor environment. In August 2008, h e succ essfully completed the Professional Land Surveying Certificate Program at Wentworth Institute of Technology in Bost on, MA. Desiring more advanced education in surveying and mapping, Adam enrolled as a graduate assistant in the geomatics department in the School of Forestry and Resource Conservation at the University of Florida in August 2009. He received his master's degree from the University of Florida in the summer of 2011. Adam's academic interests include photogrammetry, LiDAR, mobile mapping systems hydrographic surveying and coastal mapping He is currently employed by the University of Florida as a Geomatics Specialist and Program Assistant in the Fort

PAGE 181

181 Lauderdale Research and Education Center. In this position Adam teach es geomatics courses, adv ise s students, collaborate s on geomatics research projects, and continue s his education al pursuit of a doctoral degree in geomatics.