Camera Based Robotic Control

MISSING IMAGE

Material Information

Title:
Camera Based Robotic Control
Physical Description:
1 online resource (68 p.)
Language:
english
Creator:
Niu,Angela
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Master's ( M.S.)
Degree Grantor:
University of Florida
Degree Disciplines:
Mechanical Engineering, Mechanical and Aerospace Engineering
Committee Chair:
Crane, Carl D
Committee Members:
Dixon, Warren E

Subjects

Subjects / Keywords:
recognition -- sift -- stitching
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
Genre:
Mechanical Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Computer vision is a new field of study that can greatly be used in autonomous vehicle navigation or other industrial automation. In this thesis, the SIFT feature which is a scale invariant feature transform is applied in multiple computer vision tasks such as image stitching and object recognition. The full derivation of the Lowe SIFT features is discussed in detail in this thesis. In the image stitching part, first the SIFT features are extracted from images. Then k-d tree method is used for feature matching between the sets of the SIFT features of two images. Once the matches are obtained, the homography transformation matrices are obtained with the matches using the RANSAC method. Image stitching is achieved after the image is transformed by the homography transform matrix. Panorama stitching is a further development of image stitching. The goal is to produce a panoramic stitched image while the vehicle is moving through an unknown environment. There are some additional processes that need to be developed for the panorama stitching, such as color correction. Due to the illumination differences, there may be artificial edges in the stitched image, so color correction is used to minimize the artificial edges. The SIFT feature can also be applied in object recognition which is called feature based object recognition. Since the SIFT features are distinctive and invariant, it is plausible to use them to achieve object recognition. Through the feature matching and verification processes, the SIFT feature is able to recognize the object from different scene images. The number of feature matches is also returned as a standard for object recognition.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Angela Niu.
Thesis:
Thesis (M.S.)--University of Florida, 2011.
Local:
Adviser: Crane, Carl D.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043417:00001


This item is only available as the following downloads:


Full Text

PAGE 1

1 CAMERA BASED ROBOTIC CONTROL By QIANG NIU A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE UNIVERSITY OF FLORIDA 2011

PAGE 2

2 2011 Qiang Niu

PAGE 3

3 To my dear parents, thank you so much for your full support!

PAGE 4

4 ACKNOWLEDGMENTS I would like to acknowledge the inspirational guidance to Dr. Carl D. Crane. Without his patient support and kind advice, I could not have comp leted this thesis. I also would like to thank manage r s of Center for Intelligence Machines and Robotics, Dave Armstrong and Shannon Ridgeway for their valuable advice. I would like to thank all my lab mates at CIMAR. Special thanks to Vishesh Vikas for hi s explanations of some terminologies and concept s Junsu Shin and Sujin Jang have been very generous in support ing of programming assistance. I would also like to thank Anubi Olugbenga Moses and his fiance Serena Sealy for the ir patient support. Finally I would like to thank all my friends for their support and encouragement, Joel Oliver, Andrew Sharkey, Ayman Abdellatief and Divik Schueller They helped me a lot for revising my writing. Again, special thank my parents, Ying Hai and Jingyuan Niu I could never have finished my thesis without their selfless love and support!

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 7 LIST OF FIGURES ................................ ................................ ................................ .......... 8 LIST OF ABBREVIATIONS ................................ ................................ ........................... 10 ABSTRACT ................................ ................................ ................................ ................... 11 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 13 Motivation ................................ ................................ ................................ ............... 13 Literature Review ................................ ................................ ................................ .... 14 F AST Corner Detection ................................ ................................ .................... 14 SIFT Feature Detection ................................ ................................ .................... 14 PCA SIFT Feature Detector ................................ ................................ ............. 15 SURF Feature Detector ................................ ................................ .................... 15 2 RESEARCH GOAL ................................ ................................ ................................ 17 Problem Statement ................................ ................................ ................................ 17 De velopment ................................ ................................ ................................ ........... 18 3 IMAGE STITCHING ................................ ................................ ................................ 19 Background ................................ ................................ ................................ ............. 19 Features Finding ................................ ................................ ................................ ..... 19 Scale Space Extrema Detection ................................ ................................ ....... 20 Keypoint Localization ................................ ................................ ....................... 22 Orientation Assignme nt ................................ ................................ .................... 24 Keypoint Descriptor ................................ ................................ .......................... 25 Implementation ................................ ................................ ................................ 26 Features Matching ................................ ................................ ................................ .. 26 Nearest Neighbor Indexing ................................ ................................ ............... 27 Implementation ................................ ................................ ................................ 29 Homography Transformation ................................ ................................ .................. 30 Homography ................................ ................................ ................................ ..... 30 Random Sample Consensus ................................ ................................ ............ 32 Implementation ................................ ................................ ................................ 33 4 PANORAMA STITCHING ................................ ................................ ....................... 42

PAGE 6

6 Background ................................ ................................ ................................ ............. 42 Finding Overlap Area ................................ ................................ .............................. 43 Color and Luminance Compensation ................................ ................................ ...... 45 5 OBJECT RECOGNITION ................................ ................................ ....................... 53 Background ................................ ................................ ................................ ............. 53 Object Recognition Using SIFT Features ................................ ................................ 54 6 IMPLEMENTATIONS AND CONCLUSIONS ................................ .......................... 59 Results ................................ ................................ ................................ .................... 59 Conclusions ................................ ................................ ................................ ............ 59 LIST OF REFERENCES ................................ ................................ ............................... 66 BIOGRAPHICAL SKETCH ................................ ................................ ............................ 68

PAGE 7

7 LIST OF TABLES Table page 3 1 Procedure of RANSAC a lgorithm ................................ ................................ ....... 33

PAGE 8

8 LIST OF FIGURES Figure page 3 1 Bu ilding image pyramid and derivation of DOG ................................ .................. 35 3 2 Detection of local minimum and maximum by comparing with the neighbors ..... 35 3 3 Gradient orientation histogram ................................ ................................ ........... 36 3 4 Producing the keypoint descriptor ................................ ................................ ...... 36 3 5 Implementation of finding the SIFT features ................................ ....................... 37 3 6 K d tree decomposition ................................ ................................ ....................... 37 3 7 Features matching of two images using k d tree ................................ ................ 38 3 8 Homography transformation of image. A) Source image. B) Image after homography transform. ................................ ................................ ...................... 39 3 9 Example of image stitching using SURF A) First ima ge to be stitched. B) Second image to be stitched. C) Stitched image using SURF ............................ 40 3 10 Implementation of image stitching using the SIFT A) First image to be stitched. B) Second image to be stitched. C) Stitched image using SIFT ........... 41 4 1 Three cases that image A and B are intersected ................................ ................ 48 4 2 The overlap area with points of intersection ................................ ....................... 48 4 3 Finding overlap area of two images. A) Image1 B) Image2. C) Overlap area of image1. D) Overlap area of image2. ................................ ............................... 49 4 4 Overlapping source images notations ................................ ................................ 50 4 5 Work flow of the color correction ................................ ................................ ........ 50 4 6 Implementation of Gamma correction.. ................................ ............................... 52 5 1 Attempt testing of object recognition. Training image is shown on the top; Image captured from camera is shown on the bottom ................................ ........ 57 5 2 Comparison of object recognition using t he SIFT with verification. A) T he recognition before verification; B ) T he one after verification. .............................. 58 6 1 T he parameters of the c amera MvBlueFOX ................................ ....................... 61 6 2 Panorama stitching ................................ ................................ ............................. 63

PAGE 9

9 6 3 Object recognition using the SIFT features. ................................ ....................... 65

PAGE 10

10 LIST OF ABBREVIATION S DOG Di fference of Gaussian DOH Difference of Hessian LOG Log of Gaussian NN Neural Network PCA Principal Components Analysis RANSAC Random Sample Consensus SIFT Scale Invariant Feature Transform SSD Sum of Squared Difference SURF Speed Up Robust Features SUSAN S mall Univalue Segment Assimilating Nucleus

PAGE 11

11 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Science CAMERA BASED ROBOTIC CONTROL By Qiang Niu August 2011 Chair: Carl D. Crane III Major: Mechanical Engineering Computer vision is a new field of study that can greatly be used in autonomous vehicle navigation or other industrial automation. In this thesis, the SIFT feature which is a scale invariant feature transform is applied in multiple computer vision tasks such as image stitching and object recognition. The full derivation of the Lowe SIFT features is discussed in detail in this thesis. In the image stitching p art, first the SIFT features are extracted from images. Then k d tree method is used for feature matching between the sets of the SIFT features of two images. Once the matches are obtained, the homography transformation matrices are obtained with the match es using the RANSAC method. Image stitching is achieved after the image is transformed by the homography transform matrix. Panorama stitching is a further development of image stitching. The goal is to produce a panoramic stitched image while the vehicle i s moving through an unknown environment. There are some additional processes that need to be developed for the panorama stitching, such as color correction. Due to the illumination differences, there may be artificial edges in the stitched image, so color correction is used to minimize the artificial edges.

PAGE 12

12 The SIFT feature can also be applied in object recognition which is called feature based object recognition. Since the SIFT features are distinctive and invariant, it is plausible to use them to achieve object recognition. Through the feature matching and verification processes, the SIFT feature is able to recognize the object from different scene images. The number of feature matches is also returned as a standard for object recognition.

PAGE 13

13 CHAPTER 1 INT RODUCTION Motivation Computer Vision is a relatively new field of study and it can be applied to a variety of fields. It enables the machine to utilize images to extract information that is necessary to solve tasks. Many methods and applications about Comp uter Vision are still in the state of basic research, but more and more efforts have found their way into commercial products, such as automobile driver assistance, digital photography, people tracking, medical and biomedical images and in the area of ind ustrial automation and inspection. Camera sensors are becoming economical and thus problem solving is becoming more prominent with the use of camera. Camera based robotic control is one of the applications of Computer Vision I t is widely used in the fiel d of navigation for unmanned autonomous vehicle s It enables a robot to be control led with the feedback information extracted from camera sensors. The images captured from the camera sensors provide a large amount of information. But image data analysis pr ocess es are computationally expensive so that making concise use of images becomes necessary. Instead of processing all the image data, image features are more representative to be used for future tasks. There a re varieties of methods to detect the feature s, such as corner detection, edge detection, or blob detection The exact extraction of features depends on the problem or the type of application. To achieve the task of image stitching, features that are invariant to translations, rotations, scaling and other transformations will be important There are some methods that are able to detect critical features and they are discussed in the literature review sections.

PAGE 14

14 Literature Review There are different methods to detect features from images Through compa rison of these methods in this literature review, the most suitable feature is chosen for image stitching. F AST Corner Detection F AST corner detection is used as the first step of computer vision tasks such as tracking, localization, image matching and r ecognition. Corner detectors have been widely used as feature point detectors because corners correspond to image locations with high information content and they can be matched between images reliably. F AST corner detection is high speed detection co mpare d with Harris, SUSAN, or LOG, DOG, and DO H detection which yield high quality features but are computationally intensive (Luo Juan and Oubong Gwun in 2010). This is important because FAST can be used in real time applications FAST corner detection is able to achieve high quality feature detection and it is repeatable between images. But FAST corner detection also has its cons. The high speed doesn't generalize well so the choice of high speed is not optimal of noise and it is dependent on threshold. But according to the high speed detection, FAST corner detection is often used in real time video tracking such as the application in smart phones developed by Nokia Research Center SIFT Feature Detection Lowe (2004) presented the SIFT for extracting distinguished invariant features from images that can be invariant to image scale and rotation The SIFT features can also provide robust matching across the substantial range of affine distortion, change in 3D viewpoint, addition o f noise and change in illumination

PAGE 15

15 Developed by Lowe in distinctive image features from scale invariant keypoints, t he SIFT feature detection includes four stages: scale space extrema detection, keypoint localization, orientation assignment and keypoin t descriptor. DO G was used instead of Gaussian to identify the potential interested points in order to imp rove the computational speed. He ssian Matrix was used to compute the principal curvatures in order to eliminate low contrasts points and edge response s. Dominant orientations are assi gned to localized keypoints. The se steps ensure the invariant features. More details about the SIFT will be discussed in Chapter 3. PCA SIFT Feature Detector The PCA SIFT feature detector is a te chnique for dimension reduct ion. PCA SIFT feature is able to represent the keypoint patches and enables one to linearly project high dimensional samples into a low dimensional feature space. PCA SIFT uses PCA instead of a histogram to normalize patch es T he feature vector is signifi cantly smaller than the standard SIFT feature vector and it can be used with the same matching method The PCA based feature descriptors are also distinctive and robust to image deformation s. However PCA still has its shortcomings such as its implicit ass umption of Gaussian distributions and its restriction to orthogonal linear combinations T he methods of extracting robust features are time intensive As a result, the PCA SIFT feature detector is often used in high dimension situation s SURF Feature Detec tor The SURF feature is partly inspired by the SIFT descriptor, faster and more robust against different image transformations than the SIFT. SURF employs a slightly different way to detect features from the SIFT. The SIFT builds an

PAGE 16

16 image pyramid and then uses DO G. SURF creates a stack without down sampling for higher levels in the pyramid resulting in images of the same resolution. The invariant features are always preferred but in some cases, rotation invariance can be left out, re sulting in a scale SURF). In the applications like mobile robot navigation, the camera only rotates about the vertical axis. The benefit of avoiding the rotation invariance is not only increased speed, but also increased discriminative power. SURF relies on integral images to reduce the computation time so it is call ed a wavelet responses within the interest point neighborh ood, which can be computed extremely fast. However, SURF does have some loss of accuracy from the SIFT in the SIFT.

PAGE 17

17 CHAPTER 2 RESEARCH GOAL P roblem Statement From the pr evious literature review each type of image feature has advantages and drawbacks. For the image stitching task, it is desired to get the image features which do not vary with differences in translations, rotations and change s in the 3D view In order to produce correct matches for stitching, not only the features need to be distinguished but also the number of features detected need s to be sufficient The speed of finding the features is less critical than the accuracy. So the SIFT features are used t o accomplish the image stitching task For image stitching, it is desired to seamlessly stitch together the images that hav e overlap area and make the stitched image result look natural Once the features are indentified the next step is to determine the common features that match between the two images In order to stitch the image together, one of the image s is transformed into the image plane of the other image using the matches between two images. A further use of stitching is panorama stitching. T he vehicle is expected to get multiple images while it is traversing through an unknown environment then produce a single image of the surroundings for the unknown environment However, there are some problems to solve when stitching several images t ogether. The image may not be straighten ed after stitching and t here may be some artificial edges between the images due to the differences in illumination. Straightening the image and gain compensation are necessary process es after the normal stitching. T he SIFT feature is very distinguished and large number of the SIFT features can be detected effectively from an image I t is plausible to achieve feature based object

PAGE 18

18 recognition with the SIFT features This feature based object recognition can be used when traini ng images are stored as local data. O bject recognition is then performed while the vehicle is capturing images. It recognizes an object defined in the training image is detected. Development As described in the problem statement section, three main tasks a re discussed in this thesis : image stitching, panorama stitching and object recognition using the SIFT features The developmen t of the tasks in this thesis is as follow s : Lowe Approach to detect the SIFT features A method is developed to find the matches between the SIFT features. This method can be used in image stitching and in object recognition. Finding the homogenous transformation with the SIFT matches to achieve image stitching Perform panorama stitching using the image stitching method. Color cor rection is applied additionally for elimina tion of the artificial e dges caused by the differences in illumination. Feature based object recognition is achieved by matching the detected the SIFT features. A verification method is introduced to find better m atches with fewer errors involved

PAGE 19

19 CHAPTER 3 IMAGE STITCHING Background Image stitching is a fundamental problem in computer vision I t is a process of combining multiple images with overlapping fields of view to produce a panoramic panel of a high reso lution image. On an unmanned autonomous vehicle, image stitching can be used when process ing images captured from a camera mounted on the vehicle, in order to extract more information from smaller image data files There are four steps to process image st itching: to find features of the overlapped area of two images, to get feature matches from each feature set of the images, to find the homogenous transformation of the two image s, and to get warp image and blending. From the literature review section, res earch suggests use of the SIFT feature method, which is invariant to scaling, rotation and other changes. In feat ure matching, a special k d tree method called Best Bin First search is used to get match ing sets. Once the matching feature sets are obtained the homogenous transformation matrix can be determined using the RANSAC method. Details are discussed in the following sections. Features Finding As the literature review explains the SIFT features are used for image stitching. The reason of using the SI FT features is that the SIFT features are invari ant to image scale and rotation. They can provide robust matching across the substantial range of affine distortion, change in 3D viewpoint, addition of noise and change in illumination. Large number s of dis tinctive features can be extracted from a typical image, which allows a single feature to be correctly matched with high probability from a database of many features

PAGE 20

20 There are four steps to get the SIFT features: scale space extrema detection, k eypoint lo calization, orientation assignment and k eypoint descriptor. Scale Space Extrema Detection The first step of detection is the Scale Space Extrema detection. It searches over all scales of the image to get the potential points that are invariant to scale a nd rotation The first stage of keypoint detection is to find the locations and scales that can be assigned under different views of the same object Detecting the locations that are invariant to the scales can be achieved by searching for the stable featu res across all the possible scales using scale space. The scale space representation of the image is usually parameterized by one parameter called the scale parameter such as the size of smoothing kernel, to represent the images as a one parameter family of smoothed images. The main reason for generating a scale space representation is based on the observation that the real world objects are composed of different structures at different scales. When an unknown scene is analyzed it is impossible to know a priori what scales are appropriate to describe the interesting structures from the image data. To combat this factor, description s at multiple scales will be able to capture the unknown scale variations. A scale space representation is considered to repres ent at all scales. Koenderink (1984) and Lindeberg (1994) have shown that under a variety of reasonable assumptions the only possible scale space kernel is the Gaussian function The Gaussian filter is widely used because it is able to reduce noise and detail. It is a space. It is important that the smoothing filter does not introduce new spurious orresponding

PAGE 21

21 structures at finer scales. The Gaussian filter is presented to generate linear scale space based on the foresaid requirement. Therefore the scale space of an image I(x,y) is defined as L(x,y, which is produced from the convolution of the G aussian function G(x,y, and image I(x,y): ( 3 1) Where is the convolution operation in x and y, and ( 3 2) In order to detect stable keypoint locations in scal e space, difference of Gaussian (DOG) is used to convolve with the image to co nstruct the image pyramid (3 3 ) The reason to use DO G is that firstly it is an efficient function to compute to smooth the image so is the image subtraction. Secondly it provid es a close approximation to the scale normalized Laplacian of Gaussian (LOG) From the heat diffusion equation 3 4 equation 3 5 can be derived as follows : ( 3 4) ( 3 5) In 2002, Mikolajczyk found that the maxima and minima of the LO G produce the most stable image features compared to other image functions such as gradient, Hessian, or the Harris corner function through detailed experimental comparisons. The factor (k To build the image pyramid D(x,y, ), t he initial image is convolve d with Gaussians to produce one octave. The images in the same octave are separated by a constant factor

PAGE 22

22 k in scale space, shown o n the left in F igure 3 1. E ach octave is divided into an integral number s, of intervals, so k= It is necessary to produce s+3 images to cover the complete octave. Once a complete octave has been processed, the Gaussian image which has twice the initial value of is resampled and the process repeat s By this method, the accuracy of sampling r elative to is the same as the starting octave but the computation is greatly reduced. Adjacent image scales are subtracted to produce the DO G image shown on the right in F igure 3 1. Once the DO G is produced the maxima and minima of D(x,y, ) can be dete cted by comparing each sample point with its eight neighbors in the current image and nine neighbors in the scale above and below (Figure 3 2). The point is selected when it is larger or smaller than all of its neighbors. Keypo int Localization After the ke ypoint candidates are found by comparison to its neighbors, i t is necessary to find the accurate keypoint location, scale and ratio of the principal curvatures The ratio of the principal curvatures is for eliminating low contrast keypoints and edge respo nses. In 2002, Brown and Lowe developed a method for fitting a 3D quadratic function to the local sample points to determine the interpolated location of the maximum This approach uses the Taylor expansion of the scale space function D(x,y, ). Let X=(x,y ), ( 3 6)

PAGE 23

23 Where D and its derivatives are evaluated at the sample point and X is the offset from the point. The lo cation of the extremum is determined by taking the derivative of this function with respect to X and set ting it to zero: ( 3 7) ( 3 8) The Hessian and derivative of D are approximated by the differences of neig hboring sample points. The function value at the extremum can be obtained by: ( 3 9) If the value of | | is less than 0.03, the p oint will be discarded. Therefore the low contrast points can be eliminated by setting the contrast threshold As the edge responses which were caused by the DO G function the poorly defined peaks in DO G will have large principal curvatures across the edge but a small one in the perpendicular direction. T he principal curvature s can be produced using the Hessian Matrix. ( 3 10) The eigenvalues of H are proportional to the principal curvatures of D. By calculating the ratio computing the eigenvalues can be avoided Let be the ei genvalue with the larger magnitude and be the smaller one, equations 3 11 and 3 12 are obtained as follows : ( 3 11) ( 3 12) L et be the ratio between the larger magnitude eigenvalue and the smaller one, so:

PAGE 24

24 ( 3 13) w hich depends only on the ratio of the eigenvalues The value goes to minimum when two eigenvalues are equal and it increases with Therefore, in order to discard the edge responses, the threshold value for needs to be assigned and it is implemented as: ( 3 14) Orientation Assignment Once the ke ypo ints that are invariant to scal ing are found, the next step is to achieve invariance to image rotation. By assigning consistent orientation to each keypoint based on the local image properties the keypoint descriptor can be represented relat ive to this orientation. Lowe ( 2004 ) showed that using the flowing approach to assign orientation gives the most stable results through experiments The scale of the keypoint is used to select the Gaussian smoothed image L. For a certain scale, assign each image L(x,y) the gradient magnitude m(x,y) and orientation (x,y) as: ( 3 15) ( 3 16) By assigning the gradient orientations of sample points within a region around the keypoint, an orientation histogram is formed It has 36 bins cove ring the 360 degree range of orientations shown in Figure 3 3 E ach sample to the histogram is added by weighting it with its gradient magnitude and a Gaussian weighted circular window with the scale parameter that is 1.5 times that of the scale of the ke ypoint

PAGE 25

25 Keypoint Descriptor By doing the previous operations, the image location, scale, and orientation have been assigned to the keypoint T herefore it is able to provide invariance to scale and o rientation. The next step is to compute the keypoint desc riptor for the highly distinctive image region so it can be invariant to the change in illumination or 3D viewpoint. Edelman, Intrator, and Poggio (1997 ) proposed a representation based on a biological vision model particularly in complex neurons in the v isual cortex. They performed detailed experiments showing that matching gradients while allowing for position shift results in better classification. First the image gradient magnitudes and orientations are computed around the keypoint locations. Next the level of the Gaussian blur is chosen by the scale of the keypoint shown on the left side of Figure 3 4 For orientation invariance, the coordinates of the keypoint and the gradient orientation are rotated relative to the keypoint orientation. A Gaussian w eighting function is shown as the window on the left side of Figure 3 4. The reason of using the Gaussian window is to avoid sudden changes in the descriptor and to give less emphasis to the gradients on the edge as they are mostly affected by misregistrat ion errors. The keypoint descriptor is shown on the right side of Figure 3 4. It shows eight directions for each orientation histogram over 4 4 sample regions, with the length corresponding to the magnitude of the histogram entry A gradient samp le on the left can shift up to four sample positions while remaining the same histogram on the right, so that it achieves the objective of allowing large positional shifts The feature vector is modified to reduce the change of illumination. First, t he vector is n ormalized to unit length. Image contrast change is the multiplication by a constant with each pixel value, the same as with the gradients, so it will be canceled by

PAGE 26

26 vector normalization A brightness change has no effect on the gradient values which are co mputed from the pixel differences. Therefore, the descriptor is invariant to affine changes in illumination. But there are also non linear changes of illumination, which causes large change in magnitudes for some gradients but are less likely to affect the gradient orientations. By setting the threshold value in the unite feature vector, the influence of large gradient magnitudes can be reduced Implement ation T here are four main steps of fin ding the SIFT features of the image. the feature structure is def ined to have the following parameters: feature x coordinate, feature y coordinate, oxford type affine region parameter, scale of the Lowe style feature, orientation of the Lowe style feature, descriptor length, descriptor, feature type, and all purpose fea ture class. T he result is shown in Figure 3 5. The length of the arrow shows the gradient magnitude of the feature point and the orientation of the arrow is the orientation of the descriptor in the histogram As shown in Figure 3 5, a 512 384 image is abl e to produce 305 SIFT features These large numbers of keypoints extracted from the sample image are not only useful for the correct match for a keypoint from a large database of other image keypoints due to their distinctiveness, but also enable s the robu stness in extracting small objects among cluster s In the following section the use of the SIFT features for matching will be discussed Features Matching Through features finding, the set of the SIFT features is able to be extracted from a sample image Given two images that are used for image stitching, two sets of the SIFT features can be detected The next step is to find the matching keypoints from the first set of SIFT features to the other set.

PAGE 27

27 neighbor in the database of keypoints from a training image, the best candidate match for the keypoint can be found The nearest neighbor is defined as the keypoint with the minimum Euclidean distance for the invariant descriptor vector However there may be some incorre ct matches in the database, so the features that are not well matched in the database should be discarded Since some descriptors are more discriminative than others, it is not well performed to set the threshold on the distance to the closet neighbor. An effective way is to compare the distance of the closest neighbor to that of the second closest neighbor. The reason is that the correct matches need to have the closest neighbor significantly closer than the closest incorrect match to achieve reliable matc hing. For false matches, there will possibly be a number of other matches within similar distances due to the high dimensionality of the feature space. So the matching process is basically using the k d tree to identify the nearest neighbors of keypoints. But the k d tree provides no speedup over the search for high dimensional spaces as the keypoint descriptor has a high dimensional feature vector. So a modified k d tree searching method called Best Bin First (BBF) search is used which was first examined by Arya and Mount (1993) The details about BBF will be discussed in the next section. After indexing the set of the SIFT feature s for the second image using a BBF search, each SIFT feature from the first image is matched to the k d tree index of the secon d image features. Nearest Neighbor Indexing Before in troducing the BBF search algorith m, first the standard k d tree is reviewed A k d tree is a data structure for storing a finite set of points from a k dimensional space. It was examined in detail by J. Bentley (1980) It is a binary tree in

PAGE 28

28 which every node is a k dimensional point. The domain vector or the range vector component is the index for the node. Each non leaf node is implicitly generating a splitting hyperplane that divides the space into two parts called subspace s All the points in the left subspace are represented by the left subtree, and the points in the right subspace by the right subtree. The splitting hyperplane is a plan which passes through the domain vector and which is perpendicula r to the left of the domain vector if and only if its i th component is less than the i th component of the domain vector. So for example, the node will be on the left subtree. An example is shown in Figure 3 6. Beginning with a complete set of N points, the data space is split on the dimension i in which the data exhibits the greatest variance. A cut is made at the median value m of the data in that dimension, so that an equal number of points fall to one side or the other. An internal node is created to store i and m and the process iterates with both halves of the data. To look up the Neural Network ( NN ) to a query point q a backtracking branch and bound search is used. First the tree is traversed to find the bin containing the query point. In the backtracking stage, whole branches of the tree can be pruned if the region of space they represent is further from the query point than the distance from q to the clos est neighbor yet seen. However, the backtracking is still inefficient because the order of examining the leaf nodes is according to the tree structure, which depends only A better idea is to look in bins in order of increasing distance from the query point. The distance to a bin is defined to be the minimum distance between q and any point on

PAGE 29

29 the bin boundary. It can be implemented using a priority queue. During the lookup, when a decision is made at an internal node to branch in one direction, an entry is added to the priority queue to hold information about the option not taken. It includes the current tree position and the distance of the query point from the node. After a leaf node has been examined, the top entry in the priority queue is removed and used to continue the search at the branch containing the next closest bin. The Best Bin First (BBF) search provides a great improvement in the NN search for moderate dimensio nality, making indexing in these regimes practical (Jeffrey S. Beis and David G. Lowe 2004). Implement ation In order to build a k d tree database from keypoints in an array, a k d tree node is initialized with a set of features. Define d k d node structure s consist of the following parameters: partition key index, partition key value, leaf, features, number of features, k d node left and k d node right. The Partition key index is that along which descriptors have most variance. The Partition key value is the median of the descriptor values at the partition key index. To expand the k d tree, first the descriptor index at which the value with which to partition a k is determined Then partition the features at this specified k d tree nod e to creat e its two children. When all records fall on same side of the partitio n, a leaf is found. When exploring a k d tree from a given node to a leaf, branching decisions are made at each node based on the descriptor of a given feature. Each node exami ned but not explored is put into a priority queue to be explored later, descriptor.

PAGE 30

30 Once the k d tree is built, the image feature approximate k nearest neighbors are found i n a k d tree using the BBF search. It returns k which is the number of neighbors f ound and stored in nbrs n brs is a pointer to an array in which to store pointers to neighbo rs in order to increase the descriptor distance. If k equals to 2, the squ ared Euclidian distance between the two feature descriptors is calculated When the first distance is less than two times threshold on squared ratio, a match is found. A l ine is drawn between the matched keypoints in each image and then nbrs is released An example of the implementation is shown in Figure 3 7. Homog raphy Transformation Homography Any two images of the same planar surface in space are related by a homography. A homography is an invertible transformation from the real projective plane to th e projective plan e that maps two straight line s together It describes what happens to the perceived positions of observed objects when the point of view of the observer changes From the previous section, the feature matches are extracted from adjacent im ages. These feature matches are used to get the homography of two images. Assuming that the camera rotates about its optical center, the group of transformations which the images undergo is a special group of homographies. E ach camera rotation is paramete rized by the rotation vector and the focal length f The homography can be expressed as ( 3 17) w here are the homogeneous image positions (3 dimentional) and i is the number of the captured images. The camera model is defined by

PAGE 31

31 ( 3 18) Using the exponential representation for rotations, the following equation is obtained: ( 3 19) Ideally the image features would be invariant under the group of transformation. But for small changes in image position ( 3 20) o r ( 3 21) w he re ( 3 22) is a n affine transformation obtained by lineari z ing the homography about It implies that each image patch undergoes an affine transformation, a nd justifies the use of the SIFT features which are only partially invariant to affine changes. The homogeneous matrix can be calculate d using two methods. The f irst method is when t he camera calibration and rotation matrix are known The s econd method is when one know s the correspondences between the two images. Since one can find the correspondences which are the SIFT feature matches between two images, the second method is used to find the homogeneous matrix.

PAGE 32

32 Random Sample Consensus As discussed in the previous section, the homography transformation matrix is a 3 3 homogeneous matrix with eight unknown variables , (each of which has three components) and and In order to get the transf ormation matrix, at least four pairs of correspondences should be given Since many SIFT features are found, RANSAC ( Fischler and Bolles in 1981) is used in this parameter estimation problem. RANSAC is an iterative method to estimate parameters from a set of observed data which contains outliers Outliers are the data that do not fit in the model Rather than using as much of the data as possible to obtain an initial solution and then attempting to eliminate the invalid da ta points, RANSAC uses as small of an initial data set as feasible and enlarges this set with consistent data when possible It produces a reasonable result only with a certain probability, w ith this probability increasing as more iterations are allowed. R ANSAC has two basic assumptions. Firstly, it assumes the data consists of inliers and outliers The outliers can be subject to noise. It also assumes that, given a set of inliers a procedure can estimate the parameters of a model that optimally explains o r fits the data. The input of RANSAC is a s et of observed data which is the c orrespondences between the two images a parameterized model which is the transformation matrix and some confidence parameters. RANSAC achieves its goal by iteratively selecting a random subset of the input observed data which are hypothetical inliers Afterwards, the hypothesis is tested as shown in T able 3 1:

PAGE 33

33 Table 3 1. Procedure of RANSAC A lgorith m Step Action 1 A model is fitted to the hypothesis inliers 2 All other data a re tested against the fitted model (if a point fits well to the estimated model, considered as a hypothetical inlier) 3 The estimated model is reasonably good if sufficiently many points have been classified as hypothetical inliers 4 The model is reestim ated from all hypothetical inliers 5 The model is evaluated by estimating the error of the inliers relative to the model Let parameter k be the number of iterations, p be the pro bability that the RANSAC algorith m in some iterations selects only inliers from the input data set from n points, w be the probability that an inlier is chosen each time a single point is selected. So that: w = number of inliers/ number of points in the data is the probability t hat a bad model will be estimated from the point set. ( 3 23) ( 3 24) T hese parameters assume that the n data points are select ed independently. However this does not happen when points are selected. Therefore, k should be the upper limit when the points are selected without replacement. Implement ation From the previous step all the correspondences between the two images are obtained Using RANSAC one can get the homography transformation matrix which can be implemented in OpenCV with the function cvFindHomography. Once the transformation matrix is known the image can be t ransformed to the other image perspective. An example is shown in Figure 3 8. Here the transformation matrix is produced from the correspondences transformed from the second image to the first

PAGE 34

34 one Then by warping the second image using the cvWarpPerspecti ve function, the second image is transformed to the perspective of the first image. Next, the first image is copied to the warped image of the second one. T he final stitched image is produced and shown in Figure 3 10 Compared with Figure 3 9 which uses th e SURF features, it shows that the SIFT features perform better than the SURF.

PAGE 35

35 Figure 3 1. Building image pyramid and derivation of DOG Figure 3 2 Detection of local minimum and maximum by comparing with the neighbors

PAGE 36

36 Figure 3 3 G radient orientation histogram Figure 3 4 Producing the keypoint descriptor Photo courtesy of David Lowe 0 20 40 60 80 100 120 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 y percentage x number of bins

PAGE 37

37 Figure 3 5 Implementation of finding the SIFT features Photo taken by Qiang Niu Figure 3 6 K d tree decomposition

PAGE 38

38 Figure 3 7 Features matching of two images using k d tree Photo taken by Qiang Niu

PAGE 39

39 A B Figure 3 8 Homography transformation of image A) Source image. B) Image after homography transform Photo taken by Qiang Niu

PAGE 40

40 A B C Figure 3 9 Example of image st itching using SURF A) First image to be stitched. B) Second image to be stitched. C) Stitched image using SURF Photo taken by Qiang Niu

PAGE 41

41 A B C Figure 3 10 Implementation of image stitching using the SIFT A) First image to be stitched. B) Second image to be stitched. C) Stitched image using SIFT Photo taken by Qiang Niu

PAGE 42

42 CHAPTER 4 PANORAMA STITCHING Background From the previous chapter, two images are able to be stitched together using the SIFT features. One use of that is to form a panorama which is a wide angle view of a scene. It can be used on an unmanned vehicle when the vehicle is moving around an unknown scene and taking pictures through the camera mounted on it. Instead of storing all the images taken from the camera, a panorama image is stored after stitching. A p anorama image is a high resolution image, so that more information can be extracted from less stored data. To produce a panorama stitch ed image, the first step is to get a series of images from the camera mounted on the vehicle. The c amera that was used for obtaining the images is an mvBlueFOX camera A CameraDevice class is defined to hold information about the camera. The basic functions are defined in this class by the interface function called mvIMPACT _acquire such as GetSerialID GetImage and so. After the camera configuration, images are obtained which means quit getting images The images are shown and saved for further processing and stitchi ng. However, once two images are stitched together, an artificial edge between two images appears It is because of the colors and luminance levels of the two images are different sometimes. In order to construct a good panoramic image, one need s to red uce the color differences between the source images and smooth the color transition for the image sequence before stitching the image together. In order to reduce the color

PAGE 43

43 differences the overlap area between two images need s to be found first which is d iscussed in the next section. Find ing Overlap Area It is a common task to find out the overlap area of coverage between two images a nd there are several methods available to find it of varying time complexities. A method based on the Monte Carlo method in 1946 was used This method takes less time than compared to the image matching methods through correlations when complete images are given. The method is discussed below. If A and B are two 4 gons then three cases arise between A and B. First is that A an d B intersect on two different edges with one point of B in A second is that A and B intersect on the same edge with two points of B in A third is that A and B intersect on the same edge with three points of B in A The three cases are illustrated in Fig ure 4 1. When image B is entire within image A, the overlapping area is set to be image A The first two cases are ruled out for finding the common points of intersection between them. In the last case, the number of intersection points may vary from one t o eight depending on the orientation of both 4 gons. The common points between two polygons are the intersection of edges of one polygon with edges of the other. In this method, the eight points of corner coordina tes are known. L et ( ), i=0, 1, 2, 3 be the corner coordinates of A, and ( ), i=0, 1, 2, 3 be the corner coordinates of B ordered along a clock wise direction as shown in Figure 4 2. Let L1: be a line of ( ) a nd ( ), where , and L2: : be a line of ( ) and ( ), where

PAGE 44

44 The common point of intersection between L1 and L2 if they intersect (when ) is given by ( equals to ( ). The parameters and are used to decide which solution is an intersecting point inside of both polygons. Any point on t he line segment joining ( ) and ( ) in parametric form is given by ( ( ( ), similarly any point on the line of ( ) and ( ) is ( ( ( ), where 0 1. At the point of intersection it follows ( ( and ( ( Solving for and gives : ( 4 1 ) ( 4 2 ) When both and are between [0, 1], the common solution is ( ( ( ). The algorith m of finding the point s of intersection is as follows, the overlap area of A and B are shown in Figure 4 2. 1. Read ( ) and ( ) for i = 0, 1, 2, 3. 2. For i = 0:3 (i loop) 3. Calculate and of line segment 4. For j = 0:3 (j loop) 5. Calculate and of line segment 6. If go to step 4 7. Calculate , 8. If 0 1, then ( is the point of intersection 9. End j loop, end i loop, and stop. An example is implemented for finding the overlap area between image1 and image2 shown in Figure 4 3 A and B Given two images, the second image needs to be transformed to the plane of the first image thr ough the homography transformation

PAGE 45

45 matrix discussed previously. Then the algorith m is used to find the overlap area. Figure 4 3, C is the overlap are a of the first image, and D is the overlap are a of the second one. Color a nd Luminance Compensation As stat ed previously, different view angles of the camera and changes in illumination in the scene will cause diff erent exposure and colors in consecutive images. So it is necessary to perform color and luminance compensation for the source image to reduce the ar tifacts. The gamma correction for the luminance component of the source image is performed to avoid pixel saturation problems and obtain good transitions between source images. And the linear correction is performed for the color components of the source i mage, so that a good quality of color correction is obtained Gamma correction is a nonlinear operation used to code and decode luminance in images or videos. It is simply defined by the power law expression as: ( 4 3 ) When is larger than 1, it is called encoding gamma, which makes the output image darker When is smaller than 1, it is called decoding gamma, which makes the output image brighter The gamma correction is use d to match colors of overlapping images areas in the RGB color space. From the previous section of finding the overlap area, the overlapping areas are obtained and the notation s are illustrated in Figure 4 4. Let (p) be the color value of pixel p in image and (p) be the color value of corresponding pixel p in image the following result is expected to be achieved through gamma correction:

PAGE 46

46 ( 4 4 ) w here and are gamma correction coefficients for image i 1and i is the number of pixels in the overlap area. To simplify t he computation and make the algorith m robus t to spatial alignment errors and speed up the algorith m, the means over the overlapping area is calculated then the gamma correction is applied so the equation 4 4 becomes: ( 4 5 ) Applying a logarithm to both sides of the equation 4 5 yields ( 4 6 ) E quation 4 6 becomes ( 4 7 ) In order to get the gamma correction coefficients ( i = 1, 2 an error function is defined over all source images in the image sequence. The error function is the sum of the squared differences of gamma corrected colors between all corresponding pixels in the overlapping area. It is calculated as follow s : ( 4 8 ) Sinc e is added to keep the gamma correction coefficients close to unity since it makes no change when Also the normalization between color and the gamma correction coefficients errors should be considered So the error function becomes: ( 4 9 )

PAGE 47

47 To minimize this function to get (i=1, the derivatives of the error function are set to zero, which gives : ( 4 10 ) ( 4 11 ) ( 4 12 ) 1) By setting a tri diagonal linear equations are obtained and can be solved by LU decompo sition. Once the optimal gamma correction coefficients have been obtained color c orrection is performed for all source images in sequence. This method enables the compensation to have no color saturation and cumulative error problems. So before stitching the images together, the work flow of color correction is illustrated in Figure 4 5. An example is implemented for the color compensation of the image shown in Figure 4 6. C is the stitched image without any color compensation; D is the result after proce ssing color compensation.

PAGE 48

48 Figure 4 1. Three cases that image A and B are intersected Figure 4 2 The overlap area with points of intersection

PAGE 49

49 A B C D Figure 4 3 Finding overlap area of two images A) Image1. B) Image2. C) Overlap area of image1. D) Overlap area of image2. Photo taken by Qiang Niu

PAGE 50

50 Figure 4 4 Overlapping source images notations Photo courtesy of D. Sudheer Reddy Start Load the images Figure 4 5. Work flow of the color correction Extract overlapping area between the source image Compute color means in the overlapping area Construct an error function and minimize it to color correction coefficients for each image Perform gamma correction and obt ain image End

PAGE 51

51 A B C Figure 4 6. Implementation of Gamma correction. A) First image to be stitched. B) Second image to be stitched. C ) T he image stitching without gamma correction. D ) T he result after g amma correction. Photo taken by Qiang Niu

PAGE 52

52 D Figure 4 6. Continued

PAGE 53

53 CHAPTER 5 OBJECT RECOGNITION Background Object recognition is widely used in the manufacturing industry such as for machine vision for registration, manipulation an d so on But current commercial object recognition system s are mainly based on template matching. Template matching is effective for some types of engineering environment s but when it comes to changes in illumination, rotation or scale, it becomes infeasi ble. More and more progress has been made in developing real world object recognition systems based on the use of invariant features. As the local features are complex they are distinctive enough to determine the likely matches from a large database feat ures. Also they are insensitive to clutter and occlusion. The difficulty of the object recognition problem is strongly due to the lack of success in finding such features. However f rom the previous discussion, the SIFT features are invariant to image scali ng, translation, rotation, and partially invariant to illumination changes and affine transformations So the SIFT features can be used efficiently as local image features for object recognition. The process of object recognition using the SIFT is describ ed as follows: First the SIFT features are detected from a local training image through filtering in scale space as discussed previously. Secondly, image keys are created which represent the blurred image gradients in different orientation planes and at di fferent scales, as the input to a nearest neighbor indexing method to identify the matches. Thirdly, a verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters.

PAGE 54

54 Object Recognition Using SIFT F eatures From the previous discussion, the images are represented as a set of SIFT features. The approach that is used for object recognition is to match the features to individual training images. Fir st the SIFT features extracted from the training image a re stored as local data Then images are captured from the camera and present ed as sets of the SIFT features T he local data is matched to the sets of the SIFT features produced con tinuously O nce the matches are produced, the object is found. An example o f finding a stop sign is shown in Figure 5 1. There are some requirements on the training image. It is better taken on a uniform background, thus only few SIFT features will be produced from the uniform background. In this way, incorporation of spurious f eatures will be minimized From experiments (David Lowe 2004), the extra computation and memory usage. the training image w ithout a uniform background as long as the majority of the image is the single object But there may be situations in which false matches are produced or a different number of matches is found from different images. So finding f inal matches should be achieved through verification The geometric v erification is performed by using an affine transform ation to verify the fit between local data and image features Let ting [x,y] T be a local data, [u,v] T be an image point, the affine transformation can be expressed as: ( 5 1 )

PAGE 55

55 Where is the translation parameter and ar e the affine parameters including rotation, scale and stretch. In order to solve the parameters, the above equation 5 1 is rewritten as: ( 5 2 ) There are six unknown paramet ers in equation 5 2 so at least three sets of matches are needed to solve the equation. The linear system can be expressed as: ( 5 3 ) Parameter x can be determined by: ( 5 4 ) In this way, the sum of the squares of the distances from local data to image points will be minimized. Outliers can now be removed by checking the verification with the affine parameter solution. In th is test, each match is defined to agree within 15 degrees orientation, change s in scale, and 0.2 times maximum in location. If fewer than 3 points remain after the test, the match will be rejected. If any outliers are discarded, the le a st square solut ion is resolved within the remaining points. An example processed after v erification is shown in Figure 5 2. When there is a case of more than one image having the same object, the images should be able to be distinguished from the extent of recognizing t he object. So another parameter probability is introduced. From the previous discussion, the number of the SIFT features is obtained from the training image. T he number of the feature matches

PAGE 56

56 between the object features from the training image and the im age features from the captured image can also be obtained. T he number of matches divided by the number of the SIFT features is used as the matching probability to measure the extent of object reco gnition, a nd the probability parameter is returned with the images. As object recognition is an important and useful area of computer vision, there are varieties of approaches in feature based recognition, such as pose clustering using Hough Transform (Paul Hough in 1962). The SIFT feature s approach is only one of the feasible methods So this chapter just briefly described another contribution of the SIFT feature s

PAGE 57

57 Figure 5 1 Attempt testing of object recognition. Training image is shown on the top; I mage captured from camera is shown on the bottom Photo taken by Qiang Niu

PAGE 58

58 A B Figure 5 2 Comparison of object recognition using the SIFT with verification. A) T he recognition before verification. B ) T he one after verification. Photo courtesy of David Lowe

PAGE 59

59 CHAPTER 6 IMPLEMENTATIONS AND CONCLUSIONS Results The camera that was used for capturing image s is the MvBlueFOX camera which has the parameters are shown in Figure 6 1. Chapter 3 provides a technique to find the SIFT features W ith the invariant features further process ing f or image stitching c an be achieved Chapter 4 further discussed color correction for panorama stitching, which further improves the performance of image stitching in Chapter 3. The panorama stitching after color correction is shown in Figure 6 2. Chapter 5 briefly discussed a nother SIFT feature application which is feature based object recognition It is faster and easier, it also requires less memory storage and computational time. Figure 6 3 shows how an object from a training image is recognized from sets of images captured from the camera. Conclusions Scale Invariant Feature Transform is the essential technique used in this thesis. It is an important invariant local feature for varieties of use. The idea of building an image pyramid by convolution with Gaussian function is essential for detection. In the image stitching area, large number of the SIFT features can be detected effectively. From the implementation, it only takes few seconds to extract the SIFT features. Feature descriptors are assigned for image matching, also for object recognition as it is able to achieve the invariance. Stitching using the SIFT features does have better alignment than SURF as tested but it is mostly only better aligned around the region of features that are detected. By just copying the un transformed image onto the transformed image this

PAGE 60

60 type of stitching ca nnot guarantee full alignment on the image edges as shown in Figure 6 2. For further improvement, an optimal seam method can be use to find the cutting curve in the overlapping area. Th is method requires one to minimize the difference between the pixels in overlapping area from both images. In addition, there are various algorith m s to find the cutting curve such as Min Cut (Kwatra et al in 2003) With this method finding seamless edges may provides better alignment for image stitching. There i s a constraint on the direction that the camera may move for capturing images. Since the or der that images transform to the adjacent one s affects the order of stitching there are different algorith m s for the different directions that the camera moves. C lockwise and counter clockwise directions were defined here to capture the images. For future development, there should be a way to combine them together. O bject recognition is another application of the SIFT features. Th e recognition from feature matches was briefly tested in this thesis. It works when the vehicle is driving around trying to find an object like a stop sign from the picture it captures. This method requires high feature based training image that contains mostly only the object. In this test, it was decided to use a black background image as the training image. There are also ways to find shape s that contain all the matched features from the training image within the captured image such as the Hough Transform, which would be developed in further object recognition problems. Also building a probability model helps the object recognition perform better. In addition, t here are also various applications of the SIFT features such as 3D scene m odeling, video tracking and so. It is a powerful invariant type of feature so it has the potential to make great contributions to many computer vision researches.

PAGE 61

61 Figure 6 1 T he parameters of the c amera MvBlueFOX

PAGE 62

62 A B C Figure 6 2. Panorama stitching. A) First image to be stitched. B) Second image to be stitched. C) Third image to be stitched. D) Window of the process. E) Stitched imag e. Photo taken by Qiang Niu

PAGE 63

63 D E Figure 6 2 Continued

PAGE 64

64 A B Figure 6 3. O bject recognition using the SIFT features. A) Process of finding object in sets of images, returning the number of matches. B) Recognition between training image and the first captured image. C) Recognition between training image and the second captured im age. D) Recognition between training image and the third captured imag e. Photo taken by Qiang Niu

PAGE 65

65 C D Figure 6 3 Continued

PAGE 66

66 LIST OF REFERENCES Ayers, B. and Boutell, M. 2007. Home Interior classification using SIFT Keypoint Histograms. Computer Visi on and Pattern Recognition pp. 1 6. Beis, J. S. and Lowe, D. G. 1997. Shape Indexing Using Approximate Nearest Neighbor Sea rch in High Dimensional Spaces. Computer Vision and Pattern Recognition, Vancouver B.C., Canada, pp. 1000 1006. Brown, M. and Lowe, D. G. Automatic Panoramic Image Stit ching Using Invariant Features. International Journal of Computer Vision Vancouver, B.C., Canada, vol.74, pp. 59 73. Fis chler, M. A., and Bolles, R. C. 1981. Random Sample Consensus: A paradigm for Model Fitting with A pplications to Image Anal ysis and Automated Cartography. SRI International, ACM vol. 24, pp. 381 395 Li, Y., W ang Y., Huang, W. and Zhang, Z. 2008. Autom atic Image Stitching Using SIFT. International Conference on Language and Image Processing ICALIP, pp. 568 571. Lowe, D. G. 2004. Distinctive Image Features from Scale I nvariant Keypoints. I n the International Journal of Computer Vision Vancouver, B. C., Canada, vol. 60, pp. 91 110. Lowe, D. G. 2001. Local Feature View Clustering for 3D Object Recogn ition Conference on Computer Vision and Pattern Recognition Vancouver, Canada, vol.1, pp. 682 688. Lowe, D. G. 1999. Object Recognition from Local Scale Invariant Features International Conference on Computer Vision vol.2, pp. 1150. Reddy, D. S. 2010. Monte Carlo Method of Finding Overlap Area Between Two Image Scenes when On ly Corner Coordinates are Known. J Indian Society of Remote Sensing vol. 38, pp. 664 669. Sa deghi, M. A., Hejrati, S. M. M. and Gheissari, N. 2008. Poisson Local Color Correction for Image Stitching. Computer Vision Group pp. 275 282. Stegmann, M. B. 2001. Image Warping. Informatics and Mathematical Modeling Sun, X., Pitsi anis, N. P. and Bientinesi, P. Fast Computation of Local Correction Coefficients Advanced Signal Processing Algorithms vol.7074, pp. 1 8. Szeliski, R. 2005. Image Alignment and Stitching. Handbook of Mathematical Models in Computer Vision pp. 273 292.

PAGE 67

67 Wu, F. and Fang X.2007. An Improved RANSAC homography Algorithm for Feature Based Image Mosaic. Conference on Signal Processing Computer Geometry and Artificial Vision Athens, Greece, pp. 202 206. Xiong, Y. and Pulli, K. 2010. Color and Luminace Compensation fo r Mobile Panorama Construction. International Conference on ACM Multimedia Firenze, Italy, pp. 1547 1 550. Xiong, Y. and Pulli, K. 2010. Color Matching of Image Sequences with Combined Gamma and Linear Corre ctions. International Conference on ACM Multimedia Firenze, Italy, pp. 261 270. Xiong, Y. and Pulli, K. 2009. Color Correcti on for Mobile Panorama Im aging. Proceedings of the First International Conference on the Internet Multimedia Computing and Service New York, 2009, pp. 219 226 Xu, Y. Hu, K., Tian, Y. and Peng, F. 2008. Classification of Hyperspectral Imagery us ing SIFT for spectral matching. C ongress on Image and Signal Processing pp. 704 708. Yianilos, P. N. Data Structures and Algorithms for Nearest Neighbor Search in General Metric Spaces NEC Research Institute Princeton, New Jersey, pp. 311 321.

PAGE 68

68 BIOGRAPHICAL SKETCH Qiang Niu graduated from Nanjing University of Aeronautics and Astronautics in China with a Bachelor of Science degree in vehicle e ngineering. After graduating, Qiang Niu attended the University of Florida for her master e ngineering. At the sam e time, she pursued a minor in electrical and c omputer e ngineering at the University of Florida. Qiang Niu has worked on some projects involving vehicle dynamics and mechanical practice about automotives. She started doing research on computer v ision, image processing and machine intelligence during her master Her research goal is develop computer vision system for autonomous vehicles and she will continue her career as an engineer in robotics field.