<%BANNER%>

An Automatic Pose Recognition System for Knee Implants

Permanent Link: http://ufdc.ufl.edu/UFE0024447/00001

Material Information

Title: An Automatic Pose Recognition System for Knee Implants
Physical Description: 1 online resource (36 p.)
Language: english
Creator: Vardhan, Avantika
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: biomechanics, fluoroscopy, imaging, registration, xray
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The problem of the registration of two dimensional fluoroscopic and X ray images with respect to a three dimensional joint implant model has been solved using a variety of techniques. In the case of knee joints, this problem is very important because accurate pose determination results in better design of implants and useful data for clinical studies. Current joint implant registration techniques require an initial model placement by a human operator. This work attempts to find an initial pose with minimal to no user interaction and hence make the pose recognition system fully automated. This study uses a two level approach to the pose identification problem. An object recognition system has been designed which can find the Regions of Interest (ROI). The Regions of Interest in this case are the femur and tibia boundaries, which are found using a cascaded implementation of image processing and image segmentation blocks. The resulting candidate regions are then passed through an image recognition system and the final femur and tibia regions are identified. AdaBoost (adaptive boosting) performed with Haar rectangular features and a neural network classifier used on SIFT (Scale Invariant Feature Transform) generated features have been implemented for object recognition. In the second stage of pose recognition a SIFT based classifier is used. The position and angular orientations of the image are found by matching corresponding features of the 2D image to a library of 3D models. The 3D models are generated by a computer which uses limiting biomechanical constraints. The nearest neighbor approach is used for feature matching between the image sets. It is shown that using this method the model achieved a pose matching accuracy within the acceptable range for a correct optimized result and is relatively invariant to model type, illumination and noise factors.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Avantika Vardhan.
Thesis: Thesis (M.S.)--University of Florida, 2009.
Local: Adviser: Slatton, Kenneth C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024447:00001

Permanent Link: http://ufdc.ufl.edu/UFE0024447/00001

Material Information

Title: An Automatic Pose Recognition System for Knee Implants
Physical Description: 1 online resource (36 p.)
Language: english
Creator: Vardhan, Avantika
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: biomechanics, fluoroscopy, imaging, registration, xray
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The problem of the registration of two dimensional fluoroscopic and X ray images with respect to a three dimensional joint implant model has been solved using a variety of techniques. In the case of knee joints, this problem is very important because accurate pose determination results in better design of implants and useful data for clinical studies. Current joint implant registration techniques require an initial model placement by a human operator. This work attempts to find an initial pose with minimal to no user interaction and hence make the pose recognition system fully automated. This study uses a two level approach to the pose identification problem. An object recognition system has been designed which can find the Regions of Interest (ROI). The Regions of Interest in this case are the femur and tibia boundaries, which are found using a cascaded implementation of image processing and image segmentation blocks. The resulting candidate regions are then passed through an image recognition system and the final femur and tibia regions are identified. AdaBoost (adaptive boosting) performed with Haar rectangular features and a neural network classifier used on SIFT (Scale Invariant Feature Transform) generated features have been implemented for object recognition. In the second stage of pose recognition a SIFT based classifier is used. The position and angular orientations of the image are found by matching corresponding features of the 2D image to a library of 3D models. The 3D models are generated by a computer which uses limiting biomechanical constraints. The nearest neighbor approach is used for feature matching between the image sets. It is shown that using this method the model achieved a pose matching accuracy within the acceptable range for a correct optimized result and is relatively invariant to model type, illumination and noise factors.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Avantika Vardhan.
Thesis: Thesis (M.S.)--University of Florida, 2009.
Local: Adviser: Slatton, Kenneth C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024447:00001


This item has the following downloads:


Full Text

PAGE 1

1 AN AUTOMATIC POSE RECOGNITION SYSTEM FOR KNEE IMPLANTS By AVANTIKA VARDHAN A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCI ENCE UNIVERSITY OF FLORIDA 2009

PAGE 2

2 2009 Avantika Vardhan

PAGE 3

3 To Saraswathi, the source of all learning

PAGE 4

4 ACKNOWLEDGMENTS I would like to express my sincere thanks to my advisor, Dr. Scott Banks for his unfa iling enthusiasm and support during my thesis. This research work would not have been possible without his guidance and encouragement. I am very grateful to Dr. Kenneth Slatton and Dr. Amauri Arroyo for being a part of my committee. I thank my lab members, particularly Mu Shang John Yamakoski and Dr. Mikashima, who have helped me in several ways. I want to acknowledge my friends Shivakeshavan Giridharan, Venkat Jayaraman, Anirudh Sudarshan and Adithya Rajan for helping me obtain computing resources. I am grateful to all the members of my extended family, who have always believed in me and encouraged me in academic pursuits. Finally, I am de eply thankful to my parents for supporting my endeavors in innumerable ways and being a gre at source of strength durin g every stage of my thesis

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................................................................................................... 4 LIST OF TABLES ................................................................................................................................ 6 LIST OF FIGURES .............................................................................................................................. 7 ABSTRACT .......................................................................................................................................... 8 CHAPTER 1 INTRODUCTION ....................................................................................................................... 10 Motivation .................................................................................................................................... 10 Related Work ............................................................................................................................... 12 2 METHODS .................................................................................................................................. 16 Overview ...................................................................................................................................... 16 Object Recognition ...................................................................................................................... 17 Pose Determination ..................................................................................................................... 24 3 RESULTS .................................................................................................................................... 29 Results of Object Recognition .................................................................................................... 29 Results of Pose Determination ................................................................................................... 29 Discussion .................................................................................................................................... 32 LIST OF REFERENCES ................................................................................................................... 34 BIOGRAPHICAL SKETCH ............................................................................................................. 36

PAGE 6

6 LIST OF TABLES Table page 3 1 Object recognition .................................................................................................................. 29 3 2 Femur Orientation Determination ......................................................................................... 31 3 3 Tibia Orientation D etermination ........................................................................................... 32

PAGE 7

7 LIST OF FIGURES Figure page 1 1 Joint track GUI: final optimized result of image registration .............................................. 12 2 1 Initial pose estimation block set ............................................................................................ 16 2 2 The algorithm flow of the object recognition code .............................................................. 17 2 3 An example of the adaptive thresholding technique ............................................................ 18 2 4 An example of the N -Cut segmentation appli ed on a subimage ........................................ 19 2 5 Edge and line rectangular Haar features ............................................................................... 20 2 6 Integral images and Haar features ......................................................................................... 21 2 7 Integral images ...................................................................................................................... 22 2 8 SIFT Feature matching process ............................................................................................. 25 2 9 Illustration of orientation values of SIFT key points ........................................................... 26 3 1 Tibia pose ................................................................................................................................ 30 3 2 Femur pose .............................................................................................................................. 31

PAGE 8

8 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Science AN AUTOMATIC POSE RECOGNITION SYSTEM FOR KNEE IMPLANTS By Avantika Vardhan M ay 2009 Chair: Kenneth Slatton Major: Electrical and Computer Engineering The problem of t he registration of two dimensional fluoroscopic and X ray images with respec t to a three dimensional joint implant model has been solved using a variety of techniques. In the case of knee joints this problem is very important because accurate pose determi nation results in better design of implants and useful data for clinical studies. Current joint implant registration techniques require an initial model placement by a human operator. This work attempts to find an initial pose with minimal to no user inter action and hence make the pose recognition system fully automated. This study uses a two level approach to the pose identification problem. An object recognition system has been designed which can find the Regions of Interest (ROI). The Regions of Interest in this case are the femur and tibia boundaries which are found using a cascaded imp lementation of image processing and i mage segmentation block s. The resulting candidate regions are then passed through an image recognition system and the final femur and tibia regions a re identified AdaBoost (adaptive boosting) performed with Haar rectangular features and a neural network classifier used on SIFT (Scale Invariant Feature Transform) generated features have been implemented for object recognition

PAGE 9

9 In the se cond stage of pose recognition a SIFT based classifier is used. The position and angular orientations of the image a re found by matching corresponding features of the 2D image to a library of 3D models. The 3D models a re generated by a computer which uses limiting biomechanical constraints. The nearest neighbor approach i s used for feature matching between the image sets. It is shown that using this method the model achieved a pose matching accuracy within the acceptable range for a correct optimized resul t and is relatively invariant to model type, illumination and noise factors

PAGE 10

10 CHAPTER 1 INTRODUCTION Motivation Total Knee Arthroplasty, commonl y known as knee replacement ha s been frequently used to treat patients with sever e knee damage. In TKA, a man made surface of metal and plastic is used to replace damaged bone and cartilage. In the period between 1979 and 2002, the number of TKA procedures performed almost increased by an eightfold factor [1]. With this r apid increase, the design of better joint implants is of great importance. The determination of the exact pose of knee implants during load bearing activities and motion gives us valuable information which could help in improving current design technology and in overcoming problems such as the pre mature failure of implants. In addition it could assist in clinical studies analysis of effectiveness of new treatment s development of better rehabilitation strategies, and the development of new surgical techniques [2 ],[3] In order to study the in vivo kinematics of joint function, the current techniques used for joint implant matching depend on a manually generated initial pose. In this case, the accuracy of the final result depends to a large extent on the users input pose. Our system, which would au tomate finding the pose of the model would hence eliminate errors caused by the observers and by differences b etween observers [4 ]. In addition, it would speed up the process and ensure repeatability of pose matching results. The approach we have taken to tackle the problem of pose recognition is two fold. Initially, object recognition is performed to find the femoral and tibial regions of interest. Prior to recognition, several ima ge processing techniques are applied to remove the high level of noise pres ent in fluoroscopic and x ray images. We have obtained the different candidate ROI s (Regions of I nterest) using a n ormalized cut vertex segmentation algorithm

PAGE 11

11 T hese candidate regions once passed through an image classifier are then classified as either the femur or tibia. The best matches for the femur and tibia are chosen for further processing. Two kinds of image recognition approache s ar e used. First, an AdaB oost (adaptive boosting) classifier i s used along with a large set of rectangular Haar Featu res, which compute the difference in pixel intensities ove r various regions of the image [5]. Secondly, SIFT (Scale Invariant Feature Transform) generated features a re used along with a three layer Perceptron model The SIFT key points a re then used to buil d a machine learning algorithm using a neural network classifier. Once the femoral and tibial regions are found, the pose determination algorithm is applied to match the exact pose along the 6 DOF (degrees of freedom). SIFT extracts the dominant key points using a DoG (D ifference of Gaussian) function by finding the local peaks and dominant feature orientations [6]. The use of SIFT ensure s that key points generated remain invariant to changes in scale, illumination and projective distortions. A large model image library is generated which consists of projections of the model provided by manufacturers The images in the library have been generated for various rotations along all the X, Y and Z axes. Once a set of SIFT key points are generated for the 2 dimens ional image and the library of models, matching of the key points i s then performed using a nearest neighbor search [6]. Coarse to fine library matching i s used, in which higher levels of pose accuracy are found with each correct image to model match. In a ddition to finding the right orienta tion, the position information i s found by performing a matching of the location of the origins of the key point descriptors. Once the translations and rotations are found, currently available methods for 2D to 3D joint image registration can be applied to find the accurate pose.

PAGE 12

12 Software for the final optimization steps are already available in an open source format known as Joint track. This implementation uses the Mahfouz metric to compute the cost function of an image match and then performs simulated annealing to obtain an optimized result [7]. Figure 1 1. Joint track GUI: final optimized result of image registration Related Work Various methods have been used previously to tackle the problem of 2D to 3D image re gistration for knee kinematics. These methods have been developed for the registration of bone images to CT (computed tomography) and DRR (Digitally Reconstructed Radiographs ). The second application of this technique is for the registration of an implant image with CAD models or drawings of knee implants provided by the manufacturers.

PAGE 13

13 Among the several approaches that exist to perform this registra tion, commonly used ones have been intensity based and contour based matching algorithms [4]. In the contour m atching approach, the edge or exterior surface of the object wa s compared with the silhouette of the model. The pixel points close to the contour we re assigned a weight which wa s inversely proportional to their distance to the contours. Con tour matching wa s then performed by taking the cross correlation between the edge images belonging to both the model and the image The intensity matching done consisted of taking a cross correlation between the silhouette and the radiograph and in essence finding the ave rage pixel intensity across the area of overlap between the two. Combinations of both contour and intensity matching were used to produce minima when exact matching of model and image was obtained [4]. Although this technique is relatively robust and gives a close optimization match, it requires user supervision at several levels and manual pose estimation. Another method for improved registration of bone images uses bony reference points located at certain anatomical landmarks of the tibia and femur for im age fitting [8]. In addition, an anatomical coordinate system was found for the femur and tibia based on landmarks such as the medial and lateral points of the bone. A set of 3D models were generated and then fitted with the images by applying deformations in such a way that the surface shape and reference points match between the two. By extending this technique to joint implants, t he TKA implants were also fitted with respect to the 3 D mechanical axis of the bones. The advantage of this technique is that the component pose was found in relation to the entire lower extremity of the bone and was not affected by changes in xray source position or exposure parameters. In template matching procedures, the implant model was matched with a library of generated model templates over a wide range of rotation and translation parameters. The use of

PAGE 14

14 Fourier descriptors found by the application of the DFT (Discrete Fourier Transform) operation was used to summarize contour information of a joint image during in vivo ki nematics [9]. These normalized Fourier descriptors provided a highly compressed form of the contour data which can be visualized as a number of periodic functions. However, the use of Fourier descriptors requires complete segmentation of the image and the presence of a completely closed contour. Therefore, in noisy images and occluded components, this method is much less efficient. Another template matching algorithm involved the use of differences in template areas as opposed to contours [10] Although thi s overcomes the limitations of generating closed contours, it nevertheless requires a complete segmentation of components. Apart from template based techniques, the alignment of the component with a projection from a camera using a distance metric as a cos t function which was optimized using a simple nonlinear minimization approach [11] The fitted object satisfied the tangent condition between the camera projection lines and the model surface if the Euclidean distance between each point of the projection r ays and each point of the model was zero. However, the use of distance maps is considerably slow and also requires highly intensive computations. The success of this technique also depends largely on the resolution of the distance map and on obtaining perf ect match between the geometry of the CAD models and the implants used. Since the highest level of inaccuracy is present in the depth position in the plane perpendicular to the image, new techniques have focused on improving the value of this coordinate. Two orthogonal images have been used in order to ensure that the depth accuracy from a single view can be enhanced by the in -plane translation in an orthogonal view [12]. Apart from the non-invasive techniques outlined above, a number of invasive methods h ave also been found to perform registration for TKA. For instance, bone markers or tantalum

PAGE 15

15 beads were used and point and intensity based registration was performed with these points as references [13]. Such methods involve painful surgical procedures and pose the risk of infec tion. The technique which we implement overcomes many problems faced by the above mentioned algorithms. A rough segmentation which yields the ROI is required, hence eliminating the need for a very accurate segmentation step. Since the SIFT features are scale invariant, a single model library can be used with reasonable accuracy over several image datasets hence reducing the computational time for generating a model library. The limitations which exist in the current technique include the need for very good noise removal and adaptive Thresholding steps. The results produced using this technique depends to a large extent on the effectiveness of these image pre processing steps. The primary purpose of our registration algorithm is to repl ace the manual intervention of the operator in finding the initial pose of the implant. Hence although it can yield a reasonable initial guess it is beyond the scope of this algorithm to gain a very accurate ly optimized final pose result.

PAGE 16

16 CHAPT ER 2 METHODS O verview Our method of finding the initial guess is composed of first finding an initial boundary for the femur and tibia and later performing pose recognition on these images. The bounded regions are passed to an object recognition algorithm which then classifies them into a femoral or tibial implant. This is done to ensure that cross -matching of SIFT key points between the femur and tibia doesnt take place, especially since they have some similar components. Figure 2 1. I nitial pose estim ation b lock set In addition, finding the exact region of interest eliminates the need for a highly accurate segmentation algorithm. Once roughly segmented, t he classified regions matched with a pose determination algorithm based on SIFT key points. The pos e is initially matched for all in plane and out of plane rotations and later registration is performed to ensure that position is also accurately matched. As a result, the initial estimate of pose is generated which will automate the optimization procedure involved in finding the implant pose.

PAGE 17

17 Object Recognition Algorithm flow: Object Recognition consists of extracting the femoral and tibial regions before the pose determinat ion algorithm is applied Therefore, the effectiveness of the pose determination al gorithm depends to a large extent on finding the correct boundaries of the femur and tibia. Since a rough segmentation is sufficient for the later stages it is merely important to find the approximate bounding box of the two implants. In cases where there is an overlap between the implants, a few seconds of manual intervention would ensure that a line is drawn dividing the image into the femur and tibia regions. Prior to applying the object recognition algorithm, the images used are subjected to several pr eprocessing steps. In particular, the challenges posed to object recognition such as illumination changes, occlusions and noise must be effectively removed. Figure 2 2 The algorithm flow of the object recognition code Image Preprocessing: Before applyi ng any object recognition technique, first the images have to be suitably processed to form the training and testing datasets. Initially a Hough transform with a curvature range is applied to detect the circular shape which frames

PAGE 18

18 fluoroscopic images. A si mple normalization procedure is performed to create a uniform range and remove variability in illumination. The adaptive t hresholding algorithm has then been applied to all images in order to remove noise and extract only the essential components. As oppos ed to setting a global threshold the image is divided into several windows and in each of these an adaptive threshold value is found [14], [15]. The use of an adaptive threshold is especially effective since the intensity values of the implants are not con stant or within a fixed range but vary in their pixel value s over different image regions. Once this Thresholding is performed, the surrounding bone regions and soft tissues are simultaneously removed and only the required component s are retained. Figure 2 3 An example of the adaptive t hresholding technique Normalized -cut vertex Segmentation : Once image preprocessing has been performed, a sub -image or ROI (region of interest) which contains only the relevant components which are necessary is extracted. On this sub image, N -cut segmentation is applied to find the approximate boundaries of the femur and tibia. In cases where a clear match is not found by the segmentation technique, a Hough transform or a bounding line can be used to divide the image into the femoral

PAGE 19

19 and tibial regions. The N -cut segmentation is a technique which treats the problem of segmentation as one similar to clustering. Each pixel in the image is considered a node and pixel pairs are considered edges which are assigned weights. The v alue of the edge weights correspond to the degree of similarity between the edges. In completely disconnected edges, this similarity measure has a value of zero. Cut (A, B) = B j A iwij, (2 1) Figure 2 4 An example of the N -Cut segmentation applied on a sub-image Each possible cut hence divides the image into two segments A and B (Equation 2 1) and results in a summation of weights of each edge between nodes in the two regions Therefore, an optimal partition minimizes the value of the Cut metric in Equation 2 1. Each partition is then checked for stability and recursive segmentation is performed if necessary [16]. The number of cuts and the number of segments in an image can be specified by the user. As shown in Figure 2 4, the femur and tibia regions segmented within the sub-image show a high degree of accuracy. The main advantages of using the N -cut segmentation method are that it is highly texture

PAGE 20

20 sensitive which is a useful property while dealing with dis tinguishing implant regions from bone, soft tissue and surrounding noise. A bounding box surrounding the femoral and tibial regions is now extracted and these regions are sent to a classifier. AdaBoost for object recognition: AdaBoost (Adaptive Boosting) is an algorithm which is popularly applied in the fields of face and object recognition. The underlying principle of AdaBoost is the process of combining several weak classifiers with different weights to result in a strong classifier [18] The features ex tracted from AdaBoost are usually in the order of hundreds of thousands which are then pas sed through a Boosting cascade which extracts the best features. Figure 2 5 Edge and line r ectangular Haar features The AdaBoost program in this case uses rectangular edge and line Haar features which can be computed at very high speeds. These features, which add and subtract combinations of rectangular regions, exploit the inherent symmetry present in the implant components. Haar features are computed over all possible varying window sizes and shapes to form the complete feature set. A method which is used to speed up computing times is the Integral Image technique. Every pixel of the integral image consists of the summation of all the previous pixels up to that point. x i y jj i y x I1 1) Im( ) ( (2 2)

PAGE 21

21 Therefore, all the Haar features which need to be computed can be easily generated using the in tegral image as illustrated in Figure 2 6. A B Figure 2 6 Integral images and Haar features A) Integral ima ge B) Haar feature generation F(x) = T t t th1 (2 3) Once features are generated, the AdaBoost classifier undergoes several cycles with each cycle corresponding to a weak classifier chosen for building the final strong classifier as shown by Equation 2 3 Although in the initial stage of boosting each sample is given a uniform weight, with each iteration of AdaBoost the misclassified samples assume greater weight before the subsequent cycle [19] The advantages of AdaBoost lie in extremely fast computation, a tendency to work even with highly skewed or biased datasets and robustness with noise. The weak classifier employed is a simple model of a one layer perceptron consisting of the parameters parity, error and threshold value. S ome other positive attributes of AdaBoost include the facts that the result converges to the log likelihood ratio, it can be generalized and the chances of over fitting are low. Its main drawbacks are that it requires a large training set and uses a subopt imal Greedy learning algorithm for convergence. AdaBoost has also been used recently in multi view face recognition implementations by using training images which have undergone several rotations [18].

PAGE 22

22 A B C D Figure 2 7 Integral images A) Image of Femur, B) Integral Image of Femur C) Image of Tibia and D) Integral Image of Tibia A diverse database covering all possibil ities is used as a training dataset in order to make AdaBoost rotation invariant and to accommodate for the changes in view point and model rotation. Since the rotations of the femur and tibia are not over an extremely large range detection is easily possi ble despite varying angles. AdaBoost cycle implementation: The following steps illustrate an implementation of AdaBoost. Given that,

PAGE 23

23 } 1 1 { 1 1, ); ( ),..., ( i i m my x y x y x (2 4) Where, ) ( ),..., (1 1 m my x y x in Equation 2 4 refers to images mxx ,...,1 a nd my y ,...,1 refers to the image labels (femur or tibia) The weights are first initialized as (2 5) In Equation 2 5 m stands for the total number of images in the dataset. For t=1 ,., T (where T stands for the total number of Haar Features) (2 6) If the error, then the loop is discontinued Set the weight factor (2 7) The weights for each of the i samples is then updated as given by Equation 28 (2 8) In Equation 2 8 Zt refers to a normalization factor. Finally, the strong classifier is given by the weighted sum of weak classifiers (2 9) SIFT classifier for object recognition: In cases where the dataset might not be large enou gh or there are very wide variations in rotation and viewpoints of objects, a SIFT classifier can be used instead of AdaBoost to ensure that object recognition is performed. The SIFT algorithm generates key points which remain invariant to several external factors such as illumination, rotation and camera viewpoints while staying partially invariant to affine transformations and the presence of occlusions and noise. Once the SIFT key points were generated the magnitude and orientation of these key points w ere used as a feature set to m i Dt/ 1 ) ( m i i i t j tx h y i D h1) ( ) ( min arg 2 / 1t t t t 1 log 2 / 1 t i t t t t tZ x h y i D i D ) ( exp( ) ( ) (1 T t t tx h sign x H1) ( ) (

PAGE 24

24 implement a three layer perceptron based neural network classifier. This text will delve into SIFT features in greater detail in the next section and illustrate exactly how pattern matching occurs. The performance of SIFT and A daBoost was found to be comparable for the dataset used without any major differences in parameters such as computational time or error rate. Pose Determination The central problem which our implementation attempts to solve is that of arriving at a good ap proximate initial pose to replace the users manually generated guess. Therefore, once the two rectangular regions representing the femur and tibia have been extracted, pose matching is performed between these regions and a generated library of the project ions of the CAD model. While currently popular techniques employ contour matching and edge detection [4], these result in several local minima and are error prone when the image illumination or object model changes. Therefore, we have used the SIFT key poi nts as the primary parameter for matching between the model pose library and the implant of interest. The model library consists of projections of the manufacturers model under certain biomechanical constraints. Since the most important parameters for 2D to 3D registration are the out of plane rota tions, the library consists of variations over the entire range of these out of plane rotations at a resolution of 5 degrees. The model library further computes the range of in plane rotation at a resolution of 10 degrees. Once the matching of the SIFT key points yields an estimated orientation, conventional 2 D image registration techniques can be applied to arrive at a complete initial pose which includes all 6 DOFs (Degrees of Freedom). SIFT classifier for pose recognition: SIFT Features are based on the generation of key points from images such that they are invariant to several external factors such as illumination, vie wpoint, noise and occlusions [6]. Since we seek to replace the function of a human oper ator with an automate d algorithm, it is of interest to note that SIFT features have been shown to

PAGE 25

25 correspond to the biological basis of vision in studies of the primate visual cortex [20]. According to recent studies the use of SIFT Features for correspond ence matching has been evaluated as superior to almost all other currently existing techniques [21]. Figure 2 8 SIFT Feature matching process SIFT key point generation: SIFT key points are generated by finding the local peaks for DoG (Difference of Gaus sian) images. The steps involved in finding SIFT key points are: 1 Scale Space Extrema Generation 2 Key point Localization 3 Orientation Assignment 4 Key point Descriptor Scale Space Extrema are used to find the locations and scales which remain constant for sev eral views of the given object. This is found by convolving the image with a scale space operator which is gen erally a Gaussian, as shown in E quation 2 10. ) ( ) ( ) ( y x I y x G y x L (2 10) As these images are scaled, for instance by a factor k, the Difference of Gaussians between different scales of the image are computed as given in Equation 2 11 ) ( ) ( ) ( y x L k y x L y x D (2 11)

PAGE 26

26 The local maxima and minima of the function D is found by generally comparing it with 8 nearest neighbors of the sam e scale and 9 nearest neighbors at one scale higher and lower. This procedure ensures that the features are consistent even for progressively blurred images. The next step consists of testing that each of the key points is above a certain fixed threshold t o ensure that low contrast regions are not used for generating key points. This is because low contrast regions usually correspond to noisy portions of images and not the actual features. In addition, adjacent key points are also tested for principle curva ture in order to reject edge points. A B Figure 2 9 Illustration of orientation values of SIFT key points A) Various gradients over different parts of a local patch, B) Histogram of the gradient orientations (Source www.open.ou.nl/Touw/TouW dagen%202008/TouW dag%2083 2008/touw2.ppt. Last accessed March, 2009). A fter thresholding, each key point is assigned a magnitude and orientation based on the local image properties as given in Equation 2 12 and Equation 2 13. The gradient histograms of sample points are used to generate an orientation histogram for a region. As indicated in the Figure 2 10, t he angle corresponding to the peak of that histogram is used t o create a key point. The magnitude is given by, 2 2)) 1 ( ) 1 ( ( ) 1 ( ) 1 ( ( ) ( y x L y x L y x L y x L y x m (2 12) The orientation is given by,

PAGE 27

27 )) 1 ( ) 1 ( ( )) 1 ( ) 1 ( ( tan ) (1y x L y x L y x L y x L y x (2 13) SIFT key point matching and Library generation : With the application of reasonable biomechanical const raints, the projection of a CAD model of the femoral and tibial implants is obtained to form an extensive image library. SIFT key points are generated for each model pose available in the library and a close matching of the set of features for the model li brary with respect to the original image features is done [23] The matching is performed by first calculating the distance function between every combination of possible descriptor pairs between the library model and image. Instead of merely using the res ults with the minimum value of the distance metric, we first obtain those pairs of images with the largest number of matched key points. In the second stage we find the pair corresponding to the minimum distance metric within that subset. A coarse to fine registration can be performed by changing the resolution and angular spacing between adjacent library entries once an approximate match is found. However, since the pose results which have been generated are not used as a final result this is usually not n ecessary. A salient feature of using SIFT is scale invariance, as a result of which several types of image can be matched with a single model library for pose matching in the first stage. As image libraries with finer accuracies are used, the matching err or also decreases to an acceptable value in the range of 2 to 5 degrees [4]. In a typical case, the number of features which are generally matched range from 3 to 25. In a counter intuitive argument, it was observed that more features being matched does not always correspond to the best matching result. The best results were those which had a large number of feature matches as well as a low value for the distance between the corresponding features.

PAGE 28

28 A B Figure 2 10. Femur matching A) Femur key points B) Matched model with key p oints A B Figure 2 11. Tibia matching A) Tibia key points B) Matched model with key points

PAGE 29

29 CHAPTER 3 RESULTS Results of Object Recognition The r esults for the object recognition section was found for both the AdaBoost as well as the SIFT based neural network algorithm. A training data set of 1400 images consisting of 700 femur and 700 tibia images were used to train AdaBoost. The classifier underw ent 25 cycles and 25 weak classifiers were combined. The computational time for AdaBoost training on a high speed Intel Quad Core Processor was 20 minutes and the computational time for testing was less than 5 minutes. The testing data set consisted of 50 images of the femur and tibia in a random proportion. The training data set included noisy samples to test the efficiency of the program. In the case of the SIFT algorithm, a training data set of 1600 images consisting of 800 femur and 800 tibia images wer e used. However, it was observed that a training data set of just 50 images was sufficient to train the neural network to achieve a detection error rate of less than 10%. T able 3 1. Object recognition Classifier Training images Test images Error r a te AdaBoost 1400 images, 700 femur, 700 tibia 50 images 8 % Neural N/w 1600 images 800 femur, 800 tibia 50 images 6 % It is noted that matching is currently not accurate in the case of X ray images in comparison with fluoroscopic images due to the presence of bone components in X rays. Results of Pose Determination Pose determination results were compiled using a library of 936 femur models and 2364 tibia models. Biomechanical constraints which were applied to the CAD models such that most

PAGE 30

30 model poses were created in the library. Initially the resolution for the out of plane rotations were maintained at 5 degrees and the resolution for in plane rotation was 10 degrees. Once a match of reasonable accuracy has been found, a finer match ca n be found by creating a new image library of 5x5x5 images. The Figures 3 1 illustrate examples of pose matching in the femur and tibia. The number of key points generated for the femur and tibia ranged between 5 and 15. In a good match the minimum value o f distance between the key points found was 200. Figure 3 1 Tibia pose A), C) Tibia Models B), D) Tibia image matching It was observed that errors were primarily in the in plane rotation which is easy to correct in comparison with the out -of -plane rotations. Most of the images which underwent proper thresholding and noise removal showed a highly accurate pose result.

PAGE 31

31 Figure 3 2 Femur pose A), C) Femur images, B), D) Matched f emur models It is noted that matching is currently not accurate in the case of X ray images in comparison with fluoroscopi c images. This could be due to the presence of several soft tissues and bone components which are imaged more clearly in the case of X rays. The tabulated results for the x, y and z axes orientations have been shown in Table 32. The results were calcula ted for 15 images while the subject was performing Kneel, Lunge and Stair motions. It can be noted here that since more emphasis was placed on obtaining an accurate out of plane rotation, the number of in plane rotation models were fewer and hence resulted in lower accuracy of the in plane rotation around the x axis It is observed that the mean of the deviation of the predicted value from the actual results are within the acceptable range of 5 degrees. In the case of in plane rotations, the lower accuracy can be easily compensated by increased library size or further angular refinements. T able 3 2 Femur Orientation Determination Axis of r otation Mean error Variance of e rror X Y Z 1.5 1.5714 4.0909 2.2909 2.4 0.7774

PAGE 32

32 T able 3 3 Tibia Orientation Determination Axis of r otation Mean e rror Variance of e rror X Y Z 1.167 1.5714 4.16 1.75 3.12 2.18 Discussion The problem of pose estimation has been tackled in several invasive and non invas ive methods. While usually contour or intensity based techniques are commonly employed, the SIFT based technique aims to instantly extract the portions of the contour which correspond to image key points. By matching these key points which are invariant to numerous factors such as occlusions, noise, and illumination we can arrive at an accurate initial pose estimate. Once this is complete, the model in the resulting pose is then matched using currently available software to get an accurate pose. In the case of the SIFT transform the robustness to noise is not as high as required for medical images. As a result the effectiveness of the SIFT transform depends to a large extent on the successful implementation of image processing techniques such as noise filter ing and thresholding. Hence, finding a closer segmentation for the femur and tibia especially in the images in which they intersect or overlap is very important. A common drawback of most template based matching schemes is the generation of a large image l ibrary which is computationally expensive and time consuming. However, in the case of SIFT a single image library works for matching most images which actually use different models. This can be attributed to the fact that it is scale invariant and is used to train large datasets of objects with a wide variation such as cars, vehicles and other objects. By generating an automatic initial pose, manual errors generated by different observers are overcome. Repeatability of results is ensured and expertise on t he part of the user is not needed

PAGE 33

33 for successful operation. In possible future improvements, the initial pose of a sequence of images taken from a video can be computed by using the value of the final pose of the previous image, thus reducing the number of computations. In the future, this algorithm could also be used for the registration of bones and implants in other regions of the body such as the shoulder.

PAGE 34

34 LIST OF REFERENCES [1] Centers for Disease Control and Prevention. National Center for Health Statistics. Data W arehouse on Trends in Health and Aging. National Hospital Discharge Survey. Health Data Interactive. www.cdc.gov/nchs/hdi.htm [Date of access March 20 09]. [2] Banks, S. A., Harman, M.K., Be llemans, J., Hodge, W.A., 2003, Making sense of knee A rthroplast y kinematics: news you can use, Am. J. Bone Joint Surg., 85(A), pp. 64 72. [3] Yamazaki, T., Watanabe, T., Nakajima, Y., Sugamoto, K., Tomita, T., Yoshikawa, H., Tamura, S., 2004, Improvement of depth position in 2 D/3 D registration of knee implants using single -plane fluoroscopy. IEEE Trans on Medical Imaging, 23(5), pp. 602612. [4] Mahfouz, M.R., Hoff, W .A., Kom istek, R.D., Dennis, D.A., 2003, A robust method for registration of three -dimensional knee implant models to two-dimensional fluoroscopy images, IEEE Trans on Med Imaging 22 (12), pp. 15611574. [5] Viola, P., Jones, M., 20 01, Rapid object detection using a boost ed cascade of simple features, IEEE Comp Soc Conf on Computer Vision and Pattern Recognition 2001, vol.1. I EEE, pp. 511518. [6] Lowe, D.G., 1999, Object recognition from local scale invariant features, Proceedings of the Seventh IEEE International Conference on Computer Vision 1999, vol.2. IEEE, pp.11501157. [7] Shang, M., 2007, Jointtrack: An Open -Source, Easily Expandable Program For Skeletal Kinematic M easurement Using Model Image Registration Masters Thesis, University of Florida Dept of Mechanical and Aerospace Eng pp 4 24. [8] Sato, T., Koga, Y., Omori, G., 2004, Three -dimensional lower extremity alignment assessment system: Application to evaluation of component position after total knee Arthroplasty, The J. of Arthroplasty, 19 (5), pp. 620628. [9] Banks, S.A., Hodge, W.A., 1996, Accurate measurement of three -dimensional knee replacement kinemati cs using single -plane fluoroscopy, IEEE Trans. Biomed. Eng. 43 (6), pp.638649. [10] Komistek, R. D., Dennis, D.A., Hoff, W.A., Gabriel, S.M., and Walker, S.A., 200 2. Three -dimensional determination of femoral tibial contact positions under in vivo conditions using fluoroscopy, The J. Arthroplasty, 17 (2), pp. 209216. [11] Zuffi, S., Leardini, A., Catani, F., Fanto zzi, S., and Cappello, A., 1999. A Model Based Method for the Reconstruction of To tal Knee Replacement Kinematics, IEEE Trans. On Med.Imaging, 18 (10), pp. 981991.

PAGE 35

35 [12] Li, G., Wuerz, T. H., and DeFrate, L. E., 2004, Feasibility of Using Orthogonal Fluoroscopic Images to Measure In Vivo Joint Kine matics. J. B iomech. Engg 126(2), pp. 314318. [13] Tang, T. S. Y., MacIntyre, N. J., Gill, H. S., Fellows, R. A., Hill, N. A., Wilson, D. R., and Ellis, R. E., 2004, Accurate assessment of patellar tracking using fiducial and int ensity based fluoroscopic techniques. In: Medical Image Analysis, Medical Image Computing and Computer -Assisted Intervention MICCAI, 8(3), pp. 343351 [14] Jain, A., 1986, Fundamentals of digital image p rocessing PrenticeHall, p p 408. [15] Chow, C.K., and Kaneko T., 1972, Automatic Boundary Detection of the Left Ventricle from Cineangiograms, Comp. Biomed. Res., 5, pp. 388410. [16] Shi, J., Malik, J., 2000, Normali zed cuts and image segmentation, IEEE Trans. Pat tern Analysis and Machine Intelligence, 22 (8), pp. 888905. [17] Viola, P., Jones, M.,2002, Fast and robust classification using asymmetric adaboost and a detector cascade, Advances in Neural Information Processing Systems [18] Wu, B ., Ai, H., Huang, C., Lao, S., 2004, Fast rotation invariant multi -view face detection based on real Adaboost, Proc. of Sixth IEEE Intl. Conf. on Automatic Face and Gesture Recognition. vol., no., pp. 79 84. [19] Sochman, J., and Matas J., Center for Machine Perception, Czech Technical University, http://cmp.felk.cvut.cz [Date of access March 2009]. [20] Serre, T., Wolf, L., Poggio, T., 2005, Object recognition with features i nspired by visual Cortex, IEEE Comp Soc. Conf. on Comp. Vision and Pattern Rec, 2 pp.9941000. [21] Mikolajczyk, K., Schmid, C., 2003, A performance evaluation of Local Descriptors. IEEE Comp. Soc. Conf. on Comp. Vision and Pattern Rec., 2, pp. 257264. [22] Gordon, I. and Lowe, D.G., 2006, What and w here: 3D object recognition with accurate p ose, Springer -Verlag Toward Category -Level Object Recognition Berlin, Germany, pp. 6782. [23] Vedaldi, A., and Fulkerson, B., 2008, The VL Feat Open and Portable Computer Vision Library, has been used for SIFT Ke y point Generation.

PAGE 36

36 BIOGRAPHICAL SKETCH Avantika Vardhan was born in Mumbai in 1986. She lived in Chennai, India for 21 years and completed her b achelor s degree i n electronics and communication engineering from S ri Ramaswamy M emorial University Sh e pursued her master s in the Department of Electrical and Computer Engineering at University of Florida Her research interests lie in the field of c omputer vision and image processing with medical applications.