SCALEINDEPENDENT BIOMECHANICAL OPTIMIZATION
J.F. Schutte (1), B. Koh (2), J. A. Reinbolt (3), R.T. Haftka (1), A. George (2), B.J. Fregly (1,3,4)
(1) Dept. of Mechanical & Aerospace Engineering
University of Florida
Gainesville, FL
(3) Department of Biomedical Engineering
University of Florida
Gainesville, FL
INTRODUCTION
In this paper we present a scaleindependent method of
optimization with a stochastic global optimization approach introduced
by Kennedy and Eberhart [1], the Particle Swarm Optimizer (PSO).
We apply this method to the biomechanical system identification
problem of finding positions and orientations of joint axes in body
segments through the processing of experimental movement data [2,3].
We compare its performance to the BFGS optimizer which falls under
a class of optimizers more commonly used for this application.
Traditionally, gradientbased methods such as the BFGS
algorithm have been used to solve joint parameter identification
problems, but major drawbacks to these methods are their sensitivity
to problem scaling and algorithm parameter selection. These
drawbacks require a costly and timeconsuming parameter sensitivity
studies to be carried out for a problem before consistently acceptable
results can be obtained. In addition, the presence of noise in the data
will often cause premature convergence to an incorrect solution.
The PSO method has some very desirable qualities that can be
exploited in these types of problems. First, because it requires no
gradient evaluations and because of the way it is formulated, the
algorithm is insensitive to scaling of the design variables. Second,
because of the algorithm's simplicity, there are very few parameters to
tune, and even these have been shown to be relatively problem
independent. Finally, the concurrent nature of the swarm algorithm
lends it to parallelization, enabling the solution of problems that are
too computationally challenging for singleprocessor machines. The
need for greater computational power is common in the search for
more realistic and accurate engineering models [4], which currently
can only be addressed by the use of parallel algorithms.
METHODOLOGY
For our test problem, we have elected to use a 2 degreeof
freedom (DOF), 3dimensional ankle kinematic model requiring 12
parameters [2]. This model is used to generate synthetic trajectories of
markers fixed to the foot and shank (three per segment). An
optimization approach was then followed to recover the original joint
(2) Dept. of Electrical & Computer Engineering
University of Florida
Gainesville, FL
(4) Department of Orthopaedics & Rehabilitation
University of Florida
Gainesville, FL
parameters from this synthetic marker trajectory data. One of the
primary reasons for this approach is that any solution found by either
algorithm can be quantified in terms of final design variable errors.
Therefore, we are able to make an immediate evaluation of the
performance of the optimization algorithm that was used. In addition,
we have control over the magnitude and other characteristics of any
numerical noise we introduce into the system to emulate experimental
measurement errors, and we are able to observe the impact varying
any of these has on the optimization algorithm.
The unconstrained optimization (or system identification)
problem can be stated as follows:
min f(x) e(x,t) (1)
x
t
m 3
e(x,t)=min (m (t) c (x,p)) (2)
P
I J
where x is a proposed joint position and orientation in the body
segment, which will have a corresponding marker configuration c(x).
The fitness (Eq. 1) of this marker configuration is evaluated according
to how closely it can be aligned (in three dimensions j) to all of the m
markers in configuration m(t) over all n recorded time frames t. This
matching or alignment of proposed virtual to observed marker
locations is done in a separate optimization step (Eq. 2) by means of a
nonlinear least squares fit, where p is an alignment operator [5].
In order to demonstrate the potential sensitivity to scaling in both
PSO and BFGS methods, the ankle joint identification problem was
first defined using the original units of cm and radians for the location
and orientation design variables respectively. Bounds on these design
variables were chosen to enclose a physically realistic interval around
the solution point in the design space.
2003 Summer Bioengineering Conference, June 2529, Sonesta Beach Resort in Key Biscayne, Florida
The scaled version of this problem was then obtained by normalizing
all 12 variables to be bounded within [1,1].
Numerical noise was introduced by means of superimposing a
sine wave with a random period, phase, and amplitude (limited to a
maximum of 1 cm) onto the marker data. This was done to emulate
artifacts caused by skin and soft tissue movement and by camera
resolution limitations found in real data.
In generating the synthetic marker trajectory data, we endeavored
to simulate real life data which are analyzed in exactly the same
manner. As such, the problem, while still being analytical, required the
use of parallel processing due to the sheer amount of data to be
processed. Both the PSO and the unconstrained BFGS gradientbased
method were parallelized and evaluated on a cluster of 29 Linux based
PCs in the UF HCS Research Laboratory (1.33 GHz Athlons with
256MB memory on a 100Mbps switched Fast Ethernet network).
RESULTS
104
103
102
 Unscaled BFGS
Scaled BFGS
Scaled/Unscaled PSO
D 101 Iterations 102 10
Figure 1: Fitness history for typical BFGS and PSO runs
Before using the parallelized VisualDOC (Vanderplaats R & D,
Colorado Spring, CO) implementation of the BFGS algorithm, a
parameter sensitivity study was performed in order to obtain the
optimum initial forward finite difference and termination parameters
of 102 and 105 respectively. For the parallel PSO, the standard general
recommended parameters were used [6].
A total of 10 unsealed and 10 scaled (or normalized) optimization
runs were performed for both the PSO and BFGS algorithms (Table
1). For both methods, the optimizations were started at randomly
chosen points within the bounds. The same starting point locations
were used throughout all of the runs when switching to the scaled
problem in order to obtain a fair comparison for both algorithms.
The BFGS algorithm converged prematurely in all 10 unsealed
runs, never obtaining even approximately correct joint positions and
orientations, resluting in poor fitness values (large cumulative marker
errors). Restarting the algorithm at the termination points yielded no
improvement. After scaling the problem, however, we obtained
convergence in 7 out of the 10 runs to an approximately correct
solution. Again, all attempts to further improve these by a restart of the
algorithm are unsuccessful.
In contrast, with the PSO method, both unsealed and scaled
optimizations produced a final fitness errors on the order of the noise
level with very little standard deviation as compared to the BFGS
results. However, this reliability and accuracy came at a high cost in
terms of function evaluations (see Table 1).
When comparing fitness value drift between the scaled and
unsealed problems during the optimization (Figure 1), we observed
only a minute difference for the PSO algorithm, on the order of 104,
due to numerical roundoff and truncation. In contrast, the scaled
BFGS solution very quickly diverged from the unsealed solution and
in all but two cases terminated at an entirely different result than found
for the unsealed problem. In these two cases, both the scaled and
unsealed optimizations failed to converge to the approximate solution.
Algorithm Unscaled Scaled
Mean fitness error BFGS 3806 2477 889 1601
std. dev. PSO 69.28 4 69.31 2
BFGS 65 305
Mean function evals
PSO 19700 20040
Table 1: Comparison of unsealed and scaled optimization
results for BFGS and PSO algorithms
CONCLUSIONS
The BFGS requires scaling of the problem and an extensive
parameter sensitivity study before it is able to find the general solution
region consistently. Even with both these measures, it still tends to
become trapped in local minima as can be seen from the large standard
deviation values in Table 1. The main advantage of this method is its
efficiency in terms of function evaluations. In contrast, the PSO is very
reliable in finding the correct solution region, and is insensitive to both
the scaling of the problem and initial algorithm parameter selection. Its
main drawback is the high cost in terms of function evaluations
because of slow convergence in the final stages of the optimization. A
hybrid approach could be advantageous if an efficient transition
criterion could be found and optimum parameters for the BFGS
algorithm are known in advance.
ACKNOWLEDGEMENTS
This study was funded by NIH National Library of Medicine (R03
LM0733201) and Whitaker Foundation grants to B.J.F and AFOSR
grant F496200910070 to R.T.H.
REFERENCES
1. Kennedy, J. and Eberhart, R.C., 1995, "Particle swarm
optimization", Proceedings of the 1995 IEEE International
Conference on Neural Networks, Perth, Australia, Vol. 4, pp.
19421948.
2. van den Bogert AJ, Smith GD, Nigg BM, 1994, "In vivo
determination of the anatomical axes of the ankle joint complex:
an optimization approach", J. Biomech. Vol 12, pp. 147788.
3. Sommer, H. J. III, Miller, N. R., 1980, "Technique for kinematic
modeling of anatomical joints", J Biomech. Eng Trans ASME,
Vol 102, pp. 311317.
4. Venter, G., and Watson, B.C., "Exploiting parallelism in general
purpose optimization", 2000, Proceedings, 6th International
Conference on Applications of HighPerformance Computers in
Engineering, Maui, Hawaii.
5. Lu, T.W., and O'Connor, J.J., 1999, "Bone position estimation
from skin marker coordinates using global optimization with joint
constraints," Journal of Biomechanics, Vol. 32, pp. 129134.
6. Carlisle, A., Dozier, G., 2001, "An offtheshelf PSO",
Proceedings of the Workshop on Particle Swarm Optimization,
Purdue School of Engineering and Technology 2001,
Indianapolis, USA.
2003 Summer Bioengineering Conference, June 2529, Sonesta Beach Resort in Key Biscayne, Florida
