
Citation 
 Permanent Link:
 https://ufdc.ufl.edu/UF00097439/00001
Material Information
 Title:
 Testing lack of fit in a mixture model
 Creator:
 Shelton, John Thomas, 1952 ( Dissertant )
Khuri, Andre I. ( Thesis advisor )
Cornell, John A. ( Reviewer )
Fisher, Richard F. ( Reviewer )
 Place of Publication:
 Gainesville, Fla.
 Publisher:
 University of Florida
 Publication Date:
 1982
 Copyright Date:
 1982
 Language:
 English
 Physical Description:
 viii, 202 leaves : ill. ; 28 cm.
Subjects
 Subjects / Keywords:
 Degrees of freedom ( jstor )
Eigenvalues ( jstor ) Least squares ( jstor ) Matrices ( jstor ) Modeling ( jstor ) Parametric models ( jstor ) Point estimators ( jstor ) Polynomials ( jstor ) Statistical models ( jstor ) Statistics ( jstor ) Dissertations, Academic  Statistics  UF Mixtures ( lcsh ) Mixtures  Mathematical models ( lcsh ) Statistics thesis Ph. D
 Genre:
 bibliography ( marcgt )
nonfiction ( marcgt )
Notes
 Abstract:
 A common problem in modeling the response surface in
most systems, and in particular in a mixture system, is that
of detecting lack of fit, or inadequancy, of a fitted model
of the form E(Y) = Xg, in comparison to a model of the form
E{Y) = Xe,+ X B postulated as the true model. One method
for detecting lack of fit involves comparing the value of
the response observed at certain locations in the factor
space, called "check points," with the value of the response
that the fitted model predicts at these same check points.
The observations at the check points are used only for
testing lack of fit and are not used in fitting the model.
It is shown that under the usual assumptions of
independent and normally distributed errors, the lack of fit
test statistic which uses the data at the check points is an F statistic. When no lack of fit is present the statistic
possesses a central F distribution, but in general, in the
presence of lack of fit, the statistic possesses a doubly
noncentral F distribution. The power of this F test depends
on the location of the check points in the factor space
through its noncentrality parameters. A method of selecting
check points that maximize the power of the test for lack of
fit through their influence on the numerator noncentrality
parameter is developed.
A second method for detecting lack of fit relies on
replicated response observations. The residual sum of
squares from the fitted model is partitioned into a pure
error variation component and into a lack of fit variation
component. Lack of fit is detected if the lack of fit
variation is large in comparison to the pure error
variation. This method can be generalized when "near
neighbor" observations must be substituted for replicates.
In this case, the test statistic (assuming independent and
normally distributed errors) has a central F distribution
when the fitted model is adequate and a doubly noncentral F
distribution under lack of fit. The arrangement of near
neighbors is seen to affect the testing procedure and its
power.
 Thesis:
 Thesis (Ph. D.)University of Florida, 1982.
 Bibliography:
 Includes bibliographic references (leaves 198201).
 General Note:
 Typescript.
 General Note:
 Vita.
 Statement of Responsibility:
 by John Thomas Shelton.
Record Information
 Source Institution:
 University of Florida
 Holding Location:
 University of Florida
 Rights Management:
 Copyright [name of dissertation author]. Permission granted to the University of Florida to digitize, archive and distribute this item for nonprofit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
 Resource Identifier:
 028552175 ( AlephBibNum )
09205562 ( OCLC ) ABU5710 ( NOTIS )

Downloads 
This item has the following downloads:

Full Text 
TESTING LACK OF FIT IN A MIXTURE MODEL
BY
JOHN THOMAS SHELTON
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1982
To Nydra
and
My Parents
ACKNOWLEDGEMENTS
I would like to express my deepest appreciation to Drs.
Andre' Khuri and John Cornell for suggesting this topic to
me and for providing constant guidance and assistance. They
have made this project not only a rewarding educational
experience but an enjoyable one as well. A special word of
thanks goes to Mrs. Carol Rozear for her diligence in
transforming my handwritten draft into an expertly typed
manuscript.
iii
TABLE OF CONTENTS
page
ACKNOWLEDGEMENTS............ ............. .. .............iii
ABSTRACT .............................................. vii
CHAPTER
ONE INTRODUCTION................................... 1
1.1 The Response Surface Problem............... 1
1.2 The Mixture Problem....................... 5
1.2.1 Mixture Models..................... 6
1.2.2 Experimental Designs for Mixtures.. 12
1.3 The Purpose of this Research:
Investigation of Procedures for Testing
a Model Fitted in a Mixture System for
Lack of Fit ............................... 17
TWO LITERATURE REVIEWTESTING FOR LACK OF FIT..... 19
2.1 Introduction............................... 19
2.2 Partitioning the Residual Sum of Squares.. 21
2.3 Testing for Lack of Fit Without
Replicated ObservationsNear Neighbor
Procedures ................................ 26
2.4 Testing for Lack of Fit with Check Points. 33
THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL................. 40
3.1 Introduction............................... 40
3.2 Testing for Lack of Fit in the Presence
of an External Estimate of Experimental
Error Variation............................ 41
3.2.1 The Test Statistic................. 41
3.2.2 The Testing Procedure and an
Expression for the Power of
The Test........................... 45
3.2.3 A Method for Locating Optimal
Check Points ....................... 47
3.3 Testing for Lack of Fit When MSE Is
Used to Estimate Experimental Error
Variation..................................... 51
3.3.1 The Test Statistic................. 51
3.3.2 The Rejection Region and its
Relation to the Power of the Test.. 53
iv
3.3.3 A Method for Locating Optimal
Check Points ....................... 56
3.3.4 Determining Whether the Test Is
Upper Tailed or Lower Tailed....... 58
3.4 Examples. ................................ 67
3.4.1 Theoretical Examples............... 67
3.4.2 Numerical Examples................. 83
3.5 Discussion. ............................... 95
FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR
TESTING LACK OF FIT .............................. 99
4.1 Introduction ............................. 99
4.2 Notation......... ..........................101
4.3 Shillington's Procedure ...................106
4.3.1 Development of MSEB ................109
4.3.2 Development of MSEW................110
4.4 Development of SSE(weighted) .............112
4.5 Equivalence of SSEW and SSEW(weighted)....116
4.6 The Test Statistic .........................118
4.7 The Testing Procedure and its Power........122
4.8 Selection of Near Neighbor Groupings....... 125
4.8.1 Example 1Stack Loss Data.........129
4.8.2 Example 2Glass Leaching Data.....134
4.9 Discussion................................ 142
FIVE CONCLUSIONS AND RECOMMENDATIONS ................145
APPENDICES
1 INFLUENCE OF X1 ON
P{F" > F } ...................156
v 1'2 ;11 '12 a;v ,v2
2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR
GLOBAL OPTIMIZATION ............................159
3 STATISTICAL INDEPENDENCE OF d'V d/o2
2 0
AND SSE/o .................................... 164
4 THEOREM 3.1................................... 168
5 THEOREM 3.2...................................... 169
6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL
F DISTRIBUTION.................................. 171
7 EQUIVALENCE OF SSEB AND SSLOF WHEN
REPLICATES REPLACE NEAR NEIGHBOR
OBSERVATIONS...................................... 172
8 LEMMA 4.1...................................... 175
9 PROOF OF THEOREM 4.1(i) ............ ......... 178
10 PROOF OF THEOREM 4.1(ii)......................182
11 PROOF OF THEOREM 4.1(iii).............. ........185
12 PROOF OF THEOREM 4.2.............................191
1
13 PROOF OF THE EQUALITY SSE = d'V d + SSE .....193
A 0 
REFERENCES .... ....................................... 198
BIOGRAPHICAL SKETCH.. .......... .. ..... ............202
Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in
Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy
TESTING LACK OF FIT IN A MIXTURE MODEL
By
John Thomas Shelton
May 1982
Chairman: Andre' I. Khuri
Cochairman: John A. Cornell
Major Department: Statistics
A common problem in modeling the response surface in
most systems, and in particular in a mixture system, is that
of detecting lack of fit, or inadequancy, of a fitted model
of the form E(Y) = X 1 in comparison to a model of the form
E(Y) = X1 + X 22 postulated as the true model. One method
for detecting lack of fit involves comparing the value of
the response observed at certain locations in the factor
space, called "check points," with the value of the response
that the fitted model predicts at these same check points.
The observations at the check points are used only for
testing lack of fit and are not used in fitting the model.
It is shown that under the usual assumptions of
independent and normally distributed errors, the lack of fit
test statistic which uses the data at the check points is an
vii
F statistic. When no lack of fit is present the statistic
possesses a central F distribution, but in general, in the
presence of lack of fit, the statistic possesses a doubly
noncentral F distribution. The power of this F test depends
on the location of the check points in the factor space
through its noncentrality parameters. A method of selecting
check points that maximize the power of the test for lack of
fit through their influence on the numerator noncentrality
parameter is developed.
A second method for detecting lack of fit relies on
replicated response observations. The residual sum of
squares from the fitted model is partitioned into a pure
error variation component and into a lack of fit variation
component. Lack of fit is detected if the lack of fit
variation is large in comparison to the pure error
variation. This method can be generalized when "near
neighbor" observations must be substituted for replicates.
In this case, the test statistic (assuming independent and
normally distributed errors) has a central F distribution
when the fitted model is adequate and a doubly noncentral F
distribution under lack of fit. The arrangement of near
neighbors is seen to affect the testing procedure and its
power.
viii
CHAPTER ONE
INTRODUCTION
1.1 The Response Surface Problem
A mixture problem is a special type of a response
surface problem. First we shall define the general response
surface problem and indicate the basic objectives sought in
its analysis, and follow this development with a discussion
of the mixture problem.
In the general response surface problem, we are inter
ested in studying the relationship between an observable
response, Y, and a set of q independent variables or
factors, xl, x2, ..., Xq, whose levels are assumed con
trolled by the experimenter. The independent variables are
quantitative and continuous. We express this relationship
in terms of a continuous response function, p, as
Y = (ul, x2 uq ) + u
where Yu is the uth of N observations of the response col
lected in an experiment, and xui represents the uth level of
the ith independent variable, u = 1, 2, ..., N, i = 1, 2,
..., q. The exact functional relationship, p, is unknown.
The term Eu is the experimental error of the uth
observation. It is assumed that E(cu) = 0, E(Eueu,) = 0,
for u u', and E(e ) = 2, for u = 1, 2, ..., N.
As the form of ( is unknown and may be quite complex, a
low order polynomial (usually first or second order) in the
independent variables xl, x2, ..., Xq is generally used to
approximate p. This may be justified by noting that such
polynomials constitute low order terms of a Taylor series
expansion of < about the point xl = x2 = ... = xq = 0,
(Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point
out that these low order polynomials may give a poor approx
imation to q when extrapolated beyond the experimental
region, and thus should not be used for this purpose.
A linear response surface model may be written in
matrix notation as
Y = X4 + E (1.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, a is a pxl vector of
unknown parameters (regression coefficients), and a is the
Nxl vector of random errors. When the model is a first or a
second degree polynomial, the columns of X correspond to the
first or second degree powers of the independent variables
xl, x2, ..., Xq, or their cross products. If the model
contains a constant term, 80, the first column of X will
correspond to this term, and will consist of N ones. Since
E(c) = 0, an alternative representation for the response
surface model of (1.1) is
E(Y) = X .
Once the form of the model that will be used to approx
imate ((xl, x2, ..., Xq) is chosen, the next step is to
estimate the regression coefficients, a, and then use the
estimated model to make inferences about the true response
function, (. The estimation of the elements of a is usually
accomplished by ordinary least squares techniques. For the
purpose of testing hypotheses concerning the regression
coefficients, ., it is assumed that L has a normal distribu
tion, that is, e ~ NN(O, a 2N).
Perhaps the most common objective in the exploration of
a response system is the determination of its optimum
operating conditions. By this we mean that it is desired to
find the settings of xl, x2, ..., xq that optimize (, which
in some applications may be interpreted as maximizing p,
while in other applications a minimum value of ( may be of
interest. It is also often desirable to determine the be
havior of the response function in the neighborhood of the
optimum. For second order response models, such an investi
gation can be carried out by performing a canonical analysis
of the second order surface as discussed in Myers (1971).
For simple systems having only one or two independent
variables, the response surface may be explored by just
plotting the fitted response values against values taken by
the independent variables. If q = 1, implying only one
independent variable, say xl, then a twodimensional plot of
the fitted response against xl may be used to locate the
optimum, as well as to investigate the response behavior in
other parts of the experimental range of xl. If q = 2, and
the two independent variables are xl and x2, then a plot of
the contours of constant response over the region specified
by the ranges of the values for xI and x2 can be used to
describe the response surface.
The properties that the fitted model possesses in terms
of its ability to represent the true surface, p, depend on
the settings of xl, x2, ..., Xq at which values of Y are
observed. Thus the experimental design is of great impor
tance. Much work has been done on the construction of
designs that are optimal with respect to one criterion or
another involving the fitted response and/or the true unfit
ted model. Box and Draper (1975) list fourteen criteria to
consider when choosing a design for investigating response
surfaces. Myers (1971) gives several designs for fitting
first and second order polynomial models. A discussion of
specific design considerations will not be attempted here,
as such a discussion is not the focus of this dissertation,
and would necessarily be lengthy.
The initial steps in the analysis of a response system
may be described as follows: First an attempt is made to
approximate the true response function, p(xl, x2, ..., Xq),
usually with a low order polynomial in xl, x2, ..., xq.
After the form of the model has been chosen, then comes the
selection of an appropriate experimental design, which
specifies the settings of the independent variables at which
observed values of the response will be collected. The
observed values of the response are used in estimating the
regression coefficients in the model, using, in general,
ordinary least squares. After a test for "goodness of fit"
of the model verifies the fitted model is adequate, the
fitted model is used in determining optimum operating condi
tions for the response system.
1.2 The Mixture Problem
A mixture system is a response system in which the
response depends only on the relative proportions of the
components or ingredients present in a mixture, and not on
the total amount of the mixture. For example, the response
might be the octane rating of a blend of gasolines where the
rating is a function only of the relative percentages of the
gasoline types present in the blend. The proportion of each
ingredient in the mixture, denoted by xi, must lie between
zero and unity, i = 1, 2, ..., q. The sum of the propor
tions of all the components will equal unity, that is,
q
0 < x. < 1, i = 1,2,...,q, E x. = 1. (1.2)
i=l
The factor space containing the q components is represented
by a (q 1)dimensional simplex. For q = 2 components, the
factor space is a straight line, whereas for q = 3
components, the factor space is an equilateral triangle, and
for q = 4 components, the factor space is represented by a
regular tetrahedron.
The objectives in the analysis of a mixture response
system are, in general, the same as in any response surface
exploration. That is, one seeks to approximate the surface
with a model equation by fitting an equation to observations
taken at preselected combinations of the mixture com
ponents. Another objective is to determine the roles played
by the individual components. We shall not concern our
selves with this but rather concentrate on the empirical
model fit. Once the model equation is deemed adequate an
attempt is made to determine which of the component combina
tions yield the optimal response. The models used to repre
sent the response in a mixture system are in most cases
different in form from the standard polynomial models. The
first type of model form that we discuss is the canonical
polynomial suggested by Scheffe.
1.2.1 Mixture Models
Scheffe (1958) introduced a canonical form of the poly
nomial model for representing the response in a mixture
system. These canonical polynomial models are derived from
the standard polynomials using the restrictions on the xi
shown in (1.2). With q = 2 mixture components, for example,
the standard second order polynomial model is of the form
2 2
E(Y) = a0 + alx1 + a2x2 + a12 x2 + all x + a22x2 (1.3)
Restrictions (1.2) imply that a0 = a0(xl + x2)'
x2 = l( and x2 = 2( thus (1.3) can be
1 xl(l x2), x2 x2(l xl)
written in the canonical form
E(Y) = 81x1 + 2x2 + 12XlX2
where 1= a0 + al + all' 82 = a0 + a2 + a22' and 812= al2
 a11 a22. There is no constant term in the above canoni
cal form and the pure quadratic terms in equation (1.3) have
been absorbed in the xixj terms.
The general form of the canonical polynomial of degree
d in q mixture components can be written as
q
E(Y) = E 8ixi, for d = 1,
i=l
q q
E(Y) = Z i.x. + Z E .x.x. for d = 2, and
i=l 1. i
q q q
E(Y) = a .x. + EZ .x.x. + E E 6..x.x.(x. x.)
i=l 1 i
q
+ E a ijkx.x.xk for d = 3. (1.4)
1Ki
The fourth degree canonical polynomial in q components is
given in Cornell (1981, p. 64). The general canonical poly
nomial of degree d > 4 in q components does not explicitly
appear in the literature, but is mentioned in Scheffe
(1958). If terms of the form 6ijxixj(xi xj) are removed
from the full cubic model (1.4), then the remaining terms
make up what is referred to as the special cubic model. For
example, for q = 3 components, the special cubic model is
E(Y) = 1x1 + 82x2 + 3x3 + a12 2 + 813X1X3
+ 823x2x3 + 123XlX2X3
Scheffe's canonical polynomial models are used for
approximating the response surface in many mixture systems.
Their popularity stems from the ease in interpreting the
coefficient estimates, especially when the models are fitted
to data collected at the points of the associated designs
(see Section 1.2.2). However, other models have been intro
duced which seem to better represent the response when the
components have strictly additive blending effects. We
present some of them now.
Becker (1968) introduced three forms of homogeneous
models of degree one which he recommends instead of the
polynomial models when one or more of the mixture components
have an additive effect or when one or more components are
inert. A function f(x, y, ..., z) is said to be homogeneous
of degree n when f(tx, ty, ..., tz) = tnf(x, y, ..., z), for
every positive value of t and (x, y, ..., z) (0, 0, ...,
0). These models, which Becker refers to as models HI, H2,
and H3, are of the form
q q
HI: E(Y) = Z B.x. + Z Bijmin(xi, x ) + ...
i=1 1 i
+ 12...qmin(xl, x 2, ..., x )
q q
q q 21
H2: E(Y) = E Bx. + Z Bix. .x ./(x. + x .) +
i= 13 1 3 1
i=1 14i
+ 812...qxlx2..x /(x + x2 + ... + xq)q
q q 1
H3: E(Y) = 6 .x. + Z . (x.x 1/2
i=1 l i
+ Bl2...q(X l2...xq)/q
Each term in the H2 model is defined to be zero when the
denominator of the term is zero.
Draper and St. John (1977) suggest a model which in
cludes inverse terms, 1/xi, in addition to terms in the
Scheffe polynomials. Such a term is used to model an
extreme change in the response as xi approaches zero. The
experimental region of interest is assumed to include the
region near the zero boundary (xi = 0), but does not include
the boundary itself. One example of this type of model is
the Scheffe linear polynomial model with inverse terms
q q 1
E(Y) = Z Bixi + Z B .x .
i=l i=l 
Another model form that is useful in the study of the
response in a mixture system is the model containing ratios
of the component proportions. A term such as xi/xj measures
the relationship of xi to xj rather than the percentage of
each in the blends. Snee (1973) points out that the ratio
model presents a useful alternative to the Scheffe and
Becker models in that the ratio model describes a different
type of curvature. He notes that the curvilinear terms for
the Scheffe and Becker models, when plotted as a function of
xi, are symmetric functions about xi = 1/2, whereas the
ratio term xi/xj is a monotone function when plotted against
xi.
The terms in the ratio models may also contain sums of
the components. For example, with q = 3 components, we
might express the second order model
q1 q1 q1
E(Y) = B + E izi + EE ijziz. + E 68iz.
i=l l
(note the constant term) where zl and z2 are defined as
z1 = xl/(x2 + x3) and z2 = x2/x3. Some terms will be unde
fined if points from the boundary of the experimental sim
plex are included in the design, for example, if x3 = 0,
then z2 = x2/x3 is not defined. Snee (1973) suggests adding
a small positive quantity, c, to each xi in this case.
This, of course, will not be of concern if the experimental
region is entirely inside of the simplex.
When one or more of the components is inactive, Becker
(1978) suggests that a ratio model that is homogeneous of
degree zero in the remaining components is appropriate. In
three components, such a model is of the form
E(Y) = 80 + 1X/(x1 + x2) + 2x2/(x2 + x3)
3
+ 83x /(x + x ) + Z BZ ..h..(x., x )
s3 3 1 3 1i
li
+ 8123h123(xl, x2, x3), (1.5)
where hij and h123 are specified functions that are homoge
neous of degree zero. The function h123 is intended to
represent the joint effect of all three components simulta
neously. If in fitting a model of the form (1.5) we deter
mine the model should be
E(Y) = 80 + a1X1/(X1 + x2) + 812h12(xl, x2)
then component three is said to be inactive and is removed
from further consideration. The model of equation (1.5) may
produce an extreme value near the vertices of the simplex
factor space when there are no inactive components. In this
case it is suggested that a model of the form (1.5) be used
only when the proportions are restricted so that no two of
the xi are simultaneously very close to zero. Becker notes
that other authors who have suggested ratio models have also
used them primarily over a subregion inside the simplex
factor space. Apparently this is where they are most appro
priate.
1.2.2 Experimental Designs for Mixtures
As in the general response surface problem, one of the
major concerns in exploring a mixture system is that of
choosing the experimental design for collecting observed
values of the response that will be used in fitting the
model. Scheffe (1958) proposed the {q,m} simplex lattice
designs for exploring the entire qcomponent simplex factor
space. In these designs, the proportions used for each
component have the m + 1 values spaced equally from zero to
one, xi = 0, 1/m, 2/m, ..., (m l)/m, 1, and all possible
mixtures with these proportions for each component are
used. The number of design points in the {q,m} simplex
lattice design is (m + q 1). The main appeal of these
m
designs is that they provide a uniform coverage of the fac
tor space. Another feature, which Scheffe (1958) demon
strates, is that the parameters of the canonical polynomial
of degree m in q components are expressible as simple linear
combinations of the true response values at the design
points of the {q,m} simplex lattice. The {3,2} simplex
lattice, which consists of six design points, is represented
on the two dimensional simplex in Figure 1 along with the
triangular coordinates (xl, x2, x3).
Scheffe (1963) also developed the simplex centroid
designs consisting of 2q 1 points, where the only mixtures
considered are the ones in which the components present
appear in equal proportions. That is, in a qcomponent
simplex centroid design, the design points correspond to the
q
q permutations of (1, 0, 0, ..., 0), the () permutations of
(1/2, 1/2, 0, ..., 0), the (3) permutations of (1/3, 1/3,
1/3, 0, ..., 0), ..., and the point (1/q, l/q, ..., l/q).
This design alleviates the problem inherent in the {q,m}
simplex lattice designs of observing responses at mixtures
containing at most m components. To give an example, the
q = 3 simplex centroid design is made up of 23 1 = 7
design points, and is equivalent to the {3,2} simplex
lattice design augmented by the center point (xl, x2, x3)
(1/3, 1/3, 1/3). This design is represented in Figure 2.
Scheffe (1963) mentions that the number of parameters
in the polynomial model of the form
q q q
E(Y) = .ix. + EE B. .x.x. + EE 8ikx. xxk
i=l 1 i
+ ... + B12...qxlx2 ... q (1.6)
is 2q 1 and therefore these models have a special rela
tionship with the simplex centroid design in q components.
This relationship is that the number of terms in the model
equals the number of points in the design and as a result
the parameters in model (1.6) are expressible as simple
functions of the responses at the 2q 1 points of the sim
plex centroid design. Polynomial models of the form (1.6)
(2 2
(0,1,0)
Figure 1.
(
(0,1 ,0)
(0,0,1 )
x =I ( x, I )
2 2'2 3
The {3,2}
simplex lattice design.
x =I
(0,0,1)
X3
The q = 3 simplex centroid design.
x I)
~~2 2'2
Figure 2.
therefore are natural models to fit using the simplex cen
troid design.
Ratio models may be desirable when the interest in one
or more of the mixture components is in terms of their rela
tionship to one another, rather than in terms of their per
centages in blends. Kenworthy (1963) proposed factorial
arrangements for ratio variables. An example of the use of
ratios is the following three component system in which the
mixture components are constrained by the upper and lower
bounds:
.2 < x1 < .4, .2 < x2 < .4, .3 < x3 < .5. (1.7)
The ratio variables of interest are zl = x2/xl and
z2 = x2/x3, and it is desired to fit either a first or a
second order polynomial model in zI and z2. For such a
problem, we can define a 22 and a 32 factorial design that
can be used for fitting the first and second order poly
nomial models, respectively, by taking as design points the
intersection of rays passing from two of the three vertices
of the twodimensional simplex through the region of
interest defined by the constraints (1.7). Kenworthy's 22
factorial design is shown in Figure 3.
Becker (1978) uses rays extending from one or more
vertices of the simplex factor space to the opposite bound
aries in developing "radial designs." These designs are
suggested for detecting the presence of an inactive
x :
0 Design Points
x :1 x :I
2 3
Figure 3. Kenworthy's 22 factorial design.
component or in another case a component which has an addi
tive effect, when models containing ratio terms that are
homogeneous of degree zero are fitted.
McLean and Anderson (1966) suggest an algorithm for
locating the vertices of a restricted region of the simplex
factor space which is defined by the placing of upper and
lower bounds on the mixture component proportions. The
vertices of the factor space and convex combinations of the
vertices are the candidates for design points for fitting a
first or second degree polynomial model in the mixture com
ponents. One drawback of the "extreme vertices" design is
that the design points are not uniformly distributed over
the factor space resulting in an imbalance in the variances
of Y(x), see Cornell (1973).
Another method that has been suggested for studying the
response over a subregion of the simplex mixture space is
to transform the q mixture components into q 1 independent
variables. Transforming to an independent variable system
was first suggested by Claringbold (1955) and later proposed
by Draper and Lawrence (1965a, 1965b) and Thompson and Myers
(1968). Standard response surface polynomial models in the
transformed variables can be fitted to data values collected
on standard designs and a design criterion such as the aver
age mean square error of the response can be employed when
distinguishing between designs. Thompson and Myers (1968)
suggest the use of rotatable designs (see also Cornell and
Good, 1970).
Designs other than rotatable designs, such as multiple
lattices and symmetricsimplex designs, to name a few, have
been suggested in the literature for fitting models to a
mixture system which may be appropriate depending on par
ticular experimental situations. However, as the intent
here is not to give an exhaustive list but only a sampling
of available designs, we shall not discuss designs further
but instead state the purpose of this work.
1.3 The Purpose of this Research:
Investigation of Procedures tor Testing a Model
Fitted in A Mixture System for Lack of Fit
A common problem in modeling the response in a mixture
system is that of detecting lack of fit, or inadequacy, of a
fitted model of the form E(Y) = XI when the true model is
of the form E(Y) = X2 + X2 2. The statistical literature
suggests several procedures for testing lack of fit, which
will be described in Chapter Two. In general, the authors
of these procedures are not specific in stating hypotheses
to be tested and do not adequately discuss the power of
their procedures.
The major purpose of this research is to investigate
the power of two of the testing procedures appearing in the
literature in detecting the inadequacy of a fitted model
when the general form of the true model is specified. Our
findings for a "check points" lack of fit testing procedure
are presented in Chapter Three while Chapter Four contains
findings for a "near neighbor" lack of fit testing proce
dure. For both procedures, explicit formulas for the power
of the test are given in terms of cumulative probabilities
of either the noncentral F or doubly noncentral F distribu
tion, which are derived by assuming that the response obser
vations are independent and normally distributed. Addition
ally, we propose methods for maximizing the power of the
testing procedures. In the final chapter, we make some
concluding comments concerning both of these procedures.
CHAPTER TWO
LITERATURE REVIEWTESTING FOR LACK OF FIT
2.1 Introduction
Let us return to the general response surface problem
and assume the true response is to be approximated by
fitting a model of the form
E(Y) = XI1 (2.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, and El is a pxl vector of
unknown regression coefficients. We wish to consider the
situation in which the true model contains terms in addition
to those in the fitted model. Then the true model has the
form
E(Y) = X1 + X2a2 (2.2)
where X2 is an Nxp2 matrix of known constants, and 12 is a
P2xl vector of unknown regression coefficients. We assume
that the vector Y has the normal distribution with
var(Y) = a 2N
It is desirable to determine the suitability of the
fitted model given by Eq. (2.1) when in reality the true
model is of the form given by Eq. (2.2). The process of
19
making this determination is referred to as testing for lack
of fit of the fitted model.
There are three general approaches to testing for lack
of fit. The first approach requires that there be replicate
observations of the response at one or more design points,
and involves partitioning the residual sum of squares from
the fitted model into a sum of squares due to lack of fit
and a sum of squares due to pure error. A large value for
the ratio of the mean square due to lack of fit to the mean
square due to pure error provides evidence for lack of fit.
If replicate observations are not available then the
above approach to testing for lack of fit cannot be used.
Green (1971), Daniel and Wood (1971), and Shillington (1979)
have proposed alternative methods that are applicable in
this case. Their approach is to group values of the
response which are observed at similar settings of the
independent variables and to call these grouped values
"pseudoreplicates" or "near neighbor observations." They
then treat these pseudoreplicates as they would treat true
replicates to form statistics for lack of fit testing,
although arriving at their respective statistics through
different approaches.
A third approach to testing for lack of fit involves
the use of "check points." In this method a model of the
form (2.1) is fitted to data at the design points and
additional observations are collected at other points in the
experimental region. The additional points other than the
design points are called check points, and the data at these
check points are not used in fitting the model. Lack of fit
is tested by comparing the values of the response observed
at the check points to the values of the response which the
fitted model predicts at these same check points.
We now discuss the first method mentioned above of
testing for lack of fit which involves partitioning the
residual sum of squares.
2.2 Partitioning the Residual Sum of Squares
The method for testing lack of fit which makes use of a
partitioning of the residual sum of squares from the fitted
model requires there be replicate observations of the
response at some of the design points (Draper and Smith,
1981, p. 120). When a model of the form (2.1) is fitted,
the residual sum of squares is defined as
n.
n 1 2
SSE = EZ (Y Y )
i=1 j=l 1
1
=Y'(I' X(X'X) X')Y
where n is the number of distinct design points, ni > 1 is
the number of replicate observations at the ith design
point, Yij is the jth observed value of the response at the
ith design point, Yi is the value which the model of the
form in Eq. (2.1), fitted by ordinary least squares
techniques, predicts for the response at the ith design
n
point, and N = Z n. Using the replicated observations
i=l 1
only, a pure error sum of squares can be calculated as
n "i 2
SSE = E E (Y. Y. )
pure i=l 13 '
i=1 j=1
where Yi. is the average of the values of the response
observed at the ith design point. The sum of squares due to
lack of fit can be obtained by taking the difference
LOF
=SSE SSE
pure
This partitioning of the residual sum of squares is
displayed in the analysis of variance table in Table 1.
Table 1. Analysis of Variance
Partitioning the Residual Sum of Squares.
Source
of Variation
Regression
Residual
Pure Error
Lack of Fit
Total(corrected)
Sum
of Squares
b{X'Y (1'Y)2/N
SSE
SSEpure
SSLOF
Y'Y (l'Y)2/N
Degrees
of Freedom
p 1
N p
N n
n p
N 1
bl represents the ordinary least squares estimator of B in
1
model (2.1), b = (X'X) X'Y, and 1 is an Nxl vector of
ones.
ones.
Mean
Square
MSE
MSEpure
MSLOF
To test the hypothesis of zero lack of fit, that is
HO: lack of fit = 0 or E(X) = XEI, an F statistic is formed
MS
LOF
F = F (2.3)
MSE
pure
which possesses a central F distribution if the true model
is of the form (2.1), but has a noncentral F distribution if
the true model is of the form (2.2). In other words
F ~ F
np,Nn
under H : E(Y) = X8_ and
F ~ F'
np,Nn;X2
under H : E(Y) = XB + X2_ where X2 is the noncentrality
a 1+ 22 2
parameter 2 = B(X2XA)'(X2XA)B2/22 and A = (X'X) X'X2
Under H E(MSLOF) = 2 + (X2 XA)'(X2 XA) 2/(np) and
E(MSEpure) = o2 (Draper and Smith, 1981, p. 120), hence HO
is rejected in favor of Ha if the value of F in (2.3)
exceeds the upper 100a percentage point of the central F
distribution, Fa;np,Nn. When HO is rejected, we conclude
that a significant lack of fit is present.
Draper and Herzberg (1971) demonstrated that the lack
of fit sum of squares can be partitioned into two
statistically independent sums of squares, SSL1 and SSL2'
when there are replicate observations at the center of the
response surface design and when the basic design without
center points is nonsingular. If the true model and the
fitted model are of the same form as in equation (2.1) then
the two F ratios FL1 = [SSLl/(n p 1)]/MSEpure and
FL2 = SSL2/MSEpure are both distributed as central F random
variates, with respective numerator and denominator degrees
of freedom (n p 1), (N n) for FL1 and 1, (N n) for
FL2. If the true model is of the form shown in equation
(2.2), then FL1 and FL2 are both distributed as noncentral F
random variates. The expected values of SSL1 and SSL2 are
used to show what functions of E2 are testable with FL1 and
FL2*
Two examples are presented by Draper and Herzberg to
illustrate this testing for lack of fit. The first example
makes use of a first order orthogonal design in k factors
augmented with center point replicates for fitting a first
order polynomial model. If the true model is of the second
order, then FL2 can be used to test a hypothesis concerning
the parameters associated with the pure quadratic terms in
the model. If all such parameters are zero, then FL1
provides a check on interaction terms. The second example
illustrates the fitting of a second order polynomial model
to a second order design with all odd design moments of
order six or less zero. If the true model is third degree,
then FL1 can be used to test the significance of the third
order terms, while FL2 tests terms of order greater than
three. The partitioning of SSLOF into SSL1 and SSL2 and the
corresponding tests of hypotheses are also given in Myers
(1971, p. 114119), for the special case of fitting a first
order polynomial model to a 2q factorial or a fraction of a
2q factorial design augmented with center point replicates
and the true model is of the second degree.
A more complete partitioning of the lack of fit sum of
squares in an attempt to obtain a more detailed diagnosis of
the lack of fit of the fitted model is given in a technical
report written by Khuri and Cornell (1981). The lack of fit
sum of squares, which has n p degrees of freedom, is
partitioned into n p independent sums of squares, each
having one degree of freedom. The expected values of these
single degreeoffreedom sums of squares are used to
identify at most n p linearly independent causes for the
lack of fit variation. Tests of significance are performed
on the assumed contributing causes. This method enables the
screening of all subsets of E2 in order to identify those
subsets which are most responsible for lack of fit of the
fitted model.
We shall now discuss the second general approach used
in lack of fit testing, which is to test for lack of fit by
making use of response values observed at points which are
near neighbors in the factor space when true replicate
observations are not available.
2.3 Testing for Lack of Fit Without
Replicated ObservationsNear Neighbor Procedures
Green (1971) suggests the following approach when
testing for lack of fit if there are no design points at
which replicate observations of the response are
available. The N observed values of a response, Y,
considered a function of only one variable, x, are divided
into g groups, by grouping observations which have similar
values of x. Green hypothesizes a model of the form Y= Ha +
e, where Y is an Nxl vector of observable responses, H is an
Nxm matrix whose columns correspond to known functions of
the variable, x, is an mxl vector of unknown regression
coefficients, and e is the Nxl vector of random errors,
2
N ~ N (0, o N).
Green's method assumes that the vector of differences
(EY Hg) can be well approximated by a dth order polynomial
in x within each of the g groups, d > 1. An alternative
model of the form
Y = H v + n +
is given, where is distributed as NN(Q, o21N), H1 is an
Nx [g(d + 1) + ml] matrix of known constants, u is a
[g(d + 1) + ml]xl vector of regression coefficients, and .,
as Green states is "a small vector." The first g(d + 1)
columns of H1 correspond to the polynomial terms for the g
groups (with (d + 1) terms for each group), the rightmost
mI < m columns in H1 correspond to terms that are in the
fitted model, but are not represented among the g(d + 1)
polynomial terms in the alternative model.
Under the assumption that a = Q, the presence of lack
of fit is tested by using the test statistic:
Y'[H (HHl) H H(H'H) 1H']Y/[g(d + 1) + mi m]
F = *
1
Y'[I H (HH1I) H{]Y/[N g(d + 1) mi]
(2.4)
This statistic is of the same form as the F statistic used
in the standard multiple regression test of a postulated
model against a more general one which includes the
postulated model as a special case. Lack of fit is
suspected if the calculated F ratio in (2.4) is greater
than Fa;g(d+l)+mlm, Ng(d+l)ml where this latter quantity
is the upper 100a percentage point of the central F
distribution.
Green notes that when there is no lack of fit, the
quadratic forms Y'[H1(HIH ) H H(H'H) H']Y and
Y'[I H (HIH,) Hi]Y are distributed independently as
o2X2 with g(d + 1) + mi m and N g(d + 1) mi degrees of
freedom, respectively. In this case the F ratio in (2.4)
possesses a central F distribution. If there is lack of fit
on the other hand, then these two quadratic forms are
distributed as noncentral chisquares, multiplied by 02,
with respective noncentrality parameters
I = [H + '[H (HiH1) H' H(H'H) H'][H + n]
and ; = n'[I H1(HH ) HI]n Thus the assumption that
n = 0 can affect the power of the test, since if n # 0 the
expected value of MSE is greater than a2, where MSE is the
quadratic form in the denominator of the F ratio. Hence if
n 0 the probability of calculating a large F value is
reduced, and we are less likely to detect lack of fit using
an upper tailed rejection region.
Daniel and Wood (1971) suggest another method for lack
of fit testing when replicated observations of the response
are not available. They make use of "near replicates" to
obtain an estimate of a, which is the standard deviation of
the observable responses in the true model. The value of
the estimate a is compared to the square root of the
residual mean square from the analysis of the fitted
model. Lack of fit is indicated if the square root of the
residual mean square is large compared to the estimate o.
To determine when observations are near replicates so that
an estimate of a can be found, they define the squared
distance between any two data points, j and j', to be
measured by
K
D2 = x. ij.)/5y]2
3J' i=1
where xij and xij, are the values of the ith independent
variable corresponding to the observations yj and yj,,
respectively, i = 1, 2, ..., K, and bi is the ordinary least
squares estimate of the ith regression coefficient. In the
denominator, s is the square root of the residual mean
square for the fitted model.
To obtain an estimate of a from near replicates, let
And = dj d ,j, n = 1, 2, ..., (2), where dj and d are
the residuals at points j and j', respectively, and where
there are N data observations in the experiment. Since the
expected value of the range for pairs of independent
observations from a normal distribution is 1.1280, a running
average of the And's is calculated and their average is
multiplied by .886 = (1/1.128) to get a running estimate,
sn, of a. That is, s = .886 n And/n The closest pair
of observations as judged by Dj, is used to begin the
running estimate, the next closest pair (next "nearest
neighbors") is used for A2d, and the procedure continues
until sn "stabilizes." The stabilized value of sn is used
to estimate a.
A third method for testing for lack of fit without
replication is given by Shillington (1979). The fitted
model is of the form
Y = XB + E (2.5)
where Y (Nxl), X (Nxp), and (pxl) are defined as in (1.2)
and E ~ N (0, a IN) The test for lack of fit of the
fitted model is a test for whether the true model has the
form
y = X1 + 6 + E 1
where 6 (Nxl) is a fixed effect quantifying the departure of
(2.5) from the true model.
Shillington assumes that the data can be grouped into g
cells, with nj observations in the jth cell, determined in
advance. Letting Cj refer to the jth cell, j = 1, 2, ...,
g, a vector of cell averages is written YC (gxl), where the
jth element of YC is the average of the observed responses
in Cj. The matrix XC of independent variables associated
with Y is the gxp matrix where the elements in the jth row
n.
are x'. = i x!./n. that is, row j of X is the row
i. =1 
vector x'. The matrix XC is assumed to be of full rank
p < g. Also within each cell are defined the differences
W.. = Y.. Y. i C. j = 1, 2, ..., g, where Y is
the jth element of Yc.
The two independent data sets, YC and {Wij} with g and
N g degrees of freedom, respectively, are used to find two
independent estimates of o2. The first estimate is written
as
9 2
MSEB = E n (Y x' /(g )
j=l B
where g4 is the weighted least squares estimate of g using
the regression of cell means, YC, on XC. The second
estimate of o2 uses the within cell deviations on cell
means, {Wij}, and is
n.
g n 2
MSEJ = Z (W. W. ) /(N g r),
j=1 i=l
where r is the rank of an Nxp matrix with rows equal to
x! x' i C C., j = 1, 2, .., g. If the matrix of
13j .j 3
independent variables, corrected for cell means, is of full
rank, then r = p. Here 7ij is the estimate of Wij from the
regression of cell residuals {Wij} on the associated vectors
of independent variates, x'. x'.
If the fitted model is the correct model, then MSEB and
MSEW are independent estimates of a2 and the ratio MSEB/MSEW
is an F statistic with g p and N g r degrees of
freedom. When all observations in a cell have the same
settings of the independent variables, that is, the
observations are truly replicates for all cells, then this F
statistic is identical to the F statistic in the usual lack
of fit test in which the residual sum of squares is
partitioned into lack of fit and pure error sums of squares,
as given in Draper and Smith (1981, p. 120).
If the true model is Y = X8 + 6 + E however, and if
we let X'6 = 0 and 2 = 6/N, then
E(MSEB) = o2 + [I XC(X'XC)1'] B/(gP)
n.
where B (gxl) has jth component equal to E 6../n .
B i=
Furthermore, with this latter true model form
E(MSE ) = 02 + 67(I XW(XJ XW) X)6W/(N g r)
where _W has the components ij .j, i e Cj, j = 1, 2,
..., g. The matrix XW (Nxp) has the rows x x'
i e C., j = 1, 2, ..., g. The power of the F test,
F = MSEB/MSEW, depends on the relative bias of the estimates
of 2, that is, the biases in MSEB and MSEW.
Shillington states that the power of the F test which
makes use of F = MSEB/MSEW is maximized by forming cells so
that the bias of E(MSEW) is minimized. This is the same as
forming cells so that the within cell variation in 6 is
minimized. Shillington (1979, p. 141) also states,
"Observations with near covariate (independent variable)
values might be expected to have similar 6 values, since we
assume that 6 varies in some continuous but unknown fashion
with X. This justifies the usual procedure of forming
groups by collapsing observations with adjacent covariate
values. Indeed, if covariates do not vary within cells we
have the usual lack of fit test and maximum power."
By imposing a further structure on the form of 6, it is
shown that if the F test has an upper tailed rejection
region, the power is maximized by selecting the group sizes
as nj = 2, j = 1, 2, ..., g. Finally, Shillington suggests
that in the presence of more than one independent variable
problems in grouping may arise, and in this case it may be
wise to perform a different lack of fit test for each
parameter. Following this approach, an example is given
which suggests testing lack of fit for each of the p
independent variables separately may be more powerful than
trying to form groups based on all independent variables at
once.
In summary, all the approaches we have discussed for
testing for lack of fit when replicate observations of the
response are not available at any of the settings of the
independent variables make use of grouping the observed
response values according to similar values of the
independent variables. The observations falling in such
groups are referred to as "pseudoreplicates" or "near
neighbor observations." These pseudoreplicates are used to
estimate the true variance of the observations, a2, but a
completely unbiased estimate of 02 cannot be attained unless
true replicate observations are available. In each case,
the power of the lack of fit testing procedure is reduced
because an unbiased estimate of a2 is not attainable. We
now turn to the use of check points for lack of fit testing.
2.4 Testing for Lack of Fit with Check Points
An alternative to the two approaches to lack of fit
testing already discussed is the method which makes use of
check points. We assume a model of the form E(Y) = Xl as
given in (2.1), is fitted in a response surface system, but
that the true model is of the form E(Y) = X1I + X22 as
given in (2.2). The parameters, a8, in the fitted model are
estimated by ordinary least squares techniques, making use
of the values of the response observed at the design
points. After the model is fitted, values of the response
are observed at additional points in the experimental region
called "check points." The observed response values at the
check points are compared to the values which the fitted
model predicts at these same check points. It is important
to note that the observed values of the response at the
check points are not used in fitting the model initially.
Snee (1977) gives four methods of validating regression
models, one of which is the collection of new data to check
predictions from a previously fitted model. In a designed
experiment these new data take the form of check points.
Snee suggests that the inclusion of a small number of check
points in any designed experiment is a "worthwhile"
procedure.
Scheffe (1958) proposed a test for lack of fit when the
{3,2} simplex lattice design is used for fitting a second
order canonical polynomial model in three mixture
components. It is desired to use the observed value of the
response at (1/3, 1/3, 1/3) as a check point blend. The
test statistic proposed is the t statistic of the form
Y Y
t = (2.6)
[var(Y )]
where Y is the observed value of the response at the check
point, and Y is the value of the response predicted at the
same point by the second order model which is fitted by
ordinary least squares techniques to the observed response
values at the six design points of the {3,2} simplex
lattice. The response value observed at the point
(1/3, 1/3, 1/3) is not used in fitting the model. Lack of
fit is inferred if the absolute value of the calculated t
value in equation (2.6) is larger than the corresponding
tabled t value.
In the denominator of the t test of equation (2.6), the
variance of the difference Y Y is shown to be
var(Y Y) = var(Y) + var(Y)
= (44/27r)o
when r replicates are taken at each design point. The
estimate of the variance of Y Y is (44/27r)o2, where o2 is
calculated from the replicated response values at the design
points.
Scheffe (1958) also alludes to a test for lack of fit
when several check points are used simultaneously. When
there are k check points, the test for lack of fit is an F
statistic of the form
1
F = 2 (2.7)
ka
where d' = (Y Y 2 2' ... Yk Yk) and V = o2V
var(d). Formulas are given for the elements of VO in the
special case when the check points are the design points of
the {3,2} simplex lattice. Lack of fit is suspected if the
calculated value of the F statistic given in (2.7) is larger
than the corresponding tabled F value.
Gorman and Hinman (1962) suggest the same t test in
equation (2.6) that Scheffe (1958) suggested for a check
point taken at (1/3, 1/3, 1/3) to test for lack of fit in a
second order polynomial model fitted from a {3,2} simplex
lattice design. They suggest using (1/3, 1/3, 1/3) as the
location of the check point because the observation at this
point may later be used to fit the next more complex model,
the special cubic, if the second order model is found to be
inadequate. They state that in general for the second order
polynomial model as well as higher order models, check
points should be taken in regions of particular interest, of
which there are usually many in any blending study.
Further, they suggest that the number of check points
depends on individual experimental situationstechnical
background, precision required, cost of materials and
analyses, and probability of requiring a more complex
model. However, no specific criterion is given by Gorman
and Hinman for selecting the location of the check points.
Gorman and Hinman (1962) indicate that a t test at a
check point other than at (1/3, 1/3, 1/3) takes the same
form as the statistic of equation (2.6),
Y Y
t 1/2
[var(Y) + var(Y)]/2
with the additional condition that if several check points
are taken, say for example k points, the method of checking
the fit is to compute the t value at each location and refer
these calculated t values to the 100(a/2k) percentage point
of the central t distribution rather than the 100(a/2)
percentage point.
Kurotori (1966) gives an example of a mixture
experiment where the response is the modulus of elasticity
of a rocket fuel, which is a mixture of three components,
binder (xl), oxidizer (x2), and fuel (x3). The factor space
of feasible mixtures is a subspace inside the two
dimensional simplex or triangle where all three components
are present simultaneously. "Pseudocomponents" are defined
and in the pseudocomponent system a special cubic model is
fitted to data collected at the points of the q = 3 simplex
centroid design (Figure 4). A check for adequacy of fit is
made by using three check points and the response values at
the check points are used only for testing the fit of the
model and not for fitting the model initially.
The reason for the choice of the particular check point
locations by Kurotori is that, as he states, "They are the
most remote mixtures from the seven design points." The
lack of fit test is an F statistic of the form
2
F = (2.8)
2 3
where s (Y Y ,) for the i = 1, 2, 3 check points
2 i=l
and 02 is an estimate of measurement error from a previous
analysis. Kurotori admits that the use of the F statistic
x :1
I
(1,0,0)
9 Design Points
O 0 Check Points
SE 3 ') o
S 2 I 112
e(o,,o) a the ser)
2 '2 '2)
Figure 4. Kurotori's rocket fuel example,
xI', x2', and x3' represent pseudocomponents.
in Eq. (2.8) for lack of fit testing may be risky because
the predicted values at the check points are correlated
(correlation of .5), although the observed values are not
correlated. Kurotori suggests individual t tests as
proposed by Scheffe (1958) might be the preferred procedure.
Snee (1971) repeats Kurotori's rocket fuel example
using the same F test for lack of fit as Kurotori and makes
the comment that the Yi's at the check points are
correlated. In stating that the F test is not an exact
test, he nevertheless offers no solution in the form of an
exact test.
39
In summary, only Scheffe refers to an exact F test when
several check points are considered simultaneously for
testing for possible lack of fit of a model fitted in a
mixture space, and his development is limited to the special
case where the check points are the design points used to
fit the model initially. No criterion is proposed by
Scheffe for selecting other locations for the check points.
CHAPTER THREE
AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL
3.1 Introduction
In Chapter Three we investigate the problem of testing
for lack of fit of a linear model fitted in a mixture
space. The testing is to be accomplished with the use of
check points. We assume that an experimental design is
specified, and that the fitted model is of the form
E(Y) = X1 (3.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants and rank p, and 81 is a
vector of p unknown regression coefficients. The true model
is assumed to be of the form
E(Y) = X1 + X 2 (3.2)
1 22
where X2 is an Nxp2 matrix of known constants and g2 is a
vector of p2 unknown regression coefficients. Throughout
our development, we will assume that the random vector Y has
the normal distribution with variancecovariance matrix
equal to a IN.
In our investigation we wish to determine the proper
testing procedure to follow in deciding whether the fitted
model exhibits lack of fit. In order to optimize the lack
of fit testing procedure, we will determine the location of
the check points so that the power of the test is maximized.
3.2 Testing for Lack of Fit in the Presence of
an External Estimate of Experimental Error Variation
3.2.1 The Test Statistic
We wish to test the performance or fit of a fitted
model in a mixture space when the true model possibly
contains terms in addition to those in the fitted model.
The fit of the model is to be tested by a test which makes
use of the response values observed at certain locations
called "check points" in the experimental region, by
comparing them to the values which the fitted model predicts
at the same check points. The observed values at the check
points are not used for estimating the coefficients in the
fitted model and are assumed to represent the values of the
true surface at the check points.
Let us define the vector of differences
d = (Y* Y*)
(Y* Y Y Y* Y* y*)i
1 1' 2 21 k k
where Y* = 1, 2, ..., k are observed response values at
k check points and YV, i = 1, 2, ..., k are response values
predicted at the k check points by the fitted model,
VY = xt'b where b is the ordinary least squares estimator
of al and where x*' is the ith row of X*, the kxp matrix
1 1
whose columns are of the same form as the columns of X but
with its rows evaluated at the k check points. Note that
if 3 = 0, then E(d) = 0 and if a 2 0, then
2 2 2
E(d) = (X2 X*(X'X)X'X2). Let V represent the
variancecovariance matrix of the random vector d.
Then V = o V where
1
V = Ik + X*(X'X) X*'
and where Ik is the identity matrix of order kxk.
We assume that an unbiased estimate of a2 is available
^2
and we denote this estimate by a where the subscript ext
Sext'
^2
stands for external, and a is independent of the model
being fitted. The test statistic for the hypothesis of zero
lack of fit H : E(d) = 0 is
1
d'V d/k
0
F = (3.3)
^2
ext
(see Scheffe, 1958, p.358). It will be shown later in this
section that the F ratio in Eq. (3.3) possesses either a
central F distribution or a noncentral F distribution,
depending upon whether the true model is represented by Eq.
(3.1) or Eq. (3.2).
^2
The variance estimate a that appears in equation
ext
(3.3) is ordinarily generated from replicated observations
at some of the design points in the experiment. We assume
^2
that aext is a constant multiple of a central chisquare
random variable with v degrees of freedom. This is written
as
^2
a e = SSE /v
ext pure
2 2
= (a 2/v)(SSEpe/
pure
2 2
where SSEpure/ ~ X. Note that SSEpure denotes the
portion of the residual sum of squares due to replication
variation from the fitted model. The residual sum of
squares from the fitted model may be partitioned into
SSEpure and SSLOF only if replicated observations are
collected at one or more design points. For the case where
replicate observations are collected at all of the design
points
n n.
SSE = Z (Yij Yi. )
i=l j=1
where n is the number of distinct design points, n. > 2 is
1
the number of replicates at the ith design point, Yij is the
jth observation at the ith design point, and Y. is the
1.
average of the ni observations at the ith design point.
n
Here SSEpure has v = Z (ni 1) degrees of freedom.
i=l1
When the fitted model and the true model are of the
same form as defined by Eq. (3.1), the quantity d'Vld/a2
(3.lr th quntit d'
possesses a central chisquare distribution (Searle, 1971,
p.57, Theorem 2). However, when the true model is of the
1 2
form specified by Eq. (3.2), d'V d/a possesses a
noncentral chisquare distribution. Thus when the true
model is of the form in Eq. (3.1),
1 2 2
d'V d/ao 2 X
0 Xk'
but when the true model is of the form in Eq. (3.2),
1 2 2
d'V d/o2 ~ x ,
where in the second case the noncentrality parameter X has
the form
S= E(d)'V E(d)/2a2
1 0
1 2
= X* X*A)'Vo (X X*A) 2/2
2 2 0 2 2
1
The matrix A = (X'X) X'X is called the alias matrix and is
2
of order pxp2. In X1, the matrix X* is of order kxp2 and
has the same relationship to X2 as X* has to X.
2
Since SSE /a is statistically independent of
pure
1 2
d'V d/a then under model (3.1) the test statistic
0 0
1 2
d'V d/ko
F = 2
SSE pure/v
pure
1
d'V d/k
= ^2
ext
will have a central F distribution. When the true model
contains terms in addition to those in the fitted model then
F will have a noncentral F distribution. We write these two
cases as
F ~ Fk,
k,v
under model (3.1), and
F ~ F'
k,v;Xi
under model (3.2), where the noncentrality parameter is
x = (X X*A)'V0 (X X*A)_2/2 2.
3.2.2 The Testing Procedure and an Expression for the Power
of the Test
Given that the form of the fitted model is defined as
Eq. (3.1), the expected value of the numerator of the F
statistic in Eq. (3.3) will depend on the form of the true
model. For the case where the true model is expressed as
Eq. (3.2),
1
E(numerator) = E(d'V d/k)
0
2 2
= ( /k)EXx2
= (a /k)(k + 211)
2 2
= 2 + 20 X1/k
= o2 + BA1 /k, (3.4)
1
where A1 = (X* X*A)'V0 (X X*A). However, when the true
model is Eq. (3.1), 8 = 0 and in this case A = 0 so that
2 1
2 ^2
E(numerator) = 02. Also a is an unbiased estimator of
ext
o2 and
^2 2
E(a ) = a (3.5)
ext
Therefore the ratio E(numerator)/E(denominator) where
^2
the denominator is ext will equal unity under model (3.1),
that is, when there is no lack of fit. Under model (3.2),
the ratio will be greater than or equal to unity so lack of
fit should be suspected if the calculated F ratio in
equation (3.3) is large. We can thus use an upper tailed
rejection region to reject the hypothesis of zero lack of
fit. The power of the test is
PI k ,v ;X > a; k, v 1
P{Fv > F,; };k,I
where F is the upper 100a percentage point of the
a ;k,v
central F distribution with k numerator degrees of freedom
and v denominator degrees of freedom.
It is worth noting that from Eq. (3.4) and Eq. (3.5)
testing the hypothesis that 2 = 0 is equivalent to testing
the hypothesis that X1 = 0, assuming A1 is positive
definite. Thus testing a null hypothesis of zero lack of
fit using the proposed testing procedure involving the F
ratio in (3.3) may be expressed as a test of the hypotheses:
H: 1 = 0
H: a1 > 0.
a 1
3.2.3 A Method for Locating Optimal Check Points
Once a design for fitting model (3.1) in a mixture
space is chosen and the number of simultaneous check points
is decided on, say k > 1, the next step is to determine
where in the mixture space we should place the k check
points so as to maximize the power of the test for lack of
fit. The location of the check points is to be made
independently of the value of .
2
The power of the upper tailed F test for lack of fit is
an increasing function of X1 (see Appendix 1 for proof, with
X2 = 0). Therefore, to maximize the power of the test we
maximize the value of X1 defined as
1
Xl = S2AI82/2a2
where A = (X X*A)'V0 (X* X*A), by properly selecting
the k check points whose coordinates are defined in X*. To
maximize the value of X1, we shall concentrate on the matrix
A1.
The matrix A1 is a square matrix of order p2xp2 and is
a scalar quantity when p2 = 1. By maximizing the scalar
quantity A1 with respect to the k check points, the power is
maximized no matter what the value of .2 Maximizing the
2
scalar A1 can be accomplished by using The Controlled Random
Search Procedure given by Price (1977). This procedure is
described in Appendix 2. As a computational aid, A1 can be
expressed as
V + (X* X*A)(X* X*A)'
A = V2 1 (3.6)
1 V
when p2 = 1, where the symbol IBI denotes the determinant of
the square matrix B. Thus the computations reduce to
evaluating two determinants rather than inverting VO (see
Scheffe, 1959, Appendix V, p.417).
When p2 > 1 and A1 is no longer a scalar, maximizing X1
(and thus maximizing the power of the test) cannot be
accomplished without specifying 02 In this case we make
use of a lower bound for 1l (Graybill, 1969, p.330, Theorem
12.2.14(9)) defined as
lmin/22 < X1
(where min is the smallest eigenvalue of A1) to be used in
place of \l. Hence an approximate solution to the
maximization of X1 will be achieved by finding the k
simultaneous check points (using Price's procedure) that
maximize min' the smallest eigenvalue of A1. In other
words when p2 > 1, and in order to avoid specifying 2' we
seek to maximize a lower bound value for X1. This
maximization does not depend on the value of 02.
There are cases where the matrix A1 is of less than
full rank (less than rank p2) or equivalently where the
matrix A1 is positive semidefinite so that umin will be
equal to zero no matter which check points are selected.
One such case occurs when k < p2 (when the number of check
points is less than the number of parameters in the true
model which are not in the fitted model) since when k < p2
rank(Al) = rank[V I X* X*A)]
rank(X* X*A),
2
and so rank(Al) < min(k, p2) because the matrix (X* X*A)
is of order kxP2. Therefore when k < p2, the rank of A1 is
at most k so that A1 is of less than full rank. Since umin
must be equal to zero when Al is positive semidefinite, an
alternative method to that of maximizing umin to select
optimal check points must be found when A1 is positive semi
definite in order to produce a positive lower bound for X1.
In this pursuit, let us write X1 as
1 = 2AA2/202
= PAP'_2/202
= 8[P1:P2] diag[Al, A2=0][P1:P2] '2/2o2
= PlA 1P 12/202
where A is a diagonal matrix with elements equal to the
eigenvalues of Al, P is an orthogonal matrix whose columns
are orthonormal eigenvectors of AI, A1 and P1 correspond to
the positive eigenvalues of AI, while A2 = 0 and P2
correspond to zero eigenvalues of AI. Then by Theorem
12.2.14(9) in Graybill (1969) we can write
Sz'z/2o2 < 1 (3.7)
min p
where in is the smallest positive eigenvalue of A,, and
min
z = P'S Thus by Eq. (3.7), an approach to maximizing a
 12
positive lower bound for X1 when A1 is positive semi
definite is to select check points that maximize the
smallest positive eigenvalue of A1. It must be noted,
however, that this method can only be used when
a2 e n C(Pl), where C(P1) denotes the column space of P1
and n C(PI) denotes the intersection of all such spaces
which can be obtained at all possible check points
locations. This is because, in general, z'z in (3.7)
depends on the location of the check points through its
dependency on Pi. If, however, 2 e nC(P ), then
zz = P1 = P = 22 since 'P2 = 0.
It follows that when 2 e n C(P ), mn z'z/2o
2 1 min 
+ 2 +
= min 2+/2o and only mn depends on the location of the
check points.
3.3 Testing for Lack of Fit When MSE Is Used
to Estimate Experimental Error Variation
3.3.1 The Test Statistic
In this section we shall show that when an external
estimate of a2 is not available and the residual mean square
(MSE) from the fitted model of the form (3.1) must be used
as an estimate of a2, the test statistic
l
d'V d/k
F ME (3.8)
MSEpossesses a central F distribution when the true model is
possesses a central F distribution when the true model is
Eq. (3.1), but possesses a doubly noncentral F distribution
when the true model is Eq. (3.2).
In the initial section of this chapter, the quantity
d'V d/2 was said to possess a central chisquare
distribution or to possess a noncentral chisquare
distribution, depending on whether the true model was
specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum
of squares from the fitted model is defined as
N
SSE = E (Y Y)
i=l
1
= Y'(I X(X'X) X')Y
and it is easy to show (Searle, 1971, p.57, Theorem 2) that
SSE/a2 possesses a central chisquare distribution if the
true model is Eq. (3.1), but under model (3.2), SSE/a2
possesses a noncentral chisquare distribution. This is
expressed as
2 2
SSE/a xp
Np
under model (3.1), and
2 2
SSE/a X.
Np,12
under model (3.2), where the noncentrality parameter \2 is
2
X2 = (X2 XA)'(X2 XA)2/2o0
The distributional form of the test statistic in Eq.
(3.8) is derived by knowing that the quantities
d'Vo d/o2 and SSE/o are statistically independent (see
Appendix 3), so that
d'vold/ko2
12
MSE/o
d'V d/k
MSE
is distributed as a central F when the true model is Eq.
(3.1), but when the true model is Eq. (3.2) the F ratio is a
doubly noncentral F, that is, under model (3.1),
F ~ F
k,Np
and under model (3.2),
F~ F"
k,Np;X ,X2
3.3.2 The Rejection Region and its Relation to the Power of
the Test
In Appendix 1 it is shown that if k, Np, and X2 are
fixed, then the power of the F test using the ratio (3.8) is
a function of the location of the rejection region (upper
tailed or lower tailed) of the test. The power increases
with increasing values of the numerator noncentrality
parameter, X1, when the test is an upper tailed test. The
power decreases with increasing values of X1 when the test
is a lower tailed test. This means that to study ways of
increasing the power of the test, we have to determine
whether the test is an upper tailed test or a lower tailed
test. Similarly, for fixed values of k, Np, and X1, the
power of the F test is a decreasing function of X2 for an
upper tailed test, and is an increasing function of X2 when
the F test is a lower tailed test (Scheffe, 1959, p. 136
137).
To decide if the test is an upper tailed test or a
lower tailed test, we recall from Section 3.2.2 that if the
true model is Eq. (3.1) then the expected value of the
numerator of the F statistic in (3.8) can be written as
E(numerator) = 02,
and if the true model is Eq. (3.2),
2 2
E(numerator) = a + 2a X1/k (3.9)
= 02 + QA 2/k
1
where the P2xp2 matrix A is A = (X* X*A)'V (X* X*A).
Similarly, it can be shown that if the true model is Eq.
(3.1), the expected value of the denominator of the F
statistic in (3.8), where the denominator equals MSE, is
E(denominator) = E(MSE)
= 2,
but if the true model is Eq.(3.2),
E(denominator) = E(MSE)
[o2/(N p)]Eyp,2
= [a2/(N p)][N p + 2X2]
= a2 + 22 2X/(N p) (3.10)
= 2 + a2A2 2/(N P)
where the P2xP2 matrix A2 is A2 = (X2 XA)'(X2 XA). Thus
the ratio E(numerator)/E(denominator) will equal unity if
the true model is Eq. (3.1), but if the true model is Eq.
(3.2), the ratio is greater than unity if B_~A12/k >
8_A 2 2/(N p). In this latter case we reject the null
hypothesis of zero lack of fit if the calculated value of
the F ratio in (3.8) is large. An upper tailed rejection
region seems reasonable for this test. When the true model
is Eq. (3.2), and if a2Al2/k < 2A2 2/(N p), then a lower
tailed rejection region is preferred.
3.3.3. A Method for Locating Optimal Check Points
Given a design for fitting a model of the form in Eq.
(3.1) in a mixture space (note that fixing the design fixes
A2 and (N p)), and given the number of simultaneous check
points desired, k > 1, we now wish to determine where in the
mixture space the k check points should be located so as to
maximize the power of the F test for lack of fit, where the
test statistic is given in Eq. (3.8). We also wish to
position the optimal check points in a manner that is
independent of the values of the elements in 2 .
2
The case of an upper tailed test. To help us find k
simultaneous check points that maximize the power of an
upper tailed test, we shall make use of the fact that the
power is an increasing function of Xi. Therefore to
maximize the power of the upper tailed F test, we shall seek
the locations of the k check points that maximize X1.
As in the case considered in Section 3.2.3, where the
test statistic had a noncentral F distribution, if the
number of extra terms in the true model is p2 = 1, then
maximizing X1 is equivalent to maximizing the scalar A1.
However, as before, if p2 > 1, then the P2xP2 matrix A1 is
not a scalar and we will have to approximate the
maximization of X1 by maximizing a lower bound for X1. This
is done by finding the maximum value of min' the smallest
eigenvalue of A1, since
minm2/22 X1
When the number of check points is less than the order
of the square matrix A1, that is, k < p2, then rank(A1) <
min(k, p2), and A1 will have Umin = 0. For this case, we
again try to maximize the smallest positive eigenvalue of A1
which we denote by in' while remembering from Section
min'
3.2.3 that this technique is limited to situations where
B2 e nC(P1)
21
The case of a lower tailed test. To find k check
points to maximize the power of a lower tailed test, we make
use of the fact that the power of the lower tailed F test
increases as X1 decreases. Then if P2 = 1 and A1 is a
scalar quantity, X1 can be minimized with respect to the k
check points by finding the check points that minimize A1.
If P2 > 1, then by Theorem 12.2.14(9) in Graybill (1969), we
see that an upper bound for X1 is
X1 max_22 /202 (3.11)
where umax is the largest eigenvalue of Al. An approximate
solution to minimizing X1 in (3.11) can be achieved by
minimizing max. It is not necessary to treat the case
'max
of k < p2 separately here, although X1 will equal zero if
_2 is in the column space of P2, where P2 is the matrix
whose columns are orthonormalized eigenvectors corresponding
to the zero eigenvalues of the matrix A .
3.3.4 Determining Whether the Test Is Upper Tailed or Lower
Tailed
The procedures outlined in Section 3.3.3 produce a set
of k check points that simultaneously maximize the power of
the upper tailed test as well as a second set of k check
points that simultaneously maximize the power of the lower
tailed test. The check points that are selected maximize
the power, given A2, k, and N p without specification
of a2' except that when A1 is positive semidefinite we
require that 82 e n C(P ).
It is now necessary to decide which of our two
candidates will be used for a lack of fit test. To choose
between the upper tailed test and the lower tailed test, let
us consider the quantity
R = [A /k] [A2/(N p)].
If R is positive definite when the true model is Eq. (3.2),
then no matter what the value of 82 is, the ratio
E(numerator)/E(denominator) will be greater than unity,
implying an upper tailed test is to be used. Similarly, if
R is negative definite, then a lower tailed test should be
used. Finally, if R is not definite, then neither an upper
nor a lower tailed test is implicated and further
investigation is necessary. The criterion of R = [A /k] 
[A2/(N p)] may yield any of the four following cases.
Case 1. If R = [A /k] [A2/(N p)] is positive
definite when A1 is generated by the k optimal upper tailed
test check points, and R is not negative definite when A1 is
generated by the k optimal lower tailed test check points,
then we recommend that the check points be used that yield
the optimal upper tailed test with an upper tailed rejection
region.
For Case 1 it is necessary for A1 to be positive
definite (see Appendix 4). Since A1 is a square matrix of
order p2xP2 with rank(A ) < min(k, p2), then A1 can be
positive definite only if k > p2. Thus, there must be at
least p2 check points for Case 1 to hold, where p2 is the
number of terms in the model of Eq. (3.2) that are not in
the model of Eq. (3.1).
From inspection of equations (3.9) and (3.10), it is
apparent that the testing for lack of fit in Case 1 is
equivalent to testing the hypothesis
1 2
H0 0 N p = (3.12)
against the alternative
1 2
H > 0
a k N p
60
since R = [A /k] [A2/(Np)] is positive definite when the
true model is Eq. (3.2). In Appendix 5(a) it is shown that
under Case 1, the hypothesis given by (3.12) is equivalent
to the hypothesis
H X = X = 0.
Case 2. In Case 2 we assume that R = [A /k] 
[A2/(N p)] is not positive definite for the k optimal
upper tailed test check points, but that R is negative
definite for the k optimal lower tailed test check points.
Here we recommend that the lower tailed test check points be
used with a lower tailed rejection region.
It is necessary for A2 to be positive definite for Case
2 to occur (see Appendix 4). However, A1 need not be
positive definite, and so k need not be greater than p2. In
Case 2 then, it is possible that lack of fit may be tested
with only one check point.
By inspection of equations (3.9) and (3.10), a
hypothesis of no lack of fit is equivalent to
X1 X
1 2
Hk = 0 (3.13)
0' k N p
while the alternative hypothesis that lack of fit is present
is equivalent to
X1 X
1 2
H N < 0
a k N p
since R = [A /k] [A2/(N p)] is negative definite. In
Appendix 5(b) it is shown that the hypothesis given by
(3.13) is equivalent to the hypothesis
H0: = 2 = 0.
Case 3. We assume R is positive definite for the k
optimal upper tailed test check points, and R is negative
definite for the k optimal lower tailed test check points.
Hence either an upper or lower tailed test may be considered
as a possible test for lack of fit. If the quantity
2
_'_2//o can be specified, then the minimum power for both
the optimal upper and optimal lower tailed tests can be
approximated, and the test with the greater minimum power is
recommended. In Appendix 4 it is shown that Case 3 can
occur only when A1 is positive definite for the upper tailed
test. Thus Case 3 can only occur when there are at least p2
check points.
The minimum power of the upper tailed test may be found
by calculating
P IF" > F ), (3.14)
k,Np;1 IL' 2U a;k,Np
where F;k,Np is the upper 100a percentage point of the
central F distribution,
2
IL = /min2/2a2
and
X2U = max22/2 02
where min is the smallest eigenvalue of A1 and max is the
mmn 1 max
largest eigenvalue of A2. Formula (3.14) yields a
conservative lower bound for the power of the optimal upper
tailed test. Note that A1 is generated using the optimal
upper tailed test check points. The cumulative distribution
function of F" can be approximated by multiplying the
cumulative probabilities of the central F distribution by a
constant (Johnson and Kotz, 1970, p.197). This
approximation is described in Appendix 6. Other
approximations for F" (such as the Edgeworth series
approximation suggested by Mudholkar, Chaubey, and Lin,
1976) exist which are generally more accurate, but we chose
to use the approximation given in Johnson and Kotz (1970,
p.197) due to its simplicity. Additionally, the
approximation of Mudholkar, Chaubey, and Lin (1976) produced
negative probabilties when only one degree of freedom was
available in either the numerator or denominator of F".
This problem was avoided by using the approximation given by
Johnson and Kotz (1970).
The minimum power of the optimal lower tailed test can
be approximated similarly (if B~22/o2 is specified) by
calculating
P IFF"
< (la);k,Npp
where
X lu = maxJ.2 2/2o 2
and
S2L = 6min22/20 2
with Umax equal to the largest eigenvalue of A1 and 6min
max n
equal to the smallest eigenvalue of A2. Note that A1 is
generated by using the optimal lower tailed test check
points. For the lower tailed test, A1 may be positive semi
definite, and if 82 is in the column space of P2 then A1 = 0.
In Case 3, the upper tailed test is a test of
HO: 1 = = 0
0 1 2
H 2 > 0
a k N p
while the lower tailed test is a test of
H: X =X2 0
H0 1 2 = 2
1 2
H P < 0.
a k N P < 0
Case 4. In Case 4 we assume that R = [Al/k] 
[A2/(N p)] is not positive definite for the k optimal
upper tailed test check points and R is not negative
definite for the k optimal lower tailed test check points.
Here it is useful to write the difference between the
expected value of the numerator and the expected value of
the denominator of the F ratio in (3.8) as
al[A /k A2/(N P)] = sns'S
2 1 2 2 = 2
= 8[S1:S2:S3] diag[l2'r2=0,03 [Sl:S2:S3] '2
= 8_2S"l I2 + 2S3 332
where 0 = diag(ll, 02' 23) is a diagonal matrix consisting
of the eigenvalues of R, 01 is a diagonal matrix of the
positive eigenvalues of R, 02 is a diagonal matrix of the
zero eigenvalues of R, and 03 is a diagonal matrix of the
negative eigenvalues of R. The orthogonal matrix S can be
expressed as S = [S1:S2:S3], where the matrices SI, S2, and
S3 have columns which are orthonormalized eigenvectors
corresponding to nl, 02, and 03, respectively.
In Case 4, neither the optimal upper tailed test nor
the optimal lower tailed test is applicable for all values
of _2 For completeness, we note that Case 4 actually
consists of nine subcases, where R may be positive semi
definite, negative semidefinite, or indefinite for either
the optimal upper tailed test or lower tailed test check
points. These subcases are listed in Table 2.
Table 2. Nine Subcases of Case 4.
RUpper RLower
Subcase Tailed Test Tailed Test
1 PSD PSD
2 PSD NSD
3 PSD I
4 NSD PSD
5 NSD NSD
6 NSD I
7 I PSD
8 I NSD
9 I I
PSD = positive semidefinite, NSD = negative semi
definite, I = indefinite.
If _2 lies in the column space of S2, then 8'[A /k 
A2/(N p)]s82 is zero, and therefore lack of fit is not
testable with either an upper or lower tailed test. A
sufficient condition for the test for lack of fit to be
upper tailed in Case 4 is that 0 be in the column space
2
of [S1:S2], but not entirely in the column space of S2. In
this case
;[A1/k A2/(N p)]_2 = 2S01812 + 2S303S32
= 2S1~lS 2 + 0
= S~Sl2lSI2,
and 8_[ Al/k 
indicating an
condition for
that _2 be in
in the column
A2/(N )]_2 will be greater than zero,
upper tailed test. Similarly, a sufficient
the test for lack of fit to be lower tailed is
the column space of [S2:S3], but not entirely
space of S2. Then
2[A1/k A2/(N P)]g2 = 0 + 2S33S3_2
= 2S3 3S 2
which makes _2[Al/k A2/(N p)] 2 less than zero,
indicating a lower tailed test.
To determine whether 2 is in the column space of
[S :S2], let us define the augmented matrix
Q1 = [f2:Sl:S2] If Q!Q1 has a zero eigenvalue, then 02 is
in the column space of [S :S2]. Similarly, if we define
Q2 = [82:S2] and Q3 = [82:S2:S3]' then a2 is in the column
space of S2 if Q'Q2 has a zero eigenvalue, and 2 is in the
column space of [S2:S3] if Q3Q3 has a zero eigenvalue.
Given that we are in a particular subcase of the nine
subcases described in Table 2, we recommend that lack of fit
be tested with the upper tailed test check points if it is
determined that 2 is such that '[A /k A2/(N p)]2 is
positive when A1 is generated from the upper tailed test
check points. Likewise, for the same given subcase, if the
value of 02 of interest is determined to produce a negative
value for i2[Al/k A2/(N P)]@2 when A1 is generated from
the lower tailed test check points, then we recommend that
lack of fit be tested with the lower tailed test.
We see then that Case 4 is an undesirable situation in
practice, since, in order to test for lack of fit, we must
assume a priori that any lack of fit is due to a nonzero
value of 82 that produces an upper tailed or lower tailed
rejection region. However, it would seem rare that such
knowledge would be available.
3.4 Examples
We now present several examples to illustrate the
technique for locating optimal check points to be used in
testing for lack of fit in a mixture model.
3.4.1 Theoretical Examples
Example 1. In this example a second order canonical
polynomial model is fitted in three mixture components using
the {3,2} simplex lattice design, which is presented in
Figure 1 of Chapter 1. The true model is assumed to be the
special cubic model containing the term 123x x 23 in
addition to the six terms of the fitted model. The expected
values of the response at the six design points are assumed
to be represented by the fitted model in the form
E(Y) = X01,
but with the true model the expectations are written as
E(Y) = X 1 + X22 ,
where X is a 6x6 matrix with rows that define the
coordinates of the six design points and columns that
correspond to the six terms in the fitted model (xi, x2, x3,
xlx2, xlx3' x2x3)' 8a is the 6x1 vector of regression
coefficients (81, 82, 83, 812, 813, 823), X2 is a 6x1
column vector containing the values of the term xlx2x3 at
the design points, and 82 is the single regression
coefficient 8123'
The {3,2} simplex lattice design consists of only six
design points, and since six parameters are estimated in the
second order fitted model, there are no degrees of freedom
remaining for obtaining an estimate of the experimental
error, 02. We assume therefore that an external estimate of
a is available, a2 which will be used in the denominator
of the lack of fit F statistic given in Eq. (3.3).
Since there is one term in the true model in addition
to those in the fitted model, that is p2 = 1, we know that
in order to locate k simultaneous check points that maximize
the power of the test for lack of fit it is necessary to
maximize the scalar quantity
1
A = (X* X*A)'V (X* X*A)
1 2 0 2
with respect to the coordinates of the k check points. Here
X* is a kelement column vector with ith element equal to
the value of x* x* x* at the ith check point, X* is a kx6
il i2 i3
matrix with ith row equal to the value of (x* x* x*
il 12' i3'
S x* x* x xt ) at the ith check point,
11 1i2' 11 Xi3' 12 i3
A = (X'X)X'X2 is the 6x1 alias vector, and
V = Ik + X*(X'X) X*'. This maximization is accomplished
by use of the Controlled Random Search Procedure (Price,
1977), which is described in Appendix 2.
When only a single (k = 1) optimal check point is
desired the Controlled Random Search Procedure locates a
point (x*, x*) which maximizes
1
A1 = (X* X*A)'V0 (X2 X*A),
where
S= x*x*x* = *x*( x* 
2 123 12 1 2
X* = (x*, x*, x* x* x*x*, x*x*)
= (x*, X* (1 x* x*), x*x*, x*(l *),
1 2 1 2 12' 1l 1 2
x*(l x* x*)),
2 1 2
l
and V0 = 1 + X*(X'X) *'. The value of A1 is calculated
using the formula of Eq. (3.6). Following this procedure,
we find that the single check point that maximizes A1, and
thus maximizes the power of the test, is the centroid of the
triangular factor space (1/3, 1/3, 1/3). The value of A1 at
this centroid point is A1 = 0.00084.
When the Controlled Random Search Procedure is used to
locate k = 2 simultaneous check points that maximize A,, the
centroid (1/3, 1/3, 1/3) is selected twice, and A1 =
0.00121. For three simultaneous optimal check points, the
centroid is selected three times, and A1 = 0.00142.
To test whether the second order model exhibits lack of
fit, when we suspect the special cubic model is the true
model, we form the F ratio
1
d'V d/k
0
F = ^ 2
^2
ext
with the single check point (1/3, 1/3, 1/3) where d =
Y* Y*, Y* is the observed response, Y* is the response
1 1 1 1
predicted by the second order fitted model at (1/3, 1/3,
1/3), and V0 = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(X'X)1
(1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of
the F ratio exceeds F where v equals the number of
degrees of freedom associated with a then we reject the
ext
null hypothesis that the second order model is the true
model in favor of the alternative hypothesis that the
special cubic model is the true model. Equivalently, we
reject H: I1 = 0 in favor of Ha: 1 > 0. For k = 2 or
k = 3 check points, the value of the F ratio is calculated
using the observed and predicted responses at the two or
three replicates at the centroid. The hypothesis
H0: X = 0 is rejected in favor of Ha: X > 0 if F
exceeds F;kv
a ;k,v
Example 2. In Example 2 we illustrate the second of
the four cases that could arise when MSE is used as an
estimate of 02 in the lack of fit test statistic (see
Section 3.3.4). We again fit a second order canonical
polynomial model in three mixture components, and assume the
true model is special cubic. The design to be used is the
q = 3 simplex centroid design, which consists of seven
design points, and is illustrated in Figure 2 of Chapter 1.
There are six parameters to be estimated and seven
design points hence one degree of freedom can be used to
calculate MSE. We shall use MSE to estimate a2. Optimal
upper and lower tailed test check points must be located,
and then a decision is made as to which test should
be used. The actual testing for lack of fit involves the F
statistic in (3.8).
As in Example 1, p2 = 1, since there is one term in the
true model in addition to those in the fitted model. Thus
Al is a scalar whose value we seek to optimize with respect
to the desired number of check points, k. When only a
single check point is sought for the purpose of testing lack
of fit, the Controlled Random Search Procedure has two
functions. First, the procedure is used to locate the
optimal candidate check point for an upper tailed test by
locating the check point that maximizes the scalar Al.
Secondly, the procedure is used to locate the optimal
candidate check point for a lower tailed test, which is
accomplished by locating the point that minimizes A1. The
quantity R = [Al/k] [A2/(N p)] is then calculated to
determine whether the upper or lower tailed test will be
used. If R is positive for the candidate check point for an
upper tailed test, then the test is upper tailed, and the
test is lower tailed if the candidate check point for a
lower tailed test produces a negative value for R. Note
that A2 = (X2 XA)'(X2 XA) is fixed once the design is
specified, since A2 does not depend on the check points.
Using the Controlled Random Search Procedure it is found
that the maximum value of A1 occurs at (xl, X*, x*) = (1/3,
1/3, 1/3), which will be the location for the check point
for the upper tailed test. Calculating A1 at this centroid
point, we find that R = [A /k] [A2/(N p)] = [(3.7258
x 104)/l] [(8.4175 x 104)/l] = 4.6917 x 104. Since R
is negative, the test is not upper tailed.
Using the Controlled Random Search Procedure to
minimize AI, we find that a subregion of the factor space
exists in which all points yield a near minimum value for
A1. We choose the point (0.0189, 0.9269, 0.0542) at random
from this subregion to be used as the optimal candidate for
a lower tailed test. Here R = 0 [(8.4175 x 104)/] =
8.4175 x 104.
Since R is negative for both the optimal upper tailed
test check point and for the optimal lower tailed test check
point, we have Case 2 of Section 3.3.4. The upper tailed
test check point is disregarded, and the lower tailed test
check point (0.0189, 0.9269, 0.0542) is used to test for
lack of fit. If the calculated F ratio,
1
d'V d
F =
MSE
is less than F ( );,then H: X = X = 0 is rejected in
favor of Ha: [X1/1] [X2/1] < 0, that is we conclude that
the second order model exhibits lack of fit, and the true
model is special cubic.
When two simultaneous check points are desired for
testing lack of fit, we can again use the Controlled Random
Search Procedure to locate the optimal settings. To
maximize the scalar A,, we find that both check points
should be selected at (1/3, 1/3, 1/3), for an upper tailed
test. With our calculations R = [(5.8275 x 104)/2] 
[(8.4175 x 104)/1] = 5.5038 x 104, but since R is
negative, the test is not upper tailed.
Minimizing A1 to locate two optimal lower tailed test
check points yields a subregion in the factor space of
optimal check points. The pair of check points (0.3749,
0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at
random from this subregion, and these check points yield
R = 0 [(8.4175 x 104)/1] = 8.4175 x 104.
Since R is negative for the upper tailed test points
and the lower tailed test points, we have Case 2 of Section
3.3.4 again and the lower tailed test check points are used
to test for lack of fit. The hypothesis H0: X1 = 2 = 0 is
rejected in favor of Ha : [1/2] [x2/1] < 0 if the cal
1
culated value of F = (d'Vo d/2)/MSE is less than
F(1a);2,1, in which case we say lack of fit of the model is
present.
^2
If an external estimate aext had been available for
ext
this example, then the optimal upper tailed test check
points could have been used in the F ratio,
1 ^2
F = (d'V d/k)/o t and lack of fit would then be detected
0 k exta
if the calculated value of F exceeded F ,
a;k,v
Example 3. Example 3 illustrates the procedure for
locating optimal check points when there are two terms in
the true model in addition to those in the fitted model. A
second order canonical polynomial model in three mixture
components is fitted using a q = 3 simplex centroid design.
The true model is assumed to contain eight terms, six of
which are the same terms as in the fitted model, with the
additional two terms being the third order terms
612x x2(X1 x2) and B123x x2x3. As in Example 2, there is
one degree of freedom for MSE which is used to estimate
2 T
2. The test statistic, F = (d'V d/k)/MSE, is given in
equation (3.8).
Since p2 = 2 and A1 is a 2x2 matrix, locating the
optimal upper tailed test check points by the procedure of
maximizing X1 is assisted by the maximizing of a lower bound
for 1 namely maximizing u min /2a where u is the
I min22 mm
smallest eigenvalue of A1. Since 82 and a2 are unknown,
this is equivalent to maximizing min For min to exceed
zero, it is necessary that A1 be of full rank, and since
rank(A1) ( min(k, p2), it is necessary to select k > 2 check
points. If A1 is less than full rank, and thus is positive
semidefinite, only a subset of possible values of 2 could
be considered to make it possible to test for lack of fit
with an upper tailed test.
Using the Controlled Random Search Procedure, the
points that maximize min are found to be (0.418, 0.277,
mln
0.305) and (0.277, 0.418, 0.305). These points are thus
optimal candidates for upper tailed test check points. At
these check points we have umin = 5.1623 x 104, A1 =
diag[5.1623 x 104, 5.1916 x 104], A2 = diag[0, 8.4175 x
104], and R = [A1/2] [A2/I] = diag[2.5811 x 104, 5.8217
76
x 104]. Since the eigenvalues of R are 5.8217 x 104 and
2.5811 x 104, R is indefinite. Following the suggested
procedure for Case 4 of Section 3.3.4, we note that an upper
tailed test for lack of fit exists if the value of
'2 = [512' 8123]' is in the column space of [Sl:S2] but not
entirely in the column space of S2, where S1 is the matrix
whose columns are the orthonormalized eigenvectors of R
corresponding to the positive eigenvalues of R, and S2 is
the matrix whose columns are the orthonormalized
eigenvectors of R corresponding to the zero eigenvalues of
R. Since R has no zero eigenvalues in this example, S2 does
not exist, but S1 is the column vector, S1 = [1,0]'. Thus
if 2 is of the form 2 = [612 0]', where 12 0, then
a2 is in the column space of S1 and the test is upper
tailed.
The matrix A2 has rank one and therefore is positive
semidefinite. Hence it is impossible to locate two check
points that minimize umax and also make R = [A1/2] [A2/1]
negative definite (see Appendix 4), that is, it is
impossible to find a lower tailed test that is capable of
testing lack of fit for all values of 2. However, if we
use the Controlled Random Search Procedure to locate two
check points that minimize an upper bound for X1 which is
umax S2 /2 2, then by minimizing max, we find that any of
the check points in a particular subregion of the factor
space yield a near minimum for umax. One pair of points in
this subregion is selected as the points to be used as
optimal lower tailed test check points, namely the pair
consisting of the point (0.053, 0, 0.947) replicated twice.
Replicating this check point, we find max = 7.3900
x 1011, A = diag[0, 7.3900 x 1011], A2 = diag[0, 8.4175 x
104], and R = [A1/2] [A2/1] = diag[0, 8.4175 x 104].
The eigenvalues of R are 0 and 8.4175 x 104 implying that
R is negative semidefinite. The values of 8 that are in
2
the column space of [S2:S31 but not entirely in the column
space of S2 will provide a lower tailed test. Here, [S2:S3]
= diag[l,l] and S2 = [11,0]'. Thus, the test for lack of fit
is lower tailed if 8123 0.
For values of 8 that produce an upper tailed test we
2
use the check points (0.418, 0.277, 0.305) and (0.277,
0.418, 0.305) with the F ratio
1
d'Vo d/2
F =
MSE
and conclude there is lack of fit if the calculated value of
F exceeds Fa;21. For values of a2 that produce a lower
tailed test, we use two replicates of the check point
(0.053, 0, 0.947), and'conclude there is lack of fit if F is
less than F(1a);2,1, where again F is calculated by
1
F = (d'V d/2)/MSE.
0O
Example 4. Example 4 illustrates Case 3 of Section
3.3.4 in which MSE is used to estimate 02 in the lack of fit
test statistic. A second order canonical polynomial model
in three mixture components is fitted using the {3,3}
simplex lattice design, which appears in Figure 5. The true
model is assumed to be special cubic, thus p2 = 1 and A1 is
a scalar. The {3,3} design consists of ten design points
and since there are six parameters to be estimated in the
fitted model, 02 can be estimated by MSE with N p = 10 6
= 4 degrees of freedom.
We first suppose that a single check point is to be
used to test for lack of fit. Using the Controlled Random
Search Procedure we find the single check point that
maximizes the scalar
1
A= (X X*A)'V (X* X*A)
1 2 0 2
is located at the centroid of the simplex factor space.
Thus (x*, x*, x*) = (1/3, 1/3, 1/3) is the optimal candidate
for an upper tailed test check point. At this centroid
point, A1 = 4.9076 x 104. For the {3,3} design the scalar
quantity A2 = (X2 XA)'(X2 XA) is fixed and is equal to
A2 = 9.4062 x 104 and thus, R = [Al/k] [A2/(N p)] =
[(4.9076 x 104)/1] [(9.4062 x 104)/4] = 2.5560 x 104
The point that is the optimal candidate for a lower
tailed test check point is chosen randomly from a subregion
of points in the factor space, in which all points minimize
A1. The point selected has the value (x*, x*, x*) = (0.560,
0.410, 0.030). Here A1 = 9.6590 x 107 and R = [(9.6590 x
107)/1] [(9.4062 x 104)/4] = 2.3419 x 104.
(1,0,0)
3 3 a0o 0'x
( 0)/ 3 3
3' 3'3
(0o,,0)  (0,0,1)
x2 0, ) 3 3' ) x3= 1
Figure 5. The {3,3} simplex lattice design.
Since R is positive for the optimal upper tailed test
check point (1/3, 1/3, 1/3) and R is negative for the
optimal lower tailed test check point (0.560, 0.410, 0.030)
we are in Case 3 of Section 3.3.4. Either the upper or
lower tailed test could be used to test for lack of fit, but
if the quantity B B2/o2 can be specified, then we will
choose to use the test that has greater minimum power, since
greater power means that we are more likely to detect lack
of fit when in fact lack of fit exists. In this example
2 123*
For illustrative purposes, we arbitrarily choose
2
8'8 /C2 = 2000, so that an approximate conservative lower
bound for the power of the upper tailed test is found by
calculating
P {F" > F k
k,Np;XIL' 2U a;k,Np
where F;k,Np is the upper 100a percentage point of the
central F distribution, k is the number of check points, N
is the total number of response observations, p is the
number of parameters in the fitted model,
2 2
IlL = UminB2/2a and 2U 6 axI/20 The
quantity umin is the smallest eigenvalue of AI, where A1 is
evaluated at the optimal upper tailed test check point.
Since A is a scalar, mn = A. Likewise, 6 is the
1 mim 1 fmax
largest eigenvalue of A2, and since in this example A2 is a
scalar, 6max = A2. In this example we have k = 1, N p =
10 6 = 4, AlL = UminS2/2a2 = (4.9076 x 104)(2000/2)
1 2
= 4.9076 x 101, and 2U = 6 m /2a2
= (9.4062 x 10 )(2000/2) = 9.4062 x 101. Using the
approximation to the cumulative probabilities of the doubly
noncentral F distribution given by Johnson and Kotz (1970,
p.197) which is described in Appendix 6, and taking a = .05,
we find that a conservative lower bound for the power of the
optimal upper tailed test is approximately equal to .0649.
The minimum power for the optimal lower tailed test is
2
approximated (assuming 8_Y2/a = 2000) by calculating
P F" < FN
k,Np;lU' 2L (1a);k,Np
2
The quantities A1U and 2L are taken as A U = maxJ2 /2o
lJU 2L lU maxz22
and 2L = 6 mn /2o where max is the largest eigenvalue
2L min22 max
of A1 with A1 calculated using the optimal lower tailed test
check point, and where 6 in is the smallest eigenvalue of
A2. Since A1 and A2 are scalars, max = A and 6 min = A .
2 1 2 max 1 mm 2
In this example, k = 1, N p = 4,
AlU = (9.6590 x 10 )(2000/2) = 9.6590 x 104, and
4 i
x2L = (9.4062 x 10 )(2000/2) = 9.4062 x 101 Again if the
approximation to the doubly noncentral F distribution given
in Johnson and Kotz is used, an approximate conservative
lower bound for the power of the optimal lower tailed test
is .0555.
Having specified a 2/02 = 2000, the optimal upper
tailed test is chosen over the optimal lower tailed test,
because the approximate minimum power of the upper tailed
test is greater than the approximate minimum power of the
lower tailed test. Using the optimal upper tailed test
check point (1/3, 1/3, 1/3) in the test statistic
1
d'V d
F =
MSE
we conclude that lack of fit is significant if the
calculated value of F exceeds F a1,4 in which case we
a;1,4'
reject H0: XI = 2 = 0 in favor of Ha: A/l 12/4 > 0.
When two simultaneous check points are used for testing
lack of fit, the Controlled Random Search Procedure locates
the optimal upper tailed test and optimal lower tailed test
check points. It turns out that two replicates at (1/3,
1/3, 1/3) maximize Al, and are used as optimal check points
for an upper tailed test. The value of R = [A,/2] [A2/4]
is [(7.9210 x 104)/2] [(9.4062 x 104)/4] = 1.6090 x
104.
In searching for two optimal lower tailed test check
points, again a subregion of the factor space is found in
which any of the points nearly minimize A From this
subregion are chosen the points (0.6386, 0.3263, 0.0351) and
(0.7257, 0.2421, 0.0322) resulting in a value of R = [A,/2]
 [A2/4] of [(1.5216 x 109)/2] [(9.4062 x 104)/4] =
2.3516 x 104.
In conclusion, when two simultaneous check points are
used in the test for lack of fit in this example, R is
positive for the optimal upper tailed test and R is negative
for the optimal lower tailed test, and we have Case 3 of
2
Section 3.3.4. Selecting 002 /o = 2000 arbitrarily, we
found the approximate lower bound for the power of the upper
tailed test to be .0504, and the approximate lower bound for
the power of the lower tailed test to be .0612. Since the
power is higher with the lower tailed test it is our choice
for testing lack of fit when two check points are used
simultaneously. Lack of fit is detected and we reject
H0: 1 2 = 0 in favor of Ha: [1 /2] [ 2/4] < 0 if the F
1
ratio, F = (d'V0 d/2)/MSE, using the optimal lower tailed
test check points (0.6386, 0.3263, 0.0351) and (0.7257,
0.2421, 0.0322) is calculated to be less than F1a);2,4'
(ia);2 ,4"
3.4.2 Numerical Examples
Numerical Example 1. In this example we illustrate
numerically some of the findings in the first theoretical
example of Section 3.4.1. Data that were collected in a
rocket fuel experiment (Kurotori, 1966) will be used to
investigate the power of the lack of fit F test. The test
is set up to detect the inadequacy of a fitted second order
canonical polynomial model when the true model is special
cubic. Calculated values of the power of the test which
detects lack of fit through large values of
1
d'V0 d/k
F =
^2
ext
will be compared for several check point locations, includ
ing the location (1/3, 1/3, 1/3) at which the power was
found to be maximum in Example 1 of Section 3.4.1.
In Kurotori's experiment the modulus of elasticity (Y)
of a rocket fuel is expressed as a function of the
proportions of three componentsbinder (xl), oxidizer (x2),
and fuel (x3). Since lower bounds are placed on the
component proportions xl, x2, and x3, in the form of
0.20 < xl, 0.40 < x2, and 0.20 < x3, pseudocomponents (x!)
are defined in terms of the original components in the form
of x1 = (xl 0.20)/(1 .80), x' = (x2 0.40)/(1 .80),
and x' = (x3 0.20)/(1 .80). The true special cubic
model in the pseudocomponents, which is obtained by using
the data at the seven points of the simplex centroid design
in the pseudocomponent system, is
E(Y) = 2350x' + 2450x' + 2650x' + Ox'x'
1 2 3 1 2
+ lOOOx'x3 + 1600x'x' + 6150x'x'x'.
The second order canonical polynomial model that is fitted
to the six boundary points only, and which will be tested
for lack of fit, is given by
Y = 2350x' + 2450x' + 2650x'
3
+ 1000x'x' + 1600xNx'.
The configuration of the experimental design as well as the
check point locations are depicted in Figure 4 of Chapter 2
and the observed response values are given in Table 3 of
this chapter.
1 ^2
A value of the ratio F = [d'V d]/o is calculated at
0 ext
each of the four individual check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3)
"2 ^2
where ext is assumed to have the value et = 144 as
suggested by Kurotori (1966). We also assume without loss
of generality that the degrees of freedom associated
^2
with aet are v = 10. The power of the F test is calculated
at each of the four check points by using the approximation
to the cumulative probabilities of the noncentral F
Table 3. Observed Response Values at the
Pseudocomponent Settings for Kurotori's Rocket Fuel
ExperimentNumerical Example 1.
Observation
Number x'
______ _____
1
2
3
4
5
6
7*
8*
9*
10*
Binder Oxidizer Fuel
Modulus
x 3
0
1/2
1/2
0
1/3
2/3
1/6
1/6
0
1
0
1/2
0
1/2
1/3
1/6
2/3
1/6
0
0
1
0
1/2
1/2
1/3
1/6
1/6
2/3
of Elasticity
Y
2350
2450
2650
2400
2750
2950
3000
2690
2770
2980
*Check Points.
distribution given by Johnson and Kotz (1970, p. 197) to
evaluate
Power = P{FI' ,;,
11, 10;Xi
F.05;1,10}
2 ^2
where 1 = A 8 23/2 ext.
I 1 123 ex
The value of
A = (X* X*A)'V0 (X* X*A) is calculated for each check
point using the values of X*, X*, v and the value of
A = (X'X) X'X2 which is fixed by the {3,2} simplex lattice
design. Since the {3,2} simplex lattice consists of points
only on the boundaries of the triangle (and therefore at
each point at least one of the x! values is equal to zero),
1
then X2 = 0 and A = 0. From the true special cubic model,
123 = 6150.
The calculated value of F as well as the approximate
value for the power at each of the four check points is
given in Table 4. The check point (1/3, 1/3, 1/3) produced
the highest power of the four check points investigated,
supporting the previous results of Example 1 in Section
3.4.1 where (1/3, 1/3, 1/3) was selected as the check point
location with the maximum power when a second order
canonical polynomial was fitted using the {3,2} simplex
lattice design, but the true model was assumed to be special
cubic. Additional support for the point (1/3, 1/3, 1/3)
being optimal is given by the contour plot of values of A1
in Figure 6(d). The highest values of A1 appear near the
centroid (1/3, 1/3, 1/3) where high A1 values translate into
2 2
high X1 values, since Xi = A a123/22 which in turn implies
high power since we know the power is an increasing function
of X1.
As a second part of this example the power of the F
test that is obtained when three replicates are taken at
(1/3, 1/3, 1/3) is compared to the power of the F test that
is obtained when one replicate is taken at the three check
points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6,
2/3). These latter three point locations were suggested by
Kurotori for testing lack of fit of his fitted special
o 0 0 0
4
r1 4 r 4
, 4 , O
OC
11
mm
M N\N
i '
 '
0 *
0w
O
.JU
Q .H
3
Q)
4J)
U
4
0
U
0
m
4a
4J
(0
>1
cu
4Ja
CO
00
S0O
Cu
A 0
u r
0 0
c
o
4 r4
UO4
A U( 
S*
I 4 4 J
U K M
14
o
o
o
o
rI CN
, q 
CN .4 ,I
(m)
cMNrN
S S
0 
4J 4
Z 1
cubic model. The result of this comparison, see Table 4, is
that the three replicates at (1/3, 1/3, 1/3) produce the
test with greater power which again supports the findings of
Example 1 of Section 3.4.1.
All of the check point locations listed in Table 4
produce very high power values (> .999) which is due in part
to the high value of 123 (123 = 6150). If S1 were of
123 123 6123
lower magnitude, then the three replicates at (1/3, 1/3,
1/3) would show a still greater superiority in the power
value compared to the power using the other check points.
This superiority is demonstrated in Table 5 where values of
8123 are listed as 3000 and 1500 and the comparative power
values are listed as 0.998 compared to 0.795 and 0.662
compared to 0.249, respectively. Table 5 also demonstrates
the superior power value for the point (1/3, 1/3, 1/3) when
123 = 3000 or 123 = 1500 and each of the four check points
is used one at a time.
Finally, (1/3, 1/3, 1/3) being the optimal check point
location is seen in Figure 6(c), where contour plots of the
expected difference in the heights of the surfaces are
drawn. The differences in the heights are found by
subtracting the estimated height of the surface obtained
with the fitted second order model from the estimated height
of the surface obtained with the true special cubic model.
The expected difference between the true and fitted surfaces
approaches a maximum the closer one moves to the centroid of
the simplex factor space, so that the optimal check point
89
<4 0O CD CD C
a) m Co C MN C N c
C 0 *
U a4
I COD0 CO
U co co 1 o CD m
0
a4
41J 4
C a) m co o o co m a) 4)
0 (a D aN O Oo *10 10 m r En
C >1* *
0
U a)
0 a U1 m a) 0T co
m 2' 0 0
'0 Z O*o r * r* *< u
Z ; 0 O
Oi 0
*q 4J V
4 L 0 4 0 *
q m o .m .
0 )
En *' m
a) 0
0 Cr rn)Cr)Cr)
^ N. N. N.. N. 'NN.'N N
a) N N N N NNN N.N U
; 4 (N i 4 C '4 M 4 4
U .4'
x :I
I
X =1 x= I
2 3
(a) True special cubic surface.
X =I
x :1 x :1
2 3
(c) Expected difference between the
true special cubic surface and
the fitted second order surface.
x2
(b) Fitted second order surface.
x =1 X :1
2 3
(d) A( X X XA)' V'( X X*A)
Figure 6. Contour plots for Numerical Example 1.
location (1/3, 1/3, 1/3) coincides with the point where the
expected difference between the true special cubic surface
and the fitted second order surface is maximum.
Numerical Example 2. In this second numerical example,
we investigate the power of the F test for detecting lack of
fit when a second order canonical polynomial model is fitted
in a mixture system which is in truth represented by a
special cubic model. The true model is assumed to be
E(Y) = 2350x1 + 2450x2 + 2650x3
+ 1000x x3 + 1600x2x3 + 6150x 2X3
which is used to generate hypothetical response observations
at the seven points of the q = 3 simplex centroid design as
well as at three check points. The values of the response
are obtained by adding the value of a pseudorandom normal
variate with mean 0 and variance 144 to each true predicted
response value. The data are given in Table 6.
The response values at the seven points of the simplex
centroid design are used in the least squares normal
equations to obtain the fitted second order model
Y = 2341x1 + 2438x2 + 2630x3
+ 310x1x2 + 1304x x3 + 1970x2x3
Table 6. Generated Response ValuesNumerical Example 2.
xl x2 x3 Y
1 0 0 2357
0 1 0 2454
0 0 1 2646
1/2 1/2 0 2403
1/2 0 1/2 2747
0 1/2 1/2 2962
1/3 1/3 1/3 3013
1/3 1/3 1/3 2993
2/3 1/6 1/6 2693
.02 .93 .05 2550
*Check points.
which is to be tested for lack of fit using the test
1
statistic F = d'V0 d/MSE. The F statistic will be evaluated
at each of the three check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time,
and the power of the test at the three check point locations
will be calculated and compared. The test is lower tailed
for all check point locations (since R = A1 A2 is negative
for all check point locations) and thus the power is defined
as
PI" ( F;, }.
1,1;XIX2 .95;1,1

Full Text 
PAGE 1
TESTING LACK OF FIT IN A MIXTURE MODEL BY JOHN THOMAS SHELTON A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1982
PAGE 2
To Nydra and My Parents
PAGE 3
ACKNOWLEDGEMENTS I would like to express my deepest appreciation to Drs . Andre ' Khuri and John Cornell for suggesting this topic to me and for providing constant guidance and assistance. They have made this project not only a rewarding educational experience but an enjoyable one as well. A special word of thanks goes to Mrs. Carol Pvozear for her diligence in transforming my handwritten draft into an expertly typed manuscript. Ill
PAGE 4
TABLE OF CONTENTS page ACKNOWLEDGEMENTS iii ABSTRACT vii CHAPTER ONE INTRODUCTION 1 1.1 The Response Surface Problem 1 1.2 The Mixture Problem 5 1.2.1 Mixture Models 6 1.2.2 Experimental Designs for Mixtures.. 12 1.3 The Purpose of this Research: Investigation of Procedures for Testing a Model Fitted in a Mixture System for Lack of Fit 17 TWO LITERATURE REVIEW Â— TESTING FOR LACK OF FIT 19 2.1 Introduction 19 2.2 Partitioning the Residual Sum of Squares.. 21 2.3 Testing for Lack of Fit Without Replicated Observations Â— Near Neighbor Procedures 26 2.4 Testing for Lack of Fit with Check Points. 33 THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING LACK OF FIT IN A MIXTURE MODEL 40 3.1 Introduction 40 3.2 Testing for Lack of Fit in the Presence of an External Estimate of Experimental Error Variation 41 3.2.1 The Test Statistic 41 3.2.2 The Testing Procedure and an Expression for the Power of The Test 45 3.2.3 A Method for Locating Optimal Check Points 47 3.3 Testing for Lack of Fit When MSE Is Used to Estimate Experimental Error Variation 51 3.3.1 The Test Statistic 51 3.3.2 The Rejection Region and its Relation to the Power of the Test.. 53 iv
PAGE 5
3.3.3 A Method for Locating Optimal Check Points 56 3.3.4 Determining VVhether the Test Is Upper Tailed or Lower Tailed 58 3 . 4 Examples 67 3.4.1 Theoretical Examples 67 3.4.2 Numerical Examples 83 3.5 Discussion 95 FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR TESTING LACK OF FIT 99 4.1 Introduction 99 4.2 Notation 101 4.3 Shillington ' s Procedure 106 4.3.1 Development of MSEg 109 4.3.2 Development of MSE^ 110 4.4 Development of SSE^^( weighted ) 112 4.5 Equivalence of SSE^^ and SSEy^(weighted ) . . . . 116 4.6 The Test Statistic 118 4.7 The Testing Procedure and its Power 122 4.8 Selection of Near Neighbor Groupings 125 4.8.1 Example 1 Â— Stack Loss Data 129 4.8.2 Example 2 Â— Glass Leaching Data 134 4.9 Discussion 142 FIVE CONCLUSIONS AND RECOMMENDATIONS 145 APPENDICES 1 INFLUENCE OF X, ON P{F" , , > F } 156 vi,V2;Xi,X2 a;vi,V2 2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR GLOBAL OPTIMIZATION 159 3 STATISTICAL INDEPENDENCE OF d'V"'d/a^ AND SSE/o ^ 164 4 THEOREM 3.1 168 5 THEOREM 3.2 169 6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION 171 7 EQUIVALENCE OF SSEg AND SSloF WHEN REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS 172 V
PAGE 6
8 LEMMA 4.1 175 9 PROOF OF THEOREM 4.1(1) 178 10 PROOF OF THEOREM 4.1(11) 182 11 PROOF OF THEOREM 4.1(111) 185 12 PROOF OF THEOREM 4.2 191 13 PROOF OF THE EQUALITY SSE = d'V~''"d + SSE 193 REFERENCES 198 BIOGRAPHICAL SKETCH ; 202 VI
PAGE 7
Abstract of Dissertation Presented to the Graduate Council of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy TESTING LACK OF FIT IN A MIXTURE MODEL By John Thomas Shelton May 1982 Chairman: Andre' I. Khuri Cochairman: John A. Cornell Major Department: Statistics A common problem in modeling the response surface in most systems, and in particular in a mixture system, is that of detecting lack of fit, or inadequancy, of a fitted model of the form E(Y) = Xg, in comparison to a model of the form E{Y) = Xe,+ X B postulated as the true model. One method for detecting lack of fit involves comparing the value of the response observed at certain locations in the factor space, called "check points," with the value of the response that the fitted model predicts at these same check points. The observations at the check points are used only for testing lack of fit and are not used in fitting the model. It is shown that under the usual assumptions of independent and normally distributed errors, the lack of fit test statistic which uses the data at the check points is an vii
PAGE 8
F statistic. When no lack of fit is present the statistic possesses a central F distribution, but in general, in the presence of lack of fit, the statistic possesses a doubly noncentral F distribution. The power of this F test depends on the location of the check points in the factor space through its noncentrality parameters. A method of selecting check points that maximize the power of the test for lack of fit through their influence on the numerator noncentrality parameter is developed. A second method for detecting lack of fit relies on replicated response observations. The residual sum of squares from the fitted model is partitioned into a pure error variation component and into a lack of fit variation component. Lack of fit is detected if the lack of fit variation is large in comparison to the pure error variation. This method can be generalized when "near neighbor" observations must be substituted for replicates. In this case, the test statistic (assuming independent and normally distributed errors) has a central F distribution when the fitted model is adequate and a doubly noncentral F distribution under lack of fit. The arrangement of near neighbors is seen to affect the testing procedure and its power. Vlll
PAGE 9
CHAPTER ONE INTRODUCTION 1.1 The Response Surface Problem A mixture problem is a special type of a response surface problem. First we shall define the general response surface problem and indicate the basic objectives sought in its analysis, and follow this development with a discussion of the mixture problem. In the general response surface problem, we are interested in studying the relationship between an observable response, Y, and a set of q independent variables or factors, x^, X2Â» Â•Â•Â•f Xq, whose levels are assumed controlled by the experimenter. The independent variables are quantitative and continuous. We express this relationship in terms of a continuous response function, (j) , as ^u = f(^ur '^U2' Â•Â•Â•' ^uq) " 'u where Y^ is the uth of N observations of the response collected in an experiment, and x^^^ represents the uth level of the ith independent variable, u = 1, 2, . . . , N, i = 1, 2, ..., q. The exact functional relationship,
PAGE 10
observation. It is assumed that E(e^J) = 0, E(e^e^l) = 0, for u * u\ and E(e^) = a^, for u = 1, 2, ..., N. As the form of (j) is unknown and may be quite complex, a low order polynomial (usually first or second order) in the independent variables xj^, X2, ..., Xg is generally used to approximate 41. This may be justified by noting that such polynomials constitute low order terms of a Taylor series expansion of <{Â» about the point ^Cj^ = X2 = ... = x = 0, (Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point out that these low order polynomials may give a poor approximation to (\) when extrapolated beyond the experimental region, and thus should not be used for this purpose. A linear response surface model may be written in matrix notation as Y = XB + e (1.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants, a. is a pxl vector of unknown parameters (regression coefficients), and Â£. is the Nxl vector of random errors. When the model is a first or a second degree polynomial, the columns of X correspond to the first or second degree powers of the independent variables x^, X2, Â•Â•Â•, Xg, or their cross products. If the model contains a constant term, Sq, the first column of X will correspond to this term, and will consist of N ones. Since E(e) = 0, an alternative representation for the response
PAGE 11
surface model of (1.1) is E(Y) = Xg . Once the form of the model that will be used to approximate 4)(X2/ X2Â» ...f Xq) is chosen, the next step is to estimate the regression coefficients, a., and then use the estimated model to make inferences about the true response function, (j) . The estimation of the elements of a. is usually accomplished by ordinary least squares techniques. For the purpose of testing hypotheses concerning the regression coefficients, a., it is assumed that g. has a normal distribu2 tion, that is, e_ ~ N (0, a !Â«) Â• Perhaps the most common objective in the exploration of a response system is the determination of its optimum operating conditions. By this we mean that it is desired to find the settings of x^, X2r ...f Xq that optimize (^ , which in some applications may be interpreted as maximizing (j) , while in other applications a minimum value of <^ may be of interest. It is also often desirable to determine the behavior of the response function in the neighborhood of the optimum. For second order response models, such an investigation can be carried out by performing a canonical analysis of the second order surface as discussed in Myers (1971). For simple systems having only one or two independent variables, the response surface may be explored by just plotting the fitted response values against values taken by
PAGE 12
the independent variables. If q = 1, implying only one independent variable, say x^^, then a twodimensional plot of the fitted response against x^^ may be used to locate the optimum, as well as to investigate the response behavior in other parts of the experimental range of x^. If q = 2, and the two independent variables are xj^ and X2f then a plot of the contours of constant response over the region specified by the ranges of the values for x^ and X2 can be used to describe the response surface. The properties that the fitted model possesses in terms of its ability to represent the true surface, <^ , depend on the settings of x^, X2, ...f Xg at which values of Y are observed. Thus the experimental design is of great importance. Much work has been done on the construction of designs that are optimal with respect to one criterion or another involving the fitted response and/or the true unfitted model. Box and Draper (1975) list fourteen criteria to consider when choosing a design for investigating response surfaces. Myers (1971) gives several designs for fitting first and second order polynomial models. A discussion of specific design considerations will not be attempted here, as such a discussion is not the focus of this dissertation, and would necessarily be lengthy. The initial steps in the analysis of a response system may be described as follows: First an attempt is made to approximate the true response function, (})(X]^, X2, Â•Â•Â•/ Xq), usually with a low order polynomial in x^, X21 ...f Xg.
PAGE 13
After the form of the model has been chosen, then comes the selection of an appropriate experimental design, which specifies the settings of the independent variables at which observed values of the response will be collected. The observed values of the response are used in estimating the regression coefficients in the model, using, in general, ordinary least squares. After a test for "goodness of fit" of the model verifies the fitted model is adequate, the fitted model is used in determining optimum operating conditions for the response system. 1.2 The Mixture Problem A mixture system is a response system in which the response depends only on the relative proportions of the components or ingredients present in a mixture, and not on the total amount of the mixture. For example, the response might be the octane rating of a blend of gasolines where the rating is a function only of the relative percentages of the gasoline types present in the blend. The proportion of each ingredient in the mixture, denoted by xj^, must lie between zero and unity, i = 1, 2, ..., q. The sum of the proportions of all the components will equal unity, that is. q < X. < 1, i = 1,2,. ..,q, I. X. = 1. (1.2) i=l The factor space containing the q components is represented by a (q l)dimensional simplex. For q = 2 components, the factor space is a straight line, whereas for q = 3
PAGE 14
components, the factor space is an equilateral triangle, and for q = 4 components, the factor space is represented by a regular tetrahedron. The objectives in the analysis of a mixture response system are, in general, the same as in any response surface exploration. That is, one seeks to approximate the surface with a model equation by fitting an equation to observations taken at preselected combinations of the mixture components. Another objective is to determine the roles played by the individual components. We shall not concern ourselves with this but rather concentrate on the empirical model fit. Once the model equation is deemed adequate an attempt is made to determine which of the component combinations yield the optimal response. The models used to represent the response in a mixture system are in most cases different in form from the standard polynomial models. The first type of model form that we discuss is the canonical polynomial suggested by Scheffe. 1.2.1 Mixture Models Scheffe (1958) introduced a canonical form of the polynomial model for representing the response in a mixture system. These canonical polynomial models are derived from the standard polynomials using the restrictions on the Xj^ shown in (1.2). With q = 2 mixture components, for example, the standard second order polynomial model is of the form 2 2
PAGE 15
Restrictions (1.2) imply that ag = aQ(Xj^ + X2)f ^1 = ^1^1 " ^2^' ^"^ ^2 "" ^2^1 " ^1^' ^^^^ (1.3) can be written in the canonical form E(Y) = S^x^ + ^2X2 + e^2^^2' where e^= a^ + a^ + a^^, ^2 = ^0 "" '"2 "^ ^22' ^"^ ^12= "l2 a a . There is no constant term in the above canonical form and the pure quadratic terms in equation (1.3) have been absorbed in the x^Xj terms. The general form of the canonical polynomial of degree d in q mixture components can be written as E(Y) = E e, X. , for d = 1, i=l E(Y) = Z g.x. + Z Z B..X.X. , for d = 2, and ill ^. . i;] ID 1=1 l 4 in q components does not explicitly appear in the literature, but is mentioned in Scheffe (1958). If terms of the form 6ijXiXj(xi Xj) are removed from the full cubic model (1.4), then the remaining terms
PAGE 16
8 make up what is referred to as the special cubic model. For example, for q = 3 components, the special cubic model is E(Y) = B^x^ f 02^2 ^ ^3^3 ^ ^12^1^2 "^ ^3^1^3 ^ ^23^2^3 ^ H23^l"2"3 * Scheffe's canonical polynomial models are used for approximating the response surface in many mixture systems. Their popularity stems from the ease in interpreting the coefficient estimates, especially when the models are fitted to data collected at the points of the associated designs (see Section 1.2.2). However, other models have been introduced which seem to better represent the response when the components have strictly additive blending effects. We present some of them now. Becker (1968) introduced three forms of homogeneous models of degree one which he recommends instead of the polynomial models when one or more of the mixture components have an additive effect or when one or more components are inert. A function f(x, y, ..., z) is said to be homogeneous of degree n when f(tx, ty, ..., tz ) = t'^f(x, y, ..., z), for every positive value of t and (x, y, ..., z) * (0, 0, ..., 0) . These models, which Becker refers to as models HI, H2, and H3 , are of the form
PAGE 17
q q HI: E(Y) = E g.x. + E Z B..min(x., x.) + ... i=l ^ l
PAGE 18
10 Another model form that is useful in the study of the response in a mixture system is the model containing ratios of the component proportions. A term such as x.^/x^ measures the relationship of x^^ to x^ rather than the percentage of each in the blends. Snee (1973) points out that the ratio model presents a useful alternative to the Scheffe and Becker models in that the ratio model describes a different type of curvature. He notes that the curvilinear terms for the Scheffe and Becker models, when plotted as a function of XÂ£, are symmetric functions about Xj^ = 1/2, whereas the ratio term Xj^/xj is a monotone function when plotted against XiThe terms in the ratio models may also contain sums of the components. For example, with q = 3 components, we might express the second order model q1 q1 q1 E(Y)=e+ EB.Z.+ ZZ B..Z.Z.+ E Q . . z. Â° i=l ^ ^ l
PAGE 19
11 When one or more of the components is inactive, Becker (1978) suggests that a ratio model that is homogeneous of degree zero in the remaining components is appropriate. In three components, such a model is of the form E(Y) = Bq + ^^x^/{x^ + x^) + e2X2/(X2 + x^) + 33X3/(x^. X3) H ^^E^Z Bijh..(x., x.) ^ ^123^123^^1' ^2' ^3^' ^^'^^ where h^^^ and hj^23 ^^Â® specified functions that are homogeneous of degree zero. The function hj^23 ^^ intended to represent the joint effect of all three components simultaneously. If in fitting a model of the form (1.5) we determine the model should be E(Y) = 3q + e^Xj^/Cx^ + X2) + B;l2^12^^1' ^2^ then component three is said to be inactive and is removed from further consideration. The model of equation (1.5) may produce an extreme value near the vertices of the simplex factor space when there are no inactive components. In this case it is suggested that a model of the form (1.5) be used only when the proportions are restricted so that no two of the x^ are simultaneously very close to zero. Beclcer notes that other authors who have suggested ratio models have also
PAGE 20
12 used them primarily over a subregion inside the simplex factor space. Apparently this is where they are most appropriate . 1.2.2 Experimental Designs for Mixtures As in the general response surface problem, one of the major concerns in exploring a mixture system is that of choosing the experimental design for collecting observed values of the response that will be used in fitting the model. Scheffe (1958) proposed the {q,m} simplex lattice designs for exploring the entire qcomponent simplex factor space. In these designs, the proportions used for each component have the m + 1 values spaced equally from zero to one, Xj^ = 0, 1/m, 2/m, ..., (m l)/m, 1, and all possible mixtures with these proportions for each component are used. The number of design points in the {q,m} simplex lattice design is ('"*"'' ^ ~ 'Â•) . The main appeal of these designs is that they provide a uniform coverage of the factor space. Another feature, which Scheffe (1958) demonstrates, is that the parameters of the canonical polynomial of degree m in q components are expressible as simple linear combinations of the true response values at the design points of the {q,m} simplex lattice. The {3,2} simplex lattice, which consists of six design points, is represented on the two dimensional simplex in Figure 1 along with the triangular coordinates (xj^, X2, X3). Scheffe (1963) also developed the simplex centroid designs consisting of 2^ 1 points, where the only mixtures
PAGE 21
13 considered are the ones in which the components present appear in equal proportions. That is, in a qcomponent simplex centroid design, the design points correspond to the q q permutations of (1, 0, 0, ..., 0), the (2) permutations of q (1/2, 1/2, 0, ,.., 0), the (3) permutations of (1/3, 1/3, 1/3, 0, ..., 0), ..., and the point (1/q, 1/q, ..., 1/q). This design alleviates the problem inherent in the {q,m} simplex lattice designs of observing responses at mixtures containing at most m components. To give an example, the q = 3 simplex centroid design is made up of 2^ 1 = 7 design points, and is equivalent to the {3,2} simplex lattice design augmented by the center point (xj^, X2, X3) = (1/3, 1/3, 1/3). This design is represented in Figure 2. Scheffe (1963) mentions that the number of parameters in the polynomial model of the form q q q E(Y) = E 3.x. + E E e..x.x. + E E Z e..,x.x.x, i=i ' ' i
PAGE 22
14 (^^0) (0.1,0) X =1 Z v ( 1,0,0) ("i'i) ii'Â°'i) (0,0,1 ) x = l 3 Figure 1. The {3,2} simplex lattice design. ii'h") (0,1 ,0) (1,0,0) ("4.]:) (5'Â°'i) (0,0,1) 2'2 *3=' Figure 2. The q = 3 simplex centroid design.
PAGE 23
15 therefore are natural models to fit using the simplex centroid design. Ratio models may be desirable when the interest in one or more of the mixture components is in terms of their relationship to one another, rather than in terms of their percentages in blends. Kenworthy (1963) proposed factorial arrangements for ratio variables. An example of the use of ratios is the following three component system in which the mixture components are constrained by the upper and lower bounds : .2 < X < .4, .2 < X < .4, .3 < x < .5. (1.7) The ratio variables of interest are z^ = X2/X]^ and ^2 ~ ^2/^3' ^^^ ^^ ^^ desired to fit either a first or a second order polynomial model in z^ and 22^ For such a problem, we can define a 2^ and a 3^ factorial design that can be used for fitting the first and second order polynomial models, respectively, by taking as design points the intersection of rays passing from two of the three vertices of the twodimensional simplex through the region of interest defined by the constraints (1.7). Kenworthy 's 2 factorial design is shown in Figure 3. Becker (1978) uses rays extending from one or more vertices of the simplex factor space to the opposite boundaries in developing "radial designs." These designs are suggested for detecting the presence of an inactive
PAGE 24
16 X = I Design Points X =1 2 X : 3 Figure 3. Kenworthy's 2^ factorial design component or in another case a component which has an additive effect, when models containing ratio terms that are homogeneous of degree zero are fitted. McLean and Anderson (1966) suggest an algorithm for locating the vertices of a restricted region of the simplex factor space which is defined by the placing of upper and lower bounds on the mixture component proportions. The vertices of the factor space and convex combinations of the vertices are the candidates for design points for fitting a first or second degree polynomial model in the mixture components. One drawback of the "extreme vertices" design is that the design points are not uniformly distributed over the factor space resulting in an imbalance in the variances of Y(x), see Cornell (1973).
PAGE 25
17 Another method that has been suggested for studying the response over a subregion of the simplex mixture space is to transform the q mixture components into q 1 independent variables. Transforming to an independent variable system was first suggested by Claringbold (1955) and later proposed by Draper and Lawrence (1965a, 1965b) and Thompson and Myers (1968). Standard response surface polynomial models in the transformed variables can be fitted to data values collected on standard designs and a design criterion such as the average mean square error of the response can be employed when distinguishing between designs. Thompson and Myers (1968) suggest the use of rotatable designs (see also Cornell and Good, 1970). Designs other than rotatable designs, such as multiple lattices and symmetricsimplex designs, to name a few, have been suggested in the literature for fitting models to a mixture system which may be appropriate depending on particular experimental situations. However, as the intent here is not to give an exhaustive list but only a sampling of available designs, we shall not discuss designs further but instead state the purpose of this work. 1.3 The Purpose of this Research; Investigation of Procedures tor Testing "a" Model Fitted in A Mixture System for Lack of Fit A common problem in modeling the response in a mixture system is that of detecting lack of fit, or inadequacy, of a fitted model of the form E(Y) = Xg when the true model is of the form E(Y) = XQ^ + X B2The statistical literature
PAGE 26
18 suggests several procedures for testing lack of fit, which will be described in Chapter Two. In general, the authors of these procedures are not specific in stating hypotheses to be tested and do not adequately discuss the power of their procedures. The major purpose of this research is to investigate the power of two of the testing procedures appearing in the literature in detecting the inadequacy of a fitted model when the general form of the true model is specified. Our findings for a "check points" lack of fit testing procedure are presented in Chapter Three while Chapter Four contains findings for a "near neighbor" lack of fit testing procedure. For both procedures, explicit formulas for the power of the test are given in terms of cumulative probabilities of either the noncentral F or doubly noncentral F distribution, which are derived by assuming that the response observations are independent and normally distributed. Additionally, we propose methods for maximizing the power of the testing procedures. In the final chapter, we make some concluding comments concerning both of these procedures.
PAGE 27
CHAPTER TWO LITERATURE REVIEl'V Â— TESTING FOR LACK OF FIT 2.1 Introduction Let us return to the general response surface problem and assume the true response is to be approximated by fitting a model of the form E(Y) = XSj^ (2.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants, and ij^ is a px 1 vector of unknown regression coefficients. We wish to consider the situation in which the true model contains terms in addition to those in the fitted model. Then the true model has the form E(Y) = Xe^ + Y.^_^ (2.2) where X2 is an Nxp2 matrix of known constants, and e.2 is a P2>:1 vector of unknown regression coefficients. We assume that the vector Y has the normal distribution with 2 var(Y) = Ij^ . It is desirable to determine the suitability of the fitted model given by Eq. (2.1) when in reality the true model is of the form given by Eq. (2.2). The process of 19
PAGE 28
20 making this determination is referred to as testing for lack of fit of the fitted model. There are three general approaches to testing for lack of fit. The first approach requires that there be replicate observations of the response at one or more design points, and involves partitioning the residual sum of squares from the fitted model into a sum of squares due to lack of fit and a sum of squares due to pure error. A large value for the ratio of the mean square due to lack of fit to the mean square due to pure error provides evidence for lack of fit. If replicate observations are not available then the above approach to testing for lack of fit cannot be used. Green (1971), Daniel and Wood (1971), and Shillington (1979) have proposed alternative methods that are applicable in this case. Their approach is to group values of the response which are observed at similar settings of the independent variables and to call these grouped values "pseudoreplicates" or "near neighbor observations." They then treat these pseudoreplicates as they would treat true replicates to form statistics for lack of fit testing, although arriving at their respective statistics through different approaches. A third approach to testing for lack of fit involves the use of "check points." In this method a model of the form (2.1) is fitted to data at the design points and additional observations are collected at other points in the experimental region. The additional points other than the
PAGE 29
21 design points are called check points, and the data at these check points are not used in fitting the model. Lack of fit is tested by comparing the values of the response observed at the check points to the values of the response which the fitted model predicts at these same check points. We now discuss the first method mentioned above of testing for lack of fit which involves partitioning the residual sum of squares. 2.2 Partitioning the Residual Sum of Squares The method for testing lack of fit which makes use of a partitioning of the residual sum of squares from the fitted model requires there be replicate observations of the response at some of the design points (Draper and Smith, 1981, p. 120). When a model of the form (2.1) is fitted, the residual sum of squares is defined as n. SSE = E Z (Y. . Y. )^ i=l j=l ^J = Y'(Ifj X(X'X)~''"X' )Y where n is the number of distinct design points, nj^ > 1 is the number of replicate observations at the ith design point, Yj^^ is the jth observed value of the response at the ith design point, Yis the value which the model of the form in Eq. (2.1), fitted by ordinary least squares techniques, predicts for the response at the ith design n point, and N = i: n. . Using the replicated observations i=l ^
PAGE 30
22 only, a pure error sum of squares can be calculated as n . " ^ 2 SSE = E I (Y. . Y. ) , Pl^re ^^^ j^^ 1] ^1.' ' where Y^ ^ is the average of the values of the response observed at the ith design point. The sum of squares due to lack of fit can be obtained by taking the difference SS^^^ = SSE SSE LOF pure This partitioning of the residual sum of squares is displayed in the analysis of variance table in Table 1, Table 1. Analysis of Variance Â— Partitioning the Residual Sum of Squares. Source
PAGE 31
23 To test the hypothesis of zero lack of fit, that is Hq: lack of fit = or E(I) = X^x' ^^ F statistic is formed ^ = MSe'' (2.3) pure which possesses a central F distribution if the true model is of the form (2.1), but has a noncentral F distribution if the true model is of the form (2.2). In other words F ~ F np,Nn under H : E(Y) = X3,^ , and und F ~ F' np,Nn;A er H : E(Y) = XB, + X232 ' where X 2 is the noncentrality parameter X = ^^(X2XA) ' (X2XA)B2/2a ' ^"^ ^ = (X'X)"^X'X2' Under H3, E(MS^Qp) = a^ + e^(X2 XA)'(X2 XA)B2/(i^P) ^"^^ E(MSE j.g) = a^ (Draper and Smith, 1981, p. 120), hence Hq is rejected in favor of H^ if the value of F in (2.3) exceeds the upper 100a percentage point of the central F distribution, Fa;np,Nn* ^^^" ^0 ^^ rejected, we conclude that a significant lack of fit is present. Draper and Herzberg (1971) demonstrated that the lack of fit sum of squares can be partitioned into two statistically independent sums of squares, SSj^j and 53^2' when there are replicate observations at the center of the
PAGE 32
24 response surface design and when the basic design without center points is nonsingular. If the true model and the fitted model are of the same form as in equation (2.1) then the two F ratios F^ , = rss^,/(n p 1)1/MSE and LI 'LI J ' pure Fl2 = SSL2/^S^pure ^^^ both distributed as central F random variates, with respective numerator and denominator degrees of freedom (n p 1), (N n) for F^j^ and 1, (N n) for Fl2' ^f ^^^ true model is of the form shown in equation (2.2), then F^j^ and Fl2 ^^^ both distributed as noncentral F random variates. The expected values of SSlj^ and 83^2 ^^^ used to show what functions of e.2 ^^Â® testable with F^jl and fL2Two examples are presented by Draper and Herzberg to illustrate this testing for lack of fit. The first example makes use of a first order orthogonal design in k factors augmented with center point replicates for fitting a first order polynomial model . If the true model is of the second order, then Ft2 can be used to test a hypothesis concerning the parameters associated with the pure quadratic terms in the model. If all such parameters are zero, then Fj^j^ provides a check on interaction terms. The second example illustrates the fitting of a second order polynomial model to a second order design with all odd design moments of order six or less zero. If the true model is third degree, then Fj^j^ can be used to test the significance of the third order terms, while Fl2 tests terms of order greater than three. The partitioning of SS^qf into SSlj^ and SSl2 ^^^ ^^^
PAGE 33
25 corresponding tests of hypotheses are also given in Myers (1971, p. 114119), for the special case of fitting a first order polynomial model to a 2*3 factorial or a fraction of a 2*3 factorial design augmented with center point replicates and the true model is of the second degree. A more complete partitioning of the lack of fit sum of squares in an attempt to obtain a more detailed diagnosis of the lack of fit of the fitted model is given in a technical report written by Khuri and Cornell (1981). The lack of fit sum of squares, which has n p degrees of freedom, is partitioned into n p independent sums of squares, each having one degree of freedom. The expected values of these single degreeoffreedom sums of squares are used to identify at most n p linearly independent causes for the lack of fit variation. Tests of significance are performed on the assumed contributing causes. This method enables the screening of all subsets of 2.2 in order to identify those subsets which are most responsible for lack of fit of the fitted model. We shall now discuss the second general approach used in lack of fit testing, which is to test for lack of fit by making use of response values observed at points which are near neighbors in the factor space when true replicate observations are not available.
PAGE 34
26 2.3 Testing for Lack of Fit Without Replicated Observations Â— Near Neighbor Procedures Green (1971) suggests the following approach when testing for lack of fit if there are no design points at which replicate observations of the response are available. The N observed values of a response, Y, considered a function of only one variable, x, are divided into g groups, by grouping observations which have similar values of x. Green hypothesizes a model of the form Y= Ha + Â£., where X is an Nxl vector of observable responses, H is an Nxm matrix whose columns correspond to known functions of the variable, x, a. is an mxl vector of unknown regression coefficients, and e. is the Nxl vector of random errors, e ~ N^(0, a^Ijj). Green's method assumes that the vector of differences ( EY Hg.) can be well approximated by a dth order polynomial in X within each of the g groups, d > 1. An alternative model of the form Y = H V + n + Â£ is given, where S. is distributed as N^(Q, a^I^)/ Hj^ is an Nx[g(d + 1) + mj^] matrix of known constants, ii is a [g(d + 1) + mj^]xl vector of regression coefficients, and u., as Green states is "a small vector." The first g(d + 1) columns of H^ correspond to the polynomial terms for the g groups (with (d + 1) terms for each group), the rightmost m < m columns in Hj^ correspond to terms that are in the
PAGE 35
27 fitted model, but are not represented among the g(d + 1) polynomial terms in the alternative model. Under the assumption that n = Q, the presence of lack of fit is tested by using the test statistic: Y' [H (H'H )"'H H(H'H)"'H']Y/[g(d +1) + m^ m] F = ^^^ . Y'[I H^(HjH^)"^H]Y/[N g(d + 1) m^] (2.4) This statistic is of the same form as the F statistic used in the standard multiple regression test of a postulated model against a more general one which includes the postulated model as a special case. Lack of fit is suspected if the calculated F ratio in (2.4) is greater than Fcc;g(d+l)+mim, Ng(d+l)mi "^^^^ ^his latter quantity is the upper 100a percentage point of the central F distribution. Green notes that when there is no lack of fit, the quadratic forms Y' [ H , (H 'H , ) ""'"H' H(H'H)~ H'JY and Y'[l H (H'H )"'H']y are distributed independently as a\^ with g(d +1) + mj^ m and N g(d +1) mj^ degrees of freedom, respectively. In this case the F ratio in (2.4) possesses a central F distribution. If there is lack of fit on the other hand, then these two quadratic forms are distributed as noncentral chisquares, multiplied by a'', with respective noncentrality parameters
PAGE 36
28 ?j_ = [Hj_v + n] Â•[Hj^(HJ_Hj^)"^H'^ H(H'H)""'"H' ][h^v + n] and C = ri_'[l H (H'H )~ H']ri . Thus the assumption that n = can affect the power of the test, since if n * , the expected value of MSE is greater than o^ , where MSE is the quadratic form in the denominator of the F ratio. Hence if r\ * , the probability of calculating a large F value is reduced, and we are less likely to detect lack of fit using an upper tailed rejection region. Daniel and Vtood (1971) suggest another method for lack of fit testing when replicated observations of the response are not available. They make use of "near replicates" to obtain an estimate of a, which is the standard deviation of the observable responses in the true model. The value of the estimate a is compared to the square root of the residual mean square from the analysis of the fitted model. Lack of fit is indicated if the square root of the residual mean square is large compared to the estimate a. To determine when observations are near replicates so that an estimate of a can be found, they define the squared distance between any two data points, j and j', to be measured by where Xj_j and x^ji are the values of the ith independent variable corresponding to the observations yj and Yj > ' respectively, i = 1, 2, ..., K, and b^ is the ordinary least
PAGE 37
29 squares estimate of the ith regression coefficient. In the denominator, s is the square root of the residual mean square for the fitted model. To obtain an estimate of a from near replicates, let Aj^d = d4 dj,, n = 1, 2, ..., (2), where dj and dj 1 are the residuals at points j and j', respectively, and where there are N data observations in the experiment. Since the expected value of the range for pairs of independent observations from a normal distribution is 1.128a, a running average of the A^d's is calculated and their average is multiplied by .886 = (1/1.128) to get a running estimate, Sj^, of a. That is, s^ = .836 ^ A^d/n . The closest pair of observations as judged by D?; 1 is used to begin the running estimate, the next closest pair (next "nearest neighbors") is used for A2d, and the procedure continues until s^ "stabilizes." The stabilized value of s^ is used to estimate a. A third method for testing for lack of fit without replication is given by Shillington (1979). The fitted model is of the form Y = X3 + e (2.5) where Y (Nxl), X (N^p), and Â§. (p^l) are defined as in (1.2) and e ~ '^m^' Â°^^N^ * '^^^ ^^^^ ^Â°^ "''^^^ Â°^ ^"""^ Â°^ ^^Â® fitted model is a test for whether the true model has the form Y = X6 +6 + Â£ ,
PAGE 38
30 where 6 (Nxl) is a fixed effect quantifying the departure of (2.5) from the true model. Shillington assumes that the data can be grouped into g cells, with nj observations in the jth cell, determined in advance. Letting Cj refer to the jth cell, j =1, 2, ..., g, a vector of cell averages is written Yp (gxl), where the jth element of Y^ is the average of the observed responses in Cj . The matrix X^ of independent variables associated with Y^ is the gxp matrix where the elements in the jth row n . 3 are x' . = Z x'. ./n. , that is, row j of Xp is the row O i^i 1] J vector x' . . The matrix Xp is assumed to be of full rank p < g. Also within each cell are defined the differences W. . = Y. . Y . , i e C. , j = 1, 2, ..., g, where Y . is ID ID D _ D D the jth element of Yp. The two independent data sets, Yp and {WÂ•} with g and N g degrees of freedom, respectively, are used to find two independent estimates of a^. The first estimate is written as MSE^ = Z n.(Y . x'.gÂ„)V(g " P) / where g_g is the weighted least squares estimate of g. using the regression of cell means, Y^^,, on X^. The second estimate of a uses the within cell deviations on cell means, {Wj^^}, and is g "d . , MSE,, = 2 Z (W.. W. )^/(N g r), ^^ j=l i=l ^^ ^^
PAGE 39
31 where r is the rank of an N^p matrix v/ith rows equal to x* _ xÂ« , i e C, j = 1, 2, .., g. If the matrix of ij O D independent variables, corrected for cell means, is of full rank, then r = p. Here VJ^is the estimate of W^j from the regression of cell residuals i^W^^j} on the associated vectors of independent variates, x! . x'. . If the fitted model is the correct model, then MSEg and MSE^ are independent estimates of a^ and the ratio MSEg/MSE^ is an F statistic with g p and N g r degrees of freedom. When all observations in a cell have the same settings of the independent variables, that is, the observations are truly replicates for all cells, then this F statistic is identical to the F statistic in the usual lack of fit test in which the residual sum of squares is partitioned into lack of fit and pure error sums of squares, as given in Draper and Smith (1981, p. 120). If the true model is Y = XB + 6 + e , however, and if we let X'6 =0 and <^^ = 5'6/N, then 2 + ?1[I X^(X'X^)"^X']^Â„/(gp) E(MSEg) = a+ 6^[I X^{X^X^) X^]^^/ n . 3 where 6 (g^l) has jth component equal to I ^^^/n^ Â• B .^^ 13 3 Furthermore, with this latter true model form E(MSE^) = a2 + 6^j(l X^(X^ X^) "^X^) 6^/(N g r)
PAGE 40
32 where Â§_^ has the components 5^^S", i e c.;, j = 1, 2, ..., g. The matrix Xr., (Nxp) has the rows x! . x' . , w _ 1  _ , j ' i e C., j = 1, 2, ..., g. The power of the F test, F = MSEg/MSE^, depends on the relative bias of the estimates of a^, that is, the biases in MSEg and MSE^. Shillington states that the power of the F test which makes use of F = MSEg/MSE;^ is maximized by forming cells so that the bias of E(MSE^) is minimized. This is the same as forming cells so that the within cell variation in 6 is minimized. Shillington (1979, p. 141) also states, "Observations with near covariate (independent variable) values might be expected to have similar 6 values, since we assume that Â§_ varies in some continuous but unknown fashion with X. This justifies the usual procedure of forming groups by collapsing observations with adjacent covariate values. Indeed, if covariates do not vary within cells we have the usual lack of fit test and maximum power." By imposing a further structure on the form of Â§_, it is shown that if the F test has an upper tailed rejection region, the power is maximized by selecting the group sizes as n; = 2, j = 1, 2, ..., g. Finally, Shillington suggests that in the presence of more than one independent variable problems in grouping may arise, and in this case it may be wise to perform a different lack of fit test for each parameter. Following this approach, an example is given which suggests testing lack of fit for each of the p independent variables separately may be more powerful than
PAGE 41
33 trying to form groups based on all independent variables at once. In summary, all the approaches we have discussed for testing for lack of fit when replicate observations of the response are not available at any of the settings of the independent variables make use of grouping the observed response values according to similar values of the independent variables. The observations falling in such groups are referred to as "pseudoreplicates" or "near neighbor observations." These pseudoreplicates are used to estimate the true variance of the observations, a^, but a completely unbiased estimate of o^ cannot be attained unless true replicate observations are available. In each case, the power of the lack of fit testing procedure is reduced because an unbiased estimate of o^ is not attainable. We now turn to the use of check points for lack of fit testing. 2.4 Testing for Lack of Fit with Check Points An alternative to the two approaches to lack of fit testing already discussed is the method which makes use of check points. We assume a model of the form E(Y) = XB, , as given in (2.1), is fitted in a response surface system, but that the true model is of the form E(Y) = Xg + X 3 as given in (2.2). The parameters, dif in the fitted model are estimated by ordinary least squares techniques, making use of the values of the response observed at the design points. After the model is fitted, values of the response are observed at additional points in the experimental region
PAGE 42
34 > called "check points." The observed response values at the check points are compared to the values which the fitted model predicts at these same check points. It is important to note that the observed values of the response at the check points are not used in fitting the model initially. Snee (1977) gives four methods of validating regression models, one of which is the collection of new data to check predictions from a previously fitted model. In a designed experiment these new data take the form of check points. Snee suggests that the inclusion of a small number of check points in any designed experiment is a "worthwhile" procedure. Scheffe (1958) proposed a test for lack of fit when the {3,2} simplex lattice design is used for fitting a second order canonical polynomial model in three mixture components. It is desired to use the observed value of the response at (1/3, 1/3, 1/3) as a check point blend. The test statistic proposed is the t statistic of the form t = Â— ^ ^ 1 Â— ,7oÂ— (2.6) [var(Y Y)]"/^ where Y is the observed value of the response at the check point, and Y is the value of the response predicted at the same point by the second order model which is fitted by ordinary least squares techniques to the observed response values at the six design points of the {3,2} simplex lattice. The response value observed at the point
PAGE 43
35 (1/3, 1/3, 1/3) is not used in fitting the model. Lack of fit is inferred if the absolute value of the calculated t value in equation (2.6) is larger than the corresponding tabled t value. In the denominator of the t test of equation (2.6), the variance of the difference Y Y is shown to be var(Y Y) = var(Y) + var(Y) = (44/27r)a^ , when r replicates are taken at each design point. The estimate of the variance of Y Y is (44/27r)a2, where a^ is calculated from the replicated response values at the design points . Scheffe (1958) also alludes to a test for lack of fit when several check points are used simultaneously. When there are k check points, the test for lack of fit is an F statistic of the form F = V(2.7) ko^ where d' = (Y^ Y^, Y^ X ^, ..., Yj^ Yj^ ) , and V = a ^Vq = var(d). Formulas are given for the elements of Vq in the special case when the check points are the design points of the {3,2} simplex lattice. Lack of fit is suspected if the calculated value of the F statistic given in (2.7) is larger than the corresponding tabled F value.
PAGE 44
36 Gorman and Hinman (1962) suggest the same t test in equation (2.6) that Scheffe (1958) suggested for a check point taken at (1/3, 1/3, 1/3) to test for lack of fit in a second order polynomial model fitted from a {3,2} simplex lattice design. They suggest using (1/3, 1/3, 1/3) as the location of the check point because the observation at this point may later be used to fit the next more complex model, the special cubic, if the second order model is found to be inadequate. They state that in general for the second order polynomial model as well as higher order models, check points should be taken in regions of particular interest, of which there are usually many in any blending study. Further, they suggest that the number of check points depends on individual experimental situations Â— technical background, precision required, cost of materials and analyses, and probability of requiring a more complex model. However, no specific criterion is given by Gorman and Hinman for selecting the location of the check points. Gorman and Hinman (1962) indicate that a t test at a check point other than at (1/3, 1/3, 1/3) takes the same form as the statistic of equation (2.6), t = Y ^ [var(Y) + var(Y)]'/^ with the additional condition that if several check points are taken, say for example k points, the method of checking
PAGE 45
37 the fit is to compute the t value at each location and refer these calculated t values to the 100(a/2k) percentage point of the central t distribution rather than the 100 (a/2) percentage point. Kurotori (1966) gives an example of a mixture experiment where the response is the modulus of elasticity of a rocket fuel, which is a mixture of three components, binder (X]^), oxidizer {^2)1 and fuel (X3). The factor space of feasible mixtures is a subspace inside the twodimensional simplex or triangle where all three components are present simultaneously. "Pseudocomponents" are defined and in the pseudocomponent system a special cubic model is fitted to data collected at the points of the q = 3 simplex centroid design (Figure 4). A check for adequacy of fit is made by using three check points and the response values at the check points are used only for testing the fit of the model and not for fitting the model initially. The reason for the choice of the particular check point locations by Kurotori is that, as he states, "They are the most remote mixtures from the seven design points." The lack of fit test is an F statistic of the form 2 F = ly(2.8) a 2 3 9 where s = z (Y. Y.)^ , for the i = 1, 2, 3 check points .2 . i=l and a is an estimate of measurement error from a previous analysis. Kurotori admits that the use of the F statistic
PAGE 46
38 (1,0,0) # Design Points O Check Points (i'a'")/ r {5.0,1) (0,1,0) Â« 1^^_^ 1 "'" ^ (0,0,1) Hi) Â« =1 [
PAGE 47
39 In summary, only Scheffe refers to an exact F test when several check points are considered simultaneously for testing for possible lack of fit of a model fitted in a mixture space, and his development is limited to the special case where the check points are the design points used to fit the model initially. No criterion is proposed by Scheffe for selecting other locations for the check points.
PAGE 48
CHAPTER THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING LACK OF FIT IN A MIXTURE MODEL 3.1 Introduction In Chapter Three we investigate the problem of testing for lack of fit of a linear model fitted in a mixture space. The testing is to be accomplished with the use of check points. We assume that an experimental design is specified, and that the fitted model is of the form E(Y) = X3j^ (3.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants and rank p, and 3 is a vector of p unknown regression coefficients. The true model is assumed to be of the form E(Y) = XBj^ + X^^^ (3.2) where X2 is an Nxp2 matrix of known constants and ^2 ^^ ^ vector of po unknown regression coefficients. Throughout our development, we will assume that the random vector Y has the normal distribution with variancecovariance matrix equal to o I^. 40
PAGE 49
41 In our investigation we wish to determine the proper testing procedure to follow in deciding whether the fitted model exhibits lack of fit. In order to optimize the lack of fit testing procedure, we will determine the location of the check points so that the power of the test is maximized. 3.2 Testing for Lack of Fit in the Presence of an~ External Estimate of Experimental Error Variation 3.2.1 The Test Statistic We wish to test the performance or fit of a fitted model in a mixture space when the true model possibly contains terms in addition to those in the fitted model. The fit of the model is to be tested by a test which makes use of the response values observed at certain locations called "check points" in the experimental region, by comparing them to the values which the fitted model predicts at the same check points. The observed values at the check points are not used for estimating the coefficients in the fitted model and are assumed to represent the values of the true surface at the check points. Let us define the vector of differences d = (Y* Y*) (Y* Y*, Y* Y*, ..., Y* Y*)' where Y*, i = 1, 2, ..., k are observed response values at k check points and Y*, i = 1, 2, ..., k are response values predicted at the k check points by the fitted model.
PAGE 50
42 Y^ = x^'b,, where b, is the ordinary least squares estimator of 3,/ and where x*' is the ith row of X*, the kxp matrix whose columns are of the same form as the columns of X but with its rows evaluated at the k check points. Note that if 3 = 0, then E(d) =0 and if ^ * 0, then E(d) = (X* X*(X'X)~''X'X2)62Let V represent the variancecovariance matrix of the random vector d. 2 Then V = a VÂ» where Vq = Ij^ + X*(X'X) 'X*' and where Ij^ is the identity matrix of order kxk. We assume that an unbiased estimate of a^ is available and we denote this estimate by a ^ , where the subscript ext ext " 2 stands for external, and o . is independent of the model being fitted. The test statistic for the hypothesis of zero lack of fit H : E(d) = is d'V''d/k F = :rfI (3.3) ''ext (see Scheffe, 1958, p. 358). It will be shown later in this section that the F ratio in Eq. (3.3) possesses either a central F distribution or a noncentral F distribution, depending upon whether the true model is represented by Eq. (3.1) or Eq. (3.2). "2 . . The variance estimate a ^ that appears m equation ext (3.3) is ordinarily generated from replicated observations
PAGE 51
43 at some of the design points in the experiment. We assume * 2 that a ^ IS a constant multiple of a central chisquare ext random variable with v degrees of freedom. This is written as a^ ^ = SSE /v ext pure^ = (aVv)(SSEp^^^/a^ 2 2 where ^^E ^^^/a ~ x^* Note that SSEpy^g denotes the portion of the residual sum of squares due to replication variation from the fitted model. The residual sum of squares from the fitted model may be partitioned into SSEp^jg and SSj^Qp only if replicated observations are collected at one or more design points. For the case where replicate observations are collected at all of the design points n n . _ SSE = Z E''' ( Y. . Y. ) , PUi^e i=i j=i ^3 1where n is the number of distinct design points, n. > 2 is the number of replicates at the ith design point, Yj^^ is the jth observation at the ith design point, and Y. is the average of the n^ observations at the ith design point. n Here SSE j^^ has v = Z (n. 1) degrees of freedom. i=l ^ When the fitted model and the true model are of the same form as defined by Eq. (3.1), the quantity d'V~''"d/a^
PAGE 52
44 possesses a central chisquare distribution (Searle, 1971, p. 57, Theorem 2). However, when the true model is of the form specified by Eq. (3.2), d'V~''"d/a^ possesses a noncentral chisquare distribution. Thus when the true model is of the form in Eq. (3.1), d'VVa' ~ X^ but when the true model is of the form in Eq. (3.2), d'VVc^2 ^ ^.2^^^ where in the second case the noncentrality parameter X. has the form Xj_ = E(d)'VQ^E(d)/2a^ = 3^(X* X*A)'Vq^(X* X*A)^^/2a^. The matrix A = (X'X) X'XÂ„ is called the alias matrix and is of order pxp . In X,, the matrix X* is of order kxp2 a has the same relationship to X2 as X* has to X. 2 Since SSE /a is statistically independent of pure"^ 12 . . d'V_ d/a , then under model (3.1) the test statistic nd
PAGE 53
45 d'V """d/ka^ F = SSE /va^ pure^ ''ext will have a central F distribution. When the true model contains terms in addition to those in the fitted model then F will have a noncentral F distribution. We write these two cases as F ~ F k,v under model (3.1), and K,v ;Xi under model (3.2), where the noncentral ity parameter is Xj_ = S^(X* X*A)'Vq'(X* X*A)l^/2a . 3.2.2 The Testing Procedure and an Expression for the Power of the Test Given that the form of the fitted model is defined as Eq. (3.1), the expected value of the numerator of the F statistic in Eq. (3.3) will depend on the form of the true model. For the case where the true model is expressed as
PAGE 54
46 Eq . (3.2), E(numerator) = E (d ' V~"'"d/k ) (aVk)Ex^,^ (a^/k)(k + 2X^) 2 2 = o^ + 2a^Xj^/k = a^ + e^A^B^A, (3.4) where A^ = (X* X*A) 'V~ (X* X*A). However, when the true model is Eq. (3.1), 3 = and in this case X = so that 2 * 2 E (numerator) = a . Also a . is an unbiased estimator of ext a 2 and E(a2^^) = a\ (3.5) Therefore the ratio E(numerator )/E(denominator ) where " 2 the denominator is a , will equal unity under model (3.1), that is, when there is no lack of fit. Under model (3.2), the ratio will be greater than or equal to unity so lack of fit should be suspected if the calculated F ratio in equation (3.3) is large. We can thus use an upper tailed rejection region to reject the hypothesis of zero lack of fit. The power of the test is
PAGE 55
47 ^(^i,v;X. > ^a;k.vl where F is the upper 100a percentage point of the ex ; K / V central F distribution with k numerator degrees of freedom and V denominator degrees of freedom. It is worth noting that from Eq. (3.4) and Eq. (3.5) testing the hypothesis that S^ = is equivalent to testing the hypothesis that X^ = 0, assuming A, is positive definite. Thus testing a null hypothesis of zero lack of fit using the proposed testing procedure involving the F ratio in (3.3) may be expressed as a test of the hypotheses Hq: X^ = H : A, > 0. a X 3.2.3 A Method for Locating Optimal Check Points Once a design for fitting model (3.1) in a mixture space is chosen and the number of simultaneous check points is decided on, say k > 1, the next step is to determine where in the mixture space we should place the k check points so as to maximize the power of the test for lack of fit. The location of the check points is to be made independently of the value of 8 .
PAGE 56
48 The power of the upper tailed F test for lack of fit is an increasing function of X , (see Appendix 1 for proof, with X = 0). Therefore, to maximize the power of the test we maximize the value of X defined as Xj_ = &^A^Q^/2a where A^^ = (X* X*A) 'V~ (X* X*A), by properly selecting the k check points whose coordinates are defined in X*. To maximize the value of Xj^, we shall concentrate on the matrix A^. The matrix A^ is a square matrix of order P^^P^ and is a scalar quantity when P2 = 1. By maximizing the scalar quantity Aj^ with respect to the k check points, the power is maximized no matter what the value of 8 . Maximizing the scalar Aj^ can be accomplished by using The Controlled Random Search Procedure given by Price (1977). This procedure is described in Appendix 2. As a computational aid, Aj^ can be expressed as V + (X* X*A) (X* X*A) ' I when P2 = 1Â» where the symbol B denotes the determinant of the square matrix B. Thus the computations reduce to evaluating two determinants rather than inverting Vq (see Scheffe, 1959, Appendix V, p. 417).
PAGE 57
49 When p> 1 and A, is no longer a scalar, maximizing X j^ (and thus maximizing the power of the test) cannot be accomplished without specifying B . In this case we make use of a lower bound for X^ (Graybill, 1969, p. 330, Theorem 12.2.14(9) ) defined as 2 min22^ 1 (where u . is the smallest eigenvalue of A,) to be used in place of Xi. Hence an approximate solution to the maximization of X^ will be achieved by finding the k simultaneous check points (using Price's procedure) that maximize u Â• , the smallest eigenvalue of Ai . In other mm ^ words when p > 1, and in order to avoid specifying ^ , we 2 ^ seek to maximize a lower bound value for X^^. This maximization does not depend on the value of Q^. There are cases where the matrix Aj^ ^^ Â°^ less than full rank (less than rank P2) or equivalently where the matrix Ai is positive semidefinite so that u^^^ will be equal to zero no matter which check points are selected. One such case occurs when k < p (when the number of check points is less than the number of parameters in the true model which are not in the fitted model) since when k < p^ rank(Aj^) = rank[VQ Ax* X*A)] = rank(X* X*A) ,
PAGE 58
50 and so rank(A]^) < min(k, P2) because the matrix (X* X*A) is of order kxp2. Therefore when k < p , the rank of A, is at most k so that A, is of less than full rank. Since u . Jmin must be equal to zero when A, is positive semidefinite, an alternative method to that of maximizing y to select ^ mm optimal check points must be found when A, is positive semidefinite in order to produce a positive lower bound for Xi. In this pursuit, let us write X^ as ^1 = i2^ig.2/2a^ 6_^PAP'3 2/2a^ Q^^lP^zP^] diag[Aj^, K2 = 0] [P ^iP^] 'Q_^/2a e^PlAlPU2/2c>^ where A is a diagonal matrix with elements equal to the eigenvalues of An, P is an orthogonal matrix whose columns are orthonormal eigenvectors of h^, Aj^ and P^^ correspond to the positive eigenvalues of Aj^, while A2 = ^rid P2 correspond to zero eigenvalues of A^. Then by Theorem 12.2.14(9) in Graybill (1969) we can write y"^. z'z/2a^ < X, (3.7) mm' 1 whe re y . is the smallest positive eigenvalue of Ai , and ^min iT 3 J.
PAGE 59
51 z = f{^2* Thus by Eq . (3.7), an approach to maximizing a positive lower bound for X ^ when Aj^ is positive semidefinite is to select check points that maximize the smallest positive eigenvalue of A^. It must be noted, however, that this method can only be used when 02 E n C(P^), where C(Pj^) denotes the column space of Pj^ and n C(P^) denotes the intersection of all such spaces which can be obtained at all possible check points locations. This is because, in general, z'z in (3.7) depends on the location of the check points through its dependency on Pj_. if, however, 0^ ^ nC(P ), then z'z = ep^P]^02 " Â§2^P'Â§.2 " ^22' ^^"^^ ^2^2 " Â°* It follows that when ^^ e n C(P,), u^z'z/2o^ = *^inin22^^Â° ^"^ only u^^^ depends on the location of the check points. 3.3 Testing for Lack of Fit When MSE Is Used to Estimate Experimental Error Variation 3.3.1 The Test Statistic In this section we shall show that when an external estimate of o^ is not available and the residual mean square (MSE) from the fitted model of the form (3.1) must be used as an estimate of a^, the test statistic d'V^'d/k ^ = ^llSE^ (38) possesses a central F distribution when the true model is
PAGE 60
52 Eq. (3.1), but possesses a doubly noncentral F distribution when the true model is Eq. (3.2). In the initial section of this chapter, the quantity 1 2 d'Vd/a was said to possess a central chisquare distribution or to possess a noncentral chisquare distribution, depending on whether the true model was specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum of squares from the fitted model is defined as '^ 2 SSE = E (Y. Y. ) i=l ^ ^ = Y' (Ij^ X(X'X) 'Â•X' )Y and it is easy to show (Searle, 1971, p. 57, Theorem 2) that SSE/a^ possesses a central chisquare distribution if the true model is Eq. (3.1), but under model (3.2), SSE/a^ possesses a noncentral chisquare distribution. This is expressed as SSE/a^ ~ Xn_p under model (3.1), and SSE/a^ ~ x^fp^x.
PAGE 61
53 under model (3.2), where the noncentrality parameter X 2 is ^2 " ^2^^2 " ^^)'(^2 " ^A)Â§.2/2a^ The distributional form of the test statistic in Eq. (3.8) is derived by knowing that the quantities 12 2 d'Vp, d/a and SSE/a are statistically independent (see Appendix 3), so that F = d'VQ^d/ka^ MSE/a^ d'V~''"d/k MSE is distributed as a central F when the true model is Eq. (3.1), but when the true model is Eq. (3.2) the F ratio is a doubly noncentral F, that is, under model (3.1), F ~ F k,Np and under model (3.2), k,Np;Xi ,A2 * 3.3.2 The Rejection Region and its Relation to the Power of the Test In Appendix 1 it is shown that if k, Np, and X2 are fixed, then the power of the F test using the ratio (3.8) is
PAGE 62
54 a function of the location of the rejection region (upper tailed or lower tailed) of the test. The power increases with increasing values of the numerator noncentrality parameter, Xi, when the test is an upper tailed test. The power decreases with increasing values ot X ^ when the test is a lower tailed test. This means that to study ways of increasing the power of the test, we have to determine whether the test is an upper tailed test or a lower tailed test. Similarly, for fixed values of k, Np, and X^, the power of the F test is a decreasing function of X2 ^^^ ^" upper tailed test, and is an increasing function of X 2 "hen the F test is a lower tailed test (Scheffe, 1959, p. 136137) . To decide if the test is an upper tailed test or a lower tailed test, we recall from Section 3.2.2 that if the true model is Eq. (3.1) then the expected value of the numerator of the F statistic in (3.8) can be written as E( numerator) = 0^, and if the true model is Eq. (3.2), 2 2 E( numerator) = a + 2a X ^/'k. (3.9) = 0^ + 3^A^B2A
PAGE 63
55 where the P2XP2 matrix A^ is A^^ = (X* X*A)'Vq (X* X*A) Similarly, it can be shown that if the true model is Eq. (3.1), the expected value of the denominator of the F statistic in (3.8), where the denominator equals MSE, is E(denominator) = E(MSE) = a2, but if the true model is Eq.(3.2), E (denominator) = E(MSE) [c^'/(N P)]Ex'fp,,^ [a^/(N p)][N p + 2X2] a^ + 2a^\^/{n P) (3.10) 0^ + e2A2e2/(^ " P) where the P2' 3 ' A e /(N p) . In this latter case we reject the null
PAGE 64
56 hypothesis of zero lack of fit if the calculated value of the F ratio in (3.8) is large. An upper tailed rejection region seems reasonable for this test. When the true model is Eq. (3.2), and if g^Aj^g^A < l^P^2^2'^''^ ~ P^' ^^en a lower tailed rejection region is preferred. 3.3.3. A Method for Locating Optimal Check Points Given a design for fitting a model of the form in Eq. (3.1) in a mixture space (note that fixing the design fixes ^2 and (N p)), and given the number of simultaneous check points desired, k > 1, we now wish to determine where in the mixture space the k check points should be located so as to maximize the power of the F test for lack of fit, where the test statistic is given in Eq. (3.8). We also wish to position the optimal check points in a manner that is independent of the values of the elements in g . The case of an upper tailed test. To help us find k simultaneous check points that maximize the power of an upper tailed test, we shall make use of the fact that the power is an increasing function of X,. Therefore to maximize the power of the upper tailed F test, we shall seek the locations of the k check points that maximize X,. As in the case considered in Section 3.2.3, where the test statistic had a noncentral F distribution, if the number of extra terms in the true model is P2 = 1, then maximizing X j^ is equivalent to maximizing the scalar A,. However, as before, if p > l, then the P^^Po "^^^rix A^ is not a scalar and we will have to approximate the
PAGE 65
57 maximization of X ^ by maximizing a lower bound for X^. This is done by finding the maximum value of u . , the smallest eigenvalue of A]^, since 2 ^min22' 1 When the number of check points is less than the order of the square matrix A^, that is, k < P2f then rank(A]^) < min(k, po), and A, will have y . = 0. For this case, we again try to maximize the smallest positive eigenvalue of h^ which we denote by u'*'. , while remembering from Section min 3.2.3 that this technique is limited to situations where B^ e nC(P^) . The case of a lower tailed test. To find k check points to maximize the power of a lower tailed test, we make use of the fact that the power of the lower tailed F test increases as X j^ decreases. Then if P2 = 1 and A^ is a scalar quantity, X ^ can be minimized with respect to the k check points by finding the check points that minimize Aj^. If p > 1, then by Theorem 12.2.14(9) in Graybill (1969), we see that an upper bound for Xj^ is ^1 < ^max^2^2/2<^'' ^''^^^ where u is the largest eigenvalue of Ai . An approximate ^max ^3 ^ i solution to minimizing X j^ in (3.11) can be achieved by minimizing u . It is not necessary to treat the case ^ max
PAGE 66
58 of k < p separately here, although X ^ will equal zero if g_2 is in the column space of P2 , where P2 is the matrix whose columns are orthonormalized eigenvectors corresponding to the zero eigenvalues of the matrix A, . 3.3.4 Determining Whether the Test Is Upper Tailed or Lower Tailed The procedures outlined in Section 3.3.3 produce a set of k check points that simultaneously maximize the power of the upper tailed test as well as a second set of k check points that simultaneously maximize the power of the lower tailed test. The check points that are selected maximize the power, given A2f k, and N p without specification of g_2' except that when Aj^ is positive semidefinite we require that e n C(P ). It is now necessary to decide which of our two candidates will be used for a lack of fit test. To choose between the upper tailed test and the lower tailed test, let us consider the quantity R = [A^/k] [A^/(N p)] . If R is positive definite when the true model is Eq. (3.2), then no matter what the value of is, the ratio E (numerator )/E (denominator) will be greater than unity, implying an upper tailed test is to be used. Similarly, if R is negative definite, then a lower tailed test should be used. Finally, if R is not definite, then neither an upper nor a lower tailed test is implicated and further
PAGE 67
59 investigation is necessary. The criterion of R = [A /k] [a /(N p)] may yield any of the four following cases. Case 1. If R = [Aj^/k] [a^/CN p)] is positive definite when A^ is generated by the k optimal upper tailed test check points, and R is not negative definite when A, is generated by the k optimal lower tailed test check points, then we recommend that the check points be used that yield the optimal upper tailed test with an upper tailed rejection region. For Case 1 it is necessary for A, to be positive definite (see Appendix 4). Since A, is a square matrix of order P2^P2 with rank(A ) < min(k, P2)' then A^ can be positive definite only if k > p . Thus, there must be at least P2 check points for Case 1 to hold, where P2 is the number of terms in the model of Eq. (3.2) that are not in the model of Eq. (3.1). From inspection of equations (3.9) and (3.10), it is apparent that the testing for lack of fit in Case 1 is equivalent to testing the hypothesis ^1 ^2 ^0'' Â— FN^ = (3.12) against the alternative Â«a= XNH >
PAGE 68
60 since R = [ A^^/k] [A2/(Np)] is positive definite when the true model is Eq. (3.2). In i^pendix 5(a) it is shown that under Case 1, the hypothesis given by (3.12) is equivalent to the hypothesis Hq: X^ = X2 = 0. Case 2 . In Case 2 we assume that R = [A /k] [a /(N p)] is not positive definite for the k optimal upper tailed test check points, but that R is negative definite for the k optimal lower tailed test check points. Here we recommend that the lower tailed test check points be used with a lower tailed rejection region. It is necessary for A2 to be positive definite for Case 2 to occur (see Appendix 4). However, k^ need not be positive definite, and so k need not be greater than P2. In Case 2 then, it is possible that lack of fit may be tested with only one check point. By inspection of equations (3.9) and (3.10), a hypothesis of no lack of fit is equivalent to X , X 2 while the alternative hypothesis that lack of fit is present is equivalent to X X a k N p
PAGE 69
61 since R = [ A^^/k] [a^/CN p)] is negative definite. In Appendix 5(b) it is shown that the hypothesis given by (3.13) is equivalent to the hypothesis Hq: A^ = X^ = 0. Case 3. We assume R is positive definite for the k optimal upper tailed test check points, and R is negative definite for the k optimal lower tailed test check points. Hence either an upper or lower tailed test may be considered as a possible test for lack of fit. If the quantity 2 22^" can be specified, then the minimum power for both the optimal upper and optimal lower tailed tests can be approximated, and the test with the greater minimum power is recommended. In Appendix 4 it is shown that Case 3 can occur only when Aj^ is positive definite for the upper tailed test. Thus Case 3 can only occur when there are at least pn check points. The minimum power of the upper tailed test may be found by calculating "^ (^J,Np,X^^,X,Â„ > ^Â«;k,Npl' '^Â•"' where F central F distribution. ak Np """^ ^^^ upper 100a percentage point of the
PAGE 70
62 ^IL = ^min^2&2/2^^' and ^2U = ^ax^2^2/2<^^' where \i . is the smallest eigenvalue of An and 5 is the mm ^ 1 max largest eigenvalue of A2. Formula (3.14) yields a conservative lower bound for the power of the optimal upper tailed test. Note that Aj^ is generated using the optimal upper tailed test check points. The cumulative distribution function of F" can be approximated by multiplying the cumulative probabilities of the central F distribution by a constant (Johnson and Kotz, 1970, p. 197). This approximation is described in Appendix 6. Other approximations for F" (such as the Edgeworth series approximation suggested by Mudholkar, Chaubey, and Lin, 1976) exist which are generally more accurate, but we chose to use the approximation given in Johnson and Kotz (1970, p. 197) due to its simplicity. Additionally, the approximation of Mudholkar, Chaubey, and Lin (1976) produced negative probabilties when only one degree of freedom was available in either the numerator or denominator of F" . This problem was avoided by using the approximation given by Johnson and Kotz (1970). The minimum power of the optimal lower tailed test can 2 be approximated similarly (if SAio/'' ^^ specified) by
PAGE 71
63 calculating P I F" < F 1 ^ k,Np;X^y,X2L (lct);k,Np^ where ^lU = ^max^2^2/2o^ and ^2L = Vin^2&2/2a^ with uÂ„aÂ„ equal to the largest eigenvalue of Ai and 6 . max 3 :3 j_ j^j^j^ equal to the smallest eigenvalue of A2. Note that Aj^ is generated by using the optimal lower tailed test check points. For the lower tailed test, Aj^ may be positive semidefinite, and if 3 is in the column space of P2 then X, = 0. In Case 3, the upper tailed test is a test of Hq: Xi = A^ = X X while the lower tailed test is a test of
PAGE 72
64 %'' ^=^2 = ' X X Case 4 . In Case 4 we assume that R = [ A /k] [k^/(U p)] is not positive definite for the k optimal upper tailed test check points and R is not negative definite for the k optimal lower tailed test check points. Here it is useful to write the difference between the expected value of the numerator and the expected value of the denominator of the F ratio in (3.8) as s^[Aj^/k A^/CN p)]32 = e^sns'e^ = 3'S^.^S'3 2 + 3^33.33.32 where ^ = diag(n,, ^2' ^3) is a diagonal matrix consisting of the eigenvalues of R, J^j^ is a diagonal matrix of the positive eigenvalues of R, ^2 is a diagonal matrix of the zero eigenvalues of R, and ^^3 is a diagonal matrix of the negative eigenvalues of R. The orthogonal matrix S can be expressed as S = [Sj^ :S2:S3] , where the matrices S^^, S2f and S3 have columns which are orthonormalized eigenvectors corresponding to Q^, Q.2, and .3, respectively.
PAGE 73
65 In Case 4, neither the optimal upper tailed test nor the optimal lower tailed test is applicable for all values of e . For completeness, we note that Case 4 actually consists of nine subcases, where R may be positive semidefinite, negative semidefinite, or indefinite for either the optimal upper tailed test or lower tailed test check points. These subcases are listed in Table 2. Table 2. Nine Subcases of Case 4.
PAGE 74
66 e^[A^/k A^/(N p)]g2 = e22i"iSiB2 + Â§.2^3^3231= Â§.2Sii2j^Se2 + Â§.2^i"iSie2f and 3 2[ A^/k A2/(N P)]e2 ^^^^ '^^ greater than zero, indicating an upper tailed test. Similarly, a sufficient condition for the test for lack of fit to be lower tailed is that 3 be in the column space of [S :S ], but not entirely in the column space of S2. Then 3^[A^/k A^/CN p)]32 = + 3^32^33^3.2 3^33^33^3.2 which makes g.^[ A^^/k A2/(N p)]g.2 less than zero, indicating a lower tailed test. To determine whether 3, is in the column space of [3 :3 ], let us define the augmented matrix ^1 ~ [Â§.9*^i*3t] Â• If ^1^1 ^^^ ^ zero eigenvalue, then 3Â„ is in the column space of [3 :S ]. Similarly, if we define ^2 ~ [^2*^21 ^"*^ ^1, ~ [Â§.9 '^9 Â•^^] ' then 3 is in the column space of Sp if Q'QÂ„ has a zero eigenvalue, and 3^ is in the column space of [ 3 :3 ] if QAQ, has a zero eigenvalue.
PAGE 75
67 Given that we are in a particular subcase of the nine subcases described in Table 2, we recommend that lack of fit be tested with the upper tailed test check points if it is determined that is such that e^[A^/k A^/CN P)]e2 ^^ positive when A, is generated from the upper tailed test check points. Likewise, for the same given subcase, if the value of 6__ of interest is determined to produce a negative value for e.A[A,/k A^/(N P)]32 ^'hen A^ is generated from the lower tailed test check points, then we recommend that lack of fit be tested with the lower tailed test. We see then that Case 4 is an undesirable situation in practice, since, in order to test for lack of fit, we must assume a priori that any lack of fit is due to a nonzero value of B Â„ that produces an upper tailed or lower tailed rejection region. However, it would seem rare that such knowledge would be available. 3.4 Examples We now present several examples to illustrate the technique for locating optimal check points to be used in testing for lack of fit in a mixture model. 3.4.1 Theoretical Examples Example 1. In this example a second order canonical polynomial model is fitted in three mixture components using the {3,2} simplex lattice design, which is presented in Figure 1 of Chapter 1. The true model is assumed to be the special cubic model containing the term 3 x x x in addition to the six terms of the fitted model. The expected
PAGE 76
68 values of the response at the six design points are assumed to be represented by the fitted model in the form E(Y) = X6^, but with the true model the expectations are written as E(Y) = X3j^ + y^^t^, where X is a 6x6 matrix with rows that define the coordinates of the six design points and columns that correspond to the six terms in the fitted model (xj^, X2f X3, x,x, ^I'^T' xÂ„x^), B, is the 6x1 vector of regression coefficients (g^, ^, g^, g^^' ^ j^^ ' ^23^' ^2 ^^ ^ ^"^^ column vector containing the values of the term '^\'^2^1 ^^ the design points, and 3 is the single regression coefficient 3i^o' The {3,2} simplex lattice design consists of only six design points, and since six parameters are estimated in the second order fitted model, there are no degrees of freedom remaining for obtaining an estimate of the experimental error, a^. We assume therefore that an external estimate of o'' is available, a ^t which will be used in the denominator ext of the lack of fit F statistic given in Eq. (3.3). Since there is one term in the true model in addition to those in the fitted model, that is P2 = 1, we know that in order to locate k simultaneous check points that maximize
PAGE 77
69 the power of the test for lack of fit it is necessary to maximize the scalar quantity A^ = (X* X*A) Â•Vq'(X* X*A) with respect to the coordinates of the k check points. Here X* is a kelement column vector with ith element equal to the value of x* x* x* at the ith check point, X* is a kx 6 matrix with ith row equal to the value of (x* , x* , x* , il i2 i3 X* x* , x* x* , x* X* ) at the ith check point, A = (X'X)"lx'X2 is the 6x1 alias vector, and V = I + X*(X'X)~ X*'. This maximization is accomplished by use of the Controlled Random Search Procedure (Price, 1977), which is described in Appendix 2. When only a single (k = 1) optimal check point is desired the Controlled Random Search Procedure locates a point (x*, X*) which maximizes A^ = (X* X*A)'Vq'(X* X*A), where X* = xjx*x* = x*x*(l X* X*),
PAGE 78
70 X* = (Xj, X*, X*, x*x*, x*x*, x*x*) = (X*, X*, (1 xj X*), xjx*, xj(l xj X*), X*(l Xj X*)), and Vq = 1 + X*(X'X)~ X*'. The value of A^^ is calculated using the formula of Eq. (3.6). Following this procedure, we find that the single check point that maximizes Aj^, and thus maximizes the power of the test, is the centroid of the triangular factor space (1/3, 1/3, 1/3). The value of A^ at this centroid point is Aj^ = 0.00084. When the Controlled Random Search Procedure is used to locate k = 2 simultaneous check points that maximize Aj^, the centroid (1/3, 1/3, 1/3) is selected twice, and Aj^ = 0.00121. For three simultaneous optimal check points, the centroid is selected three times, and A, = 0.00142. To test whether the second order model exhibits lack of fit, when we suspect the special cubic model is the true model, we form the F ratio d'V^'d/k F = Â— " '2 ''ext with the single check point (1/3, 1/3, 1/3) where d = Y* Y*, Y* is the observed response, Y* is the response
PAGE 79
71 predicted by the second order fitted model at (1/3, 1/3, 1/3), and Vq = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(X'X)"1 (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of the F ratio exceeds F , where v equals the number of a ; 1 , V * 9 degrees of freedom associated with a then we reiect the ext " null hypothesis that the second order model is the true model in favor of the alternative hypothesis that the special cubic model is the true model. Equivalently, we reject H^: X j^ = in favor of H : A, > 0. For k = 2 or k = 3 check points, the value of the F ratio is calculated using the observed and predicted responses at the two or three replicates at the centroid. The hypothesis H : X, = is rejected in favor of H : X, > if F U 1 a 1 exceeds F , a ;k,v Example 2. In Example 2 we illustrate the second of the four cases that could arise when MSE is used as an estimate of a^ in the lack of fit test statistic (see Section 3.3.4). We again fit a second order canonical polynomial model in three mixture components, and assume the true model is special cubic. The design to be used is the q = 3 simplex centroid design, which consists of seven design points, and is illustrated in Figure 2 of Chapter 1. There are six parameters to be estimated and seven design points hence one degree of freedom can be used to calculate MSE. We shall use MSE to estimate a 2. Optimal upper and lower tailed test check points must fc>e located, and then a decision is made as to which test should
PAGE 80
72 be used. The actual testing for lack of fit involves the F statistic in (3.8). As in Example 1, P2 = 1, since there is one term in the true model in addition to those in the fitted model. Thus Aj^ is a scalar whose value we seek to optimize with respect to the desired number of check points, k. When only a single check point is sought for the purpose of testing lack of fit, the Controlled Random Search Procedure has two functions. First, the procedure is used to locate the optimal candidate check point for an upper tailed test by locating the check point that maximizes the scalar A.. Secondly, the procedure is used to locate the optimal candidate check point for a lower tailed test, which is accomplished by locating the point that minimizes A,. The quantity R = [A,/k] [A2/(N p)] is then calculated to determine whether the upper or lower tailed test will be used. If R is positive for the candidate check point for an upper tailed test, then the test is upper tailed, and the test is lower tailed if the candidate check point for a lower tailed test produces a negative value for R. Note that A2 = (X2 XA)'(X2 XA) is fixed once the design is specified, since A2 does not depend on the check points. Using the Controlled Random Search Procedure it is found that the maximum value of A^^ occurs at (xj, x^, x^) = (1/3, 1/3, 1/3), which will be the location for the check point for the upper tailed test. Calculating Aj^ at this centroid
PAGE 81
73 point, we find that R = [A^/k] [A2/(N p)] = [(3.7258 X lO""*)/!] [(8.4175 X 10"'^)/1] = 4.6917 x 10"^. Since R is negative, the test is not upper tailed. Using the Controlled Random Search Procedure to minimize Aj^, we find that a subregion of the factor space exists in which all points yield a near minimum value for A]^. We choose the point (0.0189, 0.9269, 0.0542) at random from this subregion to be used as the optimal candidate for a lower tailed test. Here R = [(8.4175 x 10~'^)/1] = 8.4175 X lO"'^. Since R is negative for both the optimal upper tailed test check point and for the optimal lower tailed test check point, we have Case 2 of Section 3.3.4. The upper tailed test check point is disregarded, and the lower tailed test check point (0.0189, 0.9269, 0.0542) is used to test for lack of fit. If the calculated F ratio. MSE is less than F,, . ^ , then HÂ„ : X , = X ^ = is rejected in ( 1 a ) ; 1 , 1 1 2 favor of H : [x /l] [x /l] < 0, that is we conclude that a J. 2, the second order model exhibits lack of fit, and the true model is special cubic. When two simultaneous check points are desired for testing lack of fit, we can again use the Controlled Random Search Procedure to locate the optimal settings. To maximize the scalar An, we find that both check points
PAGE 82
74 should be selected at (1/3, 1/3, 1/3), for an upper tailed test. With our calculations R = [(5.8275 x 10"'*)/2] [(8.4175 X 104)/l] = 5.5038 x lO'^, but since R is negative, the test is not upper tailed. Minimizing A^ to locate two optimal lower tailed test check points yields a subregion in the factor space of optimal check points. The pair of check points (0.3749, 0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at random from this subregion, and these check points yield R = [(8.4175 X 10'^)/l] = 8.4175 x lO"'*. Since R is negative for the upper tailed test points and the lower tailed test points, we have Case 2 of Section 3.3.4 again and the lower tailed test check points are used to test for lack of fit. The hypothesis H : X = X = is rejected in favor of H : [x /2] [x /l] < if the cala X z culated value of F = (d'V~ d/2)/MSE is less than F.^ . Â„ , , in which case we say lack of fit of the model is ( lo ) ; 2 , 1 present. * 2 If an external estimate a ^ had been available for ext this example, then the optimal upper tailed test check points could have been used in the F ratio, F = (d'V~ d/k)/a^ ^, and lack of fit would then be detected ^" ext if the calculated value of F exceeded F , a ;k, V Example 3. Example 3 illustrates the procedure for locating optimal check points when there are two terms in the true model in addition to those in the fitted model. A second order canonical polynomial model in three mixture
PAGE 83
75 components is fitted using a q = 3 simplex centroid design. The true model is assumed to contain eight terms, six of which are the same terms as in the fitted model, with the additional two terms being the third order terms *^12^1^2^^1 ~ ^2^ ^^^ ^123^1^2^3* ^ ^^ Example 2, there is one degree of freedom for MSE which is used to estimate a2. The test statistic, F = (d ' v''d/k )/MSE, is given in equation (3.8). Since p2 = 2 and Aj^ is a 2^2 matrix, locating the optimal upper tailed test check points by the procedure of maximizing X ^^ is assisted by the maximizing of a lower bound for A,, namely maximizing u . Bl3_/2o^, where y . is the minÂ—zÂ—z min 2 smallest eigenvalue of A^. Since gand a are unknown, this is equivalent to maximizing u . . For u . to exceed mm min zero, it is necessary that A^ be of full rank, and since rankCAj^) < min(k, P2), it is necessary to select k > 2 check points. If A^ is less than full rank, and thus is positive semidefinite, only a subset of possible values of 3 Â„ could be considered to make it possible to test for lack of fit with an upper tailed test. Using the Controlled Random Search Procedure, the points that maximize w . are found to be (0.418, 0.277, 0.305) and (0.277, 0.418, 0.305). These points are thus optimal candidates for upper tailed test check points. At these check points we have v . = 5.1623 x 10"^, A, = mm 1 diag[5.1623 x io4, 5. 1916 x lO'^] , A2 = diag[0, 8.4175 x 104], 3f^^ R ^ [A^/2] [A2/I] = diag[2.5811 x lo'^, 5.8217
PAGE 84
76 X 10"^]. Since the eigenvalues of R are 5.8217 ^ 10"^ and 2.5811 X 10"'^, R is indefinite. Following the suggested procedure for Case 4 of Section 3.3.4, we note that an upper tailed test for lack of fit exists if the value of 2 ~ ^"^12' ^123^' ^^ """ ^^^ column space of [S2:S2] but not entirely in the column space of S2, where S^ is the matrix whose columns are the orthonormalized eigenvectors of R corresponding to the positive eigenvalues of R, and S2 is the matrix whose columns are the orthonormalized eigenvectors of R corresponding to the zero eigenvalues of R. Since R has no zero eigenvalues in this example, S2 does not exist, but S^ is the column vector, S^ = [1,0]'. Thus if 3 is of the form 3 = [^,2' Â°^ ' ' ^^^^^ "^12 * Â°' ^^^^ 3^ is in the column space of Sj^ and the test is upper tailed . The matrix A2 has rank one and therefore is positive semidefinite. Hence it is impossible to locate two check points that minimize w^^j^ and also make R = [h^/2] [A2/I] negative definite (see Appendix 4), that is, it is impossible to find a lower tailed test that is capable of testing lack of fit for all values of 3 . However, if we use the Controlled Random Search Procedure to locate two check points that minimize an upper bound for A which is y 8 13^/20 , then by minimizing m , we find that any of max22^ ^ max the check points in a particular subregion of the factor space yield a near minimum for u . One pair of points in ^ max this subregion is selected as the points to be used as
PAGE 85
77 optimal lower tailed test check points, namely the pair consisting of the point (0.053, 0, 0.947) replicated twice. Replicating this check point, we find u = 7.3900 max X 10"11, AL = diag[0, 7.3900 x 10"11], A2 = diag[0, 8.4175 x 10""^], and R = [A^/2] [A2/I] = diag[0, 8.4175 x lO"'^]. The eigenvalues of R are and 8.4175 x 10"'* implying that R is negative semidefinite. The values of g that are in the column space of [82:33] but not entirely in the column space of S2 will provide a lower tailed test. Here, [82:33] = diag[l,l] and S2 = [1,0]'. Thus, the test for lack of fit is lower tailed if Si^:* * 0. For values of 3Â„ that produce an upper tailed test we use the check points (0.418, 0.277, 0.305) and (0.277, 0.418, 0.305) with the F ratio d'v/d/2 F = MSE and conclude there is lack of fit if the calculated value of F exceeds F^,2 i* ^^1^ values of 3^2 that produce a lower tailed test, we use two replicates of the check point (0.053, 0, 0.947), and conclude there is lack of fit if F is less than F , where again F is calculated by F = (d'Vg'd/2)/MSE. Example 4. Example 4 illustrates Case 3 of Section 3.3.4 in which MSE is used to estimate a^ in the lack of fit test statistic. A second order canonical polynomial model
PAGE 86
78 in three mixture components is fitted using the {3,3} simplex lattice design, which appears in Figure 5. The true model is assumed to be special cubic, thus p2 = 1 and An is a scalar. The {3,3} design consists of ten design points and since there are six parameters to be estimated in the fitted model, o^ can be estimated by MSE with N p = 10 6 = 4 degrees of freedom. We first suppose that a single check point is to be used to test for lack of fit. Using the Controlled Random Search Procedure we find the single check point that maximizes the scalar A^ = (X* X*A) 'Vq'(X* X*A) is located at the centroid of the simplex factor space. Thus (X*, X*, x*) = (1/3, 1/3, 1/3) is the optimal candidate for an upper tailed test check point. At this centroid point, A^ = 4.9076 x 10"^. For the {3,3} design the scalar quantity A2 = (X2 XA)'(X2 XA) is fixed and is equal to A2 = 9.4062 X 10"'* and thus, R = [Aj^/k] [A2/(N p)] = [(4.9076 X 10"^)/1] [(9.4062 x 10"^)/4] = 2.5560 x 10""*. The point that is the optimal candidate for a lower tailed test check point is chosen randomly from a subregion of points in the factor space, in which all points minimize Aj^. The point selected has the value (x?, xi, xJ) = (0.560, 0.410, 0.030). Here A^ = 9.6590 x 10"'^ and R = [(9.6590 x 10"'7)/1] [(9.4062 X 104)/4] = 2.3419 x 10*4.
PAGE 87
(ffÂ°) (i'fo) (1,0,0) (0,1,0) {i'Â°'i) (0,0,1) Figure 5. The {3,3} simplex lattice design 79 Since R is positive for the optimal upper tailed test check point (1/3, 1/3, 1/3) and R is negative for the optimal lower tailed test check point (0.560, 0.410, 0.030) we are in Case 3 of Section 3.3.4. Either the upper or lower tailed test could be used to test for lack of fit, but 2 if the quantity S'B_/a can be specified, then we will choose to use the test that has greater minimum power, since greater power means that we are more likely to detect lack of fit when in fact lack of fit exists. In this example ^2= ^123For illustrative purposes, we arbitrarily choose 2 3 '3 /a = 2000, so that an approximate conservative lower bound for the power of the upper tailed test is found by
PAGE 88
80 calculating P I F" > F 1 ^ k,Np;A j^^,X2u a;k,NpJ where F ,, ,, _ is the upper 100a percentage point of the a ; K , NÂ— p central F distribution, k is the number of check points, N is the total number of response observations, p is the number of parameters in the fitted model, ^IL = ^min^2^2/2^^' ^"d A^u = 'S^ax^2^2/2<'^The quantity y ^^ is the smallest eigenvalue of Aj^, where Ai is evaluated at the optimal upper tailed test check point. Since A, is a scalar, y . = A^ . Likewise, 6 is the L mm 1 max largest eigenvalue of A2, and since in this example A2 is a scalar, 6 = A.. In this example we have k = 1, N p = max 2 10 6 = 4, \ = u^. SiB/2a^ = (4.9076 x lO""^ )( 2000/2 ) = 4.9076 X 101, and X ^^ = &^^^&.^S,2/2o^ = (9.4062 X 10^)(2000/2) = 9.4062 x lo'"*". Using the approximation to the cumulative probabilities of the doubly noncentral F distribution given by Johnson and Kotz (1970, p. 197) which is described in Appendix 6, and taking a = .05, we find that a conservative lower bound for the power of the optimal upper tailed test is approximately equal to .0649. The minimum power for the optimal lower tailed test is 2 approximated (assuming 3A3,/a = 2000) by calculating ^ l^k,Np;Aj^y,X2L '^ ^( 1a ) ;k,Np^ *
PAGE 89
81 The quantities \ ^^ and x ^^ are taken as x = y B'g /2a^ 2 and X= 6^. el3o/2o where u ^ is the largest eigenvalue zL min22 max ^ ^ of A]^ with Aj^ calculated using the optimal lower tailed test check point, and where 6 . is the smallest eigenvalue of mm ^ A^. Since Ai and An are scalars, u = A, and 6 . = A . ^ J. z Â•^max 1 min 2 In this example, k = 1, N p = 4, X^^ = (9.6590 X 10"'^)(2000/2) = 9.6590 x lo""*, and X2L = (9.4062 x lo""* )( 2000/2 ) = 9.4062 x lo""". Again if the approximation to the doubly noncentral F distribution given in Johnson and Kotz is used, an approximate conservative lower bound for the power of the optimal lower tailed test is .0555. Having specified l^^^^a'^ = 2000, the optimal upper tailed test is chosen over the optimal lower tailed test, because the approximate minimum power of the upper tailed test is greater than the approximate minimum power of the lower tailed test. Using the optimal upper tailed test check point (1/3, 1/3, 1/3) in the test statistic MSE we conclude that lack of fit is significant if the calculated value of F exceeds F , , , in which case we a ; 1 , 4 reject HQ:XL=X2 = 0in favor of H^ : X j^/1 X 2/4 > . When two simultaneous check points are used for testing lack of fit, the Controlled Random Search Procedure locates the optimal upper tailed test and optimal lower tailed test
PAGE 90
82 check points. It turns out that two replicates at (1/3, 1/3, 1/3) maximize A^, and are used as optimal check points for an upper tailed test. The value of R = [A^/2] [A2/4] is [(7.9210 X 10"'^)/2] [(9.4062 x 10"'^)/4] = 1.6090 x 104. In searching for two optimal lower tailed test check points, again a subregion of the factor space is found in which any of the points nearly minimize A,. From this subregion are chosen the points (0.6386, 0.3263, 0.0351) and (0.7257, 0.2421, 0.0322) resulting in a value of R = [A.j^/2] [A2/4] of [(1.5216 x 109)/2] [(9.4062 x 104)/4] = 2.3516 X 104. In conclusion, when two simultaneous check points are used in the test for lack of fit in this example, R is positive for the optimal upper tailed test and R is negative for the optimal lower tailed test, and we have Case 3 of 2 Section 3.3.4. Selecting 3A3^/a = 2000 arbitrarily, we found the approximate lower bound for the power of the upper tailed test to be .0504, and the approximate lower bound for the power of the lower tailed test to be .0612. Since the power is higher with the lower tailed test it is our choice for testing lack of fit when two check points are used simultaneously. Lack of fit is detected and we reject H : X = X = in favor of H : [X /2] [X /4] < if the F U X ^ Si 1. ^ ratio, F = (d'V~ d/2)/MSE, using the optimal lower tailed test check points (0.6386, 0.3263, 0.0351) and (0.7257, 0.2421, 0.0322) is calculated to be less than F . ( Â±~ct J ; z , '1
PAGE 91
83 3.4.2 Numerical Examples Numerical Example 1. In this example we illustrate numerically some of the findings in the first theoretical example of Section 3.4.1. Data that were collected in a rocket fuel experiment (Kurotori, 1966) will be used to investigate the power of the lack of fit F test. The test is set up to detect the inadequacy of a fitted second order canonical polynomial model when the true model is special cubic. Calculated values of the power of the test which detects lack of fit through large values of d'v""'d/k F = Â— " ^2 Â°ext will be compared for several check point locations, including the location (1/3, 1/3, 1/3) at which the power was found to be maximum in Example 1 of Section 3.4.1. In Kurotori 's experiment the modulus of elasticity (Y) of a rocket fuel is expressed as a function of the proportions of three components Â— binder (x^) , oxidizer (X2)/ and fuel (X3). Since lower bounds are placed on the component proportions x^, x^, and X3, in the form of 0.20 < x^, 0.40 < x^/ and 0.20 < x , pseudocomponents (x!) are defined in terms of the original components in the form of xj = (x^ 0.20)/(1 .80), x^ = (x^ 0.40)/(l .80), and x^ = (x^ 0.20)/(l .80). The true special cubic model in the pseudocomponents, which is obtained by using the data at the seven points of the simplex centroid design
PAGE 92
84 in the pseudocomponent system, is E(Y) = 2350X' + 2450X' + 2650x' + Ox'xl i. ^ J X ^ + lOOOx^x^ + leOOx^x' + 6150x'x'x' The second order canonical polynomial model that is fitted to the six boundary points only, and which will be tested for lack of fit, is given by Y = 2350x + 2450X' + 2650x' + lOOOx^x^ + leOOx^x'. The configuration of the experimental design as well as the check point locations are depicted in Figure 4 of Chapter 2 and the observed response values are given in Table 3 of this chapter. 1 "2 A value of the ratio F = [d'V_ d]/a is calculated at ^ ext each of the four individual check points (1/3, 1/3, 1/3), (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3) * 2 "2 where a . is assumed to have the value a ^ = 144 as ext ext suggested by Kurotori (1966). We also assume without loss of generality that the degrees of freedom associated " 2 with o ^ are v = lo. The power of the F test is calculated ext at each of the four check points by using the approximation to the cumulative probabilities of the noncentral F
PAGE 93
85 Table 3. Observed Response Values at the Pseudocomponent Settings for Kurotori's Rocket Fuel Experiment Â— Numerical Example 1. Observation Binder Oxidizer Fuel Modulus of Elasticity Number
PAGE 94
86 only on the boundaries of the triangle (and therefore at each point at least one of the x! values is equal to zero), then X2 = and A = 0. From the true special cubic model, ^123 = ^^5Â°The calculated value of F as well as the approximate value for the power at each of the four check points is given in Table 4. The check point (1/3, 1/3, 1/3) produced the highest power of the four check points investigated, supporting the previous results of Example 1 in Section 3.4.1 where (1/3, 1/3, 1/3) was selected as the check point location with the maximum power when a second order canonical polynomial was fitted using the {3,2} simplex lattice design, but the true model was assumed to be special cubic. Additional support for the point (1/3, 1/3, 1/3) being optimal is given by the contour plot of values of A, in Figure 6(d). The highest values of A^ appear near the centroid (1/3, 1/3, 1/3) where high A^ values translate into 2 2 high \ ^ values, since ^i = ^1^123/^^ ' which in turn implies high power since we know the power is an increasing function of X j^. As a second part of this example the power of the F test that is obtained when three replicates are taken at (1/3, 1/3, 1/3) is compared to the power of the F test that is obtained when one replicate is taken at the three check points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3). These latter three point locations were suggested by Kurotori for testing lack of fit of his fitted special
PAGE 95
87 u
PAGE 96
88 cubic model. The result of this comparison, see Table 4, is that the three replicates at (1/3, 1/3, 1/3) produce the test with greater power which again supports the findings of Example 1 of Section 3.4.1. All of the check point locations listed in Table 4 produce very high power values (> .999) which is due in part to the high value of & ^^^ ^^123 " 6150). If 3 were of lower magnitude, then the three replicates at (1/3, 1/3, 1/3) would show a still greater superiority in the power value compared to the power using the other check points. This superiority is demonstrated in Table 5 where values of 3^23 3^^ listed as 3000 and 1500 and the comparative power values are listed as 0.998 compared to 0.795 and 0.662 compared to 0.249, respectively. Table 5 also demonstrates the superior power value for the point (1/3, 1/3, 1/3) when ^123 ~ ^^^^ Â°^ ^123 ^ 1500 and each of the four check points is used one at a time. Finally, (1/3, 1/3, 1/3) being the optimal check point location is seen in Figure 6(c), where contour plots of the expected difference in the heights of the surfaces are drawn. The differences in the heights are found by subtracting the estimated height of the surface obtained with the fitted second order model from the estimated height of the surface obtained with the true special cubic model. The expected difference between the true and fitted surfaces approaches a maximum the closer one moves to the centroid of the simplex factor space, so that the optimal check point
PAGE 97
89 u
PAGE 98
90 ^2^' V (a) True special cubic surface. (b) Fitted second order surface. X. =1 (c) Expected difference between the true special cubic surface and the fitted second order surface. (d) A,=(X* X*A)' Vq'(X*X*A Figure 6. Contour plots for Numerical Example 1.
PAGE 99
91 location (1/3, 1/3, 1/3) coincides with the point where the expected difference between the true special cubic surface and the fitted second order surface is maximum. Numerical Example 2. In this second numerical example, we investigate the power of the F test for detecting lack of fit when a second order canonical polynomial model is fitted in a mixture system which is in truth represented by a special cubic model. The true model is assumed to be E(Y) = 2350X + 2450X + 2650x ^ ^ J + lOOOXj^x^ + leOOx^x^ + 6150x X x which is used to generate hypothetical response observations at the seven points of the q = 3 simplex centroid design as well as at three check points. The values of the response are obtained by adding the value of a pseudorandom normal variate with mean and variance 144 to each true predicted response value. The data are given in Table 6. The response values at the seven points of the simplex centroid design are used in the least squares normal equations to obtain the fitted second order model Y = 2341x^ + 2438X2 + 2630x2 + 310x^X2 + 1304x^^X2 + 1970x x
PAGE 100
92 Table 6. Generated Response Values Â— Numerical Example 2, Xi X2 X3 Y 10 2357 10 2454 1 2646 1/2 1/2 2403 1/2 1/2 2747 1/2 1/2 2962 1/3 1/3 1/3 3013 * 1/3 1/3 1/3 2993 2/3 1/6 1/6 2693 .02 .93 .05 2550 * Check points which is to be tested for lack of fit using the test statistic F = d'v" d/MSE. The F statistic will be evaluated at each of the three check points (1/3, 1/3, 1/3), (2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time, and the power of the test at the three check point locations will be calculated and compared. The test is lower tailed for all check point locations (since R = A, A2 is negative for all check point locations) and thus the power is defined as pi F" < F 1 ^ l,l;Xi ,X2 .95;1,1^
PAGE 101
93 X I (.02 ,.93, .05 ) X =1 2 x =1 3 Figure 7. Contours of R = Aj^ A2 for Numerical Example 2, 2 2 2 2 The values of X , = 3 123^1'^^'' ^^^ ^2 ~ ^123^2'^^'' ^^^ found 2 by taking 6 , j^ = 6150 and a = 144. The results of this power investigation are given in Table 7. Since the check point (.02, .93, .05) produces the greatest power of the three check points investigated, this supports the result in Example 2 of Section 3.4.1, where we saw that the point (.02, .93, .05) yielded the maximum power of all points for detecting lack of fit of a fitted second order canonical polynomial model, using the q = 3 simplex centroid design in the presence of a true special cubic surface. Additional evidence for the point (.02, .93, .05) being an optimal check point is shown in Figure 7, where contours of the values of R = Aj^ A2 are presented . The point (.02, .93, .05) is seen to belong to an area of the
PAGE 102
94 u 1^
PAGE 103
95 simplex factor space where R is minimum, which implies that At (and in turn X.) is also minimum in this area, since R = ^1 " ^^2 ^^'^ ^2 ^^^ ^^^ fixed value of A2 = .00084 for the simplex centroid design. Thus the check point (.02, .93, .05) produces a minimal X , value and maximum power, since the power increases with decreasing values of X^3.5 Discussion When check points are used for testing lack of fit in a mixture model, the appropriate testing procedure, assuming a normally distributed response, involves an F statistic. If " 2 an external estimate, a *' Â°^ ^^^ experimental error variance is available so that the test statistic is given by dv^Vk F = Â— '2 ^ext then the power of the test for lack of fit is maximized by choosing k check points that maximize the value of the noncentrality parameter X^, When P2 = If maximizing X ^ is achieved without knowing the value of the elements of g by selecting check points that maximize the scalar A^ . When P2 > If the maximization of X j_ is approximated by maximizing a lower bound for Xj_. This is achieved also without knowing the values of the elements of 6 by selecting check points that maximize the smallest eigenvalue of the matrix Aj^ . The test is upper tailed, and for given values of Xi, the actual
PAGE 104
96 power of the test can be calculated from the cumulative probabilities of the noncentral F distribution. A problem arises when A^ is positive semidefinite and its smallest eigenvalue is equal to zero. In this case check points that maximize the smallest positive eigenvalue of Ai are selected, and lack of fit is only detectable for a subset of the possible values of the elements of g . When an external estimate of a^ is not available, testing lack of fit at the check points is further complicated. The F statistic is F = MSE and the rejection region for the lack of fit test can be upper tailed or lower tailed. The power of the test is determined by using the doubly noncentral F distribution, which depends on the parameters k, N p, X^, and A 2Of these four parameters, only k and X^ are influenced by check points, and if the value of k is fixed, the power of the test is maximized by choosing check points that affect the value of Xj^* Regardless of the values of the elements of , check points that maximize X, are selected for maximizing the power of an upper tailed test, and check points that minimize X Â•, are selected for maximizing the power of a lower tailed test.
PAGE 105
97 Lack of fit can be tested with the upper tailed test for all nonzero values of the elements of g if the check points are selected so that [A^/k] [A2/(H p)] is positive definite, since this forces the expected value of the numerator mean square in the F ratio to be greater than the expected value of the denominator mean square. Similarly, lack of fit can be tested with a lower tailed test if check points are selected which make [A, /k] [A2/(N p)] negative definite. When it is not possible to select check points that make [A^/k] [A^/CN p)] either positive definite or negative definite then detection of lack of fit is only possible for a subset of all nonzero values of the elements of 3 . The power of the test for lack of fit using the F statistic in (3.8) is a function of X^ and X2* Since the magnitudes of Xj^ and X2 are influenced by the experimental design, an area for future study is the investigation of the effect of the experimental design on the power of the lack of fit test. In the presence of an external estimate of 2 a , Atkinson (1972) suggested designs that maximize the determinant of A2, A2i, when lack of fit is to be detected by a large value of X using a procedure which is, in general, equivalent to the lack of fit testing procedure that partitions the residual sum of squares into pure error and lack of fit sums of squares. It might be useful to apply Atkinson's (1972) methodology not only to A , but to \h^\ or \h^/k A2/(N p) in order to find an
PAGE 106
98 appropriate design when testing lack of fit with the F ratio in Eq. (3.8). Since the power of the test in Eq. (3.8) is also affected by the values of k and N p, which are the numerator and denominator degrees of freedom of the doubly noncentral F distribution, respectively, optimal settings for these parameters can also be considered. For a given fitted model, p is fixed so that the degrees of freedom would be influenced by the number of check points selected, k, and by N, the total number of observations. Finally, the experimental design and the number of check points also " 2 affect the power of the F test when o ^ is used to ext 2 estimate a . Thus the effect of the experimental design and the number of check points can also be investigated for the situation where the lack of fit test statistic is given by Eq . ( 3 . 3 ) . We now conclude our investigation of the check point approach to lack of fit testing and in the next chapter turn to an investigation of a near neighbor method for testing lack of fit.
PAGE 107
CHAPTER FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR TESTING LACK OF FIT 4.1 Introduction In an experiment in which replicate response observations are available at one or more design points, lack of fit of a fitted model can be tested by a procedure which involves partitioning the residual sum of squares into two statistically independent portions. One portion is the sum of squares due to lack of fit (SSj^Qp), and the second portion is the sum of squares due to pure error (SSEp^_,j.g) obtained from the replicates. As discussed in Section 2.2, this procedure was suggested by Draper and Smith (1981, p. 120). Lack of fit is inferred if the calculated value of the ratio ""^LOF ^=^iSE (4.1) pure exceeds the corresponding upper 100a percentage point of the central F distribution, where MSlof ^"d MSEp^jg are the mean square values found by dividing SSlof ^""^ ^^^pure ^^ their respective degrees of freedom. In order to test the fitted model for lack of fit when replicate observations are not available, Shillington (1979) suggested a procedure which uses observed response values 99
PAGE 108
100 collected at points which are "near neighbors" in the factor space in place of replicates (see Section 2.3). Lack of fit is inferred when the calculated value of the ratio MSEg ^ ^ MSEÂ„ (4.2) W exceeds the upper 100a percentage point of the central F distribution. The numerator, MSEg, of the F ratio in Eq. (4.2) is a generalization of the numerator, MSlqf' Â°^ *^^ ^ ratio in Eq. (4.1). The form of MSEg will be given in Eq. (4.5) of Section 4.3. The denominator, MSE^^, in Eq. (4.2) is a generalization of MSEp^j.^ in Eq. (4.1), and the value of MSEy^ is calculated using near neighbor observations in place of replicates (see Eq. (4.6) in Section 4.3). Shillington 's near neighbor method provides an alternative to the check points method when replicate observations are not available. Typically, near neighbors might appear either because an experiment was not designed to provide replicate observations or with a designed experiment consisting of a large number of design points in a bounded factor space which results in some points being near one another. In this chapter we shall further study Shillington Â• s (1979) near neighbor procedure for testing lack of fit. A question involving the correctness of the ordinary least squares technique suggested by Shillington for deriving the denominator, MSE^, of the F ratio in Eq. (4.2) will
PAGE 109
101 be raised. The question will be resolved by showing the equivalence of deriving MSE^ by ordinary least squares and of deriving MSE^^ by a generalization of weighted least squares. We will verify that when the observable response values are assumed to have the normal distribution with 2 homogeneous variance, a , the F ratio in Eq. (4.2) possesses a central F distribution when the fitted model is adequate, but possesses a doubly noncentral F distribution when the fitted model suffers from lack of fit. We shall also show that the F test for lack of fit which uses the statistic in Eq. (4.2) can have either an upper tailed or a lower tailed rejection region. Finally, the use of a clustering algorithm for defining groups of near neighbors will be proposed. 4.2 Notation In this section we introduce the notation to be used in this chapter. Throughout our investigation of Shillington 's near neighbor procedure for testing lack of fit, we shall assume the observed response values collected in an experiment can be grouped into g cells where the jth cell contains nj observations, j = 1, 2, ..., g. The observations in a cell are from points that are "near neighbors" in the sense that they are near one another in the factor (mixture) space. A model of the form E(Y) = XBj_ (4.3)
PAGE 110
102 is fitted to the data using ordinary least squares, but the true model is assumed to have the form E(Y) = X3j^ + X^^^, (4.4) where Y is an Nxl vector of response values observable at 2 the design points with var(Y) = a !Â„, X and X2 are Nxp and Nxp2 matrices of known constants, respectively, and g, and g^are pxl and P2^1 vectors of unknown regression coefficients, respectively. The vector Y is assumed to have the normal distribution. Let us now define the following vector and matrix quantities to be used in developing the numerator, MSEg, of the F ratio in Eq. (4.2): Y = a gx 1 vector with jth element equal to the average of the n^ observed response values in the jth cell of near neighbor observations, j = 1, 2, . . . , g . Xq = a gxp matrix whose jth row is the average of the nj rows of X corresponding to the jth cell, j = 1, z f ..., g. X2C = a g>
PAGE 111
103 Gr. = a gxg diagonal matrix of the form G = diag[l/nj^, l/n2/ Â•Â•Â•/ l/"q]' To further illustrate the forms of Y^, X^, X2C' ^rid Gg/ we present the following numerical example. Consider a data set consisting of response observations (Y) taken at N = 8 different combinations of the settings of the factors xi and X2, where the eight response observations are divided into g = 4 near neighbor cells. The vector of observed response values, Y, and the matrix X corresponding to the first order model, E(Y) = 3q + Pj^x^^ + 3 2^2' ^^^ Y = 10 13 16 15 18 21 27 30 X = 112 12 5 12 4 13 2 13 1 14 2 I 5 5 15 4 The horizontal lines in Y and X delineate the four cells of near neighbors. In this example ^C = 10 (13 + 16)/2 (15 + 18 + 21)/3 (27 + 30)/2 10 14.5 18 28.5
PAGE 112
104 ^C = 1 (1 + l)/2 (l+l+l)/3 (1 + l)/2 1 1 1 1 1 (2 + 2)/2 (3+3+4)/3 (5 + 5)/2 1 2 3.3 5 2 (5 + 4)/2 (2+l+2)/3 (5 + 4)/2 2 4.5 1.7 4.5 and Gq = diagd, 1/2, 1/3, 1/2). If the true model is second degree, E{Y) = 2 2 8q + Bj^Xj^ + 3 2^2 "*" ^12^1^2 '*' ^11^1 '*' ^22^2' ^^^" ^^^ ^2 ^"^ X2C matrices have three columns corresponding to the 2 2 terms x,Xp, x, , and x, respectively. For this numerical example we have ^2 = and '2C 2 10 8 6 3 8 25 20 2 9 5.7 22.5 1 4 4 9 9 16 25 25 1 4 11.3 25 4 25 16 4 1 4 25 16 4 20.5 3 20.5
PAGE 113
105 Next let us define the following quantities to be used in developing the denominator, MSE^, of the F ratio in Eq. (4.2): W = an Nx 1 vector of within cell deviations where the ith element, W^ , of W is equal to the difference between the ith element, Y^/ of Y and the average of the response values observed in the near neighbor cell containing Y^ , i = 1, 2, . . . , N. X^ = an Nxp matrix whose ith row is equal to the ith row of the X matrix minus the row of X^ corresponding to the cell containing the response value observed at the ith row of X. r = rank(X^^). ^2W ~ ^" Nxp2 matrix whose ith row is equal to the ith row of the X2 matrix minus the row of X2P corresponding to the cell containing the response value observed at the ith row of X. Zq = an NxN idempotent matrix of the form ^0 " ^N " di^gni/n^)J^, (l/n2)J2/ . . . , ( 1/n )J ] where Jj is an njxnj matrix of ones, and If^ is g the identity matrix of order NxN, with N = En. j=l ' Let us illustrate the forms of W, X^, X2^^, and J: q by using the numerical example presented earlier in this section, where the eight response observations were distributed among four cells. For these data we have
PAGE 114
106 W = ^w X 2W
PAGE 115
107 and vector quantities defined in Section 4.2. The test that Shillington proposed involves the use of an F ratio (see Eq. (4.2)) of two statistically independent mean square values, 2 each of which is an unbiased estimate of a when the fitted model is the correct true model. The two independent mean 2 squares become biased estimates of a when the fitted model suffers from lack of fit. Shillington 's methodology detects lack of fit when the calculated value of the F ratio in Eq. (4.2) is large, thus his test is upper tailed. We shall see later in Section 4.7 that the test is not always upper tailed, and may be lower tailed. Shillington points out that the power of the test depends on the relative magnitudes of E(MSE3) and E(MSE^^), that is, the power depends on the difference between the expected values of the numerator and of the denominator in the F ratio in Eq. (4.2). We shall be more specific than Shillington by discussing the power of the test in terms of parameters of the doubly noncentral F distribution. We now turn to defining Shillington 's test statistic in matrix notation. Shillington 's F ratio takes the form (see Eq. (4.2)) SSE /(g p) F = ^ SSE^/(N g r) MSE3 MSE^
PAGE 116
108 where SSEg is the residual sum of squares from a weighted least squares regression analysis in which Y is regressed on Xq, g is the number of cells of near neighbors, p is the number of terms in the fitted model, and r is the rank of Xy^. The quantity SSEg can be written as the quadratic form (see Graybill, 1976, p. 329; also see Draper and Smith, 1981, p. 109). The quantity SSE^^ is defined as SSE^ = W'[In XÂ„(X^'XÂ„) X^;,]W, (4.6) where (XJLXÂ„)~ is any generalized inverse of (XAX ). [A matrix A~ is defined as a generalized inverse of the matrix A if AA~A = A.] The quadratic form SSE^ is the residual sum of squares from an ordinary least squares regression analysis in which W is regressed on X^. In the next two sections we shall discuss the development of the numerator and denominator, MSE3 and MSE^, respectively, of the F ratio given in Eq. (4.2). We then suggest an alternative representation for MSE^ which relies on a generalization of weighted least squares. This alternative representation for MSE^ will be shown to be equivalent to Shillington 's expression for MSE^.
PAGE 117
109 4.3.1 Development of MSEg The quantity MSEg = SSEg/Cg P) is the numerator of the F ratio in Eq. (4.2). As mentioned in Section 4.3, the quantity SSEg is the residual sum of squares from a weighted least squares regression analysis in which Yp is regressed on Xp. The weighting is appropriate because var(Yp) = a GÂ„ not only when the fitted model is adequate (under model (4.3)), but also when the fitted model suffers from lack of fit (under model (4.4)). In order to further explain the (Yp, Xp ) system, we define the matrix M as M = diag[(l/n^)l, ..., (l/n^)!^] where 1 . is an nxl vector of ones, j =1, 2, ..., g. Then the (Yp^ X ) system can be derived as a linear transformation of the (Y, X) system. That is, application of the transformation matrix M yields the following equalities Xc = ^'X' and X^ = MX, ^2C ~ "^^2 From this it follows that var(Y ) = M var(Y)M' = a^MM' 2 a G_ , since MM' = G.. Under the hypothesized model of
PAGE 118
110 Eq. (4.3), E(Y^) = ME(Y) = MXB^ = X^gj^, whereas under the model of Eq. (4.4), E(Y^) = M(Xe^ +^2^2^ " ^C^l "^ ^2CÂ§2' We now consider the distribution of the random quantity 2 SSEg/a . It can be shown (Theorem 2, Searle, 1971, p. 57) that under the model of Eq. (4.3), SSE_,/a^ possesses a central chisquare distribution with g p degrees of freedom, but that under the model of Eq. (4.4), SSE /a possesses a noncentral chisquare distribution with g p degrees of freedom and noncentrality parameter n , , where "l = (l/2^)Â§.2^2cfS^ S\(^C^0^^c)"^^cS^J^2C^2^'''^ Here we point out that the noncentrality parameter for 2 SSEg/a given by Shillington (1979) is not correct and should be written as in Eq. (4.7). Finally we note that SSEg is equivalent to the usual lack of fit sum of squares, SSlqF' "here SSlqf/^^ ~ P) = MSlqp is the numerator of the F ratio in Eq. (4.1), when all observations in each cell are true replicates rather than near neighbors. Shillington pointed out this fact, but did not give a proof. We offer a proof in Appendix 7. 4.3.2 Development of MSE^^ The quantity MSE^ = SSE^^/(N g r), where r denotes the rank of X^^, is the denominator of the F ratio in Eq. (4.2). As mentioned in Section 4.3, the quantity SSE^ is the residual sum of squares from an ordinary least squares regression analysis in which W is regressed on X^^. Using
PAGE 119
Ill Theorem 2 (Searle, 1971, p. 57) and noting that W = E Y and E^Z^ = Z^, it can be shown that under the 02 hypothesized model of Eq. (4.3), SSE /a possesses a central chisquare distribution with N g r degrees of freedom, 2 but under the model of Eq. (4.4), SSE.ya possesses a w noncentral chisquare distribution with N g r degrees of freedom and noncentrality parameter H Â„ , where ^ = (l/2a2)3.X'^tI^ X^(X^'X^)X^]X2/_2. (4.8) Shillington (1979) points out that SSE^ reduces to the usual pure error sum of squares, SSEp^J.g, when all cells contain true replicates. This is easily seen by using the fact that X^ = when all cells are composed entirely of true replicates so that SSE,, = WW = Y'E.I.Y = Y'l.Y = SSE ^ W0 00pure We saw in Section 4.3.1 that the (Yp, XÂ„) system is derived as a linear transformation of the (Y, X) system. Similarly, the (W, X^) system can be derived by applying the transformation matrix Zq. Thus W = ^^.Y, X^^ = ^^.X, and X = ^p,X . It follows that E(W) = 2f.E(Y) = ^^XP = X^S , under the model of Eq. (4.3), and E(W) = I (X^ + X 3 ) = X^ + X g under the model of Eq. (4.4). Furthermore, var(VjJ) = Z var(Y)5:^ = (J^^q^q = <^^^o' ^^^^^ ^0 is symmetric and idempotent. Since the variancecovariance matrix of W is not equal to alj^, for some positive constant a, SSE^ should have been derived as the residual sum of squares from a weighted least
PAGE 120
112 squares regression analysis of W on X^ rather than from the ordinary least squares regression of W on X^ that Shillington (1979) suggested. We shall use the weighted least squares regression of W on X^ in an attempt to replace SSE^ in the F ratio in Eq. (4.2) by an expression we will call SSEy^(weighted) . We later show that SSE^^ and SSE^( weighted) are equivalent. 2 The variancecovariance matrix of W, which is a H ^, is of rank N g and is thus singular. Therefore the residual sum of squares from a weighted least squares regression analysis of W on X^ which is w'l^o^ z%(x;i%)x;,zi]w cannot be used as an expression for SSE^^( weighted ) , since Sq does not exist. The problem of performing a weighted least squares regression analysis when the variancecovariance matrix of W is singular is considered in the next section. 4.4 Development of SSEyj( weighted ) 2 If the variancecovariance matrix of W, a Eq, is nonsingular then the weighted least squares formula for SSEY^(weighted ) is SSE^(weighted) = (W X^p^^ ^ '^ o"^ ^ " Vl^
PAGE 121
113 where Q_^ is a solution to X^Eq~'X^3 = X^J^z"'W and can be written as B^^ = ( X^ ~ X^)~X^J^Z ^''W. The quantity SSE^( weighted) divided by the appropriate degrees of freedom 2 provides an unbiased estimate of a under the model of Eq. (4.3) . However, since 2: q is singular, the weighted regression formula above cannot be used to calculate SSE^^(weighted ) . C. R. Rao (1971, 1972, and 1973) suggests an analog of weighted least squares for the case of a singular variancecovariance matrix. Rao suggests the existence of a matrix H such that 6, is a stationary vector value of (W X^0j^)'H(W X^Bj^) in which case o^ may be estimated using a^ = (W X^3^)'H(W X^3i)/(N g r) where (N g r) = rank(EQ:X^) rank(X^). The rank of the matrix (EqTX^) is equal to N g because X^ = EqX^ so that the columns of X^ are spanned by the columns of z ^, thus, rankCZ^rX^) = rank(EQ) = N g. One form of the matrix H is defined (Rao, 1971 and 1972) as H = [Eq + c^X^X^] (4.9) where c is an arbitrary nonzero constant, so that with the model of Eq. (4.3), o^ = (W X^g_j^)'H(W X^^^)/(li g r)
PAGE 122
114 2 IS an unbiased estimator for a . Thus a stationary vector value of (W X^3,)'H(W ^^,) is given by gj^ = (X^HX^)~X^HW . Rao indicates (1972, p. 3) that a^ is invariant to the choices of the generalized inverses ^ 2 involved in a Â• Rao's proofs for obtaining an unbiased estimator *2 2 a for a are not given m detail. Therefore we shall state and prove the following theorem which will be used to develop an expression for SSE^( weighted ) . The notation A" will be used to denote any generalized inverse of a matrix A, such that AA~A = A. 2 Theorem 4.1 . Let Y ~ (Xg, a G), where G is singular, then a^ = f""'(Y xe)'T~(y Xg) 2 (i) is an unbiased estimate of a , and (ii) is unique with probability one, and (iii) is a scalar multiple of a central chisquare variable with f degrees of freedom of the form 2 2 (a /f )Xf if Y has the multivariate normal distribution. The vector Y is of order Nxl, 3 is a px 1 vector of unknown regression coefficients, X is an Nxp matrix of known constants, G is an NxN positive semidefinite matrix of known constants, T = G + XX' , 3 is any solution to X'T~xe = X'T~Y, that is, 3 = {X'T~X)~X'T~Y, and f = rank(G:X) rank(X).
PAGE 123
115 The proofs of parts (i), (ii) and (iii) of Theorem 4.1 are given in Appendices 9, 10, and 11, respectively. Lemma 4.1 which is used in the proof of Theorem 4.1 is stated and proved in Appendix 8. The results of Theorem 4.1 can now be applied to our problem of finding an expression for SSE^^( weighted ) . We define SSE^( weighted ) as SSE^(weighted) = W[T" TX^(X^TX^)X^;,T] W (4.10) where Tq = S q + X^X^. Writing SSE^(weighted ) in Eq. (4.10) as SSE^( weighted) = (W X^ej^)'T~(W X^3,^), from Theorem 4.1 we see that if the true model is of the form in Eq. 2 (4.3) then SSE^( weighted )/a has a central chisquare distribution with f = rank(z :X^) rank(X^) = N g r degrees of freedom. However, if the true model is of the form in Eq. (4.4), then SSE^( weighted )/a^ has a noncentral chisquare distribution with N g r degrees of freedom and noncentrality parameter n*, where n* = (l/2a2)e.X^^[TQ TqXÂ„(X^TqX^) X^Tq]X2^02 The distribution of SSE^( weighted )/a under model (4.4) is verified by the following theorem.
PAGE 124
116 Theorem 4.2 . Let Y ~ N^(X3 + X202' a^G), G singular, ^22 2 * 2 Â—1 * Â— then ta /a ~ Xf ^ where a =f (YXB)'T(YXB), f = rank(G:X) rank(X), T = G + XX', and \ = (l/2a^)0^X^[T~ T~X(X'T~X)~X'T~]X23 2The proof of Theorem 4.2 is given in Appendix 12. 4.5 Equivalence of SSE^ and SSE^^( weighted ) In this section we shall show that our expression for SSEy^ ( weighted ) in Eq. (4.10) is equal to Shillington ' s unweighted SSE^ in Eq. (4.6). Thus the complex calculations required for evaluating SSEy^(weighted ) can be avoided by calculating the simpler form SSEy^. Zyskind (1967) investigated conditions under which ordinary least squares estimators are BLUE (best linear unbiased estimators) even though Y in the model Y = X3 + E, where E(e) = 0, does not have variance2 covariance matrix equal to a I . Zyskind assumes that 2 var(Y) = a V, where V is nonnegative and possibly singular, and then states and proves the following necessary and sufficient condition for ordinary least squares estimators to be BLUE. Theorem 1 (Zyskind, 1967) . A necessary and sufficient condition for all simple least squares linear estimators to be also best linear unbiased estimators of the
PAGE 125
117 corresponding estimable parametric function X'g in the linear model Y = Xg + e, E(e) = 0, E ( e e ' ) = a^V, where V is a symmetric nonnegative matrix and X is of rank r, is that there exist a subset of r orthogonal eigenvectors of V which forms a basis for the column space of the matrix X. In a second theorem, Zyskind (1967) gives several other necessary and sufficient conditions for ordinary least squares estimators to be BLUE. These conditions are shown to be equivalent to the condition in Theorem 1 (Zyskind, 1967). The fifth of these conditions in Zyskind 's second theorem is that VP = PV, where P = X(X'X)~X'. Applying condition 5 of Theorem 2 (Zyskind, 1967) to our problem of regressing W on X^ we have ^ = ^0 and p = x^(xÂ„'V ^ and therefore
PAGE 126
118 ^^ = ^o^w(Vw^ ^ ^w^Vw^ ^ ' since E^X^ = S^E^X = E^X = X^. It follows that VP = X^(X^Â„) X^Eq = PV. Therefore by Theorem 2 (Zyskind, 1967) the ordinary least squares solutions from regressing W on X^ are BLUE estimators, and thus are equivalant to the solutions obtained from weighted least squares. We conclude therefore that SSE^ = SSE^( weighted ) . 4,6 The Test Statistic As stated in Section 4.1, Shillington (1979) proposed that a fitted model be tested for lack of fit by using the F ratio MSBÂ„ F = ^ MSE^ given in Eq. (4.2). In this section we shall verify that Shillington ' s F ratio possesses a central F distribution when the true model is of the form in Eq. (4.3), and possesses a doubly noncentral F distribution when the true
PAGE 127
119 model is of the form in Eq. (4.4). This information on the distribution of the F ratio will be needed in Section 4.7, where the power of the test is discussed. Additionally, we shall give the form of the expected values of both the numerator, MSEg, and the denominator, MSE^, of the F ratio in Eq. (4.2). These expected values will aid us in developing a procedure for calculating the power of the test, since they will be used to determine whether the test is upper tailed or lower tailed. In developing the distribution of the F ratio in Eq. (4.2), we shall show that SSE^/a'^ and SSEÂ„/a^ are Statistically independent. In this pursuit, let us use the expression for SSEg in Eq. (4.5) and the fact that Y^ = MY to express SSEg as SSEg = Y'M'[Gq^Gq^X^(X^Gq^X^)"^X^Gq^]MY. Also, using the expression for SSEÂ„ in Eq. (4.6) (which is allowed because we showed in Section 4.5 that the correct form, SSE^^(weighted) , is equal to SSEy^) and using the fact that W = IqY, we can express SSE^ as 2^^W = r^ot^N Ww)"^JS^ By Theorem 4 (Searle, 1971, p. 59), to show that 2 9 SSEg/a and SSE^/a are statistically independent, it suffices to show that the matrix product
PAGE 128
120 is equal to the zero matrix. This is seen to be true since MEq = 0, and therefore SSE^/a and SSE^/a^ are independent. When the fitted model and the true model are both of the form in Eq. (4.3), then the F ratio in Eq. (4.2) possesses a central F distribution with g p and N g r degrees of freedom in the numerator and denominator, respectively. Furthermore, the numerator, MSEg, of the F ratio in Eq. (4.2) has expectation equal to E(MSE3) = [oV(g P)]EXg_p 2 = a . Similarly, under model (4.3), the expected value of the denominator, MSE^^, of the F ratio has expectation equal to E(MSE^) = [0V(N g r)]Ex^_g_^ _ 2 = a When the fitted model suffers from lack of fit and the true model is given by Eq. (4.4), the F ratio in Eq. (4.2) is a ratio of two statistically independent noncentral chisquare random variables, each divided by its respective
PAGE 129
121 degrees of freedom. Thus the F ratio in Eq. (4.2) possesses a doubly noncentral F distribution with g p and N g r degrees of freedom and noncentral ity parameters n , and nÂ„, where n ^^ and n were given in Eqs. (4.7) and (4.8), respectively. The expected value of the numerator, MSEg, of the F ratio can be written as E(MSE3) = [aV(g P)]Ex^l^^^^ = a^ + t^C^^^/iq p) where ^1 = X2^*tG^ Go\(^C^A)"'^cGo'^^^2(4.11) Similarly under model (4.4), the expected value of the denominator, MSE^, of the F ratio can be written as E(MSE^) = [aV(N g r)]Ex^,? Ngr ,Il2 = a + 0^C232/(N g r) where S = ^2^0f^N ^W^^wV ^^^0^2(412)
PAGE 130
122 4.7 The Testing Procedure and its Power As discussed in Section 4.1, Shillington (1979) suggested that lack of fit of the fitted model be inferred when the value of the F statistic in Eq. (4.2) exceeds F ^, . The test, however, is not always upper a;gp,Ngr j tttailed, and in fact can be lower tailed. The test is considered lower tailed when, because of lack of fit, the expected value of the numerator of the F ratio is less than the expected value of the denominator of the F ratio. We suggest that lack of fit be tested with an upper tailed test using the F ratio F = MSEg/MSE^ when the matrix D, which is defined as C C 1 2 D = Â— J. = (4.13) gp Ngr is found to be positive definite (which can only occur when Ci is positive definite, by Theorem 3.1 in Appendix 4). The matrices C, and C2 in Eq. (4.13) are defined in Eqs. (4.11) and (4.12), respectively. An upper tailed test is appropriate v;hen the matrix D is positive definite because no matter what the value of g is, the expected value of the numerator, MSEg, of the F ratio will be greater than the expected value of the denominator, MSE^, of the F ratio. However, there may be cases where D is negative definite (which can only occur when C2 is positive definite), and in this case lack of fit should be tested with a lower tailed
PAGE 131
123 rejection region. If D is indefinite, then the F test for lack of fit may be upper tailed, lower tailed, or lack of fit may not be testable depending upon the value of g . In those cases where D is indefinite, it is helpful to write the quantity S^D6_2^ which represents the difference between the expected value of the numerator and the expected value of the denominator of the F ratio, F = MSEg/MSE^, as 6^6.2 = Â§.2["i*"2"3l diag[r^, T^ = 0, r 3 ] [U^ :U2 103 ] Â• g . = ^^U^T^U[^^ + ^F3r3U^i2' where U^, U2, and U3 are matrices whose columns are orthonormal eigenvectors of D, and r,, r ^, and r^ are diagonal matrices whose elements are the positive, zero, and negative eigenvalues of D, respectively. Lack of fit is testable with an upper tailed test if B is in the column space of [Uj^:U2], but not entirely in the column space of U2, since then 6^062 is positive. Similarly, lack of fit is testable with a lower tailed test if g is in the column space of ["2:03] , but not entirely in the column space of U2, since then S^D6 2 is negative. If 62 is in the column space of U2f then lack of fit cannot be tested, since 2^2 ^^"''^ equal zero.
PAGE 132
124 We define g__ to be in the column space of U2 if the matrix LIL has a zero eigenvalue, where L = [g :U ]. Similarly, letting L^ = [^^:U^:U^] and L^ = [025U2:U2], e is in the column space of [u,:U2] if L'LÂ„ has a zero eigenvalue, and g is in the column space of [u :U ] if L^L^ has a zero eigenvalue. When D is positive definite or D is indefinite but 8 is upper tailed testable, then the F test for lack of fit which makes use of the F ratio F = MSEg/MSE^ is a test of the hypotheses (see Theorem 3.2, Appendix 5) %'Â• ^^="2 = "a* "j^/Cg p) n2/(N g r) > 0. When D is negative definite or D is indefinite but g is lower tailed testable, then the F test tests HÂ„: n, = n^ = 12 H,: n,/(g p) n,/(N g r) < 0, In the case where D is indefinite and g is in the column space of U2Â» then no hypotheses concerning lack of fit of the fitted model can be tested. When the test is upper tailed, the power of the F test for lack of fit is defined as
PAGE 133
125 Power = p{f" Â„ Â„ Â„ > F ^^ } (4.14) where F Â„ is the upper 100a percentage point of the a;gp,Ngr central F distribution with g p and N g r degrees of freedom. In the case of a lower tailed test, the power of the test is defined as Power = P{F" Â„ Â„ Â„ < F, Â„ }. (4.15) ^ gp,Ngr;ni ,n2 1a ;gp,Ngr ^ 4.8 Selection of Near Neighbor Groupings In the preceding sections of this chapter, we have discussed a near neighbor procedure which uses the F ratio F = MSEg/MSE^ to test a fitted model for lack of fit. In this section we shall investigate the effect that different groupings of response observations into near neighbor cells have on the testing procedure and its power. From equations (4.14) and (4.15) in the previous section it is evident that the power of the F test for lack of fit, which makes use of the F ratio F = MSE3/MSE^, depends on the values of the numerator and denominator noncentrality parameters, n, and II Â„. Assuming the numerator and denominator degrees of freedom are fixed, and the test is upper tailed, then the power is an increasing function of increasing values of II and is a decreasing function of increasing values of II (see Appendix 1). When the test is lower tailed, the
PAGE 134
126 power is an increasing function of n _ and is a decreasing function of n . Since both the numerator and denominator noncentrality parameters, n and U , are functions of the groupings of the data points into near neighbor cells (as are the numerator and denominator degrees of freedom), we would like to investigate the effect of the number and composition of cells on the power of the F test. Intuitively, it would seem that homogeneous near neighbor cells would minimize the 2 bias inherent in estimating a with MSE^^, and thus would minimize R and maximize the power of an upper tailed test. However, any grouping of the data points would also influence the numerator noncentrality parameter and the numerator and denominator degrees of freedom. Therefore while a grouping of data points into homogenous cells might decrease II Â„ and thus apparently increase the power of an upper tailed test, the result of the grouping on the power also depends on how the degrees of freedom, g p and N g r, and the numerator noncentrality parameter, IT,, are affected. We will attempt to find homogeneous cells of near neighbor points by using an iterative partitioning clustering algorithm. Two examples will be presented. The first example makes use of the stack loss data presented by Daniel and Wood (1971) and later analyzed by Shillington (1979). The second example involves data from a mixture experiment discussed by Piepel (1981).
PAGE 135
127 Our objective is to investigate the effect of forming homogeneous cells of near neighbors on the F test for lack of fit which makes use of the test statistic F = MSEg/MSE^. It is hoped that such homogeneous groupings will increase the power of the F test (assuming that the rejection region is upper tailed) by decreasing n for a fixed number of near neighbor cells (and thus fixed values for the degrees of freedom). The effect that homogeneous grouping has on n , is not clear, but is of interest, since the magnitude of n , also affects the power of the test. Additionally, we will vary the number of cells of near neighbors in an attempt to determine how this affects the power of the test, since the number of cells affects both the noncentrality parameters and the degrees of freedom. The algorithm used for grouping the data points into homogeneous near neighbor cells can be described as an iterative partitioning type of cluster analysis. The computations involved were accomplished using the RELOC procedure available in the CLUSTAN IC computer package (Wishart, 1975). All computations were performed using data points whose coordinates were standardized by subtracting off sample means and dividing by sample standard deviations. Initially, k clusters (near neighbor cells) of the N data points in the factor space are arbitrarily defined. Then the Euclidean distance between each point and the centroid (average vector value) of each of the k clusters is determined. If a point is found to be closer to the
PAGE 136
128 centroid of one of the other k 1 clusters than to the centroid of the cluster in which it is currently classified, then the point is reclassified into that nearest cluster (cell). The centroids of the k clusters are then recalculated, taking into account any reclassified point. The entire set of N points is scanned repeatedly in this manner until no reclassification occurs. This method of assigning points to clusters will be referred to as iterative relocation. In the second stage of the algorithm, two of the k clusters arrived at by the iterative relocation procedure are fused, resulting in k 1 clusters. The two clusters to be fused are selected as those which when fused produce the k 1 clusters with minimum "error sum of squares." The error sum of squares is defined as the sum of squared Euclidean distances between every point and the centroid point of the cluster to which it belongs. After k 1 clusters are determined using the error sum of squares criterion, iterative relocation is applied to the k 1 groups in an effort to improve the clusterings. This alternation of fusion and iterative relocation continues for k 2 clusters, k 3 clusters, ..., 2 clusters, or until a specified minimum number of clusters is reached. The question of determining an "optimal" number of clusters is not addressed by this procedure.
PAGE 137
129 4.8.1 Example 1 Â— Stack Loss Data The first example we investigate is the 21 observation stack loss data of Daniel and Wood (1971), which was analyzed by Shillington (1979). The data (see Table 8) consist of the values of three factors, x^ (air flow), Xj (cooling water inlet temperature), and X3 (acid concentration) along with the values of a response variable, Y (stack loss). A first order regression equation of the form E(y) = 3o + B^x^ + ,^x^ . 33X3 is fitted using 17 of the original 21 observations (Shillington discarded 4 of the original 21 observations as outliers). We assume the true model to have the form E(Y) = 3o + B^XjL + 32X2 + B3X3 + 3;l1^J ^ ^22^2 "^ ^33^3 and thus contains P2 = 3 terms in addition to the p = 4 terms in the fitted model. We wish to investigate the capability of the F test MSE F = B M^ in detecting lack of fit of the fitted model. We first consider the use of the F statistic with the six cells of near neighbors used by Shillington (1979), which is the same near neighbor grouping suggested by Daniel
PAGE 138
130 and Wood (1971). This 6 cell grouping (see Table 8 under the column heading "6**") is found to yield a matrix D = [Cj^/(g p)] [C2/(N g r)] which is indefinite, since the eigenvalues of D have the values 12110/ 7, and 1415 (see Table 9). Thus the test is not upper tailed, since D must be positive definite for an upper tailed test to exist for all values of ^j, where in this example, ^2 = ^^11' ^22' ^33^'* When the 6 cell grouping of near neighbors generated by the iterative partitioning clustering algorithm (see Table 8 under the column heading "6") is used, the values of the eigenvalues of D are 49090, 379, and 43, and the test is still not upper tailed for all values of 3_. We then use the iterative partitioning clustering algorithm to determine homogeneous cell groupings for 5, 7, 8, 9, 10, 11, and 12 cells. The matrix D is found to be indefinite for the groupings into 5, 7, or 8 cells, but D is positive definite for 9, 10, 11, or 12 cells of near neighbor groupings. Thus, no matter what the value of g_2' lack of fit can be tested with an upper tailed test using the 9, 10, 11, or 12 cell groupings of near neighbors. The value of F = MSEb/MSE^ was calculated using the matrix procedure from the 1979 version of SAS. None of the near neighbor groupings provided evidence of lack of fit, and thus we cannot conclude that there is lack of fit when the fitted model is E(Y) = e + 3,x + ^j^o "*" ^3^3
PAGE 139
131 CM CO
PAGE 140
(N 00 LD CN r~
PAGE 141
133 and the true model contains only pure quadratic terms in addition to the first degree terms. For the groupings of near neighbors into cells which provide an upper tailed test (9, 10, 11, or 12 cells), the power of the upper tailed test can be approximated if n , and II can be specified. This approximate power can be calculated using an approximation to the doubly noncentral F distribution given in Johnson and Kotz (1970, p. 197), which is described in Appendix 6 of this dissertation. Thus we calculate an approximation for P( F" > F I ^ gp,Ngr;ni ,n2 a;gp,NgrJ * In order to compare the power of the upper tailed F test for 9, 10, 11, and 12 cells, we will assume arbitrarily that the true value of the parameter vector B^^ ^2 "" ^^11' ^22' ^33^' ^ (.044, .329, .033)' which is arrived at by taking eÂ„ = lOg^ where 3 is the least squares estimate of gÂ„ calculated from the data. Furthermore, 2 taking a =1.6 (since the residual mean square value from fitting the "true" second degree model is MSE = 1.6) we 2 calculate the values for n , = 3AC,3/2a and II2 = 0^020 2/20^ for each of the 9, 10, 11, and 12 cell groupings. The calculated values of H and n , as well as the approximate power values for each of the four F tests (calculated using the approximation to F" from Johnson and Kotz (1970, p. 197)) are presented in Table 9. The power is
PAGE 142
134 quite high {> .96) for 9, 10, or 11 cell groupings, but drops off to .79 for the 12 cell grouping. This drop in power seems to be due to the effect of having only 3 degrees of freedom in the denominator of the F ratio. In summary, this example illustrates that the F test for lack of fit that makes use of the statistic F = MSEg/MSE^ is upper tailed only for certain groupings of the design points into near neighbor cells. For the near neighbor cell groupings that provide an upper tailed test, the power is generally high for the values of 6_ and 2 a that we selected, but decreases slightly as we move from 9 to 10 to 11 cells and decreases more severely as we move from 11 to 12 cells. This more severe decrease in power is due to the decrease to only 3 denominator degrees of freedom. 4.8.2 Example 2 Â— Glass Leaching Data The second example we investigate is one in which the leachability, Y, of glass is assumed to be a function of the proportions of eleven chemicals of which the glass is composed (Piepel, 1981). A first order Scheffe polynomial model was fitted to the common logarithms of the leachability values, that is, the fitted model is of the form 11 E(log Y) = Z g.x.
PAGE 143
135 The experimental design coordinates and the values of the 44 data observations are presented in Table 10. For illustrative purposes, the true model is assumed to contain the P2 = 8 second order cross product terras, ^es^'e^'s' ^3,ii''3''ii' ^79''7''9' Se'^s^'e' ^7,ii''7''ii' g_x.x_, 6in^c^in' ^^^ ^cQ^c^^Q ^^ addition to the p = 11 15 1 5 6,10 6 10 59 5 9 first order terms in the fitted model. The 19term model is the final fitted model proposed by Piepel and serves as our true model. Piepel (1981) suggested that the four sets of observations (see Table 10) (a) 14 and 15 (b) 18, 19, and 20 (c) 25, 26, and 27 (d) 39, 40, and 41 were intended to constitute four cells of replicate observations for use in estimating pure experimental error. However, the settings of the mixture components were not well controlled, so that each of the four cells contained near neighbors rather than replicates. By defining each of the remaining 33 data points as 33 cells containing one observation each, the 44 data observations are partitioned into 37 cells. If we choose to use the 37 cells to test the fitted model for lack of fit, with the test statistic F = MSEg/MSE^, we find that there are no degrees of freedom for MSE^ so that the F statistic cannot be calculated in this case. However, the F statistic for
PAGE 144
136 *
PAGE 145
0'^o^o>^(^Ln>x)rnLno in CN n cnr^r^r^'^cN 137 ^ocNorooo CNOCMOCNOOO oooooooo LD 1.D VO Â—( m ^ ITl CN CN CM CN (N CN CN O O O O O O O >^ o Â—( ooooounoo OOOOOCNOf^ oooooooo o o o o o o o ro O O O O O O o o o o o o o ^OrOO^OOOOOOOOrH* mOrooroOOOOOOOPOrOCN OOOOOOOOOOOOOOO O'^r^'^'r^ootN oo or^rrr~oor~oo oooooooooo o o o o r^ o o o o r^ o o o o o OOOOOOOOOCTlrHOrHOOm OLDvCOOOOLDinvDVOVOOOVO OOOOOOOOOOOOOOO c o a; E u o Vl Ou CO CO >1 .Â—I c < vÂ£3^TjCT\Ln'a<":rv^^on ^nHfHHM^HOOM^rHH^^ CO CO Â«3 0) 3 C Â•H 4J c o o I I o JQ m E< ooorooomorooooooo oooocx3oocx)0 ooooo oooo oooooooo oooo ooo OOOO'*C0V000^r^r^'Â«j'^i^vocNmi^ MlÂ— lOMOOOrHrHOOOHÂ— lO r^cyiTO^'^'^rocNjTrcNfHooo*^ MCNincNrcovoronmrorOiÂ— (rocT( T3 T^ TJ OrHCNrOTl'Ln^Dt^(X5cyiO'(fNn'<* CO Ul 0) O 4J (0 to O CO H O H rH a. a* c O (U 0) c Qj a; u u c CP Â•H yi O; O CO 0) Oi 3 (0 o CO en 3 ij (0 (0 ^ (tj
PAGE 146
138 lack of fit can be calculated using from 12 to 33 cells of near neighbors, and thus we refer to the iterative partitioning clustering algorithm discussed earlier in this section to generate near neighbor groupings of 15, 20, 25, and 30 cells (see Table 11). The clusterings of observations into 15, 20, or 25 cells each yields a D matrix that is indefinite (see Table 12), so that the test is not upper tailed. The 30 cell clustering produces a positive definite D matrix, so that the test is upper tailed for all nonzero values of g_, where 3.2 = (egg, 33^3^3^, 379, 35g, Sy,!!' 3j^5, 3g^3^Q, 359)'. Taking 3.2 = (15.141, 112.429, 78.761, 78.275, 87.996, 13.356, 76.948, 34.721)', which is the least squares 2 estimate of 3^ from the data, taking a = .008 (which is MSEp,jj.g with seven degrees of freedom from Piepel's analysis of the data), and using the approximation of Johnson and Kotz (1970, p. 197) to approximate Pf F" > F 1 ^ 19,4;ni ,Jl2 .05;19,4^ where n, = 9.79, n^ = 0.08 and F ^_ ,^ ^ = 5.81, we find 1 2 . 05;19,4 that (using 30 near neighbor cells) a value for the power of the F test is .10. The power increases as the magnitudes of the elements of 3 are increased, so for example if all the elements of 3 , above are doubled, then n , = 39.16, H _ = 0.32, and the approximate power is .25. If the elements of 3_~ above are each multiplied by 5, then
PAGE 147
139 Table 11. Near Neighbor Cells for Glass Leaching Data. Membership in Near Observation Neighbor cells* Number 15 20 25 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 6 14 11 11 11 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 6 14 15 15 15 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 14 15 16 16 16 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 14 15 16 16 16 3 17 Membership in Near Observation Neighbor Cells Number 15 20 25 30 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 1 3 15 15 15 4 12 10 8 1 10 2 13 11 8 9 7 7 7 10 14 2 1 3 16 16 16 4 12 17 8 14 18 2 19 11 8 9 7 7 7 17 20 2 1 3 17 17 17 18 12 19 8 15 20 2 21 22 23 9 7 7 7 19 24 25 18 19 20 20 20 21 12 22 23 15 24 2 25 26 27 28 7 7 7 22 29 30 *Cell groupings generated by an iterative partitioning cluster analysis using the CLUSTAN computer package. Numbers in the table refer to cell membership.
PAGE 148
140 o
PAGE 149
141 vD vD vc r^ r~ CO I I I I I I o o o o o o X VD X X X X r^ 00 00 00 X o ro CN CTi 00 00 I I o o X 00 X IT) 00 CN K in in Â• o CN o^ o^ 00 o vovovDrr~oooor~ I I I I I I I I oooooooo XXX C) o\ o CN ro X in X (N X X X 00 m VÂ£> vD r~ r~r~00 I I I I I I o o o o o o 00 r^ I I o o X X in 00 o m X n XXX in rH ^ ^ ^ 00 X m X CN in CN
PAGE 150
142 n j^ = 244.75, n = 2.00 and the approximate power is .83. From the entry in Table 12, we see that the calculated F value of 20.55 with the 30 cell clustering exceeds ^ ni;.iQ A ~ 5.81, and we conclude that the fitted first degree model is inadequate. 4.9 Discussion When a designed experiment includes replicated points, the adequacy of a fitted model can be tested by comparing the portion of the residual sum of squares due to lack of fit to a second portion due to pure error from the replicates. The test statistic is an F ratio of the mean square due to lack of fit to the mean square due to pure error, and lack of fit is inferred when the calculated value of this ratio is large (Draper and Smith, 1981, p. 120). When replicate points do not exist, lack of fit can be tested using near neighbor observations with the test statisic F = MSEg/MSE^. This F ratio has been shown to possess a central F distribution when the fitted model is adequate, and a doubly noncentral F distribution when the fitted model suffers from lack of fit. When the fitted model is adequate, the expected values of both MSEg and MSE^ are equal to a^, so that the ratio E [MSEg]/E[MSE^] equals unity. However, when lack of fit is present, both MSEg and MSE^^ are biased estimates of a^, and we compare the magnitudes of the biases of these estimates (which are functions of the noncentrality parameters and degrees of freedom of the doubly noncentral F distribution)
PAGE 151
143 in the F test. The test has an upper tailed rejection region if the bias corresponding to MSEg exceeds the bias corresponding to MSEÂ„. The rejection region is lower tailed if the bias corresponding to MSE^ exceeds the bias corresponding to MSEg. In other words, the test is upper tailed if the matrix D (see Eq. 4.13) is positive definite, and the test is lower tailed if D is negative definite. If D is indefinite then the test may be upper tailed, lower tailed or still yet lack of fit may not be testable depending upon the value of 3 . In two examples an iterative partitioning clustering algorithm is used to assign the data points to a preselected number of near neighbor cells. When the number of cells is low, the matrix D is found to be indefinite, so that the F test is not strictly upper tailed or lower tailed. However, by increasing the number of cells, it is possible in both examples to produce a positive definite matrix D, so that the test is upper tailed. Increasing the number of cells not only produces an upper tailed test, but also affects the values of the parameters of the doubly noncentral F distribution. As the number of cells is increased (moving from left to right in Tables 9 and 12) we see that the smallest eigenvalue of C^ increases and that the largest eigenvalue of C2 decreases. Therefore a lower bound for n, , ^2^2^min , , TÂ— ^ '^l 2a
PAGE 152
144 increases as the number of cells increases (where ib . ^min denotes the smallest eigenvalue of Cj^). In addition, an upper bound for n 2 ' 8lBÂ„p TT c 22 max ^2 ^ _ 2 2a decreases as the number of cells increases (where pÂ„=,Â„ denotes the largest eigenvalue of C2). Finally, as the number of cells increases (moving from left to right in Tables 9 and 12), the numerator degrees of freedom, g p, increase and the denominator degrees of freedom, N g r, decrease. Since the parameters of the doubly noncentral F distribution change as the number of cells changes, the power of the F test can be affected. For the stack loss data example, we see in Table 9 that the power of the upper tailed test decreases as we move from 9 to 10 to 11 to 12 cells. An area for future study can be a further investigation of the effect of the number and composition of near neighbor cells on the power of the F test which makes use of F = MSEq/MSE^. This investigation would involve the effect of near neighbor cell selections on the parameters II,, II/ g p, and N g r of the doubly noncentral F distribution. It would be desirable to develop a method (perhaps an alternative to the iterative partitioning clustering algorithm) which could be used to select the number and composition of cells so as to maximize the power of the F test.
PAGE 153
CHAPTER FIVE CONCLUSIONS AND RECOMMENDATIONS Two general methods for testing a linear model fitted in a mixture space for lack of fit have been investigated in this dissertation. The first method makes use of response values observed at check points while the second method makes use of response values observed at design points which are near neighbors in the factor space. In Chapter Two we discussed the work of several authors (Scheffe (1958), Gorman and Hinman (1962), Kurotori (1966), and Snee (1971)) for testing lack of fit which centered on measuring bias inherent in the fitted model when estimating the response at check points. Only the method suggested by Scheffe (1958) was an exact test. In Chapter Three, a method for selecting check points that maximizes the power of Scheffe 's F test was devised. When replicate response observations were available, so that the experimental error " 2 variance could be estimated by a ^ from the replicates, we ^ ext ^ saw that the power of this upper tailed F test was maximized by selecting check points that maximize (or approximately maximize) the noncentrality parameter A, of the noncentral F distribution. When the matrix K^ (where x, = 3AA,3Â„/2a ) was found to be positive semi definite it was determined that only a subset of possible values of the g parameter vector could be detected as contributing to lack of fit. 145
PAGE 154
146 When an estimate of the experimental error variance was not available from replicates, an extension of Scheffe's F test for lack of fit which replaced n with MSE (the ext residual mean square error from the fitted model) in the denominator was developed. We found that to maximize the power of the test it was necessary to select check points to maximize (or approximately maximize) the numerator noncentrality parameter, X , of the doubly noncentral F distribution when the test was upper tailed. When the test was lower tailed, we sought check point locations that minimized (or approximately minimized) X . A criterion was developed for determining whether the test was upper tailed or lower tailed by comparing the expected values of the numerator and denominator of the F ratio when the fitted model was inadequate. Finally, we discovered cases where, for some values of g , lack of fit could not be tested. An alternative to the check points method for testing lack of fit in a fitted model is a procedure that involves measuring the bias that is present in estimates of the response at the design points (the number of design points must exceed the number of terras in the model). When replicate observations are available, the well known procedure in which the test statistic is a ratio of the lack of fit mean square to the pure error mean square can be used to test for lack of fit (see Draper and Smith, 1981, p. 120). When replicate observations are not available, several techniques which make use of near neighbor
PAGE 155
147 observations in place of replicates for testing lack of fit have been proposed in the literature (see Green (1971), Daniel and Wood (1971), and Shillington (1979)). Additionally, it has been suggested by Draper and Smith (1981, p. 42) that lack of fit can be tested by using near neighbor observations as substitutes for replicate observations in the usual lack of fit, pure error F ratio. However, the exact distributions of the test statistics proposed by Daniel and Wood (1971) and Draper and Smith (1981, p. 42) have not been defined, and Green's (1971) procedure requires an inordinately large number of observations. Thus because of these reasons we chose Shillington 's (1979) procedure to study in greater detail in Chapter Four. In Chapter Four the distributional properties of Shillington 's test statistic were developed, and a method based on an iterative partitioning clustering algorithm for defining groups of near neighbor observations was proposed. It was shown that the power of Shillington ' s test depends on the parameters of the doubly noncentral F distribution, and that the manner in which observations are grouped as near neighbors can alter the values of the parameters of the doubly noncentral F distribution and thus affect the power of the test. We found that increasing the number of near neighbor cells so that individual cells become more compact produced an upper tailed F test in the
PAGE 156
148 two examples studied, but that there are many other cases where the test will not be upper tailed. Now that we have briefly summarized our findings from investigating the check point and near neighbor methods of testing lack of fit in a mixture model, a logical question is, "Which of the two methods is better?" It was not our original intent to address this question in this dissertation, but an interesting result that has been discovered in the latter stages of our investigations is as follows: Under certain circumstances, the check point method for testing lack of fit is equivalent to the usual method which partitions the residual sum of squares into sums of squares due to lack of fit and due to pure error (which was shown in Chapter Four to be a special case of the near neighbor method). Because we have not found a derivation of the equality of these methods in the literature, we shall show it here. In Chapter Three, check points were used to test lack of fit in a fitted model of the form E(Y) = Xg , . With k check points, the test statistic was of the form (see Eq. (3.3)) d'v/d/k F = *2 ^'ext ^2 2 where a ^ is an external estimate of a which can be ext
PAGE 157
149 calculated from replicates, if they exist. The vector d in the F ratio was defined to be a vector of differences between observed and predicted response values at the k check points having the form d = Y* X*(X'X)~'^X' Y, where Y* is the kx 1 vector of observed response values at the k check points and X* is the corresponding settings of 2 the model terms at the check points. The matrix a V^ was defined as the variancecovariance matrix of d where Vq has the form V^ = I, + X*(X'X) "x*' . Ok It can be shown (see ^pendix 13) that if we define the vector Y, as A ^A = Y Y* observations at the check points and similarly define the matrix X^ as ^A = X X* design P2iD^_Â§2^^i'^5Â£ check point settingi so that the original design points as well as the check points are all taken at once as design points in regressing Yon X, then A A
PAGE 158
150 SSE^ = X^f^N+k) ^a(^A^a)~^^A^Xa = ^'V^d + SSE. (5.1) Thus, the residual sum of squares, SSE^, from the analysis of the fitted model when both the original design points and the check points are used to fit the model is equal to the sum of the quadratic form, d'V~ d, used in the numerator of the check point F test and the residual sum of squares, SSE, from the analysis of the fitted model using data collected only from the original design points. If we perform the usual partitioning of SSE^ into a lack of fit sum of squares, SSj^Qp/^x, and a pure error sum of squares due to replicates, SSEp^_jj.g( j^j , then from Eq. (5.1) we can write SSlOF(A) ^ SSEp^re(A) = ^'^o'^+ S^^' ^^.2) Thus from Eq. (5.2), when SSE^^j^^^j is equal to SSE, then SSlof(a) becomes equal to d'v" d so that the check point F 1 * 2 ratio, F = (d'VÂ„ d/k)/a ^, and the usual lack of fit F ^ext ratio, F = MS,^Â„,,,/MSE ,^,, are equivalent. We now LOF(A)'^ pure (A) present an example to illustrate the result in Eq. (5.2). Let us fit a second degree Scheffe polynomial model to the following hypothetical or artificial response observations collected at the six points of the {3,2} simplex lattice design:
PAGE 159
151 Y = 2350 2370 2450 2430 2650 2670 2400 2420 2750 2730 2950 2970 X = 1
PAGE 160
152 If we use all of the observed response values to fit the second order Scheffe polynomial, then the model is Y = 2360. 7Xj^ + 2437.7x2 + 2661.7x3 + 183.3x^X2 + 1071.3x^X3 + 1785.3X2X3 and the residual sum of squares is SSE^ = 25746.7. This residual sum of squares can be partitioned into SSlof(A) = 24546.7 and SSEpuj^e(^) = 1200. The F ratio for testing lack of fit is calculated to be F = MSLOF(A)/MSEpure(A) = [ 24546 . 7/3] / [1200/6] = 40.91, which is identical to the previously calculated F value. In the above example we note that SSEp^j.g/^j is equal to SSE (SSE = SSEpy^g(^p so that SSlof(A) ^^ equal to d'VÂ„ d. Since both the check point F ratio and the usual lack of fit F ratio have produced the same value, F = 40.91, we conclude that the two methods for testing lack of fit in the fitted model are equivalent. In order to put this dissertation in a better perspective, we now make some concluding remarks on the lack of fit testing procedures investigated, including possible drawbacks, extensions, and recommendations for future work. An aspect of our investigations that may raise some questions is that our methods are dependent on the specification of the form of the true model believed to be responsible for lack of fit in the fitted model. Requiring the form of the true model to be specified was necessary in order to be able to investigate the power of the testing procedures. There are situations, however, where a complete or true model can reasonably be specified. One example
PAGE 161
153 could be in fitting polynomial models, where the polynomial of one degree higher than the fitted model could be taken as the true model. We now mention two ways in which our results can be applied to more general situations than may be readily apparent from our previous discussions. First, we point out that all examples in Chapters Three and Four dealt with polynomial models. This type of model was selected because of its popularity and v/ide applicability, however, our methods can be applied not only to polynomial models but to any models which are linear in their parameters. Secondly, it was our intent in this dissertation to discuss methods for testing lack of fit in a mixture model, but the methods discussed can certainly be used not only in mixture problems but also in general response surface problems in which a linear model is fitted. This generalization is illustrated for the near neighbor approach to lack of fit testing through the stack loss example in Chapter Four. Topics for future research stemming from this dissertation were listed in the concluding paragraphs of Chapters Three and Four. One area suggested in Chapter Three was to investigate the effect of experimental design on the selection of check points and on the resulting power of the test. Perhaps a "minimum bias" design could be used for fitting the model, while lack of fit could be detected with "high bias" check points, but this in only speculation, and needs to be investigated.
PAGE 162
154 The fact that the check points method and the standard method that partitions the residual sum of squares into lack of fit and pure error portions were found, under a certain condition, to be equivalent suggests that selecting check points to maximize the power of the check point F test may in general be equivalent to choosing points to augment the original design. The augmented points would be chosen to maximize the power of the F test that partitions the residual sum of squares into lack of fit and pure error sums of squares. An investigation of the selection of optimal check points versus the selection of optimal augmented design points would be of interest. For the near neighbor test for lack of fit it was recommended in Chapter Four that other methods besides the iterative partitioning clustering algorithm might be considered for selecting groups of near neighbors. The effect of the number and composition of the groups selected on the power of the test through their effect on the parameters of the doubly noncentral F distribution could then be investigated. In view of the equivalence of the check point method and the method that partitions the residual sum of squares when replicates exist (see Eq. (5.2)), it would be of interest to investigate whether there is also some equivalence between Shillington ' s near neighbor F ratio and the check point F ratio, F = (d'V~ d/k)/MSE, to be used when an external estimate of a is not available. If the methods
PAGE 163
155 are not equivalent, perhaps one could be shown to be preferable to the other as judged by comparing the power of the two procedures in testing for lack of fit. Finally, the focus of this dissertation has been on testing lack of fit in linear models so that another area for future investigation can be the problem of testing lack of fit in models which are nonlinear in their parameters.
PAGE 164
APPENDIX 1 INFLUENCE OF X . ON p{ F" > F 1 Vi,V2'^l/^2 a;vi,V2 In this appendix we show that P{ F" > f } is an increasing function of X^. Let X , ..., X , Y., ..., Y be independent N(0,1). i V 1 i V2 \ r ' Then F = (V /v )[(X + x/2)2 ^ J, x2]/[(Y. + xy^)^ + E Y^] i=2 1 / i=2 ^ is distributed as F" where vi and v^ are the respective numerator and denominator degrees of freedom and XjL and X2 are the respective numerator and denominator noncentrality parameters (Scheffe, 1959, p. 412413). Fixing the values of v^, v 21 and X 2 we wish to show p{F" > , > f 1 is a strictly increasing function VlfV2;Xi,X2 a;vi,V2^ "^ ^ of X 1 , where F represents the upper 100a percentage Ja ;vi ,V2 rrc^ point of the central F distribution with vi and V2 degrees of freedom. Let f(xl/2) ^ . , ,. . ^2 = P{ (v^/v^)! (X^ + xJ/2)^ + Z xJ]/[(Y^ + X^/2)2 ^ J, y2^ 1=2 i=2 156 a ;v^ ,^2
PAGE 165
157 1/2 then f(^/ ) niay be rewritten as f(xl/2) = p((x^ + xl/2)2 > u} = 1 P{(X^ + X^/2j2 , Â„ ^ ( Al . 1 ) where U = (v^/v^)[(Y^ . X ^^ ) 2 , _z^ yJJF^^^^^^^ ^E^ X^ . Note that the random variable U is independent of X^* If X / and XT'! denote any two values of Xy such that X^/2 ^ x/2, then we shall prove that for f(XJ/2) defined as in (Al.l), f(^]^{^) < ^^^Yl^^' ^Â°^ f(x/^) =17 g,^ (u)p(u)du where p(u) is the p.d.f. of U, and for any positive number, u', g,^ (u') denotes the ^ 1 1/2 2 conditional probability that (X^ + X^ ) < u', given U = u'. However, this conditional probability must be the same as the unconditional probability, since Xi and U are statistically independent. Thus g^'s (u') is the probability that the random ^1 1/2 variable Xj^ falls in an interval of half length u' 1/2 centered at X, . Since X ~ N(0,1), this is a decreasing function of X ;/ . Therefore g,*2 (u*) g,^ (u') > for all u' > 0. Hence, f(xl/2) _ f(x]/2) = J [g^^^(u) + g^^^(u)]p(u)du < 0.
PAGE 166
158 Thus P{ F" , , > F } is a strictly increasing function of X i . We note that this proof is a modification of the proof that Pf F" , , > F } is decreasing in X, ^ Vi,V27Xi,X2 a;vi,V2' ^ (Scheffe, 1959, p. 136).
PAGE 167
APPENDIX 2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR GLOBAL OPTIMIZATION W. L. Price (1977) describes a conceptually simple random search procedure, called "a controlled random search procedure for global optimization," which is effective in searching for global minima of a function of n variables, with or without constraints. The procedure does not require the function to be dif ferentiable or the variables to be continuous. An initial search domain, V, is defined by specifying upper and lower bounds for each of the n variables, and a predetermined number, N, of trial points are chosen at random over V, consistent with any constraints. The function is evaluated at each of the N trial points and the position as well as the value of the function at each point are stored in an array. A' . At each iteration a new trial point, P, is selected randomly from a set of possible trial points whose positions are related to the configuration of the N points currently in storage. If P satisfies the constraints, the function is evaluated at P and the function value, fp, is compared with fj^, which is the greatest function value for the N points already in storage. If fp < f^ then M, the point in storage corresponding to f^, is replaced, in the array A', by P. If p fails to satisfy 159
PAGE 168
160 the constraints or if f > f then the trial is discarded and a new point is chosen from the potential trial set. As the algorithm proceeds, the set of N points in storage tend to cluster around minima. As Price states, "the probability that the points ultimately coverge onto the global minimum (minima) depends on the value of N, the complexity of the function, the nature of the constraints and the way in which the set of potential trial points is chosen. " Price notes that since the procedure is intended to find global minima, thoroughness of search is more important than speed of convergence, but if the procedure is to be more efficient than pure random search the probability of success (f < f ) at each iteration must be sufficiently p m high. His procedure reaches a compromise between the requirements of search and convergence by defining the set of potential trial points in terms of the configuration of the N points already in storage. At each iteration n + 1 distinct points, R, , RÂ„, ..., R ,, are chosen at random 1 2 n+1 from the N (N > n) currently in storage and these constitute a simplex of points in nspace . The point R is arbitrarily chosen as the vertex of the simplex, and the next trial point, P, is taken as the image of the vertex with respect to the centroid, G, of the remaining n points. Thus P = 2G R ,. He notes that it is possible n+1 to speed up covergence by selecting the vertex as the point R. , i = 1, 2, ..., n + 1, which has the largest
PAGE 169
161 function value of the points Rw R/ ...Â» R but this would be detrimental to the thoroughness of the search. The version of Price's procedure used in the work in this dissertation was programmed in the FORTRAN language by Michael Conlon of the Center for Instructional and Research Computing Activities, the University of Florida, Gainesville, Florida. This version of Price's procedure selects new trial points using the suggested criterion P = 2G R , . The algorithm continues until an iteration n+1 limit is reached or a desired tolerance between the minimum and maximum function values in storage is achieved. In our particular application, if P2 = 1 so that A^ is a scalar, we wish to maximize A^ = (X* X*A)'Vq^(X* X*A), with respect to k check points, in order to maximize the power of an upper tailed test. For locating check points that maximize the power of a lower tailed test it is necessary to minimize Aj^. If p > 1 so that Aj^ is not a scalar, but is a P2XP2 matrix, then it will be necessary to maximize or minimize certain eigenvalues of A,. All of these optimization problems can be handled by Price's procedure. Since the procedure finds minima, then to find maxima, we simply minimize the negative of the function under consideration. The restriction that the check points must be located within the experimental simplex
PAGE 170
162 (or a subregion of the simplex) is taken care of by specifying constraints in the program. To give a specific example, suppose we fit a second order canonical polynomial model in a three component mixture space, using a simplex centroid design. If we assume the true model is special cubic in the three components, then P2 = 1, and ^1 ^ ^^2 " X*A) Â•Vq'(X* X*A) is a scalar quantity. In order to locate a single check point that maximizes the power of an upper tailed test for lack of fit, we select the check point that maximizes Ai. Since the experimental region we wish to search is the entire two dimensional simplex, we define the check point as x*' = (x , x , X ), and in our program impose the constraints : and < X < 1, < X < 1. We then define x, as x^ = 1 x, y.^, while requiring that < X < 1. Price's random search procedure is used to search the twodimensional simplex for the point (xj^, X2) that maximizes K^.
PAGE 171
163 Price suggests the use of N = 50 storage points for such a twodimensional search, and we have generally found this to be adequate. For k > 1 check points to be located simultaneously in a three component system, the problem becomes one of searching in 2k dimensions. For the applications considered, N = 50k appears to be adequate. The only real problem encountered has been that of economics in that the procedure becomes costly in terms of computer time for these situations where the optimal value of the function is assumed by all points in a region. In these cases the algorithm searches in vain for points that will improve upon the functional values already in storage, which all lie in this optimum region. However, in other applications, the procedure converged quickly to an optimum (those that converged did so in 10,000 iterations or less, at a small cost in computational time).
PAGE 172
APPENDIX 3 _, STATISTICAL INDEPENDENCE OF d Â• V d /a AND SSE/a^ Let us write d'v" d as 1 * * * _i * * * d'V d = (Y Y )'V "(Y Y ) Y 'Vq Y Y 'Vq Y Y 'Vq'y + Y 'Vq^Y * 1* * 1** '* 1** Y Â•Vq'y 2 Y Â•Vq'y + Y 'Vq^Y . Now let us write SSE as SSE = Y'(I^ X(X'X)~^X')Y. Since Y and Y are independent, SSE is independent of * _l * * _i ^ * Y 'VÂ» Y . Rewriting y 'VÂ„ Y as * Â—1 * * * Â—1 * 1 '^0 " '^0^ 1' where b is the least squares estimator of g , we have * Â— 1"* * Â—1* Â—1 1 '^0 " '^0^ (X'X) X'Y. 164
PAGE 173
165 We now show that the second portion of d'V^ d is independent of SSE if and only if [v''x (x'x) 'x'llijg x(x'x) "x'] = 0. Define COv[Y 'Vq'"X (X'X) ""XY, Y'(I^ X(X'X) '"XMY] E[Y 'V '''X (X'X)~''"X'YY' (Ij^ X(X'X) '"X')Y] E[Y Â•Vq'X (X'X) 'Â•X'Y] E[Y'(Ijj X(X'X) "X'Y] * 1 * 1 1 E(Y ') E[Vq X (X'X) X'YY'dj^ X(X'X) X')Y] E(Y ') E[Vq''"X (X'X)~'"X'Y] E[Y'(Ij^ X(X'X) 'Â•X'Y] E(Y ') [cov(Vq"'"X (X'X)~'''X'Y, Y'(Ij^ X(X'X)~'''X'Y)] = 0, if Vq X (X'X) '"X'Y is independent of Y'(I X( X' X) ""'"X ' ) Y. This occurs if and only if. [Vq'X*(X'X)"'X' ] [Ij^ X(X'X)"'X'] = 0, see Searle (1971) p. 59, Theorem 3. Now,
PAGE 174
166 [Vq^X (X'X)"'X'] [Ij^ X(X'X) ^x'] = Vq'x (x'x)"^x' Vq^x*(x'x) 'x'x(x'x)"'x' = 0. Therefore SSE is independent of the second portion of d'V~ d. Now we must show that SSE is independent of the third portion of d'V~ d. Write Y*'V~'Y* as Y 'Vq Y = (X bj^)'VQ'x b^ = Y'x(x'x)"'x Â•Vq'x (X'X)~'X'Y. Then SSE is independent of the third portion of d'v" d if and only if [X(X'X)"'X*'Vq'"X*(X'X)""^X'] [Ijj X(X'X) "'"X'] = 0, see Searle (1971), p. 59, Theorem 4. Continuing then. [X(X'X) "'X*'Vq''"X*(X'X) """XMIIj^ X(X'X) """X'] X(X'X) Â•'X*'Vq'"X*(X'X)"'X'X(X'X) 'Â•X*'Vq"'X*(X'X)""'"X'X(X'X)~'X' X(X'X)~''X*'Vq'X*(X'X) """XX(X'X) 'X*'Vq'X (X'X)~ X' = 0.
PAGE 175
167 Therefore SSE is independent of the third portion of Finally, since SSE is independent of each of the three portions of d'V d, we can conclude that SSE is independent of d'v" d and therefore SSE/a^ is independent of d'V~'''d/a^.
PAGE 176
APPENDIX 4 THEOREM 3.1 Theorem 3 . 1 Let A and B be kxk matrices. If (A B) is positive definite and B is positive semidefinite, then A is positive definite . Proof We assume that (A B) is positive definite. Then z'(A B)z > 0, for all z * 0. Thus z ' Az z Â• Bz > 0, for all z* 0, so that z ' Az > z Â• Bz > 0, for all z t 0, since B is positive semidefinite. Therefore, z'Az > 0, all z t 0. Now if z'Az = 0, then z = for if z + 0, then z'(A B)z > implies z Â• Bz < 0. But this is a contradiction since by assumption z ' Bz > 0. Therefore z must be and A must be positive definite. 168
PAGE 177
APPENDIX 5 THEOREM 3.2 Theorem 3.2 Let Aj^ and A2 be P2> 0. Let k>0, N>0, p>0, and N > p. (a) If [A^/k A^/ili p)] is positive definite then [Xj_/k A2/(N p)] =0 if and only if X ,_ = x 2 = 0. (b) If [Aj^/k A2/(N p)] is negative definite then [Xj^/k X2/(N p)] =0 if and only if x ^ = X 2 = 0. Proof of part (a) . Necessity . Let [A^^/k A^/CN p)] be positive definite and suppose that [X^/k X^/i^ P)] = 0. We show that Xj^ = X2 = 0. The matrix Aj^/k A2/(N p) being positive definite implies 8'[a /k A /(N p)]0 =0 iff 3 = , that is, 2 169
PAGE 178
170 a^A^B^A ^2^22'^^'^ P) = iff e^ = 0^ut. e^Aj^e^/^ka^ e_^A232/2(N p)a^ = iff g^ = 0. Hence X^/k X2/(N p) = iff B^ = 0' It follows that if X l/Ic X2/(N p) =0, then x ]^ = X 2 = 0. Sufficiency . Obviously, if X^ = X2 = 0, then X^/k X^/CN p) =00=0. Proof of part (b). This follows from part (a), since in this case A /(N p) A /k is positive definite.
PAGE 179
APPENDIX 6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION Johnson and Kotz (1970, p. 197) indicate the following approximation for P{ F,"^ ^,^ ., ^ ^,^< F^.,^,,^} where vi and .^ are the numerator and denominator degrees of freedom, respectively, and X ^ and A 2 are the numerator and denominator noncentrality parameters, respectively: P{F" Vl/V2;Xi,X2 "^ ^a;vi,V2J ^^^^v,v' ^ ^aj\>i,V2^ = P{F , < (l/c)F } VfV ' ^ a ;vi ,V2^ where F^.^^^^^ is the upper 100a percentage point of the central F distribution with v^ and V2 degrees of freedom, and where c = [1 + X^/v^]/[l + X ^^v ^] , v = [v^ + X^]^/lv^ + 2X^], v' = [^2 +^2^ /^^? "^ 2X], and F , is a central F random variable with v and v' degrees of freedom. 171
PAGE 180
APPENDIX 7 EQUIVALENCE OF SSEg AND SSrQp WHEN REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS In this appendix we show that SSEg = SS^qf ^^^^ response observations are partitioned into g groups of true replicates rather than g groups of near neighbor observations. From Chapter Two, Section 2.2, if each cell consists entirely of true replicates, then the sum of squares due to lack of fit can be expressed as SS,^^ = SSE SSE LOF pure where SSE is the residual sum of squares from a least squares regression of Y on X and where SSEp^^.^ is the sum of squares due to pure error, calculated from replicates. Since SSE = Y'E.Y, where Z . is defined as in Section pure 04.2, we have ^^LOF = ^'^^N " X(X'X) ^X')Y YTqY = Y'(Ijj J:q)Y Y'X(X'X)"'X' Y. (A7.1) We wish to show that when each cell is composed entirely of true replicates, SSEg ig equal to the expression in (A7.1). 172
PAGE 181
173 Recalling from Section 4.3.1 that Y = MY, where M = diag[(l/n )1', ..., (1/n )1'], we write SSE3 = Y'[G1 G\(X^G\)^X^G^]Y^ = X'M'[Go^ G^Xc(X^G^Xc)^X^G^]MY, where from Section 4.2, G = diag[l/n , 1/n , ..., 1/n ] Recognizing that Gq = MM' and X^ = MX, we have SSE = Y'[M' (MM* ) M B M' (MM' )~''"MX{X'M' (MM* )~'''MX} X'M'(MM') M]Y. Since M'(MM')~ M = I z , we have SSE3 = Y'(I^ Zq)Y Y'd^ ^o^^^^'^^N ^O^^J'^^'^^N ^0^^ and since E qX = when all cells are composed entirely of true replicates, we have SSEÂ„ = Y'(I,, J:^)Y Y'X(X'X)"'X'Y B Â— N Â— Â— = S^LOF'
PAGE 182
174 from (A7.1). Therefore, SSEg is equal to the usual SSlqf when cells are composed entirely of true replicates.
PAGE 183
APPENDIX 8 LEMMA 4.1 Lemma 4 . 1 2 Let Y ~ (XB, a G), G singular. Define T = G + XX'. Define T~ such that TT~T = T. 1. (i) TT~X = X (ii) X'T~T = X' 2. rank(X'T~X) = rank(X) 3. (i) X(X'T~X)~(X'T~X) = X (ii) (X'T~X)(X'T~X)~X' = X' 4. Y is in the column space of T (Ye C(T)), with probability one, by which we mean that there exists a vector a such that letting Y = (y , y , ..., y )' and Ta = (x, , x^, ..., x^J ' , then 1 2 N P{ y^ x^l > e} = ,for all e > 0, i = 1, 2, ..., N. Proof 1. (i) T = XX' + G = XX' + W, where G = W = CC , where C = [X:V] . Now, CC'(CC')~C = C, from Pringle and Rayner (1971, p. 26), and therefore TT~[X:V] = [X:V] from which it follows that TT~X = X (and TT~V = V). 175
PAGE 184
176 (ii) The proof of (ii) follows directly from (i) by taking the transpose. 2. The proof of part 2 is given in Rao, 1973, p. 77, #30. 3. The proof of part 3 is given in Rao and Mitra, 1971, p. 22, Lemma 2.2.6(c). 4. By definition, Y ~ ( Xg , a^G) so that an equivalent representation for Y is Y = Xg + e , where e ~ (0, a^G). We wish to show that the random vector Y is in the column space of T, with probability one. It is sufficient to show that TT~Y = Y, w.p.l. (see Pringle and Rayner, 1971, p. 9). Rewriting TT~Y we have TT~Y = TT~(Xe + Â£) = TT~X6 + TT~Â£. By part 1 of Lemma 4.1, TT~X = X, and therefore X3 e C(T). The proof is complete if we show TT~e = e, w.p.l. The difference TT~e e can be written as TT~~e e = (TT~ Ifj)e/ therefore we must show (see explanation below) that E[e'(TT~ Ij^)'(TT~ I^)e] = 0. (A8.1) The expectation in Eq. (A8.1) can be written as
PAGE 185
177 :[Â£'(TT 1^) '(TT l^)e] = trace [ (TT IÂ„)'(TT I^Ja^G 1 2 o trace [ (TT I^^ ) ' ( TT I )W'] = since TT~V = V, by proof of part l(i) of Lemma 4.1. Therefore Y e C(T) , w.p.l. We now show that proving the equality in (A8.1) is equivalent to proving that TT~Â£ = e, w.p.l. By the Markov Inequality P{ u. v. > e} < [E(u. v.)2]a2 and therefore if E(Uj^ Vj^ ) = 0, we have Uj^ = v^, w.p.l. If u' = (u^, U2, ...,Uj^), V' = (Vj^, w^, ..., Vj^), and if 2 E(u^ v^) = 0, for i = 1, 2, ..., N, then u^ = Vj^, w.p.l, for i = 1, 2, ..., N, which implies that u = v, w.p.l. But E(u^ Vj^)^ = 0, for i = 1, 2, ..., N if and only if "^ 2 Z E(u. V.) =0, and since i=l ^ ^ ^ 2 I E(u. V. ) = E(u V) ' (u V) i=l ^ ^ we have u = v, w.p.l, if E(u v)'(u v) = 0. In (A8.1) we take u = TT~Â£ and v = e .
PAGE 186
APPENDIX 9 PROOF OF THEOREM 4 . 1 ( i ) In this appendix we give the proof of Theorem 4.1(i). We show that E(a ) = a , where a = f~ (Y X3)'T~(Y Xe ) " 2 First we write a as ^2 ^ f Ir Â„,m = f ^[Y'T Y 2B'X'T Y + 3'X'T Xg] , where $ = (X'T X) X'T Y. Now, e'X'T Xe = B'X'T X(X'T X) X'T Y = g 'X'T Y, by Lemma 4.1, part 3(ii). Therefore a^ = f"'[Y'T~Y B'X'T~Y] = f"'[Y'T~Y { (X'T~X)~X'T~Y} 'X'T~Y] = f~'[Y'T~Y Y' (T~) Â•X(X'T~X)~X'T~Y] f''Y'A Y (A9.1) where A = T (T )'X(X'T X) X'T . Using equation ( A9 . 1 ) , Â° 178
PAGE 187
179 and applying Theorem l(i) (Searle, 1971, p. 55), we can write the expected value of a as E(P) = f "^E[Y'A Y] 1 2 f [traceJA^a G} + E(Y)'AqE(Y)] Â—1 2 f trace [AqO G] , (A9.2) since E(Y)'AqE(Y) = 0_'X'[T (T )'X(X'T X) X'T ] Xg = B'X'T Xg B'(X'T X)(X'T X) (X'T~X)e = as X'(T )'X = X'T X, because T is symmetric and X'T~X is unique (see proof of Theorem 4.1{ii)). Thus 2. _ .1 .Â„_Â„r^^2. E(a') = f " trace[A a^G] 2 1 f trace [AG] 2 1 a f trace [ AÂ»(T XX' ) ] By writing Aq as in Eq. ( A9 . 1 ) , we get
PAGE 188
180 ^2 2 Â—1 Â— E(a ) = a f trace [{t (T )'X(X'T X) X'T }{t XX'}] 2 _) = a f trace [t T T XX' (T ) ' X( X 'T~X)~X'T~T + (T ) 'X(X'T X) X'T XX'] 2 _] o f [trace T T trace T XX' trace (T )'X(X'T X) X' + trace (T~)'XX'], by Lemma 4.1, parts 1 and 3, and so ^2 2 Â—1 E(a ) = o f [trace T T trace (X'T X) (X'T X)], since X'(T~)'X = X'T~X. Since T~T and ( X'T~X)~( X'T~X) are idempotent, and rank(AA~)= rank (A) for any matrix A, we see that E(a^) = o^f 'Â•[rank(T) rank(X'T X)] and by Lemma 4.1 part 2 we have E(a^) = a^f "'[rank(T) rank(X)].
PAGE 189
181 Finally, since f = rank(G:X) rank(X), we can write '^ 2 2 Â—1 o E(0 ) = f [rank(G;X) rank(X)] = a . ( A9 . 3 ) The proof of Theorem 4.1(i) is now completed by justifying the equality in (A9.3) by showing that rank (T) = rank (G:X). First we write rank(T) = rank(G + XX'). Replacing G by W , we have rank(T) = rank(W' + XX') = rank(CC'), where C = (V:X) = rank(C) = rank(V:X) = rank(G:X), since the column space of G is the same as the column space of V. The column space of G is the same as the column space of V if the columns of V belong to the column space of G, and vice versa, if the columns of G belong to the column space of V. Symbolically, this is written as V <= C(G), and G c C(V). To show that V c C(G), it is sufficient to show that GG~V = v, but this is true because GG~V = (W')(W')V = V. Now, G = C(V) since by definition W = G.
PAGE 190
APPENDIX 10 PROOF OF THEOREM 4.1(ii) In Appendix 10 we prove part (ii) of Theorem 4.1, thus we show that a = f~ (y X8)'T~(Y Xg) is unique with probability one. The following theorem will be useful in our proof . Theorem vi(c) (Rao, 1973, p. 26). Let B and D be nonnull matrices. Then BA~D is invariant for any choice of A~ if and only if C(B') c C(A') and C(D) = C(A), where C( . ) denotes column space. The relationship C(B') c C(A') holds if and only if BA~A = B, and similarly C(D) <= C(A) holds if and only if AA~D = D (see " 2 Pringle and Rayner, 1971, p. 9). Since the quantity a is written as ^2 a = f 'Â•[Y'T Y Y'(T )'X(X'T X) X'T Y] , " 2 to show that a is unique with probabilty one, it suffices to show that Y'T Y Y'(T )'X(X'T X) X'T Y (AlO.l) 182
PAGE 191
183 is invariant with probability one to the choice of the generalized inverses involved. First we show that Y'T~Y is unique with probability one (w.p.l). From part 4 of Lemma 4.1, Y e C(T), w.p.l. Therefore Y' e C(T'), w.p.l, since T is symmetric, and then by Theorem vi(c) (Rao, 1973, p. 26), Y'T~Y is unique, w.p.l. Secondly we show that Y' (T~) 'X( X'T~X)~X'T~Y is unique with probability one in the following four part proof. (1) Show X'T~X is unique. From part l(i) of Lemma 4.1, TT~X = X and thus X<=C(T). Since T is symmetric, we have X'^CCT'). By Theorem vi(c) (Rao, 1973, p. 26), X'T~X is unique. (2) Show X'T~Y is unique, w.p.l. By (1) above, XcC(T') and by part 4 of Lemma 4.1, Y e C(T), w.p.l. Thus applying Theorem vi(c) (Rao, 1973, p. 26), X'T~Y is unique, w.p.l. (3) Show Y'(T~)'X is unique, w.p.l. This follows from part (2), since Y'(T~)'X is equal to the transpose of X'T""Y , which was shown in (2) to be unique, w.p.l. (4) Using (1), (2), and (3) above, the second quantity in (AlO.l) is unique, w.p.l, by Theorem vi(c) (Rao, 1973, p. 26) if (a) [Y'('r~)'X]' e C[(X"r^X)'], w.p.l, and (b) X'T~Y e C(X'T~X), w.p.l. Part (a) is true not only with probability one but always because Y' (T~) Â•X(X'T~X)~(X'T~X) = Y'(T~)'X, since by part
PAGE 192
184 3(i) of Lemma 4.1, X = X(X'T~X)~ (X'T~X). Part (b) is true not only w.p.l but always because (X'T~X)(X'T~X)~ X'T~Y = X'T~Y, by part 3(ii) of Lemma 4.1. Therefore we have shown that both Y'T~Y and Y* (T~) Â•X(X"I^X)~X"r"Y are unique with probability one, which ^ 2 allows us to conclude that a is unique with probability one .
PAGE 193
APPENDIX 11 PROOF OF THEOREM 4.1(iii) In this appendix we prove Theorem 4.1(iii), that is we show that if Y possesses an Nvariate normal distribution ^22 2 then fa /a ~ Xf ' where f = rank(G:X) rank(X). Recall that a = f Y'AqY where A = T (T )'X(X'T X) X'T . Since we have shown in Theorem 4.1(ii) that a^ is unique with probability one, the choice of the generalized inverses " 2 m the expression for a may be made arbitrarily. Thus we choose each of the generalized inverses to be the unique MoorePenrose inverse, and we denote the unique MoorePenrose inverse of a matrix B by B"^. The MoorePenrose inverse has the following four properties (see Searle, 1971, p. 16): 1. BB+B = B 2. B+BB+ = B"^ 3. (BB""") ' = BB"*" 4. (B+B)' = B+B. "2 2 The quantity fa /a can be expressed as fa^/a^ = Y'AY, (All.l) where a = (l/a^)[T"^ t"^X( X't'^X) "^X't"^] . We wish to show 185
PAGE 194
186 2 that Y'AY ~ Xf ' which can be done by making use of the following corollary. Corollary 2s. 1 (Searle^ 1971, p. 69). When X is N(u,V) whether V be singular or nonsingular, 2 X ' Ax ~ x' with degrees of freedom equal to trace (AV) and noncentrality parameter equal to (l/2)y'Au, where 2 x' denotes a noncentral chisquare random variable, if and only if (i) VAVAV = VAV ( ii ) M_ Â• AV = u Â• AVAV, and ( i i i ) U ' Ay = jj ' AVAy . In our application, the matrices A and V in Corollary 2s. 1 (Searle, 1971, p. 69) are defined as A = (l/a^)[l T'^X(X'T'^X)'*'X']t'^, and 2 V = a G The proof of Theorem 4.1(iii) follows from Corollary 2s. 1 (Searle, 1971, p. 69) if we can show that AVA = A. To show that AVA = A, we first show that AG = AT, where as we recall, AG = A(T XX'). Thus AG = AT if AXX' = 0. Using the complete expression for A, we have
PAGE 195
187 AXX' = {l/a^)[T'*" t'^X(X't''"x) "''x't'^JXX' = (1/o^)[t'^XX' t"^X(X'T"^X)'*'(X'T"^X)X' ] , and so by Lemma 4 . 1 part 3 ( i ) , AXX' = (l/a^)[T'^XX' t'^XX'] = 0. 2 2 Therefore, since AG = AT, we have AVA = a AGA = a ATA. We 2 now show that a ATA = A: ^ATA = (l/a^)[l T"^X(X'T'^X)'*'X']t'''t[I T'''X( X'T'^X) "^X' ] T"^ (l/a^)[T'^TT'^X(X'T"^X)'*'X'T'^T ] [ t'^t'^X( X ' t'*'x) '*"x'T'^] (1/o^)[t"^tt'^ t'^x(x't"^x)'*"x't"^tt'*" t'''tt"*"x(X't'*"x)"'"x*t''' + t'*"x(x't"''x)''"x't"*"tt''"x(x't'''x)'*"x't''"] (l/a^)[T"^ t'^X(X'T'*"X)''"X'T'^ T'''X( X 'T'''X) """x* t"*" + T"'"X(X'T'*"X)"''X'T'*'X(X'T'''X)'^X'T"^]
PAGE 196
188 since t+TT"^ = T+, by property 2 of the MoorePenrose inverse. Therefore, a^ATA = (l/a^)[T'^ 2T'^X( X'T'*'x) "^X't"^ + t'^XC X'T'^X) "^X' t"^] = (l/a^)[l t'^X(X'T'^X)'^X']t'^ = A. 2 Since we have verified that AVA = a ATA = A we can conclude that fa^/a^ = V ^1 ~ Xf ' ^Y Corollary 2s. 1 (Searle, 1971, p. 69). The quantity fa /a ~ Xf ^^<^ "ot Xf^' since the noncentrality parameter equals zero, which we now show. The noncentrality parameter, from Corollary 2s. 1 (Searle, 1971, p. 69) is of the form (l/2)u'Au, where in our application, y = Xg. Thus, M ' Ay = ' X Â• AX6 (l/a^)0 'X'[t'^ t'^X(X't'*"x)'^X't'^]XB = (l/a^)[0 X't''"X3 e 'X'T'^X(X't"^X)'^(X'T'^X)0] and so by Lemma 4.1 part 3(i), y'Ay_ = (l/a^)[0 Â•X'T''"X3 S'X'T'^XB] = 0.
PAGE 197
189 We now verify that the degrees of freedom are f = rank(G:X) rank(X). From Corollary 2s. 1 (Searle, 1971, p. 69) the degrees of freedom associated with Y'AY are equal 2 to f = trace (o AG), and so trace(a^AG) = trace[ I t'^X( X'T"^X) "''X' Jt'^G = trace(T"^G) trace[T"*"X( X'T'''X) "^X'T'*'g] = trace(T"*'T t'''xX') trace[T'*'X(X'T'*"x) '''x't'^'t] + trace[T'''x(X'T'^X) '^X'T"''xX' ] , since G = T XX'. It follows that trace(a AG) = trace t'*"t trace t"''xX' trace X( X't'''x) """x' t"^ + trace t'''xX' , since trace AB = trace BA for arbitrary matrices A and B, T'^TT'^ = T"^, and X( X'T'^X) "*( X Â•T'^X) = X by Lemma 4.1 part 3( ii ) . Therefore trace(o AG) = trace t"^! trace (X'T"^X) (X'T"*'X) '*' = ran]c(T) rank( X't'^X) ,
PAGE 198
190 since TT and (X'T X)(X'T X) are idempotent, and rank(AA ) = rank A, for any matrix A. Finally, by Lemma 4.1 part 2 we have trace (a AG) = rank(T) rank(X) and by the argument in the proof of Theorem 4.1(i), 2 trace (a AG) = rank(G:X) rank(X)
PAGE 199
APPENDIX 12 PROOF OF THEOREM 4.2 In this appendix we prove Theorem 4.2, thus we show that when Y~ Nj^(X3 + X B , a^G) then fa^/a^ ~ X^^^, where X = ( l/2a ^ )e Â• x' [T~ T~X(X'T~X)~X'T~]X 3 . "11 1 From the proof of Theorem 4.1, we have fa /a ~ \\,\ By Corollary 2s. 1 (Searle, 1971, p. 69) the noncentrality parameter is X = (1/2)(X0 + Y.^_^)'A{Y.^_ + X^P^), where A = (l/a^)[T T X(X'T X) X'T ]. Thus X = (l/2)[6 'X'AX3 + 3'X'AX 3 + 3'X'AX3 + ?.^X' AX 3 ] . From the proof of Theorem 4.1(iii), 3'X'AX3 = 0. We now show that 3'X'AX3 = 0: 3'X'AX3 = lo^U'^ ~ '^ X(X'T X) X'T ] X3 / a2 = 3'X'[t X3 T X(X'T X) (X'T X)3]/a2, and so by Lemma 4.1 part 3(i), 3'X'AX3 = B'X'[t X3 T X3 ] / = 0. 191 a2
PAGE 200
192 Thus we conclude that X = (l/2)e^X^AX2e2 = (l/2a )eiXi[T T X(X'T X) X'T~]x^S,. z z J 2Â—2
PAGE 201
APPENDIX 13 PROOF OF THE EQUALITY SSE, = d'v"^d + SSE A ~ (J " In this appendix we show that the check point method for testing a fitted model for lack of fit and the method in which the residual sum of squares is partitioned into a lack of fit sum of squares and a pure error sum of squares are equivalent in the sense that SSE = d'V~''"d + SSE. A ~ (J " Let us define Y, and X, as A A Xa = Y Y* (A13.1) and ^A = X X* (A13.2) Then the residual sum of squares from regressinq Y ng Y^ on X, is A A 1, SSE^ = Y. [I X^(X'X^)X]Y^. 193
PAGE 202
194 Using Eqs. (A13.1) and (A13.2) we can write SSE^ as SSE Y Y* I X X' (X'X + X*'X*) 1 X I Â• X*l Y Y* 1. = Y*'[I X*(X'X + X*'X*) X*']Y 2 Y*'X*(X'X + X*'X*) "'"X'Y + Y'[I X(X'X + X*'X*)"''"X' ] Y Y*'V~ Y* 2Y*'X*(X'X + X*'X*)~'''X' Y + Y' [I X(X'X + X*'X*) 'X'lY. (A13.3) Eq. (A13.3) is true because from Eq. (8) (Morrison, 1976, p. 69) we can write vl as V~ = [I + X*(X'X) 'X*' ] '" = I X*(X'X + X*'X*) """X * I
PAGE 203
195 We now write the quadratic from d'V^''"d as d'vJ'd = (Y* Y*)'V^(Y* Y*) Y*'v~'Y* 2Y*'V"'Y* + Y*'V"'Y* = T'^o^I* 2 Y*'Vq'X*(X'X)"^X'Y + Y'X(X'X)"'X*'Vq'X*(X'X)"^X'Y. (A13.4) The first portion in Eq. (A13.3) is equal to the first portion in Eq. (A13.4). We now show that the second portions of Eqs. (A13.3) and (A13.4) are equal. It can be verified using Eq. (8) (Morrison, 1976, p. 69) that (X'X + X*'X*) ^ = (X'X) ^ (X'X)"^X*'Vq^X*(X'X)"^. = iAA} (X'X) X*'VqX*(X'X) ( A13.5) Using Eq. (A13.5) the second portion of SSE^ in Eq. (A13.3) can be written as "v I Â• 2 Y*'X*(X'X + X*'X*) X'Y 2Y*'X*[(X'X) ^ (X'X) ^X*'V'^X*(X'X)~^]X'Y Â•2Y*'[I X*(X'X) 'X*'Vq']X*(X'X)"'X'Y. (A13. 6)
PAGE 204
196 The second portion of SSE^, given in Eq. (A13.6), is seen to equal the second portion of d'v" d in Eq. (A13.4) using the fact that I X*(X'X)~'X*'Vq'= I (Vq DVq^ 0^We now show that the third portion of the expression for SSEp^ in Eq. (A13.3) is equal to the sum of the third portion of d'v" d in Eq. (A13.4) and SSE, where SSE = Y'[l X(X'X)~"'"X' ] Y. Using the result in Eq. (A13.5), the third portion of SSE^ in Eq. (A13.3) can be written as Y' [I X(X'X + X*'X*) 'Â•X' ]Y Y'[I X{(X'X) * (X'X)~''"X*'Vq''"X*(X'X) "IXMY Y' [I X(X'X)"'X']Y + Y'X(X'X) 'Â•X*'Vq'X*(X'X)"'X'Y. Therefore, since the first two portions of SSE^ in Eq. (A13.3) are equal to the first two portions of d'v" d in Eq. (A13.4), respectively, and the third portion of SSE^ in Eq. (A13.3) is equal to the sura of the third portion of d'Vd
PAGE 205
197 in Eq. {A13.4) and Y'[I X(X'X) 'Â•X*]Y, we must have then SSE = d'V~"'"d + Y'[I X(X'X) ''"X'jY A " U " ^ Â— = d'V~"'"d + SSE.
PAGE 206
REFERENCES Atkinson, A. C. (1972). Planning experiments to detect inadequate regression models. Biometrika , Vol. 59, pp. 275293. Barr, A. J., J. H. Goodnight, and J. P. Sail (1979). SAS Users Guide, 1979 Edition . SAS Institute, Inc., Raleigh, North Carolina. Becker, N.G. (1968). Models for the response of a mixture. Journal of the Royal Statistical Society , _B, Vol. 30, pp. 349358. ~ Becker, N.G. (1978). Models and designs for experiments with mixtures. Australian Journal of Statistics , Vol. 20, pp. 195208. Box, G. E. P., and N. R. Draper (1975). Robust designs. Biometrika , Vol. 62, pp. 347352. Claringbold, P. J. (1955). Use of the simplex design in the study of the joint action of related hormones. Biometrics , Vol. 11, pp. 174185. Cochran, W. G., and G. M. Cox (1957). Experimental Designs , 2nd Ed. John Wiley and Sons, New York. Cornell, J. A. (1973). Experiments with mixtures: A review. Technometrics , Vol. 15, pp. 437455. Cornell, J. A. (1981). Experiments with Mixtures; Designs, Models, and the Analysis of Mixture Data . John Wiley and Sons, New York. Cornell, J. A., and I. J. Good (1970). The mixture problem for categorized components. Journal of the American Statistical Association , Vol. 65, pp. 339355. Daniel, C, and F. S. Wood (1971). Fitting Equations to Data; Computer Analysis of Multif actor Data for Scientists and Engineers . John Wiley and Sons, ^Nev/ York. Draper N. R. , and A. M. Herzberg (1971). On lack of fit. Technometrics , Vol. 13, pp. 231241. 198
PAGE 207
199 Draper, N. R. , and W. E. Lawrence (1965a). Mixture designs for three factors. Journal of the Royal Statistical Society , B_, Vol. 27, pp. 450465. Draper, N. R., and W. E. Lawrence (1965b). Mixture designs for four factors. Journal of the Royal Statistical Society , B_, Vol. 27, pp. 473478. Draper, N. R. , and H. Smith (1981). Applied Regression Analysis , 2nd Ed. John Wiley and Sons, New York. Draper, N. R. , and R. C. St. John (1977). A mixtures model with inverse terms. Technometrics , Vol. 19, pp. 3746. Gorman, J. W. , and J. E. Hinman (1962). Simplexlattice designs for multicomponent systems. Technometrics , Vol. 4, pp. 463487. Graybill, F. A. (1969). Introduction to Matrices with Applications in Statistics . Wadsworth Publishing Co., Inc., Belmont, California. Graybill, F. A. (1976). Theory and Application of the Linear Model . Duxbury Press, North Scituate, Massachusetts . Green, J. R. (1971). Testing departure from a regression, without using replicaton. Technometrics , Vol. 13, pp. 609615. Johnson, N. L. , and S. Kotz (1970). Distributions in Statistics: Continuous Univariate Distributions 2 . Houghton Mifflin, Boston. Kenworthy, 0. 0. (1963). Factorial experiments with mixtures using ratios. Industrial Quality Control , Vol. 19, pp. 2426. Khuri , A. I., and J. A. Cornell (1981). Lack of fit revisited. Technical Report No. 167, Department of Statistics, University of Florida, Gainesville, Florida. Kurotori, I. S. (1966). Experiments with mixtures of components having lower bounds. Industrial Quality Control , Vol. 22, pp. 592596. McLean, R. A., and V. L. Anderson (1966). Extreme vertices design of mixture experiments. Technometrics , Vol. 8, pp. 447454. Morrison, D. F. (1976). Multivariate Statistical Methods , 2nd Ed. McGrawHill, New York.
PAGE 208
200 Mudholkar, G. S., Y. P. Chaubey, and C. Lin (1976). Approximations for the doubly noncentralF distribution. Communications in Statistics ^ A, Vol. 5, pp. 4963. ~ Myers, R. H. (1971). Response Surface Methodology . Allyn and Bacon, Inc., Boston. Piepel, G. F. (1981). Measuring component effects in constrained mixture experiments. Unpublished manuscript presented at 1981 Joint Statistical Meetings of the American Statistical Association and the Biometric Societies, Detroit, Michigan, August 1981. Price, W. L. (1977). A controlled random search procedure for global optimisation. The Computer Journal , Vol. 20, pp. 367370. Pringle, R. M. , and A. A. Rayner (1971). Generalized Inverse Matrices with Applications to Statistics . Hafner Publishing Company, New York. Rao, C. R. (1971). Unified theory of linear estimation. Sankhya , _A, Vol. 33, pp. 371394. Rao, C. R. (1972). Unified theory of least squares. Communications in Statistics , Vol. 1, pp. 18. Rao, C. R. (1973). Linear Statistical Inference and its Applications , 2nd Ed. John Wiley and Sons, New York. Rao, C. R., and S. K. Mitra (1971). Generalized Inverse of Matrices and its Applications . John Wiley and Sons, New York. Scheffe, H. (1958). Experiments with mixtures. Journal of the Royal Statistical Society , B_, Vol. 20, pp. 344360. Scheffe, H. (1959). The Analysis of Variance . John Wiley and Sons, New York. Scheffe, H. (1963). The simplexcentroid design for experiments with mixtures. Journal of the Royal Statistical Society , _B, Vol. 25, pp. 235263. Searle, S. R. (1971). Linear Models . John Wiley and Sons, New York . Shillington, E. R. (1979). Testing lack of fit in regression without replication. The Canadian Journal of Statistics, Vol. 7, pp. 137146.
PAGE 209
201 Snee, R. D. (1971). Design and analysis of mixture experiments. Journal of Qu ality Technoloqy, Vol. 3, pp. 159169. " ^^ Snee, R. D. (1973). Techniques for the analysis of mixture data. Technonetrics , Vol. 15, pp. 517528. Snee, R. D. (1977). Validation of regression models: methods and examples. Technome tries. Vol. 19, pp. 415428. Thompson, W. 0., and R. K. Myers (1968). Response surface designs for experiments with mixtures. Technometrics, Vol. 10, pp. 739756. " Wishart, D. (1975). Clustan IC User Manual . David IVishart, University College, London. Zyskind, G. (1967). On canonical forms, nonnegative covariance matrices and best and simple least squares linear estimators in linear models. Annals of Mathematical Statistics , Vol. 38, pp. 10921109.
PAGE 210
BIOGRAPHICAL SKETCH John Thomas Shelton was born on March 30, 1952, in Jacksonville, Florida, where he resided until graduating from Englewood High School in June, 1970. He then entered the University of Florida where he received a Bachelor of Science degree in mathematics in June, 1974. John began graduate study at Virginia Polytechnic Institute and State University in Blacksburg, Virginia, in September, 1975, and there received a Master of Science degree in statistics in the summer of 1976. After two years as a Research Associate at Auburn University in Auburn, Alabama, he returned to the University of Florida in September, 1978, where he has since been pursuing a doctoral degree in statistics. While a graduate student at the University of Florida, John has worked as a Graduate Assistant performing statistical consulting duties in the School of Forest Resources and Conservation. 202
PAGE 211
I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a disseration for the degree o^f Doctor of Philosophy. T /^k, W' Andre' I. Khun, Chairman Assistant Professor of Statistics I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a disseration for the degree of Doctor of Philosophy.
PAGE 212
?^
TESTING LACK OF FIT IN A MIXTURE MODEL
BY
JOHN THOMAS SHELTON
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1982
To Nydra
and
My Parents
ACKNOWLEDGEMENTS
I would like to express my deepest appreciation to Drs.
Andre' Khuri and John Cornell for suggesting this topic to
me and for providing constant guidance and assistance. They
have made this project not only a rewarding educational
experience but an enjoyable one as well. A special word of
thanks goes to Mrs. Carol Rozear for her diligence in
transforming my handwritten draft into an expertly typed
manuscript.
iii
TABLE OF CONTENTS
page
ACKNOWLEDGEMENTS iii
ABSTRACT vii
CHAPTER
ONE INTRODUCTION 1
1.1 The Response Surface Problem 1
1.2 The Mixture Problem 5
1.2.1 Mixture Models 6
1.2.2 Experimental Designs for Mixtures.. 12
1.3 The Purpose of this Research:
Investigation of Procedures for Testing
a Model Fitted in a Mixture System for
Lack of Fit 17
TWO LITERATURE REVIEWâ€”TESTING FOR LACK OF FIT 19
2.1 Introduction 19
2.2 Partitioning the Residual Sum of Squares.. 21
2.3 Testing for Lack of Fit Without
Replicated Observationsâ€”Near Neighbor
Procedures 26
2.4 Testing for Lack of Fit with Check Points. 33
THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL 40
3.1 Introduction 40
3.2 Testing for Lack of Fit in the Presence
of an External Estimate of Experimental
Error Variation 41
3.2.1 The Test Statistic 41
3.2.2 The Testing Procedure and an
Expression for the Power of
The Test 45
3.2.3 A Method for Locating Optimal
Check Points 47
3.3 Testing for Lack of Fit When MSE Is
Used to Estimate Experimental Error
Variation 51
3.3.1 The Test Statistic 51
3.3.2 The Rejection Region and its
Relation to the Power of the Test.. 53
iv
3.3.3 A Method for Locating Optimal
Check Points 56
3.3.4 Determining Whether the Test Is
Upper Tailed or Lower Tailed 58
3.4 Examples 67
3.4.1 Theoretical Examples 67
3.4.2 Numerical Examples 83
3.5 Discussion 95
FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR
TESTING LACK OF FIT 99
4.1 Introduction 99
4.2 Notation 101
4.3 Shill ington ' s Procedure 106
4.3.1 Development of MSEB 109
4.3.2 Development of MSEW 110
4.4 Development of SSEW(weighted) 112
4.5 Equivalence of SSEW and SSEw(weighted)....116
4.6 The Test Statistic 118
4.7 The Testing Procedure and its Power 122
4.8 Selection of Near Neighbor Groupings 125
4.8.1 Example 1â€”Stack Loss Data 129
4.8.2 Example 2â€”Glass Leaching Data 134
4.9 Discussion 142
FIVE CONCLUSIONS AND RECOMMENDATIONS 145
APPENDICES
1
INFLUENCE OF \ ON
P{ F" . , > F }
Vj ,V2 ;Ai (Ã2 a;v^,V2
156
2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR
GLOBAL OPTIMIZATION 159
3 STATISTICAL INDEPENDENCE OF d'Vâ€œ1d/o2
AND SSE/a2 164
4 THEOREM 3.1 168
5 THEOREM 3.2 169
6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL
F DISTRIBUTION 171
7 EQUIVALENCE OF SSEg AND SSL0F WHEN
REPLICATES REPLACE NEAR NEIGHBOR
OBSERVATIONS 17 2
v
8 LEMMA 4.1 175
9 PROOF OF THEOREM 4.1(i) 178
10 PROOF OF THEOREM 4.1(11) 182
11 PROOF OF THEOREM 4.1(iii) 185
12 PROOF OF THEOREM 4.2 191
13 PROOF OF THE EQUALITY SSEA = d'v"1! + SSE 193
REFERENCES 198
BIOGRAPHICAL SKETCH '. 202
vi
Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in
Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy
TESTING LACK OF FIT IN A MIXTURE MODEL
By
John Thomas Shelton
May 1982
Chairman: Andre' I. Khuri
Cochairman: John A. Cornell
Major Department: Statistics
A common problem in modeling the response surface in
most systems, and in particular in a mixture system, is that
of detecting lack of fit, or inadequancy, of a fitted model
of the form E(Y) = XÂ§^ in comparison to a model of the form
E(Y) = xe1+ X^g2 postulated as the true model. One method
for detecting lack of fit involves comparing the value of
the response observed at certain locations in the factor
space, called "check points," with the value of the response
that the fitted model predicts at these same check points.
The observations at the check points are used only for
testing lack of fit and are not used in fitting the model.
It is shown that under the usual assumptions of
independent and normally distributed errors, the lack of fit
test statistic which uses the data at the check points is an
Vll
F statistic. When no lack of fit is present the statistic
possesses a central F distribution, but in general, in the
presence of lack of fit, the statistic possesses a doubly
noncentral F distribution. The power of this F test depends
on the location of the check points in the factor space
through its noncentrality parameters. A method of selecting
check points that maximize the power of the test for lack of
fit through their influence on the numerator noncentrality
parameter is developed.
A second method for detecting lack of fit relies on
replicated response observations. The residual sum of
squares from the fitted model is partitioned into a pure
error variation component and into a lack of fit variation
component. Lack of fit is detected if the lack of fit
variation is large in comparison to the pure error
variation. This method can be generalized when "near
neighbor" observations must be substituted for replicates.
In this case, the test statistic (assuming independent and
normally distributed errors) has a central F distribution
when the fitted model is adequate and a doubly noncentral F
distribution under lack of fit. The arrangement of near
neighbors is seen to affect the testing procedure and its
viii
power.
CHAPTER ONE
INTRODUCTION
1.1 The Response Surface Problem
A mixture problem is a special type of a response
surface problem. First we shall define the general response
surface problem and indicate the basic objectives sought in
its analysis, and follow this development with a discussion
of the mixture problem.
In the general response surface problem, we are interÂ¬
ested in studying the relationship between an observable
response, Y, and a set of q independent variables or
factors, x^, x2/ . .., Xq, whose levels are assumed conÂ¬
trolled by the experimenter. The independent variables are
quantitative and continuous. We express this relationship
in terms of a continuous response function, 4Â» r as
Y = d> (x
u y '
ul' u2'
. ., x ) + e
' uq' u
where Yu is the uth of N observations of the response colÂ¬
lected in an experiment, and xuÂ¿ represents the uth level of
the ith independent variable, u = 1, 2, ..., N, i = 1, 2,
..., q. The exact functional relationship, 4, is unknown.
The term eu is the experimental error of the uth
1
2
observation. It is assumed that E(eu) = 0, E(eueui) = 0,
for u * u', and E(eÂ¿j) = a2, for u = 1, 2, N.
As the form of is unknown and may be quite complex, a
low order polynomial (usually first or second order) in the
independent variables x^, X2/ ..., Xg is generally used to
approximate . This may be justified by noting that such
polynomials constitute low order terms of a Taylor series
expansion of 41 about the point xp = x2 = â€¢â€¢â€¢ = xq = 0,
(Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point
out that these low order polynomials may give a poor approxÂ¬
imation to
region, and thus should not be used for this purpose.
A linear response surface model may be written in
matrix notation as
Y = X0 + Â£ (1.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, ft is a pxl vector of
unknown parameters (regression coefficients), and g. is the
Nxl vector of random errors. When the model is a first or a
second degree polynomial, the columns of X correspond to the
first or second degree powers of the independent variables
x^, X2/ . .., Xg, or their cross products. If the model
contains a constant term, 3q, the first column of X will
correspond to this term, and will consist of N ones. Since
E(e) = 0, an alternative representation for the response
3
surface model of (1.1) is
E(Y) = XB .
Once the form
of the model that will
be
used
to
approx
imate (X2, . .
â€¢, Xg)
is chosen, the next
step
is
to
estimate the regression
coefficients, a.,
and
then
use
the
estimated model to
make
inferences about
the
true
response
function, <> . The estimation of the elements of a. is usually
accomplished by ordinary least squares techniques. For the
purpose of testing hypotheses concerning the regression
coefficients, B., it is assumed that Â£. has a normal distribu
2
tion, that is, e ~ NN(0, a IN).
Perhaps the most common objective in the exploration of
a response system is the determination of its optimum
operating conditions. By this we mean that it is desired to
find the settings of x^, X21 . .., Xg that optimize , which
in some applications may be interpreted as maximizing ,
while in other applications a minimum value of may be of
interest. It is also often desirable to determine the beÂ¬
havior of the response function in the neighborhood of the
optimum. For second order response models, such an investiÂ¬
gation can be carried out by performing a canonical analysis
of the second order surface as discussed in Myers (1971).
For simple systems having only one or two independent
variables, the response surface may be explored by just
plotting the fitted response values against values taken by
4
the independent variables. If q = 1, implying only one
independent variable, say x^, then a twodimensional plot of
the fitted response against x^ may be used to locate the
optimum, as well as to investigate the response behavior in
other parts of the experimental range of x^. If q = 2, and
the two independent variables are x^ and x2> then a plot of
the contours of constant response over the region specified
by the ranges of the values for x^ and X2 can be used to
describe the response surface.
The properties that the fitted model possesses in terms
of its ability to represent the true surface, , depend on
the settings of Xj_, X2* ..., Xq at which values of Y are
observed. Thus the experimental design is of great imporÂ¬
tance. Much work has been done on the construction of
designs that are optimal with respect to one criterion or
another involving the fitted response and/or the true unfitÂ¬
ted model. Box and Draper (1975) list fourteen criteria to
consider when choosing a design for investigating response
surfaces. Myers (1971) gives several designs for fitting
first and second order polynomial models. A discussion of
specific design considerations will not be attempted here,
as such a discussion is not the focus of this dissertation,
and would necessarily be lengthy.
The initial steps in the analysis of a response system
may be described as follows: First an attempt is made to
approximate the true response function, (xj_, X21 â€¢â€¢â€¢* Xq) ,
usually with a low order polynomial in xj_, x2/ â€¢â€¢â€¢/ Xq.
5
After the form of the model has been chosen, then comes the
selection of an appropriate experimental design, which
specifies the settings of the independent variables at which
observed values of the response will be collected. The
observed values of the response are used in estimating the
regression coefficients in the model, using, in general,
ordinary least squares. After a test for "goodness of fit"
of the model verifies the fitted model is adequate, the
fitted model is used in determining optimum operating condiÂ¬
tions for the response system.
1.2 The Mixture Problem
A mixture system is a response system in which the
response depends only on the relative proportions of the
components or ingredients present in a mixture, and not on
the total amount of the mixture. For example, the response
might be the octane rating of a blend of gasolines where the
rating is a function only of the relative percentages of the
gasoline types present in the blend. The proportion of each
ingredient in the mixture, denoted by xÂ¿, must lie between
zero and unity, i = 1, 2, ..., q. The sum of the proporÂ¬
tions of all the components will equal unity, that is,
q
0 < x. < 1, i = 1,2,...,q, Â£ x. = 1. (1.2)
i=l
The factor space containing the q components is represented
by a (q  1)dimensional simplex. For q = 2 components, the
factor space is a straight line, whereas for q = 3
6
components, the factor space is an equilateral triangle, and
for q = 4 components, the factor space is represented by a
regular tetrahedron.
The objectives in the analysis of a mixture response
system are, in general, the same as in any response surface
exploration. That is, one seeks to approximate the surface
with a model equation by fitting an equation to observations
taken at preselected combinations of the mixture comÂ¬
ponents. Another objective is to determine the roles played
by the individual components. We shall not concern ourÂ¬
selves with this but rather concentrate on the empirical
model fit. Once the model equation is deemed adequate an
attempt is made to determine which of the component combinaÂ¬
tions yield the optimal response. The models used to repreÂ¬
sent the response in a mixture system are in most cases
different in form from the standard polynomial models. The
first type of model form that we discuss is the canonical
polynomial suggested by ScheffÃ©.
1.2.1 Mixture Models
Scheffe (1958) introduced a canonical form of the polyÂ¬
nomial model for representing the response in a mixture
system. These canonical polynomial models are derived from
the standard polynomials using the restrictions on the xÂ¿
shown in (1.2). With q = 2 mixture components, for example,
the standard second order polynomial model is of the form
a 0 + alXl +a2X2 + a12XlX2 + allXl + a 22X 2
E(Y)
(1.3)
7
Restrictions (1.2) imply that ciq = ag(xL + x2),
X1 = xl(1 â€œ x 2), and x2 = x2(1 " xl)f thus (1*3) can be
written in the canonical form
E(Y) = 0x + 02x2 + 612x1x2'
where 6^ <Â»â€ž + ^ + 2 + and S12= Â«12
 ct^  ot 22 â€¢ There is no constant term in the above canoniÂ¬
cal form and the pure quadratic terms in equation (1.3) have
been absorbed in the xÂ¿xj terms.
The general form of the canonical polynomial of degree
d in q mixture components can be written as
E(Y) = Z 3,x., for d = 1,
i = 1
E (Y) = Z 3x. + Z Z g.x.x. , for d = 2, and
.,11 , . .. IT 1 i
x=l l
q q q
E (Y) = Z g.x. + Z Z S.x.x. + Z Z 6..x.x.(x.
i1 1 1 Ki
 V
+ ZZZ g.., x.x.x. , for d = 3.
i â€¢ , â€¢ ilk ink
l
(1.4)
The fourth degree canonical polynomial in q components is
given in Cornell (1981, p. 64). The general canonical polyÂ¬
nomial of degree d > 4 in q components does not explicitly
*
appear in the literature, but is mentioned in Scheffe
(1958). If terms of the form 6ijXiXj(xÂ¿  Xj) are removed
from the full cubic model (1.4), then the remaining terms
8
make up what is referred to as the special cubic model. For
example, for q = 3 components, the special cubic model is
E(Y) = 3^ + 8 2X2 + P3X3 + 012X1X2 + ^13X1X3
+ + e,â€ž,x,xâ€žx,
23 2 3 123 123
Scheffe's canonical polynomial models are used for
approximating the response surface in many mixture systems.
Their popularity stems from the ease in interpreting the
coefficient estimates, especially when the models are fitted
to data collected at the points of the associated designs
(see Section 1.2.2). However, other models have been introÂ¬
duced which seem to better represent the response when the
components have strictly additive blending effects. We
present some of them now.
Becker (1968) introduced three forms of homogeneous
models of degree one which he recommends instead of the
polynomial models when one or more of the mixture components
have an additive effect or when one or more components are
inert. A function f(x, y, ..., z) is said to be homogeneous
of degree n when f(tx, ty, ..., tz) = tnf(x, y, ..., z), for
every positive value of t and (x, y, ..., z) * (0, 0, ...,
0). These models, which Becker refers to as models HI, H2,
and H3, are of the form
9
q q
HI: E(Y) = Z 3.x. + El 0..min(x., x.) + ...
..li . Â» . ii i 1
1=1 1<. l <3 J J
+ B12...qmin(Xr X2 V '
q q 21
H2: E(Y) = E 3x. + Z Z 0..x.x./(x. + x.) + ...
11 ..... ^11 l 1/v l 1
1<1<3 J J J
i =1
+ 0
12 . qxix2â€œ *xq/(xi + x2 + **' + xq>q
q q 1/2
H3: E (Y) = Z 0,x, + Z Z 3 (x x ^ + ...
l
i=i 1 1 i*Ã/Ã
Bi2...q(XlX2"Xq>
i/q
Each term in the H2 model is defined to be zero when the
denominator of the term is zero.
Draper and St. John (1977) suggest a model which inÂ¬
cludes inverse terms, l/xÂ¿, in addition to terms in the
*
Scheffe polynomials. Such a term is used to model an
extreme change in the response as xÂ¿ approaches zero. The
experimental region of interest is assumed to include the
region near the zero boundary (xÂ¿ = 0), but does not include
the boundary itself. One example of this type of model is
the Scheffe linear polynomial model with inverse terms
q q
= Z 0 . x . + Z 0.x.
i=l 1 1 i=l _1 1
1
E(Y)
10
Another model form that is useful in the study of the
response in a mixture system is the model containing ratios
of the component proportions. A term such as Xj_/xj measures
the relationship of Xj_ to Xj rather than the percentage of
each in the blends. Snee (1973) points out that the ratio
model presents a useful alternative to the ScheffÃ© and
Becker models in that the ratio model describes a different
type of curvature. He notes that the curvilinear terms for
the Scheffe and Becker models, when plotted as a function of
xÂ¿, are symmetric functions about xÂ¿ = 1/2, whereas the
ratio term x^/xj is a monotone function when plotted against
xi*
The terms in the ratio models may also contain sums of
the components. For example, with q = 3 components, we
might express the second order model
E(Y)
q1
B0 +
1=1
q1
+ Z Z
1< i < j
B . . z . z .
13 1 3
q1
+ Z
i =1
2
i
(note the constant term) where z^ and Z2 are defined as
Zj_ = x1/(x2 + X3) and z2 = x2/x3. Some terms will be undeÂ¬
fined if points from the boundary of the experimental simÂ¬
plex are included in the design, for example, if x3 = 0,
then z2 = x2/x3 is not defined. Snee (1973) suggests adding
a small positive quantity, c, to each xÂ¿ in this case.
This, of course, will not be of concern if the experimental
region is entirely inside of the simplex.
11
When one or more of the components is inactive, Becker
(1978) suggests that a ratio model that is homogeneous of
degree zero in the remaining components is appropriate. In
three components, such a model is of the form
E(Y) = 0O + e1x1/(x1 + x 2) + e2x2/(x2 + x3)
3
+ 3 x./(x, + x ) + E E 0â€¢ â€¢hâ€¢ . (x . , x.)
3 J 1 3 l
+ 3123h123(Xlr X 2â€™ X3)' (1*5)
where h^j and h^23 are specified functions that are homogeÂ¬
neous of degree zero. The function hjl23 is intended to
represent the joint effect of all three components simultaÂ¬
neously. If in fitting a model of the form (1.5) we deterÂ¬
mine the model should be
E(Y) = 0O + + x2) + B 12h i2 1' x2)
then component three is said to be inactive and is removed
from further consideration. The model of equation (1.5) may
produce an extreme value near the vertices of the simplex
factor space when there are no inactive components. In this
case it is suggested that a model of the form (1.5) be used
only when the proportions are restricted so that no two of
the x^ are simultaneously very close to zero. Becker notes
that other authors who have suggested ratio models have also
12
used them primarily over a subregion inside the simplex
factor space. Apparently this is where they are most approÂ¬
priate .
1.2.2 Experimental Designs for Mixtures
As in the general response surface problem, one of the
major concerns in exploring a mixture system is that of
choosing the experimental design for collecting observed
values of the response that will be used in fitting the
*
model. Scheffe (1958) proposed the {q,m} simplex lattice
designs for exploring the entire qcomponent simplex factor
space. In these designs, the proportions used for each
component have the m + 1 values spaced equally from zero to
one, xÂ¿ = 0, 1/m, 2/m, ..., (m  l)/m, 1, and all possible
mixtures with these proportions for each component are
used. The number of design points in the {q,m} simplex
lattice design is (m + ^ ~ The main appeal of these
designs is that they provide a uniform coverage of the fac
*
tor space. Another feature, which Scheffe (1958) demonÂ¬
strates, is that the parameters of the canonical polynomial
of degree m in q components are expressible as simple linear
combinations of the true response values at the design
points of the {q,m} simplex lattice. The {3,2} simplex
lattice, which consists of six design points, is represented
on the two dimensional simplex in Figure 1 along with the
triangular coordinates (xj_, X2# X3).
Scheffe (1963) also developed the simplex centroid
designs consisting of 2^  1 points, where the only mixtures
13
considered are the ones in which the components present
appear in equal proportions. That is, in a qcomponent
simplex centroid design, the design points correspond to the
q
q permutations of (1, 0, 0, 0), the (2) permutations of
q
(1/2, 1/2, 0, ..., 0), the (3) permutations of (1/3, 1/3,
1/3, 0, . .., 0), . .., and the point (1/q, 1/q, ..., 1/q).
This design alleviates the problem inherent in the {q,m}
simplex lattice designs of observing responses at mixtures
containing at most m components. To give an example, the
q = 3 simplex centroid design is made up of 2^  1 = 7
design points, and is equivalent to the {3,2} simplex
lattice design augmented by the center point (x^, x2, X3) =
(1/3, 1/3, 1/3). This design is represented in Figure 2.
ScheffÃ© (1963) mentions that the number of parameters
in the polynomial model of the form
E(Y) =
q
E
q
E E
g.x. +
i =1 1 1 l
q
6..X.X. + E E E 6 â€¢ x. x . x.
^ 1 3 i
+
+ B12...qXlX2
X
q
(1.6)
is 2*3  1 and therefore these models have a special relaÂ¬
tionship with the simplex centroid design in q components.
This relationship is that the number of terms in the model
equals the number of points in the design and as a result
the parameters in model (1.6) are expressible as simple
functions of the responses at the 2^1 points of the simÂ¬
plex centroid design. Polynomial models of the form (1.6)
14
V
(0,1 ,
( 0,0,1 )
Figure 1. The {3/2} simplex lattice design.
x.r i
(0,0,1)
Figure 2. The q = 3 simplex centroid design.
15
therefore are natural models to fit using the simplex cenÂ¬
troid design.
Ratio models may be desirable when the interest in one
or more of the mixture components is in terms of their relaÂ¬
tionship to one another, rather than in terms of their perÂ¬
centages in blends. Kenworthy (1963) proposed factorial
arrangements for ratio variables. An example of the use of
ratios is the following three component system in which the
mixture components are constrained by the upper and lower
bounds:
.2 < *2 < x2 < *4/ .3 < < *5* (1*7)
The ratio variables of interest are zÂ± = x2/xl anc^
z2 = x2/x3' anc^ desired to fit either a first or a
second order polynomial model in and Z2* For such a
problem, we can define a 22 and a 32 factorial design that
can be used for fitting the first and second order polyÂ¬
nomial models, respectively, by taking as design points the
intersection of rays passing from two of the three vertices
of the twodimensional simplex through the region of
O
interest defined by the constraints (1.7). Kenworthy's 2
factorial design is shown in Figure 3.
Becker (1978) uses rays extending from one or more
vertices of the simplex factor space to the opposite boundÂ¬
aries in developing "radial designs." These designs are
suggested for detecting the presence of an inactive
16
Figure 3. Kenworthy's 2^ factorial design.
component or in another case a component which has an addiÂ¬
tive effect, when models containing ratio terms that are
homogeneous of degree zero are fitted.
McLean and Anderson (1966) suggest an algorithm for
locating the vertices of a restricted region of the simplex
factor space which is defined by the placing of upper and
lower bounds on the mixture component proportions. The
vertices of the factor space and convex combinations of the
vertices are the candidates for design points for fitting a
first or second degree polynomial model in the mixture comÂ¬
ponents. One drawback of the "extreme vertices" design is
that the design points are not uniformly distributed over
the factor space resulting in an imbalance in the variances
of Y(x), see Cornell (1973).
17
Another method that has been suggested for studying the
response over a subregion of the simplex mixture space is
to transform the q mixture components into q  1 independent
variables. Transforming to an independent variable system
was first suggested by Claringbold (1955) and later proposed
by Draper and Lawrence (1965a, 1965b) and Thompson and Myers
(1968). Standard response surface polynomial models in the
transformed variables can be fitted to data values collected
on standard designs and a design criterion such as the averÂ¬
age mean square error of the response can be employed when
distinguishing between designs. Thompson and Myers (1968)
suggest the use of rotatable designs (see also Cornell and
Good, 1970).
Designs other than rotatable designs, such as multiple
lattices and symmetricsimplex designs, to name a few, have
been suggested in the literature for fitting models to a
mixture system which may be appropriate depending on parÂ¬
ticular experimental situations. However, as the intent
here is not to give an exhaustive list but only a sampling
of available designs, we shall not discuss designs further
but instead state the purpose of this work.
1.3 The Purpose of this Research:
Investigation of Procedures tor Testing a Model
Fitted in A Mixture System for Lack of Fit
A common problem in modeling the response in a mixture
system is that of detecting lack of fit, or inadequacy, of a
fitted model of the form E(Y) = XQ ^ when the true model is
of the form E(Y) = Xg ^ + X22* T^e stat^stliterature
18
suggests several procedures for testing lack of fit, which
will be described in Chapter Two. In general, the authors
of these procedures are not specific in stating hypotheses
to be tested and do not adequately discuss the power of
their procedures.
The major purpose of this research is to investigate
the power of two of the testing procedures appearing in the
literature in detecting the inadequacy of a fitted model
when the general form of the true model is specified. Our
findings for a "check points" lack of fit testing procedure
are presented in Chapter Three while Chapter Four contains
findings for a "near neighbor" lack of fit testing proceÂ¬
dure. For both procedures, explicit formulas for the power
of the test are given in terms of cumulative probabilities
of either the noncentral F or doubly noncentral F distribuÂ¬
tion, which are derived by assuming that the response obserÂ¬
vations are independent and normally distributed. AdditionÂ¬
ally, we propose methods for maximizing the power of the
testing procedures. In the final chapter, we make some
concluding comments concerning both of these procedures.
CHAPTER TWO
LITERATURE REVIEWâ€”TESTING FOR LACK OF FIT
2.1 Introduction
Let us return to the general response surface problem
and assume the true response is to be approximated by
fitting a model of the form
E(Y) = X0 (2.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, and g.]_ is a px 1 vector of
unknown regression coefficients. We wish to consider the
situation in which the true model contains terms in addition
to those in the fitted model. Then the true model has the
form
E(Y) = X0x + X202 (2.2)
where X2 is an Nxp2 matrix of known constants, and Â£L2 is a
P2xl vector of unknown regression coefficients. We assume
that the vector Y has the normal distribution with
var(Y) = a2IN .
It is desirable to determine the suitability of the
fitted model given by Eq. (2.1) when in reality the true
model is of the form given by Eq. (2.2). The process of
19
20
making this determination is referred to as testing for lack
of fit of the fitted model.
There are three general approaches to testing for lack
of fit. The first approach requires that there be replicate
observations of the response at one or more design points,
and involves partitioning the residual sum of squares from
the fitted model into a sum of squares due to lack of fit
and a sum of squares due to pure error. A large value for
the ratio of the mean square due to lack of fit to the mean
square due to pure error provides evidence for lack of fit.
If replicate observations are not available then the
above approach to testing for lack of fit cannot be used.
Green (1971), Daniel and Wood (1971), and Shillington (1979)
have proposed alternative methods that are applicable in
this case. Their approach is to group values of the
response which are observed at similar settings of the
independent variables and to call these grouped values
"pseudoreplicates" or "near neighbor observations." They
then treat these pseudoreplicates as they would treat true
replicates to form statistics for lack of fit testing,
although arriving at their respective statistics through
different approaches.
A third approach to testing for lack of fit involves
the use of "check points." In this method a model of the
form (2.1) is fitted to data at the design points and
additional observations are collected at other points in the
experimental region. The additional points other than the
21
design points are called check points, and the data at these
check points are not used in fitting the model. Lack of fit
is tested by comparing the values of the response observed
at the check points to the values of the response which the
fitted model predicts at these same check points.
We now discuss the first method mentioned above of
testing for lack of fit which involves partitioning the
residual sum of squares.
2.2 Partitioning the Residual Sum of Squares
The method for testing lack of fit which makes use of a
partitioning of the residual sum of squares from the fitted
model requires there be replicate observations of the
response at some of the design points (Draper and Smith,
1981, p. 120). When a model of the form (2.1) is fitted,
the residual sum of squares is defined as
n.
n i â€ž
SSE = Z Z (Y..Y.)
i=l j=l ^
= Y'(IN  X(X'X)1X')Y
where n is the number of distinct design points, nÂ¡_ > 1 is
the number of replicate observations at the ith design
point, YÂ¿j is the jth observed value of the response at the
ith design point, Y^ is the value which the model of the
form in Eq. (2.1), fitted by ordinary least squares
techniques, predicts for the response at the ith design
n
point, and N = z n. . Using the replicated observations
i =1 1
22
only, a pure error sura of squares can be calculated as
SSE
pure
E E
i=l j=l
Y .
1
2
t
where . is the average of the values of the response
observed at the ith design point. The sura of squares due to
lack of fit can be obtained by taking the difference
SSrnp = SSE
LOF
SSE
pure
This partitioning of the residual sum of squares is
displayed in the analysis of variance table in Table 1.
Table 1. Analysis of Varianceâ€”
Partitioning the Residual Sura of Squares.
Source
Sum
Degrees
Mean
of Variation
of Squares
of Freedom
Square
Regression
bjX'Y  (l'Y)2/N
p  1
Residual
SSE
N  p
MSE
Pure Error
SSEpure
N  n
MSEpure
Lack of Fit
SSLOF
n  p
msLOF
Total(corrected)
Y'Y  (l'Y)2/N
N  1
b^ represents the ordinary least squares estimator of (3 in
model (2.1), b^ = (X'X)1X'Y, and 1 is an Nxl vector of
ones.
23
To test the hypothesis of zero lack of fit, that is
Hq: lack of fit = 0 or E(X) = Xfcj., an F statistic is formed
F
MS
LOF
MSE
pure
(2.3)
which possesses a central F distribution if the true model
is of the form (2.1), but has a noncentral F distribution if
the true model is of the form (2.2). In other words
F
F
np,Nn
under HQ: E(Y) = Xg^ , and
F ~ F'
np,Nn;X 2
under H^: E(Y) = Xg^ + X2Â£2 â€™ where ^2 *s tlie noricentrali
parameter x2 = g^(X2~XA)'(X2XA)g2/2a2, and A = (X,X)_1X,X2*
Under Ha, E(MSLQF) = o2 + g^(X2  XA)'(X2  XA)g2/(np) and
E(MSEpure) = a^ (Draper and Smith, 1981, p. 120), hence Hq
is rejected in favor of Ha if the value of F in (2.3)
exceeds the upper 100a percentage point of the central F
distribution, Fa;np,Nn* when Hq is rejected, we conclude
that a significant lack of fit is present.
Draper and Herzberg (1971) demonstrated that the lack
of fit sum of squares can be partitioned into two
statistically independent sums of squares, SSL^ and SSL2,
when there are replicate observations at the center of the
24
response surface design and when the basic design without
center points is nonsingular. If the true model and the
fitted model are of the same form as in equation (2.1) then
the two F ratios Frl = [ssr1/(n  p  1)1/MSE and
FL2 = ssL2/MSEpure are both distributed as central F random
variates, with respective numerator and denominator degrees
of freedom (n  p  1), (N  n) for FL1 and 1, (N  n) for
FL2* the true model is of the form shown in equation
(2.2), then FLj_ and FL2 are both distributed as noncentral F
random variates. The expected values of SSLi and SSL2 are
used to show what functions of S2 are testable with FL1 and
fL2*
Two examples are presented by Draper and Herzberg to
illustrate this testing for lack of fit. The first example
makes use of a first order orthogonal design in k factors
augmented with center point replicates for fitting a first
order polynomial model. If the true model is of the second
order, then FL2 can be used to test a hypothesis concerning
the parameters associated with the pure quadratic terms in
the model. If all such parameters are zero, then F^
provides a check on interaction terms. The second example
illustrates the fitting of a second order polynomial model
to a second order design with all odd design moments of
order six or less zero. If the true model is third degree,
then Fl1 can be used to test the significance of the third
order terms, while FL2 tests terms of order greater than
three. The partitioning of SSLQF into SSLi and SSL2 and the
25
corresponding tests of hypotheses are also given in Myers
(1971, p. 114119), for the special case of fitting a first
order polynomial model to a 2^ factorial or a fraction of a
2^ factorial design augmented with center point replicates
and the true model is of the second degree.
A more complete partitioning of the lack of fit sum of
squares in an attempt to obtain a more detailed diagnosis of
the lack of fit of the fitted model is given in a technical
report written by Khuri and Cornell (1981). The lack of fit
sum of squares, which has n  p degrees of freedom, is
partitioned into n  p independent sums of squares, each
having one degree of freedom. The expected values of these
single degreeoffreedom sums of squares are used to
identify at most n  p linearly independent causes for the
lack of fit variation. Tests of significance are performed
on the assumed contributing causes. This method enables the
screening of all subsets of g.2 in order to identify those
subsets which are most responsible for lack of fit of the
fitted model.
We shall now discuss the second general approach used
in lack of fit testing, which is to test for lack of fit by
making use of response values observed at points which are
near neighbors in the factor space when true replicate
observations are not available.
26
2.3 Testing for Lack of Fit Without
Replicated Observationsâ€”Near Neighbor Procedures
Green (1971) suggests the following approach when
testing for lack of fit if there are no design points at
which replicate observations of the response are
available. The N observed values of a response, Y,
considered a function of only one variable, x, are divided
into g groups, by grouping observations which have similar
values of x. Green hypothesizes a model of the form Y= H& +
e., where Y is an Nxl vector of observable responses, H is an
Nxm matrix whose columns correspond to known functions of
the variable, x, g. is an mxl vector of unknown regression
coefficients, and g. is the Nxl vector of random errors,
e ~ Nn(0, Â»2In).
Green's method assumes that the vector of differences
(EY  Hg.) can be well approximated by a dth order polynomial
in x within each of the g groups, d > 1. An alternative
model of the form
Y = H v + n + Â£
is given, where Â£. is distributed as NN(Q, a2!^) , Hj_ is an
Nx [g (d + 1) + mjJ matrix of known constants, y is a
[g(d + 1) + ntjjxl vector of regression coefficients, and el,
as Green states is "a small vector." The first g(d + 1)
columns of H^ correspond to the polynomial terms for the g
groups (with (d + 1) terms for each group), the rightmost
m^ < m columns in correspond to terms that are in the
27
fitted model, but are not represented among the g(d + 1)
polynomial terms in the alternative model.
Under the assumption that a = D, the presence of lack
of fit is tested by using the test statistic:
Y'[H1(HH1)"1H  ]Y/[g(d +1) +  m]
Y'[I  Y/[N  g(d + 1)  mj
(2.4)
This statistic is of the same form as the F statistic used
in the standard multiple regression test of a postulated
model against a more general one which includes the
postulated model as a special case. Lack of fit is
suspected if the calculated F ratio in (2.4) is greater
than Fa ;g (d+1 )+m1m, ugid+Dnij. where this latter quantity
is the upper 100a percentage point of the central F
distribution.
Green notes that when there is no lack of fit, the
quadratic forms Y'[HÂ±(HjHÂ±)_1H^  H(H'H)_1H']Y and
Y'[l  (HH^)~^H]Y are distributed independently as
a2x^ with g(d +1) + m^  m and N  g(d +1)  m^ degrees of
freedom, respectively. In this case the F ratio in (2.4)
possesses a central F distribution. If there is lack of fit
on the other hand, then these two quadratic forms are
distributed as noncentral chisquares, multiplied by a2,
with respective noncentrality parameters
28
S 1 = [hxv + n ] â€™ [ ( HH x) 1Hâ€™_  H(H'H) 1Hâ€™][h;lv + n]
and ? ^ = n'[l  ^H^]n . Thus the assumption that
n = 0 can affect the power of the test, since if n * o , the
expected value of MSE is greater than o^, where MSE is the
quadratic form in the denominator of the F ratio. Hence if
n * 0 , the probability of calculating a large F value is
reduced, and we are less likely to detect lack of fit using
an upper tailed rejection region.
Daniel and Wood (1971) suggest another method for lack
of fit testing when replicated observations of the response
are not available. They make use of "near replicates" to
obtain an estimate of a, which is the standard deviation of
the observable responses in the true model. The value of
the estimate o is compared to the square root of the
residual mean square from the analysis of the fitted
model. Lack of fit is indicated if the square root of the
A
residual mean square is large compared to the estimate o.
To determine when observations are near replicates so that
an estimate of o can be found, they define the squared
distance between any two data points, j and jâ€™, to be
measured by
K
I [b
i =1
where j and x^j. are the values of the ith independent
variable corresponding to the observations yj and Yj'/
respectively, i = 1, 2, ..., K, and bj_ is the ordinary least
29
squares estimate of the ith regression coefficient. In the
denominator, s^ is the square root of the residual mean
square for the fitted model.
To obtain an estimate of a from near replicates, let
And = dj  dji , n = 1, 2, ..., (^), where dj and dj. are
the residuals at points j and j', respectively, and where
there are N data observations in the experiment. Since the
expected value of the range for pairs of independent
observations from a normal distribution is 1.128o, a running
average of the And's is calculated and their average is
multiplied by .886 = (1/1.128) to get a running estimate,
sn, of o. That is, sn = .886 Â£ And/n . The closest pair
of observations as judged by D?j, is used to begin the
running estimate, the next closest pair (next "nearest
neighbors") is used for A2d/ an<^ the Procedure continues
until sn "stabilizes." The stabilized value of sn is used
to estimate a.
A third method for testing for lack of fit without
replication is given by Shillington (1979). The fitted
model is of the form
Y = X8 + e (2.5)
where Y (N*1), X (Hxp), and 8 (pxl) are defined as in (1.2)
and e ~ NN(0, o^i ) . The test for lack of fit of the
fitted model is a test for whether the true model has the
form
Y = X8 +5 + e ,
30
where 5 (Nxl) is a fixed effect quantifying the departure of
(2.5) from the true model.
Shillington assumes that the data can be grouped into g
cells, with nj observations in the jth cell, determined in
advance. Letting Cj refer to the jth cell, j =1, 2, ...,
g, a vector of cell averages is written (gxl), where the
jth element of is the average of the observed responses
in Cj. The matrix X^ of independent variables associated
with Yq is the gxp matrix where the elements in the jth row
are x'. = E x!./n . , that is, row j of Xr is the row
O i=1 "ID D
vector x'. . The matrix Xr is assumed to be of full rank
â€¢ D c
p < g. Also within each cell are defined the differences
W. . = Y..  Y . , i â‚¬ C. , j =1, 2, ..., g, where Y . is
1D ID D D J D
the jth element of Yc.
The two independent data sets, Y^ and (W â€”} with g and
N  g degrees of freedom, respectively, are used to find two
independent estimates of a2. The first estimate is written
as
MSE
B
9
E
j=l
n â€¢ (Y
D
 s.yB>
V(g  p)
A
where is the weighted least squares estimate of Â§. using
the regression of cell means, Yc, on Xc. The second
estimate of a2 uses the within cell deviations on cell
means, and is
MSET7 =
W
g
E
j=i
n .
D
E
i=l
(W..  W..) /(N  g  r),
31
where r is the rank of an N*p matrix with rows equal to
x!.  x'. , i e C., j =1, 2, .., g. If the matrix of
1D â€” â€¢ D D
independent variables, corrected for cell means, is of full
rank, then r = p. Here VJâ€” is the estimate of W^j from the
regression of cell residuals ÃW^j} on the associated vectors
of independent variates, x! .  x' . .
13 ~ â€¢ 3
If the fitted model is the correct model, then MSEg and
MSEW are independent estimates of o2 and the ratio MSEB/MSEW
is an F statistic with g  p and N  g  r degrees of
freedom. When all observations in a cell have the same
settings of the independent variables, that is, the
observations are truly replicates for all cells, then this F
statistic is identical to the F statistic in the usual lack
of fit test in which the residual sum of squares is
partitioned into lack of fit and pure error sums of squares,
as given in Draper and Smith (1981, p. 120).
If the true model is Y = X8 + 6 + e , however, and if
we let X'<5 =0 and a2 = 6'6/N, then
E(MSEb) = a2 + ÃÂ¿[I  XC(XÂ¿XC) 1XÂ¿]?B/(gp)
where Ã³ (g*l) has jth component equal to l
~B i=i
Furthermore, with this latter true model form
<5 . . /n .
ID D
E(MSEw) =
6 ' (i 
wl
xwlxw
V"lxw>V,N  9  r)
32
where 6_w has the components <$ â€”  i e Cj, j = 1, 2,
g. The matrix XTs7 (N*p) has the rows x! .  x'
w 1 j . j
i Â£ Cy j = 1, 2, g. The power of the F test,
F = MSEB/MSEW, depends on the relative bias of the estimates
of o2f that is, the biases in MSEB and MSEW.
Shillington states that the power of the F test which
makes use of F = MSEB/MSEW is maximized by forming cells so
that the bias of E(MSEW) is minimized. This is the same as
forming cells so that the within cell variation in 6 is
minimized. Shillington (1979, p. 141) also states,
"Observations with near covariate (independent variable)
values might be expected to have similar Â§. values, since we
assume that 6 varies in some continuous but unknown fashion
with X. This justifies the usual procedure of forming
groups by collapsing observations with adjacent covariate
values. Indeed, if covariates do not vary within cells we
have the usual lack of fit test and maximum power."
By imposing a further structure on the form of <$, it is
shown that if the F test has an upper tailed rejection
region, the power is maximized by selecting the group sizes
as nj = 2, j = 1, 2, ..., g. Finally, Shillington suggests
that in the presence of more than one independent variable
problems in grouping may arise, and in this case it may be
wise to perform a different lack of fit test for each
parameter. Following this approach, an example is given
which suggests testing lack of fit for each of the p
independent variables separately may be more powerful than
33
trying to form groups based on all independent variables at
once.
In summary, all the approaches we have discussed for
testing for lack of fit when replicate observations of the
response are not available at any of the settings of the
independent variables make use of grouping the observed
response values according to similar values of the
independent variables. The observations falling in such
groups are referred to as "pseudoreplicates" or "near
neighbor observations." These pseudoreplicates are used to
estimate the true variance of the observations, o^, but a
completely unbiased estimate of a^ cannot be attained unless
true replicate observations are available. In each case,
the power of the lack of fit testing procedure is reduced
because an unbiased estimate of is not attainable. We
now turn to the use of check points for lack of fit testing.
2.4 Testing for Lack of Fit with Check Points
An alternative to the two approaches to lack of fit
testing already discussed is the method which makes use of
check points. We assume a model of the form E(Y) = , as
given in (2.1), is fitted in a response surface system, but
that the true model is of the form E(Y) = Xf^ + X2B2 as
given in (2.2). The parameters, s.^, the fitted model are
estimated by ordinary least squares techniques, making use
of the values of the response observed at the design
points. After the model is fitted, values of the response
are observed at additional points in the experimental region
34
called "check points." The observed response values at the
check points are compared to the values which the fitted
model predicts at these same check points. It is important
to note that the observed values of the response at the
check points are not used in fitting the model initially.
Snee (1977) gives four methods of validating regression
models, one of which is the collection of new data to check
predictions from a previously fitted model. In a designed
experiment these new data take the form of check points.
Snee suggests that the inclusion of a small number of check
points in any designed experiment is a "worthwhile"
procedure.
Scheffe (1958) proposed a test for lack of fit when the
{3,2} simplex lattice design is used for fitting a second
order canonical polynomial model in three mixture
components. It is desired to use the observed value of the
response at (1/3, 1/3, 1/3) as a check point blend. The
test statistic proposed is the t statistic of the form
[var(Y  Y)]1/2
where Y is the observed value of the response at the check
point, and Y is the value of the response predicted at the
same point by the second order model which is fitted by
ordinary least squares techniques to the observed response
values at the six design points of the {3,2} simplex
lattice. The response value observed at the point
35
(1/3, 1/3, 1/3) is not used in fitting the model. Lack of
fit is inferred if the absolute value of the calculated t
value in equation (2.6) is larger than the corresponding
tabled t value.
In the denominator of the t test of equation (2.6), the
variance of the difference Y  Y is shown to be
A A
var(Y  Y) = var(Y) + var(Y)
= ( 44/27r)c 2 ,
when r replicates are taken at each design point. The
estimate of the variance of Y  Y is (44/27r)o2, where a2 is
calculated from the replicated response values at the design
points.
Scheffe (1958) also alludes to a test for lack of fit
when several check points are used simultaneously. When
there are k check points, the test for lack of fit is an F
statistic of the form
F
d
(2.7)
where d* = (Y1  Yr Y2  Y2, ..., Yk  Yk) , and V = o2VQ =
var(cl). Formulas are given for the elements of Vg in the
special case when the check points are the design points of
the {3,2} simplex lattice. Lack of fit is suspected if the
calculated value of the F statistic given in (2.7) is larger
than the corresponding tabled F value.
36
Gorman and Hinman (1962) suggest the same t test in
equation (2.6) that Scheffe (1958) suggested for a check
point taken at (1/3, 1/3, 1/3) to test for lack of fit in a
second order polynomial model fitted from a {3,2} simplex
lattice design. They suggest using (1/3, 1/3, 1/3) as the
location of the check point because the observation at this
point may later be used to fit the next more complex model,
the special cubic, if the second order model is found to be
inadequate. They state that in general for the second order
polynomial model as well as higher order models, check
points should be taken in regions of particular interest, of
which there are usually many in any blending study.
Further, they suggest that the number of check points
depends on individual experimental situations â€” technical
background, precision required, cost of materials and
analyses, and probability of requiring a more complex
model. However, no specific criterion is given by Gorman
and Hinman for selecting the location of the check points.
Gorman and Hinman (1962) indicate that a t test at a
check point other than at (1/3, 1/3, 1/3) takes the same
form as the statistic of equation (2.6),
[var(Y) + var ( Y) ]
with the additional condition that if several check points
are taken, say for example k points, the method of checking
37
the fit is to compute the t value at each location and refer
these calculated t values to the 100(a/2k) percentage point
of the central t distribution rather than the 100(a/2)
percentage point.
Kurotori (1966) gives an example of a mixture
experiment where the response is the modulus of elasticity
of a rocket fuel, which is a mixture of three components,
binder (x^), oxidizer (X2), and fuel (X3). The factor space
of feasible mixtures is a subspace inside the two
dimensional simplex or triangle where all three components
are present simultaneously. "Pseudocomponents" are defined
and in the pseudocomponent system a special cubic model is
fitted to data collected at the points of the q = 3 simplex
centroid design (Figure 4). A check for adequacy of fit is
made by using three check points and the response values at
the check points are used only for testing the fit of the
model and not for fitting the model initially.
The reason for the choice of the particular check point
locations by Kurotori is that, as he states, "They are the
most remote mixtures from the seven design points." The
lack of fit test is an F statistic of the form
2
F = I2 (2.8)
a
3
2 J â– * 2
where s = Z (Y.  Y.) , for the i = 1, 2, 3 check points
2 . 1=1 1 1
and a is an estimate of measurement error from a previous
analysis. Kurotori admits that the use of the F statistic
38
Figure 4. Kurotori's rocket fuel example,
xl'f x2'' anc^ x3' rePresent pseudocomponents.
in Eq. (2.8) for lack of fit testing may be risky because
the predicted values at the check points are correlated
(correlation of .5), although the observed values are not
correlated. Kurotori suggests individual t tests as
proposed by ScheffÃ© (1958) might be the preferred procedure.
Snee (1971) repeats Kurotori's rocket fuel example
using the same F test for lack of fit as Kurotori and makes
the comment that the Y^'s at the check points are
correlated. In stating that the F test is not an exact
test, he nevertheless offers no solution in the form of an
exact test.
39
In summary, only Scheffe refers to an exact F test when
several check points are considered simultaneously for
testing for possible lack of fit of a model fitted in a
mixture space, and his development is limited to the special
case where the check points are the design points used to
fit the model initially. No criterion is proposed by
Scheffe for selecting other locations for the check points.
CHAPTER THREE
AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL
3.1 Introduction
In Chapter Three we investigate the problem of testing
for lack of fit of a linear model fitted in a mixture
space. The testing is to be accomplished with the use of
check points. We assume that an experimental design is
specified, and that the fitted model is of the form
E(Y) = XSi (3.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants and rank p, and 3^ is a
vector of p unknown regression coefficients. The true model
is assumed to be of the form
E (Y) = XB + X Â¡J (3.2)
where X2 is an Nxp2 matrix of known constants and is a
vector of p2 unknown regression coefficients. Throughout
our development, we will assume that the random vector Y has
the normal distribution with variancecovariance matrix
equal to a^IN*
40
41
In our investigation we wish to determine the proper
testing procedure to follow in deciding whether the fitted
model exhibits lack of fit. In order to optimize the lack
of fit testing procedure, we will determine the location of
the check points so that the power of the test is maximized.
3.2 Testing for Lack of Fit in the Presence of
an' External Estimate of Experimental Error Variation
3.2.1 The Test Statistic
We wish to test the performance or fit of a fitted
model in a mixture space when the true model possibly
contains terms in addition to those in the fitted model.
The fit of the model is to be tested by a test which makes
use of the response values observed at certain locations
called "check points" in the experimental region, by
comparing them to the values which the fitted model predicts
at the same check points. The observed values at the check
points are not used for estimating the coefficients in the
fitted model and are assumed to represent the values of the
true surface at the check points.
Let us define the vector of differences
d = (Y*  Y*)
 /V* â€” Y* Y* â€” Y* Y* â€” Y*\'
^ 1 1' 2 2' â€¢**' k k'
where YÃ, i = 1, 2, ..., k are observed response values at
k check points and YÂ£, i =1, 2, ..., k are response values
predicted at the k check points by the fitted model,
42
Yi = x*'b^, where is the ordinary least squares estimator
of and where xÂ£' is the ith row of X*, the kxp matrix
whose columns are of the same form as the columns of X but
with its rows evaluated at the k check points. Note that
if S2 = 0, then E(d) = 0 and if 02 * 0/ then
E(d) = (X*  X*(X'X)â€œ1X,X2)e2. Let V represent the
variancecovariance matrix of the random vector d.
2
Then v = a Vq where
VQ = Ik + X*(X,X)â€œ1X*'
and where 1^ is the identity matrix of order kxk.
We assume that an unbiased estimate of a^ is available
~ 2
and we denote this estimate by a where the subscript ext
" 2
stands for external, and aext is independent of the model
being fitted. The test statistic for the hypothesis of zero
lack of fit Hq : E(d) = 0 is
d 'V'â€™d/k
F = (3.3)
aext
(see Scheffe, 1958, p.358). It will be shown later in this
section that the F ratio in Eq. (3.3) possesses either a
central F distribution or a noncentral F distribution,
depending upon whether the true model is represented by Eq.
(3.1) or Eq. (3.2).
A 2 .
The variance estimate a ^ that appears m equation
ext
(3.3) is ordinarily generated from replicated observations
43
at some of the design points in the experiment. We assume
 2
that a ^ is a constant multiple of a central chisquare
ext
random variable with v degrees of freedom. This is written
as
a2 . = SSE /v
ext pure'
= (a2/v)(SSEpure/a2)
2 2
where SSEpUre/Â° ~ Xvâ€¢ Note that SSEpUre denotes the
portion of the residual sum of squares due to replication
variation from the fitted model. The residual sum of
squares from the fitted model may be partitioned into
SSEpUre and SSL0F only if replicated observations are
collected at one or more design points. For the case where
replicate observations are collected at all of the design
points
SSE
pure
n n.
E E1
i=l j=l
2
where n is the number of distinct design points, n, > 2 is
the number of replicates at the ith design point, Y^j is the
jth observation at the ith design point, and Y. is the
1 â€¢
average of the n^ observations at the ith design point.
n
Here SSEpure has v = E (n.  1) degrees of freedom.
i=l
When the fitted model and the true model are of the
same form as defined by Eq. (3.1), the quantity d'Y^d/a2
44
possesses a central chisquare distribution (Searle, 1971,
p.57, Theorem 2). However, when the true model is of the
form specified by Eq. (3.2), d'V^d/a2 possesses a
noncentral chisquare distribution. Thus when the true
model is of the form in Eq. (3.1),
d'V'Va2 ~
but when the true model is of the form in Eq. (3.2),
dv'Va2 ~
where in the second case the noncentrality parameter X^ has
the form
= E (d ) â€¢VQ1E(d)/2a2
= Â§.Â¿(X*  X*A)'V"1(X*  X* A) g 2/2a2.
The matrix A = (X'X) ^X'X2 is called the alias matrix and is
of order PXP2* In the matrix X* is of order kxp2 and
has the same relationship to X2 as X* has to X.
2
Since SSEpure/a is statistically independent of
12 .
d'Vg d/a , then under model (3.1) the test statistic
45
F
dâ€¢V01d/ko2
s^Epure^vo 2
d'V^d/k
= *~2
Â°ext
will have a central F distribution. When the true model
contains terms in addition to those in the fitted model then
F will have a noncentral F distribution. We write these two
cases as
F ~
F
k, v
under model (3.1), and
under model (3.2), where the noncentrality parameter is
X1 = Â£2(X2 " X*A) â€™ Vq1 ( x2 " x*a)Â§.2/2Â°2
3.2.2 The Testing Procedure and an Expression for the Power
the Test
Given that the form of the fitted model is defined as
Eq. (3.1), the expected value of the numerator of the F
statistic in Eq. (3.3) will depend on the form of the true
model. For the case where the true model is expressed as
Eg. (3.2),
46
E(numerator) = Efd'v'^d/k)
= Â«â€™2/k >â– *]Â£,
= ( o 2/k ) (k + 2A 1)
2 2
= a + 2o ^A /k
= o2 + i^A^/k, (3.4)
where A^ = (X^  X*A)'V01(X^  X*A). However, when the true
model is Eq. (3.1), 3^ = 0 and in this case A^ = o so that
2 ~ 2
E(numerator) = a . Also o , is an unbiased estimator of
ext
o 2 and
E(aext} = Â°2â€™ (3.5)
Therefore the ratio E(numerator)/E(denominator) where
* 2
the denominator is a , will equal unity under model (3.1),
that is, when there is no lack of fit. Under model (3.2),
the ratio will be greater than or equal to unity so lack of
fit should be suspected if the calculated F ratio in
equation (3.3) is large. We can thus use an upper tailed
rejection region to reject the hypothesis of zero lack of
fit. The power of the test is
47
v j\i
> F
a ; k, v
}
where F^ ^ v is the upper 100a percentage point of the
central F distribution with k numerator degrees of freedom
and v denominator degrees of freedom.
It is worth noting that from Eq. (3.4) and Eq. (3.5)
testing the hypothesis that 0 = 0 is equivalent to testing
the hypothesis that Xj_ = 0, assuming A^ is positive
definite. Thus testing a null hypothesis of zero lack of
fit using the proposed testing procedure involving the F
ratio in (3.3) may be expressed as a test of the hypotheses:
H
o:
0
3.2.3 A Method for Locating Optimal Check Points
Once a design for fitting model (3.1) in a mixture
space is chosen and the number of simultaneous check points
is decided on, say k > 1, the next step is to determine
where in the mixture space we should place the k check
points so as to maximize the power of the test for lack of
fit. The location of the check points is to be made
independently of the value of 02Â«
48
The power of the upper tailed F test for lack of fit is
an increasing function of X ^ (see Appendix 1 for proof, with
X^ = 0). Therefore, to maximize the power of the test we
maximize the value of X^ defined as
2
Xf = 02Ai&2//^a
where A^ = (X*  X*A)'Vq1(X*  X*A), by properly selecting
the k check points whose coordinates are defined in X*. To
maximize the value of X^, we shall concentrate on the matrix
A1â€˜
The matrix A^ is a square matrix of order P2X?2 and
a scalar quantity when p2 = 1. By maximizing the scalar
quantity A^ with respect to the k check points, the power is
maximized no matter what the value of 0^* Maximizing the
scalar A^ can be accomplished by using The Controlled Random
Search Procedure given by Price (1977). This procedure is
described in Appendix 2. As a computational aid, Aj_ can be
expressed as
VQ + (X*  X*A)(X*  X*A)'
when p2 = 1/ where the symbol B denotes the determinant of
the square matrix B. Thus the computations reduce to
evaluating two determinants rather than inverting VQ (see
Scheffe, 1959, Appendix V, p.417).
49
When p2 > 1 and A^ is no longer a scalar, maximizing X
(and thus maximizing the power of the test) cannot be
accomplished without specifying 0 . In this case we make
use of a lower bound for Xj_ (Graybill, 1969, p. 330, Theorem
12.2.14(9)) defined as
tJmin22/2Â°2 < X1
(where u . is the smallest eigenvalue of A, ) to be used in
min r i
place of X^. Hence an approximate solution to the
maximization of X^ will be achieved by finding the k
simultaneous check points (using Price's procedure) that
maximize \i . , the smallest eigenvalue of Ai . In other
min i
words when p^ > 1* and in order to avoid specifying 0^, we
seek to maximize a lower bound value for X^. This
maximization does not depend on the value of 02Â«
There are cases where the matrix Aj. is of less than
full rank (less than rank P2) or equivalently where the
matrix Ai is positive semidefinite so that u â€¢ will be
equal to zero no matter which check points are selected.
One such case occurs when k < p^ (when the number of check
points is less than the number of parameters in the true
model which are not in the fitted model) since when k < p^
rank(A^) = rank[v~ /2
= rank(X*  X*A),
50
and so rankfA^ $ min(k, p2) because the matrix (X*  X*A)
is of order kxp2. Therefore when k < p , the rank of A^ is
at most k so that A, is of less than full rank. Since u
J min
must be equal to zero when A^ is positive semidefinite, an
alternative method to that of maximizing y . to select
optimal check points must be found when A^ is positive semi
definite in order to produce a positive lower bound for X ^.
In this pursuit, let us write X^ as
X^ = Â§2 A^Ã‰.2^^a
= 0Â¿PAP'02/2 a2
= 0 Â¿[ Pj. :P2] ^ ia9 [A !, A2 = 0] [P1:P2] '02/2a2
= 0^P1A1P[e2/2a2
where A is a diagonal matrix with elements equal to the
eigenvalues of A^, P is an orthogonal matrix whose columns
are orthonormal eigenvectors of A^, A^ and P^ correspond to
the positive eigenvalues of A^, while A2 = 0 and P2
correspond to zero eigenvalues of A^. Then by Theorem
12.2.14(9) in Graybill (1969) we can write
y + . z'z/2a2 < X. (3.7)
mm  1
where y + . is the smallest positive eigenvalue of Ai , and
Mmm
51
z = PÃ^2* T^us by Eq. (3.7), an approach to maximizing a
positive lower bound for X^ when is positive semi
definite is to select check points that maximize the
smallest positive eigenvalue of A^. It must be noted,
however, that this method can only be used when
0_2 e n C(P^), where C(P^) denotes the column space of Pj_
and n CiP^) denotes the intersection of all such spaces
which can be obtained at all possible check points
locations. This is because, in general, z'z in (3.7)
depends on the location of the check points through its
dependency on P^. If, however, &2 e n CfP^, then
z'z = b^Pj^Ã^ = eÂ¿pp'e2 = *22â€™ since Â§2p2 = Â°*
+ 2
It follows that when &2 e n CfP^, %inz'z/2o
+ 2 +
= u â€¢ and only n . depends on the location of the
mm22' 1 Hmm
check points.
3.3 Testing for Lack of Fit When MSE Is Used
to Estimate Experimental Error Variation
3.3.1 The Test Statistic
In this section we shall show that when an external
estimate of is not available and the residual mean square
(MSE) from the fitted model of the form (3.1) must be used
as an estimate of o^, the test statistic
F
d'v01d/k
MSE
(3.8)
possesses a central F distribution when the true model is
52
Eq. (3.1), but possesses a doubly noncentral F distribution
when the true model is Eq. (3.2).
In the initial section of this chapter, the quantity
1 2
d'V0 d/a was said to possess a central chisquare
distribution or to possess a noncentral chisquare
distribution, depending on whether the true model was
specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum
of squares from the fitted model is defined as
N
SSE = I (Y.  Yâ– )2
i=l 1 1
= Y'(I  X(X,X)1X')Y
and it is easy to show (Searle, 1971, p.57, Theorem 2) that
SSE/a2 possesses a central chisquare distribution if the
true model is Eq. (3.1), but under model (3.2), SSE/a2
possesses a noncentral chisquare distribution. This is
expressed as
SSE/a2
2
XNp
SSE/a2
X
,2
Np,
X2
under model (3.1), and
53
under model (3.2), where the noncentrality parameter X2 is
x2 = eÂ¿(x2  XA)'(X2  XA)32/2a2.
The distributional form of the test statistic in Eq.
(3.8) is derived by knowing that the quantities
12 2
d'V0 d/a and SSE/a are statistically independent (see
Appendix 3), so that
d'V^d/ko2
F = 2
MSE/a
d'V^d/k
MSE
is distributed as a central F when the true model is Eq.
(3.1), but when the true model is Eq. (3.2) the F ratio is a
doubly noncentral F, that is, under model (3.1),
F
F
k, Np
and under model (3.2),
F ~ F"
k, Nâ€”p ; X 2. i X 2
3.3.2 The Rejection Region and its Relation to the Power of
the Test
In Appendix 1 it is shown that if k, Np, and X2 are
fixed, then the power of the F test using the ratio (3.8) is
54
a function of the location of the rejection region (upper
tailed or lower tailed) of the test. The power increases
with increasing values of the numerator noncentrality
parameter, X^, when the test is an upper tailed test. The
power decreases with increasing values of when the test
is a lower tailed test. This means that to study ways of
increasing the power of the test, we have to determine
whether the test is an upper tailed test or a lower tailed
test. Similarly, for fixed values of k, Np, and X^, the
power of the F test is a decreasing function of Xj for an
upper tailed test, and is an increasing function of X2 when
the F test is a lower tailed test (ScheffÃ©, 1959, p. 136
137) .
To decide if the test is an upper tailed test or a
lower tailed test, we recall from Section 3.2.2 that if the
true model is Eq. (3.1) then the expected value of the
numerator of the F statistic in (3.8) can be written as
E(numerator) = o^,
and if the true model is Eq. (3.2),
2 2
E(numerator) = a + 2a X ^/k
= a ^
(3.9)
55
where the p^xp^ matrix A^ is A^ = (X*  X*A) ' Vq^ (X*  X*A) .
Similarly, it can be shown that if the true model is Eq.
(3.1), the expected value of the denominator of the F
statistic in (3.8), where the denominator equals MSE, is
E(denominator) = E(MSE)
but if the true model is Eq.(3.2),
E(denominator) = E(MSE)
â–  tÂ°2/(N  p)1Exnp,a2
 [o2/(N  p) ] [ N  p + 2A2]
= a2 + 2a2X2/(N  p) (3.10)
= a2 + 8Â¿A2B2/(N  p)
where the P2XP2 matrix A2 is A2 = (X2  XA)'(X2  XA). Thus
the ratio Enumerator )/E(denominator) will equal unity if
the true model is Eq. (3.1), but if the true model is Eq.
(3.2), the ratio is greater than unity if B^A^B2/k >
2A22^N  P) â€¢ In this latter case we reject the null
56
hypothesis of zero lack of fit if the calculated value of
the F ratio in (3.8) is large. An upper tailed rejection
region seems reasonable for this test. When the true model
is Eq. (3.2), and if & /k < 2A22/(N â€œ p)' then a lower
tailed rejection region is preferred.
3.3.3. A Method for Locating Optimal Check Points
Given a design for fitting a model of the form in Eq.
(3.1) in a mixture space (note that fixing the design fixes
A2 and (N  p)), and given the number of simultaneous check
points desired, k > 1, we now wish to determine where in the
mixture space the k check points should be located so as to
maximize the power of the F test for lack of fit, where the
test statistic is given in Eq. (3.8). We also wish to
position the optimal check points in a manner that is
independent of the values of the elements in 32*
The case of an upper tailed test. To help us find k
simultaneous check points that maximize the power of an
upper tailed test, we shall make use of the fact that the
power is an increasing function of XTherefore to
maximize the power of the upper tailed F test, we shall seek
the locations of the k check points that maximize X^.
As in the case considered in Section 3.2.3, where the
test statistic had a noncentral F distribution, if the
number of extra terms in the true model is P2 = 1, then
maximizing Xis equivalent to maximizing the scalar A^ â€¢
However, as before, if p2 > 1, then the P2XP2 raatr:'x Ai is
not a scalar and we will have to approximate the
57
maximization of X^ bY maximizing a lower bound for X. This
is done by finding the maximum value of y . , the smallest
3 nun
eigenvalue of A^, since
u â€¢ 813^/2a
min22'
< X
1â€™
When the number of check points is less than the order
of the square matrix A^, that is, k < pÂ£, then rank(A^) <
min(k, p9), and A, will have y . =0. For this case, we
again try to maximize the smallest positive eigenvalue of A^
which we denote by /. , while remembering from Section
min
3.2.3 that this technique is limited to situations where
82 Â£ n C(P^) .
The case of a lower tailed test. To find k check
points to maximize the power of a lower tailed test, we make
use of the fact that the power of the lower tailed F test
increases as X ^ decreases. Then if P2 = 1 and Aj_ is a
scalar quantity, X^ can be minimized with respect to the k
check points by finding the check points that minimize A^â€¢
If p > 1, then by Theorem 12.2.14(9) in Graybill (1969), we
see that an upper bound for X1 is
X1 < umax^2^2/2a
(3.11)
where umax is the largest eigenvalue of A^. An approximate
solution to minimizing X^ in (3.11) can be achieved by
minimizing u . It is not necessary to treat the case
3 max J
58
of k < p2 separately here, although will equal zero if
32 is in the column space of P2, where P2 is the matrix
whose columns are orthonormalized eigenvectors corresponding
to the zero eigenvalues of the matrix A^.
3.3.4 Determining Whether the Test Is Upper Tailed or Lower
Tailed
The procedures outlined in Section 3.3.3 produce a set
of k check points that simultaneously maximize the power of
the upper tailed test as well as a second set of k check
points that simultaneously maximize the power of the lower
tailed test. The check points that are selected maximize
the power, given A2, k, and N  p without specification
of g_2' except that when A^ is positive semidefinite we
require that 3^ e n C(P^).
It is now necessary to decide which of our two
candidates will be used for a lack of fit test. To choose
between the upper tailed test and the lower tailed test, let
us consider the quantity
R = [A1/k]  [A2/(N  p)].
If R is positive definite when the true model is Eq. (3.2),
then no matter what the value of 32 is, the ratio
E(numerator )/E(denominator) will be greater than unity,
implying an upper tailed test is to be used. Similarly, if
R is negative definite, then a lower tailed test should be
used. Finally, if R is not definite, then neither an upper
nor a lower tailed test is implicated and further
59
investigation is necessary. The criterion of R = [ A /k]
[A^/(N  p)] may yield any of the four following cases.
Case 1. If R = [ A /k]  [ A /(N  p) ] is positive
definite when is generated by the k optimal upper tailed
test check points, and R is not negative definite when A^ is
generated by the k optimal lower tailed test check points,
then we recommend that the check points be used that yield
the optimal upper tailed test with an upper tailed rejection
region.
For Case 1 it is necessary for A^ to be positive
definite (see Appendix 4). Since A^ is a square matrix of
order P2XP2 with rank(A^) < min(k, p2), then A^ can be
positive definite only if k > p^. Thus, there must be at
least p2 check points for Case 1 to hold, where p2 is the
number of terms in the model of Eq. (3.2) that are not in
the model of Eq. (3.1).
From inspection of equations (3.9) and (3.10), it is
apparent that the testing for lack of fit in Case 1 is
equivalent to testing the hypothesis
1
(3.12)
against the alternative
X
1
k
60
since R = [A^/k]  [A2/(Np)] is positive definite when the
true model is Eq. (3.2). In Appendix 5(a) it is shown that
under Case 1, the hypothesis given by (3.12) is equivalent
to the hypothesis
H
o:
0.
Case 2. In Case 2 we assume that R = [A^/k] 
[A^/iN  p)] is not positive definite for the k optimal
upper tailed test check points, but that R is negative
definite for the k optimal lower tailed test check points.
Here we recommend that the lower tailed test check points be
used with a lower tailed rejection region.
It is necessary for A2 to be positive definite for Case
2 to occur (see Appendix 4). However, A^ need not be
positive definite, and so k need not be greater than P2. In
Case 2 then, it is possible that lack of fit may be tested
with only one check point.
By inspection of equations (3.9) and (3.10), a
hypothesis of no lack of fit is equivalent to
while the alternative
is equivalent to
hypothesis that lack of fit is present
N  p
< 0
61
since R = [ A /k]  [a^/ÃN  p)] is negative definite. In
Appendix 5(b) it is shown that the hypothesis given by
(3.13) is equivalent to the hypothesis
Case 3. We assume R is positive definite for the k
optimal upper tailed test check points, and R is negative
definite for the k optimal lower tailed test check points.
Hence either an upper or lower tailed test may be considered
as a possible test for lack of fit. If the quantity
2
22//Â° C3n be specified, then the minimum power for both
the optimal upper and optimal lower tailed tests can be
approximated, and the test with the greater minimum power is
recommended. In Appendix 4 it is shown that Case 3 can
occur only when A^ is positive definite for the upper tailed
test. Thus Case 3 can only occur when there are at least P2
check points.
The minimum power of the upper tailed test may be found
by calculating
( F" >
k,Np;A1L,X2U
a;k,Np
oh
(3.14)
where F^ ^ N_^ is the upper 100a percentage point of the
central F distribution,
62
B'B0/2a
2
and
2
where u .
min
is the smallest eigenvalue of A, and 6 is the
^ 1 max
max
largest eigenvalue of A2* Formula (3.14) yields a
conservative lower bound for the power of the optimal upper
tailed test. Note that is generated using the optimal
upper tailed test check points. The cumulative distribution
function of F" can be approximated by multiplying the
cumulative probabilities of the central F distribution by a
constant (Johnson and Kotz, 1970, p.197). This
approximation is described in Appendix 6. Other
approximations for F" (such as the Edgeworth series
approximation suggested by Mudholkar, Chaubey, and Lin,
1976) exist which are generally more accurate, but we chose
to use the approximation given in Johnson and Kotz (1970,
p.197) due to its simplicity. Additionally, the
approximation of Mudholkar, Chaubey, and Lin (1976) produced
negative probabilties when only one degree of freedom was
available in either the numerator or denominator of F".
This problem was avoided by using the approximation given by
Johnson and Kotz (1970).
The minimum power of the optimal lower tailed test can
2
be approximated similarly (if is specified) by
63
calculating
P ( F" < F }
k,Np;XlufA2L (1a);k,Np
where
A
1U
lJmax22/^a
2
and
X2L ~ (^min22//2a '
with umax equal to the largest eigenvalue of A^ and <5m^n
equal to the smallest eigenvalue of A2. Note that A^ is
generated by using the optimal lower tailed test check
points. For the lower tailed test, A^ may be positive semi
definite, and if g is in the column space of P2 then A ^ = 0.
In Case 3, the upper tailed test is a test of
while the lower tailed test is a test of
64
X
1
H
k
a
Case 4. In Case 4 we assume that R = [A^/k]
[A^/(N  p)] is not positive definite for the k optimal
upper tailed test check points and R is not negative
definite for the k optimal lower tailed test check points.
Here it is useful to write the difference between the
expected value of the numerator and the expected value of
the denominator of the F ratio in (3.8) as
8Â¿[AiA  A /(N  p)]82 = 02sas'e2
where 15 = diag(15^, 152, 15 ^) is a diagonal matrix consisting
of the eigenvalues of Rf 15^ is a diagonal matrix of the
positive eigenvalues of R, 152 is a diagonal matrix of the
zero eigenvalues of R, and 15 3 is a diagonal matrix of the
negative eigenvalues of R. The orthogonal matrix S can be
expressed as S = [si:S2:S3], where the matrices S^, S2, and
S3 have columns which are orthonormalized eigenvectors
corresponding to 15^, 152, and 153, respectively.
65
In Case 4, neither the optimal upper tailed test nor
the optimal lower tailed test is applicable for all values
of 0 ^â€¢ For completeness, we note that Case 4 actually
consists of nine subcases, where R may be positive semi
definite, negative semidefinite, or indefinite for either
the optimal upper tailed test or lower tailed test check
points. These subcases are listed in Table 2.
Table 2.
Nine Subcases
of Case 4.
Râ€”Upper
Râ€”Lower
Subcase
Tailed Test
Tailed Test
1
PSD
PSD
2
PSD
NSD
3
PSD
I
4
NSD
PSD
5
NSD
NSD
6
NSD
I
7
I
PSD
8
I
NSD
9
I
I
PSD = positive semidefinite, NSD = negative semi
definite, I = indefinite.
If ^2 lies in the column space of S2/ then /k ~
a2/(N  p) ] 0_2 is zero, and therefore lack of fit is not
testable with either an upper or lower tailed test. A
sufficient condition for the test for lack of fit to be
upper tailed in Case 4 is that be in the column space
of [S^:S2], but not entirely in the column space of S2* In
this case
and BÂ£[ A^/k  A2/(N  P)]62 will be greater than zero,
indicating an upper tailed test. Similarly, a sufficient
condition for the test for lack of fit to be lower tailed is
that b_2 be in the column space of [S :S ] , but not entirely
in the column space of S2. Then
S^A^/k  A2/(N  p) ] B 2 = 0 + eÂ¿S3fl3S^B2
= BÂ£S3a3S^B2
which makes A^/k  A2/(N  p)]g.2 less than zero,
indicating a lower tailed test.
To determine whether B2 is in the column space of
[S :S2], let us define the augmented matrix
= [ B_2 :S2] * bas a zero eigenvalue, then B2 is
in the column space of [S :S ]. Similarly, if we define
Q2 = [ 6_2: S 2] and Q3 = [b2:S2:S3], then B2 is in the column
space of S2 if Q^Q2 has a zero eigenvalue, and B2 is in the
column space of [S2:S3] if Q^Q3 has a zero eigenvalue.
67
Given that we are in a particular subcase of the nine
subcases described in Table 2, we recommend that lack of fit
be tested with the upper tailed test check points if it is
determined that g_2 is such that g^fA^k " A2/(N â€œP) ] Â§.2 is
positive when A^ is generated from the upper tailed test
check points. Likewise, for the same given subcase, if the
value of S2 Â°f interest is determined to produce a negative
value for B 2[ A^/k ~ A2/(N  p)]g2 when A^ is generated from
the lower tailed test check points, then we recommend that
lack of fit be tested with the lower tailed test.
We see then that Case 4 is an undesirable situation in
practice, since, in order to test for lack of fit, we must
assume a priori that any lack of fit is due to a nonzero
value of Â§_2 that produces an upper tailed or lower tailed
rejection region. However, it would seem rare that such
knowledge would be available.
3.4 Examples
We now present several examples to illustrate the
technique for locating optimal check points to be used in
testing for lack of fit in a mixture model.
3.4.1 Theoretical Examples
Example 1. In this example a second order canonical
polynomial model is fitted in three mixture components using
the {3,2} simplex lattice design, which is presented in
Figure 1 of Chapter 1. The true model is assumed to be the
special cubic model containing the term ^^23X1X2X3
addition to the six terms of the fitted model. The expected
68
values of the response at the six design points are assumed
to be represented by the fitted model in the form
E(Y) = Xf^,
but with the true model the expectations are written as
E (Y) = Xf^ + X282,
where X is a 6x6 matrix with rows that define the
coordinates of the six design points and columns that
correspond to the six terms in the fitted model (x^, X2/ X3,
X1X2' xlx3' X2X3^' 1 vector Â°f regression
coefficients (0 , 02, 03, 012, 0 , 0 23) / *2 is a 6x1
column vector containing the values of the term x^x2x3 at
the design points, and 0 is the single regression
coefficient 0^23*
The {3,2} simplex lattice design consists of only six
design points, and since six parameters are estimated in the
second order fitted model, there are no degrees of freedom
remaining for obtaining an estimate of the experimental
error, aWe assume therefore that an external estimate of
o^ is available, a which will be used m the denominator
ext
of the lack of fit F statistic given in Eq. (3.3).
Since there is one term in the true model in addition
to those in the fitted model, that is P2 = 1, we know that
in order to locate k simultaneous check points that maximize
69
the power of the test for lack of fit it is necessary to
maximize the scalar quantity
Aj^ = (X*  X*A)'V~1(X*  X*A)
with respect to the coordinates of the k check points. Here
X* is a kelement column vector with ith element equal to
the value of x* x* x* at the ith check point, X* is a kx6
ll i2 i3
matrix with ith row equal to the value of (x?^, Xi2r Xi3'
x* x* , x* x* , x* x* ) at the ith check point,
ili2 il i3 i2i3
A = (X1X)â€œ*X1X2 is the 6xl alias vector, and
Vg = + X*(X'X)1X*'. This maximization is accomplished
by use of the Controlled Random Search Procedure (Price,
1977), which is described in Appendix 2.
When only a single (k = 1) optimal check point is
desired the Controlled Random Search Procedure locates a
point (x*, x*) which maximizes
A1 = (X*  X*A)'V"1(X* â€”X*A),
where
x*
X2
= x*x *x
â˜… â€”
1 2 3
= x*x*(l
 X* 
X*) ,
70
X* = (xj, X*, X*, x*x*, x*x*, x*x*)
= (X*, X*, (1  X*  x*), x*xj, x*(1  X*  x*),
X*(1  X*  X*)),
and VQ = 1 + X*(X,X)_1X*'. The value of A^ is calculated
using the formula of Eq. (3.6). Following this procedure,
we find that the single check point that maximizes A^, and
thus maximizes the power of the test, is the centroid of the
triangular factor space (1/3, 1/3, 1/3). The value of at
this centroid point is A^ = 0.00084.
When the Controlled Random Search Procedure is used to
locate k = 2 simultaneous check points that maximize A^, the
centroid (1/3, 1/3, 1/3) is selected twice, and A^ =
0.00121. For three simultaneous optimal check points, the
centroid is selected three times, and A^ = 0.00142.
To test whether the second order model exhibits lack of
fit, when we suspect the special cubic model is the true
model, we form the F ratio
F
d'V^d/k
Â°ext
with the single check point (1/3, 1/3, 1/3) where d =
a *
Y* ~ Yl' Y1 tlie observed response, Y* is the response
71
predicted by the second order fitted model at (1/3, 1/3,
1/3), and V0 = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(Xâ€™X)_1
(1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of
the F ratio exceeds F , where v equals the number of
a ; 1, v
*â– o
degrees of freedom associated with
ext
null hypothesis that the second order model is the true
model in favor of the alternative hypothesis that the
special cubic model is the true model. Equivalently, we
reject Hn: X. = 0 in favor of H : X , > 0. For k = 2 or
k = 3 check points, the value of the F ratio is calculated
using the observed and predicted responses at the two or
three replicates at the centroid. The hypothesis
H_: X. = 0 is rejected in favor of H : X, > 0 if F
0 1 J a 1
exceeds F
a ; k ,v
Example 2. In Example 2 we illustrate the second of
the four cases that could arise when MSE is used as an
estimate of in the lack of fit test statistic (see
Section 3.3.4). We again fit a second order canonical
polynomial model in three mixture components, and assume the
true model is special cubic. The design to be used is the
q = 3 simplex centroid design, which consists of seven
design points, and is illustrated in Figure 2 of Chapter 1.
There are six parameters to be estimated and seven
design points hence one degree of freedom can be used to
calculate MSE. We shall use MSE to estimate aOptimal
upper and lower tailed test check points must be located,
and then a decision is made as to which test should
72
be used. The actual testing for lack of fit involves the F
statistic in (3.8).
As in Example 1, P2 = 1, since there is one term in the
true model in addition to those in the fitted model. Thus
is a scalar whose value we seek to optimize with respect
to the desired number of check points, k. When only a
single check point is sought for the purpose of testing lack
of fit, the Controlled Random Search Procedure has two
functions. First, the procedure is used to locate the
optimal candidate check point for an upper tailed test by
locating the check point that maximizes the scalar A^.
Secondly, the procedure is used to locate the optimal
candidate check point for a lower tailed test, which is
accomplished by locating the point that minimizes A^. The
quantity R = [A^/k]  [A2/(N  p)] is then calculated to
determine whether the upper or lower tailed test will be
used. If R is positive for the candidate check point for an
upper tailed test, then the test is upper tailed, and the
test is lower tailed if the candidate check point for a
lower tailed test produces a negative value for R. Note
that A2 = (X2  XA)'(X2  XA) is fixed once the design is
specified, since A2 does not depend on the check points.
Using the Controlled Random Search Procedure it is found
that the maximum value of A^ occurs at (xj, xij, x^) = (1/3,
1/3, 1/3), which will be the location for the check point
for the upper tailed test. Calculating A^ at this centroid
73
point, we find that R = [A^/k]  [A2/(N  p)] = [(3.7258
x 10_4)/1]  [(8.4175 x 10_4)/1] = 4.6917 x 10~4. Since R
is negative, the test is not upper tailed.
Using the Controlled Random Search Procedure to
minimize A^, we find that a subregion of the factor space
exists in which all points yield a near minimum value for
A^. We choose the point (0.0189, 0.9269 , 0.0542) at random
from this subregion to be used as the optimal candidate for
a lower tailed test. Here R = 0  [(8.4175 x 104)/l] =
8.4175 x 104.
Since R is negative for both the optimal upper tailed
test check point and for the optimal lower tailed test check
point, we have Case 2 of Section 3.3.4. The upper tailed
test check point is disregarded, and the lower tailed test
check point (0.0189, 0.9269, 0.0542) is used to test for
lack of fit. If the calculated F ratio,
F
MSE
is less than F
then Hq: X^ = X^ = 0 is rejected in
(1 a ) ; 1,1
favor of H : [x /l]  [X /l] < 0, that is we conclude that
the second order model exhibits lack of fit, and the true
model is special cubic.
When two simultaneous check points are desired for
testing lack of fit, we can again use the Controlled Random
Search Procedure to locate the optimal settings. To
maximize the scalar A^, we find that both check points
74
should be selected at (1/3, 1/3, 1/3), for an upper tailed
test. With our calculations R = [(5.8275 x 104)/2) 
[(8.4175 x 104)/l] = 5.5038 x 10â€œ4, but since R is
negative, the test is not upper tailed.
Minimizing A^ to locate two optimal lower tailed test
check points yields a subregion in the factor space of
optimal check points. The pair of check points (0.3749,
0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at
random from this subregion, and these check points yield
R = 0  [(8.4175 x 10_4)/1] = 8.4175 x 104.
Since R is negative for the upper tailed test points
and the lower tailed test points, we have Case 2 of Section
3.3.4 again and the lower tailed test check points are used
to test for lack of fit. The hypothesis : X^ = X^ = 0 is
rejected in favor of H : [x /2]  [x /l] < 0 if the cal
culated value of F = (d'Vg^d/2)/MSE is less than
F,, . _ ,, in which case we say lack of fit of the model is
(1a ) ; 2,1
present.
a 2
If an external estimate a . had been available for
ext
this example, then the optimal upper tailed test check
points could have been used in the F ratio,
1 â– Â» 2
F = (d'Vg d/k)/aext, an<^ lack Â°f fit would then be detected
if the calculated value of F exceeded F .
a ;k, v
Example 3. Example 3 illustrates the procedure for
locating optimal check points when there are two terms in
the true model in addition to those in the fitted model. A
second order canonical polynomial model in three mixture
75
components is fitted using a q = 3 simplex centroid design.
The true model is assumed to contain eight terms, six of
which are the same terms as in the fitted model, with the
additional two terms being the third order terms
512X1X2^X1 â€œ X2^ and ^123X1X2X3* ^ ExaraPle 2' there is
one degree of freedom for MSE which is used to estimate
a2. The test statistic, F = (dâ€™Vg^d/k)/MSE, is given in
equation (3.8).
Since P2 = 2 and A^ is a 2*2 matrix, locating the
optimal upper tailed test check points by the procedure of
maximizing X^ is assisted by the maximizing of a lower bound
2
for 11, namely maximizing p . Bâ€™&_/2a , where P . is the
1 2 ^ min22/ mm
2
smallest eigenvalue of A]_. Since Ã©$2 and 0 are unknown,
this is equivalent to maximizing P . . For P . to exceed
mm mm
zero, it is necessary that A^ be of full rank, and since
rank(Aj,) < min(k, p2), it is necessary to select k > 2 check
points. If Aj_ is less than full rank, and thus is positive
semidefinite, only a subset of possible values of could
be considered to make it possible to test for lack of fit
with an upper tailed test.
Using the Controlled Random Search Procedure, the
points that maximize ymin are found to be (0.418, 0.277,
0.305) and (0.277, 0.418, 0.305). These points are thus
optimal candidates for upper tailed test check points. At
these check points we have p . = 5.1623 * 104, A, =
mm I
diag[5.1623 * 104, 5.1916 x 104], A2 = diag[0, 8.4175 *
10 4], and R = [A,/2]  [A2/l] = diag[2.5811 * 104, 5.8217
4
76
x 104] . Since the eigenvalues of R are 5.8217 x 104 and
2.5811 x 104, R is indefinite. Following the suggested
procedure for Case 4 of Section 3.3.4, we note that an upper
tailed test for lack of fit exists if the value of
2 = [5i2' ^123^' in the column space of [si:S2l but not
entirely in the column space of S2, where Sj^ is the matrix
whose columns are the orthonormalized eigenvectors of R
corresponding to the positive eigenvalues of R, and S2 is
the matrix whose columns are the orthonormalized
eigenvectors of R corresponding to the zero eigenvalues of
R. Since R has no zero eigenvalues in this example, S2 does
not exist, but S^ is the column vector, S^ = [1,0]'. Thus
if &2 is Â°f tbe foâ„¢ &2 = ^ ' â€™ wbere ^^2 * tben
82 is in the column space of Sj^ and the test is upper
tailed .
The matrix A2 has rank one and therefore is positive
semidefinite. Hence it is impossible to locate two check
points that minimize iJmax and also make R = [Aj^/2]  [A2/l]
negative definite (see Appendix 4), that is, it is
impossible to find a lower tailed test that is capable of
testing lack of fit for all values of However, if we
use the Controlled Random Search Procedure to locate two
check points that minimize an upper bound for X which is
y 3l3â€ž/2a2, then by minimizing y , we find that any of
the check points in a particular subregion of the factor
space yield a near minimum for y . One pair of points in
1 max
this subregion is selected as the points to be used as
77
optimal lower tailed test check points, namely the pair
consisting of the point (0.053, 0, 0.947) replicated twice.
Replicating this check point, we find umax = 7.3900
x 10â€**, A^ = diag[0, 7.3900 x 10â€œ^], A2 = diag[0, 8.4175 x
104], and R = [AÂ±/2]  [A2/l] = diag[0, 8.4175 x 104].
The eigenvalues of R are 0 and 8.4175 x 104 implying that
R is negative semidefinite. The values of ^ that are in
the column space of [S2: S3] but not entirely in the column
space of S2 will provide a lower tailed test. Here, [S2:S3]
= diag[l,l] and S2 = [1,0]'. Thus, the test for lack of fit
is lower tailed if 3122 * 0.
For values of that produce an upper tailed test we
use the check points (0.418, 0.277, 0.305) and (0.277,
0.418, 0.305) with the F ratio
dâ€™VQ1d/2
F =
MSE
and conclude there is lack of fit if the calculated value of
F exceeds F .0 ,. For values of g~ that produce a lower
tailed test, we use two replicates of the check point
(0.053, 0, 0.947), and'conclude there is lack of fit if F is
less than F . _ ., where again F is calculated by
v 1 a ) ; / , 1
F = (d'V~1d/2)/MSE.
Example 4. Example 4 illustrates Case 3 of Section
3.3.4 in which MSE is used to estimate in the lack of fit
test statistic. A second order canonical polynomial model
78
in three mixture components is fitted using the {3/3}
simplex lattice design, which appears in Figure 5. The true
model is assumed to be special cubic, thus p2 = 1 and A^ is
a scalar. The {3,3} design consists of ten design points
and since there are six parameters to be estimated in the
fitted model, a2 can be estimated by MSE with N  p = 10  6
= 4 degrees of freedom.
We first suppose that a single check point is to be
used to test for lack of fit. Using the Controlled Random
Search Procedure we find the single check point that
maximizes the scalar
A1 = (X*  X*A)'V~1(X*  X*A)
is located at the centroid of the simplex factor space.
Thus (x*, x*, x*) = (1/3, 1/3, 1/3) is the optimal candidate
for an upper tailed test check point. At this centroid
point, A^ = 4.9076 * 104. For the {3,3} design the scalar
quantity A2 = (X2  XA)'(X2  XA) is fixed and is equal to
A2 = 9.4062 x 10"4 and thus, R = [Aj/k]  [A2/(N  p)] =
[(4.9076 x 10~4)/1]  [(9.4062 x 10â€œ4)/4] = 2.5560 x 104.
The point that is the optimal candidate for a lower
tailed test check point is chosen randomly from a subregion
of points in the factor space, in which all points minimize
A^. The point selected has the value (x*, xÂ£, x^) = (0.560,
0.410, 0.030). Here Aj. = 9.6590 x 107 and R = [(9.6590 x
10_7)/1]  [(9.4062 x 104)/4] = 2.3419 x 10~4.
79
V
(0,0,1)
Figure 5. The {3,3} simplex lattice design
Since R is positive for the optimal upper tailed test
check point (1/3, 1/3, 1/3) and R is negative for the
optimal lower tailed test check point (0.560, 0.410, 0.030)
we are in Case 3 of Section 3.3.4. Either the upper or
lower tailed test could be used to test for lack of fit, but
2
if the quantity &22^Â° can be specified, then we will
choose to use the test that has greater minimum power, since
greater power means that we are more likely to detect lack
of fit when in fact lack of fit exists. In this example
0
123*
For illustrative purposes, we arbitrarily choose
2
022//CT = 2000, so that an approximate conservative lower
bound for the power of the upper tailed test is found by
80
calculating
P I F" > F 1
kfNp;X1L,X2u a;k,Np
where F^^ N_p is the upper 100a percentage point of the
central F distribution, k is the number of check points, N
is the total number of response observations, p is the
number of parameters in the fitted model,
A1L ' Mmin22^Â°2' and A2U = {max^2/2Â° 2 The
quantity wm^n is the smallest eigenvalue of A^, where is
evaluated at the optimal upper tailed test check point.
Since A, is a scalar, u . = A.. Likewise, 6 is the
1 mm 1 max
largest eigenvalue of A2, an<3 since in this example A2 is a
scalar, 6 = A.. In this example we have k = 1, N  p =
max 2 c
10  6 = 4, x.T = 3â€™30/2a2 = ( 4.9076 x 10"4 )( 2000/2)
= 4.9076 X 101, and X2â€ž = Â«max8Â¿S2/2a2
= (9.4062 x 10"4)(2000/2) = 9.4062 x 10_1. Using the
approximation to the cumulative probabilities of the doubly
noncentral F distribution given by Johnson and Kotz (1970,
p.197) which is described in Appendix 6, and taking a = .05,
we find that a conservative lower bound for the power of the
optimal upper tailed test is approximately equal to .0649.
The minimum power for the optimal lower tailed test is
2
approximated (assuming = ^000) by calculating
p ( F" <â– F 1
1 k,Np;XltI,X2L ( 1a ) ;k,Np; *
81
2
The quantities Alr, and A are taken as A1n = y 0'8o/2o
JL u 2 ij jl u ma x ~ z â„¢ z
2
and A or =6 . g'80/2o where y is the largest eigenvalue
Zi^ mmzz max
of Aj_ with A^ calculated using the optimal lower tailed test
check point, and where Ã³ . is the smallest eigenvalue of
mm 3
a2 â€¢
Since A^ and A2
are scalars
' umax
A1
and
Ã³ . = A .
min 2
In
this example, k =
1, N  p =
4,
7
4
X 1U
= (9.6590 x 10
)(2000/2) =
9.6590 x
10
t
and
X 2L
= (9.4062 x 10"4
)(2000/2) =
9.4062 x
10
1
â€¢
Again if the
approximation to the doubly noncentral F distribution given
in Johnson and Kotz is used, an approximate conservative
lower bound for the power of the optimal lower tailed test
is .0555.
2
Having specified B2B2/o = the optimal upper
tailed test is chosen over the optimal lower tailed test,
because the approximate minimum power of the upper tailed
test is greater than the approximate minimum power of the
lower tailed test. Using the optimal upper tailed test
check point (1/3, 1/3, 1/3) in the test statistic
MSE
we conclude that lack of fit is significant if the
calculated value of F exceeds F , ., in which case we
a ; 1,4
reject HQ: Aj. = A2 = 0 in favor of Ha : A j/1  A 2/4 > 0.
When two simultaneous check points are used for testing
lack of fit, the Controlled Random Search Procedure locates
the optimal upper tailed test and optimal lower tailed test
82
check points. It turns out that two replicates at (1/3,
1/3, 1/3) maximize A^, and are used as optimal check points
for an upper tailed test. The value of R = [Aj^/2]  [A2/4]
is [(7.9210 x 10 â€” 4 )/2]  [(9.4062 x 10"4)/4] = 1.6090 x
io4.
In searching for two optimal lower tailed test check
points, again a subregion of the factor space is found in
which any of the points nearly minimize A^. From this
subregion are chosen the points (0.6386, 0.3263, 0.0351) and
(0.7257, 0.2421, 0.0322) resulting in a value of R = [A^/2]
 [A2/4] of [(1.5216 x 10â€œ9 )/2]  [(9.4062 x 104 )/4] =
2.3516 x 10"4.
In conclusion, when two simultaneous check points are
used in the test for lack of fit in this example, R is
positive for the optimal upper tailed test and R is negative
for the optimal lower tailed test, and we have Case 3 of
2
Section 3.3.4. Selecting 3 23_2/a = 2000 arbitrarily, we
found the approximate lower bound for the power of the upper
tailed test to be .0504, and the approximate lower bound for
the power of the lower tailed test to be .0612. Since the
power is higher with the lower tailed test it is our choice
for testing lack of fit when two check points are used
simultaneously. Lack of fit is detected and we reject
H : X = X = 0 in favor of H : [X /2]  [X /4] < 0 if the F
U J. d 1 /
ratio, F = (d' V~'*'d/2 )/MSE, using the optimal lower tailed
test check points (0.6386, 0.3263, 0.0351) and (0.7257,
0.2421, 0.0322) is calculated to be less than F
(1a);2,4 *
83
3.4.2 Numerical Examples
Numerical Example 1. In this example we illustrate
numerically some of the findings in the first theoretical
example of Section 3.4.1. Data that were collected in a
rocket fuel experiment (Kurotori, 1966) will be used to
investigate the power of the lack of fit F test. The test
is set up to detect the inadequacy of a fitted second order
canonical polynomial model when the true model is special
cubic. Calculated values of the power of the test which
detects lack of fit through large values of
dâ€¢vâ€œ1d/k
F = ~
aext
will be compared for several check point locations, includÂ¬
ing the location (1/3, 1/3, 1/3) at which the power was
found to be maximum in Example 1 of Section 3.4.1.
In Kurotori's experiment the modulus of elasticity (Y)
of a rocket fuel is expressed as a function of the
proportions of three componentsâ€”binder (x^), oxidizer (x2),
and fuel (x2). Since lower bounds are placed on the
component proportions x^, x2, and x2, in the form of
0.20 < x^, 0.40 < x^/ and 0.20 < x^, pseudocomponents (x)
are defined in terms of the original components in the form
of xj =  0.20 )/( 1  .80 ), x^ = (x  0 40)/(1  .80),
and x^ = (x^  0.20)/(1  .80). The true special cubic
model in the pseudocomponents, which is obtained by using
the data at the seven points of the simplex centroid design
84
in the pseudocomponent system, is
E(Y) = 2 3 5 0 x' + 2450x' + 2650x' + Ox'xl
^ Ãº X c.
+ lOOOxjx^ + 1600x ^x ^ + 6150x^x^x^.
The second order canonical polynomial model that is fitted
to the six boundary points only, and which will be tested
for lack of fit, is given by
Y = 23 50x + 2450x2 + 2650x^
+ lOOOxJx^ + 1600xJ,x^.
The configuration of the experimental design as well as the
check point locations are depicted in Figure 4 of Chapter 2
and the observed response values are given in Table 3 of
this chapter.
1 ~ 2
A value of the ratio F = [d'V_ d]/a is calculated at
0  ' ext
each of the four individual check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3)
* 2 ~ 2
where a . is assumed to have the value a ^ = 144 as
ext ext
suggested by Kurotori (1966). We also assume without loss
of generality that the degrees of freedom associated
* 2
with Â°ext are v = 10. The power of the F test is calculated
at each of the four check points by using the approximation
to the cumulative probabilities of the noncentral F
85
Table 3. Observed Response Values at the
Pseudocomponent Settings for Kurotori's Rocket Fuel
Experimentâ€”Numerical Example 1.
Observation Binder Oxidizer Fuel Modulus of Elasticity
Number x x^ x^ Y
1
1
0
0
2350
2
0
1
0
2450
3
0
0
1
2650
4
1/2
1/2
0
2400
5
1/2
0
1/2
2750
6
0
1/2
1/2
2950
7*
1/3
1/3
1/3
3000
8*
2/3
1/6
1/6
2690
9*
1/6
2/3
1/6
2770
10*
1/6
1/6
2/3
2980
* Check
Points .
distribution given by Johnson and Kotz (1970, p. 197) to
evaluate
Power = PÃ F' , n , > F nc ,
1 1,10;A! .05;1,10J
where A. = a . . The value of
1 1 123 ext
Al= ^X2 " x*a)'vo"^^X2 â€œ X*A) is calculated for each check
point using the values of X*, X*, and the value of
A = (x'X) "*"X'X2 which is fixed by the {3,2} simplex lattice
design. Since the {3,2} simplex lattice consists of points
86
only on the boundaries of the triangle (and therefore at
each point at least one of the x values is equal to zero),
then X2 = 0 and A = 0. From the true special cubic model,
6123 = 6150.
The calculated value of F as well as the approximate
value for the power at each of the four check points is
given in Table 4. The check point (1/3, 1/3, 1/3) produced
the highest power of the four check points investigated,
supporting the previous results of Example 1 in Section
3.4.1 where (1/3, 1/3, 1/3) was selected as the check point
location with the maximum power when a second order
canonical polynomial was fitted using the {3,2} simplex
lattice design, but the true model was assumed to be special
cubic. Additional support for the point (1/3, 1/3, 1/3)
being optimal is given by the contour plot of values of A^
in Figure 6(d). The highest values of appear near the
centroid (1/3, 1/3, 1/3) where high A^ values translate into
2 2
high X values, since X^ = A^^^/2a , which in turn implies
high power since we know the power is an increasing function
of X !.
As a second part of this example the power of the F
test that is obtained when three replicates are taken at
(1/3, 1/3, 1/3) is compared to the power of the F test that
is obtained when one replicate is taken at the three check
points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6,
2/3). These latter three point locations were suggested by
Kurotori for testing lack of fit of his fitted special
Table 4. Effect of Check Point Location on the Power
of the Lack of Fit F Testâ€”Numerical Example 1.
Check Point**
Numerator
d.f. (k)
Denominator
d.f. (v)
A1
A1
F
Power
(1/3,1/3,1/3)
1
10
.00084
110.32
221.1*
1.000
(2/3,1/6,1/6)
1
10
.00023
30.21
65.0*
.999
(1/6,2/3,1/6)
1
10
.00023
30.21
44.8*
. 999
(1/6,1/6,2/3)
1
10
.00023
30.21
72.6*
.999
(1/3,1/3,1/3)
(1/3,1/3,1/3)
(1/3,1/3,1/3)
3
10
.00142
186.49
1.000
(2/3,1/6,1/6)
(1/6,2/3,1/6)
(1/6,1/6,2/3)
3
10
.00051
66.98
45.3*
.999
**Check point coordinates are in the pseudocomponent system.
â€”F ratio cannot be calculated since only one response observation was collected
at (1/3,1/3,1/3).
88
cubic model. The result of this comparison, see Table 4, is
that the three replicates at (1/3, 1/3, 1/3) produce the
test with greater power which again supports the findings of
Example 1 of Section 3.4.1.
All of the check point locations listed in Table 4
produce very high power values (> .999) which is due in part
to the high value of $123 (3123 = 615Â°)Â« If 3123 were of
lower magnitude, then the three replicates at (1/3, 1/3,
1/3) would show a still greater superiority in the power
value compared to the power using the other check points.
This superiority is demonstrated in Table 5 where values of
8123 are listed as 3000 and 1500 and the comparative power
values are listed as 0.998 compared to 0.795 and 0.662
compared to 0.249, respectively. Table 5 also demonstrates
the superior power value for the point (1/3, 1/3, 1/3) when
&123 = 3000 or 8^3 = 150Â° and each of the four check points
is used one at a time.
Finally, (1/3, 1/3, 1/3) being the optimal check point
location is seen in Figure 6(c), where contour plots of the
expected difference in the heights of the surfaces are
drawn. The differences in the heights are found by
subtracting the estimated height of the surface obtained
with the fitted second order model from the estimated height
of the surface obtained with the true special cubic model.
The expected difference between the true and fitted surfaces
approaches a maximum the closer one moves to the centroid of
the simplex factor space, so that the optimal check point
on the Power
Table 5. Effect of the Magnitude of 3^
of the Lack of Fit F Testâ€”Numerical Example 1.
3 123
= 6150
3123
= 3000
3 123
= 1500
Check Points**
X1
Power
X1
Power
X1
Power
(1/3,1/3/1/3)
110.32
1.000
26.25
. 998
6. 56
. 638
(2/3,1/6,1/6)
30.21
.999
7.19
.680
1.80
. 220
(1/6,2/3,1/6)
30.21
. 999
7.19
. 680
1.80
. 220
(1/6,1/6,2/3)
30.21
.999
7.19
.680
1.80
.220
(1/3,1/3,1/3)
(1/3,1/3,1/3)
(1/3,1/3,1/3)
186.49
1.000
44.38
.998
11.09
.662
(2/3,1/6,1/6)
(1/6,2/3,1/6)
(1/6,1/6,2/3)
66.98
.999
15.94
.795
3.98
.249
**Check point coordinates are in the pseudocomponent system.
CD
CD
90
(c) Expected difference between the
true special cubic surface and
the fitted second order surface.
Figure 6.
Contour plots for Numerical Example 1.
91
location (1/3, 1/3, 1/3) coincides with the point where the
expected difference between the true special cubic surface
and the fitted second order surface is maximum.
Numerical Example 2. In this second numerical example,
we investigate the power of the F test for detecting lack of
fit when a second order canonical polynomial model is fitted
in a mixture system which is in truth represented by a
special cubic model. The true model is assumed to be
E(Y) = 2350x1 + 2450x2 + 2650x3
+ lOOOx^x^ + leOOx^x^ + 6150x^x2x3
which is used to generate hypothetical response observations
at the seven points of the q = 3 simplex centroid design as
well as at three check points. The values of the response
are obtained by adding the value of a pseudorandom normal
variate with mean 0 and variance 144 to each true predicted
response value. The data are given in Table 6.
The response values at the seven points of the simplex
centroid design are used in the least squares normal
equations to obtain the fitted second order model
Y = 234lx^ + 2438x2 + 2630x3
+ 310x.xâ€ž + 1304x,x_ + 1970x_x_
92
Table 6. Generated Response Valuesâ€”Numerical Example 2.
X1
x 2
x3
Y
â€”
'
â€”
â€”
1
0
0
2357
0
1
0
2454
0
0
1
2646
1/2
1/2
0
2403
1/2
0
1/2
2747
0
1/2
1/2
2962
1/3
1/3
1/3
3013
* 1/3
1/3
1/3
2993
* 2/3
1/6
1/6
2693
* . 02
*Check points.
.93
. 05
2550
which is to be tested for lack of fit using the test
statistic F = d1VQ^d/MSE. The F statistic will be evaluated
at each of the three check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time,
and the power of the test at the three check point locations
will be calculated and compared. The test is lower tailed
for all check point locations (since R = A^  A2 is negative
for all check point locations) and thus the power is defined
as
F1,1;X1 ,X 2 < F. 95 ; 1,1^ *
93
X =1
I
Figure 7. Contours of R = A^  A2 for Numerical Example 2.
2 2 2 2
The values of X ^ = &i23Al//2a and X2 = g123A2//2Â° are found
2
by taking 3123 = 6150 and 0 = 144â€¢ The results of this
power investigation are given in Table 7.
Since the check point (.02, .93, .05) produces the
greatest power of the three check points investigated, this
supports the result in Example 2 of Section 3.4.1, where we
saw that the point (.02, .93, .05) yielded the maximum power
of all points for detecting lack of fit of a fitted second
order canonical polynomial model, using the q = 3 simplex
centroid design in the presence of a true special cubic
surface. Additional evidence for the point (.02, .93, .05)
being an optimal check point is shown in Figure 7, where
contours of the values of R = A^  A2 are presented. The
point (.02, .93, .05) is seen to belong to an area of the
Table 7. Effect of Check Point Location on the Power
of the Lack of Fit F Testâ€”Numerical Example 2.
Numerator Denominator
Check Point
d.f.(k)
d.f.(N 
P) Ax
A2
R
X1
A 2
F Power**
(1/3,1/3,1/3)
1
1
3.70x104
.00084
.00047
48.59
110.3
.333 .000
(2/3,1/6,1/6)
1
1
5.20x105
.00084
.00079
6. 84
110.3
.063 .012
(.02,.93,.05)
1
1
8.12x109
.00084
.00084
.001
110.3
.0005* .590
*Significant
at a =
.05 (F < F
â€¢
95;1,1,}*
**Power is approximated using an approximation found in Johnson and Kotz (1970, p.197).
95
simplex factor space where R is minimum, which implies that
Aj_ (and in turn \ is also minimum in this area, since R =
A1 ~ a2 an(^ a2 ^as t^ie fixec^ value of A2 = .00084 for the
simplex centroid design. Thus the check point (.02, .93,
.05) produces a minimal A ^ value and maximum power, since
the power increases with decreasing values of A^.
3.5 Discussion
When check points are used for testing lack of fit in a
mixture model, the appropriate testing procedure, assuming a
normally distributed response, involves an F statistic. If
" 2
an external estimate, o , of the experimental error
variance is available so that the test statistic is given by
dâ€™V^d/k
F = ^2
Â°ext
then the power of the test for lack of fit is maximized by
choosing k check points that maximize the value of the nonÂ¬
centrality parameter A ^. When p2 = 1, maximizing A^ is
achieved without knowing the value of the elements of by
selecting check points that maximize the scalar Aj_. When
P2 > 1, the maximization of A^ is approximated by maximizing
a lower bound for A^. This is achieved also without knowing
the values of the elements of g^ by selecting check points
that maximize the smallest eigenvalue of the matrix A^. The
test is upper tailed, and for given values of Aj_,
the actual
96
power of the test can be calculated from the cumulative
probabilities of the noncentral F distribution. A problem
arises when A^ is positive semidefinite and its smallest
eigenvalue is equal to zero. In this case check points that
maximize the smallest positive eigenvalue of are
selected, and lack of fit is only detectable for a subset of
the possible values of the elements of & .
When an external estimate of a2 is not available,
testing lack of fit at the check points is further
complicated. The F statistic is
(d'V^d/k)
F =
MSE
and the rejection region for the lack of fit test can be
upper tailed or lower tailed. The power of the test is
determined by using the doubly noncentral F distribution,
which depends on the parameters k, N  p, A^, and A 2* Of
these four parameters, only k and A^ are influenced by check
points, and if the value of k is fixed, the power of the
test is maximized by choosing check points that affect the
value of A^. Regardless of the values of the elements of
Â£S_2, check points that maximize Aj_ are selected for
maximizing the power of an upper tailed test, and check
points that minimize A ^ are selected for maximizing the
power of a lower tailed test.
97
Lack of fit can be tested with the upper tailed test
for all nonzero values of the elements of g 0 if the check
points are selected so that [A^/k]  [A2/(N  p)] is
positive definite, since this forces the expected value of
the numerator mean square in the F ratio to be greater than
the expected value of the denominator mean square.
Similarly, lack of fit can be tested with a lower tailed
test if check points are selected which make [A^ /k] 
[A2/(N  p)] negative definite. When it is not possible to
select check points that make [A^/k]  [A2/(N  p)] either
positive definite or negative definite then detection of
lack of fit is only possible for a subset of all nonzero
values of the elements of
The power of the test for lack of fit using the F
statistic in (3.8) is a function of X ^ and X2. Since the
magnitudes of X^ and X2 are influenced by the experimental
design, an area for future study is the investigation of the
effect of the experimental design on the power of the lack
of fit test. In the presence of an external estimate of
2
a , Atkinson (1972) suggested designs that maximize the
determinant of A2, A^Â¡, when lack of fit is to be detected
by a large value of X^ using a procedure which is, in
general, equivalent to the lack of fit testing procedure
that partitions the residual sum of squares into pure error
and lack of fit sums of squares. It might be useful to
apply Atkinson's (1972) methodology not only to A2, but
to Â¡A1 or jA^/k  A2/(N  p) in order to find an
98
appropriate design when testing lack of fit with the F ratio
in Eq. (3.8). Since the power of the test in Eq. (3.8) is
also affected by the values of k and N  p, which are the
numerator and denominator degrees of freedom of the doubly
noncentral F distribution, respectively, optimal settings
for these parameters can also be considered. For a given
fitted model, p is fixed so that the degrees of freedom
would be influenced by the number of check points selected,
k, and by N, the total number of observations. Finally, the
experimental design and the number of check points also
affect the power of the F test when o is used to
t A L
2
estimate a . Thus the effect of the experimental design and
the number of check points can also be investigated for the
situation where the lack of fit test statistic is given by
Eq. (3.3).
We now conclude our investigation of the check point
approach to lack of fit testing and in the next chapter turn
to an investigation of a near neighbor method for testing
lack of fit.
CHAPTER FOUR
USE OF NEAR NEIGHBOR OBSERVATIONS
FOR TESTING LACK OF FIT
4.1 Introduction
In an experiment in which replicate response
observations are available at one or more design points,
lack, of fit of a fitted model can be tested by a procedure
which involves partitioning the residual sum of squares into
two statistically independent portions. One portion is the
sum of squares due to lack of fit (SSL0F), and the second
portion is the sum of squares due to pure error (SSEpUre)
obtained from the replicates. As discussed in Section 2.2,
this procedure was suggested by Draper and Smith (1981,
p.120). Lack of fit is inferred if the calculated value of
the ratio
F
MS
LOF
MSE
pure
(4.1)
exceeds the corresponding upper 100a percentage point of the
central F distribution, where MSLqF and MSEpure are the mean
square values found by dividing SSLOf and SSEpure their
respective degrees of freedom.
In order to test the fitted model for lack of fit when
replicate observations are not available, Shillington (1979)
suggested a procedure which uses observed response values
99
100
collected at points which are "near neighbors" in the factor
space in place of replicates (see Section 2.3). Lack of fit
is inferred when the calculated value of the ratio
MSEb
MSEw
(4.2)
exceeds the upper 100a percentage point of the central F
distribution. The numerator, MSEB, of the F ratio in Eq.
(4.2) is a generalization of the numerator, MSLqF, of the F
ratio in Eq. (4.1). The form of MSEg will be given in Eq.
(4.5) of Section 4.3. The denominator, MSEW, in Eq. (4.2)
is a generalization of MSEpure in Eq. (4.1), and the value
of MSEW is calculated using near neighbor observations in
place of replicates (see Eq. (4.6) in Section 4.3).
Shillington's near neighbor method provides an
alternative to the check points method when replicate
observations are not available. Typically, near neighbors
might appear either because an experiment was not designed
to provide replicate observations or with a designed
experiment consisting of a large number of design points in
a bounded factor space which results in some points being
near one another.
In this chapter we shall further study Shillington' s
(1979) near neighbor procedure for testing lack of fit. A
question involving the correctness of the ordinary least
squares technique suggested by Shillington for deriving the
denominator, MSEW, of the F ratio in Eq. (4.2) will
101
be raised. The question will be resolved by showing the
equivalence of deriving MSEW by ordinary least squares and
of deriving MSEW by a generalization of weighted least
squares. We will verify that when the observable response
values are assumed to have the normal distribution with
2
homogeneous variance, a , the F ratio in Eq. (4.2) possesses
a central F distribution when the fitted model is adequate,
but possesses a doubly noncentral F distribution when the
fitted model suffers from lack of fit. We shall also show
that the F test for lack of fit which uses the statistic in
Eq. (4.2) can have either an upper tailed or a lower tailed
rejection region. Finally, the use of a clustering
algorithm for defining groups of near neighbors will be
proposed.
4.2 Notation
In this section we introduce the notation to be used in
this chapter. Throughout our investigation of Shillington's
near neighbor procedure for testing lack of fit, we shall
assume the observed response values collected in an
experiment can be grouped into g cells where the jth cell
contains nj observations, j =1, 2, ..., g. The
observations in a cell are from points that are "near
neighbors" in the sense that they are near one another in
the factor (mixture) space. A model of the form
E(Y)
(4.3)
102
is fitted to the data using ordinary least squares, but the
true model is assumed to have the form
E(Y) = XB1 + X202, (4.4)
where Y is an Nxl vector of response values observable at
the design points with var(Y) = a IN, X and X2 are Nxp and
Nxp2 matrices of known constants, respectively, and 0^ and
02 are P*1 and p2xl vectors of unknown regression
coefficients, respectively. The vector Y is assumed to have
the normal distribution.
Let us now define the following vector and matrix
quantities to be used in developing the numerator, MSEB, of
the F ratio in Eq. (4.2):
Yc = a gx1 vector with jth element equal to the
average of the nj observed response values in the
jth cell of near neighbor observations, j = 1, 2,
â€¢ â€¢ â€¢, g .
X( = a gxp matrix whose jth row is the average of the
nj rows of X corresponding to the jth cell, j =
1, 2, ..., g.
X2c = a gxp2 matrix whose jth row is the average of the
nj rows of X2 corresponding to the jth cell, j =
1/ 2,
â€¢ â€¢ â€¢ , g.
103
Gq = a gxg diagonal matrix of the form
Gq = diag [1/n^, l/n2, 1/n ] .
To further illustrate the forms of Y^, Xq, X2C1 an<^ G0'
we present the following numerical example. Consider a data
set consisting of response observations (Y) taken at N = 8
different combinations of the settings of the factors x^ and
x2, where the eight response observations are divided into
g = 4 near neighbor cells. The vector of observed response
values, Y, and the matrix X corresponding to the first order
model, E(Y) = 0 + 0.x + 0 x , are
10
1
1
2
13
1
2
5
16
1
2
4
15
X =
1
3
2
18
1
3
1
21
1
4
2
27
T~
5
5
30
1
5
4
â€”
â€”
The horizontal lines in Y and X delineate the four cells of
near neighbors. In this example
10
10
(13 + 16)/2
=
14.5
(15 + 18 + 21)/3
18
(27 + 30)/2
28.5
â€” _
_.
104
X
c
1
(1 + 1)/2
(l+l+D/3
(1 + 1)/2
(2 + 2 )/2
(3+3+4)/3
(5 + 5 )/2
(5 + 4)/2
(2+1+2)/3
(5 + 4)/2
1
1
1
1
1 2
2 4.5
3.3 1.7
5 4.5
and
G0 = diag(1, 1/2, 1/3, 1/2).
If the true model is second degree, E(Y) =
2 2
+ 2ixi + g2X2 + ^ 12X1X 2 + ^11X1 + ^ 22X 2' then the x2 and
X2C matrices have three columns corresponding to the
2 2
terms x^x2/ x^, and x2, respectively. For this numerical
example we have
X
2
and
2
10
8
6
3
8
25
20
X2C
2
9
5.7
22.5
1
4
4
9
9
16
25
25
1
4
11.3
25
4
25
16
4
1
4
25
16
4
20.5
3
20.5
105
Next let us define the following quantities to be used
in developing the denominator, MSEW, of the F ratio in Eq.
(4.2) :
W = an Nxl vector of within cell deviations where the
ith element, , of W is equal to the difference
between the ith element, Y^r of Y and the average
of the response values observed in the near
neighbor cell containing Yj_, i = 1, 2, ..., N.
Xw = an Nxp matrix whose ith row is equal to the ith
row of the X matrix minus the row of Xc
corresponding to the cell containing the response
value observed at the ith row of X.
r = rank(Xw).
X2W = an NxP2 matrix whose ith row is equal to the ith
row of the X2 matrix minus the row of ^2C
corresponding to the cell containing the response
value observed at the ith row of X.
Eg = an NxN idempotent matrix of the form
Eq = â€” diag [ (1/n ^ , (l/n2)J2r â€˜Â«Â«rÃl/n) Jg ]
where Jj is an njxnj matrix of ones, and IN is
9
the identity matrix of order NxN, with N = E n..
j=l D
Let us illustrate the forms of W, X^Â¡r %2VI' anc^ Â£ q
using the numerical example presented earlier in this
section, where the eight response observations were
distributed among four cells. For these data we have
106
â€” 
10  10
0
13  29/2
1.5
16  29/2
1.5
15  18
=
3
18  18
0
21  18
3
27  57/2
1.5
30  57/2
1.5
1 
1
1 
1
2 
2
0
0
0
1 
1
2 
2
5 
9/2
0
0
. 5
1 
1
2 
2
4 
9/2
0
0
.5
1 
1
3 
10/3
2 
5/3
=
0
.3
.3
1 
1
3 
10/3
1 
5/3
0
.3
.7
1 
1
4 
10/3
2 
5/3
0
.7
.3
1 
1
5 
5
5 
9/2
0
0
.5
1 
1
5 
5
4 
9/2
0
0
.5
X2W
and
Z
0
2 
2
1 
1
4 
4
0
0
0
10 
9
4 
4
25 
41/2
1
0
4.5
8 
9
4 
4
16 
41/2
1
0
4.5
6 
17/3
9 
34/3
4 
3
=
.3
2.3
1
3 
17/3
9 
34/3
1 
3
2.7
2.3
2
8 
17/3
16 
34/3
4 
3
2.3
4.7
1
25 
45/2
25 
25
25 
41/2
2.5
0
4.5
20 
45/2
25 
25
16 
41/2
2.5
0
4.5
0
0
0
0
0
0
0
0 ~
0
1/2
1/2
0
0
0
0
0
0
1/2
1/2
0
0
0
0
0
0
0
0
2/3
1/3
1/3
0
0
0
0
0
1/3
2/3
1/3
0
0
0
0
0
1/3
1/3
2/3
0
0
0
0
0
0
0
0
1/2
1/2
0
0
0
0
0
0
1/2
1/2 J
â€¢
4.3 Shillington1s Procedure
We originally described Shillington1s near neighbor
procedure for testing lack of fit in Section 2.3 of Chapter
Two. We now reintroduce the procedure by using the matrix
107
and vector quantities defined in Section 4.2. The test that
Shillington proposed involves the use of an F ratio (see Eq.
(4.2)) of two statistically independent mean square values,
2
each of which is an unbiased estimate of a when the fitted
model is the correct true model. The two independent mean
2
squares become biased estimates of a when the fitted model
suffers from lack of fit. Shillington's methodology detects
lack of fit when the calculated value of the F ratio in Eq.
(4.2) is large, thus his test is upper tailed. We shall see
later in Section 4.7 that the test is not always upper
tailed, and may be lower tailed. Shillington points out
that the power of the test depends on the relative
magnitudes of E(MSEB) and E(MSEW), that is, the power
depends on the difference between the expected values of the
numerator and of the denominator in the F ratio in Eq.
(4.2). We shall be more specific than Shillington by
discussing the power of the test in terms of parameters of
the doubly noncentral F distribution.
We now turn to defining Shillington's test statistic in
matrix notation. Shillington's F ratio takes the form (see
Eq. (4.2))
SSEB/(g  p)
SSEW/(N  g  r)
MSEB
MSEW
108
where SSEg is the residual sum of squares from a weighted
least squares regression analysis in which Yc is regressed
on Xc, g is the number of cells of near neighbors, p is the
number of terms in the fitted model, and r is the rank of
X^. The quantity SSEB can be written as the quadratic form
sseb  (45)
(see Graybill, 1976, p. 329; also see Draper and Smith,
1981, p. 109). The quantity SSEW is defined as
SSEW = W*[IN  XW(XW'XW)~XÂ¿]W, (4.6)
where is any generalized inverse of (X^XW). [A
matrix A is defined as a generalized inverse of the matrix
A if AA~A = A.] The quadratic form SSEW is the residual sum
of squares from an ordinary least squares regression
analysis in which W is regressed on XyÂ¡.
In the next two sections we shall discuss the
development of the numerator and denominator, MSEB and MSEW,
respectively, of the F ratio given in Eq. (4.2). We then
suggest an alternative representation for MSE^ which relies
on a generalization of weighted least squares. This
alternative representation for MSEW will be shown to be
equivalent to Shillingtonâ€™s expression for MSE^.
109
4.3.1 Development of MSEB
The quantity MSEg = SSEB/(g  p) is the numerator of
the F ratio in Eq. (4.2). As mentioned in Section 4.3, the
quantity SSEB is the residual sum of squares from a weighted
least squares regression analysis in which Yc is regressed
on Xc. The weighting is appropriate because
 2 .
var(Yc) = a Gq not only when the fitted model is adequate
(under model (4.3)), but also when the fitted model suffers
from lack of fit (under model (4.4)). In order to further
explain the (Y^, ) system, we define the matrix M as
M = diag[(l/n1)1
â€¢ â€¢ â€¢ 9
where 1. is an
"3
f 9
Then the (Yc, Xc) system can be derived as a linear
transformation of the (Y, X) system. That is, application
of the transformation matrix M yields the following
equalities
X
C
MX
and
X
2C
MX
2
From this it follows that var(Y^) = M var(Y)M' = a^MM'
2
a Gq , since MM' = G^. Under the hypothesized model of
110
Eq. (4.3), E(fc) = ME (Y) = MXg^ = X^^, whereas under the
model of Eq. (4.4), E(YC) = M( XB ^ +X2$2) = XC^1 + X2C^2â€˜
We now consider the distribution of the random quantity
2
SSEg/a . It can be shown (Theorem 2, Searle, 1971, p. 57)
that under the model of Eq. (4.3), SSED/a2 possesses a
D
central chisquare distribution with g  p degrees of
O
freedom, but that under the model of Eq. (4.4), SSEg/a
possesses a noncentral chisquare distribution with g  p
degrees of freedom and noncentrality parameter n^, where
"l = (1/2<â€™2)62X2CIG01  GÃ“1XC(XCGÃ“1XC>'1XÂ¿GÃ“11X2C^2' (4'7)
Here we point out that the noncentrality parameter for
2
SSEg/a given by Shillington (1979) is not correct and
should be written as in Eq. (4.7).
Finally we note that SSEB is equivalent to the usual
lack of fit sum of squares, SSL0F, where SSlqf/^  P) =
MSLof is the numerator of the F ratio in Eq. (4.1), when
all observations in each cell are true replicates rather
than near neighbors. Shillington pointed out this fact, but
did not give a proof. We offer a proof in Appendix 7.
4.3.2 Development of MSEW
The quantity MSEW = SSE^/(N  g  r), where r denotes
the rank of X^, is the denominator of the F ratio in Eq.
(4.2). As mentioned in Section 4.3, the quantity SSEW is
the residual sum of squares from an ordinary least squares
regression analysis in which W is regressed on Xw. Using
Ill
Theorem 2 (Searle, 1971, p. 57) and noting that
W = E y and EE = E . it can be shown that under the
 0 000
2
hypothesized model of Eq. (4.3), SSEw/a possesses a central
chisquare distribution with N  g  r degrees of freedom,
2
but under the model of Eq. (4.4), SSET,/o possesses a
w
noncentral chisquare distribution with N  g  r degrees of
freedom and noncentrality parameter n^, where
n2 = (l/202)BÂ¿X'w[In  vw~xÂ¿lW2 (48)
Shillington (1979) points out that SSEW reduces to the usual
pure error sum of squares, SSEpUre, when all cells contain
true replicates. This is easily seen by using the fact that
X^ = 0 when all cells are composed entirely of true
replicates so that SSET7 = W'W = Y'E E y = Y'E v = SSE
We saw in Section 4.3.1 that the (Y^, X^) system is
derived as a linear transformation of the (Y, X) system.
Similarly, the (W, X^) system can be derived by applying the
transformation matrix Eq. Thus W = E^y, Xw = E^x,
and Xâ€žT7 = E x*. It follows that E(W) = E E(Y) = E.XB.
= X^B^, under the model of Eq. (4.3), and E(W) =
+ X22^ = XW^2 + X2W2 un<^er mo<3el Â°f Eq. (4.4).
Furthermore, var(W) = EQ var(Y)E^ = a2EQE^ = a2EQ, since EQ
is symmetric and idempotent.
Since the variancecovariance matrix of W is not equal
to aIN, for some positive constant a, SSEW should have been
derived as the residual sum of squares from a weighted least
112
squares regression analysis of W on Xw rather than from the
ordinary least squares regression of W on X^. that
Shillington (1979) suggested. We shall use the weighted
least squares regression of W on Xw in an attempt to replace
SSEW in the F ratio in Eq. (4.2) by an expression we will
call SSEW(weighted). We later show that SSEW and
SSEW(weighted) are equivalent.
2
The variancecovariance matrix of W, which is a Eq, is
of rank N  g and is thus singular. Therefore the residual
sum of squares from a weighted least squares regression
analysis of W on Xw which is
cannot be used as an expression for SSEW(weighted), since
Eq1 does not exist. The problem of performing a weighted
least squares regression analysis when the variance
covariance matrix of W is singular is considered in the next
section.
4.4 Development of SSEW(weighted)
2
If the variancecovariance matrix of W, a Eq, is
nonsingular then the weighted least squares formula for
SSEw(weighted) is
SSEW( weighted) = (W  X^TEq^W  Xw3 x)
113
where ^ is a solution to X^Eg'^X^^ = X^E^W and can be
written as = (X^E q"*â€™Xw)â€” X^E q^W. The quantity
SSEW(weighted) divided by the appropriate degrees of freedom
2
provides an unbiased estimate of a under the model of Eq.
(4.3) .
However, since Eq is singular, the weighted regression
formula above cannot be used to calculate SSEW(weighted).
C. R. Rao (1971, 1972, and 1973) suggests an analog of
weighted least squares for the case of a singular variance
covariance matrix. Rao suggests the existence of a matrix H
such that g^ is a stationary vector value of
2
(W   Xw0^) in whlch case a may be estimated
using
a2 = (W  XW61)Â«H(W  Xw01)/(N  g  r)
where (N  g  r) = rank(EQ:Xw)  rank(Xw). The rank of the
matrix (EQ:XW) is equal to N  g because Xw = EqXw so that
the columns of Xw are spanned by the columns of Eg, thus,
rank(EQ:Xw) = rank(EQ) = N  g.
One form of the matrix H is defined (Rao, 1971 and
1972) as
H = [Eq + c2XwXÂ¿] (4.9)
where c is an arbitrary nonzero constant, so that with the
model of Eq. (4.3), a2 = (W  X^rHÃW  X^J/iN  g  r)
114
2
is an unbiased estimator for a . Thus a stationary vector
value of (W  _ ^{J ) is given by
= (X^HXW)~X^HW . Rao indicates (1972, p.3) that 02 is
invariant to the choices of the generalized inverses
* 2
involved m a â€¢
Rao's proofs for obtaining an unbiased estimator
~ 2 2 . .
a for a are not given m detail. Therefore we shall state
and prove the following theorem which will be used to
develop an expression for SSEW(weighted). The notation A
will be used to denote any generalized inverse of a matrix
A, such that AAA = A.
2
Theorem 4.1. Let Y ~ (X0, a G), where G is singular,
then a2 = f_1(Y  X0)'T(Y  X0)
2
(i) is an unbiased estimate of a , and
(ii) is unique with probability one, and
(iii) is a scalar multiple of a central chisquare
variable with f degrees of freedom of the form
2 2
(a /f)Xf if Y has the multivariate normal
distribution.
The vector Y is of order Nxl, g is a pxl vector of
unknown regression coefficients, X is an Nxp matrix of
known constants, G is an NxN positive semidefinite
A
matrix of known constants, T = G + XX', 0 is any
solution to X'T_X0 = X'Tâ€œY, that is, 0 = (X'TX)~X'TY,
and f = rank(G:X)  rank(X).
115
The proofs of parts (i), (ii) and (iii) of Theorem 4.1 are
given in Appendices 9, 10, and 11, respectively. Lemma 4.1
which is used in the proof of Theorem 4.1 is stated and
proved in Appendix 8.
The results of Theorem 4.1 can now be applied to our
problem of finding an expression for SSEW(weighted). We
define SSEW(weighted) as
SSEW(weighted) = Wâ€™[Tâ€œ  T~XW(X^T~XW)â€œX^T"]W (4.10)
where tq = ZQ + XWX^. Writing SSEw(weighted) in Eq. (4.10)
as SSEW(weighted) = (W  ^B^J'T^W  XWB1), from Theorem
4.1 we see that if the true model is of the form in Eq.
2
(4.3) then SSEW(weighted)/a has a central chisquare
distribution with f = rankU^rX^  rankfx^) = N  g  r
degrees of freedom. However, if the true model is of the
2
form in Eq. (4.4), then SSEW(weighted)/o has a noncentral
chisquare distribution with N  g  r degrees of freedom
and noncentrality parameter II*, where
n 2 * (1'/2'â€™2>Â®2XÂ¿WIT0 '
The distribution of SSEW(weighted)/o^ under model (4.4) is
verified by the following theorem.
116
Theorem 4.2. Let Y ~ NN(XÂ¡3 + X 0 , o2G), G singular,
, "22 2 ^21  _
then fa /a ~ xÂ¿ A where a = f (Y  X0 )'T (Y  X0 ) ,
f = rank(G:X)  rank(X), T = G + XX', and
A = (l/2a2)0Â¿XÂ¿[T~  T~X(X'T_X)~X,T_]X2e2.
The proof of Theorem 4.2 is given in Appendix 12.
4.5 Equivalence of SSEW and SSEW(weighted)
In this section we shall show that our expression for
SSE^(weighted) in Eq. (4.10) is equal to Shillington's
unweighted SSE^ in Eq. (4.6). Thus the complex calculations
required for evaluating SSEW(weighted) can be avoided by
calculating the simpler form SSEW.
Zyskind (1967) investigated conditions under which
ordinary least squares estimators are BLUE (best linear
unbiased estimators) even though Y in the model
Y = X3 + e, where E(e) = 0, does not have variance
2
covariance matrix equal to a I. Zyskind assumes that
2
var(Y) = o V, where V is nonnegative and possibly singular,
and then states and proves the following necessary and
sufficient condition for ordinary least squares estimators
to be BLUE.
Theorem 1 (Zyskind, 1967). A necessary and sufficient
condition for all simple least squares linear estimators
to be also best linear unbiased estimators of the
117
corresponding estimable parametric function A'3 in the
linear model
Y = X3 + e, E(e) = 0, E(ee ' ) = a2V,
where V is a symmetric nonnegative matrix and X is of
rank r, is that there exist a subset of r orthogonal
eigenvectors of V which forms a basis for the column
space of the matrix X.
In a second theorem, Zyskind (1967) gives several other
necessary and sufficient conditions for ordinary least
squares estimators to be BLUE. These conditions are shown
to be equivalent to the condition in Theorem 1 (Zyskind,
1967). The fifth of these conditions in Zyskind's second
theorem is that VP = PV, where P = X(X'X)X'.
Applying condition 5 of Theorem 2 (Zyskind, 1967) to
our problem of regressing W on Xw we have
V
I
0
and
P =
vv
V
'*w
and therefore
118
VP = W W *W
= Ww)_xw '
since Eq= EqEqX = EqX = XwÂ« It follows that
vp = V W^o
= PV.
Therefore by Theorem 2 (Zyskind, 1967) the ordinary least
squares solutions from regressing W on X^ are BLUE
estimators, and thus are equivalant to the solutions
obtained from weighted least squares. We conclude therefore
that SSEW = SSEW(weighted).
4.6 The Test Statistic
As stated in Section 4.1, Shillington (1979) proposed
that a fitted model be tested for lack of fit by using the F
ratio
MSB_
F =
MSEW
w
given in Eq. (4.2). In this section we shall verify that
Shillington's F ratio possesses a central F distribution
when the true model is of the form in Eq. (4.3), and
possesses a doubly noncentral F distribution when the true
119
model is of the form in Eq. (4.4). This information on the
distribution of the F ratio will be needed in Section 4.7,
where the power of the test is discussed. Additionally, we
shall give the form of the expected values of both the
numerator, MSEg, and the denominator, MSEW, of the F ratio
in Eq. (4.2). These expected values will aid us in
developing a procedure for calculating the power of the
test, since they will be used to determine whether the test
is upper tailed or lower tailed.
In developing the distribution of the F ratio in Eq.
(4.2), we shall show that SSEB/a2 and SSEw/o2 are
statistically independent. In this pursuit, let us use the
expression for SSEg in Eq. (4.5) and the fact that Yc = MY
to express SSEg as
SSEb = Y'M'IG1 G1Xc(XÂ¿G1Xc)1XÂ¿G1]MY.
Also, using the expression for SSEW in Eq. (4.6) (which is
allowed because we showed in Section 4.5 that the correct
form, SSEW(weighted), is equal to SSEw) and using the fact
that W = zqY, we can express SSE^ as
SSEw = TV1Â»  xw(xwxw)_xw1Eoâ€™
By Theorem 4 (Searle, 1971, p.59), to show that
2 2
SSEg/a and SSE /a are statistically independent, it
suffices to show that the matrix product
120
EM'fGÃ“1 
G:1)M]tr0{IN  VW
V
is equal to the zero matrix. This is seen to be true since
MEq = 0, and therefore SSEB/a2 and SSEw/a2 are independent.
When the fitted model and the true model are both of
the form in Eq. (4.3), then the F ratio in Eq. (4.2)
possesses a central F distribution with g  p and N  g  r
degrees of freedom in the numerator and denominator,
respectively. Furthermore, the numerator, MSEB, of the F
ratio in Eq. (4.2) has expectation equal to
E(MSEb) = [o2/(g  P)]EXg_p
2
a .
Similarly, under model (4.3), the expected value of the
denominator, MSEW, of the F ratio has expectation equal to
E(MSEw) = [a 2/(N  g  r)]Ex2_g_r
When the fitted model suffers from lack of fit and the
true model is given by Eq. (4.4), the F ratio in Eq. (4.2)
is a ratio of two statistically independent noncentral chi
square random variables, each divided by its respective
121
degrees of freedom. Thus the F ratio in Eq. (4.2) possesses
a doubly noncentral F distribution with g  p and N  g  r
degrees of freedom and noncentrality parameters and n^â€¢
where n^ and n2 were given in Eqs. (4.7) and (4.8),
respectively. The expected value of the numerator, MSEg, of
the F ratio can be written as
E(MSEb) = (a2/(g  P *!EXgP,n,
= a
+ 6Â¿C1
s,/(g  p)
where
C1  W'^01  GÃ³lxc(XÂ¿GÃ³lxC>'lxÂ¿GÃ³llMX2 (4U>
Similarly under model (4.4), the expected value of the
denominator, MSE^, of the F ratio can be written as
E(MSEW) = [c2/
= a2 + BÂ¿C2b2/(N  g  r)
where
â€œ X2E0[IN
 vw
VZ0X2
(4.12)
122
4.7 The Testing Procedure and its Power
As discussed in Section 4.1, Shillington (1979)
suggested that lack of fit of the fitted model be inferred
when the value of the F statistic in Eq. (4.2) exceeds
F . The test, however, is not always upper
tailed, and in fact can be lower tailed. The test is
considered lower tailed when, because of lack of fit, the
expected value of the numerator of the F ratio is less than
the expected value of the denominator of the F ratio.
We suggest that lack of fit be tested with an upper
tailed test using the F ratio F = MSEB/MSEW when the matrix
D, which is defined as
is found to be positive definite (which can only occur when
is positive definite, by Theorem 3.1 in Appendix 4). The
matrices and C2 in Eq. (4.13) are defined in Eqs. (4.11)
and (4.12), respectively. An upper tailed test is
appropriate when the matrix D is positive definite because
no matter what the value of is/ the expected value of the
numerator, MSEg, of the F ratio will be greater than the
expected value of the denominator, MSE^, of the F ratio.
However, there may be cases where D is negative definite
(which can only occur when C2 is positive definite), and in
this case lack of fit should be tested with a lower tailed
123
rejection region. If D is indefinite, then the F test for
lack of fit may be upper tailed, lower tailed, or lack of
fit may not be testable depending upon the value of 8 .
In those cases where D is indefinite, it is helpful to
write the quantity 8^D8^r which represents the difference
between the expected value of the numerator and the expected
value of the denominator of the F ratio, F = MSEB/MSEW, as
2 Â°2 = Â§.Â£[ ui:U2:U3] diag[r1, r2 = 0, r 3 ] [U1 :U2 :U3 ] â€™ B 2
= g_2UlrlUi^2 + ^2^3r 3^322'
where U^, Ãœ2/ and U3 are matrices whose columns are
orthonormal eigenvectors of D, and r^, r2, and are
diagonal matrices whose elements are the positive, zero, and
negative eigenvalues of D, respectively. Lack of fit is
testable with an upper tailed test if B2 is in the column
space of [U^:U ], but not entirely in the column space of
U2/ since then B^Df^ is positive. Similarly, lack of fit is
testable with a lower tailed test if b2 is in the column
space of [U2;u3] â€¢ but not entirely in the column space of
U2, since then 32DS2 is negative. If B2 is in the column
space of U2, then lack of fit cannot be tested, since
B_2DB_2 would equal zero.
124
We define Â§_2 to h>e in the column space of U2 if the
matrix has a zero eigenvalue, where = [Â§.2:U2].
Similarly, letting L2 = and L3 = [0 2: U2: U3 ^ '
3_2 is in the column space of [U^:U2] if LÂ£L2 has a zero
eigenvalue, and 3~ is in the column space of [u :U ]
^ ^ J
if l3l3 has a zero eigenvalue.
When D is positive definite or D is indefinite but
32 is upper tailed testable, then the F test for lack of fit
which makes use of the F ratio F = MSEB/MSEW is a test of
the hypotheses (see Theorem 3.2, Appendix 5)
Ha: ni/(9 " P) " n2/(N " 9 " r> > Â°*
When D is negative definite or D is indefinite but 32 is
lower tailed testable, then the F test tests
0
Ha: nl/(g " P) " n2/(N ~ 9 ~ r) < 0.
In the case where D is indefinite and 3_2 is in the column
space of U2, then no hypotheses concerning lack of fit of
the fitted model can be tested.
When the test is upper tailed, the power of the F test
for lack of fit is defined as
125
Power = p{ F"
(4.14)
where F
a;gp,Ngr
is the upper 100a percentage point of the
central F distribution with g  p and Ngr degrees of
freedom. In the case of a lower tailed test, the power of
the test is defined as
Power = P{F"
M < F __ } . (4.15)
gp,Ngr ;ll i ,Ãœ2 1a ;gp,NgrJ
4.8 Selection of Near Neighbor Groupings
In the preceding sections of this chapter, we have
discussed a near neighbor procedure which uses the F ratio
F = MSEB/MSEW to test a fitted model for lack of fit. In
this section we shall investigate the effect that different
groupings of response observations into near neighbor cells
have on the testing procedure and its power. From equations
(4.14) and (4.15) in the previous section it is evident that
the power of the F test for lack of fit, which makes use of
the F ratio F = MSEB/MSEW, depends on the values of the
numerator and denominator noncentrality parameters,
n and n2Â« Assuming the numerator and denominator degrees
of freedom are fixed, and the test is upper tailed, then the
power is an increasing function of increasing values
of n^ and is a decreasing function of increasing values of
n2 (see Appendix 1). When the test is lower tailed, the
126
power is an increasing function of n2 and is a decreasing
function of n .
Since both the numerator and denominator noncentrality
parameters, II ^ and II , are functions of the groupings of the
data points into near neighbor cells (as are the numerator
and denominator degrees of freedom), we would like to
investigate the effect of the number and composition of
cells on the power of the F test. Intuitively, it would
seem that homogeneous near neighbor cells would minimize the
2
bias inherent in estimating a with MSEW, and thus would
minimize n^ and maximize the power of an upper tailed
test. However, any grouping of the data points would also
influence the numerator noncentrality parameter and the
numerator and denominator degrees of freedom. Therefore
while a grouping of data points into homogenous cells might
decrease n ^ and thus apparently increase the power of an
upper tailed test, the result of the grouping on the power
also depends on how the degrees of freedom, g  p and
N  g  r, and the numerator noncentrality parameter, II
are affected.
We will attempt to find homogeneous cells of near
neighbor points by using an iterative partitioning
clustering algorithm. Two examples will be presented. The
first example makes use of the stack loss data presented by
Daniel and Wood (1971) and later analyzed by Shillington
(1979). The second example involves data from a mixture
experiment discussed by Piepel (1981).
127
Our objective is to investigate the effect of forming
homogeneous cells of near neighbors on the F test for lack of
fit which makes use of the test statistic F = MSEg/MSE^.
It is hoped that such homogeneous groupings will increase
the power of the F test (assuming that the rejection region
is upper tailed) by decreasing for a fixed number of near
neighbor cells (and thus fixed values for the degrees of
freedom). The effect that homogeneous grouping has on n^ is
not clear, but is of interest, since the magnitude of
n1 also affects the power of the test. Additionally, we
will vary the number of cells of near neighbors in an
attempt to determine how this affects the power of the test,
since the number of cells affects both the noncentrality
parameters and the degrees of freedom.
The algorithm used for grouping the data points into
homogeneous near neighbor cells can be described as an
iterative partitioning type of cluster analysis. The
computations involved were accomplished using the RELOC
procedure available in the CLUSTAN 1C computer package
(Wishart, 1975). All computations were performed using data
points whose coordinates were standardized by subtracting
off sample means and dividing by sample standard deviations.
Initially, k clusters (near neighbor cells) of the N
data points in the factor space are arbitrarily defined.
Then the Euclidean distance between each point and the
centroid (average vector value) of each of the k clusters is
determined. If a point is found to be closer to the
128
centroid of one of the other k  1 clusters than to the
centroid of the cluster in which it is currently classified,
then the point is reclassified into that nearest cluster
(cell). The centroids of the k clusters are then
recalculated, taking into account any reclassified point.
The entire set of N points is scanned repeatedly in this
manner until no reclassification occurs. This method of
assigning points to clusters will be referred to as
iterative relocation.
In the second stage of the algorithm, two of the k
clusters arrived at by the iterative relocation procedure
are fused, resulting in k  1 clusters. The two clusters to
be fused are selected as those which when fused produce the
k  1 clusters with minimum "error sum of squares." The
error sum of squares is defined as the sum of squared
Euclidean distances between every point and the centroid
point of the cluster to which it belongs. After k  1
clusters are determined using the error sum of squares
criterion, iterative relocation is applied to the k  1
groups in an effort to improve the clusterings. This
alternation of fusion and iterative relocation continues for
k  2 clusters, k  3 clusters, ..., 2 clusters, or until a
specified minimum number of clusters is reached. The
question of determining an "optimal" number of clusters is
not addressed by this procedure.
129
4.8.1 Example 1â€”Stack Loss Data
The first example we investigate is the 21 observation
stack loss data of Daniel and Wood (1971), which was
analyzed by Shillington (1979). The data (see Table 8)
consist of the values of three factors, (air flow), X2
(cooling water inlet temperature), and x^ (acid concenÂ¬
tration) along with the values of a response variable, Y
(stack loss). A first order regression equation of the form
E (Y) = eQ + B 1x x + 8 2X 2 + B3x3
is fitted using 17 of the original 21 observations
(Shillington discarded 4 of the original 21 observations as
outliers). We assume the true model to have the form
E(Y) = 8
+ 81x1 + 82x2 +
83X3 +
Bllxl + 622x2 + S33x3
and thus contains p2 = 3 terms in addition to the p = 4
terms in the fitted model. We wish to investigate the
capability of the F test
MSE
F = â€ž B
msett
w
in detecting lack of fit of the fitted model.
We first consider the use of the F statistic with the
six cells of near neighbors used by Shillington (1979),
which is the same near neighbor grouping suggested by Daniel
130
and Wood (1971). This 6 cell grouping (see Table 8 under
the column heading "6**") is found to yield a matrix
D = [c^/(9 ~ P)J  [C2^N â€œ 9 â€ r)] which is indefinite,
since the eigenvalues of D have the values 12110, 7, and
1415 (see Table 9). Thus the test is not upper tailed,
since D must be positive definite for an upper tailed test
to exist for all values of 32* where in this example,
2
= (3
11'
B 22'
B33} '
When the 6 cell grouping of near neighbors generated by
the iterative partitioning clustering algorithm (see Table 8
under the column heading "6") is used, the values of the
eigenvalues of D are 49090, 379, and 43, and the test is
still not upper tailed for all values of
We then use the iterative partitioning clustering
algorithm to determine homogeneous cell groupings for 5, 7,
8, 9, 10, 11, and 12 cells. The matrix D is found to be
indefinite for the groupings into 5, 7, or 8 cells, but D is
positive definite for 9, 10, 11, or 12 cells of near
neighbor groupings. Thus, no matter what the value of
lack of fit can be tested with an upper tailed test using
the 9, 10, 11, or 12 cell groupings of near neighbors.
The value of F = MSEB/MSEW was calculated using the
matrix procedure from the 1979 version of SAS. None of the
near neighbor groupings provided evidence of lack of fit,
and thus we cannot conclude that there is lack of fit when
the fitted model is E(Y) = 8^ + 8^x^ + 32X2 + ^3X3
Table 8. Near Neighbor Cells for Stack Loss Data.
Observation
Number
Air
Flow
X1
Water
Temp.
x2
Acid
Concen.
x3
Stack
Loss
Y
Membership in
for Five
5 6** 6 7
Near Neighbor Cells
to Twelve Cells
8 9 10 11
12
1*
80
27
89
42
2
80
27
88
37
1
1
1
1
1
1
1
1
1
3*
75
25
90
37
4*
62
24
87
28
5
62
22
87
18
2
2
2
2
2
2
2
2
2
6
62
23
87
18
2
2
2
2
2
2
2
2
2
7
62
24
93
19
2
2
2
3
3
3
3
3
3
8
62
24
93
20
2
2
2
3
3
3
3
3
3
9
58
23
87
15
2
3
2
2
2
2
2
4
4
10
58
18
80
14
3
4
3
4
4
4
4
5
5
11
58
18
89
14
4
4
4
5
5
5
5
6
6
12
58
17
88
13
4
4
4
5
5
5
5
6
6
13
58
18
82
11
3
4
3
4
4
4
4
5
5
14
58
19
93
12
4
4
4
5
5
6
6
7
7
15
50
18
89
8
4
5
5
6
6
7
7
8
8
16
50
18
â€¢ 86
7
4
5
5
6
6
7
7
8
9
17
50
19
72
8
5
5
6
7
7
8
8
9
10
18
50
19
79
8
5
5
6
7
8
9
9
10
11
19
50
20
80
9
5
5
6
7
8
9
9
10
11
20
56
20
82
15
3
6
3
4
4
4
10
11
12
21*
70
20
91
15
*Outliers, not included in analyses.
**This near neighbor cell grouping was proposed by Daniel and Wood (1971) and
used by Shillington (1979). All other cell groupings were generated by an
iterative partitioning cluster analysis using the CLUSTAN computer package.
Numbers in the table refer to cell membership.
131
Table 9. Effect of Near Neighbor Cells on
F = MSEb/MSEw for Stack Loss Data.
Number of Cells
Numerator
(g) 5
6**
6
7
8
9
10
11
12
d .f . (g  p)
Denominator
1
2
2
3
4
5
6
7
8
d.f. (Ngr)
Eigenvalues
9
9
8
7
6
5
4
4
3
of C,
65525
24408
98191
99915
102192
102572
102859
103214
103445
0
622
4082
9098
20054
21985
23317
22394
22642
Eigenvalues
0
0
0
17
17
74
85
106
107
of C2
14900
16325
13539
8297
1017
327
318
318
104
1478
121
112
50
48
48
24
24
10
Eigenvalues
111
0
39
20
18
18
0
0
0
of D
69297
12110
49090
33300
25540
20507
17143
14745
12931
13
7
379
1852
4847
4332
3640
3120
2800
Calculated
1592
1415
43
4
2
8
8
9
6
value of F
1.14NS
1.08NS
0.96NS
0.91NS
0.83NS
1.05NS
1.73NS
1.49NS
2.52NS
ni
â€”
â€”
â€”
â€”
â€”
133
133
136
136
n 2
Approximate
Power of Upper
1
1
1
1
Tailed Test
Note: Fitted Model:
E(Y) = B
0 + 61X1
+ 8 2x2
+ 83X3
1.00
0.98
0.96
0.79
Assumed
** Daniel and
NS Not greater
true model: E (Y) = BQ + 3 jX 1 + B 2X 2 + 3 3x + 3^x
Wood (1971) grouping, other groupings generated by
than i)1) :aD.Mnr .
2
1 + 622X
CLUSTAN.
2 2
2 + 3 33X3
132
133
and the true model contains only pure quadratic terms in
addition to the first degree terms.
For the groupings of near neighbors into cells which
provide an upper tailed test (9, 10, 11, or 12 cells), the
power of the upper tailed test can be approximated if
and n can be specified. This approximate power can be
calculated using an approximation to the doubly noncentral F
distribution given in Johnson and Kotz (1970, p.197), which
is described in Appendix 6 of this dissertation. Thus we
calculate an approximation for
P{ F"
1 gp,Ngr ;H i ,n2.
>
F } .
a ;gp,NgrJ
In order to compare the power of the upper tailed F
test for 9, 10, 11, and 12 cells, we will assume arbitrarily
that the true value of the parameter vector g^ is
2 = ^11' e22' e33^' = (* 044' * 329,  . 033)' which is
arrived at by taking = 1002 w^ere ^2 t*ie ieast squares
estimate of 0^ calculated from the data. Furthermore,
2
taking a =1.6 (since the residual mean square value from
fitting the "true" second degree model is MSE = 1.6) we
2
calculate the values for and
H 2 = &2C22/'2Â°2 ^or eacl1 the 9, 10, 11, and 12 cell
groupings. The calculated values of and nas well as
the approximate power values for each of the four F tests
(calculated using the approximation to F" from Johnson and
Kotz (1970, p.197)) are presented in Table 9. The power is
134
quite high (> .96) for 9, 10, or 11 cell groupings, but
drops off to .79 for the 12 cell grouping. This drop in
power seems to be due to the effect of having only 3 degrees
of freedom in the denominator of the F ratio.
In summary, this example illustrates that the F test
for lack of fit that makes use of the statistic
F = MSEg/MSEw is upper tailed only for certain groupings of
the design points into near neighbor cells. For the near
neighbor cell groupings that provide an upper tailed test,
the power is generally high for the values of Â£5^ and
2
a that we selected, but decreases slightly as we move from
9 to 10 to 11 cells and decreases more severely as we move
from 11 to 12 cells. This more severe decrease in power is
due to the decrease to only 3 denominator degrees of
freedom.
4.8.2 Example 2â€”Glass Leaching Data
The second example we investigate is one in which the
leachability, Y, of glass is assumed to be a function of the
proportions of eleven chemicals of which the glass is
composed (Piepel, 1981). A first order ScheffÃ© polynomial
model was fitted to the common logarithms of the
leachability values, that is, the fitted model is of the
11
E(log1(JY) =_Z 8ixi.
form
135
The experimental design coordinates and the values of
the 44 data observations are presented in Table 10. For
illustrative purposes, the true model is assumed to contain
the P2 = 8 second order cross product terms,
B68X6X8' 83,11X3X11' B79X7X9' B56X5X6' B7,11X7X11'
B15X1X5' B6,10Vl0' and B59X5X9 ln addition to the P = 11
first order terms in the fitted model. The 19term model is
the final fitted model proposed by Piepel and serves as our
true model.
Piepel (1981) suggested that the four sets of
observations (see Table 10)
(a)
14 and
15
(b)
18,
19,
and
20
(c)
25,
26,
and
27
(d)
39,
40,
and
41
were intended
to
constitute four
cells of replicate
observations
for
use
in estimating pure experimental
error.
However,
the
settings of
the mixture components
not well controlled, so that each of the four cells
contained near neighbors rather than replicates. By
defining each of the remaining 33 data points as 33 cells
containing one observation each, the 44 data observations
are partitioned into 37 cells. If we choose to use the 37
cells to test the fitted model for lack of fit, with the
test statistic F = MSEg/MSE^, we find that there are no
degrees of freedom for MSEW so that the F statistic cannot
be calculated in this case. However, the F statistic for
Table 10. G'
Observation
Number
X1
Si O2
x 2
B2Â°3
x 3
a12Â°3
x4
CaO
x 5
MgO
1
.448
.063
.143
.000
.000
2
. 544
. 063
.000
. 138
. 000
3
.565
.120
.000
.000
.000
4
. 581
. 061
.012
.000
. 079
5
.432
.126
.000
.098
.079
6
.486
.064
.000
. 140
.000
7
.429
.060
.145
. 139
.013
8
.429
. 124
. 144
. 138
.000
9
.443
.120
.000
.099
.080
10
.447
.114
. 148
.012
.077
11
.427
.121
.145
.010
.078
12
. 550
. 060
.146
.000
.076
13a
. 434
.058
.148
.127
.000
14a
. 580
.122
.000
. 135
.000
15
. 575
.123
.000
.134
.000
16
.498
. 066
.000
. 134
. 080
17b
.435
.118
.145
.000
.031
18b
. 496
. 092
. 072
. 067
.038
19b
.497
.091
.072
.066
. 039
20
. 497
. 091
. 072
. 067
. 039
21
. 565
.125
.000
.010
. 036
22
. 428
. 068
. 073
. 117
. 084
23
.433
.064
. 155
.047
.000
24c
. 550
.141
. 000
. 000
. 000
25c
. 577
.127
.023
. 000
.086
26c
. 586
.126
.024
.000
. 079
27
. 585
. 128
. 023
.000
.082
28
. 579
. 068
.000
.000
. 088
29
. 541
.064
.153
.072
. 000
ss Leaching Data.
x 6
Na 2O
x 7
ZnO
x 8
Ti02
x9
Cr203
x10
Fe2Â°3
X11
NiO
Y*
Leach
Value
.165
.059
.069
.000
.029
.025
8.10
. 154
. 000
. 071
. 030
. 000
. 000
0.14
.153
.061
.070
.031
.000
.000
0.01
.108
. 061
. 070
. 000
. 028
.000
0.01
.158
.000
.070
.000
.009
.028
7.40
.159
. 063
.000
. 028
. 031
.029
3.90
. 152
. 062
.000
.000
.000
.000
91.40
.105
. 060
. 000
. 000
. 000
. 000
49.00
.101
.061
.070
.000
.026
.000
0.11
. 150
.000
.000
.027
.025
. 000
33.40
.161
.000
.000
.029
.029
.000
39.50
.111
.000
.030
.000
.000
.027
0.42
. 109
.000
.069
.030
. 025
.000
9.60
. 110
.000
.000
. 000
.025
. 028
0.20
. 112
.000
.000
.000
.028
. 028
0.24
. 105
. 059
.000
.030
. 000
. 028
6.20
.102
.060
.051
.031
.000
.027
2.20
. 125
. 031
.035
. 015
. 015
. 014
0.86
.129
.029
.034
.014
.015
.014
0.80
. 127
. 030
.034
. 014
.015
. 014
0.86
.163
.000
. 070
.031
. 000
.000
0.20
. 108
. 000
.071
. 000
. 027
.024
1.47
.159
.000
.073
.028
.022
.019
16.40
. 156
. 063
. 079
.000
. Oil
. 000
0.15
.094
. 060
.000
. 033
.000
. 000
0.15
. 093
. 058
. 000
.034
. 000
.000
0.08
. 091
.059
.000
.032
.000
.000
0.15
. 146
. 061
. 000
. 000
. 031
.027
0.18
.113
.000
.000
.031
.026
.000
2.04
136
Table 10â€”continued.
30
.417
.127
.152
. 010
â€¢
31
.4 29
.133
. 031
.130
â€¢
32
.454
.067
.142
.000
â€¢
33
.420
.130
. 138
.000
â€¢
34
. 574
.066
.000
.114
â€¢
35
. 584
.067
. 031
.128
â€¢
36
.464
.067
.142
. 126
â€¢
37
. 433
.134
.142
. 038
â€¢
38d
.432
. 134
.012
.129
â€¢
39d
.434
. 068
. 127
. 131
â€¢
40d
.432
.067
.138
.131
â€¢
41
.431
. 067
.139
.132
â€¢
42
.413
. 127
.151
.011
â€¢
43
.430
. 133
.144
. 127
â€¢
44
. 59 6
.068
.000
.011
â€¢
* Leach values are weight percent loss
a,b,c,d Four groups of intended replica
162
.000
.000
. 031
.000
.021
51.80
144
. 059
. 074
. 000
. 000
. 000
9.04
145
.060
.077
.033
.000
.022
2.09
155
. 000
. 074
. 000
.000
. 000
24.70
108
.000
.077
.034
.000
.027
0.02
165
. 000
. 000
. 000
.025
. 000
0.08
118
.000
. 000
.000
.000
.000
37.20
093
.058
. 072
. 000
. 030
. 000
7.39
096
.059
. 000
.000
.030
.025
96.60
153
. 061
. 000
. 000
. 000
.026
77.80
146
. 060
.000
.000
.000
.026
75.60
149
. 061
. 000
. 000
. 000
.021
76.40
165
.000
.000
.030
.000
.023
43.30
111
. 000
.000
. 031
.000
. 024
25.40
136
.063
.077
.024
.000
.025
0.07
ass. Analyses performed on log^Q(Y).
080
000
000
083
000
000
083
000
083
000
000
000
080
000
000
of gl
tes.
137
138
lack of fit can be calculated using from 12 to 33 cells of
near neighbors, and thus we refer to the iterative
partitioning clustering algorithm discussed earlier in this
section to generate near neighbor groupings of 15, 20, 25,
and 30 cells (see Table 11).
The clusterings of observations into 15, 20, or 25
cells each yields a D matrix that is indefinite (see Table
12), so that the test is not upper tailed. The 30 cell
clustering produces a positive definite D matrix, so that
the test is upper tailed for all nonzero values of where
82 = (0 68' 0 3,11' 0 79 ' 0 56 ' 07,11' 015' 06,1O' 059)'*
Taking 02 = (15.141, 112.429, 78.761, 78.275, 87.996,
13.356, 76.948, 34.721)', which is the least squares
2
estimate of 8^ froâ„¢ the data, taking a = .008 (which is
MSEpure with seven degrees of freedom from Piepel's analysis
of the data), and using the approximation of Johnson and
Kotz (1970, p.197) to approximate
PÃ F" > F )
1 19,4,Hi ,n2 . 05 ; 19,4 J
where n^ = 9.79, U^ = 0.08 and F 4 = 5.81, we find
that (using 30 near neighbor cells) a value for the power of
the F test is .10. The power increases as the magnitudes of
the elements of ^ are increased, so for example if all the
elements of above are doubled, then = 39.16,
n2 = 0.32, and the approximate power is .25. If the
elements of 0_2 above are each multiplied by 5, then
139
Table 11. Near Neighbor Cells for Glass Leaching Data.
Membership in Near
Observation
Neighbor
cells
Number
15
20
25
30
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
6
6
6
6
6
7
7
7
7
7
8
8
8
8
8
9
9
9
9
9
10
10
10
10
10
11
10
10
10
10
12
11
11
11
11
13
12
12
12
12
14
13
13
13
13
15
13
13
13
13
16
6
6
14
14
17
14
14
15
15
18
11
15
16
16
19
11
15
16
16
20
11
15
16
16
21
3
3
3
3
22
5
5
5
17
Membership in Near
Observation
Neighbor
Cells
Number
15
20
25
30
23
1
1
1
18
24
3
3
3
19
25
15
16
17
20
26
15
16
17
20
27
15
16
17
20
28
4
4
18
21
29
12
12
12
12
30
10
17
19
22
31
8
8
8
23
32
1
14
15
15
33
10
18
20
24
34
2
2
2
2
35
13
19
21
25
36
11
11
22
26
37
8
8
23
27
38
9
9
9
28
39
7
7
7
7
40
7
7
7
7
41
7
7
7
7
42
10
17
19
22
43
14
20
24
29
44
2
2
25
30
*Cell groupings generated by an iterative partitioning
cluster analysis using the CLUSTAN computer package.
Numbers in the table refer to cell membership.
Table 12. Effect of Near Neighbor Cells on
F = MSEb/MSEw for
Glass Leaching
Data.
Number of Cells (g)
15
20
25
30
Numerator d.f. (g  p)
4
9
14
19
Denominator d.f. (Ngr)
19
14
9
4
Eigenvalues of C^
2.92x105
6.57xl05
1.03xl04
1.27xl04
1.92x105
2.56xl05
2.84xl05
3.24x105
3.13xl06
1.07xl05
1.74xl05
2.37x105
6.60xl07
4.34x106
6.45x106
7.38x106
0
3.02x106
4.86xl06
5.51x10â€œ6
0
1.07xl0â€œ6
1.23xl06
1.88xl6
0
l.lOxlO7
9.00x107
1.44x106
0
4.76x108
3.33xl07
7.97xl07
Eigenvalues of C2
1.60xl05
1.05x10â€œ5
6.69xl06
5.Olx107
1.28xl05
5.03x106
5.06x10â€œ7
5.89xl0â€œ8
7.65xl06
2.64x106
1.51xl07
3.22x109
1.84x106
8.10x107
9.50xl08
1.21x109
1.09xl0â€œ6
4.36xl0â€œ7
2.65x10â€œ8
0
7.43xl07
7.31xl0â€œ8
9.49xl09
0
6.36xl0â€œ7
2.58xl08
1.71xl09
0
2.30xl08
2.72x109
0
0
140
Table 12â€”continued
Eigenvalues of D
6.54xl06
7.05x106
7.23xl06
6.36x106
4.44xl06
2.38xl0â€œ6
1.79x10"6
1.67x 106
6.17xl07
9.39xl0â€œ7
1.03x106
1.24x106
1.85xl0â€œ8
4.15x107
4.54x107
3.87x10â€œ7
4.43x108
2.41x10â€œ7
3.42x107
2.88x10~7
4.87xl0â€œ8
6.86x108
7.97xl0~8
9.30x108
1.38xl07
2.93x108
3.77x 10**8
7.18x108
4.80x10â€œ7
1.52x10â€œ7
1.9 7x107
2.85x 108
Calculated Value
of F
0.42
1.01
5.69*
20.55*
J^**
â€”
â€”
â€”
9.79
n 2
â€”
â€”
â€”
0.08
Approximate Power
of Upper Tailed Test
â€”
â€”
â€”
0.10
Greater than F
** Values of II
1'
.05;gp,Ngr *
n 2, and the power of the test were calculated by assuming
0 = (15.141, 122.429, 78.761, 78.275, 87.996, 13.456, 76.948, 34.721)â€™ and
Ã . 2
assuming a = .008.
008
141
142
= 244.75, n2 = 2.00 and the approximate power is .83.
From the entry in Table 12, we see that the calculated F
value of 20.55 with the 30 cell clustering exceeds
F nc;. i a a = 5*81/ and we conclude that the fitted first
â€¢ u j y i y f 4
degree model is inadequate.
4.9 Discussion
When a designed experiment includes replicated points,
the adequacy of a fitted model can be tested by comparing
the portion of the residual sum of squares due to lack of
fit to a second portion due to pure error from the
replicates. The test statistic is an F ratio of the mean
square due to lack of fit to the mean square due to pure
error, and lack of fit is inferred when the calculated value
of this ratio is large (Draper and Smith, 1981, p.120).
When replicate points do not exist, lack of fit can be
tested using near neighbor observations with the test
statisic F = MSEB/MSEW. This F ratio has been shown to
possess a central F distribution when the fitted model is
adequate, and a doubly noncentral F distribution when the
fitted model suffers from lack of fit.
When the fitted model is adequate, the expected values
of both MSEg and MSEW are equal to a^, so that the ratio
E[MSEg]/E[MSEW] equals unity. However, when lack of fit is
present, both MSEg and MSEW are biased estimates of a^, and
we compare the magnitudes of the biases of these estimates
(which are functions of the noncentrality parameters and
degrees of freedom of the doubly noncentral F distribution)
143
in the F test. The test has an upper tailed rejection
region if the bias corresponding to MSEg exceeds the bias
corresponding to MSEW. The rejection region is lower tailed
if the bias corresponding to MSEW exceeds the bias
corresponding to MSEg. In other words, the test is upper
tailed if the matrix D (see Eq. 4.13) is positive definite,
and the test is lower tailed if D is negative definite. If
D is indefinite then the test may be upper tailed, lower
tailed or still yet lack of fit may not be testable
depending upon the value of B^â€¢
In two examples an iterative partitioning clustering
algorithm is used to assign the data points to a preselected
number of near neighbor cells. When the number of cells is
low, the matrix D is found to be indefinite, so that the F
test is not strictly upper tailed or lower tailed. However,
by increasing the number of cells, it is possible in both
examples to produce a positive definite matrix D, so that
the test is upper tailed.
Increasing the number of cells not only produces an
upper tailed test, but also affects the values of the
parameters of the doubly noncentral F distribution. As the
number of cells is increased (moving from left to right in
Tables 9 and 12) we see that the smallest eigenvalue of Cj_
increases and that the largest eigenvalue of C2 decreases.
Therefore a lower bound for ,
22rmin
< n
2a
144
increases as the number of cells increases (where ii> .
ymin
denotes the smallest eigenvalue of C^). In addition, an
upper bound for n ,
n2
22pmax
2a2
decreases as the number of cells increases (where Pmax
denotes the largest eigenvalue of C2). Finally, as the
number of cells increases (moving from left to right in
Tables 9 and 12), the numerator degrees of freedom, g  p,
increase and the denominator degrees of freedom, N  g  r,
decrease. Since the parameters of the doubly noncentral F
distribution change as the number of cells changes, the
power of the F test can be affected. For the stack loss
data example, we see in Table 9 that the power of the upper
tailed test decreases as we move from 9 to 10 to 11 to 12
cells.
An area for future study can be a further investigation
of the effect of the number and composition of near neighbor
cells on the power of the F test which makes use of
F = MSEb/MSEw. This investigation would involve the effect
of near neighbor cell selections on the parameters n^, I^/
g  p, and N  g  r of the doubly noncentral F distribuÂ¬
tion. It would be desirable to develop a method (perhaps an
alternative to the iterative partitioning clustering algoÂ¬
rithm) which could be used to select the number and composiÂ¬
tion of cells so as to maximize the power of the F test.
CHAPTER FIVE
CONCLUSIONS AND RECOMMENDATIONS
Two general methods for testing a linear model fitted
in a mixture space for lack of fit have been investigated in
this dissertation. The first method makes use of response
values observed at check points while the second method
makes use of response values observed at design points which
are near neighbors in the factor space.
In Chapter Two we discussed the work of several authors
(Scheffe (1958), Gorman and Hinman (1962), Kurotori (1966),
and Snee (1971)) for testing lack of fit which centered on
measuring bias inherent in the fitted model when estimating
the response at check points. Only the method suggested by
Scheffe (1958) was an exact test. In Chapter Three, a
method for selecting check points that maximizes the power
*
of Scheffe's F test was devised. When replicate response
observations were available, so that the experimental error
~ 2
variance could be estimated by oext from the replicates, we
saw that the power of this upper tailed F test was maximized
by selecting check points that maximize (or approximately
maximize) the noncentrality parameter of the noncentral F
distribution. When the matrix A^ (where \^ '
was found to be positive semidefinite it was determined
that only a subset of possible values of the parameter
vector could be detected as contributing to lack of fit.
145
146
When an estimate of the experimental error variance was
not available from replicates, an extension of ScheffÃ©'s F
~ 9
test for lack of fit which replaced a ^ with MSE (the
ext
residual mean square error from the fitted model) in the
denominator was developed. We found that to maximize the
power of the test it was necessary to select check points to
maximize (or approximately maximize) the numerator
noncentrality parameter, A , of the doubly noncentral F
distribution when the test was upper tailed. When the test
was lower tailed, we sought check point locations that
minimized (or approximately minimized) X . A criterion was
developed for determining whether the test was upper tailed
or lower tailed by comparing the expected values of the
numerator and denominator of the F ratio when the fitted
model was inadequate. Finally, we discovered cases where,
for some values of $ lack of fit could not be tested.
An alternative to the check points method for testing
lack of fit in a fitted model is a procedure that involves
measuring the bias that is present in estimates of the
response at the design points (the number of design points
must exceed the number of terms in the model). When
replicate observations are available, the well known
procedure in which the test statistic is a ratio of the lack
of fit mean square to the pure error mean square can be used
to test for lack of fit (see Draper and Smith, 1981,
p. 120). When replicate observations are not available,
several techniques which make use of near neighbor
147
observations in place of replicates for testing lack of fit
have been proposed in the literature (see Green (1971),
Daniel and Wood (1971), and Shillington (1979)).
Additionally, it has been suggested by Draper and Smith
(1981, p. 42) that lack of fit can be tested by using near
neighbor observations as substitutes for replicate
observations in the usual lack of fit, pure error F ratio.
However, the exact distributions of the test statistics
proposed by Daniel and Wood (1971) and Draper and Smith
(1981, p. 42) have not been defined, and Green's (1971)
procedure requires an inordinately large number of
observations. Thus because of these reasons we chose
Shillington1s (1979) procedure to study in greater detail in
Chapter Four.
In Chapter Four the distributional properties of
Shillington1s test statistic were developed, and a method
based on an iterative partitioning clustering algorithm for
defining groups of near neighbor observations was
proposed. It was shown that the power of Shillington1s test
depends on the parameters of the doubly noncentral F
distribution, and that the manner in which observations are
grouped as near neighbors can alter the values of the
parameters of the doubly noncentral F distribution and thus
affect the power of the test. We found that increasing the
number of near neighbor cells so that individual cells
become more compact produced an upper tailed F test in the
148
two examples studied, but that there are many other cases
where the test will not be upper tailed.
Now that we have briefly summarized our findings from
investigating the check point and near neighbor methods of
testing lack of fit in a mixture model, a logical question
is, "Which of the two methods is better?" It was not our
original intent to address this question in this
dissertation, but an interesting result that has been
discovered in the latter stages of our investigations is as
follows: Under certain circumstances, the check point
method for testing lack of fit is equivalent to the usual
method which partitions the residual sum of squares into
sums of squares due to lack of fit and due to pure error
(which was shown in Chapter Four to be a special case of the
near neighbor method). Because we have not found a
derivation of the equality of these methods in the
literature, we shall show it here.
In Chapter Three, check points were used to test lack
of fit in a fitted model of the form E(Y) = X8^. With k
check points, the test statistic was of the form (see Eq.
(3.3))
d'V^d/k
Â°ext
where
â€œ 2
Â°ext
is an external estimate of
2
a
which can be
149
calculated from replicates, if they exist. The vector d in
the F ratio was defined to be a vector of differences
between observed and predicted response values at the k
check points having the form
d = Y*  X*(X'X) 1X'Y,
where Y* is the kxl vector of observed response values at
the k check points and X* is the corresponding settings of
. 2
the model terms at the check points. The matrix a Vq was
defined as the variancecovariance matrix of d where Vq has
the form
V. = I, + X*(X'X) 1X*' .
Ok
It can be shown (see Appendix 13) that if we define the
vector Y, as
A
r
Y
observations at the original design points
Y*
observations at the check points
and similarly define the matrix XA as
â€œ â€œ
â€” â€œ
X
design point settings
X*
check point settings
so that the original design points as well as the check
points are all taken at once as design points in regressing
*A Â°n V
then
150
SSE.
= y ' r T
A1 (N+k)
 XA(XÃXA>'lxÃ^A
= d
'vÃ³^
+ SSE,
(5.1)
Thus, the residual sum of squares, SSEA, from the analysis
of the fitted model when both the original design points and
the check points are used to fit the model is equal to the
sum of the quadratic form, d'y^d, used in the numerator of
the check point F test and the residual sum of squares, SSE,
from the analysis of the fitted model using data collected
only from the original design points.
If we perform the usual partitioning of SSEA into a
lack of fit sum of squares, SSLqF^Aj, and a pure error sum
of squares due to replicates, SSEpure^Aj, then from Eq.
(5.1) we can write
SS
LOF ( A)
+ SSE
pure(A)
= d
'v0la
+ SSE.
(5.2)
Thus from Eq. (5.2), when SSEpure(A) e<3ua* to SSE' then
SSLOF(A) becomes equal to d'vâ€1^ so that the check point F
1 "2
ratio, F = (d'VQ d/k)/a0xt, and the usual lack of fit F
ratio, F = MS_ __._./MSE ..., are equivalent. We now
LOF (A)' pure (A)
present an example to illustrate the result in Eq. (5.2).
Let us fit a second degree Scheffe polynomial model to
the following hypothetical or artificial response
observations collected at the six points of the {3,2}
simplex lattice design:
151
" 2350
1
0
0
0
0
0
2370
1
0
0
0
0
0
2450
0
1
0
0
0
0
2430
0
1
0
0
0
0
2650
0
0
1
0
0
0
2670
0
0
1
0
0
0
2400
X =
. 5
.5
0
.25
0
0
2420
.5
.5
0
. 25
0
0
2750
. 5
0
. 5
0
. 25
0
2730
.5
0
.5
0
. 25
0
2950
0
. 5
.5
0
0
. 25
2970 _
0
.5
.5
0
0
. 25
The model is Y = 2360x^ + 2440x^ + 2660x^ + 40x^x^ +
920x^2 + 1640X2X2* Since each of the six design points is
replicated twice, there are six degrees of freedom available
for estimating the experimental error variance. Let an
* 2
estimate of the error variance be 0 . = MSE =
ext pure
SSE /6 = 1200/6 = 200, and this value will be the
pure' '
denominator of the check point lack of fit F ratio.
Let us choose the three points (2/3, 1/6, 1/6),
(1/6, 2/3, 1/6), and (1/6, 1/6, 2/3) as check points and
assume that we have observed the following values at these
points
2690
2/3
1/6
1/6
1/9
1/9
1/36
Y* =
2770
X*
=
1/6
2/3
1/6
1/9
1/36
1/9
2980
1/6
1/6
2/3
1/36
1/9
1/9
The numerator
of
the
check
point
lack
of fit F
ratio
is then
calculated
to
be
24546.
7, so
that
F = (d' V01d/k)/(jgxt = (24546.7/3)/200 = 40.91.
152
If we use all of the observed response values to fit
the second order ScheffÃ© polynomial, then the model is
Y = 2360.7x^ + 2437.7x2 + 2661.7x2 + I83.3X2X2 + 1071.3x^x^
+ 1785.3x2X2 and the residual sum of squares is SSEA =
25746.7. This residual sum of squares can be partitioned
into SSlof(A) = 24546.7 and SSEpure(A) = 1200. The F ratio
for testing lack of fit is calculated to be
F = MSL0F(A)/MSEpure(A) = [24546.7/3]/[1200/6] = 40.91,
which is identical to the previously calculated F value.
In the above example we note that SSEpUre(A) â– 'â€¢s ec3ual
to SSE (SSE = SSEpUre^A)) so that sSL0F^Aj is equal
to d'y^d. Since both the check point F ratio and the usual
lack of fit F ratio have produced the same value, F = 40.91,
we conclude that the two methods for testing lack of fit in
the fitted model are equivalent.
In order to put this dissertation in a better
perspective, we now make some concluding remarks on the lack
of fit testing procedures investigated, including possible
drawbacks, extensions, and recommendations for future work.
An aspect of our investigations that may raise some
questions is that our methods are dependent on the
specification of the form of the true model believed to be
responsible for lack of fit in the fitted model. Requiring
the form of the true model to be specified was necessary in
order to be able to investigate the power of the testing
procedures. There are situations, however, where a complete
or true model can reasonably be specified. One example
153
could be in fitting polynomial models, where the polynomial
of one degree higher than the fitted model could be taken as
the true model.
We now mention two ways in which our results can be
applied to more general situations than may be readily
apparent from our previous discussions. First, we point out
that all examples in Chapters Three and Four dealt with
polynomial models. This type of model was selected because
of its popularity and wide applicability, however, our
methods can be applied not only to polynomial models but to
any models which are linear in their parameters. Secondly,
it was our intent in this dissertation to discuss methods
for testing lack of fit in a mixture model, but the methods
discussed can certainly be used not only in mixture problems
but also in general response surface problems in which a
linear model is fitted. This generalization is illustrated
for the near neighbor approach to lack of fit testing
through the stack loss example in Chapter Four.
Topics for future research stemming from this
dissertation were listed in the concluding paragraphs of
Chapters Three and Four. One area suggested in Chapter
Three was to investigate the effect of experimental design
on the selection of check points and on the resulting power
of the test. Perhaps a "minimum bias" design could be used
for fitting the model, while lack of fit could be detected
with "high bias" check points, but this in only speculation,
and needs to be investigated.
154
The fact that the check points method and the standard
method that partitions the residual sum of squares into lack
of fit and pure error portions were found, under a certain
condition, to be equivalent suggests that selecting check
points to maximize the power of the check point F test may
in general be equivalent to choosing points to augment the
original design. The augmented points would be chosen to
maximize the power of the F test that partitions the
residual sum of squares into lack of fit and pure error sums
of squares. An investigation of the selection of optimal
check points versus the selection of optimal augmented
design points would be of interest.
For the near neighbor test for lack of fit it was
recommended in Chapter Four that other methods besides the
iterative partitioning clustering algorithm might be
considered for selecting groups of near neighbors. The
effect of the number and composition of the groups selected
on the power of the test through their effect on the
parameters of the doubly noncentral F distribution could
then be investigated.
In view of the equivalence of the check point method
and the method that partitions the residual sum of squares
when replicates exist (see Eq. (5.2)), it would be of
interest to investigate whether there is also some
equivalence between Shillington's near neighbor F ratio and
the check point F ratio, F = (d'Vg^d/k)/MSE, to be used when
2
an external estimate of a is not available. If the methods
155
are not equivalent, perhaps one could be shown to be
preferable to the other as judged by comparing the power of
the two procedures in testing for lack of fit.
Finally, the focus of this dissertation has been on
testing lack of fit in linear models so that another area
for future investigation can be the problem of testing lack
of fit in models which are nonlinear in their parameters.
APPENDIX 1
INFLUENCE OF A, ON p{ F" > F }
1 vx ,v2 ;Ai , X 2 a ;vx ,v2
In this appendix we show that PÃ F" > F }
v 1 (^2 iX i ,X2 a ; v i , v 2
is an increasing function of
Let X , . .., X , Y., ..., Y be independent N(0,1).
I v i I v 2
Then
F = (v /v )[(X + \]/2)2 + E X?]/[(Y + X1/2)2 + E Y2]
i=2 1 1 Â¿ i=2 1
is distributed as F" , , where v, and v0 are the
respective numerator and denominator degrees of freedom and
X^ and X2 are the respective numerator and denominator
noncentrality parameters (ScheffÃ©, 1959, p. 412413).
Fixing the values of v2, and X2 we wish to show
PÃF" , , > F } is a strictly increasing function
v 1 ,v2 ,*a 1 ,X2 a;vj ,v2
of Xi, where F represents the upper 100a percentage
*â– a ;v 1 ,v2
point of the central F distribution with and v2 degrees
of freedom. Let
fU^2)
= P( l.j/Vjll (+ X2/2)2 + t X?]/[(Y1 + X2/2)2 + E Y?)
V .
2
i=2
> F
a ;vj ,v2
156
157
then
) may be rewritten as
f (X
p{ (Xi + x^/2)2 > u} = i  p{ (x1 + x^/2)2 < u} ,
( Al. 1)
where u = + X^2)2 + Â£ ^]^.Vl,V2  *â€¢
Note that the random variable U is independent of X^.
If X^/2 and denote any two values of X^A such
that x/2 < *122f tlien we shall prove that for f(A^/2)
defined as in (Al. 1) , < f(X^^). Now
f(x}/2) = 1  / q^h (u)p(u)du where p(u) is the p.d.f. of Ãœ,
1 0 A 1
and for any positive number, u', g,^ (u') denotes the
A l
1/2 2
conditional probability that (X^ + X^ ) < u', given
1/2
1/2
U = u'. However, this conditional probability must be the
same as the unconditional probability, since X^ and U are
statistically independent.
Thus qM (u') is the probability that the random
A1
1/2
variable X^ falls in an interval of half length u'
1/2
centered at X^' . Since X^ ~ N(0,1), this is a decreasing
function of x\^2. Therefore q,h (u')  q.h (u') > 0
1 All Al2
for all u' > 0. Hence,
f(Xll2) " f(X122) = o ^â– _gX^1(u) + 9x^2 (U)^P( U)dU < Â°*
158
Thus PÃF" , , > F } is a strictly increasing
v 1 f v2 1 2
function of Xp
We note that this proof is a modification of the proof
that PÃ F" , , > F } is decreasing in
V1/V2/A1/A2 a ; v 1 , v 2 Â¿
(ScheffÃ©, 1959, p.136).
APPENDIX 2
A CONTROLLED RANDOM SEARCH PROCEDURE
FOR GLOBAL OPTIMIZATION
W. L. Price (1977) describes a conceptually simple
random search procedure, called "a controlled random search
procedure for global optimization," which is effective in
searching for global minima of a function of n variables,
with or without constraints. The procedure does not require
the function to be differentiable or the variables to be
continuous.
An initial search domain, V, is defined by specifying
upper and lower bounds for each of the n variables, and a
predetermined number, N, of trial points are chosen at
random over V, consistent with any constraints. The
function is evaluated at each of the N trial points and the
position as well as the value of the function at each point
are stored in an array, A'. At each iteration a new trial
point, P, is selected randomly from a set of possible trial
points whose positions are related to the configuration of
the N points currently in storage. If P satisfies the
constraints, the function is evaluated at P and the function
value, fp, is compared with fm, which is the greatest
function value for the N points already in storage.
If fp < fm then M, the point in storage corresponding to fm,
is replaced, in the array A', by P. if p fails to satisfy
159
160
the constraints or if f > f then the trial is discarded
p m
and a new point is chosen from the potential trial set.
As the algorithm proceeds, the set of N points in
storage tend to cluster around minima. As Price states,
"the probability that the points ultimately coverge onto the
global minimum (minima) depends on the value of N, the
complexity of the function, the nature of the constraints
and the way in which the set of potential trial points is
chosen."
Price notes that since the procedure is intended to
find global minima, thoroughness of search is more important
than speed of convergence, but if the procedure is to be
more efficient than pure random search the probability of
success (f < f ) at each iteration must be sufficiently
pm
high. His procedure reaches a compromise between the
requirements of search and convergence by defining the set
of potential trial points in terms of the configuration of
the N points already in storage. At each iteration n + 1
distinct points, R,, R_, ..., R ., are chosen at random
1 2 n+1
from the N (N > n) currently in storage and these constitute
a simplex of points in nspace. The point R^+^ is
arbitrarily chosen as the vertex of the simplex, and the
next trial point, P, is taken as the image of the vertex
with respect to the centroid, G, of the remaining n
points. Thus P = 2G  R , . He notes that it is possible
 n+1
to speed up covergence by selecting the vertex as the
point R^, i = 1, 2, ..., n + 1, which has the largest
161
function value of the points R , R, . .., Rn+j_ but this
would be detrimental to the thoroughness of the search.
The version of Price's procedure used in the work in
this dissertation was programmed in the FORTRAN language by
Michael Conlon of the Center for Instructional and Research
Computing Activities, the University of Florida,
Gainesville, Florida. This version of Price's procedure
selects new trial points using the suggested criterion
P = 2G  R The algorithm continues until an iteration
  n+1
limit is reached or a desired tolerance between the minimum
and maximum function values in storage is achieved.
In our particular application, if P2 = 1 so that A^ is
a scalar, we wish to maximize
Ax = (X*  X*A)'V"1(X*  X*A),
with respect to k check points, in order to maximize the
power of an upper tailed test. For locating check points
that maximize the power of a lower tailed test it is
necessary to minimize A^. If p > 1 so that Aj_ is not a
scalar, but is a P2XP2 matrix/ then it will be necessary to
maximize or minimize certain eigenvalues of A^.
All of these optimization problems can be handled by
Price's procedure. Since the procedure finds minima, then
to find maxima, we simply minimize the negative of the
function under consideration. The restriction that the
check points must be located within the experimental simplex
162
(or a subregion of the simplex) is taken care of by
specifying constraints in the program.
To give a specific example, suppose we fit a second
order canonical polynomial model in a three component
mixture space, using a simplex centroid design. If we
assume the true model is special cubic in the three
components, then P2 = 1, and
A1 = {X2 " X* A) ' Vg1 ( XÃp  X* A)
is a scalar quantity. In order to locate a single check
point that maximizes the power of an upper tailed test for
lack of fit, we select the check point that maximizes A^.
Since the experimental region we wish to search is the
entire two dimensional simplex, we define the check point
as x*' = (x^, x2, x^), and in our program impose the
constraints:
and
0 < x < 1,
0 < x 2 < 1.
We then define x^ as x^ = 1  , while requiring
that 0 < < 1. Price's random search procedure is used to
search the twodimensional simplex for the point (x^, x2)
1*
that maximizes A
163
Price suggests the use of N = 50 storage points for
such a twodimensional search, and we have generally found
this to be adequate. For k > 1 check points to be located
simultaneously in a three component system, the problem
becomes one of searching in 2k dimensions. For the
applications considered, N = 50k appears to be adequate.
The only real problem encountered has been that of
economics in that the procedure becomes costly in terms of
computer time for these situations where the optimal value
of the function is assumed by all points in a region. In
these cases the algorithm searches in vain for points that
will improve upon the functional values already in storage,
which all lie in this optimum region. However, in other
applications, the procedure converged quickly to an optimum
(those that converged did so in 10,000 iterations or less,
at a small cost in computational time).
APPENDIX 3
STATISTICAL INDEPENDENCE OF d'Vgd/o AND SSE/oÂ¿
Let us write d'V^d as
d'V^d = (Y*  V*)*V1(Y*  Y*)
^ â€” 1 * ^ _ 1 *
= Y 'VQ Y  Y 'Vq Y
* _ i ~ * * * _i* *
X â€¢Vq Y + Y V0 Y
* _ i * * _ i ~ ~ * _ i * *
= Y ' VQ Y  2 Y 'Vq Y + Y ' VQ Y .
Now let us write SSE as
SSE = Y'(IN  X(X'X) 1X')Y.
k
Since Y and Y are
Y*'Vq1Y*. Rewriti
*
Y
independent,
ng
Y V^Y as
SSE is independent of
where is the least squares estimator of 3^, we have
* _ i ~ * * â€”1* â€”1
X 'v0 X = X 'Vq x (x'x) X'X
164
165
We now show that the second portion of d'y^d is independent
of SSE if and only if
[Vâ€œ1X*(X'X)_1X'][IN  X(X'X)_1X'] = 0.
Define
COv[Y*'V~1X*(X'X)â€œ1X,Y, Y'(I  X(X'X)"1X')Y]
= E[Y*,vâ€œ1X*(X,X)â€1X,YY'(I  X(X'X)_1X' ) Y]
 E[Y*'Vq1X*(X,X)"1X'Y] E[Y'(I  X(X'X)_1X1Y]
= E ( Y* ' ) EIV^X* (X'X)"1X' YY' ( I  X ( X ' X ) _1X â€™ ) Y ]
 E(Y* ' ) E[Vq1X*(X'X)_1X'Y] E[Y'(In  X(X'X)â€œ1X'Y]
= E(Y*') [cov(VqXX*(X1X)_1X'Y, Y'(In  X(X'X)_1X'Y)]
= 0,
if vâ€XX*(X'X)1X'Y is independent of Y'(I  X(X'X)â€1X')Y.
This occurs if and only if,
[Vq1X*(X'X)"1X' ] [I  X(X'X)_:LX'] = 0,
see Searle (1971) p.59, Theorem 3. Now,
166
[v01x*(x,x) 1x,][iN  X(X'X)1x1]
= vâ€œ1X*(X'X)â€œ1X'  V01X*(X'X) 1X'X(X'X) 1x
= 0.
Therefore SSE is independent of the second portion of
d'Vg^d. Now we must show that SSE is independent of the
third portion of d'Y'^d. Write Y*'V~^Y* as
Â»*_]Â»* * l *
Y 'V0XY = (X b^'VgX b1
= Y'X(X'X) 1X*'Vâ€œ1X*(X'X) 1X'Y.
Then SSE is independent of the third portion of d'Vg^d if
and only if
[X(X'X)1X* ,V()1X* (X'X) 1X'][IN  X(X'X) 1X'] = 0,
see Searle (1971), p.59, Theorem 4. Continuing then,
[X(Xâ€™X) 1X* ,V()1X* (X'X) 1X'][IN  X(X'X) 1X']
x(x'x) 1x*'v01x*(x,x)â€œ1x,x(x,x) 1x*,vâ€œ1x*(x,x)1x,x(x,x)1x'
X(X'X) 1X*,V01X*(X'X) 1X'  X(X'X) 1X*'vâ€œ1X*(X'X) 1X'
0.
167
Therefore SSE is independent of the third portion of
S'vÃ³V
Finally, since SSE is independent of each of the three
portions of d'Vg^d, we can conclude that SSE is independent
of d'V~^d and therefore SSE/a^ is independent of d'Y^d/a^.
APPENDIX 4
THEOREM 3.1
Theorem 3.1
Let A and B be kxk matrices. If (A  B) is positive
definite and B is positive semidefinite, then A is positive
definite.
Proof
We assume that (A  B) is positive definite. Then
z'(A  B)z > 0, for all z * 0. Thus z'Az  z'Bz > 0,
for all z * 0, so that z'Az > z'Bz > 0, for all z * 0,
since B is positive semidefinite. Therefore,
z'Az > 0, all z * 0.
Now if z'Az = 0, then z = 0 for if z * 0,
then z'(A  B)z > 0 implies z'Bz < 0. But this is a
contradiction since by assumption z'Bz > 0.
Therefore z must be 0 and A must be positive definite.
168
Theorem 3.2
APPENDIX 5
THEOREM 3.2
Let and A2 be p2xp2 positive semidefinite
matrices. Let g be a p2dimensional vector and define A^
and X2 as
2
Ai = 62A162/2a , and
A2 = BÂ¿A232/2a2,
2
where a >0. Let k>0,N>0,p>0, and N > p.
(a) If [A^/k  A^/(N  p)] is positive definite then
[A ^/k  A2/(N  p)] = 0 if and only if A_ = A2 = 0.
(b) If [A^/k  A2/(N  p)] is negative definite then
[A ^/k  A2/(N  p)] = 0 if and only if X^ = A2 = 0.
Proof of part (a).
Necessity. Let [A^/k  A2/(N  p)] be positive
definite and suppose that [A^k  A2/(N  p)] = 0. We show
that Xi = A 2 = 0. The matrix A^/k  A2/(N  p) being
positive definite implies g^fA^/k  A2/(N  p)]g2 = 0 iff
g = 0 , that is,
2
169
170
2Al2/k â€ 52A22//(N â€œ P) = 0 iff 2 = * But'
2Al2//2k
xiA  X2/^N â€œ P) = 0 iff e2 = 0.
It follows that if Aj/k  A2/(N  p) =0, then A j. = A2 = 0.
Sufficiency. Obviously, if A^ = A2 = 0, then
Aj/k  A /(N  p) =00=0.
Proof of part (b). This follows from part (a), since in
this case A2/(N  P)  A^/k is positive definite.
APPENDIX 6
AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION
Johnson and Kotz (1970, p.197) indicate the following
approximation for PÃF" , , , < F 1 where vi and v
lv^,v2;A^,X2 a;v j ,V2j L
are the numerator and denominator degrees of freedom,
respectively, and X^ and X2 are the numerator and
denominator noncentrality parameters, respectively:
2
PÃ F" . < F
1 V1(v2 ;Xi ,X2 a;vj ,v2
PÃ cF
1 v
V '
< F }
a;vx ,v2 J
= P{F
v , v
< (1/C)F
a ; v i , v 2
where F is the upper 100a percentage point of the cen
a ;v j ,v 2
tral F distribution with v^ and V2 degrees of freedom, and
where c = [1 + ^/v^/U + \ 2/v 2] , v = [vj + X1]2/[v1 + 2\ ^
2
v' = [v^ +^2^ + an<^ Fv v ' a central F random
variable with v and v' degrees of freedom.
171
APPENDIX 7
EQUIVALENCE OF SSEB AND SSL0F WHEN
REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS
In this appendix we show that SSEB = SSL0F when
response observations are partitioned into g groups of true
replicates rather than g groups of near neighbor
observations.
From Chapter Two, Section 2.2, if each cell consists
entirely of true replicates, then the sum of squares due to
lack of fit can be expressed as
SS
LOF
= SSE  SSE
pure
where SSE is the residual sum of squares from a least
squares regression of Y on X and where SSEpure is the sum of
squares due to pure error, calculated from replicates.
Since SSE = Y'Eâ€žY, where T. n is defined as in Section
pure  0 0
4.2, we have
SSL0F = Y'(IN  X(X'X) 1X,)Y  Y'EqY
= Y'(IN  Eq)Y  Y,X(X'X)â€œ1X'Y. (A7.1)
We wish to show that when each cell is composed entirely of
true replicates, SSEg is equal to the expression in (A7.1).
172
173
Recalling from Section 4.3.1 that = MY, where
M = diag[(l/n )1', ..., (1/n ) 1'], we write
L j. y y
SSEâ€ž = ^[G1  G1XC(XÂ¿G1XC)1XÂ¿G1]?C
= Y'M'[G0  Gq Xc(XÂ¿G0 Xc) xÂ¿G0 ]my,
where from Section 4.2, = diag[l/n^, l/n^, . .., 1/n^].
Recognizing that Gq = MM' and Xq = MX, we have
SSEd = Y'[M'(MM')1M
o â€œ L
 M'(MM')1MX{X'M'(MM1)1MX}1X'M'(MM')_1M]Y.
Since M'(MM') ^M = IN  eq, we have
SSEB = ^(IN 
 rÃ
and since E qX = 0 when all cells are composed entirely of
true replicates, we have
SSE = Y'(I  Ert)Y  Y'X(X'X)_1X'Y
B  N 0   
174
from (A7.1). Therefore, SSEB is equal to the usual SSLOp
when cells are composed entirely of true replicates.
APPENDIX tÃ
LEMMA 4.1
Lemma 4.1
2
Let Y ~ (Xg, o G), G singular. Define T = G + XX'.
Define Tâ€” such that TTâ€”T = T.
1. (i) TTâ€”X = X
(ii) X'T~T = X'
2. rank(X'Tâ€”X) = rank(X)
3. (i) X(X'Tâ€”X)â€”(X'T~X) = X
(ii) (X'Tâ€”X)(X'Tâ€”X)â€œX' = X'
4. Y is in the column space of T (Y e C(T)), with
probability one, by which we mean that there exists a
vector a such that letting Y = (y^, Y2/ ..., yN)'
and Ta = (x., xâ€ž, ..., xâ€ž)', then
1 2 N
P{ y^  xÂ¿ > e} = 0 ,for all e > 0, i = 1, 2, ..., N
Proof
1. (i) T = XX' + G
= XX' + W, where G = W
= CC' , where C = [X:V].
Now, CC'(CC')â€”C = C, from Pringle and Rayner (1971, p
26), and therefore
TT [ X: V] = [ X:V]
from which it follows that TTâ€”X = X (and TT~V = V).
175
(ii) The proof of (ii) follows directly from (i) by
taking the transpose.
The proof of part 2 is given in Rao, 1973, p. 77, #30.
The proof of part 3 is given in Rao and Mitra, 1971,
p.22, Lemma 2.2.6(c).
. 2
By definition, Y ~ (Xg , a G) so that an equivalent
representation for Y is Y = Xg + e , where e ~ (0, a2G).
We wish to show that the random vector Y is in the
column space of T, with probability one. It is
sufficient to show that TTâ€”Y = Y, w.p.l. (see Pringle
and Rayner, 1971, p.9). Rewriting TT~Y we have
TTâ€”Y = TT~(Xg + e)
= TTâ€”Xg + TTâ€”e .
By part 1 of Lemma 4.1, TT~X = X, and therefore
Xg e C(T). The proof is complete if we show TTâ€”e = e,
w.p.l. The difference TT~e  e can be written as
TTâ€”e  e = (TTâ€”  I )e, therefore we must show (see
  N 
explanation below) that
E[e1(TTâ€”  In)'(TT_  I )e] = 0. (A8.1)
The expectation in Eq. (A8.1) can be written as
177
E[e'(TT  IN)'(TT  IN)e]
= trace [(TT  I )'(TT  I )
L N N J
= a2 trace [ (TT  I ) ' (TT  I )W']
= 0
since TTâ€”V = V, by proof of part 1(i) of Lemma 4.1.
Therefore Y e C(T), w.p.l.
We now show that proving the equality in (A8.1) is
equivalent to proving that TTâ€”e = e, w.p.l. By the Markov
Inequality
P{ ui  vi > e} < [ E (u Â¿  vi)2]/e2
and therefore if E(uÂ¿  vÂ¿) = 0, we have uÂ¿ = vÂ¿, w.p.l.
If u' = (Uy u2, ...,uN), v' = {Vy v2/ ..., vN) / and if
2
E(u^  v^) = 0, for i = 1, 2, ..., N, then uÂ¿ = vÂ¿, w.p.l,
for i = 1, 2, ..., N, which implies that u = v, w.p.l. But
E(u^  v^)2 = o, for i = 1, 2, ..., N if and only if
N 2
I E(u. v.) = 0, and since
i=l 1 1
N 2
I E(u.  v. ) = E(u  v)'(u  v)
i=l 1 1
we have u = v, w.p.l, if E(u  v)â€™(u  v) =0. In (A8.1) we
= e .
take u
TT e and v
APPENDIX 9
PROOF OF THEOREM 4.1(i)
In this appendix we give the proof of Theorem 41(i).
We show that E(o2) = a2, where o2 = f_1(Y  XB)'T(Y  XB ) .
~ 2
First we write a as
o 2 = f1[Y'Tâ€”Y  2B'X'T_Y + B'X'T~XB] ,
where B = (X1Tâ€”X)â€”X1Tâ€”Y. Now,
B'X'T~XB = B'X1Tâ€”X(X'Tâ€”X)â€”X'Tâ€”Y
= B'X'T_Y,
by Lemma 4.1, part 3(ii). Therefore
a2 = f1[Y'Tâ€”Y  B'X'T~Y]
= f_1[ Y'T~Y  { ( X 1 Tâ€” X)â€”X' T~ Yj'X'T y]
= f_1[ Y'T~Y  Y ' (Tâ€”) ' X ( X' Tâ€”X) ~X ' Tâ€”Y]
= f 1y,aqy
( A9.1)
where A
0
(T )1X(X'T X) X'T . Using equation (A9.1),
178
T
179
and applying Theorem 1(i) (Searle, 1971, p.55), we can write
~ 2
the expected value of a as
E(o2) = f1E[Y'A0Y]
= f_1[ trace{AQa2G} + E(Y)'AQE(Y)]
= f ^ trace[A^a2G] , (A9.2)
since
E(Y)'AqE(Y) = 0'X'[Tâ€œ  (T~)1X(X1T~X)_X'T~]X3
= e'X'T~Xe  3'(X1T~X)(X'TX)â€”(X'Tâ€”X)0
= 0
as X'(Tâ€”)'X = X'T~X, because T is symmetric and
X'T~X is unique (see proof of Theorem 4.1(ii)). Thus
E(a2) = f trace[Ag
= a2f 1 trace[ A^G]
= a2f 1 trace[Ag(T  XX')].
By writing Aq as in Eq. (A9.1), we get
180
 2 2 â€”1
E(a ) = a f trace [{T  (T )'X(X'T X) X'T }{T  XX'}]
2 _ i
= a f trace [T T  T XX'  (T )'X(X1Tâ€”X)â€”X1Tâ€”T
+ (T_) 'X(X'T~X)_X'T~XX']
2 â€”1
= a f [trace T T  trace T XX'
 trace (T) 'X(X'TX)X' + trace (T_)'XX'],
by Lemma 4.1, parts 1 and 3, and so
2 2 â€”1
E(a ) = a f [trace T T  trace (X'T X) (X'T X)],
since X'(Tâ€”)'X = X'Tâ€œX. Since Tâ€”T and (X'T~X)â€œ(X'T_X) are
idempotent, and rank(AA~)= rank(A) for any matrix A, we see
that
E(a 2) = a 2f1[rank(T)  rank(X'TX)]
and by Lemma 4.1 part 2 we have
E(a2) = a2f ^[rank(T)  rank(X)].
181
Finally, since f = rank(G:X)  rank(X), we can write
E(a2) = a2f ^[rank(G:X)  rank(X)] = a2. (A9.3)
The proof of Theorem 4.1(i) is now completed by
justifying the equality in (A9.3) by showing that rank (T) =
rank (G:X). First we write
rank(T) = rank(G + XX').
Replacing G by W , we have
rank(T)
rank(W + XX')
rank(CC ' ) / where C = (V:X)
rank(C)
rank(V:X)
rank(G:X),
since the column space of G is the same as the column space
of V. The column space of G is the same as the column space
of V if the columns of V belong to the column space of G,
and vice versa, if the columns of G belong to the column
space of V. Symbolically, this is written as
V c C(G), and G C(V). To show that V <= C(G), it is
sufficient to show that GGâ€”V = V, but this is true because
GGâ€œV = (W')(W)V = V. Now, G c C(V) since by
definition W = G.
APPENDIX 10
PROOF OF THEOREM 4.1(ii)
In Appendix 10 we prove part (ii) of Theorem 4.1, thus
we show that a2 = f ^(Y  Xg)'T (Y  X3) is unique with
probability one. The following theorem will be useful in
our proof.
Theorem vi(c) (Rao, 1973, p.26).
Let B and D be nonnull matrices. Then BAâ€”D is
invariant for any choice of A if and only if
C(B') c C(A') and C(D) c C(A), where C(.) denotes column
space.
The relationship C(B') <= C(A') holds if and only if BAA = B,
and similarly C(D) c C(A) holds if and only if AAâ€”D = D (see
* 2
Pringle and Rayner, 1971, p.9). Since the quantity a is
written as
a2 = fâ€œ1[Y'Tâ€œY  Y'(Tâ€”)'X(X'Tâ€”X)â€”X'Tâ€”Y],
* 2
to show that a is unique with probabilty one, it suffices
to show that
Y'T Y  Y'(T )'X(X'T X) X'T Y
(A10.1)
182
183
is invariant with probability one to the choice of the
generalized inverses involved.
First we show that Y'TY is unique with probability one
(w.p.l). From part 4 of Lemma 4.1, Y e C(T), w.p.l.
Therefore Y' e C(T'), w.p.l, since T is symmetric, and then
by Theorem vi(c) (Rao, 1973, p.26), Y'TY is unique, w.p.l.
Secondly we show that Y'(Tâ€”)'X(X'Tâ€”X)â€”X'Tâ€”Y is unique
with probability one in the following four part proof.
(1) Show Xâ€™TX is unique.
From part 1 (i) of Lemma 4.1, TTâ€”X = X and thus X<=C(T).
Since T is symmetric, we have X cC(T'). By Theorem vi(c)
(Rao, 1973, p.26), X'TX is unique.
(2) Show X'TY is unique, w.p.l.
By (1) above, X<=C(T') and by part 4 of Lemma 4.1, Ye C(T),
w.p.l. Thus applying Theorem vi(c) (Rao, 1973, p.26),
X'TY is unique, w.p.l.
(3) Show Yâ€™(Tâ€”)'X is unique, w.p.l.
This follows from part (2), since Y'(Tâ€”)'X is equal to the
transpose of X'TY , which was shown in (2) to be unique,
w.p.l.
(4) Using (1), (2), and (3) above, the second quantity
in (A10.1) is unique, w.p.l, by Theorem vi(c) (Rao, 1973,
p.26) if
(a) [Y'CD'X]' e C[(X"Tâ€X)'], w.p.l, and
(b) X'TY Â£ C(X'Tâ€”X), w.p.l.
Part (a) is true not only with probability one but always
because Y'(Tâ€”)'X(X'TX)(X'TX) = Yâ€™(T)'X, since by part
184
3(i) of Lemma 4.1, X = X(X'T X) (X'T X). Part (b) is true
not only w.p.l but always because
(X'Tâ€”X)(X'Tâ€”X)~ X'Tâ€”Y = X'Tâ€”Y, by part 3(ii) of Lemma 4.1.
Therefore we have shown that both Y'TY and
Y'(T~)'X(X'T~X)â€”X'TY are unique with probability one, which
* 2
allows us to conclude that 0 is unique with probability
one.
APPENDIX 11
PROOF OF THEOREM 4 1(iii)
In this appendix we prove Theorem 4.1(iii), that is we
show that if Y possesses an Nvariate normal distribution
A 2 2 2
then fa /a ~ Xf > where f = rank(G:X)  rank(X). Recall
that a2 = f_1Y'AqY where AQ = T  (T_)'X(X'T~X)_X'T~.
â€¢* 2
Since we have shown in Theorem 4.1(ii) that a is unique
with probability one, the choice of the generalized inverses
a 2
in the expression for a may be made arbitrarily. Thus we
choose each of the generalized inverses to be the unique
MoorePenrose inverse, and we denote the unique Moore
Penrose inverse of a matrix B by B+. The MoorePenrose
inverse has the following four properties (see Searle, 1971,
p.16):
1. BB+B = B
2. B+BB+ = B+
3. (BB+)' = BB+
4. (B+B)' = B+B.
~ 2 2
The quantity fa /a can be expressed as
fa2/a2 = Y'AY, (All  1)
where A = (l/a2)[T+  T+X(X'T+X)+X'T+]. We wish to show
185
186
2
that Y'AY ~ Xf â€¢ which can be done by making use of the
following corollary.
Corollary 2s.1 (Searle, 1971, p.69).
When x is N(y,V) whether V be singular or nonsingular,
2
x' Ax ~ x1 with degrees of freedom equal to trace(AV)
and noncentrality parameter equal to (l/2)y'Ay, where
2
x' denotes a noncentral chisquare random variable,
if and only if
(i)
VAVAV ;
= VAV
(ii)
ii
%
P.I
y'AVAV,
(iii)
P ' Ay =
y'AVAy.
In our application, the matrices A and V in Corollary 2s.1
(Searle, 1971, p.69) are defined as
A = (1/a2)[I  T+X(X'T+X)+X']T+,
and
V = a2G .
The proof of Theorem 4.1(iii) follows from Corollary 2s.1
(Searle, 1971, p.69) if we can show that AVA = A. To show
that AVA = A, we first show that AG = AT, where as we
recall, AG = A(T  XX'). Thus AG = AT if AXX' =
the complete expression for A, we have
0.
Using
187
AXX' = (1/a 2)[T+  T+X(X'T+X)+X'T+]XX'
= (1/a 2)[T+XX'  T+X(X'T+X)+(X'T+X)X'],
and so by Lenuna 4.1 part 3 ( i ) ,
AXX' = (1/a 2)[T+XX'  T + XX']
= 0.
2 2
Therefore, since AG = AT, we have AVA = a AGA = a ATA. We
2
now show that a ATA = A:
a2ATA = (1/a 2)[I  T+X(X'T+X)+X']T+T[I  T+X(X'T+X)+X']T +
= (l/a2)[T+TT+X(X'T+X)+X'T+T ][T+T+X(X'T+X)+X'T+]
= (1/a 2)[T+TT+  T+X(X'T+X)+X'T+TT +
 T+TT+X(X'T+X)+X'T+
+ T+X(X'T+X)+X'T+TT+X(X'T+X)+X'T+]
= (1/a 2)[T+  T+X(X'T+X)+X'T+  T+X(X'T+X)+X'T +
+ T+X(X'T+X)+X'T+X(X'T+X)+X'T+]
188
since T+TT + = T+, by property 2 of the MoorePenrose
inverse. Therefore,
ATA = (1/a 2)[T+  2T+X(X'T+X)+X'T+ + T+X(Xâ€™T+X)+X'T + ]
= (1/a ^)[I  T+X(X'T+X)+X']T +
= A.
2
Since we have verified that AVA = a ATA = A we can conclude
that fa 2/o2 = Y' AY ~ ^ , by Corollary 2s.1 (Searle, 1971,
p.69). The quantity fa2/a2 ~ Xf and not xÂ¿2' since the
noncentrality parameter equals zero, which we now show.
The noncentrality parameter, from Corollary 2s.1
(Searle, 1971, p.69) is of the form (l/2)y'Au, where in our
application, y = Xg. Thus,
y'Ay = g'X'AXg
= (1/a 2 )g ' X' [ T+  T+X( X'T+X)+X'T + ] Xg
= (1/a 2)[g 'X'T+Xg  g 'X'T+X(X'T+X) + (X'T+X)g]
and so by Lemma 4.1 part 3(i),
y'Ay_ = ( 1/a 2 ) [ g 1 X ' T+Xg  g'X'T+Xg]
0.
189
We now verify that the degrees of freedom are
f = rank(G:X)  rank(X). From Corollary 2s.1 (Searle, 1971,
p.69) the degrees of freedom associated with Y1 AY are equal
2
to f = trace(a AG), and so
trace(a 2AG) = trace[I  T+X(X'T+X)+Xâ€™]T+G
= trace(T+G)  trace[T+X(X'T+X)+X'T+G]
= trace(T+T  T+XX')  trace[T+X(X'T+X)+X'T+T]
+ trace[T+X(X1T+X)+X'T+XX'],
since G = T  XX1. It follows that
trace(a2AG) = trace T+T  trace T+XX'  trace X(X'T+X)+X'T+
+ trace T+XX',
since trace AB = trace BA for arbitrary matrices A and B,
t+tt+ = T+, and X(X'T+X)+(X'T+X) = X by Lemma 4.1 part
3 ( ii ) . Therefore
trace(a2AG) = trace T+T  trace (X'T+X)(X'T+X)+
= rank(T)  rank(X'T+X),
190
since TT+ and (X1T+X)(X'T+X)+ are idempotent, and
rank(AA+) = rank A, for any matrix A. Finally, by Lemma 4.1
part 2 we have
trace(a2AG) = rank(T)  rank(X)
and by the argument in the proof of Theorem 4.1(i),
trace(a2AG) = rank(G:X)  rank(X).
APPENDIX 12
PROOF OF THEOREM 4.2
In this appendix we prove Theorem 4.2, thus we show
that when Y~ N..(X3 + X03_, o2G) then fa2/o2 ~ xl2w
N â€œ Z 4 Â£ f A
where X = (l/2o 2 )3 ^ [T  T_X(X'T_X)~X'T~]X23 .
~ 2 2 2
From the proof of Theorem 4.1, we have fa /a ~ xÂ¿ ,
t, a
By Corollary 2s.1 (Searle, 1971, p.69) the noncentrality
parameter is
X = (1/2)(X0 + X202)'A(X3 + X232),
where A = (1/o2)[t~  T~X(X'TX)_Xâ€™Tâ€œ]. Thus
X = (1/2) [ 3 'X1AX3 + S'Xâ€™AX^ + B^X^AXB + BÂ£XÂ£AX B ].
From the proof of Theorem 4.1(iii), 3'X'AXB =0. We now
show that 3j,X^AX3 = 0:
3Â£X^AX3 = 3_^X2[ Tâ€”  T_X ( X â€¢ TX) _X' T_] X3 /o 2
= 3J;x^[t_ X " T_ X(X,T~" X)â€œ(X'T X) 3 ] /o 2 ,
and so by Lemma 4.1 part 3(i),
3^X^AX3 = 3^X^[tX3  TXB]/o2
= 0.
191
Thus we conclude that
X = (l/2)gÂ¿X^AX20
= (l/2a2)gÂ¿X^[T
APPENDIX 13
PROOF OF THE EQUALITY SSE = dâ€™V^d + SSE
In this appendix we show that the check point method
for testing a fitted model for lack of fit and the method in
which the residual sum of squares is partitioned into a lack
of fit sum of squares and a pure error sum of squares are
equivalent in the sense that SSE = d'y^d + SSE.
Let us define Y. and X, as
A A
(A13.1)
and
X
A
X
X*
(A13.2)
Then the residual sum of squares from regressing
Y on
A
is
SSE
= rA[i 
XA(XÃXA)_lxÃ]^A*
193
Using Eqs. (A13.1) and (A13.2) we can write SSEA as
_
ssea =
Y
Y*
1
I 
X
X*
(Xâ€™X + X*^*)1
X
X*
1
1

Y
Y*
= Y*'[I  X*(X'X + X*'X*) 1X*,]Y*
 2 Y*1X*(X1X + X*'X*) 1X'Y
+ Y'[I  X(X'X + X*'X*) 1X']Y
Y*'V01Y*  2Y*'X*(X1X + X*'X*)_1X,Y
+ Y'[I  X(X'X + X*'X*) 1X']Y.
(A13.3)
Eq. (A13.3) is true because from Eq. (8) (Morrison, 1976,
69) we can write Vq1 as
v"1 = [I + X*(X'X) 1X*'] 1
= I  X*(X'X + X*'X*) 1x*
195
We now write the quadratic from d'Vg^d as
d ' V^d = (Y*  Y* ) ' Vq 1 (Y*  Y* )
= Y* ' Vâ€"1 Y*  2Y* ' Vâ€1 Y* + Y* ' Vâ€1 Y*
= Y*Vâ€1Y*  2 Y* 1 V^X* ( X ' X ) _1X' Y
+ Y'X(X'X)"1X*,Vâ€œ1X*(X,X)_1X'Y. (A13.4)
The first portion in Eq. (A13.3) is equal to the first
portion in Eq. (A13.4). We now show that the second
portions of Eqs. (A13.3) and (A13.4) are equal. It can be
verified using Eq. (8) (Morrison, 1976, p. 69) that
(X'X + X^X*)1 = (X'X)"1  (X,X)1X*,Vâ€1X*(X,X)_1.
(A13.5)
Using Eq. (A13.5) the second portion of SSEA in Eq. (A13.3)
can be written as
 2 Y*'X*(X1X + X*'X*)_1X'Y
= 2Y*'X* [(X'X)â€1  (X'X)_1X*,V~;LX*(X,X)â€œ1]X, Y
= 2Y*1[I  X*(X'X)_1X*'V"1]X*(X,X)"1X,Y.
(A13.6)
196
The second portion of SSEA, given in Eq. (A13.6),
equal the second portion of d'vâ€1^ in Eq. (A13.4)
fact that
I  X*(X'X)"1X*'Vq1 = I  (VQ  I)Vq1
We now show that the third portion of the expression
for SSEA in Eq. (A13.3) is equal to the sum of the third
portion of d'y^d in Eq. (A13.4) and SSE, where
SSE = Y'[I  X(X,X)â€œ1X']Y. Using the result in Eq. (A13.5),
the third portion of SSEA in Eq. (A13.3) can be written as
Y'[I  X(X'X + X*'X*)1X']Y
= Y'[I  X{(X'X)_1  (X'X)"1X*,Vq1X*(X'X)"1}X,]Y
= Y'[I  X(X'X)_1X']Y + Y'X(X,X)_1X*,Vq1X*(X'X)1X,Y.
Therefore, since the first two portions of SSEA in Eq.
(A13.3) are equal to the first two portions of d'y^d in Eq.
(A13.4), respectively, and the third portion of SSEA in Eq.
(A13.3) is equal to the sum of the third portion of d'Y^d
is seen to
using the
197
in Eq. (A13.4) and Y'[I  X(X'X) ^X']Y, we must have
SSEa = d1V~^d + Yâ€™[I  X(X,X)â€1X']Y
= d 1 V^d + SSE .
then
REFERENCES
Atkinson, A. C. (1972). Planning
inadequate regression models.
275293.
experiments to detect
Biometrika, Vol. 59, pp.
Barr, A. J., J. H. Goodnight, and J. P. Sail (1979). SAS
Users Guide, 1979 Edition. SAS Institute, Inc.,
Raleigh, North Carolina.
Becker, N.G. (1968). Models for the response of a mixture.
Journal of the Royal Statistical Society, _B_, Vol. 30,
pp. 349358.
Becker, N.G. (1978). Models and designs for experiments
with mixtures. Australian Journal of Statistics, Vol.
20, pp. 195208.
Box, G. E. P., and N. R. Draper (1975). Robust designs.
Biometrika, Vol. 62, pp. 347352.
Claringbold, P. J. (1955). Use of the simplex design in the
study of the joint action of related hormones.
Biometrics, Vol. 11, pp. 174185.
Cochran, W.
2nd Ed .
G., and G. M. Cox (1957). Experimental Designs,
John Wiley and Sons, New York.
Cornell, J.
review.
A. (1973). Experiments with mixtures:
Technometrics, Vol. 15, pp. 437455.
A
Cornell, J. A. (1981). Experiments with Mixtures: Designs,
Models, and the Analysis of Mixture Data. John Wiley
and Sons, New York.
Cornell, J. A., and I. J. Good (1970). The mixture problem
for categorized components. Journal of the American
Statistical Association, Vol. 65, pp. 339355.
Daniel, C., and F. S. Wood (1971). Fitting Equations to
Data: Computer Analysis of Multifactor Data for
Scientists and Engineers. John Wiley and Sons, New
York.
Draper N. R., and A. M. Herzberg (1971). On lack of fit.
Technometrics, Vol. 13, pp. 231241.
198
199
Draper, N. R., and W. E. Lawrence (1965a). Mixture designs
for three factors. Journal of the Royal Statistical
Society, _B, Vol. 27, pp. 450465.
Draper, N. R., and W. E. Lawrence (1965b). Mixture designs
for four factors. Journal of the Royal Statistical
Society, _B_, Vol. 27, pp. 473478 .
Draper, N. R., and H. Smith (1981). Applied Regression
Analysis, 2nd Ed. John Wiley and Sons, New York.
Draper, N. R., and R. C. St. John (1977). A mixtures model
with inverse terms. Technometrics, Vol. 19, pp. 3746.
Gorman, J. W., and J. E. Hinman (1962). Simplexlattice
designs for multicomponent systems. Technometrics, Vol.
4, pp. 463487.
Graybill, F. A. (1969). Introduction to Matrices with
Applications in Statistics. Wadsworth Publishing Co.,
Inc., Belmont, California.
Graybill, F. A. (1976). Theory and Application of the
Linear Model. Duxbury Press, North Scituate,
Massachusetts.
Green, J. R. (1971). Testing departure from a regression,
without using replicaton. Technometrics, Vol. 13, pp.
609615.
Johnson, N. L., and S. Kotz (1970). Distributions in
Statistics: Continuous Univariate Distributions  2.
Houghton Mifflin, Boston.
Kenworthy, 0. 0. (1963). Factorial experiments with
mixtures using ratios. Industrial Quality Control, Vol.
19, pp. 2426.
Khuri, A. I., and J. A. Cornell (1981). Lack of fit
revisited. Technical Report No. 167, Department of
Statistics, University of Florida, Gainesville, Florida.
Kurotori, I. S. (1966). Experiments with mixtures of
components having lower bounds. Industrial Quality
Control, Vol. 22, pp. 592596.
McLean, R. A., and V. L. Anderson (1966). Extreme vertices
design of mixture experiments. Technometrics, Vol. 8,
pp. 447454.
Morrison, D. F. (1976). Multivariate Statistical Methods,
2nd Ed. McGrawHill, New York.
200
Mudholkar, G. S., Y. P. Chaubey, and C. Lin (1976).
Approximations for the doubly noncentralF
distribution. Communications in Statistics/ A, Vol. 5,
pp. 4963. â€”
Myers, R. H. (1971). Response Surface Methodology. Allyn
and Bacon, Inc., Boston.
Piepel, G. F. (1981). Measuring component effects in
constrained mixture experiments. Unpublished manuscript
presented at 1981 Joint Statistical Meetings of the
American Statistical Association and the Biometric
Societies, Detroit, Michigan, August 1981.
Price, W. L. (1977). A controlled random search procedure
for global optimisation. The Computer Journal, Vol. 20,
pp. 367370.
Pringle, R. M., and A. A. Rayner (1971). Generalized
Inverse Matrices with Applications to Statistics.
Hafner Publishing Company, New York.
Rao, C. R. (1971). Unified theory of linear estimation.
SankhyÃ¡, A., Vol. 33, pp. 371394.
Rao, C. R. (1972). Unified theory of least squares.
Communications in Statistics, Vol. 1, pp. 18.
Rao, C. R. (1973). Linear Statistical Inference and its
Applications, 2nd Ed. John Wiley and Sons, New York.
Rao, C. R., and S. K. Mitra (1971). Generalized Inverse of
Matrices and its Applications. John Wiley and Sons, New
York.
Scheffe, H. (1958). Experiments with mixtures. Journal of
the Royal Statistical Society, _B, Vol. 20, pp. 344360.
Scheffe, H. (1959). The Analysis of Variance. John Wiley
and Sons, New York.
Scheffe, H. (1963). The simplexcentroid design for
experiments with mixtures. Journal of the Royal
Statistical Society, _B, Vol. 25, pp. 235263 .
Searle, S. R. (1971). Linear Models. John Wiley and Sons,
New York.
Shillington, E. R. (1979). Testing lack of fit in
regression without replication. The Canadian Journal of
Statistics, Vol. 7, pp. 137146.
201
Snee, R. D. (1971). Design and analysis of mixture
experiments. Journal of Quality Technology, Vol. 3, pp.
159169.
Snee, R. D. (1973). Techniques for the analysis of mixture
data. Technometrics, Vol. 15, pp. 517528.
Snee, R. D. (1977). Validation of regression models:
methods and examples. Technometrics, Vol. 19, pp. 415
428.
Thompson, W. O., and R. K. Myers (1968). Response surface
designs for experiments with mixtures. Technometrics,
Vol. 10, pp. 739756.
Wishart, D. (1975). Clustan 1C User Manual. David Wishart,
University College, London.
Zyskind, G. (1967). On canonical forms, nonnegative
covariance matrices and best and simple least squares
linear estimators in linear models. Annals of
Mathematical Statistics, Vol. 38, pp. 10921109.
BIOGRAPHICAL SKETCH
John Thomas Shelton was born on March 30, 1952, in
Jacksonville, Florida, where he resided until graduating
from Englewood High School in June, 1970. He then entered
the University of Florida where he received a Bachelor of
Science degree in mathematics in June, 1974.
John began graduate study at Virginia Polytechnic
Institute and State University in Blacksburg, Virginia, in
September, 1975, and there received a Master of Science
degree in statistics in the summer of 1976. After two years
as a Research Associate at Auburn University in Auburn,
Alabama, he returned to the University of Florida in
September, 1978, where he has since been pursuing a doctoral
degree in statistics. While a graduate student at the
University of Florida, John has worked as a Graduate
Assistant performing statistical consulting duties in the
School of Forest Resources and Conservation.
202
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
Andre' I. KhurT, Chairman
Assistant Professor of Statistics
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
John A. Cornell, Cochairman
Professor of Statistics
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
x
Richard F~. Fisher
Professor of Forest
and Conservation
Resources
This dissertation was submitted to the Graduate Faculty of
the Department of Statistics in the College of Liberal Arts
and Sciences and to the Graduate Council, and was accepted
as partial fulfillment of the requirements for the degree of
Doctor of Philosophy.
May, 1982
Dean for Graduate Studies
and Research

