Citation
Testing lack of fit in a mixture model

Material Information

Title:
Testing lack of fit in a mixture model
Creator:
Shelton, John Thomas, 1952- ( Dissertant )
Khuri, Andre I. ( Thesis advisor )
Cornell, John A. ( Reviewer )
Fisher, Richard F. ( Reviewer )
Place of Publication:
Gainesville, Fla.
Publisher:
University of Florida
Publication Date:
Copyright Date:
1982
Language:
English
Physical Description:
viii, 202 leaves : ill. ; 28 cm.

Subjects

Subjects / Keywords:
Degrees of freedom ( jstor )
Eigenvalues ( jstor )
Least squares ( jstor )
Matrices ( jstor )
Modeling ( jstor )
Parametric models ( jstor )
Point estimators ( jstor )
Polynomials ( jstor )
Statistical models ( jstor )
Statistics ( jstor )
Dissertations, Academic -- Statistics -- UF
Mixtures ( lcsh )
Mixtures -- Mathematical models ( lcsh )
Statistics thesis Ph. D
Genre:
bibliography ( marcgt )
non-fiction ( marcgt )

Notes

Abstract:
A common problem in modeling the response surface in most systems, and in particular in a mixture system, is that of detecting lack of fit, or inadequancy, of a fitted model of the form E(Y) = Xg, in comparison to a model of the form E{Y) = Xe,+ X B postulated as the true model. One method for detecting lack of fit involves comparing the value of the response observed at certain locations in the factor space, called "check points," with the value of the response that the fitted model predicts at these same check points. The observations at the check points are used only for testing lack of fit and are not used in fitting the model. It is shown that under the usual assumptions of independent and normally distributed errors, the lack of fit test statistic which uses the data at the check points is an F statistic. When no lack of fit is present the statistic possesses a central F distribution, but in general, in the presence of lack of fit, the statistic possesses a doubly noncentral F distribution. The power of this F test depends on the location of the check points in the factor space through its noncentrality parameters. A method of selecting check points that maximize the power of the test for lack of fit through their influence on the numerator noncentrality parameter is developed. A second method for detecting lack of fit relies on replicated response observations. The residual sum of squares from the fitted model is partitioned into a pure error variation component and into a lack of fit variation component. Lack of fit is detected if the lack of fit variation is large in comparison to the pure error variation. This method can be generalized when "near neighbor" observations must be substituted for replicates. In this case, the test statistic (assuming independent and normally distributed errors) has a central F distribution when the fitted model is adequate and a doubly noncentral F distribution under lack of fit. The arrangement of near neighbors is seen to affect the testing procedure and its power.
Thesis:
Thesis (Ph. D.)--University of Florida, 1982.
Bibliography:
Includes bibliographic references (leaves 198-201).
General Note:
Typescript.
General Note:
Vita.
Statement of Responsibility:
by John Thomas Shelton.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
Copyright [name of dissertation author]. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Resource Identifier:
028552175 ( AlephBibNum )
09205562 ( OCLC )
ABU5710 ( NOTIS )

Downloads

This item has the following downloads:


Full Text












TESTING LACK OF FIT IN A MIXTURE MODEL


BY

JOHN THOMAS SHELTON



















A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY


UNIVERSITY OF FLORIDA


1982































To Nydra

and

My Parents













ACKNOWLEDGEMENTS

I would like to express my deepest appreciation to Drs.

Andre' Khuri and John Cornell for suggesting this topic to

me and for providing constant guidance and assistance. They

have made this project not only a rewarding educational

experience but an enjoyable one as well. A special word of

thanks goes to Mrs. Carol Rozear for her diligence in

transforming my handwritten draft into an expertly typed

manuscript.


iii













TABLE OF CONTENTS
page

ACKNOWLEDGEMENTS............ ............. .. .............iii

ABSTRACT .............................................. vii

CHAPTER

ONE INTRODUCTION................................... 1

1.1 The Response Surface Problem............... 1
1.2 The Mixture Problem....................... 5
1.2.1 Mixture Models..................... 6
1.2.2 Experimental Designs for Mixtures.. 12
1.3 The Purpose of this Research:
Investigation of Procedures for Testing
a Model Fitted in a Mixture System for
Lack of Fit ............................... 17

TWO LITERATURE REVIEW--TESTING FOR LACK OF FIT..... 19

2.1 Introduction............................... 19
2.2 Partitioning the Residual Sum of Squares.. 21
2.3 Testing for Lack of Fit Without
Replicated Observations--Near Neighbor
Procedures ................................ 26
2.4 Testing for Lack of Fit with Check Points. 33

THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL................. 40

3.1 Introduction............................... 40
3.2 Testing for Lack of Fit in the Presence
of an External Estimate of Experimental
Error Variation............................ 41
3.2.1 The Test Statistic................. 41
3.2.2 The Testing Procedure and an
Expression for the Power of
The Test........................... 45
3.2.3 A Method for Locating Optimal
Check Points ....................... 47
3.3 Testing for Lack of Fit When MSE Is
Used to Estimate Experimental Error
Variation..................................... 51
3.3.1 The Test Statistic................. 51
3.3.2 The Rejection Region and its
Relation to the Power of the Test.. 53
iv








3.3.3 A Method for Locating Optimal
Check Points ....................... 56
3.3.4 Determining Whether the Test Is
Upper Tailed or Lower Tailed....... 58
3.4 Examples. ................................ 67
3.4.1 Theoretical Examples............... 67
3.4.2 Numerical Examples................. 83
3.5 Discussion. ............................... 95

FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR
TESTING LACK OF FIT .............................. 99

4.1 Introduction ............................. 99
4.2 Notation......... ..........................101
4.3 Shillington's Procedure ...................106
4.3.1 Development of MSEB ................109
4.3.2 Development of MSEW................110
4.4 Development of SSE(weighted) .............112
4.5 Equivalence of SSEW and SSEW(weighted)....116
4.6 The Test Statistic .........................118
4.7 The Testing Procedure and its Power........122
4.8 Selection of Near Neighbor Groupings....... 125
4.8.1 Example 1--Stack Loss Data.........129
4.8.2 Example 2--Glass Leaching Data.....134
4.9 Discussion................................ 142

FIVE CONCLUSIONS AND RECOMMENDATIONS ................145

APPENDICES

1 INFLUENCE OF X1 ON
P{F" > F } ...................156
v 1'2 ;11 '12 a;v ,v2

2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR
GLOBAL OPTIMIZATION ............................159

3 STATISTICAL INDEPENDENCE OF d'V d/o2
2 -0-
AND SSE/o .................................... 164

4 THEOREM 3.1................................... 168

5 THEOREM 3.2...................................... 169

6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL
F DISTRIBUTION.................................. 171

7 EQUIVALENCE OF SSEB AND SSLOF WHEN
REPLICATES REPLACE NEAR NEIGHBOR
OBSERVATIONS...................................... 172








8 LEMMA 4.1...................................... 175

9 PROOF OF THEOREM 4.1(i) ............ ......... 178

10 PROOF OF THEOREM 4.1(ii)......................182

11 PROOF OF THEOREM 4.1(iii).............. ........185

12 PROOF OF THEOREM 4.2.............................191
-1
13 PROOF OF THE EQUALITY SSE = d'V d + SSE .....193
A -0 -
REFERENCES .... ....................................... 198

BIOGRAPHICAL SKETCH.. .......... .. ..... ............202













Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in
Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy



TESTING LACK OF FIT IN A MIXTURE MODEL

By

John Thomas Shelton

May 1982



Chairman: Andre' I. Khuri
Cochairman: John A. Cornell
Major Department: Statistics

A common problem in modeling the response surface in

most systems, and in particular in a mixture system, is that

of detecting lack of fit, or inadequancy, of a fitted model

of the form E(Y) = X 1 in comparison to a model of the form

E(Y) = X1 + X 22 postulated as the true model. One method

for detecting lack of fit involves comparing the value of

the response observed at certain locations in the factor

space, called "check points," with the value of the response

that the fitted model predicts at these same check points.

The observations at the check points are used only for

testing lack of fit and are not used in fitting the model.

It is shown that under the usual assumptions of

independent and normally distributed errors, the lack of fit

test statistic which uses the data at the check points is an
vii







F statistic. When no lack of fit is present the statistic

possesses a central F distribution, but in general, in the

presence of lack of fit, the statistic possesses a doubly

noncentral F distribution. The power of this F test depends

on the location of the check points in the factor space

through its noncentrality parameters. A method of selecting

check points that maximize the power of the test for lack of

fit through their influence on the numerator noncentrality

parameter is developed.

A second method for detecting lack of fit relies on

replicated response observations. The residual sum of

squares from the fitted model is partitioned into a pure

error variation component and into a lack of fit variation

component. Lack of fit is detected if the lack of fit

variation is large in comparison to the pure error

variation. This method can be generalized when "near

neighbor" observations must be substituted for replicates.

In this case, the test statistic (assuming independent and

normally distributed errors) has a central F distribution

when the fitted model is adequate and a doubly noncentral F

distribution under lack of fit. The arrangement of near

neighbors is seen to affect the testing procedure and its

power.


viii














CHAPTER ONE
INTRODUCTION

1.1 The Response Surface Problem

A mixture problem is a special type of a response

surface problem. First we shall define the general response

surface problem and indicate the basic objectives sought in

its analysis, and follow this development with a discussion

of the mixture problem.

In the general response surface problem, we are inter-

ested in studying the relationship between an observable

response, Y, and a set of q independent variables or

factors, xl, x2, ..., Xq, whose levels are assumed con-

trolled by the experimenter. The independent variables are

quantitative and continuous. We express this relationship

in terms of a continuous response function, p, as



Y = (ul, x2 uq ) + u



where Yu is the uth of N observations of the response col-

lected in an experiment, and xui represents the uth level of

the ith independent variable, u = 1, 2, ..., N, i = 1, 2,

..., q. The exact functional relationship, p, is unknown.

The term Eu is the experimental error of the uth







observation. It is assumed that E(cu) = 0, E(Eueu,) = 0,

for u u', and E(e ) = 2, for u = 1, 2, ..., N.

As the form of ( is unknown and may be quite complex, a

low order polynomial (usually first or second order) in the

independent variables xl, x2, ..., Xq is generally used to

approximate p. This may be justified by noting that such

polynomials constitute low order terms of a Taylor series

expansion of < about the point xl = x2 = ... = xq = 0,

(Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point

out that these low order polynomials may give a poor approx-

imation to q when extrapolated beyond the experimental

region, and thus should not be used for this purpose.

A linear response surface model may be written in

matrix notation as



Y = X4 + E (1.1)



where Y is an Nxl vector of observable response values, X is

an Nxp matrix of known constants, a is a pxl vector of

unknown parameters (regression coefficients), and a is the

Nxl vector of random errors. When the model is a first or a

second degree polynomial, the columns of X correspond to the

first or second degree powers of the independent variables

xl, x2, ..., Xq, or their cross products. If the model

contains a constant term, 80, the first column of X will

correspond to this term, and will consist of N ones. Since

E(c) = 0, an alternative representation for the response







surface model of (1.1) is



E(Y) = X .



Once the form of the model that will be used to approx-

imate ((xl, x2, ..., Xq) is chosen, the next step is to

estimate the regression coefficients, a, and then use the

estimated model to make inferences about the true response

function, (. The estimation of the elements of a is usually

accomplished by ordinary least squares techniques. For the

purpose of testing hypotheses concerning the regression

coefficients, ., it is assumed that L has a normal distribu-

tion, that is, e ~ NN(O, a 2N).

Perhaps the most common objective in the exploration of

a response system is the determination of its optimum

operating conditions. By this we mean that it is desired to

find the settings of xl, x2, ..., xq that optimize (, which

in some applications may be interpreted as maximizing p,

while in other applications a minimum value of ( may be of

interest. It is also often desirable to determine the be-

havior of the response function in the neighborhood of the

optimum. For second order response models, such an investi-

gation can be carried out by performing a canonical analysis

of the second order surface as discussed in Myers (1971).

For simple systems having only one or two independent

variables, the response surface may be explored by just

plotting the fitted response values against values taken by







the independent variables. If q = 1, implying only one

independent variable, say xl, then a two-dimensional plot of

the fitted response against xl may be used to locate the

optimum, as well as to investigate the response behavior in

other parts of the experimental range of xl. If q = 2, and

the two independent variables are xl and x2, then a plot of

the contours of constant response over the region specified

by the ranges of the values for xI and x2 can be used to

describe the response surface.

The properties that the fitted model possesses in terms

of its ability to represent the true surface, p, depend on

the settings of xl, x2, ..., Xq at which values of Y are

observed. Thus the experimental design is of great impor-

tance. Much work has been done on the construction of

designs that are optimal with respect to one criterion or

another involving the fitted response and/or the true unfit-

ted model. Box and Draper (1975) list fourteen criteria to

consider when choosing a design for investigating response

surfaces. Myers (1971) gives several designs for fitting

first and second order polynomial models. A discussion of

specific design considerations will not be attempted here,

as such a discussion is not the focus of this dissertation,

and would necessarily be lengthy.

The initial steps in the analysis of a response system

may be described as follows: First an attempt is made to

approximate the true response function, p(xl, x2, ..., Xq),

usually with a low order polynomial in xl, x2, ..., xq.







After the form of the model has been chosen, then comes the

selection of an appropriate experimental design, which

specifies the settings of the independent variables at which

observed values of the response will be collected. The

observed values of the response are used in estimating the

regression coefficients in the model, using, in general,

ordinary least squares. After a test for "goodness of fit"

of the model verifies the fitted model is adequate, the

fitted model is used in determining optimum operating condi-

tions for the response system.

1.2 The Mixture Problem

A mixture system is a response system in which the

response depends only on the relative proportions of the

components or ingredients present in a mixture, and not on

the total amount of the mixture. For example, the response

might be the octane rating of a blend of gasolines where the

rating is a function only of the relative percentages of the

gasoline types present in the blend. The proportion of each

ingredient in the mixture, denoted by xi, must lie between

zero and unity, i = 1, 2, ..., q. The sum of the propor-

tions of all the components will equal unity, that is,


q
0 < x. < 1, i = 1,2,...,q, E x. = 1. (1.2)
i=l


The factor space containing the q components is represented

by a (q 1)-dimensional simplex. For q = 2 components, the

factor space is a straight line, whereas for q = 3







components, the factor space is an equilateral triangle, and

for q = 4 components, the factor space is represented by a

regular tetrahedron.

The objectives in the analysis of a mixture response

system are, in general, the same as in any response surface

exploration. That is, one seeks to approximate the surface

with a model equation by fitting an equation to observations

taken at preselected combinations of the mixture com-

ponents. Another objective is to determine the roles played

by the individual components. We shall not concern our-

selves with this but rather concentrate on the empirical

model fit. Once the model equation is deemed adequate an

attempt is made to determine which of the component combina-

tions yield the optimal response. The models used to repre-

sent the response in a mixture system are in most cases

different in form from the standard polynomial models. The

first type of model form that we discuss is the canonical

polynomial suggested by Scheffe.

1.2.1 Mixture Models

Scheffe (1958) introduced a canonical form of the poly-

nomial model for representing the response in a mixture

system. These canonical polynomial models are derived from

the standard polynomials using the restrictions on the xi

shown in (1.2). With q = 2 mixture components, for example,

the standard second order polynomial model is of the form


2 2
E(Y) = a0 + alx1 + a2x2 + a12 x2 + all x + a22x2 (1.3)







Restrictions (1.2) imply that a0 = a0(xl + x2)'

x2 = l(- and x2 = 2(- thus (1.3) can be
1 xl(l x2), x2 x2(l -xl)
written in the canonical form



E(Y) = 81x1 + 2x2 + 12XlX2



where 1= a0 + al + all' 82 = a0 + a2 + a22' and 812= al2

- a11 a22. There is no constant term in the above canoni-

cal form and the pure quadratic terms in equation (1.3) have

been absorbed in the xixj terms.

The general form of the canonical polynomial of degree

d in q mixture components can be written as


q
E(Y) = E 8ixi, for d = 1,
i=l

q q
E(Y) = Z i.x. + Z E .x.x. for d = 2, and
i=l 1. i
q q q
E(Y) = a .x. + EZ .x.x. + E E 6..x.x.(x. x.)
i=l 1 i
q
+ E a ijkx.x.xk for d = 3. (1.4)
1Ki

The fourth degree canonical polynomial in q components is

given in Cornell (1981, p. 64). The general canonical poly-

nomial of degree d > 4 in q components does not explicitly

appear in the literature, but is mentioned in Scheffe

(1958). If terms of the form 6ijxixj(xi xj) are removed

from the full cubic model (1.4), then the remaining terms







make up what is referred to as the special cubic model. For

example, for q = 3 components, the special cubic model is



E(Y) = 1x1 + 82x2 + 3x3 + a12 2 + 813X1X3



+ 823x2x3 + 123XlX2X3



Scheffe's canonical polynomial models are used for

approximating the response surface in many mixture systems.

Their popularity stems from the ease in interpreting the

coefficient estimates, especially when the models are fitted

to data collected at the points of the associated designs

(see Section 1.2.2). However, other models have been intro-

duced which seem to better represent the response when the

components have strictly additive blending effects. We

present some of them now.

Becker (1968) introduced three forms of homogeneous

models of degree one which he recommends instead of the

polynomial models when one or more of the mixture components

have an additive effect or when one or more components are

inert. A function f(x, y, ..., z) is said to be homogeneous

of degree n when f(tx, ty, ..., tz) = tnf(x, y, ..., z), for

every positive value of t and (x, y, ..., z) (0, 0, ...,

0). These models, which Becker refers to as models HI, H2,

and H3, are of the form








q q
HI: E(Y) = Z B.x. + Z Bijmin(xi, x ) + ...
i=1 1 i

+ 12...qmin(xl, x 2, ..., x )


q q
q q 2-1
H2: E(Y) = E Bx. + Z Bix. .x ./(x. + x .) +
i= 13 1 3 1
i=1 14i

+ 812...qxlx2..x /(x + x2 + ... + xq)q-


q q 1
H3: E(Y) = 6 .x. + Z . (x.x 1/2
i=1 l i

+ Bl2...q(X l2...xq)/q


Each term in the H2 model is defined to be zero when the

denominator of the term is zero.

Draper and St. John (1977) suggest a model which in-

cludes inverse terms, 1/xi, in addition to terms in the

Scheffe polynomials. Such a term is used to model an

extreme change in the response as xi approaches zero. The

experimental region of interest is assumed to include the

region near the zero boundary (xi = 0), but does not include

the boundary itself. One example of this type of model is

the Scheffe linear polynomial model with inverse terms


q q -1
E(Y) = Z Bixi + Z B .x .
i=l i=l -







Another model form that is useful in the study of the

response in a mixture system is the model containing ratios

of the component proportions. A term such as xi/xj measures

the relationship of xi to xj rather than the percentage of

each in the blends. Snee (1973) points out that the ratio

model presents a useful alternative to the Scheffe and

Becker models in that the ratio model describes a different

type of curvature. He notes that the curvilinear terms for

the Scheffe and Becker models, when plotted as a function of

xi, are symmetric functions about xi = 1/2, whereas the

ratio term xi/xj is a monotone function when plotted against

xi.

The terms in the ratio models may also contain sums of

the components. For example, with q = 3 components, we

might express the second order model


q-1 q-1 q-1
E(Y) = B + E izi + EE ijziz. + E 68iz.
i=l l

(note the constant term) where zl and z2 are defined as

z1 = xl/(x2 + x3) and z2 = x2/x3. Some terms will be unde-

fined if points from the boundary of the experimental sim-

plex are included in the design, for example, if x3 = 0,

then z2 = x2/x3 is not defined. Snee (1973) suggests adding

a small positive quantity, c, to each xi in this case.

This, of course, will not be of concern if the experimental

region is entirely inside of the simplex.







When one or more of the components is inactive, Becker

(1978) suggests that a ratio model that is homogeneous of

degree zero in the remaining components is appropriate. In

three components, such a model is of the form



E(Y) = 80 + 1X/(x1 + x2) + 2x2/(x2 + x3)


3
+ 83x /(x + x ) + Z BZ ..h..(x., x )
s3 3 1 3 1i li

+ 8123h123(xl, x2, x3), (1.5)



where hij and h123 are specified functions that are homoge-

neous of degree zero. The function h123 is intended to

represent the joint effect of all three components simulta-

neously. If in fitting a model of the form (1.5) we deter-

mine the model should be



E(Y) = 80 + a1X1/(X1 + x2) + 812h12(xl, x2)



then component three is said to be inactive and is removed

from further consideration. The model of equation (1.5) may

produce an extreme value near the vertices of the simplex

factor space when there are no inactive components. In this

case it is suggested that a model of the form (1.5) be used

only when the proportions are restricted so that no two of

the xi are simultaneously very close to zero. Becker notes

that other authors who have suggested ratio models have also







used them primarily over a subregion inside the simplex

factor space. Apparently this is where they are most appro-

priate.

1.2.2 Experimental Designs for Mixtures

As in the general response surface problem, one of the

major concerns in exploring a mixture system is that of

choosing the experimental design for collecting observed

values of the response that will be used in fitting the

model. Scheffe (1958) proposed the {q,m} simplex lattice

designs for exploring the entire q-component simplex factor

space. In these designs, the proportions used for each

component have the m + 1 values spaced equally from zero to

one, xi = 0, 1/m, 2/m, ..., (m l)/m, 1, and all possible

mixtures with these proportions for each component are

used. The number of design points in the {q,m} simplex

lattice design is (m + q 1). The main appeal of these
m
designs is that they provide a uniform coverage of the fac-

tor space. Another feature, which Scheffe (1958) demon-

strates, is that the parameters of the canonical polynomial

of degree m in q components are expressible as simple linear

combinations of the true response values at the design

points of the {q,m} simplex lattice. The {3,2} simplex

lattice, which consists of six design points, is represented

on the two dimensional simplex in Figure 1 along with the

triangular coordinates (xl, x2, x3).

Scheffe (1963) also developed the simplex centroid

designs consisting of 2q 1 points, where the only mixtures







considered are the ones in which the components present

appear in equal proportions. That is, in a q-component

simplex centroid design, the design points correspond to the
q
q permutations of (1, 0, 0, ..., 0), the () permutations of

(1/2, 1/2, 0, ..., 0), the (3) permutations of (1/3, 1/3,

1/3, 0, ..., 0), ..., and the point (1/q, l/q, ..., l/q).

This design alleviates the problem inherent in the {q,m}

simplex lattice designs of observing responses at mixtures

containing at most m components. To give an example, the

q = 3 simplex centroid design is made up of 23 1 = 7

design points, and is equivalent to the {3,2} simplex

lattice design augmented by the center point (xl, x2, x3)

(1/3, 1/3, 1/3). This design is represented in Figure 2.

Scheffe (1963) mentions that the number of parameters

in the polynomial model of the form


q q q
E(Y) = .ix. + EE B. .x.x. + EE 8ikx. xxk
i=l 1 i

+ ... + B12...qxlx2 ... q (1.6)



is 2q 1 and therefore these models have a special rela-

tionship with the simplex centroid design in q components.

This relationship is that the number of terms in the model

equals the number of points in the design and as a result

the parameters in model (1.6) are expressible as simple

functions of the responses at the 2q 1 points of the sim-

plex centroid design. Polynomial models of the form (1.6)



















(2 2


(0,1,0)




Figure 1.













(





(0,1 ,0)


(0,0,1 )


x =I ( x, I )
2 2'2 3


The {3,2}


simplex lattice design.


x =I


(0,0,1)


X3


The q = 3 simplex centroid design.


x I)
~~2 2'2


Figure 2.







therefore are natural models to fit using the simplex cen-

troid design.

Ratio models may be desirable when the interest in one

or more of the mixture components is in terms of their rela-

tionship to one another, rather than in terms of their per-

centages in blends. Kenworthy (1963) proposed factorial

arrangements for ratio variables. An example of the use of

ratios is the following three component system in which the

mixture components are constrained by the upper and lower

bounds:



.2 < x1 < .4, .2 < x2 < .4, .3 < x3 < .5. (1.7)



The ratio variables of interest are zl = x2/xl and

z2 = x2/x3, and it is desired to fit either a first or a

second order polynomial model in zI and z2. For such a

problem, we can define a 22 and a 32 factorial design that

can be used for fitting the first and second order poly-

nomial models, respectively, by taking as design points the

intersection of rays passing from two of the three vertices

of the two-dimensional simplex through the region of

interest defined by the constraints (1.7). Kenworthy's 22

factorial design is shown in Figure 3.

Becker (1978) uses rays extending from one or more

vertices of the simplex factor space to the opposite bound-

aries in developing "radial designs." These designs are

suggested for detecting the presence of an inactive











x :




0 Design Points









x :1 x :I
2 3


Figure 3. Kenworthy's 22 factorial design.



component or in another case a component which has an addi-

tive effect, when models containing ratio terms that are

homogeneous of degree zero are fitted.

McLean and Anderson (1966) suggest an algorithm for

locating the vertices of a restricted region of the simplex

factor space which is defined by the placing of upper and

lower bounds on the mixture component proportions. The

vertices of the factor space and convex combinations of the

vertices are the candidates for design points for fitting a

first or second degree polynomial model in the mixture com-

ponents. One drawback of the "extreme vertices" design is

that the design points are not uniformly distributed over

the factor space resulting in an imbalance in the variances

of Y(x), see Cornell (1973).







Another method that has been suggested for studying the

response over a sub-region of the simplex mixture space is

to transform the q mixture components into q 1 independent

variables. Transforming to an independent variable system

was first suggested by Claringbold (1955) and later proposed

by Draper and Lawrence (1965a, 1965b) and Thompson and Myers

(1968). Standard response surface polynomial models in the

transformed variables can be fitted to data values collected

on standard designs and a design criterion such as the aver-

age mean square error of the response can be employed when

distinguishing between designs. Thompson and Myers (1968)

suggest the use of rotatable designs (see also Cornell and

Good, 1970).

Designs other than rotatable designs, such as multiple

lattices and symmetric-simplex designs, to name a few, have

been suggested in the literature for fitting models to a

mixture system which may be appropriate depending on par-

ticular experimental situations. However, as the intent

here is not to give an exhaustive list but only a sampling

of available designs, we shall not discuss designs further

but instead state the purpose of this work.

1.3 The Purpose of this Research:
Investigation of Procedures tor Testing a Model
Fitted in A Mixture System for Lack of Fit

A common problem in modeling the response in a mixture

system is that of detecting lack of fit, or inadequacy, of a

fitted model of the form E(Y) = XI when the true model is

of the form E(Y) = X2 + X2 2. The statistical literature







suggests several procedures for testing lack of fit, which

will be described in Chapter Two. In general, the authors

of these procedures are not specific in stating hypotheses

to be tested and do not adequately discuss the power of

their procedures.

The major purpose of this research is to investigate

the power of two of the testing procedures appearing in the

literature in detecting the inadequacy of a fitted model

when the general form of the true model is specified. Our

findings for a "check points" lack of fit testing procedure

are presented in Chapter Three while Chapter Four contains

findings for a "near neighbor" lack of fit testing proce-

dure. For both procedures, explicit formulas for the power

of the test are given in terms of cumulative probabilities

of either the noncentral F or doubly noncentral F distribu-

tion, which are derived by assuming that the response obser-

vations are independent and normally distributed. Addition-

ally, we propose methods for maximizing the power of the

testing procedures. In the final chapter, we make some

concluding comments concerning both of these procedures.













CHAPTER TWO
LITERATURE REVIEW--TESTING FOR LACK OF FIT

2.1 Introduction

Let us return to the general response surface problem

and assume the true response is to be approximated by

fitting a model of the form



E(Y) = XI1 (2.1)



where Y is an Nxl vector of observable response values, X is

an Nxp matrix of known constants, and El is a pxl vector of

unknown regression coefficients. We wish to consider the

situation in which the true model contains terms in addition

to those in the fitted model. Then the true model has the

form



E(Y) = X1 + X2a2 (2.2)



where X2 is an Nxp2 matrix of known constants, and 12 is a

P2xl vector of unknown regression coefficients. We assume
that the vector Y has the normal distribution with

var(Y) = a 2N

It is desirable to determine the suitability of the

fitted model given by Eq. (2.1) when in reality the true

model is of the form given by Eq. (2.2). The process of
19







making this determination is referred to as testing for lack

of fit of the fitted model.

There are three general approaches to testing for lack

of fit. The first approach requires that there be replicate

observations of the response at one or more design points,

and involves partitioning the residual sum of squares from

the fitted model into a sum of squares due to lack of fit

and a sum of squares due to pure error. A large value for

the ratio of the mean square due to lack of fit to the mean

square due to pure error provides evidence for lack of fit.

If replicate observations are not available then the

above approach to testing for lack of fit cannot be used.

Green (1971), Daniel and Wood (1971), and Shillington (1979)

have proposed alternative methods that are applicable in

this case. Their approach is to group values of the

response which are observed at similar settings of the

independent variables and to call these grouped values

"pseudoreplicates" or "near neighbor observations." They

then treat these pseudoreplicates as they would treat true

replicates to form statistics for lack of fit testing,

although arriving at their respective statistics through

different approaches.

A third approach to testing for lack of fit involves

the use of "check points." In this method a model of the

form (2.1) is fitted to data at the design points and

additional observations are collected at other points in the

experimental region. The additional points other than the







design points are called check points, and the data at these

check points are not used in fitting the model. Lack of fit

is tested by comparing the values of the response observed

at the check points to the values of the response which the

fitted model predicts at these same check points.

We now discuss the first method mentioned above of

testing for lack of fit which involves partitioning the

residual sum of squares.

2.2 Partitioning the Residual Sum of Squares

The method for testing lack of fit which makes use of a

partitioning of the residual sum of squares from the fitted

model requires there be replicate observations of the

response at some of the design points (Draper and Smith,

1981, p. 120). When a model of the form (2.1) is fitted,

the residual sum of squares is defined as

n.
n 1 2
SSE = EZ (Y Y )
i=1 j=l 1

-1
=Y'(I' X(X'X) X')Y



where n is the number of distinct design points, ni > 1 is

the number of replicate observations at the ith design

point, Yij is the jth observed value of the response at the

ith design point, Yi is the value which the model of the

form in Eq. (2.1), fitted by ordinary least squares

techniques, predicts for the response at the ith design
n
point, and N = Z n. Using the replicated observations
i=l 1







only, a pure error sum of squares can be calculated as


n "i 2
SSE = E E (Y. Y. )
pure i=l 13 '
i=1 j=1


where Yi. is the average of the values of the response

observed at the ith design point. The sum of squares due to

lack of fit can be obtained by taking the difference


LOF


=SSE SSE
pure


This partitioning of the residual sum of squares is

displayed in the analysis of variance table in Table 1.


Table 1. Analysis of Variance--
Partitioning the Residual Sum of Squares.


Source
of Variation

Regression

Residual

Pure Error

Lack of Fit

Total(corrected)


Sum
of Squares

b{X'Y (1'Y)2/N

SSE

SSEpure

SSLOF
Y'Y (l'Y)2/N


Degrees
of Freedom

p 1

N p

N n

n p

N- 1


bl represents the ordinary least squares estimator of B in
-1
model (2.1), b = (X'X) X'Y, and 1 is an Nxl vector of
ones.
ones.


Mean
Square



MSE

MSEpure

MSLOF







To test the hypothesis of zero lack of fit, that is

HO: lack of fit = 0 or E(X) = XEI, an F statistic is formed


MS
LOF
F = F (2.3)
MSE
pure


which possesses a central F distribution if the true model

is of the form (2.1), but has a noncentral F distribution if

the true model is of the form (2.2). In other words



F ~ F
n-p,N-n


under H : E(Y) = X8_ and



F ~ F'
n-p,N-n;X2


under H : E(Y) = XB + X2_ where X2 is the noncentrality
a 1+ 22 2
parameter 2 = B(X2-XA)'(X2-XA)B2/22 and A = (X'X)- X'X2

Under H E(MSLOF) = 2 + (X2 XA)'(X2 XA) 2/(n-p) and

E(MSEpure) = o2 (Draper and Smith, 1981, p. 120), hence HO

is rejected in favor of Ha if the value of F in (2.3)

exceeds the upper 100a percentage point of the central F

distribution, Fa;n-p,N-n. When HO is rejected, we conclude

that a significant lack of fit is present.

Draper and Herzberg (1971) demonstrated that the lack

of fit sum of squares can be partitioned into two

statistically independent sums of squares, SSL1 and SSL2'

when there are replicate observations at the center of the







response surface design and when the basic design without

center points is nonsingular. If the true model and the

fitted model are of the same form as in equation (2.1) then

the two F ratios FL1 = [SSLl/(n p 1)]/MSEpure and

FL2 = SSL2/MSEpure are both distributed as central F random

variates, with respective numerator and denominator degrees

of freedom (n p 1), (N n) for FL1 and 1, (N n) for

FL2. If the true model is of the form shown in equation

(2.2), then FL1 and FL2 are both distributed as noncentral F

random variates. The expected values of SSL1 and SSL2 are

used to show what functions of E2 are testable with FL1 and

FL2*

Two examples are presented by Draper and Herzberg to

illustrate this testing for lack of fit. The first example

makes use of a first order orthogonal design in k factors

augmented with center point replicates for fitting a first

order polynomial model. If the true model is of the second

order, then FL2 can be used to test a hypothesis concerning

the parameters associated with the pure quadratic terms in

the model. If all such parameters are zero, then FL1

provides a check on interaction terms. The second example

illustrates the fitting of a second order polynomial model

to a second order design with all odd design moments of

order six or less zero. If the true model is third degree,

then FL1 can be used to test the significance of the third

order terms, while FL2 tests terms of order greater than

three. The partitioning of SSLOF into SSL1 and SSL2 and the







corresponding tests of hypotheses are also given in Myers

(1971, p. 114-119), for the special case of fitting a first

order polynomial model to a 2q factorial or a fraction of a

2q factorial design augmented with center point replicates

and the true model is of the second degree.

A more complete partitioning of the lack of fit sum of

squares in an attempt to obtain a more detailed diagnosis of

the lack of fit of the fitted model is given in a technical

report written by Khuri and Cornell (1981). The lack of fit

sum of squares, which has n p degrees of freedom, is

partitioned into n p independent sums of squares, each

having one degree of freedom. The expected values of these

single degree-of-freedom sums of squares are used to

identify at most n p linearly independent causes for the

lack of fit variation. Tests of significance are performed

on the assumed contributing causes. This method enables the

screening of all subsets of E2 in order to identify those

subsets which are most responsible for lack of fit of the

fitted model.

We shall now discuss the second general approach used

in lack of fit testing, which is to test for lack of fit by

making use of response values observed at points which are

near neighbors in the factor space when true replicate

observations are not available.







2.3 Testing for Lack of Fit Without
Replicated Observations--Near Neighbor Procedures

Green (1971) suggests the following approach when

testing for lack of fit if there are no design points at

which replicate observations of the response are

available. The N observed values of a response, Y,

considered a function of only one variable, x, are divided

into g groups, by grouping observations which have similar

values of x. Green hypothesizes a model of the form Y= Ha +

e, where Y is an Nxl vector of observable responses, H is an

Nxm matrix whose columns correspond to known functions of

the variable, x, is an mxl vector of unknown regression

coefficients, and e is the Nxl vector of random errors,
2
N ~ N (0, o N).

Green's method assumes that the vector of differences

(EY Hg) can be well approximated by a dth order polynomial

in x within each of the g groups, d > 1. An alternative

model of the form



Y = H v + n +



is given, where is distributed as NN(Q, o21N), H1 is an

Nx [g(d + 1) + ml] matrix of known constants, u is a

[g(d + 1) + ml]xl vector of regression coefficients, and .,

as Green states is "a small vector." The first g(d + 1)

columns of H1 correspond to the polynomial terms for the g

groups (with (d + 1) terms for each group), the rightmost

mI < m columns in H1 correspond to terms that are in the







fitted model, but are not represented among the g(d + 1)

polynomial terms in the alternative model.

Under the assumption that a = Q, the presence of lack

of fit is tested by using the test statistic:




Y'[H (HHl) -H H(H'H) 1H']Y/[g(d + 1) + mi m]
F = *----------------------
-1
Y'[I H (HH1I) H{]Y/[N g(d + 1) mi]


(2.4)



This statistic is of the same form as the F statistic used

in the standard multiple regression test of a postulated

model against a more general one which includes the

postulated model as a special case. Lack of fit is

suspected if the calculated F ratio in (2.4) is greater

than Fa;g(d+l)+ml-m, N-g(d+l)-ml where this latter quantity

is the upper 100a percentage point of the central F

distribution.

Green notes that when there is no lack of fit, the

quadratic forms Y'[H1(HIH ) -H H(H'H) H']Y and

Y'[I H (HIH,) Hi]Y are distributed independently as

o2X2 with g(d + 1) + mi m and N g(d + 1) mi degrees of

freedom, respectively. In this case the F ratio in (2.4)

possesses a central F distribution. If there is lack of fit

on the other hand, then these two quadratic forms are

distributed as noncentral chi-squares, multiplied by 02,

with respective noncentrality parameters







I = [H + '[H (HiH1) -H' H(H'H)- H'][H + n]
and ; = n'[I H1(HH ) -HI]n Thus the assumption that

n = 0 can affect the power of the test, since if n # 0 the

expected value of MSE is greater than a2, where MSE is the

quadratic form in the denominator of the F ratio. Hence if

n 0 the probability of calculating a large F value is

reduced, and we are less likely to detect lack of fit using

an upper tailed rejection region.

Daniel and Wood (1971) suggest another method for lack

of fit testing when replicated observations of the response

are not available. They make use of "near replicates" to

obtain an estimate of a, which is the standard deviation of

the observable responses in the true model. The value of

the estimate a is compared to the square root of the

residual mean square from the analysis of the fitted

model. Lack of fit is indicated if the square root of the

residual mean square is large compared to the estimate o.

To determine when observations are near replicates so that

an estimate of a can be found, they define the squared

distance between any two data points, j and j', to be

measured by

K

D2 = x. ij.)/5y]2
3J' i=1


where xij and xij, are the values of the ith independent

variable corresponding to the observations yj and yj,,

respectively, i = 1, 2, ..., K, and bi is the ordinary least







squares estimate of the ith regression coefficient. In the

denominator, s is the square root of the residual mean

square for the fitted model.

To obtain an estimate of a from near replicates, let

And = |dj d ,j, n = 1, 2, ..., (2), where dj and d are

the residuals at points j and j', respectively, and where

there are N data observations in the experiment. Since the

expected value of the range for pairs of independent

observations from a normal distribution is 1.1280, a running

average of the And's is calculated and their average is

multiplied by .886 = (1/1.128) to get a running estimate,

sn, of a. That is, s = .886 n And/n The closest pair

of observations as judged by Dj, is used to begin the

running estimate, the next closest pair (next "nearest

neighbors") is used for A2d, and the procedure continues

until sn "stabilizes." The stabilized value of sn is used

to estimate a.

A third method for testing for lack of fit without

replication is given by Shillington (1979). The fitted

model is of the form

Y = XB + E (2.5)

where Y (Nxl), X (Nxp), and (pxl) are defined as in (1.2)

and E ~ N (0, a IN) The test for lack of fit of the

fitted model is a test for whether the true model has the

form


y = X1 + 6 + E 1







where 6 (Nxl) is a fixed effect quantifying the departure of

(2.5) from the true model.

Shillington assumes that the data can be grouped into g

cells, with nj observations in the jth cell, determined in

advance. Letting Cj refer to the jth cell, j = 1, 2, ...,

g, a vector of cell averages is written YC (gxl), where the

jth element of YC is the average of the observed responses

in Cj. The matrix XC of independent variables associated

with Y is the gxp matrix where the elements in the jth row
n.
are x'. = i x!./n. that is, row j of X is the row
i-. =1 -
vector x'. The matrix XC is assumed to be of full rank

p < g. Also within each cell are defined the differences

W.. = Y.. Y. i C. j = 1, 2, ..., g, where Y is

the jth element of Yc.

The two independent data sets, YC and {Wij} with g and

N g degrees of freedom, respectively, are used to find two

independent estimates of o2. The first estimate is written

as


9 2
MSEB = E n (Y x' /(g -)
j=l B


where g4 is the weighted least squares estimate of g using

the regression of cell means, YC, on XC. The second

estimate of o2 uses the within cell deviations on cell

means, {Wij}, and is

n.
g n 2
MSEJ = Z (W. W. ) /(N g r),
j=1 i=l







where r is the rank of an Nxp matrix with rows equal to

x! -x' i C C., j = 1, 2, .., g. If the matrix of
-13j -.j 3
independent variables, corrected for cell means, is of full

rank, then r = p. Here 7ij is the estimate of Wij from the

regression of cell residuals {Wij} on the associated vectors

of independent variates, x'. x'.

If the fitted model is the correct model, then MSEB and

MSEW are independent estimates of a2 and the ratio MSEB/MSEW

is an F statistic with g p and N g r degrees of

freedom. When all observations in a cell have the same

settings of the independent variables, that is, the

observations are truly replicates for all cells, then this F

statistic is identical to the F statistic in the usual lack

of fit test in which the residual sum of squares is

partitioned into lack of fit and pure error sums of squares,

as given in Draper and Smith (1981, p. 120).

If the true model is Y = X8 + 6 + E however, and if

we let X'6 = 0 and 2 = 6/N, then



E(MSEB) = o2 + [I XC(X'XC-)1'] B/(g-P)

n.
where B (gxl) has jth component equal to E 6../n .
-B i=
Furthermore, with this latter true model form



E(MSE ) = 02 + 67(I XW(XJ XW)- X)6W/(N g r)







where _W has the components ij .j, i e Cj, j = 1, 2,

..., g. The matrix XW (Nxp) has the rows x x'

i e C., j = 1, 2, ..., g. The power of the F test,

F = MSEB/MSEW, depends on the relative bias of the estimates

of 2, that is, the biases in MSEB and MSEW.

Shillington states that the power of the F test which

makes use of F = MSEB/MSEW is maximized by forming cells so

that the bias of E(MSEW) is minimized. This is the same as

forming cells so that the within cell variation in 6 is

minimized. Shillington (1979, p. 141) also states,

"Observations with near covariate (independent variable)

values might be expected to have similar 6 values, since we

assume that 6 varies in some continuous but unknown fashion

with X. This justifies the usual procedure of forming

groups by collapsing observations with adjacent covariate

values. Indeed, if covariates do not vary within cells we

have the usual lack of fit test and maximum power."

By imposing a further structure on the form of 6, it is

shown that if the F test has an upper tailed rejection

region, the power is maximized by selecting the group sizes

as nj = 2, j = 1, 2, ..., g. Finally, Shillington suggests

that in the presence of more than one independent variable

problems in grouping may arise, and in this case it may be

wise to perform a different lack of fit test for each

parameter. Following this approach, an example is given

which suggests testing lack of fit for each of the p

independent variables separately may be more powerful than







trying to form groups based on all independent variables at

once.

In summary, all the approaches we have discussed for

testing for lack of fit when replicate observations of the

response are not available at any of the settings of the

independent variables make use of grouping the observed

response values according to similar values of the

independent variables. The observations falling in such

groups are referred to as "pseudoreplicates" or "near

neighbor observations." These pseudoreplicates are used to

estimate the true variance of the observations, a2, but a

completely unbiased estimate of 02 cannot be attained unless

true replicate observations are available. In each case,

the power of the lack of fit testing procedure is reduced

because an unbiased estimate of a2 is not attainable. We

now turn to the use of check points for lack of fit testing.

2.4 Testing for Lack of Fit with Check Points

An alternative to the two approaches to lack of fit

testing already discussed is the method which makes use of

check points. We assume a model of the form E(Y) = Xl as

given in (2.1), is fitted in a response surface system, but

that the true model is of the form E(Y) = X1I + X22 as

given in (2.2). The parameters, a8, in the fitted model are

estimated by ordinary least squares techniques, making use

of the values of the response observed at the design

points. After the model is fitted, values of the response

are observed at additional points in the experimental region







called "check points." The observed response values at the

check points are compared to the values which the fitted

model predicts at these same check points. It is important

to note that the observed values of the response at the

check points are not used in fitting the model initially.

Snee (1977) gives four methods of validating regression

models, one of which is the collection of new data to check

predictions from a previously fitted model. In a designed

experiment these new data take the form of check points.

Snee suggests that the inclusion of a small number of check

points in any designed experiment is a "worthwhile"

procedure.

Scheffe (1958) proposed a test for lack of fit when the

{3,2} simplex lattice design is used for fitting a second

order canonical polynomial model in three mixture

components. It is desired to use the observed value of the

response at (1/3, 1/3, 1/3) as a check point blend. The

test statistic proposed is the t statistic of the form


Y Y
t = (2.6)
[var(Y -)]


where Y is the observed value of the response at the check

point, and Y is the value of the response predicted at the

same point by the second order model which is fitted by

ordinary least squares techniques to the observed response

values at the six design points of the {3,2} simplex

lattice. The response value observed at the point







(1/3, 1/3, 1/3) is not used in fitting the model. Lack of

fit is inferred if the absolute value of the calculated t

value in equation (2.6) is larger than the corresponding

tabled t value.

In the denominator of the t test of equation (2.6), the

variance of the difference Y Y is shown to be



var(Y Y) = var(Y) + var(Y)

= (44/27r)o



when r replicates are taken at each design point. The

estimate of the variance of Y Y is (44/27r)o2, where o2 is

calculated from the replicated response values at the design

points.

Scheffe (1958) also alludes to a test for lack of fit

when several check points are used simultaneously. When

there are k check points, the test for lack of fit is an F

statistic of the form


-1
F = 2 (2.7)
ka

where d' = (Y Y 2 2' ... Yk Yk) and V = o2V

var(d). Formulas are given for the elements of VO in the

special case when the check points are the design points of

the {3,2} simplex lattice. Lack of fit is suspected if the

calculated value of the F statistic given in (2.7) is larger

than the corresponding tabled F value.







Gorman and Hinman (1962) suggest the same t test in

equation (2.6) that Scheffe (1958) suggested for a check

point taken at (1/3, 1/3, 1/3) to test for lack of fit in a

second order polynomial model fitted from a {3,2} simplex

lattice design. They suggest using (1/3, 1/3, 1/3) as the

location of the check point because the observation at this

point may later be used to fit the next more complex model,

the special cubic, if the second order model is found to be

inadequate. They state that in general for the second order

polynomial model as well as higher order models, check

points should be taken in regions of particular interest, of

which there are usually many in any blending study.

Further, they suggest that the number of check points

depends on individual experimental situations--technical

background, precision required, cost of materials and

analyses, and probability of requiring a more complex

model. However, no specific criterion is given by Gorman

and Hinman for selecting the location of the check points.

Gorman and Hinman (1962) indicate that a t test at a

check point other than at (1/3, 1/3, 1/3) takes the same

form as the statistic of equation (2.6),


Y Y
t 1/2
[var(Y) + var(Y)]/2



with the additional condition that if several check points

are taken, say for example k points, the method of checking







the fit is to compute the t value at each location and refer

these calculated t values to the 100(a/2k) percentage point

of the central t distribution rather than the 100(a/2)

percentage point.

Kurotori (1966) gives an example of a mixture

experiment where the response is the modulus of elasticity

of a rocket fuel, which is a mixture of three components,

binder (xl), oxidizer (x2), and fuel (x3). The factor space

of feasible mixtures is a subspace inside the two-

dimensional simplex or triangle where all three components

are present simultaneously. "Pseudocomponents" are defined

and in the pseudocomponent system a special cubic model is

fitted to data collected at the points of the q = 3 simplex

centroid design (Figure 4). A check for adequacy of fit is

made by using three check points and the response values at

the check points are used only for testing the fit of the

model and not for fitting the model initially.

The reason for the choice of the particular check point

locations by Kurotori is that, as he states, "They are the

most remote mixtures from the seven design points." The

lack of fit test is an F statistic of the form


2
F = (2.8)


2 3
where s (Y Y ,) for the i = 1, 2, 3 check points
2 i=l
and 02 is an estimate of measurement error from a previous

analysis. Kurotori admits that the use of the F statistic











x :1
I
(1,0,0)
9 Design Points

O 0 Check Points




SE 3 ') o
S 2 I 112



e(o,,o) a the ser)
2 '2 '2)




Figure 4. Kurotori's rocket fuel example,
xI', x2', and x3' represent pseudocomponents.





in Eq. (2.8) for lack of fit testing may be risky because

the predicted values at the check points are correlated

(correlation of .5), although the observed values are not

correlated. Kurotori suggests individual t tests as

proposed by Scheffe (1958) might be the preferred procedure.

Snee (1971) repeats Kurotori's rocket fuel example

using the same F test for lack of fit as Kurotori and makes

the comment that the Yi's at the check points are

correlated. In stating that the F test is not an exact

test, he nevertheless offers no solution in the form of an

exact test.





39

In summary, only Scheffe refers to an exact F test when

several check points are considered simultaneously for

testing for possible lack of fit of a model fitted in a

mixture space, and his development is limited to the special

case where the check points are the design points used to

fit the model initially. No criterion is proposed by

Scheffe for selecting other locations for the check points.














CHAPTER THREE
AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL

3.1 Introduction

In Chapter Three we investigate the problem of testing

for lack of fit of a linear model fitted in a mixture

space. The testing is to be accomplished with the use of

check points. We assume that an experimental design is

specified, and that the fitted model is of the form



E(Y) = X-1 (3.1)



where Y is an Nxl vector of observable response values, X is

an Nxp matrix of known constants and rank p, and 81 is a

vector of p unknown regression coefficients. The true model

is assumed to be of the form



E(Y) = X1 + X 2 (3.2)
-1 2-2


where X2 is an Nxp2 matrix of known constants and g2 is a

vector of p2 unknown regression coefficients. Throughout

our development, we will assume that the random vector Y has

the normal distribution with variance-covariance matrix

equal to a IN.








In our investigation we wish to determine the proper

testing procedure to follow in deciding whether the fitted

model exhibits lack of fit. In order to optimize the lack

of fit testing procedure, we will determine the location of

the check points so that the power of the test is maximized.

3.2 Testing for Lack of Fit in the Presence of
an External Estimate of Experimental Error Variation

3.2.1 The Test Statistic

We wish to test the performance or fit of a fitted

model in a mixture space when the true model possibly

contains terms in addition to those in the fitted model.

The fit of the model is to be tested by a test which makes

use of the response values observed at certain locations

called "check points" in the experimental region, by

comparing them to the values which the fitted model predicts

at the same check points. The observed values at the check

points are not used for estimating the coefficients in the

fitted model and are assumed to represent the values of the

true surface at the check points.

Let us define the vector of differences



d = (Y* Y*)



(Y* Y- Y Y* Y* y*)i
1 1' 2 21 k k


where Y* = 1, 2, ..., k are observed response values at

k check points and YV, i = 1, 2, ..., k are response values

predicted at the k check points by the fitted model,







VY = xt'b where b is the ordinary least squares estimator

of al and where x*' is the ith row of X*, the kxp matrix
1 -1
whose columns are of the same form as the columns of X but

with its rows evaluated at the k check points. Note that

if 3 = 0, then E(d) = 0 and if a 2 0, then

2 2 2
E(d) = (X2 X*(X'X)-X'X2)-. Let V represent the

variance-covariance matrix of the random vector d.

Then V = o V where


-1
V = Ik + X*(X'X) X*'



and where Ik is the identity matrix of order kxk.

We assume that an unbiased estimate of a2 is available
^2
and we denote this estimate by a where the subscript ext
Sext'
^2
stands for external, and a is independent of the model

being fitted. The test statistic for the hypothesis of zero

lack of fit H : E(d) = 0 is

-1
d'V d/k
0-
F = (3.3)
^2
ext


(see Scheffe, 1958, p.358). It will be shown later in this

section that the F ratio in Eq. (3.3) possesses either a

central F distribution or a noncentral F distribution,

depending upon whether the true model is represented by Eq.

(3.1) or Eq. (3.2).
^2
The variance estimate a that appears in equation
ext
(3.3) is ordinarily generated from replicated observations








at some of the design points in the experiment. We assume
^2
that aext is a constant multiple of a central chi-square

random variable with v degrees of freedom. This is written

as


^2
a e = SSE /v
ext pure


2 2
= (a 2/v)(SSEpe/
pure


2 2
where SSEpure/ ~ X. Note that SSEpure denotes the

portion of the residual sum of squares due to replication

variation from the fitted model. The residual sum of

squares from the fitted model may be partitioned into

SSEpure and SSLOF only if replicated observations are

collected at one or more design points. For the case where

replicate observations are collected at all of the design

points


n n.
SSE = Z (Yij Yi. )
i=l j=1


where n is the number of distinct design points, n. > 2 is
1
the number of replicates at the ith design point, Yij is the

jth observation at the ith design point, and Y. is the
1.
average of the ni observations at the ith design point.
n
Here SSEpure has v = Z (ni 1) degrees of freedom.
i=l1
When the fitted model and the true model are of the

same form as defined by Eq. (3.1), the quantity d'Vld/a2
(3.lr th quntit d'







possesses a central chi-square distribution (Searle, 1971,

p.57, Theorem 2). However, when the true model is of the
-1 2
form specified by Eq. (3.2), d'V d/a possesses a

noncentral chi-square distribution. Thus when the true

model is of the form in Eq. (3.1),


-1 2 2
d'V- d/ao 2 X
0 Xk'


but when the true model is of the form in Eq. (3.2),


-1 2 2
d'V d/o2 ~ x ,



where in the second case the noncentrality parameter X has

the form



S= E(d)'V E(d)/2a2
1 0


-1 2
= X* X*A)'Vo (X X*A) 2/2
2 2 0 2 2


-1
The matrix A = (X'X) X'X is called the alias matrix and is
2
of order pxp2. In X1, the matrix X* is of order kxp2 and

has the same relationship to X2 as X* has to X.
2
Since SSE /a is statistically independent of
pure
-1 2
d'V d/a then under model (3.1) the test statistic
0 0








-1 2
d'V d/ko
F = 2
SSE pure/v
pure


-1
d'V d/k
= ^2
ext


will have a central F distribution. When the true model

contains terms in addition to those in the fitted model then

F will have a noncentral F distribution. We write these two

cases as



F ~ Fk,
k,v



under model (3.1), and



F ~ F'
k,v;Xi



under model (3.2), where the noncentrality parameter is



x = (X X*A)'V0 (X- X*A)_2/2 2.



3.2.2 The Testing Procedure and an Expression for the Power
of the Test

Given that the form of the fitted model is defined as

Eq. (3.1), the expected value of the numerator of the F

statistic in Eq. (3.3) will depend on the form of the true

model. For the case where the true model is expressed as







Eq. (3.2),



-1
E(numerator) = E(d'V d/k)
0-


2 2
= ( /k)EXx2



= (a /k)(k + 211)


2 2
= 2 + 20 X1/k



= o2 + BA1 /k, (3.4)


-1
where A1 = (X* X*A)'V0 (X X*A). However, when the true

model is Eq. (3.1), 8 = 0 and in this case A = 0 so that
-2 1
2 ^2
E(numerator) = 02. Also a is an unbiased estimator of
ext
o2 and



^2 2
E(a ) = a (3.5)
ext


Therefore the ratio E(numerator)/E(denominator) where
^2
the denominator is ext will equal unity under model (3.1),

that is, when there is no lack of fit. Under model (3.2),

the ratio will be greater than or equal to unity so lack of

fit should be suspected if the calculated F ratio in

equation (3.3) is large. We can thus use an upper tailed

rejection region to reject the hypothesis of zero lack of

fit. The power of the test is










PI k ,v ;X > a; k, v 1
P{Fv > F,; };k,I



where F is the upper 100a percentage point of the
a ;k,v
central F distribution with k numerator degrees of freedom

and v denominator degrees of freedom.

It is worth noting that from Eq. (3.4) and Eq. (3.5)

testing the hypothesis that -2 = 0 is equivalent to testing

the hypothesis that X1 = 0, assuming A1 is positive

definite. Thus testing a null hypothesis of zero lack of

fit using the proposed testing procedure involving the F

ratio in (3.3) may be expressed as a test of the hypotheses:



H: 1 = 0



H: a1 > 0.
a 1


3.2.3 A Method for Locating Optimal Check Points

Once a design for fitting model (3.1) in a mixture

space is chosen and the number of simultaneous check points

is decided on, say k > 1, the next step is to determine

where in the mixture space we should place the k check

points so as to maximize the power of the test for lack of

fit. The location of the check points is to be made

independently of the value of .
-2







The power of the upper tailed F test for lack of fit is

an increasing function of X1 (see Appendix 1 for proof, with

X2 = 0). Therefore, to maximize the power of the test we

maximize the value of X1 defined as







-1
Xl = S2AI82/2a2



where A = (X X*A)'V0 (X* X*A), by properly selecting

the k check points whose coordinates are defined in X*. To

maximize the value of X1, we shall concentrate on the matrix

A1.

The matrix A1 is a square matrix of order p2xp2 and is

a scalar quantity when p2 = 1. By maximizing the scalar

quantity A1 with respect to the k check points, the power is

maximized no matter what the value of .2 Maximizing the
-2
scalar A1 can be accomplished by using The Controlled Random

Search Procedure given by Price (1977). This procedure is

described in Appendix 2. As a computational aid, A1 can be

expressed as


V + (X* X*A)(X* X*A)'
A = V-2 1 (3.6)
1 V


when p2 = 1, where the symbol IBI denotes the determinant of

the square matrix B. Thus the computations reduce to

evaluating two determinants rather than inverting VO (see

Scheffe, 1959, Appendix V, p.417).








When p2 > 1 and A1 is no longer a scalar, maximizing X1

(and thus maximizing the power of the test) cannot be

accomplished without specifying 02- In this case we make

use of a lower bound for 1l (Graybill, 1969, p.330, Theorem

12.2.14(9)) defined as



lmin-/22 < X1



(where min is the smallest eigenvalue of A1) to be used in

place of \l. Hence an approximate solution to the

maximization of X1 will be achieved by finding the k

simultaneous check points (using Price's procedure) that

maximize min' the smallest eigenvalue of A1. In other

words when p2 > 1, and in order to avoid specifying 2' we

seek to maximize a lower bound value for X1. This

maximization does not depend on the value of 02.

There are cases where the matrix A1 is of less than

full rank (less than rank p2) or equivalently where the

matrix A1 is positive semi-definite so that umin will be

equal to zero no matter which check points are selected.

One such case occurs when k < p2 (when the number of check

points is less than the number of parameters in the true

model which are not in the fitted model) since when k < p2



rank(Al) = rank[V I X* X*A)]


rank(X* -X*A),
2







and so rank(Al) < min(k, p2) because the matrix (X* X*A)

is of order kxP2. Therefore when k < p2, the rank of A1 is

at most k so that A1 is of less than full rank. Since umin

must be equal to zero when Al is positive semi-definite, an

alternative method to that of maximizing umin to select

optimal check points must be found when A1 is positive semi-

definite in order to produce a positive lower bound for X1.

In this pursuit, let us write X1 as



1 = -2AA2/202


= PAP'_2/202



= 8[P1:P2] diag[Al, A2=0][P1:P2] '2/2o2


= P-lA 1P 12/202


where A is a diagonal matrix with elements equal to the

eigenvalues of Al, P is an orthogonal matrix whose columns

are orthonormal eigenvectors of AI, A1 and P1 correspond to

the positive eigenvalues of AI, while A2 = 0 and P2

correspond to zero eigenvalues of AI. Then by Theorem

12.2.14(9) in Graybill (1969) we can write



Sz'z/2o2 < 1 (3.7)
min p


where in is the smallest positive eigenvalue of A,, and
min







z = P'S Thus by Eq. (3.7), an approach to maximizing a
- -1-2
positive lower bound for X1 when A1 is positive semi-

definite is to select check points that maximize the

smallest positive eigenvalue of A1. It must be noted,

however, that this method can only be used when

a2 e n C(Pl), where C(P1) denotes the column space of P1
and n C(PI) denotes the intersection of all such spaces

which can be obtained at all possible check points

locations. This is because, in general, z'z in (3.7)

depends on the location of the check points through its

dependency on Pi. If, however, 2 e nC(P ), then

zz = P1 = P = 22 since 'P2 = 0.

It follows that when 2 e n C(P ), mn z'z/2o
2 1 min- -
+ 2 +
= min 2+-/2o and only mn depends on the location of the

check points.

3.3 Testing for Lack of Fit When MSE Is Used
to Estimate Experimental Error Variation

3.3.1 The Test Statistic

In this section we shall show that when an external

estimate of a2 is not available and the residual mean square

(MSE) from the fitted model of the form (3.1) must be used

as an estimate of a2, the test statistic




-l
d'V d/k
F ME (3.8)

MSEpossesses a central F distribution when the true model is

possesses a central F distribution when the true model is







Eq. (3.1), but possesses a doubly noncentral F distribution

when the true model is Eq. (3.2).

In the initial section of this chapter, the quantity

d'V- d/2 was said to possess a central chi-square

distribution or to possess a noncentral chi-square

distribution, depending on whether the true model was

specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum

of squares from the fitted model is defined as


N
SSE = E (Y Y)
i=l

-1
= Y'(I X(X'X) X')Y



and it is easy to show (Searle, 1971, p.57, Theorem 2) that

SSE/a2 possesses a central chi-square distribution if the

true model is Eq. (3.1), but under model (3.2), SSE/a2

possesses a noncentral chi-square distribution. This is

expressed as


2 2
SSE/a xp
N-p


under model (3.1), and



2 2
SSE/a X.
N-p,12







under model (3.2), where the noncentrality parameter \2 is


2
X2 = -(X2 XA)'(X2 XA)2/2o0



The distributional form of the test statistic in Eq.

(3.8) is derived by knowing that the quantities

d'Vo d/o2 and SSE/o are statistically independent (see

Appendix 3), so that


d'vold/ko2



-12
MSE/o



d'V d/k
MSE


is distributed as a central F when the true model is Eq.

(3.1), but when the true model is Eq. (3.2) the F ratio is a

doubly noncentral F, that is, under model (3.1),



F ~ F
k,N-p


and under model (3.2),



F~ F"
k,N-p;X ,X2


3.3.2 The Rejection Region and its Relation to the Power of
the Test

In Appendix 1 it is shown that if k, N-p, and X2 are

fixed, then the power of the F test using the ratio (3.8) is







a function of the location of the rejection region (upper

tailed or lower tailed) of the test. The power increases

with increasing values of the numerator noncentrality

parameter, X1, when the test is an upper tailed test. The

power decreases with increasing values of X1 when the test

is a lower tailed test. This means that to study ways of

increasing the power of the test, we have to determine

whether the test is an upper tailed test or a lower tailed

test. Similarly, for fixed values of k, N-p, and X1, the

power of the F test is a decreasing function of X2 for an

upper tailed test, and is an increasing function of X2 when

the F test is a lower tailed test (Scheffe, 1959, p. 136-

137).

To decide if the test is an upper tailed test or a

lower tailed test, we recall from Section 3.2.2 that if the

true model is Eq. (3.1) then the expected value of the

numerator of the F statistic in (3.8) can be written as



E(numerator) = 02,



and if the true model is Eq. (3.2),




2 2
E(numerator) = a + 2a X1/k (3.9)



= 02 + QA 2/k








-1
where the P2xp2 matrix A is A = (X* X*A)'V (X* X*A).

Similarly, it can be shown that if the true model is Eq.

(3.1), the expected value of the denominator of the F

statistic in (3.8), where the denominator equals MSE, is



E(denominator) = E(MSE)



= 2,



but if the true model is Eq.(3.2),



E(denominator) = E(MSE)



[o2/(N p)]Eyp,2


= [a2/(N p)][N p + 2X2]



= a2 + 22 2X/(N p) (3.10)



= 2 + a2A2 2/(N P)



where the P2xP2 matrix A2 is A2 = (X2 XA)'(X2 XA). Thus

the ratio E(numerator)/E(denominator) will equal unity if

the true model is Eq. (3.1), but if the true model is Eq.

(3.2), the ratio is greater than unity if B_~A12/k >

8_A 2 2/(N p). In this latter case we reject the null







hypothesis of zero lack of fit if the calculated value of

the F ratio in (3.8) is large. An upper tailed rejection

region seems reasonable for this test. When the true model

is Eq. (3.2), and if a2Al2/k < 2A2 2/(N p), then a lower

tailed rejection region is preferred.

3.3.3. A Method for Locating Optimal Check Points

Given a design for fitting a model of the form in Eq.

(3.1) in a mixture space (note that fixing the design fixes

A2 and (N p)), and given the number of simultaneous check

points desired, k > 1, we now wish to determine where in the

mixture space the k check points should be located so as to

maximize the power of the F test for lack of fit, where the

test statistic is given in Eq. (3.8). We also wish to

position the optimal check points in a manner that is

independent of the values of the elements in 2 .
-2
The case of an upper tailed test. To help us find k

simultaneous check points that maximize the power of an

upper tailed test, we shall make use of the fact that the

power is an increasing function of Xi. Therefore to

maximize the power of the upper tailed F test, we shall seek

the locations of the k check points that maximize X1.

As in the case considered in Section 3.2.3, where the

test statistic had a noncentral F distribution, if the

number of extra terms in the true model is p2 = 1, then

maximizing X1 is equivalent to maximizing the scalar A1.

However, as before, if p2 > 1, then the P2xP2 matrix A1 is

not a scalar and we will have to approximate the







maximization of X1 by maximizing a lower bound for X1. This

is done by finding the maximum value of min' the smallest

eigenvalue of A1, since



minm-2-/22 X1


When the number of check points is less than the order

of the square matrix A1, that is, k < p2, then rank(A1) <

min(k, p2), and A1 will have Umin = 0. For this case, we

again try to maximize the smallest positive eigenvalue of A1

which we denote by in' while remembering from Section
min'
3.2.3 that this technique is limited to situations where

B2 e nC(P1)
-21
The case of a lower tailed test. To find k check

points to maximize the power of a lower tailed test, we make

use of the fact that the power of the lower tailed F test

increases as X1 decreases. Then if P2 = 1 and A1 is a

scalar quantity, X1 can be minimized with respect to the k

check points by finding the check points that minimize A1.

If P2 > 1, then by Theorem 12.2.14(9) in Graybill (1969), we

see that an upper bound for X1 is



X1 max_22 /202 (3.11)



where umax is the largest eigenvalue of Al. An approximate

solution to minimizing X1 in (3.11) can be achieved by

minimizing max. It is not necessary to treat the case
-'max







of k < p2 separately here, although X1 will equal zero if

_2 is in the column space of P2, where P2 is the matrix
whose columns are orthonormalized eigenvectors corresponding

to the zero eigenvalues of the matrix A .

3.3.4 Determining Whether the Test Is Upper Tailed or Lower
Tailed

The procedures outlined in Section 3.3.3 produce a set

of k check points that simultaneously maximize the power of

the upper tailed test as well as a second set of k check

points that simultaneously maximize the power of the lower

tailed test. The check points that are selected maximize

the power, given A2, k, and N p without specification

of a2' except that when A1 is positive semi-definite we
require that 82 e n C(P ).

It is now necessary to decide which of our two

candidates will be used for a lack of fit test. To choose

between the upper tailed test and the lower tailed test, let

us consider the quantity



R = [A /k] [A2/(N p)].



If R is positive definite when the true model is Eq. (3.2),

then no matter what the value of 82 is, the ratio

E(numerator)/E(denominator) will be greater than unity,

implying an upper tailed test is to be used. Similarly, if

R is negative definite, then a lower tailed test should be

used. Finally, if R is not definite, then neither an upper

nor a lower tailed test is implicated and further







investigation is necessary. The criterion of R = [A /k] -

[A2/(N p)] may yield any of the four following cases.

Case 1. If R = [A /k] [A2/(N p)] is positive

definite when A1 is generated by the k optimal upper tailed

test check points, and R is not negative definite when A1 is

generated by the k optimal lower tailed test check points,

then we recommend that the check points be used that yield

the optimal upper tailed test with an upper tailed rejection

region.

For Case 1 it is necessary for A1 to be positive

definite (see Appendix 4). Since A1 is a square matrix of

order p2xP2 with rank(A ) < min(k, p2), then A1 can be

positive definite only if k > p2. Thus, there must be at

least p2 check points for Case 1 to hold, where p2 is the

number of terms in the model of Eq. (3.2) that are not in

the model of Eq. (3.1).

From inspection of equations (3.9) and (3.10), it is

apparent that the testing for lack of fit in Case 1 is

equivalent to testing the hypothesis




1 2
H0 0 N p = (3.12)


against the alternative




1 2
H > 0
a k N p





60

since R = [A /k] [A2/(N-p)] is positive definite when the

true model is Eq. (3.2). In Appendix 5(a) it is shown that

under Case 1, the hypothesis given by (3.12) is equivalent

to the hypothesis



H X = X = 0.



Case 2. In Case 2 we assume that R = [A /k] -

[A2/(N p)] is not positive definite for the k optimal

upper tailed test check points, but that R is negative

definite for the k optimal lower tailed test check points.

Here we recommend that the lower tailed test check points be

used with a lower tailed rejection region.

It is necessary for A2 to be positive definite for Case

2 to occur (see Appendix 4). However, A1 need not be

positive definite, and so k need not be greater than p2. In

Case 2 then, it is possible that lack of fit may be tested

with only one check point.

By inspection of equations (3.9) and (3.10), a

hypothesis of no lack of fit is equivalent to


X1 X
1 2
Hk = 0 (3.13)
0' k N p


while the alternative hypothesis that lack of fit is present

is equivalent to


X1 X
1 2
H N < 0
a k N p







since R = [A /k] [A2/(N p)] is negative definite. In

Appendix 5(b) it is shown that the hypothesis given by

(3.13) is equivalent to the hypothesis



H0: = 2 = 0.


Case 3. We assume R is positive definite for the k

optimal upper tailed test check points, and R is negative

definite for the k optimal lower tailed test check points.

Hence either an upper or lower tailed test may be considered

as a possible test for lack of fit. If the quantity
2
_'_2//o can be specified, then the minimum power for both
the optimal upper and optimal lower tailed tests can be

approximated, and the test with the greater minimum power is

recommended. In Appendix 4 it is shown that Case 3 can

occur only when A1 is positive definite for the upper tailed

test. Thus Case 3 can only occur when there are at least p2

check points.

The minimum power of the upper tailed test may be found

by calculating



P IF" > F ), (3.14)
k,N-p;1 IL' 2U a;k,N-p


where F;k,N-p is the upper 100a percentage point of the

central F distribution,






2
IL = /min-2/2a2


and



X2U = max2-2/2 02



where min is the smallest eigenvalue of A1 and max is the
mmn 1 max
largest eigenvalue of A2. Formula (3.14) yields a

conservative lower bound for the power of the optimal upper

tailed test. Note that A1 is generated using the optimal

upper tailed test check points. The cumulative distribution

function of F" can be approximated by multiplying the

cumulative probabilities of the central F distribution by a

constant (Johnson and Kotz, 1970, p.197). This

approximation is described in Appendix 6. Other

approximations for F" (such as the Edgeworth series

approximation suggested by Mudholkar, Chaubey, and Lin,

1976) exist which are generally more accurate, but we chose

to use the approximation given in Johnson and Kotz (1970,

p.197) due to its simplicity. Additionally, the

approximation of Mudholkar, Chaubey, and Lin (1976) produced

negative probabilties when only one degree of freedom was

available in either the numerator or denominator of F".

This problem was avoided by using the approximation given by

Johnson and Kotz (1970).

The minimum power of the optimal lower tailed test can

be approximated similarly (if B~22/o2 is specified) by







calculating


P IFF"


< (l-a);k,N-pp


where


X lu = maxJ.2 2/2o 2


and


S2L = 6min-22/20 2


with Umax equal to the largest eigenvalue of A1 and 6min
max n
equal to the smallest eigenvalue of A2. Note that A1 is

generated by using the optimal lower tailed test check

points. For the lower tailed test, A1 may be positive semi-

definite, and if 82 is in the column space of P2 then A1 = 0.

In Case 3, the upper tailed test is a test of



HO: 1 = = 0




0 1 2
H 2 > 0
a k N- p


while the lower tailed test is a test of







H: X =X2 0
H0 1 2 = 2



1 2
H -P < 0.
a k N P < 0


Case 4. In Case 4 we assume that R = [Al/k] -

[A2/(N p)] is not positive definite for the k optimal

upper tailed test check points and R is not negative

definite for the k optimal lower tailed test check points.

Here it is useful to write the difference between the

expected value of the numerator and the expected value of

the denominator of the F ratio in (3.8) as



al[A /k A2/(N P)] = -sns'S
-2 1 2 -2 = -2


= 8[S1:S2:S3] diag[l2'r2=0,03 [Sl:S2:S3] '2



= 8_2S"l I2 + 2S3 332


where 0 = diag(ll, 02' 23) is a diagonal matrix consisting

of the eigenvalues of R, 01 is a diagonal matrix of the

positive eigenvalues of R, 02 is a diagonal matrix of the

zero eigenvalues of R, and 03 is a diagonal matrix of the

negative eigenvalues of R. The orthogonal matrix S can be

expressed as S = [S1:S2:S3], where the matrices SI, S2, and

S3 have columns which are orthonormalized eigenvectors

corresponding to nl, 02, and 03, respectively.







In Case 4, neither the optimal upper tailed test nor

the optimal lower tailed test is applicable for all values

of _2 For completeness, we note that Case 4 actually

consists of nine subcases, where R may be positive semi-

definite, negative semi-definite, or indefinite for either

the optimal upper tailed test or lower tailed test check

points. These subcases are listed in Table 2.




Table 2. Nine Subcases of Case 4.


R--Upper R--Lower
Subcase Tailed Test Tailed Test

1 PSD PSD
2 PSD NSD
3 PSD I
4 NSD PSD
5 NSD NSD
6 NSD I
7 I PSD
8 I NSD
9 I I

PSD = positive semi-definite, NSD = negative semi-
definite, I = indefinite.





If _2 lies in the column space of S2, then 8'[A /k -

A2/(N p)]s82 is zero, and therefore lack of fit is not

testable with either an upper or lower tailed test. A

sufficient condition for the test for lack of fit to be

upper tailed in Case 4 is that 0 be in the column space
-2
of [S1:S2], but not entirely in the column space of S2. In

this case








;[A1/k A2/(N p)]_2 = 2S01812 + 2S303S32


= 2S1~lS 2 + 0


= S~Sl2lSI2,


and 8_[ Al/k -
indicating an
condition for
that _2 be in
in the column


A2/(N )]_2 will be greater than zero,
upper tailed test. Similarly, a sufficient
the test for lack of fit to be lower tailed is
the column space of [S2:S3], but not entirely
space of S2. Then


2[A1/k A2/(N P)]g2 = 0 + 2S33S3_2


= 2S3 3S 2


which makes _2[Al/k A2/(N p)] 2 less than zero,
indicating a lower tailed test.

To determine whether 2 is in the column space of

[S :S2], let us define the augmented matrix

Q1 = [f2:Sl:S2] If Q!Q1 has a zero eigenvalue, then 02 is
in the column space of [S :S2]. Similarly, if we define

Q2 = [82:S2] and Q3 = [82:S2:S3]' then a2 is in the column
space of S2 if Q'Q2 has a zero eigenvalue, and 2 is in the
column space of [S2:S3] if Q3Q3 has a zero eigenvalue.







Given that we are in a particular subcase of the nine

subcases described in Table 2, we recommend that lack of fit

be tested with the upper tailed test check points if it is

determined that 2 is such that '[A /k A2/(N -p)]2 is

positive when A1 is generated from the upper tailed test

check points. Likewise, for the same given subcase, if the

value of 02 of interest is determined to produce a negative

value for i2[Al/k A2/(N P)]@2 when A1 is generated from

the lower tailed test check points, then we recommend that

lack of fit be tested with the lower tailed test.

We see then that Case 4 is an undesirable situation in

practice, since, in order to test for lack of fit, we must

assume a priori that any lack of fit is due to a nonzero

value of 82 that produces an upper tailed or lower tailed

rejection region. However, it would seem rare that such

knowledge would be available.

3.4 Examples

We now present several examples to illustrate the

technique for locating optimal check points to be used in

testing for lack of fit in a mixture model.

3.4.1 Theoretical Examples

Example 1. In this example a second order canonical

polynomial model is fitted in three mixture components using

the {3,2} simplex lattice design, which is presented in

Figure 1 of Chapter 1. The true model is assumed to be the

special cubic model containing the term 123x x 23 in

addition to the six terms of the fitted model. The expected







values of the response at the six design points are assumed

to be represented by the fitted model in the form



E(Y) = X01,



but with the true model the expectations are written as



E(Y) = X 1 + X22 ,



where X is a 6x6 matrix with rows that define the

coordinates of the six design points and columns that

correspond to the six terms in the fitted model (xi, x2, x3,

xlx2, xlx3' x2x3)' 8a is the 6x1 vector of regression

coefficients (81, 82, 83, 812, 813, 823), X2 is a 6x1

column vector containing the values of the term xlx2x3 at

the design points, and 82 is the single regression

coefficient 8123'

The {3,2} simplex lattice design consists of only six

design points, and since six parameters are estimated in the

second order fitted model, there are no degrees of freedom

remaining for obtaining an estimate of the experimental

error, 02. We assume therefore that an external estimate of

a is available, a2 which will be used in the denominator

of the lack of fit F statistic given in Eq. (3.3).

Since there is one term in the true model in addition

to those in the fitted model, that is p2 = 1, we know that

in order to locate k simultaneous check points that maximize







the power of the test for lack of fit it is necessary to

maximize the scalar quantity



-1
A = (X* X*A)'V (X* X*A)
1 2 0 2


with respect to the coordinates of the k check points. Here

X* is a k-element column vector with ith element equal to

the value of x* x* x* at the ith check point, X* is a kx6
il i2 i3
matrix with ith row equal to the value of (x* x* x*
il 12' i3'
S x* x* x xt ) at the ith check point,
11 1i2' 11 Xi3' 12 i3
A = (X'X)-X'X2 is the 6x1 alias vector, and

V = Ik + X*(X'X)- X*'. This maximization is accomplished

by use of the Controlled Random Search Procedure (Price,

1977), which is described in Appendix 2.

When only a single (k = 1) optimal check point is

desired the Controlled Random Search Procedure locates a

point (x*, x*) which maximizes


-1
A1 = (X* X*A)'V0 (X2 -X*A),



where


S= x*x*x* = *x*( x* -
2 123 12 1 2







X* = (x*, x*, x* x* x*x*, x*x*)



= (x*, X* (1 x* x*), x*x*, x*(l *),
1 2 1 2 12' 1l 1 2


x*(l x* x*)),
2 1 2

-l
and V0 = 1 + X*(X'X)- *'. The value of A1 is calculated

using the formula of Eq. (3.6). Following this procedure,

we find that the single check point that maximizes A1, and

thus maximizes the power of the test, is the centroid of the

triangular factor space (1/3, 1/3, 1/3). The value of A1 at

this centroid point is A1 = 0.00084.

When the Controlled Random Search Procedure is used to

locate k = 2 simultaneous check points that maximize A,, the

centroid (1/3, 1/3, 1/3) is selected twice, and A1 =

0.00121. For three simultaneous optimal check points, the

centroid is selected three times, and A1 = 0.00142.

To test whether the second order model exhibits lack of

fit, when we suspect the special cubic model is the true

model, we form the F ratio



-1
d'V d/k
-0-
F = ^ 2
^2
ext




with the single check point (1/3, 1/3, 1/3) where d =

Y* Y*, Y* is the observed response, Y* is the response
1 1 1 1







predicted by the second order fitted model at (1/3, 1/3,

1/3), and V0 = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(X'X)-1

(1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of

the F ratio exceeds F where v equals the number of

degrees of freedom associated with a then we reject the
ext
null hypothesis that the second order model is the true

model in favor of the alternative hypothesis that the

special cubic model is the true model. Equivalently, we

reject H: I1 = 0 in favor of Ha: 1 > 0. For k = 2 or

k = 3 check points, the value of the F ratio is calculated

using the observed and predicted responses at the two or

three replicates at the centroid. The hypothesis

H0: X = 0 is rejected in favor of Ha: X > 0 if F

exceeds F;kv
a ;k,v
Example 2. In Example 2 we illustrate the second of

the four cases that could arise when MSE is used as an

estimate of 02 in the lack of fit test statistic (see

Section 3.3.4). We again fit a second order canonical

polynomial model in three mixture components, and assume the

true model is special cubic. The design to be used is the

q = 3 simplex centroid design, which consists of seven

design points, and is illustrated in Figure 2 of Chapter 1.

There are six parameters to be estimated and seven

design points hence one degree of freedom can be used to

calculate MSE. We shall use MSE to estimate a2. Optimal

upper and lower tailed test check points must be located,

and then a decision is made as to which test should







be used. The actual testing for lack of fit involves the F

statistic in (3.8).

As in Example 1, p2 = 1, since there is one term in the

true model in addition to those in the fitted model. Thus

Al is a scalar whose value we seek to optimize with respect

to the desired number of check points, k. When only a

single check point is sought for the purpose of testing lack

of fit, the Controlled Random Search Procedure has two

functions. First, the procedure is used to locate the

optimal candidate check point for an upper tailed test by

locating the check point that maximizes the scalar Al.

Secondly, the procedure is used to locate the optimal

candidate check point for a lower tailed test, which is

accomplished by locating the point that minimizes A1. The

quantity R = [Al/k] [A2/(N p)] is then calculated to

determine whether the upper or lower tailed test will be

used. If R is positive for the candidate check point for an

upper tailed test, then the test is upper tailed, and the

test is lower tailed if the candidate check point for a

lower tailed test produces a negative value for R. Note

that A2 = (X2 XA)'(X2 XA) is fixed once the design is

specified, since A2 does not depend on the check points.

Using the Controlled Random Search Procedure it is found

that the maximum value of A1 occurs at (xl, X*, x*) = (1/3,

1/3, 1/3), which will be the location for the check point

for the upper tailed test. Calculating A1 at this centroid







point, we find that R = [A /k] [A2/(N p)] = [(3.7258

x 10-4)/l] [(8.4175 x 10-4)/l] = -4.6917 x 10-4. Since R

is negative, the test is not upper tailed.

Using the Controlled Random Search Procedure to

minimize AI, we find that a subregion of the factor space

exists in which all points yield a near minimum value for

A1. We choose the point (0.0189, 0.9269, 0.0542) at random

from this subregion to be used as the optimal candidate for

a lower tailed test. Here R = 0 [(8.4175 x 10-4)/] =

-8.4175 x 10-4.

Since R is negative for both the optimal upper tailed

test check point and for the optimal lower tailed test check

point, we have Case 2 of Section 3.3.4. The upper tailed

test check point is disregarded, and the lower tailed test

check point (0.0189, 0.9269, 0.0542) is used to test for

lack of fit. If the calculated F ratio,

-1
d'V d
F =
MSE


is less than F ( );,then H: X = X = 0 is rejected in

favor of Ha: [X1/1] [X2/1] < 0, that is we conclude that

the second order model exhibits lack of fit, and the true

model is special cubic.

When two simultaneous check points are desired for

testing lack of fit, we can again use the Controlled Random

Search Procedure to locate the optimal settings. To

maximize the scalar A,, we find that both check points







should be selected at (1/3, 1/3, 1/3), for an upper tailed

test. With our calculations R = [(5.8275 x 10-4)/2] -

[(8.4175 x 10-4)/1] = -5.5038 x 10-4, but since R is

negative, the test is not upper tailed.

Minimizing A1 to locate two optimal lower tailed test

check points yields a subregion in the factor space of

optimal check points. The pair of check points (0.3749,

0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at

random from this subregion, and these check points yield

R = 0 [(8.4175 x 10-4)/1] = -8.4175 x 10-4.

Since R is negative for the upper tailed test points

and the lower tailed test points, we have Case 2 of Section

3.3.4 again and the lower tailed test check points are used

to test for lack of fit. The hypothesis H0: X1 = 2 = 0 is

rejected in favor of Ha : [1/2] [x2/1] < 0 if the cal-
-1
culated value of F = (d'Vo d/2)/MSE is less than

F(1-a);2,1, in which case we say lack of fit of the model is

present.
^2
If an external estimate aext had been available for
ext
this example, then the optimal upper tailed test check

points could have been used in the F ratio,
-1 ^2
F = (d'V d/k)/o t and lack of fit would then be detected
0 k exta

if the calculated value of F exceeded F ,
a;k,v
Example 3. Example 3 illustrates the procedure for

locating optimal check points when there are two terms in

the true model in addition to those in the fitted model. A

second order canonical polynomial model in three mixture







components is fitted using a q = 3 simplex centroid design.

The true model is assumed to contain eight terms, six of

which are the same terms as in the fitted model, with the

additional two terms being the third order terms

612x x2(X1 x2) and B123x x2x3. As in Example 2, there is

one degree of freedom for MSE which is used to estimate
2 T-
2. The test statistic, F = (d'V d/k)/MSE, is given in

equation (3.8).

Since p2 = 2 and A1 is a 2x2 matrix, locating the

optimal upper tailed test check points by the procedure of

maximizing X1 is assisted by the maximizing of a lower bound

for 1 namely maximizing u min /2a where u is the
I min-2-2 mm
smallest eigenvalue of A1. Since 82 and a2 are unknown,

this is equivalent to maximizing min For min to exceed

zero, it is necessary that A1 be of full rank, and since

rank(A1) ( min(k, p2), it is necessary to select k > 2 check

points. If A1 is less than full rank, and thus is positive

semi-definite, only a subset of possible values of 2 could

be considered to make it possible to test for lack of fit

with an upper tailed test.

Using the Controlled Random Search Procedure, the

points that maximize min are found to be (0.418, 0.277,
mln
0.305) and (0.277, 0.418, 0.305). These points are thus

optimal candidates for upper tailed test check points. At

these check points we have umin = 5.1623 x 10-4, A1 =

diag[5.1623 x 10-4, 5.1916 x 10-4], A2 = diag[0, 8.4175 x

10-4], and R = [A1/2] [A2/I] = diag[2.5811 x 10-4, -5.8217




76

x 10-4]. Since the eigenvalues of R are -5.8217 x 10-4 and

2.5811 x 10-4, R is indefinite. Following the suggested

procedure for Case 4 of Section 3.3.4, we note that an upper

tailed test for lack of fit exists if the value of

'2 = [512' 8123]' is in the column space of [Sl:S2] but not
entirely in the column space of S2, where S1 is the matrix

whose columns are the orthonormalized eigenvectors of R

corresponding to the positive eigenvalues of R, and S2 is

the matrix whose columns are the orthonormalized

eigenvectors of R corresponding to the zero eigenvalues of

R. Since R has no zero eigenvalues in this example, S2 does

not exist, but S1 is the column vector, S1 = [1,0]'. Thus

if 2 is of the form 2 = [612 0]', where 12 0, then

a2 is in the column space of S1 and the test is upper
tailed.

The matrix A2 has rank one and therefore is positive

semi-definite. Hence it is impossible to locate two check

points that minimize umax and also make R = [A1/2] [A2/1]

negative definite (see Appendix 4), that is, it is

impossible to find a lower tailed test that is capable of

testing lack of fit for all values of 2. However, if we

use the Controlled Random Search Procedure to locate two

check points that minimize an upper bound for X1 which is

umax S2 /2 2, then by minimizing max, we find that any of
the check points in a particular subregion of the factor

space yield a near minimum for umax. One pair of points in

this subregion is selected as the points to be used as








optimal lower tailed test check points, namely the pair

consisting of the point (0.053, 0, 0.947) replicated twice.

Replicating this check point, we find max = 7.3900

x 10-11, A = diag[0, 7.3900 x 10-11], A2 = diag[0, 8.4175 x

10-4], and R = [A1/2] [A2/1] = diag[0, -8.4175 x 10-4].

The eigenvalues of R are 0 and -8.4175 x 10-4 implying that

R is negative semi-definite. The values of 8 that are in
-2
the column space of [S2:S31 but not entirely in the column

space of S2 will provide a lower tailed test. Here, [S2:S3]

= diag[l,l] and S2 = [11,0]'. Thus, the test for lack of fit

is lower tailed if 8123 0.

For values of 8 that produce an upper tailed test we
-2
use the check points (0.418, 0.277, 0.305) and (0.277,

0.418, 0.305) with the F ratio



-1
d'Vo d/2
F =
MSE


and conclude there is lack of fit if the calculated value of

F exceeds Fa;21. For values of a2 that produce a lower

tailed test, we use two replicates of the check point

(0.053, 0, 0.947), and'conclude there is lack of fit if F is

less than F(1-a);2,1, where again F is calculated by
-1
F = (d'V d/2)/MSE.
0O
Example 4. Example 4 illustrates Case 3 of Section

3.3.4 in which MSE is used to estimate 02 in the lack of fit

test statistic. A second order canonical polynomial model







in three mixture components is fitted using the {3,3}

simplex lattice design, which appears in Figure 5. The true

model is assumed to be special cubic, thus p2 = 1 and A1 is

a scalar. The {3,3} design consists of ten design points

and since there are six parameters to be estimated in the

fitted model, 02 can be estimated by MSE with N p = 10 6

= 4 degrees of freedom.

We first suppose that a single check point is to be

used to test for lack of fit. Using the Controlled Random

Search Procedure we find the single check point that

maximizes the scalar


-1
A= (X X*A)'V (X* X*A)
1 2 0 2


is located at the centroid of the simplex factor space.

Thus (x*, x*, x*) = (1/3, 1/3, 1/3) is the optimal candidate

for an upper tailed test check point. At this centroid

point, A1 = 4.9076 x 10-4. For the {3,3} design the scalar

quantity A2 = (X2 XA)'(X2 XA) is fixed and is equal to

A2 = 9.4062 x 10-4 and thus, R = [Al/k] [A2/(N p)] =

[(4.9076 x 10-4)/1] [(9.4062 x 10-4)/4] = 2.5560 x 10-4

The point that is the optimal candidate for a lower

tailed test check point is chosen randomly from a subregion

of points in the factor space, in which all points minimize

A1. The point selected has the value (x*, x*, x*) = (0.560,

0.410, 0.030). Here A1 = 9.6590 x 10-7 and R = [(9.6590 x

10-7)/1] [(9.4062 x 10-4)/4] = -2.3419 x 10-4.












(1,0,0)







3 3 a0o 0'x
( 0)/ 3 3

3' 3'3

(0o,,0) --- (0,0,1)
x2 0, ) 3 3' ) x3= 1

Figure 5. The {3,3} simplex lattice design.





Since R is positive for the optimal upper tailed test

check point (1/3, 1/3, 1/3) and R is negative for the

optimal lower tailed test check point (0.560, 0.410, 0.030)

we are in Case 3 of Section 3.3.4. Either the upper or

lower tailed test could be used to test for lack of fit, but

if the quantity B B2/o2 can be specified, then we will

choose to use the test that has greater minimum power, since

greater power means that we are more likely to detect lack

of fit when in fact lack of fit exists. In this example

-2 123*
For illustrative purposes, we arbitrarily choose
2
8'8 /C2 = 2000, so that an approximate conservative lower
bound for the power of the upper tailed test is found by







calculating



P {F" > F k
k,N-p;XIL' 2U a;k,N-p


where F;k,N-p is the upper 100a percentage point of the

central F distribution, k is the number of check points, N

is the total number of response observations, p is the

number of parameters in the fitted model,
2 2
IlL = UminB-2/2a and 2U 6 axI/20 The

quantity umin is the smallest eigenvalue of AI, where A1 is

evaluated at the optimal upper tailed test check point.

Since A is a scalar, mn = A. Likewise, 6 is the
1 mim 1 fmax
largest eigenvalue of A2, and since in this example A2 is a

scalar, 6max = A2. In this example we have k = 1, N p =

10 6 = 4, AlL = UminS2/2a2 = (4.9076 x 10-4)(2000/2)
-1 2
= 4.9076 x 101, and 2U = 6 m /2a2

= (9.4062 x 10 )(2000/2) = 9.4062 x 101. Using the

approximation to the cumulative probabilities of the doubly

noncentral F distribution given by Johnson and Kotz (1970,

p.197) which is described in Appendix 6, and taking a = .05,

we find that a conservative lower bound for the power of the

optimal upper tailed test is approximately equal to .0649.

The minimum power for the optimal lower tailed test is
2
approximated (assuming 8_Y2/a = 2000) by calculating



P F" < FN
k,N-p;lU' 2L (1-a);k,N-p







2
The quantities A1U and 2L are taken as A U = maxJ2 /2o
lJU 2L lU max-z22
and 2L = 6 mn /2o where max is the largest eigenvalue
2L min-2-2 max
of A1 with A1 calculated using the optimal lower tailed test

check point, and where 6 in is the smallest eigenvalue of

A2. Since A1 and A2 are scalars, max = A and 6 min = A .
2 1 2 max 1 mm 2
In this example, k = 1, N p = 4,

AlU = (9.6590 x 10 )(2000/2) = 9.6590 x 104, and
-4 -i
x2L = (9.4062 x 10 )(2000/2) = 9.4062 x 101 Again if the

approximation to the doubly noncentral F distribution given

in Johnson and Kotz is used, an approximate conservative

lower bound for the power of the optimal lower tailed test

is .0555.

Having specified a 2/02 = 2000, the optimal upper

tailed test is chosen over the optimal lower tailed test,

because the approximate minimum power of the upper tailed

test is greater than the approximate minimum power of the

lower tailed test. Using the optimal upper tailed test

check point (1/3, 1/3, 1/3) in the test statistic

-1
d'V d
F =
MSE


we conclude that lack of fit is significant if the

calculated value of F exceeds F a1,4 in which case we
a;1,4'
reject H0: XI = 2 = 0 in favor of Ha: A/l 12/4 > 0.

When two simultaneous check points are used for testing

lack of fit, the Controlled Random Search Procedure locates

the optimal upper tailed test and optimal lower tailed test







check points. It turns out that two replicates at (1/3,

1/3, 1/3) maximize Al, and are used as optimal check points

for an upper tailed test. The value of R = [A,/2] [A2/4]

is [(7.9210 x 10-4)/2] [(9.4062 x 10-4)/4] = 1.6090 x

10-4.

In searching for two optimal lower tailed test check

points, again a subregion of the factor space is found in

which any of the points nearly minimize A From this

subregion are chosen the points (0.6386, 0.3263, 0.0351) and

(0.7257, 0.2421, 0.0322) resulting in a value of R = [A,/2]

- [A2/4] of [(1.5216 x 10-9)/2] [(9.4062 x 10-4)/4] =

-2.3516 x 10-4.

In conclusion, when two simultaneous check points are

used in the test for lack of fit in this example, R is

positive for the optimal upper tailed test and R is negative

for the optimal lower tailed test, and we have Case 3 of
2
Section 3.3.4. Selecting 002 /o = 2000 arbitrarily, we

found the approximate lower bound for the power of the upper

tailed test to be .0504, and the approximate lower bound for

the power of the lower tailed test to be .0612. Since the

power is higher with the lower tailed test it is our choice

for testing lack of fit when two check points are used

simultaneously. Lack of fit is detected and we reject

H0: 1 2 = 0 in favor of Ha: [1 /2] [ 2/4] < 0 if the F
-1
ratio, F = (d'V0 d/2)/MSE, using the optimal lower tailed

test check points (0.6386, 0.3263, 0.0351) and (0.7257,

0.2421, 0.0322) is calculated to be less than F1-a);2,4'
(i-a);2 ,4"







3.4.2 Numerical Examples

Numerical Example 1. In this example we illustrate

numerically some of the findings in the first theoretical

example of Section 3.4.1. Data that were collected in a

rocket fuel experiment (Kurotori, 1966) will be used to

investigate the power of the lack of fit F test. The test

is set up to detect the inadequacy of a fitted second order

canonical polynomial model when the true model is special

cubic. Calculated values of the power of the test which

detects lack of fit through large values of

-1
d'V0 d/k
F =
^2
ext

will be compared for several check point locations, includ-

ing the location (1/3, 1/3, 1/3) at which the power was

found to be maximum in Example 1 of Section 3.4.1.

In Kurotori's experiment the modulus of elasticity (Y)

of a rocket fuel is expressed as a function of the

proportions of three components--binder (xl), oxidizer (x2),

and fuel (x3). Since lower bounds are placed on the

component proportions xl, x2, and x3, in the form of

0.20 < xl, 0.40 < x2, and 0.20 < x3, pseudocomponents (x!)

are defined in terms of the original components in the form

of x1 = (xl 0.20)/(1 .80), x' = (x2 0.40)/(1 .80),

and x' = (x3 0.20)/(1 .80). The true special cubic

model in the pseudocomponents, which is obtained by using

the data at the seven points of the simplex centroid design







in the pseudocomponent system, is



E(Y) = 2350x' + 2450x' + 2650x' + Ox'x'
1 2 3 1 2


+ lOOOx'x3 + 1600x'x' + 6150x'x'x'.



The second order canonical polynomial model that is fitted

to the six boundary points only, and which will be tested

for lack of fit, is given by



Y = 2350x' + 2450x' + 2650x'
3


+ 1000x'x' + 1600xNx'.



The configuration of the experimental design as well as the

check point locations are depicted in Figure 4 of Chapter 2

and the observed response values are given in Table 3 of

this chapter.
-1 ^2
A value of the ratio F = [d'V d]/o is calculated at
0 ext
each of the four individual check points (1/3, 1/3, 1/3),

(2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3)
"2 ^2
where ext is assumed to have the value et = 144 as

suggested by Kurotori (1966). We also assume without loss

of generality that the degrees of freedom associated
^2
with aet are v = 10. The power of the F test is calculated

at each of the four check points by using the approximation

to the cumulative probabilities of the noncentral F












Table 3. Observed Response Values at the
Pseudocomponent Settings for Kurotori's Rocket Fuel
Experiment--Numerical Example 1.


Observation


Number x'
______ _____


1
2
3
4
5
6
7*
8*
9*
10*


Binder Oxidizer Fuel


Modulus


x 3


0
1/2
1/2
0
1/3
2/3
1/6
1/6


0
1
0
1/2
0
1/2
1/3
1/6
2/3
1/6


0
0
1
0
1/2
1/2
1/3
1/6
1/6
2/3


of Elasticity
Y


2350
2450
2650
2400
2750
2950
3000
2690
2770
2980


*Check Points.


distribution given by Johnson and Kotz (1970, p. 197) to

evaluate


Power = P{FI' ,;,
11, 10;Xi


F.05;1,10}


2 ^2
where 1 = A 8 23/2 ext.
I 1 123 ex


The value of


A = (X* X*A)'V0 (X* X*A) is calculated for each check

point using the values of X*, X*, v and the value of

A = (X'X) X'X2 which is fixed by the {3,2} simplex lattice

design. Since the {3,2} simplex lattice consists of points







only on the boundaries of the triangle (and therefore at

each point at least one of the x! values is equal to zero),
1
then X2 = 0 and A = 0. From the true special cubic model,

123 = 6150.

The calculated value of F as well as the approximate

value for the power at each of the four check points is

given in Table 4. The check point (1/3, 1/3, 1/3) produced

the highest power of the four check points investigated,

supporting the previous results of Example 1 in Section

3.4.1 where (1/3, 1/3, 1/3) was selected as the check point

location with the maximum power when a second order

canonical polynomial was fitted using the {3,2} simplex

lattice design, but the true model was assumed to be special

cubic. Additional support for the point (1/3, 1/3, 1/3)

being optimal is given by the contour plot of values of A1

in Figure 6(d). The highest values of A1 appear near the

centroid (1/3, 1/3, 1/3) where high A1 values translate into
2 2
high X1 values, since Xi = A a123/22 which in turn implies

high power since we know the power is an increasing function

of X1.

As a second part of this example the power of the F

test that is obtained when three replicates are taken at

(1/3, 1/3, 1/3) is compared to the power of the F test that

is obtained when one replicate is taken at the three check

points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6,

2/3). These latter three point locations were suggested by

Kurotori for testing lack of fit of his fitted special

















































o 0 0 0
-4


r-1 -4- r- -4
,- -4 ,- O


OC










1-1




mm







M N\N
i- '

- '--


0 *


0w






O


.JU
Q .H

-3


Q)
4J-)
U

-4

0
U





0
m-
4a
4J






(0
>1
cu






4Ja



CO
















00
S0O

-Cu

A 0


u r-
0 0







-c




o




4 r-4








UO-4








A U( -
S*
I -4 4 J
U K M


1-4
o
o
o







o
























r-I- CN


, q --



CN -.4 ,-I
(m)











cMNrN
S S


0 -
4J -4




Z 1







cubic model. The result of this comparison, see Table 4, is

that the three replicates at (1/3, 1/3, 1/3) produce the

test with greater power which again supports the findings of

Example 1 of Section 3.4.1.

All of the check point locations listed in Table 4

produce very high power values (> .999) which is due in part

to the high value of 123 (123 = 6150). If S1 were of
123 123 6123
lower magnitude, then the three replicates at (1/3, 1/3,

1/3) would show a still greater superiority in the power

value compared to the power using the other check points.

This superiority is demonstrated in Table 5 where values of

8123 are listed as 3000 and 1500 and the comparative power

values are listed as 0.998 compared to 0.795 and 0.662

compared to 0.249, respectively. Table 5 also demonstrates

the superior power value for the point (1/3, 1/3, 1/3) when

123 = 3000 or 123 = 1500 and each of the four check points

is used one at a time.

Finally, (1/3, 1/3, 1/3) being the optimal check point

location is seen in Figure 6(c), where contour plots of the

expected difference in the heights of the surfaces are

drawn. The differences in the heights are found by

subtracting the estimated height of the surface obtained

with the fitted second order model from the estimated height

of the surface obtained with the true special cubic model.

The expected difference between the true and fitted surfaces

approaches a maximum the closer one moves to the centroid of

the simplex factor space, so that the optimal check point







89







<4 0O CD CD C
a) m Co C MN C N c
C 0 *

U a4




-I COD0 CO
U- co co 1 o CD m

0
a4


41J -4

C a) m co o o co m a) 4)
0 (a D aN O Oo *10 10 m r- En

C >1* *





0
U a)


0 a U1 m a) 0T co
m 2' 0 0



'0 Z O*o r- *- r*- *< u
Z ; 0 O
Oi 0
*-q 4J V





4 L 0 4 0 *



q m o .m .
0 )








En *' m

a) 0








0 Cr rn)Cr)Cr)

^ N. N. N.. N. 'NN.'N N

a) N N N N NNN N.N U
-; -4 (N i- -4 C '4 M |-4 -4
U .4'













x :I
I


X =1 x= I
2 3
(a) True special cubic surface.








X =I


x :1 x :1
2 3
(c) Expected difference between the
true special cubic surface and
the fitted second order surface.


x2-
(b) Fitted second order surface.


x =1 X :1
2 3

(d) A( X -X XA)' V'( X- X*A)


Figure 6. Contour plots for Numerical Example 1.







location (1/3, 1/3, 1/3) coincides with the point where the

expected difference between the true special cubic surface

and the fitted second order surface is maximum.

Numerical Example 2. In this second numerical example,

we investigate the power of the F test for detecting lack of

fit when a second order canonical polynomial model is fitted

in a mixture system which is in truth represented by a

special cubic model. The true model is assumed to be



E(Y) = 2350x1 + 2450x2 + 2650x3



+ 1000x x3 + 1600x2x3 + 6150x 2X3



which is used to generate hypothetical response observations

at the seven points of the q = 3 simplex centroid design as

well as at three check points. The values of the response

are obtained by adding the value of a pseudorandom normal

variate with mean 0 and variance 144 to each true predicted

response value. The data are given in Table 6.

The response values at the seven points of the simplex

centroid design are used in the least squares normal

equations to obtain the fitted second order model



Y = 2341x1 + 2438x2 + 2630x3


+ 310x1x2 + 1304x x3 + 1970x2x3











Table 6. Generated Response Values--Numerical Example 2.

xl x2 x3 Y

1 0 0 2357
0 1 0 2454
0 0 1 2646
1/2 1/2 0 2403
1/2 0 1/2 2747
0 1/2 1/2 2962
1/3 1/3 1/3 3013
1/3 1/3 1/3 2993
2/3 1/6 1/6 2693
.02 .93 .05 2550

*Check points.









which is to be tested for lack of fit using the test
-1
statistic F = d'V0 d/MSE. The F statistic will be evaluated

at each of the three check points (1/3, 1/3, 1/3),

(2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time,

and the power of the test at the three check point locations

will be calculated and compared. The test is lower tailed

for all check point locations (since R = A1 A2 is negative

for all check point locations) and thus the power is defined

as



PI" ( F;, }.
1,1;XIX2 .95;1,1




Full Text

PAGE 1

TESTING LACK OF FIT IN A MIXTURE MODEL BY JOHN THOMAS SHELTON A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1982

PAGE 2

To Nydra and My Parents

PAGE 3

ACKNOWLEDGEMENTS I would like to express my deepest appreciation to Drs . Andre ' Khuri and John Cornell for suggesting this topic to me and for providing constant guidance and assistance. They have made this project not only a rewarding educational experience but an enjoyable one as well. A special word of thanks goes to Mrs. Carol Pvozear for her diligence in transforming my handwritten draft into an expertly typed manuscript. Ill

PAGE 4

TABLE OF CONTENTS page ACKNOWLEDGEMENTS iii ABSTRACT vii CHAPTER ONE INTRODUCTION 1 1.1 The Response Surface Problem 1 1.2 The Mixture Problem 5 1.2.1 Mixture Models 6 1.2.2 Experimental Designs for Mixtures.. 12 1.3 The Purpose of this Research: Investigation of Procedures for Testing a Model Fitted in a Mixture System for Lack of Fit 17 TWO LITERATURE REVIEW — TESTING FOR LACK OF FIT 19 2.1 Introduction 19 2.2 Partitioning the Residual Sum of Squares.. 21 2.3 Testing for Lack of Fit Without Replicated Observations — Near Neighbor Procedures 26 2.4 Testing for Lack of Fit with Check Points. 33 THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING LACK OF FIT IN A MIXTURE MODEL 40 3.1 Introduction 40 3.2 Testing for Lack of Fit in the Presence of an External Estimate of Experimental Error Variation 41 3.2.1 The Test Statistic 41 3.2.2 The Testing Procedure and an Expression for the Power of The Test 45 3.2.3 A Method for Locating Optimal Check Points 47 3.3 Testing for Lack of Fit When MSE Is Used to Estimate Experimental Error Variation 51 3.3.1 The Test Statistic 51 3.3.2 The Rejection Region and its Relation to the Power of the Test.. 53 iv

PAGE 5

3.3.3 A Method for Locating Optimal Check Points 56 3.3.4 Determining VVhether the Test Is Upper Tailed or Lower Tailed 58 3 . 4 Examples 67 3.4.1 Theoretical Examples 67 3.4.2 Numerical Examples 83 3.5 Discussion 95 FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR TESTING LACK OF FIT 99 4.1 Introduction 99 4.2 Notation 101 4.3 Shillington ' s Procedure 106 4.3.1 Development of MSEg 109 4.3.2 Development of MSE^ 110 4.4 Development of SSE^^( weighted ) 112 4.5 Equivalence of SSE^^ and SSEy^(weighted ) . . . . 116 4.6 The Test Statistic 118 4.7 The Testing Procedure and its Power 122 4.8 Selection of Near Neighbor Groupings 125 4.8.1 Example 1 — Stack Loss Data 129 4.8.2 Example 2 — Glass Leaching Data 134 4.9 Discussion 142 FIVE CONCLUSIONS AND RECOMMENDATIONS 145 APPENDICES 1 INFLUENCE OF X, ON P{F" , , > F } 156 vi,V2;Xi,X2 a;vi,V2 2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR GLOBAL OPTIMIZATION 159 3 STATISTICAL INDEPENDENCE OF d'V"-'-d/a^ AND SSE/o ^ 164 4 THEOREM 3.1 168 5 THEOREM 3.2 169 6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION 171 7 EQUIVALENCE OF SSEg AND SSloF WHEN REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS 172 V

PAGE 6

8 LEMMA 4.1 175 9 PROOF OF THEOREM 4.1(1) 178 10 PROOF OF THEOREM 4.1(11) 182 11 PROOF OF THEOREM 4.1(111) 185 12 PROOF OF THEOREM 4.2 191 13 PROOF OF THE EQUALITY SSE = d'V~''"d + SSE 193 REFERENCES 198 BIOGRAPHICAL SKETCH ; 202 VI

PAGE 7

Abstract of Dissertation Presented to the Graduate Council of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy TESTING LACK OF FIT IN A MIXTURE MODEL By John Thomas Shelton May 1982 Chairman: Andre' I. Khuri Cochairman: John A. Cornell Major Department: Statistics A common problem in modeling the response surface in most systems, and in particular in a mixture system, is that of detecting lack of fit, or inadequancy, of a fitted model of the form E(Y) = Xg, in comparison to a model of the form E{Y) = Xe,+ X B postulated as the true model. One method for detecting lack of fit involves comparing the value of the response observed at certain locations in the factor space, called "check points," with the value of the response that the fitted model predicts at these same check points. The observations at the check points are used only for testing lack of fit and are not used in fitting the model. It is shown that under the usual assumptions of independent and normally distributed errors, the lack of fit test statistic which uses the data at the check points is an vii

PAGE 8

F statistic. When no lack of fit is present the statistic possesses a central F distribution, but in general, in the presence of lack of fit, the statistic possesses a doubly noncentral F distribution. The power of this F test depends on the location of the check points in the factor space through its noncentrality parameters. A method of selecting check points that maximize the power of the test for lack of fit through their influence on the numerator noncentrality parameter is developed. A second method for detecting lack of fit relies on replicated response observations. The residual sum of squares from the fitted model is partitioned into a pure error variation component and into a lack of fit variation component. Lack of fit is detected if the lack of fit variation is large in comparison to the pure error variation. This method can be generalized when "near neighbor" observations must be substituted for replicates. In this case, the test statistic (assuming independent and normally distributed errors) has a central F distribution when the fitted model is adequate and a doubly noncentral F distribution under lack of fit. The arrangement of near neighbors is seen to affect the testing procedure and its power. Vlll

PAGE 9

CHAPTER ONE INTRODUCTION 1.1 The Response Surface Problem A mixture problem is a special type of a response surface problem. First we shall define the general response surface problem and indicate the basic objectives sought in its analysis, and follow this development with a discussion of the mixture problem. In the general response surface problem, we are interested in studying the relationship between an observable response, Y, and a set of q independent variables or factors, x^, X2» •••f Xq, whose levels are assumed controlled by the experimenter. The independent variables are quantitative and continuous. We express this relationship in terms of a continuous response function, (j) , as ^u = f(^ur '^U2' •••' ^uq) " 'u where Y^ is the uth of N observations of the response collected in an experiment, and x^^^ represents the uth level of the ith independent variable, u = 1, 2, . . . , N, i = 1, 2, ..., q. The exact functional relationship,
PAGE 10

observation. It is assumed that E(e^J) = 0, E(e^e^l) = 0, for u * u\ and E(e^) = a^, for u = 1, 2, ..., N. As the form of (j) is unknown and may be quite complex, a low order polynomial (usually first or second order) in the independent variables x-j^, X2, ..., Xg is generally used to approximate 41. This may be justified by noting that such polynomials constitute low order terms of a Taylor series expansion of <{» about the point ^Cj^ = X2 = ... = x = 0, (Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point out that these low order polynomials may give a poor approximation to (\) when extrapolated beyond the experimental region, and thus should not be used for this purpose. A linear response surface model may be written in matrix notation as Y = XB + e (1.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants, a. is a pxl vector of unknown parameters (regression coefficients), and £. is the Nxl vector of random errors. When the model is a first or a second degree polynomial, the columns of X correspond to the first or second degree powers of the independent variables x^, X2, •••, Xg, or their cross products. If the model contains a constant term, Sq, the first column of X will correspond to this term, and will consist of N ones. Since E(e) = 0, an alternative representation for the response

PAGE 11

surface model of (1.1) is E(Y) = Xg . Once the form of the model that will be used to approximate 4)(X2/ X2» ...f Xq) is chosen, the next step is to estimate the regression coefficients, a., and then use the estimated model to make inferences about the true response function, (j) . The estimation of the elements of a. is usually accomplished by ordinary least squares techniques. For the purpose of testing hypotheses concerning the regression coefficients, a., it is assumed that g. has a normal distribu2 tion, that is, e_ ~ N (0, a !«) • Perhaps the most common objective in the exploration of a response system is the determination of its optimum operating conditions. By this we mean that it is desired to find the settings of x-^, X2r ...f Xq that optimize (^ , which in some applications may be interpreted as maximizing (j) , while in other applications a minimum value of <^ may be of interest. It is also often desirable to determine the behavior of the response function in the neighborhood of the optimum. For second order response models, such an investigation can be carried out by performing a canonical analysis of the second order surface as discussed in Myers (1971). For simple systems having only one or two independent variables, the response surface may be explored by just plotting the fitted response values against values taken by

PAGE 12

the independent variables. If q = 1, implying only one independent variable, say x^^, then a two-dimensional plot of the fitted response against x^^ may be used to locate the optimum, as well as to investigate the response behavior in other parts of the experimental range of x-^. If q = 2, and the two independent variables are xj^ and X2f then a plot of the contours of constant response over the region specified by the ranges of the values for x^ and X2 can be used to describe the response surface. The properties that the fitted model possesses in terms of its ability to represent the true surface, <^ , depend on the settings of x-^, X2, ...f Xg at which values of Y are observed. Thus the experimental design is of great importance. Much work has been done on the construction of designs that are optimal with respect to one criterion or another involving the fitted response and/or the true unfitted model. Box and Draper (1975) list fourteen criteria to consider when choosing a design for investigating response surfaces. Myers (1971) gives several designs for fitting first and second order polynomial models. A discussion of specific design considerations will not be attempted here, as such a discussion is not the focus of this dissertation, and would necessarily be lengthy. The initial steps in the analysis of a response system may be described as follows: First an attempt is made to approximate the true response function, (})(X]^, X2, •••/ Xq), usually with a low order polynomial in x^, X21 ...f Xg.

PAGE 13

After the form of the model has been chosen, then comes the selection of an appropriate experimental design, which specifies the settings of the independent variables at which observed values of the response will be collected. The observed values of the response are used in estimating the regression coefficients in the model, using, in general, ordinary least squares. After a test for "goodness of fit" of the model verifies the fitted model is adequate, the fitted model is used in determining optimum operating conditions for the response system. 1.2 The Mixture Problem A mixture system is a response system in which the response depends only on the relative proportions of the components or ingredients present in a mixture, and not on the total amount of the mixture. For example, the response might be the octane rating of a blend of gasolines where the rating is a function only of the relative percentages of the gasoline types present in the blend. The proportion of each ingredient in the mixture, denoted by xj^, must lie between zero and unity, i = 1, 2, ..., q. The sum of the proportions of all the components will equal unity, that is. q < X. < 1, i = 1,2,. ..,q, I. X. = 1. (1.2) i=l The factor space containing the q components is represented by a (q l)-dimensional simplex. For q = 2 components, the factor space is a straight line, whereas for q = 3

PAGE 14

components, the factor space is an equilateral triangle, and for q = 4 components, the factor space is represented by a regular tetrahedron. The objectives in the analysis of a mixture response system are, in general, the same as in any response surface exploration. That is, one seeks to approximate the surface with a model equation by fitting an equation to observations taken at preselected combinations of the mixture components. Another objective is to determine the roles played by the individual components. We shall not concern ourselves with this but rather concentrate on the empirical model fit. Once the model equation is deemed adequate an attempt is made to determine which of the component combinations yield the optimal response. The models used to represent the response in a mixture system are in most cases different in form from the standard polynomial models. The first type of model form that we discuss is the canonical polynomial suggested by Scheffe. 1.2.1 Mixture Models Scheffe (1958) introduced a canonical form of the polynomial model for representing the response in a mixture system. These canonical polynomial models are derived from the standard polynomials using the restrictions on the Xj^ shown in (1.2). With q = 2 mixture components, for example, the standard second order polynomial model is of the form 2 2

PAGE 15

Restrictions (1.2) imply that ag = aQ(Xj^ + X2)f ^1 = ^1^1 " ^2^' ^"^ ^2 "" ^2^1 " ^1^' ^^^^ (1.3) can be written in the canonical form E(Y) = S^x^ + ^2X2 + e^2^^2' where e^= a^ + a^ + a^^, ^2 = ^0 "" '"2 "^ ^22' ^"^ ^12= "l2 -a a . There is no constant term in the above canonical form and the pure quadratic terms in equation (1.3) have been absorbed in the x^Xj terms. The general form of the canonical polynomial of degree d in q mixture components can be written as E(Y) = E e, X. , for d = 1, i=l E(Y) = Z g.x. + Z Z B..X.X. , for d = 2, and --ill ^. . i;] ID 1=1 l 4 in q components does not explicitly appear in the literature, but is mentioned in Scheffe (1958). If terms of the form 6ijXiXj(xi Xj) are removed from the full cubic model (1.4), then the remaining terms

PAGE 16

8 make up what is referred to as the special cubic model. For example, for q = 3 components, the special cubic model is E(Y) = B^x^ f 02^2 -^ ^3^3 ^ ^12^1^2 "^ ^3^1^3 ^ ^23^2^3 -^ H23^l"2"3 * Scheffe's canonical polynomial models are used for approximating the response surface in many mixture systems. Their popularity stems from the ease in interpreting the coefficient estimates, especially when the models are fitted to data collected at the points of the associated designs (see Section 1.2.2). However, other models have been introduced which seem to better represent the response when the components have strictly additive blending effects. We present some of them now. Becker (1968) introduced three forms of homogeneous models of degree one which he recommends instead of the polynomial models when one or more of the mixture components have an additive effect or when one or more components are inert. A function f(x, y, ..., z) is said to be homogeneous of degree n when f(tx, ty, ..., tz ) = t'^f(x, y, ..., z), for every positive value of t and (x, y, ..., z) * (0, 0, ..., 0) . These models, which Becker refers to as models HI, H2, and H3 , are of the form

PAGE 17

q q HI: E(Y) = E g.x. + E Z B..min(x., x.) + ... i=l ^ l
PAGE 18

10 Another model form that is useful in the study of the response in a mixture system is the model containing ratios of the component proportions. A term such as x.^/x^ measures the relationship of x^^ to x^ rather than the percentage of each in the blends. Snee (1973) points out that the ratio model presents a useful alternative to the Scheffe and Becker models in that the ratio model describes a different type of curvature. He notes that the curvilinear terms for the Scheffe and Becker models, when plotted as a function of X£, are symmetric functions about Xj^ = 1/2, whereas the ratio term Xj^/xj is a monotone function when plotted against XiThe terms in the ratio models may also contain sums of the components. For example, with q = 3 components, we might express the second order model q-1 q-1 q-1 E(Y)=e-+ EB.Z.+ ZZ B..Z.Z.+ E Q . . z. ° i=l ^ ^ l
PAGE 19

11 When one or more of the components is inactive, Becker (1978) suggests that a ratio model that is homogeneous of degree zero in the remaining components is appropriate. In three components, such a model is of the form E(Y) = Bq + ^^x^/{x^ + x^) + e2X2/(X2 + x^) + 33X3/(x^. X3) -H ^^E^Z Bijh..(x., x.) ^ ^123^123^^1' ^2' ^3^' ^^'^^ where h^^^ and hj^23 ^^® specified functions that are homogeneous of degree zero. The function hj^23 ^^ intended to represent the joint effect of all three components simultaneously. If in fitting a model of the form (1.5) we determine the model should be E(Y) = 3q + e-^Xj^/Cx^ + X2) + B;l2^12^^1' ^2^ then component three is said to be inactive and is removed from further consideration. The model of equation (1.5) may produce an extreme value near the vertices of the simplex factor space when there are no inactive components. In this case it is suggested that a model of the form (1.5) be used only when the proportions are restricted so that no two of the x^ are simultaneously very close to zero. Beclcer notes that other authors who have suggested ratio models have also

PAGE 20

12 used them primarily over a subregion inside the simplex factor space. Apparently this is where they are most appropriate . 1.2.2 Experimental Designs for Mixtures As in the general response surface problem, one of the major concerns in exploring a mixture system is that of choosing the experimental design for collecting observed values of the response that will be used in fitting the model. Scheffe (1958) proposed the {q,m} simplex lattice designs for exploring the entire q-component simplex factor space. In these designs, the proportions used for each component have the m + 1 values spaced equally from zero to one, Xj^ = 0, 1/m, 2/m, ..., (m l)/m, 1, and all possible mixtures with these proportions for each component are used. The number of design points in the {q,m} simplex lattice design is ('"*"'' ^ ~ '•) . The main appeal of these designs is that they provide a uniform coverage of the factor space. Another feature, which Scheffe (1958) demonstrates, is that the parameters of the canonical polynomial of degree m in q components are expressible as simple linear combinations of the true response values at the design points of the {q,m} simplex lattice. The {3,2} simplex lattice, which consists of six design points, is represented on the two dimensional simplex in Figure 1 along with the triangular coordinates (xj^, X2, X3). Scheffe (1963) also developed the simplex centroid designs consisting of 2^ 1 points, where the only mixtures

PAGE 21

13 considered are the ones in which the components present appear in equal proportions. That is, in a q-component simplex centroid design, the design points correspond to the q q permutations of (1, 0, 0, ..., 0), the (2) permutations of q (1/2, 1/2, 0, ,.., 0), the (3) permutations of (1/3, 1/3, 1/3, 0, ..., 0), ..., and the point (1/q, 1/q, ..., 1/q). This design alleviates the problem inherent in the {q,m} simplex lattice designs of observing responses at mixtures containing at most m components. To give an example, the q = 3 simplex centroid design is made up of 2-^ 1 = 7 design points, and is equivalent to the {3,2} simplex lattice design augmented by the center point (xj^, X2, X3) = (1/3, 1/3, 1/3). This design is represented in Figure 2. Scheffe (1963) mentions that the number of parameters in the polynomial model of the form q q q E(Y) = E 3.x. + E E e..x.x. + E E Z e..,x.x.x, i=i ' ' i
PAGE 22

14 (^^0) (0.1,0) X =1 Z v ( 1,0,0) ("i'i) ii'°'i) (0,0,1 ) x = l 3 Figure 1. The {3,2} simplex lattice design. ii'h") (0,1 ,0) (1,0,0) ("4.]:) (5'°'i) (0,0,1) 2'2 *3=' Figure 2. The q = 3 simplex centroid design.

PAGE 23

15 therefore are natural models to fit using the simplex centroid design. Ratio models may be desirable when the interest in one or more of the mixture components is in terms of their relationship to one another, rather than in terms of their percentages in blends. Kenworthy (1963) proposed factorial arrangements for ratio variables. An example of the use of ratios is the following three component system in which the mixture components are constrained by the upper and lower bounds : .2 < X < .4, .2 < X < .4, .3 < x < .5. (1.7) The ratio variables of interest are z^ = X2/X]^ and ^2 ~ ^2/^3' ^^^ ^^ ^^ desired to fit either a first or a second order polynomial model in z^ and 22^ For such a problem, we can define a 2^ and a 3^ factorial design that can be used for fitting the first and second order polynomial models, respectively, by taking as design points the intersection of rays passing from two of the three vertices of the two-dimensional simplex through the region of interest defined by the constraints (1.7). Kenworthy 's 2 factorial design is shown in Figure 3. Becker (1978) uses rays extending from one or more vertices of the simplex factor space to the opposite boundaries in developing "radial designs." These designs are suggested for detecting the presence of an inactive

PAGE 24

16 X = I Design Points X =1 2 X :| 3 Figure 3. Kenworthy's 2^ factorial design component or in another case a component which has an additive effect, when models containing ratio terms that are homogeneous of degree zero are fitted. McLean and Anderson (1966) suggest an algorithm for locating the vertices of a restricted region of the simplex factor space which is defined by the placing of upper and lower bounds on the mixture component proportions. The vertices of the factor space and convex combinations of the vertices are the candidates for design points for fitting a first or second degree polynomial model in the mixture components. One drawback of the "extreme vertices" design is that the design points are not uniformly distributed over the factor space resulting in an imbalance in the variances of Y(x), see Cornell (1973).

PAGE 25

17 Another method that has been suggested for studying the response over a sub-region of the simplex mixture space is to transform the q mixture components into q 1 independent variables. Transforming to an independent variable system was first suggested by Claringbold (1955) and later proposed by Draper and Lawrence (1965a, 1965b) and Thompson and Myers (1968). Standard response surface polynomial models in the transformed variables can be fitted to data values collected on standard designs and a design criterion such as the average mean square error of the response can be employed when distinguishing between designs. Thompson and Myers (1968) suggest the use of rotatable designs (see also Cornell and Good, 1970). Designs other than rotatable designs, such as multiple lattices and symmetric-simplex designs, to name a few, have been suggested in the literature for fitting models to a mixture system which may be appropriate depending on particular experimental situations. However, as the intent here is not to give an exhaustive list but only a sampling of available designs, we shall not discuss designs further but instead state the purpose of this work. 1.3 The Purpose of this Research; Investigation of Procedures tor Testing "a" Model Fitted in A Mixture System for Lack of Fit A common problem in modeling the response in a mixture system is that of detecting lack of fit, or inadequacy, of a fitted model of the form E(Y) = Xg when the true model is of the form E(Y) = XQ^ + X B2The statistical literature

PAGE 26

18 suggests several procedures for testing lack of fit, which will be described in Chapter Two. In general, the authors of these procedures are not specific in stating hypotheses to be tested and do not adequately discuss the power of their procedures. The major purpose of this research is to investigate the power of two of the testing procedures appearing in the literature in detecting the inadequacy of a fitted model when the general form of the true model is specified. Our findings for a "check points" lack of fit testing procedure are presented in Chapter Three while Chapter Four contains findings for a "near neighbor" lack of fit testing procedure. For both procedures, explicit formulas for the power of the test are given in terms of cumulative probabilities of either the noncentral F or doubly noncentral F distribution, which are derived by assuming that the response observations are independent and normally distributed. Additionally, we propose methods for maximizing the power of the testing procedures. In the final chapter, we make some concluding comments concerning both of these procedures.

PAGE 27

CHAPTER TWO LITERATURE REVIEl'V — TESTING FOR LACK OF FIT 2.1 Introduction Let us return to the general response surface problem and assume the true response is to be approximated by fitting a model of the form E(Y) = XSj^ (2.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants, and ij^ is a px 1 vector of unknown regression coefficients. We wish to consider the situation in which the true model contains terms in addition to those in the fitted model. Then the true model has the form E(Y) = Xe^ + Y.^_^ (2.2) where X2 is an Nxp2 matrix of known constants, and e.2 is a P2>:1 vector of unknown regression coefficients. We assume that the vector Y has the normal distribution with 2 var(Y) = Ij^ . It is desirable to determine the suitability of the fitted model given by Eq. (2.1) when in reality the true model is of the form given by Eq. (2.2). The process of 19

PAGE 28

20 making this determination is referred to as testing for lack of fit of the fitted model. There are three general approaches to testing for lack of fit. The first approach requires that there be replicate observations of the response at one or more design points, and involves partitioning the residual sum of squares from the fitted model into a sum of squares due to lack of fit and a sum of squares due to pure error. A large value for the ratio of the mean square due to lack of fit to the mean square due to pure error provides evidence for lack of fit. If replicate observations are not available then the above approach to testing for lack of fit cannot be used. Green (1971), Daniel and Wood (1971), and Shillington (1979) have proposed alternative methods that are applicable in this case. Their approach is to group values of the response which are observed at similar settings of the independent variables and to call these grouped values "pseudoreplicates" or "near neighbor observations." They then treat these pseudoreplicates as they would treat true replicates to form statistics for lack of fit testing, although arriving at their respective statistics through different approaches. A third approach to testing for lack of fit involves the use of "check points." In this method a model of the form (2.1) is fitted to data at the design points and additional observations are collected at other points in the experimental region. The additional points other than the

PAGE 29

21 design points are called check points, and the data at these check points are not used in fitting the model. Lack of fit is tested by comparing the values of the response observed at the check points to the values of the response which the fitted model predicts at these same check points. We now discuss the first method mentioned above of testing for lack of fit which involves partitioning the residual sum of squares. 2.2 Partitioning the Residual Sum of Squares The method for testing lack of fit which makes use of a partitioning of the residual sum of squares from the fitted model requires there be replicate observations of the response at some of the design points (Draper and Smith, 1981, p. 120). When a model of the form (2.1) is fitted, the residual sum of squares is defined as n. SSE = E Z (Y. . Y. )^ i=l j=l ^J = Y'(Ifj X(X'X)~''"X' )Y where n is the number of distinct design points, nj^ > 1 is the number of replicate observations at the ith design point, Yj^^ is the jth observed value of the response at the ith design point, Yis the value which the model of the form in Eq. (2.1), fitted by ordinary least squares techniques, predicts for the response at the ith design n point, and N = i: n. . Using the replicated observations i=l ^

PAGE 30

22 only, a pure error sum of squares can be calculated as n . " ^ 2 SSE = E I (Y. . Y. ) , Pl^re ^^^ j^^ 1] ^1.' ' where Y^ ^ is the average of the values of the response observed at the ith design point. The sum of squares due to lack of fit can be obtained by taking the difference SS^^^ = SSE SSE LOF pure This partitioning of the residual sum of squares is displayed in the analysis of variance table in Table 1, Table 1. Analysis of Variance — Partitioning the Residual Sum of Squares. Source

PAGE 31

23 To test the hypothesis of zero lack of fit, that is Hq: lack of fit = or E(I) = X^x' ^^ F statistic is formed ^ = MSe'' (2.3) pure which possesses a central F distribution if the true model is of the form (2.1), but has a noncentral F distribution if the true model is of the form (2.2). In other words F ~ F n-p,N-n under H : E(Y) = X3-,^ , and und F ~ F' n-p,N-n;A er H : E(Y) = XB, + X232 ' where X 2 is the noncentrality parameter X = ^^(X2-XA) ' (X2-XA)B2/2a ' ^"^ ^ = (X'X)"^X'X2' Under H3, E(MS^Qp) = a^ + e^(X2 XA)'(X2 XA)B2/(i^-P) ^"^^ E(MSE j.g) = a^ (Draper and Smith, 1981, p. 120), hence Hq is rejected in favor of H^ if the value of F in (2.3) exceeds the upper 100a percentage point of the central F distribution, Fa;n-p,N-n* ^^^" ^0 ^^ rejected, we conclude that a significant lack of fit is present. Draper and Herzberg (1971) demonstrated that the lack of fit sum of squares can be partitioned into two statistically independent sums of squares, SSj^j and 53^2' when there are replicate observations at the center of the

PAGE 32

24 response surface design and when the basic design without center points is nonsingular. If the true model and the fitted model are of the same form as in equation (2.1) then the two F ratios F^ , = rss^,/(n p 1)1/MSE and LI 'LI J ' pure Fl2 = SSL2/^S^pure ^^^ both distributed as central F random variates, with respective numerator and denominator degrees of freedom (n p 1), (N n) for F^j^ and 1, (N n) for Fl2' ^f ^^^ true model is of the form shown in equation (2.2), then F^j^ and Fl2 ^^^ both distributed as noncentral F random variates. The expected values of SSlj^ and 83^2 ^^^ used to show what functions of e.2 ^^® testable with F^jl and fL2Two examples are presented by Draper and Herzberg to illustrate this testing for lack of fit. The first example makes use of a first order orthogonal design in k factors augmented with center point replicates for fitting a first order polynomial model . If the true model is of the second order, then Ft2 can be used to test a hypothesis concerning the parameters associated with the pure quadratic terms in the model. If all such parameters are zero, then Fj^-j^ provides a check on interaction terms. The second example illustrates the fitting of a second order polynomial model to a second order design with all odd design moments of order six or less zero. If the true model is third degree, then Fj^j^ can be used to test the significance of the third order terms, while Fl2 tests terms of order greater than three. The partitioning of SS^qf into SSlj^ and SSl2 ^^^ ^^^

PAGE 33

25 corresponding tests of hypotheses are also given in Myers (1971, p. 114-119), for the special case of fitting a first order polynomial model to a 2*3 factorial or a fraction of a 2*3 factorial design augmented with center point replicates and the true model is of the second degree. A more complete partitioning of the lack of fit sum of squares in an attempt to obtain a more detailed diagnosis of the lack of fit of the fitted model is given in a technical report written by Khuri and Cornell (1981). The lack of fit sum of squares, which has n p degrees of freedom, is partitioned into n p independent sums of squares, each having one degree of freedom. The expected values of these single degree-of-freedom sums of squares are used to identify at most n p linearly independent causes for the lack of fit variation. Tests of significance are performed on the assumed contributing causes. This method enables the screening of all subsets of 2.2 in order to identify those subsets which are most responsible for lack of fit of the fitted model. We shall now discuss the second general approach used in lack of fit testing, which is to test for lack of fit by making use of response values observed at points which are near neighbors in the factor space when true replicate observations are not available.

PAGE 34

26 2.3 Testing for Lack of Fit Without Replicated Observations — Near Neighbor Procedures Green (1971) suggests the following approach when testing for lack of fit if there are no design points at which replicate observations of the response are available. The N observed values of a response, Y, considered a function of only one variable, x, are divided into g groups, by grouping observations which have similar values of x. Green hypothesizes a model of the form Y= Ha + £., where X is an Nxl vector of observable responses, H is an Nxm matrix whose columns correspond to known functions of the variable, x, a. is an mxl vector of unknown regression coefficients, and e. is the Nxl vector of random errors, e ~ N^(0, a^Ijj). Green's method assumes that the vector of differences ( EY Hg.) can be well approximated by a dth order polynomial in X within each of the g groups, d > 1. An alternative model of the form Y = H V + n + £ is given, where S. is distributed as N^(Q, a^I^)/ Hj^ is an Nx[g(d + 1) + mj^] matrix of known constants, ii is a [g(d + 1) + m-j^]xl vector of regression coefficients, and u., as Green states is "a small vector." The first g(d + 1) columns of H^ correspond to the polynomial terms for the g groups (with (d + 1) terms for each group), the rightmost m < m columns in H-j^ correspond to terms that are in the

PAGE 35

27 fitted model, but are not represented among the g(d + 1) polynomial terms in the alternative model. Under the assumption that n = Q, the presence of lack of fit is tested by using the test statistic: Y' [H (H'H )"-'-H| H(H'H)"-'-H']Y/[g(d +1) + m^ m] F = ^^^ . Y'[I H^(HjH^)"-^H|]Y/[N g(d + 1) m^] (2.4) This statistic is of the same form as the F statistic used in the standard multiple regression test of a postulated model against a more general one which includes the postulated model as a special case. Lack of fit is suspected if the calculated F ratio in (2.4) is greater than Fcc;g(d+l)+mi-m, N-g(d+l)-mi "^^^^ ^his latter quantity is the upper 100a percentage point of the central F distribution. Green notes that when there is no lack of fit, the quadratic forms Y' [ H , (H 'H , ) ""'"H' H(H'H)~ H'JY and Y'[l H (H'H )"-'-H']y are distributed independently as a\^ with g(d +1) + m-j^ m and N g(d +1) mj^ degrees of freedom, respectively. In this case the F ratio in (2.4) possesses a central F distribution. If there is lack of fit on the other hand, then these two quadratic forms are distributed as noncentral chi-squares, multiplied by a'', with respective noncentrality parameters

PAGE 36

28 ?j_ = [Hj_v + n] •[Hj^(HJ_Hj^)"^H'^ H(H'H)""'"H' ][h^v + n] and C = ri_'[l H (H'H )~ H']ri . Thus the assumption that n = can affect the power of the test, since if n * , the expected value of MSE is greater than o^ , where MSE is the quadratic form in the denominator of the F ratio. Hence if r\ * , the probability of calculating a large F value is reduced, and we are less likely to detect lack of fit using an upper tailed rejection region. Daniel and Vtood (1971) suggest another method for lack of fit testing when replicated observations of the response are not available. They make use of "near replicates" to obtain an estimate of a, which is the standard deviation of the observable responses in the true model. The value of the estimate a is compared to the square root of the residual mean square from the analysis of the fitted model. Lack of fit is indicated if the square root of the residual mean square is large compared to the estimate a. To determine when observations are near replicates so that an estimate of a can be found, they define the squared distance between any two data points, j and j', to be measured by where Xj_j and x^ji are the values of the ith independent variable corresponding to the observations yj and Yj > ' respectively, i = 1, 2, ..., K, and b^ is the ordinary least

PAGE 37

29 squares estimate of the ith regression coefficient. In the denominator, s is the square root of the residual mean square for the fitted model. To obtain an estimate of a from near replicates, let Aj^d = |d4 d-j,|, n = 1, 2, ..., (2), where dj and dj 1 are the residuals at points j and j', respectively, and where there are N data observations in the experiment. Since the expected value of the range for pairs of independent observations from a normal distribution is 1.128a, a running average of the A^d's is calculated and their average is multiplied by .886 = (1/1.128) to get a running estimate, Sj^, of a. That is, s^ = .836 ^ A^d/n . The closest pair of observations as judged by D?-; 1 is used to begin the running estimate, the next closest pair (next "nearest neighbors") is used for A2d, and the procedure continues until s^ "stabilizes." The stabilized value of s^ is used to estimate a. A third method for testing for lack of fit without replication is given by Shillington (1979). The fitted model is of the form Y = X3 + e (2.5) where Y (Nxl), X (N^p), and §. (p^l) are defined as in (1.2) and e ~ '^m^-' °^^N^ * '^^^ ^^^^ ^°^ "''^^^ °^ ^"""^ °^ ^^® fitted model is a test for whether the true model has the form Y = X6 +6 + £ ,

PAGE 38

30 where 6 (Nxl) is a fixed effect quantifying the departure of (2.5) from the true model. Shillington assumes that the data can be grouped into g cells, with nj observations in the jth cell, determined in advance. Letting Cj refer to the jth cell, j =1, 2, ..., g, a vector of cell averages is written Yp (gxl), where the jth element of Y^ is the average of the observed responses in C-j . The matrix X^ of independent variables associated with Y^ is the gxp matrix where the elements in the jth row n . 3 are x' . = Z x'. ./n. , that is, row j of Xp is the row -O i^i -1] J vector x' . . The matrix Xp is assumed to be of full rank p < g. Also within each cell are defined the differences W. . = Y. . Y . , i e C. , j = 1, 2, ..., g, where Y . is ID ID -D _ D -D the jth element of Yp. The two independent data sets, Yp and {W•} with g and N g degrees of freedom, respectively, are used to find two independent estimates of a^. The first estimate is written as MSE^ = Z n.(Y . x'.g„)V(g " P) / where g_g is the weighted least squares estimate of g. using the regression of cell means, Y^^,, on X^. The second estimate of a uses the within cell deviations on cell means, {Wj^^}, and is g "d . , MSE,, = 2 Z (W.. W. )^/(N g r), ^^ j=l i=l ^^ ^^

PAGE 39

31 where r is the rank of an N^p matrix v/ith rows equal to x* _ x« , i e C, j = 1, 2, .., g. If the matrix of -ij -O D independent variables, corrected for cell means, is of full rank, then r = p. Here VJ^is the estimate of W^j from the regression of cell residuals i^W^^j} on the associated vectors of independent variates, x! . x'. . If the fitted model is the correct model, then MSEg and MSE^ are independent estimates of a^ and the ratio MSEg/MSE^ is an F statistic with g p and N g r degrees of freedom. When all observations in a cell have the same settings of the independent variables, that is, the observations are truly replicates for all cells, then this F statistic is identical to the F statistic in the usual lack of fit test in which the residual sum of squares is partitioned into lack of fit and pure error sums of squares, as given in Draper and Smith (1981, p. 120). If the true model is Y = XB + 6 + e , however, and if we let X'6 =0 and <^^ = 5'6/N, then 2 + ?1[I X^(X'X^)"^X']^„/(g-p) E(MSEg) = a+ 6^[I X^{X^X^) X^]^^/ n . 3 where 6 (g^l) has jth component equal to I ^^^/n^ • -B .^^ 13 3 Furthermore, with this latter true model form E(MSE^) = a2 + 6^j(l X^(X^ X^) "^X^) 6^/(N g r)

PAGE 40

32 where §_^ has the components 5^^S"-, i e c.;, j = 1, 2, ..., g. The matrix Xr., (Nxp) has the rows x! . x' . , w _ 1 -| _ , j ' i e C., j = 1, 2, ..., g. The power of the F test, F = MSEg/MSE^, depends on the relative bias of the estimates of a^, that is, the biases in MSEg and MSE^. Shillington states that the power of the F test which makes use of F = MSEg/MSE;^ is maximized by forming cells so that the bias of E(MSE^) is minimized. This is the same as forming cells so that the within cell variation in 6 is minimized. Shillington (1979, p. 141) also states, "Observations with near covariate (independent variable) values might be expected to have similar 6 values, since we assume that §_ varies in some continuous but unknown fashion with X. This justifies the usual procedure of forming groups by collapsing observations with adjacent covariate values. Indeed, if covariates do not vary within cells we have the usual lack of fit test and maximum power." By imposing a further structure on the form of §_, it is shown that if the F test has an upper tailed rejection region, the power is maximized by selecting the group sizes as n-; = 2, j = 1, 2, ..., g. Finally, Shillington suggests that in the presence of more than one independent variable problems in grouping may arise, and in this case it may be wise to perform a different lack of fit test for each parameter. Following this approach, an example is given which suggests testing lack of fit for each of the p independent variables separately may be more powerful than

PAGE 41

33 trying to form groups based on all independent variables at once. In summary, all the approaches we have discussed for testing for lack of fit when replicate observations of the response are not available at any of the settings of the independent variables make use of grouping the observed response values according to similar values of the independent variables. The observations falling in such groups are referred to as "pseudoreplicates" or "near neighbor observations." These pseudoreplicates are used to estimate the true variance of the observations, a^, but a completely unbiased estimate of o^ cannot be attained unless true replicate observations are available. In each case, the power of the lack of fit testing procedure is reduced because an unbiased estimate of o^ is not attainable. We now turn to the use of check points for lack of fit testing. 2.4 Testing for Lack of Fit with Check Points An alternative to the two approaches to lack of fit testing already discussed is the method which makes use of check points. We assume a model of the form E(Y) = XB, , as given in (2.1), is fitted in a response surface system, but that the true model is of the form E(Y) = Xg + X 3 as given in (2.2). The parameters, dif in the fitted model are estimated by ordinary least squares techniques, making use of the values of the response observed at the design points. After the model is fitted, values of the response are observed at additional points in the experimental region

PAGE 42

34 > called "check points." The observed response values at the check points are compared to the values which the fitted model predicts at these same check points. It is important to note that the observed values of the response at the check points are not used in fitting the model initially. Snee (1977) gives four methods of validating regression models, one of which is the collection of new data to check predictions from a previously fitted model. In a designed experiment these new data take the form of check points. Snee suggests that the inclusion of a small number of check points in any designed experiment is a "worthwhile" procedure. Scheffe (1958) proposed a test for lack of fit when the {3,2} simplex lattice design is used for fitting a second order canonical polynomial model in three mixture components. It is desired to use the observed value of the response at (1/3, 1/3, 1/3) as a check point blend. The test statistic proposed is the t statistic of the form t = — ^ ^ 1 — ,-7o— (2.6) [var(Y Y)]-"-/^ where Y is the observed value of the response at the check point, and Y is the value of the response predicted at the same point by the second order model which is fitted by ordinary least squares techniques to the observed response values at the six design points of the {3,2} simplex lattice. The response value observed at the point

PAGE 43

35 (1/3, 1/3, 1/3) is not used in fitting the model. Lack of fit is inferred if the absolute value of the calculated t value in equation (2.6) is larger than the corresponding tabled t value. In the denominator of the t test of equation (2.6), the variance of the difference Y Y is shown to be var(Y Y) = var(Y) + var(Y) = (44/27r)a^ , when r replicates are taken at each design point. The estimate of the variance of Y Y is (44/27r)a2, where a^ is calculated from the replicated response values at the design points . Scheffe (1958) also alludes to a test for lack of fit when several check points are used simultaneously. When there are k check points, the test for lack of fit is an F statistic of the form F = V(2.7) ko^ where d' = (Y^ Y^, Y^ X ^, ..., Yj^ Yj^ ) , and V = a ^Vq = var(d). Formulas are given for the elements of Vq in the special case when the check points are the design points of the {3,2} simplex lattice. Lack of fit is suspected if the calculated value of the F statistic given in (2.7) is larger than the corresponding tabled F value.

PAGE 44

36 Gorman and Hinman (1962) suggest the same t test in equation (2.6) that Scheffe (1958) suggested for a check point taken at (1/3, 1/3, 1/3) to test for lack of fit in a second order polynomial model fitted from a {3,2} simplex lattice design. They suggest using (1/3, 1/3, 1/3) as the location of the check point because the observation at this point may later be used to fit the next more complex model, the special cubic, if the second order model is found to be inadequate. They state that in general for the second order polynomial model as well as higher order models, check points should be taken in regions of particular interest, of which there are usually many in any blending study. Further, they suggest that the number of check points depends on individual experimental situations — technical background, precision required, cost of materials and analyses, and probability of requiring a more complex model. However, no specific criterion is given by Gorman and Hinman for selecting the location of the check points. Gorman and Hinman (1962) indicate that a t test at a check point other than at (1/3, 1/3, 1/3) takes the same form as the statistic of equation (2.6), t = Y ^ [var(Y) + var(Y)]-'-/^ with the additional condition that if several check points are taken, say for example k points, the method of checking

PAGE 45

37 the fit is to compute the t value at each location and refer these calculated t values to the 100(a/2k) percentage point of the central t distribution rather than the 100 (a/2) percentage point. Kurotori (1966) gives an example of a mixture experiment where the response is the modulus of elasticity of a rocket fuel, which is a mixture of three components, binder (X]^), oxidizer {^2)1 and fuel (X3). The factor space of feasible mixtures is a subspace inside the twodimensional simplex or triangle where all three components are present simultaneously. "Pseudocomponents" are defined and in the pseudocomponent system a special cubic model is fitted to data collected at the points of the q = 3 simplex centroid design (Figure 4). A check for adequacy of fit is made by using three check points and the response values at the check points are used only for testing the fit of the model and not for fitting the model initially. The reason for the choice of the particular check point locations by Kurotori is that, as he states, "They are the most remote mixtures from the seven design points." The lack of fit test is an F statistic of the form 2 F = -ly(2.8) a 2 3 9 where s = z (Y. Y.)^ , for the i = 1, 2, 3 check points .2 . i=l and a is an estimate of measurement error from a previous analysis. Kurotori admits that the use of the F statistic

PAGE 46

38 (1,0,0) # Design Points O Check Points (i'a-'")/ r {5.0,1) (0,1,0) « 1^^_^ 1 "'" ^ (0,0,1) H-i) « =1 [
PAGE 47

39 In summary, only Scheffe refers to an exact F test when several check points are considered simultaneously for testing for possible lack of fit of a model fitted in a mixture space, and his development is limited to the special case where the check points are the design points used to fit the model initially. No criterion is proposed by Scheffe for selecting other locations for the check points.

PAGE 48

CHAPTER THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING LACK OF FIT IN A MIXTURE MODEL 3.1 Introduction In Chapter Three we investigate the problem of testing for lack of fit of a linear model fitted in a mixture space. The testing is to be accomplished with the use of check points. We assume that an experimental design is specified, and that the fitted model is of the form E(Y) = X3j^ (3.1) where Y is an Nxl vector of observable response values, X is an Nxp matrix of known constants and rank p, and 3 is a vector of p unknown regression coefficients. The true model is assumed to be of the form E(Y) = XBj^ + X^^^ (3.2) where X2 is an Nxp2 matrix of known constants and ^2 ^^ ^ vector of po unknown regression coefficients. Throughout our development, we will assume that the random vector Y has the normal distribution with variance-covariance matrix equal to o I^. 40

PAGE 49

41 In our investigation we wish to determine the proper testing procedure to follow in deciding whether the fitted model exhibits lack of fit. In order to optimize the lack of fit testing procedure, we will determine the location of the check points so that the power of the test is maximized. 3.2 Testing for Lack of Fit in the Presence of an~ External Estimate of Experimental Error Variation 3.2.1 The Test Statistic We wish to test the performance or fit of a fitted model in a mixture space when the true model possibly contains terms in addition to those in the fitted model. The fit of the model is to be tested by a test which makes use of the response values observed at certain locations called "check points" in the experimental region, by comparing them to the values which the fitted model predicts at the same check points. The observed values at the check points are not used for estimating the coefficients in the fitted model and are assumed to represent the values of the true surface at the check points. Let us define the vector of differences d = (Y* Y*) (Y* Y*, Y* Y*, ..., Y* Y*)' where Y*, i = 1, 2, ..., k are observed response values at k check points and Y*, i = 1, 2, ..., k are response values predicted at the k check points by the fitted model.

PAGE 50

42 Y^ = x^'b,, where b, is the ordinary least squares estimator of 3,/ and where x*' is the ith row of X*, the kxp matrix whose columns are of the same form as the columns of X but with its rows evaluated at the k check points. Note that if 3 = 0, then E(d) =0 and if ^ * 0, then E(d) = (X* X*(X'X)~-''X'X2)62Let V represent the variance-covariance matrix of the random vector d. 2 Then V = a V» where Vq = Ij^ + X*(X'X) 'X*' and where Ij^ is the identity matrix of order kxk. We assume that an unbiased estimate of a^ is available and we denote this estimate by a ^ , where the subscript ext ext " 2 stands for external, and o . is independent of the model being fitted. The test statistic for the hypothesis of zero lack of fit H : E(d) = is d'V'-'-d/k F = :rf-I (3.3) ''ext (see Scheffe, 1958, p. 358). It will be shown later in this section that the F ratio in Eq. (3.3) possesses either a central F distribution or a noncentral F distribution, depending upon whether the true model is represented by Eq. (3.1) or Eq. (3.2). "2 . . The variance estimate a ^ that appears m equation ext (3.3) is ordinarily generated from replicated observations

PAGE 51

43 at some of the design points in the experiment. We assume * 2 that a ^ IS a constant multiple of a central chi-square ext random variable with v degrees of freedom. This is written as a^ ^ = SSE /v ext pure^ = (aVv)(SSEp^^^/a^ 2 2 where ^^E ^^^/a ~ x^* Note that SSEpy^g denotes the portion of the residual sum of squares due to replication variation from the fitted model. The residual sum of squares from the fitted model may be partitioned into SSEp^j-g and SSj^Qp only if replicated observations are collected at one or more design points. For the case where replicate observations are collected at all of the design points n n . _ SSE = Z E''' ( Y. . Y. ) , PUi^e i=i j=i ^3 1where n is the number of distinct design points, n. > 2 is the number of replicates at the ith design point, Yj^^ is the jth observation at the ith design point, and Y. is the average of the n^ observations at the ith design point. n Here SSE j^^ has v = Z (n. 1) degrees of freedom. i=l ^ When the fitted model and the true model are of the same form as defined by Eq. (3.1), the quantity d'V~''"d/a^

PAGE 52

44 possesses a central chi-square distribution (Searle, 1971, p. 57, Theorem 2). However, when the true model is of the form specified by Eq. (3.2), d'V~''"d/a^ possesses a noncentral chi-square distribution. Thus when the true model is of the form in Eq. (3.1), d'V-Va' ~ X^ but when the true model is of the form in Eq. (3.2), d'V-Vc^2 ^ ^.2^^^ where in the second case the noncentrality parameter X. has the form Xj_ = E(d)'VQ^E(d)/2a^ = 3^(X* X*A)'Vq^(X* X*A)^^/2a^. The matrix A = (X'X) X'X„ is called the alias matrix and is of order pxp . In X-,, the matrix X* is of order kxp2 a has the same relationship to X2 as X* has to X. 2 Since SSE /a is statistically independent of pure"^ -12 . . d'V_ d/a , then under model (3.1) the test statistic nd

PAGE 53

45 d'V """d/ka^ F = SSE /va^ pure^ ''ext will have a central F distribution. When the true model contains terms in addition to those in the fitted model then F will have a noncentral F distribution. We write these two cases as F ~ F k,v under model (3.1), and K,v ;Xi under model (3.2), where the noncentral ity parameter is Xj_ = S^(X* X*A)'Vq-'-(X* X*A)l^/2a . 3.2.2 The Testing Procedure and an Expression for the Power of the Test Given that the form of the fitted model is defined as Eq. (3.1), the expected value of the numerator of the F statistic in Eq. (3.3) will depend on the form of the true model. For the case where the true model is expressed as

PAGE 54

46 Eq . (3.2), E(numerator) = E (d ' V~"'"d/k ) (aVk)Ex-^,^ (a^/k)(k + 2X^) 2 2 = o^ + 2a^Xj^/k = a^ + e^A^B^A, (3.4) where A^ = (X* X*A) 'V~ (X* X*A). However, when the true model is Eq. (3.1), 3 = and in this case X = so that 2 * 2 E (numerator) = a . Also a . is an unbiased estimator of ext a 2 and E(a2^^) = a\ (3.5) Therefore the ratio E(numerator )/E(denominator ) where " 2 the denominator is a , will equal unity under model (3.1), that is, when there is no lack of fit. Under model (3.2), the ratio will be greater than or equal to unity so lack of fit should be suspected if the calculated F ratio in equation (3.3) is large. We can thus use an upper tailed rejection region to reject the hypothesis of zero lack of fit. The power of the test is

PAGE 55

47 ^(^i,v;X. > ^a;k.vl where F is the upper 100a percentage point of the ex ; K / V central F distribution with k numerator degrees of freedom and V denominator degrees of freedom. It is worth noting that from Eq. (3.4) and Eq. (3.5) testing the hypothesis that S^ = is equivalent to testing the hypothesis that X-^ = 0, assuming A, is positive definite. Thus testing a null hypothesis of zero lack of fit using the proposed testing procedure involving the F ratio in (3.3) may be expressed as a test of the hypotheses Hq: X^ = H : A, > 0. a X 3.2.3 A Method for Locating Optimal Check Points Once a design for fitting model (3.1) in a mixture space is chosen and the number of simultaneous check points is decided on, say k > 1, the next step is to determine where in the mixture space we should place the k check points so as to maximize the power of the test for lack of fit. The location of the check points is to be made independently of the value of 8 .

PAGE 56

48 The power of the upper tailed F test for lack of fit is an increasing function of X , (see Appendix 1 for proof, with X = 0). Therefore, to maximize the power of the test we maximize the value of X defined as Xj_ = &^A^Q^/2a where A^^ = (X* X*A) 'V~ (X* X*A), by properly selecting the k check points whose coordinates are defined in X*. To maximize the value of Xj^, we shall concentrate on the matrix A^. The matrix A-^ is a square matrix of order P^^P^ and is a scalar quantity when P2 = 1. By maximizing the scalar quantity Aj^ with respect to the k check points, the power is maximized no matter what the value of 8 . Maximizing the scalar Aj^ can be accomplished by using The Controlled Random Search Procedure given by Price (1977). This procedure is described in Appendix 2. As a computational aid, Aj^ can be expressed as V + (X* X*A) (X* X*A) ' I when P2 = 1» where the symbol |B| denotes the determinant of the square matrix B. Thus the computations reduce to evaluating two determinants rather than inverting Vq (see Scheffe, 1959, Appendix V, p. 417).

PAGE 57

49 When p> 1 and A-, is no longer a scalar, maximizing X -j^ (and thus maximizing the power of the test) cannot be accomplished without specifying B . In this case we make use of a lower bound for X^ (Graybill, 1969, p. 330, Theorem 12.2.14(9) ) defined as 2 min-2-2^ 1 (where u . is the smallest eigenvalue of A-,) to be used in place of Xi. Hence an approximate solution to the maximization of X-^ will be achieved by finding the k simultaneous check points (using Price's procedure) that maximize u • , the smallest eigenvalue of A-i . In other mm -^ words when p > 1, and in order to avoid specifying ^ , we 2 ^ seek to maximize a lower bound value for X^^. This maximization does not depend on the value of Q^. There are cases where the matrix Aj^ ^^ °^ less than full rank (less than rank P2) or equivalently where the matrix Ai is positive semi-definite so that u^^^ will be equal to zero no matter which check points are selected. One such case occurs when k < p (when the number of check points is less than the number of parameters in the true model which are not in the fitted model) since when k < p^ rank(Aj^) = rank[VQ Ax* X*A)] = rank(X* X*A) ,

PAGE 58

50 and so rank(A-]^) < min(k, P2) because the matrix (X* X*A) is of order kxp2. Therefore when k < p , the rank of A, is at most k so that A, is of less than full rank. Since u . Jmin must be equal to zero when A-, is positive semi-definite, an alternative method to that of maximizing y to select ^ mm optimal check points must be found when A, is positive semidefinite in order to produce a positive lower bound for Xi. In this pursuit, let us write X-^ as ^1 = i2^ig.2/2a^ 6_^PAP'3 2/2a^ Q^^lP^zP^] diag[Aj^, K2 = 0] [P ^iP^] 'Q_^/2a e^PlAlPU2/2c>^ where A is a diagonal matrix with elements equal to the eigenvalues of An, P is an orthogonal matrix whose columns are orthonormal eigenvectors of h-^, Aj^ and P^^ correspond to the positive eigenvalues of Aj^, while A2 = ^rid P2 correspond to zero eigenvalues of A^. Then by Theorem 12.2.14(9) in Graybill (1969) we can write y"^. z'z/2a^ < X, (3.7) mm-' 1 whe re y . is the smallest positive eigenvalue of Ai , and ^min iT 3 J.

PAGE 59

51 z = f{^2* Thus by Eq . (3.7), an approach to maximizing a positive lower bound for X -^ when Aj^ is positive semidefinite is to select check points that maximize the smallest positive eigenvalue of A-^. It must be noted, however, that this method can only be used when 02 E n C(P^), where C(Pj^) denotes the column space of Pj^ and n C(P-|^) denotes the intersection of all such spaces which can be obtained at all possible check points locations. This is because, in general, z'z in (3.7) depends on the location of the check points through its dependency on Pj_. if, however, 0^ ^ nC(P ), then z'z = ep^P]^02 " §-2^P'§.2 " ^2-2' ^^"^^ ^2^2 " °* It follows that when ^^ e n C(P,), u^z'z/2o^ = *^inin-2-2^^° ^"^ only u^^^ depends on the location of the check points. 3.3 Testing for Lack of Fit When MSE Is Used to Estimate Experimental Error Variation 3.3.1 The Test Statistic In this section we shall show that when an external estimate of o^ is not available and the residual mean square (MSE) from the fitted model of the form (3.1) must be used as an estimate of a^, the test statistic d'V^-'-d/k ^ = -^llSE^ (3-8) possesses a central F distribution when the true model is

PAGE 60

52 Eq. (3.1), but possesses a doubly noncentral F distribution when the true model is Eq. (3.2). In the initial section of this chapter, the quantity -1 2 d'Vd/a was said to possess a central chi-square distribution or to possess a noncentral chi-square distribution, depending on whether the true model was specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum of squares from the fitted model is defined as '^ 2 SSE = E (Y. Y. ) i=l ^ ^ = Y' (Ij^ X(X'X) '•X' )Y and it is easy to show (Searle, 1971, p. 57, Theorem 2) that SSE/a^ possesses a central chi-square distribution if the true model is Eq. (3.1), but under model (3.2), SSE/a^ possesses a noncentral chi-square distribution. This is expressed as SSE/a^ ~ Xn_p under model (3.1), and SSE/a^ ~ x^fp^x.

PAGE 61

53 under model (3.2), where the noncentrality parameter X 2 is ^2 " ^2^^2 " ^^)'(^2 " ^A)§.2/2a^ The distributional form of the test statistic in Eq. (3.8) is derived by knowing that the quantities -12 2 d'Vp, d/a and SSE/a are statistically independent (see Appendix 3), so that F = d'VQ^d/ka^ MSE/a^ d'V~''"d/k MSE is distributed as a central F when the true model is Eq. (3.1), but when the true model is Eq. (3.2) the F ratio is a doubly noncentral F, that is, under model (3.1), F ~ F k,N-p and under model (3.2), k,N-p;Xi ,A2 * 3.3.2 The Rejection Region and its Relation to the Power of the Test In Appendix 1 it is shown that if k, N-p, and X2 are fixed, then the power of the F test using the ratio (3.8) is

PAGE 62

54 a function of the location of the rejection region (upper tailed or lower tailed) of the test. The power increases with increasing values of the numerator noncentrality parameter, Xi, when the test is an upper tailed test. The power decreases with increasing values ot X ^ when the test is a lower tailed test. This means that to study ways of increasing the power of the test, we have to determine whether the test is an upper tailed test or a lower tailed test. Similarly, for fixed values of k, N-p, and X-^, the power of the F test is a decreasing function of X2 ^^^ ^" upper tailed test, and is an increasing function of X 2 "hen the F test is a lower tailed test (Scheffe, 1959, p. 136137) . To decide if the test is an upper tailed test or a lower tailed test, we recall from Section 3.2.2 that if the true model is Eq. (3.1) then the expected value of the numerator of the F statistic in (3.8) can be written as E( numerator) = 0^, and if the true model is Eq. (3.2), 2 2 E( numerator) = a + 2a X ^/'k. (3.9) = 0^ + 3^A^B2A

PAGE 63

55 where the P2XP2 matrix A^ is A^^ = (X* X*A)'Vq (X* X*A) Similarly, it can be shown that if the true model is Eq. (3.1), the expected value of the denominator of the F statistic in (3.8), where the denominator equals MSE, is E(denominator) = E(MSE) = a2, but if the true model is Eq.(3.2), E (denominator) = E(MSE) [c^'/(N P)]Ex'fp,,^ [a^/(N p)][N p + 2X2] a^ + 2a^\^/{n P) (3.10) 0^ + e2A2e2/(^ " P) where the P2' 3 ' A e /(N p) . In this latter case we reject the null

PAGE 64

56 hypothesis of zero lack of fit if the calculated value of the F ratio in (3.8) is large. An upper tailed rejection region seems reasonable for this test. When the true model is Eq. (3.2), and if g^Aj^g^A < l^P^2^2'^''^ ~ P^' ^^en a lower tailed rejection region is preferred. 3.3.3. A Method for Locating Optimal Check Points Given a design for fitting a model of the form in Eq. (3.1) in a mixture space (note that fixing the design fixes ^2 and (N p)), and given the number of simultaneous check points desired, k > 1, we now wish to determine where in the mixture space the k check points should be located so as to maximize the power of the F test for lack of fit, where the test statistic is given in Eq. (3.8). We also wish to position the optimal check points in a manner that is independent of the values of the elements in g . The case of an upper tailed test. To help us find k simultaneous check points that maximize the power of an upper tailed test, we shall make use of the fact that the power is an increasing function of X-,. Therefore to maximize the power of the upper tailed F test, we shall seek the locations of the k check points that maximize X,. As in the case considered in Section 3.2.3, where the test statistic had a noncentral F distribution, if the number of extra terms in the true model is P2 = 1, then maximizing X -j^ is equivalent to maximizing the scalar A,. However, as before, if p > l, then the P^^Po "^^^rix A-^ is not a scalar and we will have to approximate the

PAGE 65

57 maximization of X -^ by maximizing a lower bound for X-^. This is done by finding the maximum value of u . , the smallest eigenvalue of A-]^, since 2 ^min-2-2' 1 When the number of check points is less than the order of the square matrix A-^, that is, k < P2f then rank(A]^) < min(k, po), and A-, will have y . = 0. For this case, we again try to maximize the smallest positive eigenvalue of h-^ which we denote by u'*'. , while remembering from Section min 3.2.3 that this technique is limited to situations where B^ e nC(P^) . The case of a lower tailed test. To find k check points to maximize the power of a lower tailed test, we make use of the fact that the power of the lower tailed F test increases as X j^ decreases. Then if P2 = 1 and A-|^ is a scalar quantity, X -^ can be minimized with respect to the k check points by finding the check points that minimize A-j^. If p > 1, then by Theorem 12.2.14(9) in Graybill (1969), we see that an upper bound for Xj^ is ^1 < ^max^2^2/2<^'' ^''^^^ where u is the largest eigenvalue of A-i . An approximate ^max ^3 ^ i solution to minimizing X j^ in (3.11) can be achieved by minimizing u . It is not necessary to treat the case ^ max

PAGE 66

58 of k < p separately here, although X ^ will equal zero if g_2 is in the column space of P2 , where P2 is the matrix whose columns are orthonormalized eigenvectors corresponding to the zero eigenvalues of the matrix A, . 3.3.4 Determining Whether the Test Is Upper Tailed or Lower Tailed The procedures outlined in Section 3.3.3 produce a set of k check points that simultaneously maximize the power of the upper tailed test as well as a second set of k check points that simultaneously maximize the power of the lower tailed test. The check points that are selected maximize the power, given A2f k, and N p without specification of g_2' except that when Aj^ is positive semi-definite we require that e n C(P ). It is now necessary to decide which of our two candidates will be used for a lack of fit test. To choose between the upper tailed test and the lower tailed test, let us consider the quantity R = [A^/k] [A^/(N p)] . If R is positive definite when the true model is Eq. (3.2), then no matter what the value of is, the ratio E (numerator )/E (denominator) will be greater than unity, implying an upper tailed test is to be used. Similarly, if R is negative definite, then a lower tailed test should be used. Finally, if R is not definite, then neither an upper nor a lower tailed test is implicated and further

PAGE 67

59 investigation is necessary. The criterion of R = [A /k] [a /(N p)] may yield any of the four following cases. Case 1. If R = [Aj^/k] [a^/CN p)] is positive definite when A-^ is generated by the k optimal upper tailed test check points, and R is not negative definite when A-, is generated by the k optimal lower tailed test check points, then we recommend that the check points be used that yield the optimal upper tailed test with an upper tailed rejection region. For Case 1 it is necessary for A, to be positive definite (see Appendix 4). Since A-, is a square matrix of order P2^P2 with rank(A ) < min(k, P2)' then A^ can be positive definite only if k > p . Thus, there must be at least P2 check points for Case 1 to hold, where P2 is the number of terms in the model of Eq. (3.2) that are not in the model of Eq. (3.1). From inspection of equations (3.9) and (3.10), it is apparent that the testing for lack of fit in Case 1 is equivalent to testing the hypothesis ^1 ^2 ^0'' — FN-^ = (3.12) against the alternative «a= -X--N-H >

PAGE 68

60 since R = [ A^^/k] [A2/(N-p)] is positive definite when the true model is Eq. (3.2). In i^pendix 5(a) it is shown that under Case 1, the hypothesis given by (3.12) is equivalent to the hypothesis Hq: X^ = X2 = 0. Case 2 . In Case 2 we assume that R = [A /k] [a /(N p)] is not positive definite for the k optimal upper tailed test check points, but that R is negative definite for the k optimal lower tailed test check points. Here we recommend that the lower tailed test check points be used with a lower tailed rejection region. It is necessary for A2 to be positive definite for Case 2 to occur (see Appendix 4). However, k-^ need not be positive definite, and so k need not be greater than P2. In Case 2 then, it is possible that lack of fit may be tested with only one check point. By inspection of equations (3.9) and (3.10), a hypothesis of no lack of fit is equivalent to X , X 2 while the alternative hypothesis that lack of fit is present is equivalent to X X a k N p

PAGE 69

61 since R = [ A^^/k] [a^/CN p)] is negative definite. In Appendix 5(b) it is shown that the hypothesis given by (3.13) is equivalent to the hypothesis Hq: A^ = X^ = 0. Case 3. We assume R is positive definite for the k optimal upper tailed test check points, and R is negative definite for the k optimal lower tailed test check points. Hence either an upper or lower tailed test may be considered as a possible test for lack of fit. If the quantity 2 -2-2^" can be specified, then the minimum power for both the optimal upper and optimal lower tailed tests can be approximated, and the test with the greater minimum power is recommended. In Appendix 4 it is shown that Case 3 can occur only when A-j^ is positive definite for the upper tailed test. Thus Case 3 can only occur when there are at least pn check points. The minimum power of the upper tailed test may be found by calculating "^ (^J,N-p,X^^,X,„ > ^«;k,N-pl' '^•"' where F central F distribution. a-k N-p """^ ^^^ upper 100a percentage point of the

PAGE 70

62 ^IL = ^min^2&2/2^^' and ^2U = ^ax^2^2/2<^^' where \i . is the smallest eigenvalue of An and 5 is the mm ^ 1 max largest eigenvalue of A2. Formula (3.14) yields a conservative lower bound for the power of the optimal upper tailed test. Note that Aj^ is generated using the optimal upper tailed test check points. The cumulative distribution function of F" can be approximated by multiplying the cumulative probabilities of the central F distribution by a constant (Johnson and Kotz, 1970, p. 197). This approximation is described in Appendix 6. Other approximations for F" (such as the Edgeworth series approximation suggested by Mudholkar, Chaubey, and Lin, 1976) exist which are generally more accurate, but we chose to use the approximation given in Johnson and Kotz (1970, p. 197) due to its simplicity. Additionally, the approximation of Mudholkar, Chaubey, and Lin (1976) produced negative probabilties when only one degree of freedom was available in either the numerator or denominator of F" . This problem was avoided by using the approximation given by Johnson and Kotz (1970). The minimum power of the optimal lower tailed test can 2 be approximated similarly (if SAio/'' ^^ specified) by

PAGE 71

63 calculating P I F" < F 1 ^ k,N-p;X^y,X2L (l-ct);k,N-p^ where ^lU = ^max^2^2/2o^ and ^2L = Vin^2&2/2a^ with u„a„ equal to the largest eigenvalue of Ai and 6 . max 3 :3 j_ j^j^j^ equal to the smallest eigenvalue of A2. Note that A-j^ is generated by using the optimal lower tailed test check points. For the lower tailed test, A-j^ may be positive semidefinite, and if 3 is in the column space of P2 then X-, = 0. In Case 3, the upper tailed test is a test of Hq: Xi = A^ = X X while the lower tailed test is a test of

PAGE 72

64 %'' ^=^2 = ' X X Case 4 . In Case 4 we assume that R = [ A /k] [k^/(U p)] is not positive definite for the k optimal upper tailed test check points and R is not negative definite for the k optimal lower tailed test check points. Here it is useful to write the difference between the expected value of the numerator and the expected value of the denominator of the F ratio in (3.8) as s^[Aj^/k A^/CN p)]32 = e^sns'e^ = 3'S^.^S'3 2 + 3^33.33.32 where ^ = diag(n-,, ^2' ^3) is a diagonal matrix consisting of the eigenvalues of R, J^j^ is a diagonal matrix of the positive eigenvalues of R, ^2 is a diagonal matrix of the zero eigenvalues of R, and ^^3 is a diagonal matrix of the negative eigenvalues of R. The orthogonal matrix S can be expressed as S = [Sj^ :S2:S3] , where the matrices S^^, S2f and S3 have columns which are orthonormalized eigenvectors corresponding to Q-^, Q.2, and .3, respectively.

PAGE 73

65 In Case 4, neither the optimal upper tailed test nor the optimal lower tailed test is applicable for all values of e . For completeness, we note that Case 4 actually consists of nine subcases, where R may be positive semidefinite, negative semi-definite, or indefinite for either the optimal upper tailed test or lower tailed test check points. These subcases are listed in Table 2. Table 2. Nine Subcases of Case 4.

PAGE 74

66 e^[A^/k A^/(N p)]g2 = e22i"iSiB2 + §.2^3^3231= §.2Sii2j^S|e2 + §.2^i"iSie2f and 3 2[ A^/k A2/(N P)]e2 ^^^^ '^^ greater than zero, indicating an upper tailed test. Similarly, a sufficient condition for the test for lack of fit to be lower tailed is that 3 be in the column space of [S :S ], but not entirely in the column space of S2. Then 3^[A^/k A^/CN p)]32 = + 3^32^33^3.2 3^33^33^3.2 which makes g.^[ A^^/k A2/(N p)]g.2 less than zero, indicating a lower tailed test. To determine whether 3-, is in the column space of [3 :3 ], let us define the augmented matrix ^1 ~ [§.9*^i*3t] • If ^1^1 ^^^ ^ zero eigenvalue, then 3„ is in the column space of [3 :S ]. Similarly, if we define ^2 ~ [^2*^2-1 ^"*^ ^1, ~ [§.9 '^9 •^^] ' then 3 is in the column space of S-p if Q'Q„ has a zero eigenvalue, and 3^ is in the column space of [ 3 :3 ] if QAQ, has a zero eigenvalue.

PAGE 75

67 Given that we are in a particular subcase of the nine subcases described in Table 2, we recommend that lack of fit be tested with the upper tailed test check points if it is determined that is such that e^[A^/k A^/CN -P)]e2 ^^ positive when A-, is generated from the upper tailed test check points. Likewise, for the same given subcase, if the value of 6__ of interest is determined to produce a negative value for e.A[A,/k A^/(N P)]32 ^'hen A-^ is generated from the lower tailed test check points, then we recommend that lack of fit be tested with the lower tailed test. We see then that Case 4 is an undesirable situation in practice, since, in order to test for lack of fit, we must assume a priori that any lack of fit is due to a nonzero value of B „ that produces an upper tailed or lower tailed rejection region. However, it would seem rare that such knowledge would be available. 3.4 Examples We now present several examples to illustrate the technique for locating optimal check points to be used in testing for lack of fit in a mixture model. 3.4.1 Theoretical Examples Example 1. In this example a second order canonical polynomial model is fitted in three mixture components using the {3,2} simplex lattice design, which is presented in Figure 1 of Chapter 1. The true model is assumed to be the special cubic model containing the term 3 x x x in addition to the six terms of the fitted model. The expected

PAGE 76

68 values of the response at the six design points are assumed to be represented by the fitted model in the form E(Y) = X6^, but with the true model the expectations are written as E(Y) = X3j^ + y^^t^, where X is a 6x6 matrix with rows that define the coordinates of the six design points and columns that correspond to the six terms in the fitted model (xj^, X2f X3, x,x-, ^I'^T' x„x^), B-, is the 6x1 vector of regression coefficients (g^, ^, g^, g^^' ^ j^^ ' ^23^' ^2 ^^ ^ ^"^^ column vector containing the values of the term '^\'^2^1 ^^ the design points, and 3 is the single regression coefficient 3i^o' The {3,2} simplex lattice design consists of only six design points, and since six parameters are estimated in the second order fitted model, there are no degrees of freedom remaining for obtaining an estimate of the experimental error, a^. We assume therefore that an external estimate of o'' is available, a ^t which will be used in the denominator ext of the lack of fit F statistic given in Eq. (3.3). Since there is one term in the true model in addition to those in the fitted model, that is P2 = 1, we know that in order to locate k simultaneous check points that maximize

PAGE 77

69 the power of the test for lack of fit it is necessary to maximize the scalar quantity A^ = (X* X*A) •Vq-'-(X* X*A) with respect to the coordinates of the k check points. Here X* is a k-element column vector with ith element equal to the value of x* x* x* at the ith check point, X* is a kx 6 matrix with ith row equal to the value of (x* , x* , x* , il i2 i3 X* x* , x* x* , x* X* ) at the ith check point, A = (X'X)"lx'X2 is the 6x1 alias vector, and V = I + X*(X'X)~ X*'. This maximization is accomplished by use of the Controlled Random Search Procedure (Price, 1977), which is described in Appendix 2. When only a single (k = 1) optimal check point is desired the Controlled Random Search Procedure locates a point (x*, X*) which maximizes A^ = (X* X*A)'Vq-'-(X* -X*A), where X* = xjx*x* = x*x*(l X* X*),

PAGE 78

70 X* = (Xj, X*, X*, x*x*, x*x*, x*x*) = (X*, X*, (1 xj X*), xjx*, xj(l xj X*), X*(l Xj X*)), and Vq = 1 + X*(X'X)~ X*'. The value of A^^ is calculated using the formula of Eq. (3.6). Following this procedure, we find that the single check point that maximizes A-j^, and thus maximizes the power of the test, is the centroid of the triangular factor space (1/3, 1/3, 1/3). The value of A-^ at this centroid point is A-j^ = 0.00084. When the Controlled Random Search Procedure is used to locate k = 2 simultaneous check points that maximize A-j^, the centroid (1/3, 1/3, 1/3) is selected twice, and A-j^ = 0.00121. For three simultaneous optimal check points, the centroid is selected three times, and A, = 0.00142. To test whether the second order model exhibits lack of fit, when we suspect the special cubic model is the true model, we form the F ratio d'V^-'-d/k F = — " '2 ''ext with the single check point (1/3, 1/3, 1/3) where d = Y* Y*, Y* is the observed response, Y* is the response

PAGE 79

71 predicted by the second order fitted model at (1/3, 1/3, 1/3), and Vq = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(X'X)"1 (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of the F ratio exceeds F , where v equals the number of a ; 1 , V * 9 degrees of freedom associated with a then we reiect the ext -" null hypothesis that the second order model is the true model in favor of the alternative hypothesis that the special cubic model is the true model. Equivalently, we reject H^: X j^ = in favor of H : A, > 0. For k = 2 or k = 3 check points, the value of the F ratio is calculated using the observed and predicted responses at the two or three replicates at the centroid. The hypothesis H : X, = is rejected in favor of H : X, > if F U 1 a 1 exceeds F , a ;k,v Example 2. In Example 2 we illustrate the second of the four cases that could arise when MSE is used as an estimate of a^ in the lack of fit test statistic (see Section 3.3.4). We again fit a second order canonical polynomial model in three mixture components, and assume the true model is special cubic. The design to be used is the q = 3 simplex centroid design, which consists of seven design points, and is illustrated in Figure 2 of Chapter 1. There are six parameters to be estimated and seven design points hence one degree of freedom can be used to calculate MSE. We shall use MSE to estimate a 2. Optimal upper and lower tailed test check points must fc>e located, and then a decision is made as to which test should

PAGE 80

72 be used. The actual testing for lack of fit involves the F statistic in (3.8). As in Example 1, P2 = 1, since there is one term in the true model in addition to those in the fitted model. Thus Aj^ is a scalar whose value we seek to optimize with respect to the desired number of check points, k. When only a single check point is sought for the purpose of testing lack of fit, the Controlled Random Search Procedure has two functions. First, the procedure is used to locate the optimal candidate check point for an upper tailed test by locating the check point that maximizes the scalar A-.. Secondly, the procedure is used to locate the optimal candidate check point for a lower tailed test, which is accomplished by locating the point that minimizes A-,. The quantity R = [A-,/k] [A2/(N p)] is then calculated to determine whether the upper or lower tailed test will be used. If R is positive for the candidate check point for an upper tailed test, then the test is upper tailed, and the test is lower tailed if the candidate check point for a lower tailed test produces a negative value for R. Note that A2 = (X2 XA)'(X2 XA) is fixed once the design is specified, since A2 does not depend on the check points. Using the Controlled Random Search Procedure it is found that the maximum value of A^^ occurs at (xj, x^, x^) = (1/3, 1/3, 1/3), which will be the location for the check point for the upper tailed test. Calculating Aj^ at this centroid

PAGE 81

73 point, we find that R = [A^/k] [A2/(N p)] = [(3.7258 X lO""*)/!] [(8.4175 X 10"'^)/1] = -4.6917 x 10"^. Since R is negative, the test is not upper tailed. Using the Controlled Random Search Procedure to minimize Aj^, we find that a subregion of the factor space exists in which all points yield a near minimum value for A-]^. We choose the point (0.0189, 0.9269, 0.0542) at random from this subregion to be used as the optimal candidate for a lower tailed test. Here R = [(8.4175 x 10~'^)/1] = -8.4175 X lO"'^. Since R is negative for both the optimal upper tailed test check point and for the optimal lower tailed test check point, we have Case 2 of Section 3.3.4. The upper tailed test check point is disregarded, and the lower tailed test check point (0.0189, 0.9269, 0.0542) is used to test for lack of fit. If the calculated F ratio. MSE is less than F,, . ^ , then H„ : X , = X ^ = is rejected in ( 1 -a ) ; 1 , 1 1 2 favor of H : [x /l] [x /l] < 0, that is we conclude that a J. 2, the second order model exhibits lack of fit, and the true model is special cubic. When two simultaneous check points are desired for testing lack of fit, we can again use the Controlled Random Search Procedure to locate the optimal settings. To maximize the scalar An, we find that both check points

PAGE 82

74 should be selected at (1/3, 1/3, 1/3), for an upper tailed test. With our calculations R = [(5.8275 x 10"'*)/2] [(8.4175 X 10-4)/l] = -5.5038 x lO'^, but since R is negative, the test is not upper tailed. Minimizing A-^ to locate two optimal lower tailed test check points yields a subregion in the factor space of optimal check points. The pair of check points (0.3749, 0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at random from this subregion, and these check points yield R = [(8.4175 X 10-'^)/l] = -8.4175 x lO"'*. Since R is negative for the upper tailed test points and the lower tailed test points, we have Case 2 of Section 3.3.4 again and the lower tailed test check points are used to test for lack of fit. The hypothesis H : X = X = is rejected in favor of H : [x /2] [x /l] < if the cala X z culated value of F = (d'V~ d/2)/MSE is less than F.^ . „ , , in which case we say lack of fit of the model is ( l-o ) ; 2 , 1 present. * 2 If an external estimate a ^ had been available for ext this example, then the optimal upper tailed test check points could have been used in the F ratio, F = (d'V~ d/k)/a^ ^, and lack of fit would then be detected ^" ext if the calculated value of F exceeded F , a ;k, V Example 3. Example 3 illustrates the procedure for locating optimal check points when there are two terms in the true model in addition to those in the fitted model. A second order canonical polynomial model in three mixture

PAGE 83

75 components is fitted using a q = 3 simplex centroid design. The true model is assumed to contain eight terms, six of which are the same terms as in the fitted model, with the additional two terms being the third order terms *^12^1^2^^1 ~ ^2^ ^^^ ^123^1^2^3* ^ ^^ Example 2, there is one degree of freedom for MSE which is used to estimate a2. The test statistic, F = (d ' v'-'-d/k )/MSE, is given in equation (3.8). Since p2 = 2 and A-j^ is a 2^2 matrix, locating the optimal upper tailed test check points by the procedure of maximizing X ^^ is assisted by the maximizing of a lower bound for A,, namely maximizing u . Bl3_/2o^, where y . is the min—z—z min 2 smallest eigenvalue of A-^. Since gand a are unknown, this is equivalent to maximizing u . . For u . to exceed mm min zero, it is necessary that A-^ be of full rank, and since rankCA-j^) < min(k, P2), it is necessary to select k > 2 check points. If A^ is less than full rank, and thus is positive semi-definite, only a subset of possible values of 3 „ could be considered to make it possible to test for lack of fit with an upper tailed test. Using the Controlled Random Search Procedure, the points that maximize w . are found to be (0.418, 0.277, 0.305) and (0.277, 0.418, 0.305). These points are thus optimal candidates for upper tailed test check points. At these check points we have v . = 5.1623 x 10"^, A, = mm 1 diag[5.1623 x io-4, 5. 1916 x lO'^] , A2 = diag[0, 8.4175 x 10-4], 3f^^ R ^ [A^/2] [A2/I] = diag[2.5811 x lo'^, -5.8217

PAGE 84

76 X 10"^]. Since the eigenvalues of R are -5.8217 ^ 10"^ and 2.5811 X 10"'^, R is indefinite. Following the suggested procedure for Case 4 of Section 3.3.4, we note that an upper tailed test for lack of fit exists if the value of -2 ~ ^"^12' ^123-^' ^^ """ ^^^ column space of [S2:S2] but not entirely in the column space of S2, where S-^ is the matrix whose columns are the orthonormalized eigenvectors of R corresponding to the positive eigenvalues of R, and S2 is the matrix whose columns are the orthonormalized eigenvectors of R corresponding to the zero eigenvalues of R. Since R has no zero eigenvalues in this example, S2 does not exist, but S^ is the column vector, S-^ = [1,0]'. Thus if 3 is of the form 3 = [^-,2' °^ ' ' ^^^^^ "^12 * °' ^^^^ 3^ is in the column space of Sj^ and the test is upper tailed . The matrix A2 has rank one and therefore is positive semi-definite. Hence it is impossible to locate two check points that minimize w^^j^ and also make R = [h-^/2] [A2/I] negative definite (see Appendix 4), that is, it is impossible to find a lower tailed test that is capable of testing lack of fit for all values of 3 . However, if we use the Controlled Random Search Procedure to locate two check points that minimize an upper bound for A which is y 8 13^/20 , then by minimizing m , we find that any of max-2-2^ ^ max the check points in a particular subregion of the factor space yield a near minimum for u . One pair of points in -^ max this subregion is selected as the points to be used as

PAGE 85

77 optimal lower tailed test check points, namely the pair consisting of the point (0.053, 0, 0.947) replicated twice. Replicating this check point, we find u = 7.3900 max X 10"11, A-L = diag[0, 7.3900 x 10"11], A2 = diag[0, 8.4175 x 10""^], and R = [A-^/2] [A2/I] = diag[0, -8.4175 x lO"'^]. The eigenvalues of R are and -8.4175 x 10"'* implying that R is negative semi-definite. The values of g that are in the column space of [82:33] but not entirely in the column space of S2 will provide a lower tailed test. Here, [82:33] = diag[l,l] and S2 = [1,0]'. Thus, the test for lack of fit is lower tailed if Si^-:* * 0. For values of 3„ that produce an upper tailed test we use the check points (0.418, 0.277, 0.305) and (0.277, 0.418, 0.305) with the F ratio d'v/d/2 F = MSE and conclude there is lack of fit if the calculated value of F exceeds F^,2 i* ^^1^ values of 3^2 that produce a lower tailed test, we use two replicates of the check point (0.053, 0, 0.947), and conclude there is lack of fit if F is less than F , where again F is calculated by F = (d'Vg-'-d/2)/MSE. Example 4. Example 4 illustrates Case 3 of Section 3.3.4 in which MSE is used to estimate a^ in the lack of fit test statistic. A second order canonical polynomial model

PAGE 86

78 in three mixture components is fitted using the {3,3} simplex lattice design, which appears in Figure 5. The true model is assumed to be special cubic, thus p2 = 1 and An is a scalar. The {3,3} design consists of ten design points and since there are six parameters to be estimated in the fitted model, o^ can be estimated by MSE with N p = 10 6 = 4 degrees of freedom. We first suppose that a single check point is to be used to test for lack of fit. Using the Controlled Random Search Procedure we find the single check point that maximizes the scalar A^ = (X* X*A) 'Vq-'-(X* X*A) is located at the centroid of the simplex factor space. Thus (X*, X*, x*) = (1/3, 1/3, 1/3) is the optimal candidate for an upper tailed test check point. At this centroid point, A-^ = 4.9076 x 10"^. For the {3,3} design the scalar quantity A2 = (X2 XA)'(X2 XA) is fixed and is equal to A2 = 9.4062 X 10"'* and thus, R = [Aj^/k] [A2/(N p)] = [(4.9076 X 10"^)/1] [(9.4062 x 10"^)/4] = 2.5560 x 10""*. The point that is the optimal candidate for a lower tailed test check point is chosen randomly from a subregion of points in the factor space, in which all points minimize Aj^. The point selected has the value (x?, xi, xJ) = (0.560, 0.410, 0.030). Here A-^ = 9.6590 x 10"'^ and R = [(9.6590 x 10"'7)/1] [(9.4062 X 10-4)/4] = -2.3419 x 10*4.

PAGE 87

(ff°) (i'fo) (1,0,0) (0,1,0) {i'°'i) (0,0,1) Figure 5. The {3,3} simplex lattice design 79 Since R is positive for the optimal upper tailed test check point (1/3, 1/3, 1/3) and R is negative for the optimal lower tailed test check point (0.560, 0.410, 0.030) we are in Case 3 of Section 3.3.4. Either the upper or lower tailed test could be used to test for lack of fit, but 2 if the quantity S'B_/a can be specified, then we will choose to use the test that has greater minimum power, since greater power means that we are more likely to detect lack of fit when in fact lack of fit exists. In this example ^2= ^123For illustrative purposes, we arbitrarily choose 2 3 '3 /a = 2000, so that an approximate conservative lower bound for the power of the upper tailed test is found by

PAGE 88

80 calculating P I F" > F 1 ^ k,N-p;A j^^,X2u a;k,N-pJ where F ,, ,, _ is the upper 100a percentage point of the a ; K , N— p central F distribution, k is the number of check points, N is the total number of response observations, p is the number of parameters in the fitted model, ^IL = ^min^-2^-2/2^^' ^"d A^u = 'S^ax^2^2/2<'^The quantity y ^^ is the smallest eigenvalue of Aj^, where Ai is evaluated at the optimal upper tailed test check point. Since A, is a scalar, y . = A^ . Likewise, 6 is the -L mm 1 max largest eigenvalue of A2, and since in this example A2 is a scalar, 6 = A.. In this example we have k = 1, N p = max 2 10 6 = 4, \ = u^. SiB-/2a^ = (4.9076 x lO""^ )( 2000/2 ) = 4.9076 X 10-1, and X ^^ = &^^^&.^S,2/2o^ = (9.4062 X 10-^)(2000/2) = 9.4062 x lo'"*". Using the approximation to the cumulative probabilities of the doubly noncentral F distribution given by Johnson and Kotz (1970, p. 197) which is described in Appendix 6, and taking a = .05, we find that a conservative lower bound for the power of the optimal upper tailed test is approximately equal to .0649. The minimum power for the optimal lower tailed test is 2 approximated (assuming 3A3-,/a = 2000) by calculating ^ l^k,N-p;Aj^y,X2L '^ ^( 1-a ) ;k,N-p^ *

PAGE 89

81 The quantities \ ^^ and x ^^ are taken as x = y B'g /2a^ 2 and X= 6^. el3o/2o where u ^ is the largest eigenvalue zL min-2-2 max ^ ^ of A]^ with Aj^ calculated using the optimal lower tailed test check point, and where 6 . is the smallest eigenvalue of mm ^ A^. Since A-i and An are scalars, u = A, and 6 . = A . ^ J. z •^max 1 min 2 In this example, k = 1, N p = 4, X^^ = (9.6590 X 10"'^)(2000/2) = 9.6590 x lo""*, and X2L = (9.4062 x lo""* )( 2000/2 ) = 9.4062 x lo"""-. Again if the approximation to the doubly noncentral F distribution given in Johnson and Kotz is used, an approximate conservative lower bound for the power of the optimal lower tailed test is .0555. Having specified l^^^^a'^ = 2000, the optimal upper tailed test is chosen over the optimal lower tailed test, because the approximate minimum power of the upper tailed test is greater than the approximate minimum power of the lower tailed test. Using the optimal upper tailed test check point (1/3, 1/3, 1/3) in the test statistic MSE we conclude that lack of fit is significant if the calculated value of F exceeds F , , , in which case we a ; 1 , 4 reject HQ:X-L=X2 = 0in favor of H^ : X j^/1 X 2/4 > . When two simultaneous check points are used for testing lack of fit, the Controlled Random Search Procedure locates the optimal upper tailed test and optimal lower tailed test

PAGE 90

82 check points. It turns out that two replicates at (1/3, 1/3, 1/3) maximize A^, and are used as optimal check points for an upper tailed test. The value of R = [A^/2] [A2/4] is [(7.9210 X 10"'^)/2] [(9.4062 x 10"'^)/4] = 1.6090 x 10-4. In searching for two optimal lower tailed test check points, again a subregion of the factor space is found in which any of the points nearly minimize A-,. From this subregion are chosen the points (0.6386, 0.3263, 0.0351) and (0.7257, 0.2421, 0.0322) resulting in a value of R = [A.j^/2] [A2/4] of [(1.5216 x 10-9)/2] [(9.4062 x 10-4)/4] = -2.3516 X 10-4. In conclusion, when two simultaneous check points are used in the test for lack of fit in this example, R is positive for the optimal upper tailed test and R is negative for the optimal lower tailed test, and we have Case 3 of 2 Section 3.3.4. Selecting 3A3^/a = 2000 arbitrarily, we found the approximate lower bound for the power of the upper tailed test to be .0504, and the approximate lower bound for the power of the lower tailed test to be .0612. Since the power is higher with the lower tailed test it is our choice for testing lack of fit when two check points are used simultaneously. Lack of fit is detected and we reject H : X = X = in favor of H : [X /2] [X /4] < if the F U X ^ Si 1. ^ ratio, F = (d'V~ d/2)/MSE, using the optimal lower tailed test check points (0.6386, 0.3263, 0.0351) and (0.7257, 0.2421, 0.0322) is calculated to be less than F . ( ±~ct J ; z , '1

PAGE 91

83 3.4.2 Numerical Examples Numerical Example 1. In this example we illustrate numerically some of the findings in the first theoretical example of Section 3.4.1. Data that were collected in a rocket fuel experiment (Kurotori, 1966) will be used to investigate the power of the lack of fit F test. The test is set up to detect the inadequacy of a fitted second order canonical polynomial model when the true model is special cubic. Calculated values of the power of the test which detects lack of fit through large values of d'v""'-d/k F = — " ^2 °ext will be compared for several check point locations, including the location (1/3, 1/3, 1/3) at which the power was found to be maximum in Example 1 of Section 3.4.1. In Kurotori 's experiment the modulus of elasticity (Y) of a rocket fuel is expressed as a function of the proportions of three components — binder (x^) , oxidizer (X2)/ and fuel (X3). Since lower bounds are placed on the component proportions x^, x^, and X3, in the form of 0.20 < x^, 0.40 < x^/ and 0.20 < x , pseudocomponents (x!) are defined in terms of the original components in the form of xj = (x^ 0.20)/(1 .80), x^ = (x^ 0.40)/(l .80), and x^ = (x^ 0.20)/(l .80). The true special cubic model in the pseudocomponents, which is obtained by using the data at the seven points of the simplex centroid design

PAGE 92

84 in the pseudocomponent system, is E(Y) = 2350X' + 2450X' + 2650x' + Ox'xl -i. ^ J X ^ + lOOOx^x^ + leOOx^x' + 6150x'x'x' The second order canonical polynomial model that is fitted to the six boundary points only, and which will be tested for lack of fit, is given by Y = 2350x| + 2450X' + 2650x' + lOOOx^x^ + leOOx^x'. The configuration of the experimental design as well as the check point locations are depicted in Figure 4 of Chapter 2 and the observed response values are given in Table 3 of this chapter. -1 "2 A value of the ratio F = [d'V_ d]/a is calculated at -^ ext each of the four individual check points (1/3, 1/3, 1/3), (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3) * 2 "2 where a . is assumed to have the value a ^ = 144 as ext ext suggested by Kurotori (1966). We also assume without loss of generality that the degrees of freedom associated " 2 with o ^ are v = lo. The power of the F test is calculated ext at each of the four check points by using the approximation to the cumulative probabilities of the noncentral F

PAGE 93

85 Table 3. Observed Response Values at the Pseudocomponent Settings for Kurotori's Rocket Fuel Experiment — Numerical Example 1. Observation Binder Oxidizer Fuel Modulus of Elasticity Number

PAGE 94

86 only on the boundaries of the triangle (and therefore at each point at least one of the x! values is equal to zero), then X2 = and A = 0. From the true special cubic model, ^123 = ^^5°The calculated value of F as well as the approximate value for the power at each of the four check points is given in Table 4. The check point (1/3, 1/3, 1/3) produced the highest power of the four check points investigated, supporting the previous results of Example 1 in Section 3.4.1 where (1/3, 1/3, 1/3) was selected as the check point location with the maximum power when a second order canonical polynomial was fitted using the {3,2} simplex lattice design, but the true model was assumed to be special cubic. Additional support for the point (1/3, 1/3, 1/3) being optimal is given by the contour plot of values of A, in Figure 6(d). The highest values of A-^ appear near the centroid (1/3, 1/3, 1/3) where high A^ values translate into 2 2 high \ -^ values, since ^i = ^1^123/^^ ' which in turn implies high power since we know the power is an increasing function of X j^. As a second part of this example the power of the F test that is obtained when three replicates are taken at (1/3, 1/3, 1/3) is compared to the power of the F test that is obtained when one replicate is taken at the three check points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3). These latter three point locations were suggested by Kurotori for testing lack of fit of his fitted special

PAGE 95

87 u

PAGE 96

88 cubic model. The result of this comparison, see Table 4, is that the three replicates at (1/3, 1/3, 1/3) produce the test with greater power which again supports the findings of Example 1 of Section 3.4.1. All of the check point locations listed in Table 4 produce very high power values (> .999) which is due in part to the high value of & ^^^ ^^123 " 6150). If 3 were of lower magnitude, then the three replicates at (1/3, 1/3, 1/3) would show a still greater superiority in the power value compared to the power using the other check points. This superiority is demonstrated in Table 5 where values of 3^23 3^^ listed as 3000 and 1500 and the comparative power values are listed as 0.998 compared to 0.795 and 0.662 compared to 0.249, respectively. Table 5 also demonstrates the superior power value for the point (1/3, 1/3, 1/3) when ^123 ~ ^^^^ °^ ^123 ^ 1500 and each of the four check points is used one at a time. Finally, (1/3, 1/3, 1/3) being the optimal check point location is seen in Figure 6(c), where contour plots of the expected difference in the heights of the surfaces are drawn. The differences in the heights are found by subtracting the estimated height of the surface obtained with the fitted second order model from the estimated height of the surface obtained with the true special cubic model. The expected difference between the true and fitted surfaces approaches a maximum the closer one moves to the centroid of the simplex factor space, so that the optimal check point

PAGE 97

89 u

PAGE 98

90 ^2^' V (a) True special cubic surface. (b) Fitted second order surface. X. =1 (c) Expected difference between the true special cubic surface and the fitted second order surface. (d) A,=(X* X*A)' Vq'(X*-X*A Figure 6. Contour plots for Numerical Example 1.

PAGE 99

91 location (1/3, 1/3, 1/3) coincides with the point where the expected difference between the true special cubic surface and the fitted second order surface is maximum. Numerical Example 2. In this second numerical example, we investigate the power of the F test for detecting lack of fit when a second order canonical polynomial model is fitted in a mixture system which is in truth represented by a special cubic model. The true model is assumed to be E(Y) = 2350X + 2450X + 2650x ^ ^ J + lOOOXj^x^ + leOOx^x^ + 6150x X x which is used to generate hypothetical response observations at the seven points of the q = 3 simplex centroid design as well as at three check points. The values of the response are obtained by adding the value of a pseudorandom normal variate with mean and variance 144 to each true predicted response value. The data are given in Table 6. The response values at the seven points of the simplex centroid design are used in the least squares normal equations to obtain the fitted second order model Y = 2341x^ + 2438X2 + 2630x2 + 310x^X2 + 1304x^^X2 + 1970x x

PAGE 100

92 Table 6. Generated Response Values — Numerical Example 2, Xi X2 X3 Y 10 2357 10 2454 1 2646 1/2 1/2 2403 1/2 1/2 2747 1/2 1/2 2962 1/3 1/3 1/3 3013 * 1/3 1/3 1/3 2993 2/3 1/6 1/6 2693 .02 .93 .05 2550 * Check points which is to be tested for lack of fit using the test statistic F = d'v" d/MSE. The F statistic will be evaluated at each of the three check points (1/3, 1/3, 1/3), (2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time, and the power of the test at the three check point locations will be calculated and compared. The test is lower tailed for all check point locations (since R = A-, A2 is negative for all check point locations) and thus the power is defined as pi F" < F 1 ^ l,l;Xi ,X2 .95;1,1^

PAGE 101

93 X -I (.02 ,.93, .05 ) X =1 2 x =1 3 Figure 7. Contours of R = Aj^ A2 for Numerical Example 2, 2 2 2 2 The values of X , = 3 123^1'^^'' ^^^ ^2 ~ ^123^2'^^'' ^^^ found 2 by taking 6 , j-^ = 6150 and a = 144. The results of this power investigation are given in Table 7. Since the check point (.02, .93, .05) produces the greatest power of the three check points investigated, this supports the result in Example 2 of Section 3.4.1, where we saw that the point (.02, .93, .05) yielded the maximum power of all points for detecting lack of fit of a fitted second order canonical polynomial model, using the q = 3 simplex centroid design in the presence of a true special cubic surface. Additional evidence for the point (.02, .93, .05) being an optimal check point is shown in Figure 7, where contours of the values of R = A-j^ A2 are presented . The point (.02, .93, .05) is seen to belong to an area of the

PAGE 102

94 u 1^
PAGE 103

95 simplex factor space where R is minimum, which implies that At (and in turn X-.) is also minimum in this area, since R = ^1 " ^^2 ^^'^ ^2 ^^^ ^^^ fixed value of A2 = .00084 for the simplex centroid design. Thus the check point (.02, .93, .05) produces a minimal X , value and maximum power, since the power increases with decreasing values of X-^3.5 Discussion When check points are used for testing lack of fit in a mixture model, the appropriate testing procedure, assuming a normally distributed response, involves an F statistic. If " 2 an external estimate, a *-' °^ ^^^ experimental error variance is available so that the test statistic is given by d-v^Vk F = — '2 ^ext then the power of the test for lack of fit is maximized by choosing k check points that maximize the value of the noncentrality parameter X-^, When P2 = If maximizing X -^ is achieved without knowing the value of the elements of g by selecting check points that maximize the scalar A^ . When P2 > If the maximization of X j_ is approximated by maximizing a lower bound for Xj_. This is achieved also without knowing the values of the elements of 6 by selecting check points that maximize the smallest eigenvalue of the matrix A-j^ . The test is upper tailed, and for given values of Xi, the actual

PAGE 104

96 power of the test can be calculated from the cumulative probabilities of the noncentral F distribution. A problem arises when A-^ is positive semi-definite and its smallest eigenvalue is equal to zero. In this case check points that maximize the smallest positive eigenvalue of A-i are selected, and lack of fit is only detectable for a subset of the possible values of the elements of g . When an external estimate of a^ is not available, testing lack of fit at the check points is further complicated. The F statistic is F = MSE and the rejection region for the lack of fit test can be upper tailed or lower tailed. The power of the test is determined by using the doubly noncentral F distribution, which depends on the parameters k, N p, X^, and A 2Of these four parameters, only k and X-^ are influenced by check points, and if the value of k is fixed, the power of the test is maximized by choosing check points that affect the value of X-j^* Regardless of the values of the elements of , check points that maximize X, are selected for maximizing the power of an upper tailed test, and check points that minimize X •, are selected for maximizing the power of a lower tailed test.

PAGE 105

97 Lack of fit can be tested with the upper tailed test for all nonzero values of the elements of g if the check points are selected so that [A-^/k] [A2/(H p)] is positive definite, since this forces the expected value of the numerator mean square in the F ratio to be greater than the expected value of the denominator mean square. Similarly, lack of fit can be tested with a lower tailed test if check points are selected which make [A, /k] [A2/(N p)] negative definite. When it is not possible to select check points that make [A-^/k] [A^/CN p)] either positive definite or negative definite then detection of lack of fit is only possible for a subset of all nonzero values of the elements of 3 . The power of the test for lack of fit using the F statistic in (3.8) is a function of X^ and X2* Since the magnitudes of Xj^ and X2 are influenced by the experimental design, an area for future study is the investigation of the effect of the experimental design on the power of the lack of fit test. In the presence of an external estimate of 2 a , Atkinson (1972) suggested designs that maximize the determinant of A2, |A2i, when lack of fit is to be detected by a large value of X using a procedure which is, in general, equivalent to the lack of fit testing procedure that partitions the residual sum of squares into pure error and lack of fit sums of squares. It might be useful to apply Atkinson's (1972) methodology not only to |A |, but to \h^\ or \h^/k A2/(N p)| in order to find an

PAGE 106

98 appropriate design when testing lack of fit with the F ratio in Eq. (3.8). Since the power of the test in Eq. (3.8) is also affected by the values of k and N p, which are the numerator and denominator degrees of freedom of the doubly noncentral F distribution, respectively, optimal settings for these parameters can also be considered. For a given fitted model, p is fixed so that the degrees of freedom would be influenced by the number of check points selected, k, and by N, the total number of observations. Finally, the experimental design and the number of check points also " 2 affect the power of the F test when o ^ is used to ext 2 estimate a . Thus the effect of the experimental design and the number of check points can also be investigated for the situation where the lack of fit test statistic is given by Eq . ( 3 . 3 ) . We now conclude our investigation of the check point approach to lack of fit testing and in the next chapter turn to an investigation of a near neighbor method for testing lack of fit.

PAGE 107

CHAPTER FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR TESTING LACK OF FIT 4.1 Introduction In an experiment in which replicate response observations are available at one or more design points, lack of fit of a fitted model can be tested by a procedure which involves partitioning the residual sum of squares into two statistically independent portions. One portion is the sum of squares due to lack of fit (SSj^Qp), and the second portion is the sum of squares due to pure error (SSEp^_,j.g) obtained from the replicates. As discussed in Section 2.2, this procedure was suggested by Draper and Smith (1981, p. 120). Lack of fit is inferred if the calculated value of the ratio ""^LOF ^=^iSE (4.1) pure exceeds the corresponding upper 100a percentage point of the central F distribution, where MSlof ^"d MSEp^j-g are the mean square values found by dividing SSlof ^""^ ^^^pure ^^ their respective degrees of freedom. In order to test the fitted model for lack of fit when replicate observations are not available, Shillington (1979) suggested a procedure which uses observed response values 99

PAGE 108

100 collected at points which are "near neighbors" in the factor space in place of replicates (see Section 2.3). Lack of fit is inferred when the calculated value of the ratio MSEg ^ ^ MSE„ (4.2) W exceeds the upper 100a percentage point of the central F distribution. The numerator, MSEg, of the F ratio in Eq. (4.2) is a generalization of the numerator, MSlqf' °^ *-^^ ^ ratio in Eq. (4.1). The form of MSEg will be given in Eq. (4.5) of Section 4.3. The denominator, MSE^^, in Eq. (4.2) is a generalization of MSEp^j.^ in Eq. (4.1), and the value of MSEy^ is calculated using near neighbor observations in place of replicates (see Eq. (4.6) in Section 4.3). Shillington 's near neighbor method provides an alternative to the check points method when replicate observations are not available. Typically, near neighbors might appear either because an experiment was not designed to provide replicate observations or with a designed experiment consisting of a large number of design points in a bounded factor space which results in some points being near one another. In this chapter we shall further study Shillington • s (1979) near neighbor procedure for testing lack of fit. A question involving the correctness of the ordinary least squares technique suggested by Shillington for deriving the denominator, MSE^, of the F ratio in Eq. (4.2) will

PAGE 109

101 be raised. The question will be resolved by showing the equivalence of deriving MSE^ by ordinary least squares and of deriving MSE^^ by a generalization of weighted least squares. We will verify that when the observable response values are assumed to have the normal distribution with 2 homogeneous variance, a , the F ratio in Eq. (4.2) possesses a central F distribution when the fitted model is adequate, but possesses a doubly noncentral F distribution when the fitted model suffers from lack of fit. We shall also show that the F test for lack of fit which uses the statistic in Eq. (4.2) can have either an upper tailed or a lower tailed rejection region. Finally, the use of a clustering algorithm for defining groups of near neighbors will be proposed. 4.2 Notation In this section we introduce the notation to be used in this chapter. Throughout our investigation of Shillington 's near neighbor procedure for testing lack of fit, we shall assume the observed response values collected in an experiment can be grouped into g cells where the jth cell contains nj observations, j = 1, 2, ..., g. The observations in a cell are from points that are "near neighbors" in the sense that they are near one another in the factor (mixture) space. A model of the form E(Y) = XBj_ (4.3)

PAGE 110

102 is fitted to the data using ordinary least squares, but the true model is assumed to have the form E(Y) = X3j^ + X^^^, (4.4) where Y is an Nxl vector of response values observable at 2 the design points with var(Y) = a !„, X and X2 are Nxp and Nxp2 matrices of known constants, respectively, and g, and g^are pxl and P2^1 vectors of unknown regression coefficients, respectively. The vector Y is assumed to have the normal distribution. Let us now define the following vector and matrix quantities to be used in developing the numerator, MSEg, of the F ratio in Eq. (4.2): Y = a gx 1 vector with jth element equal to the average of the n^ observed response values in the jth cell of near neighbor observations, j = 1, 2, . . . , g . Xq = a gxp matrix whose jth row is the average of the nj rows of X corresponding to the jth cell, j = 1, z f ..., g. X2C = a g>
PAGE 111

103 Gr. = a gxg diagonal matrix of the form G = diag[l/nj^, l/n2/ •••/ l/"q]' To further illustrate the forms of Y^, X^, X2C' ^rid Gg/ we present the following numerical example. Consider a data set consisting of response observations (Y) taken at N = 8 different combinations of the settings of the factors xi and X2, where the eight response observations are divided into g = 4 near neighbor cells. The vector of observed response values, Y, and the matrix X corresponding to the first order model, E(Y) = 3q + Pj^x^^ + 3 2^2' ^^^ Y = 10 13 16 15 18 21 27 30 X = 112 12 5 12 4 13 2 13 1 14 2 I 5 5 15 4 The horizontal lines in Y and X delineate the four cells of near neighbors. In this example ^C = 10 (13 + 16)/2 (15 + 18 + 21)/3 (27 + 30)/2 10 14.5 18 28.5

PAGE 112

104 ^C = 1 (1 + l)/2 (l+l+l)/3 (1 + l)/2 1 1 1 1 1 (2 + 2)/2 (3+3+4)/3 (5 + 5)/2 1 2 3.3 5 2 (5 + 4)/2 (2+l+2)/3 (5 + 4)/2 2 4.5 1.7 4.5 and Gq = diagd, 1/2, 1/3, 1/2). If the true model is second degree, E{Y) = 2 2 8q + Bj^Xj^ + 3 2^2 "*" ^12^1^2 '*' ^11^1 '*' ^22^2' ^^^" ^^^ ^2 ^"^ X2C matrices have three columns corresponding to the 2 2 terms x,Xp, x, , and x-, respectively. For this numerical example we have ^2 = and '2C 2 10 8 6 3 8 25 20 2 9 5.7 22.5 1 4 4 9 9 16 25 25 1 4 11.3 25 4 25 16 4 1 4 25 16 4 20.5 3 20.5

PAGE 113

105 Next let us define the following quantities to be used in developing the denominator, MSE^, of the F ratio in Eq. (4.2): W = an Nx 1 vector of within cell deviations where the ith element, W^ , of W is equal to the difference between the ith element, Y-^/ of Y and the average of the response values observed in the near neighbor cell containing Y^ , i = 1, 2, . . . , N. X^ = an Nxp matrix whose ith row is equal to the ith row of the X matrix minus the row of X^ corresponding to the cell containing the response value observed at the ith row of X. r = rank(X^^). ^2W ~ ^" Nxp2 matrix whose ith row is equal to the ith row of the X2 matrix minus the row of X2P corresponding to the cell containing the response value observed at the ith row of X. Zq = an NxN idempotent matrix of the form ^0 " -^N " di^gni/n^)J^, (l/n2)J2/ . . . , ( 1/n )J ] where Jj is an njxnj matrix of ones, and If^ is g the identity matrix of order NxN, with N = En. j=l ' Let us illustrate the forms of W, X^, X2^^, and J: q by using the numerical example presented earlier in this section, where the eight response observations were distributed among four cells. For these data we have

PAGE 114

106 W = ^w X 2W

PAGE 115

107 and vector quantities defined in Section 4.2. The test that Shillington proposed involves the use of an F ratio (see Eq. (4.2)) of two statistically independent mean square values, 2 each of which is an unbiased estimate of a when the fitted model is the correct true model. The two independent mean 2 squares become biased estimates of a when the fitted model suffers from lack of fit. Shillington 's methodology detects lack of fit when the calculated value of the F ratio in Eq. (4.2) is large, thus his test is upper tailed. We shall see later in Section 4.7 that the test is not always upper tailed, and may be lower tailed. Shillington points out that the power of the test depends on the relative magnitudes of E(MSE3) and E(MSE^^), that is, the power depends on the difference between the expected values of the numerator and of the denominator in the F ratio in Eq. (4.2). We shall be more specific than Shillington by discussing the power of the test in terms of parameters of the doubly noncentral F distribution. We now turn to defining Shillington 's test statistic in matrix notation. Shillington 's F ratio takes the form (see Eq. (4.2)) SSE /(g p) F = ^ SSE^/(N g r) MSE3 MSE^

PAGE 116

108 where SSEg is the residual sum of squares from a weighted least squares regression analysis in which Y is regressed on Xq, g is the number of cells of near neighbors, p is the number of terms in the fitted model, and r is the rank of Xy^. The quantity SSEg can be written as the quadratic form (see Graybill, 1976, p. 329; also see Draper and Smith, 1981, p. 109). The quantity SSE^^ is defined as SSE^ = W'[In X„(X^'X„) X^;,]W, (4.6) where (XJLX„)~ is any generalized inverse of (XAX ). [A matrix A~ is defined as a generalized inverse of the matrix A if AA~A = A.] The quadratic form SSE^ is the residual sum of squares from an ordinary least squares regression analysis in which W is regressed on X^. In the next two sections we shall discuss the development of the numerator and denominator, MSE3 and MSE^, respectively, of the F ratio given in Eq. (4.2). We then suggest an alternative representation for MSE^ which relies on a generalization of weighted least squares. This alternative representation for MSE^ will be shown to be equivalent to Shillington 's expression for MSE^.

PAGE 117

109 4.3.1 Development of MSEg The quantity MSEg = SSEg/Cg P) is the numerator of the F ratio in Eq. (4.2). As mentioned in Section 4.3, the quantity SSEg is the residual sum of squares from a weighted least squares regression analysis in which Yp is regressed on Xp. The weighting is appropriate because var(Yp) = a G„ not only when the fitted model is adequate (under model (4.3)), but also when the fitted model suffers from lack of fit (under model (4.4)). In order to further explain the (Yp, Xp ) system, we define the matrix M as M = diag[(l/n^)l|, ..., (l/n^)!^] where 1 . is an n-xl vector of ones, j =1, 2, ..., g. Then the (Yp^ X ) system can be derived as a linear transformation of the (Y, X) system. That is, application of the transformation matrix M yields the following equalities Xc = ^'X' and X^ = MX, ^2C ~ "^^2 From this it follows that var(Y ) = M var(Y)M' = a^MM' 2 a G_ , since MM' = G.. Under the hypothesized model of

PAGE 118

110 Eq. (4.3), E(Y^) = ME(Y) = MXB^ = X^gj^, whereas under the model of Eq. (4.4), E(Y^) = M(Xe^ +^2^2^ " ^C^l "^ ^2C§-2' We now consider the distribution of the random quantity 2 SSEg/a . It can be shown (Theorem 2, Searle, 1971, p. 57) that under the model of Eq. (4.3), SSE_,/a^ possesses a central chi-square distribution with g p degrees of freedom, but that under the model of Eq. (4.4), SSE /a possesses a noncentral chi-square distribution with g p degrees of freedom and noncentrality parameter n , , where "l = (l/2-^)§.2^2cfS^ S\(^C^0^^c)"^^cS^J^2C^-2^'''^ Here we point out that the noncentrality parameter for 2 SSEg/a given by Shillington (1979) is not correct and should be written as in Eq. (4.7). Finally we note that SSEg is equivalent to the usual lack of fit sum of squares, SSlqF' "here SSlqf/^^ ~ P) = MSlqp is the numerator of the F ratio in Eq. (4.1), when all observations in each cell are true replicates rather than near neighbors. Shillington pointed out this fact, but did not give a proof. We offer a proof in Appendix 7. 4.3.2 Development of MSE^^ The quantity MSE^ = SSE^^/(N g r), where r denotes the rank of X^^, is the denominator of the F ratio in Eq. (4.2). As mentioned in Section 4.3, the quantity SSE^ is the residual sum of squares from an ordinary least squares regression analysis in which W is regressed on X^^. Using

PAGE 119

Ill Theorem 2 (Searle, 1971, p. 57) and noting that W = E Y and E^Z^ = Z^, it can be shown that under the 02 hypothesized model of Eq. (4.3), SSE /a possesses a central chi-square distribution with N g r degrees of freedom, 2 but under the model of Eq. (4.4), SSE.ya possesses a w noncentral chi-square distribution with N g r degrees of freedom and noncentrality parameter H „ , where ^ = (l/2a2)3.X'^tI^ X^(X^'X^)-X^]X2/_2. (4.8) Shillington (1979) points out that SSE^ reduces to the usual pure error sum of squares, SSEp^J.g, when all cells contain true replicates. This is easily seen by using the fact that X^ = when all cells are composed entirely of true replicates so that SSE,, = WW = Y'E.I.Y = Y'l.Y = SSE ^ W---0 0--0pure We saw in Section 4.3.1 that the (Yp, X„) system is derived as a linear transformation of the (Y, X) system. Similarly, the (W, X^) system can be derived by applying the transformation matrix Zq. Thus W = ^^.Y, X^^ = ^^.X, and X = ^p,X . It follows that E(W) = 2f.E(Y) = ^^XP = X^S , under the model of Eq. (4.3), and E(W) = I (X^ + X 3 ) = X^ + X g under the model of Eq. (4.4). Furthermore, var(VjJ) = Z var(Y)5:|^ = (J^^q^q = <^^^o' ^^^^^ ^0 is symmetric and idempotent. Since the variance-covariance matrix of W is not equal to alj^, for some positive constant a, SSE^ should have been derived as the residual sum of squares from a weighted least

PAGE 120

112 squares regression analysis of W on X^ rather than from the ordinary least squares regression of W on X^ that Shillington (1979) suggested. We shall use the weighted least squares regression of W on X^ in an attempt to replace SSE^ in the F ratio in Eq. (4.2) by an expression we will call SSEy^(weighted) . We later show that SSE^^ and SSE^( weighted) are equivalent. 2 The variance-covariance matrix of W, which is a H ^, is of rank N g and is thus singular. Therefore the residual sum of squares from a weighted least squares regression analysis of W on X^ which is w'l^o^ z-%(x;i-%)-x;,z-i]w cannot be used as an expression for SSE^^( weighted ) , since Sq does not exist. The problem of performing a weighted least squares regression analysis when the variancecovariance matrix of W is singular is considered in the next section. 4.4 Development of SSEyj( weighted ) 2 If the variance-covariance matrix of W, a Eq, is nonsingular then the weighted least squares formula for SSEY^(weighted ) is SSE^(weighted) = (W X^p^^ ^ '^ o"^ ^ " Vl^

PAGE 121

113 where Q_^ is a solution to X^Eq~-'-X^3 = X^J^z"-'-W and can be written as B^^ = ( X^ ~ X^)~X^J^Z ^''-W. The quantity SSE^( weighted) divided by the appropriate degrees of freedom 2 provides an unbiased estimate of a under the model of Eq. (4.3) . However, since 2: q is singular, the weighted regression formula above cannot be used to calculate SSE^^(weighted ) . C. R. Rao (1971, 1972, and 1973) suggests an analog of weighted least squares for the case of a singular variancecovariance matrix. Rao suggests the existence of a matrix H such that 6, is a stationary vector value of (W X^0j^)'H(W X^Bj^) in which case o^ may be estimated using a^ = (W X^3^)'H(W X^3i)/(N g r) where (N g r) = rank(EQ:X^) rank(X^). The rank of the matrix (EqTX^) is equal to N g because X^ = EqX^ so that the columns of X^ are spanned by the columns of z ^, thus, rankCZ^rX^) = rank(EQ) = N g. One form of the matrix H is defined (Rao, 1971 and 1972) as H = [Eq + c^X^X^] (4.9) where c is an arbitrary nonzero constant, so that with the model of Eq. (4.3), o^ = (W X^g_j^)'H(W X^^^)/(li g r)

PAGE 122

114 2 IS an unbiased estimator for a . Thus a stationary vector value of (W X^3,)'H(W ^^,) is given by gj^ = (X^HX^)~X^HW . Rao indicates (1972, p. 3) that a^ is invariant to the choices of the generalized inverses ^ 2 involved in a • Rao's proofs for obtaining an unbiased estimator *2 2 a for a are not given m detail. Therefore we shall state and prove the following theorem which will be used to develop an expression for SSE^( weighted ) . The notation A" will be used to denote any generalized inverse of a matrix A, such that AA~A = A. 2 Theorem 4.1 . Let Y ~ (Xg, a G), where G is singular, then a^ = f""'-(Y xe)'T~(y Xg) 2 (i) is an unbiased estimate of a , and (ii) is unique with probability one, and (iii) is a scalar multiple of a central chi-square variable with f degrees of freedom of the form 2 2 (a /f )Xf if Y has the multivariate normal distribution. The vector Y is of order Nxl, 3 is a px 1 vector of unknown regression coefficients, X is an Nxp matrix of known constants, G is an NxN positive semi-definite matrix of known constants, T = G + XX' , 3 is any solution to X'T~xe = X'T~Y, that is, 3 = {X'T~X)~X'T~Y, and f = rank(G:X) rank(X).

PAGE 123

115 The proofs of parts (i), (ii) and (iii) of Theorem 4.1 are given in Appendices 9, 10, and 11, respectively. Lemma 4.1 which is used in the proof of Theorem 4.1 is stated and proved in Appendix 8. The results of Theorem 4.1 can now be applied to our problem of finding an expression for SSE^^( weighted ) . We define SSE^( weighted ) as SSE^(weighted) = W[T" T-X^(X^T-X^)-X^;,T-] W (4.10) where Tq = S q + X^X^. Writing SSE^(weighted ) in Eq. (4.10) as SSE^( weighted) = (W X^ej^)'T~(W X^3-,^), from Theorem 4.1 we see that if the true model is of the form in Eq. 2 (4.3) then SSE^( weighted )/a has a central chi-square distribution with f = rank(z :X^) rank(X^) = N g r degrees of freedom. However, if the true model is of the form in Eq. (4.4), then SSE^( weighted )/a^ has a noncentral chi-square distribution with N g r degrees of freedom and noncentrality parameter n*, where n* = (l/2a2)e.X^^[TQ TqX„(X^TqX^) X^Tq]X2^02 The distribution of SSE^( weighted )/a under model (4.4) is verified by the following theorem.

PAGE 124

116 Theorem 4.2 . Let Y ~ N^(X3 + X202' a^G), G singular, ^22 2 * 2 —1 * — then ta /a ~ Xf ^ where a =f (Y-XB)'T(Y-XB), f = rank(G:X) rank(X), T = G + XX', and \ = (l/2a^)0^X^[T~ T~X(X'T~X)~X'T~]X23 2The proof of Theorem 4.2 is given in Appendix 12. 4.5 Equivalence of SSE^ and SSE^^( weighted ) In this section we shall show that our expression for SSEy^ ( weighted ) in Eq. (4.10) is equal to Shillington ' s unweighted SSE^ in Eq. (4.6). Thus the complex calculations required for evaluating SSEy^(weighted ) can be avoided by calculating the simpler form SSEy^. Zyskind (1967) investigated conditions under which ordinary least squares estimators are BLUE (best linear unbiased estimators) even though Y in the model Y = X3 + E, where E(e) = 0, does not have variance2 covariance matrix equal to a I . Zyskind assumes that 2 var(Y) = a V, where V is non-negative and possibly singular, and then states and proves the following necessary and sufficient condition for ordinary least squares estimators to be BLUE. Theorem 1 (Zyskind, 1967) . A necessary and sufficient condition for all simple least squares linear estimators to be also best linear unbiased estimators of the

PAGE 125

117 corresponding estimable parametric function X'g in the linear model Y = Xg + e, E(e) = 0, E ( e e ' ) = a^V, where V is a symmetric non-negative matrix and X is of rank r, is that there exist a subset of r orthogonal eigenvectors of V which forms a basis for the column space of the matrix X. In a second theorem, Zyskind (1967) gives several other necessary and sufficient conditions for ordinary least squares estimators to be BLUE. These conditions are shown to be equivalent to the condition in Theorem 1 (Zyskind, 1967). The fifth of these conditions in Zyskind 's second theorem is that VP = PV, where P = X(X'X)~X'. Applying condition 5 of Theorem 2 (Zyskind, 1967) to our problem of regressing W on X^ we have ^ = ^0 and p = x^(x„'V ^ and therefore

PAGE 126

118 ^^ = ^o^w(Vw^ ^ ^w^Vw^ ^ ' since E^X^ = S^E^X = E^X = X^. It follows that VP = X^(X^„) X^Eq = PV. Therefore by Theorem 2 (Zyskind, 1967) the ordinary least squares solutions from regressing W on X^ are BLUE estimators, and thus are equivalant to the solutions obtained from weighted least squares. We conclude therefore that SSE^ = SSE|^( weighted ) . 4,6 The Test Statistic As stated in Section 4.1, Shillington (1979) proposed that a fitted model be tested for lack of fit by using the F ratio MSB„ F = ^ MSE^ given in Eq. (4.2). In this section we shall verify that Shillington ' s F ratio possesses a central F distribution when the true model is of the form in Eq. (4.3), and possesses a doubly noncentral F distribution when the true

PAGE 127

119 model is of the form in Eq. (4.4). This information on the distribution of the F ratio will be needed in Section 4.7, where the power of the test is discussed. Additionally, we shall give the form of the expected values of both the numerator, MSEg, and the denominator, MSE^, of the F ratio in Eq. (4.2). These expected values will aid us in developing a procedure for calculating the power of the test, since they will be used to determine whether the test is upper tailed or lower tailed. In developing the distribution of the F ratio in Eq. (4.2), we shall show that SSE^/a'^ and SSE„/a^ are Statistically independent. In this pursuit, let us use the expression for SSEg in Eq. (4.5) and the fact that Y^ = MY to express SSEg as SSEg = Y'M'[Gq^Gq^X^(X^Gq^X^)"^X^Gq^]MY. Also, using the expression for SSE„ in Eq. (4.6) (which is allowed because we showed in Section 4.5 that the correct form, SSE^^(weighted) , is equal to SSEy^) and using the fact that W = IqY, we can express SSE^ as 2^^W = r^ot^N Ww)"^JS^ By Theorem 4 (Searle, 1971, p. 59), to show that 2 9 SSEg/a and SSE^/a are statistically independent, it suffices to show that the matrix product

PAGE 128

120 is equal to the zero matrix. This is seen to be true since MEq = 0, and therefore SSE^/a and SSE^/a^ are independent. When the fitted model and the true model are both of the form in Eq. (4.3), then the F ratio in Eq. (4.2) possesses a central F distribution with g p and N g r degrees of freedom in the numerator and denominator, respectively. Furthermore, the numerator, MSEg, of the F ratio in Eq. (4.2) has expectation equal to E(MSE3) = [oV(g P)]EXg_p 2 = a . Similarly, under model (4.3), the expected value of the denominator, MSE^^, of the F ratio has expectation equal to E(MSE^) = [0V(N g r)]Ex^_g_^ _ 2 = a When the fitted model suffers from lack of fit and the true model is given by Eq. (4.4), the F ratio in Eq. (4.2) is a ratio of two statistically independent noncentral chisquare random variables, each divided by its respective

PAGE 129

121 degrees of freedom. Thus the F ratio in Eq. (4.2) possesses a doubly noncentral F distribution with g p and N g r degrees of freedom and noncentral ity parameters n , and n„, where n ^^ and n were given in Eqs. (4.7) and (4.8), respectively. The expected value of the numerator, MSEg, of the F ratio can be written as E(MSE3) = [aV(g P)]Ex^l^^^^ = a^ + t^C^^^/iq p) where ^1 = X2^*tG-^ Go\(^C^A)"'^cGo'^^^2(4.11) Similarly under model (4.4), the expected value of the denominator, MSE^, of the F ratio can be written as E(MSE^) = [aV(N g r)]Ex^,? N-g-r ,Il2 = a + 0^C232/(N g r) where S = ^2^0f^N ^W^^wV ^^^0^2(4-12)

PAGE 130

122 4.7 The Testing Procedure and its Power As discussed in Section 4.1, Shillington (1979) suggested that lack of fit of the fitted model be inferred when the value of the F statistic in Eq. (4.2) exceeds F ^, . The test, however, is not always upper a;g-p,N-g-r j t-ttailed, and in fact can be lower tailed. The test is considered lower tailed when, because of lack of fit, the expected value of the numerator of the F ratio is less than the expected value of the denominator of the F ratio. We suggest that lack of fit be tested with an upper tailed test using the F ratio F = MSEg/MSE^ when the matrix D, which is defined as C C 1 2 D = — -J. = (4.13) g-p N-g-r is found to be positive definite (which can only occur when Ci is positive definite, by Theorem 3.1 in Appendix 4). The matrices C-, and C2 in Eq. (4.13) are defined in Eqs. (4.11) and (4.12), respectively. An upper tailed test is appropriate v;hen the matrix D is positive definite because no matter what the value of g is, the expected value of the numerator, MSEg, of the F ratio will be greater than the expected value of the denominator, MSE^, of the F ratio. However, there may be cases where D is negative definite (which can only occur when C2 is positive definite), and in this case lack of fit should be tested with a lower tailed

PAGE 131

123 rejection region. If D is indefinite, then the F test for lack of fit may be upper tailed, lower tailed, or lack of fit may not be testable depending upon the value of g . In those cases where D is indefinite, it is helpful to write the quantity S^D6_2^ which represents the difference between the expected value of the numerator and the expected value of the denominator of the F ratio, F = MSEg/MSE^, as 6^6.2 = §.2["i*"2-"3l diag[r^, T^ = 0, r 3 ] [U^ :U2 103 ] • g . = ^^U^T^U[^^ + ^F3r3U^i2' where U-^, U2, and U3 are matrices whose columns are orthonormal eigenvectors of D, and r,, r ^, and r^ are diagonal matrices whose elements are the positive, zero, and negative eigenvalues of D, respectively. Lack of fit is testable with an upper tailed test if B is in the column space of [Uj^:U2], but not entirely in the column space of U2, since then 6^062 is positive. Similarly, lack of fit is testable with a lower tailed test if g is in the column space of ["2:03] , but not entirely in the column space of U2, since then S^D6 2 is negative. If 62 is in the column space of U2f then lack of fit cannot be tested, since -2^-2 ^^"-'-'^ equal zero.

PAGE 132

124 We define g__ to be in the column space of U2 if the matrix LIL has a zero eigenvalue, where L = [g :U ]. Similarly, letting L^ = [^^:U^:U^] and L^ = [025U2:U2], e is in the column space of [u,:U2] if L'L„ has a zero eigenvalue, and g is in the column space of [u :U ] if L^L^ has a zero eigenvalue. When D is positive definite or D is indefinite but 8 is upper tailed testable, then the F test for lack of fit which makes use of the F ratio F = MSEg/MSE^ is a test of the hypotheses (see Theorem 3.2, Appendix 5) %'• ^^="2 = "a* "j^/Cg p) n2/(N g r) > 0. When D is negative definite or D is indefinite but g is lower tailed testable, then the F test tests H„: n, = n^ = 12 H,: n,/(g p) n,/(N g r) < 0, In the case where D is indefinite and g is in the column space of U2» then no hypotheses concerning lack of fit of the fitted model can be tested. When the test is upper tailed, the power of the F test for lack of fit is defined as

PAGE 133

125 Power = p{f" „ „ „ > F ^^ } (4.14) where F „ is the upper 100a percentage point of the a;g-p,N-g-r central F distribution with g p and N g r degrees of freedom. In the case of a lower tailed test, the power of the test is defined as Power = P{F" „ „ „ < F, „ }. (4.15) ^ g-p,N-g-r;ni ,n2 1-a ;g-p,N-g-r ^ 4.8 Selection of Near Neighbor Groupings In the preceding sections of this chapter, we have discussed a near neighbor procedure which uses the F ratio F = MSEg/MSE^ to test a fitted model for lack of fit. In this section we shall investigate the effect that different groupings of response observations into near neighbor cells have on the testing procedure and its power. From equations (4.14) and (4.15) in the previous section it is evident that the power of the F test for lack of fit, which makes use of the F ratio F = MSE3/MSE^, depends on the values of the numerator and denominator noncentrality parameters, n, and II „. Assuming the numerator and denominator degrees of freedom are fixed, and the test is upper tailed, then the power is an increasing function of increasing values of II and is a decreasing function of increasing values of II (see Appendix 1). When the test is lower tailed, the

PAGE 134

126 power is an increasing function of n _ and is a decreasing function of n . Since both the numerator and denominator noncentrality parameters, n and U , are functions of the groupings of the data points into near neighbor cells (as are the numerator and denominator degrees of freedom), we would like to investigate the effect of the number and composition of cells on the power of the F test. Intuitively, it would seem that homogeneous near neighbor cells would minimize the 2 bias inherent in estimating a with MSE^^, and thus would minimize R and maximize the power of an upper tailed test. However, any grouping of the data points would also influence the numerator noncentrality parameter and the numerator and denominator degrees of freedom. Therefore while a grouping of data points into homogenous cells might decrease II „ and thus apparently increase the power of an upper tailed test, the result of the grouping on the power also depends on how the degrees of freedom, g p and N g r, and the numerator noncentrality parameter, IT,, are affected. We will attempt to find homogeneous cells of near neighbor points by using an iterative partitioning clustering algorithm. Two examples will be presented. The first example makes use of the stack loss data presented by Daniel and Wood (1971) and later analyzed by Shillington (1979). The second example involves data from a mixture experiment discussed by Piepel (1981).

PAGE 135

127 Our objective is to investigate the effect of forming homogeneous cells of near neighbors on the F test for lack of fit which makes use of the test statistic F = MSEg/MSE^. It is hoped that such homogeneous groupings will increase the power of the F test (assuming that the rejection region is upper tailed) by decreasing n for a fixed number of near neighbor cells (and thus fixed values for the degrees of freedom). The effect that homogeneous grouping has on n , is not clear, but is of interest, since the magnitude of n , also affects the power of the test. Additionally, we will vary the number of cells of near neighbors in an attempt to determine how this affects the power of the test, since the number of cells affects both the noncentrality parameters and the degrees of freedom. The algorithm used for grouping the data points into homogeneous near neighbor cells can be described as an iterative partitioning type of cluster analysis. The computations involved were accomplished using the RELOC procedure available in the CLUSTAN IC computer package (Wishart, 1975). All computations were performed using data points whose coordinates were standardized by subtracting off sample means and dividing by sample standard deviations. Initially, k clusters (near neighbor cells) of the N data points in the factor space are arbitrarily defined. Then the Euclidean distance between each point and the centroid (average vector value) of each of the k clusters is determined. If a point is found to be closer to the

PAGE 136

128 centroid of one of the other k 1 clusters than to the centroid of the cluster in which it is currently classified, then the point is reclassified into that nearest cluster (cell). The centroids of the k clusters are then recalculated, taking into account any reclassified point. The entire set of N points is scanned repeatedly in this manner until no reclassification occurs. This method of assigning points to clusters will be referred to as iterative relocation. In the second stage of the algorithm, two of the k clusters arrived at by the iterative relocation procedure are fused, resulting in k 1 clusters. The two clusters to be fused are selected as those which when fused produce the k 1 clusters with minimum "error sum of squares." The error sum of squares is defined as the sum of squared Euclidean distances between every point and the centroid point of the cluster to which it belongs. After k 1 clusters are determined using the error sum of squares criterion, iterative relocation is applied to the k 1 groups in an effort to improve the clusterings. This alternation of fusion and iterative relocation continues for k 2 clusters, k 3 clusters, ..., 2 clusters, or until a specified minimum number of clusters is reached. The question of determining an "optimal" number of clusters is not addressed by this procedure.

PAGE 137

129 4.8.1 Example 1 — Stack Loss Data The first example we investigate is the 21 observation stack loss data of Daniel and Wood (1971), which was analyzed by Shillington (1979). The data (see Table 8) consist of the values of three factors, x-^ (air flow), Xj (cooling water inlet temperature), and X3 (acid concentration) along with the values of a response variable, Y (stack loss). A first order regression equation of the form E(y) = 3o + B^x^ + ,^x^ -. 33X3 is fitted using 17 of the original 21 observations (Shillington discarded 4 of the original 21 observations as outliers). We assume the true model to have the form E(Y) = 3o + B^XjL + 32X2 + B3X3 + 3;l1^J ^ ^22^2 "^ ^33^3 and thus contains P2 = 3 terms in addition to the p = 4 terms in the fitted model. We wish to investigate the capability of the F test MSE F = B M^ in detecting lack of fit of the fitted model. We first consider the use of the F statistic with the six cells of near neighbors used by Shillington (1979), which is the same near neighbor grouping suggested by Daniel

PAGE 138

130 and Wood (1971). This 6 cell grouping (see Table 8 under the column heading "6**") is found to yield a matrix D = [Cj^/(g p)] [C2/(N g r)] which is indefinite, since the eigenvalues of D have the values 12110/ -7, and -1415 (see Table 9). Thus the test is not upper tailed, since D must be positive definite for an upper tailed test to exist for all values of ^j, where in this example, ^2 = ^^11' ^22' ^33^'* When the 6 cell grouping of near neighbors generated by the iterative partitioning clustering algorithm (see Table 8 under the column heading "6") is used, the values of the eigenvalues of D are 49090, 379, and -43, and the test is still not upper tailed for all values of 3_. We then use the iterative partitioning clustering algorithm to determine homogeneous cell groupings for 5, 7, 8, 9, 10, 11, and 12 cells. The matrix D is found to be indefinite for the groupings into 5, 7, or 8 cells, but D is positive definite for 9, 10, 11, or 12 cells of near neighbor groupings. Thus, no matter what the value of g_2' lack of fit can be tested with an upper tailed test using the 9, 10, 11, or 12 cell groupings of near neighbors. The value of F = MSEb/MSE^ was calculated using the matrix procedure from the 1979 version of SAS. None of the near neighbor groupings provided evidence of lack of fit, and thus we cannot conclude that there is lack of fit when the fitted model is E(Y) = e + 3,x + ^j^o "*" ^3^3

PAGE 139

131 CM CO

PAGE 140

(N 00 LD CN r~-

PAGE 141

133 and the true model contains only pure quadratic terms in addition to the first degree terms. For the groupings of near neighbors into cells which provide an upper tailed test (9, 10, 11, or 12 cells), the power of the upper tailed test can be approximated if n , and II can be specified. This approximate power can be calculated using an approximation to the doubly noncentral F distribution given in Johnson and Kotz (1970, p. 197), which is described in Appendix 6 of this dissertation. Thus we calculate an approximation for P( F" > F I ^ g-p,N-g-r;ni ,n2 a;g-p,N-g-rJ * In order to compare the power of the upper tailed F test for 9, 10, 11, and 12 cells, we will assume arbitrarily that the true value of the parameter vector B^^ ^2 "" ^^11' ^22' ^33^' ^ (.044, .329, .033)' which is arrived at by taking e„ = lOg^ where 3 is the least squares estimate of g„ calculated from the data. Furthermore, 2 taking a =1.6 (since the residual mean square value from fitting the "true" second degree model is MSE = 1.6) we 2 calculate the values for n , = 3AC,3-/2a and II2 = 0^020 2/20^ for each of the 9, 10, 11, and 12 cell groupings. The calculated values of H and n , as well as the approximate power values for each of the four F tests (calculated using the approximation to F" from Johnson and Kotz (1970, p. 197)) are presented in Table 9. The power is

PAGE 142

134 quite high {> .96) for 9, 10, or 11 cell groupings, but drops off to .79 for the 12 cell grouping. This drop in power seems to be due to the effect of having only 3 degrees of freedom in the denominator of the F ratio. In summary, this example illustrates that the F test for lack of fit that makes use of the statistic F = MSEg/MSE^ is upper tailed only for certain groupings of the design points into near neighbor cells. For the near neighbor cell groupings that provide an upper tailed test, the power is generally high for the values of 6_ and 2 a that we selected, but decreases slightly as we move from 9 to 10 to 11 cells and decreases more severely as we move from 11 to 12 cells. This more severe decrease in power is due to the decrease to only 3 denominator degrees of freedom. 4.8.2 Example 2 — Glass Leaching Data The second example we investigate is one in which the leachability, Y, of glass is assumed to be a function of the proportions of eleven chemicals of which the glass is composed (Piepel, 1981). A first order Scheffe polynomial model was fitted to the common logarithms of the leachability values, that is, the fitted model is of the form 11 E(log Y) = Z g.x.

PAGE 143

135 The experimental design coordinates and the values of the 44 data observations are presented in Table 10. For illustrative purposes, the true model is assumed to contain the P2 = 8 second order cross product terras, ^es^'e^'s' ^3,ii''3''ii' ^79''7''9' Se'^s^'e' ^7,ii''7''ii' g-_x.x_, 6in^c^in' ^^^ ^cQ^c^^Q ^^ addition to the p = 11 15 1 5 6,10 6 10 59 5 9 first order terms in the fitted model. The 19-term model is the final fitted model proposed by Piepel and serves as our true model. Piepel (1981) suggested that the four sets of observations (see Table 10) (a) 14 and 15 (b) 18, 19, and 20 (c) 25, 26, and 27 (d) 39, 40, and 41 were intended to constitute four cells of replicate observations for use in estimating pure experimental error. However, the settings of the mixture components were not well controlled, so that each of the four cells contained near neighbors rather than replicates. By defining each of the remaining 33 data points as 33 cells containing one observation each, the 44 data observations are partitioned into 37 cells. If we choose to use the 37 cells to test the fitted model for lack of fit, with the test statistic F = MSEg/MSE^, we find that there are no degrees of freedom for MSE^ so that the F statistic cannot be calculated in this case. However, the F statistic for

PAGE 144

136 *

PAGE 145

0'^o^o->^(^Ln>x)rnLno in CN n cnr^r^r^'^cN 137 ^ocNor--ooo CNOCMOCNOOO oooooooo LD 1.D VO —( m ^ ITl CN CN CM CN (N CN CN O O O O O O O >^ o —( ooooounoo OOOOOCNOf^ oooooooo o o o o o o o ro O O O O O O o o o o o o o ^OrOO-^OOOOOOOOrH-* mOrooroOOOOOOOPOrOCN OOOOOOOOOOOOOOO O'^r^-'^'r^ootN oo or^r--r-r-~oor~oo oooooooooo o o o o r^ o o o o r^ o o o o o OOOOOOOOOCTlrHOrHOOm OLDvCOOOOLDinvDVOVOOOVO OOOOOOOOOOOOOOO c o a; E u o Vl Ou CO CO >1 .—I c < v£3^TjCT\Ln'a<":rv^^on ^nHfH-HM^-HOOM^rH-H^^ CO CO «3 0) 3 C •H 4J c o o I I o JQ m E-< ooorooomorooooooo oooocx3oocx)0 ooooo oooo oooooooo oooo ooo OOOO'*C0V000^r^r^'«j'-^i^vocNmi^ Ml— lOMOOOrHrHOOO-H— lO r^cyiTO^'^'^rocNjTrcNfHooo*^ MCNincNr-covoronmrorOi— (rocT( T3 T^ TJ OrHCNrOTl'Ln^Dt^(X5cyiO'-(fNn'<* CO U-l 0) O 4J (0 to O CO -H O -H rH a. a* c O (U 0) c Qj a; u -u c CP •H y-i O; O CO 0) Oi 3 (0 o CO en 3 ij (0 (0 ^ (tj

PAGE 146

138 lack of fit can be calculated using from 12 to 33 cells of near neighbors, and thus we refer to the iterative partitioning clustering algorithm discussed earlier in this section to generate near neighbor groupings of 15, 20, 25, and 30 cells (see Table 11). The clusterings of observations into 15, 20, or 25 cells each yields a D matrix that is indefinite (see Table 12), so that the test is not upper tailed. The 30 cell clustering produces a positive definite D matrix, so that the test is upper tailed for all nonzero values of g_, where 3.2 = (egg, 33^3^3^, 3-79, 35g, Sy,!!' 3j^5, 3g^3^Q, 359)'. Taking 3.2 = (15.141, -112.429, -78.761, -78.275, 87.996, 13.356, -76.948, 34.721)', which is the least squares 2 estimate of 3^ from the data, taking a = .008 (which is MSEp,jj.g with seven degrees of freedom from Piepel's analysis of the data), and using the approximation of Johnson and Kotz (1970, p. 197) to approximate Pf F" > F 1 ^ 19,4;ni ,Jl2 .05;19,4^ where n, = 9.79, n^ = 0.08 and F ^_ ,^ ^ = 5.81, we find 1 2 . 05;19,4 that (using 30 near neighbor cells) a value for the power of the F test is .10. The power increases as the magnitudes of the elements of 3 are increased, so for example if all the elements of 3 -, above are doubled, then n , = 39.16, H _ = 0.32, and the approximate power is .25. If the elements of 3_~ above are each multiplied by 5, then

PAGE 147

139 Table 11. Near Neighbor Cells for Glass Leaching Data. Membership in Near Observation Neighbor cells* Number 15 20 25 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 6 14 11 11 11 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 6 14 15 15 15 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 14 15 16 16 16 3 5 1 2 3 4 5 6 7 8 9 10 10 11 12 13 13 14 15 16 16 16 3 17 Membership in Near Observation Neighbor Cells Number 15 20 25 30 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 1 3 15 15 15 4 12 10 8 1 10 2 13 11 8 9 7 7 7 10 14 2 1 3 16 16 16 4 12 17 8 14 18 2 19 11 8 9 7 7 7 17 20 2 1 3 17 17 17 18 12 19 8 15 20 2 21 22 23 9 7 7 7 19 24 25 18 19 20 20 20 21 12 22 23 15 24 2 25 26 27 28 7 7 7 22 29 30 *Cell groupings generated by an iterative partitioning cluster analysis using the CLUSTAN computer package. Numbers in the table refer to cell membership.

PAGE 148

140 o

PAGE 149

141 vD vD vc r^ r~ CO I I I I I I o o o o o o X VD X X X X r^ 00 00 00 X o ro CN CTi 00 00 I I o o X 00 X IT) 00 CN K in in • o CN o^ o^ 00 o vovovDr--r~oooor-~ I I I I I I I I oooooooo XXX C) o\ o CN ro X in X (N X X X 00 m V£> vD r~ r~r~00 I I I I I I o o o o o o 00 r^ I I o o X X in 00 o m X n XXX in rH ^ ^ ^ 00 X m X CN in CN
PAGE 150

142 n j^ = 244.75, n = 2.00 and the approximate power is .83. From the entry in Table 12, we see that the calculated F value of 20.55 with the 30 cell clustering exceeds ^ ni;.iQ A ~ 5.81, and we conclude that the fitted first degree model is inadequate. 4.9 Discussion When a designed experiment includes replicated points, the adequacy of a fitted model can be tested by comparing the portion of the residual sum of squares due to lack of fit to a second portion due to pure error from the replicates. The test statistic is an F ratio of the mean square due to lack of fit to the mean square due to pure error, and lack of fit is inferred when the calculated value of this ratio is large (Draper and Smith, 1981, p. 120). When replicate points do not exist, lack of fit can be tested using near neighbor observations with the test statisic F = MSEg/MSE^. This F ratio has been shown to possess a central F distribution when the fitted model is adequate, and a doubly noncentral F distribution when the fitted model suffers from lack of fit. When the fitted model is adequate, the expected values of both MSEg and MSE^ are equal to a^, so that the ratio E [MSEg]/E[MSE^] equals unity. However, when lack of fit is present, both MSEg and MSE^^ are biased estimates of a^, and we compare the magnitudes of the biases of these estimates (which are functions of the noncentrality parameters and degrees of freedom of the doubly noncentral F distribution)

PAGE 151

143 in the F test. The test has an upper tailed rejection region if the bias corresponding to MSEg exceeds the bias corresponding to MSE„. The rejection region is lower tailed if the bias corresponding to MSE^ exceeds the bias corresponding to MSEg. In other words, the test is upper tailed if the matrix D (see Eq. 4.13) is positive definite, and the test is lower tailed if D is negative definite. If D is indefinite then the test may be upper tailed, lower tailed or still yet lack of fit may not be testable depending upon the value of 3 . In two examples an iterative partitioning clustering algorithm is used to assign the data points to a preselected number of near neighbor cells. When the number of cells is low, the matrix D is found to be indefinite, so that the F test is not strictly upper tailed or lower tailed. However, by increasing the number of cells, it is possible in both examples to produce a positive definite matrix D, so that the test is upper tailed. Increasing the number of cells not only produces an upper tailed test, but also affects the values of the parameters of the doubly noncentral F distribution. As the number of cells is increased (moving from left to right in Tables 9 and 12) we see that the smallest eigenvalue of C^ increases and that the largest eigenvalue of C2 decreases. Therefore a lower bound for n, , ^-2^2^min , , -T— ^ '^l 2a

PAGE 152

144 increases as the number of cells increases (where ib . ^min denotes the smallest eigenvalue of Cj^). In addition, an upper bound for n 2 ' 8lB„p TT c -2-2 max ^2 ^ _ 2 2a decreases as the number of cells increases (where p„=,„ denotes the largest eigenvalue of C2). Finally, as the number of cells increases (moving from left to right in Tables 9 and 12), the numerator degrees of freedom, g p, increase and the denominator degrees of freedom, N g r, decrease. Since the parameters of the doubly noncentral F distribution change as the number of cells changes, the power of the F test can be affected. For the stack loss data example, we see in Table 9 that the power of the upper tailed test decreases as we move from 9 to 10 to 11 to 12 cells. An area for future study can be a further investigation of the effect of the number and composition of near neighbor cells on the power of the F test which makes use of F = MSEq/MSE^. This investigation would involve the effect of near neighbor cell selections on the parameters II-,, II-/ g p, and N g r of the doubly noncentral F distribution. It would be desirable to develop a method (perhaps an alternative to the iterative partitioning clustering algorithm) which could be used to select the number and composition of cells so as to maximize the power of the F test.

PAGE 153

CHAPTER FIVE CONCLUSIONS AND RECOMMENDATIONS Two general methods for testing a linear model fitted in a mixture space for lack of fit have been investigated in this dissertation. The first method makes use of response values observed at check points while the second method makes use of response values observed at design points which are near neighbors in the factor space. In Chapter Two we discussed the work of several authors (Scheffe (1958), Gorman and Hinman (1962), Kurotori (1966), and Snee (1971)) for testing lack of fit which centered on measuring bias inherent in the fitted model when estimating the response at check points. Only the method suggested by Scheffe (1958) was an exact test. In Chapter Three, a method for selecting check points that maximizes the power of Scheffe 's F test was devised. When replicate response observations were available, so that the experimental error " 2 variance could be estimated by a ^ from the replicates, we ^ ext ^ saw that the power of this upper tailed F test was maximized by selecting check points that maximize (or approximately maximize) the noncentrality parameter A, of the noncentral F distribution. When the matrix K^ (where x, = 3AA,3„/2a ) was found to be positive semi -definite it was determined that only a subset of possible values of the g parameter vector could be detected as contributing to lack of fit. 145

PAGE 154

146 When an estimate of the experimental error variance was not available from replicates, an extension of Scheffe's F test for lack of fit which replaced n with MSE (the ext residual mean square error from the fitted model) in the denominator was developed. We found that to maximize the power of the test it was necessary to select check points to maximize (or approximately maximize) the numerator noncentrality parameter, X , of the doubly noncentral F distribution when the test was upper tailed. When the test was lower tailed, we sought check point locations that minimized (or approximately minimized) X . A criterion was developed for determining whether the test was upper tailed or lower tailed by comparing the expected values of the numerator and denominator of the F ratio when the fitted model was inadequate. Finally, we discovered cases where, for some values of g , lack of fit could not be tested. An alternative to the check points method for testing lack of fit in a fitted model is a procedure that involves measuring the bias that is present in estimates of the response at the design points (the number of design points must exceed the number of terras in the model). When replicate observations are available, the well known procedure in which the test statistic is a ratio of the lack of fit mean square to the pure error mean square can be used to test for lack of fit (see Draper and Smith, 1981, p. 120). When replicate observations are not available, several techniques which make use of near neighbor

PAGE 155

147 observations in place of replicates for testing lack of fit have been proposed in the literature (see Green (1971), Daniel and Wood (1971), and Shillington (1979)). Additionally, it has been suggested by Draper and Smith (1981, p. 42) that lack of fit can be tested by using near neighbor observations as substitutes for replicate observations in the usual lack of fit, pure error F ratio. However, the exact distributions of the test statistics proposed by Daniel and Wood (1971) and Draper and Smith (1981, p. 42) have not been defined, and Green's (1971) procedure requires an inordinately large number of observations. Thus because of these reasons we chose Shillington 's (1979) procedure to study in greater detail in Chapter Four. In Chapter Four the distributional properties of Shillington 's test statistic were developed, and a method based on an iterative partitioning clustering algorithm for defining groups of near neighbor observations was proposed. It was shown that the power of Shillington ' s test depends on the parameters of the doubly noncentral F distribution, and that the manner in which observations are grouped as near neighbors can alter the values of the parameters of the doubly noncentral F distribution and thus affect the power of the test. We found that increasing the number of near neighbor cells so that individual cells become more compact produced an upper tailed F test in the

PAGE 156

148 two examples studied, but that there are many other cases where the test will not be upper tailed. Now that we have briefly summarized our findings from investigating the check point and near neighbor methods of testing lack of fit in a mixture model, a logical question is, "Which of the two methods is better?" It was not our original intent to address this question in this dissertation, but an interesting result that has been discovered in the latter stages of our investigations is as follows: Under certain circumstances, the check point method for testing lack of fit is equivalent to the usual method which partitions the residual sum of squares into sums of squares due to lack of fit and due to pure error (which was shown in Chapter Four to be a special case of the near neighbor method). Because we have not found a derivation of the equality of these methods in the literature, we shall show it here. In Chapter Three, check points were used to test lack of fit in a fitted model of the form E(Y) = Xg , . With k check points, the test statistic was of the form (see Eq. (3.3)) d'v/d/k F = *2 ^'ext ^2 2 where a ^ is an external estimate of a which can be ext

PAGE 157

149 calculated from replicates, if they exist. The vector d in the F ratio was defined to be a vector of differences between observed and predicted response values at the k check points having the form d = Y* X*(X'X)~'^X' Y, where Y* is the kx 1 vector of observed response values at the k check points and X* is the corresponding settings of 2 the model terms at the check points. The matrix a V^ was defined as the variance-covariance matrix of d where Vq has the form V^ = I, + X*(X'X) -"-x*' . Ok It can be shown (see ^pendix 13) that if we define the vector Y, as -A ^A = Y Y* observations at the check points and similarly define the matrix X^ as ^A = X X* design P2iD^_§2^^i'^5£ check point settingi so that the original design points as well as the check points are all taken at once as design points in regressing Yon X-, then -A A

PAGE 158

150 SSE^ = X^f^N+k) ^a(^A^a)~^^A^Xa = ^'V-^d + SSE. (5.1) Thus, the residual sum of squares, SSE^, from the analysis of the fitted model when both the original design points and the check points are used to fit the model is equal to the sum of the quadratic form, d'V~ d, used in the numerator of the check point F test and the residual sum of squares, SSE, from the analysis of the fitted model using data collected only from the original design points. If we perform the usual partitioning of SSE^ into a lack of fit sum of squares, SSj^Qp/^x, and a pure error sum of squares due to replicates, SSEp^_jj.g( j^j , then from Eq. (5.1) we can write SSlOF(A) -^ SSEp^re(A) = ^'^o'^+ S^^' ^^.2) Thus from Eq. (5.2), when SSE-^^j-^^^j is equal to SSE, then SSlof(a) becomes equal to d'v" d so that the check point F -1 * 2 ratio, F = (d'V„ d/k)/a ^, and the usual lack of fit F ^ext ratio, F = MS,^„,,,/MSE ,^,, are equivalent. We now LOF(A)'^ pure (A) present an example to illustrate the result in Eq. (5.2). Let us fit a second degree Scheffe polynomial model to the following hypothetical or artificial response observations collected at the six points of the {3,2} simplex lattice design:

PAGE 159

151 Y = 2350 2370 2450 2430 2650 2670 2400 2420 2750 2730 2950 2970 X = 1

PAGE 160

152 If we use all of the observed response values to fit the second order Scheffe polynomial, then the model is Y = 2360. 7Xj^ + 2437.7x2 + 2661.7x3 + 183.3x^X2 + 1071.3x-|^X3 + 1785.3X2X3 and the residual sum of squares is SSE^ = 25746.7. This residual sum of squares can be partitioned into SSlof(A) = 24546.7 and SSEpuj^e(^) = 1200. The F ratio for testing lack of fit is calculated to be F = MSLOF(A)/MSEpure(A) = [ 24546 . 7/3] / [1200/6] = 40.91, which is identical to the previously calculated F value. In the above example we note that SSEp^j.g/^j is equal to SSE (SSE = SSEpy^g(^p so that SSlof(A) ^^ equal to d'V„ d. Since both the check point F ratio and the usual lack of fit F ratio have produced the same value, F = 40.91, we conclude that the two methods for testing lack of fit in the fitted model are equivalent. In order to put this dissertation in a better perspective, we now make some concluding remarks on the lack of fit testing procedures investigated, including possible drawbacks, extensions, and recommendations for future work. An aspect of our investigations that may raise some questions is that our methods are dependent on the specification of the form of the true model believed to be responsible for lack of fit in the fitted model. Requiring the form of the true model to be specified was necessary in order to be able to investigate the power of the testing procedures. There are situations, however, where a complete or true model can reasonably be specified. One example

PAGE 161

153 could be in fitting polynomial models, where the polynomial of one degree higher than the fitted model could be taken as the true model. We now mention two ways in which our results can be applied to more general situations than may be readily apparent from our previous discussions. First, we point out that all examples in Chapters Three and Four dealt with polynomial models. This type of model was selected because of its popularity and v/ide applicability, however, our methods can be applied not only to polynomial models but to any models which are linear in their parameters. Secondly, it was our intent in this dissertation to discuss methods for testing lack of fit in a mixture model, but the methods discussed can certainly be used not only in mixture problems but also in general response surface problems in which a linear model is fitted. This generalization is illustrated for the near neighbor approach to lack of fit testing through the stack loss example in Chapter Four. Topics for future research stemming from this dissertation were listed in the concluding paragraphs of Chapters Three and Four. One area suggested in Chapter Three was to investigate the effect of experimental design on the selection of check points and on the resulting power of the test. Perhaps a "minimum bias" design could be used for fitting the model, while lack of fit could be detected with "high bias" check points, but this in only speculation, and needs to be investigated.

PAGE 162

154 The fact that the check points method and the standard method that partitions the residual sum of squares into lack of fit and pure error portions were found, under a certain condition, to be equivalent suggests that selecting check points to maximize the power of the check point F test may in general be equivalent to choosing points to augment the original design. The augmented points would be chosen to maximize the power of the F test that partitions the residual sum of squares into lack of fit and pure error sums of squares. An investigation of the selection of optimal check points versus the selection of optimal augmented design points would be of interest. For the near neighbor test for lack of fit it was recommended in Chapter Four that other methods besides the iterative partitioning clustering algorithm might be considered for selecting groups of near neighbors. The effect of the number and composition of the groups selected on the power of the test through their effect on the parameters of the doubly noncentral F distribution could then be investigated. In view of the equivalence of the check point method and the method that partitions the residual sum of squares when replicates exist (see Eq. (5.2)), it would be of interest to investigate whether there is also some equivalence between Shillington ' s near neighbor F ratio and the check point F ratio, F = (d'V~ d/k)/MSE, to be used when an external estimate of a is not available. If the methods

PAGE 163

155 are not equivalent, perhaps one could be shown to be preferable to the other as judged by comparing the power of the two procedures in testing for lack of fit. Finally, the focus of this dissertation has been on testing lack of fit in linear models so that another area for future investigation can be the problem of testing lack of fit in models which are nonlinear in their parameters.

PAGE 164

APPENDIX 1 INFLUENCE OF X . ON p{ F" > F 1 Vi,V2'^l/^2 a;vi,V2 In this appendix we show that P{ F" > f } is an increasing function of X-^. Let X , ..., X , Y., ..., Y be independent N(0,1). i V 1 i V2 \ r ' Then F = (V /v )[(X + x|/2)2 ^ J, x2]/[(Y. + xy^)^ + E Y^] i=2 1 / i=2 ^ is distributed as F" where vi and v^ are the respective numerator and denominator degrees of freedom and XjL and X2 are the respective numerator and denominator noncentrality parameters (Scheffe, 1959, p. 412-413). Fixing the values of v-^, v 21 and X 2 we wish to show p{F" > , > f 1 is a strictly increasing function VlfV2;Xi,X2 a;vi,V2^ "^ ^ of X -1 , where F represents the upper 100a percentage Ja ;vi ,V2 r-rc^ point of the central F distribution with vi and V2 degrees of freedom. Let f(xl/2) ^ . , ,. . ^2 = P{ (v^/v^)! (X^ + xJ/2)^ + Z xJ]/[(Y^ + X^/2)2 ^ J, y2^ 1=2 i=2 156 a ;v^ ,^2

PAGE 165

157 1/2 then f(^/ ) niay be rewritten as f(xl/2) = p((x^ + xl/2)2 > u} = 1 P{(X^ + X^/2j2 , „| ^ ( Al . 1 ) where U = (v^/v^)[(Y^ . X ^^ ) 2 , _z^ yJJF^^^^^^^ ^E^ X^ . Note that the random variable U is independent of X^* If X |/ and XT'! denote any two values of Xy such that X^/2 ^ x|-/2, then we shall prove that for f(XJ-/2) defined as in (Al.l), f(^]^{^) < ^^^Yl^^' ^°^ f(x|/^) =1-7 g,^ (u)p(u)du where p(u) is the p.d.f. of U, and for any positive number, u', g,^ (u') denotes the ^ 1 1/2 2 conditional probability that (X^ + X^ ) < u', given U = u'. However, this conditional probability must be the same as the unconditional probability, since Xi and U are statistically independent. Thus g^'s (u') is the probability that the random ^1 1/2 variable X-j^ falls in an interval of half length u' 1/2 centered at -X, . Since X ~ N(0,1), this is a decreasing function of X ;/ . Therefore g,*2 (u*) g,^ (u') > for all u' > 0. Hence, f(xl/2) _ f(x]-/2) = J [-g^^^(u) + g^^^(u)]p(u)du < 0.

PAGE 166

158 Thus P{ F" , , > F } is a strictly increasing function of X -i . We note that this proof is a modification of the proof that Pf F" , , > F } is decreasing in X-, ^ Vi,V27Xi,X2 a;vi,V2' ^ (Scheffe, 1959, p. 136).

PAGE 167

APPENDIX 2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR GLOBAL OPTIMIZATION W. L. Price (1977) describes a conceptually simple random search procedure, called "a controlled random search procedure for global optimization," which is effective in searching for global minima of a function of n variables, with or without constraints. The procedure does not require the function to be dif ferentiable or the variables to be continuous. An initial search domain, V, is defined by specifying upper and lower bounds for each of the n variables, and a predetermined number, N, of trial points are chosen at random over V, consistent with any constraints. The function is evaluated at each of the N trial points and the position as well as the value of the function at each point are stored in an array. A' . At each iteration a new trial point, P, is selected randomly from a set of possible trial points whose positions are related to the configuration of the N points currently in storage. If P satisfies the constraints, the function is evaluated at P and the function value, fp, is compared with fj^, which is the greatest function value for the N points already in storage. If fp < f^ then M, the point in storage corresponding to f^, is replaced, in the array A', by P. If p fails to satisfy 159

PAGE 168

160 the constraints or if f > f then the trial is discarded and a new point is chosen from the potential trial set. As the algorithm proceeds, the set of N points in storage tend to cluster around minima. As Price states, "the probability that the points ultimately coverge onto the global minimum (minima) depends on the value of N, the complexity of the function, the nature of the constraints and the way in which the set of potential trial points is chosen. " Price notes that since the procedure is intended to find global minima, thoroughness of search is more important than speed of convergence, but if the procedure is to be more efficient than pure random search the probability of success (f < f ) at each iteration must be sufficiently p m high. His procedure reaches a compromise between the requirements of search and convergence by defining the set of potential trial points in terms of the configuration of the N points already in storage. At each iteration n + 1 distinct points, R, , R„, ..., R ,, are chosen at random -1 -2 -n+1 from the N (N > n) currently in storage and these constitute a simplex of points in n-space . The point R is arbitrarily chosen as the vertex of the simplex, and the next trial point, P, is taken as the image of the vertex with respect to the centroid, G, of the remaining n points. Thus P = 2G R ,. He notes that it is possible -n+1 to speed up covergence by selecting the vertex as the point R. , i = 1, 2, ..., n + 1, which has the largest

PAGE 169

161 function value of the points Rw R-/ ...» R but this would be detrimental to the thoroughness of the search. The version of Price's procedure used in the work in this dissertation was programmed in the FORTRAN language by Michael Conlon of the Center for Instructional and Research Computing Activities, the University of Florida, Gainesville, Florida. This version of Price's procedure selects new trial points using the suggested criterion P = 2G R , . The algorithm continues until an iteration -n+1 limit is reached or a desired tolerance between the minimum and maximum function values in storage is achieved. In our particular application, if P2 = 1 so that A-^ is a scalar, we wish to maximize A^ = (X* X*A)'Vq^(X* X*A), with respect to k check points, in order to maximize the power of an upper tailed test. For locating check points that maximize the power of a lower tailed test it is necessary to minimize A-j^. If p > 1 so that A-j^ is not a scalar, but is a P2XP2 matrix, then it will be necessary to maximize or minimize certain eigenvalues of A-,. All of these optimization problems can be handled by Price's procedure. Since the procedure finds minima, then to find maxima, we simply minimize the negative of the function under consideration. The restriction that the check points must be located within the experimental simplex

PAGE 170

162 (or a subregion of the simplex) is taken care of by specifying constraints in the program. To give a specific example, suppose we fit a second order canonical polynomial model in a three component mixture space, using a simplex centroid design. If we assume the true model is special cubic in the three components, then P2 = 1, and ^1 ^ ^^2 " X*A) •Vq-'-(X* X*A) is a scalar quantity. In order to locate a single check point that maximizes the power of an upper tailed test for lack of fit, we select the check point that maximizes Ai. Since the experimental region we wish to search is the entire two dimensional simplex, we define the check point as x*' = (x , x , X ), and in our program impose the constraints : and < X < 1, < X < 1. We then define x-, as x^ = 1 x, y.^, while requiring that < X < 1. Price's random search procedure is used to search the two-dimensional simplex for the point (xj^, X2) that maximizes K-^.

PAGE 171

163 Price suggests the use of N = 50 storage points for such a two-dimensional search, and we have generally found this to be adequate. For k > 1 check points to be located simultaneously in a three component system, the problem becomes one of searching in 2k dimensions. For the applications considered, N = 50k appears to be adequate. The only real problem encountered has been that of economics in that the procedure becomes costly in terms of computer time for these situations where the optimal value of the function is assumed by all points in a region. In these cases the algorithm searches in vain for points that will improve upon the functional values already in storage, which all lie in this optimum region. However, in other applications, the procedure converged quickly to an optimum (those that converged did so in 10,000 iterations or less, at a small cost in computational time).

PAGE 172

APPENDIX 3 _, STATISTICAL INDEPENDENCE OF d • V d /a AND SSE/a^ Let us write d'v" d as -1 * * * _i * * * d'V d = (Y Y )'V -"(Y Y ) Y 'Vq Y Y 'Vq Y Y 'Vq-'y + Y 'Vq-^Y * -1* * -1** '* -1** Y •Vq-'y 2 Y •Vq-'y + Y 'Vq-^Y . Now let us write SSE as SSE = Y'(I^ X(X'X)~^X')Y. Since Y and Y are independent, SSE is independent of * _l * * _i ^ * Y 'V» Y . Rewriting y 'V„ Y as * —1 * * * —1 * 1 '^0 " '^0^ -1' where b is the least squares estimator of g , we have * — 1"* * —1* —1 1 '^0 " '^0^ (X'X) X'Y. 164

PAGE 173

165 We now show that the second portion of d'V^ d is independent of SSE if and only if [v'-'-x (x'x) 'x'llijg x(x'x) -"-x'] = 0. Define COv[Y 'Vq-'"X (X'X) -""X-Y, Y'(I^ X(X'X) '"XMY] E[Y 'V '''X (X'X)~''"X'YY' (Ij^ X(X'X) '"X')Y] E[Y •Vq-'-X (X'X) '•X'Y] E[Y'(Ijj X(X'X) -"-X'Y] * -1 * -1 -1 E(Y ') E[Vq X (X'X) X'YY'dj^ X(X'X) X')Y] E(Y ') E[Vq''"X (X'X)~-'"X'Y] E[Y'(Ij^ X(X'X) '•X'Y] E(Y ') [cov(Vq"'"X (X'X)~'''X'Y, Y'(Ij^ X(X'X)~'''X'Y)] = 0, if Vq X (X'X) '"X'Y is independent of Y'(I X( X' X) ""'"X ' ) Y. This occurs if and only if. [Vq-'-X*(X'X)"-'-X' ] [Ij^ X(X'X)"-'-X'] = 0, see Searle (1971) p. 59, Theorem 3. Now,

PAGE 174

166 [Vq^X (X'X)"-'-X'] [Ij^ --X(X'X) -^x'] = Vq-'-x (x'x)"-^x' Vq-^x*(x'x) 'x'x(x'x)"-'-x' = 0. Therefore SSE is independent of the second portion of d'V~ d. Now we must show that SSE is independent of the third portion of d'V~ d. Write Y*'V~-'-Y* as Y 'Vq Y = (X bj^)'VQ-'x b^ = Y'x(x'x)"-'-x •Vq-'-x (X'X)~-'-X'Y. Then SSE is independent of the third portion of d'v" d if and only if [X(X'X)"-'-X*'Vq-'"X*(X'X)""^X'] [Ijj X(X'X) "'"X'] = 0, see Searle (1971), p. 59, Theorem 4. Continuing then. [X(X'X) "'X*'Vq''"X*(X'X) """XMIIj^ X(X'X) """X'] X(X'X) •'X*'Vq-'"X*(X'X)"-'-X'-X(X'X) '•X*'Vq"'-X*(X'X)""'"X'X(X'X)~-'-X' X(X'X)~''-X*'Vq-'-X*(X'X) """XX(X'X) 'X*'Vq-'-X (X'X)~ X' = 0.

PAGE 175

167 Therefore SSE is independent of the third portion of Finally, since SSE is independent of each of the three portions of d'V d, we can conclude that SSE is independent of d'v" d and therefore SSE/a^ is independent of d'V~'''d/a^.

PAGE 176

APPENDIX 4 THEOREM 3.1 Theorem 3 . 1 Let A and B be kxk matrices. If (A B) is positive definite and B is positive semi-definite, then A is positive definite . Proof We assume that (A B) is positive definite. Then z'(A B)z > 0, for all z * 0. Thus z ' Az z • Bz > 0, for all z* 0, so that z ' Az > z • Bz > 0, for all z t 0, since B is positive semi-definite. Therefore, z'Az > 0, all z t 0. Now if z'Az = 0, then z = for if z + 0, then z'(A B)z > implies z • Bz < 0. But this is a contradiction since by assumption z ' Bz > 0. Therefore z must be and A must be positive definite. 168

PAGE 177

APPENDIX 5 THEOREM 3.2 Theorem 3.2 Let A-j^ and A2 be P2> 0. Let k>0, N>0, p>0, and N > p. (a) If [A^/k A^/ili p)] is positive definite then [Xj_/k A2/(N p)] =0 if and only if X ,_ = x 2 = 0. (b) If [Aj^/k A2/(N p)] is negative definite then [Xj^/k X2/(N p)] =0 if and only if x -^ = X 2 = 0. Proof of part (a) . Necessity . Let [A^^/k A^/CN p)] be positive definite and suppose that [X^/k X^/i^ P)] = 0. We show that Xj^ = X2 = 0. The matrix Aj^/k A2/(N p) being positive definite implies 8'[a /k A /(N p)]0 =0 iff 3 = , that is, -2 169

PAGE 178

170 a^A^B^A ^2^2-2'^^'^ P) = iff e^ = 0^ut. e^Aj^e^/^ka^ e_^A232/2(N p)a^ = iff g^ = 0. Hence X^/k X2/(N p) = iff B^ = 0' It follows that if X -l/Ic X2/(N p) =0, then x ]^ = X 2 = 0. Sufficiency . Obviously, if X^ = X2 = 0, then X^/k X^/CN p) =0-0=0. Proof of part (b). This follows from part (a), since in this case A /(N p) A /k is positive definite.

PAGE 179

APPENDIX 6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION Johnson and Kotz (1970, p. 197) indicate the following approximation for P{ F,"^ ^,^ ., ^ ^,^< F^.,^,,^} where vi and .^ are the numerator and denominator degrees of freedom, respectively, and X -^ and A 2 are the numerator and denominator noncentrality parameters, respectively: P{F" Vl/V2;Xi,X2 "^ ^a;vi,V2J ^^^^v,v' ^ ^aj\>i,V2^ = P{F , < (l/c)F } VfV ' ^ a ;vi ,V2^ where F^.^^^^^ is the upper 100a percentage point of the central F distribution with v^ and V2 degrees of freedom, and where c = [1 + X^/v^]/[l + X ^^v ^] , v = [v^ + X^]^/lv^ + 2X^], v' = [^2 +^2^ /^^? "^ 2X], and F , is a central F random variable with v and v' degrees of freedom. 171

PAGE 180

APPENDIX 7 EQUIVALENCE OF SSEg AND SSrQp WHEN REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS In this appendix we show that SSEg = SS^qf ^^^^ response observations are partitioned into g groups of true replicates rather than g groups of near neighbor observations. From Chapter Two, Section 2.2, if each cell consists entirely of true replicates, then the sum of squares due to lack of fit can be expressed as SS,^^ = SSE SSE LOF pure where SSE is the residual sum of squares from a least squares regression of Y on X and where SSEp^^.^ is the sum of squares due to pure error, calculated from replicates. Since SSE = Y'E.Y, where Z . is defined as in Section pure 04.2, we have ^^LOF = ^'^^N " X(X'X) ^X')Y YTqY = Y'(Ijj J:q)Y Y'X(X'X)"-'-X' Y. (A7.1) We wish to show that when each cell is composed entirely of true replicates, SSEg ig equal to the expression in (A7.1). 172

PAGE 181

173 Recalling from Section 4.3.1 that Y = MY, where M = diag[(l/n )1', ..., (1/n )1'], we write SSE3 = Y'[G-1 G-\(X^G-\)-^X^G-^]Y^ = X'M'[Go^ G-^Xc(X^G-^Xc)-^X^G-^]MY, where from Section 4.2, G = diag[l/n , 1/n , ..., 1/n ] Recognizing that Gq = MM' and X^ = MX, we have SSE = Y'[M' (MM* ) M B M' (MM' )~''"MX{X'M' (MM* )~'''MX} X'M'(MM') M]Y. Since M'(MM')~ M = I z , we have SSE3 = Y'(I^ Zq)Y Y'd^ ^o^^^^'^^N ^O^^J'^^'^^N ^0^^ and since E qX = when all cells are composed entirely of true replicates, we have SSE„ = Y'(I,, J:^)Y Y'X(X'X)"-'-X'Y B — N — — = S^LOF'

PAGE 182

174 from (A7.1). Therefore, SSEg is equal to the usual SSlqf when cells are composed entirely of true replicates.

PAGE 183

APPENDIX 8 LEMMA 4.1 Lemma 4 . 1 2 Let Y ~ (XB, a G), G singular. Define T = G + XX'. Define T~ such that TT~T = T. 1. (i) TT~X = X (ii) X'T~T = X' 2. rank(X'T~X) = rank(X) 3. (i) X(X'T~X)~(X'T~X) = X (ii) (X'T~X)(X'T~X)~X' = X' 4. Y is in the column space of T (Ye C(T)), with probability one, by which we mean that there exists a vector a such that letting Y = (y , y , ..., y )' and Ta = (x, , x^, ..., x^J ' , then 1 2 N P{ |y^ x^l > e} = ,for all e > 0, i = 1, 2, ..., N. Proof 1. (i) T = XX' + G = XX' + W, where G = W = CC , where C = [X:V] . Now, CC'(CC')~C = C, from Pringle and Rayner (1971, p. 26), and therefore TT~[X:V] = [X:V] from which it follows that TT~X = X (and TT~V = V). 175

PAGE 184

176 (ii) The proof of (ii) follows directly from (i) by taking the transpose. 2. The proof of part 2 is given in Rao, 1973, p. 77, #30. 3. The proof of part 3 is given in Rao and Mitra, 1971, p. 22, Lemma 2.2.6(c). 4. By definition, Y ~ ( Xg , a^G) so that an equivalent representation for Y is Y = Xg + e , where e ~ (0, a^G). We wish to show that the random vector Y is in the column space of T, with probability one. It is sufficient to show that TT~Y = Y, w.p.l. (see Pringle and Rayner, 1971, p. 9). Rewriting TT~Y we have TT~Y = TT~(Xe + £) = TT~X6 + TT~£. By part 1 of Lemma 4.1, TT~X = X, and therefore X3 e C(T). The proof is complete if we show TT~e = e, w.p.l. The difference TT~e e can be written as TT~~e e = (TT~ Ifj)e/ therefore we must show (see explanation below) that E[e'(TT~ Ij^)'(TT~ I^)e] = 0. (A8.1) The expectation in Eq. (A8.1) can be written as

PAGE 185

177 :[£'(TT 1^) '(TT l^)e] = trace [ (TT I„)'(TT I^Ja^G 1 2 o trace [ (TT I^^ ) ' ( TT I )W'] = since TT~V = V, by proof of part l(i) of Lemma 4.1. Therefore Y e C(T) , w.p.l. We now show that proving the equality in (A8.1) is equivalent to proving that TT~£ = e, w.p.l. By the Markov Inequality P{ |u. v.| > e} < [E(u. v.)2]a2 and therefore if E(Uj^ Vj^ ) = 0, we have Uj^ = v^, w.p.l. If u' = (u^, U2, ...,Uj^), V' = (Vj^, w^, ..., Vj^), and if 2 E(u^ v^) = 0, for i = 1, 2, ..., N, then u^ = Vj^, w.p.l, for i = 1, 2, ..., N, which implies that u = v, w.p.l. But E(u^ Vj^)^ = 0, for i = 1, 2, ..., N if and only if "^ 2 Z E(u. -V.) =0, and since i=l ^ ^ ^ 2 I E(u. V. ) = E(u V) ' (u V) i=l ^ ^ we have u = v, w.p.l, if E(u v)'(u v) = 0. In (A8.1) we take u = TT~£ and v = e .

PAGE 186

APPENDIX 9 PROOF OF THEOREM 4 . 1 ( i ) In this appendix we give the proof of Theorem 4.1(i). We show that E(a ) = a , where a = f~ (Y X3)'T~(Y Xe ) " 2 First we write a as ^2 ^ f Ir „,m = f -^[Y'T Y 2B'X'T Y + 3'X'T Xg] , where $ = (X'T X) X'T Y. Now, e'X'T Xe = B'X'T X(X'T X) X'T Y = g 'X'T Y, by Lemma 4.1, part 3(ii). Therefore a^ = f"-'-[Y'T~Y B'X'T~Y] = f"-'-[Y'T~Y { (X'T~X)~X'T~Y} 'X'T~Y] = f~-'-[Y'T~Y Y' (T~) •X(X'T~X)~X'T~Y] f'-'-Y'A Y (A9.1) where A = T (T )'X(X'T X) X'T . Using equation ( A9 . 1 ) , ° 178

PAGE 187

179 and applying Theorem l(i) (Searle, 1971, p. 55), we can write the expected value of a as E(P) = f "^E[Y'A Y] -1 2 f [traceJA^a G} + E(Y)'AqE(Y)] —1 2 f trace [AqO G] , (A9.2) since E(Y)'AqE(Y) = 0_'X'[T (T )'X(X'T X) X'T ] Xg = B'X'T Xg B'(X'T X)(X'T X) (X'T~X)e = as X'(T )'X = X'T X, because T is symmetric and X'T~X is unique (see proof of Theorem 4.1{ii)). Thus 2. _ .-1 .„_„r^^2. E(a') = f -" trace[A a^G] 2 -1 f trace [A-G] 2 -1 a f trace [ A»(T XX' ) ] By writing Aq as in Eq. ( A9 . 1 ) , we get

PAGE 188

180 ^2 2 —1 — E(a ) = a f trace [{t (T )'X(X'T X) X'T }{t XX'}] 2 _) = a f trace [t T T XX' (T ) ' X( X 'T~X)~X'T~T + (T ) 'X(X'T X) X'T XX'] 2 _] o f [trace T T trace T XX' trace (T )'X(X'T X) X' + trace (T~)'XX'], by Lemma 4.1, parts 1 and 3, and so ^2 2 —1 E(a ) = o f [trace T T trace (X'T X) (X'T X)], since X'(T~)'X = X'T~X. Since T~T and ( X'T~X)~( X'T~X) are idempotent, and rank(AA~)= rank (A) for any matrix A, we see that E(a^) = o^f '•[rank(T) rank(X'T X)] and by Lemma 4.1 part 2 we have E(a^) = a^f "'[rank(T) -rank(X)].

PAGE 189

181 Finally, since f = rank(G:X) rank(X), we can write '^ 2 2 —1 o E(0 ) = f [rank(G;X) rank(X)] = a . ( A9 . 3 ) The proof of Theorem 4.1(i) is now completed by justifying the equality in (A9.3) by showing that rank (T) = rank (G:X). First we write rank(T) = rank(G + XX'). Replacing G by W , we have rank(T) = rank(W' + XX') = rank(CC'), where C = (V:X) = rank(C) = rank(V:X) = rank(G:X), since the column space of G is the same as the column space of V. The column space of G is the same as the column space of V if the columns of V belong to the column space of G, and vice versa, if the columns of G belong to the column space of V. Symbolically, this is written as V <= C(G), and G c C(V). To show that V c C(G), it is sufficient to show that GG~V = v, but this is true because GG~V = (W')(W')-V = V. Now, G = C(V) since by definition W = G.

PAGE 190

APPENDIX 10 PROOF OF THEOREM 4.1(ii) In Appendix 10 we prove part (ii) of Theorem 4.1, thus we show that a = f~ (y X8)'T~(Y Xg) is unique with probability one. The following theorem will be useful in our proof . Theorem vi(c) (Rao, 1973, p. 26). Let B and D be non-null matrices. Then BA~D is invariant for any choice of A~ if and only if C(B') c C(A') and C(D) = C(A), where C( . ) denotes column space. The relationship C(B') c C(A') holds if and only if BA~A = B, and similarly C(D) <= C(A) holds if and only if AA~D = D (see " 2 Pringle and Rayner, 1971, p. 9). Since the quantity a is written as ^2 a = f '•[Y'T Y Y'(T )'X(X'T X) X'T Y] , " 2 to show that a is unique with probabilty one, it suffices to show that Y'T Y Y'(T )'X(X'T X) X'T Y (AlO.l) 182

PAGE 191

183 is invariant with probability one to the choice of the generalized inverses involved. First we show that Y'T~Y is unique with probability one (w.p.l). From part 4 of Lemma 4.1, Y e C(T), w.p.l. Therefore Y' e C(T'), w.p.l, since T is symmetric, and then by Theorem vi(c) (Rao, 1973, p. 26), Y'T~Y is unique, w.p.l. Secondly we show that Y' (T~) 'X( X'T~X)~X'T~Y is unique with probability one in the following four part proof. (1) Show X'T~X is unique. From part l(i) of Lemma 4.1, TT~X = X and thus X<=C(T). Since T is symmetric, we have X'^CCT'). By Theorem vi(c) (Rao, 1973, p. 26), X'T~X is unique. (2) Show X'T~Y is unique, w.p.l. By (1) above, XcC(T') and by part 4 of Lemma 4.1, Y e C(T), w.p.l. Thus applying Theorem vi(c) (Rao, 1973, p. 26), X'T~Y is unique, w.p.l. (3) Show Y'(T~)'X is unique, w.p.l. This follows from part (2), since Y'(T~)'X is equal to the transpose of X'T""Y , which was shown in (2) to be unique, w.p.l. (4) Using (1), (2), and (3) above, the second quantity in (AlO.l) is unique, w.p.l, by Theorem vi(c) (Rao, 1973, p. 26) if (a) [Y'('r~)'X]' e C[(X"r^X)'], w.p.l, and (b) X'T~Y e C(X'T~X), w.p.l. Part (a) is true not only with probability one but always because Y' (T~) •X(X'T~X)~(X'T~X) = Y'(T~)'X, since by part

PAGE 192

184 3(i) of Lemma 4.1, X = X(X'T~X)~ (X'T~X). Part (b) is true not only w.p.l but always because (X'T~X)(X'T~X)~ X'T~Y = X'T~Y, by part 3(ii) of Lemma 4.1. Therefore we have shown that both Y'T~Y and Y* (T~) •X(X"I^X)~X"r"Y are unique with probability one, which ^ 2 allows us to conclude that a is unique with probability one .

PAGE 193

APPENDIX 11 PROOF OF THEOREM 4.1(iii) In this appendix we prove Theorem 4.1(iii), that is we show that if Y possesses an N-variate normal distribution ^22 2 then fa /a ~ Xf ' where f = rank(G:X) rank(X). Recall that a = f Y'AqY where A = T (T )'X(X'T X) X'T . Since we have shown in Theorem 4.1(ii) that a^ is unique with probability one, the choice of the generalized inverses " 2 m the expression for a may be made arbitrarily. Thus we choose each of the generalized inverses to be the unique Moore-Penrose inverse, and we denote the unique MoorePenrose inverse of a matrix B by B"^. The Moore-Penrose inverse has the following four properties (see Searle, 1971, p. 16): 1. BB+B = B 2. B+BB+ = B"^ 3. (BB""") ' = BB"*" 4. (B+B)' = B+B. "2 2 The quantity fa /a can be expressed as fa^/a^ = Y'AY, (All.l) where a = (l/a^)[T"^ t"^X( X't'^X) "^X't"^] . We wish to show 185

PAGE 194

186 2 that Y'AY ~ Xf ' which can be done by making use of the following corollary. Corollary 2s. 1 (Searle^ 1971, p. 69). When X is N(u,V) whether V be singular or non-singular, 2 X ' Ax ~ x' with degrees of freedom equal to trace (AV) and noncentrality parameter equal to (l/2)y'Au, where 2 x' denotes a noncentral chi-square random variable, if and only if (i) VAVAV = VAV ( ii ) M_ • AV = u • AVAV, and ( i i i ) U ' Ay = jj ' AVAy . In our application, the matrices A and V in Corollary 2s. 1 (Searle, 1971, p. 69) are defined as A = (l/a^)[l T'^X(X'T'^X)'*'X']t'^, and 2 V = a G The proof of Theorem 4.1(iii) follows from Corollary 2s. 1 (Searle, 1971, p. 69) if we can show that AVA = A. To show that AVA = A, we first show that AG = AT, where as we recall, AG = A(T XX'). Thus AG = AT if AXX' = 0. Using the complete expression for A, we have

PAGE 195

187 AXX' = {l/a^)[T'*" t'^X(X't''"x) "''x't'^JXX' = (1/o^)[t'^XX' t"^X(X'T"^X)'*'(X'T"^X)X' ] , and so by Lemma 4 . 1 part 3 ( i ) , AXX' = (l/a^)[T'^XX' t'^XX'] = 0. 2 2 Therefore, since AG = AT, we have AVA = a AGA = a ATA. We 2 now show that a ATA = A: ^ATA = (l/a^)[l T"^X(X'T'^X)'*'X']t'''t[I T'''X( X'T'^X) "^X' ] T"^ (l/a^)[T'^T-T'^X(X'T"^X)'*'X'T'^T ] [ t'^-t'^X( X ' t'*'x) '*"x'T'^] (1/o^)[t"^tt'^ t'^x(x't"^x)'*"x't"^tt'*" t'''tt"*"x(X't'*"x)"'"x*t''' + t'*"x(x't"''x)''"x't"*"tt''"x(x't'''x)'*"x't''"] (l/a^)[T"^ t'^X(X'T'*"X)''"X'T'^ T'''X( X 'T'''X) """x* t"*" + T"'"X(X'T'*"X)"''X'T'*'X(X'T'''X)'^X'T"^]

PAGE 196

188 since t+TT"^ = T+, by property 2 of the Moore-Penrose inverse. Therefore, a^ATA = (l/a^)[T'^ 2T'^X( X'T'*'x) "^X't"^ + t'^XC X'T'^X) "^X' t"^] = (l/a^)[l t'^X(X'T'^X)'^X']t'^ = A. 2 Since we have verified that AVA = a ATA = A we can conclude that fa^/a^ = V ^1 ~ Xf ' ^Y Corollary 2s. 1 (Searle, 1971, p. 69). The quantity fa /a ~ Xf ^^<^ "ot Xf^' since the noncentrality parameter equals zero, which we now show. The noncentrality parameter, from Corollary 2s. 1 (Searle, 1971, p. 69) is of the form (l/2)u'Au, where in our application, y = Xg. Thus, M ' Ay = ' X • AX6 (l/a^)0 'X'[t'^ t'^X(X't'*"x)'^X't'^]XB = (l/a^)[0 X't''"X3 e 'X'T'^X(X't"^X)'^(X'T'^X)0] and so by Lemma 4.1 part 3(i), y'Ay_ = (l/a^)[0 •X'T''"X3 S'X'T'^XB] = 0.

PAGE 197

189 We now verify that the degrees of freedom are f = rank(G:X) rank(X). From Corollary 2s. 1 (Searle, 1971, p. 69) the degrees of freedom associated with Y'AY are equal 2 to f = trace (o AG), and so trace(a^AG) = trace[ I t'^X( X'T"^X) "''X' Jt'^G = trace(T"^G) trace[T"*"X( X'T'''X) "^X'T'*'g] = trace(T"*'T t'''xX') trace[T'*'X(X'T'*"x) '''x't'^'t] + trace[T'''x(X'T'^X) '^X'T"''xX' ] , since G = T XX'. It follows that trace(a AG) = trace t'*"t trace t"''xX' trace X( X't'''x) """x' t"^ + trace t'''xX' , since trace AB = trace BA for arbitrary matrices A and B, T'^TT'^ = T"^, and X( X'T'^X) "*( X •T'^X) = X by Lemma 4.1 part 3( ii ) . Therefore trace(o AG) = trace t"^! trace (X'T"^X) (X'T"*'X) '*' = ran]c(T) rank( X't'^X) ,

PAGE 198

190 since TT and (X'T X)(X'T X) are idempotent, and rank(AA ) = rank A, for any matrix A. Finally, by Lemma 4.1 part 2 we have trace (a AG) = rank(T) rank(X) and by the argument in the proof of Theorem 4.1(i), 2 trace (a AG) = rank(G:X) rank(X)

PAGE 199

APPENDIX 12 PROOF OF THEOREM 4.2 In this appendix we prove Theorem 4.2, thus we show that when Y~ Nj^(X3 + X B , a^G) then fa^/a^ ~ X^^^, where X = ( l/2a ^ )e • x' [T~ T~X(X'T~X)~X'T~]X 3 . "11 1 From the proof of Theorem 4.1, we have fa /a ~ \\,\ By Corollary 2s. 1 (Searle, 1971, p. 69) the noncentrality parameter is X = (1/2)(X0 + Y.^_^)'A{Y.^_ + X^P^), where A = (l/a^)[T T X(X'T X) X'T ]. Thus X = (l/2)[6 'X'AX3 + 3'X'AX 3 + 3'X'AX3 + ?.^X' AX 3 ] . From the proof of Theorem 4.1(iii), 3'X'AX3 = 0. We now show that 3'X'AX3 = 0: 3'X'AX3 = lo^U'^ ~ '^ X(X'T X) X'T ] X3 / a2 = 3'X'[t X3 T X(X'T X) (X'T X)3]/a2, and so by Lemma 4.1 part 3(i), 3'X'AX3 = B'X'[t X3 T X3 ] / = 0. 191 a2

PAGE 200

192 Thus we conclude that X = (l/2)e^X^AX2e2 = (l/2a )eiXi[T T X(X'T X) X'T~]x^S,. z z J 2—2

PAGE 201

APPENDIX 13 PROOF OF THE EQUALITY SSE, = d'v"-^d + SSE A ~ (J " In this appendix we show that the check point method for testing a fitted model for lack of fit and the method in which the residual sum of squares is partitioned into a lack of fit sum of squares and a pure error sum of squares are equivalent in the sense that SSE = d'V~''"d + SSE. A ~ (J " Let us define Y, and X, as -A A Xa = Y Y* (A13.1) and ^A = X X* (A13.2) Then the residual sum of squares from regressinq Y ng Y^ on X, is -A A -1, SSE^ = Y. [I X^(X'X^)-X]Y^. 193

PAGE 202

194 Using Eqs. (A13.1) and (A13.2) we can write SSE^ as SSE Y Y* I X X' (X'X + X*'X*) -1 X I • X*l Y Y* -1. = Y*'[I X*(X'X + X*'X*) X*']Y 2 Y*'X*(X'X + X*'X*) "'"X'Y + Y'[I X(X'X + X*'X*)"''"X' ] Y Y*'V~ Y* 2Y*'X*(X'X + X*'X*)~'''X' Y + Y' [I X(X'X + X*'X*) 'X'lY. (A13.3) Eq. (A13.3) is true because from Eq. (8) (Morrison, 1976, p. 69) we can write vl as V~ = [I + X*(X'X) 'X*' ] '" = I X*(X'X + X*'X*) """X * I

PAGE 203

195 We now write the quadratic from d'V^''"d as d'vJ'-d = (Y* Y*)'V^(Y* Y*) Y*'v~-'-Y* 2Y*'V"-'-Y* + Y*'V"-'-Y* = T'^o^I* 2 Y*'Vq-'-X*(X'X)"-^X'Y + Y'X(X'X)"-'-X*'Vq-'-X*(X'X)"-^X'Y. (A13.4) The first portion in Eq. (A13.3) is equal to the first portion in Eq. (A13.4). We now show that the second portions of Eqs. (A13.3) and (A13.4) are equal. It can be verified using Eq. (8) (Morrison, 1976, p. 69) that (X'X + X*'X*) ^ = (X'X) ^ (X'X)"^X*'Vq^X*(X'X)"^. = iA-A} (X'X) -X*'Vq-X*(X'X) ( A13.5) Using Eq. (A13.5) the second portion of SSE^ in Eq. (A13.3) can be written as -"v I • 2 Y*'X*(X'X + X*'X*) X'Y -2Y*'X*[(X'X) ^ (X'X) ^X*'V'^X*(X'X)~^]X'Y •2Y*'[I X*(X'X) 'X*'Vq-'-]X*(X'X)"-'-X'Y. (A13. 6)

PAGE 204

196 The second portion of SSE^, given in Eq. (A13.6), is seen to equal the second portion of d'v" d in Eq. (A13.4) using the fact that I X*(X'X)~-'-X*'Vq-'= I (Vq DVq-^ -0^We now show that the third portion of the expression for SSEp^ in Eq. (A13.3) is equal to the sum of the third portion of d'v" d in Eq. (A13.4) and SSE, where SSE = Y'[l X(X'X)~"'"X' ] Y. Using the result in Eq. (A13.5), the third portion of SSE^ in Eq. (A13.3) can be written as Y' [I X(X'X + X*'X*) '•X' ]Y Y'[I X{(X'X) * (X'X)~''"X*'Vq''"X*(X'X) -"-IXMY Y' [I X(X'X)"-'-X']Y + Y'X(X'X) '•X*'Vq-'-X*(X'X)"-'-X'Y. Therefore, since the first two portions of SSE^ in Eq. (A13.3) are equal to the first two portions of d'v" d in Eq. (A13.4), respectively, and the third portion of SSE^ in Eq. (A13.3) is equal to the sura of the third portion of d'Vd

PAGE 205

197 in Eq. {A13.4) and Y'[I X(X'X) '•X*]Y, we must have then SSE = d'V~"'"d + Y'[I X(X'X) ''"X'jY A " U " ^ — = d'V~"'"d + SSE.

PAGE 206

REFERENCES Atkinson, A. C. (1972). Planning experiments to detect inadequate regression models. Biometrika , Vol. 59, pp. 275-293. Barr, A. J., J. H. Goodnight, and J. P. Sail (1979). SAS Users Guide, 1979 Edition . SAS Institute, Inc., Raleigh, North Carolina. Becker, N.G. (1968). Models for the response of a mixture. Journal of the Royal Statistical Society , _B, Vol. 30, pp. 349-358. ~ Becker, N.G. (1978). Models and designs for experiments with mixtures. Australian Journal of Statistics , Vol. 20, pp. 195-208. Box, G. E. P., and N. R. Draper (1975). Robust designs. Biometrika , Vol. 62, pp. 347-352. Claringbold, P. J. (1955). Use of the simplex design in the study of the joint action of related hormones. Biometrics , Vol. 11, pp. 174-185. Cochran, W. G., and G. M. Cox (1957). Experimental Designs , 2nd Ed. John Wiley and Sons, New York. Cornell, J. A. (1973). Experiments with mixtures: A review. Technometrics , Vol. 15, pp. 437-455. Cornell, J. A. (1981). Experiments with Mixtures; Designs, Models, and the Analysis of Mixture Data . John Wiley and Sons, New York. Cornell, J. A., and I. J. Good (1970). The mixture problem for categorized components. Journal of the American Statistical Association , Vol. 65, pp. 339-355. Daniel, C, and F. S. Wood (1971). Fitting Equations to Data; Computer Analysis of Multif actor Data for Scientists and Engineers . John Wiley and Sons, ^Nev/ York. Draper N. R. , and A. M. Herzberg (1971). On lack of fit. Technometrics , Vol. 13, pp. 231-241. 198

PAGE 207

199 Draper, N. R. , and W. E. Lawrence (1965a). Mixture designs for three factors. Journal of the Royal Statistical Society , B_, Vol. 27, pp. 450-465. Draper, N. R., and W. E. Lawrence (1965b). Mixture designs for four factors. Journal of the Royal Statistical Society , B_, Vol. 27, pp. 473-478. Draper, N. R. , and H. Smith (1981). Applied Regression Analysis , 2nd Ed. John Wiley and Sons, New York. Draper, N. R. , and R. C. St. John (1977). A mixtures model with inverse terms. Technometrics , Vol. 19, pp. 37-46. Gorman, J. W. , and J. E. Hinman (1962). Simplex-lattice designs for multicomponent systems. Technometrics , Vol. 4, pp. 463-487. Graybill, F. A. (1969). Introduction to Matrices with Applications in Statistics . Wadsworth Publishing Co., Inc., Belmont, California. Graybill, F. A. (1976). Theory and Application of the Linear Model . Duxbury Press, North Scituate, Massachusetts . Green, J. R. (1971). Testing departure from a regression, without using replicaton. Technometrics , Vol. 13, pp. 609-615. Johnson, N. L. , and S. Kotz (1970). Distributions in Statistics: Continuous Univariate Distributions 2 . Houghton Mifflin, Boston. Kenworthy, 0. 0. (1963). Factorial experiments with mixtures using ratios. Industrial Quality Control , Vol. 19, pp. 24-26. Khuri , A. I., and J. A. Cornell (1981). Lack of fit revisited. Technical Report No. 167, Department of Statistics, University of Florida, Gainesville, Florida. Kurotori, I. S. (1966). Experiments with mixtures of components having lower bounds. Industrial Quality Control , Vol. 22, pp. 592-596. McLean, R. A., and V. L. Anderson (1966). Extreme vertices design of mixture experiments. Technometrics , Vol. 8, pp. 447-454. Morrison, D. F. (1976). Multivariate Statistical Methods , 2nd Ed. McGraw-Hill, New York.

PAGE 208

200 Mudholkar, G. S., Y. P. Chaubey, and C. Lin (1976). Approximations for the doubly noncentral-F distribution. Communications in Statistics ^ A, Vol. 5, pp. 49-63. ~ Myers, R. H. (1971). Response Surface Methodology . Allyn and Bacon, Inc., Boston. Piepel, G. F. (1981). Measuring component effects in constrained mixture experiments. Unpublished manuscript presented at 1981 Joint Statistical Meetings of the American Statistical Association and the Biometric Societies, Detroit, Michigan, August 1981. Price, W. L. (1977). A controlled random search procedure for global optimisation. The Computer Journal , Vol. 20, pp. 367-370. Pringle, R. M. , and A. A. Rayner (1971). Generalized Inverse Matrices with Applications to Statistics . Hafner Publishing Company, New York. Rao, C. R. (1971). Unified theory of linear estimation. Sankhya , _A, Vol. 33, pp. 371-394. Rao, C. R. (1972). Unified theory of least squares. Communications in Statistics , Vol. 1, pp. 1-8. Rao, C. R. (1973). Linear Statistical Inference and its Applications , 2nd Ed. John Wiley and Sons, New York. Rao, C. R., and S. K. Mitra (1971). Generalized Inverse of Matrices and its Applications . John Wiley and Sons, New York. Scheffe, H. (1958). Experiments with mixtures. Journal of the Royal Statistical Society , B_, Vol. 20, pp. 344-360. Scheffe, H. (1959). The Analysis of Variance . John Wiley and Sons, New York. Scheffe, H. (1963). The simplex-centroid design for experiments with mixtures. Journal of the Royal Statistical Society , _B, Vol. 25, pp. 235-263. Searle, S. R. (1971). Linear Models . John Wiley and Sons, New York . Shillington, E. R. (1979). Testing lack of fit in regression without replication. The Canadian Journal of Statistics, Vol. 7, pp. 137-146.

PAGE 209

201 Snee, R. D. (1971). Design and analysis of mixture experiments. Journal of Qu ality Technoloqy, Vol. 3, pp. 159-169. " ^^ Snee, R. D. (1973). Techniques for the analysis of mixture data. Technonetrics , Vol. 15, pp. 517-528. Snee, R. D. (1977). Validation of regression models: methods and examples. Technome tries. Vol. 19, pp. 415428. Thompson, W. 0., and R. K. Myers (1968). Response surface designs for experiments with mixtures. Technometrics, Vol. 10, pp. 739-756. " Wishart, D. (1975). Clustan IC User Manual . David IVishart, University College, London. Zyskind, G. (1967). On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. Annals of Mathematical Statistics , Vol. 38, pp. 1092-1109.

PAGE 210

BIOGRAPHICAL SKETCH John Thomas Shelton was born on March 30, 1952, in Jacksonville, Florida, where he resided until graduating from Englewood High School in June, 1970. He then entered the University of Florida where he received a Bachelor of Science degree in mathematics in June, 1974. John began graduate study at Virginia Polytechnic Institute and State University in Blacksburg, Virginia, in September, 1975, and there received a Master of Science degree in statistics in the summer of 1976. After two years as a Research Associate at Auburn University in Auburn, Alabama, he returned to the University of Florida in September, 1978, where he has since been pursuing a doctoral degree in statistics. While a graduate student at the University of Florida, John has worked as a Graduate Assistant performing statistical consulting duties in the School of Forest Resources and Conservation. 202

PAGE 211

I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a disseration for the degree o^f Doctor of Philosophy. T /^k, W' Andre' I. Khun, Chairman Assistant Professor of Statistics I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a disseration for the degree of Doctor of Philosophy.
PAGE 212

?^


TESTING LACK OF FIT IN A MIXTURE MODEL
BY
JOHN THOMAS SHELTON
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1982

To Nydra
and
My Parents

ACKNOWLEDGEMENTS
I would like to express my deepest appreciation to Drs.
Andre' Khuri and John Cornell for suggesting this topic to
me and for providing constant guidance and assistance. They
have made this project not only a rewarding educational
experience but an enjoyable one as well. A special word of
thanks goes to Mrs. Carol Rozear for her diligence in
transforming my handwritten draft into an expertly typed
manuscript.
iii

TABLE OF CONTENTS
page
ACKNOWLEDGEMENTS iii
ABSTRACT vii
CHAPTER
ONE INTRODUCTION 1
1.1 The Response Surface Problem 1
1.2 The Mixture Problem 5
1.2.1 Mixture Models 6
1.2.2 Experimental Designs for Mixtures.. 12
1.3 The Purpose of this Research:
Investigation of Procedures for Testing
a Model Fitted in a Mixture System for
Lack of Fit 17
TWO LITERATURE REVIEW—TESTING FOR LACK OF FIT 19
2.1 Introduction 19
2.2 Partitioning the Residual Sum of Squares.. 21
2.3 Testing for Lack of Fit Without
Replicated Observations—Near Neighbor
Procedures 26
2.4 Testing for Lack of Fit with Check Points. 33
THREE AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL 40
3.1 Introduction 40
3.2 Testing for Lack of Fit in the Presence
of an External Estimate of Experimental
Error Variation 41
3.2.1 The Test Statistic 41
3.2.2 The Testing Procedure and an
Expression for the Power of
The Test 45
3.2.3 A Method for Locating Optimal
Check Points 47
3.3 Testing for Lack of Fit When MSE Is
Used to Estimate Experimental Error
Variation 51
3.3.1 The Test Statistic 51
3.3.2 The Rejection Region and its
Relation to the Power of the Test.. 53
iv

3.3.3 A Method for Locating Optimal
Check Points 56
3.3.4 Determining Whether the Test Is
Upper Tailed or Lower Tailed 58
3.4 Examples 67
3.4.1 Theoretical Examples 67
3.4.2 Numerical Examples 83
3.5 Discussion 95
FOUR USE OF NEAR NEIGHBOR OBSERVATIONS FOR
TESTING LACK OF FIT 99
4.1 Introduction 99
4.2 Notation 101
4.3 Shill ington ' s Procedure 106
4.3.1 Development of MSEB 109
4.3.2 Development of MSEW 110
4.4 Development of SSEW(weighted) 112
4.5 Equivalence of SSEW and SSEw(weighted)....116
4.6 The Test Statistic 118
4.7 The Testing Procedure and its Power 122
4.8 Selection of Near Neighbor Groupings 125
4.8.1 Example 1—Stack Loss Data 129
4.8.2 Example 2—Glass Leaching Data 134
4.9 Discussion 142
FIVE CONCLUSIONS AND RECOMMENDATIONS 145
APPENDICES
1
INFLUENCE OF \ ON
P{ F" . , > F }
Vj ,V2 ;Ai (Á2 a;v^,V2
156
2 A CONTROLLED RANDOM SEARCH PROCEDURE FOR
GLOBAL OPTIMIZATION 159
3 STATISTICAL INDEPENDENCE OF d'V“1d/o2
AND SSE/a2 164
4 THEOREM 3.1 168
5 THEOREM 3.2 169
6 AN APPROXIMATION TO THE DOUBLY NONCENTRAL
F DISTRIBUTION 171
7 EQUIVALENCE OF SSEg AND SSL0F WHEN
REPLICATES REPLACE NEAR NEIGHBOR
OBSERVATIONS 17 2
v

8 LEMMA 4.1 175
9 PROOF OF THEOREM 4.1(i) 178
10 PROOF OF THEOREM 4.1(11) 182
11 PROOF OF THEOREM 4.1(iii) 185
12 PROOF OF THEOREM 4.2 191
13 PROOF OF THE EQUALITY SSEA = d'v"1! + SSE 193
REFERENCES 198
BIOGRAPHICAL SKETCH '. 202
vi

Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in
Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy
TESTING LACK OF FIT IN A MIXTURE MODEL
By
John Thomas Shelton
May 1982
Chairman: Andre' I. Khuri
Cochairman: John A. Cornell
Major Department: Statistics
A common problem in modeling the response surface in
most systems, and in particular in a mixture system, is that
of detecting lack of fit, or inadequancy, of a fitted model
of the form E(Y) = X§^ in comparison to a model of the form
E(Y) = xe1+ X^g2 postulated as the true model. One method
for detecting lack of fit involves comparing the value of
the response observed at certain locations in the factor
space, called "check points," with the value of the response
that the fitted model predicts at these same check points.
The observations at the check points are used only for
testing lack of fit and are not used in fitting the model.
It is shown that under the usual assumptions of
independent and normally distributed errors, the lack of fit
test statistic which uses the data at the check points is an
Vll

F statistic. When no lack of fit is present the statistic
possesses a central F distribution, but in general, in the
presence of lack of fit, the statistic possesses a doubly
noncentral F distribution. The power of this F test depends
on the location of the check points in the factor space
through its noncentrality parameters. A method of selecting
check points that maximize the power of the test for lack of
fit through their influence on the numerator noncentrality
parameter is developed.
A second method for detecting lack of fit relies on
replicated response observations. The residual sum of
squares from the fitted model is partitioned into a pure
error variation component and into a lack of fit variation
component. Lack of fit is detected if the lack of fit
variation is large in comparison to the pure error
variation. This method can be generalized when "near
neighbor" observations must be substituted for replicates.
In this case, the test statistic (assuming independent and
normally distributed errors) has a central F distribution
when the fitted model is adequate and a doubly noncentral F
distribution under lack of fit. The arrangement of near
neighbors is seen to affect the testing procedure and its
viii
power.

CHAPTER ONE
INTRODUCTION
1.1 The Response Surface Problem
A mixture problem is a special type of a response
surface problem. First we shall define the general response
surface problem and indicate the basic objectives sought in
its analysis, and follow this development with a discussion
of the mixture problem.
In the general response surface problem, we are inter¬
ested in studying the relationship between an observable
response, Y, and a set of q independent variables or
factors, x^, x2/ . .., Xq, whose levels are assumed con¬
trolled by the experimenter. The independent variables are
quantitative and continuous. We express this relationship
in terms of a continuous response function, 4» r as
Y = d> (x
u y '
ul' u2'
. ., x ) + e
' uq' u
where Yu is the uth of N observations of the response col¬
lected in an experiment, and xu¿ represents the uth level of
the ith independent variable, u = 1, 2, ..., N, i = 1, 2,
..., q. The exact functional relationship, 4, is unknown.
The term eu is the experimental error of the uth
1

2
observation. It is assumed that E(eu) = 0, E(eueui) = 0,
for u * u', and E(e¿j) = a2, for u = 1, 2, N.
As the form of is unknown and may be quite complex, a
low order polynomial (usually first or second order) in the
independent variables x-^, X2/ ..., Xg is generally used to
approximate . This may be justified by noting that such
polynomials constitute low order terms of a Taylor series
expansion of 41 about the point xp = x2 = ••• = xq = 0,
(Myers, 1971, p. 62). Cochran and Cox (1957, p. 336) point
out that these low order polynomials may give a poor approx¬
imation to region, and thus should not be used for this purpose.
A linear response surface model may be written in
matrix notation as
Y = X0 + £ (1.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, ft is a pxl vector of
unknown parameters (regression coefficients), and g. is the
Nxl vector of random errors. When the model is a first or a
second degree polynomial, the columns of X correspond to the
first or second degree powers of the independent variables
x^, X2/ . .., Xg, or their cross products. If the model
contains a constant term, 3q, the first column of X will
correspond to this term, and will consist of N ones. Since
E(e) = 0, an alternative representation for the response

3
surface model of (1.1) is
E(Y) = XB .
Once the form
of the model that will
be
used
to
approx-
imate (X2, . .
•, Xg)
is chosen, the next
step
is
to
estimate the regression
coefficients, a.,
and
then
use
the
estimated model to
make
inferences about
the
true
response
function, <|> . The estimation of the elements of a. is usually
accomplished by ordinary least squares techniques. For the
purpose of testing hypotheses concerning the regression
coefficients, B., it is assumed that £. has a normal distribu-
2
tion, that is, e ~ NN(0, a IN).
Perhaps the most common objective in the exploration of
a response system is the determination of its optimum
operating conditions. By this we mean that it is desired to
find the settings of x^, X21 . .., Xg that optimize , which
in some applications may be interpreted as maximizing ,
while in other applications a minimum value of may be of
interest. It is also often desirable to determine the be¬
havior of the response function in the neighborhood of the
optimum. For second order response models, such an investi¬
gation can be carried out by performing a canonical analysis
of the second order surface as discussed in Myers (1971).
For simple systems having only one or two independent
variables, the response surface may be explored by just
plotting the fitted response values against values taken by

4
the independent variables. If q = 1, implying only one
independent variable, say x-^, then a two-dimensional plot of
the fitted response against x^ may be used to locate the
optimum, as well as to investigate the response behavior in
other parts of the experimental range of x^. If q = 2, and
the two independent variables are x^ and x2> then a plot of
the contours of constant response over the region specified
by the ranges of the values for x^ and X2 can be used to
describe the response surface.
The properties that the fitted model possesses in terms
of its ability to represent the true surface, , depend on
the settings of Xj_, X2* ..., Xq at which values of Y are
observed. Thus the experimental design is of great impor¬
tance. Much work has been done on the construction of
designs that are optimal with respect to one criterion or
another involving the fitted response and/or the true unfit¬
ted model. Box and Draper (1975) list fourteen criteria to
consider when choosing a design for investigating response
surfaces. Myers (1971) gives several designs for fitting
first and second order polynomial models. A discussion of
specific design considerations will not be attempted here,
as such a discussion is not the focus of this dissertation,
and would necessarily be lengthy.
The initial steps in the analysis of a response system
may be described as follows: First an attempt is made to
approximate the true response function, (xj_, X21 •••* Xq) ,
usually with a low order polynomial in xj_, x2/ •••/ Xq.

5
After the form of the model has been chosen, then comes the
selection of an appropriate experimental design, which
specifies the settings of the independent variables at which
observed values of the response will be collected. The
observed values of the response are used in estimating the
regression coefficients in the model, using, in general,
ordinary least squares. After a test for "goodness of fit"
of the model verifies the fitted model is adequate, the
fitted model is used in determining optimum operating condi¬
tions for the response system.
1.2 The Mixture Problem
A mixture system is a response system in which the
response depends only on the relative proportions of the
components or ingredients present in a mixture, and not on
the total amount of the mixture. For example, the response
might be the octane rating of a blend of gasolines where the
rating is a function only of the relative percentages of the
gasoline types present in the blend. The proportion of each
ingredient in the mixture, denoted by x¿, must lie between
zero and unity, i = 1, 2, ..., q. The sum of the propor¬
tions of all the components will equal unity, that is,
q
0 < x. < 1, i = 1,2,...,q, £ x. = 1. (1.2)
i=l
The factor space containing the q components is represented
by a (q - 1)-dimensional simplex. For q = 2 components, the
factor space is a straight line, whereas for q = 3

6
components, the factor space is an equilateral triangle, and
for q = 4 components, the factor space is represented by a
regular tetrahedron.
The objectives in the analysis of a mixture response
system are, in general, the same as in any response surface
exploration. That is, one seeks to approximate the surface
with a model equation by fitting an equation to observations
taken at preselected combinations of the mixture com¬
ponents. Another objective is to determine the roles played
by the individual components. We shall not concern our¬
selves with this but rather concentrate on the empirical
model fit. Once the model equation is deemed adequate an
attempt is made to determine which of the component combina¬
tions yield the optimal response. The models used to repre¬
sent the response in a mixture system are in most cases
different in form from the standard polynomial models. The
first type of model form that we discuss is the canonical
polynomial suggested by Scheffé.
1.2.1 Mixture Models
Scheffe (1958) introduced a canonical form of the poly¬
nomial model for representing the response in a mixture
system. These canonical polynomial models are derived from
the standard polynomials using the restrictions on the x¿
shown in (1.2). With q = 2 mixture components, for example,
the standard second order polynomial model is of the form
a 0 + alXl +a2X2 + a12XlX2 + allXl + a 22X 2
E(Y)
(1.3)

7
Restrictions (1.2) imply that ciq = ag(x-L + x2),
X1 = xl(1 “ x 2), and x2 = x2(1 " xl)f thus (1*3) can be
written in the canonical form
E(Y) = 0x + 02x2 + 612x1x2'
where 6^ <»„ + ^ + 2 + and S12= «12
- ct^ - ot 22 • There is no constant term in the above canoni¬
cal form and the pure quadratic terms in equation (1.3) have
been absorbed in the x¿xj terms.
The general form of the canonical polynomial of degree
d in q mixture components can be written as
E(Y) = Z 3,x., for d = 1,
i = 1
E (Y) = Z 3-x. + Z Z g-.x.x. , for d = 2, and
.,11 , . .. IT 1 i
x=l l q q q
E (Y) = Z g.x. + Z Z S-.x.x. + Z Z 6..x.x.(x.
i-1 1 1 Ki - V
+ ZZZ g.., x.x.x. , for d = 3.
i • , • ilk ink
l (1.4)
The fourth degree canonical polynomial in q components is
given in Cornell (1981, p. 64). The general canonical poly¬
nomial of degree d > 4 in q components does not explicitly
*
appear in the literature, but is mentioned in Scheffe
(1958). If terms of the form 6ijXiXj(x¿ - Xj) are removed
from the full cubic model (1.4), then the remaining terms

8
make up what is referred to as the special cubic model. For
example, for q = 3 components, the special cubic model is
E(Y) = 3-^ + 8 2X2 + P3X3 + 012X1X2 + ^13X1X3
+ + e,„,x,x„x,
23 2 3 123 123
Scheffe's canonical polynomial models are used for
approximating the response surface in many mixture systems.
Their popularity stems from the ease in interpreting the
coefficient estimates, especially when the models are fitted
to data collected at the points of the associated designs
(see Section 1.2.2). However, other models have been intro¬
duced which seem to better represent the response when the
components have strictly additive blending effects. We
present some of them now.
Becker (1968) introduced three forms of homogeneous
models of degree one which he recommends instead of the
polynomial models when one or more of the mixture components
have an additive effect or when one or more components are
inert. A function f(x, y, ..., z) is said to be homogeneous
of degree n when f(tx, ty, ..., tz) = tnf(x, y, ..., z), for
every positive value of t and (x, y, ..., z) * (0, 0, ...,
0). These models, which Becker refers to as models HI, H2,
and H3, are of the form

9
q q
HI: E(Y) = Z 3.x. + El 0..min(x., x.) + ...
..li . » . ii i 1
1=1 1<. l <3 J J
+ B12...qmin(Xr X2 V '
q q 2-1
H2: E(Y) = E 3-x. + Z Z 0..x.x./(x. + x.) + ...
11 ..... ^11 l 1/v l 1
1<1<3 J J J
i =1
+ 0
12 . qxix2“ *xq/(xi + x2 + **' + xq>q
q q 1/2
H3: E (Y) = Z 0,x, + Z Z 3 (x x ^ + ...
l i=i 1 1 i*í/í
Bi2...q(XlX2-"Xq>
i/q
Each term in the H2 model is defined to be zero when the
denominator of the term is zero.
Draper and St. John (1977) suggest a model which in¬
cludes inverse terms, l/x¿, in addition to terms in the
*
Scheffe polynomials. Such a term is used to model an
extreme change in the response as x¿ approaches zero. The
experimental region of interest is assumed to include the
region near the zero boundary (x¿ = 0), but does not include
the boundary itself. One example of this type of model is
the Scheffe linear polynomial model with inverse terms
q q
= Z 0 . x . + Z 0.x.
i=l 1 1 i=l _1 1
-1
E(Y)

10
Another model form that is useful in the study of the
response in a mixture system is the model containing ratios
of the component proportions. A term such as Xj_/xj measures
the relationship of Xj_ to Xj rather than the percentage of
each in the blends. Snee (1973) points out that the ratio
model presents a useful alternative to the Scheffé and
Becker models in that the ratio model describes a different
type of curvature. He notes that the curvilinear terms for
the Scheffe and Becker models, when plotted as a function of
x¿, are symmetric functions about x¿ = 1/2, whereas the
ratio term x^/xj is a monotone function when plotted against
xi*
The terms in the ratio models may also contain sums of
the components. For example, with q = 3 components, we
might express the second order model
E(Y)
q-1
B0 +
1=1
q-1
+ Z Z
1< i < j
B . . z . z .
13 1 3
q-1
+ Z
i =1
2
i
(note the constant term) where z-^ and Z2 are defined as
Zj_ = x1/(x2 + X3) and z2 = x2/x3. Some terms will be unde¬
fined if points from the boundary of the experimental sim¬
plex are included in the design, for example, if x3 = 0,
then z2 = x2/x3 is not defined. Snee (1973) suggests adding
a small positive quantity, c, to each x¿ in this case.
This, of course, will not be of concern if the experimental
region is entirely inside of the simplex.

11
When one or more of the components is inactive, Becker
(1978) suggests that a ratio model that is homogeneous of
degree zero in the remaining components is appropriate. In
three components, such a model is of the form
E(Y) = 0O + e1x1/(x1 + x 2) + e2x2/(x2 + x3)
3
+ 3 x./(x, + x ) + E E 0• •h• . (x . , x.)
3 J 1 3 l + 3123h123(Xlr X 2’ X3)' (1*5)
where h^j and h^23 are specified functions that are homoge¬
neous of degree zero. The function hjl23 is intended to
represent the joint effect of all three components simulta¬
neously. If in fitting a model of the form (1.5) we deter¬
mine the model should be
E(Y) = 0O + + x2) + B 12h i2 1' x2)
then component three is said to be inactive and is removed
from further consideration. The model of equation (1.5) may
produce an extreme value near the vertices of the simplex
factor space when there are no inactive components. In this
case it is suggested that a model of the form (1.5) be used
only when the proportions are restricted so that no two of
the x^ are simultaneously very close to zero. Becker notes
that other authors who have suggested ratio models have also

12
used them primarily over a subregion inside the simplex
factor space. Apparently this is where they are most appro¬
priate .
1.2.2 Experimental Designs for Mixtures
As in the general response surface problem, one of the
major concerns in exploring a mixture system is that of
choosing the experimental design for collecting observed
values of the response that will be used in fitting the
*
model. Scheffe (1958) proposed the {q,m} simplex lattice
designs for exploring the entire q-component simplex factor
space. In these designs, the proportions used for each
component have the m + 1 values spaced equally from zero to
one, x¿ = 0, 1/m, 2/m, ..., (m - l)/m, 1, and all possible
mixtures with these proportions for each component are
used. The number of design points in the {q,m} simplex
lattice design is (m + ^ ~ The main appeal of these
designs is that they provide a uniform coverage of the fac-
*
tor space. Another feature, which Scheffe (1958) demon¬
strates, is that the parameters of the canonical polynomial
of degree m in q components are expressible as simple linear
combinations of the true response values at the design
points of the {q,m} simplex lattice. The {3,2} simplex
lattice, which consists of six design points, is represented
on the two dimensional simplex in Figure 1 along with the
triangular coordinates (xj_, X2# X3).
Scheffe (1963) also developed the simplex centroid
designs consisting of 2^ - 1 points, where the only mixtures

13
considered are the ones in which the components present
appear in equal proportions. That is, in a q-component
simplex centroid design, the design points correspond to the
q
q permutations of (1, 0, 0, 0), the (2) permutations of
q
(1/2, 1/2, 0, ..., 0), the (3) permutations of (1/3, 1/3,
1/3, 0, . .., 0), . .., and the point (1/q, 1/q, ..., 1/q).
This design alleviates the problem inherent in the {q,m}
simplex lattice designs of observing responses at mixtures
containing at most m components. To give an example, the
q = 3 simplex centroid design is made up of 2^ - 1 = 7
design points, and is equivalent to the {3,2} simplex
lattice design augmented by the center point (x^, x2, X3) =
(1/3, 1/3, 1/3). This design is represented in Figure 2.
Scheffé (1963) mentions that the number of parameters
in the polynomial model of the form
E(Y) =
q
E
q
E E
g.x. +
i =1 1 1 l q
6..X.X. + E E E 6 • x. x . x.
^ 1 3 i +
+ B12...qXlX2
X
q
(1.6)
is 2*3 - 1 and therefore these models have a special rela¬
tionship with the simplex centroid design in q components.
This relationship is that the number of terms in the model
equals the number of points in the design and as a result
the parameters in model (1.6) are expressible as simple
functions of the responses at the 2^-1 points of the sim¬
plex centroid design. Polynomial models of the form (1.6)

14
V
(0,1 ,
( 0,0,1 )
Figure 1. The {3/2} simplex lattice design.
x.r i
(0,0,1)
Figure 2. The q = 3 simplex centroid design.

15
therefore are natural models to fit using the simplex cen¬
troid design.
Ratio models may be desirable when the interest in one
or more of the mixture components is in terms of their rela¬
tionship to one another, rather than in terms of their per¬
centages in blends. Kenworthy (1963) proposed factorial
arrangements for ratio variables. An example of the use of
ratios is the following three component system in which the
mixture components are constrained by the upper and lower
bounds:
.2 < *2 < x2 < *4/ .3 < < *5* (1*7)
The ratio variables of interest are z± = x2/xl anc^
z2 = x2/x3' anc^ desired to fit either a first or a
second order polynomial model in and Z2* For such a
problem, we can define a 22 and a 32 factorial design that
can be used for fitting the first and second order poly¬
nomial models, respectively, by taking as design points the
intersection of rays passing from two of the three vertices
of the two-dimensional simplex through the region of
O
interest defined by the constraints (1.7). Kenworthy's 2
factorial design is shown in Figure 3.
Becker (1978) uses rays extending from one or more
vertices of the simplex factor space to the opposite bound¬
aries in developing "radial designs." These designs are
suggested for detecting the presence of an inactive

16
Figure 3. Kenworthy's 2^ factorial design.
component or in another case a component which has an addi¬
tive effect, when models containing ratio terms that are
homogeneous of degree zero are fitted.
McLean and Anderson (1966) suggest an algorithm for
locating the vertices of a restricted region of the simplex
factor space which is defined by the placing of upper and
lower bounds on the mixture component proportions. The
vertices of the factor space and convex combinations of the
vertices are the candidates for design points for fitting a
first or second degree polynomial model in the mixture com¬
ponents. One drawback of the "extreme vertices" design is
that the design points are not uniformly distributed over
the factor space resulting in an imbalance in the variances
of Y(x), see Cornell (1973).

17
Another method that has been suggested for studying the
response over a sub-region of the simplex mixture space is
to transform the q mixture components into q - 1 independent
variables. Transforming to an independent variable system
was first suggested by Claringbold (1955) and later proposed
by Draper and Lawrence (1965a, 1965b) and Thompson and Myers
(1968). Standard response surface polynomial models in the
transformed variables can be fitted to data values collected
on standard designs and a design criterion such as the aver¬
age mean square error of the response can be employed when
distinguishing between designs. Thompson and Myers (1968)
suggest the use of rotatable designs (see also Cornell and
Good, 1970).
Designs other than rotatable designs, such as multiple
lattices and symmetric-simplex designs, to name a few, have
been suggested in the literature for fitting models to a
mixture system which may be appropriate depending on par¬
ticular experimental situations. However, as the intent
here is not to give an exhaustive list but only a sampling
of available designs, we shall not discuss designs further
but instead state the purpose of this work.
1.3 The Purpose of this Research:
Investigation of Procedures tor Testing a Model
Fitted in A Mixture System for Lack of Fit
A common problem in modeling the response in a mixture
system is that of detecting lack of fit, or inadequacy, of a
fitted model of the form E(Y) = XQ ^ when the true model is
of the form E(Y) = Xg ^ + X2-2* T^e stat^stliterature

18
suggests several procedures for testing lack of fit, which
will be described in Chapter Two. In general, the authors
of these procedures are not specific in stating hypotheses
to be tested and do not adequately discuss the power of
their procedures.
The major purpose of this research is to investigate
the power of two of the testing procedures appearing in the
literature in detecting the inadequacy of a fitted model
when the general form of the true model is specified. Our
findings for a "check points" lack of fit testing procedure
are presented in Chapter Three while Chapter Four contains
findings for a "near neighbor" lack of fit testing proce¬
dure. For both procedures, explicit formulas for the power
of the test are given in terms of cumulative probabilities
of either the noncentral F or doubly noncentral F distribu¬
tion, which are derived by assuming that the response obser¬
vations are independent and normally distributed. Addition¬
ally, we propose methods for maximizing the power of the
testing procedures. In the final chapter, we make some
concluding comments concerning both of these procedures.

CHAPTER TWO
LITERATURE REVIEW—TESTING FOR LACK OF FIT
2.1 Introduction
Let us return to the general response surface problem
and assume the true response is to be approximated by
fitting a model of the form
E(Y) = X0 (2.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants, and g.]_ is a px 1 vector of
unknown regression coefficients. We wish to consider the
situation in which the true model contains terms in addition
to those in the fitted model. Then the true model has the
form
E(Y) = X0x + X202 (2.2)
where X2 is an Nxp2 matrix of known constants, and £L2 is a
P2xl vector of unknown regression coefficients. We assume
that the vector Y has the normal distribution with
var(Y) = a2IN .
It is desirable to determine the suitability of the
fitted model given by Eq. (2.1) when in reality the true
model is of the form given by Eq. (2.2). The process of
19

20
making this determination is referred to as testing for lack
of fit of the fitted model.
There are three general approaches to testing for lack
of fit. The first approach requires that there be replicate
observations of the response at one or more design points,
and involves partitioning the residual sum of squares from
the fitted model into a sum of squares due to lack of fit
and a sum of squares due to pure error. A large value for
the ratio of the mean square due to lack of fit to the mean
square due to pure error provides evidence for lack of fit.
If replicate observations are not available then the
above approach to testing for lack of fit cannot be used.
Green (1971), Daniel and Wood (1971), and Shillington (1979)
have proposed alternative methods that are applicable in
this case. Their approach is to group values of the
response which are observed at similar settings of the
independent variables and to call these grouped values
"pseudoreplicates" or "near neighbor observations." They
then treat these pseudoreplicates as they would treat true
replicates to form statistics for lack of fit testing,
although arriving at their respective statistics through
different approaches.
A third approach to testing for lack of fit involves
the use of "check points." In this method a model of the
form (2.1) is fitted to data at the design points and
additional observations are collected at other points in the
experimental region. The additional points other than the

21
design points are called check points, and the data at these
check points are not used in fitting the model. Lack of fit
is tested by comparing the values of the response observed
at the check points to the values of the response which the
fitted model predicts at these same check points.
We now discuss the first method mentioned above of
testing for lack of fit which involves partitioning the
residual sum of squares.
2.2 Partitioning the Residual Sum of Squares
The method for testing lack of fit which makes use of a
partitioning of the residual sum of squares from the fitted
model requires there be replicate observations of the
response at some of the design points (Draper and Smith,
1981, p. 120). When a model of the form (2.1) is fitted,
the residual sum of squares is defined as
n.
n i „
SSE = Z Z (Y..-Y.)
i=l j=l ^
= Y'(IN - X(X'X)-1X')Y
where n is the number of distinct design points, n-¡_ > 1 is
the number of replicate observations at the ith design
point, Y¿j is the jth observed value of the response at the
ith design point, Y^ is the value which the model of the
form in Eq. (2.1), fitted by ordinary least squares
techniques, predicts for the response at the ith design
n
point, and N = z n. . Using the replicated observations
i =1 1

22
only, a pure error sura of squares can be calculated as
SSE
pure
E E
i=l j=l
Y .
1
2
t
where . is the average of the values of the response
observed at the ith design point. The sura of squares due to
lack of fit can be obtained by taking the difference
SSrnp = SSE
LOF
SSE
pure
This partitioning of the residual sum of squares is
displayed in the analysis of variance table in Table 1.
Table 1. Analysis of Variance—
Partitioning the Residual Sura of Squares.
Source
Sum
Degrees
Mean
of Variation
of Squares
of Freedom
Square
Regression
bjX'Y - (l'Y)2/N
p - 1
Residual
SSE
N - p
MSE
Pure Error
SSEpure
N - n
MSEpure
Lack of Fit
SSLOF
n - p
msLOF
Total(corrected)
Y'Y - (l'Y)2/N
N - 1
b-^ represents the ordinary least squares estimator of (3 in
model (2.1), b^ = (X'X)-1X'Y, and 1 is an Nxl vector of
ones.

23
To test the hypothesis of zero lack of fit, that is
Hq: lack of fit = 0 or E(X) = Xfcj., an F statistic is formed
F
MS
LOF
MSE
pure
(2.3)
which possesses a central F distribution if the true model
is of the form (2.1), but has a noncentral F distribution if
the true model is of the form (2.2). In other words
F
F
n-p,N-n
under HQ: E(Y) = Xg^ , and
F ~ F'
n-p,N-n;X 2
under H^: E(Y) = Xg^ + X2£2 ’ where ^2 *s tlie noricentrali
parameter x2 = g^(X2~XA)'(X2-XA)g2/2a2, and A = (X,X)_1X,X2*
Under Ha, E(MSLQF) = o2 + g^(X2 - XA)'(X2 - XA)g2/(n-p) and
E(MSEpure) = a^ (Draper and Smith, 1981, p. 120), hence Hq
is rejected in favor of Ha if the value of F in (2.3)
exceeds the upper 100a percentage point of the central F
distribution, Fa;n-p,N-n* when Hq is rejected, we conclude
that a significant lack of fit is present.
Draper and Herzberg (1971) demonstrated that the lack
of fit sum of squares can be partitioned into two
statistically independent sums of squares, SSL^ and SSL2,
when there are replicate observations at the center of the

24
response surface design and when the basic design without
center points is nonsingular. If the true model and the
fitted model are of the same form as in equation (2.1) then
the two F ratios Frl = [ssr1/(n - p - 1)1/MSE and
FL2 = ssL2/MSEpure are both distributed as central F random
variates, with respective numerator and denominator degrees
of freedom (n - p - 1), (N - n) for FL1 and 1, (N - n) for
FL2* the true model is of the form shown in equation
(2.2), then FLj_ and FL2 are both distributed as noncentral F
random variates. The expected values of SSLi and SSL2 are
used to show what functions of S-2 are testable with FL1 and
fL2*
Two examples are presented by Draper and Herzberg to
illustrate this testing for lack of fit. The first example
makes use of a first order orthogonal design in k factors
augmented with center point replicates for fitting a first
order polynomial model. If the true model is of the second
order, then FL2 can be used to test a hypothesis concerning
the parameters associated with the pure quadratic terms in
the model. If all such parameters are zero, then F^
provides a check on interaction terms. The second example
illustrates the fitting of a second order polynomial model
to a second order design with all odd design moments of
order six or less zero. If the true model is third degree,
then Fl1 can be used to test the significance of the third
order terms, while FL2 tests terms of order greater than
three. The partitioning of SSLQF into SSLi and SSL2 and the

25
corresponding tests of hypotheses are also given in Myers
(1971, p. 114-119), for the special case of fitting a first
order polynomial model to a 2^ factorial or a fraction of a
2^ factorial design augmented with center point replicates
and the true model is of the second degree.
A more complete partitioning of the lack of fit sum of
squares in an attempt to obtain a more detailed diagnosis of
the lack of fit of the fitted model is given in a technical
report written by Khuri and Cornell (1981). The lack of fit
sum of squares, which has n - p degrees of freedom, is
partitioned into n - p independent sums of squares, each
having one degree of freedom. The expected values of these
single degree-of-freedom sums of squares are used to
identify at most n - p linearly independent causes for the
lack of fit variation. Tests of significance are performed
on the assumed contributing causes. This method enables the
screening of all subsets of g.2 in order to identify those
subsets which are most responsible for lack of fit of the
fitted model.
We shall now discuss the second general approach used
in lack of fit testing, which is to test for lack of fit by
making use of response values observed at points which are
near neighbors in the factor space when true replicate
observations are not available.

26
2.3 Testing for Lack of Fit Without
Replicated Observations—Near Neighbor Procedures
Green (1971) suggests the following approach when
testing for lack of fit if there are no design points at
which replicate observations of the response are
available. The N observed values of a response, Y,
considered a function of only one variable, x, are divided
into g groups, by grouping observations which have similar
values of x. Green hypothesizes a model of the form Y= H& +
e., where Y is an Nxl vector of observable responses, H is an
Nxm matrix whose columns correspond to known functions of
the variable, x, g. is an mxl vector of unknown regression
coefficients, and g. is the Nxl vector of random errors,
e ~ Nn(0, »2In).
Green's method assumes that the vector of differences
(EY - Hg.) can be well approximated by a dth order polynomial
in x within each of the g groups, d > 1. An alternative
model of the form
Y = H v + n + £
is given, where £. is distributed as NN(Q, a2!^) , Hj_ is an
Nx [g (d + 1) + mjJ matrix of known constants, y is a
[g(d + 1) + ntjjxl vector of regression coefficients, and el,
as Green states is "a small vector." The first g(d + 1)
columns of H-^ correspond to the polynomial terms for the g
groups (with (d + 1) terms for each group), the rightmost
m^ < m columns in correspond to terms that are in the

27
fitted model, but are not represented among the g(d + 1)
polynomial terms in the alternative model.
Under the assumption that a = D, the presence of lack
of fit is tested by using the test statistic:
Y'[H1(H|H1)"1H| - ]Y/[g(d +1) + - m]
Y'[I - Y/[N - g(d + 1) - mj
(2.4)
This statistic is of the same form as the F statistic used
in the standard multiple regression test of a postulated
model against a more general one which includes the
postulated model as a special case. Lack of fit is
suspected if the calculated F ratio in (2.4) is greater
than Fa ;g (d+1 )+m1-m, u-gid+D-nij. where this latter quantity
is the upper 100a percentage point of the central F
distribution.
Green notes that when there is no lack of fit, the
quadratic forms Y'[H±(HjH±)_1H^ - H(H'H)_1H']Y and
Y'[l - (H|H^)~^H|]Y are distributed independently as
a2x^ with g(d +1) + m-^ - m and N - g(d +1) - m^ degrees of
freedom, respectively. In this case the F ratio in (2.4)
possesses a central F distribution. If there is lack of fit
on the other hand, then these two quadratic forms are
distributed as noncentral chi-squares, multiplied by a2,
with respective noncentrality parameters

28
S 1 = [hxv + n ] ’ [ ( H|H x) 1H’_ - H(H'H) 1H’][h;lv + n]
and ? ^ = n'[l - ^H^]n . Thus the assumption that
n = 0 can affect the power of the test, since if n * o , the
expected value of MSE is greater than o^, where MSE is the
quadratic form in the denominator of the F ratio. Hence if
n * 0 , the probability of calculating a large F value is
reduced, and we are less likely to detect lack of fit using
an upper tailed rejection region.
Daniel and Wood (1971) suggest another method for lack
of fit testing when replicated observations of the response
are not available. They make use of "near replicates" to
obtain an estimate of a, which is the standard deviation of
the observable responses in the true model. The value of
the estimate o is compared to the square root of the
residual mean square from the analysis of the fitted
model. Lack of fit is indicated if the square root of the
A
residual mean square is large compared to the estimate o.
To determine when observations are near replicates so that
an estimate of o can be found, they define the squared
distance between any two data points, j and j’, to be
measured by
K
I [b
i =1
where j and x^j. are the values of the ith independent
variable corresponding to the observations yj and Yj'/
respectively, i = 1, 2, ..., K, and bj_ is the ordinary least

29
squares estimate of the ith regression coefficient. In the
denominator, s^ is the square root of the residual mean
square for the fitted model.
To obtain an estimate of a from near replicates, let
And = |dj - dji |, n = 1, 2, ..., (^), where dj and dj. are
the residuals at points j and j', respectively, and where
there are N data observations in the experiment. Since the
expected value of the range for pairs of independent
observations from a normal distribution is 1.128o, a running
average of the And's is calculated and their average is
multiplied by .886 = (1/1.128) to get a running estimate,
sn, of o. That is, sn = .886 £ And/n . The closest pair
of observations as judged by D?j, is used to begin the
running estimate, the next closest pair (next "nearest
neighbors") is used for A2d/ an<^ the Procedure continues
until sn "stabilizes." The stabilized value of sn is used
to estimate a.
A third method for testing for lack of fit without
replication is given by Shillington (1979). The fitted
model is of the form
Y = X8 + e (2.5)
where Y (N*1), X (Hxp), and 8 (pxl) are defined as in (1.2)
and e ~ NN(0, o^i ) . The test for lack of fit of the
fitted model is a test for whether the true model has the
form
Y = X8 +5 + e ,

30
where 5 (Nxl) is a fixed effect quantifying the departure of
(2.5) from the true model.
Shillington assumes that the data can be grouped into g
cells, with nj observations in the jth cell, determined in
advance. Letting Cj refer to the jth cell, j =1, 2, ...,
g, a vector of cell averages is written (gxl), where the
jth element of is the average of the observed responses
in Cj. The matrix X^ of independent variables associated
with Yq is the gxp matrix where the elements in the jth row
are x'. = E x!./n . , that is, row j of Xr is the row
-O i=1 "ID D
vector x'. . The matrix Xr is assumed to be of full rank
-• D c
p < g. Also within each cell are defined the differences
W. . = Y.. - Y . , i € C. , j =1, 2, ..., g, where Y . is
1D ID -D D J -D
the jth element of Yc.
The two independent data sets, Y^ and (W —} with g and
N - g degrees of freedom, respectively, are used to find two
independent estimates of a2. The first estimate is written
as
MSE
B
9
E
j=l
n • (Y
D
- s.y-B>
V(g - p)
A
where is the weighted least squares estimate of §. using
the regression of cell means, Yc, on Xc. The second
estimate of a2 uses the within cell deviations on cell
means, and is
MSET7 =
W
g
E
j=i
n .
D
E
i=l
(W.. - W..) /(N - g - r),

31
where r is the rank of an N*p matrix with rows equal to
x!. - x'. , i e C., j =1, 2, .., g. If the matrix of
-1D — • D D
independent variables, corrected for cell means, is of full
rank, then r = p. Here VJ— is the estimate of W^j from the
regression of cell residuals ÍW^j} on the associated vectors
of independent variates, x! . - x' . .
-13 ~ • 3
If the fitted model is the correct model, then MSEg and
MSEW are independent estimates of o2 and the ratio MSEB/MSEW
is an F statistic with g - p and N - g - r degrees of
freedom. When all observations in a cell have the same
settings of the independent variables, that is, the
observations are truly replicates for all cells, then this F
statistic is identical to the F statistic in the usual lack
of fit test in which the residual sum of squares is
partitioned into lack of fit and pure error sums of squares,
as given in Draper and Smith (1981, p. 120).
If the true model is Y = X8 + 6 + e , however, and if
we let X'<5 =0 and a2 = 6'6/N, then
E(MSEb) = a2 + Í¿[I - XC(X¿XC) 1X¿]?B/(g-p)
where ó (g*l) has jth component equal to l
~B i=i
Furthermore, with this latter true model form
<5 . . /n .
ID D
E(MSEw) =
6 ' (i -
-wl
xwlxw
V"lxw>V,N - 9 - r)

32
where 6_w has the components <$ — - i e Cj, j = 1, 2,
g. The matrix XTs7 (N*p) has the rows x! . - x'
w -1 j -. j
i £ Cy j = 1, 2, g. The power of the F test,
F = MSEB/MSEW, depends on the relative bias of the estimates
of o2f that is, the biases in MSEB and MSEW.
Shillington states that the power of the F test which
makes use of F = MSEB/MSEW is maximized by forming cells so
that the bias of E(MSEW) is minimized. This is the same as
forming cells so that the within cell variation in 6 is
minimized. Shillington (1979, p. 141) also states,
"Observations with near covariate (independent variable)
values might be expected to have similar §. values, since we
assume that 6 varies in some continuous but unknown fashion
with X. This justifies the usual procedure of forming
groups by collapsing observations with adjacent covariate
values. Indeed, if covariates do not vary within cells we
have the usual lack of fit test and maximum power."
By imposing a further structure on the form of <$, it is
shown that if the F test has an upper tailed rejection
region, the power is maximized by selecting the group sizes
as nj = 2, j = 1, 2, ..., g. Finally, Shillington suggests
that in the presence of more than one independent variable
problems in grouping may arise, and in this case it may be
wise to perform a different lack of fit test for each
parameter. Following this approach, an example is given
which suggests testing lack of fit for each of the p
independent variables separately may be more powerful than

33
trying to form groups based on all independent variables at
once.
In summary, all the approaches we have discussed for
testing for lack of fit when replicate observations of the
response are not available at any of the settings of the
independent variables make use of grouping the observed
response values according to similar values of the
independent variables. The observations falling in such
groups are referred to as "pseudoreplicates" or "near
neighbor observations." These pseudoreplicates are used to
estimate the true variance of the observations, o^, but a
completely unbiased estimate of a^ cannot be attained unless
true replicate observations are available. In each case,
the power of the lack of fit testing procedure is reduced
because an unbiased estimate of is not attainable. We
now turn to the use of check points for lack of fit testing.
2.4 Testing for Lack of Fit with Check Points
An alternative to the two approaches to lack of fit
testing already discussed is the method which makes use of
check points. We assume a model of the form E(Y) = , as
given in (2.1), is fitted in a response surface system, but
that the true model is of the form E(Y) = Xf^ + X2B2 as
given in (2.2). The parameters, s.^, the fitted model are
estimated by ordinary least squares techniques, making use
of the values of the response observed at the design
points. After the model is fitted, values of the response
are observed at additional points in the experimental region

34
called "check points." The observed response values at the
check points are compared to the values which the fitted
model predicts at these same check points. It is important
to note that the observed values of the response at the
check points are not used in fitting the model initially.
Snee (1977) gives four methods of validating regression
models, one of which is the collection of new data to check
predictions from a previously fitted model. In a designed
experiment these new data take the form of check points.
Snee suggests that the inclusion of a small number of check
points in any designed experiment is a "worthwhile"
procedure.
Scheffe (1958) proposed a test for lack of fit when the
{3,2} simplex lattice design is used for fitting a second
order canonical polynomial model in three mixture
components. It is desired to use the observed value of the
response at (1/3, 1/3, 1/3) as a check point blend. The
test statistic proposed is the t statistic of the form
[var(Y - Y)]1/2
where Y is the observed value of the response at the check
point, and Y is the value of the response predicted at the
same point by the second order model which is fitted by
ordinary least squares techniques to the observed response
values at the six design points of the {3,2} simplex
lattice. The response value observed at the point

35
(1/3, 1/3, 1/3) is not used in fitting the model. Lack of
fit is inferred if the absolute value of the calculated t
value in equation (2.6) is larger than the corresponding
tabled t value.
In the denominator of the t test of equation (2.6), the
variance of the difference Y - Y is shown to be
A A
var(Y - Y) = var(Y) + var(Y)
= ( 44/27r)c 2 ,
when r replicates are taken at each design point. The
estimate of the variance of Y - Y is (44/27r)o2, where a2 is
calculated from the replicated response values at the design
points.
Scheffe (1958) also alludes to a test for lack of fit
when several check points are used simultaneously. When
there are k check points, the test for lack of fit is an F
statistic of the form
F
d
(2.7)
where d* = (Y1 - Yr Y2 - Y2, ..., Yk - Yk) , and V = o2VQ =
var(cl). Formulas are given for the elements of Vg in the
special case when the check points are the design points of
the {3,2} simplex lattice. Lack of fit is suspected if the
calculated value of the F statistic given in (2.7) is larger
than the corresponding tabled F value.

36
Gorman and Hinman (1962) suggest the same t test in
equation (2.6) that Scheffe (1958) suggested for a check
point taken at (1/3, 1/3, 1/3) to test for lack of fit in a
second order polynomial model fitted from a {3,2} simplex
lattice design. They suggest using (1/3, 1/3, 1/3) as the
location of the check point because the observation at this
point may later be used to fit the next more complex model,
the special cubic, if the second order model is found to be
inadequate. They state that in general for the second order
polynomial model as well as higher order models, check
points should be taken in regions of particular interest, of
which there are usually many in any blending study.
Further, they suggest that the number of check points
depends on individual experimental situations — technical
background, precision required, cost of materials and
analyses, and probability of requiring a more complex
model. However, no specific criterion is given by Gorman
and Hinman for selecting the location of the check points.
Gorman and Hinman (1962) indicate that a t test at a
check point other than at (1/3, 1/3, 1/3) takes the same
form as the statistic of equation (2.6),
[var(Y) + var ( Y) ]
with the additional condition that if several check points
are taken, say for example k points, the method of checking

37
the fit is to compute the t value at each location and refer
these calculated t values to the 100(a/2k) percentage point
of the central t distribution rather than the 100(a/2)
percentage point.
Kurotori (1966) gives an example of a mixture
experiment where the response is the modulus of elasticity
of a rocket fuel, which is a mixture of three components,
binder (x^), oxidizer (X2), and fuel (X3). The factor space
of feasible mixtures is a subspace inside the two-
dimensional simplex or triangle where all three components
are present simultaneously. "Pseudocomponents" are defined
and in the pseudocomponent system a special cubic model is
fitted to data collected at the points of the q = 3 simplex
centroid design (Figure 4). A check for adequacy of fit is
made by using three check points and the response values at
the check points are used only for testing the fit of the
model and not for fitting the model initially.
The reason for the choice of the particular check point
locations by Kurotori is that, as he states, "They are the
most remote mixtures from the seven design points." The
lack of fit test is an F statistic of the form
2
F = -I2- (2.8)
a
3
2 J â– * 2
where s = Z (Y. - Y.) , for the i = 1, 2, 3 check points
-2 . 1=1 1 1
and a is an estimate of measurement error from a previous
analysis. Kurotori admits that the use of the F statistic

38
Figure 4. Kurotori's rocket fuel example,
xl'f x2'' anc^ x3' rePresent pseudocomponents.
in Eq. (2.8) for lack of fit testing may be risky because
the predicted values at the check points are correlated
(correlation of .5), although the observed values are not
correlated. Kurotori suggests individual t tests as
proposed by Scheffé (1958) might be the preferred procedure.
Snee (1971) repeats Kurotori's rocket fuel example
using the same F test for lack of fit as Kurotori and makes
the comment that the Y^'s at the check points are
correlated. In stating that the F test is not an exact
test, he nevertheless offers no solution in the form of an
exact test.

39
In summary, only Scheffe refers to an exact F test when
several check points are considered simultaneously for
testing for possible lack of fit of a model fitted in a
mixture space, and his development is limited to the special
case where the check points are the design points used to
fit the model initially. No criterion is proposed by
Scheffe for selecting other locations for the check points.

CHAPTER THREE
AN OPTIMAL CHECK POINT METHOD FOR TESTING
LACK OF FIT IN A MIXTURE MODEL
3.1 Introduction
In Chapter Three we investigate the problem of testing
for lack of fit of a linear model fitted in a mixture
space. The testing is to be accomplished with the use of
check points. We assume that an experimental design is
specified, and that the fitted model is of the form
E(Y) = XSi (3.1)
where Y is an Nxl vector of observable response values, X is
an Nxp matrix of known constants and rank p, and 3^ is a
vector of p unknown regression coefficients. The true model
is assumed to be of the form
E (Y) = XB + X ¡J (3.2)
where X2 is an Nxp2 matrix of known constants and is a
vector of p2 unknown regression coefficients. Throughout
our development, we will assume that the random vector Y has
the normal distribution with variance-covariance matrix
equal to a^IN*
40

41
In our investigation we wish to determine the proper
testing procedure to follow in deciding whether the fitted
model exhibits lack of fit. In order to optimize the lack
of fit testing procedure, we will determine the location of
the check points so that the power of the test is maximized.
3.2 Testing for Lack of Fit in the Presence of
an' External Estimate of Experimental Error Variation
3.2.1 The Test Statistic
We wish to test the performance or fit of a fitted
model in a mixture space when the true model possibly
contains terms in addition to those in the fitted model.
The fit of the model is to be tested by a test which makes
use of the response values observed at certain locations
called "check points" in the experimental region, by
comparing them to the values which the fitted model predicts
at the same check points. The observed values at the check
points are not used for estimating the coefficients in the
fitted model and are assumed to represent the values of the
true surface at the check points.
Let us define the vector of differences
d = (Y* - Y*)
- /V* — Y* Y* — Y* Y* — Y*\'
^ 1 1' 2 2' •**' k k'
where Yí, i = 1, 2, ..., k are observed response values at
k check points and Y£, i =1, 2, ..., k are response values
predicted at the k check points by the fitted model,

42
Yi = x*'b^, where is the ordinary least squares estimator
of and where x£' is the ith row of X*, the kxp matrix
whose columns are of the same form as the columns of X but
with its rows evaluated at the k check points. Note that
if S2 = 0, then E(d) = 0 and if 02 * 0/ then
E(d) = (X* - X*(X'X)“1X,X2)e2. Let V represent the
variance-covariance matrix of the random vector d.
2
Then v = a Vq where
VQ = Ik + X*(X,X)“1X*'
and where 1^ is the identity matrix of order kxk.
We assume that an unbiased estimate of a^ is available
~ 2
and we denote this estimate by a where the subscript ext
" 2
stands for external, and aext is independent of the model
being fitted. The test statistic for the hypothesis of zero
lack of fit Hq : E(d) = 0 is
d 'V'-’-d/k
F = (3.3)
aext
(see Scheffe, 1958, p.358). It will be shown later in this
section that the F ratio in Eq. (3.3) possesses either a
central F distribution or a noncentral F distribution,
depending upon whether the true model is represented by Eq.
(3.1) or Eq. (3.2).
A 2 .
The variance estimate a ^ that appears m equation
ext
(3.3) is ordinarily generated from replicated observations

43
at some of the design points in the experiment. We assume
- 2
that a ^ is a constant multiple of a central chi-square
ext
random variable with v degrees of freedom. This is written
as
a2 . = SSE /v
ext pure'
= (a2/v)(SSEpure/a2)
2 2
where SSEpUre/° ~ Xv• Note that SSEpUre denotes the
portion of the residual sum of squares due to replication
variation from the fitted model. The residual sum of
squares from the fitted model may be partitioned into
SSEpUre and SSL0F only if replicated observations are
collected at one or more design points. For the case where
replicate observations are collected at all of the design
points
SSE
pure
n n.
E E1
i=l j=l
2
where n is the number of distinct design points, n, > 2 is
the number of replicates at the ith design point, Y^j is the
jth observation at the ith design point, and Y. is the
1 •
average of the n^ observations at the ith design point.
n
Here SSEpure has v = E (n. - 1) degrees of freedom.
i=l
When the fitted model and the true model are of the
same form as defined by Eq. (3.1), the quantity d'Y^d/a2

44
possesses a central chi-square distribution (Searle, 1971,
p.57, Theorem 2). However, when the true model is of the
form specified by Eq. (3.2), d'V^d/a2 possesses a
noncentral chi-square distribution. Thus when the true
model is of the form in Eq. (3.1),
d'V'Va2 ~
but when the true model is of the form in Eq. (3.2),
d-v'Va2 ~
where in the second case the noncentrality parameter X^ has
the form
= E (d ) •VQ1E(d)/2a2
= §.¿(X* - X*A)'V"1(X* - X* A) g 2/2a2.
The matrix A = (X'X) ^X'X2 is called the alias matrix and is
of order PXP2* In the matrix X* is of order kxp2 and
has the same relationship to X2 as X* has to X.
2
Since SSEpure/a is statistically independent of
-12 .
d'Vg d/a , then under model (3.1) the test statistic

45
F
d•V01d/ko2
s^Epure^vo 2
d'V^d/k
= *~2
°ext
will have a central F distribution. When the true model
contains terms in addition to those in the fitted model then
F will have a noncentral F distribution. We write these two
cases as
F ~
F
k, v
under model (3.1), and
under model (3.2), where the noncentrality parameter is
X1 = £-2(X2 " X*A) ’ Vq1 ( x2 " x*a)§.2/2°2-
3.2.2 The Testing Procedure and an Expression for the Power
the Test
Given that the form of the fitted model is defined as
Eq. (3.1), the expected value of the numerator of the F
statistic in Eq. (3.3) will depend on the form of the true
model. For the case where the true model is expressed as

Eg. (3.2),
46
E(numerator) = Efd'v'^d/k)
= «’2/k >■*]£,
= ( o 2/k ) (k + 2A 1)
2 2
= a + 2o ^A /k
= o2 + i^A^/k, (3.4)
where A^ = (X^ - X*A)'V01(X^ - X*A). However, when the true
model is Eq. (3.1), 3^ = 0 and in this case A^ = o so that
2 ~ 2
E(numerator) = a . Also o , is an unbiased estimator of
ext
o 2 and
E(aext} = °2’ (3.5)
Therefore the ratio E(numerator)/E(denominator) where
* 2
the denominator is a , will equal unity under model (3.1),
that is, when there is no lack of fit. Under model (3.2),
the ratio will be greater than or equal to unity so lack of
fit should be suspected if the calculated F ratio in
equation (3.3) is large. We can thus use an upper tailed
rejection region to reject the hypothesis of zero lack of
fit. The power of the test is

47
v j\i
> F
a ; k, v
}
where F^ ^ v is the upper 100a percentage point of the
central F distribution with k numerator degrees of freedom
and v denominator degrees of freedom.
It is worth noting that from Eq. (3.4) and Eq. (3.5)
testing the hypothesis that 0 = 0 is equivalent to testing
the hypothesis that Xj_ = 0, assuming A-^ is positive
definite. Thus testing a null hypothesis of zero lack of
fit using the proposed testing procedure involving the F
ratio in (3.3) may be expressed as a test of the hypotheses:
H
o:
0
3.2.3 A Method for Locating Optimal Check Points
Once a design for fitting model (3.1) in a mixture
space is chosen and the number of simultaneous check points
is decided on, say k > 1, the next step is to determine
where in the mixture space we should place the k check
points so as to maximize the power of the test for lack of
fit. The location of the check points is to be made
independently of the value of 02«

48
The power of the upper tailed F test for lack of fit is
an increasing function of X ^ (see Appendix 1 for proof, with
X^ = 0). Therefore, to maximize the power of the test we
maximize the value of X^ defined as
2
Xf = 02Ai&2//^a
where A^ = (X* - X*A)'Vq1(X* - X*A), by properly selecting
the k check points whose coordinates are defined in X*. To
maximize the value of X-^, we shall concentrate on the matrix
A1‘
The matrix A-^ is a square matrix of order P2X?2 and
a scalar quantity when p2 = 1. By maximizing the scalar
quantity A^ with respect to the k check points, the power is
maximized no matter what the value of 0^* Maximizing the
scalar A^ can be accomplished by using The Controlled Random
Search Procedure given by Price (1977). This procedure is
described in Appendix 2. As a computational aid, Aj_ can be
expressed as
VQ + (X* - X*A)(X* - X*A)'
when p2 = 1/ where the symbol |B| denotes the determinant of
the square matrix B. Thus the computations reduce to
evaluating two determinants rather than inverting VQ (see
Scheffe, 1959, Appendix V, p.417).

49
When p2 > 1 and A^ is no longer a scalar, maximizing X
(and thus maximizing the power of the test) cannot be
accomplished without specifying 0 . In this case we make
use of a lower bound for Xj_ (Graybill, 1969, p. 330, Theorem
12.2.14(9)) defined as
tJmin-2-2/2°2 < X1
(where u . is the smallest eigenvalue of A-, ) to be used in
min r i
place of X-^. Hence an approximate solution to the
maximization of X^ will be achieved by finding the k
simultaneous check points (using Price's procedure) that
maximize \i . , the smallest eigenvalue of Ai . In other
min i
words when p^ > 1* and in order to avoid specifying 0^, we
seek to maximize a lower bound value for X^. This
maximization does not depend on the value of 02«
There are cases where the matrix Aj. is of less than
full rank (less than rank P2) or equivalently where the
matrix Ai is positive semi-definite so that u • will be
equal to zero no matter which check points are selected.
One such case occurs when k < p^ (when the number of check
points is less than the number of parameters in the true
model which are not in the fitted model) since when k < p^
rank(A^) = rank[v~ /2 = rank(X* - X*A),

50
and so rankfA^ $ min(k, p2) because the matrix (X* - X*A)
is of order kxp2. Therefore when k < p , the rank of A^ is
at most k so that A, is of less than full rank. Since u
J- min
must be equal to zero when A^ is positive semi-definite, an
alternative method to that of maximizing y . to select
optimal check points must be found when A^ is positive semi-
definite in order to produce a positive lower bound for X ^.
In this pursuit, let us write X-^ as
X^ = §-2 A^É.2^^a
= 0¿PAP'02/2 a2
= 0 ¿[ Pj. :P2] ^ ia9 [A !, A2 = 0] [P1:P2] '02/2a2
= 0^P1A1P[e2/2a2
where A is a diagonal matrix with elements equal to the
eigenvalues of A-^, P is an orthogonal matrix whose columns
are orthonormal eigenvectors of A^, A^ and P^ correspond to
the positive eigenvalues of A^, while A2 = 0 and P2
correspond to zero eigenvalues of A^. Then by Theorem
12.2.14(9) in Graybill (1969) we can write
y + . z'z/2a2 < X. (3.7)
mm- - 1
where y + . is the smallest positive eigenvalue of Ai , and
Mmm

51
z = Pí^2* T^us by Eq. (3.7), an approach to maximizing a
positive lower bound for X^ when is positive semi-
definite is to select check points that maximize the
smallest positive eigenvalue of A^. It must be noted,
however, that this method can only be used when
0_2 e n C(P^), where C(P^) denotes the column space of Pj_
and n CiP^) denotes the intersection of all such spaces
which can be obtained at all possible check points
locations. This is because, in general, z'z in (3.7)
depends on the location of the check points through its
dependency on P^. If, however, &2 e n CfP^, then
z'z = b^Pj^í^ = e¿pp'e2 = *2-2’ since §-2p2 = °*
+ 2
It follows that when &2 e n CfP^, %inz'z/2o
+ 2 +
= u • and only n . depends on the location of the
mm-2-2' 1 Hmm
check points.
3.3 Testing for Lack of Fit When MSE Is Used
to Estimate Experimental Error Variation
3.3.1 The Test Statistic
In this section we shall show that when an external
estimate of is not available and the residual mean square
(MSE) from the fitted model of the form (3.1) must be used
as an estimate of o^, the test statistic
F
d'v01d/k
MSE
(3.8)
possesses a central F distribution when the true model is

52
Eq. (3.1), but possesses a doubly noncentral F distribution
when the true model is Eq. (3.2).
In the initial section of this chapter, the quantity
-1 2
d'V0 d/a was said to possess a central chi-square
distribution or to possess a noncentral chi-square
distribution, depending on whether the true model was
specified by Eq. (3.1) or Eq. (3.2). Now, the residual sum
of squares from the fitted model is defined as
N
SSE = I (Y. - Yâ– )2
i=l 1 1
= Y'(I - X(X,X)-1X')Y
and it is easy to show (Searle, 1971, p.57, Theorem 2) that
SSE/a2 possesses a central chi-square distribution if the
true model is Eq. (3.1), but under model (3.2), SSE/a2
possesses a noncentral chi-square distribution. This is
expressed as
SSE/a2
2
XN-p
SSE/a2
X
,2
N-p,
X2
under model (3.1), and

53
under model (3.2), where the noncentrality parameter X2 is
x2 = e¿(x2 - XA)'(X2 - XA)32/2a2.
The distributional form of the test statistic in Eq.
(3.8) is derived by knowing that the quantities
-12 2
d'V0 d/a and SSE/a are statistically independent (see
Appendix 3), so that
d'V^d/ko2
F = 2
MSE/a
d'V^d/k
MSE
is distributed as a central F when the true model is Eq.
(3.1), but when the true model is Eq. (3.2) the F ratio is a
doubly noncentral F, that is, under model (3.1),
F
F
k, N-p
and under model (3.2),
F ~ F"
k, N—p ; X 2. i X 2
3.3.2 The Rejection Region and its Relation to the Power of
the Test
In Appendix 1 it is shown that if k, N-p, and X2 are
fixed, then the power of the F test using the ratio (3.8) is

54
a function of the location of the rejection region (upper
tailed or lower tailed) of the test. The power increases
with increasing values of the numerator noncentrality
parameter, X^, when the test is an upper tailed test. The
power decreases with increasing values of when the test
is a lower tailed test. This means that to study ways of
increasing the power of the test, we have to determine
whether the test is an upper tailed test or a lower tailed
test. Similarly, for fixed values of k, N-p, and X-^, the
power of the F test is a decreasing function of Xj for an
upper tailed test, and is an increasing function of X2 when
the F test is a lower tailed test (Scheffé, 1959, p. 136-
137) .
To decide if the test is an upper tailed test or a
lower tailed test, we recall from Section 3.2.2 that if the
true model is Eq. (3.1) then the expected value of the
numerator of the F statistic in (3.8) can be written as
E(numerator) = o^,
and if the true model is Eq. (3.2),
2 2
E(numerator) = a + 2a X ^/k
= a ^
(3.9)

55
where the p^xp^ matrix A-^ is A^ = (X* - X*A) ' Vq^ (X* - X*A) .
Similarly, it can be shown that if the true model is Eq.
(3.1), the expected value of the denominator of the F
statistic in (3.8), where the denominator equals MSE, is
E(denominator) = E(MSE)
but if the true model is Eq.(3.2),
E(denominator) = E(MSE)
■- t°2/(N - p)1Exn-p,a2
- [o2/(N - p) ] [ N - p + 2A2]
= a2 + 2a2X2/(N - p) (3.10)
= a2 + 8¿A2B2/(N - p)
where the P2XP2 matrix A2 is A2 = (X2 - XA)'(X2 - XA). Thus
the ratio Enumerator )/E(denominator) will equal unity if
the true model is Eq. (3.1), but if the true model is Eq.
(3.2), the ratio is greater than unity if B^A^B2/k >
-2A2-2^N - P) • In this latter case we reject the null

56
hypothesis of zero lack of fit if the calculated value of
the F ratio in (3.8) is large. An upper tailed rejection
region seems reasonable for this test. When the true model
is Eq. (3.2), and if & /k < -2A2-2/(N “ p)' then a lower
tailed rejection region is preferred.
3.3.3. A Method for Locating Optimal Check Points
Given a design for fitting a model of the form in Eq.
(3.1) in a mixture space (note that fixing the design fixes
A2 and (N - p)), and given the number of simultaneous check
points desired, k > 1, we now wish to determine where in the
mixture space the k check points should be located so as to
maximize the power of the F test for lack of fit, where the
test statistic is given in Eq. (3.8). We also wish to
position the optimal check points in a manner that is
independent of the values of the elements in 32*
The case of an upper tailed test. To help us find k
simultaneous check points that maximize the power of an
upper tailed test, we shall make use of the fact that the
power is an increasing function of XTherefore to
maximize the power of the upper tailed F test, we shall seek
the locations of the k check points that maximize X-^.
As in the case considered in Section 3.2.3, where the
test statistic had a noncentral F distribution, if the
number of extra terms in the true model is P2 = 1, then
maximizing Xis equivalent to maximizing the scalar A^ •
However, as before, if p2 > 1, then the P2XP2 raatr:'-x Ai is
not a scalar and we will have to approximate the

57
maximization of X^ bY maximizing a lower bound for X. This
is done by finding the maximum value of y . , the smallest
3 nun
eigenvalue of A-^, since
u • 813^/2a
min-2-2'
< X
1’
When the number of check points is less than the order
of the square matrix A-^, that is, k < p£, then rank(A^) <
min(k, p9), and A, will have y . =0. For this case, we
again try to maximize the smallest positive eigenvalue of A-^
which we denote by /. , while remembering from Section
min
3.2.3 that this technique is limited to situations where
82 £ n C(P^) .
The case of a lower tailed test. To find k check
points to maximize the power of a lower tailed test, we make
use of the fact that the power of the lower tailed F test
increases as X ^ decreases. Then if P2 = 1 and Aj_ is a
scalar quantity, X^ can be minimized with respect to the k
check points by finding the check points that minimize A^•
If p > 1, then by Theorem 12.2.14(9) in Graybill (1969), we
see that an upper bound for X1 is
X1 < umax^2^2/2a
(3.11)
where umax is the largest eigenvalue of A^. An approximate
solution to minimizing X^ in (3.11) can be achieved by
minimizing u . It is not necessary to treat the case
3 max J

58
of k < p2 separately here, although will equal zero if
32 is in the column space of P2, where P2 is the matrix
whose columns are orthonormalized eigenvectors corresponding
to the zero eigenvalues of the matrix A^.
3.3.4 Determining Whether the Test Is Upper Tailed or Lower
Tailed
The procedures outlined in Section 3.3.3 produce a set
of k check points that simultaneously maximize the power of
the upper tailed test as well as a second set of k check
points that simultaneously maximize the power of the lower
tailed test. The check points that are selected maximize
the power, given A2, k, and N - p without specification
of g_2' except that when A-^ is positive semi-definite we
require that 3^ e n C(P^).
It is now necessary to decide which of our two
candidates will be used for a lack of fit test. To choose
between the upper tailed test and the lower tailed test, let
us consider the quantity
R = [A1/k] - [A2/(N - p)].
If R is positive definite when the true model is Eq. (3.2),
then no matter what the value of 32 is, the ratio
E(numerator )/E(denominator) will be greater than unity,
implying an upper tailed test is to be used. Similarly, if
R is negative definite, then a lower tailed test should be
used. Finally, if R is not definite, then neither an upper
nor a lower tailed test is implicated and further

59
investigation is necessary. The criterion of R = [ A /k]
[A^/(N - p)] may yield any of the four following cases.
Case 1. If R = [ A /k] - [ A /(N - p) ] is positive
definite when is generated by the k optimal upper tailed
test check points, and R is not negative definite when A-^ is
generated by the k optimal lower tailed test check points,
then we recommend that the check points be used that yield
the optimal upper tailed test with an upper tailed rejection
region.
For Case 1 it is necessary for A^ to be positive
definite (see Appendix 4). Since A^ is a square matrix of
order P2XP2 with rank(A^) < min(k, p2), then A^ can be
positive definite only if k > p^. Thus, there must be at
least p2 check points for Case 1 to hold, where p2 is the
number of terms in the model of Eq. (3.2) that are not in
the model of Eq. (3.1).
From inspection of equations (3.9) and (3.10), it is
apparent that the testing for lack of fit in Case 1 is
equivalent to testing the hypothesis
1
(3.12)
against the alternative
X
1
k

60
since R = [A^/k] - [A2/(N-p)] is positive definite when the
true model is Eq. (3.2). In Appendix 5(a) it is shown that
under Case 1, the hypothesis given by (3.12) is equivalent
to the hypothesis
H
o:
0.
Case 2. In Case 2 we assume that R = [A^/k] -
[A^/iN - p)] is not positive definite for the k optimal
upper tailed test check points, but that R is negative
definite for the k optimal lower tailed test check points.
Here we recommend that the lower tailed test check points be
used with a lower tailed rejection region.
It is necessary for A2 to be positive definite for Case
2 to occur (see Appendix 4). However, A-^ need not be
positive definite, and so k need not be greater than P2. In
Case 2 then, it is possible that lack of fit may be tested
with only one check point.
By inspection of equations (3.9) and (3.10), a
hypothesis of no lack of fit is equivalent to
while the alternative
is equivalent to
hypothesis that lack of fit is present
N - p
< 0

61
since R = [ A /k] - [a^/ÍN - p)] is negative definite. In
Appendix 5(b) it is shown that the hypothesis given by
(3.13) is equivalent to the hypothesis
Case 3. We assume R is positive definite for the k
optimal upper tailed test check points, and R is negative
definite for the k optimal lower tailed test check points.
Hence either an upper or lower tailed test may be considered
as a possible test for lack of fit. If the quantity
2
-2-2//° C3n be specified, then the minimum power for both
the optimal upper and optimal lower tailed tests can be
approximated, and the test with the greater minimum power is
recommended. In Appendix 4 it is shown that Case 3 can
occur only when A^ is positive definite for the upper tailed
test. Thus Case 3 can only occur when there are at least P2
check points.
The minimum power of the upper tailed test may be found
by calculating
( F" >
k,N-p;A1L,X2U
a;k,N-p
-oh
(3.14)
where F^ ^ N_^ is the upper 100a percentage point of the
central F distribution,

62
B'B0/2a
2
and
2
where u .
min
is the smallest eigenvalue of A, and 6 is the
^ 1 max
max
largest eigenvalue of A2* Formula (3.14) yields a
conservative lower bound for the power of the optimal upper
tailed test. Note that is generated using the optimal
upper tailed test check points. The cumulative distribution
function of F" can be approximated by multiplying the
cumulative probabilities of the central F distribution by a
constant (Johnson and Kotz, 1970, p.197). This
approximation is described in Appendix 6. Other
approximations for F" (such as the Edgeworth series
approximation suggested by Mudholkar, Chaubey, and Lin,
1976) exist which are generally more accurate, but we chose
to use the approximation given in Johnson and Kotz (1970,
p.197) due to its simplicity. Additionally, the
approximation of Mudholkar, Chaubey, and Lin (1976) produced
negative probabilties when only one degree of freedom was
available in either the numerator or denominator of F".
This problem was avoided by using the approximation given by
Johnson and Kotz (1970).
The minimum power of the optimal lower tailed test can
2
be approximated similarly (if is specified) by

63
calculating
P ( F" < F }
k,N-p;XlufA2L (1-a);k,N-p
where
A
1U
lJmax-2-2/^a
2
and
X2L ~ (^min-2-2//2a '
with umax equal to the largest eigenvalue of A^ and <5m^n
equal to the smallest eigenvalue of A2. Note that A^ is
generated by using the optimal lower tailed test check
points. For the lower tailed test, A-^ may be positive semi-
definite, and if g is in the column space of P2 then A ^ = 0.
In Case 3, the upper tailed test is a test of
while the lower tailed test is a test of

64
X
1
H
k
a
Case 4. In Case 4 we assume that R = [A^/k]
[A^/(N - p)] is not positive definite for the k optimal
upper tailed test check points and R is not negative
definite for the k optimal lower tailed test check points.
Here it is useful to write the difference between the
expected value of the numerator and the expected value of
the denominator of the F ratio in (3.8) as
8¿[AiA - A /(N - p)]82 = 02sas'e2
where 15 = diag(15^, 152, 15 ^) is a diagonal matrix consisting
of the eigenvalues of Rf 15^ is a diagonal matrix of the
positive eigenvalues of R, 152 is a diagonal matrix of the
zero eigenvalues of R, and 15 3 is a diagonal matrix of the
negative eigenvalues of R. The orthogonal matrix S can be
expressed as S = [si:S2:S3], where the matrices S^, S2, and
S3 have columns which are orthonormalized eigenvectors
corresponding to 15^, 152, and 153, respectively.

65
In Case 4, neither the optimal upper tailed test nor
the optimal lower tailed test is applicable for all values
of 0 ^• For completeness, we note that Case 4 actually
consists of nine subcases, where R may be positive semi-
definite, negative semi-definite, or indefinite for either
the optimal upper tailed test or lower tailed test check
points. These subcases are listed in Table 2.
Table 2.
Nine Subcases
of Case 4.
R—Upper
R—Lower
Subcase
Tailed Test
Tailed Test
1
PSD
PSD
2
PSD
NSD
3
PSD
I
4
NSD
PSD
5
NSD
NSD
6
NSD
I
7
I
PSD
8
I
NSD
9
I
I
PSD = positive semi-definite, NSD = negative semi-
definite, I = indefinite.
If ^2 lies in the column space of S2/ then /k ~
a2/(N - p) ] 0_2 is zero, and therefore lack of fit is not
testable with either an upper or lower tailed test. A
sufficient condition for the test for lack of fit to be
upper tailed in Case 4 is that be in the column space
of [S^:S2], but not entirely in the column space of S2* In
this case

and B£[ A^/k - A2/(N - P)]62 will be greater than zero,
indicating an upper tailed test. Similarly, a sufficient
condition for the test for lack of fit to be lower tailed is
that b_2 be in the column space of [S :S ] , but not entirely
in the column space of S2. Then
S^A^/k - A2/(N - p) ] B 2 = 0 + e¿S3fl3S^B2
= B£S3a3S^B2
which makes A-^/k - A2/(N - p)]g.2 less than zero,
indicating a lower tailed test.
To determine whether B2 is in the column space of
[S :S2], let us define the augmented matrix
= [ B_2 :S2] * bas a zero eigenvalue, then B2 is
in the column space of [S :S ]. Similarly, if we define
Q2 = [ 6_2: S 2] and Q3 = [b2:S2:S3], then B2 is in the column
space of S2 if Q^Q2 has a zero eigenvalue, and B2 is in the
column space of [S2:S3] if Q^Q3 has a zero eigenvalue.

67
Given that we are in a particular subcase of the nine
subcases described in Table 2, we recommend that lack of fit
be tested with the upper tailed test check points if it is
determined that g_2 is such that g^fA^k " A2/(N “P) ] §.2 is
positive when A^ is generated from the upper tailed test
check points. Likewise, for the same given subcase, if the
value of S2 °f interest is determined to produce a negative
value for B 2[ A^/k ~ A2/(N - p)]g2 when A^ is generated from
the lower tailed test check points, then we recommend that
lack of fit be tested with the lower tailed test.
We see then that Case 4 is an undesirable situation in
practice, since, in order to test for lack of fit, we must
assume a priori that any lack of fit is due to a nonzero
value of §_2 that produces an upper tailed or lower tailed
rejection region. However, it would seem rare that such
knowledge would be available.
3.4 Examples
We now present several examples to illustrate the
technique for locating optimal check points to be used in
testing for lack of fit in a mixture model.
3.4.1 Theoretical Examples
Example 1. In this example a second order canonical
polynomial model is fitted in three mixture components using
the {3,2} simplex lattice design, which is presented in
Figure 1 of Chapter 1. The true model is assumed to be the
special cubic model containing the term ^^23X1X2X3
addition to the six terms of the fitted model. The expected

68
values of the response at the six design points are assumed
to be represented by the fitted model in the form
E(Y) = Xf^,
but with the true model the expectations are written as
E (Y) = Xf^ + X282,
where X is a 6x6 matrix with rows that define the
coordinates of the six design points and columns that
correspond to the six terms in the fitted model (x^, X2/ X3,
X1X2' xlx3' X2X3^' -1 vector °f regression
coefficients (0 , 02, 03, 012, 0 , 0 23) / *2 is a 6x1
column vector containing the values of the term x^x2x3 at
the design points, and 0 is the single regression
coefficient 0^23*
The {3,2} simplex lattice design consists of only six
design points, and since six parameters are estimated in the
second order fitted model, there are no degrees of freedom
remaining for obtaining an estimate of the experimental
error, aWe assume therefore that an external estimate of
o^ is available, a which will be used m the denominator
ext
of the lack of fit F statistic given in Eq. (3.3).
Since there is one term in the true model in addition
to those in the fitted model, that is P2 = 1, we know that
in order to locate k simultaneous check points that maximize

69
the power of the test for lack of fit it is necessary to
maximize the scalar quantity
Aj^ = (X* - X*A)'V~1(X* - X*A)
with respect to the coordinates of the k check points. Here
X* is a k-element column vector with ith element equal to
the value of x* x* x* at the ith check point, X* is a kx6
ll i2 i3
matrix with ith row equal to the value of (x?^, Xi2r Xi3'
x* x* , x* x* , x* x* ) at the ith check point,
ili2 il i3 i2i3
A = (X1X)“-*-X1X2 is the 6xl alias vector, and
Vg = + X*(X'X)-1X*'. This maximization is accomplished
by use of the Controlled Random Search Procedure (Price,
1977), which is described in Appendix 2.
When only a single (k = 1) optimal check point is
desired the Controlled Random Search Procedure locates a
point (x*, x*) which maximizes
A1 = (X* - X*A)'V"1(X* —X*A),
where
x*
X2
= x*x *x
★ —
1 2 3
= x*x*(l
- X* -
X*) ,

70
X* = (xj, X*, X*, x*x*, x*x*, x*x*)
= (X*, X*, (1 - X* - x*), x*xj, x*(1 - X* - x*),
X*(1 - X* - X*)),
and VQ = 1 + X*(X,X)_1X*'. The value of A^ is calculated
using the formula of Eq. (3.6). Following this procedure,
we find that the single check point that maximizes A^, and
thus maximizes the power of the test, is the centroid of the
triangular factor space (1/3, 1/3, 1/3). The value of at
this centroid point is A^ = 0.00084.
When the Controlled Random Search Procedure is used to
locate k = 2 simultaneous check points that maximize A-^, the
centroid (1/3, 1/3, 1/3) is selected twice, and A-^ =
0.00121. For three simultaneous optimal check points, the
centroid is selected three times, and A^ = 0.00142.
To test whether the second order model exhibits lack of
fit, when we suspect the special cubic model is the true
model, we form the F ratio
F
d'V^d/k
°ext
with the single check point (1/3, 1/3, 1/3) where d =
a *
Y* ~ Yl' Y1 tlie observed response, Y* is the response

71
predicted by the second order fitted model at (1/3, 1/3,
1/3), and V0 = 1 + (1/3, 1/3, 1/3, 1/9, 1/9, 1/9)(X’X)_1
(1/3, 1/3, 1/3, 1/9, 1/9, 1/9)'. If the calculated value of
the F ratio exceeds F , where v equals the number of
a ; 1, v
*â–  o
degrees of freedom associated with ext
null hypothesis that the second order model is the true
model in favor of the alternative hypothesis that the
special cubic model is the true model. Equivalently, we
reject Hn: X. = 0 in favor of H : X , > 0. For k = 2 or
k = 3 check points, the value of the F ratio is calculated
using the observed and predicted responses at the two or
three replicates at the centroid. The hypothesis
H_: X. = 0 is rejected in favor of H : X, > 0 if F
0 1 J a 1
exceeds F
a ; k ,v
Example 2. In Example 2 we illustrate the second of
the four cases that could arise when MSE is used as an
estimate of in the lack of fit test statistic (see
Section 3.3.4). We again fit a second order canonical
polynomial model in three mixture components, and assume the
true model is special cubic. The design to be used is the
q = 3 simplex centroid design, which consists of seven
design points, and is illustrated in Figure 2 of Chapter 1.
There are six parameters to be estimated and seven
design points hence one degree of freedom can be used to
calculate MSE. We shall use MSE to estimate aOptimal
upper and lower tailed test check points must be located,
and then a decision is made as to which test should

72
be used. The actual testing for lack of fit involves the F
statistic in (3.8).
As in Example 1, P2 = 1, since there is one term in the
true model in addition to those in the fitted model. Thus
is a scalar whose value we seek to optimize with respect
to the desired number of check points, k. When only a
single check point is sought for the purpose of testing lack
of fit, the Controlled Random Search Procedure has two
functions. First, the procedure is used to locate the
optimal candidate check point for an upper tailed test by
locating the check point that maximizes the scalar A-^.
Secondly, the procedure is used to locate the optimal
candidate check point for a lower tailed test, which is
accomplished by locating the point that minimizes A-^. The
quantity R = [A^/k] - [A2/(N - p)] is then calculated to
determine whether the upper or lower tailed test will be
used. If R is positive for the candidate check point for an
upper tailed test, then the test is upper tailed, and the
test is lower tailed if the candidate check point for a
lower tailed test produces a negative value for R. Note
that A2 = (X2 - XA)'(X2 - XA) is fixed once the design is
specified, since A2 does not depend on the check points.
Using the Controlled Random Search Procedure it is found
that the maximum value of A^ occurs at (xj, xij, x^) = (1/3,
1/3, 1/3), which will be the location for the check point
for the upper tailed test. Calculating A^ at this centroid

73
point, we find that R = [A^/k] - [A2/(N - p)] = [(3.7258
x 10_4)/1] - [(8.4175 x 10_4)/1] = -4.6917 x 10~4. Since R
is negative, the test is not upper tailed.
Using the Controlled Random Search Procedure to
minimize A^, we find that a subregion of the factor space
exists in which all points yield a near minimum value for
A^. We choose the point (0.0189, 0.9269 , 0.0542) at random
from this subregion to be used as the optimal candidate for
a lower tailed test. Here R = 0 - [(8.4175 x 10-4)/l] =
-8.4175 x 10-4.
Since R is negative for both the optimal upper tailed
test check point and for the optimal lower tailed test check
point, we have Case 2 of Section 3.3.4. The upper tailed
test check point is disregarded, and the lower tailed test
check point (0.0189, 0.9269, 0.0542) is used to test for
lack of fit. If the calculated F ratio,
F
MSE
is less than F
then Hq: X^ = X^ = 0 is rejected in
(1 -a ) ; 1,1
favor of H : [x /l] - [X /l] < 0, that is we conclude that
the second order model exhibits lack of fit, and the true
model is special cubic.
When two simultaneous check points are desired for
testing lack of fit, we can again use the Controlled Random
Search Procedure to locate the optimal settings. To
maximize the scalar A^, we find that both check points

74
should be selected at (1/3, 1/3, 1/3), for an upper tailed
test. With our calculations R = [(5.8275 x 10-4)/2) -
[(8.4175 x 10-4)/l] = -5.5038 x 10“4, but since R is
negative, the test is not upper tailed.
Minimizing A-^ to locate two optimal lower tailed test
check points yields a subregion in the factor space of
optimal check points. The pair of check points (0.3749,
0.5752, 0.0499) and (0.5332, 0.4169, 0.0499) is selected at
random from this subregion, and these check points yield
R = 0 - [(8.4175 x 10_4)/1] = -8.4175 x 10-4.
Since R is negative for the upper tailed test points
and the lower tailed test points, we have Case 2 of Section
3.3.4 again and the lower tailed test check points are used
to test for lack of fit. The hypothesis : X^ = X^ = 0 is
rejected in favor of H : [x /2] - [x /l] < 0 if the cal-
culated value of F = (d'Vg^d/2)/MSE is less than
F,, . _ ,, in which case we say lack of fit of the model is
(1-a ) ; 2,1
present.
a 2
If an external estimate a . had been available for
ext
this example, then the optimal upper tailed test check
points could have been used in the F ratio,
-1 ■» 2
F = (d'Vg d/k)/aext, an<^ lack °f fit would then be detected
if the calculated value of F exceeded F .
a ;k, v
Example 3. Example 3 illustrates the procedure for
locating optimal check points when there are two terms in
the true model in addition to those in the fitted model. A
second order canonical polynomial model in three mixture

75
components is fitted using a q = 3 simplex centroid design.
The true model is assumed to contain eight terms, six of
which are the same terms as in the fitted model, with the
additional two terms being the third order terms
512X1X2^X1 “ X2^ and ^123X1X2X3* ^ ExaraPle 2' there is
one degree of freedom for MSE which is used to estimate
a2. The test statistic, F = (d’Vg^d/k)/MSE, is given in
equation (3.8).
Since P2 = 2 and A-^ is a 2*2 matrix, locating the
optimal upper tailed test check points by the procedure of
maximizing X^ is assisted by the maximizing of a lower bound
2
for 11, namely maximizing p . B’&_/2a , where P . is the
1 2 ^ min-2-2/ mm
2
smallest eigenvalue of A]_. Since é$2 and 0 are unknown,
this is equivalent to maximizing P . . For P . to exceed
mm mm
zero, it is necessary that A^ be of full rank, and since
rank(Aj,) < min(k, p2), it is necessary to select k > 2 check
points. If Aj_ is less than full rank, and thus is positive
semi-definite, only a subset of possible values of could
be considered to make it possible to test for lack of fit
with an upper tailed test.
Using the Controlled Random Search Procedure, the
points that maximize ymin are found to be (0.418, 0.277,
0.305) and (0.277, 0.418, 0.305). These points are thus
optimal candidates for upper tailed test check points. At
these check points we have p . = 5.1623 * 10-4, A, =
mm I
diag[5.1623 * 10-4, 5.1916 x 10-4], A2 = diag[0, 8.4175 *
10 4], and R = [A-,/2] - [A2/l] = diag[2.5811 * 10-4, -5.8217
-4

76
x 10-4] . Since the eigenvalues of R are -5.8217 x 10-4 and
2.5811 x 10-4, R is indefinite. Following the suggested
procedure for Case 4 of Section 3.3.4, we note that an upper
tailed test for lack of fit exists if the value of
-2 = [5i2' ^123-^' in the column space of [si:S2l but not
entirely in the column space of S2, where Sj^ is the matrix
whose columns are the orthonormalized eigenvectors of R
corresponding to the positive eigenvalues of R, and S2 is
the matrix whose columns are the orthonormalized
eigenvectors of R corresponding to the zero eigenvalues of
R. Since R has no zero eigenvalues in this example, S2 does
not exist, but S^ is the column vector, S^ = [1,0]'. Thus
if &2 i-s °f tbe fo™ &2 = ^ ' ’ wbere ^^2 * tben
82 is in the column space of Sj^ and the test is upper
tailed .
The matrix A2 has rank one and therefore is positive
semi-definite. Hence it is impossible to locate two check
points that minimize iJmax and also make R = [Aj^/2] - [A2/l]
negative definite (see Appendix 4), that is, it is
impossible to find a lower tailed test that is capable of
testing lack of fit for all values of However, if we
use the Controlled Random Search Procedure to locate two
check points that minimize an upper bound for X which is
y 3l3„/2a2, then by minimizing y , we find that any of
the check points in a particular subregion of the factor
space yield a near minimum for y . One pair of points in
-1 max
this subregion is selected as the points to be used as

77
optimal lower tailed test check points, namely the pair
consisting of the point (0.053, 0, 0.947) replicated twice.
Replicating this check point, we find umax = 7.3900
x 10”-*--*-, A^ = diag[0, 7.3900 x 10“-^], A2 = diag[0, 8.4175 x
10-4], and R = [A±/2] - [A2/l] = diag[0, -8.4175 x 10-4].
The eigenvalues of R are 0 and -8.4175 x 10-4 implying that
R is negative semi-definite. The values of ^ that are in
the column space of [S2: S3] but not entirely in the column
space of S2 will provide a lower tailed test. Here, [S2:S3]
= diag[l,l] and S2 = [1,0]'. Thus, the test for lack of fit
is lower tailed if 3122 * 0.
For values of that produce an upper tailed test we
use the check points (0.418, 0.277, 0.305) and (0.277,
0.418, 0.305) with the F ratio
d’VQ1d/2
F =
MSE
and conclude there is lack of fit if the calculated value of
F exceeds F .0 ,. For values of g~ that produce a lower
tailed test, we use two replicates of the check point
(0.053, 0, 0.947), and'conclude there is lack of fit if F is
less than F . _ ., where again F is calculated by
v 1 a ) ; / , 1
F = (d'V~1d/2)/MSE.
Example 4. Example 4 illustrates Case 3 of Section
3.3.4 in which MSE is used to estimate in the lack of fit
test statistic. A second order canonical polynomial model

78
in three mixture components is fitted using the {3/3}
simplex lattice design, which appears in Figure 5. The true
model is assumed to be special cubic, thus p2 = 1 and A^ is
a scalar. The {3,3} design consists of ten design points
and since there are six parameters to be estimated in the
fitted model, a2 can be estimated by MSE with N - p = 10 - 6
= 4 degrees of freedom.
We first suppose that a single check point is to be
used to test for lack of fit. Using the Controlled Random
Search Procedure we find the single check point that
maximizes the scalar
A1 = (X* - X*A)'V~1(X* - X*A)
is located at the centroid of the simplex factor space.
Thus (x*, x*, x*) = (1/3, 1/3, 1/3) is the optimal candidate
for an upper tailed test check point. At this centroid
point, A-^ = 4.9076 * 10-4. For the {3,3} design the scalar
quantity A2 = (X2 - XA)'(X2 - XA) is fixed and is equal to
A2 = 9.4062 x 10"4 and thus, R = [Aj/k] - [A2/(N - p)] =
[(4.9076 x 10~4)/1] - [(9.4062 x 10“4)/4] = 2.5560 x 10-4.
The point that is the optimal candidate for a lower
tailed test check point is chosen randomly from a subregion
of points in the factor space, in which all points minimize
A^. The point selected has the value (x*, x£, x^) = (0.560,
0.410, 0.030). Here Aj. = 9.6590 x 10-7 and R = [(9.6590 x
10_7)/1] - [(9.4062 x 10-4)/4] = -2.3419 x 10~4.

79
V
(0,0,1)
Figure 5. The {3,3} simplex lattice design
Since R is positive for the optimal- upper tailed test
check point (1/3, 1/3, 1/3) and R is negative for the
optimal lower tailed test check point (0.560, 0.410, 0.030)
we are in Case 3 of Section 3.3.4. Either the upper or
lower tailed test could be used to test for lack of fit, but
2
if the quantity &2-2^° can be specified, then we will
choose to use the test that has greater minimum power, since
greater power means that we are more likely to detect lack
of fit when in fact lack of fit exists. In this example
0
123*
For illustrative purposes, we arbitrarily choose
2
02-2//CT = 2000, so that an approximate conservative lower
bound for the power of the upper tailed test is found by

80
calculating
P I F" > F 1
kfN-p;X1L,X2u a;k,N-p
where F^^ N_p is the upper 100a percentage point of the
central F distribution, k is the number of check points, N
is the total number of response observations, p is the
number of parameters in the fitted model,
A1L ' Mmin-2-2^°2' and A2U = {max^2/2° 2- The
quantity wm^n is the smallest eigenvalue of A-^, where is
evaluated at the optimal upper tailed test check point.
Since A, is a scalar, u . = A.. Likewise, 6 is the
1 mm 1 max
largest eigenvalue of A2, an<3 since in this example A2 is a
scalar, 6 = A.. In this example we have k = 1, N - p =
max 2 c
10 - 6 = 4, x.T = 3’30/2a2 = ( 4.9076 x 10"4 )( 2000/2)
= 4.9076 X 10-1, and X2„ = «max8¿S2/2a2
= (9.4062 x 10"4)(2000/2) = 9.4062 x 10_1. Using the
approximation to the cumulative probabilities of the doubly
noncentral F distribution given by Johnson and Kotz (1970,
p.197) which is described in Appendix 6, and taking a = .05,
we find that a conservative lower bound for the power of the
optimal upper tailed test is approximately equal to .0649.
The minimum power for the optimal lower tailed test is
2
approximated (assuming = ^000) by calculating
p ( F" <â–  F 1
1 k,N-p;XltI,X2L ( 1-a ) ;k,N-p; *

81
2
The quantities Alr, and A are taken as A1n = y 0'8o/2o
JL u 2 i-j jl u ma x ~ z â„¢ z
2
and A or =6 . g'80/2o where y is the largest eigenvalue
Zi^ mm-z-z max
of Aj_ with A-^ calculated using the optimal lower tailed test
check point, and where ó . is the smallest eigenvalue of
mm 3
a2 •
Since A^ and A2
are scalars
' umax
A1
and
ó . = A .
min 2
In
this example, k =
1, N - p =
4,
-7
-4
X 1U
= (9.6590 x 10
)(2000/2) =
9.6590 x
10
t
and
X 2L
= (9.4062 x 10"4
)(2000/2) =
9.4062 x
10
-1
•
Again if the
approximation to the doubly noncentral F distribution given
in Johnson and Kotz is used, an approximate conservative
lower bound for the power of the optimal lower tailed test
is .0555.
2
Having specified B2B2/o = the optimal upper
tailed test is chosen over the optimal lower tailed test,
because the approximate minimum power of the upper tailed
test is greater than the approximate minimum power of the
lower tailed test. Using the optimal upper tailed test
check point (1/3, 1/3, 1/3) in the test statistic
MSE
we conclude that lack of fit is significant if the
calculated value of F exceeds F , ., in which case we
a ; 1,4
reject HQ: A-j. = A2 = 0 in favor of Ha : A j/1 - A 2/4 > 0.
When two simultaneous check points are used for testing
lack of fit, the Controlled Random Search Procedure locates
the optimal upper tailed test and optimal lower tailed test

82
check points. It turns out that two replicates at (1/3,
1/3, 1/3) maximize A-^, and are used as optimal check points
for an upper tailed test. The value of R = [Aj^/2] - [A2/4]
is [(7.9210 x 10 — 4 )/2] - [(9.4062 x 10"4)/4] = 1.6090 x
io-4.
In searching for two optimal lower tailed test check
points, again a subregion of the factor space is found in
which any of the points nearly minimize A^. From this
subregion are chosen the points (0.6386, 0.3263, 0.0351) and
(0.7257, 0.2421, 0.0322) resulting in a value of R = [A^/2]
- [A2/4] of [(1.5216 x 10“9 )/2] - [(9.4062 x 10-4 )/4] =
-2.3516 x 10"4.
In conclusion, when two simultaneous check points are
used in the test for lack of fit in this example, R is
positive for the optimal upper tailed test and R is negative
for the optimal lower tailed test, and we have Case 3 of
2
Section 3.3.4. Selecting 3 23_2/a = 2000 arbitrarily, we
found the approximate lower bound for the power of the upper
tailed test to be .0504, and the approximate lower bound for
the power of the lower tailed test to be .0612. Since the
power is higher with the lower tailed test it is our choice
for testing lack of fit when two check points are used
simultaneously. Lack of fit is detected and we reject
H : X = X = 0 in favor of H : [X /2] - [X /4] < 0 if the F
U J. d 1 /
ratio, F = (d' V~'*'d/2 )/MSE, using the optimal lower tailed
test check points (0.6386, 0.3263, 0.0351) and (0.7257,
0.2421, 0.0322) is calculated to be less than F
(1-a);2,4 *

83
3.4.2 Numerical Examples
Numerical Example 1. In this example we illustrate
numerically some of the findings in the first theoretical
example of Section 3.4.1. Data that were collected in a
rocket fuel experiment (Kurotori, 1966) will be used to
investigate the power of the lack of fit F test. The test
is set up to detect the inadequacy of a fitted second order
canonical polynomial model when the true model is special
cubic. Calculated values of the power of the test which
detects lack of fit through large values of
d•v“1d/k
F = ~
aext
will be compared for several check point locations, includ¬
ing the location (1/3, 1/3, 1/3) at which the power was
found to be maximum in Example 1 of Section 3.4.1.
In Kurotori's experiment the modulus of elasticity (Y)
of a rocket fuel is expressed as a function of the
proportions of three components—binder (x^), oxidizer (x2),
and fuel (x2). Since lower bounds are placed on the
component proportions x^, x2, and x2, in the form of
0.20 < x^, 0.40 < x^/ and 0.20 < x^, pseudocomponents (x|)
are defined in terms of the original components in the form
of xj = - 0.20 )/( 1 - .80 ), x^ = (x - 0- 40)/(1 - .80),
and x^ = (x^ - 0.20)/(1 - .80). The true special cubic
model in the pseudocomponents, which is obtained by using
the data at the seven points of the simplex centroid design

84
in the pseudocomponent system, is
E(Y) = 2 3 5 0 x' + 2450x' + 2650x' + Ox'xl
^ ú X c.
+ lOOOxjx^ + 1600x ^x ^ + 6150x^x^x^.
The second order canonical polynomial model that is fitted
to the six boundary points only, and which will be tested
for lack of fit, is given by
Y = 23 50x| + 2450x2 + 2650x^
+ lOOOxJx^ + 1600xJ,x^.
The configuration of the experimental design as well as the
check point locations are depicted in Figure 4 of Chapter 2
and the observed response values are given in Table 3 of
this chapter.
-1 ~ 2
A value of the ratio F = [d'V_ d]/a is calculated at
0 - ' ext
each of the four individual check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6, 2/3)
* 2 ~ 2
where a . is assumed to have the value a ^ = 144 as
ext ext
suggested by Kurotori (1966). We also assume without loss
of generality that the degrees of freedom associated
* 2
with °ext are v = 10. The power of the F test is calculated
at each of the four check points by using the approximation
to the cumulative probabilities of the noncentral F

85
Table 3. Observed Response Values at the
Pseudocomponent Settings for Kurotori's Rocket Fuel
Experiment—Numerical Example 1.
Observation Binder Oxidizer Fuel Modulus of Elasticity
Number x| x^ x^ Y
1
1
0
0
2350
2
0
1
0
2450
3
0
0
1
2650
4
1/2
1/2
0
2400
5
1/2
0
1/2
2750
6
0
1/2
1/2
2950
7*
1/3
1/3
1/3
3000
8*
2/3
1/6
1/6
2690
9*
1/6
2/3
1/6
2770
10*
1/6
1/6
2/3
2980
* Check
Points .
distribution given by Johnson and Kotz (1970, p. 197) to
evaluate
Power = PÍ F' , n , > F nc ,
1 1,10;A! .05;1,10J
where A. = a . . The value of
1 1 123 ext
Al= ^X2 " x*a)'vo"^^X2 “ X*A) is calculated for each check
point using the values of X*, X*, and the value of
A = (x'X) "*"X'X2 which is fixed by the {3,2} simplex lattice
design. Since the {3,2} simplex lattice consists of points

86
only on the boundaries of the triangle (and therefore at
each point at least one of the x| values is equal to zero),
then X2 = 0 and A = 0. From the true special cubic model,
6123 = 6150.
The calculated value of F as well as the approximate
value for the power at each of the four check points is
given in Table 4. The check point (1/3, 1/3, 1/3) produced
the highest power of the four check points investigated,
supporting the previous results of Example 1 in Section
3.4.1 where (1/3, 1/3, 1/3) was selected as the check point
location with the maximum power when a second order
canonical polynomial was fitted using the {3,2} simplex
lattice design, but the true model was assumed to be special
cubic. Additional support for the point (1/3, 1/3, 1/3)
being optimal is given by the contour plot of values of A^
in Figure 6(d). The highest values of appear near the
centroid (1/3, 1/3, 1/3) where high A^ values translate into
2 2
high X values, since X^ = A^^^/2a , which in turn implies
high power since we know the power is an increasing function
of X !.
As a second part of this example the power of the F
test that is obtained when three replicates are taken at
(1/3, 1/3, 1/3) is compared to the power of the F test that
is obtained when one replicate is taken at the three check
points (2/3, 1/6, 1/6), (1/6, 2/3, 1/6), and (1/6, 1/6,
2/3). These latter three point locations were suggested by
Kurotori for testing lack of fit of his fitted special

Table 4. Effect of Check Point Location on the Power
of the Lack of Fit F Test—Numerical Example 1.
Check Point**
Numerator
d.f. (k)
Denominator
d.f. (v)
A1
A1
F
Power
(1/3,1/3,1/3)
1
10
.00084
110.32
221.1*
1.000
(2/3,1/6,1/6)
1
10
.00023
30.21
65.0*
.999
(1/6,2/3,1/6)
1
10
.00023
30.21
44.8*
. 999
(1/6,1/6,2/3)
1
10
.00023
30.21
72.6*
.999
(1/3,1/3,1/3)
(1/3,1/3,1/3)
(1/3,1/3,1/3)
3
10
.00142
186.49
1.000
(2/3,1/6,1/6)
(1/6,2/3,1/6)
(1/6,1/6,2/3)
3
10
.00051
66.98
45.3*
.999
**Check point coordinates are in the pseudocomponent system.
—F ratio cannot be calculated since only one response observation was collected
at (1/3,1/3,1/3).

88
cubic model. The result of this comparison, see Table 4, is
that the three replicates at (1/3, 1/3, 1/3) produce the
test with greater power which again supports the findings of
Example 1 of Section 3.4.1.
All of the check point locations listed in Table 4
produce very high power values (> .999) which is due in part
to the high value of $123 (3123 = 615°)« If 3123 were of
lower magnitude, then the three replicates at (1/3, 1/3,
1/3) would show a still greater superiority in the power
value compared to the power using the other check points.
This superiority is demonstrated in Table 5 where values of
8123 are listed as 3000 and 1500 and the comparative power
values are listed as 0.998 compared to 0.795 and 0.662
compared to 0.249, respectively. Table 5 also demonstrates
the superior power value for the point (1/3, 1/3, 1/3) when
&123 = 3000 or 8^3 = 150° and each of the four check points
is used one at a time.
Finally, (1/3, 1/3, 1/3) being the optimal check point
location is seen in Figure 6(c), where contour plots of the
expected difference in the heights of the surfaces are
drawn. The differences in the heights are found by
subtracting the estimated height of the surface obtained
with the fitted second order model from the estimated height
of the surface obtained with the true special cubic model.
The expected difference between the true and fitted surfaces
approaches a maximum the closer one moves to the centroid of
the simplex factor space, so that the optimal check point

on the Power
Table 5. Effect of the Magnitude of 3^
of the Lack of Fit F Test—Numerical Example 1.
3 123
= 6150
3123
= 3000
3 123
= 1500
Check Points**
X1
Power
X1
Power
X1
Power
(1/3,1/3/1/3)
110.32
1.000
26.25
. 998
6. 56
. 638
(2/3,1/6,1/6)
30.21
.999
7.19
.680
1.80
. 220
(1/6,2/3,1/6)
30.21
. 999
7.19
. 680
1.80
. 220
(1/6,1/6,2/3)
30.21
.999
7.19
.680
1.80
.220
(1/3,1/3,1/3)
(1/3,1/3,1/3)
(1/3,1/3,1/3)
186.49
1.000
44.38
.998
11.09
.662
(2/3,1/6,1/6)
(1/6,2/3,1/6)
(1/6,1/6,2/3)
66.98
.999
15.94
.795
3.98
.249
**Check point coordinates are in the pseudocomponent system.
CD
CD

90
(c) Expected difference between the
true special cubic surface and
the fitted second order surface.
Figure 6.
Contour plots for Numerical Example 1.

91
location (1/3, 1/3, 1/3) coincides with the point where the
expected difference between the true special cubic surface
and the fitted second order surface is maximum.
Numerical Example 2. In this second numerical example,
we investigate the power of the F test for detecting lack of
fit when a second order canonical polynomial model is fitted
in a mixture system which is in truth represented by a
special cubic model. The true model is assumed to be
E(Y) = 2350x1 + 2450x2 + 2650x3
+ lOOOx^x^ + leOOx^x^ + 6150x^x2x3
which is used to generate hypothetical response observations
at the seven points of the q = 3 simplex centroid design as
well as at three check points. The values of the response
are obtained by adding the value of a pseudorandom normal
variate with mean 0 and variance 144 to each true predicted
response value. The data are given in Table 6.
The response values at the seven points of the simplex
centroid design are used in the least squares normal
equations to obtain the fitted second order model
Y = 234lx^ + 2438x2 + 2630x3
+ 310x.x„ + 1304x,x_ + 1970x_x_

92
Table 6. Generated Response Values—Numerical Example 2.
X1
x 2
x3
Y
—
'
—
—
1
0
0
2357
0
1
0
2454
0
0
1
2646
1/2
1/2
0
2403
1/2
0
1/2
2747
0
1/2
1/2
2962
1/3
1/3
1/3
3013
* 1/3
1/3
1/3
2993
* 2/3
1/6
1/6
2693
* . 02
*Check points.
.93
. 05
2550
which is to be tested for lack of fit using the test
statistic F = d1VQ^d/MSE. The F statistic will be evaluated
at each of the three check points (1/3, 1/3, 1/3),
(2/3, 1/6, 1/6), and (.02, .93, .05), taken one at a time,
and the power of the test at the three check point locations
will be calculated and compared. The test is lower tailed
for all check point locations (since R = A-^ - A2 is negative
for all check point locations) and thus the power is defined
as
F1,1;X1 ,X 2 < F. 95 ; 1,1^ *

93
X =1
I
Figure 7. Contours of R = A^ - A2 for Numerical Example 2.
2 2 2 2
The values of X ^ = &i23Al//2a and X2 = g123A2//2° are found
2
by taking 3123 = 6150 and 0 = 144• The results of this
power investigation are given in Table 7.
Since the check point (.02, .93, .05) produces the
greatest power of the three check points investigated, this
supports the result in Example 2 of Section 3.4.1, where we
saw that the point (.02, .93, .05) yielded the maximum power
of all points for detecting lack of fit of a fitted second
order canonical polynomial model, using the q = 3 simplex
centroid design in the presence of a true special cubic
surface. Additional evidence for the point (.02, .93, .05)
being an optimal check point is shown in Figure 7, where
contours of the values of R = A^ - A2 are presented. The
point (.02, .93, .05) is seen to belong to an area of the

Table 7. Effect of Check Point Location on the Power
of the Lack of Fit F Test—Numerical Example 2.
Numerator Denominator
Check Point
d.f.(k)
d.f.(N -
P) Ax
A2
R
X1
A 2
F Power**
(1/3,1/3,1/3)
1
1
3.70x10-4
.00084
-.00047
48.59
110.3
.333 .000
(2/3,1/6,1/6)
1
1
5.20x10-5
.00084
-.00079
6. 84
110.3
.063 .012
(.02,.93,.05)
1
1
8.12x10-9
.00084
-.00084
.001
110.3
.0005* .590
*Significant
at a =
.05 (F < F
•
95;1,1,}*
**Power is approximated using an approximation found in Johnson and Kotz (1970, p.197).

95
simplex factor space where R is minimum, which implies that
Aj_ (and in turn \ is also minimum in this area, since R =
A1 ~ a2 an(^ a2 ^as t^ie fixec^ value of A2 = .00084 for the
simplex centroid design. Thus the check point (.02, .93,
.05) produces a minimal A ^ value and maximum power, since
the power increases with decreasing values of A^.
3.5 Discussion
When check points are used for testing lack of fit in a
mixture model, the appropriate testing procedure, assuming a
normally distributed response, involves an F statistic. If
" 2
an external estimate, o , of the experimental error
variance is available so that the test statistic is given by
d’V^d/k
F = ^2
°ext
then the power of the test for lack of fit is maximized by
choosing k check points that maximize the value of the non¬
centrality parameter A ^. When p2 = 1, maximizing A^ is
achieved without knowing the value of the elements of by
selecting check points that maximize the scalar Aj_. When
P2 > 1, the maximization of A^ is approximated by maximizing
a lower bound for A^. This is achieved also without knowing
the values of the elements of g^ by selecting check points
that maximize the smallest eigenvalue of the matrix A^. The
test is upper tailed, and for given values of Aj_,
the actual

96
power of the test can be calculated from the cumulative
probabilities of the noncentral F distribution. A problem
arises when A-^ is positive semi-definite and its smallest
eigenvalue is equal to zero. In this case check points that
maximize the smallest positive eigenvalue of are
selected, and lack of fit is only detectable for a subset of
the possible values of the elements of & .
When an external estimate of a2 is not available,
testing lack of fit at the check points is further
complicated. The F statistic is
(d'V^d/k)
F =
MSE
and the rejection region for the lack of fit test can be
upper tailed or lower tailed. The power of the test is
determined by using the doubly noncentral F distribution,
which depends on the parameters k, N - p, A-^, and A 2* Of
these four parameters, only k and A-^ are influenced by check
points, and if the value of k is fixed, the power of the
test is maximized by choosing check points that affect the
value of A^. Regardless of the values of the elements of
£S_2, check points that maximize Aj_ are selected for
maximizing the power of an upper tailed test, and check
points that minimize A ^ are selected for maximizing the
power of a lower tailed test.

97
Lack of fit can be tested with the upper tailed test
for all nonzero values of the elements of g 0 if the check
points are selected so that [A^/k] - [A2/(N - p)] is
positive definite, since this forces the expected value of
the numerator mean square in the F ratio to be greater than
the expected value of the denominator mean square.
Similarly, lack of fit can be tested with a lower tailed
test if check points are selected which make [A^ /k] -
[A2/(N - p)] negative definite. When it is not possible to
select check points that make [A^/k] - [A2/(N - p)] either
positive definite or negative definite then detection of
lack of fit is only possible for a subset of all nonzero
values of the elements of
The power of the test for lack of fit using the F
statistic in (3.8) is a function of X ^ and X2. Since the
magnitudes of X^ and X2 are influenced by the experimental
design, an area for future study is the investigation of the
effect of the experimental design on the power of the lack
of fit test. In the presence of an external estimate of
2
a , Atkinson (1972) suggested designs that maximize the
determinant of A2, |A^¡, when lack of fit is to be detected
by a large value of X^ using a procedure which is, in
general, equivalent to the lack of fit testing procedure
that partitions the residual sum of squares into pure error
and lack of fit sums of squares. It might be useful to
apply Atkinson's (1972) methodology not only to |A2|, but
to ¡A1| or jA^/k - A2/(N - p)| in order to find an

98
appropriate design when testing lack of fit with the F ratio
in Eq. (3.8). Since the power of the test in Eq. (3.8) is
also affected by the values of k and N - p, which are the
numerator and denominator degrees of freedom of the doubly
noncentral F distribution, respectively, optimal settings
for these parameters can also be considered. For a given
fitted model, p is fixed so that the degrees of freedom
would be influenced by the number of check points selected,
k, and by N, the total number of observations. Finally, the
experimental design and the number of check points also
affect the power of the F test when o is used to
t A L
2
estimate a . Thus the effect of the experimental design and
the number of check points can also be investigated for the
situation where the lack of fit test statistic is given by
Eq. (3.3).
We now conclude our investigation of the check point
approach to lack of fit testing and in the next chapter turn
to an investigation of a near neighbor method for testing
lack of fit.

CHAPTER FOUR
USE OF NEAR NEIGHBOR OBSERVATIONS
FOR TESTING LACK OF FIT
4.1 Introduction
In an experiment in which replicate response
observations are available at one or more design points,
lack, of fit of a fitted model can be tested by a procedure
which involves partitioning the residual sum of squares into
two statistically independent portions. One portion is the
sum of squares due to lack of fit (SSL0F), and the second
portion is the sum of squares due to pure error (SSEpUre)
obtained from the replicates. As discussed in Section 2.2,
this procedure was suggested by Draper and Smith (1981,
p.120). Lack of fit is inferred if the calculated value of
the ratio
F
MS
LOF
MSE
pure
(4.1)
exceeds the corresponding upper 100a percentage point of the
central F distribution, where MSLqF and MSEpure are the mean
square values found by dividing SSLOf and SSEpure their
respective degrees of freedom.
In order to test the fitted model for lack of fit when
replicate observations are not available, Shillington (1979)
suggested a procedure which uses observed response values
99

100
collected at points which are "near neighbors" in the factor
space in place of replicates (see Section 2.3). Lack of fit
is inferred when the calculated value of the ratio
MSEb
MSEw
(4.2)
exceeds the upper 100a percentage point of the central F
distribution. The numerator, MSEB, of the F ratio in Eq.
(4.2) is a generalization of the numerator, MSLqF, of the F
ratio in Eq. (4.1). The form of MSEg will be given in Eq.
(4.5) of Section 4.3. The denominator, MSEW, in Eq. (4.2)
is a generalization of MSEpure in Eq. (4.1), and the value
of MSEW is calculated using near neighbor observations in
place of replicates (see Eq. (4.6) in Section 4.3).
Shillington's near neighbor method provides an
alternative to the check points method when replicate
observations are not available. Typically, near neighbors
might appear either because an experiment was not designed
to provide replicate observations or with a designed
experiment consisting of a large number of design points in
a bounded factor space which results in some points being
near one another.
In this chapter we shall further study Shillington' s
(1979) near neighbor procedure for testing lack of fit. A
question involving the correctness of the ordinary least
squares technique suggested by Shillington for deriving the
denominator, MSEW, of the F ratio in Eq. (4.2) will

101
be raised. The question will be resolved by showing the
equivalence of deriving MSEW by ordinary least squares and
of deriving MSEW by a generalization of weighted least
squares. We will verify that when the observable response
values are assumed to have the normal distribution with
2
homogeneous variance, a , the F ratio in Eq. (4.2) possesses
a central F distribution when the fitted model is adequate,
but possesses a doubly noncentral F distribution when the
fitted model suffers from lack of fit. We shall also show
that the F test for lack of fit which uses the statistic in
Eq. (4.2) can have either an upper tailed or a lower tailed
rejection region. Finally, the use of a clustering
algorithm for defining groups of near neighbors will be
proposed.
4.2 Notation
In this section we introduce the notation to be used in
this chapter. Throughout our investigation of Shillington's
near neighbor procedure for testing lack of fit, we shall
assume the observed response values collected in an
experiment can be grouped into g cells where the jth cell
contains nj observations, j =1, 2, ..., g. The
observations in a cell are from points that are "near
neighbors" in the sense that they are near one another in
the factor (mixture) space. A model of the form
E(Y)
(4.3)

102
is fitted to the data using ordinary least squares, but the
true model is assumed to have the form
E(Y) = XB1 + X202, (4.4)
where Y is an Nxl vector of response values observable at
the design points with var(Y) = a IN, X and X2 are Nxp and
Nxp2 matrices of known constants, respectively, and 0^ and
02 are P*1 and p2xl vectors of unknown regression
coefficients, respectively. The vector Y is assumed to have
the normal distribution.
Let us now define the following vector and matrix
quantities to be used in developing the numerator, MSEB, of
the F ratio in Eq. (4.2):
Yc = a gx1 vector with jth element equal to the
average of the nj observed response values in the
jth cell of near neighbor observations, j = 1, 2,
• • •, g .
X(- = a gxp matrix whose jth row is the average of the
nj rows of X corresponding to the jth cell, j =
1, 2, ..., g.
X2c = a gxp2 matrix whose jth row is the average of the
nj rows of X2 corresponding to the jth cell, j =
1/ 2,
• • • , g.

103
Gq = a gxg diagonal matrix of the form
Gq = diag [1/n^, l/n2, 1/n ] .
To further illustrate the forms of Y^, Xq, X2C1 an<^ G0'
we present the following numerical example. Consider a data
set consisting of response observations (Y) taken at N = 8
different combinations of the settings of the factors x^ and
x2, where the eight response observations are divided into
g = 4 near neighbor cells. The vector of observed response
values, Y, and the matrix X corresponding to the first order
model, E(Y) = 0 + 0.x + 0 x , are
10
1
1
2
13
1
2
5
16
1
2
4
15
X =
1
3
2
18
1
3
1
21
1
4
2
27
T~
5
5
30
1
5
4
—
—
The horizontal lines in Y and X delineate the four cells of
near neighbors. In this example
10
10
(13 + 16)/2
=
14.5
(15 + 18 + 21)/3
18
(27 + 30)/2
28.5
— _
_.

104
X
c
1
(1 + 1)/2
(l+l+D/3
(1 + 1)/2
(2 + 2 )/2
(3+3+4)/3
(5 + 5 )/2
(5 + 4)/2
(2+1+2)/3
(5 + 4)/2
1
1
1
1
1 2
2 4.5
3.3 1.7
5 4.5
and
G0 = diag(1, 1/2, 1/3, 1/2).
If the true model is second degree, E(Y) =
2 2
+ 2ixi + g2X2 + ^ 12X1X 2 + ^11X1 + ^ 22X 2' then the x2 and
X2C matrices have three columns corresponding to the
2 2
terms x^x2/ x^, and x2, respectively. For this numerical
example we have
X
2
and
2
10
8
6
3
8
25
20
X2C
2
9
5.7
22.5
1
4
4
9
9
16
25
25
1
4
11.3
25
4
25
16
4
1
4
25
16
4
20.5
3
20.5

105
Next let us define the following quantities to be used
in developing the denominator, MSEW, of the F ratio in Eq.
(4.2) :
W = an Nxl vector of within cell deviations where the
ith element, , of W is equal to the difference
between the ith element, Y^r of Y and the average
of the response values observed in the near
neighbor cell containing Yj_, i = 1, 2, ..., N.
Xw = an Nxp matrix whose ith row is equal to the ith
row of the X matrix minus the row of Xc
corresponding to the cell containing the response
value observed at the ith row of X.
r = rank(Xw).
X2W = an NxP2 matrix whose ith row is equal to the ith
row of the X2 matrix minus the row of ^2C
corresponding to the cell containing the response
value observed at the ith row of X.
Eg = an NxN idempotent matrix of the form
Eq = — diag [ (1/n ^ , (l/n2)J2r ‘««ríl/n) Jg ]
where Jj is an njxnj matrix of ones, and IN is
9
the identity matrix of order NxN, with N = E n..
j=l D
Let us illustrate the forms of W, X^¡r %2VI' anc^ £ q
using the numerical example presented earlier in this
section, where the eight response observations were
distributed among four cells. For these data we have

106
— -
10 - 10
0
13 - 29/2
-1.5
16 - 29/2
1.5
15 - 18
=
-3
18 - 18
0
21 - 18
3
27 - 57/2
-1.5
30 - 57/2
1.5
1 -
1
1 -
1
2 -
2
0
0
0
1 -
1
2 -
2
5 -
9/2
0
0
. 5
1 -
1
2 -
2
4 -
9/2
0
0
-.5
1 -
1
3 -
10/3
2 -
5/3
=
0
-.3
.3
1 -
1
3 -
10/3
1 -
5/3
0
-.3
-.7
1 -
1
4 -
10/3
2 -
5/3
0
.7
.3
1 -
1
5 -
5
5 -
9/2
0
0
.5
1 -
1
5 -
5
4 -
9/2
0
0
-.5
X2W
and
Z
0
2 -
2
1 -
1
4 -
4
0
0
0
10 -
9
4 -
4
25 -
41/2
1
0
4.5
8 -
9
4 -
4
16 -
41/2
-1
0
-4.5
6 -
17/3
9 -
34/3
4 -
3
=
.3
-2.3
1
3 -
17/3
9 -
34/3
1 -
3
-2.7
-2.3
-2
8 -
17/3
16 -
34/3
4 -
3
2.3
4.7
1
25 -
45/2
25 -
25
25 -
41/2
2.5
0
4.5
20 -
45/2
25 -
25
16 -
41/2
-2.5
0
-4.5
0
0
0
0
0
0
0
0 ~
0
1/2
-1/2
0
0
0
0
0
0
-1/2
1/2
0
0
0
0
0
0
0
0
2/3
-1/3
-1/3
0
0
0
0
0
-1/3
2/3
-1/3
0
0
0
0
0
-1/3
-1/3
2/3
0
0
0
0
0
0
0
0
1/2
-1/2
0
0
0
0
0
0
-1/2
1/2 J
•
4.3 Shillington1s Procedure
We originally described Shillington1s near neighbor
procedure for testing lack of fit in Section 2.3 of Chapter
Two. We now reintroduce the procedure by using the matrix

107
and vector quantities defined in Section 4.2. The test that
Shillington proposed involves the use of an F ratio (see Eq.
(4.2)) of two statistically independent mean square values,
2
each of which is an unbiased estimate of a when the fitted
model is the correct true model. The two independent mean
2
squares become biased estimates of a when the fitted model
suffers from lack of fit. Shillington's methodology detects
lack of fit when the calculated value of the F ratio in Eq.
(4.2) is large, thus his test is upper tailed. We shall see
later in Section 4.7 that the test is not always upper
tailed, and may be lower tailed. Shillington points out
that the power of the test depends on the relative
magnitudes of E(MSEB) and E(MSEW), that is, the power
depends on the difference between the expected values of the
numerator and of the denominator in the F ratio in Eq.
(4.2). We shall be more specific than Shillington by
discussing the power of the test in terms of parameters of
the doubly noncentral F distribution.
We now turn to defining Shillington's test statistic in
matrix notation. Shillington's F ratio takes the form (see
Eq. (4.2))
SSEB/(g - p)
SSEW/(N - g - r)
MSEB
MSEW

108
where SSEg is the residual sum of squares from a weighted
least squares regression analysis in which Yc is regressed
on Xc, g is the number of cells of near neighbors, p is the
number of terms in the fitted model, and r is the rank of
X^. The quantity SSEB can be written as the quadratic form
sseb - (4-5)
(see Graybill, 1976, p. 329; also see Draper and Smith,
1981, p. 109). The quantity SSEW is defined as
SSEW = W*[IN - XW(XW'XW)~X¿]W, (4.6)
where is any generalized inverse of (X^XW). [A
matrix A- is defined as a generalized inverse of the matrix
A if AA~A = A.] The quadratic form SSEW is the residual sum
of squares from an ordinary least squares regression
analysis in which W is regressed on Xy¡.
In the next two sections we shall discuss the
development of the numerator and denominator, MSEB and MSEW,
respectively, of the F ratio given in Eq. (4.2). We then
suggest an alternative representation for MSE^ which relies
on a generalization of weighted least squares. This
alternative representation for MSEW will be shown to be
equivalent to Shillington’s expression for MSE^.

109
4.3.1 Development of MSEB
The quantity MSEg = SSEB/(g - p) is the numerator of
the F ratio in Eq. (4.2). As mentioned in Section 4.3, the
quantity SSEB is the residual sum of squares from a weighted
least squares regression analysis in which Yc is regressed
on Xc. The weighting is appropriate because
- 2 .
var(Yc) = a Gq not only when the fitted model is adequate
(under model (4.3)), but also when the fitted model suffers
from lack of fit (under model (4.4)). In order to further
explain the (Y^, ) system, we define the matrix M as
M = diag[(l/n1)1|
• • • 9
where 1. is an
"3
f 9
Then the (Yc, Xc) system can be derived as a linear
transformation of the (Y, X) system. That is, application
of the transformation matrix M yields the following
equalities
X
C
MX
and
X
2C
MX
2
From this it follows that var(Y^) = M var(Y)M' = a^MM'
2
a Gq , since MM' = G^. Under the hypothesized model of

110
Eq. (4.3), E(fc) = ME (Y) = MXg^ = X^^, whereas under the
model of Eq. (4.4), E(YC) = M( XB ^ +X2$-2) = XC^-1 + X2C^2‘
We now consider the distribution of the random quantity
2
SSEg/a . It can be shown (Theorem 2, Searle, 1971, p. 57)
that under the model of Eq. (4.3), SSED/a2 possesses a
D
central chi-square distribution with g - p degrees of
O
freedom, but that under the model of Eq. (4.4), SSEg/a
possesses a noncentral chi-square distribution with g - p
degrees of freedom and noncentrality parameter n^, where
"l = (1/2<’2)6-2X2CIG01 - GÓ1XC(XCGÓ1XC>'1X¿GÓ11X2C^2' (4'7)
Here we point out that the noncentrality parameter for
2
SSEg/a given by Shillington (1979) is not correct and
should be written as in Eq. (4.7).
Finally we note that SSEB is equivalent to the usual
lack of fit sum of squares, SSL0F, where SSlqf/^ - P) =
MSLof i-s the numerator of the F ratio in Eq. (4.1), when
all observations in each cell are true replicates rather
than near neighbors. Shillington pointed out this fact, but
did not give a proof. We offer a proof in Appendix 7.
4.3.2 Development of MSEW
The quantity MSEW = SSE^/(N - g - r), where r denotes
the rank of X^, is the denominator of the F ratio in Eq.
(4.2). As mentioned in Section 4.3, the quantity SSEW is
the residual sum of squares from an ordinary least squares
regression analysis in which W is regressed on Xw. Using

Ill
Theorem 2 (Searle, 1971, p. 57) and noting that
W = E y and EE = E . it can be shown that under the
- 0- 000
2
hypothesized model of Eq. (4.3), SSEw/a possesses a central
chi-square distribution with N - g - r degrees of freedom,
2
but under the model of Eq. (4.4), SSET,/o possesses a
w
noncentral chi-square distribution with N - g - r degrees of
freedom and noncentrality parameter n^, where
n2 = (l/202)B¿X'w[In - vw~x¿lW-2- (4-8)
Shillington (1979) points out that SSEW reduces to the usual
pure error sum of squares, SSEpUre, when all cells contain
true replicates. This is easily seen by using the fact that
X^ = 0 when all cells are composed entirely of true
replicates so that SSET7 = W'W = Y'E E y = Y'E v = SSE
We saw in Section 4.3.1 that the (Y^, X^) system is
derived as a linear transformation of the (Y, X) system.
Similarly, the (W, X^) system can be derived by applying the
transformation matrix Eq. Thus W = E^y, Xw = E^x,
and X„T7 = E x*. It follows that E(W) = E E(Y) = E.XB.
= X^B^, under the model of Eq. (4.3), and E(W) =
+ X2-2^ = XW^-2 + X2W-2 un<^er mo<3el °f Eq. (4.4).
Furthermore, var(W) = EQ var(Y)E^ = a2EQE^ = a2EQ, since EQ
is symmetric and idempotent.
Since the variance-covariance matrix of W is not equal
to aIN, for some positive constant a, SSEW should have been
derived as the residual sum of squares from a weighted least

112
squares regression analysis of W on Xw rather than from the
ordinary least squares regression of W on X^. that
Shillington (1979) suggested. We shall use the weighted
least squares regression of W on Xw in an attempt to replace
SSEW in the F ratio in Eq. (4.2) by an expression we will
call SSEW(weighted). We later show that SSEW and
SSEW(weighted) are equivalent.
2
The variance-covariance matrix of W, which is a Eq, is
of rank N - g and is thus singular. Therefore the residual
sum of squares from a weighted least squares regression
analysis of W on Xw which is
cannot be used as an expression for SSEW(weighted), since
Eq1 does not exist. The problem of performing a weighted
least squares regression analysis when the variance-
covariance matrix of W is singular is considered in the next
section.
4.4 Development of SSEW(weighted)
2
If the variance-covariance matrix of W, a Eq, is
nonsingular then the weighted least squares formula for
SSEw(weighted) is
SSEW( weighted) = (W - X^TEq^W - Xw3 x)

113
where ^ is a solution to X^Eg'^X^^ = X^E^W and can be
written as = (X^E q"*’Xw)— X^E q^W. The quantity
SSEW(weighted) divided by the appropriate degrees of freedom
2
provides an unbiased estimate of a under the model of Eq.
(4.3) .
However, since Eq is singular, the weighted regression
formula above cannot be used to calculate SSEW(weighted).
C. R. Rao (1971, 1972, and 1973) suggests an analog of
weighted least squares for the case of a singular variance-
covariance matrix. Rao suggests the existence of a matrix H
such that g^ is a stationary vector value of
2
(W - - Xw0^) in whlch case a may be estimated
using
a2 = (W - XW61)«H(W - Xw01)/(N - g - r)
where (N - g - r) = rank(EQ:Xw) - rank(Xw). The rank of the
matrix (EQ:XW) is equal to N - g because Xw = EqXw so that
the columns of Xw are spanned by the columns of Eg, thus,
rank(EQ:Xw) = rank(EQ) = N - g.
One form of the matrix H is defined (Rao, 1971 and
1972) as
H = [Eq + c2XwX¿] (4.9)
where c is an arbitrary nonzero constant, so that with the
model of Eq. (4.3), a2 = (W - X^rHÍW - X^J/iN - g - r)

114
2
is an unbiased estimator for a . Thus a stationary vector
value of (W - _ ^{J ) is given by
= (X^HXW)~X^HW . Rao indicates (1972, p.3) that 02 is
invariant to the choices of the generalized inverses
* 2
involved m a •
Rao's proofs for obtaining an unbiased estimator
~ 2 2 . .
a for a are not given m detail. Therefore we shall state
and prove the following theorem which will be used to
develop an expression for SSEW(weighted). The notation A-
will be used to denote any generalized inverse of a matrix
A, such that AA-A = A.
2
Theorem 4.1. Let Y ~ (X0, a G), where G is singular,
then a2 = f_1(Y - X0)'T-(Y - X0)
2
(i) is an unbiased estimate of a , and
(ii) is unique with probability one, and
(iii) is a scalar multiple of a central chi-square
variable with f degrees of freedom of the form
2 2
(a /f)Xf if Y has the multivariate normal
distribution.
The vector Y is of order Nxl, g is a pxl vector of
unknown regression coefficients, X is an Nxp matrix of
known constants, G is an NxN positive semi-definite
A
matrix of known constants, T = G + XX', 0 is any
solution to X'T_X0 = X'T“Y, that is, 0 = (X'T-X)~X'T-Y,
and f = rank(G:X) - rank(X).

115
The proofs of parts (i), (ii) and (iii) of Theorem 4.1 are
given in Appendices 9, 10, and 11, respectively. Lemma 4.1
which is used in the proof of Theorem 4.1 is stated and
proved in Appendix 8.
The results of Theorem 4.1 can now be applied to our
problem of finding an expression for SSEW(weighted). We
define SSEW(weighted) as
SSEW(weighted) = W’[T“ - T~XW(X^T~XW)“X^T"]W (4.10)
where tq = ZQ + XWX^. Writing SSEw(weighted) in Eq. (4.10)
as SSEW(weighted) = (W - ^B^J'T^W - XWB1), from Theorem
4.1 we see that if the true model is of the form in Eq.
2
(4.3) then SSEW(weighted)/a has a central chi-square
distribution with f = rankU^rX^ - rankfx^) = N - g - r
degrees of freedom. However, if the true model is of the
2
form in Eq. (4.4), then SSEW(weighted)/o has a noncentral
chi-square distribution with N - g - r degrees of freedom
and noncentrality parameter II*, where
n 2 * (1'/2'’2>®2X¿WIT0 '
The distribution of SSEW(weighted)/o^ under model (4.4) is
verified by the following theorem.

116
Theorem 4.2. Let Y ~ NN(X¡3 + X 0 , o2G), G singular,
, "22 2 ^2-1 - _
then fa /a ~ x¿ A where a = f (Y - X0 )'T (Y - X0 ) ,
f = rank(G:X) - rank(X), T = G + XX', and
A = (l/2a2)0¿X¿[T~ - T~X(X'T_X)~X,T_]X2e2.
The proof of Theorem 4.2 is given in Appendix 12.
4.5 Equivalence of SSEW and SSEW(weighted)
In this section we shall show that our expression for
SSE^(weighted) in Eq. (4.10) is equal to Shillington's
unweighted SSE^ in Eq. (4.6). Thus the complex calculations
required for evaluating SSEW(weighted) can be avoided by
calculating the simpler form SSEW.
Zyskind (1967) investigated conditions under which
ordinary least squares estimators are BLUE (best linear
unbiased estimators) even though Y in the model
Y = X3 + e, where E(e) = 0, does not have variance-
2
covariance matrix equal to a I. Zyskind assumes that
2
var(Y) = o V, where V is non-negative and possibly singular,
and then states and proves the following necessary and
sufficient condition for ordinary least squares estimators
to be BLUE.
Theorem 1 (Zyskind, 1967). A necessary and sufficient
condition for all simple least squares linear estimators
to be also best linear unbiased estimators of the

117
corresponding estimable parametric function A'3 in the
linear model
Y = X3 + e, E(e) = 0, E(ee ' ) = a2V,
where V is a symmetric non-negative matrix and X is of
rank r, is that there exist a subset of r orthogonal
eigenvectors of V which forms a basis for the column
space of the matrix X.
In a second theorem, Zyskind (1967) gives several other
necessary and sufficient conditions for ordinary least
squares estimators to be BLUE. These conditions are shown
to be equivalent to the condition in Theorem 1 (Zyskind,
1967). The fifth of these conditions in Zyskind's second
theorem is that VP = PV, where P = X(X'X)-X'.
Applying condition 5 of Theorem 2 (Zyskind, 1967) to
our problem of regressing W on Xw we have
V
I
0
and
P =
vv
V
'*w
and therefore

118
VP = W W *W
= Ww)_xw '
since Eq= EqEqX = EqX = Xw« It follows that
vp = V W^o
= PV.
Therefore by Theorem 2 (Zyskind, 1967) the ordinary least
squares solutions from regressing W on X^ are BLUE
estimators, and thus are equivalant to the solutions
obtained from weighted least squares. We conclude therefore
that SSEW = SSEW(weighted).
4.6 The Test Statistic
As stated in Section 4.1, Shillington (1979) proposed
that a fitted model be tested for lack of fit by using the F
ratio
MSB_
F =
MSEW
w
given in Eq. (4.2). In this section we shall verify that
Shillington's F ratio possesses a central F distribution
when the true model is of the form in Eq. (4.3), and
possesses a doubly noncentral F distribution when the true

119
model is of the form in Eq. (4.4). This information on the
distribution of the F ratio will be needed in Section 4.7,
where the power of the test is discussed. Additionally, we
shall give the form of the expected values of both the
numerator, MSEg, and the denominator, MSEW, of the F ratio
in Eq. (4.2). These expected values will aid us in
developing a procedure for calculating the power of the
test, since they will be used to determine whether the test
is upper tailed or lower tailed.
In developing the distribution of the F ratio in Eq.
(4.2), we shall show that SSEB/a2 and SSEw/o2 are
statistically independent. In this pursuit, let us use the
expression for SSEg in Eq. (4.5) and the fact that Yc = MY
to express SSEg as
SSEb = Y'M'IG-1- G-1Xc(X¿G-1Xc)-1X¿G-1]MY.
Also, using the expression for SSEW in Eq. (4.6) (which is
allowed because we showed in Section 4.5 that the correct
form, SSEW(weighted), is equal to SSEw) and using the fact
that W = zqY, we can express SSE^ as
SSEw = TV1» - xw(xwxw)_xw1Eo-’
By Theorem 4 (Searle, 1971, p.59), to show that
2 2
SSEg/a and SSE /a are statistically independent, it
suffices to show that the matrix product

120
EM'fGÓ1 -
G:1)M]tr0{IN - VW
V
is equal to the zero matrix. This is seen to be true since
MEq = 0, and therefore SSEB/a2 and SSEw/a2 are independent.
When the fitted model and the true model are both of
the form in Eq. (4.3), then the F ratio in Eq. (4.2)
possesses a central F distribution with g - p and N - g - r
degrees of freedom in the numerator and denominator,
respectively. Furthermore, the numerator, MSEB, of the F
ratio in Eq. (4.2) has expectation equal to
E(MSEb) = [o2/(g - P)]EXg_p
2
a .
Similarly, under model (4.3), the expected value of the
denominator, MSEW, of the F ratio has expectation equal to
E(MSEw) = [a 2/(N - g - r)]Ex2_g_r
When the fitted model suffers from lack of fit and the
true model is given by Eq. (4.4), the F ratio in Eq. (4.2)
is a ratio of two statistically independent noncentral chi-
square random variables, each divided by its respective

121
degrees of freedom. Thus the F ratio in Eq. (4.2) possesses
a doubly noncentral F distribution with g - p and N - g - r
degrees of freedom and noncentrality parameters and n^•
where n^ and n2 were given in Eqs. (4.7) and (4.8),
respectively. The expected value of the numerator, MSEg, of
the F ratio can be written as
E(MSEb) = (a2/(g - P *!EXg-P,n,
= a
+ 6¿C1
s,/(g - p)
where
C1 - W'^01 - Gólxc(X¿GólxC>'lx¿GóllMX2- (4-U>
Similarly under model (4.4), the expected value of the
denominator, MSE^, of the F ratio can be written as
E(MSEW) = [c2/ = a2 + B¿C2b2/(N - g - r)
where
“ X2E0[IN
- vw
VZ0X2-
(4.12)

122
4.7 The Testing Procedure and its Power
As discussed in Section 4.1, Shillington (1979)
suggested that lack of fit of the fitted model be inferred
when the value of the F statistic in Eq. (4.2) exceeds
F . The test, however, is not always upper
tailed, and in fact can be lower tailed. The test is
considered lower tailed when, because of lack of fit, the
expected value of the numerator of the F ratio is less than
the expected value of the denominator of the F ratio.
We suggest that lack of fit be tested with an upper
tailed test using the F ratio F = MSEB/MSEW when the matrix
D, which is defined as
is found to be positive definite (which can only occur when
is positive definite, by Theorem 3.1 in Appendix 4). The
matrices and C2 in Eq. (4.13) are defined in Eqs. (4.11)
and (4.12), respectively. An upper tailed test is
appropriate when the matrix D is positive definite because
no matter what the value of is/ the expected value of the
numerator, MSEg, of the F ratio will be greater than the
expected value of the denominator, MSE^, of the F ratio.
However, there may be cases where D is negative definite
(which can only occur when C2 is positive definite), and in
this case lack of fit should be tested with a lower tailed

123
rejection region. If D is indefinite, then the F test for
lack of fit may be upper tailed, lower tailed, or lack of
fit may not be testable depending upon the value of 8 .
In those cases where D is indefinite, it is helpful to
write the quantity 8^D8^r which represents the difference
between the expected value of the numerator and the expected
value of the denominator of the F ratio, F = MSEB/MSEW, as
-2 °-2 = §.£[ ui:U2:U3] diag[r1, r2 = 0, r 3 ] [U1 :U2 :U3 ] ’ B 2
= g_2UlrlUi^2 + ^-2^3r 3^32-2'
where U^, Ü2/ and U3 are matrices whose columns are
orthonormal eigenvectors of D, and r^, r2, and are
diagonal matrices whose elements are the positive, zero, and
negative eigenvalues of D, respectively. Lack of fit is
testable with an upper tailed test if B2 is in the column
space of [U^:U ], but not entirely in the column space of
U2/ since then B^Df^ is positive. Similarly, lack of fit is
testable with a lower tailed test if b2 is in the column
space of [U2;u3] • but not entirely in the column space of
U2, since then 32DS2 is negative. If B2 is in the column
space of U2, then lack of fit cannot be tested, since
B_2DB_2 would equal zero.

124
We define §_2 to h>e i-n the column space of U2 if the
matrix has a zero eigenvalue, where = [§.2:U2].
Similarly, letting L2 = and L3 = [0 2: U2: U3 ^ '
3_2 is in the column space of [U^:U2] if L£L2 has a zero
eigenvalue, and 3~ is in the column space of [u :U ]
^ ^ J
if l3l3 has a zero eigenvalue.
When D is positive definite or D is indefinite but
32 is upper tailed testable, then the F test for lack of fit
which makes use of the F ratio F = MSEB/MSEW is a test of
the hypotheses (see Theorem 3.2, Appendix 5)
Ha: ni/(9 " P) " n2/(N " 9 " r> > °*
When D is negative definite or D is indefinite but 32 is
lower tailed testable, then the F test tests
0
Ha: nl/(g " P) " n2/(N ~ 9 ~ r) < 0.
In the case where D is indefinite and 3_2 is in the column
space of U2, then no hypotheses concerning lack of fit of
the fitted model can be tested.
When the test is upper tailed, the power of the F test
for lack of fit is defined as

125
Power = p{ F"
(4.14)
where F
a;g-p,N-g-r
is the upper 100a percentage point of the
central F distribution with g - p and N-g-r degrees of
freedom. In the case of a lower tailed test, the power of
the test is defined as
Power = P{F"
M < F __ } . (4.15)
g-p,N-g-r ;ll i ,Ü2 1-a ;g-p,N-g-rJ
4.8 Selection of Near Neighbor Groupings
In the preceding sections of this chapter, we have
discussed a near neighbor procedure which uses the F ratio
F = MSEB/MSEW to test a fitted model for lack of fit. In
this section we shall investigate the effect that different
groupings of response observations into near neighbor cells
have on the testing procedure and its power. From equations
(4.14) and (4.15) in the previous section it is evident that
the power of the F test for lack of fit, which makes use of
the F ratio F = MSEB/MSEW, depends on the values of the
numerator and denominator noncentrality parameters,
n and n2« Assuming the numerator and denominator degrees
of freedom are fixed, and the test is upper tailed, then the
power is an increasing function of increasing values
of n^ and is a decreasing function of increasing values of
n2 (see Appendix 1). When the test is lower tailed, the

126
power is an increasing function of n2 and is a decreasing
function of n .
Since both the numerator and denominator noncentrality
parameters, II ^ and II , are functions of the groupings of the
data points into near neighbor cells (as are the numerator
and denominator degrees of freedom), we would like to
investigate the effect of the number and composition of
cells on the power of the F test. Intuitively, it would
seem that homogeneous near neighbor cells would minimize the
2
bias inherent in estimating a with MSEW, and thus would
minimize n^ and maximize the power of an upper tailed
test. However, any grouping of the data points would also
influence the numerator noncentrality parameter and the
numerator and denominator degrees of freedom. Therefore
while a grouping of data points into homogenous cells might
decrease n ^ and thus apparently increase the power of an
upper tailed test, the result of the grouping on the power
also depends on how the degrees of freedom, g - p and
N - g - r, and the numerator noncentrality parameter, II
are affected.
We will attempt to find homogeneous cells of near
neighbor points by using an iterative partitioning
clustering algorithm. Two examples will be presented. The
first example makes use of the stack loss data presented by
Daniel and Wood (1971) and later analyzed by Shillington
(1979). The second example involves data from a mixture
experiment discussed by Piepel (1981).

127
Our objective is to investigate the effect of forming
homogeneous cells of near neighbors on the F test for lack of
fit which makes use of the test statistic F = MSEg/MSE^.
It is hoped that such homogeneous groupings will increase
the power of the F test (assuming that the rejection region
is upper tailed) by decreasing for a fixed number of near
neighbor cells (and thus fixed values for the degrees of
freedom). The effect that homogeneous grouping has on n^ is
not clear, but is of interest, since the magnitude of
n1 also affects the power of the test. Additionally, we
will vary the number of cells of near neighbors in an
attempt to determine how this affects the power of the test,
since the number of cells affects both the noncentrality
parameters and the degrees of freedom.
The algorithm used for grouping the data points into
homogeneous near neighbor cells can be described as an
iterative partitioning type of cluster analysis. The
computations involved were accomplished using the RELOC
procedure available in the CLUSTAN 1C computer package
(Wishart, 1975). All computations were performed using data
points whose coordinates were standardized by subtracting
off sample means and dividing by sample standard deviations.
Initially, k clusters (near neighbor cells) of the N
data points in the factor space are arbitrarily defined.
Then the Euclidean distance between each point and the
centroid (average vector value) of each of the k clusters is
determined. If a point is found to be closer to the

128
centroid of one of the other k - 1 clusters than to the
centroid of the cluster in which it is currently classified,
then the point is reclassified into that nearest cluster
(cell). The centroids of the k clusters are then
recalculated, taking into account any reclassified point.
The entire set of N points is scanned repeatedly in this
manner until no reclassification occurs. This method of
assigning points to clusters will be referred to as
iterative relocation.
In the second stage of the algorithm, two of the k
clusters arrived at by the iterative relocation procedure
are fused, resulting in k - 1 clusters. The two clusters to
be fused are selected as those which when fused produce the
k - 1 clusters with minimum "error sum of squares." The
error sum of squares is defined as the sum of squared
Euclidean distances between every point and the centroid
point of the cluster to which it belongs. After k - 1
clusters are determined using the error sum of squares
criterion, iterative relocation is applied to the k - 1
groups in an effort to improve the clusterings. This
alternation of fusion and iterative relocation continues for
k - 2 clusters, k - 3 clusters, ..., 2 clusters, or until a
specified minimum number of clusters is reached. The
question of determining an "optimal" number of clusters is
not addressed by this procedure.

129
4.8.1 Example 1—Stack Loss Data
The first example we investigate is the 21 observation
stack loss data of Daniel and Wood (1971), which was
analyzed by Shillington (1979). The data (see Table 8)
consist of the values of three factors, (air flow), X2
(cooling water inlet temperature), and x-^ (acid concen¬
tration) along with the values of a response variable, Y
(stack loss). A first order regression equation of the form
E (Y) = eQ + B 1x x + 8 2X 2 + B3x3
is fitted using 17 of the original 21 observations
(Shillington discarded 4 of the original 21 observations as
outliers). We assume the true model to have the form
E(Y) = 8
+ 81x1 + 82x2 +
83X3 +
Bllxl + 622x2 + S33x3
and thus contains p2 = 3 terms in addition to the p = 4
terms in the fitted model. We wish to investigate the
capability of the F test
MSE
F = „ B
msett
w
in detecting lack of fit of the fitted model.
We first consider the use of the F statistic with the
six cells of near neighbors used by Shillington (1979),
which is the same near neighbor grouping suggested by Daniel

130
and Wood (1971). This 6 cell grouping (see Table 8 under
the column heading "6**") is found to yield a matrix
D = [c^/(9 ~ P)J - [C2^N “ 9 ” r)] which is indefinite,
since the eigenvalues of D have the values 12110, -7, and
-1415 (see Table 9). Thus the test is not upper tailed,
since D must be positive definite for an upper tailed test
to exist for all values of 32* where in this example,
-2
= (3
11'
B 22'
B33} '
When the 6 cell grouping of near neighbors generated by
the iterative partitioning clustering algorithm (see Table 8
under the column heading "6") is used, the values of the
eigenvalues of D are 49090, 379, and -43, and the test is
still not upper tailed for all values of
We then use the iterative partitioning clustering
algorithm to determine homogeneous cell groupings for 5, 7,
8, 9, 10, 11, and 12 cells. The matrix D is found to be
indefinite for the groupings into 5, 7, or 8 cells, but D is
positive definite for 9, 10, 11, or 12 cells of near
neighbor groupings. Thus, no matter what the value of
lack of fit can be tested with an upper tailed test using
the 9, 10, 11, or 12 cell groupings of near neighbors.
The value of F = MSEB/MSEW was calculated using the
matrix procedure from the 1979 version of SAS. None of the
near neighbor groupings provided evidence of lack of fit,
and thus we cannot conclude that there is lack of fit when
the fitted model is E(Y) = 8^ + 8-^x^ + 32X2 + ^3X3

Table 8. Near Neighbor Cells for Stack Loss Data.
Observation
Number
Air
Flow
X1
Water
Temp.
x2
Acid
Concen.
x3
Stack
Loss
Y
Membership in
for Five
5 6** 6 7
Near Neighbor Cells
to Twelve Cells
8 9 10 11
12
1*
80
27
89
42
2
80
27
88
37
1
1
1
1
1
1
1
1
1
3*
75
25
90
37
4*
62
24
87
28
5
62
22
87
18
2
2
2
2
2
2
2
2
2
6
62
23
87
18
2
2
2
2
2
2
2
2
2
7
62
24
93
19
2
2
2
3
3
3
3
3
3
8
62
24
93
20
2
2
2
3
3
3
3
3
3
9
58
23
87
15
2
3
2
2
2
2
2
4
4
10
58
18
80
14
3
4
3
4
4
4
4
5
5
11
58
18
89
14
4
4
4
5
5
5
5
6
6
12
58
17
88
13
4
4
4
5
5
5
5
6
6
13
58
18
82
11
3
4
3
4
4
4
4
5
5
14
58
19
93
12
4
4
4
5
5
6
6
7
7
15
50
18
89
8
4
5
5
6
6
7
7
8
8
16
50
18
• 86
7
4
5
5
6
6
7
7
8
9
17
50
19
72
8
5
5
6
7
7
8
8
9
10
18
50
19
79
8
5
5
6
7
8
9
9
10
11
19
50
20
80
9
5
5
6
7
8
9
9
10
11
20
56
20
82
15
3
6
3
4
4
4
10
11
12
21*
70
20
91
15
*Outliers, not included in analyses.
**This near neighbor cell grouping was proposed by Daniel and Wood (1971) and
used by Shillington (1979). All other cell groupings were generated by an
iterative partitioning cluster analysis using the CLUSTAN computer package.
Numbers in the table refer to cell membership.
131

Table 9. Effect of Near Neighbor Cells on
F = MSEb/MSEw for Stack Loss Data.
Number of Cells
Numerator
(g) 5
6**
6
7
8
9
10
11
12
d .f . (g - p)
Denominator
1
2
2
3
4
5
6
7
8
d.f. (N-g-r)
Eigenvalues
9
9
8
7
6
5
4
4
3
of C,
65525
24408
98191
99915
102192
102572
102859
103214
103445
0
622
4082
9098
20054
21985
23317
22394
22642
Eigenvalues
0
0
0
17
17
74
85
106
107
of C2
14900
16325
13539
8297
1017
327
318
318
104
1478
121
112
50
48
48
24
24
10
Eigenvalues
111
0
39
20
18
18
0
0
0
of D
69297
12110
49090
33300
25540
20507
17143
14745
12931
-13
-7
379
1852
4847
4332
3640
3120
2800
Calculated
-1592
-1415
-43
-4
-2
8
8
9
6
value of F
1.14NS
1.08NS
0.96NS
0.91NS
0.83NS
1.05NS
1.73NS
1.49NS
2.52NS
ni
—
—
—
—
—
133
133
136
136
n 2
Approximate
Power of Upper
1
1
1
1
Tailed Test
Note: Fitted Model:
E(Y) = B
0 + 61X1
+ 8 2x2
+ 83X3
1.00
0.98
0.96
0.79
Assumed
** Daniel and
NS Not greater
true model: E (Y) = BQ + 3 jX 1 + B 2X 2 + 3 3x + 3-^x
Wood (1971) grouping, other groupings generated by
than i)1) :a-D.M-n-r .
2
1 + 622X
CLUSTAN.
2 2
2 + 3 33X3
132

133
and the true model contains only pure quadratic terms in
addition to the first degree terms.
For the groupings of near neighbors into cells which
provide an upper tailed test (9, 10, 11, or 12 cells), the
power of the upper tailed test can be approximated if
and n can be specified. This approximate power can be
calculated using an approximation to the doubly noncentral F
distribution given in Johnson and Kotz (1970, p.197), which
is described in Appendix 6 of this dissertation. Thus we
calculate an approximation for
P{ F"
1 g-p,N-g-r ;H i ,n2.
>
F } .
a ;g-p,N-g-rJ
In order to compare the power of the upper tailed F
test for 9, 10, 11, and 12 cells, we will assume arbitrarily
that the true value of the parameter vector g^ is
-2 = ^11' e22' e33^' = (* 044' * 329, - . 033)' which is
arrived at by taking = 1002 w^ere ^2 t*ie ieast squares
estimate of 0^ calculated from the data. Furthermore,
2
taking a =1.6 (since the residual mean square value from
fitting the "true" second degree model is MSE = 1.6) we
2
calculate the values for and
H 2 = &2C2-2/'2°2 ^or eacl1 the 9, 10, 11, and 12 cell
groupings. The calculated values of and nas well as
the approximate power values for each of the four F tests
(calculated using the approximation to F" from Johnson and
Kotz (1970, p.197)) are presented in Table 9. The power is

134
quite high (> .96) for 9, 10, or 11 cell groupings, but
drops off to .79 for the 12 cell grouping. This drop in
power seems to be due to the effect of having only 3 degrees
of freedom in the denominator of the F ratio.
In summary, this example illustrates that the F test
for lack of fit that makes use of the statistic
F = MSEg/MSEw is upper tailed only for certain groupings of
the design points into near neighbor cells. For the near
neighbor cell groupings that provide an upper tailed test,
the power is generally high for the values of £5^ and
2
a that we selected, but decreases slightly as we move from
9 to 10 to 11 cells and decreases more severely as we move
from 11 to 12 cells. This more severe decrease in power is
due to the decrease to only 3 denominator degrees of
freedom.
4.8.2 Example 2—Glass Leaching Data
The second example we investigate is one in which the
leachability, Y, of glass is assumed to be a function of the
proportions of eleven chemicals of which the glass is
composed (Piepel, 1981). A first order Scheffé polynomial
model was fitted to the common logarithms of the
leachability values, that is, the fitted model is of the
11
E(log1(JY) =_Z 8ixi.
form

135
The experimental design coordinates and the values of
the 44 data observations are presented in Table 10. For
illustrative purposes, the true model is assumed to contain
the P2 = 8 second order cross product terms,
B68X6X8' 83,11X3X11' B79X7X9' B56X5X6' B7,11X7X11'
B15X1X5' B6,10Vl0' and B59X5X9 ln addition to the P = 11
first order terms in the fitted model. The 19-term model is
the final fitted model proposed by Piepel and serves as our
true model.
Piepel (1981) suggested that the four sets of
observations (see Table 10)
(a)
14 and
15
(b)
18,
19,
and
20
(c)
25,
26,
and
27
(d)
39,
40,
and
41
were intended
to
constitute four
cells of replicate
observations
for
use
in estimating pure experimental
error.
However,
the
settings of
the mixture components
not well controlled, so that each of the four cells
contained near neighbors rather than replicates. By
defining each of the remaining 33 data points as 33 cells
containing one observation each, the 44 data observations
are partitioned into 37 cells. If we choose to use the 37
cells to test the fitted model for lack of fit, with the
test statistic F = MSEg/MSE^, we find that there are no
degrees of freedom for MSEW so that the F statistic cannot
be calculated in this case. However, the F statistic for

Table 10. G'
Observation
Number
X1
Si O2
x 2
B2°3
x 3
a12°3
x4
CaO
x 5
MgO
1
.448
.063
.143
.000
.000
2
. 544
. 063
.000
. 138
. 000
3
.565
.120
.000
.000
.000
4
. 581
. 061
.012
.000
. 079
5
.432
.126
.000
.098
.079
6
.486
.064
.000
. 140
.000
7
.429
.060
.145
. 139
.013
8
.429
. 124
. 144
. 138
.000
9
.443
.120
.000
.099
.080
10
.447
.114
. 148
.012
.077
11
.427
.121
.145
.010
.078
12
. 550
. 060
.146
.000
.076
13a
. 434
.058
.148
.127
.000
14a
. 580
.122
.000
. 135
.000
15
. 575
.123
.000
.134
.000
16
.498
. 066
.000
. 134
. 080
17b
.435
.118
.145
.000
.031
18b
. 496
. 092
. 072
. 067
.038
19b
.497
.091
.072
.066
. 039
20
. 497
. 091
. 072
. 067
. 039
21
. 565
.125
.000
.010
. 036
22
. 428
. 068
. 073
. 117
. 084
23
.433
.064
. 155
.047
.000
24c
. 550
.141
. 000
. 000
. 000
25c
. 577
.127
.023
. 000
.086
26c
. 586
.126
.024
.000
. 079
27
. 585
. 128
. 023
.000
.082
28
. 579
. 068
.000
.000
. 088
29
. 541
.064
.153
.072
. 000
ss Leaching Data.
x 6
Na 2O
x 7
ZnO
x 8
Ti02
x9
Cr203
x10
Fe2°3
X11
NiO
Y*
Leach
Value
.165
.059
.069
.000
.029
.025
8.10
. 154
. 000
. 071
. 030
. 000
. 000
0.14
.153
.061
.070
.031
.000
.000
0.01
.108
. 061
. 070
. 000
. 028
.000
0.01
.158
.000
.070
.000
.009
.028
7.40
.159
. 063
.000
. 028
. 031
.029
3.90
. 152
. 062
.000
.000
.000
.000
91.40
.105
. 060
. 000
. 000
. 000
. 000
49.00
.101
.061
.070
.000
.026
.000
0.11
. 150
.000
.000
.027
.025
. 000
33.40
.161
.000
.000
.029
.029
.000
39.50
.111
.000
.030
.000
.000
.027
0.42
. 109
.000
.069
.030
. 025
.000
9.60
. 110
.000
.000
. 000
.025
. 028
0.20
. 112
.000
.000
.000
.028
. 028
0.24
. 105
. 059
.000
.030
. 000
. 028
6.20
.102
.060
.051
.031
.000
.027
2.20
. 125
. 031
.035
. 015
. 015
. 014
0.86
.129
.029
.034
.014
.015
.014
0.80
. 127
. 030
.034
. 014
.015
. 014
0.86
.163
.000
. 070
.031
. 000
.000
0.20
. 108
. 000
.071
. 000
. 027
.024
1.47
.159
.000
.073
.028
.022
.019
16.40
. 156
. 063
. 079
.000
. Oil
. 000
0.15
.094
. 060
.000
. 033
.000
. 000
0.15
. 093
. 058
. 000
.034
. 000
.000
0.08
. 091
.059
.000
.032
.000
.000
0.15
. 146
. 061
. 000
. 000
. 031
.027
0.18
.113
.000
.000
.031
.026
.000
2.04
136

Table 10—continued.
30
.417
.127
.152
. 010
•
31
.4 29
.133
. 031
.130
•
32
.454
.067
.142
.000
•
33
.420
.130
. 138
.000
•
34
. 574
.066
.000
.114
•
35
. 584
.067
. 031
.128
•
36
.464
.067
.142
. 126
•
37
. 433
.134
.142
. 038
•
38d
.432
. 134
.012
.129
•
39d
.434
. 068
. 127
. 131
•
40d
.432
.067
.138
.131
•
41
.431
. 067
.139
.132
•
42
.413
. 127
.151
.011
•
43
.430
. 133
.144
. 127
•
44
. 59 6
.068
.000
.011
•
* Leach values are weight percent loss
a,b,c,d Four groups of intended replica
162
.000
.000
. 031
.000
.021
51.80
144
. 059
. 074
. 000
. 000
. 000
9.04
145
.060
.077
.033
.000
.022
2.09
155
. 000
. 074
. 000
.000
. 000
24.70
108
.000
.077
.034
.000
.027
0.02
165
. 000
. 000
. 000
.025
. 000
0.08
118
.000
. 000
.000
.000
.000
37.20
093
.058
. 072
. 000
. 030
. 000
7.39
096
.059
. 000
.000
.030
.025
96.60
153
. 061
. 000
. 000
. 000
.026
77.80
146
. 060
.000
.000
.000
.026
75.60
149
. 061
. 000
. 000
. 000
.021
76.40
165
.000
.000
.030
.000
.023
43.30
111
. 000
.000
. 031
.000
. 024
25.40
136
.063
.077
.024
.000
.025
0.07
ass. Analyses performed on log^Q(Y).
080
000
000
083
000
000
083
000
083
000
000
000
080
000
000
of gl
tes.
137

138
lack of fit can be calculated using from 12 to 33 cells of
near neighbors, and thus we refer to the iterative
partitioning clustering algorithm discussed earlier in this
section to generate near neighbor groupings of 15, 20, 25,
and 30 cells (see Table 11).
The clusterings of observations into 15, 20, or 25
cells each yields a D matrix that is indefinite (see Table
12), so that the test is not upper tailed. The 30 cell
clustering produces a positive definite D matrix, so that
the test is upper tailed for all nonzero values of where
8-2 = (0 68' 0 3,11' 0 79 ' 0 56 ' 07,11' 015' 06,1O' 059)'*
Taking 02 = (15.141, -112.429, -78.761, -78.275, 87.996,
13.356, -76.948, 34.721)', which is the least squares
2
estimate of 8^ froâ„¢ the data, taking a = .008 (which is
MSEpure with seven degrees of freedom from Piepel's analysis
of the data), and using the approximation of Johnson and
Kotz (1970, p.197) to approximate
PÍ F" > F )
1 19,4,-Hi ,n2 . 05 ; 19,4 J
where n^ = 9.79, U^ = 0.08 and F 4 = 5.81, we find
that (using 30 near neighbor cells) a value for the power of
the F test is .10. The power increases as the magnitudes of
the elements of ^ are increased, so for example if all the
elements of above are doubled, then = 39.16,
n2 = 0.32, and the approximate power is .25. If the
elements of 0_2 above are each multiplied by 5, then

139
Table 11. Near Neighbor Cells for Glass Leaching Data.
Membership in Near
Observation
Neighbor
cells
Number
15
20
25
30
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
6
6
6
6
6
7
7
7
7
7
8
8
8
8
8
9
9
9
9
9
10
10
10
10
10
11
10
10
10
10
12
11
11
11
11
13
12
12
12
12
14
13
13
13
13
15
13
13
13
13
16
6
6
14
14
17
14
14
15
15
18
11
15
16
16
19
11
15
16
16
20
11
15
16
16
21
3
3
3
3
22
5
5
5
17
Membership in Near
Observation
Neighbor
Cells
Number
15
20
25
30
23
1
1
1
18
24
3
3
3
19
25
15
16
17
20
26
15
16
17
20
27
15
16
17
20
28
4
4
18
21
29
12
12
12
12
30
10
17
19
22
31
8
8
8
23
32
1
14
15
15
33
10
18
20
24
34
2
2
2
2
35
13
19
21
25
36
11
11
22
26
37
8
8
23
27
38
9
9
9
28
39
7
7
7
7
40
7
7
7
7
41
7
7
7
7
42
10
17
19
22
43
14
20
24
29
44
2
2
25
30
*Cell groupings generated by an iterative partitioning
cluster analysis using the CLUSTAN computer package.
Numbers in the table refer to cell membership.

Table 12. Effect of Near Neighbor Cells on
F = MSEb/MSEw for
Glass Leaching
Data.
Number of Cells (g)
15
20
25
30
Numerator d.f. (g - p)
4
9
14
19
Denominator d.f. (N-g-r)
19
14
9
4
Eigenvalues of C-^
2.92x10-5
6.57xl0-5
1.03xl0-4
1.27xl0-4
1.92x10-5
2.56xl0-5
2.84xl0-5
3.24x10-5
3.13xl0-6
1.07xl0-5
1.74xl0-5
2.37x10-5
6.60xl0-7
4.34x10-6
6.45x10-6
7.38x10-6
0
3.02x10-6
4.86xl0-6
5.51x10“6
0
1.07xl0“6
1.23xl0-6
1.88xl--6
0
l.lOxlO-7
9.00x10-7
1.44x10-6
0
4.76x10-8
3.33xl0-7
7.97xl0-7
Eigenvalues of C2
1.60xl0-5
1.05x10“5
6.69xl0-6
5.Olx10-7
1.28xl0-5
5.03x10-6
5.06x10“7
5.89xl0“8
7.65xl0-6
2.64x10-6
1.51xl0-7
3.22x10-9
1.84x10-6
8.10x10-7
9.50xl0-8
1.21x10-9
1.09xl0“6
4.36xl0“7
2.65x10“8
0
7.43xl0-7
7.31xl0“8
9.49xl0-9
0
6.36xl0“7
2.58xl0-8
1.71xl0-9
0
2.30xl0-8
2.72x10-9
0
0
140

Table 12—continued
Eigenvalues of D
6.54xl0-6
7.05x10-6
7.23xl0-6
6.36x10-6
4.44xl0-6
2.38xl0“6
1.79x10"6
1.67x 10-6
6.17xl0-7
9.39xl0“7
1.03x10-6
1.24x10-6
1.85xl0“8
4.15x10-7
4.54x10-7
3.87x10“7
-4.43x10-8
2.41x10“7
3.42x10-7
2.88x10~7
-4.87xl0“8
6.86x10-8
7.97xl0~8
9.30x10-8
-1.38xl0-7
-2.93x10-8
3.77x 10**8
7.18x10-8
-4.80x10“7
-1.52x10“7
-1.9 7x10-7
2.85x 10-8
Calculated Value
of F
0.42
1.01
5.69*
20.55*
J^**
—
—
—
9.79
n 2
—
—
—
0.08
Approximate Power
of Upper Tailed Test
—
—
—
0.10
Greater than F
** Values of II
1'
.05;g-p,N-g-r *
n 2, and the power of the test were calculated by assuming
0 = (15.141, -122.429, -78.761, -78.275, 87.996, 13.456, -76.948, 34.721)’ and
Á . 2
assuming a = .008.
008
141

142
= 244.75, n2 = 2.00 and the approximate power is .83.
From the entry in Table 12, we see that the calculated F
value of 20.55 with the 30 cell clustering exceeds
F nc;. i a a = 5*81/ and we conclude that the fitted first
• u j y i y f 4
degree model is inadequate.
4.9 Discussion
When a designed experiment includes replicated points,
the adequacy of a fitted model can be tested by comparing
the portion of the residual sum of squares due to lack of
fit to a second portion due to pure error from the
replicates. The test statistic is an F ratio of the mean
square due to lack of fit to the mean square due to pure
error, and lack of fit is inferred when the calculated value
of this ratio is large (Draper and Smith, 1981, p.120).
When replicate points do not exist, lack of fit can be
tested using near neighbor observations with the test
statisic F = MSEB/MSEW. This F ratio has been shown to
possess a central F distribution when the fitted model is
adequate, and a doubly noncentral F distribution when the
fitted model suffers from lack of fit.
When the fitted model is adequate, the expected values
of both MSEg and MSEW are equal to a^, so that the ratio
E[MSEg]/E[MSEW] equals unity. However, when lack of fit is
present, both MSEg and MSEW are biased estimates of a^, and
we compare the magnitudes of the biases of these estimates
(which are functions of the noncentrality parameters and
degrees of freedom of the doubly noncentral F distribution)

143
in the F test. The test has an upper tailed rejection
region if the bias corresponding to MSEg exceeds the bias
corresponding to MSEW. The rejection region is lower tailed
if the bias corresponding to MSEW exceeds the bias
corresponding to MSEg. In other words, the test is upper
tailed if the matrix D (see Eq. 4.13) is positive definite,
and the test is lower tailed if D is negative definite. If
D is indefinite then the test may be upper tailed, lower
tailed or still yet lack of fit may not be testable
depending upon the value of B^•
In two examples an iterative partitioning clustering
algorithm is used to assign the data points to a preselected
number of near neighbor cells. When the number of cells is
low, the matrix D is found to be indefinite, so that the F
test is not strictly upper tailed or lower tailed. However,
by increasing the number of cells, it is possible in both
examples to produce a positive definite matrix D, so that
the test is upper tailed.
Increasing the number of cells not only produces an
upper tailed test, but also affects the values of the
parameters of the doubly noncentral F distribution. As the
number of cells is increased (moving from left to right in
Tables 9 and 12) we see that the smallest eigenvalue of Cj_
increases and that the largest eigenvalue of C2 decreases.
Therefore a lower bound for ,
-2-2rmin
< n
2a

144
increases as the number of cells increases (where ii> .
ymin
denotes the smallest eigenvalue of C^). In addition, an
upper bound for n ,
n2
-2-2pmax
2a2
decreases as the number of cells increases (where Pmax
denotes the largest eigenvalue of C2). Finally, as the
number of cells increases (moving from left to right in
Tables 9 and 12), the numerator degrees of freedom, g - p,
increase and the denominator degrees of freedom, N - g - r,
decrease. Since the parameters of the doubly noncentral F
distribution change as the number of cells changes, the
power of the F test can be affected. For the stack loss
data example, we see in Table 9 that the power of the upper
tailed test decreases as we move from 9 to 10 to 11 to 12
cells.
An area for future study can be a further investigation
of the effect of the number and composition of near neighbor
cells on the power of the F test which makes use of
F = MSEb/MSEw. This investigation would involve the effect
of near neighbor cell selections on the parameters n^, I^/
g - p, and N - g - r of the doubly noncentral F distribu¬
tion. It would be desirable to develop a method (perhaps an
alternative to the iterative partitioning clustering algo¬
rithm) which could be used to select the number and composi¬
tion of cells so as to maximize the power of the F test.

CHAPTER FIVE
CONCLUSIONS AND RECOMMENDATIONS
Two general methods for testing a linear model fitted
in a mixture space for lack of fit have been investigated in
this dissertation. The first method makes use of response
values observed at check points while the second method
makes use of response values observed at design points which
are near neighbors in the factor space.
In Chapter Two we discussed the work of several authors
(Scheffe (1958), Gorman and Hinman (1962), Kurotori (1966),
and Snee (1971)) for testing lack of fit which centered on
measuring bias inherent in the fitted model when estimating
the response at check points. Only the method suggested by
Scheffe (1958) was an exact test. In Chapter Three, a
method for selecting check points that maximizes the power
*
of Scheffe's F test was devised. When replicate response
observations were available, so that the experimental error
~ 2
variance could be estimated by oext from the replicates, we
saw that the power of this upper tailed F test was maximized
by selecting check points that maximize (or approximately
maximize) the noncentrality parameter of the noncentral F
distribution. When the matrix A^ (where \^ '
was found to be positive semi-definite it was determined
that only a subset of possible values of the parameter
vector could be detected as contributing to lack of fit.
145

146
When an estimate of the experimental error variance was
not available from replicates, an extension of Scheffé's F
~ 9
test for lack of fit which replaced a ^ with MSE (the
ext
residual mean square error from the fitted model) in the
denominator was developed. We found that to maximize the
power of the test it was necessary to select check points to
maximize (or approximately maximize) the numerator
noncentrality parameter, A , of the doubly noncentral F
distribution when the test was upper tailed. When the test
was lower tailed, we sought check point locations that
minimized (or approximately minimized) X . A criterion was
developed for determining whether the test was upper tailed
or lower tailed by comparing the expected values of the
numerator and denominator of the F ratio when the fitted
model was inadequate. Finally, we discovered cases where,
for some values of $ lack of fit could not be tested.
An alternative to the check points method for testing
lack of fit in a fitted model is a procedure that involves
measuring the bias that is present in estimates of the
response at the design points (the number of design points
must exceed the number of terms in the model). When
replicate observations are available, the well known
procedure in which the test statistic is a ratio of the lack
of fit mean square to the pure error mean square can be used
to test for lack of fit (see Draper and Smith, 1981,
p. 120). When replicate observations are not available,
several techniques which make use of near neighbor

147
observations in place of replicates for testing lack of fit
have been proposed in the literature (see Green (1971),
Daniel and Wood (1971), and Shillington (1979)).
Additionally, it has been suggested by Draper and Smith
(1981, p. 42) that lack of fit can be tested by using near
neighbor observations as substitutes for replicate
observations in the usual lack of fit, pure error F ratio.
However, the exact distributions of the test statistics
proposed by Daniel and Wood (1971) and Draper and Smith
(1981, p. 42) have not been defined, and Green's (1971)
procedure requires an inordinately large number of
observations. Thus because of these reasons we chose
Shillington1s (1979) procedure to study in greater detail in
Chapter Four.
In Chapter Four the distributional properties of
Shillington1s test statistic were developed, and a method
based on an iterative partitioning clustering algorithm for
defining groups of near neighbor observations was
proposed. It was shown that the power of Shillington1s test
depends on the parameters of the doubly noncentral F
distribution, and that the manner in which observations are
grouped as near neighbors can alter the values of the
parameters of the doubly noncentral F distribution and thus
affect the power of the test. We found that increasing the
number of near neighbor cells so that individual cells
become more compact produced an upper tailed F test in the

148
two examples studied, but that there are many other cases
where the test will not be upper tailed.
Now that we have briefly summarized our findings from
investigating the check point and near neighbor methods of
testing lack of fit in a mixture model, a logical question
is, "Which of the two methods is better?" It was not our
original intent to address this question in this
dissertation, but an interesting result that has been
discovered in the latter stages of our investigations is as
follows: Under certain circumstances, the check point
method for testing lack of fit is equivalent to the usual
method which partitions the residual sum of squares into
sums of squares due to lack of fit and due to pure error
(which was shown in Chapter Four to be a special case of the
near neighbor method). Because we have not found a
derivation of the equality of these methods in the
literature, we shall show it here.
In Chapter Three, check points were used to test lack
of fit in a fitted model of the form E(Y) = X8^. With k
check points, the test statistic was of the form (see Eq.
(3.3))
d'V^d/k
°ext
where
“ 2
°ext
is an external estimate of
2
a
which can be

149
calculated from replicates, if they exist. The vector d in
the F ratio was defined to be a vector of differences
between observed and predicted response values at the k
check points having the form
d = Y* - X*(X'X) 1X'Y,
where Y* is the kxl vector of observed response values at
the k check points and X* is the corresponding settings of
. 2
the model terms at the check points. The matrix a Vq was
defined as the variance-covariance matrix of d where Vq has
the form
V. = I, + X*(X'X) 1X*' .
Ok
It can be shown (see Appendix 13) that if we define the
vector Y, as
-A
r
Y
observations at the original design points
Y*
observations at the check points
and similarly define the matrix XA as
“ “
— “
X
design point settings
X*
check point settings
so that the original design points as well as the check
points are all taken at once as design points in regressing
*A °n V
then

150
SSE.
= y ' r T
-A1 (N+k)
- XA(XÁXA>'lxÁ^A
= d
'vó^
+ SSE,
(5.1)
Thus, the residual sum of squares, SSEA, from the analysis
of the fitted model when both the original design points and
the check points are used to fit the model is equal to the
sum of the quadratic form, d'y^d, used in the numerator of
the check point F test and the residual sum of squares, SSE,
from the analysis of the fitted model using data collected
only from the original design points.
If we perform the usual partitioning of SSEA into a
lack of fit sum of squares, SSLqF^Aj, and a pure error sum
of squares due to replicates, SSEpure^Aj, then from Eq.
(5.1) we can write
SS
LOF ( A)
+ SSE
pure(A)
= d
'v0la-
+ SSE.
(5.2)
Thus from Eq. (5.2), when SSEpure(A) e<3ua-*- to SSE' then
SSLOF(A) becomes equal to d'v”1^ so that the check point F
-1 "2
ratio, F = (d'VQ d/k)/a0xt, and the usual lack of fit F
ratio, F = MS_ __._./MSE ..., are equivalent. We now
LOF (A)' pure (A)
present an example to illustrate the result in Eq. (5.2).
Let us fit a second degree Scheffe polynomial model to
the following hypothetical or artificial response
observations collected at the six points of the {3,2}
simplex lattice design:

151
" 2350
1
0
0
0
0
0
2370
1
0
0
0
0
0
2450
0
1
0
0
0
0
2430
0
1
0
0
0
0
2650
0
0
1
0
0
0
2670
0
0
1
0
0
0
2400
X =
. 5
.5
0
.25
0
0
2420
.5
.5
0
. 25
0
0
2750
. 5
0
. 5
0
. 25
0
2730
.5
0
.5
0
. 25
0
2950
0
. 5
.5
0
0
. 25
2970 _
0
.5
.5
0
0
. 25
The model is Y = 2360x^ + 2440x^ + 2660x^ + 40x^x^ +
920x^2 + 1640X2X2* Since each of the six design points is
replicated twice, there are six degrees of freedom available
for estimating the experimental error variance. Let an
* 2
estimate of the error variance be 0 . = MSE =
ext pure
SSE /6 = 1200/6 = 200, and this value will be the
pure' '
denominator of the check point lack of fit F ratio.
Let us choose the three points (2/3, 1/6, 1/6),
(1/6, 2/3, 1/6), and (1/6, 1/6, 2/3) as check points and
assume that we have observed the following values at these
points
2690
2/3
1/6
1/6
1/9
1/9
1/36
Y* =
2770
X*
=
1/6
2/3
1/6
1/9
1/36
1/9
2980
1/6
1/6
2/3
1/36
1/9
1/9
The numerator
of
the
check
point
lack
of fit F
ratio
is then
calculated
to
be
24546.
7, so
that
F = (d' V01d/k)/(jgxt = (24546.7/3)/200 = 40.91.

152
If we use all of the observed response values to fit
the second order Scheffé polynomial, then the model is
Y = 2360.7x^ + 2437.7x2 + 2661.7x2 + I83.3X2X2 + 1071.3x^x^
+ 1785.3x2X2 and the residual sum of squares is SSEA =
25746.7. This residual sum of squares can be partitioned
into SSlof(A) = 24546.7 and SSEpure(A) = 1200. The F ratio
for testing lack of fit is calculated to be
F = MSL0F(A)/MSEpure(A) = [24546.7/3]/[1200/6] = 40.91,
which is identical to the previously calculated F value.
In the above example we note that SSEpUre(A) ■'•s ec3ual
to SSE (SSE = SSEpUre^A)) so that sSL0F^Aj is equal
to d'y^d. Since both the check point F ratio and the usual
lack of fit F ratio have produced the same value, F = 40.91,
we conclude that the two methods for testing lack of fit in
the fitted model are equivalent.
In order to put this dissertation in a better
perspective, we now make some concluding remarks on the lack
of fit testing procedures investigated, including possible
drawbacks, extensions, and recommendations for future work.
An aspect of our investigations that may raise some
questions is that our methods are dependent on the
specification of the form of the true model believed to be
responsible for lack of fit in the fitted model. Requiring
the form of the true model to be specified was necessary in
order to be able to investigate the power of the testing
procedures. There are situations, however, where a complete
or true model can reasonably be specified. One example

153
could be in fitting polynomial models, where the polynomial
of one degree higher than the fitted model could be taken as
the true model.
We now mention two ways in which our results can be
applied to more general situations than may be readily
apparent from our previous discussions. First, we point out
that all examples in Chapters Three and Four dealt with
polynomial models. This type of model was selected because
of its popularity and wide applicability, however, our
methods can be applied not only to polynomial models but to
any models which are linear in their parameters. Secondly,
it was our intent in this dissertation to discuss methods
for testing lack of fit in a mixture model, but the methods
discussed can certainly be used not only in mixture problems
but also in general response surface problems in which a
linear model is fitted. This generalization is illustrated
for the near neighbor approach to lack of fit testing
through the stack loss example in Chapter Four.
Topics for future research stemming from this
dissertation were listed in the concluding paragraphs of
Chapters Three and Four. One area suggested in Chapter
Three was to investigate the effect of experimental design
on the selection of check points and on the resulting power
of the test. Perhaps a "minimum bias" design could be used
for fitting the model, while lack of fit could be detected
with "high bias" check points, but this in only speculation,
and needs to be investigated.

154
The fact that the check points method and the standard
method that partitions the residual sum of squares into lack
of fit and pure error portions were found, under a certain
condition, to be equivalent suggests that selecting check
points to maximize the power of the check point F test may
in general be equivalent to choosing points to augment the
original design. The augmented points would be chosen to
maximize the power of the F test that partitions the
residual sum of squares into lack of fit and pure error sums
of squares. An investigation of the selection of optimal
check points versus the selection of optimal augmented
design points would be of interest.
For the near neighbor test for lack of fit it was
recommended in Chapter Four that other methods besides the
iterative partitioning clustering algorithm might be
considered for selecting groups of near neighbors. The
effect of the number and composition of the groups selected
on the power of the test through their effect on the
parameters of the doubly noncentral F distribution could
then be investigated.
In view of the equivalence of the check point method
and the method that partitions the residual sum of squares
when replicates exist (see Eq. (5.2)), it would be of
interest to investigate whether there is also some
equivalence between Shillington's near neighbor F ratio and
the check point F ratio, F = (d'Vg^d/k)/MSE, to be used when
2
an external estimate of a is not available. If the methods

155
are not equivalent, perhaps one could be shown to be
preferable to the other as judged by comparing the power of
the two procedures in testing for lack of fit.
Finally, the focus of this dissertation has been on
testing lack of fit in linear models so that another area
for future investigation can be the problem of testing lack
of fit in models which are nonlinear in their parameters.

APPENDIX 1
INFLUENCE OF A, ON p{ F" > F }
1 vx ,v2 ;Ai , X 2 a ;vx ,v2
In this appendix we show that PÍ F" > F }
v 1 (^2 iX i ,X2 a ; v i , v 2
is an increasing function of
Let X , . .., X , Y., ..., Y be independent N(0,1).
I v i I v 2
Then
F = (v /v )[(X + \]/2)2 + E X?]/[(Y + X1/2)2 + E Y2]
i=2 1 1 ¿ i=2 1
is distributed as F" , , where v, and v0 are the
respective numerator and denominator degrees of freedom and
X^ and X2 are the respective numerator and denominator
noncentrality parameters (Scheffé, 1959, p. 412-413).
Fixing the values of v2, and X2 we wish to show
PÍF" , , > F } is a strictly increasing function
v 1 ,v2 ,*a 1 ,X2 a;vj ,v2
of Xi, where F represents the upper 100a percentage
-*â–  a ;v 1 ,v2
point of the central F distribution with and v2 degrees
of freedom. Let
fU^2)
= P( l.j/Vjll (+ X2/2)2 + t X?]/[(Y1 + X2/2)2 + E Y?)
V .
2
i=2
> F
a ;vj ,v2
156

157
then
) may be rewritten as
f (X
p{ (Xi + x^/2)2 > u} = i - p{ (x1 + x^/2)2 < u} ,
( Al. 1)
where u = + X^2)2 + £ ^]^.Vl,V2 - *•
Note that the random variable U is independent of X^.
If X^/2 and denote any two values of X^A such
that x|/2 < *122f tlien we shall prove that for f(A^/2)
defined as in (Al. 1) , < f(X^^). Now
f(x}/2) = 1 - / q^h (u)p(u)du where p(u) is the p.d.f. of Ü,
1 0 A 1
and for any positive number, u', g,^ (u') denotes the
A l
1/2 2
conditional probability that (X^ + X^ ) < u', given
1/2
1/2
U = u'. However, this conditional probability must be the
same as the unconditional probability, since X-^ and U are
statistically independent.
Thus qM (u') is the probability that the random
A1
1/2
variable X-^ falls in an interval of half length u'
1/2
centered at -X^' . Since X^ ~ N(0,1), this is a decreasing
function of x\^2. Therefore q,h (u') - q.h (u') > 0
1 All Al2
for all u' > 0. Hence,
f(Xll2) " f(X122) = o ^■_gX^1(u) + 9x^2 (U)^P( U)dU < °*

158
Thus PÍF" , , > F } is a strictly increasing
v 1 f v2 1 2
function of Xp
We note that this proof is a modification of the proof
that PÍ F" , , > F } is decreasing in
V1/V2/A1/A2 a ; v 1 , v 2 ¿
(Scheffé, 1959, p.136).

APPENDIX 2
A CONTROLLED RANDOM SEARCH PROCEDURE
FOR GLOBAL OPTIMIZATION
W. L. Price (1977) describes a conceptually simple
random search procedure, called "a controlled random search
procedure for global optimization," which is effective in
searching for global minima of a function of n variables,
with or without constraints. The procedure does not require
the function to be differentiable or the variables to be
continuous.
An initial search domain, V, is defined by specifying
upper and lower bounds for each of the n variables, and a
predetermined number, N, of trial points are chosen at
random over V, consistent with any constraints. The
function is evaluated at each of the N trial points and the
position as well as the value of the function at each point
are stored in an array, A'. At each iteration a new trial
point, P, is selected randomly from a set of possible trial
points whose positions are related to the configuration of
the N points currently in storage. If P satisfies the
constraints, the function is evaluated at P and the function
value, fp, is compared with fm, which is the greatest
function value for the N points already in storage.
If fp < fm then M, the point in storage corresponding to fm,
is replaced, in the array A', by P. if p fails to satisfy
159

160
the constraints or if f > f then the trial is discarded
p m
and a new point is chosen from the potential trial set.
As the algorithm proceeds, the set of N points in
storage tend to cluster around minima. As Price states,
"the probability that the points ultimately coverge onto the
global minimum (minima) depends on the value of N, the
complexity of the function, the nature of the constraints
and the way in which the set of potential trial points is
chosen."
Price notes that since the procedure is intended to
find global minima, thoroughness of search is more important
than speed of convergence, but if the procedure is to be
more efficient than pure random search the probability of
success (f < f ) at each iteration must be sufficiently
pm
high. His procedure reaches a compromise between the
requirements of search and convergence by defining the set
of potential trial points in terms of the configuration of
the N points already in storage. At each iteration n + 1
distinct points, R,, R_, ..., R ., are chosen at random
-1 -2 -n+1
from the N (N > n) currently in storage and these constitute
a simplex of points in n-space. The point R^+^ is
arbitrarily chosen as the vertex of the simplex, and the
next trial point, P, is taken as the image of the vertex
with respect to the centroid, G, of the remaining n
points. Thus P = 2G - R , . He notes that it is possible
- -n+1
to speed up covergence by selecting the vertex as the
point R^, i = 1, 2, ..., n + 1, which has the largest

161
function value of the points R , R, . .., Rn+j_ but this
would be detrimental to the thoroughness of the search.
The version of Price's procedure used in the work in
this dissertation was programmed in the FORTRAN language by
Michael Conlon of the Center for Instructional and Research
Computing Activities, the University of Florida,
Gainesville, Florida. This version of Price's procedure
selects new trial points using the suggested criterion
P = 2G - R The algorithm continues until an iteration
- - -n+1
limit is reached or a desired tolerance between the minimum
and maximum function values in storage is achieved.
In our particular application, if P2 = 1 so that A^ is
a scalar, we wish to maximize
Ax = (X* - X*A)'V"1(X* - X*A),
with respect to k check points, in order to maximize the
power of an upper tailed test. For locating check points
that maximize the power of a lower tailed test it is
necessary to minimize A^. If p > 1 so that Aj_ is not a
scalar, but is a P2XP2 matri-x/ then it will be necessary to
maximize or minimize certain eigenvalues of A-^.
All of these optimization problems can be handled by
Price's procedure. Since the procedure finds minima, then
to find maxima, we simply minimize the negative of the
function under consideration. The restriction that the
check points must be located within the experimental simplex

162
(or a subregion of the simplex) is taken care of by
specifying constraints in the program.
To give a specific example, suppose we fit a second
order canonical polynomial model in a three component
mixture space, using a simplex centroid design. If we
assume the true model is special cubic in the three
components, then P2 = 1, and
A1 = {X2 " X* A) ' Vg1 ( Xíp - X* A)
is a scalar quantity. In order to locate a single check
point that maximizes the power of an upper tailed test for
lack of fit, we select the check point that maximizes A-^.
Since the experimental region we wish to search is the
entire two dimensional simplex, we define the check point
as x*' = (x^, x2, x^), and in our program impose the
constraints:
and
0 < x < 1,
0 < x 2 < 1.
We then define x^ as x^ = 1 - , while requiring
that 0 < < 1. Price's random search procedure is used to
search the two-dimensional simplex for the point (x-^, x2)
1*
that maximizes A

163
Price suggests the use of N = 50 storage points for
such a two-dimensional search, and we have generally found
this to be adequate. For k > 1 check points to be located
simultaneously in a three component system, the problem
becomes one of searching in 2k dimensions. For the
applications considered, N = 50k appears to be adequate.
The only real problem encountered has been that of
economics in that the procedure becomes costly in terms of
computer time for these situations where the optimal value
of the function is assumed by all points in a region. In
these cases the algorithm searches in vain for points that
will improve upon the functional values already in storage,
which all lie in this optimum region. However, in other
applications, the procedure converged quickly to an optimum
(those that converged did so in 10,000 iterations or less,
at a small cost in computational time).

APPENDIX 3
STATISTICAL INDEPENDENCE OF d'Vgd/o AND SSE/o¿
Let us write d'V^d as
d'V^d = (Y* - V*)*V-1(Y* - Y*)
^ — 1 * ^ _ 1 *
= Y 'VQ Y - Y 'Vq Y
* _ i ~ * * * _i* *
X •Vq Y + Y -V0 Y
* _ i * * _ i ~ ~ * _ i * *
= Y ' VQ Y - 2 Y 'Vq Y + Y ' VQ Y .
Now let us write SSE as
SSE = Y'(IN - X(X'X) 1X')Y.
k
Since Y and Y are
Y*'Vq1Y*. Rewriti
*
Y
independent,
ng
Y -V^Y as
SSE is independent of
where is the least squares estimator of 3^, we have
* _ i ~ * * —1* —1
X 'v0 X = X 'Vq x (x'x) X'X-
164

165
We now show that the second portion of d'y^d is independent
of SSE if and only if
[V“1X*(X'X)_1X'][IN - X(X'X)_1X'] = 0.
Define
COv[Y*'V~1X*(X'X)“1X,Y, Y'(I - X(X'X)"1X')Y]
= E[Y*,v“1X*(X,X)”1X,YY'(I - X(X'X)_1X' ) Y]
- E[Y*'Vq1X*(X,X)"1X'Y] E[Y'(I - X(X'X)_1X1Y]
= E ( Y* ' ) EIV^X* (X'X)"1X' YY' ( I - X ( X ' X ) _1X ’ ) Y ]
- E(Y* ' ) E[Vq1X*(X'X)_1X'Y] E[Y'(In - X(X'X)“1X'Y]
= E(Y*') [cov(VqXX*(X1X)_1X'Y, Y'(In - X(X'X)_1X'Y)]
= 0,
if v”XX*(X'X)-1X'Y is independent of Y'(I - X(X'X)”1X')Y.
This occurs if and only if,
[Vq1X*(X'X)"1X' ] [I - X(X'X)_:LX'] = 0,
see Searle (1971) p.59, Theorem 3. Now,

166
[v01x*(x,x) 1x,][iN - X(X'X)-1x1]
= v“1X*(X'X)“1X' - V01X*(X'X) 1X'X(X'X) 1x
= 0.
Therefore SSE is independent of the second portion of
d'Vg^d. Now we must show that SSE is independent of the
third portion of d'Y'^d. Write Y*'V~^Y* as
»*_]»* * -l *
Y 'V0XY = (X b^'VgX b1
= Y'X(X'X) 1X*'V“1X*(X'X) 1X'Y.
Then SSE is independent of the third portion of d'Vg^d if
and only if
[X(X'X)-1X* ,V()1X* (X'X) 1X'][IN - X(X'X) 1X'] = 0,
see Searle (1971), p.59, Theorem 4. Continuing then,
[X(X’X) 1X* ,V()1X* (X'X) 1X'][IN - X(X'X) 1X']
x(x'x) 1x*'v01x*(x,x)“1x,-x(x,x) 1x*,v“1x*(x,x)-1x,x(x,x)-1x'
X(X'X) 1X*,V01X*(X'X) 1X' - X(X'X) 1X*'v“1X*(X'X) 1X'
0.

167
Therefore SSE is independent of the third portion of
S'vóV
Finally, since SSE is independent of each of the three
portions of d'Vg^d, we can conclude that SSE is independent
of d'V~^d and therefore SSE/a^ is independent of d'Y^d/a^.

APPENDIX 4
THEOREM 3.1
Theorem 3.1
Let A and B be kxk matrices. If (A - B) is positive
definite and B is positive semi-definite, then A is positive
definite.
Proof
We assume that (A - B) is positive definite. Then
z'(A - B)z > 0, for all z * 0. Thus z'Az - z'Bz > 0,
for all z * 0, so that z'Az > z'Bz > 0, for all z * 0,
since B is positive semi-definite. Therefore,
z'Az > 0, all z * 0.
Now if z'Az = 0, then z = 0 for if z * 0,
then z'(A - B)z > 0 implies z'Bz < 0. But this is a
contradiction since by assumption z'Bz > 0.
Therefore z must be 0 and A must be positive definite.
168

Theorem 3.2
APPENDIX 5
THEOREM 3.2
Let and A2 be p2xp2 positive semi-definite
matrices. Let g be a p2-dimensional vector and define A^
and X2 as
2
Ai = 62A162/2a , and
A2 = B¿A232/2a2,
2
where a >0. Let k>0,N>0,p>0, and N > p.
(a) If [A^/k - A^/(N - p)] is positive definite then
[A ^/k - A2/(N - p)] = 0 if and only if A-|_ = A2 = 0.
(b) If [A^/k - A2/(N - p)] is negative definite then
[A ^/k - A2/(N - p)] = 0 if and only if X^ = A2 = 0.
Proof of part (a).
Necessity. Let [A^/k - A2/(N - p)] be positive
definite and suppose that [A-^k - A2/(N - p)] = 0. We show
that Xi = A 2 = 0. The matrix A^/k - A2/(N - p) being
positive definite implies g^fA^/k - A2/(N - p)]g2 = 0 iff
g = 0 , that is,
-2
169

170
-2Al-2/k ” 5-2A2-2//(N “ P) = 0 iff -2 = -* But'
-2Al-2//2k xiA - X2/^N “ P) = 0 iff e2 = 0.
It follows that if Aj/k - A2/(N - p) =0, then A j. = A2 = 0.
Sufficiency. Obviously, if A^ = A2 = 0, then
Aj/k - A /(N - p) =0-0=0.
Proof of part (b). This follows from part (a), since in
this case A2/(N - P) - A^/k is positive definite.

APPENDIX 6
AN APPROXIMATION TO THE DOUBLY NONCENTRAL F DISTRIBUTION
Johnson and Kotz (1970, p.197) indicate the following
approximation for PÍF" , , , < F 1 where vi and v
lv^,v2;A^,X2 a;v j ,V2j -L
are the numerator and denominator degrees of freedom,
respectively, and X^ and X2 are the numerator and
denominator noncentrality parameters, respectively:
2
PÍ F" . < F
1 V1(v2 ;Xi ,X2 a;vj ,v2
PÍ cF
1 v
V '
< F }
a;vx ,v2 J
= P{F
v , v
< (1/C)F
a ; v i , v 2
where F is the upper 100a percentage point of the cen
a ;v j ,v 2
tral F distribution with v-^ and V2 degrees of freedom, and
where c = [1 + ^/v^/U + \ 2/v 2] , v = [vj + X1]2/[v1 + 2\ ^
2
v' = [v^ +^2^ + an<^ Fv v ' a central F random
variable with v and v' degrees of freedom.
171

APPENDIX 7
EQUIVALENCE OF SSEB AND SSL0F WHEN
REPLICATES REPLACE NEAR NEIGHBOR OBSERVATIONS
In this appendix we show that SSEB = SSL0F when
response observations are partitioned into g groups of true
replicates rather than g groups of near neighbor
observations.
From Chapter Two, Section 2.2, if each cell consists
entirely of true replicates, then the sum of squares due to
lack of fit can be expressed as
SS
LOF
= SSE - SSE
pure
where SSE is the residual sum of squares from a least
squares regression of Y on X and where SSEpure is the sum of
squares due to pure error, calculated from replicates.
Since SSE = Y'E„Y, where T. n is defined as in Section
pure - 0- 0
4.2, we have
SSL0F = Y'(IN - X(X'X) 1X,)Y - Y'EqY
= Y'(IN - Eq)Y - Y,X(X'X)“1X'Y. (A7.1)
We wish to show that when each cell is composed entirely of
true replicates, SSEg is equal to the expression in (A7.1).
172

173
Recalling from Section 4.3.1 that = MY, where
M = diag[(l/n )1', ..., (1/n ) 1'], we write
-L j. y y
SSE„ = ^[G-1 - G-1XC(X¿G-1XC)-1X¿G-1]?C
= Y'M'[G0 - Gq Xc(X¿G0 Xc) x¿G0 ]my,
where from Section 4.2, = diag[l/n^, l/n^, . .., 1/n^].
Recognizing that Gq = MM' and Xq = MX, we have
SSEd = Y'[M'(MM')-1M
o “ L
- M'(MM')-1MX{X'M'(MM1)-1MX}-1X'M'(MM')_1M]Y.
Since M'(MM') ^M = IN - eq, we have
SSEB = ^(IN -
- rí
and since E qX = 0 when all cells are composed entirely of
true replicates, we have
SSE = Y'(I - Ert)Y - Y'X(X'X)_1X'Y
B - N 0 - - -

174
from (A7.1). Therefore, SSEB is equal to the usual SSLOp
when cells are composed entirely of true replicates.

APPENDIX tí
LEMMA 4.1
Lemma 4.1
2
Let Y ~ (Xg, o G), G singular. Define T = G + XX'.
Define T— such that TT—T = T.
1. (i) TT—X = X
(ii) X'T~T = X'
2. rank(X'T—X) = rank(X)
3. (i) X(X'T—X)—(X'T~X) = X
(ii) (X'T—X)(X'T—X)“X' = X'
4. Y is in the column space of T (Y e C(T)), with
probability one, by which we mean that there exists a
vector a such that letting Y = (y^, Y2/ ..., yN)'
and Ta = (x., x„, ..., x„)', then
1 2 N
P{ |y^ - x¿| > e} = 0 ,for all e > 0, i = 1, 2, ..., N
Proof
1. (i) T = XX' + G
= XX' + W, where G = W
= CC' , where C = [X:V].
Now, CC'(CC')—C = C, from Pringle and Rayner (1971, p
26), and therefore
TT [ X: V] = [ X:V]
from which it follows that TT—X = X (and TT~V = V).
175

(ii) The proof of (ii) follows directly from (i) by
taking the transpose.
The proof of part 2 is given in Rao, 1973, p. 77, #30.
The proof of part 3 is given in Rao and Mitra, 1971,
p.22, Lemma 2.2.6(c).
. 2
By definition, Y ~ (Xg , a G) so that an equivalent
representation for Y is Y = Xg + e , where e ~ (0, a2G).
We wish to show that the random vector Y is in the
column space of T, with probability one. It is
sufficient to show that TT—Y = Y, w.p.l. (see Pringle
and Rayner, 1971, p.9). Rewriting TT~Y we have
TT—Y = TT~(Xg + e)
= TT—Xg + TT—e .
By part 1 of Lemma 4.1, TT~X = X, and therefore
Xg e C(T). The proof is complete if we show TT—e = e,
w.p.l. The difference TT~e - e can be written as
TT—e - e = (TT— - I )e, therefore we must show (see
- - N -
explanation below) that
E[e1(TT— - In)'(TT_ - I )e] = 0. (A8.1)
The expectation in Eq. (A8.1) can be written as

177
E[e'(TT - IN)'(TT - IN)e]
= trace [(TT- - I )'(TT- - I ) L N N J
= a2 trace [ (TT- - I ) ' (TT- - I )W']
= 0
since TT—V = V, by proof of part 1(i) of Lemma 4.1.
Therefore Y e C(T), w.p.l.
We now show that proving the equality in (A8.1) is
equivalent to proving that TT—e = e, w.p.l. By the Markov
Inequality
P{ |ui - vi| > e} < [ E (u ¿ - vi)2]/e2
and therefore if E(u¿ - v¿) = 0, we have u¿ = v¿, w.p.l.
If u' = (Uy u2, ...,uN), v' = {Vy v2/ ..., vN) / and if
2
E(u^ - v^) = 0, for i = 1, 2, ..., N, then u¿ = v¿, w.p.l,
for i = 1, 2, ..., N, which implies that u = v, w.p.l. But
E(u^ - v^)2 = o, for i = 1, 2, ..., N if and only if
N 2
I E(u. -v.) = 0, and since
i=l 1 1
N 2
I E(u. - v. ) = E(u - v)'(u - v)
i=l 1 1
we have u = v, w.p.l, if E(u - v)’(u - v) =0. In (A8.1) we
= e .
take u
TT e and v

APPENDIX 9
PROOF OF THEOREM 4.1(i)
In this appendix we give the proof of Theorem 4-1(i).
We show that E(o2) = a2, where o2 = f_1(Y - XB)'T-(Y - XB ) .
~ 2
First we write a as
o 2 = f-1[Y'T—Y - 2B'X'T_Y + B'X'T~XB] ,
where B = (X1T—X)—X1T—Y. Now,
B'X'T~XB = B'X1T—X(X'T—X)—X'T—Y
= B'X'T_Y,
by Lemma 4.1, part 3(ii). Therefore
a2 = f-1[Y'T—Y - B'X'T~Y]
= f_1[ Y'T~Y - { ( X 1 T— X)—X' T~ Yj'X'T- y]
= f_1[ Y'T~Y - Y ' (T—) ' X ( X' T—X) ~X ' T—Y]
= f 1y,aqy
( A9.1)
where A
0
(T )1X(X'T X) X'T . Using equation (A9.1),
178
T

179
and applying Theorem 1(i) (Searle, 1971, p.55), we can write
~ 2
the expected value of a as
E(o2) = f-1E[Y'A0Y]
= f_1[ trace{AQa2G} + E(Y)'AQE(Y)]
= f ^ trace[A^a2G] , (A9.2)
since
E(Y)'AqE(Y) = 0'X'[T“ - (T~)1X(X1T~X)_X'T~]X3
= e'X'T~Xe - 3'(X1T~X)(X'T-X)—(X'T—X)0
= 0
as X'(T—)'X = X'T~X, because T is symmetric and
X'T~X is unique (see proof of Theorem 4.1(ii)). Thus
E(a2) = f trace[Ag = a2f 1 trace[ A^G]
= a2f 1 trace[Ag(T - XX')].
By writing Aq as in Eq. (A9.1), we get

180
- 2 2 —1
E(a ) = a f trace [{T - (T )'X(X'T X) X'T }{T - XX'}]
2 _ i
= a f trace [T T - T XX' - (T )'X(X1T—X)—X1T—T
+ (T_) 'X(X'T~X)_X'T~XX']
2 —1
= a f [trace T T - trace T XX'
- trace (T-) 'X(X'T-X)-X' + trace (T_)'XX'],
by Lemma 4.1, parts 1 and 3, and so
2 2 —1
E(a ) = a f [trace T T - trace (X'T X) (X'T X)],
since X'(T—)'X = X'T“X. Since T—T and (X'T~X)“(X'T_X) are
idempotent, and rank(AA~)= rank(A) for any matrix A, we see
that
E(a 2) = a 2f-1[rank(T) - rank(X'T-X)]
and by Lemma 4.1 part 2 we have
E(a2) = a2f ^[rank(T) - rank(X)].

181
Finally, since f = rank(G:X) - rank(X), we can write
E(a2) = a2f ^[rank(G:X) - rank(X)] = a2. (A9.3)
The proof of Theorem 4.1(i) is now completed by
justifying the equality in (A9.3) by showing that rank (T) =
rank (G:X). First we write
rank(T) = rank(G + XX').
Replacing G by W , we have
rank(T)
rank(W + XX')
rank(CC ' ) / where C = (V:X)
rank(C)
rank(V:X)
rank(G:X),
since the column space of G is the same as the column space
of V. The column space of G is the same as the column space
of V if the columns of V belong to the column space of G,
and vice versa, if the columns of G belong to the column
space of V. Symbolically, this is written as
V c C(G), and G C(V). To show that V <= C(G), it is
sufficient to show that GG—V = V, but this is true because
GG“V = (W')(W)-V = V. Now, G c C(V) since by
definition W = G.

APPENDIX 10
PROOF OF THEOREM 4.1(ii)
In Appendix 10 we prove part (ii) of Theorem 4.1, thus
we show that a2 = f ^(Y - Xg)'T (Y - X3) is unique with
probability one. The following theorem will be useful in
our proof.
Theorem vi(c) (Rao, 1973, p.26).
Let B and D be non-null matrices. Then BA—D is
invariant for any choice of A- if and only if
C(B') c C(A') and C(D) c C(A), where C(.) denotes column
space.
The relationship C(B') <= C(A') holds if and only if BA-A = B,
and similarly C(D) c C(A) holds if and only if AA—D = D (see
* 2
Pringle and Rayner, 1971, p.9). Since the quantity a is
written as
a2 = f“1[Y'T“Y - Y'(T—)'X(X'T—X)—X'T—Y],
* 2
to show that a is unique with probabilty one, it suffices
to show that
Y'T Y - Y'(T )'X(X'T X) X'T Y
(A10.1)
182

183
is invariant with probability one to the choice of the
generalized inverses involved.
First we show that Y'T-Y is unique with probability one
(w.p.l). From part 4 of Lemma 4.1, Y e C(T), w.p.l.
Therefore Y' e C(T'), w.p.l, since T is symmetric, and then
by Theorem vi(c) (Rao, 1973, p.26), Y'T-Y is unique, w.p.l.
Secondly we show that Y'(T—)'X(X'T—X)—X'T—Y is unique
with probability one in the following four part proof.
(1) Show X’T-X is unique.
From part 1 (i) of Lemma 4.1, TT—X = X and thus X<=C(T).
Since T is symmetric, we have X cC(T'). By Theorem vi(c)
(Rao, 1973, p.26), X'T-X is unique.
(2) Show X'T-Y is unique, w.p.l.
By (1) above, X<=C(T') and by part 4 of Lemma 4.1, Ye C(T),
w.p.l. Thus applying Theorem vi(c) (Rao, 1973, p.26),
X'T-Y is unique, w.p.l.
(3) Show Y’(T—)'X is unique, w.p.l.
This follows from part (2), since Y'(T—)'X is equal to the
transpose of X'T-Y , which was shown in (2) to be unique,
w.p.l.
(4) Using (1), (2), and (3) above, the second quantity
in (A10.1) is unique, w.p.l, by Theorem vi(c) (Rao, 1973,
p.26) if
(a) [Y'CD'X]' e C[(X"T”X)'], w.p.l, and
(b) X'T-Y £ C(X'T—X), w.p.l.
Part (a) is true not only with probability one but always
because Y'(T—)'X(X'T-X)-(X'T-X) = Y’(T-)'X, since by part

184
3(i) of Lemma 4.1, X = X(X'T X) (X'T X). Part (b) is true
not only w.p.l but always because
(X'T—X)(X'T—X)~ X'T—Y = X'T—Y, by part 3(ii) of Lemma 4.1.
Therefore we have shown that both Y'T-Y and
Y'(T~)'X(X'T~X)—X'T-Y are unique with probability one, which
* 2
allows us to conclude that 0 is unique with probability
one.

APPENDIX 11
PROOF OF THEOREM 4 -1(iii)
In this appendix we prove Theorem 4.1(iii), that is we
show that if Y possesses an N-variate normal distribution
A 2 2 2
then fa /a ~ Xf > where f = rank(G:X) - rank(X). Recall
that a2 = f_1Y'AqY where AQ = T- - (T_)'X(X'T~X)_X'T~.
•* 2
Since we have shown in Theorem 4.1(ii) that a is unique
with probability one, the choice of the generalized inverses
a 2
in the expression for a may be made arbitrarily. Thus we
choose each of the generalized inverses to be the unique
Moore-Penrose inverse, and we denote the unique Moore-
Penrose inverse of a matrix B by B+. The Moore-Penrose
inverse has the following four properties (see Searle, 1971,
p.16):
1. BB+B = B
2. B+BB+ = B+
3. (BB+)' = BB+
4. (B+B)' = B+B.
~ 2 2
The quantity fa /a can be expressed as
fa2/a2 = Y'AY, (All - 1)
where A = (l/a2)[T+ - T+X(X'T+X)+X'T+]. We wish to show
185

186
2
that Y'AY ~ Xf • which can be done by making use of the
following corollary.
Corollary 2s.1 (Searle, 1971, p.69).
When x is N(y,V) whether V be singular or non-singular,
2
x' Ax ~ x1 with degrees of freedom equal to trace(AV)
and noncentrality parameter equal to (l/2)y'Ay, where
2
x' denotes a noncentral chi-square random variable,
if and only if
(i)
VAVAV ;
= VAV
(ii)
ii
%
P.I
y'AVAV,
(iii)
P ' Ay =
y'AVAy.
In our application, the matrices A and V in Corollary 2s.1
(Searle, 1971, p.69) are defined as
A = (1/a2)[I - T+X(X'T+X)+X']T+,
and
V = a2G .
The proof of Theorem 4.1(iii) follows from Corollary 2s.1
(Searle, 1971, p.69) if we can show that AVA = A. To show
that AVA = A, we first show that AG = AT, where as we
recall, AG = A(T - XX'). Thus AG = AT if AXX' =
the complete expression for A, we have
0.
Using

187
AXX' = (1/a 2)[T+ - T+X(X'T+X)+X'T+]XX'
= (1/a 2)[T+XX' - T+X(X'T+X)+(X'T+X)X'],
and so by Lenuna 4.1 part 3 ( i ) ,
AXX' = (1/a 2)[T+XX' - T + XX']
= 0.
2 2
Therefore, since AG = AT, we have AVA = a AGA = a ATA. We
2
now show that a ATA = A:
a2ATA = (1/a 2)[I - T+X(X'T+X)+X']T+T[I - T+X(X'T+X)+X']T +
= (l/a2)[T+T-T+X(X'T+X)+X'T+T ][T+-T+X(X'T+X)+X'T+]
= (1/a 2)[T+TT+ - T+X(X'T+X)+X'T+TT +
- T+TT+X(X'T+X)+X'T+
+ T+X(X'T+X)+X'T+TT+X(X'T+X)+X'T+]
= (1/a 2)[T+ - T+X(X'T+X)+X'T+ - T+X(X'T+X)+X'T +
+ T+X(X'T+X)+X'T+X(X'T+X)+X'T+]

188
since T+TT + = T+, by property 2 of the Moore-Penrose
inverse. Therefore,
ATA = (1/a 2)[T+ - 2T+X(X'T+X)+X'T+ + T+X(X’T+X)+X'T + ]
= (1/a ^)[I - T+X(X'T+X)+X']T +
= A.
2
Since we have verified that AVA = a ATA = A we can conclude
that fa 2/o2 = Y' AY ~ ^ , by Corollary 2s.1 (Searle, 1971,
p.69). The quantity fa2/a2 ~ Xf and not x¿2' si-nce the
noncentrality parameter equals zero, which we now show.
The noncentrality parameter, from Corollary 2s.1
(Searle, 1971, p.69) is of the form (l/2)y'Au, where in our
application, y = Xg. Thus,
y'Ay = g'X'AXg
= (1/a 2 )g ' X' [ T+ - T+X( X'T+X)+X'T + ] Xg
= (1/a 2)[g 'X'T+Xg - g 'X'T+X(X'T+X) + (X'T+X)g]
and so by Lemma 4.1 part 3(i),
y'Ay_ = ( 1/a 2 ) [ g 1 X ' T+Xg - g'X'T+Xg]
0.

189
We now verify that the degrees of freedom are
f = rank(G:X) - rank(X). From Corollary 2s.1 (Searle, 1971,
p.69) the degrees of freedom associated with Y1 AY are equal
2
to f = trace(a AG), and so
trace(a 2AG) = trace[I - T+X(X'T+X)+X’]T+G
= trace(T+G) - trace[T+X(X'T+X)+X'T+G]
= trace(T+T - T+XX') - trace[T+X(X'T+X)+X'T+T]
+ trace[T+X(X1T+X)+X'T+XX'],
since G = T - XX1. It follows that
trace(a2AG) = trace T+T - trace T+XX' - trace X(X'T+X)+X'T+
+ trace T+XX',
since trace AB = trace BA for arbitrary matrices A and B,
t+tt+ = T+, and X(X'T+X)+(X'T+X) = X by Lemma 4.1 part
3 ( ii ) . Therefore
trace(a2AG) = trace T+T - trace (X'T+X)(X'T+X)+
= rank(T) - rank(X'T+X),

190
since TT+ and (X1T+X)(X'T+X)+ are idempotent, and
rank(AA+) = rank A, for any matrix A. Finally, by Lemma 4.1
part 2 we have
trace(a2AG) = rank(T) - rank(X)
and by the argument in the proof of Theorem 4.1(i),
trace(a2AG) = rank(G:X) - rank(X).

APPENDIX 12
PROOF OF THEOREM 4.2
In this appendix we prove Theorem 4.2, thus we show
that when Y~ N..(X3 + X03_, o2G) then fa2/o2 ~ xl2w
N “ Z 4 £ f A
where X = (l/2o 2 )3 ^ [T- - T_X(X'T_X)~X'T~]X23 .
~ 2 2 2
From the proof of Theorem 4.1, we have fa /a ~ x¿ ,
t, a
By Corollary 2s.1 (Searle, 1971, p.69) the noncentrality
parameter is
X = (1/2)(X0 + X202)'A(X3 + X232),
where A = (1/o2)[t~ - T~X(X'T-X)_X’T“]. Thus
X = (1/2) [ 3 'X1AX3 + S'X’AX^ + B^X^AXB + B£X£AX B ].
From the proof of Theorem 4.1(iii), 3'X'AXB =0. We now
show that 3j,X^AX3 = 0:
3£X^AX3 = 3_^X2[ T— - T_X ( X • T-X) _X' T_] X3 /o 2
= 3J;x^[t_ X- " T_ X(X,T~" X)“(X'T- X) 3 ] /o 2 ,
and so by Lemma 4.1 part 3(i),
3^X^AX3 = 3^X^[t-X3 - T-XB]/o2
= 0.
191

Thus we conclude that
X = (l/2)g¿X^AX20
= (l/2a2)g¿X^[T

APPENDIX 13
PROOF OF THE EQUALITY SSE = d’V^d + SSE
In this appendix we show that the check point method
for testing a fitted model for lack of fit and the method in
which the residual sum of squares is partitioned into a lack
of fit sum of squares and a pure error sum of squares are
equivalent in the sense that SSE = d'y^d + SSE.
Let us define Y. and X, as
-A A
(A13.1)
and
X
A
X
X*
(A13.2)
Then the residual sum of squares from regressing
Y on
-A
is
SSE
= rA[i -
XA(XÁXA)_lxÁ]^A*
193

Using Eqs. (A13.1) and (A13.2) we can write SSEA as
_
ssea =
Y
Y*
1
I -
X
X*
(X’X + X*^*)-1
X
X*
1
1
-
Y
Y*
= Y*'[I - X*(X'X + X*'X*) 1X*,]Y*
- 2 Y*1X*(X1X + X*'X*) 1X'Y
+ Y'[I - X(X'X + X*'X*) 1X']Y
Y*'V01Y* - 2Y*'X*(X1X + X*'X*)_1X,Y
+ Y'[I - X(X'X + X*'X*) 1X']Y.
(A13.3)
Eq. (A13.3) is true because from Eq. (8) (Morrison, 1976,
69) we can write Vq1 as
v"1 = [I + X*(X'X) 1X*'] 1
= I - X*(X'X + X*'X*) 1x*

195
We now write the quadratic from d'Vg^d as
d ' V^d = (Y* - Y* ) ' Vq 1 (Y* - Y* )
= Y* ' V”"1 Y* - 2Y* ' V”1 Y* + Y* ' V”1 Y*
= Y*V”1Y* - 2 Y* 1 V^X* ( X ' X ) _1X' Y
+ Y'X(X'X)"1X*,V“1X*(X,X)_1X'Y. (A13.4)
The first portion in Eq. (A13.3) is equal to the first
portion in Eq. (A13.4). We now show that the second
portions of Eqs. (A13.3) and (A13.4) are equal. It can be
verified using Eq. (8) (Morrison, 1976, p. 69) that
(X'X + X^X*)-1 = (X'X)"1 - (X,X)-1X*,V”1X*(X,X)_1.
(A13.5)
Using Eq. (A13.5) the second portion of SSEA in Eq. (A13.3)
can be written as
- 2 Y*'X*(X1X + X*'X*)_1X'Y
= -2Y*'X* [(X'X)”1 - (X'X)_1X*,V~;LX*(X,X)“1]X, Y
= -2Y*1[I - X*(X'X)_1X*'V"1]X*(X,X)"1X,Y.
(A13.6)

196
The second portion of SSEA, given in Eq. (A13.6),
equal the second portion of d'v”1^ in Eq. (A13.4)
fact that
I - X*(X'X)"1X*'Vq1 = I - (VQ - I)Vq1
We now show that the third portion of the expression
for SSEA in Eq. (A13.3) is equal to the sum of the third
portion of d'y^d in Eq. (A13.4) and SSE, where
SSE = Y'[I - X(X,X)“1X']Y. Using the result in Eq. (A13.5),
the third portion of SSEA in Eq. (A13.3) can be written as
Y'[I - X(X'X + X*'X*)-1X']Y
= Y'[I - X{(X'X)_1 - (X'X)"1X*,Vq1X*(X'X)"1}X,]Y
= Y'[I - X(X'X)_1X']Y + Y'X(X,X)_1X*,Vq1X*(X'X)-1X,Y.
Therefore, since the first two portions of SSEA in Eq.
(A13.3) are equal to the first two portions of d'y^d in Eq.
(A13.4), respectively, and the third portion of SSEA in Eq.
(A13.3) is equal to the sum of the third portion of d'Y^d
is seen to
using the

197
in Eq. (A13.4) and Y'[I - X(X'X) ^X']Y, we must have
SSEa = d1V~^d + Y’[I - X(X,X)”1X']Y
= d 1 V^d + SSE .
then

REFERENCES
Atkinson, A. C. (1972). Planning
inadequate regression models.
275-293.
experiments to detect
Biometrika, Vol. 59, pp.
Barr, A. J., J. H. Goodnight, and J. P. Sail (1979). SAS
Users Guide, 1979 Edition. SAS Institute, Inc.,
Raleigh, North Carolina.
Becker, N.G. (1968). Models for the response of a mixture.
Journal of the Royal Statistical Society, _B_, Vol. 30,
pp. 349-358.
Becker, N.G. (1978). Models and designs for experiments
with mixtures. Australian Journal of Statistics, Vol.
20, pp. 195-208.
Box, G. E. P., and N. R. Draper (1975). Robust designs.
Biometrika, Vol. 62, pp. 347-352.
Claringbold, P. J. (1955). Use of the simplex design in the
study of the joint action of related hormones.
Biometrics, Vol. 11, pp. 174-185.
Cochran, W.
2nd Ed .
G., and G. M. Cox (1957). Experimental Designs,
John Wiley and Sons, New York.
Cornell, J.
review.
A. (1973). Experiments with mixtures:
Technometrics, Vol. 15, pp. 437-455.
A
Cornell, J. A. (1981). Experiments with Mixtures: Designs,
Models, and the Analysis of Mixture Data. John Wiley
and Sons, New York.
Cornell, J. A., and I. J. Good (1970). The mixture problem
for categorized components. Journal of the American
Statistical Association, Vol. 65, pp. 339-355.
Daniel, C., and F. S. Wood (1971). Fitting Equations to
Data: Computer Analysis of Multifactor Data for
Scientists and Engineers. John Wiley and Sons, New
York.
Draper N. R., and A. M. Herzberg (1971). On lack of fit.
Technometrics, Vol. 13, pp. 231-241.
198

199
Draper, N. R., and W. E. Lawrence (1965a). Mixture designs
for three factors. Journal of the Royal Statistical
Society, _B, Vol. 27, pp. 450-465.
Draper, N. R., and W. E. Lawrence (1965b). Mixture designs
for four factors. Journal of the Royal Statistical
Society, _B_, Vol. 27, pp. 473-478 .
Draper, N. R., and H. Smith (1981). Applied Regression
Analysis, 2nd Ed. John Wiley and Sons, New York.
Draper, N. R., and R. C. St. John (1977). A mixtures model
with inverse terms. Technometrics, Vol. 19, pp. 37-46.
Gorman, J. W., and J. E. Hinman (1962). Simplex-lattice
designs for multicomponent systems. Technometrics, Vol.
4, pp. 463-487.
Graybill, F. A. (1969). Introduction to Matrices with
Applications in Statistics. Wadsworth Publishing Co.,
Inc., Belmont, California.
Graybill, F. A. (1976). Theory and Application of the
Linear Model. Duxbury Press, North Scituate,
Massachusetts.
Green, J. R. (1971). Testing departure from a regression,
without using replicaton. Technometrics, Vol. 13, pp.
609-615.
Johnson, N. L., and S. Kotz (1970). Distributions in
Statistics: Continuous Univariate Distributions - 2.
Houghton Mifflin, Boston.
Kenworthy, 0. 0. (1963). Factorial experiments with
mixtures using ratios. Industrial Quality Control, Vol.
19, pp. 24-26.
Khuri, A. I., and J. A. Cornell (1981). Lack of fit
revisited. Technical Report No. 167, Department of
Statistics, University of Florida, Gainesville, Florida.
Kurotori, I. S. (1966). Experiments with mixtures of
components having lower bounds. Industrial Quality
Control, Vol. 22, pp. 592-596.
McLean, R. A., and V. L. Anderson (1966). Extreme vertices
design of mixture experiments. Technometrics, Vol. 8,
pp. 447-454.
Morrison, D. F. (1976). Multivariate Statistical Methods,
2nd Ed. McGraw-Hill, New York.

200
Mudholkar, G. S., Y. P. Chaubey, and C. Lin (1976).
Approximations for the doubly noncentral-F
distribution. Communications in Statistics/ A, Vol. 5,
pp. 49-63. —
Myers, R. H. (1971). Response Surface Methodology. Allyn
and Bacon, Inc., Boston.
Piepel, G. F. (1981). Measuring component effects in
constrained mixture experiments. Unpublished manuscript
presented at 1981 Joint Statistical Meetings of the
American Statistical Association and the Biometric
Societies, Detroit, Michigan, August 1981.
Price, W. L. (1977). A controlled random search procedure
for global optimisation. The Computer Journal, Vol. 20,
pp. 367-370.
Pringle, R. M., and A. A. Rayner (1971). Generalized
Inverse Matrices with Applications to Statistics.
Hafner Publishing Company, New York.
Rao, C. R. (1971). Unified theory of linear estimation.
Sankhyá, A., Vol. 33, pp. 371-394.
Rao, C. R. (1972). Unified theory of least squares.
Communications in Statistics, Vol. 1, pp. 1-8.
Rao, C. R. (1973). Linear Statistical Inference and its
Applications, 2nd Ed. John Wiley and Sons, New York.
Rao, C. R., and S. K. Mitra (1971). Generalized Inverse of
Matrices and its Applications. John Wiley and Sons, New
York.
Scheffe, H. (1958). Experiments with mixtures. Journal of
the Royal Statistical Society, _B, Vol. 20, pp. 344-360.
Scheffe, H. (1959). The Analysis of Variance. John Wiley
and Sons, New York.
Scheffe, H. (1963). The simplex-centroid design for
experiments with mixtures. Journal of the Royal
Statistical Society, _B, Vol. 25, pp. 235-263 .
Searle, S. R. (1971). Linear Models. John Wiley and Sons,
New York.
Shillington, E. R. (1979). Testing lack of fit in
regression without replication. The Canadian Journal of
Statistics, Vol. 7, pp. 137-146.

201
Snee, R. D. (1971). Design and analysis of mixture
experiments. Journal of Quality Technology, Vol. 3, pp.
159-169.
Snee, R. D. (1973). Techniques for the analysis of mixture
data. Technometrics, Vol. 15, pp. 517-528.
Snee, R. D. (1977). Validation of regression models:
methods and examples. Technometrics, Vol. 19, pp. 415-
428.
Thompson, W. O., and R. K. Myers (1968). Response surface
designs for experiments with mixtures. Technometrics,
Vol. 10, pp. 739-756.
Wishart, D. (1975). Clustan 1C User Manual. David Wishart,
University College, London.
Zyskind, G. (1967). On canonical forms, non-negative
covariance matrices and best and simple least squares
linear estimators in linear models. Annals of
Mathematical Statistics, Vol. 38, pp. 1092-1109.

BIOGRAPHICAL SKETCH
John Thomas Shelton was born on March 30, 1952, in
Jacksonville, Florida, where he resided until graduating
from Englewood High School in June, 1970. He then entered
the University of Florida where he received a Bachelor of
Science degree in mathematics in June, 1974.
John began graduate study at Virginia Polytechnic
Institute and State University in Blacksburg, Virginia, in
September, 1975, and there received a Master of Science
degree in statistics in the summer of 1976. After two years
as a Research Associate at Auburn University in Auburn,
Alabama, he returned to the University of Florida in
September, 1978, where he has since been pursuing a doctoral
degree in statistics. While a graduate student at the
University of Florida, John has worked as a Graduate
Assistant performing statistical consulting duties in the
School of Forest Resources and Conservation.
202

I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
Andre' I. KhurT, Chairman
Assistant Professor of Statistics
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
John A. Cornell, Cochairman
Professor of Statistics
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality, as
a disseration for the degree of Doctor of Philosophy.
x
Richard F~. Fisher
Professor of Forest
and Conservation
Resources
This dissertation was submitted to the Graduate Faculty of
the Department of Statistics in the College of Liberal Arts
and Sciences and to the Graduate Council, and was accepted
as partial fulfillment of the requirements for the degree of
Doctor of Philosophy.
May, 1982
Dean for Graduate Studies
and Research