﻿
 UFDC Home myUFDC Home  |   Help
<%BANNER%>

# A MEMOIR ON MATHEMATICAL AND STATISTICAL CHARACTERISTICS OF NONLINEAR REGRESSION MODELS

## Material Information

Title:
A MEMOIR ON MATHEMATICAL AND STATISTICAL CHARACTERISTICS OF NONLINEAR REGRESSION MODELS
Physical Description:
Memoir
Creator:
Overman, Allen
Publication Date:

## Subjects

Subjects / Keywords:
models
regression
elements of probability

## Notes

Abstract:
This memoir is focused on mathematical and statistical characteristics of nonlinear regression models, and includes a discussion on elements of probability. Particular models are chosen to illustrate various aspects of the procedures. A simple exponential model with two parameters is chosen as the first example. The model is rearranged to a linear form by performing logarithms on the response variable Y. This is referred to as the ‘linearized’ form of the model. Linear regression is then performed on ln Y vs. X (the control variable) to obtain estimates for the exponential parameter b along with the linear correlation coefficient r. Since the correlation coefficient is a measure of system response to the input variable X and reflects scatter in the response data, a decision is then made as to whether the linearized model is adequate or whether nonlinear regression is then needed. The ‘least squares criterion’ is used to determine ‘goodness of fit’ of the model to the data. Second order Newton-Raphson procedure is then selected to minimize the error sum of squares of deviations E between measured and estimated values of the response variable and to obtain optimum estimates of model parameters. In addition standard errors of parameter estimates are calculated using the Hessian matrix for the 2nd derivatives of E with respect to the parameters. This requires the inverse of the Hessian matrix along with the variance of the estimate, from which the standard errors of parameter estimates is obtained. The nonlinear correlation coefficient R is also used as a measure of goodness of fit of the model to the data. Contours of equal probability are then estimated for various levels of uncertainty using the Fisher F statistic. The memoir includes extensive discussion of elements of probability using the binomial expansion first for the natural numbers and then for n equal to fractions and negative values. For the natural numbers the expansion leads to finite series, whereas for fractions and negative values it leads to infinite series. All of this was established by Isaac Newton before he invented the calculus and for which he was appointed the second Lucasian professor of mathematics at Cambridge University, and led to his first memoir On the Analysis of Infinite Series. Coupling between discrete and continuous distributions are illustrated using the simple pegboard for linear, triangular, and rectangular configurations. This approach provides a logical foundation for the continuous Gaussian distribution of mathematical statistics. The procedure is further applied to role of dice for a single, two, three, and four dice. The pegboard is judged to be a simpler procedure to grasp and use in practice.
Acquisition:
Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Allen Overman.
Publication Status:
Unpublished

## Record Information

Source Institution:
University of Florida Institutional Repository
Holding Location:
University of Florida
Rights Management:
System ID:
IR00003500:00001

## Material Information

Title:
A MEMOIR ON MATHEMATICAL AND STATISTICAL CHARACTERISTICS OF NONLINEAR REGRESSION MODELS
Physical Description:
Memoir
Creator:
Overman, Allen
Publication Date:

## Subjects

Subjects / Keywords:
models
regression
elements of probability

## Notes

Abstract:
This memoir is focused on mathematical and statistical characteristics of nonlinear regression models, and includes a discussion on elements of probability. Particular models are chosen to illustrate various aspects of the procedures. A simple exponential model with two parameters is chosen as the first example. The model is rearranged to a linear form by performing logarithms on the response variable Y. This is referred to as the ‘linearized’ form of the model. Linear regression is then performed on ln Y vs. X (the control variable) to obtain estimates for the exponential parameter b along with the linear correlation coefficient r. Since the correlation coefficient is a measure of system response to the input variable X and reflects scatter in the response data, a decision is then made as to whether the linearized model is adequate or whether nonlinear regression is then needed. The ‘least squares criterion’ is used to determine ‘goodness of fit’ of the model to the data. Second order Newton-Raphson procedure is then selected to minimize the error sum of squares of deviations E between measured and estimated values of the response variable and to obtain optimum estimates of model parameters. In addition standard errors of parameter estimates are calculated using the Hessian matrix for the 2nd derivatives of E with respect to the parameters. This requires the inverse of the Hessian matrix along with the variance of the estimate, from which the standard errors of parameter estimates is obtained. The nonlinear correlation coefficient R is also used as a measure of goodness of fit of the model to the data. Contours of equal probability are then estimated for various levels of uncertainty using the Fisher F statistic. The memoir includes extensive discussion of elements of probability using the binomial expansion first for the natural numbers and then for n equal to fractions and negative values. For the natural numbers the expansion leads to finite series, whereas for fractions and negative values it leads to infinite series. All of this was established by Isaac Newton before he invented the calculus and for which he was appointed the second Lucasian professor of mathematics at Cambridge University, and led to his first memoir On the Analysis of Infinite Series. Coupling between discrete and continuous distributions are illustrated using the simple pegboard for linear, triangular, and rectangular configurations. This approach provides a logical foundation for the continuous Gaussian distribution of mathematical statistics. The procedure is further applied to role of dice for a single, two, three, and four dice. The pegboard is judged to be a simpler procedure to grasp and use in practice.
Acquisition:
Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Allen Overman.
Publication Status:
Unpublished

## Record Information

Source Institution:
University of Florida Institutional Repository
Holding Location:
University of Florida
Rights Management:
System ID:
IR00003500:00001

Full Text

PAGE 1

A MEMOIR ON MATHEMATICAL AND STATISTICAL CHARACTERISTICS OF NONLINEAR REGRESSION MODELS Allen R. Overman Agricultural and Biological Engineering University of Florida Copyright 201 3 Allen R. Overman

PAGE 2

Statistics and Nonlinear Regression Allen R. Overman i ABSTRACT This memoir is focused on mathematical and statistical characteristics of nonlinear regression models, and includes a discussion on elements of probability. Particular models are chosen to illustrate various aspects of the procedures. A simple exponential model with two parameters is chosen as the first example. The model is rearranged to a linear form by performing logarithms on the response variable Y This is referred to as the linearized form of the model. Linear regression is then performed on ln Y vs. X ( the control variable ) to obtain estimates for the exponential parameter b along with the linear correlation coefficient r Since the correlation coefficient is a measure of system response to the input variable X and reflects scatter in the response data, a decision is then made as to whether the linearized model is adequate or whether nonlinear regression is then needed. The least squares criterion is used to determine goodness of fit of the model to the data. Second order NewtonRaphson procedure is then s elected to minimize the error sum of squares of deviations E between measured and estimated values of the response variable and to obtain optimum estimates of model parameters. In addition standard errors of parameter estimates are calculated using the Hes sian matrix for the 2nd derivatives of E with respect to the parameters. This requires the inverse of the Hessian matrix along with the variance of the estimate, from which the standard errors of parameter estimates is obtained. The nonlinear correlation c oefficient R is also used as a measure of goodness of fit of the model to the data. Contours of equal probability are then estimated for various levels of uncertainty using the Fisher F statistic. The memoir includes extensive discussion of elements of pr obability using the binomial expansion first for the natural numbers ,3,2,1n and then for n equal to fractions and negative values. For the natural numbers the expansion leads to finite series, whereas for fractions and negative values it l eads to infinite series. All of this was established by Isaac Newton before he invented the calculus and for which he was appointed the second Lucasian professor of mathematics at Cambridge University and led to his first memoir On the Analysis of Infinite Series Coupling between discrete and continuous distributions are illustrated using the simple pegboard for linear, triangular, and rectangular configurations. This approach provides a logical foundation for the continuous Gaussian distribution of mathe matical statistics. The procedure is further applied to role of dice for a single, two, three, and four dice. The pegboard is judged to be a simpler procedure to grasp and use in practice. Keywords : Models, regression, elements of probability. Acknowled gement: The author expresses appreciation to Amy G. Buhler, Associate University Librarian, University of Florida, for assistance with preparation of this memoir as part of the UF digital library.

PAGE 3

Statistics and Nonlinear Regression Allen R. Overman 1 Table of Contents Introduction Mathematical Char acteristics Nonlinear Model Linearized Form of the Model Least Squares Criterion Newton Raphson Procedure for Nonlinear Regression Standard Errors of the Estimates Equal Probability Contours of Parameters Error Sum of Squares Near the Optimum Maxim um Likelihood and Least Squares Analysis Statistical Analysis of the Model Linearized Form of the Model Nonlinear Regression Standard Errors of the Estimates Equal Probability Contours Dependence of E on b Near Minimum E Summary References Tables 1. Dependence of a response variable ( Y ) to a control variable ( X ). 2. Newton Raphson iterations for nonlinear regression of the exponential model. 3. Newton Raphson iterations of the exponential model for initial b = 0.5000. 4. Combinations of A and b to satisfy equal probability equation near minimum E 5. Combinations of A and b to satisfy equal probability equation for 75% probability. 6. Combinations of A and b to satisfy equal probability equation for 95% probability. 7. Combi nations of A and b to satisfy equal probability equation for 99% probability. 8. Correlation of E with b using a parabolic model. Figures 1. Dependence of response variable ( Y ) on control variable ( X ). Data from Table 1. Curve drawn from Eq. ( 35 ). 2. Dependence of ln Y on X Data from Table 1. Line drawn from Eq. (33). 3. Scatter plot for estimated response variable ( Y ) vs. measured response variable ( Y ). Line represents the 45% diagonal. 4. Equal probability contours between parameters A and b. Contours drawn from Table 5 (75%), Table 6 (95%), and Table 7 (99%) probability levels, respectively. Optimum and standard error values of 075.0016.5A and 0129.05161.0b are also shown. 5. Dependence of erro r sum of squares ( E ) on exponential parameter ( b) for linear parameter ( A = 5.016). Parabola drawn from Eq. (67).

PAGE 4

Statistics and Nonlinear Regression Allen R. Overman 2 Introduction Scientific analysis generally involves two essential components: (1) a set of data (measurements or observation s ) and (2) a co nceptual model. The process of drawing inference about the system involves uncertainty in both the data and in the model. In the case of an algebraic mathematical model, regression analysis is used to evaluate the parameters in the model. In regression ana lysis it is common to minimize the sum of squares of deviations between measured and estimated values of the response variable as the criterion of goodness of fit of the model to the data. If all the parameters in the model occur in linear form (such as linear, quadratic, cubic, etc.) then the procedure is called linear regression If one or more of the parameters in the model occur in nonlinear form (such as exponential), then the procedure is called nonlinear regression. Linear regression is the simple r of the two since it involves linear algebra, whereas nonlinear regression involves an iterative procedure to estimate the parameters. Both methods are illustrated in this memoir. A variety of statistical measures are used to describe the quality of a mo del with a particular set of data. The first step is optimization of the model to obtain best estimates of the parameters. The next step is to calculate standard errors of the parameter estimates to determine uncertainty in the parameters. Relative error o f an estimate is then calculated as standard error divided by the estimate. A scatter plot of estimated vs. measured response variable is often included to illustrate scatter of values and any evidence of bias in the model. It is also possible to draw cont ours of equal probability (uncertainty) between two parameters by use of Fishers F statistic. Many of these points have been addressed in a previous publication (Overman et al., 1990) describing cr op response to applied nitrogen with a logistic model. In this document a simple exponential model with one linear and one nonlinear parameter is applied to a set of data. Mathematical and statistical characteristics are discussed in detail to illustrate the various steps involved. Mathematical Characteristics Nonlinear Model Consider the nonlinear regression model bXAYexp (1) where X is the control variable Y is the response variable A is the linear model parameter, and b is the exponential model parameter. For this discussion we conside r X to be positive ( 0X ) and Y to be positive ( 0 Y ). It follows that parameter A must be positive as well. Parameter b can be positive or negative. For positive b it turns out that 0 / dX dY A Y whereas fo r negative b we have 0/,dXdYAY Equation (1) is considered nonlinear in the regression sense because of the exponential parameter b. Linearized Form of the Model Now Eq. (1) can be converted to a linear form by performing logarithms on Y

PAGE 5

Statistics and Nonlinear Regression Allen R. Overman 3 bX a bX A Y Z ln ln (2) where a = ln A Parameters a and b can then be estimated by linear regression of Z vs. X In Eq. (2) ln represents the natural logarithm. In some cases these estimates of parameters may be deemed sufficient for the purpose at hand In other cases a more rigorous procedure may be desired such as nonlinear regression. Least Squares Criterion For regression analysis we begin with a criterion for goodness of fit of the model to the data. Define the error sum of squares of devi ations ( E ) between data and model by n i i iY Y E1 2 (3) where n is the number of observations, Yi is the observed value, and iY is the estimated value from the model. The goal is to choose parameters A and b to minimize E T his is call ed the least squares criterion for goodness of fit of the model to the data. For the exponential model this takes the form niiibXAYE12exp (4) For regression purposes think of E as a function of A and b, say E = E ( A, b ). At the minimum value of E it can be shown from calculus that 0dbbEdAAEdE (5) To minimize E w.r.t. (with respect to) A and b requires that bEAE0 (6) simultaneously. This is called the necessary condition for a minimum. To insure a mini mum, the sufficient condition from calculus is 0 and 02 2 2 2 b E A E (7)

PAGE 6

Statistics and Nonlinear Regression Allen R. Overman 4 The partial derivatives can be obtained from Eq. (4) and are given by the equations bX A bX Y A E 2 exp exp 2 (8) bXAE2exp222 (9) bX X A bX XY A b E b A E 2 exp 2 exp 22 2 (10) bX X A bX XY A b E 2 exp exp 2 (11) bXXAbXYXAbE2exp2exp22222 (12) The subscripts have been omitted for convenience and the cross derivative s have been included for later use in the analysis. The derivative in Eq. (8) can be set to zero, which leads to bXbXYA2expexp (13) Equation (13) gives the optimum estimate of linear parameter A for an assumed value of b. Setting Eq. (11) to zero leads to an implicit equation in parameter b. An iterative procedure is needed to find b which will cause Eq. (11) to vanish. The second order Newton Raphson procedure is chosen for this purpose (Adby and Dempster, 1974) Newton Raphson Procedure for Nonlinear Regression An initial estimate of parameter b is chosen in the neighborhood of minimum E Since we can treat E as a continuous function of b, the derivative at a new value, say bb can be related to the derivative at b by Taylor series expansion 3 4 4 2 3 3 2 2) ( 3 1 ) ( 2 1 b b E b b E b b E b E b Eb b b b b (14) It is implicitly assumed that the series represented by Eq. (14) con verges to a finite value. The strategy is to set this new derivative to zero and to truncate t he series with the linear term in b which leads to b bb E b E b2 2/ / (15)

PAGE 7

Statistics and Nonlinear Regression Allen R. Overman 5 A new estimate of parameter b is obtained from b b b (16) New estimates are then obtained for A bE/ 2 2/ b E b and b The procedure is repeated until the criterion is met bb (17) where is typically chosen as 103 to 105. The final values obtained by this procedure are chosen as optimum b b A A and minimum E E assuming that the procedure converges It i s necessary to choose the initial value of b near the optimum value to insure convergence of the procedure. Convergence requires that the second derivative in Eq. (15) be positive Standard Errors of the Estimates The next step is to calculate the stan dard errors of the estimates of the parameters. This procedure requires calculation of the Hessian matrix [ H ] of the second order derivatives given by AbbbbAAbAAbEAbEbAEAEHHHHH,222222 (18) where the derivatives are evaluated at ( A b ). Since th e cross derivatives are equal, it follows that the Hessian matrix is symmetric The inverse of the Hessian matrix yields the elements 11111bbbAAbAAHHHHH (19) where the inverse Hessian is also symmetric. The variance of the estimate 2 X YS is defined by pnYYSiiXY22 (20) where p is the number of parameters in the model. The standard errors of the estimates are then given by 2 / 1 1 2 AA X YH S A (21)

PAGE 8

Statistics and Nonlinear Regression Allen R. Overman 6 2/112bbXYHSb (22) The covariance of the estimate is give n by 12,COVAbXYHSbA (23) Standard errors of the estimates provide a measure of uncertainty in the parameter estimates for a given model and a particular set of data. Equal Probability Contours of Parameters The next mathematical characteristi c which we explore is equal probability contours of A vs. b around the optimum for a chosen level of uncertainty. Note that minimum error is calculated from n i i iX b A Y b A E1 2exp (24) Now the error at some level of probability, q, is related to E by (Draper and Smith, 1981, p. 472) q p n p F p n p b A E b A E , 1 , (25) w here p is the number of parameters in the model, q is the probability level, and F is taken from tables for Fishers analysis of variance ( F statistic) The goal is to obtain combinations of parameters A and b which satisfy bAEbXAYniii,exp12 = constant (26) This leads to a plot of A vs. b which satisfies Eq. (26), and leads to an equal probability contour. Error Sum of Squares Near the Optimum The final charact eristic which we explore is to examine E vs. b at fixed value of A A For the case of a linear model this result follows a parabola. Does this relationship hold for the nonlinear exponential model? If so, then we should obtain the parabola 2bbE (27) where , are estimated from values of E vs. b near the optimum (minimum)

PAGE 9

Statistics and Nonlinear Regression Allen R. Overman 7 Maximum Likelihood and Least Squares Analysis This section focuses on the connection between the maximum likelihood method of Fisher and that of the least squares criterion (Frieden, 1983, chapter 14 ). The challenge is to calibrate a mathematical model to relate the dependent variable ( y ) to the independent variable ( x ). Assume that the error in the measurements of yi follow s a Gaussian probability density function 2 22 ) ( exp 2 1 ) ( i iy y p (28) with mean of and variance of 2 If we further assume that the error in the predicted values ( iy ) from the model also follow this same error law with the same mean and variance, then the probability density funct ion for the error between measured and estimated y is given by 222)(exp21),(iiiiyyyyp (29) For n observations we can assume the joint probability ( p ) given by the product of individual terms n i i i n i iy y p y y p1 2 2 12 ) ( exp 2 1 ) ( (30) This is referred to as the maximum likelihood principle when the parameters of the model have been chosen to maximize the function given by Eq. (30). Such a choice will also maximize the logar ithm of p n i i i n i i i ny y n y y p1 2 2 1 2 2 2 1 2 ln 2 1 2 1 ln ln (31) In order for ln p to be a maximum, it follows that the error ( E ) defined by n i i iy y E1 2 (32) must be a minimum. Equation (32) therefore defines the least squares error between measured and predicted va lues of y based on the assumptions stated.

PAGE 10

Statistics and Nonlinear Regression Allen R. Overman 8 Statistical Analysis of the Model Linearized Form of the Model In this section the procedure is applied to the particular set of data listed in Table 1. The first step is to plot the data to see the trend a nd scatter (see Figure 1). T he decrease in Y with increase in X appears t o follow an exponential pattern with negative b. T he next step is to plot ln Y vs. X to test this hypothesis (see Figure 2). Since Figure 2 appears to follow a straight line, linear r egression of ln Y vs. X leads to the regression equation X bX a Y Z 5524 0 670 1 ln r = 0.9945 ( 33) with a correlation coefficient of r = 0.9945. This leads to the prediction equation XY5524.0exp31.5 ( 34) It should be noted that Eq. ( 34) does not minimize E for Eq. (1), but instead minimizes the error sum of squares for Z For some purposes Eq. ( 34) may be deemed adequate for analysis. A more rigorous procedure follows nonlinear regression. The value of parameter b = 0.5524 is then use d as a first estimate in the iteration procedure. Nonlinear Regression We now outline the nonlinear regression procedure in detail, as given in Table 2. 5524 0 b 1141.53508.2/0858.122exp/expbXbXYA 4083.35643.11411.53737.81411.522expexp2/bXXAbXXYAbE 7920 115 7548 2 1411 5 2 0640 17 1411 5 2 2 exp 2 exp 2 /2 2 2 2 bX X A bX Y X A b E 0294.07920.115/4083.3///22bEbEb 5230 0 0294 0 5524 0 b b b 5230.0b 0409.54476.2/3381.12A 7008 0 7382 1 0409 5 8316 8 0409 5 2 / b E 5555 160 1844 3 0409 5 2 1792 16 0409 5 2 /2 2 b E 0044 0 5555 160 / 7008 0 b 5186 0 0044 0 5230 0 b

PAGE 11

Statistics and Nonlinear Regression Allen R. Overman 9 5186.0b 0251.54631.2/3773.12A 2619.07666.10251.59034.80251.52/bE 4710 164 2559 3 0251 5 2 3575 16 0251 5 2 /2 2 b E 0016.04710.164/2619.0b 5170.00016.05186.0b 5170 0 b 0194.54687.2/3915.12A 1017 0 7770 1 0194 5 9296 8 0194 5 2 / b E 9168 165 2823 3 0194 5 2 4228 16 0194 5 2 /2 2 b E 0006.09168.165/1017.0b 5164 0 0006 0 5170 0 b 5164.0b 0173 5 4708 2 / 3968 12 A 0380.07810.10173.59396.80173.52/bE 4853 166 2925 3 0173 5 2 4478 16 0173 5 2 /2 2 b E 0002 0 4853 166 / 0380 0 b 5162 0 0002 0 5164 0 b 5162.0b 0165 5 4715 2 / 3984 12 A 0200 0 7822 1 0165 5 9424 8 0165 5 2 / b E 6255 166 2954 3 0165 5 2 4550 16 0165 5 2 /2 2 b E 00012 0 6255 166 / 0200 0 b 5161.00001.05162.0b 5161 0 b 0161.547194.2/39956.12A 00645.078305.10161.594460.80161.52/bE 7581.16629766.30161.5246050.160161.52/22bE 000039 0 7581 166 / 00645 0 b 5161.00000.05161.0b 410 000076 0 5161 0 000039 0 b b

PAGE 12

Statistics and Nonlinear Regression Allen R. Overman 10 The iterations are terminated at this point and lead to the estimation equatio n X Y 5161 0 exp 016 5 ( 35) Note that the procedure converged to the final values in seven steps. The regression curve i n Figure 1 is drawn from Eq. ( 35). A scatter plot of Y vs. Y is shown in Figure 3. The question now arises as to convergence if the first estimate is greater than the true value, say b = 0.5000. The steps are outlined below for this case and are summarized in Table 3. 5000 0 b 9566.45311.2/5457.12A 4253.08937.19566.43435.99566.42/bE 9699.1815804.39566.421369.179566.42/22bE 0023 0 9699 181 / 4253 0 b 5023.00023.05000.0b 5023 0 b 9653.45224.2/5244.12A 4465.18733.19653.41757.99653.4/bE 5627 179 5381 3 9653 4 2 0537 17 9653 4 2 /2 2 b E 0081 0 5627 179 / 4465 1 b 5104.00081.05023.0b 5104.0b 9951 4 4927 2 / 4513 12 A 5872.08215.19951.40398.99951.4/bE 0235 172 3950 3 9951 4 2 6975 16 9951 4 2 /2 2 b E 0034 0 0235 172 / 5872 0 b 5138 0 0034 0 5104 0 b 5138 0 b 0077.54802.2/4201.12A 2348 0 7985 1 0077 5 9829 8 00771 5 / b E 8986 168 3369 3 0077 5 2 5565 16 0077 5 2 /2 2 b E 0014 0 8986 168 / 2348 0 b 5152.00014.05138.0b

PAGE 13

Statistics and Nonlinear Regression Allen R. Overman 11 5152.0b 0130.54751.2/4077.12A 0908 0 7891 1 0130 5 9597 8 0130 5 / b E 6221 167 3131 3 0130 5 2 4984 16 0130 5 2 /2 2 b E 0005.06221.167/0908.0b 5157.00005.05152.0b 5157 0 b 0147.54734.2/4033.12A 0376.07858.10147.59515.80147.5/bE 1526 167 3047 3 0147 5 2 4779 16 0147 5 2 /2 2 b E 0002.01526.167/0376.0b 5159 0 0002 0 5157 0 b 5159.0b 0154.54726.2/4012.12A 0118.07843.10154.59478.80154.5/bE 9252 166 3008 3 0154 5 2 4684 16 0154 5 2 /2 2 b E 00007.09252.166/0118.0b 5160 0 00007 0 5159 0 b 5160.0b 0157.54724.2/4007.12A 00744 0 7839 1 0157 5 9468 8 0157 5 / b E 8813 166 2999 3 0157 5 2 4659 16 0157 5 2 /2 2 b E 000045.08813.166/00744.0b 5161.0000045.05160.0b 410 000087 0 5160 0 000045 0 b b The proc edure again converges to Eq. ( 35 ).

PAGE 14

Statistics and Nonlinear Regression Allen R. Overman 12 Another measure of quality of fit of the model to the data is given by the nonlinear correlation coefficient defined by (Cornell and Berger, 1987) 99674.0396364.23152098.01(12/12/122YYYYRiii ( 36) which shows excellent agreement between model and data The final derivatives are given by 000012.047194.20161.539956.1222expexp2bXAbXYAE ( 37) 0 94388 4 47194 2 2 2 exp 22 2 bX A E ( 38) 0 8866 17 78305 1 0161 5 2 9446 8 2 2 exp 2 exp 22 2 bX X A bX XY A b E b A E ( 39) 0 00645 0 78305 1 0161 5 94460 8 0161 5 2 2 exp exp 2 bX X A bX XY A b E ( 40) 07581.16629766.30161.5246050.160161.522exp2exp22222bXXAbXYXAbE ( 41) Note that the first derivatives are approximately zero and the second derivatives are positive as required. Standard Errors of the Estimates The second derivatives allow calculation of the Hessian matrix 7581 166 8866 17 8866 17 94388 42 2 2 2 2 2b E A b E b A E A E H H H H Hbb bA Ab AA ( 42)

PAGE 15

Statistics and Nonlinear Regression Allen R. Overman 13 It follows that the inverse Hessian matrix becomes 0097995332.040354540022.040354540022.073305402955.011111bbbAAbAAHHHHH ( 43) which is symmetric as required. The variance of the estimate is calculated from 016900.0211152098.022pnYYSiiXY ( 44) It follows that the standard errors of the estimates and the covariance become 0747.073305402955.0016900.02/12/112AAXYHSA ( 45) 0129 0 0097995332 0 016900 02 / 1 2 / 1 1 2 bb X YH S b ( 46) 000599.040354540022.0016900.0,COV12AbXYHSbA ( 47) Under ideal circumstances the covariance would be zero to signify that the model parameters were uncorr el ated Final estimates o f parameters are 075.0016.5A ( 48) 0129 0 5161 0 b ( 49) with relative errors of %49.10149.0016.50747.0AA ( 50) %50.20250.05161.00129.0bb ( 51) which are relatively small as desired. A check of the Hessian inverse shows that 100159999999808.000000000002.02060000000000.099999999999.00097995332.040354540022.040354540022.073305402955.07581.1668866.178866.1794388.41HH within roundoff as required. We also note that the determinant of the Hessian

PAGE 16

Statistics and Nonlinear Regression Allen R. Overman 14 0 501575868 504 930459560 319 432035428 824 7581 166 8866 17 8866 17 94388 4 is positive definite as required for convergence. Equal Probability C ontours In this section we examine combinations of parameters A and b which lead to equal values of the error sum of squares E The optimized model is described by XXbAY5161.0exp016.5exp ( 52) which leads to the minimum error sum of squares of niiiiiiXYYYE1111221521.05161.0exp016.5 ( 53) Values of E are calculated for 5161.0,075.0016.5bA and for 0129.05161.0,016.5bA Results for these values are listed in Table 4. Other combinations of A and b which lead to the same E are also given. These results provide a contour of equal probabilities which pass through the standard errors for A and b. From the table we note that for A = 5.016 we obtain ( b, E ) = ( 0.5032, 0.1666) and ( b, E ) = ( 0.5290, 0.1652). This analysis is now extended to various levels of probability. 75 % probability contour It can be shown that the error at some level of probability, say 75%, can be calculated from 1112exp2069.062.1211211521.0%75,,1,iiibXAYpnpFpnpEbAE ( 54) where the value of F is obtained from statistical tables as F (2,9,75%) = 1.62. Combinations of A and b which satisfy Eq. ( 54) are listed in Table 5. A graph of the pr obability contour is shown in Figure 4, from which estimates are made of ( b, E ) = ( 0.491, 0.2069) and ( b, E ) = ( 0.543, 0.2069).

PAGE 17

Statistics and Nonlinear Regression Allen R. Overman 15 95 % probability contour It can be shown that the error at the level of pro bability of 95% can be calculated from 11 1 2exp 2961 0 26 4 2 11 2 1 1521 0 % 95 , 1 ,i i ibX A Y p n p F p n p E b A E ( 55) where the value of F is obtained from statistical tables as F (2,9,95%) = 4.26. Combinations of A and b which satisfy Eq ( 55) are listed in Table 6 and shown in Figure 4. Estimates are made of ( b, E ) = ( 0.477, 0.2961) and ( b, E ) = ( 0.560, 0.2961). 99 % probability contour It can be shown that the error at the level of pro bability of 99% can be calculated from 11 1 2exp 4232 0 02 8 2 11 2 1 1521 0 % 99 , 1 ,i i ibX A Y p n p F p n p E b A E ( 56) where the value of F is obtained from statistical tables as F (2,9,99%) = 8.02. Combinati ons of A and b which satisfy Eq. ( 56) are listed in Table 7 and shown in Figure 4. Estimates are made of ( b, E ) = ( 0.466, 0.4232 ) and ( b, E ) = ( 0.577, 0.4232). Dependence of E on b Near M inimum E A summary of E vs. b for 016.5AA whi ch satisfy various probability levels is given Table 8. The question is whether or not this relationship follows a parabola given by 2bbE ( 57) To optimize the parabolic model requires that 2 4 3 2 3 2 2Eb Eb E b b b b b b b b n ( 58) where again n is the number of observations used in the analysis. Calculations are carried out for different numbers of points listed in Table 8.

PAGE 18

Statistics and Nonlinear Regression Allen R. Overman 16 n = 9 4 6312315359 0 21038451 1 3363 2 9 6648422154 0 8 2660996762 1 42264121 2 8 2660996762 1 42264121 2 6581 4 42264121 2 6581 4 9 ( 59) which leads to the regression equation 25133 75 3473 78 4827 20 b b E ( 60) The minimum of the parabola occurs at 1608.0,5188.005133.7523473.78EbbbE ( 61) which is inconsistent with 1521 0 5161 0 E b Correlation between E and E is given by E E 9166 0 0216 0 r = 0.99837 ( 62) n = 7 44000958010.077084361.04902.125084421553.089753885152.087626821.189753885152.087626821.16191.387626821.16191.37 ( 63) whi ch leads to the regression equation 25410 83 5400 86 5631 22 b b E ( 64) The minimum of the parabola occurs at 1514 0 5179 0 0 5410 83 2 5400 86 E b b b E ( 65) which is inconsistent with 1521 0 5161 0 E b Correlation between E and E is given by E E 9958 0 00088 0 r = 0.99778 ( 66) While this is better than using all nine values, it is still off a bit.

PAGE 19

Statistics and Nonlinear Regression Allen R. Overman 17 n = 5 4 2398675041 0 46378791 0 8980 0 8 3583277494 0 8 6912411822 0 33513921 1 8 6912411822 0 33513921 1 5821 2 33513921 1 5821 2 5 ( 67) which leads to the regression equation 27507.804693.837221.21bbE ( 68) The minimum of the parabola occurs at 1522.0,5168.007507.8024693.83EbbbE ( 69) which is near 1521.0,5161.0Eb Correlation between E and E is given by EE99722.000046.0 r = 0.99905 ( 70) n = 3 4 1289835871 0 24985331 0 4842 0 1 2132717681 0 8 4127674042 0 79920921 0 8 4127674042 0 79920921 0 5481 1 79920921 5481 1 3 ( 71) which leads to the regression equation 25409 82 2091 85 1430 22 b b E ( 72) The minimum of the parabola occurs at 1520.0,5162.005409.8222091.85EbbbE ( 73) which virtually agrees with 1521 0 5161 0 E b Correlation between E and E is given by E E 00000 1 00010 0 r = 1 ( 74) Values of E vs. b are shown in Figure 5, where th e parabola is drawn from Eq. ( 72). As b is changed away from 5161.0b the values deviate from the parabola. Note for b = 0.5524 that E is within the parabolic envelope. For b = 0.5000 note that E is also within the parabolic

PAGE 20

Statistics and Nonlinear Regression Allen R. Overman 18 envelope. In both cases the procedure converges toward the minimum 1521 0 E at 5161.0b Note that it is important to carry a large number of digits to avoid roundoff errors in the m atrix computational procedure. Summary This memoir has focused on the mathematical and statistical characteristics of a nonlinear regression model. The model assumed an exponential relationship between the control variable ( X ) and the response variable ( Y ) as described by Eq. (1). Analysis was performed for a given set of data (Table 1). The first step was to linearize the model to the form of Eq. (2). A plot of the data supported this step as shown in Figure 2. Linear regression of Eq. (2) led to a firs t estimate of the parameters A and b. This estimate of b was then used to perform nonlinear regression of the model on the data to optimize the values of A and b in order to minimize the error sum of squares ( E ) between measured and estimated values of Y It was shown that the Newton Raphson procedure converged rapidly to the minimum E Standard errors of the parameters were then estimated which showed low relative errors in the parameters. A further measure of uncertainty in the parameters was illustra ted by the equal probability contours for A vs. b for various levels of probability (see Figure 4). The cross in Figure 4 represents the most probable values of parameters A and b for the exponential model and for this particular set of data, i.e. the valu es which minimize the error sum of squares between measured and predicted values of response variable y Vertical bars represent the standard error in parameter b around optimum value b while horizontal bars represent the standard er ror in parameter A around optimum value A These are equivalent to the standard deviation around the mean of a set of measurements which follow a Gaussian distribution. The contours in Figure 4 represent combinations of parameters A a nd b which produce various levels of uncertainty Following Fishers maximum likelihood method, these represents combinations of equal probability. It was further shown that E vs. b at optimum A in the neighborhood of minimum E followed parabolic depend ence. At this point it seems appropriate to call attention to several general points about data analysis and mathematical models. R. A. Fisher called attention to two elements of uncertainty in this process in his classic article of 1922 (see Bennet, 1971). Uncertainty in data led to his analysis of variance (ANOVA) procedure, while uncertainty in a model led to a subject called Fisher Information (see Frieden, 1998). In her biography of her father Joan Fisher Box noted that the passion of Fi shers life was the subject of inference (see Fisher Box, 1978, p. 447) i.e. drawing inference about a system from analysis of a specific set of data. Scientific research can be divided into two approaches: bottom/up and top/down. In the bottom/up approach data (me asurements or observations) are examined in order to identify a unifying theory or model (from specific to general), which is commonly referred to as a process of induction. In the top/down approach a general principle is postulated and the consequences of these are developed (from general to specific), which is commonly referred to as the process of deduction. Most research appears to have followed the bottom/up approach. The top/down approach was championed by Einstein and by Paul Dirac (Farmelo, 2009, p. 2, 94, 382). Following a series of lectures by Murray Gell Mann it appears that Dirac gained increased respect for the bottom/up approach, which Gell Mann had followed. The work described in this

PAGE 21

Statistics and Nonlinear Regression Allen R. Overman 19 memoir has followed the bottom/up approach. Of course we cant be certain that the simple exponential model is the very best model possible for the given data. There is always a level of uncertainty. Science progresses by assuming a theory or model and then checking the consequences of the theory through measureme nts. A final point has to do with pursuit of knowledge and understanding of how nature really works. I will call this the battle between subjective and objective criteria for judging the values of ideas in science. According to James Glanz (see Chang, 2000, p. 354) the theore tical physicist Steven Weinberg has battled with thinkers and philosophers of science over this issue. Today, one of his major battles is with postmodernist thinkers and philosophers of science who maintain that scientific theories reflect not objective reality but social negotiations among scientists. In its rawest form, this philosophy would say that the theories of the most persuasive or political ly powerful scientists become accepted fact. I was trained on the belief in objective criteria, and I still hold to this view. Otherwise, it becomes a battle for power and control of ideas in science based on personalities Unfortunately I have observed an increasing trend to cite experts as the source of truth in the ev aluation of sc ientific ideas. Some e ditors and reviewers seem to find this an attractive alternative in the peer review process. References Adby, P.R. and M.A.H. Dempster. 1974. 1974. Introduction to Optimization Methods. John Wiley & Sons. New York, NY. Bennett, J.H 1971. Collected Papers of R.A. Fisher. Vol 1 ( 19121924). University of Adelaide. Chang, L. 2000. Scientists at Work: Profiles of Todays Groundbreaking Scientists from Science Times. McGraw Hill. New York, NY. Cornell, J.A. and R.D. Berger. 1987. Fac tors that influence the value of the coefficient of determination in simple linear and nonlinear regression models. Phytopathology 77:6370. Draper, N.R. and H. Smith. 1981. Applied Regression Analysis John Wiley & Sons. New York, NY. Farmelo, G. 2009. Th e Strangest Man: The Hidden Life of Paul Dirac, Mystic of the Atom. Basic Books. New York, NY. Fisher Box, J. 1978. R.A. Fisher: The Life of a Scientist John Wiley & Sons. New York, NY. Frieden, B.R. 1983. Probability, Statistical Optics, and Data Testing. Springer Verlag. New York, NY. Frieden, B.R. 1998. Physics from Fisher Information: A Unification. Cambridge University Press. New York, NY. Overman, A.R., F.G. Martin, and S.R. Wilkinson. 1990. A logistic equation for yield response of forage grass to n itrogen. Commun. Soil Science and Plant Analysis 21:595609.

PAGE 22

Statistics and Nonlinear Regression Allen R. Overman 20 Table 1. Dependence of a response variable ( Y ) on a control variable ( X ). X Y ln Y 0.0 5.0 1.609 0.5 4.0 1.386 1.0 2.8 1.030 1.5 2.2 0.788 2.0 2.0 0.693 2.5 1.5 0.405 3.0 1.0 0.000 3.5 0.9 0.105 4.0 0.6 0.511 4.5 0.4 0.916 5.0 0.3 1.204 Table 2 Newton Raphson ite rations of the exponential model for initial b = 0.5524. X Y exp ( bX ) Y 0.0 5.0 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.00000 5.016 0.5 4.0 0.7587 0.7699 0.7716 0.7722 0.7724 0.7725 0.77256 3.875 1.0 2.8 0.5756 0.5927 0.5954 0.5963 0.5967 0.5968 0.59684 2.994 1.5 2.2 0.4367 0.4564 0.4594 0.4605 0.4609 0.4610 0.46110 2.313 2.0 2.0 0.3313 0.3513 0.3544 0.3556 0.3560 0.3561 0.35622 1.787 2.5 1.5 0.2513 0.2705 0.2735 0.2746 0.2750 0.2751 0.27520 1.380 3.0 1.0 0.1907 0.2082 0.2110 0.2120 0.2124 0.2125 0.21261 1.066 3.5 0.9 0.1447 0.1603 0.1628 0.1637 0.1641 0.1642 0.16425 0.824 4.0 0.6 0.1097 0.1234 0.1256 0.1264 0.1267 0.1268 0.12689 0.637 4.5 0.4 0.0833 0.0950 0.0969 0.0976 0.0979 0.0980 0.09803 0.492 5.0 0.3 0.0632 0.0732 0.0748 0.0754 0.0756 0.0757 0.07574 0.380 b 0.5524 0.5230 0.5186 0.5170 0.5164 0.5162 0.5161 0.5161 A 5.1411 5.0409 5.0251 5.0194 5.0173 5.0165 5.0161 5.0161 b E / 3.4083 0.7008 0.2619 0.1017 0.0380 0.0200 0.00645 22/bE 115.7920 160.5555 164.4710 165.9168 166.4853 166 .6255 166.7581 b 0.0294 0.0044 0.0016 0.0006 0.0002 0.0001 0.00004 b 0.5230 0.5186 0.5170 0.5164 0.5162 0.5161 0.51614

PAGE 23

Statistics and Nonlinear Regression Allen R. Overman 21 Table 3 Newton Raphson iterations of the exponential model for initial b = 0.5000. X Y exp ( bX ) 0.0 5.0 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0.5 4.0 0.7788 0.7779 0.7748 0.7734 0.7729 0.7727 0.7726 1.0 2.8 0.6065 0.6051 0.6003 0.5982 0.5974 0.5971 0.5970 1.5 2.2 0.4724 0.4707 0.4651 0.4627 0.4617 0.4614 0.4612 2.0 2.0 0.3679 0.3662 0.3603 0.3579 0.3569 0.3565 0.3564 2.5 1.5 0.2865 0.2849 0.2792 0.2768 0.2758 0.2755 0.2753 3.0 1.0 0.2231 0.2216 0.2163 0.2141 0.2132 0.2129 0.2127 3.5 0.9 0.1738 0.1724 0.1676 0.1656 0.1648 0.1645 0.1644 4.0 0.6 0.1353 0.1341 0.1298 0.1281 0.1274 0.1271 0.1270 4.5 0.4 0.1054 0.1043 0.1006 0.0991 0.0984 0.0982 0.0981 5.0 0.3 0.0821 0.0811 0.0779 0.0766 0.0761 0.0759 0.0758 b 0.5000 0.5023 0.5104 0.5138 0.5152 0.5157 0.5159 A 4.9566 4.9653 4.9951 5.0077 5.0130 5.0147 5.0154 b E / +0.4253 +1.4465 +0.5872 +0.2348 +0.0908 +0.0376 +0.0118 22/bE 181.9699 179.5627 172.0235 168.8986 167.6221 167.1526 166.9252 b 0.0023 0.0081 0.0034 0.0014 0.0005 0.0002 0.00007 b 0.5023 0.5104 0.5138 0.5152 0.5157 0.5159 0.5160 Table 3 ( Continued ) X Y exp ( bX ) Y YY 0.0 5.0 1.000000 1.000000 1.000000 1.000000 1.0000000 5.016 0.016 0.5 4.0 0.772607 0.772589 0.772583 0.772580 0.7725794 3.875 +0.125 1.0 2.8 0.596921 0.596894 0.596884 0.596880 0.5968789 2.994 0.194 1.5 2.2 0.461185 0.461154 0.461143 0.461138 0.4611364 2.313 0.113 2.0 2.0 0.356315 0.356283 0.356271 0.356266 0.3562644 1.787 +0.213 2.5 1.5 0.275291 0.275260 0.275249 0.275244 0.2752426 1.381 +0.119 3.0 1.0 0.212692 0.212663 0.212652 0.212648 0.2126467 1.067 0.067 3.5 0.9 0.164327 0.164301 0.164292 0.164288 0.1642865 0.824 +0.076 4.0 0.6 0.126960 0.126938 0.126929 0.126925 0.1269244 0.637 0.037 4.5 0.4 0.098090 0.098071 0.098063 0.098060 0.0980591 0.492 0.092 5.0 0.3 0.075785 0.075768 0.075762 0.075759 0.0757585 0.380 0.080 b 0.51597 0.516015 0.516032 0.516039 0.516041 0.5160 A 5.01566 5.01582 5.01588 5.01591 5.0159173 5.0160 bE/ +0.007444 +0.002792 +0.001101 +0.000301 +0.000256 22/bE 166.881252 166.840459 166.825228 166.819155 166.817429 b 0.000045 0.000017 0.000007 0.000002 0.0000015 b 0.516015 0.516032 0.516039 0.516041 0.516042

PAGE 24

Statistics and Nonlinear Regression Allen R. Overman 22 Table 4. Combinations of A and b to satisfy equal probability equation near minimum E X Y Y 0.0 5.0 4.941 5.091 5.016 5.016 4.970 5.110 5.060 4.920 0.5 4.0 3.817 3.933 3.900 3.850 3.828 3.936 3.921 3.813 1.0 2.8 2.950 3.039 3.033 2.955 2.949 3.032 3.039 2.954 1.5 2.2 2.278 2.347 2.358 2.269 2.271 2.335 2.355 2.289 2.0 2.0 1.760 1.814 1.834 1.741 1.750 1.799 1.825 1.774 2.5 1.5 1.360 1.401 1.426 1.337 1.348 1.386 1.414 1.375 3.0 1.0 1.051 1.082 1.109 1.026 1.038 1.067 1.096 1.065 3.5 0.9 0.812 0.836 0.862 0.788 0.800 0.822 0.849 0.826 4.0 0.6 0.627 0.646 0.670 0.604 0.616 0.633 0.658 0.640 4.5 0.4 0.484 0.499 0.521 0.464 0.474 0.488 0.510 0.496 5.0 0.3 0.374 0.386 0.405 0.356 0.365 0.376 0.395 0.384 b 0.5161 0.5161 0.5032 0.5290 0.522 0.522 0.510 0.510 A 4.941 5.091 5.016 5.016 4.97 5.11 5.06 4.92 E 0.1660 0.1665 0.1664 0.1657 0.1647 0.1668 0.1653 0.1673 Target is E = 0.1660 Table 4. (Continued). X Y Y 0.0 5.0 5.110 4.930 5.017 5.017 4.970 5.050 5.080 0.5 4.0 3.922 3.833 3.901 3.851 3.871 3.871 3.894 1.0 2.8 3.011 2.981 3.033 2.956 3.014 2.967 2.984 1.5 2.2 2.311 2.318 2.359 2.269 2.348 2.274 2.287 2.0 2.0 1.774 1.802 1.834 1.742 1.828 1.743 1.753 2.5 1.5 1.362 1.401 1.426 1.337 1.424 1.336 1.344 3.0 1.0 1.045 1.090 1.109 1.026 1.109 1.024 1.030 3.5 0.9 0.802 0.847 0.862 0.788 0.864 0.785 0.789 4.0 0.6 0.616 0.659 0.670 0.605 0.673 0.601 0.605 4.5 0.4 0.473 0.512 0.521 0.464 0.524 0.461 0.464 5.0 0.3 0.363 0.398 0.405 0.356 0.408 0.353 0.355 b 0.5290 0.5032 0.5032 0.5290 0.5000 0.5320 0.5320 A 5.11 4.93 5.017 5.017 4.97 5.05 5.08 E 0.1663 0.1650 0.1666 0.1652 0.1662 0.1658 0.1648 Target is E = 0.1660

PAGE 25

Statistics and Nonlinear Regression Allen R. Overman 23 Table 5. Combinations of A and b to satisfy equal probability equation for 75% probability. X Y Y 0.0 5.0 5.160 4.870 5.210 4.960 5.080 4.830 5.140 5.130 0.5 4.0 3.986 3.762 3.987 3.796 3.956 3.762 3.904 3.897 1.0 2.8 3.080 2.907 3.051 2.905 3.081 2.930 2.966 2.960 1.5 2.2 2.379 2.246 2.335 2.223 2.400 2.282 2.253 2.248 2.0 2.0 1.838 1.735 1.787 1.701 1.869 1.777 1.711 1.708 2.5 1.5 1.420 1.340 1.368 1.302 1.455 1.384 1.300 1.297 3.0 1.0 1.097 1.035 1.047 0.996 1.134 1.078 0.987 0.985 3.5 0.9 0.848 0.800 0.801 0.763 0.883 0.839 0.750 0.748 4.0 0.6 0.655 0.618 0.613 0.584 0.688 0.654 0.570 0.568 4.5 0.4 0.506 0.477 0.469 0.447 0.535 0.509 0.433 0.432 5.0 0.3 0.391 0.369 0.359 0.342 0.417 0.396 0.329 0.328 b 0.5161 0.5161 0.535 0.535 0.500 0.500 0.550 0.550 A 5.16 4.87 5.21 4.96 5.08 4.83 5.14 5.13 E 0.2035 0.2052 0.2087 0.2064 0.2044 0.2062 0.2082 0.2080 E = 0.2069 target Table 5. (Continued). X Y Y 0.0 5.0 5.000 4.840 4.920 4.960 4.860 5.050 4.830 4.910 0.5 4.0 3.914 3.788 3.861 3.796 3.813 3.943 3.771 3.855 1.0 2.8 3.063 2.965 3.029 2.905 2.992 3.078 2.944 3.026 1.5 2.2 2.398 2.321 2.377 2.223 2.348 2.403 2.299 2.376 2.0 2.0 1.877 1.817 1.865 1.701 1.842 1.876 1.795 1.865 2.5 1.5 1.469 1.422 1.463 1.302 1.446 1.465 1.401 1.464 3.0 1.0 1.150 1.113 1.148 0.996 1.134 1.144 1.094 1.149 3.5 0.9 0.900 0.871 0.901 0.763 0.890 0.893 0.854 0.902 4.0 0.6 0.704 0.682 0.707 0.584 0.698 0.697 0.667 0.708 4.5 0.4 0.551 0.534 0.555 0.447 0.548 0.544 0.521 0.556 5.0 0.3 0.431 0.418 0.435 0.342 0.430 0.425 0.407 0.437 b 0.490 0.490 0.485 0.485 0.485 0.495 0.495 0.484 A 5.00 4.84 4.92 4.96 4.86 5.05 4.83 4.91 E 0.2051 0.2042 0.2047 0.2064 0.2077 0.2074 0.2052 0.2077 E = 0.2069 target

PAGE 26

Statistics and Nonlinear Regression Allen R. Overman 24 Table 5. (Continued). X Y Y 0.0 5.0 4.840 5.130 4.900 5.190 4.990 5.210 5.040 5.190 0.5 4.0 3.754 3.979 3.769 3.992 3.809 3.977 3.838 3.952 1.0 2.8 2.912 3.087 2.899 3.070 2.908 3.036 2.922 3.009 1.5 2.2 2.259 2.394 2.229 2.361 2.220 2.318 2.225 2.292 2.0 2.0 1.752 1.857 1.715 1.816 1.695 1.769 1.695 1.745 2.5 1.5 1.359 1.441 1.319 1.397 1.294 1.351 1.290 1.329 3.0 1.0 1.054 1.118 1.014 1.074 0.988 1.031 0.983 1.012 3.5 0.9 0.818 0.867 0.780 0.826 0.754 0.787 0.748 0.770 4.0 0.6 0.634 0.672 0.600 0.636 0.575 0.601 0.570 0.587 4.5 0.4 0.492 0.522 0.462 0.489 0.439 0.459 0.434 0.447 5.0 0.3 0.382 0.405 0.355 0.376 0.335 0.350 0.330 0.340 b 0.508 0.508 0.525 0.525 0.540 0.540 0.545 0.545 A 4.84 5.13 4.90 5.19 4.99 5.21 5.04 5.19 E 0.2095 0.2074 0.2095 0.2054 0.2089 0.2095 0.2068 0.2058 E = 0.2069 target Table 5. (Continued). X Y Y 0.0 5.0 4.930 5.200 5.080 5.170 5.016 5.016 5.016 0.5 4.0 3.782 3.989 3.862 3.931 3.924 3.823 3.922 1.0 2.8 2.902 3.061 2.937 2.989 3.070 2.914 3.067 1.5 2.2 2.226 2.348 2.233 2.272 2.402 2.221 2.398 2.0 2.0 1.708 1.802 1.698 1.728 1.879 1.693 1.875 2.5 1.5 1.310 1.382 1.291 1.314 1.470 1.291 1.466 3.0 1.0 1.005 1.060 0.981 0.999 1.150 0.984 1.146 3.5 0.9 0.771 0.814 0.746 0.759 0.900 0.750 0.896 4.0 0.6 0.592 0.624 0.567 0.577 0.704 0.572 0.701 4.5 0.4 0.454 0.479 0.431 0.439 0.551 0.436 0.548 5.0 0.3 0.348 0.367 0.328 0.334 0.431 0.332 0.429 b 0.530 0.530 0.548 0.548 0.491 0.543 0.492 A 4.93 5.20 5.08 5.17 5.016 5.016 5.016 E 0.2068 0.2056 0.2071 0.2062 0.2086 0.2220 0.2037 E = 0.2069 target

PAGE 27

Statistics and Nonlinear Regression Allen R. Overman 25 Table 6. Combinations of A and b to satisfy equal probability equation for 95% probability. X Y Y 0.0 5.0 5.260 4.780 5.310 4.860 5.170 4.730 5.320 4.980 0.5 4.0 4.064 3.693 4.064 3.719 4.026 3.684 4.031 3.773 1.0 2.8 3.139 2.853 3.110 2.846 3.136 2.869 3.054 2.859 1.5 2.2 2.425 2.204 2.380 2.178 2.442 2.234 2.314 2.166 2.0 2.0 1.874 1.703 1.821 1.667 1.902 1.740 1.753 1.641 2.5 1.5 1.448 1.315 1.394 1.276 1.481 1.355 1.328 1.243 3.0 1.0 1.118 1.016 1.067 0.976 1.154 1.055 1.006 0.942 3.5 0.9 0.864 0.785 0.816 0.747 0.898 0.822 0.763 0.714 4.0 0.6 0.667 0.607 0.625 0.572 0.700 0.640 0.578 0.541 4.5 0.4 0.516 0.469 0.478 0.438 0.545 0.499 0.438 0.410 5.0 0.3 0.398 0.362 0.366 0.335 0.424 0.388 0.332 0.310 b 0.5161 0.5161 0.535 0.535 0.500 0.500 0.555 0.555 A 5.26 4.78 5.31 4.86 5.17 4.73 5.31 4.98 E 0.2986 0.2900 0.2946 0.2897 0.2949 0.2956 0.2932 0.2931 E = 0.2961 target Table 6. (Continued). X Y Y 0.0 5.0 5.120 4.710 5.050 4.710 4.900 5.070 5.290 5.320 4.910 0.5 4.0 4.007 3.687 3.972 3.705 3.874 3.822 3.988 4.051 3.739 1.0 2.8 3.137 2.885 3.125 2.914 3.062 2.882 3.007 3.085 2.847 1.5 2.2 2.455 2.258 2.458 2.293 2.421 2.172 2.267 2.349 2.168 2.0 2.0 1.922 1.768 1.934 1.803 1.914 1.638 1.709 1.789 1.651 2.5 1.5 1.504 1.384 1.521 1.419 1.513 1.235 1.288 1.362 1.257 3.0 1.0 1.177 1.083 1.196 1.116 1.196 0.931 0.971 1.037 0.957 3.5 0.9 0.921 0.848 0.941 0.878 0.946 0.702 0.732 0.790 0.729 4.0 0.6 0.721 0.663 0.740 0.691 0.748 0.529 0.552 0.601 0.555 4.5 0.4 0.564 0.519 0.582 0.543 0.591 0.399 0.416 0.458 0.423 5.0 0.3 0.442 0.406 0.458 0.427 0.467 0.301 0.314 0.349 0.322 b 0.490 0.490 0.480 0.480 0.470 0.565 0.565 0.545 0.545 A 5.12 4.71 5.05 4.71 4.90 5.07 5.29 5.32 4.91 E 0.2926 0.2989 0.2981 0.2987 0.2921 0.2944 0.2930 0.2912 0.2944 E = 0.2961 target

PAGE 28

Statistics and Nonlinear Regression Allen R. Overman 26 Table 6. (Continued). X Y Y 0.0 5.0 4.720 5.000 4.740 5.140 5.250 4.860 4.770 5.200 5.200 0.5 4.0 3.722 3.943 3.747 3.865 3.948 3.846 3.775 3.907 3.908 1.0 2.8 2.935 3.109 2.963 2.907 2.969 3.044 2.987 2.935 2.936 1.5 2.2 2.315 2.452 2.342 2.186 2.233 2.409 2.364 2.205 2.207 2.0 2.0 1.825 1.934 1.852 1.644 1.679 1.906 1.871 1.656 1.658 2.5 1.5 1.440 1.525 1.464 1.236 1.263 1.508 1.480 1.244 1.246 3.0 1.0 1.135 1.203 1.157 0.930 0.950 1.194 1.172 0.935 0.936 3.5 0.9 0.895 0.948 0.915 0.699 0.714 0.945 0.927 0.702 0.704 4.0 0.6 0.706 0.748 0.723 0.526 0.537 0.748 0.734 0.528 0.529 4.5 0.4 0.557 0.590 0.572 0.395 0.404 0.592 0.581 0.396 0.397 5.0 0.3 0.439 0.465 0.452 0.297 0.304 0.468 0.459 0.298 0.299 b 0.475 0.475 0.470 0.570 0.570 0.468 0.468 0.572 0.5715 A 4.72 5.00 4.74 5.14 5.25 4.86 4.77 5.20 5.20 E 0.2948 0.2960 0.2942 0.2967 0.2952 0.2821 0.2887 0.2994 0.2960 E = 0.2961 target Table 6. (Continued). X Y Y 0.0 5.0 5.190 5.180 5.170 4.800 4.810 4.910 5.320 4.810 5.290 0.5 4.0 3.909 3.901 3.894 3.799 3.806 3.739 4.051 3.700 4.069 1.0 2.8 2.944 2.938 2.933 3.006 3.012 2.847 3.085 2.845 3.129 1.5 2.2 2.217 2.213 2.209 2.379 2.384 2.168 2.349 2.188 2.407 2.0 2.0 1.670 1.667 1.663 1.883 1.886 1.651 1.789 1.683 1.851 2.5 1.5 1.258 1.255 1.253 1.490 1.493 1.257 1.362 1.295 1.424 3.0 1.0 0.947 0.945 0.944 1.179 1.181 0.957 1.037 0.996 1.095 3.5 0.9 0.713 0.712 0.711 0.933 0.935 0.729 0.790 0.766 0.842 4.0 0.6 0.537 0.536 0.535 0.738 0.740 0.555 0.601 0.589 0.648 4.5 0.4 0.405 0.404 0.403 0.584 0.585 0.423 0.458 0.453 0.498 5.0 0.3 0.305 0.304 0.304 0.462 0.463 0.322 0.349 0.348 0.383 b 0.567 0.567 0.567 0.468 0.468 0.545 0.545 0.525 0.525 A 5.19 5.18 5.17 4.80 4.81 4.91 5.32 4.81 5.29 E 0.2813 0.2748 0.2756 0.2809 0.2800 0.2944 0.2912 0.2940 0.2991 E = 0.2961 target

PAGE 29

Statistics and Nonlinear Regression Allen R. Overman 27 Table 6. (Continued). X Y Y 0.0 5.0 4.820 4.810 4.810 4.810 4.820 4.800 4.790 4.750 5.220 0.5 4.0 3.815 3.807 3.807 3.808 3.816 3.800 3.793 3.685 4.049 1.0 2.8 3.020 3.014 3.013 3.015 3.022 3.009 3.003 2.858 3.141 1.5 2.2 2.390 2.386 2.385 2.387 2.392 2.382 2.377 2.217 2.436 2.0 2.0 1.892 1.888 1.888 1.890 1.894 1.886 1.882 1.720 1.890 2.5 1.5 1.498 1.495 1.494 1.497 1.500 1.493 1.490 1.334 1.466 3.0 1.0 1.186 1.183 1.182 1.185 1.187 1.182 1.180 1.035 1.137 3.5 0.9 0.938 0.937 0.936 0.938 0.940 0.936 0.934 0.803 0.882 4.0 0.6 0.743 0.741 0.741 0.743 0.744 0.741 0.740 0.623 0.684 4.5 0.4 0.588 0.587 0.586 0.588 0.589 0.587 0.586 0.483 0.531 5.0 0.3 0.465 0.464 0.464 0.466 0.467 0.465 0.464 0.375 0.412 b 0.4675 0.4675 0.4677 0.4670 0.4670 0.4670 0.4670 0.508 0.508 A 4.82 4.81 4.81 4.81 4.82 4.80 4.79 4.75 5.22 E 0.2818 0.2829 0.2813 0.2853 0.2846 0.2864 0.2882 0.2932 0.2918 E = 0.2961 target Table 7. Combinations of A and b to satisfy equal probability equation for 99% probability. X Y Y 0.0 5.0 5.350 4.680 5.280 4.640 5.160 4.600 5.420 4.780 0.5 4.0 4.133 3.616 4.112 3.614 4.059 3.618 4.138 3.649 1.0 2.8 3.193 2.793 3.202 2.814 3.193 2.846 3.158 2.786 1.5 2.2 2.467 2.158 2.494 2.192 2.512 2.239 2.411 2.126 2.0 2.0 1.906 1.667 1.942 1.707 1.976 1.761 1.841 1.623 2.5 1.5 1.472 1.288 1.513 1.329 1.554 1.385 1.406 1.239 3.0 1.0 1.137 0.995 1.178 1.035 1.223 1.090 1.073 0.946 3.5 0.9 0.879 0.769 0.806 0.763 0.962 0.857 0.819 0.722 4.0 0.6 0.679 0.594 0.715 0.628 0.756 0.674 0.625 0.551 4.5 0.4 0.524 0.459 0.557 0.489 0.595 0.530 0.477 0.421 5.0 0.3 0.405 0.354 0.433 0.381 0.468 0.417 0.364 0.321 b 0.5161 0.5161 0.500 0.500 0.480 0.480 0.540 0.540 A 5.35 4.68 5.28 4.64 5.16 4.60 5.42 4.78 E 0.4274 0.4311 0.4301 0.4193 0.4285 0.4259 0.4248 0.4254 E = 0.4232 target

PAGE 30

Statistics and Nonlinear Regression Allen R. Overman 28 Table 7. (C ontinued). X Y Y 0.0 5.0 5.440 4.890 4.960 5.440 5.040 5.420 5.360 5.160 5.220 0.5 4.0 4.111 3.696 3.730 4.091 3.771 4.056 3.991 3.842 4.086 1.0 2.8 3.107 2.793 2.805 3.076 2.822 3.035 2.971 2.860 3.198 1.5 2.2 2.349 2.111 2.109 2.314 2.112 2.271 2.212 2.130 2.503 2.0 2.0 1.775 1.596 1.586 1.740 1.580 1.699 1.647 1.586 1.959 2.5 1.5 1.341 1.206 1.193 1.308 1.182 1.271 1.226 1.180 1.533 3.0 1.0 1.014 0.911 0.897 0.984 0.885 0.951 0.913 0.879 1.200 3.5 0.9 0.766 0.689 0.675 0.740 0.662 0.712 0.680 0.654 0.939 4.0 0.6 0.579 0.521 0.507 0.556 0.495 0.533 0.506 0.487 0.735 4.5 0.4 0.438 0.393 0.382 0.418 0.371 0.399 0.377 0.363 0.576 5.0 0.3 0.331 0.297 0.287 0.315 0.277 0.298 0.281 0.270 0.450 b 0.560 0.560 0.570 0.570 0.580 0.580 0.590 0.590 0.490 A 5.44 4.89 4.96 5.44 5.04 5.42 5.36 5.16 5.22 E 0.4193 0.4209 0.4188 0.4239 0.4221 0.4251 0.4241 0.4231 0.4220 E = 0.4232 target Table 7. (C ontinued). X Y Y 0.0 5.0 5.280 4.610 4.990 4.800 4.600 5.080 4.620 4.630 4.930 0.5 4.0 3.925 3.663 3.965 3.837 3.637 4.016 3.616 3.696 3.927 1.0 2.8 2.918 2.910 3.150 3.067 2.875 3.175 2.830 2.944 3.128 1.5 2.2 2.169 2.312 2.503 2.451 2.273 2.510 2.215 2.345 2.491 2.0 2.0 1.613 1.837 1.989 1.959 1.797 1.984 1.734 1.868 1.984 2.5 1.5 1.199 1.460 1.580 1.566 1.421 1.569 1.357 1.488 1.581 3.0 1.0 0.891 1.160 1.255 1.252 1.123 1.240 1.062 1.185 1.259 3.5 0.9 0.663 0.921 0.997 1.001 0.888 0.980 0.831 0.944 1.003 4.0 0.6 0.493 0.732 0.792 0.800 0.702 0.775 0.651 0.752 0.799 4.5 0.4 0.366 0.582 0.630 0.639 0.555 0.613 0.509 0.599 0.636 5.0 0.3 0.272 0.462 0.500 0.511 0.439 0.484 0.399 0.477 0.507 b 0.593 0.460 0.460 0.448 0.470 0.470 0.490 0.455 0.455 A 5.28 4.61 4.99 4.80 4.60 5.08 4.62 4.63 4.93 E 0.4207 0.4213 0.4264 0.4222 0.4192 0.4222 0.4171 0.4188 0.4252 E = 0.4232 target

PAGE 31

Statistics and Nonlinear Regression Allen R. Overman 29 Table 7. (C ontinued). X Y Y 0.0 5.0 4.730 5.390 4.830 5.440 5.430 4.660 5.310 4.720 5.380 0.5 4.0 3.629 4.135 3.669 4.132 4.124 3.615 4.119 3.630 4.138 1.0 2.8 2.784 3.173 2.787 3.139 3.133 2.804 3.195 2.792 3.183 1.5 2.2 2.136 2.434 2.117 2.384 2.380 2.175 2.478 2.148 2.448 2.0 2.0 1.639 1.867 1.608 1.811 1.807 1.687 1.922 1.652 1.883 2.5 1.5 1.257 1.433 1.221 1.375 1.373 1.309 1.491 1.270 1.448 3.0 1.0 0.965 1.099 0.928 1.045 1.043 1.015 1.157 0.977 1.114 3.5 0.9 0.740 0.843 0.705 0.794 0.792 0.787 0.897 0.751 0.857 4.0 0.6 0.568 0.647 0.535 0.603 0.602 0.611 0.696 0.578 0.659 4.5 0.4 0.436 0.496 0.407 0.458 0.457 0.474 0.540 0.445 0.507 5.0 0.3 0.334 0.381 0.309 0.348 0.347 0.368 0.419 0.342 0.390 b 0.530 0.530 0.550 0.550 0.550 0.508 0.508 0.525 0.525 A 4.73 5.39 4.83 5.44 5.43 4.66 5.31 4.72 5.38 E 0.4346 0.4174 0.4246 0.4301 0.4159 0.4221 0.4174 0.4191 0.4259 E = 0.4232 target Table 7. (C ontinued). X Y Y 0.0 5.0 4.700 4.680 4.670 4.660 4.860 4.720 4.700 4.750 0.5 4.0 3.753 3.737 3.729 3.721 3.881 3.773 3.757 3.799 1.0 2.8 2.997 2.984 2.978 2.971 3.099 3.016 3.003 3.038 1.5 2.2 2.393 2.383 2.378 2.373 2.474 2.410 2.400 2.429 2.0 2.0 1.911 1.903 1.899 1.895 1.976 1.927 1.919 1.943 2.5 1.5 1.526 1.519 1.516 1.513 1.578 1.540 1.534 1.554 3.0 1.0 1.218 1.213 1.211 1.208 1.260 1.231 1.226 1.243 3.5 0.9 0.973 0.969 0.967 0.965 1.006 0.984 0.980 0.994 4.0 0.6 0.777 0.774 0.772 0.770 0.803 0.786 0.783 0.795 4.5 0.4 0.620 0.618 0.616 0.615 0.641 0.629 0.626 0.635 5.0 0.3 0.495 0.493 0.492 0.491 0.512 0.502 0.500 0.508 b 0.450 0.450 0.450 0.450 0.450 0.448 0.448 0.447 A 4.70 4.68 4.67 4.66 4.86 4.72 4.70 4.75 E 0.4063 0.4139 0.4183 0.4229 0.4280 0.4159 0.4200 0.4226 E = 0.4232 target

PAGE 32

Statistics and Nonlinear Regression Allen R. Overman 30 Table 8. Correlation of E with b using a parabolic model with 016 5 A b E E E E E 0.577 0.4232 0.4169 0.560 0.2961 0.2892 0.2992 0.543 0.2069 0.2051 0.2039 0.2075 0.529 0.1657 0.1689 0.1616 0.1642 0.1656 0.5161 0.1521 0.1612 0.1517 0.1523 0.1520 0.503 0.1664 0.1796 0.1701 0.1677 0.1663 0.491 0.2069 0.2190 0.2121 0.2061 0.477 0.2961 0.2925 0.2915 0.462 0.4232 0.4041

PAGE 33

Statistics and Nonlinear Regression Allen R. Overman 31 List of Figures 1. Dependence of response variable ( Y ) on control variable ( X ). Data from Table 1. Curve drawn fro m Eq. ( 35). 2. Dependence of ln Y on X Data from Table 1. Line drawn from Eq. ( 33). 3. Scatter plot for estimated response variable ( Y ) vs. measured response variable ( Y ). Line represents the 45% diagonal. 4. Equal probability contours between parameters A and b. Contours drawn from Table 5 (75%), Table 6 (95 %) and Table 7 (99%) probability levels, respectively. Optimum and standard error values of 075 0 016 5 A and 0129.05161.0b are also shown. 5. Dependence of error sum of squares ( E ) on exponential parameter ( b) for linear parameter A = 5.016. Parabola drawn from Eq. ( 72).

PAGE 34

Statistics and Nonlinear Regression Allen R. Overman 32 ABE 6933 Special Topics Mathematical and Statistical Characteristics of Nonlinear Regression Models A. R. Overman I. Elements of Probability and Calculus A. Arithmetic the process of c ounting B. Natural numbers positive integers ( ,2,1,0 ) C. Rational numbers ratio of two integers ( ,3/2,3/1,2/1,,1/2,1/1 ) D. Irrational numbers (such as e, 2 etc.) E. Complex nu mbers z = x + i y with i = 1 F. Binomial theorem and Pascals triangle (a + b)0 = 1 (a + b)1 = a1 + b1 (a + b)2 = a2 + 2ab + b2 (a + b)3 = a3 + 3a2b + 3ab2 + b3 (a + b)4 = a4 + 4a3b + 6a2b2 + 4ab3 + b4 (a + b)5 = a5 + 5a4b + 10a3b2 + 10a2b3 + 5ab4 + b5 (a + b)6 = a6 + 6a5b + 15a4b2 + 20a3b3 + 15a2b4 + 6ab5 + b6 (a + b)7 = a7 + 7a6b + 21a5b2 + 35a4b3 + 35a3b4 + 21a2b5 + 7ab 6 + b7 (a + b)8 = a8 + 8a7b + 28a6b2 + 56a5b3 + 70a4b4 + 56a3b5 + 28a2b6 + 8ab7 + b8 (a + b)9 = a9 + 9a8b + 36a7b2 + 84a6b3 + 126a5b4 + 126a4b5 + 84a3b6 + 36a2b7 + 9ab8 + b9 Note symmetry in the distribution of coefficients for each expansion.

PAGE 35

Statistics and Nonlinear Regression Allen R. Overman 33 Pascals triangle for binomial coefficients 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 1 7 21 35 35 21 7 1 1 8 28 56 70 56 28 8 1 1 9 36 84 126 126 84 36 9 1 1 10 45 120 210 252 210 120 45 10 1 1 11 55 165 330 462 462 330 165 55 11 1 1 12 66 220 495 792 924 792 495 220 66 12 1 Note the pattern in the coefficients, including symmetry. G. Frequency distributions 1. Discrete distribution Consider the problem of a peg board. This is a two state system a cell (hole) is either filled or empty. Eac h cell holds one and only one object (peg), which can be viewed as a type of exclusion principle Define n as the total number of cells and x as the number of filled cells (pegs). Cells (holes) are indistinguishable (all alike), as are the objects (pegs). Order of filling the cells is irrelevant. Note that a peg board can be linear, triangular, rectangular (Eigen and Winkler, 1993, p. 40; Polster, 2004, p. 33), or even 3dimensional. The number of distinguishable combinations which are possible for each x, xnc, can be calculated from (Ruhla, 1992, p. 18; Watkins, 2000, p. 22) )!(!!,xnxnxnc (1)

PAGE 36

Statistics and Nonlinear Regression Allen R. Overman 34 where n n 3 2 1 and is called n factorial. Note that n can assume positive integers ( 3 2 1 n ) and x can also assume positive integers ( n x , 2 1 0 ). For small values of n it is easy to estimate c by intuition, but for larger n calculations of c are best performed on a pocket calculator or computer with the algorithm for computations (Eq. (1)) built in. The total number of combinations C for the system is defined as the sum of c values for all values of x and can be calculated from C = 2n. The frequency distribution of c values is then calculated from f = c/C. Cumulative frequency is calculated from the cumulative sum fF (2) so that F is normalized 10F It should be noted that F forms a discrete set of numbers for a particular case. 2. Continuous distribution The next step is to compare the discret e distribution to a continuous Gaussian distribution where x is considered a continuous variable and the cumulative distribution is described by 2 erf 1 2 1 x F (3) where and are the mean and spread of the distribution. The e rror function is defined by 202)exp(22erfxduux (4) where )exp(2u represents the Gaussian distribution (bell shaped curve). Values of the erf can be obtained from mathematical tables (cf. Abramowitz and Stegun, 1965, chp. 7). Some pr operties of the error function should be noted: erf (0) = 0, erf ( ) = 1, erf (x ) = erf (+ x ), erf ( ) = 1 Equation (3) can be rearranged to the linear form xFZ21212erf1 (5) where erf 1 is the inverse error function. For example, 00.1)8427.0(erf1 Linear regression of Z vs. x leads to values of the parameters and With these parameters now known the frequency distribution for f vs. x can be calculated for the cont inuous distribution from

PAGE 37

Statistics and Nonlinear Regression Allen R. Overman 35 22exp21xf (6) The procedure can now be applied to a linear peg board, triangular peg board, and square peg board. It can even be applied to a 3dimensional system. This analysis falls within a branch of mathematic s known as group theory Values of the error function can be calculated from the series approximation (Abramowitz and Stegun, 1965, p. 299) 4 4 3 2] 078108 0 000972 0 230389 0 278393 0 1 [ 1 1 erf x x x x x (7) for 8 1 0 x For the case where erf x is given, the inverse erf 1 an d therefore x can be obtained on a scientific calculator or computer using the solver routine Note that for the case F < 0.5 and 2F 1 < 0 (negative) the procedure is to change the value from to +, solve for the inverse by Eq. (7) and change the sign from + x to x Equation (7) does not work directly for x because the power series in Eq. (7) is not symmetric. H. Symmetry and conservation principle In all of the discrete and continuous Gaussian distributions we note symmetry in the distributions around a mean point. A mathematical consequence of this property is that something is conserved (remains constant) in the system. Note that the number of filled cells is defined by x Since this is a two state (binary) system (cells are either empty of filled), it follows that the number of unfilled cells is n x The total capacity of the system is the sum of filled and unfilled cells so that total capacity is = x + n x = n While this is obvious for our case, it illustrates the connection between symmetry and conservation. This property turns out to be very important in the various models of physics (including mechanics, electromagnetism, relativity, and quantum mechanics). It also shows up in chemistry and biology.

PAGE 38

Statistics and Nonlinear Regression Allen R. Overman 36 Frequency distributions for a linear peg board Case 1 x 2 In this case n = 1 x 2 = 2 and x = 0, 1, 2. Table A1 Frequency distribution for the linear peg board with a 1 x 2 array. x c f F Z f c 0.0000 0 1 0.2500 0.2167 0.867 0.2500 0.4767 1 2 0.5000 0.5379 2.152 0.7500 +0.4767 2 1 0.2500 0.2167 0.867 1.0000 C 4 = 22 Note symmetry in the frequency distribution. x x F Z 9534 0 9534 0 2 1 2 1 2 erf 1 r = 1 000.1,0489.12 220489.100.1exp5379.02exp21 xxf f f 2848 1 1045 0 r = 1 fc4

PAGE 39

Statistics and Nonlinear Regression Allen R. Overman 37 Case 1 x 3 In this case n = 1 x 3 = 3 and x = 0, 1, 2, 3. Table A2 Frequency distribution for the linear peg board with a 1 x 3 array. x c f F Z f c 0.0000 0 1 0.1250 0.1037 0.83 0.1250 0.8142 1 3 0.3750 0.3888 3.11 0.5000 0.0000 2 3 0.3750 0.3888 3.11 0.8750 +0.8142 3 1 0.1250 0.1037 0.83 1.0000 C 8 = 23 Note symmetry in the frequency distribution. x x F Z 8142 0 2213 1 2 1 2 1 2 erf 1 r = 1.0000 500.1,2282.12 222282.150.1exp4594.02exp21xxf ff1404.103885.0 r = 1.0000 fc8

PAGE 40

Statistics and Nonlinear Regression Allen R. Overman 38 Case 1 x 4 In this case n = 1 x 4 = 4 and x = 0, 1, 2, 3, 4. Table A3 Frequency distribution for the linear peg board with a 1 x 4 array. x c f F Z f c 0.0000 0 1 0.0625 0.0512 0.82 0.0625 1.0842 1 4 0.2500 0.2419 3.87 0.3125 0.3452 2 6 0.3750 0.4060 6.50 0.6875 +0.3452 3 4 0.2500 0.2419 3.87 0.9375 +1.0842 4 1 0.0625 0.0512 0.82 1.0000 C 16 = 24 Note symmetry in the frequency distribution. x x F Z 7196 0 4391 1 2 1 2 1 2 erf 1 r = 0. 999909 000.2,3897.12 223897.100.2exp4060.02exp21xxf f f 1052 1 0226 0 r = 0.99710 f c 16

PAGE 41

Statistics and Nonlinear Regression Allen R. Overman 39 Case 1 x 5 In this case n = 1 x 5 = 5 and x = 0, 1, 2, 3, 4, 5. Table A4 Frequency distribution for the linear peg board wit h a 1 x 5 array. x c f F Z f c 0.00000 0 1 0.03125 0.02590 0.83 0.03125 1.3148 1 5 0.15625 0.14143 4.53 0.18750 0.6277 2 10 0.31250 0.33050 10.58 0.50000 0.0000 3 10 0.31250 0.33050 10.58 0.81250 +0.6277 4 5 0.15625 0.14143 4.53 0.96875 +1.3148 5 1 0.03125 0.02590 0.83 1.00000 C 32 = 25 Note symmetry in the frequency distribution. x x F Z 65146 0 6286 1 2 1 2 1 2 erf 1 r = 0.99983 500.2,5350.12 225350.150.2exp3675.02exp21xxf f f 0882 1 01543 0 r = 0.99723 fc32

PAGE 42

Statistics and Nonlinear Regression Allen R. Overman 40 Case 1 x 6 In this case n = 1 x 6 = 6 and x = 0, 1, 2, 3, 4, 5, 6. Table A5 Frequency distribution for the linear peg board with a 1 x 6 array x c f F Z f c 0.000000 0 1 0.015625 0.013247 0.85 0.015625 1.5213 1 6 0.093750 0.080181 5.13 0.109375 0.8703 2 15 0.234375 0.236181 15.11 0.343750 0.2841 3 20 0.312500 0.338560 21.67 0.656250 +0.2841 4 15 0.234375 0.236181 15.11 0.890625 +0.8703 5 6 0.093750 0.080181 5.13 0.984375 +1.5213 6 1 0.015625 0.013247 0.85 1.000000 C 64 = 26 Note symmetry in the frequency distribution. x x F Z 60009 0 8003 1 2 1 2 1 2 erf 1 r = 0.99975 000 3 66643 1 2 2266643.100.3exp33856.02exp21xxf f f 0809 1 01187 0 r = 0.99730 f c 64

PAGE 43

Statistics and Nonlinear Regression Allen R. Overman 41 Case 1 x 7 In this case n = 1 x 7 = 7 and x = 0, 1, 2, 3, 4, 5, 6, 7. Table A6 Frequency distribution for the linear peg board with a 1 x 7 array. x c f F Z f 0.0000000 0 1 0.0078125 0.008580 0.0078125 1.7123 1 7 0.0546875 0.049286 0.0625000 1.0842 2 21 0.1640625 0.158080 0.2265625 0.5305 3 35 0.2734375 0.283108 0.5000000 0.0000 4 35 0.2734375 0.283108 0.7734375 +0.5305 5 21 0.1640625 0.158080 0.9375000 +1.0842 6 7 0.0546875 0.049286 0.9921875 +1.7123 7 1 0.0078125 0.008580 1.0000000 C 128 = 27 Note symmetry in the frequency distribution. x x F Z 53978 0 88923 1 2 1 2 1 2 erf 1 r = 0.999963 500 3 8526 1 2 2 28526 1 50 3 exp 3045 0 2 exp 2 1 x x f f f 0689 1 011875 0 r = 0.999198

PAGE 44

Statistics and Nonlinear Regression Allen R. Overman 42 Case 1 x 8 In this case n = 1 x 8 = 8 and x = 0, 1, 2, 3, 4, 5, 6, 7, 8. Table A7 Frequency distribution for the linear peg board with a 1 x 8 array. x c f F Z f 0.000000 0 1 0.003906 0.004626 0.5 0.003906 1.8934 0.012167 1 8 0.031250 0.028129 1.5 0.035156 1.2777 0.057167 2 28 0.109375 0.102126 2.5 0.144531 0.7504 0.160373 3 56 0.218750 0.221375 3.5 0.363281 0.2470 0.268612 4 70 0.273438 0.286500 4.5 0.636719 +0.2470 0.268612 5 56 0.218750 0.221375 5.5 0.855469 +0.7504 0.160373 6 28 0.109375 0.102126 6.5 0.964844 +1.2777 0.057167 7 8 0.031250 0.028129 7.5 0.996094 +1.8934 0.012167 8 1 0.003906 0.004626 1.000000 C 256 = 28 Note symmetry in the frequency distribution. xxFZ5078.00312.221212erf1 r = 0.999946 0000.4,9692.12 229692.100.4exp2865.02exp21xxf f f 0589 1 008700 0 r = 0.999084

PAGE 45

Statistics and Nonlinear Regression Allen R. Overman 43 Frequency distributions of a triangular peg board Case 1: 1 Cell ( n = 1) For this case n = 1 and x = 0 or 1. The distribution is given in Table A8. Table A8 Frequency distribution for a triangular peg board with 1 cell. x c f F 0.000 0 1 0.500 0.500 1 1 0.500 1.000 C 2 = 21 Case 2: 3 Cells ( n = 3) For this case n = 3 and x = 0, 1, 2, or 3. The pegboard is shown in the diagram. The corresponding distribution is given in Table A9. Table A9 Frequency distribution for a triangular peg board with 3 cell s x c f F Z f 0.0000 0 1 0.1250 0.1034 0.5 0.1250 0.8142 0.2367 1 3 0.3750 0.3892 1.5 0.5000 0.0000 0.4594 2 3 0.3750 0.3892 2.5 0.8750 +0.8142 0.2367 3 1 0.1250 0.1034 1.0000 C 8 = 23 xxFZ8142.02213.1212)12(erf1 r = 1.0000 500 1 2282 1 2 222282.1500.1exp4594.02exp21xxf f f 1432 1 03950 0 r = 1.000000

PAGE 46

Statistics and Nonlinear Regression Allen R. Overman 44 This system is an example of group theory in mathematics, which links principles of symmetry and conservation. Note symmetry of c around the mean value of x ( 50.1 ). Conservation comes from Filled cells + Unfilled cells = x + ( n x ) = n = total capacity of the system (number of cells) Case 3: 6 Cells ( n = 6) For this case n = 6 and x = 0, 1, 2, 3, 4, 5, 6. The distribution is given in Table A10. Table A10 Frequency distri bution for a triangular peg board with 6 cell s x c f F Z f 0.000000 0 1 0.015625 0.013246 0.015625 1.5213 1 6 0.093750 0.080177 0.109375 0.8703 2 15 0.234375 0.236178 0.343750 0.2841 3 20 0.312500 0.338560 0.656250 +0.2841 4 15 0.234375 0.236178 0.890625 +0.8703 5 6 0.093750 0.080177 0.984375 +1.5213 6 1 0.015625 0.013246 1.000000 C 64 = 26 xxFZ6001.08003.1212)12(erf1 r = 0.999748 000 3 6664 1 2 226664.100.3exp33856.02exp21xxf f f 0809 1 01187 0 r = 0.99730

PAGE 47

Statistics and Nonlinear Regression Allen R. Overman 45 Case 4: 10 Cells ( n = 10) For this case n = 10 and x = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. The distribution is given in Table A11. Table A11 Frequency distribution for a triangula r peg board with 10 cell s x c f F Z f 0.0000000 0 1 0.0009766 0.001342 0.0009766 1 10 0.0097656 0.008921 0.0107422 1.6260 2 45 0.0439453 0.038930 0.0546875 1.1311 3 120 0.1171875 0.111513 0.1718750 0.6700 4 210 0.2050781 0.209679 0.3769531 0.2215 5 252 0.2460938 0.258800 0.6230469 +0.2215 6 210 0.2050781 0.209679 0.8281250 +0.6700 7 120 0.1171875 0.111513 0.9453125 +1.1311 8 45 0.0439453 0.038930 0.9892578 +1.6260 9 10 0.0097656 0.008921 0.9990234 10 1 0.0009766 0.001342 1.0000000 C 1024 = 210 xxFZ4588.02939.2212)12(erf1 r = 0.99988 000.5,1797.22 221797.2000.5exp2588.02exp21xxf f f 0383 1 00352 0 r = 0.99894

PAGE 48

Statistics and Nonlinear Regression Allen R. Overman 46 Case 5: 15 Cells ( n = 15) For this case n = 15 and x = 15 , 3 2 1 0 The distribution is given in Table A12. Table A12 Frequency distribution for a triangular peg board with 15 cell s x c f F Z f 0.0000000 0 1 0.0000305 0.000096 0.0000305 1 15 0.0004578 0.000652 0.0004883 2 105 0.0032043 0.003355 0.0036926 3 455 0.0138855 0.013138 0.0175781 1.4875 4 1365 0.0416565 0.039158 0.0592346 1.1032 5 3003 0.0916443 0.088829 0.1508789 0.7309 6 5005 0.1527405 0.153358 0.3036194 0.3630 7 6435 0.1963806 0.201504 0.5000000 0.0000 8 6435 0.1963806 0.201504 0.6963806 +0.3630 9 5005 0.1527405 0.153358 0.8491211 +0.7309 10 3003 0.0916443 0.088829 0.9407654 +1.1032 11 1365 0.0416565 0.039158 0.9824219 +1.4875 12 455 0.0138855 0.013138 0.9963074 13 105 0.0032043 0.003355 0.9995117 14 15 0.0004578 0.000652 0.9999695 15 1 0.0000305 0.000096 1.0000000 C 32768 = 215 xxFZ3695.07711.2212)12(erf1 r = 0.999972 500.7,7065.22

PAGE 49

Statistics and Nonlinear Regression Allen R. Overman 47 2 27065 2 500 7 exp 2085 0 2 exp 2 1 x x f ff02423.1002046.0 r = 0.999637

PAGE 50

Statistics and Nonlinear Regression Allen R. Overman 48 Case 6: 21 Ce lls ( n = 21) For this case n = 21 and x = 21 , 3 2 1 0 The distribution is given in Table A13. Table A13 Frequency distribution for a triangular peg board with 21 cell s x c f F Z f 0.000000 0 1 0.000000 0.000004 0.000000 1 21 0.000010 0.000028 0.000010 2 210 0.000100 0.000159 0.000110 3 1,330 0.000634 0.000749 0.000744 4 5,985 0.002854 0.002913 0.003598 5 20,349 0.009703 0.009333 0.013301 1.5668 6 54,264 0.025875 0.024629 0.039176 1.2429 7 116,280 0.055447 0.053525 0.094623 0.9287 8 203,490 0.097032 0.095807 0.191655 0.6168 9 293,930 0.140157 0.141240 0.331812 0.3071 10 352,716 0.168188 0.171489 0.500000 0.0000 11 352,716 0.168188 0.171489 0.668188 +0.3071 12 293,930 0.140157 0.141240 0.808345 +0.6168 13 203,490 0.097032 0.095807 0.905377 +0.9287 14 116,280 0.055447 0.053525 0.960824 +1.2429 15 54,264 0.025875 0.024629 0.986699 +1.5668 16 20,349 0.009703 0.009333 0.996402 17 5,985 0.002854 0.002913 0.999256 18 1,330 0.000634 0.000749 0.999890

PAGE 51

Statistics and Nonlinear Regression Allen R. Overman 49 19 210 0.000100 0.000159 0.999990 20 21 0.000010 0.000028 1.000000 21 1 0.000000 0.000004 1.000000 C 2,097,152 = 221 x x F Z 3115 0 2707 3 2 1 2 ) 1 2 ( erf 1 r = 0.999982 500 10 2103 3 2 222103.3500.10exp1757.02exp21xxf ff0183.100135.0 r = 0.99980

PAGE 52

Statistics and Nonlinear Regression Allen R. Overman 50 Case 7: 28 Cells ( n = 28) For this case n = 28 and x = 28 , 3 2 1 0 The distribution is given in Table A14. Table A14 Frequency distribution for a triangular peg board with 28 cell s x c f F Z f 0.0000000 0 1 0.0000000 0.000000 0.0000000 1 28 0.0000001 0.000001 0.0000001 2 378 0.0000014 0.000004 0.0000015 3 3,276 0.0000122 0.000022 0.0000137 4 20,475 0.0000763 0.000104 0.0000900 5 98,280 0.0003661 0.000414 0.0004561 6 376,740 0.0014035 0.001431 0.0018596 7 1,184,040 0.0044109 0.004273 0.0062705 1.7706 8 3,108,105 0.0115786 0.011029 0.0178491 1.4830 9 6,906,900 0.0257302 0.024603 0.0435793 1.2079 10 13,123,110 0.0488874 0.047433 0.0924667 0.9378 11 21,474,180 0.0799975 0.079034 0.1724642 0.6684 12 30,421,755 0.1133299 0.113814 0.2857941 0.3996 13 37,442,160 0.1394829 0.141652 0.4252770 0.1335 14 40,116,600 0.1494460 0.152370 0.5747230 +0.1335 15 37,442,160 0.1394829 0.141652 0.7142059 +0.3996 16 30,421,755 0.1133299 0.113814 0.8275358 +0.6684 17 21,474,180 0.0799975 0.079034 0.9075333 +0.9378 18 13,123,110 0.0488874 0.047433 0.9564207 +1.2079

PAGE 53

Statistics and Nonlinear Regression Allen R. Overman 51 19 6,906,900 0.0257302 0.024603 0.9821509 +1.4830 20 3,108,105 0.0115786 0.011029 0.9937295 +1.7706 21 1,184,040 0.0044109 0.004273 0.9981404 22 376,740 0.0014035 0.001431 0.9995439 23 98,280 0.0003661 0.000414 0.9999100 24 20,475 0.0000763 0.000104 0.9999863 25 3,276 0.0000122 0.000022 0.9999985 26 378 0.0000014 0.000004 0.9999999 27 28 0.0000001 0.000001 1.0000000 28 1 0.0000000 0.000000 1.0000000 C 268,435,456 = 228 xxFZ27007.07810.3212)12(erf1 r = 0.999974 000.14,70275.32 2270275.300.14exp15237.02exp21xxf f f 01674 1 000995 0 r = 0.99985

PAGE 54

Statistics and Nonlinear Regression Allen R. Overman 52 Frequency distributions for a square peg board. Case 1: 2 x 2 For this case n = 2 x 2 = 4 and x assumes values of 0, 1, 2, 3, and 4. Corresponding values of x n c are calculated from Eq. (1). Results are given in Table A15 Table A15 Frequency dist ribution for the square peg board with a 2 x 2 array. x c f F Z f 0.0000 0 1 0.0625 0.05117 0.0625 1.0842 1 4 0.2500 0.24191 0.3125 0.3452 2 6 0.3750 0.40600 0.6875 +0.3452 3 4 0.2500 0.24191 0.9375 +1.0842 4 1 0.0625 0.05117 1.0000 C 16 = 24 Note the symmetry in the discrete frequency distribution. x x F Z 7196 0 4391 1 2 1 2 1 2 erf 1 r = 0.999909 000.2,3897.12 223897.100.2exp4060.02exp21xxf f f 1053 1 02262 0 r = 0.9971

PAGE 55

Statistics and Nonlinear Regression Allen R. Overman 53 Case 2: 3 x 3 For this case n = 3 x 3 = 9 and x assumes values of 0, 1, 2, 3, 9, Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A16. Table A16 Frequency distribution for the square peg board with a 3 x 3 array. x c f F Z f 0.000000 0 1 0.001953 0.002497 0.001953 1 9 0.017578 0.015920 0.019531 1.4568 2 36 0.070312 0.063869 0.089843 0.9491 3 84 0.164063 0.161257 0.253906 0.4680 4 126 0.246094 0.256230 0.500000 0.0000 5 126 0.246094 0.256230 0.746094 +0.4680 6 84 0.164063 0.161257 0.910157 +0.9491 7 36 0.070312 0.063869 0.980469 +1.4568 8 9 0.017578 0.015920 0.998047 9 1 0.001953 0.002497 1.000000 Total 512 = 29 Again note the symmetry in the discrete frequency distribution. xxFZ4812.01653.221212erf1 r = 0.999919 500.4,0782.22 2 20782 2 50 4 exp 2715 0 2 exp 2 1 x x f ff0369.100373.0 r = 0.99895

PAGE 56

Statistics and Nonlinear Regression Allen R. Overman 54 Case 3: 4 x 4 For this case n = 4 x 4 = 16 and x assumes values of 0, 1, 2, 3, 16, Corresponding values of xnc, are calculated f rom Eq. (1). Results are given in Table A17. Table A17 Frequency distribution for the square peg board with a 4 x 4 array. x c f F Z f 0.0000000 0 1 0.0000153 0.000044 0.0000153 1 16 0.0002441 0.000319 0.0002594 2 120 0.0018311 0.001966 0.0020905 3 560 0.0085449 0.008102 0.0106354 1.6287 4 1820 0.0277710 0.025807 0.0384064 1.2493 5 4368 0.0666504 0.063543 0.1050568 0.8868 6 8008 0.1221924 0.120946 0.2272492 0.5288 7 11440 0.1745605 0.177953 0.4018097 0.1759 8 12870 0.1963806 0.202400 0.5981903 +0.1759 9 11440 0.1745605 0.177953 0.7727508 +0.5288 10 8008 0.1221924 0.120946 0.8949432 +0.8868 11 4368 0.0666504 0.063543 0.9615936 +1.2493 12 1820 0.0277710 0.025807 0.9893646 +1.6287 13 560 0.0085449 0.008102 0.9979095 14 120 0.0018311 0.001966 0.9997406 15 16 0.0002441 0.000369 0.9999847 16 1 0.0000153 0.000053 1.0000000 C 65536 = 216 Note the symmetry in the frequency distribution.

PAGE 57

Statistics and Nonlinear Regression Allen R. Overman 55 x x F Z 3588 0 8703 2 2 1 2 1 2 erf 1 r = 0.999958 000.8,7872.22 2 27872 2 00 8 exp 2024 0 2 exp 2 1 x x f ff0248.100194.0 r = 0.99962 Several characteristics should be noted from these calculations. First, note the symmetry in the distributions in the tables. Second, note that the continuous Gaussian function approximates the discrete distributions rather well. Third, as the number o f values increases, agreement between the discrete and continuous distributions improves.

PAGE 58

Statistics and Nonlinear Regression Allen R. Overman 56 Case 3: 5 x 5 For this case n = 5 x 5 = 25 and x assumes values of 0, 1, 2, 3, 25 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A18. Table A18 Frequency distribution for the square peg board with a 5 x 5 array. x c f F Z f 0.00000000 0 1 0.00000003 0.000000 0.00000003 1 25 0.00000075 0.000003 0.00000078 2 300 0.00000894 0.000020 0.00000972 3 2,300 0.00006855 0.000101 0.00007827 4 12,650 0.00037700 0.000439 0.00045527 5 53,130 0.00158340 0.001624 0.00203867 6 177,100 0.00527799 0.005101 0.00731666 1.7297 7 480,700 0.01432598 0.013604 0.02164264 1.4266 8 1,081,575 0.03223345 0.030810 0.05387609 1.1362 9 2,042,975 0.06088540 0.059254 0.11476149 0.8503 10 3,268,760 0.09741664 0.096770 0.21217813 0.5650 11 4,457,400 0.13284087 0.134200 0.34501900 0.2816 12 5,200,300 0.15498102 0.158037 0.50000002 0.0000 13 5,200,300 0.15498087 0.158037 0.65498089 +0.2816 14 4,457,400 0.13284087 0.134200 0.78782176 +0.5650 15 3,268,760 0.09741664 0.096770 0.88523840 +0.8503 16 2,042,975 0.06088540 0.059254 0.94612380 +1.1362 17 1,081,575 0.03223345 0.030810 0.97835725 + 1.4266 18 480,700 0.01432598 0.013604

PAGE 59

Statistics and Nonlinear Regression Allen R. Overman 57 0.99268323 +1.7297 19 177,100 0.00527799 0.005101 0.99796122 20 53,130 0.00158340 0.001624 0.99954462 21 12,650 0.00037700 0.000439 0.99992162 22 2,300 0.00006855 0.000101 0.99999017 23 300 0.00000894 0.000020 0.99999911 24 25 0.00000075 0.000003 0.99999986 25 1 0.00000003 0.000000 0.99999989 C 33,554,432 = 225 Note the symmetry in the frequency distribution. x x F Z 2859 0 5740 3 2 1 2 1 2 erf 1 r = 0.999974 50 12 4975 3 2 224975.350.12exp1613.02exp21xxf ff017789.1001129.0 r = 0.999827

PAGE 60

Statistics and Nonlinear Regression Allen R. Overman 58 Case 4: 6 x 6 For this case n = 6 x 6 = 36 and x assumes values of 0, 1, 2, 3, 36 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A19. Table A19 Frequency distribution for the square peg board with a 6 x 6 array. x c f F Z f 0.000000 0 1 0.000000 1 36 0.00000000 0.000000 0.00000000 2 630 0.00000001 0.000000 0.00000001 3 7,140 0.00000010 0.000000 0.00000010 4 58,905 0.00000086 0.000002 0.00000096 5 376,992 0.00000549 0.000010 0.00000645 6 1,947,792 0.00002834 0.000039 0.00003479 7 8,347,680 0.00012147 0.000144 0.00015626 8 30,260,340 0.00044035 0.000472 0.00059661 9 94,143,280 0.00136997 0.001380 0.00196658 10 254,186,856 0.00369891 0.003606 0.00566549 1.7970 11 600,805,296 0.00874287 0.008415 0.01440836 1.5442 12 1,251,677,700 0.01821431 0.017541 0.03262267 1.3013 13 2,310,789,600 0.03362641 0.032657 0.06624908 1.0633 14 3,796,297,200 0.05524340 0.054303 0.12149248 0.8263 15 5,567,902,560 0.08102365 0.080647 0.20251613 0.5890 16 7,307,872,110 0.10634354 0.106974 0.30885967 0.3524 17 8,597,496,600 0.12511004 0.126733 0.43396971 0.1179 18 9,075,135,300 0.13206060 0.134100

PAGE 61

Statistics and Nonlinear Regression Allen R. Overman 59 0.56603031 +0.1179 19 8,597,496,600 0.12511004 0.126733 0.69114035 +0.3524 20 7,307,872,110 0.10634354 0.106974 0.79748389 +0.5890 21 5,567,902,560 0.08102365 0.080647 0.87850754 +0.8263 22 3,796,297,200 0.05524340 0.054303 0.93375094 +1.0633 23 2,310,789,600 0.03362641 0.032657 0.96737735 +1.3013 24 1,251,677,700 0.01821431 0.017541 0.98559166 +1.5442 25 600,805,296 0.00874287 0.008415 0.99433453 +1.7970 26 254,186,856 0.00369891 0.003606 0.99803344 27 94,143,280 0.00136997 0.001380 0.99940341 28 30,260,340 0.00044035 0.000472 0.99984376 29 8,347,680 0.00012147 0.000144 0.99996523 30 1,947,792 0.00002834 0.000039 0.99999357 31 376,992 0.00000549 0.000010 0.99999906 32 58,905 0.00000086 0.000002 0.99999992 33 7,140 0.00000010 0.000000 1.00000002 34 630 0.00000001 1.00000003 35 36 0.00000000 1.00000003 36 1 0.00000000 1.00000003 C 68,719,476,736 = 236 Note the symmetry in the frequency distribution.

PAGE 62

Statistics and Nonlinear Regression Allen R. Overman 60 x x F Z 2377 0 2786 4 2 1 2 1 2 erf 1 r = 0.999981 00.18,2070.42 222070.400.18exp1341.02exp21xxf ff01356.1000723.0 r = 0.999909

PAGE 63

Statistics and Nonlinear Regression Allen R. Overman 61 Case 4: 7 x 7 For this case n = 7 x 7 = 49 and x assumes values of 0, 1, 2, 3, 49 Corr esponding values of xnc, are calculated from Eq. (1). Results are given in Table A20. Table A20 Frequency distribution for the square peg board with a 7 x 7 array. x c f F Z f 0.000000 0 1 0.000000 0.000000 1 49 0.000000 0.000000 0.000000 2 1,176 0.000000 0.000000 0.000000 3 18,424 0.000000 0.000000 0.000000 4 211,876 0.000000 0.000002 0.000000 5 1,906,884 0.000000 0.000000 0.000000 6 13,983,816 0.000000 0.000000 0.000000 7 85,900,584 0.000000 0.000000 0.000000 8 450,978,066 0.000001 0.000002 0.000001 9 2,054,455,634 0.000004 0.000006 0.000005 10 8,217,822,536 0.000015 0.000020 0.000020 11 29,135,916,264 0.000052 0.000064 0.000072 12 92,263,734,836 0.000164 0.000185 0.000236 13 262,596,783,764 0.000466 0.000497 0.000702 14 675,248,872,536 0.001199 0.001228 0.001901 15 1,575,580,703,000 0.002799 0.002795 0.004700 16 3,348,108,993,000 0.005947 0.005859 0.010647 1.6285 17 6,499,270,398,000 0.011545 0.011315 0.022192 1.4192 18 11,554,258,486,000 0.020524 0.020124

PAGE 64

Statistics and Nonlinear Regression Allen R. Overman 62 0.042716 1.2145 19 18,851,684,898,000 0.033487 0.032967 0.076203 1.0119 20 28,277,527,346,000 0.050231 0.049740 0.126434 0.8093 21 39,049,918,716,000 0.069367 0.069120 0.195801 0.6061 22 49,699,896,548,000 0.088285 0.088467 0.284086 0.4032 23 58,343,356,817,000 0.103639 0.104288 0.387725 0.2016 24 63,205,303,219,000 0.112275 0.113230 0.500000 0.0000 25 63,205,303,219,000 0.112275 0.113230 0.612275 +0.2016 26 58,343,356,817,000 0.103639 0.104288 0.715914 +0.4032 27 49,699,896,548,000 0.088285 0.088467 0.804199 +0.6061 28 39,049,918,716,000 0.069367 0.069120 0.873566 +0 .8093 29 28,277,527,346,000 0.050231 0.049740 0.923797 +1.0119 30 18,851,684,898,000 0.033487 0.032967 0.957284 +1.2145 31 11,554,258,486,000 0.020524 0.020124 0.977808 +1.4192 32 6,499,270,398,000 0.011545 0.011315 0.989353 +1.6285 33 3,348,108,993,000 0.005947 0.005859 0.995300 34 1,575,580,703,000 0.002799 0.002795 0.998099 35 675,248,872,536 0.001199 0.001228 0.999298 36 262,596,783,764 0.000466 0.000497 0.999764 37 92,263,734,836 0.000164 0.000185 0.999928 38 29,135,916,264 0.000052 0.000064 0.999980 39 8,217,822,536 0.000015 0.000020 0.999995 40 2,054,455,634 0.000004 0.000006 0.999999 41 450,978,066 0.000001 0.000002

PAGE 65

Statistics and Nonlinear Regression Allen R. Overman 63 1.000000 42 85,900,584 0.000000 0.000000 43 13,983,816 44 1,906,884 45 211,876 46 18,424 47 1,176 48 49 49 1 C 562,949,953,421,000 = 249 Note the symmetry in the frequency distribution. x x F Z 20281 0 9687 4 2 1 2 1 2 erf 1 r = 0.9999962 50.24,9308.42 2 29308 4 50 24 exp 1144 0 2 exp 2 1 x x f f f 00727 1 000345 0 r = 0.999965

PAGE 66

Statistics and Nonlinear Regression Allen R. Overman 64 Case 5: 8 x 8 For this case n = 8 x 8 = 64 and x assumes values of 0, 1, 2, 3, 64 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A2 1. Table A21 Frequency distribution for the square peg board with a 8 x 8 array. x c f F Z f 0.000000 0 1 0.000000 0.000000 1 64 0.000000 0.000000 0.000000 2 3 4 7 8 0.00000000 1018 9 0.00000003 1018 10 0.00000015 1018 11 0.00000074 1018 0.0000000 0.0000000 12 0.00000328 1018 0.0000002 0.000000 0.0000002 13 0.00001314 1018 0.0000007 0.000001 0.0000009 14 0.00004786 1018 0.0000026 0.000004 0.0000035 15 0.00015952 1018 0.0000086 0.000011 0.0000121 16 0.00048853 1018 0.0000265 0.000032 0.0000386 17 0.00137937 1018 0.0000748 0.000084 0.0001134 18 0.00360169 1018 0.0001952 0.000210 0.0003086 19 0.00871988 1018 0.0004727 0.000491 0.0007813

PAGE 67

Statistics and Nonlinear Regression Allen R. Overman 65 20 0.01961973 1018 0.0010636 0.001079 0.0018449 21 0.04110800 1018 0.0022285 0.002224 0.0040734 22 0.08034745 1018 0.0043556 0.004306 0.0084390 1.6919 23 0.14672143 1018 0.0079538 0.007828 0.0163828 1.5077 24 0.25064910 1018 0.0135877 0.013363 0.0299705 1.3279 25 0.40103857 1018 0.0217403 0.021421 0.0517108 1.1504 26 0.60155785 1018 0.0326105 0.032244 0.0843213 0.9736 27 0.84663698 1018 0.0458963 0.045576 0.1302176 0.7965 28 1.11877029 1018 0.0606487 0.060493 0.1908663 0.6189 29 1.38881829 1018 0.0752880 0.075395 0.2661543 0.4412 30 1.62028801 1018 0.0878360 0.088238 0.3539903 0.2645 31 1.77709008 1018 0.0963362 0.096971 0.4503265 0.0887 32 1.83262414 1018 0.0993468 0.100070 0.5496733 +0.0887 33 1.77709008 1018 0.0963362 0.096971 0.6460095 +0.2645 34 1.62028801 1018 0.0878360 0.088238 0.7338455 +0.4412 35 1.38881829 1018 0.0752880 0.075395 0.8091335 +0.6189 36 1.11877029 1018 0.0606487 0.060493 0.8697822 +0.7965 37 0.84663698 1018 0.0458963 0.045576 0.9156785 +0.9736 38 0.60155785 1018 0.0326105 0.032244 0.9482890 +1.1504 39 0.40103857 1018 0.0217403 0.021421 0.9700293 +1.3279 40 0.25064910 1018 0.0135877 0.013363 0.9836170 +1.5077 41 0.14672143 1018 0.0079538 0.007828 0.9915708 +1.6919 42 0.08034745 1018 0.0043556 0.004306 0.9959264

PAGE 68

Statistics and Nonlinear Regression Allen R. Overman 66 43 0.04110800 1018 0.0022285 0.002224 0.9981549 44 0.01961973 1018 0.0010636 0.001079 0.9992185 45 0.00871988 1018 0.0004727 0.000491 0.9996912 46 0.00360169 1018 0.0001952 0.000210 0.9998864 47 0.00137937 1018 0.0000748 0.000084 0.9999612 48 0.00048853 1018 0.0000265 0.000032 0.9999877 49 0.00015952 1018 0.0000086 0.000011 0.9999963 50 0.00004786 1018 0.0000026 0.000004 0.9999989 51 0.00001314 1018 0.0000007 0.000001 0.9999996 52 0.00000328 1018 0.0000002 0.000000 0.9999998 53 0.00000074 1018 0.0000000 0.9999998 54 0.00000015 1018 55 0.00000003 1018 56 0.00000000 1018 57 58 59 60 61 62 63 64 64 1 C 18.4467441 1018 = 264

PAGE 69

Statistics and Nonlinear Regression Allen R. Overman 67 Note the symmetry in the frequency distribution. x x F Z 17737 0 67576 5 2 1 2 1 2 erf 1 r = 0.9999963 000.32,6380.52 226380.500.32exp10007.02exp21xxf f f 00622 1 000252 0 r = 0.999978

PAGE 70

Statistics and Nonlinear Regression Allen R. Overman 68 Case 6: 9 x 9 For this case n = 9 x 9 = 81 and x assumes values of 0, 1, 2, 3, 81 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A22. Table A22 Frequency distribution for the square p eg board with a 9 x 9 array. x c f F Z f 0.000000 0 1 0.000000 0.000000 1 81 0.000000 0.000000 0.000000 2 3 4 5 6 14 0.00000002 1023 15 0.00000008 1023 16 0.00000034 1023 0.0000000 17 0.00000128 1023 0.0000000 0.0000000 18 0.00000457 1023 0.0000002 0.000000 0.0000002 19 0.00001514 1023 0.0000006 0.000001 0.0000008 20 0.00004694 1023 0.0000019 0.000003 0.0000027 21 0.00013636 1023 0.0000056 0.000007 0.0000083 22 0.00037190 1023 0.0000154 0.000018 0.0000237 23 0.00095400 1023 0.0000395 0.000044 0.0000632 24 0.00230549 1023 0.0000953 0.000103

PAGE 71

Statistics and Nonlinear Regression Allen R. Overman 69 0.0001585 25 0.00525652 1023 0.0002174 0.000228 0.0003759 26 0.01132173 1023 0.0004683 0.000480 0.0008442 27 0.02306279 1023 0.0009539 0.000961 0.0017981 28 0.04447825 1023 0.0018396 0.001834 0.0036377 29 0.08128783 1023 0.0033620 0.003329 0.0069997 1.7415 30 0.14089890 1023 0.0058274 0.005750 0.0128271 1.5769 31 0.23180142 1023 0.0095871 0.009449 0.0224142 1.4162 32 0.36218972 1023 0.0149798 0.014777 0.0373940 1.2579 33 0.53779686 1023 0.0222428 0.021988 0.0596368 1.1008 34 0.75924263 1023 0.0314015 0.031133 0.0910383 0.9439 35 1.01955439 1023 0.0421678 0.041944 0.1332061 0.7866 36 1.30276395 1023 0.0538811 0.053771 0.1870872 0.6288 37 1.58444264 1023 0.0655310 0.065592 0.2526182 0.4709 38 1.83461779 1023 0.0758780 0.076134 0.3284962 0.3136 39 2.02278372 1023 0.0836604 0.084087 0.4121566 0.1571 40 2.12392290 1023 0.0878434 0.088370 0.5000000 0.0000 41 2.12392290 1023 0.0878434 0.088370 0.5878434 +0.1571 42 2.02278372 1023 0.0836604 0.084087 0.6715038 +0.3136 43 1.83461779 1023 0.0758780 0.076134 0.7473818 +0.4709 44 1.58444264 1023 0.0655310 0.065592 0.8129128 +0.6288 45 1.30276395 1023 0.0538811 0.053771 0.8667939 +0.7866 46 1.01955439 1023 0.0421678 0.041944 0.9089617 +0.9439 47 0.75924263 1023 0.0314015 0.031133

PAGE 72

Statistics and Nonlinear Regression Allen R. Overman 70 0.9403632 +1.1008 48 0.53779686 1023 0.0222428 0.021988 0.9626060 +1.2579 49 0.36218972 1023 0.0149798 0.014777 0.9775858 +1.4162 50 0.23180142 1023 0.0095871 0.009449 0.9871729 +1.5769 51 0.14089890 1023 0.0058274 0.005750 0.9930003 +1.7415 52 0.08128783 1023 0.0033620 0.003329 0.9963623 53 0.04447825 1023 0.0018396 0.001834 0.9982019 54 0.02306279 1023 0.0009539 0.000961 0.9991558 55 0.01132173 1023 0.0004683 0.000480 0.9996241 56 0.00525652 1023 0.0002174 0.000228 0.9998415 57 0.00230549 1023 0.0000953 0.000103 0.9999368 58 0.00095400 1023 0.0000395 0.000044 0.9999763 59 0.00037190 1023 0.0000154 0.000018 0.9999917 60 0.00013636 1023 0.0000056 0.000007 0.9999973 61 0.00004694 1023 0.0000019 0.000003 0.9999992 62 0.00001514 1023 0.0000006 0.000001 0.9999998 63 0.00000457 1023 0.0000002 0.000000 1.0000000 64 0.00000128 1023 0.0000000 0.000000 1.0000000 65 0.00000034 1023 0.0000000 1.0000000 66 0.00000008 1023 0.000000 1.0000000 67 0.00000002 1023 0.000000 1.0000000 68 0.00000000 1023 0.000000 1.0000000 69 0.00000000 1023 0.000000 1.0000000 C 24.1785164 1023 = 281

PAGE 73

Statistics and Nonlinear Regression Allen R. Overman 71 Note the symmetry in the frequency distribution. x x F Z 157606 0 38305 6 2 1 2 1 2 erf 1 r = 0.9999962 5000 40 34493 6 2 2234493.650.40exp08892.02exp21xxf f f 00595 1 000232 0 r = 0.999985

PAGE 74

Statistics and Nonlinear Regression Allen R. Overman 72 Case 7: 10 x 10 For this case n = 10 x 10 = 100 and x assumes values of 0, 1, 2, 3, 100 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A23. Table A23 Frequency distribution for the square peg board with a 10 x 10 array. x c f F Z f 0.000000 0 1 0.000000 0.000000 1 100 0.000000 0.000000 0.000000 2 3 4 5 6 20 0.00000001 1029 21 0.00000002 1029 22 0.00000007 1029 23 0.00000025 1029 0.0000000 0.0000000 24 0.00000080 1029 0.0000001 0.0000001 25 0.00000243 1029 0.0000002 0.000000 0.0000003 26 0.00000700 1029 0.0000006 0.000001 0.0000009 27 0.00001917 1029 0.0000015 0.000002 0.0000024 28 0.00004999 1029 0.0000039 0.000005 0.0000063 29 0.00012411 1029 0.0000098 0.000011 0.0000161 30 0.00029372 1029 0.0000232 0.000026

PAGE 75

Statistics and Nonlinear Regression Allen R. Overman 73 0.0000393 31 0.00066325 1029 0.0000523 0.000056 0.0000916 32 0.00143012 1029 0.0001128 0.000118 0.0002044 33 0.00294692 1029 0.0002325 0.000239 0.0004369 34 0.00580717 1029 0.0004581 0.000465 0.0008950 35 0.01095067 1029 0.0008639 0.000867 0.0017589 36 0.01977205 1029 0.0015597 0.001553 0.0033186 37 0.03420030 1029 0.0026979 0.002673 0.0060165 1.7815 38 0.05670049 1029 0.0044729 0.004420 0.0104894 1.6325 39 0.09013924 1029 0.0071107 0.007020 0.0176001 1.4871 40 0.13746234 1029 0.0108439 0.010708 0.0284440 1.3440 41 0.20116440 1029 0.0158691 0.015692 0.0443131 1.2024 42 0.28258809 1029 0.0222923 0.022088 0.0666054 1.0613 43 0.38116533 1029 0.0300686 0.029866 0.0966740 0.9202 44 0.49378236 1029 0.0389526 0.038790 0.1356266 0.7787 45 0.61448471 1029 0.0484743 0.048394 0.1841009 0.6367 46 0.73470998 1029 0.0579584 0.057996 0.2420593 0.4946 47 0.84413487 1029 0.0665905 0.066763 0.3086498 0.3529 48 0.93206559 1029 0.0735270 0.073826 0.3821768 0.2119 49 0.98913083 1029 0.0780287 0.078417 0.4602055 0.0711 50 1.00891345 1029 0.0795892 0.080010 0.5397947 +0.0711 51 0.98913083 1029 0.0780287 0.078417 0.6178234 +0.2119 52 0.93206559 1029 0.0735270 0.073826 0.6913504 +0.3529 53 0.84413487 1029 0.0665905 0.066763

PAGE 76

Statistics and Nonlinear Regression Allen R. Overman 74 0.7579409 +0.4946 54 0.73470998 1029 0.0579584 0.057996 0.8158993 +0.6367 55 0.61448471 1029 0.0484743 0.048394 0.8643736 +0.7787 56 0.49378236 1029 0.0389526 0.038790 0.9033262 +0.9202 57 0.38116533 1029 0.0300686 0.029866 0.9333948 +1.0613 58 0.28258809 1029 0.0222923 0.022088 0.9556871 +1.2024 59 0.20116440 1029 0.0158691 0.015692 0.9715562 +1.3440 60 0.13746234 1029 0.0108439 0.010708 0.9824001 +1.4871 61 0.09013924 1029 0.0071107 0.007020 0.9895108 +1.6325 62 0.05670049 1029 0.0044729 0.004420 0.9939837 +1.7815 63 0.03420030 1029 0.0026979 0.002673 0.9966816 64 0.01977205 1029 0.0015597 0.001553 0.9982413 65 0.01095067 1029 0.0008639 0.000867 0.9991052 66 0.00580717 1029 0.0004581 0.000465 0.9995633 67 0.00294692 1029 0.0002325 0.000239 0.9997958 68 0.00143012 1029 0.0001128 0.000118 0.9999086 69 0.00066325 1029 0.0000523 0.000056 0.9999609 70 0.00029372 1029 0.0000232 0.000026 0.9999841 71 0.00012411 1029 0.0000098 0.000011 0.9999939 72 0.00004999 1029 0.0000039 0.000005 0.9999978 73 0.00001917 1029 0.0000015 0.000002 0.9999993 74 0.00000700 1029 0.0000006 0.000001 0.9999999 75 0.00000243 1029 0.0000002 0.000000 1.0000001 76 0.00000080 1029 0.0000001

PAGE 77

Statistics and Nonlinear Regression Allen R. Overman 75 1.0000002 77 0.00000025 1029 0.0000000 1.0000002 78 0.00000007 1029 79 0.00000002 1029 80 0.00000001 1029 C 12.67650601 1029 = 2100 Note the symmetry in the frequency distribution. x x F Z 14181 0 0907 7 2 1 2 1 2 erf 1 r = 0.9999959 000.50,0515.72 2 20515 7 00 50 exp 080010 0 2 exp 2 1 x x f ff00521.1000181.0 r = 0.999989

PAGE 78

Statistics and Nonlinear Regression Allen R. Overman 76 Case 8: 12 x 12 For this c ase n = 12 x 12 = 144 and x assumes values of 0, 1, 2, 3, 144 Corresponding values of xnc, are calculated from Eq. (1). Results are given in Table A24. Table A24 Frequency distribution for the square peg board with a 12 x 12 array. x c f F Z f 0.000000 0 1 0.000000 0.000000 1 144 0.000000 0.000000 0.000000 2 3 4 5 6 35 0.00000000 1042 0.000000 36 0.00000001 1042 0.000000 37 0.00000003 1042 0.000000 38 0.00000009 1042 0.00000000 39 0.00000025 1042 0.00000001 0.00000001 40 0.00000066 1042 0.00000003 0.00000004 41 0.00000168 1042 0.00000008 0.00000012 42 0.00000411 1042 0.00000018 0.000000 0.00000030 43 0.00000975 1042 0.00000044 0.000001 0.00000074 44 0.00002237 1042 0.00000100 0.000001 0.00000174 45 0.00004972 1042 0.00000223 0.000003

PAGE 79

Statistics and Nonlinear Regression Allen R. Overman 77 0.00000397 46 0.00010700 1042 0.00000480 0.000005 0.00000877 47 0.00022310 1042 0.00001000 0.000011 0.00001877 48 0.00045085 1042 0.00002022 0.000022 0.00003899 49 0.00088331 1042 0.00003961 0.000042 0.00007860 50 0.00167828 1042 0.00007526 0.000079 0.00015386 51 0.00309331 1042 0.00013871 0.000143 0.00029257 52 0.00553226 1042 0.00024808 0.000254 0.00054065 53 0.00960317 1042 0.00043062 0.000436 0.00097127 54 0.01618312 1042 0.00072568 0.000731 0.00169695 55 0.02648146 1042 0.00118747 0.001190 0.00288442 56 0.04208661 1042 0.00188723 0.001884 0.00477165 57 0.06497582 1042 0.00291362 0.002901 0.00768527 1.7166 58 0.09746373 1042 0.00437042 0.004344 0.01205569 1.5942 59 0.14206578 1042 0.00637045 0.006327 0.01842614 1.4738 60 0.20125985 1042 0.00902480 0.008962 0.02745094 1.3550 61 0.27714472 1042 0.01242760 0.012346 0.03987854 1.2371 62 0.37101632 1042 0.01663695 0.016540 0.05651549 1.1197 63 0.48291012 1042 0.02165444 0.021550 0.07816993 1.0023 64 0.61118313 1042 0.02740640 0.027306 0.10557633 0.8848 65 0.75222539 1042 0.03373095 0.033650 0.13930728 0.7668 66 0.90039099 1042 0.04037493 0.040329 0.17968221 0.6486 67 1.04821638 1042 0.04700365 0.047005 0.22668586 0.5302 68 1.18695090 1042 0.05322472 0.053282

PAGE 80

Statistics and Nonlinear Regression Allen R. Overman 78 0.27991058 0.4119 69 1.30736621 1042 0.05862433 0.058738 0.33853491 0.2941 70 1.40074951 1042 0.06281178 0.062973 0.40134669 0.1767 71 1.45993611 1042 0.06546580 0.065660 0.46681249 0.0593 72 1.48021300 1042 0.06637505 0.066581 0.53318754 +0.0593 73 1.45993611 1042 0.06546580 0.065660 0.59865334 +0.1767 74 1.40074951 1042 0.06281178 0.062973 0.66146512 +0.2941 75 1.30736621 1042 0.05862433 0.058738 0.72008945 +0.4119 76 1.18695090 1042 0.05322472 0.053282 0.77331417 +0.5302 77 1.04821638 1042 0.04700365 0.047005 0.82031782 +0.6486 78 0.90039099 1042 0.04037493 0.040329 0.86069275 +0.7668 79 0.75222539 1042 0.03373095 0.033650 0.89442370 +0.8848 80 0.61118313 1042 0.02740640 0.027306 0.92183010 +1.0023 81 0.48291012 1042 0.02165444 0.021550 0.94348454 +1.1197 82 0.37101632 1042 0.01663695 0.016540 0.96012149 +1.2371 83 0.27714472 1042 0.01242760 0.012346 0.97254909 +1.3550 84 0.20125985 1042 0.00902480 0.008962 0.98157389 +1.4738 85 0.14206578 1042 0.00637045 0.006327 0.98794434 +1.5942 86 0.09746373 1042 0.00437042 0.004344 0.99231476 +1.7166 87 0.06497582 1042 0.00291362 0.002901 0.99522838 88 0.04208661 1042 0.00188723 0.001884 0.99711561 89 0.02648146 1042 0.00118747 0.001190 0.99830308 90 0.01618312 1042 0.00072568 0.000731 0.99902876 91 0.00960317 1042 0.00043062 0.000436

PAGE 81

Statistics and Nonlinear Regression Allen R. Overman 79 0.99945938 92 0.00553226 1042 0.00024808 0.000254 0.99970746 93 0.00309331 1042 0.00013871 0.000143 0.99984617 94 0.00167828 1042 0.00007526 0.000079 0.99992143 95 0.00088331 1042 0.00003961 0.000042 0.99996104 96 0.00045085 1042 0.00002022 0.000022 0.99998126 97 0.00022310 1042 0.00001000 0.000011 0.99999126 98 0.00010700 1042 0.00000480 0.000005 0.99999606 99 0.00004972 1042 0.00000223 0.000003 0.99999829 100 0.00002237 1042 0.00000100 0.000001 0.99999929 101 0.00000975 1042 0.00000044 0.000001 0.99999973 102 0.00000411 1042 0.00000018 0.000000 0.99999991 103 0.00000168 1042 0.00000008 0.99999999 104 0.00000066 1042 0.00000003 1.00000002 105 0.00000025 1042 0.00000001 1.00000003 106 0.00000009 1042 0.00000000 107 0.00000003 1042 0.000000 108 0.00000001 1042 0.000000 109 0.00000000 1042 0.000000 143 144 144 1 C 22.30074520 1042 = 2144 ok Note the symmetry in the fre quency distribution.

PAGE 82

Statistics and Nonlinear Regression Allen R. Overman 80 xxFZ11801.049688.821212erf1 r = 0.9999985 00.72,47377.82 2 247377 8 00 72 exp 066581 0 2 exp 2 1 x x f ff00290.10000840.0 r = 0.9999960 Note that the total number of combinations is controlled by 2n. For n = 500 cells, this g ives C = 3.2733906 10150 total combinations !!! Is this a large number ? Compared to what ? Compared to molecules in a container of gas it may not be so big !! Remember Avogadros number from chemistry (6.0 1023) ? Or to the number of microorganisms in a living body. Statistics entered the field of physics when James Clerk Maxwell used it in his kinetic theory of gases in the mid 1800s, and was then developed further by Ludwig Boltzmann and Willard Gibbs. This led to the branch of physics known as statis tical mechanics These concepts were later incorporated in chemical kinetics by Henry Eyring in the absolute reaction rate theory.

PAGE 83

Statistics and Nonlinear Regression Allen R. Overman 81 Table A25 Spread of the distributions for peg boards with different number of holes ( n). Board n 2 2 2n Linear 2 0.9534 1.00 1.0489 0.9534 5 1.6286 2.50 1.5351 1.0300 6 1.8003 3.00 1.6664 1.0394 7 1.8892 3.50 1.8526 1.0098 8 2.0312 4.00 1.9692 1.0156 Triangular 3 1.2213 1.50 1.2282 0.9972 6 1.8003 3.00 1.6664 1.0394 10 2.2939 5.00 2.1797 1.0259 15 2.7711 7.50 2.7065 1.0119 21 3.2707 10.50 3.2103 1.0094 28 3.7810 14.00 3.7027 1.0105 Square 4 1.4391 2.00 1.3898 1.0176 9 2.1653 4.50 2.0782 1.0207 16 2.8703 8.00 2.7872 1.0148 25 3.5740 12.50 3.4975 1.0109 36 4.2786 18.00 4.2070 1.0085 49 4.9687 24.50 4.93087 1.0038 64 5.67576 32.00 5.63801 1.0033 81 6.38305 40.50 6.34493 1.0030 100 7.0907 50.00 7.05149 1.0028 144 8.49672 72.00 8.473858 1.00135 Note: 12n or n2 spread of distribution is approaching n 2 n center of distribution is equal to n/2 References Abramowitz, M. and I.A. Stegun. 1965. Handbook of Mathematical Functions. Dover Publications. New York, NY. Eigen, M. and R. Winkler. 1993. Laws of the Game: How the Principles of Nature Govern Chance. Princeton University Press. Princeton, NJ. Polster, B 2004. Q.E.D.: Beauty in Mathematical Proof. Walker & Co. New York, NY. Ruhla, C. 1992. The Physics of Chance: From Blaise Pascal to Niels Bohr. Oxford University Press. New York, NY. Watkins, M. 2000. Useful Mathematical and Physical Formulae. Walker & C o. New York, NY.

PAGE 84

Statistics and Nonlinear Regression Allen R. Overman 82 Table A26 Correlation between discrete ( f ) and continuous Gaussian ( f ) distributions. Pegboard Cells Regression Equation Correlation Linear 1 x 2 f f 2848 1 1045 0 1 1 x 3 ff1404.103885.0 1.0000 1 x 4 ff1052.10226.0 0.9971 1 x 5 f f 0882 1 01543 0 0.99723 1 x 6 f f 0809 1 01187 0 0.99730 1 x 7 f f 0689 1 01188 0 0.999198 1 x 8 ff0589.1008700.0 0.999084 Triangular n = 3 ff1432.103950.0 1.000000 n = 6 ff0809.101187.0 0.99730 n = 10 ff0383.100352.0 0.99894 n = 15 f f 02423 1 002046 0 0.999637 n = 21 ff0183.100135.0 0.99980 n = 28 f f 01674 1 000995 0 0.99985 Square 2 x 2 ff1053.102262.0 0.9971 3 x 3 ff0369.100373.0 0.99895 4 x 4 ff0248.100194.0 099962 5 x 5 ff017789.1001129.0 0.999827 6 x 6 f f 01356 1 000723 0 0.999909 7 x 7 f f 00727 1 000345 0 0.999965 8 x 8 f f 00622 1 000252 0 0.999978 9 x 9 f f 00595 1 000232 0 0.999985 10 x 10 ff00521.1000181.0 0.999989 12 x 12 ff00290.10000840.0 0.9999960 As number of cells increases, the intercept approaches 0 and slope approaches 1, which means that fit of the continuous Gaussian distribution to discrete distribution improves as the number of cells increases.

PAGE 85

Statistics and Nonlinear Regression Allen R. Overman 83 ABE 6933 Special Topics Mathematical and Statistical Characteristics of Nonlinear Regression Models A. R. Overman I. Elemen ts of Probability and Calculus A. Arithmetic the process of counting B. Natural numbers positive integers ( ,2,1,0 ) C. Rational numbers ratio of two integers ( ,3/2,3/1,2/1,,1/2,1/1 ) D. Irrational numbers (such as e, 2 etc.) E. Complex numbers z = x + i y with i = 1 F. Binomial theorem and Pascals triangle (a + b)0 = 1 (a + b)1 = a1 + b1 (a + b)2 = a2 + 2ab + b2 (a + b)3 = a3 + 3a2b + 3ab2 + b3 (a + b)4 = a4 + 4a3b + 6a2b2 + 4ab3 + b4 (a + b)5 = a5 + 5a4b + 10a3b2 + 10a2b3 + 5ab4 + b5 (a + b)6 = a6 + 6a5b + 15a4b2 + 20a3b3 + 15a2b4 + 6ab5 + b6 (a + b)7 = a7 + 7a6b + 21a5b2 + 35a4b3 + 35a3b4 + 21a2b5 + 7ab 6 + b7 (a + b)8 = a8 + 8a7b + 28a6b2 + 56a5b3 + 70a4b4 + 56a3b5 + 28a2b6 + 8ab7 + b8 (a + b)9 = a9 + 9a8b + 36a7b2 + 84a6b3 + 126a5b4 + 126a4b5 + 84a3b6 + 36a2b7 + 9ab8 + b9 Note symmetry in the distribution of coefficients for each expansion.

PAGE 86

Statistics and Nonlinear Regression Allen R. Overman 84 Pascals triangle for binomial coefficients 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 1 7 21 35 35 21 7 1 1 8 28 56 70 56 28 8 1 1 9 36 84 126 126 84 36 9 1 1 10 45 120 210 252 210 120 45 10 1 1 11 55 165 330 462 462 330 165 55 11 1 1 12 66 220 495 792 924 792 495 220 66 12 1 Note the pattern in the coefficients, including symmetry. G. Frequency distributions 1. Discrete distribution Consider the problem of a peg board. This is a two state system a cell (hole) is either filled or empty. Each cell holds one and only one object (peg), which can be viewed as a type of exclusion principle Define n as the total number of cells and x as the number of filled cells (pegs). Cells (hole s) are indistinguishable (all alike), as are the objects (pegs). Order of filling the cells is irrelevant. Note that a peg board can be linear, triangular, rectangular (Eigen and Winkler, 1993, p. 40; Polster, 2004, p. 33), or even 3dimensional. The numbe r of distinguishable combinations which are possible for each x, xnc, can be calculated from (Ruhla, 1992, p. 18; Watkins, 2000, p. 22) )!(!!,xnxnxnc (1)

PAGE 87

Statistics and Nonlinear Regression Allen R. Overman 85 where n n 3 2 1 and is called n factorial. Note that n can assume positive integers ( 3 2 1 n ) and x can also assume positive integers ( n x , 2 1 0 ). For small values of n it is easy to estimate c by intuition, but larger n calculations of c are best performed on a pocket calculato r or computer with the algorithm for computations (Eq. (1)) built in. The total number of combinations C for the system is defined as the sum of c values for all values of x and can be calculated from C = 2n. The frequency distribution of c values is then calculated from f = c/C. Cumulative frequency is calculated from the cumulative sum fF (2) so that F is normalized 10F It should be noted that F forms a discrete set of numbers for a particular case. 2. Cont inuous distribution The next step is to compare the discrete distribution to a continuous Gaussian distribution where x is considered a continuous variable and the cumulative distribution is described by 2 erf 1 2 1 x F (3) where and are the mean and spread of the distribution. The error function is defined by 202)exp(22erfxduux (4) where )exp(2u represents the Gaussian distribution (bell shaped curve). Values of the erf can be obtained from mathematical tables (cf. Abramowitz and Stegun, 1965, chp. 7). Some properties of the error function should be noted: erf (0) = 0, erf ( ) = 1, erf (x ) = erf (+ x ), erf ( ) = 1 Equation (3) can be rearranged to the linear form xFZ21212erf1 (5) where erf 1 is the inverse error function. For example, 00.1)8427.0(erf1 Linear regression of Z vs. x leads to values of the parameters and With these parameters now known the freq uency distribution for f vs. x can be calculated for the continuous distribution from

PAGE 88

Statistics and Nonlinear Regression Allen R. Overman 86 22exp21xf (6) The procedure can now be applied to a linear peg board, triangular peg board, and square peg board. It can even be applied to a 3dimensio nal system. This analysis falls within a branch of mathematics known as group theory Values of the error function can be calculated from the series approximation (Abramowitz and Stegun, 1965, p. 299) 4 4 3 2] 078108 0 000972 0 230389 0 278393 0 1 [ 1 1 erf x x x x x (7) for 8 1 0 x For the case where erf x is given, the inverse erf 1 and therefore x can be obtained on a scientific calculator or computer using the solver routine Note that for the case F < 0.5 and 2F 1 < 0 (negative) the procedure is to change the value from t o +, solve for the inverse by Eq. (7) and change the sign from + x to x Equation (7) does not work directly for x because the power series in Eq. (7) is not symmetric. H. Symmetry and conservation principle In all of the discrete and continuous Gauss ian distributions we note symmetry in the distributions around a mean point. A mathematical consequence of this property is that something is conserved (remains constant) in the system. Note that the number of filled cells is defined by x Since this is a two state (binary) system (cells are either empty of filled), it follows that the number of unfilled cells is n x The total capacity of the system is the sum of filled and unfilled cells so that total capacity is = x + n x = n While this is obvious f or our case, it illustrates the connection between symmetry and conservation. This property turns out to be very important in the various models of physics (including mechanics, electromagnetism, relativity, and quantum mechanics). It also shows up in chem istry and biology.

PAGE 89

Statistics and Nonlinear Regression Allen R. Overman 87 Gaussian Distribution The Gauss differential equation is given by kxydxdy with y = A at x = 0 (1) where y ) 0 ( y is a continuous function of x ( )x and k is the distribution coefficient. 1. Obtain the integral solution to Eq. (1). 2. Sketch the form of the solution y vs. x on linear graph paper. 3. Perform the 2nd derivative of y on x to obtain the inflection points at x Write the constant k in terms of 4. Evaluate the constant A by normalizing the integral 1 ydx (2) 5. Write the resulting solution y in terms of variable x and parameter 6. Obta in the cumulative probability distribution F 22 erf 1 2 1xx ydx F (3) where the error function is defined by zduuz02exp2erf (4) 7. Calculate and plot F vs. 2 / x on linear graph paper.

PAGE 90

Statistics and Nonlinear Regression Allen R. Overman 88 Solutions Note : at x = 0, dy/dx = 0 (maximum or minimum) 1. 2221exp21kxAydxkdxkxydy Solution is symmetric around x = 0 3. 2 2 2 2 21 0 2 1 exp 1 k kx kx kA dx y d 2 22 expx A y at x = 0, d2y/dx2 < 0 (maximum) 4. 2 1 1 2 exp 2 2 2 exp 2 2 exp2 2 2 2 A A du u A x d x A dx x A ydx A is chosen so that the distribut ion is normalized, hence the term normal distribution. 5. 22exp21xy 6. 2erf121exp2121exp21expexp1exp120220222020222xduuduuduuduuduuydxFxxxxx where 2 0 2exp 2 2 erfxdu u x This ties the cumulative frequency distribution to the error function of mathematical physics. Note the ch aracteristics of the error function: erf (0) = 0, erf ( ) = 1, erf ( x ) = erf (+ x ). It follows that F is bounded by 10F Note also that F is a well behaved function.

PAGE 91

Statistics and Nonlinear Regression Allen R. Overman 89 Solve Eq. (1) by the power series met hod. kxy dx dy with y = A at x = 0 (1) Assume that the solution is given by a power series 77665544332210xaxaxaxaxaxaxaay (2) The first derivative is given by 6 7 5 6 4 5 3 4 2 3 2 17 6 5 4 3 2 x a x a x a x a x a x a a dx dy (3) Substitution of Eqs. (2) and (3) into Eq. (1) leads to 7665544332210665544332210675645342321765432xkaxkaxkaxkaxkaxkaxkaxaxaxaxaxaxaakxxaxaxaxaxaxaa (4) Equating like coefficients in Eq. (4) gives the recursion relations 6426054240320034635022413021akkaakaaakkaakaakaaa It follows that the solution is given by

PAGE 92

Statistics and Nonlinear Regression Allen R. Overman 90 6 3 4 2 2 0 6 0 3 4 0 2 2 0 0 6 6 5 5 4 4 3 3 2 2 1 06 4 2 4 2 2 1 6 4 2 0 4 2 0 2 0 x k x k x k a x a k x a k x ka a x a x a x a x a x a x a a y (5) Now use the substitution 22xk (6) Then Eq. (5) becomes 2 0 0 3 2 0 3 2 02 exp exp 3 2 1 3 2 1 2 1 1 x k a a a a y (7) The constant 0a is evaluated from the boundary condition, which leads to 22 exp x k A y (8) This is the famous Gaussian distribution centered at x = 0. It remains to determine k in terms of the variance of the distribution and A to normalize the distribution. Check: kxy x k kxA dx dy 22 exp correct (9)

PAGE 93

Statistics and Nonlinear Regression Allen R. Overman 91 Probability distributions with dice. Consider a single die with 6 faces numbered 1 through 6. Assume that each number is equally likely. The frequency distribution can now be calculated. Table A27 Frequency distribution for a single die x S c f F Z Z f 0.00000000 1 1 1 0.16666667 0.093758 0.16666667 0.684656 0.669385 2 2 1 0.16666667 0.146760 0.33333334 0.304151 0.334693 3 3 1 0.16666667 0.183615 0.50000001 0.000000 0.000000 4 4 1 0.16666667 0.183615 0.66666668 + 0.304151 +0.334693 5 5 1 0.16666667 0.146760 0.83333335 +0.684656 +0.669385 6 6 1 0.16666667 0.093758 1.00000002 C 6 SSFZ334693.0171424.121212erf1 r = 0.998961 5000.3,9878.22 2 29878 2 5000 3 exp 18883 0 2 exp 2 1 S S f No correlation between f and f since f = constant = 1/6.

PAGE 94

Statistics and Nonlinear Regression Allen R. Overman 92 Table A28 Fre quency distribution for two di c e x S c f F Z Z f c 0.00000000 1,1 2 1 0.02777778 0.020242 0.73 0.02777778 1.3513 1.2999 1,2; 2,1 3 2 0.05555556 0.042894 1.54 0.08333334 0.9781 1.0110 1,3; 2,2; 3,1 4 3 0.08333333 0.076922 2.77 0.16666667 0.6847 0.7222 1,4; 2,3; 3,2; 4,1 5 4 0.11111111 0.116745 4.20 0.27777778 0.4164 0.4333 1,5; 2;4; 3,3; 4,2; 5,1 6 5 0.13888889 0.149951 5.40 0.41666667 0.1490 0.1444 1,6; 2,5; 3,4; 4,3; 5,2; 6,1 7 6 0.16666667 0.163000 5.87 0.58333334 +0.1490 +0.1444 2,6; 3,5; 4,4; 5,3; 6,2 8 5 0.13888889 0.149951 5.40 0.72222223 +0.4164 +0.4333 3,6; 4,5; 5,4; 6,3 9 4 0.11111111 0.116745 4.20 0.83333334 +0.6847 +0.7222 4,6; 5,5; 6,4 10 3 0.08333333 0.076922 2.77 0.91666667 +0.9781 +1.0110 5,6; 6,5 11 2 0.05555556 0.042894 1.54 0.97222223 +1.3513 +1.2999 6,6 12 1 0.02777778 0.020242 0.73 1.00000001 C 36 = 62 S S F Z 28885 0 02195 2 2 1 2 1 2 erf 1 r = 0.999211 0000.7,4620.32 224620.30000.7exp1630.02exp21SSf f f 1360 1 0145 0 r = 0.99319 fc36 The two dice problem is discussed by Speyer (1994, p. 62)

PAGE 95

Statistics and Nonlinear Regression Allen R. Overman 93 Table A29 Fre quency distribution for three di c e x S c f F Z Z f 0.000000 1,1,1 3 1 0.004630 0.005418 0.004630 1,1,2; 1,2,1; 2,1,1 4 3 0.013889 0.012058 0.018519 1.4726 1.4344 1,1,3; 1,2,2; 1,3,1; 2,1,2; 5 6 0.027778 0.023940 2,2,1; 3,1,1 0.046297 1.1878 1.1953 1,1,4; 1,2,3; 1,3,2; 1,4,1; 6 10 0.046296 0.042397 2,1,3; 2,2,2; 2,3,1; 3,1,2; 0.092593 0.9372 0.9563 3,2,1; 4,1,1 1,1,5; 1,2,4; 1,3,3; 1,4,2; 7 15 0.069444 0.066973 1,5,1; 2,1,4; 2,2,3; 2,3,2; 0.162037 0.6979 0.7172 2,4,1; 3,1,3; 3,2,2; 3,3,1; 4,1,2; 4,2,1; 5,1,1 1,1,6; 1,2,5; 1,3,4; 1,4,3; 8 21 0.097222 0.094367 1,5,2; 1,6,1; 2,1,5; 2,2,4; 0.259259 0.4563 0.4781 2,3,3; 2,4,2; 2,5,1; 3,1,4; 3,2,3; 3,3,2; 3,4,1; 4,1,3; 4,2,2; 4,3,1; 5,1,2; 5,2,1; 6,1,1 1,2,6; 1,3,5; 1,4,4; 1,5,3; 9 25 0.115741 0.118605 1,6,2; 2,1,6; 2,2,5; 2,3,4; 0.375000 0.2251 0.2391 2,4,3; 2,5,2; 2,6,1; 3,1,5; 3,2,4; 3,3,3; 3,4,2; 3,5,1; 4,1,4; 4,2,3; 4,3,2; 4,4,1; 5,1,3; 5,2,2; 5,3,1; 6,1,2; 6,2,1 1,3,6; 1,4,5; 1,5,4; 1,6,3; 10 27 0.125000 0.132967 2,2,6; 2,3,5; 2,4,4; 2,5,3; 0.500000 0.0000 0.0000 2,6,2; 3,1,6; 3,2,5; 3,3,4; 3,4,3; 3,5,2; 3,6,1; 4,1,5; 4,2,4; 4,3,3; 4,4,2; 4,5,1; 5,1,4; 5,2,3; 5,3,2; 5,4,1; 6,1,3; 6,2,2; 6,3,1 1,4,6; 1,5,5; 1,6,4; 2,3,6; 11 27 0.125000 0.132967 2,4,5; 2,5,4; 2,6,3; 3,2,6; 0.625000 +0.2251 +0.2391 3,3,5; 3,4,4; 3,5,3; 3,6,2;

PAGE 96

Statistics and Nonlinear Regression Allen R. Overman 94 4,1,6; 4,2,5; 4,3,4; 4,4,3; 4,5,2; 4,6,1; 5,1,5; 5,2,4; 5,3,3; 5,4,2; 5,5,1; 6,1,4; 6,2,3; 6,3,2; 6,4,1 1,5,6; 1,6,5; 2,4,6; 2,5,5; 12 25 0.115741 0.118605 2,6,4; 3,3,6; 3,4,5; 3,5,4; 0.740741 +0.4563 +0.4781 3,6,3; 4,2,6; 4,3,5; 4,4,4; 4,5,3; 4,6,2; 5,1,6; 5,2,5; 5,3,4; 5,4,3; 5,5,2; 5,6,1; 6,1,5; 6,2,4; 6,3,3; 6,4,2; 6,5,1 1,6,6; 2,5,6; 2,6,5; 3,4,6; 13 21 0.097222 0.094367 3,5,5; 3,6,4; 4,3,6; 4,4,5; 0.837963 +0.6979 +0.7172 4,5,4; 4,6,3; 5,2,6; 5,3,5; 5,4,4; 5,5,3; 5,6,2; 6,1,6; 6,2,5; 6,3,4; 6,4,3; 6,5,2; 6,6,1 2,6,6; 3,5,6; 3,6,5; 4,4,6; 14 15 0.069444 0.066973 4,5,5; 4,6,4; 5,3,6; 5,4,5; 0.907407 +0.9372 +0.9563 5,5,4; 5,6,3; 6,2,6; 6,3,5; 6,4,4; 6,5,3; 6,6,2 3,6,6; 4,5,6; 4,6,5; 5,4,6; 15 10 0.046296 0.042397 5,5,5; 5,6,4; 6,3,6; 6,4,5; 0.953703 +1.1878 +1.1953 6,5,4; 6,6,3; 4,6,6; 5,5,6; 5,6,5; 6,4,6; 16 6 0.027778 0.023940 6,5,5; 6,6,4 0.981481 +1.4726 +1.4344 5,6,6; 6,5,6; 6,6,5 17 3 0.013889 0.012058 0.995370 6,6,6 18 1 0.004630 0.005418 1.000000 C 216 = 63 SSFZ23906.05102.221212erf1 r = 0.999719 5001.10,1830.42

PAGE 97

Statistics and Nonlinear Regression Allen R. Overman 95 221830.4500.10exp13488.02exp21SSf ff0752.100590.0 r = 0.99800 For three dice t he discrete distribution is closely approximated by the continuous Gaussian distribution. I conclude that the peg board is a much simpler illustration of frequency distributions than dice.

PAGE 98

Statistics and Nonlinear Regression Allen R. Overman 96 Table A30 Fre quency distribution for four di c e x S c f 1,1,1,1 4 1 0.000772 1,1,1,2; 1,1,2,1; 1,2,1,1; 5 4 0.003086 2,1,1,1 1,1,1,3; 1,1,2,2; 1,1,3,1; 6 10 0.007716 1,2,1,2; 1,2,2,1; 1,3,1,1; 2,1,1,2; 2,1,2,1; 2,2,1,1; 3,1,1,1 1,1,1,4; 1,1,2,3; 1,1,3,2; 7 20 0.015432 1,1,4,1; 1,2,1,3; 1,2,2,2; 1,2,3,1; 1,3,1,2; 1,3,2,1; 1,4,1,1; 2,1,1,3; 2,1,2,2; 2,1,3,1; 2,2,1,2; 2,2,2,1; 2,3,1,1; 3,1,1,2; 3,1,2,1; 3,2,1,1; 4,1,1,1 1,1,1,5; 1,1,2,4; 1,1,3,3; 8 35 0.027006 1,1,4,2; 1,1,5,1; 1,2,1,4; 1,2,2,3; 1,2,3,2; 1,2,4,1; 1,3,1,3; 1,3,2,2; 1,3,3,1; 1,4,1,2; 1,4,2,1; 1,5,1,1; 2,1,1,4; 2,1,2,3; 2,1,3,2; 2,1,4,1; 2,2,1,3; 2,2,2,2; 2,2,3,1; 2,3,1,2; 2,3,2,1; 2,4,1,1; 3,1,1,3; 3,1,2,2; 3,1,3,1; 3,2,1,2; 3,2,2,1; 3,3,1,1; 4,1,1,2; 4,1,2,1; 4,2,1,1; 5,1,1,1 1,1,1,6; 1,1,2,5; 1,1,3,4; 9 56 0.043210 1,1,4,3; 1,1,5,2; 1,1,6,1; 1,2,1,5; 1,2,2,4; 1,2,3,3; 1,2,4,2; 1,2,5,1; 1,3,1,4; 1,3,2,3; 1,3,3,2; 1,3,4,1; 1,4,1,3; 1,4,2,2; 1,4,3,1; 1,5,1,2; 1,5,2,1; 1,6,1,1; 2,1,1,5; 2,1,2,4; 2,1,3,3; 2,1,4,2; 2,1,5,1; 2,2,1,4; 2,2,2,3; 2,2,3,2; 2,2,4,1; 2,3,1,3; 2,3,2,2; 2,3,3,1;

PAGE 99

Statistics and Nonlinear Regression Allen R. Overman 97 2,4,1,2; 2,4,2,1; 2,5,1,1; 3,1,1,4; 3,1,2,3; 3,1,3,2; 3,1,4,1; 3,2,1,3; 3,2,2,2; 3,2,3,1; 3,3,1,2; 3,3,2,1; 3,4,1,1; 4,1,1,3; 4,1,2,2; 4,1,3,1; 4,2,1,2; 4,2,2,1; 4,3,1,1; 5,1,1,2; 5,1,2,1; 5,2,1,1; 6,1,1,1 1,1,2,6; 1,1,3,5; 1,1,4,4; 10 80 0.061728 1,1,5,3; 1,1,6,2; 1,2,1,6; 1,2,2,5; 1,2,3,4; 1,2,4,3; 1,2,5,2; 1,2,6,1; 1,3,1,5; 1,3,2,4; 1,3,3,3; 1,3,4,2; 1,3,5,1; 1,4,1,4; 1,4,2,3; 1,4,3,2; 1,4,4,1; 1,5,1,3; 1,5,2,2; 1,5,3,1; 1,6,1,2; 1,6,2,1; 2,1,1,6; 2,1,2,5; 2,1,3,4; 2,1,4,3; 2,1,5,2; 2,1,6,1; 2,2,1,5; 2,2,2,4; 2,2,3,3; 2,2,4,2; 2,2,5,1; 2,3,1,4; 2,3,2,3; 2,3,3,2; 2,3,4,1; 2,4,1,3; 2,4,2,2; 2,4,3,1; 2,5,1,2; 2,5,2,1; 2,6,1,1; 3,1,1,5; 3,1,2,4; 3,1,3,3; 3,1,4,2; 3,1,5,1; 3,2,1,4; 3,2,2,3; 3,2,3,2; 3,2,4,1; 3,3,1,3; 3,3,2,2; 3,3,3,1; 3,4,1,2; 3,4,2,1; 3,5,1,1; 4,1,1,4; 4,1,2,3; 4,1,3,2; 4,1,4,1; 4,2,1,3; 4,2,2,2; 4,2,3,1; 4,3,1,2; 4,3,2,1; 4,4,1,1; 5,1,1,3; 5,1,2,2; 5,1,3,1; 5,2,1,2; 5,2,2,1; 5,3,1,1; 6,1,1,2; 6,1,2,1; 6,2,1,1 1,1,3,6; 1,1,4,5; 1,1,5,4; 11 104 0.080247 1,1,6,3; 1,2,2,6; 1,2,3,5; 1,2,4,4; 1,2,5,3; 1,2,6,2; 1,3,1,6; 1,3,2,5; 1,3,3,4; 1,3,4,3; 1,3,5,2; 1,3,6,1; 1,4,1,5; 1,4,2,4; 1,4,3,3; 1,4,4,2; 1,4,5,1; 1,5,1,4; 1,5,2,3; 1,5,3,2; 1,5,4,1; 1,6,1,3; 1,6,2,2; 1,6,3,1;

PAGE 100

Statistics and Nonlinear Regression Allen R. Overman 98 2,1,2,6; 2,1,3,5; 2,1,4,4; 2,1,5,3; 2,1,6,2; 2,2,1,6; 2,2,2,5; 2,2,3,4; 2,2,4,3; 2,2,5,2; 2,2,6,1; 2,3,1,5; 2,3,2,4; 2,3,3,3; 2,3,4,2; 2,3,5,1; 2,4,1,4; 2,4,2,3; 2,4,3,2; 2,4,4,1; 2,5,1,3; 2,5,2,2; 2,5,3,1; 2,6,1,2; 2,6,2,1; 3,1,1,6; 3,1,2,5; 3,1,3,4; 3,1,4,3; 3,1,5,2; 3,1,6,1; 3,2,1,5; 3,2,2,4; 3,2,3,3; 3,2,4,2; 3,2,5,1; 3,3,1,4; 3,3,2,3; 3,3,3,2; 3,3,4,1; 3,4,1,3; 3,4,2,2; 3,4,3,1; 3,5,1,2; 3,5,2,1; 3,6,1,1; 4,1,1,5; 4,1,2,4; 4,1,3,3; 4,1,4,2; 4,1,5,1; 4,2,1,4; 4,2,2,3; 4,2,3,2; 4,2,4,1; 4,3,1,3; 4,3,2,2; 4,3,3,1; 4,4,1,2; 4,4,2,1; 4,5,1,1; 5,1,1,4; 5,1,2,3; 5,1,3,2; 5,1,4,1; 5,2,1,3; 5,2,2,2; 5,2,3,1; 5,3,1,2; 5,3,2,1; 5,4,1,1; 6,1,1,3; 6,1,2,2; 6,1,3,1; 6,2,1,2; 6,2,2,1; 6,3,1,1 1,1,4,6; 1,1,5,5; 1,1,6,4; 12 125 0.096451 1,2,3,6; 1,2,4,5; 1,2,5,4; 1,2,6,3; 1,3,2,6; 1,3,3,5; 1,3,4,4; 1,3,5,3; 1,3,6,2; 1,4,1,6; 1,4,2,5; 1,4,3,4; 1,4,4,3; 1,4,5,2; 1,4,6,1; 1,5,1,5; 1,5,2,4; 1,5,3,3; 1,5,4,2; 1,5,5,1; 1,6,1,4; 1,6,2,3; 1,6,3,2; 1,6,4,1; 2,1,3,6; 2,1,4,5; 2,1,5,4; 2,1,6,3; 2,2,2,6; 2,2,3,5; 2,2,4,4; 2,2,5,3; 2,2,6,2; 2,3,1,6; 2,3,2,5; 2,3,3,4; 2,3,4,3; 2,3,5,2; 2,3,6,1; 2,4,1,5; 2,4,2,4; 2,4,3,3; 2,4,4,2; 2,4,5,1; 2,5,1,4; 2,5,2,3; 2,5,3,2; 2,5,4,1; 2,6,1,3; 2,6,2,2; 2,6,3,1; 3,1,2,6; 3,1,3,5; 3,1,4,4;

PAGE 101

Statistics and Nonlinear Regression Allen R. Overman 99 3,1,5,3; 3,1,6,2; 3,2,1,6; 3,2,2,5; 3,2,3,4; 3,2,4,3; 3,2,5,2; 3,2,6,1; 3,3,1,5; 3,3,2,4; 3,3,3,3; 3,3,4,2; 3,3,5,1; 3,4,1,4; 3,4,2,3; 3,4,3,2; 3,4,4,1; 3,5,1,3; 3,5,2,2; 3,5,3,1; 3,6,1,2; 3,6,2,1; 4,1,1,6; 4,1,2,5; 4,1,3,4; 4,1,4,3; 4,1,5,2; 4,1,6,1; 4,2,1,5; 4,2,2,4; 4,2,3,3; 4,2,4,2; 4,2,5,1; 4,3,1,4; 4,3,2,3; 4,3,3,2; 4,3,4,1; 4,4,1,3; 4,4,2,2; 4,4,3,1; 4,5,1,2; 4,5,2,1; 4,6,1,1; 5,1,1,5; 5,1,2,4; 5,1,3,3; 5,1,4,2; 5,1,5,1; 5,2,1,4; 5,2,2,3; 5,2,3,2; 5,2,4,1; 5,3,1,3; 5,3,2,2; 5,3,3,1; 5,4,1,2; 5,4,2,1; 5,5,1,1; 6,1,1,4; 6,1,2,3; 6,1,3,2; 6,1,4,1; 6,2,1,3; 6,2,2,2; 6,2,3,1; 6,3,1,2; 6,3,2,1; 6,4,1,1 1,1,5,6; 1,1,6,5; 1,2,4,6; 13 140 0.108025 1,2,5,5; 1,2,6,4; 1,3,3,6; 1,3,4,5; 1,3,5,4; 1,3,6,3; 1,4,2,6; 1,4,3,5; 1,4,4,4; 1,4,5,3; 1,4,6,2; 1,5,1,6; 1,5,2,5; 1,5,3,4; 1,5,4,3; 1,5,5,2; 1,5,6,1; 1,6,1,5; 1,6,2,4; 1,6,3,3; 1,6,4,2; 1,6,5,1; 2,1,4,6; 2,1,5,5; 2,1,6,4; 2,2,3,6; 2,2,4,5; 2,2,5,4; 2,2,6,3; 2,3,2,6; 2,3,3,5; 2,3,4,4; 2,3,5,3; 2,3,6,2; 2,4,1,6; 2,4,2,5; 2,4,3,4; 2,4,4,3; 2,4,5,2; 2,4,6,1; 2,5,1,5; 2,5,2,4; 2,5,3,3; 2,5,4,2; 2,5,5,1; 2,6,1,4; 2,6,2,3; 2,6,3,2; 2,6,4,1; 3,1,3,6; 3,1,4,5; 3,1,5,4; 3,1,6,3; 3,2,2,6; 3,2,3,5; 3,2,4,4; 3,2,5,3; 3,2,6,2; 3,3,1,6; 3,3,2,5; 3,3,3,4; 3,3,4,3; 3,3,5,2;

PAGE 102

Statistics and Nonlinear Regression Allen R. Overman 100 3,3,6,1; 3,4,1,5; 3,4,2,4; 3,4,3,3; 3,4,4,2; 3,4,5,1; 3,5,1,4; 3,5,2,3; 3,5,3,2; 3,5,4,1; 3,6,1,3; 3,6,2,2; 3,6,3,1; 4,1,2,6; 4,1,3,5; 4,1,4,4; 4,1,5,3; 4,1,6,2; 4,2,1,6; 4,2,2,5; 4,2,3,4; 4,2,4,3; 4,2,5,2; 4,2,6,1; 4,3,1,5; 4,3,2,4; 4,3,3,3; 4,3,4,2; 4,3,5,1; 4,4,1,4; 4,4,2,3; 4,4,3,2; 4,4,4,1; 4,5,1,3; 4,5,2,2; 4,5,3,1; 4,6,1,2; 4,6,2,1; 5,1,1,6; 5,1,2,5; 5,1,3,4; 5,1,4,3; 5,1,5,2; 5,1,6,1; 5,2,1,5; 5,2,2,4; 5,2,3,3; 5,2,4,2; 5,2,5,1; 5,3,1,4; 5,3,2,3; 5,3,3,2; 5,3,4,1; 5,4,1,3; 5,4,2,2; 5,4,3,1; 5,5,1,2; 5,5,2,1; 5,6,1,1; 6,1,1,5; 6,1,2,4; 6,1,3,3; 6,1,4,2; 6,1,5,1; 6,2,1,4; 6,2,2,3; 6,2,3,2; 6,2,4,1; 6,3,1,3; 6,3,2,2; 6,3,3,1; 6,4,1,2; 6,4,2,1; 6,5,1,1 1,1,6,6; 1,2,5,6; 1,2,6,5; 14 146 0.112654 1,3,4,6; 1,3,5,5; 1,3,6,4; 1,4,3,6; 1,4,4,5; 1,4,5,4; 1,4,6,3; 1,5,2,6; 1,5,3,5; 1,5,4,4; 1,5,5,3; 1,5,6,2; 1,6,1,6; 1,6,2,5; 1,6,3,4; 1,6,4,3; 1,6,5,2; 1,6,6,1; 2,1,5,6; 2,1,6,5; 2,2,4,6; 2,2,5,5; 2,2,6,4; 2,3,3,6; 2,3,4,5; 2,3,5,4; 2,3,6,3; 2,4,2,6; 2,4,3,5; 2,4,4,4; 2,4,5,3; 2,4,6,2; 2,5,1,6; 2,5,2,5; 2,5,3,4; 2,5,4,3; 2,5,5,2; 2,5,6,1; 2,6,1,5; 2,6,2,4; 2,6,3,3; 2,6,4,2; 2,6,5,1; 3,1,4,6; 3,1,5,5; 3,1,6,4; 3,2,3,6; 3,2,4,5; 3,2,5,4; 3,2,6,3; 3,3,2,6; 3,3,3,5; 3,3,4,4; 3,3,5,3; 3,3,6,2; 3,4,1,6; 3,4,2,5;

PAGE 103

Statistics and Nonlinear Regression Allen R. Overman 101 3,4,3,4; 3,4,4,3; 3,4,5,2; 3,4,6,1; 3,5,1,5; 3,5,2,4; 3,5,3,3; 3,5,4,2; 3,5,5,1; 3,6,1,4; 3,6,2,3; 3,6,3,2; 3,6,4,1; 4,1,3,6; 4,1,4,5; 4,1,5,4; 4,1,6,3; 4,2,2,6; 4,2,3,5; 4,2,4,4; 4,2,5,3; 4,2,6,2; 4,3,1,6; 4,3,2,5; 4,3,3,4; 4,3,4,3; 4,3,5,2; 4,3,6,1; 4,4,1,5; 4,4,2,4; 4,4,3,3; 4,4,4,2; 4,4,5,1; 4,5,1,4; 4,5,2,3; 4,5,3,2; 4,5,4,1; 4,6,1,3; 4,6,2,2; 4,6,3,1; 5,1,2,6; 5,1,3,5; 5,1,4,4; 5,1,5,3; 5,1,6,2; 5,2,1,6; 5,2,2,5; 5,2,3,4; 5,2,4,3; 5,2,5,2; 5,2,6,1; 5,3,1,5; 5,3,2,4; 5,3,3,3; 5,3,4,2; 5,3,5,1; 5,4,1,4; 5,4,2,3; 5,4,3,2; 5,4,4,1; 5,5,1,3; 5,5,2,2; 5,5,3,1; 5,6,1,2; 5,6,2,1; 6,1,1,6; 6,1,2,5; 6,1,3,4; 6,1,4,3; 6,1,5,2; 6,1,6,1; 6,2,1,5; 6,2,2,4; 6,2,3,3; 6,2,4,2; 6,2,5,1; 6,3,1,4; 6,3,2,3; 6,3,3,2; 6,3,4,1; 6,4,1,3; 6,4,2,2; 6,4,3,1; 6,5,1,2; 6,5,2,1; 6,6,1,1 1,2,6,6; 1,3,5,6; 1,3,6,5; 15 140 0.108025 1,4,4,6; 1,4,5,5; 1,4,6,4; 1,5,3,6; 1,5,4,5; 1,5,5,4; 1,5,6,3; 1,6,2,6; 1,6,3,5; 1,6,4,4; 1,6,5,3; 1,6,6,2; 2,1,6,6; 2,2,5,6; 2,2,6,5; 2,3,4,6; 2,3,5,5; 2,3,6,4; 2,4,3,6; 2,4,4,5; 2,4,5,4; 2,4,6,3; 2,5,2,6; 2,5,3,5; 2,5,4,4; 2,5,5,3; 2,5,6,2; 2,6,1,6; 2,6,2,5; 2,6,3,4; 2,6,4,3; 2,6,5,2; 2,6,6,1; 3,1,5,6; 3,1,6,5; 3,2,4,6; 3,2,5,5; 3,2,6,4; 3,3,3,6; 3,3 4,5; 3,3,5,4; 3,3,6,3; 3,4,2,6; 3,4,3,5; 3,4,4,4;

PAGE 104

Statistics and Nonlinear Regression Allen R. Overman 102 3,4,5,3; 3,4,6,2; 3,5,1,6; 3,5,2,5; 3,5,3,4; 3,5,4,3; 3,5,5,2; 3,5,6,1; 3,6,1,5; 3,6,2,4; 3,6,3,3; 3,6,4,2; 3,6,5,1; 4,1,4,6; 4,1,5,5; 4,1,6,4; 4,2,3,6; 4,2,4,5; 4,2,5,4; 4,2,6,3; 4,3,2,6; 4,3,3,5; 4,3,4,4; 4,3,5,3; 4,3,6,2; 4,4,1,6; 4,4,2,5; 4,4,3,4; 4,4,4,3; 4,4,5,2; 4,4,6,1; 4,5,1,5; 4,5,2,4; 4,5,3,3; 4,5,4,2; 4,5,5,1; 4,6,1,4; 4,6,2,3; 4,6,3,2; 4,6,4,1; 5,1,3,6; 5,1,4,5; 5,1,5,4; 5,1,6,3; 5,2,2,6; 5,2,3,5; 5,2,4,4; 5,2,5,3; 5,2,6,2; 5,3,1,6; 5,3,2,5; 5,3,3,4; 5,3,4,3; 5,3,5,2; 5,3,6,1; 5,4,1,5; 5,4,2,4; 5,4,3,3; 5,4,4,2; 5,4,5,1; 5,5,1,4; 5,5,2,3; 5,5,3,2; 5,5,4,1; 5,6,1,3; 5,6,2,2; 5,6,3,1; 6,1,2,6; 6,1,3,5; 6,1,4,4; 6,1,5,3; 6,1,6,2; 6,2,1,6; 6,2,2,5; 6,2,3,4; 6,2,4,3; 6,2,5,2; 6,2,6,1; 6,3,1,5; 6,3,2,4; 6,3,3,3; 6,3,4,2; 6,3,5,1; 6,4,1,4; 6,4,2,3; 6,4,3,2; 6,4,4,1; 6,5,1,3; 6,5,2,2; 6,5,3,1; 6,6,1,2; 6,6,2,1 1,3,6,6; 1,4,5,6; 1,4,6,5; 16 125 0.096451 1,5,4,6; 1,5,5,5; 1,5,6,4; 1,6,3,6; 1,6,4,5; 1,6,5,4; 1,6,6,3; 2,2,6,6; 2,3,5,6; 2,3,6,5; 2,4,4,6; 2,4,5,5; 2,4,6,4; 2,5,3,6; 2,5,4,5; 2,5,5,4; 2,5,6,3; 2,6,2,6; 2,6,3,5; 2,6,4,4; 2,6,5,3; 2,6,6,2; 3,1,6,6; 3,2,5,6; 3,2,6,5; 3,3,4,6; 3,3,5,5; 3,3,6,4; 3,4,3,6; 3,4,4,5; 3,4,5,4; 3,4,6,3; 3,5,2,6; 3,5,3,5; 3,5,4,4; 3,5,5,3; 3,5,6,2; 3,6,1,6; 3,6,2,5;

PAGE 105

Statistics and Nonlinear Regression Allen R. Overman 103 3,6,3,4; 3,6,4,3; 3,6,5,2; 3,6,6,1; 4,1,5,6; 4,1,6,5; 4,2,4,6; 4,2,5,5; 4,2,6,4; 4,3,3,6; 4,3,4,5; 4,3,5,4; 4,3,6,3; 4,4,2,6; 4,4,3,5; 4,4,4,4; 4,4,5,3; 4,4,6,2; 4,5,1,6; 4,5,2,5; 4,5,3,4; 4,5,4,3; 4,5,5,2; 4,5,6,1; 4,6,1,5; 4,6,2,4; 4,6,3,3; 4,6,4,2; 4,6,5,1; 5,1,4,6; 5,1,5,5; 5,1,6,4; 5,2,3,6; 5,2,4,5; 5,2,5,4; 5,2,6,3; 5,3,2,6; 5,3,3,5; 5,3,4,4; 5,3,5,3; 5,3,6,2; 5,4,1,6; 5,4,2,5; 5,4,3,4; 5,4,4,3; 5,4,5,2; 5,4,6,1; 5,5,1,5; 5,5,2,4; 5,5,3,3; 5,5,4,2; 5,5,5,1; 5,6,1,4; 5,6,2,3; 5,6,3,2; 5,6,4,1; 6,1,3,6; 6,1,4,5; 6,1,5,4; 6,1,6,3; 6,2,2,6; 6,2,3,5; 6,2,4,4; 6,2,5,3; 6,2,6,2; 6,3,1,6; 6,3,2,5; 6,3,3,4; 6,3,4,3; 6,3,5,2; 6,3,6,1; 6,4,1,5; 6,4,2,4; 6,4,3,3; 6,4,4,2; 6,4,5,1; 6,5,1,4; 6,5,2,3; 6,5,3,2; 6,5,4,1; 6,6,1,3; 6,6,2,2; 6,6,3,1 1,4,6,6; 1,5,5,6; 1,5,6,5; 17 104 0.080247 1,6,4,6; 1,6,5,5; 1,6,6,4; 2,3,6,6; 2,4,5,6; 2,4,6,5; 2,5,4,6; 2,5,5,5; 2,5,6,4; 2,6,3,6; 2,6,4,5; 2,6,5,4; 2,6,6,3; 3,2,6,6; 3,3,5,6; 3,3,6,5; 3,4,4,6; 3,4,5,5; 3,4,6,4; 3,5,3,6; 3,5,4,5; 3,5,5,4; 3,5,6,3; 3,6,2,6; 3,6,3,5; 3,6,4,4; 3,6,5,3; 3,6,6,2; 4,1,6,6; 4,2,5,6; 4,2,6,5; 4,3,4,6; 4,3,5,5; 4,3,6,4; 4,4,3,6; 4,4,4,5; 4,4,5,4; 4,4,6,3; 4,5,2,6; 4,5,3,5; 4,5,4,4; 4,5,5,3; 4,5,6,2; 4,6,1,6; 4,6,2,5; 4,6,3,4; 4,6,4,3; 4,6,5,2;

PAGE 106

Statistics and Nonlinear Regression Allen R. Overman 104 4,6,6,1; 5,1,5,6; 5,1,6,5; 5,2,4,6; 5,2,5,5; 5,2,6,4; 5,3,3,6; 5,3,4,5; 5,3,5,4; 5,3,6,3; 5,4,2,6; 5,4,3,5; 5,4,4,4; 5,4,5,3; 5,4,6,2; 5,5,1,6; 5,5,2,5; 5,5,3,4; 5,5,4,3; 5,5,5,2; 5,5,6,1; 5,6,1,5; 5,6,2,4; 5,6,3,3; 5,6,4,2; 5,6,5,1; 6,1,4,6; 6,1,5,5; 6,1,6,4; 6,2,3,6; 6,2,4,5; 6,2,5,4; 6,2,6,3; 6,3,2,6; 6,3,3,5; 6,3,4,4; 6,3,5,3; 6,3,6,2; 6,4,1,6; 6,4,2,5; 6,4,3,4; 6,4,4,3; 6,4,5,2; 6,4,6,1; 6,5,1,5; 6,5,2,4; 6,5,3,3; 6,5,4,2; 6,5,5,1; 6,6,1,4; 6,6,2,3; 6,6,3,2; 6,6,4,1 6,6,1,5; 6,6,2,4; 6,6,3,3; 18 80 0.061728 6,6,4,2; 6,6,5,1; 6,5,1,6; 6,5,2,5; 6,5,3,4; 6,5,4,3; 6,5,5,2; 6,5,6,1; 6,4,2,6; 6,4,3,5; 6,4,4,4; 6,4,5,3; 6,4,6,2; 6,3,3,6; 6,3,4,5; 6,3,5,4; 6,3,6,3; 6,2,4,6; 6,2,5,5; 6,2,6,4; 6,1,5,6; 6,1,6,5; 5,6,1,6; 5,6,2,5; 5,6,3,4; 5,6,4,3; 5,6,5,2; 5,6,6,1; 5,5,2,6; 5,5,3,5; 5,5,4,4; 5,5,5,3; 5,5,6,2; 5,4,3,6; 5,4,4,5; 5,4,5,4; 5,4,6,3; 5,3,4,6; 5,3,5,5; 5,3,6,4; 5,2,5,6; 5,2,6,5; 5,1,6,6; 4,6,2,6; 4,6,3,5; 4,6,4,4; 4,6,5,3; 4,6,6,2; 4,5,3,6; 4,5,4,5; 4,5,5,4; 4,5,6,3; 4,4,4,6; 4,4,5,5; 4,4,6,4; 4,3,5,6; 4,3,6,5; 4,2,6,6; 3,6,3,6; 3,6,4,5; 3,6,5,4; 3,6,6,3; 3,5,4,6; 3,5,5,5; 3,5,6,4; 3,4,5,6; 3,4,6,5; 3,3,6,6; 2,6,4,6; 2,6,5,5; 2,6,6,4; 2,5,5,6; 2,5,6,5; 2,4,6,6; 1,6,5,6; 1,6,6,5; 1,5,6,6

PAGE 107

Statistics and Nonlinear Regression Allen R. Overman 105 6,6,1,6; 6,6,2,5; 6,6,3,4; 19 56 0.043210 6,6,4,3; 6,6,5,2; 6,6,6,1; 6,5,2,6; 6,5,3,5; 6,5,4,4; 6,5,5,3; 6,5,6,2; 6,4,3,6; 6,4,4,5; 6,4,5,4; 6,4,6,3; 6,3,4,6; 6,3,5,5; 6,3,6,4; 6,2,5,6; 6,2,6,5; 6,1,6,6; 5,6,2,6; 5,6,3,5; 5,6,4,4; 5,6,5,3; 5,6,6,2; 5,5,3,6; 5,5,4,5; 5,5,5,4; 5,5,6,3; 5,4,4,6; 5,4,5,5; 5,4,6,4; 5,3,5,6; 5,3,6,5; 5,2,6,6; 4,6,3,6; 4,6,4,5; 4,6,5,4; 4,6,6,3; 4,5,4,6; 4,5,5,5; 4,5,6,4; 4,4,5,6; 4,4,6,5; 4,3,6,6; 3,6,4,6; 3,6,5,5; 3,6,6,4; 3,5,5,6; 3,5,6,5; 3,4,6,6; 2,6,5,6; 2,6,6,5; 2,5,6,6; 1,6,6,6 6,6,2,6; 6,6,3,5; 6,6,4,4; 20 35 0.027006 6,6,5,3; 6,6,6,2; 6,5,3,6; 6,5,4,5; 6,5,5,4; 6,5,6,3; 6,4,4,6; 6,4,5,5; 6,4,6,4; 6,3,5,6; 6,3,6,5; 6,2,6,6; 5,6,3,6; 5,6,4,5; 5,6,5,4; 5,6,6,3; 5,5,4,6; 5,5,5,5; 5,5,6,4; 5,4,5,6; 5,4,6,5; 5,3,6,6; 4,6,4,6; 4,6,5,5; 4,6,6,4; 4,5,5,6; 4,5,6,5; 4,4,6,6; 3,6,5,6; 3,6,6,5; 3,5,6,6; 2,6,6,6; 6,6,3,6; 6,6,4,5; 6,6,5,4; 21 20 0.015432 6,6,6,3; 6,5,4,6; 6,5,5,5; 6,5,6,4; 6,4,5,6; 6,4,6,5; 6,3,6,6; 5,6,4,6; 5,6,5,5; 5,6,6,4; 5,5,5,6; 5,5,6,5; 5,4,6,6; 4,6,5,6; 4,6,6,5; 4,5,6,6; 3,6,6,6 6,6,4,6; 6,6,5,5; 6,6,6,4; 22 10 0.007716 6,5,5,6; 6,5,6,5; 6,4,6,6; 5,6,5,6; 5,6,6,5; 5,5,6,6;

PAGE 108

Statistics and Nonlinear Regression Allen R. Overman 106 4,6,6,6 6,6,6,5; 6,6,5,6; 6,5,6,6; 23 4 0.003086 5,6,6,6 6,6,6,6 24 1 0.000772

PAGE 109

Statistics and Nonlinear Regression Allen R. Overman 107 Tab le A31 Summary of frequency distribution for four dice. S c f F Z Z f 0.000000 4 1 0.000772 0.001510 0.000772 5 4 0.003086 0.003455 0.003858 6 10 0.007716 0.007246 0.011574 1.605467 1.565423 7 20 0.015432 0.013929 0.027006 1.359960 1.356700 8 35 0.027006 0.024540 0.054012 1.135382 1.147977 9 56 0.043210 0.039627 0.097222 0.917975 0.939254 10 80 0.061728 0.058651 0.158950 0.706891 0.730531 11 104 0.080247 0.079564 0.239197 0.501122 0.521808 12 125 0.096451 0.098928 0.335648 0.299664 0.313085 13 140 0.108025 0.112740 0.443673 0.100551 0.104362 14 146 0.112654 0.117760 0556327 +0.100551 +0.104362 15 140 0.108025 0.112740 0.664352 +0.299664 +0.313085 16 125 0.096451 0.098928 0.760803 +0.501122 +0.521808 17 104 0.080247 0.079564 0.841050 +0.706891 +0.521808 18 80 0.061728 0.058651 0.902778 +0.917975 +0.730531 19 56 0.043210 0.039627 0.945988 +1.135382 +0.939254 20 35 0.027006 0.024540 0.972994 +1.359960 +1.147977 21 20 0.015432 0.013929 0.988426 +1.605467 +1.565423 22 10 0.007716 0.007246 0.996142 23 4 0.003086 0.003455 0.999228 24 1 0.000772 0.001510 1.000000

PAGE 110

Statistics and Nonlinear Regression Allen R. Overman 108 C 1296 = 64 SSFZ208723.09221.221212erf1 r = 0.999770 0000 14 79104 4 2 2279104.4000.14exp11776.02exp21SSf f f 07548 1 00512 0 r = 0.99883 The frequency distribution for a set of dice more closel y conforms to the continuous Gaussian distribution as the number of dice increases. Complexity of computing the discrete distribution increases dramatically with the number of dice, and becomes unwieldy beyond four dice. For five dice the total number of c ombinations is 65 = 7776! The two dice problem illustrates how well the continuous Gaussian distribution approximates the discrete distribution (triangular). Even though the approximation is not exact, it does bring in an analytic function which we have used in the model for plant growth. The peg board offers a simpler model of the frequency distribution than does a set of dice. Reference: Speyer, E. 1994. Six Roads from Newton: Great Discoveries in Physics. John Wiley & Sons. New York. NY.

PAGE 111

Statistics and Nonlinear Regression Allen R. Overman 109 Derivatives for power functions (as defined by Cauchy) Function Derivative 0xy 011)(00xxxy 00xxy 0lim0 x y dx dyx 1xy x x x x x x x y 1 1) ( 1 x x x y 1lim0 x y dx dyx 2x y 2 2 2 2 2 2) ( 2 ) ( 2 ) ( x x x x x x x x x x x y x x x x x x x y 2 ) ( 22 x x y dx dyx2lim0 3x y 3 2 2 3 3 2 2 3 3 3) ( ) ( 3 3 ) ( ) ( 3 3 ) ( x x x x x x x x x x x x x x x y 22)(33xxxxxy 203limxxydxdyx 4xy 322343223444)(4)(64)(4)(64)(xxxxxxxxxxxxxxxxxy 223)(464xxxxxxy 304limxxydxdyx nxy 1 nnx dx dy

PAGE 112

Statistics and Nonlinear Regression Allen R. Overman 110 The reader may note that the binomial expansion with n = 0, 1, 2, ,3 leads to finite power series. Newton was the first to prove this. Then he showed that for n either negative or a fraction the expansion leads to an infinite power series (Berlinski, 2000, p. 30). This led to Newtons first memoir in 1669 entitled On Analysis by Infinite Series which preceded his development of calculus. It is common to write the general solution as an infinite power series 044332210iiixaxaxaxaxaay where ai are the expansion coefficients. Now the coefficients can be evaluated using the derivatives fro m calculus and the boundary values at x = 0. This leads to the following relationships 0 0) 0 ( a y x y 1 0a dx dyx 0 2 2 2 2 0 2 2! 2 1 1 2 x xdx y d a a dx y d 03333033!31123xxdxydaadxyd 04444044!411234xxdxydaadxyd 00!1!xnnnnxnndxydnaandxyd This leads finally to the infinite series nxnnxxxdxydnxdxydxdxdyyy0202200!1!21!11

PAGE 113

Statistics and Nonlinear Regression Allen R. Overman 111 may be recognized as the Taylor series, which was discovered in 1715 by Brook Taylor. This approach assumes that the derivatives exist. It also assumes that the series converges to a finite value for any value of x or at least for a limited domain of x Physics usually enters into the process by way of a differential equation with initial conditions at x = 0. Berlinski, D. 2000. Newtons Gift: How Sir Isaac Newton Unlocked the System of the World. Simon & Schuster. New York, NY. Example: Consider the first order differential equation k dx dy with the initial condition 0 at0 x y y where k is a constant. Solution: kdxdyx0 0022xdxyd 00044033xnnxxdxyddxyddxyd The solution becomes kx y x dx y d n x dx y d x dx dy y yn x n n x x 0 0 2 0 2 2 0 0! 1 2 1 1 1 which is the equation of a straight line.

PAGE 114

Statistics and Nonlinear Regression Allen R. Overman 112 Example: Consider the first order differential equation kydxdy with the initial condition 0 at0 x y y where k is a constant. Solution: 0 0ky dx dyx 0 2 0 0 2 2y k dx dy k dx y dx x 0 3 0 3 3y k dx y dx 0 0y k dx y dn x n n The solution becomes n x n n x xx dx y d n x dx y d x dx dy y y0 2 0 2 2 0 0! 1 2 1 1 1 n nx y k n x y k x ky y0 2 0 2 0 0! 1 2 1 nkxnkxkxy!1!21!111210 kxyexp0 where nkxnkxkxkx!1!21!111exp2 as defined by Euler.