<%BANNER%>

Accuracy of response surface approximations for weight equations based on structural optimization

University of Florida Institutional Repository

PAGE 1

ACCURACY OF RESPONSE SURFACE APPROXIMATIONS FOR WEIGHT EQUATIONS BASED ON STRUCTURAL OPTIMIZATION By MELIH PAPILA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPY UNIVERSITY OF FLORIDA 2001

PAGE 2

Aileme…

PAGE 3

iii ACKNOWLEDGEMENTS I would like to express my special thanks and appreciation to Dr. Raphael T. Haftka, chairman of my advisory committee. He has been a continuous and excellent source of guidance for my Ph.D. research. I have always been amazed and motivated by his never-ending enthusiasm to explore and willingness to share his knowledge and experience. Working and interacting with him have broadened my vision not only professionally, but also personally. I also would like to thank the members of my advisory committee, Dr. Andr I. Khuri, Dr. Andrew J. Kurdila, Dr. Bhavani V. Sankar, Dr. Peter G. Ifju and Dr. Wei Shyy. I am grateful for their willingness to review my Ph.D. research and to provide me their constructive comments that helped me to complete this dissertation. I must express my gratitude to my former advisor, Dr. Mehmet A. Akgun, who made me realize how much I like research. I have always enjoyed interacting with him in my professional and personal life. I also thank the colleagues with whom I interacted during this study. In particular, I thank Dr. Roberto Vitali, Dr. Satchi Venkataraman, Dr. Raluca Rosca and Dr. Ibrahim Ebcioglu at the University of Florida, and Dr. Hongman Kim at Virginia Tech. The financial support provided by NASA grants NAG1-2000 and NAG1-2177 is gratefully acknowledged.

PAGE 4

iv I thank my dear friends from home; Murat (MK), Canan, Selami (SYA), Senol, Tarkan, Sartuk, Bulent, Yusuf and Merve who kept constant contact with me. I am grateful for the friendship of Kamal and Alpay who always inspired me with their positive attitudes. I am also grateful to have my best friend Guray and Elif in my life. They deserve special thanks for their constant support and true friendship. The best thing happened to me during my study was getting to know wonderful people, Farshad, Sibel, and their family, and having the chance to make them an important part of my life. I thank my parents, Gonul and Mithat Papila; Meric, Ozer, Cagil and Irmak Torgal; Hurriyet, Ibrahim and Eray Uzgoren; Emine, Bulent and Yigit Topoglu: without the love and support of my family I could not have finished this dissertation. Finally, my deepest appreciation goes to my beautiful and loving wife, Nilay. Her love and presence have been and will be a tremendous inspiration for every challenge in my life. Nilay has given me her support in every respect: emotionally as my better half, professionally as my colleague. I only hope that I have been able to give her the same support in return.

PAGE 5

v TABLE OF CONTENTS page ACKNO W L E D G EME N T S ............................................................................................. iii L I ST O F T A B L E S ......................................................................................................... . vii L I ST O F F I GURE S .......................................................................................................... ix A B STRACT .................................................................................................................. xiii CHAPTERS 1. INTRODUCTION.........................................................................................................1 Focus.............................................................................................................................1 Approach....................................................................................................................... 7 Scope and Objectives....................................................................................................9 2. LITERATURE SURVEY...........................................................................................12 Wing (System level) Structural Weight Equations.....................................................13 Panel (Component Level) Weight Equations in Multi-Level Procedures...................18 Accuracy of Response Surfaces..................................................................................23 Use of RS in Uncertainty Quantification....................................................................30 3. RESPONSE SURFACE METHODOLOGY..............................................................32 Design of Experiments................................................................................................33 Fitting Procedure: Standard Least Squares.................................................................36 Adequacy and Accuracy of RS...................................................................................39 Hypothesis Testing in RS Methodology...............................................................43 Standard Lack-of-Fit Test.....................................................................................48 Lack-of-Fit with Non-Replicated Design Points...................................................52 Iteratively Re-weighted Least Squares........................................................................58 4. BIAS AND NOISE ERRORS IN RESPONSE SURFACE APPROXIMATIONS...63 Mean Squared Error Criterion.....................................................................................63 Estimated Standard Prediction Error and Residual Based Error Bound.....................69 Polynomial Examples..................................................................................................71 Coefficient of Correlation.....................................................................................72 Cubic Polynomial in Two Variables without Noise...........................................74

PAGE 6

vi Cubic Polynomial in Two Variables – with Noise................................................81 Cubic Polynomial in Five Variables – without Noise...........................................87 Cubic Polynomial in Five Variables – with Noise................................................89 Effect of Experimental Design: Eigenvalues with a Minimum Bias Design........91 Discussion...................................................................................................................94 5. HIGH SPEED CIVIL TRANSPORT (HSCT) WING WEIGHT EQUATIONS.......95 HSCT Design Definition.............................................................................................95 Structural Optimization and Finite Element Model....................................................98 Construction of Response Surface Approximations.................................................103 Five Configuration Variables..............................................................................103 Numerical noise and outlier analysis............................................................104 Lack-of-fit.....................................................................................................108 Error analysis based on MSEP eigenvalues................................................111 Cubic fitting model in the approximation.....................................................113 Data vs. RS....................................................................................................117 Repair vs. IRLS.............................................................................................118 Ten Configuration Variables...............................................................................121 Design for Uncertainty..............................................................................................123 Parameter Uncertainty: Material Properties and Geometry................................125 RS Uncertainty....................................................................................................130 Discussion.................................................................................................................132 6. CRACKED COMPOSITE PANEL WEIGHT EQUATION....................................134 Stress Intensity Factor Calculation............................................................................137 Panel Problem Definition..........................................................................................142 Optimization and Response Surface Comparisons...................................................145 Low-Fidelity Results...........................................................................................146 High-Fidelity Results..........................................................................................147 Comparison of Low-Fidelity and High-Fidelity Results.....................................150 Discussion.................................................................................................................153 7. CONCLUDING REMARKS....................................................................................155 APPENDICES A NOISE IN OPTIMIZATION RESULTS DUE TO ROUND-OFF ERRORS..........157 B FUZZY SET THEORY.............................................................................................165 REFERENCES................................................................................................................168 BIOGRAPHICAL SKETCH...........................................................................................177

PAGE 7

vii LIST OF TABLES Table page 1. Comparison of actual and predicted weights from Eq. (1.1)..................................3 2. Experimental designs used in the present work....................................................36 3. Verification of the lack-of-fit test with non-replicated design points...................57 4. Standard weighting functions for IRLS................................................................60 5. Coefficient of correlation between the absolute residual i Tje and the square-root of maximum eigenvalues max G from Eq. (4.27) for twovariable cubic polynomial example without noise................................................77 6. Coefficient of correlations between the absolute residual i Tje and the square-root of maximum eigenvalues max G from Eq. (4.27) for twovariable cubic polynomial example with noise.....................................................82 7. Coefficients of correlation between the estimated standard error and absolute true residuals i Tje for cubic polynomial example with noise................85 8. Face-centered central composite design (FCCD) and minimum-bias central composite design (MBCCD) for two-variable polynomial example........92 9. Load cases in HSCT design study.........................................................................96 10. Configuration design variables for HSCT with corresponding value ranges........97 11. Statistics for five-variable full quadratic response surface approximations for HSCT wing bending material weight............................................................105 12. The near replicate design points..........................................................................108 13. Results of the lack-of-fit test for quadratic RS fitted to 126 design points of five-variable HSCT wing problem..................................................................109 14. Results of the lack-of-fit test for quadratic RS fitted to 115 design points of five-variable HSCT wing problem..................................................................110

PAGE 8

viii 15. Coefficients of correlation r between max G and js for five-variable HSCT wing problem at 3146 design points........................................................111 16. Statistics for five-variable cubic response surface approximations for wing bending material weight......................................................................................114 17. Results of the lack-of-fit test for cubic RS fitted to data of five-variable HSCT wing problem...........................................................................................115 18. Average Residual Percentage for IRLS fit and RS after Outlier Repair, five-variable HSCT problem...............................................................................120 19. Statistics for 10-variable response surface approximations for HSCT wing bending material weight......................................................................................122 20. RScubic term estimates and statistics, five-variable HSCT problem....................124 21. Uncertainty levels and ranges for fuzzy variables in five-variable HSCT wing problem.......................................................................................................126 22. Uncertainty for blue and red design: Combined effect variable fuzziness and RS modeling uncertainty..............................................................................132 23. Material Properties for AS4/3501-6....................................................................144 24. Configurations for three-level full factorial experimental design.......................145 25. Summary of RS approximation (weight equation) based on results of lowfidelity optimization............................................................................................147 26. Summary for quadratic RS approximation (weight equation) based on high-fidelity results.............................................................................................148 27. Summary for first order RS approximation (weight equation) based on high-fidelity results.............................................................................................149 28. Ply-thicknesses of low-fidelity optima for configurations deviating from the general trend..................................................................................................152 29. Summary of RS approximation (weight equation) based on results of lowfidelity optimization after repair of the three outliers by high-fidelity...............152 30. Convergence history for Configuration-8: Convergence setting Case 1.............160 31. Objective function with different initial structural designs for Config. 8..........161

PAGE 9

ix LIST OF FIGURES Figure page 1. Aircraft gross weight variation with wing span......................................................2 2. Three-level full factorial (left) and face centered central composite (right) experimental designs for three coded-configuration variables.............................35 3. Sample F-distributions..........................................................................................45 4. The t-distributions of various degrees of freedom df............................................48 5. 3-D representation of clustering design points to generate near-replicate data for lack-of-fit test...........................................................................................56 6. 1-D example of the effect of an outlier in the data................................................58 7. 1-D example of IRLS procedure...........................................................................59 8. Weighting functions for IRLS...............................................................................61 9. Linear correlation examples..................................................................................73 10. Two-variable three-level factorial design points...................................................75 11. Square-root of maximum eigenvalue max G contours from Eq. (4.27) for a two-variable cubic polynomial, fitted with a quadratic model.....................75 12. Absolute residual i Tje contours – two-variable cubic polynomial example (polynomials defined in Table 5), without noise..................................................77 13. Comparison between square-root of eigenvalues max G and absolute residual i Tje– two-variable cubic polynomial example.......................................78 14. Comparison of square-root of eigenvalues max G and maximum absolute residual max Tje contours (maximum of residuals among the four polynomials defined in Table 5) – two-variable cubic polynomial example, without noise.........................................................................................................79

PAGE 10

x 15. Comparison between square-root of eigenvalues maxG and maximum of true residual errors max Tje due to the four polynomial in Table 5 – twovariable cubic polynomial example, without noise...............................................79 16. Conservative error bound i js contours – two-variable cubic polynomial example (polynomials defined in Table 5), without noise....................................80 17. Comparison between conservative error bound i js and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 5), without noise..........................................................................................81 18. Absolute residual i Tje contours – two-variable cubic polynomial example (polynomial defined in Table 5 and Table 6), with noise.....................................82 19. Comparison between square-root of eigenvalues maxG and absolute residual i Tje– two-variable cubic polynomial example (polynomials defined in Table 5), with noise..............................................................................83 20. Comparison of square-root of eigenvalues maxG and maximum absolute residual max Tje contours (maximum of residuals among the four polynomials defined in Table 6) – two-variable cubic polynomial example, with noise..............................................................................................................84 21. Comparison between square-root of eigenvalues maxG and maximum of residual errors max Tje due to the four polynomial in Table 5 – twovariable polynomial example, with noise..............................................................84 22. Normalized estimated standard error /ese for a two-variable...........................86 23. Comparison between conservative error bound i js and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 7), with noise...............................................................................................86 24. Comparison between square-root of eigenvalues maxG and absolute true residuals Tje and the trend line manually drawn for largest errors characterized by the eigenvalues– five-variable cubic polynomial example, without noise.........................................................................................................88

PAGE 11

xi 25. Comparison between conservative error bound js and absolute true residual Tje – five-variable cubic polynomial example, without noise...............89 26. Comparison between square-root of eigenvalues maxG x and absolute residuals je y – five-variable cubic polynomial example, with noise..............90 27. Comparison between conservative error bound js and absolute residual je – five-variable cubic polynomial example, with noise..................................91 28. Eigenvalue-contours ( max G). a) Face-centered central composite design (FCCD); b) minimum-bias central composite design (MBCCD) for twovariable polynomial example................................................................................93 29. Absolute residual-contours in two-variable polynomial example (for Polynomial 1 in Table 5). a) Face-centered central composite design (FCCD); b) Minimum-bias central composite design (MBCCD).........................93 30. HSCT wing planform and five& 10-configuration variables.............................97 31. Flow chart for the RS based wing weight equation..............................................99 32. HSCT finite element model and structural design variables.................................99 33. Close-up view for the typical wing box cell.......................................................100 34. Flow chart of the optimization in GENESIS.......................................................101 35. Effect of overly tight optimization parameters: Difference in optimization results between Case 1 and Case 3......................................................................102 36. Outliers and effect of repair on quadratic response surface, five-variable HSCT problem....................................................................................................107 37. Comparison between square-root of eigenvalues maxG and conservative error bound js – quadratic RS of HSCT 126 design points..........112 38. Comparison between square-root of eigenvalues maxG and conservative error bound js – quadratic RS of HSCT 115 design points..........112 39. Comparison of predictions of square-root of MSEP, js and js at 115 design points quadratic RS................................................................................113

PAGE 12

xii 40. Comparison between square-root of eigenvalues maxG and absolute error between reduced cubic and quadratic RS – five-variable HSCT wing problem................................................................................................................117 41. Noisy wing bending material weight Wb compared to smoother objective function (total wing structural weight)................................................................118 42. Extreme value design line...................................................................................119 43. Comparison of RS with outlier repair and with outlier elimination over the design line, five-variable HSCT problem...........................................................120 44. Description of vertex method for three-variable case.........................................127 45. Five-variable HSCT case: Contours in plane with almost constant wing bending. a) Nominal wing bending material weight contours; b) parameter uncertainty band width (upper bound minus lower bound) contours; c) Modeling (RS) uncertainty band width contours................................................129 46. Comparison of predictions of square-root of MSEP, js(=x) and js(=y) at 152 design points of five-variable HSCT wing problem....................................131 47. Blue and Red design planforms..........................................................................131 48. Fit over stress values to find K............................................................................140 49. Stress distribution in the vicinity of the crack tip................................................141 50. Composite Blade-Stiffened Panel with a Crack..................................................143 51. Comparison of prediction by RS approximation and optimization data based on low-fidelity optimization for stiffened panel with crack.....................146 52. Comparison of prediction by RS approximation and optimization data based on high-fidelity optimization for stiffened panel with crack....................149 53. Ratio of high-fidelity and low-fidelity optimal weights......................................150 54. Comparison of relative errors wrt high-fidelity results obtained by lowfidelity RS before and after repair.......................................................................153 55. Difference in results on PC and UNIX: Convergence setting Case 1.................159 56. Difference in results on PC and UNIX: Convergence setting Case 2.................159 57. Membership functions.........................................................................................166

PAGE 13

xiii Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ACCURACY OF RESPONSE SURFACE APPROXIMATIONS FOR WEIGHT EQUATIONS BASED ON STRUCTURAL OPTIMIZATION By Melih Papila December 2001 Chair: Raphael T. Haftka, Distinguished Professor Major Department: Aerospace Engineering, Mechanics and Engineering Science Accurate weight prediction methods are vitally important for aircraft design optimization. Therefore, designers seek weight prediction techniques with low computational cost and high accuracy, and usually require a compromise between the two. The compromise can be achieved by combining stress analysis and response surface (RS) methodology. While stress analysis provides accurate weight information, RS techniques help to transmit effectively this information to the optimization procedure. The focus of this dissertation is structural weight equations in the form of RS approximations and their accuracy when fitted to results of structural optimizations that are based on finite element analyses. Use of RS methodology filters out the numerical noise in structural optimization results and provides a smooth weight function that can easily be used in gradient-based configuration optimization. In engineering applications

PAGE 14

xiv RS approximations of low order polynomials are widely used, but the weight may not be modeled well by low-order polynomials, leading to bias errors. In addition, some structural optimization results may have high-amplitude errors (outliers) that may severely affect the accuracy of the weight equation. Statistical techniques associated with RS methodology are sought in order to deal with these two difficulties: (1) high-amplitude numerical noise (outliers) and (2) approximation model inadequacy. The investigation starts with reducing approximation error by identifying and repairing outliers. A potential reason for outliers in optimization results is premature convergence, and outliers of such nature may be corrected by employing different convergence settings. It is demonstrated that outlier repair can lead to accuracy improvements over the more standard approach of removing outliers. The adequacy of approximation is then studied by a modified lack-of-fit approach, and RS errors due to the approximation model are reduced by using higher order polynomials. In addition, remaining error in the RS weight equation is characterized, and two error measures are introduced. One of the measures is qualitative, and it is used to identify regions of design space where RS accuracy may be poor. The second measure is quantitative, but conservative. Its use is demonstrated for incorporating the uncertainty due to RS weight equation with parameter uncertainty in order to compare competing designs for robustness.

PAGE 15

1 CHAPTER 1 INTRODUCTION An aircraft designer would probably place weight at the top of the list of important concerns in the design of an aircraft. This is because all disciplines in the design process such as aerodynamics, structure, performance and propulsion interact with each other through the weight. The trade-off between aerodynamic and structural efficiency is a common example of such a discipline interaction. Structural weight, a big proportion of the total weight, affects the lift that must be generated, and consequently the drag. In turn aerodynamic design aspects affect the loads, and hence structural design, and the structural weight. Focus The focus of this dissertation is accurate and affordable weight estimation. The impact of the weight starts from the very beginning in aircraft design process. Definition of conceptual characteristics initiates the process, and it is followed by the preliminary design phase where prediction of the weight is one of the first steps. The final size and cost of the aircraft are strongly influenced by the estimated weight at the preliminary design phase. The more reliable and precise starting data are, the faster is the convergence to the promising configurations. Empirical weight equations. It is a common practice in conceptual design to estimate the weight by empirical equations derived from historical aircraft data. They are

PAGE 16

2 y = 2.0186x + 1.15240.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 0.000.501.001.502.002.50 x=log(Span, ft)y=log(Gross Weight, lbs) Data Linear fitone of the earliest approaches used for weight prediction. Early wing weight prediction methods, for instance, were of a purely statistical nature and included only simple relationships between wing structure weight and the most significant design parameters such as wing span and wing gross area. If only a nominal wing span is available to a designer, a purely statistical approach is simply to collect and plot weight versus wing span data for comparable existing aircraft, and then try to make a fit to obtain an expression for weight as a function of wing span. For example Figure 1 presents statistical/historical data (source: Aviation Week & Space Technology, March 16, 1992, pp. 71-119) for the gross weight versus wing span on a logarithmic scale for existing aircraft. A first order polynomial curve fit on this plot gives an empirical equation for aircraft gross weight. Figure 1: Aircraft gross weight variation with wing span determined by statistical evaluation of existing aircraft Empirical equations are still widely used for first order estimates or as reasonability checks for more detailed analysis. Most airplane manufacturers have developed their own methods based on such empirical relations (Niu, 1988; Loretti

PAGE 17

3 1997). An example of how to develop an empirical equation can be found in Staton (1996). Another example of empirical equations presented in Torenbeek (1992) is a wing weight equation, WW for subsonic transports and executive jets: 2 117 ! # $ $ % & MTO MZF wW W bS W (1.1) where b is wing span, S is wing gross area, WMZF is maximum zero fuel weight, and WMTO is maximum take-off weight. Table 1 compares the actual and estimated weight for subsonic transport and executive jet airplanes, from Eq. (1.1). Table 1: Comparison of actual and predicted weights from Eq. (1.1) from Torenbeek (1992) Aircraft WMTO, (kN) WW, estimated (kN) WW, actual (kN) Error, % Airbus A300/B2 1343.6 188.43 196.31 -4.0 Airbus A340 2487.0 295.32 340.85 -13.4 BAC 1-11/300 387.0 38.60 42.90 -10.0 Cessna Citation II 59.16 6.79 5.73 +18 Boeing 727-100 71.8 75.89 79.02 -4.0 Boeing 737-200 513.8 39.79 47.21 -15.7 Boeing 747-100 3158.4 446.15 383.35 +16.1 Fokker F-28 Mk 4000 315.8 30.22 33.28 -9.2 Lockheed 1011-1 1912.8 224.59 210.86 +6.5 McD. Douglas DC-9/30 480.4 40.60 50.71 -19.9 McD. Douglas MD-80 622.8 60.95 69.22 -11.9 McD. Douglas DC-10/10 2024.0 250.46 217.97 +14.9 McD. Douglas DC-10/30 2468.9 252.74 261.83 -5.3 The main advantage of empirical weight equations is that they are easy to use. They do not require expertise in weight engineering, so aerodynamics or performance experts can easily use them. They do not require sophisticated computational resources, but they also have some limitations that should be considered.

PAGE 18

4 Empirical equations are generally a function of exterior geometric parameters, such as wingspan and thickness to chord ratio, and they may not reflect interior structural details such as frame, stiffener shapes and spacing. Since they are based on existing aircraft data, empirical equations may be misleading for unconventional designs or for new concepts that require extrapolation instead of interpolation from the data. This may happen, for instance, when new materials or technology appear (advanced composite materials, bonded construction), or when operating temperatures become much higher than those for previous structures. When conceptual characteristics are not far beyond the available knowledge and experience (in other words when interpolation is possible) accuracy by the empirical equations may be of the order of 2-5% (Niu, 1988). However, as Schmidt et al. (1997) suggested an error of 2% in fuselage weight prediction may be equivalent to about 1.5 tons of freight or 20 passengers. Alternative to empirical weight equations. Historically, structural weight estimation of aircraft that lie outside the range of applicability of empirical methods has been determined by stress-analysis based approaches or by construction of prototypes. The latter approach is very expensive and not practical unless the design is converged, and verification is needed rather than the prediction. Stress-analysis based structural weight prediction procedures are widely used throughout the design phase. Howe (1996) presented a comparison between an empirical and a stress-analysis based wing weight equation. Data for 118 aircraft ranging from the Cessna 150 to the Boeing 747/200 were used for comparison. Howe reported that both equations gave reasonably good results,

PAGE 19

5 but the latter were more accurate. However, it was also noted that the test data were limited to aircraft of light alloy construction. Predictions by empirical equations can be misleading for aircraft of different and advanced materials such as composite materials. That is because the use of such materials has been dealt with somewhat arbitrary factoring in empirical equations as historical data includes primarily the aircraft of traditional materials. Weight determination by stress analysis can be done at different levels of complexity ranging from closed form beam and plate solutions early in the design process to advanced finite element (FE) analysis at later phases of the process. Designers seek the lightest practical arrangement of material and structural details capable of withstanding the required loads. Depending on the complexity of the analysis and weight prediction, the computational time required for a lightweight design may be substantial. With advances in computational power, the level of complexity that can be afforded in the preliminary design phase is increasing. For instance, FE analysis usage in aerospace weight engineering has progressed to the point that preliminary structural designs may be analyzed with low levels of uncertainty when combined with a corresponding knowledge of the as-built structural design. However, the multidisciplinary nature of the modern design process cannot afford long cycle times, since it entails frequent changes and extensive trade-in studies. In addition, conceptual designers are not usually experts in FE modeling. That is why these designers still demand computationally affordable, easy to use, but also reliable approaches. As a consequence, the idea of combining the reliability of stress-analysis based prediction and the ease of manipulation of analytical expressions in multi-disciplinary studies has attracted interest.

PAGE 20

6 In general, structural weight equations have been developed for various types of wing and fuselage structures since they are primary load-carrying elements. Wing and fuselage structures are composed of a large number of curved or flat, usually stiffened panels. Design optimization of all panels simultaneously, in other words the complete structure, requires detailed structural modeling and appears to be beyond the present computational capabilities since the panels have complex geometry and failure modes. The current practice is to design wing or fuselage structures at two levels, global and panel, and this can increase the required time for weight estimation. In addition, other problems that affect stress-analysis based weight prediction and design optimization are the following Numerical simulations based on stress analysis such as structural optimizations are prone to numerical noise due to reasons including discretization errors, incomplete convergence of iterative procedures, and round-off errors. When these simulations are used in a multi-disciplinary optimization procedure employing gradient-based optimization algorithms the procedure suffers from the noisy calculations, and may fail to find the optimum. As a consequence, smooth responses are desired in optimization problems. To reduce the errors and the noise generated by the computational models the complexity of the model may be increased. A finer finite element mesh, for example, can be used in structural analyses. This, however, increases the computational cost and design cycle time.

PAGE 21

7 Oftentimes interactive use of different structural codes is required for capturing various failure modes, such as buckling, delamination, or crack propagation. Integration of the different codes is not an easy task. Approach The approach taken into consideration is weight equations by response surfaces. Response surface (RS) techniques (Myers and Montgomery, 1995: Khuri and Cornell, 1996), which fit a large number of analyses with simple functions, offer an attractive set of features that address the problems mentioned in previous section: They filter out the noise and help to identify outliers They may require large computational effort beforehand to construct, but this pays off during the design process since they are easy to compute They can act as interfaces between the different levels of analysis or between different disciplines. Response surface methodology can be used to construct weight equations. Approximations by RS are similar to empirical relations in the sense that they both employ statistical fitting procedures. The source data used, however, are different. RS based equations utilize the data created by stress-analysis based structural optimizations at pre-selected design points, providing an overall perspective within the design space (design of experiments) whereas empirical relations employ historical aircraft data. Applications for RS based weight equations. Multidisciplinary optimization (MDO) has become increasingly important with the increasing complexity of engineering systems. Sobieszczanski-Sobieski and Haftka (1997) presented a survey for the recent developments in this field. They indicated the two main challenges of MDO are

PAGE 22

8 organizational complexity and computational expense. Response surface techniques are particularly suitable for MDO applications. They can be used for approximating the objective functions and constraints. For the organizational challenge these techniques provide a convenient representation of the data from one discipline to other disciplines and to the system. For the challenge of computational expense RS techniques may first seem inappropriate since they require a large number of analyses for data generation beforehand. However, the simplicity and insignificant cost of calculations by RS once constructed make them attractive in multidisciplinary design optimization studies. The organizational challenge exists also in a single-discipline application. Oftentimes design optimization uses procedures employing different levels of analysis and optimization as in global/local structural design. In these applications, integration of different codes is generally required. Approximations by RS techniques may provide an easy means to connect such codes. They may also be used to correct the low-cost analysis results with the help of a small number of high-fidelity results whose computational cost is often prohibitive to use at each design throughout the design process. Another important application of RS techniques is design against uncertainty. The random nature of this problem makes it computationally intensive, and it typically entails at least an order of magnitude higher computational cost than deterministic design (Venter and Haftka, 1998). Response surface approximations can be used to reduce this high computational cost since they are easy to calculate and implement. The designer, however, should keep in mind that the RS approximation itself may impose another type of uncertainty due to modeling error that needs to be addressed in uncertainty studies.

PAGE 23

9 Challenges associated with RS based weight equations. Structural optimizations that are often used to generate data for RS based weight equations are prone to numerical noise. Optimization based on finite element analyses, for example, may introduce noise due to discretization errors affecting the analysis results or even due to round-off error characteristics of the computation platform affecting progress by the optimizer (Papila and Haftka, 1999). Although one of the most appealing features of RS approximations is the ability to filter the data, high-amplitude-noise outliers that may be experienced in structural optimizations may cause the loss of accuracy of the approximation or weight equation. Therefore, it is important to make sure the data for constructing the weight equation are free from severe flaws before relying on the weight equation. In case of data generated by structural optimizations, for instance, optimum weight data free from flaws such as incomplete convergence would increase the reliability of the weight equation. Even when the data generated are free from severe flaws or outliers, RS based weight equations are still approximations. This is because the number of the data is usually limited and it does not cover every possible design point. In addition, traditional polynomial fitting models may not represent equally the true nature or complexity of the weight function for every region in the design space. Therefore, it is also important to study the accuracy of the weight equation and to pinpoint the design regions where the approximation may be poor so that designer may steer away while using the weight equation or at least be cautious. Scope and Objectives The main thrust of this dissertation is to address the challenges associated with the RS based weight equations as mentioned in the previous section, that is, how to deal with

PAGE 24

10 the errors that are introduced by the process of fitting structural optimizations with RS for structural weight equations. This includes reducing error by identifying and repairing outliers, studying the accuracy of approximation, reducing RS errors due to the approximation model by using higher order polynomials, and characterizing the remaining error for use in design for uncertainty. Therefore, the two main objectives of the work are the following: Reduce errors associated with RS constructed on the basis of structural optimizations: To reduce noise error the aim is to identify the highly erroneous results and label them as outliers within the data. For the data generated by structural optimizations outliers may be due to reasons such as failure of convergence and local optima. Once detected they may be studied further for possible correction. The corrected or repaired data set is expected to increase the accuracy and reliability of the RS. To deal with modeling (bias) errors, higher-order polynomials are used. Characterize the remaining error and investigate statistical tools to determine the regions of design space where RS accuracy may be poor. A literature survey of wing and panel weight equations, use of RS techniques for accurate weight estimations and for analysis/optimization code integration, and the use of RS based weight equations in design sensitivity to uncertainty are presented in Chapter 2. Chapter 3 provides the background of RS methodology. Uncertainty due to use of RS is given in Chapter 4 by deriving a measure for high modeling error locations in the design space. Chapter 5 presents an application of the methodology for weight equations of a high speed civil transport wing weight problem, and its use in design for uncertainty.

PAGE 25

11 Chapter 6 summarizes similar weight equation construction for a stiffened composite panel with a crack problem and Chapter 7 presents the concluding remarks for this dissertation.

PAGE 26

12 CHAPTER 2 LITERATURE SURVEY Weight equations allow designers to design systems without getting bogged down in the re-design of components as they change system characteristics. In aircraft design the upper level system is the aircraft itself and the main structural components are wing and fuselage. These in turn can be considered as systems of components such as panels, frames, etc. The flight optimization system (FLOPS) wing weight equation (McCullers, 1984), for example, predicts how the aircraft structural weight changes with use of composite materials without making panel and wing level detailed and computationally burdensome structural analyses or optimization. Therefore, accurate and easy-to-use weight equations are a useful and important tool for the designer. Shanley (1960) published weight prediction methods based on elementary strength/stiffness considerations, augmented by experimental results and statistical data. This approach enabled weight engineers to obtain more accurate weight estimates than traditional empirical weight equations and to consider sensitivity of weight to the design. Moreover, it also paved the way to the ‘station analysis’ methods and use of more involved analysis models. In general, structural weight equations have been developed for various types of wing and fuselage structures since they are primary load-carrying elements. Shanley (1960) reported that existing empirical formulas were sufficiently efficient for the other

PAGE 27

13 portions of the airplane structure. This appears to hold even for recent aircraft concepts. Balabanov et al. (1996, 1998 and 1999) investigated weight due to wing bendingresistant structural elements, wing bending material weight, of a futuristic high-speed civil transport (HSCT). They used empirical formulas from the FLOPS program (McCullers, 1984; 1997) for the entire structure except for the wing structural weight as in Grasmeyer et al. (1998) and Gern et al. (1999) for another unconventional concept, transport aircraft with strut-braced wing. The focus of this work is on accurate weight equations for wings and for panels. The former are used by the aircraft designer to design the entire airplane; the latter are used by the wing or fuselage designer to design the entire wing or the entire fuselage. Wing (System level) Structural Weight Equations Wing weight equations started with fitting of historical data using specially tailored expressions with variables raised to various powers. They progressed to equations based on stress analysis of simple beam models of the wing and fuselage that were adjusted to fit historical data. Shanley (1960) reviewed the work by Lipp and Kelley. Lipp (1938) developed formulas for the structural weight of wings by actual integration of assumed lift distribution in the basic equations for the conditions of shear and bending. The final equations were expressed in terms of load factor, span, root thickness, allowable stress and taper ratios for chord and thickness. Kelley (1944) used more accurate spanwise lift distributions that increased the difficulties in the integration and presented results in chart form for engineering use. Shanley (1960) modeled the wing as a box beam whose flanges carried the bending load whereas webs carried shear loads. He indicated that the

PAGE 28

14 most important terms are span, mean aerodynamic chord, and the ratio of wing span to depth. Torenbeek (1992) presented a method providing estimations for the primary parts of a wing structure. The method makes use of elementary stress analysis combined with historical data. The wing group weight is expressed as the sum of the primary weight (top and bottom covers, spars, ribs, and attachments) and the secondary weight comprised of the weight of components in front of the front spar, components behind the rear spar, plus any miscellaneous weight. The basic weight is the minimum weight of the primary structure required to resist bending and shear loads where stresses are taken into account. Formulas based on historical data were used in calculations of the secondary weight and for the correction to the primary weight due to factors such as joints and mountings for engines. Prediction errors of less than 4% were found for four representative transport aircraft with take-off weights between 50 and 3500 kN. Weight equations also progressed to using a multitude of structural design optimizations (a review for structural optimization was presented by Vanderplaats, 1999) based on simple plate models. McCullers (1984) developed the weight equations in FLOPS by combinations of historical data and structural optimizations based on equivalent plate models for wings using the wing aeroelastic synthesis procedure (WASP) and the aeroelastic tailoring and structural optimization (ATSO) program (McCullers and Lynch, 1972; 1974). Optimum wing designs data were used to account for aeroelastic tailoring via composite materials, forward sweep angle effects and flutter and divergence weight penalties for very high aspect ratio wings on the wing weight. The equations are based on analytical expressions of primary wing configuration

PAGE 29

15 parameters such as aspect ratio, thickness-to-chord ratio, sweep angle, etc. correlated by constant factors based on historical data and results of structural optimizations. For different categories of aircraft the constants and the expressions vary, so that equations are applicable for a range of aircraft types, from military fighters to civil transonic and even supersonic transport aircraft. These and similar weight equations attempt to fit any airplane, or at least a broad class of airplanes. These types of equations are referred to as quasi-analytical estimation in Staton (1996). They offer very good estimation accuracy for existing technology aircraft, and through their sensitivity to engineering parameters allow accurate trade studies. Their advantages over pure statistical equations lie in their use in extrapolating outside parameter ranges in the database of aircraft. One modern trend has been to replace wing weight equations with structural optimization based on beam models inside the aircraft optimization problem. For example, Kroo et al. (1991) used this approach in aerodynamic-structural design studies of joined-wing aircraft. They combined a vortex-lattice aerodynamic model with fully stressed beam sizing routine. Two structural box models: a symmetrical box of uniform cap and web thickness, and an asymmetrical arrangement of cap thickness so that diagonal corners of the box have the same thickness were used. They investigated selected joined-wing designs and studied the effects of design parameters on drag and structural weight. They employed a simple structural optimization following the aerodynamic analysis for each design. In later studies (Kroo et al., 1994; Braun et al., 1996), they used two-level collaborative optimization that allows the designer to include other disciplines besides structures and aerodynamics. Collaborative optimization allows some autonomy for each

PAGE 30

16 discipline to optimize on its own, but the two-level formulation can be very ill conditioned (Alexandrov and Lewis, 2000). Gern et al. (2000) also embedded a beam-model structural optimization for weight calculations of a strut-braced transport wing inside the aerodynamic optimization. They employed a fully stressed design procedure accounting for aeroelastic load distribution to estimate the wing bending material weight. The remaining portion of the wing structural weight due to flaps, slats, spoilers, ribs etc. was estimated by FLOPS weight equations. Finite element (FE) analysis based procedures have also been used in the preliminary design phase. Advances in the computational resources allow the use of at least coarse FE models in this phase. Huang et al. (1996) performed simultaneous structural-aerodynamic optimization by combining a FE based structural optimization for wing bending material weight (structural weight needed to resist bending) with a simple weight equation. Weight prediction from structural optimization was first compared with two quasi-empirical wing weight equations for ten high speed civil transport (HSCT) wing planforms. The equations were from FLOPS (McCullers, 1984; 1997) and by Grumman Aerospace Corporation (York and Labell, 1980). The comparison indicated that the weight equations predicted well the average structural weight and effects of airfoil thickness on structural weight, but did not model accurately all of the effects of planform changes on the structural weight. The FLOPS weight equation was chosen since it agreed better with the structural optimization results than Grumman equation. The structural optimization and weight equations were combined through the following interlacing procedure: structural optimization and FLOPS weight estimations were found

PAGE 31

17 for the initial design and their ratio was calculated. Five cycles of aerodynamic design optimization were performed by using the FLOPS weight estimation corrected by the ratio. After five cycles, the structural optimization was repeated for the new aerodynamic design and the ratio was updated. These steps were repeated until convergence. The interlacing procedure of the FLOPS weight equation and the FE based structural optimization results was successful in terms of obtaining designs reflecting realistic weight estimation balanced with computational cost. It did not protect, however, the optimization from getting stuck at a design where the simple equation was too optimistic. It was noted that derivative of the ratio or scale factor may be helpful for this problem. Performing repeated structural optimizations during the aerodynamic design process has the advantage that use of a general structural weight equation that must fit all transports is not required. However, this approach has three main disadvantages: noise, ill-conditioning, and code/discipline integration. Giunta et al. (1994) showed that numerical noise caused convergence problems in design optimization for HSCT employing gradient-based optimization techniques. Balabanov et al. (1996) had to deal with numerical noise of several sources in their HSCT design problem. Venter and Haftka (1998) experienced noise associated with the dependence of discretization error on their shape design variables. Multi-level treatments, such as collaborative optimization can lead to ill-conditioning and computational difficulties (Alexandrov and Lewis, 2000). In addition, the integration of structural optimization software with the other disciplines also requires substantial effort (e.g. Rohl et al., 1994). An alternate approach that avoids the above difficulties is to generate a weight equation for a particular aircraft by fitting the data generated via finite element based

PAGE 32

18 optimizations within the context of RS methodology. This approach has the advantages of the traditional weight equation, but without its disadvantages. The main difficulty is obtaining adequate accuracy, and this is one of the thrusts of the present work. Response surface techniques have become an important tool in multi-disciplinary optimization (MDO) studies where numerical noise is often an issue besides the organizational difficulty of coupling simulations from different disciplines. They filter out numerical noise, and provide a convenient representation of data from one discipline to another and are easy to interface with an optimizer due to their ease of implementation. Kaufman et al. (1996) fitted quadratic polynomial RS approximation to the structural weight obtained with multiple structural optimizations with GENESIS (VMA, 1997). The HSCT configuration optimizations performed with the RS produced superior results than with the FLOPS weight equation. Balabanov et al. (1996, 1998 and 1999) investigated RS construction for wing bending material weight of HSCT based on structural optimization results of number of different configurations and its use in the configuration optimization problem. They used a larger number of low-fidelity (LF) models and tuned the results based on small number of high-fidelity (HF) analyses (Balabanov et al., 1998) in order to handle the computational cost problem associated with FE based structural optimization. Papila and Haftka (1999, 2000a ) also constructed RS weight equations based on structural optimizations (see chapter 5). Panel (Component Level) Weight Equations in Multi-Level Procedures Wing and fuselage structures are composed of large number of curved or flat, usually stiffened panels. Designing all the wing panels simultaneously constitutes a

PAGE 33

19 complex optimization problem requiring detailed structural modeling of the entire wing and appears to be beyond present computational capabilities. For integrated structural wing (as system) and panel (as component) design, there are similar approaches to those described in the previous section. In the past, there were simple, stress-analysis based panel weight equations like Gerard's (1960). He performed minimum weight analysis of various forms of stiffened-plate construction considered as the compression covers of multi-cell wing box beams. This, however, did not take into account the effects of panel stiffness on load redistribution (designer can help one panel by beefing up another) and overall stiffness constraints. The current practice is to design the system, wing or fuselage structures at two levels; global (system) and panel (component) levels. At the global level, the structure is modeled with moderate level of detail for individual panels, and the internal load distribution is obtained. With these loads, panel-level design optimization for detailed geometry is then performed by programs such as Panel Analysis and Sizing Code (PASCO) by Stroud and Anderson (1981) or PANDA2 by Bushnell (1987). Depending of the complexity of the problem, integration of global and panel levels may become difficult. Schmit and Mehrinfar (1981) showed that the integration of the levels can be handled well for simple structural models by a two-level decomposition similar to collaborative optimization (CO) approach. One difficulty associated with the decomposition; elimination of the component level (local) variables in terms of system level (global) variables was mentioned in Haftka and Grdal (1992). For panel problems, as well as for complex truss and frame cross-sectional forms, it is impossible to find analytical expressions for eliminating local variables and replacing them with

PAGE 34

20 global variables. It is possible to keep both local and global; variables, and supplement the problem with equality constraints that guarantee the consistency of the global variables with the local variables (Haftka and Grdal, 1992). However, this approach often tends to make the optimization problem more ill-conditioned (e.g. Thareja and Haftka, 1986). As noted in the previous section two-level optimization suffers from illconditioning and the fact that panel (local) level optima are not usually smooth function of the global level design variables. A successful integration of wing (system or global) and panel (component or local) level structural optimizations was performed by Rohl et al. (1994) in three-level decomposition approach for the preliminary aerodynamic-structural design of an HSCT wing. The levels of the approach from top to bottom, made use of FLOPS as a general aircraft sizing and performance code for aerodynamic optimization, ASTROS as a structural optimization tool based on FE analyses for wing box (wing level), and PASCO for panel-level optimization. The main purpose of the paper was to demonstrate the integration of PASCO into ASTROS to supply buckling constraint evaluation for wing level optimization, and its application in a multi-level wing design procedure. The overall integration and execution of the procedure were performed by UNIX shell scripts calling specifically designed finite element pre-processor for ASTROS using the FLOPS output, PASCO pre-processor, and post-processor. The results of this integration prototype without RS application indicated that it worked, but required the designer in the loop making decisions in buckling optimization. Besides the modifications on the codes and customized interfaces, it was noted that fully automatic run requires a logic implementation.

PAGE 35

21 Response surface approximations offer an attractive means also for integrating the design of a single panel with the overall structure and for integrating different codes. Ragon et al. (1997) introduced a methodology for global/local design optimization of large wing structures. They used ADOP, Aeroelastic Design Optimization Program of McDonnel Douglas in global/local optimization, and PASCO, Panel Analysis and Sizing Code as the local design code. The methodology starts with generating many optimal designs for each panel type for a range of loading and in-plane stiffnesses (A11 and A66). The data generated were then utilized to construct an RS weight equation as a function of the in-plane loads and stiffness parameters for the optimal panel weight satisfying all the local failure constraints. However, for some values of the stiffness parameters no feasible designs exist. When the panel RS weight equation was interfaced with ADOP, during the global iterations ADOP tended to move into infeasible portions of the design space. This was because no data corresponding to the infeasible design points was included in the RS model, and the design regions bordering the infeasible regions tended to be those with low structural weights. This problem was corrected by constructing a new RS using the data that included penalty terms for those original design points for which PASCO could not generate a feasible design. The penalty terms required higher order (cubic) polynomial for accurate fitting. The use of RS in terms of stiffness constraints is an advance over traditional weight equations. It allows the global design to control the load carried by the panel by specifying the panel in-plane thickness. In other global/local procedures responses other than weight were approximated by RS easing the code integration. Venkataraman and Haftka (1997), for example, utilized separate local and global analysis codes for a liquid-hydrogen tank structure

PAGE 36

22 design made of laminated-composite material. They used NASTRAN for the global model of irregular geometry and PANDA2 for the local regular stiffened panel analysis. An iterative design procedure between NASTRAN and PANDA2 failed to converge since the latter did not have any information of the global constraints. They overcame this problem by a RS approximation for the global response used in PANDA2. Liu et al. (2000) aimed to solve the infeasibility problem faced by Ragon et al. (1997) who needed to satisfy both stiffness and strength constraints at the panel level. Instead of weight equation Liu et al. developed failure load equation. Their procedure was applied to laminated composite wing panels when the design involves discrete and combinatorial optimization. Global and local design processes were coordinated through an equation that predicts the buckling load multiplier. The equation was a cubic RS as a function of number of plies of each orientation and the loads on the panel. Response surface was fitted to a large number of panel stacking optimization based on analytical solution for simply supported unstiffened panel in the configuration space of number of plies and the loads. The reason for the cubic model is that buckling load is proportional to the cube of the thickness. They used D-optimal experimental design points where panel level optimizations by permutation genetic algorithm (GA) for the stacking sequence. The objective function was maximum buckling load. Then wing-level optimization was performed by Genesis using the cubic RS in the buckling constraint. They applied the methodology for 6-, 18-, and 54-variable cases. It was shown that a cubic RS can fit to the buckling load of the optimal panel stacking sequence as a function of loading on the panel and the given number of plies, and it can be used in order to avoid the problem of infeasible designs faced by Ragon et al. (1997).

PAGE 37

23 Ragon et al. (1997) and Liu et al. (2000) generated RS based equations with panel (component) design based on simple models. One goal of the present work is to push the procedure further in the context of general FE modeling of stiffened panels. Accuracy of Response Surfaces Increasing use of RS techniques in design optimization studies has brought attention to ways of increasing the accuracy of the RS approximations. Two sources of error have been mainly investigated: numerical noise and modeling error due to approximation functions. Efforts for identifying the cause of the numerical noise, for improving the data by reducing the noise and by proper selection of design points-Design of Experiments (DOE)-and for reducing modeling errors pay back in the accuracy of the RS and in its reliable application. Numerical experiments may be noisy due to several reasons including discretization errors, incomplete convergence of iterative procedures, and round off errors. In order to reduce the effect of noise in RS accuracy DOE offers variance optimal designs. Standard designs such as central composite design (CCD), for example, are quite efficient against noise effect (Montgomery and Myers, 1995). However, there are high-dimensional problems where more economical designs are generally required. Doptimal design criterion (Montgomery and Myers, 1995; Khuri and Cornell, 1996), for example, is commonly used in such problems. Balabanov et al. (1996) used D-optimal design for 29-variable HSCT design problem in order to reduce the number of structural optimizations. Response surface of an effective experimental design providing low variance level can be used as a data-filtering tool. Venter et al. (1998a and 1998b ) made use of RS

PAGE 38

24 approximations for filtering noise in finite element analyses. In their case the noise was associated with the dependence of discretization error on shape design variables. Giunta et al. (1994) found numerical noise in their calculation of aerodynamic drag components for the aircraft that caused convergence problems in design optimization for HSCT. They demonstrated the noise in a two dimensional example that showed variations were small so that at all points the accuracy of the drag was acceptable, but oscillatory behavior created difficulties for the gradient-based optimization technique. Quadratic RS approximation was used to filter out the noise in aerodynamic simulations. Giunta et al. (1997) applied the RS approach for 10-variable HSCT configuration optimization problem. They reported improvement in designs obtained based on the smooth RS approximation compared to the original noisy simulations. Use of RS resulted in virtually identical optimal design starting from two different initial designs whereas using the original noise-producing aerodynamic model method resulted in different optimal designs. Balabanov et al. (1996) investigated the noisy behavior of wing bending material weight of HSCT designs (that was indicated by Kaufman et al., 1996, but without identifying its source). They found that a large portion of the noise was due to incomplete optimization of the wing camber that affects the wing bending material weight via the aerodynamic loads. Much of the remaining numerical noise was due to the structural optimization process itself through incomplete convergence. Accuracy of bending material weight RS after correcting the camber optimization problem increased substantially compared to one in Kaufman et al. (1996). To improve the accuracy further, minimum bias and minimum variance design strategies were tried. They worked

PAGE 39

25 comparably well in terms of root-mean-square-error (RMSE) and average error. The maximum error, however, is about 30% less in minimum bias design based RS. While low-amplitude, random noise is filtered well by RS approximations, data with large errors--outliers--may cause significant loss of accuracy. Therefore, robust regression employs techniques for statistical methods, such as iteratively re-weighted least square (IRLS) fitting (Holland and Welsch, 1977), to detect and remove or weight down outliers. Once detected, outliers can be investigated further for possible mistakes. These techniques help also to identify flaws in analysis/optimization procedures so that improvement on the results can be achieved. Optimization procedures often produce poor results due to algorithmic difficulties, software problems, or local optima. When a single optimization is flawed, it may be difficult to detect the problem, because the ill conditioning responsible for the problem may make it difficult to apply optimality criteria unambiguously. However, in many applications a large number of optimizations are performed for a range of problem parameters (e.g. Balabanov et al., 1999). When such multiple optimizations are available, statistical analysis can be done to detect incorrect optimal results. For example, Papila and Haftka (1999 and 2000a ) and Kim et al. (2000 and 2001) used IRLS procedures to detect such outlier points. More details on these references are presented in Chapter 5. Researchers have also found ways for dealing with modeling error. For instance, it can be decreased by proper selection/definition of variables/factors. Concept of intervening design variables and intervening functions was introduced by Schmit and his coworkers for accurate approximations (e.g. Schmit and Farshi, 1974). They are transformations of the original variables and functions or arguments of the functions.

PAGE 40

26 Finding/defining intervening variables as functions of the original variables would help to model response with lower order polynomials (or lower order derivatives in case of local Taylor series approximations), and to decrease computational cost to obtain the approximation (Haftka and Grdal, 1992). Another way of decreasing modeling error and increase the RS accuracy is use of reasonable design space approach (Balabanov et al., 1997). The approach seeks inexpensive constraints such as simple geometric constraints that prevent combinations of design variables resulting in unreasonable geometry configurations. Eliminating large portions of the design space occupying those unreasonable configurations renders the design space more similar to a simplex. The approach improves the modeling accuracy of the RS by shrinking the region where the RS is fitted. Kaufman et al. (1996) investigated the ways of improving the RS accuracy for weight equations. They looked for intervening variables that may reduce the number of data points or increase the accuracy. They also used inexpensive approximate analysis methods and geometric constraints to find the regions of the design space where reasonable HSCT configurations are likely to appear. Unreasonable designs were not eliminated, but moved towards the reasonable region so that they reside on the edges of the reasonable design space. This approach did not reduce the number of structural optimizations, but caused increase in accuracy. The reasonable design space approach was also applied by Roux et al. (1998) to truss test problems and by Balabanov et al. (1999) in fitting RS weight equation for HSCT wing. Knill et al. (1999) demonstrated a method enabling the efficient implementation of accurate computational fluid dynamics (CFD) predictions into high dimensional,

PAGE 41

27 highly constrained MDO problems like HSCT. They utilized simple conceptual level models to select an appropriate set of intervening functions for which to construct RS models since accuracy with low-order polynomials can be improved by transformations of the real function or its arguments. Supersonic linear theory aerodynamics was used as low-fidelity model to generate data for full term RS approximation. Stepwise regression was then used with this data, and RS models in order to determine the significant terms/variables in calculation of the desired aerodynamic quantities. Computationally expensive Euler solutions were performed next only for set of configurations reflecting the effects of the significant terms. This reduced substantially the number of high-fidelity Euler analyses required. They used the reduced-term RS models with Euler analyses as an additive correction RS. Another way for reducing modeling error is to use higher order polynomials or more complex functions in RS. Venter et al. (1996) made use of dimensional analysis for reducing the number of variables and the design space for minimum weight design of plate with a step. The reduction in the number of variables made use of higher order polynomials in RS model affordable. They used cubic and quartic polynomials and improved the accuracy for the stress and buckling constraints. Papila and Haftka (1999 and 2000a ) also achieved substantial improvement in accuracy by using cubic polynomials in RS approximation for HSCT wing bending material weight (see in Chapter 5). Neural network (NN) based RS approximations have also been investigated as alternative to traditional polynomial-based RS approximations within the context of design optimization. They allow fitting highly non-linear response. Their main

PAGE 42

28 drawbacks are the cost of fitting the approximation and lack of statistical tools for effective design of experiments. There are comparisons in the literature of the performance of NN-based and polynomial-based RS approximations. For example, Carpenter and Barthelemy (1993) used NN-based and polynomial-based approximations to develop RS for several test problems. They found that two methods perform comparable based on the number of undetermined parameters. Papila (N.) et al. (1999) investigated the effect of data size and relative merits between polynomial and NN-based RS in handling varying data characteristics. They demonstrated that as the nature of the experimental data becomes complex NN-based approximations perform better than polynomial-based. The neural network technique and the polynomial RS were integrated to offer enhanced optimization capabilities by Shyy et al. (1999 and 2001) and Papila (N.) et al. (2001). Vaidyanathan et al. (2000) applied NN and polynomial-based RS approximations in the preliminary design of two rocket engine components, gas-gas injector and supersonic turbine. They demonstrated that NN-based and polynomial-based approximations can perform comparably for modest data sizes. Papila (N.) et al. (2000) investigated the performance of both polynomial and NN-based RS in their multiobjective turbine blade design optimization problem. Oftentimes, designers sacrifice the fidelity of the analysis model used in data generation for RS approximation due to computational cost. This may cause decrease in accuracy of the data and approximation. There is a simple way to improve accuracy in such cases: correction response surfaces (CRS) that make use of low-fidelity (LF) results with small number of high-fidelity (HF) results. Mason et al. (1994), for example, studied design optimization of curved composite frames typical of a support structure for

PAGE 43

29 semi-monocoque aircraft structures and applied two-levels of fidelity and correction of HF solutions onto the LF ones. Vitali et al. (1997) also used two levels of fidelity and RS technique in design of hat-stiffened panel for blended wing body aircraft. Venkataraman et al. (1998) demonstrated effective combination of codes/models at different fidelity levels by CRS for structural optimization of a ring-stiffened cylindrical shell of revolution. These works were examples of CRS in component design. Balabanov et al. (1998) used the CRS approach to improve the accuracy of RS based wing weight equations. They used coarse FE models for thousands of HSCT configurations, and a quadratic RS was constructed for wing bending material weight. Then about a hundred of the configurations were optimized with a refined FE model to construct a linear CRS. Both additive and multiplicative corrections were applied. The linear additive correction was found slightly better for their design problem. They tried to correct either the prediction of the RS or to correct the data themselves by the highfidelity optimizations. The correction reduced errors by more than half. Another important relevant work in the sense that LF/HF coupling of numerical simulations applied was by Knill et al. (1999). Their main contribution is that besides CRS they used the RS fitted on LF results to identify the important polynomial terms, so that the RS model of HF had fewer terms and required fewer analyses. Design of experiments offers also minimum-bias criterion for reducing modeling (bias) error. Venter and Haftka (1997) developed an algorithm implementing minimumbias based criterion, necessary for an irregularly shaped design space where no closed form solution exist for minimum-bias design. They compared minimum-bias and Doptimal designs for two problems with two and three variables. Minimum-bias design

PAGE 44

30 based RS was found more accurate than D-optimal for the two problems except for the percent maximum error. Percent maximum error was much bigger by minimum-variance design in Balabanov et al. (1996). Errors in RS approximations represent a part of the uncertainty in the design process. The studies discussed above attempt to increase the accuracy and to decrease the level of uncertainty associated with the use of RS. Another important objective of the present work is try to characterize further the errors in RS predictions so that the uncertainty introduced by them can be quantified. Use of RS in Uncertainty Quantification During the design of a new aircraft concept like HSCT, modifications and changes in geometry and material properties are likely. Therefore, it is important to determine how sensitive to change the design is. Uncertainty can be studied by probabilistic or by fuzzy set models. In both cases, design subject to uncertainty is computationally expensive. Response surface approximations alleviate the computational burden because they allow for an inexpensive evaluation of the effect of changes although they cause uncertainty of some degree themselves (Papila and Haftka, 2001a ). Reliability based design studies made use of RS approximations (Schueller et al., 1989; Rajashekhar and Ellingwood, 1993; Romero and Bankston, 1998). Xiao et al. (1999) used a reliability based MDO for shape optimization of a transport aircraft wing, treating material properties, geometric variables and loading conditions as random variables with assumed probability distributions. They constructed RS approximations for the objective function and all the constraints and applied Monte Carlo simulation for calculating reliabilities.

PAGE 45

31 Qu et al. (2000) investigated the use of RS combined with efficient Monte Carlo simulation techniques in reliability-based design optimization of angle-ply laminates subjected to mechanical loads at cryogenic temperatures. Uncertainties for the problem were in material properties due to short of characterization through the entire operating temperature range, its failure modes at cryogenic temperatures, stiffness and strength properties manufacturing dependent. Response surface approximations were used to reduce computational cost and filter out random sampling noise in the Monte Carlo simulation. Fuzzy set model of uncertainty and RS approximations, and within the vertex method (Dong and Shah, 1987) were employed by Venter and Haftka (1998b ), for calculating the possibility of failure in dropped-ply composite laminate design. Same approach was also used by Papila and Haftka (1999) for sensitivity to uncertainty of wing bending material weight in HSCT designs (See Chapter 5).

PAGE 46

32 CHAPTER 3 RESPONSE SURFACE METHODOLOGY Response surfaces (RS) approximate numerical or physical experimental data by an analytical expression that is usually a low-order polynomial. Khuri and Cornell (1996, p. 3) described the three key steps within the context of the methodology: Pre-selecting the points-design points-where experiments are performed. This step is called design of experiments to represent the design space so that the points selected will yield adequate and reliable measurements/calculations of the response of interest. Determining a mathematical approximation model that best fits the data collected/generated from the set of design points of the previous step. This is performed by conducting appropriate statistical test of hypotheses concerning the model’s parameters. Using the RS approximation for predicting the response for a given set of the experimental factors/variables. A common application is to determine the optimal settings of the factors that produce optimum (maximum or minimum) value of the response. This chapter summarizes first the design of experiments used in this dissertation. Section starting on page 36 presents the standard least-squares fitting technique. In the section starting on page 39 statistical tools to check the accuracy of the RS are given

PAGE 47

33 including a customized lack-of-fit test (page 52 ). Then, section on page 58 presents Iteratively Re-weighted Least Squares (IRLS) accommodated with the standard weighting function. Design of Experiments Experimental designs possess the following properties required to fit a pth order polynomial RS for k variables/factors: at least p+1 levels of each factor at least ) ).....( 2 )( 1 (p p k k k distinct design points Montgomery and Myers (1995, p. 280) summarized important characteristics that make choice of experimental design effective. An efficient design will Result in a good fit of the model to the data Give sufficient information to allow a test for lack of fit Allow models of increasing order to be constructed sequentially Provide an estimate of “pure” experimental error Be insensitive (robust) to the presence of outliers in the data Be robust to errors in control of design levels Be cost effective Provide a check on the homogeneous variance assumption Provide a good distribution of prediction variance Although not all of the above properties are required in every RS experience, most of them must be given serious consideration on each occasion where one designs experiments.

PAGE 48

34 In many practical situations the scientist or engineer specifies strict ranges on the design/configuration variables (experimental factors). That is region of interest and region of operability are the same and a cuboidal. In general, factors of the experiments or configuration variables iv in this work have different order of magnitudes and different units of measurement. To avoid ill-conditioning and simplify the numerical calculations used in the parameter estimates for the response surface approximation, variables are coded by Eq. (3.1) so that the smallest and the largest limits for configuration or control variables are assigned as –1 and +1, respectively. 2 / )] min( ) [max( 2 / )] min( ) [max(i i i i i iv v v v v x (3.1) For the application problems of cuboidal design regions in this study, different experimental designs and their combinations were considered based on the number of factors or configuration variables and the degree of polynomial used. As observed from the literature for the engineering design problems studied in this work quadratic polynomial RS approximations are commonly used (e.g. Balabanov et al. (1996, 1998 and 1999). Cubic or even higher order polynomials, on the other hand, were also applied (e.g. Venter et al., 1996); Papila (N.) et al. 1999). In this work, quadratic polynomials are used as an initial model and based on the accuracy achieved first order and cubic polynomials are also studied. Three-level full factorial designs (FFD) and face-centered central composite designs (FCCD) are typically applied for quadratic models depending on the number of factors involved. These experimental designs are schematically shown in Figure 2 for the case of three coded factors or configuration variables. In the cuboidal region, it is important that the points be “pushed to the extreme” of the region and it is

PAGE 49

35 important for the region to be covered in a symmetric fashion. These properties result in attractive distribution of prediction variance (Montgomery and Myers, 1995, p. 312-313). Three-level FFD and FCCD accomplish this. Figure 2: Three-level full factorial (left) and face centered central composite (right) experimental designs for three coded-configuration variables With the increase in the number of factors, initial experimental designs considered here move from the full factorial design to FCCD and with more factors added from FCCD to a D-optimal (Khuri and Cornell, 1996; SAS, 1998) subsets of FCCD. Moreover, the feasibility of the data points and their usefulness in terms of design performance is checked by quick, but representative evaluations where possible. Unreasonable designs are removed in order to reduce the number of designs from the original FCCD to provide feasible or reasonable designs for the D-optimal selection. When higher order (e.g. cubic) polynomials are needed, orthogonal arrays (Owen, 1994) with four or more levels or the addition of a sub-box in the original design space is used to provide the required number of levels. Table 2 presents a summary of the experimental designs used in this study for number of factors and degree of polynomials employed. x 3 x1 x2 x 3 x1 x2

PAGE 50

36 Table 2: Experimental designs used in the present work # of factors Quadratic polynomial Cubic polynomial 2 Three-level full factorial design Three-level full factorial design 3 Three-level full factorial design 5 FCCD + subFCCD + orthogonal array 10 D-optimal subset of (FCCD infeasible designs) + orthogonal array Fitting Procedure: Standard Least Squares The response at jth design point xj is denoted as yj [) (jyx ] and given by Eq. (3.2) j j jy (3.2) where j [) (jx] is the true mean response and j represents other random sources of variation such as measurement error and noise not accounted for in In other words if the experimenter has the luxury of performing the experiment many times at xj, the average of the observations will tend to j despite the random errors j. In RS technique the true mean response is assumed to be given in terms of coefficients is and shape functions ) (j ifx where i =1, nb as f x xT j j i n i i jfb ') ( ) (1 (3.3) where ) ( ) ( ) (2 1 j n j j jbf f f x x x fT! and T!bn !2 1 Bold face is used for vector and matrix representations, e.g. Txj j jx x2 1 for two-variable case where superscript T stands for transpose operation

PAGE 51

37 The shape functions are usually assumed as monomials. For instance, for quadratic RS in two-variables Eq. (3.4) shows the shape functions and their vector representation ( ( ( ( ) ( ( ( ( + ( ( ( ( ( ( ( ( ( ( ( ( ) ( ( ( ( + ( ( ( ( ( ( ( ( 2 2 2 1 2 1 2 1 6 5 4 3 2 11 ) ( ) ( ) ( ) ( ) ( ) ( ) (j j j j j j j j j j j j j jx x x x x x f f f f f f x x x x x x x f f (3.4) The methodology (Myers and Montgomery, 1995; Khuri and Cornell, 1996) makes assumptions, so-called ideal error assumptions (or independent and identically distributed, i.i.d. assumptions) The RS expression is exact and represents the true mean response The error j in the experiments is un-correlated, normally distributed random noise. This random variable has expected value of zero, 0 ) ( jE, and constant variance, 2) ( jVar throughout the design space. Or, ) 0 ( ~2I "N where is error vector and I is the identity matrix. The method of least squares is used to estimate the coefficients is. The least squares function L is the sum of squares of the difference between collected response data jy and the assumed true mean response j [Eq. (3.3) ] ''/ 0 1 2 3 4 N j n i j i i jbf y L2 1) ( x (3.5)

PAGE 52

38 The function L is minimized with respect to is, and least-square estimators bis of is satisfy b N j n k j k k j j i b in i f b y f Lb i,...., 2 1 0 ) ( ) ( 21 ! # $ $ % & '' x x (3.6) This yields a system of linear equations that can be solved for bis. Then the RS approximation is given as b f x xT j j i n i i jf b yb ') ( ) ( ˆ1 (3.7) The difference (residual) between the data yj and the estimate defined in Eq. (3.7) is given for the jth point xj as ) ( ˆj j jy y e x (3.8) In general case ) ( ˆjy x may not be the true mean response and the difference from the data may not be only the random error, therefore the residual error is denoted by ej instead of j. Matrix forms of Eq. (3.5), (3.6), (3.7) and (3.8) for N data points are X X y X y yT T T T T 2 L (3.9) 0 Xb X y X !T T b 2 2 L (3.10) Xb y ˆ (3.11) Xb y e (3.12)

PAGE 53

39 where X is the matrix whose terms in the row associated with the point xj are formed by shape functions ) (j ifx. For instance, for a quadratic model in two-variables with N data points [see Eqs. (3.3) and (3.4) ] X is given as / / / / / / / / 0 1 2 2 2 2 2 2 2 2 3 4 / / / / / / / / 0 1 2 2 2 2 2 2 2 2 3 4 2 2 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 22 22 12 2 12 22 12 2 21 21 11 2 11 21 11 2 11 1 1 1N N N N N N j j j j j j N jx x x x x x x x x x x x x x x x x x x x x x x x " " " " " " "T T T Tf f f f X (3.13) The system of equations, Eq. (3.10) may be written as y X Xb XT T (3.14) and the coefficient vector b can now be expressed as y X X) (X bT -1 T (3.15) If Eq. (3.14) is substituted in Eq. (3.9) the remaining error vector r e is found to satisfy y X b y y e eT T T T r r (3.16) that is called as sum of squares of the residuals SSE. Adequacy and Accuracy of RS Adequacy and accuracy of the RS approximation are mainly affected by the following three factors whose effects also interact with one another Use of finite number of data points Noise in the data Inadequacy of the fitting model

PAGE 54

40 If the true mean response is exactly represented by the RS form as assumed, the components bis of vector b in Eq. (3.15) are the unbiased estimates of the is, in other words the expected value of b is equal to as can be shown as ] ) ( ) [( )] ( ) [( ] ) [( ) (1 1 1 1" X X X X X X X X X X X y X X X bT T T T T T T T E E E E (3.17) Since 0 ) ( "E and I X X X) (XT -1 T (I is the identity matrix) b ) (E (3.18) However, when a finite number of design points are used the bis will not be exactly equal si due to effect of random error in the data. The variance of a random variable Y Var ( Y ), is the expected value of the squared difference between the variable and its expected value: ] )) ( [( ) (2Y E Y E Y Var (3.19) Similarly the variance-covariance matrix of a random vector Y, Var (Y) is defined as ] )) ( ))( ( [( ) (TY Y Y Y YE E E Var (3.20) Following Eq. (3.20) the variance-covariance of the coefficient estimator vector b is derived in Eq. (3.21) 1 2 1 2 1 1 1 1 1 1 1 1 1 1 1) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ] ) ( ) ][( ) ( ) [( ] ) ][( ) [( )] ( )][ ( [ ) ( X X X X X I X X X X X X X X X X X X "" X X X X X X "" X X X X X X X X X X X y X X X y X X X b b b b bT T N T T T T T T T T T T T T T T T T T T T T T T T T Var E E E E E E E Var (3.21)

PAGE 55

41 The variance-covariance matrix is symmetric and its ith diagonal (as bn i ... 2 1 ) characterizes the variance of estimator bi that causes variance of ) ( ˆjyx so-called prediction variance. The prediction variance can be expressed as [see also Eq. (3.7) ] j j j jVar Var y Varf b f b f xT T) ( ) ( ˆ (3.22) where ) (j jx f f Substitution of Eq. (3.21) into Eq. (3.22) gives prediction variance in Eq. (3.23). j j jy Varf X) (X f x-1 T T 2 )] ( ˆ [ (3.23) The random noise error variance 2 is not usually available and needs also to be estimated. Sum of squares of the residuals, SSE from Eq. (3.16) is used to develop an estimator 2s (error mean square) for 2. b r T r b E En N n N SS MS s e e2 (3.24) Sum of squares of the residuals ESS and evidently 2s represents variation in the data not accounted for by the fitting model. The error mean square 2s is an unbiased estimate of the noise error variance 2 provided that fitting model exactly models the true mean response Replacing 2 with 2s estimates for the variance-covariance matrix [Eq. (3.21)] and prediction variance [Eq. (3.23)] can be given as j j js y Var s Varf X) (X f x X) (X b1 T T 1 T2 2)] ( ˆ [ ) ( (3.25) Positive square root of the prediction variance is usually used as an estimate of the prediction error at a design point xj, also called estimated standard error as given in Eq. (3.26).

PAGE 56

42 j j j j ess y Var e f X) (X f x x1 T T )] ( ˆ [ ) ( (3.26) where estimator s is positive square root of EMS [Eq. (3.24) ] often called in engineering literature as root-mean-square-error (RMSE). The estimated standard error shows dependence on the location of the design point. Furthermore, as Eq. (3.3) is, in general, only an approximation to the true function, s will contain not only noise error, but also modeling (bias) error. The estimator s is also used to calculate coefficient of variation as given in Eq. (3.27) y s cv (3.27) where N y yN j j'1. Besides s and vc, the quality of the approximation is often measured by comparing the sum of squares of the residuals SSE (sum of squares unaccounted for by the fitted model) and the sum of squares due to regression SSREG (sum of squares explained by the fitted model). As SSE is given again by Eq. (3.28) the latter sum of squares is defined by Eq. (3.29) N j j j Ey y SS1 2) ˆ ( (3.28) N j j REGy y SS1 2) ˆ ( (3.29) where ) ( ˆ ˆj jy yx and y is the average of the data as used in Eq. (3.27). Their sum shows the total variation in the data set. It is called as total sum of squares SST and can also be expressed as

PAGE 57

43 N j j Ty y SS1 2) ( (3.30) The matrix forms of the sum of squares are given in Eq. (3.31) N SS N SS SST REG E 2 2) ( ) (y 1 y y y 1 y X b y X b y yT T T T T T T T (3.31) where 1 is a vector of N ones. Hypothesis Testing in RS Methodology Statistical hypothesis testing (Stark, 2001) is formalized as making a decision between rejecting or not rejecting a null hypothesis on the basis of a set of observations/calculations. Two types of errors can result from any decision rule (test): rejecting the null hypothesis when it is true (a “Type I error”), and failing to reject the null hypothesis when it is false (a “Type II error”). For any hypothesis, it is possible to develop various decision rules. Typically, the chance of a “Type I error” that researcher is willing to allow is specified ahead of time. The chance that the test erroneously rejects the null hypothesis when true, is called the significance level of the decision rule ( ). The observations are then used to calculate a test statistic. Researcher compares the test statistic (such as F-statistic) and its statistical table value (F-distribution) associated with the selected significance level If the statistic is larger than the table value, the test is considered as “significant” at level and null hypothesis is rejected. The most commonly used level of significance is 0.05 by which researcher accepts 5% probability of making a mistake by rejecting the null hypothesis.

PAGE 58

44 There are certain statistical hypotheses and relevant tests to measure the efficiency and reliability of the RS approximation model. These tests also presume the ideal error conditions. The tests are performed on a null hypothesis H0 against an alternative hypothesis Ha. A statistic customized for the testing is calculated and checked against a relevant statistical table of the statistic as a function of degree(s) of freedom that describes how many redundant results/measurements exist in an overdetermined system. One important test is the test of the significance of the fitted regression equation as formalized by the following hypotheses: zero not is intercept) n (other tha t coefficien one least At : 0 intercept) the (excluding ts coefficien All :0 a iH H (3.32) The test determines if there is a relationship between the response variable and a subset of the regression variables. Rejection of H0 implies that at least one of the variables play role on the response values. The test statistic for this test is the F-statistic as given in Eq. (3.33). ) /( ) 1 /(b E b REGn N SS n SS F (3.33) Decision based on this test at a significance level is made by comparing the Fstatistic calculated by Eq. (3.33) with the F-distribution value dfd dfnF, where dfn and dfd are degrees of freedom for numerator and degrees of freedom for denominator, respectively. For the test in Eq. (3.32) 1 bn dfn and bn N dfd The Fdistribution is the distribution of the ratio of two estimates of variance. The Fdistribution has two parameters dfn and dfd and may be denoted as dfd dfnF,. The dfn is the number of degrees of freedom that the estimate of variance used in the numerator is

PAGE 59

45 based on. The dfd is the number of degrees of freedom that the estimate used in the denominator is based on. The shape of the F-distribution dfd dfnF, depends on values of dfn and dfd. Figure 3 shows two examples for F-distribution. dfd dfnF, is the F-value where the rightmost tail area under distribution dfd dfnF, is equal to For example, 12 4 05 0F and 100 10 05 0F are equal to 3.26 and 1.93, respectively (associated tail areas equal to 0.05 are as shown on Figure 3). Figure 3: Sample F-distributions The value dfd dfnF, can also be defined as the upper 100 percentile of the distribution with degrees of freedom at numerator dfn and at denominator dfd. If the F statistic by Eq. (3.33) is beyond the upper 100 percent point of the distribution with 1 bn dfn and bn N dfd that is if b bn N nF F 1 ,, then the null hypothesis is in Eq. (3.32) rejected at the level of significance. This is interpreted that the variation accounted for by the model (through the values of ib other than intercept) is significantly greater than the unexplained variation (Khuri and Cornell, 1996, p. 31). 12 4F 100 10F

PAGE 60

46 In general, the test statistic calculated such as F in Eq. (3.33) is different for a null hypothesis than the table value of the selected significance level The statistic corresponds to a level in the table called p-value. The p-value is the smallest significance level for which any of the tests would have rejected the null hypothesis (Stark, 2001). Therefore, it provides a sense of significance of the evidence against the null hypothesis. The lower the p-value, the more significant the evidence and the test. If the significance level is set as 0.05, the test resulting in a p-value under 0.05 would be significant, and the null hypothesis is rejected in favor of the alternative hypothesis. In the test of the significance of the fitted regression equation formalized in Eq. (3.32) the p-value is determined as the rightmost tail area under the distribution b bn N nF 1 beyond the F statistic value by Eq. (3.33). For instance, if test statistic F is 2.50 on 100 10F ( Figure 3 ), then value p would be 0.01 (50 2100 10 01 0 F). This means the researcher can reject the null hypothesis at any significance level larger than 0.01 and conclude in favor of alternative hypothesis. Coefficients of determination 2 R and 2aR (2 R -adjusted), accompanying statistics to F-statistic, are respectively a measure and an adjusted measure of the proportion of total variation of the values of yj about the mean value of the response y explained by the fitted model (Khuri and Cornell, 1996, p. 31-32) and given as ) 1 /( ) ( 12 2 N SS n N SS R SS SS RT b E a T REG (3.34) An 2 aR value larger than 0.9 is typically required for an adequate approximation.

PAGE 61

47 Another hypothesis testing concerns the individual coefficients in the model: 0 : 0 :0 i a iH H (3.35) While the test for Eq. (3.32) is performed for the comparison of the two types of variance, the test for Eq. (3.35) is to check if the mean value for individual coefficients of the model is different than zero. Such a test would be useful in determining the significance of the each variable and regression coefficient in the fitting model. The test is performed by comparing the coefficient estimates to their respective estimated standard errors. The test statistic, t-statistic is defined by i i is b t (3.36) where si is the estimated standard deviation of ith coefficient bi (called standard error) and is obtained as the positive square root of the estimated ith term of the diagonal (as bn i... 2 1 ) of variance-covariance matrix given in Eq. (3.21) For the alternative hypothesis in Eq. (3.35) the test is two-sided. Absolute value of the calculated t-statistic is compared with the value obtained from the Student’s t-distribution table (denoted as dft) relevant to the degree of freedom, df=bn N Student’s t-distributions of different degree of freedom df are shown in Figure 4 The t-distribution is a symmetric, bellshaped probability distribution, similar to the standard normal curve. It differs from the standard normal curve, however, in that it has an additional parameter, called degrees of freedom, which changes its shape. The table value bn Nt 2 for a level of significance is the upper 100(2 / ) percentile of the distribution dft. For instance, the table values of different degree of

PAGE 62

48 freedom indicated on Figure 4 are values concerning level of significance 05 0 The area beyond 3.18 for df=3, for instance, is equal to 025 0 2 / Figure 4: The t-distributions of various degrees of freedom df The statistical significance or p-value for this test is determined as two times the area under the t-distribution dft and beyond the absolute value of t-statistic calculated by Eq. (3.36) For example, if one tests the null hypothesis 0 :1 0 H and calculates statistic t=4.0 with df=3, from the relevant t-distribution the p-value is found 0.028 (= 2x0.014) as shown on Figure 4. This means that, there is a 2.8% chance of making an error to reject the null and say b1 0. If researcher chooses to work with a level of significance more than or equal to 0.028 then the test concludes that there is no significant evidence to accept the coefficient being zero. Standard Lack-of-Fit Test Lack-of-fit test is used as a tool to check if the fitting model is not adequate and it does not contain a sufficient number of terms. In order to distinguish the fitting model and the missing terms subscripts, 1 and 2, respectively, will be used in matrix and /2=0.025 4.00 p -value/2=0.014

PAGE 63

49 function forms in the formulation. The fitting model and the true model are represented as in Eqs. (3.37) and (3.38), respectively. X y1 1 (3.37) 2 2 1 1! X X y ) (E (3.38) where 1X and 1! respectively denote the model previously denoted by X and in the second section of this chapter (starting at page 36 ), and 2X 2! are similar to 1X and 1! but reflect terms present in the true response that are not included in the fitting model. For example, for two variables with a quadratic fitting model, and a cubic true model, response and its expectation at design point Txj j jx x2 1 are given as j j j j j j j jx x x x x x y 2 2 6 2 1 5 2 1 4 2 3 1 2 11 1 1 1 1 1 (3.39) 3 2 4 2 2 1 3 2 2 1 2 3 1 1 2 2 6 2 1 5 2 1 4 2 3 1 2 1) (j j j j j j j j j j j j jx x x x x x x x x x x x y E2 2 2 2 1 1 1 1 1 1 (3.40) Then, the coefficient vectors (1! and 2! ), matrices 1X and 2X are given in Eq. (3.41) (3.42), and (3.43), respectively. T 2 2 2 2 2 T 1 1 1 1 1 1 1! !4 3 2 1 6 5 4 3 2 1 (3.41) / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 4 2 2 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 22 22 12 2 12 22 12 2 21 21 11 2 11 21 111 1 1 1N N N N N N j j j j j jx x x x x x x x x x x x x x x x x x x x x x x x" " " " " " "1X (3.42)

PAGE 64

50 / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 4 3 2 2 2 1 2 2 1 3 1 3 2 2 2 1 2 2 1 3 1 3 22 2 22 12 22 2 12 3 12 3 21 2 21 11 21 2 11 3 11 N N N N N N j j j j j jx x x x x x x x x x x x x x x x x x x x x x x x" " " " "2X (3.43) The lack of fit test supplies statistical evidence that the fitting model is unable to account adequately for the variation in the observed response. The test tries to isolate the portion of the residual variation unaccounted for by the fitted model. For that purpose, evidence is sought based on estimator of pure error variance that does not depend on the form of the fitted model required. An ideal scenario for obtaining such an estimate of pure experimental error variance is to have nj replications at several design points jx. Therefore, response for jth design point among M distinct design points at its kth replication can now be represented as jk j k j jky y ) ( ) ( x x (3.44) (where j=1, M and k=1, nj). The total number of design points is still denoted by N and equal to 'M j jn N1 (3.45) For the jth design point the average jy is an estimate of the true mean response ) (jx j n k jk jn y yj'1 (3.46) that is equal to the true mean response when jn.

PAGE 65

51 By these estimates, the sum of squares due to pure error SSPE can be calculated as for the design points '' M j n k j jk PEjy y SS11 2) ( (3.47) It is compared to the sum of squares due to lack-of-fit, LOFSS using the difference between the estimate y ˆ given by the model and jy. M j j j j LOFy y n SS2) ( ˆ x (3.48) Equations (3.47) and (3.48) can also be rewritten in matrix forms j j j j jn M j j n n n n PEn SS]y 1 1 [I yT T' 1 (3.49) j j j j jn M j j n n n n LOFn SS]y 1 1 [I y ]y X ) X (X X [I yT T T 1 1 1 T 1 1 N T' 1 (3.50) where jny is the response vector of nj replications at jth design point, and jn1 is a (nj x 1) vector of ones. Note that Eq. (3.50) is the difference between total sum of squares ESS [Eq. (3.31) ] and pure error sum of squares PESS [Eq. (3.49)]. Therefore, an F-ratio is formed by using these partitions of residual sum of square PE PE LOF LOFd SS d SS F/ / (3.51) where b LOFn M d and M N dPE are degrees of freedom associated with SSLOF and SSPE, respectively. Lack of fit can be detected at the level of significance if the Fratio exceeds the table value, M N n MbF , where the latter quantity is the upper 100 percentile of the central F-distribution (Khuri and Cornell, 1996, p. 38–41). Similar to

PAGE 66

52 hypothesis testing for Eq. (3.32) the p-value is determined as the rightmost tail area under the distribution M N n MbF beyond the F statistic value by Eq. (3.51) Lack-of-Fit with Non-Replicated Design Points Standard lack-of-fit test described in previous section requires exact replications at several of the design points. Unlike physical experiments, exact replications by numerical experiments/simulations do not supply additional information since they result in exactly the same response when repeated. It is still possible, however, to obtain an estimator that approximates the notion of pure experimental error (Hart, 1997, p. 123). The idea is to treat the observations at neighboring design points as “near” replicates. Then test procedures are based on a pseudo error estimator of the error variance. Neill and Johnson (1985) generalized the standard pure-error lack of fit test to accommodate the case of non-replication for first order fitting model. Here, the method is extended to a higher-order fitting model. The method considers a design point as near replicate design point xjk within a distance jk# from the associated reference point jx (subscript ‘jk’ stands for kth near-replicate around jx) jk j jk# x x (3.52) where distance or disturbance vector (for a general, n-variable case) is T# ] [2 1 njk pjk jk jk jkx x x x ! (3.53) The monomial terms at design point xjk for the approximation model are calculated using Eq. (3.52) and (3.53). In a quadratic fitting model, for instance, they are given as in Eq. (3.54)

PAGE 67

53 } { } 2 { } {2 2 2 rjk pjk pjk rj rjk pj rj pj rjk pjk pjk pjk pj pj pjk pjk pj pjkx x x x x x x x x x x x x x x x x x (3.54) where p, r =1, 2,…, n (number of design variables). The design matrix X1 for the data set augmented by the near-replicate designs is shown by Eq. (3.55) 1 1 1$ X X (3.55) where 1X is the design matrix as if the near-replicate designs are replaced by their associated reference design points jx so that it represents an experimental design with replications, and 1$ is the disturbance design matrix that turns 1X into X1. Entries in a row of the disturbance matrix 1$ correspond to the fitting model coefficient vector 1! in Eq. (3.56). T 1 1 1 1 1 1!bn i !3 2 1 (3.56) For example the (j+k)th row of the disturbance matrix corresponding to kth near replication of the jth design point is given as (assuming jth design point is followed by its k near replicates) bn jk i jk jk jk k j1 1 1 1 1$ ) ( ) ( ) ( ) ( 03 2 ! (3.57) For design points that are not near replicates the entries in the associated row are all equal to zero. The non-zero entries in Eq. (3.57) are formed by the corresponding monomial terms within the brackets [quadratic fitting model case, Eq. (3.54)].

PAGE 68

54 For instance, in a two-variable problem where the data has two near replicates (2 1 k) for design point jx and the fitting model is quadratic, design matrices, coefficient vector and disturbance design matrix are given as in the following equations. / / / / / / / / / / / 0 1 2 2 2 2 2 2 2 2 2 2 2 3 4 2 2 2 1 2 1 2 1 2 2 2 2 2 2 1 2 2 1 2 2 2 1 2 1 2 1 2 1 1 2 1 1 1 2 1 1 2 2 2 1 2 1 2 1 2 22 22 12 2 12 22 12 2 21 21 11 2 11 21 111 ) ( ) ( ) ( 1 ) ( ) ( ) ( 1 1 1 1N N N N N N j j j j j j j j j j j j j j j j j jx x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x " " " " " "1X (3.58) / / / / / / / / / / / 0 1 2 2 2 2 2 2 2 2 2 2 2 3 4 2 2 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 22 22 12 2 12 22 12 2 21 21 11 2 11 21 111 1 1 1 1 1N N N N N N j j j j j j j j j j j j j j j j j jx x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x " " " " " "1X (3.59) T 1 1 1 1 1 1 1!6 5 4 3 2 1 (3.60) / / / / / / / / / / 0 1 2 2 2 2 2 2 2 2 2 2 3 4 0 0 0 0 0 0 ) ( ) ( ) ( ) ( ) ( 0 ) ( ) ( ) ( ) ( ) ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 06 2 5 2 4 2 3 2 2 2 6 1 5 1 4 1 3 1 2 1" " " " " " "1 1 1 1 1 1 1 1 1 1 1$j j j j j j j j j j (3.61) where for the kth near replication non-zero terms in 1$ are given by Eq. (3.62).

PAGE 69

55 } 2 { ) ( } { ) ( } 2 { ) ( } { ) ( } { ) (2 2 2 2 6 2 1 1 2 2 1 5 2 1 1 1 4 2 3 1 2 jk jk j jk jk jk jk j jk j jk jk jk j jk jk jk jk jkx x x x x x x x x x x x x x 1 1 1 1 1 (3.62) For the data with near replicates, the response vector can now be represented in the following matrix form after substituting Eq. (3.55) into Eq. (3.37) $ X y1 1 1 1 (3.63) Now a corrected response vector, y can be defined by using the fitting model to project the near-replicate data to the reference points 1 1! $ y y (3.64) X y1 1 (3.65) Equation (3.65) allows a standard lack-of-fit test since the design matrix 1X now is with replicated design points that allow calculating the pure experimental error variance. The corrected response vector, y however, is not observable. Neill and Johnson (1985) proposed to replace it with an observable response vector y ˆ as given in Eq. (3.66) that, at least asymptotically, is comparable to y 1 1b $ y y ˆ (3.66) The test formalized in this approach (Neill and Johnson, 1985) is given in Eq. (3.67) 2 2 2 1 1 1 1 1 1) $ X ( ) $ X ( y ) $ X ( y ) ( : ) ( :0E H E Ha (3.67) where 2X and 2$ are respectively similar to 1X and 1$ but corresponding to the missing term coefficient vector 2! Similar to standard lack-of-fit test, the test statistic in

PAGE 70

56 Eq. (3.51) now denoted as F ˆ is based on sums of square error SSPE and SSLOF [Eqs. (3.49) and (3.50) respectively], but with y, jny and 1X replaced with y ˆ jnyˆ and 1X respectively. Neill and Johnson (1985) claimed that F ˆ is asymptotically comparable to the test statistic obtained when replication actually exists. A five-variable problem was used in order to test the lack-of-fit test with nonreplicated data. First a quadratic function in five-variables is calculated in 126 distinct design points (coded variables between –1 and +1) that will be used for the five-variable HSCT wing problem (see Chapter 5 for details of the experimental design). The data is contaminated with a random, normally distributed noise (3000 ). Then, a face centered composite design (FCCD) around the central design with a disturbance size of 0.1 in coded space (42 near replicate designs) was generated, and contaminated results were calculated similarly. Figure 5 shows the clustering near-replicate designs around the center design in 3-D case in order to visualize the procedure. Figure 5: 3-D representation of clustering design points to generate near-replicate data for lack-of-fit test cv3cv1 cv2 cv3 cv1 cv2 (1,1,1) (0.1,0.1,0.1) (-1,1,1) (1,1,-1) (1,-1,-1) (-1,-1,1) (-1,-1,-1) (-0.1,0.1,0.1) (-0.1,-0.1,0.1) (0.1,0.1,-0.1) (0.1,-0.1,-0.1) (-0.1,-0.1,-0.1)

PAGE 71

57 The left cubicle represents the whole design space with distinct points (equivalent to 126 design points in five-variable example) and the right cubicle represents the 42 near-replicates around the center design. The lack-of-fit test for this set of data obtained from a quadratic function should give no significant indication for the inadequacy of fitting model. The same approach was repeated by generating data from a cubic function while keeping the fitting model quadratic. This latter case should indicate significant statistical evidence for the inadequacy of the fitting model. Table 3 shows the results of the two tests. The second column presents the results when the true function is also quadratic polynomial of five variables. For a significance level of 05 0 the calculated statistic F ˆ is smaller than the relevant value from the F distribution table. That is, there is no significant statistical evidence for rejecting the null hypothesis; the quadratic fitting model is the true model. The p-value of almost one indicates that the chance of making an error by rejecting the null hypothesis-a quadratic model is adequate-is about 100%. In contrast, results with a cubic true function indicate that there is a statistical evidence to reject the null hypothesis, and the chance of rejecting it erroneously ( p-value ) is less than 1%. Table 3: Verification of the lack-of-fit test with non-replicated design points Statistical parameters Quadratic true function Quadratic fitting model Cubic true function Quadratic fitting model M 126 126 N 168 168 nb 21 21 F ˆ 0.6176 2.1530 M N n MbF , 05 0 1.5704 1.5704 p-value 0.9747 0.0030

PAGE 72

58 Iteratively Re-weighted Least Squares As mentioned in previous section, one of the factors affecting the accuracy of the RS is noise in the data. While low-amplitude, random noise is filtered well by RS approximations, data with large errors-outliers-may cause significant loss of accuracy. An example in one-variable case is shown graphically in Figure 6 where two noisy data set and true mean response data are compared. Data labeled as “data with outlier” is the same as the data labeled as “noisy data”, but with a larger error at the rightmost design point. While the fit of the original “noisy data” is very close to true mean response function, the slope of the function has changed substantially due to the outlier. Figure 6: 1-D example of the effect of an outlier in the data Outliers are atypical (by definition), infrequent observations. In the example, because of the way in which the regression line is determined (especially the fact that it is based on minimizing not the sum of simple distances, but the sum of squares of distances 0 1 2 3 4 5 6 7 8 9 024681012xy true function noisy data data with outlier fit on noisy data fit on data with outlier

PAGE 73

59 x y IRLSweighting 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Data LS IRLS weighting Outliersof data points from the line), the outlier has a profound influence on the slope of the regression line. A single outlier is capable of considerably changing the slope of the regression line as shown in Figure 6 Robust regression employs techniques for statistical methods, such as iteratively re-weighted least square (IRLS) fitting (Holland and Welsch, 1977), to detect and remove or weight down outliers. Once detected, outliers can be investigated further for possible mistakes. In order to detect outliers, IRLS was used in the present work. Iteratively reweighted least square procedures start with an initial response surface fitted to data and then assign low weights to points with large errors and refit the data using a weighted least square procedure. The process is repeated until convergence, and weights of the response data that do not fit the underlying model (outliers) tend to converge to small values. This effectively eliminates these points from the fitting process. A graphical representation of the procedure is shown in Figure 7 for a 1-D case. Figure 7: 1-D example of IRLS procedure

PAGE 74

60 In IRLS procedure, the weight w assigned to a data point can be determined by several weighting functions (Myers and Montgomery, 1995, pp. 667-673). Table 4 presents the standard weighting functions considered in this dissertation and collaborative research, Kim et al. (2001) focused on efficient IRLS application for results from optimizations. Table 4: Standard weighting functions for IRLS Name W ( ej) Range Tuning Constant Huber’s min-max 1 H 2s/ej 2 2ej/s 2 H 2ej/s 2 H H =1.0 Beaton and Tukey’s biweight [1-( ej/s / B )2]2 0 2ej/s 2 B 2ej/s 2 B B =1.0 ~ 1.9 Bi-weight function by Beaton and Tukey (1974) is preferred to min-max weighting function by Huber (1964) because it gives zero weight to the outliers and thus the outliers are clearly detected. The weight w assigned to a data point can be formalized as ( ( ( ( ( ( ( ( , , ( ( ( ( ( ( ( ( . . / / / / / / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 4 ! ! ! ! " " # # # # $ $ $ $ $ $ $ $ % % % % & & & & otherwise 0 if / 12 2B /s e B s e wj j (3.68) where ej is the residual, see Eq. (3.8) s is the estimate of noise, see Eq. (3.24) and B is tuning constant, usually 1
PAGE 75

61 Residual,r Weighting,w(r) -4 -2 0 2 4 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6Huber Biweight Biased Figure 8: Weighting functions for IRLS (Kim et al., 2001) The weighting function called “Biased” in Figure 8, introduced in Kim et al. (2001), is an alternative weighting function to standard bi-weight function that takes into account one-sided errors due to incomplete convergence in minimization problems. After selecting weighting function the coefficients bis for IRLS approximation can be found by solving Eq. (3.69). Wy X b WX XT 1 1 1 T 1 (3.69) where W is a diagonal weighting matrix using the weights from Eq. (3.68) / / / / 0 1 2 2 2 2 3 4 ) ( 0 0 0 ) ( 0 0 0 ) (2 1 Ne w e w e w # $ $ " $ W (3.70)

PAGE 76

62 When the weights are all one, the solution of Eq. (3.69) reduces to ordinary least square solution. However, in general Eqs. (3.69) and (3.70) are nonlinear equations, and the iterative IRLS procedure employs Eq.(3.71). ) ( ) ( ) 1 ( i (i) (i) i i1 1 T 1 1 1 T 1 1 1b X y W X X W X b b (3.71) Normally, the IRLS procedure is used only to eliminate or reduce the effect of errors by assigning them low weights (Holland and Welsch, 1977) that basically leaves them out and is referred as robust regression. However, the procedure is used here for the purpose of identifying outliers and factors causing them so that the repair of erroneous data can be sought rather than removing the erroneous points. The JMP (SAS, 1998) statistical software was used in this work for construction of RS approximations and IRLS procedure.

PAGE 77

63 CHAPTER 4 BIAS AND NOISE ERRORS IN RESPONSE SURFACE APPROXIMATIONS Applications making use of RS approximations may suffer in regions of design space where the approximation may be poor. The traditional low-order fitting models such as quadratic polynomials may not always represent the behavior of complex engineering systems. On the other hand higher order models are generally impractical to use in high dimensional problems and designers may have to live with lower-order models. In this situation, information concerning the design regions where the low-order models may suffer the most would be valuable. In this chapter a measure that can provide such information is derived. Mean Squared Error Criterion As a measure of the error in the approximation, the mean squared error of prediction MSEP defined as in Eq. (4.1) is used in this dissertation 2) ( ) ( ˆ ) ( x x x y E MSEP (4.1) where ) ( x and ) ( ˆ x y are the true mean response and the prediction by the fitted model, respectively at x MSEP is by definition an expected value that would be reached if the number of data used in approximation were unlimited. Equation (4.1) may be rewritten as given in Eq. (4.2).

PAGE 78

64 2 2 2) ( ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ˆ 2 ) ( ˆ ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ˆ ) ( x x x x x x x x x x x x x y E E y E y E y E y E y E y E y E y E MSEP (4.2) The cross-product term in the above squaring operation is 0 ) ( ) ( ˆ ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ˆ ) ( ) ( ˆ ) ( ˆ ) ( ˆ2 2 2 ) ) ) ) * * + + + + , , . . x x x x x x x x x x x x x x x x x y E y E y E y E y E y E y y E y E y E y E y E (4.3) since ) ( ) ( x x E and ) ( ˆ ) ( ˆ x x y E y E E Therefore 2 2) ( ) ( ˆ ) ( ˆ ) ( ˆ x x x x y E y E y E MSEP (4.4) The first term in Eq. (4.4) represents the variance error due to random noise [see Eq. (3.19) ] and the second term represents bias error due to inadequate modeling. This error expression is usually integrated over the design space and the integral is minimized by choosing the experimental designs that control the effect of one or both types of error (Khuri and Cornell, 1996, pp. 207-247; Venter and Haftka, 1997). Here instead it is attempted to characterize the error in the predictions of an RS approximation already constructed and to determine the design regions where RS prediction may suffer due to either or both types of error. Therefore Eq. (4.4) is used to investigate the variation of MSEP from point-to-point. The expectation of the predicted response at a given design point x can be expressed as ) ( ) ( ˆ1 T 1b f x E y E (4.5)

PAGE 79

65 where 1f is the vector of shape functions if [see Eq. (3.3) ] calculated at point x The mean and the variation for the coefficient estimates are given as [see Eq.s (3.15) and (3.21) ] ) ( ) ( y X X X bT 1 -1 1 T 1 1E E (4.6) 1 1 T 1 1 1 T 1 1 T 1 1 1 T 1 1X X X X X y X X X b 2) ( ) (Var Var (4.7) where is the standard deviation of the noise error. The first part of the mean squared error in Eq. (4.4) is equal to the prediction variance at x [ due to Eq. (3.19) ]. 1 1 T 1 T 1 1 1 T 1f X X f f b f x x x1 2 2) ( ) ( ˆ ) ( ˆ ) ( ˆ Var y Var y E y E (4.8) The second part of Eq. (4.4) is the squared error due to the inadequacy of the model used 2 2) ( ˆ ) ( ) ( ˆ x x x y Bias y E (4.9) Therefore mean squared error of prediction in Eq. (4.4) can be rewritten as 2) ( ˆ ) ( ˆ x x y Bias y Var MSEP (4.10) The form of the prediction variance, ) ( ˆ x y Var, in Eq (4.8) does not change even in the presence of inadequate modeling, since variance is only caused by random noise. However, when bias error is present, the residual error mean square 2s is not an unbiased estimate of 2. Using Eqs. (3.16) and (3.24) we get Subscripts 1 and 2 assign matrices and vectors for the fitting model and the missing terms, respectively.

PAGE 80

66 1 2p N s P)y (I yN T (4.11) where T 1 1 1 T 1 1X X X X P (4.12) The accuracy of this estimate depends on factors such as the available data or number of points and also the adequacy of the fitting model that determines whether the estimate is unbiased or biased. The true response at x can be written as 2 T 2 1 T 1! f f x ) ( (4.13) where 2f are terms missing from the assumed model. Since one usually does not know the true response, it is often assumed to be a higher order polynomial when monomials are used as shape functions. The mean of the true response is given as 2 2 1 1! X X y ) ( E (4.14) where 2X is similar to 1X but due to the terms (shape functions) present in the true response that are not included in the fitting model. After substitution of Eq. (4.14), Eq. (4.6) becomes 2 1 1A ! b ) ( E (4.15) where2 T 1 1 1 T 1X X ) X (X A is called the alias matrix. Substitution of Eq. (4.15) into Eq. (4.5) yields 2 1 T 1A ! f x ) ( ˆ y E (4.16) 2 T 2 T 1 2 1 T T 2 2 T 2 1 T 1 2 1 T 1] f A ][f f f [A ! f f A ! f x 2 2) ( ˆ y Bias (4.17) The MSEP at a given point can now be estimated as by using 2s instead of 2

PAGE 81

67 2 T 2 1 -1 1 T 1 T 1M ! f ) X (X f x 2) ( ˆ s EP S M (4.18) where ] f A ][f f f [A MT 2 T 1 2 1 T (4.19) Note that MSEP is an expectation by definition [Eq. (4.1) ], and Eq. (4.18) is its estimate by the data available and associated 2s. Using ) (yE from Eq. (4.14) the expected value of biased error mean square 2s given as (Seber, 1977) 1 2 2) ( p N s E 2 2 N T 2 T 2! P)X (I X (4.20) since P) (IN is an idempotent matrix, that is a matrix whose square is equal to itself. It is also positive semi-definite matrix, so 2 2) (s E provided that 02 P)X (I2 N. Equation (4.20) means that when bias errors are present, 2s is a biased estimate of 2. The standard lack-of-fit test detects, when significant, the presence of the second terms in Eqs. (4.18) and (4.20). If this expected value, E ( s2), is substituted in prediction variance contribution in Eq. (4.18) expected value for the estimate EP S M ˆ can be expressed as 2 T 2 1 1 1 T 1 T 1 2 T 2 2 T 2 1 1 1 T 1 T 1G ! f ) X (X f M ! K ! f ) X (X f x / 0 1 2 3 4 2 1 2)] ( ˆ [ p N EP S M E (4.21) where A) X (X X K1 2 T 2 (4.22) M K f ) X (X f G1 1 1 T 1 T 1 1p N (4.23) The aim of this chapter is to identify points where the bias error is large. It is assumed that the terms missing from the fitting model are known, but there is not enough

PAGE 82

68 data to calculate the corresponding coefficients 2! If one can estimate the size of 2!, it is possible to formulate a constrained maximization problem for the largest magnitude of the mean squared error predictor that may be experienced at any given design point for the worst possible 2! of that magnitude. c such that EP S M E 2)] ( ˆ [ max2 !! x2 (4.24) The Lagrangian for this optimization problem can be written as ) ( )] ( ˆ [ ) (2c EP S M E L 2 T 2! x (4.25) Differentiating the Lagrangian with respect to 2! 0 ) ( ) (2 c !2 T 2 2 T 2 1 1 1 T 1 T 1! G ! ] f ) X (X f [ (4.26) where T p/ / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 4 22 12 2 2 !. Equation (4.26) yields the following eigenvalue problem at a design point jx; 0 G !2 2 or 0 G !2 2 G (4.27) for which the maximum eigenvalue maxG characterizes the maximum possible mean squared and bias error associated with the assumed true model that includes shape functions missing in the fitting model. The corresponding eigenvector defines the coefficients of the missing shape functions that results in the largest bias error when fitted only with the assumed model. The eigenvectors and the function experiencing the worst possible bias error may be different point-to-point although the magnitude of the missing

PAGE 83

69 coefficient vector is constrained. So the eigenvalue calculated does not reflect the true function corresponding to the data (as the data is insufficient to calculate 2! ). It reflects instead, the assumed form of true function with the shape function coefficients 2! (among all the possible combinations such that 2c 2 T 2! !) causing the largest error. Estimated Standard Prediction Error and Residual Based Error Bound The eigenvalue measure characterizes the largest possible mean squared error at a design point for which noise also contributes as well as the insufficient fitting model. The measure, on the other hand, does not pay any attention to the actual data, and so does not include actual noise in the data. The matrix G defined in Eq. (4.23) suggests that associated eigenvalues primarily characterize the error due to missing terms (shape functions) of the true function and the effect of the experimental design. As a result, the characterization is qualitative only, and may be used to locate design points of possible large approximation error. The success of the eigenvalue measure to pinpoint the locations of high error regions may be determined by checking its correlation with the residuals. The actual error at a design point jx associated with the RS approximation ) ( ˆjy x needs the true mean response ) (jx that is not always available even for data points due to noise. For engineering problems, such as wing structural weight as studied in Chapter 5, the true function is also not known. In order to test the use of eigenvalue measure for such problems, another measure based on the available data, approximation and associated estimated standard prediction error was used. This alternate measure-conservative error

PAGE 84

70 bound-was also used as an approximate tool to quantify uncertainty due to the use of the RS approximation ) ( ˆjy x. Recalling MSEP expression in Eq. (4.4) and recognizing the first term as being the prediction variance, the problem of calculating an estimate of the MSEP at any design point stands on the second term described in Eq. (4.9) as also repeated in the following 2 2) ( ) ( ˆ ) ( ˆj j jy E y Bias x x x (4.28) Equation (4.28) cannot be calculated exactly since neither terms on right hand side are available. To be able to determine approximately, Eq. (4.28) was replaced by 2 2) ( ) ( ˆ ) ( ˆj j jy y y Bias x x x (4.29) where right hand side is square of residual je also defined in Eq. (3.8) While ) ( ˆjy x is available at any design points, ) (jy x is only available at data points, and additional experiment/simulation is needed for any other design point. Since the alternate error measure was also intended to be available for any design point, the right hand side was replaced by an RS approximation fitted on available data of residual sje. Two measures of error are collectively used for the data of residual related RS, Absolute value of residual relative to predicted response at a design point jx x x ) ( ˆ ) (j j j Ry e e (4.30) Equation (4.30) measures the combined effects of modeling and noise error. Average of the absolute relative residuals in Eq. (4.30) x N e eN j j R R') ( (4.31)

PAGE 85

71 where N is number of data points. Equation (4.31) is calculated to compare with error data obtained by Eq. (4.30) at the design points. Since the error is not completely characterized by these two measures first the maximum of the two was used as the residual error measure in order to be more conservative in modeling uncertainty analysis x xR j R j Re e e, ) ( max ) (max (4.32) Then an RS approximation ) ( ˆj Rex was constructed for ) (maxj Rex to predict the result of Eq. (4.29) by Eq. (4.33) 2 2) ( ˆ ) ( ˆ ) ( ˆj j R jy e y Bias x x x (4.33) An estimate of error at a design point jx was then defined by square-root of an alternate estimate of MSEP that is sum of prediction variance as first term in Eq. (4.18) and prediction by Eq. (4.33) at jx 2 2) ( ˆ ) ( ˆ ) ( ˆj j R j jy e s EP S AM s x x f ) X (X f x1 1 1 T 1 T 1 (4.34) Equation (4.34) was treated as a conservative error or residual bound that may be suffered in the predictions by ) ( ˆjy x, and can be calculated point-to-point. Polynomial Examples The eigenvalue measure was derived as a tool to determine the regions of potential high bias error when the fitting model is missing some of the shape functions of the true model. For instance, the traditional low-order polynomial fitting models are widely used since the higher-order models may not be applicable due to insufficient data. Therefore, the ideal case for demonstrating the use of eigenvalue estimate of bias error is

PAGE 86

72 to check the measure for problems where the true function is a polynomial of a higher degree than the fitting model used. The true function for the examples was chosen as a cubic polynomial while the fitting model was kept as quadratic. In other words for each example, the data were generated by a cubic polynomial and then fitted by quadratic model. In order to see the effect of noise, data contaminated with random and normally distributed noise were also studied. Twoand five-variable polynomials were used to demonstrate the use of the eigenvalue measure for different dimensional problems. Coefficient of Correlation As a statistical indication for the success of the measure to pinpoint the high bias error regions in the design space, coefficient of correlation between the square-root of eigenvalues calculated and the bias error (or residual calculated as the absolute difference between RS prediction and true response value) and its graphical representation can be used. The most widely-used type of correlation coefficient is Pearson r also called simple linear correlation. The Pearson coefficient of correlation, r between two variables z1 (e.g. eigenvalues) and z2 (e.g. absolute bias error or residual) is calculated as 2 12 1 2 1) (z z i iz z n z z r ' ' (4.35) where 1z and 2z are the averages, and 1z and 2zare the standard deviations in the set of values for variables. It is a measure of the linear relation between two variables and can range from -1 to +1. A value of 0 represents a lack of correlation between the variables. The value of –1 represents a perfect negative correlation while a value of +1 represents a perfect positive correlation.

PAGE 87

73 The correlation coefficient r represents the linear relationship between two variables, and if squared, the resulting value 2 r is equal to coefficient of determination [2 R in Eq. (3.34) ] as if one treats one of the variables e.g. z2 being fitted as a linear function of the other, z1. The sample plots for r are shown in Figure 9. Figure 9: Linear correlation examples (Lowry, 2001) The coefficient of determination 2 r represents the proportion of common variation in the two variables (i.e., the “strength” or “magnitude” of the relationship). On the other hand the correlation is not intended to explain or explore cause and effect type relation, and so the fit is not used to predict one from the other. In order to evaluate the correlation between variables, it is important to know “magnitude” or “strength” as well as the significance of the correlation that is the significance level at which one can reject the null hypothesis that there is zero correlation, 0 :0 r H and accept that calculated non-zero correlation is not by pure chance. The t -statistic for this test is given as r = 0 r > 0 r < 0 r = +1 r = -1 z1 z1 z2 z2 z2 z2 z2 z1 z1 z1

PAGE 88

74 ) 2 ( ) 1 (2 N r r t (4.36) where N is the number of data (or sample size) to check on the correlation between the variables. It is compared with the t -table value corresponding the significance level, and degree of freedom ( N -2) as explained in section concerning hypothesis testing (page 43 ). The p-value can also be determined from the t -table. The significance or p-value in this dissertation was calculated by a JAVA script posted on the web by Richard Lowry (2001). High positive correlation between eigenvalues and residuals indicates that if designer moves from a design point to another one where eigenvalue is higher, the error due to approximation may also increase. In other words, eigenvalue measure can warn and make the designer cautious against inaccurate regions of approximation. Cubic Polynomial in Two Variables – without Noise First a cubic polynomial in two variables was used as the true model, and a quadratic polynomial as the fitted model with no noise in the data. 2 T 2 1 T 1! f f x x ) ( ) (y (4.37) where T 2 2 2 2 2 T 1 1 1 1 1 1 1 T 2 T 1 T! f f x4 3 2 1 6 5 4 3 2 1 3 2 2 2 1 2 2 1 3 1 2 2 2 1 2 1 2 1 2 11 x x x x x x x x x x x x x x (4.38) Data for a three-level design as shown in Figure 10 is assumed to be available to form the design matrices X1 and X2 [Eq.s (3.42) and (3.43) respectively] that is, there is not enough data points to characterize all ten coefficients.

PAGE 89

75 Figure 10: Two-variable three-level factorial design points Equation (4.27) depends only on the experimental design and the fitting and assumed true models, not the response data. The eigenvalue problem in Eq. (4.27) with 1 2 T 2! was solved, at 21x21 mesh points over the design region. The maximum eigenvalues and corresponding eigenvectors were determined. Figure 11 shows a contour plot of the square-root of the eigenvalues. It is seen that high errors are expected at center of the boundaries of the design region: (-1,0), (1,0), (0,-1) and (0,1), and low values at (0,0), (-0.8,-0.8), (-0.8,0.8), (0.8,-0.8) and (0.8,0.8). Figure 11: Square-root of maximum eigenvalue maxG contours from Eq. (4.27) for a two-variable cubic polynomial, fitted with a quadratic model (T 1 1 1 1 1 1 1!6 5 4 3 2 1 ) x1,x2 1,-1 1,1 -1,1 -1,-1 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 Gmax0.8289 0.8034 0.7778 0.7522 0.7266 0.7011 0.6755 0.6499 0.6243 0.5987 0.5732 0.5476 0.5220 0.4964 0.4709 0.4453 maxG

PAGE 90

76 Next, a specific example with T 1!1 2 1 5 3 50 was used, and sets of 2! that are the eigenvectors associated with the maximum eigenvalues obtained at nine data points in Figure 10 were studied. Note that such an eigenvector at a design point (with 1 2 T 2! !) defines the cubic polynomial that introduces the largest bias error at that point, when fitting model is quadratic. Here, without noise, absolute true residual at design point jx, Tje is the bias error as given in Eq. (4.39) ) ( ˆ ) (j j Tjy ex x (4.39) There are four eigenvectors to investigate since the other five are negative of either one of the four eigenvectors, and since the absolute residuals are identical when only the sign changes. Since the eigenvalue contours consist of different third order polynomials instead of a single one, exact agreement cannot be expected for any single polynomial. Instead, a given polynomial is expected to have a positive correlation between the absolute true residuals Tje and the square-root of the maximum eigenvalues maxG. Table 5 shows the polynomial coefficients and the associated coefficient of correlation between i Tje where i denoting the polynomial/eigenvector and maxG of Eq. (4.27) Figure 12 shows the absolute true residual plot for the polynomials studied. Comparing Figure 11 and Figure 12, it is seen that each polynomial achieves maximal errors in one of the regions predicted by Figure 11 and at associated design point. For instance, Polynomial 1 is the eigenvector determined at design points (-1,-1) and (+1,+1), where errors are high as shown in Figure 12a.

PAGE 91

77 Table 5: Coefficient of correlation between the absolute residual i Tje and the square-root of maximum eigenvalues maxG from Eq. (4.27) for two-variable cubic polynomial example without noise ( T 1!1 2 1 5 3 50 ) i Corresponding design point ) (2 1x x 12 22 32 42 r 1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 0.797 2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.797 3 (-1,0) and (+1,0) 0 0 -1 0 0.406 4 (0,-1) and (0,+1) 0 -1 0 0 0.406 Maximum of four 0.973 (a) (b) (c) (d) Figure 12: Absolute residual i Tje contours – two-variable cubic polynomial example (polynomials defined in Table 5), without noise. a) Polynomial 1; b)Polynomial 2; c) Polynomial 3; d) Polynomial 4 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej1| 0.4870 0.4548 0.4225 0.3903 0.3580 0.3258 0.2935 0.2613 0.2290 0.1967 0.1645 0.1322 0.1000 0.0677 0.0355 0.0032 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej2| 0.4870 0.4548 0.4225 0.3903 0.3580 0.3258 0.2935 0.2613 0.2290 0.1967 0.1645 0.1322 0.1000 0.0677 0.0355 0.0032 1Tje 2Tje x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej3| 0.6623 0.6184 0.5746 0.5307 0.4868 0.4430 0.3991 0.3553 0.3114 0.2675 0.2237 0.1798 0.1360 0.0921 0.0482 0.0044 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej4| 0.6623 0.6184 0.5746 0.5307 0.4868 0.4430 0.3991 0.3553 0.3114 0.2675 0.2237 0.1798 0.1360 0.0921 0.0482 0.0044 3Tje 4Tje

PAGE 92

78 Comparison of absolute residual for the polynomials and the square-root of eigenvalues can also be shown by Figure 13. The correlations in Table 5 and Figure 13 show that even for a single polynomial the eigenvalue distribution can be used to identify locations of possible high bias error. Figure 13: Comparison between square-root of eigenvalues maxG and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 5), without noise. Figure indicates that eigenvalues correlate well with maximal errors. See correlation coefficients in Table 5 When the maximum of the absolute residual among the four polynomials, ) , max(4 3 2 1 maxTj Tj Tj Tj Tje e e e e was used, the correlation is 0.973 as reported in last row of Table 5 With the maximum of all four, contours given in Figure 14.b are very similar to those of Figure 11 (also repeated as Figure 14.a). High-correlation r =0.973 between maxG and maxTje can also be visualized by Figure 15 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.40.50.60.70.80.9xy Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 maxG i Tje

PAGE 93

79 (a) (b) Figure 14: Comparison of square-root of eigenvalues maxG and maximum absolute residual maxTje contours (maximum of residuals among the four polynomials defined in Table 5) – two-variable cubic polynomial example, without noise. a) Eigenvalue field; b)Maximum true residual field Figure 15: Comparison between square-root of eigenvalues maxG and maximum of true residual errors maxTje due to the four polynomial in Table 5 – two-variable cubic polynomial example, without noise. Coefficient of correlation is 0.973 Similar contour plots ( Figure 16 ) were also created for the conservative error bound i js of each polynomial in Table 5 as defined in Eq. (4.34) It seems that conservative error bound also shows high error at the design points where eigenvectors maxG 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.40.50.60.70.80.9 maxTje x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 Gmax0.8289 0.8034 0.7778 0.7522 0.7266 0.7011 0.6755 0.6499 0.6243 0.5987 0.5732 0.5476 0.5220 0.4964 0.4709 0.4453 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej|max0.6623 0.6184 0.5746 0.5307 0.4868 0.4430 0.3991 0.3553 0.3114 0.2675 0.2237 0.1798 0.1360 0.0921 0.0482 0.0044 maxG maxTje

PAGE 94

80 were determined to define the four polynomials. For instance, Polynomial 3 is the eigenvector determined at design points (-1,0) and (+1,0), where residual and conservative error bound are both high compared to other design points as shown in Figure 12 c and Figure 16c. (a) (b) (c) (d) Figure 16: Conservative error bound i js contours – two-variable cubic polynomial example (polynomials defined in Table 5), without noise. a) Polynomial 1; b) Polynomial 2; c) Polynomial 3; d) Polynomial 4 Variation of i js within each distribution is smaller compared to corresponding i Tje but i js is higher than i Tje for all design points. This can be seen better in Figure x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 sqmse1 0.7800 0.7646 0.7492 0.7338 0.7184 0.7029 0.6875 0.6721 0.6567 0.6413 0.6258 0.6104 0.5950 0.5796 0.5641 0.5487 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 sqmse2 0.7764 0.7610 0.7455 0.7301 0.7146 0.6991 0.6837 0.6682 0.6528 0.6373 0.6219 0.6064 0.5909 0.5755 0.5600 0.5446 1js 2js x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 sqmse3 0.7830 0.7657 0.7484 0.7311 0.7138 0.6965 0.6793 0.6620 0.6447 0.6274 0.6101 0.5928 0.5755 0.5582 0.5409 0.5236 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 sqmse4 0.7866 0.7691 0.7516 0.7341 0.7166 0.6990 0.6815 0.6640 0.6465 0.6290 0.6115 0.5940 0.5764 0.5589 0.5414 0.5239 3js 4js

PAGE 95

81 17 that indicates i js determined a bound for the error conservatively for no-noise example although it did not require the true function values. Figure 17: Comparison between conservative error bound i js and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 5), without noise Cubic Polynomial in Two Variables – with Noise The two-variable polynomial example presented in the previous section was repeated, with random, normally distributed noise of zero mean and standard deviation 3 0 in order to simulate more general problems with both noise and modeling error. Table 6 summarizes the coefficient of correlations for the four polynomials used also in Table 5 when noise introduced as given in Eq. (4.40) 2 T 2 1 T 1! f f x) ( y (4.40) Figure 18 shows the absolute true residual contours when noise is present in the data. Comparison of Figure 12 and Figure 18 shows that the effect of the noise is to shift slightly the location of high residual regions. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 00.10.20.30.40.50.60.70.8 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 i js i Tje 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 00.10.20.30.40.50.60.70.8 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 i js i Tje

PAGE 96

82 Table 6: Coefficient of correlations between the absolute residual i Tje and the square-root of maximum eigenvalues maxG from Eq. (4.27) for two-variable cubic polynomial example with noise (3 0 ), ( T 1!1 2 1 5 3 50 ) i Corresponding design point ) (2 1x x 12 22 32 42 r 1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 0.590 2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.370 3 (-1,0) and (+1,0) 0 0 -1 0 0.482 4 (0,-1) and (0,+1) 0 -1 0 0 0.299 Maximum of four 0.859 (a) (b) (c) (d) Figure 18: Absolute residual i Tje contours – two-variable cubic polynomial example (polynomial defined in Table 5 and Table 6), with noise (3 0 ). a) Polynomial 1; b) Polynomial 2; c) Polynomial 3; d) Polynomial 4 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej1| 0.8118 0.7581 0.7044 0.6507 0.5970 0.5433 0.4896 0.4359 0.3822 0.3285 0.2747 0.2210 0.1673 0.1136 0.0599 0.0062 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej2| 0.8988 0.8394 0.7800 0.7206 0.6611 0.6017 0.5423 0.4829 0.4235 0.3641 0.3046 0.2452 0.1858 0.1264 0.0670 0.0076 1Tje 2Tje x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej4| 0.7617 0.7112 0.6608 0.6104 0.5600 0.5095 0.4591 0.4087 0.3583 0.3078 0.2574 0.2070 0.1566 0.1062 0.0557 0.0053 4Tje x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej3| 0.7806 0.7290 0.6774 0.6258 0.5742 0.5226 0.4710 0.4195 0.3679 0.3163 0.2647 0.2131 0.1615 0.1099 0.0583 0.0067 3Tje

PAGE 97

83 Comparison curves for absolute residual and the square-root of eigenvalues for the polynomials are shown in Figure 19. Figure 13 and Figure 19 together show that noise caused the residuals spread more at an eigenvalue level calculated. Figure 19: Comparison between square-root of eigenvalues maxG and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 5), with noise (3 0 ) With the maximum of all four absolute residuals, contours given in Figure 20 .b are very similar to eigenvalue contours of Figure 11 (also repeated as Figure 20.a). The slight shift on the locations of maximum residual errors can also been seen in Figure 20.b. This shift is attributed to the noise in the data. Coefficient of correlation r =0.859 between maxG and maxTje is still high. Figure 21 shows the linear correlation between those two quantities and the increased spread due to the noise effect compared to Figure 15 In spite of the cloudy data it reflects the benefit of eigenvalue measure since high residuals are mostly mapped qualitatively by the magnitude of the eigenvalues. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.40.50.60.70.80.9 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 max G i Tje 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.40.50.60.70.80.9 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 max G i Tje

PAGE 98

84 (a) (b) Figure 20: Comparison of square-root of eigenvalues maxG and maximum absolute residual maxTje contours (maximum of residuals among the four polynomials defined in Table 6) – two-variable cubic polynomial example, with noise (3 0 ). a) Eigenvalue field; b) Maximum residual field Figure 21: Comparison between square-root of eigenvalues maxG and maximum of residual errors maxTje due to the four polynomial in Table 5 – two-variable polynomial example, with noise (3 0 ). Coefficient of correlation is 0.859 max G max Tje 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.40.50.60.70.80.9 maxG max Tje 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.40.50.60.70.80.9 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej|max0.8990 0.8417 0.7843 0.7270 0.6697 0.6123 0.5550 0.4976 0.4403 0.3829 0.3256 0.2683 0.2109 0.1536 0.0962 0.0389 x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 Gmax0.8289 0.8034 0.7778 0.7522 0.7266 0.7011 0.6755 0.6499 0.6243 0.5987 0.5732 0.5476 0.5220 0.4964 0.4709 0.4453 ma x G maxTje

PAGE 99

85 The absolute residuals in Figure 20 are due also to noise, so they should also be compared to the estimated standard error ese (square root of prediction variance). Table 7 summarizes the coefficient of correlation between the estimated standard error and the absolute residuals for four polynomials. Table 7: Coefficients of correlation between the estimated standard error and absolute true residuals i Tje for cubic polynomial example with noise (3 0 ), ( T 1!1 2 1 5 3 50 ) i Corresponding design point ) (2 1x x 12 22 32 42 s r 1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 1.127-0.043 2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.5430.363 3 (-1,0) and (+1,0) 0 0 -1 0 0.7930.020 4 (0,-1) and (0,+1) 0 -1 0 0 0.1710.433 Maximum of four 0.126 Figure 22 shows the field of estimated standard error [square root of Eq. (4.8) ] normalized by The root-mean-square-error estimator s for is different for each polynomial as reported in Table 7. The distribution of the estimated standard error of the Polynomial i i ese can be pictured as Figure 22 scaled by relevant s from Table 7. The coefficient of correlation between the estimated standard error and the maximum eigenvalues is only 0.097 for all polynomials in this example as both distributions depend only on the experimental design Similar to previous example without noise, Figure 23 shows i js determining error bound conservatively for residuals i Tje The effect of noise on the conservative representation of i Tje by i js can be seen by the comparison of Figure 17 and Figure 23

PAGE 100

86 The different magnitudes of biased estimates s for from Table 7 caused i js being conservative at different levels. Error bound 1js (Polynomial 1), for instance is the most conservative if used to determine residuals for Polynomial 1 1Tje Figure 22: Normalized estimated standard error /ese for a two-variable quadratic polynomial model with experimental design of Figure 10 Figure 23: Comparison between conservative error bound i js and absolute residual i Tje – two-variable cubic polynomial example (polynomials defined in Table 7), with noise (3 0 ) 0 0.2 0.4 0.6 0.8 1 1.2 00.20.40.60.811.2 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 i Tje i js 0 0.2 0.4 0.6 0.8 1 1.2 00.20.40.60.811.2 Polynomial 1 Polynomial 2 Polynomial 3 Polynomial 4 i Tje i js x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 ESE/s 0.8956 0.8759 0.8562 0.8364 0.8167 0.7970 0.7773 0.7576 0.7379 0.7182 0.6985 0.6788 0.6591 0.6394 0.6196 0.5999 /ese

PAGE 101

87 Cubic Polynomial in Five Variables – without Noise In order to investigate the use of eigenvalue measure in higher dimensions, a fivevariable cubic polynomial is also studied. Similar to two-variable example, a cubic true model, and a quadratic fitting polynomial are studied as summarized in Eqs. (4.41) and (4.42). 2 T 2 1 T 1! f f x x ) ( ) (y (4.41) where T 2 2 2 2 2 2 2 2 T 2 T 1 1 1 1 1 1 1 T 1 T! f f x35 26 16 7 6 2 1 3 5 3 3 3 2 3 2 1 2 2 1 2 2 1 3 1 21 12 7 6 2 1 2 5 2 2 2 1 2 1 1 5 4 3 2 11 ! # ! ! ! ! ! x x x x x x x x x x x x x x x x x x x x x x (4.42) The cubic polynomial used as a true function, ) (x is an RS approximation of cubic model constructed on the wing structural weight data discussed in next chapter. For constructing a quadratic approximation, it is assumed that the data available is only for an FCCD in five-dimension. This experimental design has 43 design points that are not enough to characterize all 56 coefficients in a full cubic approximation, but enough to estimate 21 quadratic coefficients, 1! First, no noise was used and a quadratic approximation, ) ( ˆxy based on 43 function evaluations was constructed. The approximation and the true cubic polynomial were used to find points of high errors. This was done by looking for local maxima of the error using MS Excel Solver, and twenty-one distinct local maxima of the error were determined by using different initial design points. The eigenvalues maxG from Eq.

PAGE 102

88 (4.27) true function values and predictions by the quadratic RS were calculated at these 21 points in addition to five-level full factorial design points (3125 design points). The absolute true errors Tje as given in Eq. (4.39) for those points were calculated, and compared with the square-root of eigenvalues maxG via coefficient of correlation. The coefficient of correlation was found to be 0.366 with p-value less than 0.0001. The coefficient of correlation is lower in this example although there is no noise in the data compared to two-variable polynomial example. Note, however, that only a single polynomial where 2! is not an eigenvector from Eq. (4.27) was tested unlike twovariable example. Figure 24 shows the comparison of maxG and Tje. Figure 24: Comparison between square-root of eigenvalues maxG and absolute true residuals Tje and the trend line manually drawn for largest errors characterized by the eigenvalues– five-variable cubic polynomial example, without noise Although the eigenvalues may have high values where the errors are actually low, the locations where errors are large agree well with the eigenvalue measure as indicated max G Tje 0 5000 10000 15000 20000 25000 00.511.522.533.5 Five-level full factorial design points 21 local error maxima 43 FCCD design points max G Tje 0 5000 10000 15000 20000 25000 00.511.522.533.5 Five-level full factorial design points 21 local error maxima 43 FCCD design points

PAGE 103

89 by the line manually drawn on Figure 24 The local error maxima points also show similar agreement. Note that the design points where the eigenvalues are about 11 (and their square-roots about 3.3) are vertices of the five-dimensional cuboidal, which are part of the FCCD. The conservative error bound js and the absolute residual Tje were compared. Figure 25 indicates residuals are not represented conservatively by the error bound js for some of the high residual points, but for the majority of the design points the error bound estimates are conservative although they are based on FCCD data points only. Figure 25: Comparison between conservative error bound js and absolute true residual Tje – five-variable cubic polynomial example, without noise Cubic Polynomial in Five Variables – with Noise The five-variable polynomial example presented in the previous section was repeated, with random, normally distributed noise, of zero average and standard deviation 3000 in order to simulate more general problems with both noise and Tje js 0 5000 10000 15000 20000 25000 0500010000150002000025000 Five-level full factorial design points 21 local error maxima 43 FCCD data points Tje js 0 5000 10000 15000 20000 25000 0500010000150002000025000 Five-level full factorial design points 21 local error maxima 43 FCCD data points

PAGE 104

90 modeling error. The only difference from the previous example is that function evaluations for 43 data points were performed by Eq. (4.43) instead of Eq. (4.41) that was now used only for true function evaluations 2 T 2 1 T 1! f f x) ( y (4.43) Figure 26 shows the comparison between maxG and the errors based on the noisy data. The coefficient of correlation is now about 0.225 with p-value of less than 0.0001. Reduction in the correlation coefficient is attributed to noise in the data. Figure 26: Comparison between square-root of eigenvalues maxG x and absolute residuals je y – five-variable cubic polynomial example, with noise (3000 ) The conservative error bound js offered similar performance as a substitute of residuals je [Eq. (4.44)] as shown in Figure 27 ) ( ˆ ) (j j jy y ex x (4.44) max G 0 5000 10000 15000 20000 25000 30000 00.511.522.533.5 Five-level full factorial design points 21 local error maxima 43 FCCD design points je max G 0 5000 10000 15000 20000 25000 30000 00.511.522.533.5 Five-level full factorial design points 21 local error maxima 43 FCCD design points je

PAGE 105

91 Figure 27: Comparison between conservative error bound js and absolute residual je – five-variable cubic polynomial example, with noise (3000 ) Effect of Experimental Design: Eigenvalues with a Minimum Bias Design The eigenvalue error measure is function of the fitting and the true models as well as the experimental design. In order to demonstrate the effect of the experimental design, the eigenvalue field was also studied for minimum-bias design in the two-variable polynomial example. As noted earlier, the MSEP expression is usually integrated over the design space and the integral is minimized to find an experimental design that offers minimum bias error on an average basis. This section presents the comparison of the eigenvalue field for FCCD and minimum-bias central composite design (MBCCD) that are summarized in Table 8 js je 0 5000 10000 15000 20000 25000 0500010000150002000025000 Five-level full factorial design points 21 local error maxima 43 FCCD design points js je 0 5000 10000 15000 20000 25000 0500010000150002000025000 Five-level full factorial design points 21 local error maxima 43 FCCD design points

PAGE 106

92 Table 8: Face-centered central composite design (FCCD) and minimum-bias central composite design (MBCCD) for two-variable polynomial example Design point FCCD 2 1, x x MBCCD 2 1, x x 1 -1 -1 -0.70 -0.70 2 -1 0 -0.77 0 3 -1 1 -0.70 0.70 4 0 -1 0 -0.77 5 0 0 0 0 6 0 1 0 0.77 7 1 -1 0.70 -0.70 8 1 0 0.77 0 9 1 1 0.70 0.70 Figure 28 shows the eigenvalue fields for both experimental designs. Note that the field given on the left is the same contour plot given in Figure 11 but re-plotted in accordance with contour level color map of the eigenvalue field determined for MBCCD so that the magnitude of the eigenvalues can be compared over the design space. This comparison verifies what one may expect: on average, MBCCD protects against bias error if a quadratic fitting model is used while the true function is cubic. The vertex design points, however, are much more vulnerable to the bias errors in MBCCD compared to FCCD based on the eigenvalue error measure. This observation agrees with the discussion by Ott and Cornell (1974) that is reported in Khuri and Cornell (1996, p.210) as, “ the averaging MSEP over the design region [as shown in Khuri and Cornell, (1996), p. 209] in fact mask a poor performance by the RS approximation at certain locations of the design region” Residuals due to quadratic RS constructed using MBCCD were also calculated when the cubic true function is Polynomial 1 of Table 5 They are compared with the residuals for FCCD in Figure 29 The observation made for the eigenvalue field

PAGE 107

93 comparison holds also for the residuals. That is, on average residuals are lower for MBCCD case, but the maximum residual is higher. (a) (b) Figure 28: Eigenvalue-contours ( maxG). a) Face-centered central composite design (FCCD); b) minimum-bias central composite design (MBCCD) for twovariable polynomial example (a) (b) Figure 29: Absolute residual-contours in two-variable polynomial example (for Polynomial 1 in Table 5). a) Face-centered central composite design (FCCD); b) Minimum-bias central composite design (MBCCD) x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 Gmax1.2300 1.1613 1.0927 1.0240 0.9553 0.8867 0.8180 0.7493 0.6807 0.6120 0.5433 0.4747 0.4060 0.3373 0.2687 0.2000 maxG x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 Gmax1.2300 1.1613 1.0927 1.0240 0.9553 0.8867 0.8180 0.7493 0.6807 0.6120 0.5433 0.4747 0.4060 0.3373 0.2687 0.2000 maxG x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej1| 1.0000 0.9333 0.8667 0.8000 0.7333 0.6667 0.6000 0.5333 0.4667 0.4000 0.3333 0.2667 0.2000 0.1333 0.0667 0.0000 1Tje x1 x2-1-0.500.5 -1 -0.5 0 0.5 1 |ej1| 1.0000 0.9333 0.8667 0.8000 0.7333 0.6667 0.6000 0.5333 0.4667 0.4000 0.3333 0.2667 0.2000 0.1333 0.0667 0.0000 1Tje

PAGE 108

94 Discussion An approach for identifying regions where large errors are expected due the inadequacy of the fitting model was presented. The approach leads to an eigenvalue problem with the largest eigenvalue indicating the worst error that may be experienced at a point. The examples showed that the eigenvalues may be helpful for identifying regions of high bias error. In particular, positive correlation between the square-root of maximum eigenvalues and the absolute residuals was found. The approach appears to work even for data with moderate amount of noise. The correlation obtained for the fivedimensional problem was lower than the two-dimensional problem. This may be partially due to the design of experiments used for the problems. Ability of FCCD to cover whole design space changes as the dimension of the problem increases. It should be noted that when true function is of higher order than the fitting model, proportion of the number of estimated coefficients gets lower for higher dimensional problems. Although eigenvalues appear to work for determining the potential high-error locations, they do not provide direct quantification for the magnitude of the errors. The alternate estimate of MSEP described in section at page 69 was shown to be available at any design point. Comparison with the residual error suggested that square-root of the alternate estimate of MSEP can be used as a conservative error bound. The use of conservative error bound in RS uncertainty evaluation is presented in the section starting on page 130

PAGE 109

95 CHAPTER 5 HIGH SPEED CIVIL TRANSPORT (HSCT) WING WEIGHT EQUATIONS This chapter presents applications of the methodology and statistical tools given in Chapters 3 and 4 in constructing a wing structural weight equation. In this chapter, there are two types of design parameters: wing planform/shape design parameters and wing structural design parameters. The first type is also referred as configuration design parameters iv denoted as ix when coded via Eq. (3.1) Configuration nx x x !2 1 x is a design point for the n-dimensional RS design space. The RS weight equation ) ( ˆ x y is constructed on the responses ) (xy for the configurations of experimental design. The response of interest is the optimum structural weight, found by finite element (FE) analysis based structural optimizations, which may be viewed as numerical simulations or experiments. In the structural optimizations, the second type of design parameters-structural design variables-are used. HSCT Design Definition The wing example used in this work is a 250-passenger HSCT design with a 5500 nmi. range and cruise Mach speed of 2.4. A general HSCT model developed by the Multidisciplinary Analysis and Design (MAD) Center for Advanced Vehicles at Virginia Tech includes 29 configuration design variables (Balabanov et al., 1996 and 1999; Knill et al., 1999). Of these, 26 describe the geometry; two, the mission, and one, the

PAGE 110

96 thrust. Load cases for HSCT design studies are given in Table 9, where the first two rows reflect normal flight conditions and the others represent severe and critical limit cases. Table 9: Load cases in HSCT design study Load Case Mach Number Load Factor Altitude (ft.) High-speed cruise 2.4 1.0 63175 Transonic climb 1.2 1.0 29670 Low-speed pull-up 0.6 2.5 10000 High-speed pull-up 2.4 2.5 56949 Taxiing 0.0 1.5 0 Here, following Knill et al. (1999) two simplified versions of the problem with five and 10 configuration variables were studied: The five-variable case includes fuel weight Wfuel and four wing-shape parameters: root chord croot, tip chord ctip, in-board leading edge (LE) sweep angle ILE, and the thickness-to-chord ratio for the airfoil t/c The 10-variable case includes variables of the five-variable case and five more variables, that are wing semi-span b/2 out-board leading edge sweep angle OLE, location of maximum thickness tc x max) / (, leading edge radius parameter RLE, and location of inboard nacelle. Figure 30 a and b show a typical wing planform and the configuration variables vi for the fiveand 10-variable cases, respectively. The design space for each case was determined by the lower and upper limits of the configuration variables. Table 10 summarizes the configuration design variable ranges used for fiveand 10-variable cases. Other design parameters-fuselage, vertical tail, mission and thrust-related parameterswere kept unchanged.

PAGE 111

97 (a) (b) Figure 30: HSCT wing planform and five& 10-configuration variables. a) Five configuration variable case; b) Ten configuration variable case Table 10: Configuration design variables for HSCT with corresponding value ranges Number of configuration variables Design Variable 5 10 Root chord, croot (ft.) :1v 150 – 190 :1v 150 – 190 Tip chord, ctip (ft.) :2v 7 – 13 :2v 7 – 13 Wing semi-span, b/2 (ft.) 74 :3v 58 – 78 Inboard LE sweep, ILE :3v 67 – 76 :4v 67 – 76 Outboard LE sweep, OLE 25 :5v 12 – 32 Location of max. thickness, (x/c)max-t 40% :6v38 – 52% LE radius, RLE 2.5 :7v2.1 – 4.1 Thickness to chord ratio at root, (t/c)root :4v 1.5 – 2.7% :8v1.5 – 2.7% Inboard nacelle location, ynacelle (ft.) 20 :9v10 – 35 Fuel weight, Wfuel (lb.) :5v 280000 – 350000 :10v280000 – 350000 Outboard LE sweep v4 + x Location of thickness (fixed) Leading edge (fixed) x y v2 v3 Wing semispan N acelle locations (fixed) v1 x y v2 v4 v1 v5 v3 v9 v8 + x z v6 v7

PAGE 112

98 Structural Optimization and Finite Element Model In order to perform configuration optimization, an estimate of the weight of the aircraft is required. This weight is mostly estimated by traditional weight equations taken from the FLOPS program (McCullers, 1984). However, the weight of the structure needed to carry the bending loads on the wing is not well estimated by such equations because the HSCT carries a very large amount of fuel in the wing compared to subsonic transports. Consequently, Balabanov et al. (1996) developed a procedure of estimating the bending material weight by structural optimization. The weights obtained from many structural optimizations are fitted with RS approximations as function of the configuration design variables and used in the configuration optimization. In this study, a structural optimization procedure based on a finite element model developed by Balabanov et al. (1996 and 1999) with the GENESIS program (VMA, 1997) was used to estimate the bending material weight. The procedure is summarized in Figure 31 The HSCT codes (Balabanov et al., 1996 and 1999; Knill et al., 1999) calculate aerodynamic loads for the load cases given in Table 9 A mesh generator due to Balabanov et al. (1996 and 1999) creates the finite element mesh distributes the aerodynamic and inertia loads onto the structural nodal points and generates input for GENESIS. The finite element model, shown in Figure 32 of the HSCT with a fixed arrangement of spars, ribs and skin panels was employed. The wing and fuselage skin were modeled by membrane elements. Rod elements were used to model spar and rib caps. Vertical rods and shear panels were used for spar and rib webs. The typical wing box cell used in the FE model is shown in Figure 33

PAGE 113

99 Figure 31: Flow chart for the RS based wing weight equation Figure 32: HSCT finite element model and structural design variables RESPONSE SURFACE For WEIGHT EQUATION Statistical analysis code J MP Configuration Variables Design of experiments Configurations Face-Centered Central Composite Design Factorial Design Orthogonal arrays Wing planforms ( like in Figure 4) in design space Configuration design space and data point Aerodynamic calculations and load distribution for the confi g urations Structural analysis and Optimization Codes for aerodynamic load estimations Balabanov et al. (1996) FEA and optimization code G ENE S I S Numerical simulations at data points

PAGE 114

100 Figure 33: Close-up view for the typical wing box cell The model uses 40 structural design variables, including 26 to define skin panel thicknesses, 12 for spar cap areas, and two for the rib cap areas. The objective function is the total wing structural weight, and the wing-bending material weight ( Wb) is calculated based on the values of the optimal structural design variables associated with bending resistance. This procedure generates numerical noise, and configurations of similar total structural weight can have significantly different bending material weights. The optimization methods available in GENESIS are modified feasible directions (MFD), sequential linear programming (SLP), and sequential quadratic programming (SQP). Balabanov et al. (1999) reported that none of the methods had a definite advantage over the others in terms of numerical noise in their HSCT wing structural optimizations. Therefore, in this study the default optimization method, MFD, was used. The numerical noise in the optimization depends on convergence parameters, called control parameters in GENESIS. These are categorized into move limit parameters, convergence criteria, and inner optimization control parameters. There are two loops in GENESIS as shown in Figure 34 The outer loop performs detailed FE analysis of the structure and creates approximations of structural response in order to reduce the number Wing skin panel Spar caps Rib caps Shear webs

PAGE 115

101 Minimize f( x ) such thatg( x ) 0 FEM Analysis Generate approximation f ( x ) andg ( x ) at x qwith move limits x *= x q+1 kth iteration of MMFD, SQP or SLP x q+1= x q k If (f ( x q+1) f ( x q)) / f ( x q) "for ITRMOP iterations If ( x q+1x q)/ x q # or (f( x q+1) f( x q)) / f( x q)"Yes Yes No No Outer Loop Inner Loopof expensive FE analysis. The inner loop performs optimization based on the approximation with move limits imposed to restrict the optimization to the region where the approximation is valid. The approximation and optimization is continued until the change in the design variables is small enough (called soft convergence) or until the change in the objective function is small enough (called hard convergence). Convergence parameters affect the accuracy of optimization, quality of the optimization results, and the computational cost. Figure 34: Flow chart of the optimization in GENESIS (X : relative change criterion for design variables, x; " " : relative convergence criterion for objective function) Three cases of control parameters are used in this study. Case 1 includes the default parameters provided by GENESIS. Case 2 is the same as Case 1 except that a parameter, called ITRMOP was increased from 2 to 5. Case 3, the settings used in previous HSCT studies (Balabanov et al., 1996 and 1999; Knill et al., 1999) employs tighter move limits and convergence criteria than Case 1, with ITRMOP=2. In the following, the bending material weight, ) (xy obtained with Case 1, Case 2 and Case 3

PAGE 116

102 10 20 30 40 50 60 70 80 90 100 110 120 130 -40 -30 -20 -10 0 10 20 30 40%relativechangeindesignno.Wbwere denoted by Wbd ( d for default), Wbh ( h for high accuracy), and Wbt ( t for tight move limits), respectively. Figure 35 shows how the control parameters can affect structural optimization results. Wbd and Wbt found for the five-variable configurations studied throughout this study are graphically compared by Figure 35. Although Wbt uses tighter convergence criteria than the default Wbd, it yielded higher weights than Wbd for the majority of the design points. Upon further investigation, this turned out to be the effect of allowing very small move limits that produced premature convergence. Figure 35: Effect of overly tight optimization parameters: Difference in optimization results between Case 1 and Case 3, bt bd btW W W 100 After extensive experimentation with the control parameters (Kim et al., 2001) and help from the developers of GENESIS, ITRMOP was found to be the most important parameter for improving the accuracy of the optimization for this problem. It controls the convergence of the inner optimization. The inner loop convergence criterion on objective function change must be satisfied ITRMOP consecutive times. It was found that, for some configurations, complete convergence required ITRMOP=5.

PAGE 117

103 Construction of Response Surface Approximations To improve the accuracy of the parameter estimates for the RS approximation, the HSCT configuration variables are scaled/coded to the range (–1, +1) by Eq. (3.1) The root-mean-square-error estimator s [Eq. (3.24) ], coefficient of variation vc [Eq. (3.27) ] and 2aR [Eq. (3.34) ] are used as measures of accuracy in the approximations. Note that s includes both numerical noise, which is partly suppressed by the RS (so that the RS is more accurate than the data), as well as approximation model error due to the low order of the RS. In the following, it is attempted to estimate the two components. Compared to the original RS of Balabanov et al. (1999) the following measures are employed in order to improve accuracy: (i) detection of outlier points by IRLS and their repair by reoptimization using different convergence settings, (ii) check for lack-of-fit by the loworder (quadratic) fitting model, (iii) removal of design points with excessively high weight, (iv) investigation of regions for high-modeling error, (v) application of higherorder models, (vi) check for lack-of-fit by cubic model. Five Configuration Variables Because the work was started with quadratic RS, a face-centered central composite design (FCCD) was used first. This design consists of a complete 25 vertices (32 configurations), 2x5 (10) face centers and one center configuration, for a total of 43 configurations. Next, to accommodate cubic approximations, another set of 43 design points was added, obtained by the FCCD with lower and upper limits of -0.75 and 0.75, respectively, so that this new design box is nested inside the original design box with a common center point. Then 27 configurations determined by D-optimality criterion

PAGE 118

104 (SAS, 1998) and 14 designs determined through an orthogonal array (Owen, 1994) were also added to the design space. Numerical noise and outlier analysis The results obtained for wing-bending material weight by structural optimization in GENESIS for a given configuration differed by up to 40% depending on the optimization method, optimization settings, and even computer used (see Appendix A). For example, it can be seen from Figure 35 that although base data of Wb, Wbd, are lower in general, they can be higher by up to 40% than Wbt. This indicates a substantial amount of noise and existence of outliers that can substantially degrade the accuracy of the RS approximation. Response surface approximation of full quadratic fitting model was first constructed on 126 data of Wbd. Table 11 summarizing data set modifications and quadratic response surface approximations for the five-variable HSCT wing problem, presents the statistics for this approximation in the first set of rows (Stage 1). For the improvement of the approximation the data was first checked for outliers. Of the 126 configurations, thirteen configurations were identified as candidate outliers for repair by IRLS with w [Eq. (3.68) ] smaller than 0.1. Since the data are result of minimization problem, the outliers were considered as candidates for repair only when the data were larger than the IRLS fit. The outlier configurations were repaired by repeating the optimization with ITRMOP increased from 2 to 5 (high accuracy structural optimization). This repair procedure costs one additional, more expensive structural optimization per outlier. On average, the repair required twice the CPU time spent with ITRMOP=2, but it led to substantially lower wing-bending material weight, Wb, for 11 out of 13 candidate outliers.

PAGE 119

105 Table 11: Statistics for five-variable full quadratic response surface approximations for HSCT wing bending material weight Data Stage Description Statistics s (lb.) 4874.6 1 126 points % vc 9.54 Average Wb= 51,104 (lb.) 2aR 0.9620 126 points: s (lb.) 2758.9 2 After Outlier Repair 1 (13/13/11)* % vc 5.52 Average Wb= 50,020 (lb.) 0.9868 s (lb.) 4602.6 3 115 points % vc 10.18 Average Wb= 45,229 (lb.) 2aR 0.9166 115 points: s (lb.) 2054.3 4 After Outlier Repair 1 and Reasonable Design Test % vc 4.64 Average Wb= 44,313 (lb.) 2aR 0.9816 115 points: s (lb.) 1799.5 5 After Outlier Repair 2 (15/7/5)* and Reasonable Design Test % vc 4.08 Average Wb= 44,125 (lb.) 2aR 0.9856 115 points: s (lb.) 1762.3 6 After Outlier Repair 3 (16/3/2)* and Reasonable Design Test % vc 4.00 Average Wb= 44,093 (lb.) 2aR 0.9862 115 points: s (lb.) 1722.9 7 min( Wbd, Wbh) and Reasonable Design Test % vc 3.98 Average Wb= 43,328 (lb.) 2aR 0.9868 s cv, and 2aR by Eqs. (3.24) (3.27) and (3.34) respectively. : a/b/c a, total number of outliers detected by IRLS b, number of outliers with weights above the RS c, number of outliers successfully repaired by ITRMOP=5

PAGE 120

106 The average correction for the 11 outliers was about 12,000 lb., or about 17%. The average Wb for all 126 points was reduced from 51,104 lb. to 50,020 or by about 2%. Then, new RS approximations were constructed from the repaired data. Table 11 includes also the approximation results after the first repair in the second set of rows (Stage 2), indicating improved accuracy through the reduction in root-mean-square-error estimator s and coefficient of variation vc The repair alone reduced the vc from 9.5% to 5.5%. In order to illustrate further how the process affects the approximation fit to the structural weight via structural optimization, the GENESIS results versus the corresponding approximations are plotted for the 126 points in Figure 36 The diagonal line in the figures is the ideal, corresponding to exact equality between the two values. Figure 36a shows the original RS process, and Figure 36b shows the results of the standard IRLS process. That is, the 13 outliers are assigned very low weights, so that the approximation leaves them out. The 13 points are substantially below the diagonal line, indicating that GENESIS gives a much larger weight than that IRLS estimated. For example, the outlier marked as ‘O’ in the Figure 36a, the RS fit predicts Wb of 48,423 lbs., compared to 65,185 lbs. from GENESIS. Comparing Figure 36a and Figure 36b, it is clear that the IRLS procedure moves the RS away from the outliers; for example, for outlier ‘O’, the RS now gives 45,578 lbs. Figure 36c shows a comparison of the response surface fitted through the repaired data. Even though the 13 points are included in the fit, they do not stand out as much after the repair. For example, for point ‘O’, the GENESIS result was reduced from 65,185 lbs. to 41,983 lbs., while the new response surface now

PAGE 121

107 predicts 44,606 lb. These results reflect the filtering capability of the RS, even when outliers exist in the data, and also how repair and IRLS improve accuracy. (a) (b) (c) Figure 36: Outliers and effect of repair on quadratic response surface, five-variable HSCT problem. a) RS approximation for original data (Stage 1, Table 11); b) IRLS approximation and detected outliers; c) RS approximation for repaired data (Stage 2, Table 11) O WbbyGENESIS(lbs.) RS(lbs.)1000030000500007000090000110000130000150000 10000 30000 50000 70000 90000 110000 130000 150000 ideal data O WbbyGENESIS(lbs.) RS(lbs.)1000030000500007000090000110000130000150000 10000 30000 50000 70000 90000 110000 130000 150000 ideal data WbbyGENESIS(lbs.) RS(lbs.)1000030000500007000090000110000130000150000 10000 30000 50000 70000 90000 110000 130000 150000 ideal data

PAGE 122

108 Lack-of-fit In order to test the adequacy of quadratic fitting model the approximation was checked for the lack-of-fit. Since the numerical experiments are used for data generation lack-of-fit test with exact replicates is not possible. Therefore, the near-replicate lack-offit approach (page 52 ) was applied. The near replicate design points around the center design and the data are presented in Table 12. Table 12: The near replicate design points, data and their projections to the center design (reference design point) for lack-of-fit test on quadratic RS 1x 2x 3x 4x 5x ) (xy ) ( ˆ x y ) ( ˆ x y$ $ $ $ x 0 0 0 0 0 39,973 39,973 39,973 1x -0.16 0 0 0 0 43,446 40,711 40,651 2x 0.16 0 0 0 0 37,712 40,236 40,214 3x 0 -0.16 0 0 0 39,728 39,755 39,743 4x 0 0.16 0 0 0 40,427 40,404 40,384 5x 0 0 -0.16 0 0 40,747 43,209 43,245 6x 0 0 0.16 0 0 43,857 41,210 41,242 7x 0 0 0 -0.16 0 44,111 40,720 40,974 8x 0 0 0 0.16 0 39,309 42,302 42,136 9x 0 0 0 0 -0.16 40,096 40,638 40,588 10x 0 0 0 0 0.16 43,333 42,863 42,868 by coefficient estimates of RS for 126 design points (Stage 1 in Table 10) $ by coefficient estimates of RS for 126 design points after repair 1 (Stage 2 in Table 10) Results of the lack-of-fit test using data in Table 12, are summarized in Table 13 before and after repair. Large F ˆ -statistics and small p-values suggest that the data suffers from lack-of-fit when quadratic approximations are used. Recall that the p-value

PAGE 123

109 in this test indicates the chance of making an error by rejecting the null hypothesis-a quadratic model is adequate. Table 13: Results of the lack-of-fit test for quadratic RS fitted to 126 design points of five-variable HSCT wing problem Statistical Parameters Before repair (Stage 1 in Table 11) After repair 1 (Stage 2 in Table 11) M 126 126 N 136 136 nb 21 21 F ˆ 17.57 5.61 M N n MbF , 05 0 2.59 2.59 p-value ~ 0 0.0025 In the light of this statistical evidence, fitting model of cubic polynomial or higher degree should be tried in order to reduce modeling errors. It is also possible to reduce modeling errors by reducing the size of the design space. Previous studies for HSCT (Guinta et al., 1994; Balabanov et al., 1996 and 1999) showed that Wb for feasible or near-optimal designs is in the range of about 20,000-40,000 lbs. Based on this experience, it was decided to exclude designs with Wb larger than 80,000 lb. from the response surface. This approach of excluding design points with unreasonable results is known as the reasonable design space approach (Balabanov et al., 1999). It is expected to improve the modeling accuracy of the RS by shrinking the region where the RS is fitted. Eleven designs were designated unreasonable after repair of the detected outliers, and new RS approximations were obtained by excluding them. In Table 11 without repair the reasonable design space approach did not reduce the error as reported in the third set of rows (Stage 3). After repair, however, excluding unreasonable designs

PAGE 124

110 reduced the modeling error and helped substantially the quadratic approximation accuracy (Stage 4 in Table 11 ). This indicates the importance of outlier analysis and repair. The lack-of-fit test was also applied on the data set after reasonable design space approach before and after repair. The results for these tests are given in Table 14. Substantial drop in F ˆ -value (comparing Stage 3 and 4 of Table 11) also shows the affect of the repair as in Table 13 Last column of Table 14 still suggests significant statistical evidence for lack-of-fit when quadratic fitting model is used. The lower F ˆ and higher pvalues in Table 14 compared to Table 13 is an indication of the benefit from the reasonable design space approach to reduce the modeling error. Table 14: Results of the lack-of-fit test for quadratic RS fitted to 115 design points of five-variable HSCT wing problem Statistical parameters Before repair (Stage 3 in Table 11) After repair 1 (Stage 4 in Table 11) M 115 115 N 125 125 nb 21 21 F ˆ 17.15 3.29 M N n MbF , 05 0 2.62 2.62 p-value ~0 0.0213 The usefulness of conducting additional outlier detections were also checked. The results in Table 11 show significant improvement from the second round of repair (Stage 5), but very little from the third round (Stage 6). Performance of the full quadratic RS after repair of all 115 points (Stage 7) is very close to Stage 6.

PAGE 125

111 Error analysis based on MSEP eigenvalues Before using a cubic model in order to reduce the modeling error further, the error measures presented in Chapter 4 were investigated for the wing example. Square-root of maximum eigenvalues maxG from Eq. (4.27) and the conservative error bound js from Eq. (4.34) were calculated at the 3146 design points used in polynomial examples (page 87 ). Table 15 presents the coefficients of correlations. Since eigenvalues are the function of experimental design, but not the data, they are same for 126 design points before and after repair. The increase in the correlation is attributed to the reduced effect of high amplitude of noise or outliers in shifting the RS towards themselves, and changing the locations of high prediction errors. Similar effects of removing high-weight designs also caused increase in the correlation coefficient. Table 15: Coefficients of correlation r between maxG and js for five-variable HSCT wing problem at 3146 design points RS on 126 design points (Stage 1 in Table 11) RS on 126 design points after repair (Stage 2 in Table 11) RS on 115 design points after repair (Stage 4 in Table 11) 0.233 0.305 0.654 Figure 37 a and Figure 37b compare maxG and js when RS was constructed on the original and repaired 126 data points, respectively. As mentioned previously, eigenvalues are same since it is not function of the data, but js has shown decrease after the repair. Figure 38 presents the comparison for js and maxG when RS was constructed 115 design points after repair.

PAGE 126

112 (a) (b) Figure 37: Comparison between square-root of eigenvalues maxG and conservative error bound js – quadratic RS of HSCT 126 design points. a) Original (Stage 1 in Table 11); b) After repair (Stage 2 in Table 11). Coefficient of correlations (a) 0.233 and (b) 0.305 Figure 38: Comparison between square-root of eigenvalues maxG and conservative error bound js – quadratic RS of HSCT 115 design points (Stage 4 in Table 11). Coefficient of correlation is 0.654 The conservative estimate of MSEP at the 115 RS data points were predicted as js from Eq. (4.34) as well js from a similar expression that uses directly the residual data je from Eq. (3.8) after repair instead of an RS approximation max G js 0 2000 4000 6000 8000 10000 12000 14000 01234 max G js 0 2000 4000 6000 8000 10000 12000 14000 01234 max G js 0 2000 4000 6000 8000 10000 12000 14000 01234 max G js 0 2000 4000 6000 8000 10000 12000 14000 01234 max G js 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 02468 max G js 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 02468

PAGE 127

113 2 2j je s s 1 -1 1 T 1 T 1f ) X (X f (5.1) The conservative estimate of MSEP js was compared with js in Figure 39. A perfect agreement cannot be expected with the limited number of data and noise. Having majority of errors of the design points (79 out of 115) estimated conservatively (assuming the js are exact) and positive correlation coefficient 0.484 js offers a tool to quantify the RS uncertainty throughout the design space as it can be calculated for any design point. Figure 39: Comparison of predictions of square-root of MSEP, js and js at 115 design points quadratic RS (Stage 4 in Table 11). Coefficient of correlation is 0.484. Cubic fitting model in the approximation The data generated and used for Table 11 also enabled the construction of a full cubic weight equation as well as cubic models with terms eliminated by stepwise regression (SAS, 1998). Table 16 summarizes results for cubic approximations using the same data sets of Table 11. js 0 1000 2000 3000 4000 5000 6000 0100020003000400050006000 data ideal js js 0 1000 2000 3000 4000 5000 6000 0100020003000400050006000 data ideal js

PAGE 128

114 Table 16: Statistics for five-variable cubic response surface approximations for wing bending material weight Data Full Reduced Stage Description Statistics cubic cubic s (lb.) 4844.4 4329.2 1 126 points % vc 9.48 8.47 Average Wb= 51,104 (lb.) 2aR 0.9624 0.9700 126 points: s (lb.) 2066.2 1925.8 2 After Outlier Repair 1 (13/13/11)* % vc 4.13 3.85 Average Wb= 50,020 (lb.) 2aR 0.9926 0.9936 s (lb.) 5005.3 4295 3 115 points % vc 11.07 9.50 Average Wb= 45,229 (lb.) 2aR 0.9013 0.9273 115 points: s (lb.) 1633.6 1500.8 4 After Outlier Repair 1 and Reasonable Design Test % vc 3.69 3.39 Average Wb= 44,313 (lb.) 2aR 0.9884 0.9902 115 points: s (lb.) 1408 1282.3 5 After Outlier Repair 2 (15/7/5)* and Reasonable Design Test % vc 3.19 2.91 Average Wb= 44,125 (lb.) 2aR 0.9912 0.9927 115 points: s (lb.) 1405.1 1274.3 6 After Outlier Repair 3 (16/3/2)* and Reasonable Design Test % vc 3.19 2.89 Average Wb= 44,093 (lb.) 2aR 0.9912 0.9928 115 points: s (lb.) 933 855.1 7 min( Wbd, Wbh) and Reasonable Design Test % vc 2.15 1.97 Average Wb= 43,328 (lb.) 2aR 0.9961 0.9968 s cv, and 2aR by Eqs. (3.24) (3.27) and (3.34) respectively. : a/b/c a, total number of outliers detected by IRLS b, number of outliers with weights above the RS c, number of outliers successfully repaired by ITRMOP=5

PAGE 129

115 Comparison of the first and third stages in Table 11 and Table 16 shows that addition of the cubic terms did not improve significantly the accuracy of the RS when the outliers were not repaired. This indicates that the root-mean-square-error estimator s was dominated by noise for these data sets. Comparison of Stages 2 and 4 shows that the effect of using cubic terms is more pronounced after the outliers were repaired. This indicates that the error due to noise was substantially higher than the modeling error before the repair of the data. Since the root-mean-square-error estimator s was dominated by the larger errors, it was reduced by 4-5% by repair. Lack-of-fit tests were performed for cubic RS approximations in Table 16. The results of the tests are given in Table 17. Table 17: Results of the lack-of-fit test for cubic RS fitted to data of five-variable HSCT wing problem Statistical parameters 126 points (Stage 1 in Table 16) 115 points (Stage 3 in Table 16) 115 points After repair 1 (Stage 4 in Table 16) M 126 115 115 N 136 125 125 nb 56 56 56 F ˆ 12.84 13.35 1.73 M N n MbF , 05 0 2.61 2.62 2.62 p-value ~0 ~0 0.1752 Cubic response surfaces in Stage 1 and 3 suffered from lack of fit similar to quadratic model. After repair, however, cubic RS (Stage 4) results show that there is no statistical evidence for the inadequacy of the cubic model whereas quadratic RS had suffered from the model inadequacy according to the lack-of-fit test. The p-value for Stage 4 tells that there is about 17% chance of making an error if one rejects that cubic

PAGE 130

116 model is adequate for fitting the data. It can be concluded that the observation for adequacy of the cubic model was strongly affected by the outliers before repair. Similar to quadratic RS results, second round outlier detection and repair (Stage 5) helped to improve the accuracy, but third round (Stage 6) did not make much of a difference. The root-mean-square-error estimator s was reduced by another 1% when rest of the data points other than detected outliers was also repaired (Stage 7). This improvement was not observed with the quadratic RS. Table 11 and Table 16 together with the lack-of-fit test results ( Table 13 Table 14 and Table 17 ) show that the RS fit and outlier repair reduce the noise errors while the use of cubic polynomials reduces the RS modeling error. Summarizing the results from Table 11 and Table 16, it appears that the most effective approach is to conduct a single round of repair, reducing s to 3.4% of Wb. Altogether then, the use of the RS reduced the optimization error from about 9% to about 3%. Slightly lower error level can be achieved by optimizing every configuration with ITRMOP=5. However, this is more costly; in addition, without the use of the RS techniques, it would not have been known that there are large errors with the default ITRMOP=2. The reduced cubic RS at Stage 6 of Table 16 is fairly accurate and is used for uncertainty evaluations in section from page 123 It can also be used to evaluate the eigenvalues from mean squared error criterion. Assuming this RS as the true structural weight equation prediction error of the quadratic RS at Stage 6 of Table 11 can be compared with the square-root of eigenvalues maxG. Figure 40 presents this comparison.

PAGE 131

117 Figure 40: Comparison between square-root of eigenvalues maxG and absolute error between reduced cubic and quadratic RS – five-variable HSCT wing problem, Stage 6 in Table 11 and Table 16. Coefficient of correlation is 0.763 The absolute error between quadratic RS and reduced cubic RS at Stage 6 agrees well qualitatively with maxG at 3146 design points used in polynomial example (p. 87 ) The coefficient of correlation is 0.763. Note that the correlation is especially good for the design points where the error between quadratic and cubic response surfaces is higher than 5,000 lb. There about 248 such points, most of them are of weight exceeding 80,000 lb. that was considered as the limit for the feasibility. Only 13 of the 248 points were feasible based on this limit. Data vs. RS A check was performed for the remaining error in the data after correcting the detected outliers and how RS approximation handles that. The last set of RS approximation summary Stage 7 in Table 11 and Table 16 present results based on the max G 0 5000 10000 15000 20000 25000 30000 02468 3146 CHECK POINTS 115 RS construction pointsresidual max G 0 5000 10000 15000 20000 25000 30000 02468 3146 CHECK POINTS 115 RS construction pointsresidual absolute error max G 0 5000 10000 15000 20000 25000 30000 02468 3146 CHECK POINTS 115 RS construction pointsresidual max G 0 5000 10000 15000 20000 25000 30000 02468 3146 CHECK POINTS 115 RS construction pointsresidual absolute error

PAGE 132

118 minimum Wb from the default and high accuracy structural optimizations at all points. Even in this fully repaired data, there is still noise mainly due to the process of extracting the Wb from the structural weight as shown for the design line between two outliers in Figure 41. This further correction for the remaining errors did not improve the quadratic approximation much, but the reduced noise helped the cubic approximations. Figure 41: Noisy wing bending material weight Wb compared to smoother objective function (total wing structural weight) Repair vs. IRLS The efforts for RS weight equation accuracy showed the primary factor in the improvement was the outlier analysis and repair. The repair added additional computational cost unlike the use of IRLS directly as an approximation eliminating the outliers by zero weighting. To investigate how repaired RS approximation performs compared to RS leaving out the outliers, new sets of design points were created. A design line connecting the vertex of lower limits to the vertex of upper limits (where all design variables are all -1 and all +1 respectively) is obtained. A schematic design weight(lbs.)159131721 40000 45000 50000 55000 60000 65000 70000 75000 80000 85000 90000 objectivefunction Wb

PAGE 133

119 representation of this extreme value design line is shown in Figure 42a for a threedimensional design box. Along the extreme value design line for the 5 configuration variables, 21 equally spaced points were used as shown in Figure 42b. Another design line connecting two of the outliers from the second-round IRLS application was also generated similarly to the extreme value design line. (a) (b) Figure 42: Extreme value design line. a) Design line for 3-D design box; b) Design line for five HSCT configuration variables Structural optimization results using both ITRMOP=2 and ITRMOP=5 were obtained at these 21 points of each design line as shown in Figure 43 The minimum of the two cases for each point was considered as the true optimum. These GENESIS results were compared with the predictions by the approximations. Figure 43 indicates why repair of the outliers may be advisable rather than simply leaving them out. Removing the outliers from the data makes the approximation sensitive to the location in the design space. It worked well on the extreme value design line (Figure 43a), but not as well as the repaired RS after repair for the other design line (Figure 43b). Table 18 shows a summary for average residual percentages from RS approximation constructed over the design space, but leaving out the outliers (i.e., IRLS) (-1,-1,-1) x3 x2 x1 (1,1,1) 2 1 3 19 1 -1.0 -0.9 -0.8 …… 0.8 0.9 1.0 2 -1.0 -0.9 -0.8 …… 0.8 0.9 1.0 3 -1.0 -0.9 -0.8 …… 0.8 0.9 1.0 4 -1.0 -0.9 -0.8 …… 0.8 0.9 1.0 5 -1.0 -0.9 -0.8 …… 0.8 0.9 1.0 variable 20 21

PAGE 134

120 and RS approximation after outlier repair. For the 100 design points other than outliers used in constructing the approximations and for the 21 design points of the extreme value design line, the errors in both approximations are very close. The advantage of outlier repair versus the more standard elimination by IRLS is evident at the outlier points and on the design line between two outliers. On this line, the average error after repair is about 4%, but the IRLS approximation has an average error of about 10%. (a) (b) Figure 43: Comparison of RS with outlier repair and with outlier elimination over the design line, five-variable HSCT problem. a) Extreme value design line; b) Design line connecting two outliers Table 18: Average Residual Percentage for IRLS fit and RS after Outlier Repair, five-variable HSCT problem Design Points Average of je (%) by IRLS Average je (%) by RS with repair 100 points excluding outliers 1.65 1.79 15 outlier points 7.12 3.72 21 points on extreme value design line 3.47 3.55 21 points on design line between two outliers 9.70 3.98 design Wb(lbs.)159131721 30000 35000 40000 45000 50000 55000 60000 65000 genesis RScubicexcludedoutliers RScubicrepaired design Wb(lbs.)159131721 40000 45000 50000 55000 60000 65000 genesis RScubicexcludedoutliers RScubicrepaired

PAGE 135

121 Ten Configuration Variables Similar to the five-variable case, a FCCD was first used to represent the design space of ten configuration variables. This design employs 1045 configurations, which is substantially more than the 66 coefficients. To reduce that number, a reasonable design criterion was used as suggested and done by Balabanov et al. (1999). The FLOPS program (McCullers, 1984) was used for an inexpensive estimate of the weight, and configurations with Wb <15,000 lb. or Wb 80,000 lb. were rejected. Also, using an approximate estimate of the range available with the maximum available fuel designs with range below 5000 nmi. were rejected (the actual range requirement is 5500 nmi.). 696 of the original 1045 designs were found reasonable, and the D-optimality criterion in JMP (SAS, 1998) was used to select 292 configurations. For a cubic model approximation, the required number of data points is substantially higher (286 coefficients to be characterized). In addition, the number of required levels for a cubic is at least four so even a full FCCD is inadequate. To satisfy the level requirement, an orthogonal array of 250 configurations of five levels (Owen, 1994) was added for a total of 542 points. Full quadratic, full cubic and reduced cubic (by mixed-mode stepwise regression) models were used to approximate the wing-bending material weight. The accuracy of the approximations is summarized in the first three rows of Table 19 Comparing the Stage 1 in Table 11 and Table 16 with Stage 1 in Table 19, somewhat higher improvement going from quadratic to cubic polynomials was noted in the ten-variable problem. That is an indication that level of the error due to noise and modeling error is closer for ten-variable case.

PAGE 136

122 Table 19: Statistics for 10-variable response surface approximations for HSCT wing bending material weight Stage Data Description Statistics Full quadratic Full cubic Reduced cubic s (lb.) 4576.6 3472.2 3044.9 1 542 points % vc 10.65 8.08 7.09 Average Wb= 42969 (lb.) 2aR 0.9667 0.9808 0.9853 542 points: s (lb.) 3310.4 2169.6 1895.9 2 After Outlier Repair 1 (36/27/23)* % vc 7.8 5.11 4.47 Average Wb= 42431 (lb.) 2aR 0.9813 0.9919 0.9939 492 points s (lb.) 3308.6 2805.1 2418.5 3 After Reasonable Design Test % vc 8.74 7.41 6.39 Average Wb= 37852 (lb.) 2aR 0.9709 0.9791 0.9845 492 points: s (lb.) 2229.9 1551.2 1320.2 4 After Outlier Repair 1 and Reasonable Design Test % vc 5.95 4.14 3.52 Average Wb= 37486 (lb.) 2aR 0.986 0.9932 0.9951 492 points: s (lb.) 2061 1302.9 1121.8 5 After Outlier Repair 2 (43/17/15)* and Reasonable Design Test % vc 5.52 3.49 3.01 Average Wb= 37327 (lb.) 2aR 0.9879 0.9952 0.9964 492 points: s (lb.) 2054.6 1285.6 1114.8 6 After Outlier Repair 3 (43/6/1)* and Reasonable Design Test % vc 5.51 3.44 2.99 Average Wb= 37320 (lb.) 2aR 0.9879 0.9953 0.9965 492 points: s (lb.) 2230.7 1228 1065.5 7 min( Wbd, Wbh) and Reasonable Design Test % vc 6.06 3.34 2.9 Average Wb= 36781 (lb.) 2aR 0.9854 0.9956 0.9967 s cv, and 2aR by Eqs. (3.24) (3.27) and (3.34) respectively. : a/b/c a, total number of outliers detected by IRLS b, number of outliers with weights above the RS c, number of outliers successfully repaired by ITRMOP=5

PAGE 137

123 The steps described for the five-variable case for outlier determination were also followed for the ten-variable case. 27 configurations were detected as candidate outliers by the IRLS approximation. Re-optimization by ITRMOP=5 resulted in lower Wb for 23 of the 27 designs. Stage 2 in Table 19 presents the approximation results after the first repair. Compared to Table 11 and Table 16 repair made less difference for the tenvariable case. This may reflect the lower percentage of outlier points. A reasonableness check based on GENESIS calculations resulted in improvement for quadratic approximation even before repair (Stage 3, in Table 19). Increasing the number of variables is expected to increase the importance of the modeling (bias) error. Indeed, Table 19 shows that the improvement due to the cubic terms is more pronounced than in the five-variable case, Table 11 and Table 16. Overall, Table 19 shows that a single round of repair and reasonableness check (Stage 4) reduced the optimization error from about 7% to 3.5%. Two more subsequent IRLS and repair applications were also performed. The second round repair (Stage 5) reduced vc 0.5% more, but in third round (Stage 6) only one outlier was repaired that did not cause a significant improvement. Design for Uncertainty An important application of RS techniques is design for uncertainty. Design for uncertainty typically entails much higher computational cost than the corresponding deterministic design. RS approximations can be used to reduce the high computational cost associated with the methods for designing for uncertainty with fuzzy set methods or probabilistic design via Monte Carlo simulations.

PAGE 138

124 The effects of parameter uncertainty on the wing structural weight were presented in this section for five-variable HSCT wing problem. The Fuzzy Set Theory and the interval type definition of parameter uncertainties through the theory were applied by using the RS weight equation. Unfortunately, the RS itself, being an approximation, adds another source of uncertainty. Response surface wing weight equations of fiveconfiguration variable case, created from the results of large number of structural optimizations of a high speed civil transport (HSCT), as presented in previous sections of Chapter 5, are used here for demonstrating the parameter uncertainty and effects of RS on uncertainty. The reduced cubic RS approximation for wing bending material weight after third repair cubicRS in Table 16 (Stage 6) is treated as the fuzzy function in this study. Table 20 provides more details for the RS. Table 20: RScubic term estimates and statistics, five-variable HSCT problem Term Estimate Term Estimate Term Estimate Term Estimate Intercept 41552.2 2 2x -355.2 3 2 1x x 1633.9 2 5 2x x 1141.3 1x -15287.8 3 2x x 823.8 4 2 1x x -1097.2 3 2 1x x x -882.2 2x -700.8 4 2x x -298.1 5 2 1x x 1264.4 4 2 1x x x 547.3 3x 16516.8 5 2x x -43.5 5 2 2x x -900.7 4 3 1x x x 3088.6 4x -15390.1 2 3x 2026.5 2 3 2x x -2045.4 5 3 1x x x -585.9 5x 3026.7 4 3x x -6339.5 3 3x -3788.0 5 4 1x x x 614.7 2 1x 6476.2 5 3x x 1092.3 4 2 3x x -1865.0 5 4 3x x x -536.9 2 1x x -656.3 2 4x 6780.6 2 4 1x x -2214.7 3 3x -3788.0 3x xx -8825.9 5 4x x -1403.8 2 4x x 1237.5 4 1x x 7536.9 2 5x 281.5 2 4 3x x 2584.9 5 1x x -1198.4 2 2 1x x 1081.1 3 4x -1986.8 Brief description of the Fuzzy set model can be found in Appendix B, p. 165

PAGE 139

125 The approximation, in fact, was constructed as function of five configuration variables (coded form, xi). In the calculations of uncertainty, the density, !, and yield stress, Y" were added as additional uncertain parameters. These two parameters may easily be related with structural weight when stress constraints dominate. When only stress constraints are present, it is clear that structural weight is proportional to density and inversely proportional to yield stress. Buckling and minimum gage constraints complicate the relationship. Here, the relation between the structural weights Ws for two different materials is assumed as % & r s sf W W1 2 (5.2) where 1 Y 2 Y" / / rf (5.3) With only stress constraints, & and % are equal to 1 and 0, respectively. In order to determine these constants, six configurations with different levels of bending material weight and with different number of active minimum gage constraints were studied for different andY" values. The best fit to the data was 7612 0 & and 2388 0 % The uncertain wing bending material weight problem,bW as fuzzy function can now be expressed as function of the uncertain variables, cubic al no bRS W ) 2388 0 ) / ( / 7612 0 (min Y Y (5.4) Parameter Uncertainty: Material Properties and Geometry Triangular membership functions (see Appendix B, p. 165) were assumed for all fuzzy variables. Uncertainty was studied at an alpha level cut on membership function of

PAGE 140

126 the fuzzy variables. An alpha level cut defined as the real interval where the membership function is larger than a given value, (Klir and Yuan, 1995, p. 19). The interval for a fuzzy variable is determined by the uncertainty level at the level cut. For instance, if there is 2% uncertainty about the nominal value of 1x at an cut the limits of the interval are 198 0 x and 102 1 x for the cut. The fuzzy variables, percentage membership function bound ( = 1.0) and percentage uncertainty corresponding to = 0.5 level cut definitions are summarized in Table 21. Table 21: Uncertainty levels and ranges for fuzzy variables in five-variable HSCT wing problem Fuzzy Variable Uncertainty 0 1 Uncertainty 5 0 Root chord length (ft) x1 4% 2% Tip chord length (ft) x2 10% 5% Inboard L.E. sweep angle ( ) x3 3% 1.5% Thickness to chord ratio x4 6% 3% x5, Fuel weight (lbs.) x5 20% 10% Density (lb/ft3) 10% 5% Yield stress (lb/ft2) Y" 10% and +30% 5% and +15% The uncertainty interval or band of the response, wing bending material weight, is determined through the vertex method (Dong and Shah, 1987). The method searches the lower and upper limits of the fuzzy function interval associated with the level cut. Combinations of fuzzy variable extremes at the associated level cuts construct a hypercube around the point of nominal values. The limits for the function and the uncertainty band are then sought at the 2n vertices of the hypercube and at possible interior global extreme points, if any. Thus the evaluation of a fuzzy function requires the solution of a global optimization problem for each cut. Figure 44 shows the

PAGE 141

127 method for a three-variable case. The seven uncertain variables: root chord length, tip chord length, inboard leading edge sweep angle, thickness to chord ratio, fuel weight (configuration variables), density and yield stress (material related variables) are denoted 4 3 2 1,x ,x ,x x ,5x and Y" respectively. Therefore, the fuzzy response surface in Eq. (5.4) may be written as ) (5 4 3 2 1,x ,x ,x ,x x fcubic RS (5.5) Figure 44: Description of vertex method for three-variable case Uncertainty analysis may allow a designer to choose from designs with similar performance on the basis of their level of uncertainty. In order to illustrate that point the variation in level of uncertainty of designs with similar weights was sought to measure. In the present study the magnitude of the uncertainty in a variable y is taken to be the magnitude of support of its membership function at studied level cut, l uy y (as in Figure 57 given in the Appendix B). So for the bending material weight we use bl buW W as in Figure 44. Design Response Nominal n n nx x x3 2 1, Wbn Vertex 1 l l lx x x3 2 1, Wb1 Vertex 2 u l lx x x3 2 1, Wb2 Vertex 3 l u lx x x3 2 1, Wb3 Vertex 4 u u lx x x3 2 1, Wb4 Vertex 5 l l ux x x3 2 1, Wb5 Vertex 6 u l ux x x3 2 1, Wb6 Vertex 7 l u ux x x3 2 1, Wb7 Vertex 8 u u ux x x3 2 1, Wb8 1 2 3 4 5 6 7 8 lx1ux1x1n lx2ux2x2n lx3ux3x3n buWWbn Uncertainty band blW=min(Wb1, Wb2,….,Wb8) buW=max(Wb1, Wb2,….,Wb8) blW

PAGE 142

128 A nominal value for the wing bending material weight, bnW was selected as 50,000 lbs. Then pairs of designs was sought for weights near 50,000 lbs. (500 2 000 50 ) with maximal difference in uncertainty bounds ) (bl buW W for 5 0 This is formulated as an optimization problem, 500 2 000 50 ) ( 500 2 000 50 ) ( ) ( ) ( max2 1 1 2 bn bn bl bu bl buW W W W W W (5.6) There are ten design variables for this optimization problem: the first five define the configuration variables for Point 1 and the remaining five for Point 2. In order to perform the optimization [Eq. (5.6)], two cubic response surfaces were constructed for buW and blW based on 3125 points of a full factorial with five-levels. The computational cost was not a problem although each point required the use of vertex method. This is due to the use of original RS for the bending material weight, cubicRS Each of the 3125 design points requires 27 Wb calculations (seven uncertain variables) at vertices, not practical without RS approximation. Next, the Solver tool of Microsoft EXCEL was used to solve the optimization problem given in Eq. (5.6). A number of solutions by EXCEL Solver from different initial values defining two design points were obtained. The solutions identified four design points as candidate vertex points, and three of them denoted as 2 1, ( ( and 3( were selected to define a triangle in a plane. 3 3 2 2 1 1( ( ( () ) ) (5.7) where 1i)and 1 0 i) for i =1, 2, 3. Equation (5.7) was used to obtain 66 design points distributed in the triangle. For each point the wing bending material weight and its

PAGE 143

129 uncertainty bandwidth were calculated by the vertex method. Figure 45a and Figure 45b show the contour plots over the triangle for nominal wing bending material weight and uncertainty bandwidth, respectively. (a) (b) (c) Figure 45: Five-variable HSCT case: Contours in plane with almost constant wing bending. a) Nominal wing bending material weight contours; b) parameter uncertainty band width (upper bound minus lower bound) contours; c) Modeling (RS) uncertainty band width contours nominal 48607 48345 48083 47821 47559 47297 47035 46773 46511 46249 45987 45725 45463 45201 44940 44678 modeluplow 4045 3984 3923 3862 3801 3740 3679 3618 3557 3496 3436 3375 3314 3253 3192 3131 uncer 38664 37950 37236 36522 35808 35094 34380 33666 32952 32237 31523 30809 30095 29381 28667 27953

PAGE 144

130 It is seen that for designs with weight of about 47,500 lbs. the level of weight uncertainty band can vary from 27,850 lbs. to 38,850 lbs. for 5 0 In other words, designs of similar wing bending material may vary from approximately 29 % to approximately 41 % in terms of uncertainty. RS Uncertainty Uncertainty due to the use of the RS approximation was quantified by following the procedure given in the section at page 69. cubicRS given in Table 20 was used in Eq. (4.30) as ) ( ˆjyx to calculate ) (j Rex. Design points jx are from a data set including the original data set used in cubicRS construction (115 design points) and 37 additional design configurations used to check the magnitude of noise. With the Re calculated by Eq. (4.31) a modified data ) (maxj Rex was formed, and a cubic RS approximation ) ( ˆj Rex was constructed for ) (maxj Rex to predict the relative residual due to the RS. Following Eqs. (4.33) and (4.34) the conservative error bound js can be obtained at any design point. The success of the conservative error bound was checked by comparing it with the MSEP estimate js [Eq. (5.1) ]. The comparison of these quantities is shown in Figure 46 and the coefficient of correlation was found as 0.873. The data points where js was not conservative compared to js are attributed to the remaining noise that directly affects js. For the RS uncertainty quantification js was considered conservative. The conservative error bound predictions were used to evaluate the effect of the RS uncertainty on the triangle presented in Figure 45 c, showing variations of modeling and noise uncertainty due to RS ranging from 3,000 lb. to 4,000 lb.

PAGE 145

131 blackdesign graydesign Blue desi g n Red desi g n Figure 46: Comparison of predictions of square-root of MSEP, js(=x) and js(=y) at 152 design points of five-variable HSCT wing problem – reduced cubic RS given in Table 20 (=Stage 6 in Table 16) Two vertices of the triangle checked for sensitivity to uncertainty were studied further and were labeled as Blue and Red designs (bottom and rightmost vertices of triangular plane on Figure 45 respectively). The Blue and Red planforms are shown in Figure 47. Figure 47: Blue and Red design planforms Lower and upper bound configurations found by the RS based vertex method around the Blue and Red nominal designs were optimized in GENESIS in addition to the 0 1000 2000 3000 4000 01000200030004000 data ideal js js 0 1000 2000 3000 4000 01000200030004000 data ideal js js

PAGE 146

132 Blue and Red designs. The RS based predictions for parameter uncertainty alone, the contribution due to RS modeling uncertainty and the results of optimization in GENESIS are summarized in column two, four and five of Table 22, respectively. Overall the estimated ranges taking RS modeling uncertainty into account correlate well with GENESIS results. These results indicate that even with a fairly accurate cubic RS, errors in the RS due to noise and modeling errors (due to the use of low-order polynomial) contribute significantly to the overall uncertainty. Table 22: Uncertainty for blue and red design: Combined effect variable fuzziness and RS modeling uncertainty Blue Design Wb (lbs.) RS js (1 ' ' js)RS (lbs.) Wb (lbs.) GENESIS Nominal 47500 2367 45133 – 49867 50972 Lower 34851 1688 33163 36539 34642 Upper 62715 2059 60656 64774 64921 Uncertainty band 27864 24117 31611 30279 Red Design Wb (lbs.) RS js (1 ' ' js)RS (lbs.) Wb (lbs.) GENESIS Nominal 47500 2208 45292 49708 47205 Lower 31396 1609 29787 33005 33560 Upper 70238 2452 67786 72690 71106 Uncertainty band 38842 34781 42903 37546 Discussion This chapter presented the use of RS techniques to construct a wing structural weight equation customized for a high-speed civil transport. The use of response surface techniques for handling noisy data was demonstrated for fiveand ten-variable examples of HSCT. It was shown that an iteratively re-weighted least square (IRLS) procedure can effectively identify outlier design points where the structural optimization produced over-

PAGE 147

133 heavy designs due to premature convergence. Repairing the outliers was shown to be superior to a standard IRLS procedure that simply eliminates them from consideration. It was also shown that quadratic approximation for the weight equation suffered from bias error. Elimination of unreasonable designs with high wing-bending material weights also improved the accuracy. By reducing the noise in the data, these two approaches allowed to benefit from higher-order (cubic) models. In combination, these procedures helped to reduce the error in the optimization from about 9% to about 3%. The noise-filtering effect of the RS was estimated to reduce the error to about 2%. Response surface based wing structural weight equation allowed the study of the effects of design parameter uncertainty. The parameter uncertainty analysis showed that two wing planforms of similar weight may exhibit substantially different sensitivities to uncertainty. Such information is useful for choosing between the designs. An error measure was also applied to demonstrate the uncertainty due to RS approximation itself. It improved the accuracy of uncertainty quantification.

PAGE 148

134 CHAPTER 6 CRACKED COMPOSITE PANEL WEIGHT EQUATION In design of aircraft structures, damage tolerance and fail-safe characteristics are important issues. It is vital that structures can function under damage conditions such as an induced crack. Recent damage-tolerant design approaches model damage scenarios and failure modes directly in the design process, in contrast to the traditional approach of using safety factors. Nees and Canfield (1998) implemented fatigue constraints in the design of wing panels with the automated structural optimization system (ASTROS). The fatigue constraint required panels to sustain a specified crack-length that is under the detectable size for the inspection techniques used. They employed a closed-form approximation to the stress intensity factor. Wiggenraad et al. (2000) implemented an impact damage criterion in design optimization code PANOPT to obtain structural designs of stiffened composite panels with improved impact damage resistance. Lowvelocity impact of a given target energy level is simulated by static transverse load and linear analysis of an undamaged finite element model. The critical peak impact force at which damage is initiated was calculated by a closed-form formula that employs a specific material constant and laminate thickness. Thomsen et al. (1994) studied the design of laminates subjected to transverse shear loading to improve resistance to transverse cracks. They used a mode II stress intensity factor calculated from an integral equation as the objective function to minimize at the crack tips by changing the lay-up

PAGE 149

135 parameters. Thomsen et al. (1994), Nees and Canfield (1998) and Wiggenraad et al. (2000) employed failure criteria that did not require detailed finite element modeling of the damage. For some damage scenarios, however, such detailed modeling may be required. An important damage scenario is through-the-thickness crack caused by engine blade penetration. A design to withstand such cracks until the aircraft lands safely requires a crack propagation model that includes detailed finite element modeling near the crack. However, modeling crack propagation in complex structures entails high computational cost. In addition, crack propagation constraints are not a standard capability in structural optimization software. Vitali et al. (1998 and 1999) combined high-fidelity and low-fidelity analyses for alleviating the computational burden of including crack propagation constraints. For the high-fidelity analysis, a detailed finite element model including the crack was created using the STAGS program (STructural Analyis of General Shells, Rankin et al., 1998), and fracture mechanics theory (Anderson, 1995) was applied by post-processing stress distribution around the crack tip. The low-fidelity model made use of a much coarser finite element mesh, not including the crack, and a closed-form solution for a cracked plate of infinite width. Vitali et al. (1998) performed both the high-fidelity and the low-fidelity analyses for a number of panels. Then a response surface (RS) approximation was constructed for the ratio of the stress intensity factors obtained from the models as a function of stiffener spacing and thickness. This correction response surface (CRS) was used to predict stress intensity factors by multiplying it with an RS constructed by fitting a large number of low-fidelity results. The use of the response surface also allowed easy interface of the crack

PAGE 150

136 propagation constraint in the optimization. Another approach for addressing the computational cost and absence of crack propagation constraints in structural optimization program is the use of equivalent stress and strain constraints. The crack propagation constraint is converted into a stress or strain constraint. Vitali et al. (1999) studied the use of an equivalent strain constraint for designing a composite stiffened panel with a crack. A low-fidelity finite element model that did not include the crack was used with Genesis (VMA, 1997). The higher-fidelity model including the crack used the STAGS finite element program. The stiffened panel for minimum weight in Genesis was optimized by an equivalent strain constraint determined by the high-fidelity STAGS analysis results for the crack tip location. Papila et al. (2001b ) investigated the convergence characteristics of the approach. The equivalent strain approach allowed the optimization to be performed without the use of response surfaces. However, the optimization had to be iterated several times, because replacing the crack propagation constraint with a strain constraint is only an approximation. Equivalent strain approaches as well as RS approaches are both indirect implementations of the crack propagation constraint in design optimization. The main objective of this chapter is to demonstrate direct implementation of a crack propagation constraint within a structural optimization software such as Genesis (VMA, 1997) that allows more complex constraint definition and evaluation than only stress or strain constraints. The implementation is performed with the high-fidelity approach including detailed modeling of the crack as well as a low-fidelity model of a closed form formula from linear elastic fracture mechanics. This direct application allows us to find the optimum in a single optimization run for each panel configuration

PAGE 151

137 unlike the iterative procedure needed with an equivalent strain constraint approach (Vitali et al., 1999). Wing and fuselage structures are composed of a large number of curved or flatusually stiffened-panels. Simultaneous design optimization of all the panels-in other words optimization of the complete structure-requires detailed structural modeling and appears to be beyond the present computational capabilities. The current practice is to design wing or fuselage structures at global and panel levels (e.g. Ragon et al., 1997; Liu et al., 2000; Nees and Canfield, 1998). Response surface approximations offer an attractive means for integrating the design of a single panel with the overall structure. Ragon et al. (1997) used RS approximation to construct weight equations for optimum panel weight as a function of loading and required stiffness. Liu et al. (2000) employed RS approximation for the optimal panel buckling load. The RS filters out numerical noise due to incompletely converged optimizations, but adds modeling (fitting) errors. Response surface techniques also allow detection and correction of errors that may be caused by the fidelity level afforded in the numerical simulations (as discussed in Chapter 5). The second objective of the chapter is to demonstrate the use of RS techniques to detect where the low-fidelity results for panel design optimization subject to a crack propagation constraint may be misleading and to construct a structural weight equation that may be used in upper-level optimization as a substitute for panel-level optimization. Stress Intensity Factor Calculation Stress fracture criteria for plates with notched cracks can be obtained by employing the stress intensity factor (SIF), generally denoted by K which is a measure of

PAGE 152

138 tendency to crack propagation. The average K through the thickness of the plate is computed by two different approaches. The low-fidelity approach makes use of a closed form formula for an axially loaded plate of infinite width and given in Eq. (6.1) a Kf* (6.1) where f is the far-field applied stress and a is half the crack length. In the high-fidelity approach, the stress intensity factor is calculated from the stress distribution around the crack. The normal stress component in the load direction y, near the tip of a straight crack on an infinitely wide panel, can be approximated by Eq. (6.2) r Ky* 2 (6.2) where r is the distance from the crack tip. Equation (6.2) is then used to fit the best K to the stress values. Whitney and Nuismer (1974) reported that Eq. (6.2) is generally accurate when 1 0 a r unless the crack size is too small. When K exceeds a critical value, KQ, called the fracture toughness, the crack grows. For isotropic materials, KQ is a material property determined by mechanical tests and can be used as a limit on K For laminated composites, however, KQ depends on both material and lay-up (Vaidya and Sun, 1996 and 1997). In other words, KQ may be different for each lay-up under consideration even if the same composite material is used. Vaidya and Sun’s experimental results, for eight different lay-ups including quasiisotropic and cross-ply, showed a coefficient of variation (ratio of the standard variation to mean value) of KQ of about 37%. The ratio of maximum to minimum KQ was 2.6. On the other hand, Harris and Morris (1985) reported that KQ is relatively independent of

PAGE 153

139 ply-stacking sequence and laminate thickness. They tested 27 different lay-ups including quasi-isotropic, cross-ply, and angle ply lay-ups for various total numbers of plies. Their coefficient of variation was about 11%, and the ratio of the maximum and minimum was about 1.4. To avoid the need to test each laminate, Vaidya and Sun (1996 and 1997) proposed a method for predicting crack propagation in a composite laminate. They defined the stress intensity factor in the 0+ ply, 0 K as given in Eq. (6.3) K K 0 (6.3) where is the ratio of the far-field stress in the 0+ ply, 0f, to f. Note that is not only a laminate property, but also depends on the loading. It is defined as a stress ratio under uniaxial load. Vaidya and Sun (1996 and 1997) showed that the fracture toughness of the 0+ ply, 0QK is approximately constant at failure and may be viewed as a material property. The crack is predicted to propagate when 0 K exceeds 0QK This approach is approximate since it does not take into account any stress redistribution caused by local damage in the form of matrix cracks (Vaidya and Sun, 1996). It also cannot be applied to laminates without 0+ plies. However, this criterion allows to demonstrate the implementation of a crack propagation constraint in a commercial structural optimization program. Based on the crack sizes of interest, highfidelity analyses for cracked plates without the stiffener were performed first to determine a reasonable range of r The stress intensity factor K was estimated by fitting the stress values near the crack tip and selecting the range so that 2 R [see Eq. (3.34) ] was larger than 0.99. This led to the selection of the interval 125 0 / 0 a r, used for all configurations in calculating K A typical fitting result is shown in Figure 48

PAGE 154

140 Figure 48: Fit over stress values to find K Although Genesis does not have a built-in capability for modeling the stress intensity factor, the implementation is possible through its equation utility. Equation (6.1) for the low-fidelity model was transcribed into the Genesis input file in the format needed by the equation utility and could be called within the optimization run. The farfield stress, f, is obtained from the load in a far-field element of the model divided by the total thickness. The crack size is supplied as constant input based on the panel configuration. The low-fidelity direct implementation is actually equivalent to optimization with a stress limit on 0f that is equal to a KQ*/0. In the high-fidelity approach, implementation is more complex since the leastsquare-fitting procedure for the stress distribution near the crack tip needs to be performed in Genesis. Figure 49 shows nq quadrilateral elements and the stress distribution in the vicinity of the crack tip within the range of r x y01234 0 25000 50000 75000 100000 data fit r x*2 / 1 yy 992 0 257132 R x y

PAGE 155

141 Figure 49: Stress distribution in the vicinity of the crack tip Equation (6.2) is rewritten as i iKx y (6.4) where i ir x*2 1 and yi iy is element stress at ri. Equation (6.4) in a matrix form / / / / / 0 1 2 2 2 2 2 3 4/ / / / / 0 1 2 2 2 2 2 3 4q qn nx x x K y y y" "2 1 2 1 (6.5) The least-square-fit solution for K is then obtained by pre-multiplying Eq. (6.5) by the vector of xis and solving to obtain K as Eq. (6.6) that was also implemented in the Genesis equation utility format. ! # $ $ % & ! # $ $ % &' q qn i i i n i iy x x K1 1 1 2 (6.6) Once K is estimated, K0 can also be calculated in Genesis by Eq. (6.3) and compared with 0 QK during the optimization. As mentioned earlier, Vaidya and Sun a r 1 2 3 nq elements y 1 y 2 y 3 y b yn a r 125 .

PAGE 156

142 (1996) defined as a stress ratio under uniaxial in-plane loading, Ny, in the far field. Therefore, was calculated in Genesis via its equation utility from the laminate constitutive equations using the fact that xN and xyN are zero in the far field. The inplane load yN applied to a laminate causes mid-plane strains m xy m y m x% , as given in Eq. (6.7) ( ( ( ( ) ) ) ) ( ( ( ( * * + + + + ( ( ( ( , , ( ( ( ( . . / / / / / / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 4 / / / / / / / / / / / / 0 0 0 0 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 4 4 4 40 066 26 16 26 22 12 16 12 11y m xy m y m xN a a a a a a a a a% (6.7) where aijs are elements of inverse of the laminate membrane stiffness matrix. The farfield average stress in the y -direction, f over total laminate thickness tott and far-field stress in 0 ply, 0 f, are given in Eq. (6.8) m y m x f tot y fQ Q t N 22 12 0 (6.8) where Qijs are elements of the material reduced stiffness matrix. Upon substitution of Eq. (6.7) in Eq. (6.8), the stress ratio ) (0 f f can be written as ) (22 22 12 12 22 22 12 12a Q a Q t t N N a Q N a Qtot tot y y y (6.9) Panel Problem Definition Twenty-inch square composite blade-stiffened panels loaded in tension, schematically shown in Figure 50 are optimized for minimum structural weight by changing ply thicknesses such that the crack stands as it is without propagating.

PAGE 157

143 For a given panel configuration, the stacking sequence of the plies is (45/45/90/0)s for both skin and blade-stiffener of the panel. The six design variables of the optimization problem are the thicknesses of the 45 90 and 0 skin panel plies (skint45, skint90, and skint0, respectively) and the thicknesses of the 45 90 and 0 blade stiffener plies (bladet45, bladet90, and bladet0, respectively). The -45 ply thicknesses for skin and blade stiffener are set to be equal to the 45 ply thickness of the respective panel element. h 20" 20" 2 a 5" x y zP yN 0! Figure 50: Composite Blade-Stiffened Panel with a Crack With the total laminate thickness of skin panel, Eq. (6.9) becomes ) )( 2 ( 222 22 12 12 0 90 45a Q a Q t t tskin skin skin (6.10) The variables are constrained to vary between 0.005 and 0.025 inches. Therefore the optimization problem is equivalent to 025 0 , 005 0 025 0 , 005 0 1 ) 2 ( 2 ) 2 ( 200 90 45 0 90 45 0 0 0 90 45 0 90 45 blade blade blade skin skin skin Q blade blade blade skin skin skint t t t t t K K that such t t t h t t t minimize (6.11)

PAGE 158

144 The loaded edges and the unloaded edges are simply supported and free, respectively. For the example here, the material for the panels is AS4/3501-6 (material properties are given in Table 23). Table 23: Material Properties for AS4/3501-6 Longitudinal Modulus, E11 (psi) 20 x 106 Transverse modulus, E22 (psi) 1.4 x 106 Shear Modulus, G12 (psi) 0.76 x 106 Poisson’s ratio 0.30 Density, (lb/in3) 0.057 Fracture toughness, 0 QK( in psi) 100,000 Due to the symmetry of the problem, only one quarter of the panel is modeled for the finite element analysis and optimization program Genesis. Two different models were created: the low-fidelity model uses a uniform mesh (520 quadrilateral elements, 3,238 degrees of freedom); the high-fidelity model uses a mesh refined around the crack (5,882 quadrilateral elements and 1,179 triangular elements, 39,222 degrees of freedom) and free-edge conditions along the crack. The loading P yN is uniformly distributed and is simulated by the point force, Py, on one quarter model as given in Eq. (6.12), together with the constraint of uniform displacement in y -direction at the loaded ends P y yN P 10 (6.12) Three configuration parameters considered here are: crack length 2 a blade height h and in-plane load P yN The optimal weight is fitted as a response surface (weight equation) in the three configuration parameters. For the response surface construction, a

PAGE 159

145 three-level full factorial design yielding 27 (33) configurations as presented in Table 24 is used after lower, center and upper levels for variables is scaled (Khuri and Cornell, 1996) to the range (–1, +1). The 27 optimal weights were used to estimate the ten coefficients of a quadratic polynomial in 2 a h and P yN Since the coefficients of the coded variables are used magnitude of the coefficients give their maximum effects on the weight. Table 24: Configurations for three-level full factorial experimental design No P yN h a / / 2 (in) / (in) / (lb/in) No P yN h a / / 2 (in) / (in) / (lb/in) No P yN h a / / 2 (in) / (in) / (lb/in) 1 3.0 / 2.5 / 2000 10 3.5 / 2.5 / 2000 19 4.0 / 2.5 / 2000 2 3.0 / 2.5 / 2500 11 3.5 / 2.5 / 2500 20 4.0 / 2.5 / 2500 3 3.0 / 2.5 / 3000 12 3.5 / 2.5 / 3000 21 4.0 / 2.5 / 3000 4 3.0 / 3.0 / 2000 13 3.5 / 3.0 / 2000 22 4.0 / 3.0 / 2000 5 3.0 / 3.0 / 2500 14 3.5 / 3.0 / 2500 23 4.0 / 3.0 / 2500 6 3.0 / 3.0 / 3000 15 3.5 / 3.0 / 3000 24 4.0 / 3.0 / 3000 7 3.0 / 3.5 / 2000 16 3.5 / 3.5 / 2000 25 4.0 / 3.5 / 2000 8 3.0 / 3.5 / 2500 17 3.5 / 3.5 / 2500 26 4.0 / 3.5 / 2500 9 3.0 / 3.5 / 3000 18 3.5 / 3.5 / 3000 27 4.0 / 3.5 / 3000 Optimization and Response Surface Comparisons Implementation of both low-fidelity and high-fidelity approaches allowed to find optimum weight design for a given panel configuration with a single optimization run. Fitting the RS to the stress intensity factor as a function of ply thickness requires construction of one RS approximation for each configuration. The direct implementation here allowed to obtain optimum designs without approximating the stress intensity factor as a function of configuration and/or design variables. The optimal weight data is then used to obtain a weight equation valid for a range of configurations.

PAGE 160

146 Low-Fidelity Results The optimization procedure was applied for the 27 configurations. It appears that 0 plies provide the optimum protection against the crack propagation under axial loading. Consequently, for the majority of the configurations, the minimum weight structure was obtained by driving skint45, skint90, bladet45, and bladet90 down to the minimum gauge value, bladet0 up to the upper limit, and skint0 to an intermediate value. For configuration 12, 21, 24 and 27 in the low-fidelity optimization, the 0 plies in the skin also reached their upper limits. The JMP (SAS, 1998) statistical software was used for construction of the weight equation as a quadratic RS approximation. Figure 51 compares the predictions of a weight equation fitted to low-fidelity results with the ideal case of identical results by approximation and optimization. Figure 51: Comparison of prediction by RS approximation and optimization data based on low-fidelity optimization for stiffened panel with crack Genesis(lb) RS(lb)1.61.822.22.42.62.83 1.6 1.8 2 2.2 2.4 2.6 2.8 3 ideal LFdata

PAGE 161

147 Table 25 summarizes the low-fidelity optimum weight RS approximation as a function of scaled/coded configuration variables. From the t -statistics and probability values ( p-values ) in Table 25, the intercept and the coefficients of the first-order terms in the crack size, 2 a and in-plane design load, P yN are significant in the approximation model. On the other hand, there is no strong statistical evidence for rejecting the null hypothesis H0: 0 i for the firstand second-order terms in the blade height, h That is, the blade height seemed not affecting significantly the optimal weight of the panel. Table 25: Summary of RS approximation (weight equation) based on results of lowfidelity optimization 2 R 2 aR s (lb) Average Weight W(lb) % W s/ 0.9562 0.9330 0.080445 2.095814 3.84 Terms Coefficient vector, b Estimated std. error of coefficients, is t -statistic p-value intercept 2.032828 0.040961 49.63 <.0001 2a 0.144122 0.018961 7.60 <.0001 h -0.00677 0.018961 -0.36 0.7255 Ny 0.320156 0.018961 16.88 <.0001 2) 2 (a 0.035169 0.032842 1.07 0.2992 2ah -0.04488 0.023223 -1.93 0.0701 2h 0.006078 0.032842 0.19 0.8554 P yaN 2 0.091752 0.023223 3.95 0.001 P yhN -0.05221 0.023223 -2.25 0.0381 2) (P yN 0.053233 0.032842 1.62 0.1234 High-Fidelity Results Minimum weights for the 27 configurations were found through optimization in Genesis based on high-fidelity models. A quadratic weight equation was fit to the

PAGE 162

148 optimal weight results. Table 26 summarizes the approximation, and Figure 52 compares the approximation and actual data. Table 26: Summary for quadratic RS approximation (weight equation) based on high-fidelity results 2 R 2 aR s (lb) Average Weight W(lb) % W s/ 0.9999 0.9999 0.001877 1.978859 0.09 Terms Coefficient vector, b Estimated std. error of coefficients, is t -statistic p-value intercept 1.988308 0.000956 2080.8 <.0001 2a 0.055772 0.000442 126.09 <.0001 h 0.021247 0.000442 48.03 <.0001 Ny 0.252900 0.000442 571.74 <.0001 2) 2 (a -0.01243 0.000766 -16.22 <.0001 2ah -0.00339 0.000542 -6.26 <.0001 h2 0.000606 0.000766 0.79 0.44 P yaN 2 0.014229 0.000542 26.27 <.0001 P yhN 0.00088 0.000542 1.62 0.1227 2) (P yN -0.00235 0.000766 -3.07 0.0069 Figure 52 and the small root-mean-square-error estimator, s in Table 26 show that the use of the high-fidelity model eliminated almost all the noise. With an R2 almost equal to 1.0 it can be concluded that the quadratic polynomial approximates the optimal weight very accurately and introduces no bias error for this problem. Based on t -statistics and the probabilities in Table 26, the most significant terms in the regression model are the first-order terms. Consequently, an RS of the first-order model was also fit to the data, and the results are summarized in Table 27 Although the coefficients of quadratic terms-2) 2 (a, 2 ah, and P yaN 2-are significant, eliminating them from the approximation

PAGE 163

149 did not reduce the accuracy significantly. The second-order coefficients are relatively small compared to the first order terms and the intercept, so it is concluded that the firstorder model also works well for the high-fidelity results. As in the low-fidelity case, the height has the smallest coefficient of the first-order terms, but here it passes the significance test. Figure 52: Comparison of prediction by RS approximation and optimization data based on high-fidelity optimization for stiffened panel with crack Table 27: Summary for first order RS approximation (weight equation) based on high-fidelity results 2 R 2 aR s (lb) Average Weight W(lb)% W s/ 0.997048 0.996663 0.012509 1.978859 0.63 Terms Coefficient vector, b Estimated std. error of coefficients, is t -statistic p-value intercept 1.978859 0.002407 822.03 <.0001 2a 0.055772 0.002948 18.92 <.0001 h 0.021247 0.002948 7.21 <.0001 P yN 0.2529 0.002948 85.78 <.0001 Genesis(lb) RS(lb)1.61.71.81.922.12.22.32.4 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 ideal HFdata

PAGE 164

150 Comparison of Low-Fidelity and High-Fidelity Results For all configurations, the optimal weight by the high-fidelity model is lower than by low-fidelity model. The average ratio of the high-fidelity to low-fidelity optimal weights is 0.95. The optimum design found by the low-fidelity model for Configuration 21 was analyzed with the high-fidelity model, giving a 0 K of 93,018 while the lowfidelity value was 99,999. This indicates that 0 K is overestimated about 7% by the lowfidelity model. Figure 53 shows the ratios of the weights obtained by the two models for the 27 configurations. The ratio decreases with increasing load and crack size. Configurations 12, 21 and 24 that are all loaded at the upper limit of P yN appear to deviate from the trend line. However, this observation requires both lowand highfidelity results. Figure 53: Ratio of high-fidelity and low-fidelity optimal weights An outlier analysis by Iteratively Re-weighted Least Squares analysis using a symmetrical weighting function given in Eq. (3.68) labeled configurations 12, 21 and 24 configurationno. WHF/WLF0369121518212427 0.7 0.75 0.8 0.85 0.9 0.95 1

PAGE 165

151 as outliers on the RS for low-fidelity results (the low-fidelity weight equation). In previous example, HSCT wing problem (Papila and Haftka, 1999 and 2000a ; Kim et al., 2000) outlier detection caught design points where the optimization failed due to algorithmic difficulties or due to local optima. Here the outlier detection caught points where the low-fidelity model breaks a trend. This is useful because these configurations were identified without the use of expensive high-fidelity data. Further investigation of these configurations revealed that the reason is an upper limit on the amount of zero direction plies. The high-fidelity analysis is less conservative than the low-fidelity analysis, and so its optimization does not run into this upper limit. Once the low-fidelity optimization hits the upper limit, it has to use less efficient 45 /-45 and 90 plies instead of 0 plies. This results in substantial increase in weight as well as a slope discontinuity in the low-fidelity weight function. Table 28 shows that the increases in 45 /-45 and 90 plies for configuration 27 are much smaller than the increases for configurations 12, 21 and 24. Consequently, the outlier detection catches only configurations 12, 21, 24 for which the weight penalty is high (errors of low-fidelity result relative to high-fidelity result are about 8%, 18% and 30%, respectively). As a repair procedure, the results of low-fidelity optimization were replaced by the high-fidelity results for these three configurations, and a quadratic RS was constructed for the repaired data. Table 29 presents the summary for the repaired RS. Comparison of Table 25 and Table 29 shows that repair improved the accuracy of the approximation, reducing the root-mean-square-error estimator s by a factor of three. It is seen that the significance of the intercept and the first-order terms increased including blade height, but the quadratic terms remained insignificant based on the t -test.

PAGE 166

152 Table 28: Ply-thicknesses of low-fidelity optima for configurations deviating from the general trend Config. no skint45 (in.) skint90 (in.) skint0 (in.) bladet45 (in.) bladet90 (in.) bladet0 (in.) 12 0.006218 0.005000 0.024980 0.005000 0.005000 0.024980 21 0.008784 0.005192 0.024985 0.016504 0.012048 0.025000 24 0.007749 0.005000 0.025000 0.008117 0.006245 0.024979 27 0.005347 0.005000 0.024992 0.005000 0.005000 0.024979 Table 29: Summary of RS approximation (weight equation) based on results of lowfidelity optimization after repair of the three outliers by high-fidelity results 2 R 2 aR s (lb) Average Weight W(lb)% W s/ 0.989063 0.983272 0.028747 2.047917 1.40 Terms Coefficient vector, b Estimated Std. Error of coefficients, is t -statistic p-value intercept 2.062417 0.014637 140.9 <.0001 2a 0.081692 0.006776 12.06 <.0001 h 0.041388 0.006776 6.11 <.0001 Ny 0.248312 0.006776 36.65 <.0001 2) 2 (a -0.00843 0.011736 -0.72 0.4822 2ah 0.013231 0.008299 1.59 0.1293 h2 0.005296 0.011736 0.45 0.6575 P yaN 2 -0.00189 0.008299 -0.23 0.8222 P yhN 0.020022 0.008299 2.41 0.0274 2) (P yN -0.01861 0.011736 -1.59 0.1312 Figure 54 shows the relative error (compared to high-fidelity results) of predictions by low-fidelity RS approximation before and after the repair. The maximum prediction error among the 27 configurations was reduced from about 22% to 7% by repairing outliers in low-fidelity results with high-fidelity results.

PAGE 167

153 configurationno. relativeerror0369121518212427 0 0.05 0.1 0.15 0.2 0.25 RSlow-fidelity RSlow-fidelityafterrepair Figure 54: Comparison of relative errors wrt high-fidelity results obtained by lowfidelity RS before and after repair An advantage of low-fidelity approaches is the computational cost. The optimization cost per configuration by a low-fidelity model required about 5.6 1.4 CPU seconds while the high-fidelity model required about 93 29 CPU sec on a 700 MHz Pentium III processor with 528MB RAM memory. Discussion Direct implementation of a crack propagation constraint via the equation utility of the Genesis structural optimization software was demonstrated. The constraint was implemented in two levels of fidelity: the low-fidelity approach employed a closed form estimate of the stress intensity factor. The high-fidelity approach modeled the crack and used the stress distribution around the crack tip to calculate the stress intensity factor. The low-fidelity approach is equivalent to a stress constraint and computationally much cheaper than the high-fidelity.

PAGE 168

154 Results for a blade stiffened composite panel with a crack indicate about 7% error, primarily due to more conservative results with the low-fidelity model. An outlier detection procedure labeled some configurations where the low-fidelity model produced designs that were up to 30% heavier than the high-fidelity designs. For these configurations, the accuracy of the low-fidelity model suffered due to the activation of another constraint, an upper limit for the 0 ply thickness, as a result of more conservative stress intensity prediction. Repairing the low-fidelity results at the outlier points by the high-fidelity results reduced the maximum prediction error of the lowfidelity RS approximation from 22% to 7%. This demonstrated the usefulness of outlier detection since it allowed to obtain good accuracy with only 3 out of 27 configurations optimized with expensive high-fidelity models.

PAGE 169

155 CHAPTER 7 CONCLUDING REMARKS The focus of this dissertation was accurate and affordable structural weight estimation. With this focus in mind, response surface (RS) techniques were investigated when the data are generated by finite element analyses based structural optimizations. The techniques were used to deal with two types of difficulties that degrade the weight equation accuracy: high-amplitude numerical noise (outliers) and approximation model errors or model inadequacy. In addition, measures of the remaining errors in the RS weight equation were developed. The first of the two main objectives of the present work was to reduce errors associated with RS constructed on the basis of structural optimizations. This was achieved through outlier detection by IRLS, outlier repair, reasonable design space approach, and application of cubic models. The second objective was to characterize the remaining error and investigate statistical tools to determine the regions of design space where RS accuracy may be poor. This was addressed by the derivation of a point-wise qualitative eigenvalue type error measure based on mean squared error criterion and of a conservative error bound using prediction variance and an approximation for residuals. The results presented in the dissertation suggest the following remarks for the construction and use of RS approximations

PAGE 170

156 High-amplitude noise or outliers can successfully be identified by Iteratively Re-weighted Least Squares (IRLS) procedure. The cause for outliers could then be investigated. A potential reason for outliers in optimization results is premature convergence. Outliers of such a nature may be corrected by employing different convergence settings. The accuracy of RS approximation can substantially be improved by repairing data. This is preferable, when possible, to standard IRLS approximations which merely weight down outliers. Response surface model adequacy can be checked by a lack-of-fit test based on near replicates when replicated designs are not available. If improving the approximation by higher order models is not an option, the eigenvalue error measure derived based on the Mean Squared Error criterion can determine potential high-error design regions. The eigenvalue error measure is qualitative and does not directly supply the magnitude of the error. However, a conservative error measure derived based on prediction variance and an approximation of the residuals may be used to quantify RS uncertainty.

PAGE 171

157 APPENDIX A NOISE IN OPTIMIZATION RESULTS DUE TO ROUND-OFF ERRORS One source of potential errors in numerical analysis/simulation results is roundoff error. Round-off error depends on the computer on which the numerical technique or algorithm is implemented, and thus can be considered external to the numerical algorithm. Eventually, characteristics of the computation platform may affect results particularly from iterative procedures such as optimization. This Appendix provides a comparison between structural optimization results obtained by a commercial program, but run on two different machines. Comparison of Results: PC versus UNIX The GENESIS structural optimization software used in the present dissertation was available both for Windows NT (PC) and DEC Alpha station (UNIX) platforms. Windows NT machine was used for the results reported in this dissertation (Chapter 5 and 6). A set of the results, however, was also obtained on DEC Alpha station in order to see the sensitivity to the computer used. Differences were observed although all convergence settings and parameters were kept the same. The set for comparison consisted of 43 FCCD configurations of the HSCT five-variable wing problem (Chapter 5). Structural optimizations on UNIX machine were performed both using Case 1 and Case 2 convergence settings as defined in second section of Chapter 5 starting on page 98

PAGE 172

158 Convergence Setting: Case 1 Comparison is presented both for objective function of the structural optimization (the total wing structural weight) and wing-bending material weight ( Wb) calculated based on the values of the optimal structural design variables associated with bending resistance. As shown in Figure 41 (page 118 ), for latter objective there is noise mainly due to the process of extracting the Wb from the structural weight that may mislead about the affect of computation platform and its round-off characteristics. Therefore, round-off error sensitivity of the objective function was also investigated. Percentage relative absolute difference between the objective functions from PC and UNIX machines was calculated by Eq. (A.1). PC UNIX PC UNIX PCW W W E / ) ( 100 (A.1) Figure 55 presents the comparison when convergence parameters denoted as “Case 1” are used. Difference up to 17% was observed. The differences at objective function and Wb were exactly zero for 17 and 16 configurations among the 43, respectively (they do not appear in Figure 55). Convergence Setting: Case 2 The comparison was repeated with Case 2 convergence parameters. Figure 56 summarizes the differences of the results in terms of relative error [Eq. (A.1)]. The maximum difference for objective function was around 1.5% whereas it was 5% for the bending material weight among the 43 configurations. The differences at objective function and Wb were exactly zero for five and four configurations, respectively (they do not appear in Figure 56)

PAGE 173

159 Figure 55: Difference in results on PC and UNIX: Convergence setting Case 1 Figure 56: Difference in results on PC and UNIX: Convergence setting Case 2 Discussion The comparison showed that the difference between the results from different machines may be substantial. Optimization runs using Case 1 settings for Configuration 0.0001 0.001 0.01 0.1 1 10 100 051015202530354045 configuration%(PC-UNIX)/PC objective function Wb EPC-UNIX 0.0001 0.001 0.01 0.1 1 10 100 051015202530354045 configuration%(PC-UNIX)/PC objective function Wb EPC-UNIX 0.0001 0.001 0.01 0.1 1 10 100 051015202530354045 design%(PC-UNIX)/PC objective function Wb EPC-UNIX 0.0001 0.001 0.01 0.1 1 10 100 051015202530354045 design%(PC-UNIX)/PC objective function Wb EPC-UNIX

PAGE 174

160 8 (Figure 55) were investigated further since the difference was largest difference among the 43 configurations. The convergence histories for both results can be found in Table 30. It shows that progress towards the optimum is same in the first two design cycles (0 and 1), but with cycle 2 results depart and different optima were reached. The constraint responses (stresses) were compared at cycle 1 and found to be identical, no difference due to round-off. The sensitivities at the end of cycle 1, however, are slightly different, with differences of about 0.02%. These sensitivities are used for finding a better design in cycle 2. It seems that even these slight differences in sensitivities caused the optimizer to move to different design points where progress to optimum ended up different designs satisfying convergence criteria. Table 30: Convergence history for Configuration-8: Convergence setting Case 1 # of Active Constraints Max. Violation Objective Function Design Cycle PC UNIX PC UNIX PC UNIX 0 690 690 91.3%. 91.3%. 79049 79049 1 370 370 0.0%. 0.0%. 103380 103380 2 397 520 1.4%. 35.0%. 100335 79947 3 488 377 32.0%. 0.0%. 80670 95886 4 337 454 0.0%. 11.1%. 106567 84578 5 340 427 0.0%. 0.0%. 106414 88150 In order to evaluate the possibility of getting stuck at different local optima, alternative initial designs were studied. First, the optimum design found on UNIX was set as the initial design for the optimization on PC. The optimization on PC did not move from the initial design to another design. The same applied to the case where the original PC optimum design (heavier than UNIX optimum design) was used in UNIX optimization as initial design. Next all structural variables set at their lower bounds (the

PAGE 175

161 lightest design, but with many violated stress constraints) were used as initial designs. With the lightest initial design PC and UNIX found the same design as optimum. While these results appear to indicate local optima, all the initial designs converged to virtually the same design with ITRMOP=5 (Case 2). The results are summarized in Table 31. Table 31: Objective function with different initial structural designs for Configuration 8 Initial Design ITRMOP =2* (Case 1) ITRMOP=5* (Case 2) No Description PC UNIX PC UNIX 1 Original 106414 88150 83602 83698 2 UNIX optimum (for Case 1) 88150 88150 83612 83602 3 PC optimum (for Case 1) 106414 106414 83696 83696 4 Lightest design 87249 87249 83603 83622 See in section starts on page 98 This shows the rounding-off characteristics may impose different effects on the results based on the choice of convergence parameters. Case 2 setting tries to satisfy convergence criterion on objective function change ITRMOP=5 consecutive times instead of ITRMOP=2 times (Case 1). Each time may be considered as an additional search for a better design nearby within the move limits, it moves and check the change in the objective function. Trying to move more consecutive times increases the chance to move from the current design towards a better design. The optimizations in Case 1 ended with “soft convergence” that is there occurs no more significant changes in design variables. Case 2 resulted with “hard convergence” that is although there are significant changes in design variables objective function does not improve. It seems that in Case 1 it was possible to falsely diagnose a point as being

PAGE 176

162 an optimum. Apparently, with ITRMOP=2, the optimizer often gets stuck, so the design variables cannot be changed, and soft convergence occurs. With ITRMOP=5, optimizer does not get stuck at a non-optimum, and so it stops when design is near enough an optimum and it is not worth getting any nearer because the change in the objective function is too small to bother with. Thus it appears that the round-off errors interact with the tendency of the optimizer to have premature convergence with the default value of ITRMOP. It is also important to note that one of the outliers ( Table 16 ) detected by IRLS analysis (page 58 ) was Configuration 8. The detection of the erroneous result did not actually need additional run on a different platform. This shows the benefit of IRLS procedure. While discussing different results on the two computation platforms, Prof. Layne T. Watson (Departments of Computer Science and Mathematics at Virginia Polytechnic Institute & State University) raised the possibility of using different choices of floating point exception (fpe) during the compilation of the codes. The DEC Alpha station (UNIX) does not use the “-fpe 3” option, but instead “-fpe 0” by default which invokes IEEE standard arithmetic whereas the Pentium chips only use IEEE arithmetic. The developers of GENESIS, VMA Engineering, were contacted. Their choice on the DEC Alpha is not IEEE standard arithmetic. They perform a test for the effect of fpe on Configuration 8. They found that there were no numbers during the calculations misrepresented according to IEEE arithmetic and potentially caused any differences in the results. While the deviation from the IEEE standard is potentially a source of

PAGE 177

163 significant difference between Alpha and other machines, they concluded that this is not having any effect on the observed difference for Configuration 8 The other potential sources indicated by VMA Engineering are, in their words: The Alpha processor does use the IEEE-754 double format (64-bit) for both calculation and storage (notwithstanding the treatment of exceptional values). The Intel processor uses a double extended format (80-bit) for calculation, and only converts to the 64-bit format when results are stored to memory. This is one source of difference in the results between the two. GENESIS relies heavily on tuned versions of the BLAS (Basic Linear Algebra Subprograms) for high performance. The BLAS provides vector and matrix calculations and are tuned for different processors by applying distributive and associative transformations to operations such that the processors' cache and pipeline length are used to best advantage. This creates an "order of operation" round-off sensitivity. Of course, in one pass of calculations, this does not create a 15% difference, but differences in the last one or two significant digits are quite common. In optimization, these slight differences in sensitivities cause the optimizer to select slightly different points for the next design cycle. Once the points are different, then all bets are off. The results depend on the nature of the objective and constraints. If there is one nearby optimum, then the difference in the points does not matter much, and the optimizer should bring both points close to that optimum. However, if there are many local optima nearby, then it is perfectly reasonable that two close but different points can be taken to different optima. These optima can easily be 15% or more different in objective function value.

PAGE 178

164 Ultimately, VMA confirmed that although such a dramatic effect does not occur in a single analysis, these round-off effects could become more noticeable in optimization because slight differences in sensitivities can make the optimizer move to a different point. The choice of convergence parameters, on the other hand, indicated strong influence on the potential affect of rounding-off characteristics of the computation platform.

PAGE 179

165 APPENDIX B FUZZY SET THEORY Fuzzy set theory was introduced by Zadeh (1965) as a mathematical tool for quantitative modeling of uncertainty. In contrast to classical set theory where an element either completely belongs to a specific set or does not belong at all, fuzzy set theory makes use of membership functions to denote the degree to which an element belongs to the set. The membership function, A&, of a fuzzy set A, assigns a grade of membership, ranging between 0 and 1, to each element of the universal set, Z and can be represented as ] 1 0 [ : ) ( Z zA& (B.1) Calculations with fuzzy sets make use of level cuts. An level cut A is defined as the real interval where the membership function is larger than a given value, [Klir and Yuan, p. 19, (1995)], that is & ) ( | z z AA (B.2) Figure 57 a shows Eq. (B.2) for a set A with a triangular membership function. Also shown in Figure 57a are the end points or supports, la and ua, of the level cut. A fuzzy number a fuzzy set that is both normal and convex [Klir and Yuan, p. 97, (1995)]. A normal fuzzy set has a membership function with a maximum of one, while

PAGE 180

166 all possible level cuts are convex for a convex fuzzy set. The fuzzy set A with membership function A& shown in Figure 57a is thus a fuzzy number. In fact, the triangular membership function is most often used to represent fuzzy numbers, mainly due to its simplicity. As shown in Figure 57a, any triangular fuzzy number may be represented by only three variables: N Lz z ,and Uz lower limit, nominal, and upper limit values, respectively. (a) (b) Figure 57: Membership functions. a) Triangular membership function of a fuzzy variable A; b) Membership function of fuzzy function Y A fuzzy function Y of n fuzzy variables Zi and may be written as ) ,..., (2 1nZ Z Z f Y (B.3) for the case where n fuzzy variables are considered. When all the fuzzy variables of a fuzzy function are continuous fuzzy numbers, the fuzzy function itself is also a continuous fuzzy number [Klir and Yuan (1995)], and the level cut of a fuzzy function Y" may be written in terms of the level cuts of the fuzzy variables i "Z as: ) ,..., ( max ) ,..., ( min ) ,..., (2 1 2 1 2 1 n n nZ Z Z Y Z Z Z Y Z Z Z Y YR R (B.4) 1. 1.) ( zA&) (i Yz&ualaLzNzUzUYNYLY lYlY

PAGE 181

167 where R denotes the n-dimensional hypercube formed by the level cuts of the n fuzzy variables. The membership function of a fuzzy function, as schematically represented in Figure 57 b, can be obtained by the vertex method [Dong and Shah (1987)]. The method searches the lower and upper limits of the fuzzy function for an level cut at the 2n vertices of the corresponding hypercube around the point of nominal values and at possible interior global extreme points, if any. Thus the evaluation of a fuzzy function requires the solution the solution of a global optimization problem for each cut.

PAGE 182

168 REFERENCES Alexandrov, N. M. and Lewis, R. M., “Analytical and Computational Aspects of Collaborative Optimization,” NASA/TM-2000-210104 April 2000. Anderson, T. L., Fracture Mechanics: Fundamentals and Applications 2nd edition, Boca Raton CRC Press, 1995. Aviation Week & Space Technology, pp. 71-119, March 16, 1992. Balabanov, V. O., Giunta, A. A., Golovidov, O., Grossman, B., Mason, W.H., Watson, L.T., and Haftka, R.T., “Reasonable Design Space Approach to Response Surface Approximation,” Journal of Aircraft 36(1), pp. 308-315, 1999. Balabanov, V., Haftka, R. T., Grossman, B., Mason, W.H. and Watson, L.T., “Multi-fidelity Response Surface Model for HSCT Wing Bending Material Weight,” Proceedings of 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-98-4804, pp. 778-788, St. Louis MO, Sept. 2-4, 1998. Balabanov, V., Kaufman, M., Knill, D. L., Haim, D., Golovidov, O., Giunta, A. A., Grossman, B., Mason, W. H., Watson, L. T., and Haftka, R. T., “Dependence of Optimal Structural Weight on Aerodynamic Shape for a High-Speed Civil Transport,” AIAA/NASA/USAF/ISSMO Symposium Paper AIAA-96-4046, Bellevue, WA, 1996. Beaton, A. E. and Tukey, J. W., “The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data,” Technometrics 16, pp.147-185, 1974. Braun, R.D., Kroo, I.M., and Moore, A.A.: “Use of the Collaborative Optimization Architecture for Launch Vehicle Design,” 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-96-4018, Bellevue, WA, Sept. 4-6, 1996. Carpenter, W.C. and Barthelemy, J.-F., M, “A comparison of Polynomial Approximations and Artificial Neural Nets as Response Surface,” Structural Optimization 5, pp.166-174, 1993. Dong, W., and Shah, H. C., “Vertex Method for Computing Functions of Fuzzy Variables,” Fuzzy Sets and Systems 24, pp. 65-78, 1987.

PAGE 183

169 Gerard, G. “ Minimum Weight Analysis of Orthotropic Plates Under Compressive Loading,” Journal of Aerospace Science, 27, pp. 21-26, 1960. Gern, F. H, Gundlach, J. F., Ko, A., Naghshineh-Pour, A. H., Suleaman, E., Tetrault, P., Grossman, B., Kapania, R. K., Mason, W. H., Schetz, J. A. and Haftka, R. T., “Multidisciplinary Design Optimization of a Transonic Commercial Transport with a Strut-Braced Wing,” 1999 World Aviation Congress, 99WAC-30, San Francisco, CA, Oct. 19-21, 1999. Gern, F. H, Naghshineh-Pour, A. H., Suleaman, E., Kapania, R. K., and Haftka, R. T., “Flexible Wing Model For Structural Wing Sizing And Multidisciplinary Design Optimization Of A Strut-Braced Wing,” 41th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference Paper AIAA-2000-1327, Atlanta, GA, April, 2000. Giunta, A. A., Balabanov, V., Burgee, S., Grossman, B., Haftka, R. T., Mason, W. H., and Watson, L. T., “Multidisciplinary Optimization of a Supersonic Transport Using Design of Experiments Theory and Response Surface Modeling,” Aeronautical Journal 101(1008), pp. 347-356, 1997. Giunta, A. A., Dudley, J. M., Narducci, R., Grossman, B, Haftka, R. T., Mason, W. H., and Watson, L. T., “Noisy Aerodynamic Response and Smooth Approximations in HSCT Design,” Proceedings, 5th AIAA/NASA/ISSMO Multidisciplinary Analysis and Optimization Symposium Paper AIAA-94-4376, pp. 1117-1128, Panama City, FL, September 1994. Grasmeyer, J M., Naghshineh-Pour, A. H., Tetrault, P. A., Grossman, B., Haftka, R. T., Kapania, R. K., Mason, W. H., and Schetz, J. A., “Multidisciplinary Design Optimization of a Strut-Braced Wing Aircraft with Tip Mounted Engines,” MAD Center Report 98-01-01, Virginia Tech, January 1998. Haftka, R. T. and Grdal, Z., Elements of Structural Optimization Third Revised and Expanded Edition, Boston, Kluwer Academic Publishers, 1992. Harris, C. E. and Morris, D. H., “A Damage Tolerant Design Parameter for Graphite/Epoxy Laminated Composites,” Journal of Composites Technology & Research 7 (3), pp. 77-81, 1985. Hart, J. D., Nonparametric Smoothing and Lack-of-Fit Tests New York, Springer-Verlag, 1997. Holland, P. W. and Welsch, R. E., “Robust Regression Using Iteratively Reweighted Least-Squares,” Communications in Statistics: Theory and Methods 6, pp. 813-827, 1977.

PAGE 184

170 Howe, D., “The Prediction of Aircraft Wing Mass,” Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering 210(2), pp. 135-145, 1996. Huang, X., Dudley, J., Haftka, R. T., Grossman, B. and Mason, W. H., “Structural Weight Estimation for Multidisciplinary Optimization of a High-Speed Civil Transport,” Journal of Aircraft 33(3), pp. 608-616, 1996. Huber, P. J., “Robust Estimation of a Location Parameter,” Annals of Mathematical Statistics 35, pp. 73-101, 1964. Kaufman, M., Balabanov, V., Burgee, S. L., “Variable-Complexity Response Surface Approximations for Wing Structural Weight in HSCT Design,” Computational Mechanics, 18, pp. 112-126, 1996. Kelley, J. Jr., Wing Weight Estimation Army Air Force Technical Report No. 5161, November 8, 1944. Khuri, A. I. and Cornell, J. A., Response Surfaces: Designs and Analyses 2nd edition, New York, Marcel Dekker Inc., 1996. Kim, H., Papila, M., Mason W. H., Haftka, R. T., Watson, L. T., and Grossman, B., “Detection And Correction Of Poorly Converged Optimizations By Iteratively Reweighted Least Squares,” AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, Paper AIAA-00-1525, Atlanta, GA, April 3-6, 2000. Kim, H., Papila, M., Mason W. H., Haftka, R. T., Watson, L. T., and Grossman, B., “Detection and Repair of Poorly Converged Optimization Runs,” AIAA Journal in print, 2001. Klir, G., and Yuan, B., Fuzzy Sets and Fuzzy Logic: Theory and Applications Upper Saddle River, NJ, Prentice-Hall Inc. 1995. Knill, D. L., Giunta, A. A., Baker, C.A., Grossman, B., Mason, W.H., Haftka, R.T., and Watson, L.T., “Response Surface Methods Combining Linear and Euler Aerodynamics for Supersonic Transport Design,” Journal of Aircraft 36(1), pp. 75-86, 1999. Kroo, I., Altus, S., Braun, R., Gage, P.; and Sobieski, I., “Multidisciplinary Optimization Methods for Aircraft Preliminary Design,” Fifth AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-94-4325, Panama City, FL, September 7-9, 1994. Kroo, I., Gallman, J., and Smith, S., “Aerodynamic and Structural Studies of Joined-Wing Aircraft,” Journal of Aircraft 28 (1), pp 74-81, 1991.

PAGE 185

171 Lipp, J. E., “Estimation of Wing Weight,” Journal of Aeronautical Science 6(12), October,1938. Liu, B., Haftka, R. T., and Akgun, M. A., “Two-level composite Wing Structural Optimization Using Response Surfaces,” Structural and Multidisciplinary Optimization 20(2), pp 87-96, 2000. Loretti, A. J., Generation of Response Surfaces for Aircraft Structural Wing Weight with Strength and Flutter Sizing Master Thesis, The School of Engineering and Applied Science of the George Washington University, August 1997. Lowry, R., “Concepts and Applications of Inferential Statistics,” http://departments.vassar.edu/~lowry/webtext.html accessed on 10/27/2001. Mason, B. H., Haftka, R. T. and Johnson E. R., “Analysis and Design of Composite Curved Channel Frames,” Proceedings, 5th AIAA/NASA/ISSMO Multidisciplinary Analysis and Optimization Symposium Paper AIAA-94-4364, pp. 1023-1040, Panama City, FL, September 1994. McCullers, L. A., “Aircraft Configuration Optimization Including Optimized Flight Profiles, ” Proceedings of a Symposium on Recent Experiences in Multidisciplinary Analysis and Optimization J. Sobieski, compiler, NASA CP-2327, pp. 395-412, 1984. McCullers, L. A., Flight Optimization System Release 5.92, NASA Langley Research Center, 1997. McCullers, L. A. and Lynch, R. W., “Composite Wing Design for Aeroelastic Tailoring Requirements,” Air Force Conference on Fibrous Composites in Flight Vehicle Design Dayton, Ohio, September 1972. McCullers, L. A. and Lynch, R. W., “Dynamic Characteristics of Advanced Filamentary Composite Structures,” AFFDL-TR-73-111, II, September 1974. Myers, R. H., Classical and Modern Regression with Applications Boston, PWSKent, 1990. Myers, R. H., and Montgomery, D. C., Response Surface Methodology: Process and Product Optimization Using Designed Experiments New York, John Wiley & Sons, Inc. 1995. Nees, C. D. and Canfield, R. A., “Methodology for Implementing Fracture Mechanics in Global Structural Design of Aircraft,” Journal of Aircraft 35 (1), pp. 131138, 1998.

PAGE 186

172 Neill, J. W. and Johnson, D. E., “Testing Linear Regression Function Adequacy Without Replication,” The Annals of Statistics 13(4), pp. 1482-1489, 1985. Niu, M. C., Airframe Structural Design: Practical Design Information and Data on Aircraft Structures Los Angeles, Technical Book Company, 1988. Owen A., “Lattice Sampling Revisited: Monte Carlo Variance of Means Over Randomized Orthogonal Arrays,” The Annals of Statistics 22, pp. 930-945, 1994. Papila, M. and Haftka, R. T., “Uncertainty and Wing Structural Weight Approximations,” Proceedings, 40th AIAA/ASME/ASCE/ASC Structures, Structural Dynamics, and Material Conference Paper AIAA-99-1312, pp. 988-1002, St. Louis, MO, April 1999. Papila, M. and Haftka, R. T., “Response Surface Approximations: Noise, Error Repair and Modeling Errors,” AIAA Journal 38(12), pp. 2336-2343, 2000a. Papila, M. and Haftka, R. T., “Response Surfaces for Optimal Weight of Cracked Composite Panels: Noise and Accuracy,” 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-2000-4755, September 6-8 2000b. Papila, M. and Haftka, R. T., “Uncertainty and Response Surface Approximations,” 42nd AIAA/ASME/ASCE/ASC Structures, Structural Dynamics, and Material Conference Paper AIAA-01-1680, Seattle, WA, April 2001. Papila, M., Vitali, R., Haftka, R. T. and Sankar, B. V., “Optimal Weight of Cracked Composite Panels by An Equivalent Strain Constraint,” 42nd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference and Exhibit, Paper AIAA-01-1207, Seattle, WA, April 2001. Papila, N., Shyy, W., Fitz-Coy, N. and Haftka, R.T., “Assessment of Neural Net and Polynomial-Based Techniques for Aerodynamic Applications,” 17th Applied Aerodynamics Conference Paper AIAA-99-3167, Norfolk, VA, 1999. Papila, N., Shyy, W., Griffin, L., and Dorney, D.J., “Shape Optimization of Supersonic Turbines Using Response Surface and Neural Network Methods,” AIAA 39th Aerospace Sciences Meeting & Exhibit Paper AIAA-01-1065, 2001. Papila, N., Shyy, W., Griffin, L., Huber,F., and Tran, K., “Preliminary Design Optimization For A Supersonic Turbine For Rocket Propulsion,” 36th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit Paper AIAA-00-3242, Huntsville, AL, 2000. Qu, X., Venkataraman, S., Haftka, R. T., and Johnson, T. F., “Deterministic and Reliability–based Optimization of Composite Laminates for Cryogenic Environments,”

PAGE 187

173 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-00-4760, September 6-8, 2000. Ragon, S. A., Grdal, Z., Haftka, R. T., and Tzong, T. J., “Global/Local Structural Wing Design Using Response Surface Techniques,” AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, Paper AIAA-97-1051, Kissimmee, FL, April 7-10, 1997. Rajashekhar, M. R. and Ellingwood, B. R., “A New Look at the Response Surface Approach for Reliability Analysis,” Structural Safety 12, pp 205-220, 1993. Rankin, C. C., Brogan, F. A., Loden, W. A. and Cabiness, H. D., STAGS User Manual ver. 3.0 Lockheed Martin Missiles & Space Co. Inc. Advanced Technology Center, Palo Alto, CA, 1998 Rohl, P., J., Mavris, D., N., and Schrage, D. P., “A Multilevel Wing Design Procedure Centered on the ASTROS Structural Optimization System,” 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-94-4411, Panama City Beach, FL, September 7-9, 1994. Romero, V. J. and Bankston, S. D., “Efficient Monte Carlo Probability Estimation with Finite Element Response Surfaces Built from Progressive Lattice Sampling,” Proceedings 39th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, pp. 1103-1119, Long Beach, CA, April 20-23, 1998. Roux, W.J., Stander, N., and Haftka, R.T., “Response Surface Approximations for Structural Optimization,” International Journal for Numerical Methods in Engineering 42, pp. 517-534, 1998. SAS Institute Inc., JMP Statistics and Graphics, and User’s Guides Version 3, Cary, NC, January 1998. Schmidt, A., Lapple, M. and Kelm, R., “Advanced Fuselage Weight Estimation for the New Generation of Transport Aircraft,” Weight Engineering, SAWE Journal 1997. Schmit, L. A. and Farshi, B., “Some Approximation Concepts for Structural Synthesis,” AIAA Journal 12(5), pp 692-699, 1974. Schmit, L.A. and Mehrinfar, M. “Multilevel Optimum Design of Structures with Fiber-composite Stiffened Panel Components,” AIAA Journal 20, pp 138-147, 1981. Schueller, G. I., Bucher, C. G., Bourgund, U. and Ouypornprasert, W., “On Efficient Computational Schemes to Calculate Structural Failure Probability,” Probabilistic Engineering Mechanics 4(1), pp 10-18, 1989.

PAGE 188

174 Shanley, F. R., Weight-Strength Analysis of Aircraft Structures Second Edition, New York, Dover publications, Inc. 1960. Shyy, W., Papila, N., Vaidyanathan, R. and Tucker, P.K., “Global Design Optimization for Aerodynamics and Rocket Propulsion Components,” Progress in Aerospace Sciences 37, pp.59-118, 2001. Shyy, W., Tucker, P.K. and Vaidyanathan, R., “Response Surface and Neural Network Techniques for Rocket Engine Injector Optimization,” AIAA/SAE/ASME/ASEE 35th Joint Propulsion Conference Paper AIAA-99-2455, June 20-24 1999. Sobieszczanski-Sobieski, J., and Haftka, R.T., “Multidisciplinary Aerospace Design Optimization: Survey of Recent Developments,” Structural Optimization 14(1), pp. 1-23, 1997. Stark, P. B., “SticiGui: Statistics Tools for Internet and Classroom Instruction with a Graphical User Interface,” http://oz.berkeley.edu/users/stark/SticiGui accessed 10/27/2001. Staton, R. N. editor, Introduction to Aircraft Weight Engineering Los Angeles, CA, SAWE Inc., 1996. Stroud, W. J. and Anderson, M. S., “PASCO: Structural Panel Analysis and Sizing Code, Capability and Analytical Foundations,” NASA TM-80181, 1981. Thareja, R. and Haftka, R. T., “Numerical Difficulties Associated with Using Equality Constraints to Achieve Multi-Level Decomposition in Structural Optimization,” AIAA/ASME/ASCE/AHS 27th Structures, Structural Dynamics and Material Conference Paper AIAA-86-0854CP, San Antonio, Texas, May 1986. Thomsen, N. B., Wang, J. and Karihaloo, B. L., “Optimization – A Tool in Advanced Materials Technology,” Structural Optimization 8, pp. 9 – 15, 1994. Torenbeek, E., Development and Application of a Comprehensive, Designsensitive Weight Prediction Method for Wing Structures of Transport Category Aircraft, Delft University of Technology, Report LR-693, September 1992. Vaidya R. S., Klung J. C. and Sun C. T. “Effect of Ply Thickness and Crack Tip Damage on Failure of Notched Composite Laminates” Proceedings, AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference Paper AIAA-97-1184, Vol.1 pp.2224-2237, Kissimmee, FL, 1997. Vaidya R. S. and Sun C. T. “Fracture Criterion for Notched thin Composite Laminates,” Proceedings AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, Paper AIAA-96-1354, Vol.1 pp.331-338, Salt Lake City, Utah, 1996

PAGE 189

175 Vaidyanathan, R., Papila, N., Shyy, W., Tucker, P.K., Griffin, L. W., Fitz-Coy, N. and Haftka, R.T, “Neural Network-Based and Response Surface-Based Optimization Strategies for Rocket Engine Component Design,” 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Long Beach, CA, 2000 Vanderplaats, G. N., “Structural Design Optimization Status and Direction,” Journal of Aircraft 36(1), pp. 11–20, 1999. Venkataraman, S. and Haftka, R. T., “ Integration of Finite Element Analysis Program and Panel Design Program,” AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, Paper AIAA-97-1051, Kissimmee, FL, April 7-10, 1997. Venkataraman, S., Haftka, R. T. and Johnson, T. F., “Design of Shell Structures for Buckling Using Correction Response Surface Approximations,” Proceedings, 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA 98-4855, St. Louis, Missouri, September 1998. Venter, G., and Haftka R.T., "Minimum-Bias Based Experimental Design for Constructing Response Surfaces in Structural Optimization" Proceedings,the 38th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference Paper AIAA-97-1053, Part 2, pp. 1225-1238, Kissimmee, Florida, April 710, 1997. Venter, G., and Haftka R.T., “Response Surface Approximations for Cost Optimization of Dropped Ply Composite Laminates with Uncertainty”, Proceedings, 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Paper AIAA-98-4856, pp. 1145-1163, St. Louis, Missouri, September 1998a. Venter, G., and Haftka, R. T., “Using Response Surface Methodology in Fuzzy Set Based Design Optimization,” Proceedings, 39th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference Paper AIAA-98-1776, pp. 641-651, Long Beach, CA, April 1998b. Venter, G., Haftka, R. T., and Starnes, J. J. H., “Construction of Response Surface Approximations for Design Optimization,” 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization Bellevue, Washington, 1996. Vitali, R., Haftka, R.T., and Sankar, B.V., “Correction Response Approximation for Stress Intensity Factors for Composite Stiffened Plates,” Proceedings, 39th AIAA/ASME/ASCE/ AHS/ASC Structures, Structural Dynamics and Material Conference, Paper AIAA-98-2047, pp. 2917-2922, Long Beach, CA, 1998. Vitali R., Haftka, R. T. and Sankar, B. V., “Multifidelity Design of Stiffened Composite Panel with a Crack”, World Congress of Structural and Multidisciplinary Optimization Congress Buffalo New York, May 17-21, 1999.

PAGE 190

176 Vitali, R., Park, O., Haftka, R. T. and Sankar, B. V., “Structural Optimization of Hat Stiffened Panel by Response Surface Techniques,” Proceedings, the 38th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference Kissimmee, FL, vol.4, pp. 2983-2993, 1997. VMA Engineering, GENESIS User Manual Version 4.0, Colorado Springs, CO, October 1997. Whitney, J. M. and Nuismer, R. J., “Stress Fracture Criteria for Laminated Composites Containing Stress Concentrations,” Journal of Composite Materials 8, pp. 253–265, 1974. Wiggenraad, J. F. M., Vercammen, R. W. A., Arendsen, P. and Ubels, L. C., “Design Optimization of Stiffened Composite Panels for Damage Resistance,” 41st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference and Exhibit Paper AIAA-00-1374, Atlanta, GA, 2000. Xiao, Q., Sues, R. H. and Rhodes, G. S., “Multi-disciplinary Wing Shape Optimization with Uncertain Parameters,” Proceedings, 40th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference Paper AIAA-99-1601, pp. 3005-3014, St Louis – MO, April 1999. York, P. and Labell, R., “Aircraft Wing Weight Build-up Methodology with Modification for Materials and Construction Techniques,” NASA CR-166173 1980. Zadeh, L. A., “Fuzzy Sets,” Information and Control 8, pp. 29-44, 1965.

PAGE 191

177 BIOGRAPHICAL SKETCH Melih Papila was born in Ankara, Turkey, on September 27, 1968. He received his Bachelor of Science degree in aeronautical engineering from the Middle East Technical University, Turkey, in June 1990. He was awarded first rank in his class and started his graduate studies as a research assistant at the same institution. During his assistantship he also completed 6-month on-the-job-training in the aircraft structural testing department at CASA, the major aircraft manufacturing company in Spain. Mr. Papila was hired by Roketsan, Ankara-Turkey, in July 1993. He worked for 4 years in the Engineering and Development Department of this company as a research engineer responsible for research projects on composite materials. While at Roketsan he also completed his graduate study for the Master of Science degree in aeronautical engineering from the Middle East Technical University in September 1995. His interest in conducting research continued and he decided to pursue a Ph.D. degree in aerospace engineering. Mr. Papila enrolled at the University of Florida in August 1997.


Permanent Link: http://ufdc.ufl.edu/UFE0000339/00001

Material Information

Title: Accuracy of response surface approximations for weight equations based on structural optimization
Physical Description: Mixed Material
Language: English
Creator: Papila, Melih ( Dissertant )
Haftka, Raphael T. ( Thesis advisor )
Khuri, Andre I. ( Reviewer )
Kurdila,Andrew J. ( Reviewer )
Sankar, Bhavani V. ( Reviewer )
Ifju, Peter G. ( Reviewer )
Shyy, Wei ( Reviewer )
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2001
Copyright Date: 2001

Subjects

Subjects / Keywords: Aerospace Engineering, Mechanics and Engineering Science thesis, Ph. D.
Dissertations, Academic -- UF -- Aerospace Engineering, Mechanics and Engineering Science

Notes

Abstract: Accurate weight prediction methods are vitally important for aircraft design optimization. Therefore, designers seek weight prediction techniques with low computational cost and high accuracy, and usually require a compromise between the two. The compromise can be achieved by combining stress analysis and response surface (RS) methodology. While stress analysis provides accurate weight information, RS techniques help to transmit effectively this information to the optimization procedure. The focus of this dissertation is structural weight equations in the form of RS approximations and their accuracy when fitted to results of structural optimizations that are based on finite element analyses. Use of RS methodology filters out the numerical noise in structural optimization results and provides a smooth weight function that can easily be used in gradient-based configuration optimization. In engineering applications RS approximations of low order polynomials are widely used, but the weight may not be modeled well by low-order polynomials, leading to bias errors. In addition, some structural optimization results may have high-amplitude errors (outliers) that may severely affect the accuracy of the weight equation. Statistical techniques associated with RS methodology are sought in order to deal with these two difficulties: (1) high-amplitude numerical noise (outliers) and (2) approximation model inadequacy. The investigation starts with reducing approximation error by identifying and repairing outliers. A potential reason for outliers in optimization results is premature convergence, and outliers of such nature may be corrected by employing different convergence settings. It is demonstrated that outlier repair can lead to accuracy improvements over the more standard approach of removing outliers. The adequacy of approximation is then studied by a modified lack-of-fit approach, and RS errors due to the approximation model are reduced by using higher order polynomials. In addition, remaining error in the RS weight equation is characterized, and two error measures are introduced. One of the measures is qualitative, and it is used to identify regions of design space where RS accuracy may be poor. The second measure is quantitative, but conservative. Its use is demonstrated for incorporating the uncertainty due to RS weight equation with parameter uncertainty in order to compare competing designs for robustness.
Subject: response surface techniques -- outliers -- numerical noise -- modeling errors -- structural weight -- optimization -- uncertainty -- crack propagation constraint
General Note: Title from title page of source document.
General Note: Document formatted into pages; contains xiv, 177 p.; also contains graphics.
General Note: Includes vita.
Thesis: Thesis (Ph. D.)--University of Florida, 2001.
Bibliography: Includes bibliographical references.

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0000339:00001

Permanent Link: http://ufdc.ufl.edu/UFE0000339/00001

Material Information

Title: Accuracy of response surface approximations for weight equations based on structural optimization
Physical Description: Mixed Material
Language: English
Creator: Papila, Melih ( Dissertant )
Haftka, Raphael T. ( Thesis advisor )
Khuri, Andre I. ( Reviewer )
Kurdila,Andrew J. ( Reviewer )
Sankar, Bhavani V. ( Reviewer )
Ifju, Peter G. ( Reviewer )
Shyy, Wei ( Reviewer )
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2001
Copyright Date: 2001

Subjects

Subjects / Keywords: Aerospace Engineering, Mechanics and Engineering Science thesis, Ph. D.
Dissertations, Academic -- UF -- Aerospace Engineering, Mechanics and Engineering Science

Notes

Abstract: Accurate weight prediction methods are vitally important for aircraft design optimization. Therefore, designers seek weight prediction techniques with low computational cost and high accuracy, and usually require a compromise between the two. The compromise can be achieved by combining stress analysis and response surface (RS) methodology. While stress analysis provides accurate weight information, RS techniques help to transmit effectively this information to the optimization procedure. The focus of this dissertation is structural weight equations in the form of RS approximations and their accuracy when fitted to results of structural optimizations that are based on finite element analyses. Use of RS methodology filters out the numerical noise in structural optimization results and provides a smooth weight function that can easily be used in gradient-based configuration optimization. In engineering applications RS approximations of low order polynomials are widely used, but the weight may not be modeled well by low-order polynomials, leading to bias errors. In addition, some structural optimization results may have high-amplitude errors (outliers) that may severely affect the accuracy of the weight equation. Statistical techniques associated with RS methodology are sought in order to deal with these two difficulties: (1) high-amplitude numerical noise (outliers) and (2) approximation model inadequacy. The investigation starts with reducing approximation error by identifying and repairing outliers. A potential reason for outliers in optimization results is premature convergence, and outliers of such nature may be corrected by employing different convergence settings. It is demonstrated that outlier repair can lead to accuracy improvements over the more standard approach of removing outliers. The adequacy of approximation is then studied by a modified lack-of-fit approach, and RS errors due to the approximation model are reduced by using higher order polynomials. In addition, remaining error in the RS weight equation is characterized, and two error measures are introduced. One of the measures is qualitative, and it is used to identify regions of design space where RS accuracy may be poor. The second measure is quantitative, but conservative. Its use is demonstrated for incorporating the uncertainty due to RS weight equation with parameter uncertainty in order to compare competing designs for robustness.
Subject: response surface techniques -- outliers -- numerical noise -- modeling errors -- structural weight -- optimization -- uncertainty -- crack propagation constraint
General Note: Title from title page of source document.
General Note: Document formatted into pages; contains xiv, 177 p.; also contains graphics.
General Note: Includes vita.
Thesis: Thesis (Ph. D.)--University of Florida, 2001.
Bibliography: Includes bibliographical references.

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0000339:00001


This item has the following downloads:


Full Text











ACCURACY OF RESPONSE SURFACE APPROXIMATIONS
FOR WEIGHT EQUATIONS BASED ON
STRUCTURAL OPTIMIZATION















By

MELIH PAPILA


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2001






































Aileme...















ACKNOWLEDGEMENTS


I would like to express my special thanks and appreciation to Dr. Raphael T.

Haftka, chairman of my advisory committee. He has been a continuous and excellent

source of guidance for my Ph.D. research. I have always been amazed and motivated by

his never-ending enthusiasm to explore and willingness to share his knowledge and

experience. Working and interacting with him have broadened my vision not only

professionally, but also personally.

I also would like to thank the members of my advisory committee, Dr. Andre I.

Khuri, Dr. Andrew J. Kurdila, Dr. Bhavani V. Sankar, Dr. Peter G. Ifju and Dr. Wei

Shyy. I am grateful for their willingness to review my Ph.D. research and to provide me

their constructive comments that helped me to complete this dissertation.

I must express my gratitude to my former advisor, Dr. Mehmet A. Akgun, who

made me realize how much I like research. I have always enjoyed interacting with him in

my professional and personal life.

I also thank the colleagues with whom I interacted during this study. In

particular, I thank Dr. Roberto Vitali, Dr. Satchi Venkataraman, Dr. Raluca Rosca and

Dr. Ibrahim Ebcioglu at the University of Florida, and Dr. Hongman Kim at Virginia

Tech.

The financial support provided by NASA grants NAG1-2000 and NAG1-2177 is

gratefully acknowledged.










I thank my dear friends from home; Murat (MK), Canan, Selami (SYA), Senol,

Tarkan, Sartuk, Bulent, Yusuf and Merve who kept constant contact with me.

I am grateful for the friendship of Kamal and Alpay who always inspired me with

their positive attitudes. I am also grateful to have my best friend Guray and Elif in my

life. They deserve special thanks for their constant support and true friendship.

The best thing happened to me during my study was getting to know wonderful

people, Farshad, Sibel, and their family, and having the chance to make them an

important part of my life.

I thank my parents, Gonul and Mithat Papila; Meric, Ozer, Cagil and Irmak

Torgal; Hurriyet, Ibrahim and Eray Uzgoren; Emine, Bulent and Yigit Topoglu: without

the love and support of my family I could not have finished this dissertation.

Finally, my deepest appreciation goes to my beautiful and loving wife, Nilay. Her

love and presence have been and will be a tremendous inspiration for every challenge in

my life. Nilay has given me her support in every respect: emotionally as my better half,

professionally as my colleague. I only hope that I have been able to give her the same

support in return.
















TABLE OF CONTENTS
page
A C K N O W LE D G E M EN T S ......... ................................................................................. iii

LIST OF TABLES ................... ............................ .. vii

LIST OF FIGU RES ......... .... .. ...... ........ .... .... ................... ..... ix

ABSTRACT ............................................. xiii

CHAPTERS

1. IN T R O D U C T IO N ........... ................................................. .... ........... ........

F ocus ........................................................ .............. 1
A p p ro ach ........................ ........................................................................ . 7
Scope and O objectives .................. .................... ............... 9

2. LITERA TURE SU RVEY ........................................................................ 12

W ing (System level) Structural W eight Equations ................................. ................. 13
Panel (Component Level) Weight Equations in Multi-Level Procedures ................. 18
A accuracy of R response Surfaces ......................................... ................ .............. 23
Use of RS in Uncertainty Quantification .............. ................ ....... .............. 30

3. RESPONSE SURFACE METHODOLOGY........................ ..................... 32

D design of E xperim ents ..........3......... .. ............ .... .................... ....... ............. 33
Fitting Procedure: Standard Least Squares .......................................................... 36
Adequacy and Accuracy of RS ....................................................... .......... .... 39
Hypothesis Testing in RS M ethodology .............. ............................. ....... ....... 43
Standard L ack-of-Fit T est .................................................................. ... 48
Lack-of-Fit with Non-Replicated Design Points....................................... 52
Iteratively Re-weighted Least Squares.............. ................ ...... .............. 58

4. BIAS AND NOISE ERRORS IN RESPONSE SURFACE APPROXIMATIONS ... 63

M ean Squared E rror C riterion ............................... .. ... .. .............................. 63
Estimated Standard Prediction Error and Residual Based Error Bound ................... 69
Polynomial Examples.......................................... ............ ........ 71
Coefficient of Correlation ............................................................................. 72
Cubic Polynomial in Two Variables without Noise ................ ............... 74










Cubic Polynomial in Two Variables with Noise.............................................. 81
Cubic Polynomial in Five Variables without Noise........................................... 87
Cubic Polynomial in Five Variables with Noise............................................. 89
Effect of Experimental Design: Eigenvalues with a Minimum Bias Design........ 91
Discussion ............................................ ............... 94

5. HIGH SPEED CIVIL TRANSPORT (HSCT) WING WEIGHT EQUATIONS ....... 95

H SC T D design D definition .................................................................... .............. 95
Structural Optimization and Finite Element M odel........................................ ...... 98
Construction of Response Surface Approximations .............................................. 103
Five Configuration Variables ............................... .............. 103
Numerical noise and outlier analysis .................................................... 104
Lack-of-fit ................................................... 108
Error analysis based on M SEP eigenvalues .......... ................................ 111
Cubic fitting model in the approximation ............. ..... ................. 113
Data vs. RS ....... ........ ......... ................ 117
Repair vs. IRLS .................................. ..................... 118
Ten Configuration V ariables................................................ 121
Design for Uncertainty ....................... .. .. .................... 123
Parameter Uncertainty: Material Properties and Geometry ............................ 125
R S U uncertainty .................. ............................. ................ .. 130
Discussion ............................................ 132

6. CRACKED COMPOSITE PANEL WEIGHT EQUATION................................. 134

Stress Intensity Factor C alculation .................................................. .................... 137
Panel Problem D definition ............... .................. ....................................... 142
Optimization and Response Surface Comparisons .............................................. 145
Low -Fidelity R results ..................................... ...... .. ....... .. .......... .. 146
H igh-Fidelity R results ................... .................. ...... ................ .............. .. 147
Comparison of Low-Fidelity and High-Fidelity Results................................... 150
Discussion ............................................ 153

7. CON CLUD ING REM ARK S .............. ........................................................... 155

APPENDICES

A NOISE IN OPTIMIZATION RESULTS DUE TO ROUND-OFF ERRORS.......... 157

B FU Z Z Y SE T TH E O R Y ................................................................... .................... 165

R E FER EN CE S .......... .............. ..... .............. ................ 168

BIOGRAPHICAL SKETCH .................................... ............................ 177















LIST OF TABLES


Table page

1. Comparison of actual and predicted weights from Eq. (1.1) ................................ 3

2. Experimental designs used in the present work ................................ .............. 36

3. Verification of the lack-of-fit test with non-replicated design points .................. 57

4. Standard weighting functions for IRLS ................................... .............. 60

5. Coefficient of correlation between the absolute residual eTj and the

square-root of maximum eigenvalues m from Eq. (4.27) for two-
variable cubic polynomial example without noise................................... 77

6. Coefficient of correlations between the absolute residual eTj and the

square-root of maximum eigenvalues x from Eq. (4.27) for two-
variable cubic polynomial example with noise.................................................. 82

7. Coefficients of correlation between the estimated standard error and
absolute true residuals e j for cubic polynomial example with noise............. 85

8. Face-centered central composite design (FCCD) and minimum-bias
central composite design (MBCCD) for two-variable polynomial example ........ 92

9. Load cases in H SCT design study................................ ........................ .. ....... 96

10. Configuration design variables for HSCT with corresponding value ranges........ 97

11. Statistics for five-variable full quadratic response surface approximations
for HSCT wing bending material weight......... ........... ........................... 105

12. The near replicate design points................ ...... .... ................................. 108

13. Results of the lack-of-fit test for quadratic RS fitted to 126 design points
of five-variable HSCT wing problem .................................................................. 109

14. Results of the lack-of-fit test for quadratic RS fitted to 115 design points
of five-variable HSCT wing problem ............... ......... ........................ 110










15. Coefficients of correlation r between Gma and s for five-variable
HSCT wing problem at 3146 design points ............................................. 111

16. Statistics for five-variable cubic response surface approximations for wing
bending material weight ........... ...... ......... .... .............. 114

17. Results of the lack-of-fit test for cubic RS fitted to data of five-variable
HSCT wing problem ......... ............................................ .............. 115

18. Average Residual Percentage for IRLS fit and RS after Outlier Repair,
five-variable HSCT problem................................................... 120

19. Statistics for 10-variable response surface approximations for HSCT wing
bending material weight ........... ............. ............... .............. 122

20. RScbc term estimates and statistics, five-variable HSCT problem ................ 124

21. Uncertainty levels and ranges for fuzzy variables in five-variable HSCT
wing problem...................................... .............. 126

22. Uncertainty for blue and red design: Combined effect variable fuzziness
and R S m odeling uncertainty ......... ................................................ .............. 132

23. M material Properties for A S4/3501-6........... ................................. .............. 144

24. Configurations for three-level full factorial experimental design.................... 145

25. Summary of RS approximation (weight equation) based on results of low-
fidelity optim ization ........................................... ......................... ... 147

26. Summary for quadratic RS approximation (weight equation) based on
high-fidelity results ...... ..... ........................................ ................ .. 148

27. Summary for first order RS approximation (weight equation) based on
high-fidelity results ...... ..... ........................................ ................ .. 149

28. Ply-thicknesses of low-fidelity optima for configurations deviating from
the general trend ................................................ ........ .. .......... 152

29. Summary of RS approximation (weight equation) based on results of low-
fidelity optimization after repair of the three outliers by high-fidelity ............. 152

30. Convergence history for Configuration-8: Convergence setting Case 1............. 160

31. Objective function with different initial structural designs for Config. 8.......... 161















LIST OF FIGURES


Figure page

1. Aircraft gross weight variation with wing span .................................. ................ 2

2. Three-level full factorial (left) and face centered central composite (right)
experimental designs for three coded-configuration variables ............................ 35

3. Sample F-distributions ................. .............................................. 45

4. The t-distributions of various degrees of freedom df ................ ............... 48

5. 3-D representation of clustering design points to generate near-replicate
data for lack-of-fit test............... ...................... ................... 56

6. 1-D example of the effect of an outlier in the data .................. .......... .. 58

7. 1-D example of IRLS procedure ....... ........... .. ............. ................. 59

8. W fighting functions for IRLS.................. ......... ........................ .............. 61

9. Linear correlation examples .............. .................................. 73

10. Two-variable three-level factorial design points............................ ......... ... .. 75

11. Square-root of maximum eigenvalue max contours from Eq. (4.27)
for a two-variable cubic polynomial, fitted with a quadratic model................. 75

12. Absolute residual ej contours two-variable cubic polynomial example
(polynomials defined in Table 5), without noise. ............................................ 77

13. Comparison between square-root of eigenvalues max and absolute
residual ed two-variable cubic polynomial example ...................... .............. 78

14. Comparison of square-root of eigenvalues max and maximum
absolute residual e j max contours (maximum of residuals among the four
polynomials defined in Table 5) two-variable cubic polynomial example,
without noise. ............................................ 79









15. Comparison between square-root of eigenvalues max and maximum
of true residual errors e-j max due to the four polynomial in Table 5 two-
variable cubic polynomial example, without noise................... ................. 79

16. Conservative error bound s contours two-variable cubic polynomial
example (polynomials defined in Table 5), without noise.................. ............... 80

17. Comparison between conservative error bound s, and absolute residual

ej two-variable cubic polynomial example (polynomials defined in
Table 5), without noise..................... ........................ .............. ...... .... ..... 81

18. Absolute residual ej contours two-variable cubic polynomial example
(polynomial defined in Table 5 and Table 6), with noise ................................... 82

19. Comparison between square-root of eigenvalues Gmax and absolute
residual ej two-variable cubic polynomial example (polynomials
defined in T able 5), w ith noise.................................................. .. ..... .............. 83

20. Comparison of square-root of eigenvalues max and maximum
absolute residual erj, max contours (maximum of residuals among the four
polynomials defined in Table 6) two-variable cubic polynomial example,
with noise ..... .. ...................................... ............... 84

21. Comparison between square-root of eigenvalues max and maximum
of residual errors erj, max due to the four polynomial in Table 5 two-
variable polynom ial exam ple, with noise.................................. ..................... 84

22. Normalized estimated standard error ee, / a for a two-variable........................ 86

23. Comparison between conservative error bound s, and absolute residual

ej I two-variable cubic polynomial example (polynomials defined in
Table 7), w ith noise..................... ...... ...... .... ........ ......................... ...... .. 86

24. Comparison between square-root of eigenvalues Gmax and absolute
true residuals eT and the trend line manually drawn for largest errors
characterized by the eigenvalues- five-variable cubic polynomial example,
w without noise ........... ..... ...... ......... ..... ....... ..........88










25. Comparison between conservative error bound s, and absolute true
residual eg five-variable cubic polynomial example, without noise............... 89

26. Comparison between square-root of eigenvalues x = J max and absolute
residuals y = e five-variable cubic polynomial example, with noise.............. 90

27. Comparison between conservative error bound s, and absolute residual

e1 -five-variable cubic polynomial example, with noise ..................... 91

28. Eigenvalue-contours ( kmax ). a) Face-centered central composite design
(FCCD); b) minimum-bias central composite design (MBCCD) for two-
variable polynom ial exam ple ................................................................. ....... 93

29. Absolute residual-contours in two-variable polynomial example (for
Polynomial 1 in Table 5). a) Face-centered central composite design
(FCCD); b) Minimum-bias central composite design (MBCCD)....................... 93

30. HSCT wing planform and five- & 10-configuration variables. .......................... 97

31. Flow chart for the RS based wing weight equation ....................................... 99

32. HSCT finite element model and structural design variables............................. 99

33. Close-up view for the typical wing box cell ........................................... .... 100

34. Flow chart of the optimization in GENESIS.......................... 101

35. Effect of overly tight optimization parameters: Difference in optimization
results between Case 1 and Case 3 ............................................. 102

36. Outliers and effect of repair on quadratic response surface, five-variable
H SCT problem .............. ...... .............................. ............ .. 107

37. Comparison between square-root of eigenvalues max and
conservative error bound s, quadratic RS of HSCT 126 design points.......... 112

38. Comparison between square-root of eigenvalues max and
conservative error bound s, quadratic RS of HSCT 115 design points.......... 112

39. Comparison of predictions of square-root of MSEP, s, and s, at 115
design points quadratic RS................................... .............. 113










40. Comparison between square-root of eigenvalues max and absolute
error between reduced cubic and quadratic RS five-variable HSCT wing
problem........................................... ............ 117

41. Noisy wing bending material weight Wb compared to smoother objective
function (total wing structural weight)........................ ...... .............. 118

42. Extreme value design line. ................................... .............. 119

43. Comparison of RS with outlier repair and with outlier elimination over the
design line, five-variable HSCT problem. ............. ........................... ........ 120

44. Description of vertex method for three-variable case............................ ... 127

45. Five-variable HSCT case: Contours in plane with almost constant wing
bending. a) Nominal wing bending material weight contours; b) parameter
uncertainty band width (upper bound minus lower bound) contours; c)
Modeling (RS) uncertainty band width contours........ .................................. 129

46. Comparison of predictions of square-root ofMSEP, s, (=x) and s, (=y) at
152 design points of five-variable HSCT wing problem .............................. 131

47. Blue and Red design planform s .............. ...... ......................................... 131

48. Fit over stress values to find K ......... .. .. ... .. ........................ .... .......... 140

49. Stress distribution in the vicinity of the crack tip............................................. 141

50. Composite Blade-Stiffened Panel with a Crack ....... .... ................................ 143

51. Comparison of prediction by RS approximation and optimization data
based on low-fidelity optimization for stiffened panel with crack ................... 146

52. Comparison of prediction by RS approximation and optimization data
based on high-fidelity optimization for stiffened panel with crack ...... .......... 149

53. Ratio of high-fidelity and low-fidelity optimal weights............................ 150

54. Comparison of relative errors wrt high-fidelity results obtained by low-
fidelity RS before and after repair.............................. ................. 153

55. Difference in results on PC and UNIX: Convergence setting Case 1............... 159

56. Difference in results on PC and UNIX: Convergence setting Case 2................. 159

57. M em bership functions ............................ ............ ...................... .............. 166















Abstract of Dissertation
Presented to the Graduate School of the University of Florida
in Partial Fulfillment of the Requirements for the
Degree of Doctor of Philosophy

ACCURACY OF RESPONSE SURFACE APPROXIMATIONS
FOR WEIGHT EQUATIONS BASED ON
STRUCTURAL OPTIMIZATION

By

Melih Papila

December 2001


Chair: Raphael T. Haftka, Distinguished Professor
Major Department: Aerospace Engineering, Mechanics and Engineering Science


Accurate weight prediction methods are vitally important for aircraft design

optimization. Therefore, designers seek weight prediction techniques with low

computational cost and high accuracy, and usually require a compromise between the

two. The compromise can be achieved by combining stress analysis and response surface

(RS) methodology. While stress analysis provides accurate weight information, RS

techniques help to transmit effectively this information to the optimization procedure.

The focus of this dissertation is structural weight equations in the form of RS

approximations and their accuracy when fitted to results of structural optimizations that

are based on finite element analyses. Use of RS methodology filters out the numerical

noise in structural optimization results and provides a smooth weight function that can

easily be used in gradient-based configuration optimization. In engineering applications










RS approximations of low order polynomials are widely used, but the weight may not be

modeled well by low-order polynomials, leading to bias errors. In addition, some

structural optimization results may have high-amplitude errors (outliers) that may

severely affect the accuracy of the weight equation.

Statistical techniques associated with RS methodology are sought in order to deal

with these two difficulties: (1) high-amplitude numerical noise (outliers) and (2)

approximation model inadequacy.

The investigation starts with reducing approximation error by identifying and

repairing outliers. A potential reason for outliers in optimization results is premature

convergence, and outliers of such nature may be corrected by employing different

convergence settings. It is demonstrated that outlier repair can lead to accuracy

improvements over the more standard approach of removing outliers. The adequacy of

approximation is then studied by a modified lack-of-fit approach, and RS errors due to

the approximation model are reduced by using higher order polynomials. In addition,

remaining error in the RS weight equation is characterized, and two error measures are

introduced. One of the measures is qualitative, and it is used to identify regions of design

space where RS accuracy may be poor. The second measure is quantitative, but

conservative. Its use is demonstrated for incorporating the uncertainty due to RS weight

equation with parameter uncertainty in order to compare competing designs for

robustness.















CHAPTER 1
INTRODUCTION


An aircraft designer would probably place weight at the top of the list of

important concerns in the design of an aircraft. This is because all disciplines in the

design process such as aerodynamics, structure, performance and propulsion interact with

each other through the weight. The trade-off between aerodynamic and structural

efficiency is a common example of such a discipline interaction. Structural weight, a big

proportion of the total weight, affects the lift that must be generated, and consequently

the drag. In turn aerodynamic design aspects affect the loads, and hence structural

design, and the structural weight.


Focus

The focus of this dissertation is accurate and affordable weight estimation. The

impact of the weight starts from the very beginning in aircraft design process. Definition

of conceptual characteristics initiates the process, and it is followed by the preliminary

design phase where prediction of the weight is one of the first steps. The final size and

cost of the aircraft are strongly influenced by the estimated weight at the preliminary

design phase. The more reliable and precise starting data are, the faster is the

convergence to the promising configurations.

Empirical weight equations. It is a common practice in conceptual design to

estimate the weight by empirical equations derived from historical aircraft data. They are










one of the earliest approaches used for weight prediction. Early wing weight prediction

methods, for instance, were of a purely statistical nature and included only simple

relationships between wing structure weight and the most significant design parameters

such as wing span and wing gross area. If only a nominal wing span is available to a

designer, a purely statistical approach is simply to collect and plot weight versus wing

span data for comparable existing aircraft, and then try to make a fit to obtain an

expression for weight as a function of wing span. For example Figure 1 presents

statistical/historical data (source: Aviation Week & Space Technology, March 16, 1992,

pp. 71-119) for the gross weight versus wing span on a logarithmic scale for existing

aircraft. A first order polynomial curve fit on this plot gives an empirical equation for

aircraft gross weight.

7.00
Data
6.00
Linear fit
5.00 y = 2.0186x+ 1.1524

S4.00

0 3.00

o 2.00

1.00

0.00
0.00 0.50 1.00 1.50 2.00 2.50
x=log(Span, ft)
Figure 1: Aircraft gross weight variation with wing span determined by statistical
evaluation of existing aircraft


Empirical equations are still widely used for first order estimates or as

reasonability checks for more detailed analysis. Most airplane manufacturers have

developed their own methods based on such empirical relations (Niu, 1988; Loretti










1997). An example of how to develop an empirical equation can be found in Staton

(1996). Another example of empirical equations presented in Torenbeek (1992) is a wing

weight equation, Ww for subsonic transports and executive jets:


Ww =17bS WZ (1.1)


where b is wing span, S is wing gross area, WMZF is maximum zero fuel weight, and WMTO

is maximum take-off weight. Table 1 compares the actual and estimated weight for

subsonic transport and executive jet airplanes, from Eq. (1.1).


Table 1: Comparison of actual and predicted weights from Eq. (1.1)
from Torenbeek (1992)
Aircraft WMTo, (kN) Ww, estimated (kN) Ww, actual (kN) Error, %
Airbus A300/B2 1343.6 188.43 196.31 -4.0
Airbus A340 2487.0 295.32 340.85 -13.4
BAC 1-11/300 387.0 38.60 42.90 -10.0
Cessna Citation II 59.16 6.79 5.73 +18
Boeing 727-100 71.8 75.89 79.02 -4.0
Boeing 737-200 513.8 39.79 47.21 -15.7
Boeing 747-100 3158.4 446.15 383.35 +16.1
Fokker F-28 Mk 4000 315.8 30.22 33.28 -9.2
Lockheed 1011-1 1912.8 224.59 210.86 +6.5
McD. Douglas DC-9/30 480.4 40.60 50.71 -19.9
McD. Douglas MD-80 622.8 60.95 69.22 -11.9
McD. Douglas DC-10/10 2024.0 250.46 217.97 +14.9
McD. Douglas DC-10/30 2468.9 252.74 261.83 -5.3


The main advantage of empirical weight equations is that they are easy to use.

They do not require expertise in weight engineering, so aerodynamics or performance

experts can easily use them. They do not require sophisticated computational resources,

but they also have some limitations that should be considered.










Empirical equations are generally a function of exterior geometric parameters,

such as wingspan and thickness to chord ratio, and they may not reflect

interior structural details such as frame, stiffener shapes and spacing.

Since they are based on existing aircraft data, empirical equations may be

misleading for unconventional designs or for new concepts that require

extrapolation instead of interpolation from the data. This may happen, for

instance, when new materials or technology appear (advanced composite

materials, bonded construction), or when operating temperatures become

much higher than those for previous structures.

When conceptual characteristics are not far beyond the available knowledge and

experience (in other words when interpolation is possible) accuracy by the empirical

equations may be of the order of 2-5% (Niu, 1988). However, as Schmidt et al. (1997)

suggested an error of 2% in fuselage weight prediction may be equivalent to about 1.5

tons of freight or 20 passengers.

Alternative to empirical weight equations. Historically, structural weight

estimation of aircraft that lie outside the range of applicability of empirical methods has

been determined by stress-analysis based approaches or by construction of prototypes.

The latter approach is very expensive and not practical unless the design is converged,

and verification is needed rather than the prediction. Stress-analysis based structural

weight prediction procedures are widely used throughout the design phase. Howe (1996)

presented a comparison between an empirical and a stress-analysis based wing weight

equation. Data for 118 aircraft ranging from the Cessna 150 to the Boeing 747/200 were

used for comparison. Howe reported that both equations gave reasonably good results,










but the latter were more accurate. However, it was also noted that the test data were

limited to aircraft of light alloy construction. Predictions by empirical equations can be

misleading for aircraft of different and advanced materials such as composite materials.

That is because the use of such materials has been dealt with somewhat arbitrary

factoring in empirical equations as historical data includes primarily the aircraft of

traditional materials.

Weight determination by stress analysis can be done at different levels of

complexity ranging from closed form beam and plate solutions early in the design

process to advanced finite element (FE) analysis at later phases of the process. Designers

seek the lightest practical arrangement of material and structural details capable of

withstanding the required loads. Depending on the complexity of the analysis and weight

prediction, the computational time required for a lightweight design may be substantial.

With advances in computational power, the level of complexity that can be

afforded in the preliminary design phase is increasing. For instance, FE analysis usage in

aerospace weight engineering has progressed to the point that preliminary structural

designs may be analyzed with low levels of uncertainty when combined with a

corresponding knowledge of the as-built structural design. However, the multi-

disciplinary nature of the modern design process cannot afford long cycle times, since it

entails frequent changes and extensive trade-in studies. In addition, conceptual designers

are not usually experts in FE modeling. That is why these designers still demand

computationally affordable, easy to use, but also reliable approaches. As a consequence,

the idea of combining the reliability of stress-analysis based prediction and the ease of

manipulation of analytical expressions in multi-disciplinary studies has attracted interest.










In general, structural weight equations have been developed for various types of

wing and fuselage structures since they are primary load-carrying elements. Wing and

fuselage structures are composed of a large number of curved or flat, usually stiffened

panels. Design optimization of all panels simultaneously, in other words the complete

structure, requires detailed structural modeling and appears to be beyond the present

computational capabilities since the panels have complex geometry and failure modes.

The current practice is to design wing or fuselage structures at two levels, global and

panel, and this can increase the required time for weight estimation.

In addition, other problems that affect stress-analysis based weight prediction and

design optimization are the following

Numerical simulations based on stress analysis such as structural

optimizations are prone to numerical noise due to reasons including

discretization errors, incomplete convergence of iterative procedures, and

round-off errors. When these simulations are used in a multi-disciplinary

optimization procedure employing gradient-based optimization algorithms the

procedure suffers from the noisy calculations, and may fail to find the

optimum. As a consequence, smooth responses are desired in optimization

problems.

To reduce the errors and the noise generated by the computational models the

complexity of the model may be increased. A finer finite element mesh, for

example, can be used in structural analyses. This, however, increases the

computational cost and design cycle time.










*Oftentimes interactive use of different structural codes is required for

capturing various failure modes, such as buckling, delamination, or crack

propagation. Integration of the different codes is not an easy task.


Approach

The approach taken into consideration is weight equations by response surfaces.

Response surface (RS) techniques (Myers and Montgomery, 1995: Khuri and Cornell,

1996), which fit a large number of analyses with simple functions, offer an attractive set

of features that address the problems mentioned in previous section:

They filter out the noise and help to identify outliers

They may require large computational effort beforehand to construct, but this

pays off during the design process since they are easy to compute

They can act as interfaces between the different levels of analysis or between

different disciplines.

Response surface methodology can be used to construct weight equations.

Approximations by RS are similar to empirical relations in the sense that they both

employ statistical fitting procedures. The source data used, however, are different. RS

based equations utilize the data created by stress-analysis based structural optimizations

at pre-selected design points, providing an overall perspective within the design space

(design of experiments) whereas empirical relations employ historical aircraft data.

Applications for RS based weight equations. Multidisciplinary optimization

(MDO) has become increasingly important with the increasing complexity of engineering

systems. Sobieszczanski-Sobieski and Haftka (1997) presented a survey for the recent

developments in this field. They indicated the two main challenges of MDO are










organizational complexity and computational expense. Response surface techniques are

particularly suitable for MDO applications. They can be used for approximating the

objective functions and constraints. For the organizational challenge these techniques

provide a convenient representation of the data from one discipline to other disciplines

and to the system. For the challenge of computational expense RS techniques may first

seem inappropriate since they require a large number of analyses for data generation

beforehand. However, the simplicity and insignificant cost of calculations by RS once

constructed make them attractive in multidisciplinary design optimization studies.

The organizational challenge exists also in a single-discipline application.

Oftentimes design optimization uses procedures employing different levels of analysis

and optimization as in global/local structural design. In these applications, integration of

different codes is generally required. Approximations by RS techniques may provide an

easy means to connect such codes. They may also be used to correct the low-cost

analysis results with the help of a small number of high-fidelity results whose

computational cost is often prohibitive to use at each design throughout the design

process.

Another important application of RS techniques is design against uncertainty. The

random nature of this problem makes it computationally intensive, and it typically entails

at least an order of magnitude higher computational cost than deterministic design

(Venter and Haftka, 1998). Response surface approximations can be used to reduce this

high computational cost since they are easy to calculate and implement. The designer,

however, should keep in mind that the RS approximation itself may impose another type

of uncertainty due to modeling error that needs to be addressed in uncertainty studies.










Challenges associated with RS based weight equations. Structural

optimizations that are often used to generate data for RS based weight equations are

prone to numerical noise. Optimization based on finite element analyses, for example,

may introduce noise due to discretization errors affecting the analysis results or even due

to round-off error characteristics of the computation platform affecting progress by the

optimizer (Papila and Haftka, 1999). Although one of the most appealing features of RS

approximations is the ability to filter the data, high-amplitude-noise outliers that may be

experienced in structural optimizations may cause the loss of accuracy of the

approximation or weight equation. Therefore, it is important to make sure the data for

constructing the weight equation are free from severe flaws before relying on the weight

equation. In case of data generated by structural optimizations, for instance, optimum

weight data free from flaws such as incomplete convergence would increase the

reliability of the weight equation. Even when the data generated are free from severe

flaws or outliers, RS based weight equations are still approximations. This is because the

number of the data is usually limited and it does not cover every possible design point. In

addition, traditional polynomial fitting models may not represent equally the true nature

or complexity of the weight function for every region in the design space. Therefore, it is

also important to study the accuracy of the weight equation and to pinpoint the design

regions where the approximation may be poor so that designer may steer away while

using the weight equation or at least be cautious.


Scope and Objectives

The main thrust of this dissertation is to address the challenges associated with the

RS based weight equations as mentioned in the previous section, that is, how to deal with










the errors that are introduced by the process of fitting structural optimizations with RS for

structural weight equations. This includes reducing error by identifying and repairing

outliers, studying the accuracy of approximation, reducing RS errors due to the

approximation model by using higher order polynomials, and characterizing the

remaining error for use in design for uncertainty. Therefore, the two main objectives of

the work are the following:

Reduce errors associated with RS constructed on the basis of structural

optimizations: To reduce noise error the aim is to identify the highly

erroneous results and label them as outliers within the data. For the data

generated by structural optimizations outliers may be due to reasons such as

failure of convergence and local optima. Once detected they may be studied

further for possible correction. The corrected or repaired data set is expected

to increase the accuracy and reliability of the RS. To deal with modeling

(bias) errors, higher-order polynomials are used.

Characterize the remaining error and investigate statistical tools to determine

the regions of design space where RS accuracy may be poor.

A literature survey of wing and panel weight equations, use of RS techniques for

accurate weight estimations and for analysis/optimization code integration, and the use of

RS based weight equations in design sensitivity to uncertainty are presented in Chapter 2.

Chapter 3 provides the background of RS methodology. Uncertainty due to use of RS is

given in Chapter 4 by deriving a measure for high modeling error locations in the design

space. Chapter 5 presents an application of the methodology for weight equations of a

high speed civil transport wing weight problem, and its use in design for uncertainty.






11


Chapter 6 summarizes similar weight equation construction for a stiffened composite

panel with a crack problem and Chapter 7 presents the concluding remarks for this

dissertation.















CHAPTER 2
LITERATURE SURVEY


Weight equations allow designers to design systems without getting bogged down

in the re-design of components as they change system characteristics. In aircraft design

the upper level system is the aircraft itself and the main structural components are wing

and fuselage. These in turn can be considered as systems of components such as panels,

frames, etc. The flight optimization system (FLOPS) wing weight equation (McCullers,

1984), for example, predicts how the aircraft structural weight changes with use of

composite materials without making panel and wing level detailed and computationally

burdensome structural analyses or optimization. Therefore, accurate and easy-to-use

weight equations are a useful and important tool for the designer.

Shanley (1960) published weight prediction methods based on elementary

strength/stiffness considerations, augmented by experimental results and statistical data.

This approach enabled weight engineers to obtain more accurate weight estimates than

traditional empirical weight equations and to consider sensitivity of weight to the design.

Moreover, it also paved the way to the 'station analysis' methods and use of more

involved analysis models.

In general, structural weight equations have been developed for various types of

wing and fuselage structures since they are primary load-carrying elements. Shanley

(1960) reported that existing empirical formulas were sufficiently efficient for the other










portions of the airplane structure. This appears to hold even for recent aircraft concepts.

Balabanov et al. (1996, 1998 and 1999) investigated weight due to wing bending-

resistant structural elements, wing bending material weight, of a futuristic high-speed

civil transport (HSCT). They used empirical formulas from the FLOPS program

(McCullers, 1984; 1997) for the entire structure except for the wing structural weight as

in Grasmeyer et al. (1998) and Gern et al. (1999) for another unconventional concept,

transport aircraft with strut-braced wing.

The focus of this work is on accurate weight equations for wings and for panels.

The former are used by the aircraft designer to design the entire airplane; the latter are

used by the wing or fuselage designer to design the entire wing or the entire fuselage.


Wing (System level) Structural Weight Equations

Wing weight equations started with fitting of historical data using specially

tailored expressions with variables raised to various powers. They progressed to

equations based on stress analysis of simple beam models of the wing and fuselage that

were adjusted to fit historical data.

Shanley (1960) reviewed the work by Lipp and Kelley. Lipp (1938) developed

formulas for the structural weight of wings by actual integration of assumed lift

distribution in the basic equations for the conditions of shear and bending. The final

equations were expressed in terms of load factor, span, root thickness, allowable stress

and taper ratios for chord and thickness. Kelley (1944) used more accurate spanwise lift

distributions that increased the difficulties in the integration and presented results in chart

form for engineering use. Shanley (1960) modeled the wing as a box beam whose

flanges carried the bending load whereas webs carried shear loads. He indicated that the










most important terms are span, mean aerodynamic chord, and the ratio of wing span to

depth.

Torenbeek (1992) presented a method providing estimations for the primary parts

of a wing structure. The method makes use of elementary stress analysis combined with

historical data. The wing group weight is expressed as the sum of the primary weight

(top and bottom covers, spars, ribs, and attachments) and the secondary weight comprised

of the weight of components in front of the front spar, components behind the rear spar,

plus any miscellaneous weight. The basic weight is the minimum weight of the primary

structure required to resist bending and shear loads where stresses are taken into account.

Formulas based on historical data were used in calculations of the secondary weight and

for the correction to the primary weight due to factors such as joints and mountings for

engines. Prediction errors of less than 4% were found for four representative transport

aircraft with take-off weights between 50 and 3500 kN.

Weight equations also progressed to using a multitude of structural design

optimizations (a review for structural optimization was presented by Vanderplaats, 1999)

based on simple plate models. McCullers (1984) developed the weight equations in

FLOPS by combinations of historical data and structural optimizations based on

equivalent plate models for wings using the wing aeroelastic synthesis procedure

(WASP) and the aeroelastic tailoring and structural optimization (ATSO) program

(McCullers and Lynch, 1972; 1974). Optimum wing designs data were used to account

for aeroelastic tailoring via composite materials, forward sweep angle effects and flutter

and divergence weight penalties for very high aspect ratio wings on the wing weight.

The equations are based on analytical expressions of primary wing configuration










parameters such as aspect ratio, thickness-to-chord ratio, sweep angle, etc. correlated by

constant factors based on historical data and results of structural optimizations. For

different categories of aircraft the constants and the expressions vary, so that equations

are applicable for a range of aircraft types, from military fighters to civil transonic and

even supersonic transport aircraft. These and similar weight equations attempt to fit any

airplane, or at least a broad class of airplanes. These types of equations are referred to as

quasi-analytical estimation in Staton (1996). They offer very good estimation accuracy

for existing technology aircraft, and through their sensitivity to engineering parameters

allow accurate trade studies. Their advantages over pure statistical equations lie in their

use in extrapolating outside parameter ranges in the database of aircraft.

One modem trend has been to replace wing weight equations with structural

optimization based on beam models inside the aircraft optimization problem. For

example, Kroo et al. (1991) used this approach in aerodynamic-structural design studies

of joined-wing aircraft. They combined a vortex-lattice aerodynamic model with fully

stressed beam sizing routine. Two structural box models: a symmetrical box of uniform

cap and web thickness, and an asymmetrical arrangement of cap thickness so that

diagonal corners of the box have the same thickness were used. They investigated

selected joined-wing designs and studied the effects of design parameters on drag and

structural weight. They employed a simple structural optimization following the

aerodynamic analysis for each design.

In later studies (Kroo et al., 1994; Braun et al., 1996), they used two-level

collaborative optimization that allows the designer to include other disciplines besides

structures and aerodynamics. Collaborative optimization allows some autonomy for each










discipline to optimize on its own, but the two-level formulation can be very ill

conditioned (Alexandrov and Lewis, 2000).

Gern et al. (2000) also embedded a beam-model structural optimization for

weight calculations of a strut-braced transport wing inside the aerodynamic optimization.

They employed a fully stressed design procedure accounting for aeroelastic load

distribution to estimate the wing bending material weight. The remaining portion of the

wing structural weight due to flaps, slats, spoilers, ribs etc. was estimated by FLOPS

weight equations.

Finite element (FE) analysis based procedures have also been used in the

preliminary design phase. Advances in the computational resources allow the use of at

least coarse FE models in this phase. Huang et al. (1996) performed simultaneous

structural-aerodynamic optimization by combining a FE based structural optimization for

wing bending material weight (structural weight needed to resist bending) with a simple

weight equation. Weight prediction from structural optimization was first compared with

two quasi-empirical wing weight equations for ten high speed civil transport (HSCT)

wing planforms. The equations were from FLOPS (McCullers, 1984; 1997) and by

Grumman Aerospace Corporation (York and Labell, 1980). The comparison indicated

that the weight equations predicted well the average structural weight and effects of

airfoil thickness on structural weight, but did not model accurately all of the effects of

planform changes on the structural weight. The FLOPS weight equation was chosen

since it agreed better with the structural optimization results than Grumman equation.

The structural optimization and weight equations were combined through the following

interlacing procedure: structural optimization and FLOPS weight estimations were found










for the initial design and their ratio was calculated. Five cycles of aerodynamic design

optimization were performed by using the FLOPS weight estimation corrected by the

ratio. After five cycles, the structural optimization was repeated for the new aerodynamic

design and the ratio was updated. These steps were repeated until convergence. The

interlacing procedure of the FLOPS weight equation and the FE based structural

optimization results was successful in terms of obtaining designs reflecting realistic

weight estimation balanced with computational cost. It did not protect, however, the

optimization from getting stuck at a design where the simple equation was too optimistic.

It was noted that derivative of the ratio or scale factor may be helpful for this problem.

Performing repeated structural optimizations during the aerodynamic design

process has the advantage that use of a general structural weight equation that must fit all

transports is not required. However, this approach has three main disadvantages: noise,

ill-conditioning, and code/discipline integration. Giunta et al. (1994) showed that

numerical noise caused convergence problems in design optimization for HSCT

employing gradient-based optimization techniques. Balabanov et al. (1996) had to deal

with numerical noise of several sources in their HSCT design problem. Venter and

Haftka (1998) experienced noise associated with the dependence of discretization error

on their shape design variables. Multi-level treatments, such as collaborative

optimization can lead to ill-conditioning and computational difficulties (Alexandrov and

Lewis, 2000). In addition, the integration of structural optimization software with the

other disciplines also requires substantial effort (e.g. Rohl et al., 1994).

An alternate approach that avoids the above difficulties is to generate a weight

equation for a particular aircraft by fitting the data generated via finite element based










optimizations within the context of RS methodology. This approach has the advantages

of the traditional weight equation, but without its disadvantages. The main difficulty is

obtaining adequate accuracy, and this is one of the thrusts of the present work.

Response surface techniques have become an important tool in multi-disciplinary

optimization (MDO) studies where numerical noise is often an issue besides the

organizational difficulty of coupling simulations from different disciplines. They filter

out numerical noise, and provide a convenient representation of data from one discipline

to another and are easy to interface with an optimizer due to their ease of implementation.

Kaufman et al. (1996) fitted quadratic polynomial RS approximation to the

structural weight obtained with multiple structural optimizations with GENESIS (VMA,

1997). The HSCT configuration optimizations performed with the RS produced superior

results than with the FLOPS weight equation.

Balabanov et al. (1996, 1998 and 1999) investigated RS construction for wing

bending material weight of HSCT based on structural optimization results of number of

different configurations and its use in the configuration optimization problem. They used

a larger number of low-fidelity (LF) models and tuned the results based on small number

of high-fidelity (HF) analyses (Balabanov et al., 1998) in order to handle the

computational cost problem associated with FE based structural optimization. Papila and

Haftka (1999, 2000a) also constructed RS weight equations based on structural

optimizations (see chapter 5).


Panel (Component Level) Weight Equations in Multi-Level Procedures

Wing and fuselage structures are composed of large number of curved or flat,

usually stiffened panels. Designing all the wing panels simultaneously constitutes a










complex optimization problem requiring detailed structural modeling of the entire wing

and appears to be beyond present computational capabilities. For integrated structural

wing (as system) and panel (as component) design, there are similar approaches to those

described in the previous section. In the past, there were simple, stress-analysis based

panel weight equations like Gerard's (1960). He performed minimum weight analysis of

various forms of stiffened-plate construction considered as the compression covers of

multi-cell wing box beams. This, however, did not take into account the effects of panel

stiffness on load redistribution (designer can help one panel by beefing up another) and

overall stiffness constraints.

The current practice is to design the system, wing or fuselage structures at two

levels; global (system) and panel (component) levels. At the global level, the structure is

modeled with moderate level of detail for individual panels, and the internal load

distribution is obtained. With these loads, panel-level design optimization for detailed

geometry is then performed by programs such as Panel Analysis and Sizing Code

(PASCO) by Stroud and Anderson (1981) or PANDA2 by Bushnell (1987).

Depending of the complexity of the problem, integration of global and panel

levels may become difficult. Schmit and Mehrinfar (1981) showed that the integration of

the levels can be handled well for simple structural models by a two-level decomposition

similar to collaborative optimization (CO) approach. One difficulty associated with the

decomposition; elimination of the component level (local) variables in terms of system

level (global) variables was mentioned in Haftka and Giirdal (1992). For panel

problems, as well as for complex truss and frame cross-sectional forms, it is impossible

to find analytical expressions for eliminating local variables and replacing them n il/










global variables. It is possible to keep both local and global; variables, and supplement

the problem ii i/h equality constraints that guarantee the consistency of the global

variables i/ i/h the local variables (Haftka and Giirdal, 1992). However, this approach

often tends to make the optimization problem more ill-conditioned (e.g. Thareja and

Haftka, 1986). As noted in the previous section two-level optimization suffers from ill-

conditioning and the fact that panel (local) level optima are not usually smooth function

of the global level design variables.

A successful integration of wing (system or global) and panel (component or

local) level structural optimizations was performed by Rohl et al. (1994) in three-level

decomposition approach for the preliminary aerodynamic-structural design of an HSCT

wing. The levels of the approach from top to bottom, made use of FLOPS as a general

aircraft sizing and performance code for aerodynamic optimization, ASTROS as a

structural optimization tool based on FE analyses for wing box (wing level), and PASCO

for panel-level optimization. The main purpose of the paper was to demonstrate the

integration of PASCO into ASTROS to supply buckling constraint evaluation for wing

level optimization, and its application in a multi-level wing design procedure. The

overall integration and execution of the procedure were performed by UNIX shell scripts

calling specifically designed finite element pre-processor for ASTROS using the FLOPS

output, PASCO pre-processor, and post-processor. The results of this integration

prototype without RS application indicated that it worked, but required the designer in the

loop making decisions in buckling optimization. Besides the modifications on the codes

and customized interfaces, it was noted that fully automatic run requires a logic

implementation.










Response surface approximations offer an attractive means also for integrating the

design of a single panel with the overall structure and for integrating different codes.

Ragon et al. (1997) introduced a methodology for global/local design optimization of

large wing structures. They used ADOP, Aeroelastic Design Optimization Program of

McDonnel Douglas in global/local optimization, and PASCO, Panel Analysis and Sizing

Code as the local design code. The methodology starts with generating many optimal

designs for each panel type for a range of loading and in-plane stiffnesses (All and A66).

The data generated were then utilized to construct an RS weight equation as a function of

the in-plane loads and stiffness parameters for the optimal panel weight satisfying all the

local failure constraints. However, for some values of the stiffness parameters no

feasible designs exist. When the panel RS weight equation was interfaced with ADOP,

during the global iterations ADOP tended to move into infeasible portions of the design

space. This was because no data corresponding to the infeasible design points was

included in the RS model, and the design regions bordering the infeasible regions tended

to be those with low structural weights. This problem was corrected by constructing a

new RS using the data that included penalty terms for those original design points for

which PASCO could not generate a feasible design. The penalty terms required higher

order (cubic) polynomial for accurate fitting. The use of RS in terms of stiffness

constraints is an advance over traditional weight equations. It allows the global design to

control the load carried by the panel by specifying the panel in-plane thickness.

In other global/local procedures responses other than weight were approximated

by RS easing the code integration. Venkataraman and Haftka (1997), for example,

utilized separate local and global analysis codes for a liquid-hydrogen tank structure










design made of laminated-composite material. They used NASTRAN for the global

model of irregular geometry and PANDA2 for the local regular stiffened panel analysis.

An iterative design procedure between NASTRAN and PANDA2 failed to converge

since the latter did not have any information of the global constraints. They overcame

this problem by a RS approximation for the global response used in PANDA2.

Liu et al. (2000) aimed to solve the infeasibility problem faced by Ragon et al.

(1997) who needed to satisfy both stiffness and strength constraints at the panel level.

Instead of weight equation Liu et al. developed failure load equation. Their procedure

was applied to laminated composite wing panels when the design involves discrete and

combinatorial optimization. Global and local design processes were coordinated through

an equation that predicts the buckling load multiplier. The equation was a cubic RS as a

function of number of plies of each orientation and the loads on the panel. Response

surface was fitted to a large number of panel stacking optimization based on analytical

solution for simply supported unstiffened panel in the configuration space of number of

plies and the loads. The reason for the cubic model is that buckling load is proportional

to the cube of the thickness. They used D-optimal experimental design points where

panel level optimizations by permutation genetic algorithm (GA) for the stacking

sequence. The objective function was maximum buckling load. Then wing-level

optimization was performed by Genesis using the cubic RS in the buckling constraint.

They applied the methodology for 6-, 18-, and 54-variable cases. It was shown that a

cubic RS can fit to the buckling load of the optimal panel stacking sequence as a function

of loading on the panel and the given number of plies, and it can be used in order to avoid

the problem of infeasible designs faced by Ragon et al. (1997).










Ragon et al. (1997) and Liu et al. (2000) generated RS based equations with panel

(component) design based on simple models. One goal of the present work is to push the

procedure further in the context of general FE modeling of stiffened panels.


Accuracy of Response Surfaces

Increasing use of RS techniques in design optimization studies has brought

attention to ways of increasing the accuracy of the RS approximations. Two sources of

error have been mainly investigated: numerical noise and modeling error due to

approximation functions. Efforts for identifying the cause of the numerical noise, for

improving the data by reducing the noise and by proper selection of design points--

Design of Experiments (DOE)-- and for reducing modeling errors pay back in the

accuracy of the RS and in its reliable application.

Numerical experiments may be noisy due to several reasons including

discretization errors, incomplete convergence of iterative procedures, and round off

errors. In order to reduce the effect of noise in RS accuracy DOE offers variance optimal

designs. Standard designs such as central composite design (CCD), for example, are

quite efficient against noise effect (Montgomery and Myers, 1995). However, there are

high-dimensional problems where more economical designs are generally required. D-

optimal design criterion (Montgomery and Myers, 1995; Khuri and Cornell, 1996), for

example, is commonly used in such problems. Balabanov et al. (1996) used D-optimal

design for 29-variable HSCT design problem in order to reduce the number of structural

optimizations.

Response surface of an effective experimental design providing low variance

level can be used as a data-filtering tool. Venter et al. (1998a and 1998b) made use ofRS










approximations for filtering noise in finite element analyses. In their case the noise was

associated with the dependence of discretization error on shape design variables. Giunta

et al. (1994) found numerical noise in their calculation of aerodynamic drag components

for the aircraft that caused convergence problems in design optimization for HSCT. They

demonstrated the noise in a two dimensional example that showed variations were small

so that at all points the accuracy of the drag was acceptable, but oscillatory behavior

created difficulties for the gradient-based optimization technique. Quadratic RS

approximation was used to filter out the noise in aerodynamic simulations. Giunta et al.

(1997) applied the RS approach for 10-variable HSCT configuration optimization

problem. They reported improvement in designs obtained based on the smooth RS

approximation compared to the original noisy simulations. Use of RS resulted in

virtually identical optimal design starting from two different initial designs whereas using

the original noise-producing aerodynamic model method resulted in different optimal

designs.

Balabanov et al. (1996) investigated the noisy behavior of wing bending material

weight of HSCT designs (that was indicated by Kaufman et al., 1996, but without

identifying its source). They found that a large portion of the noise was due to

incomplete optimization of the wing camber that affects the wing bending material

weight via the aerodynamic loads. Much of the remaining numerical noise was due to the

structural optimization process itself through incomplete convergence. Accuracy of

bending material weight RS after correcting the camber optimization problem increased

substantially compared to one in Kaufman et al. (1996). To improve the accuracy

further, minimum bias and minimum variance design strategies were tried. They worked










comparably well in terms of root-mean-square-error (RMSE) and average error. The

maximum error, however, is about 30% less in minimum bias design based RS.

While low-amplitude, random noise is filtered well by RS approximations, data

with large errors--outliers--may cause significant loss of accuracy. Therefore, robust

regression employs techniques for statistical methods, such as iteratively re-weighted

least square (IRLS) fitting (Holland and Welsch, 1977), to detect and remove or weight

down outliers. Once detected, outliers can be investigated further for possible mistakes.

These techniques help also to identify flaws in analysis/optimization procedures so that

improvement on the results can be achieved. Optimization procedures often produce

poor results due to algorithmic difficulties, software problems, or local optima. When a

single optimization is flawed, it may be difficult to detect the problem, because the ill

conditioning responsible for the problem may make it difficult to apply optimality criteria

unambiguously. However, in many applications a large number of optimizations are

performed for a range of problem parameters (e.g. Balabanov et al., 1999). When such

multiple optimizations are available, statistical analysis can be done to detect incorrect

optimal results. For example, Papila and Haftka (1999 and 2000a) and Kim et al. (2000

and 2001) used IRLS procedures to detect such outlier points. More details on these

references are presented in Chapter 5.

Researchers have also found ways for dealing with modeling error. For instance,

it can be decreased by proper selection/definition of variables/factors. Concept of

intervening design variables and intervening functions was introduced by Schmit and his

coworkers for accurate approximations (e.g. Schmit and Farshi, 1974). They are

transformations of the original variables and functions or arguments of the functions.










Finding/defining intervening variables as functions of the original variables would help to

model response with lower order polynomials (or lower order derivatives in case of local

Taylor series approximations), and to decrease computational cost to obtain the

approximation (Haftka and Girdal, 1992). Another way of decreasing modeling error

and increase the RS accuracy is use of reasonable design space approach (Balabanov et

al., 1997). The approach seeks inexpensive constraints such as simple geometric

constraints that prevent combinations of design variables resulting in unreasonable

geometry configurations. Eliminating large portions of the design space occupying those

unreasonable configurations renders the design space more similar to a simplex. The

approach improves the modeling accuracy of the RS by shrinking the region where the

RS is fitted.

Kaufman et al. (1996) investigated the ways of improving the RS accuracy for

weight equations. They looked for intervening variables that may reduce the number of

data points or increase the accuracy. They also used inexpensive approximate analysis

methods and geometric constraints to find the regions of the design space where

reasonable HSCT configurations are likely to appear. Unreasonable designs were not

eliminated, but moved towards the reasonable region so that they reside on the edges of

the reasonable design space. This approach did not reduce the number of structural

optimizations, but caused increase in accuracy. The reasonable design space approach

was also applied by Roux et al. (1998) to truss test problems and by Balabanov et al.

(1999) in fitting RS weight equation for HSCT wing.

Knill et al. (1999) demonstrated a method enabling the efficient implementation

of accurate computational fluid dynamics (CFD) predictions into high dimensional,










highly constrained MDO problems like HSCT. They utilized simple conceptual level

models to select an appropriate set of intervening functions for which to construct RS

models since accuracy with low-order polynomials can be improved by transformations

of the real function or its arguments. Supersonic linear theory aerodynamics was used as

low-fidelity model to generate data for full term RS approximation. Stepwise regression

was then used with this data, and RS models in order to determine the significant

terms/variables in calculation of the desired aerodynamic quantities. Computationally

expensive Euler solutions were performed next only for set of configurations reflecting

the effects of the significant terms. This reduced substantially the number of high-fidelity

Euler analyses required. They used the reduced-term RS models with Euler analyses as

an additive correction RS.

Another way for reducing modeling error is to use higher order polynomials or

more complex functions in RS. Venter et al. (1996) made use of dimensional analysis for

reducing the number of variables and the design space for minimum weight design of

plate with a step. The reduction in the number of variables made use of higher order

polynomials in RS model affordable. They used cubic and quartic polynomials and

improved the accuracy for the stress and buckling constraints. Papila and Haftka (1999

and 2000a) also achieved substantial improvement in accuracy by using cubic

polynomials in RS approximation for HSCT wing bending material weight (see in

Chapter 5).

Neural network (NN) based RS approximations have also been investigated as

alternative to traditional polynomial-based RS approximations within the context of

design optimization. They allow fitting highly non-linear response. Their main










drawbacks are the cost of fitting the approximation and lack of statistical tools for

effective design of experiments. There are comparisons in the literature of the

performance of NN-based and polynomial-based RS approximations. For example,

Carpenter and Barthelemy (1993) used NN-based and polynomial-based approximations

to develop RS for several test problems. They found that two methods perform

comparable based on the number of undetermined parameters. Papila (N.) et al. (1999)

investigated the effect of data size and relative merits between polynomial and NN-based

RS in handling varying data characteristics. They demonstrated that as the nature of the

experimental data becomes complex NN-based approximations perform better than

polynomial-based. The neural network technique and the polynomial RS were integrated

to offer enhanced optimization capabilities by Shyy et al. (1999 and 2001) and Papila

(N.) et al. (2001). Vaidyanathan et al. (2000) applied NN and polynomial-based RS

approximations in the preliminary design of two rocket engine components, gas-gas

injector and supersonic turbine. They demonstrated that NN-based and polynomial-based

approximations can perform comparably for modest data sizes. Papila (N.) et al. (2000)

investigated the performance of both polynomial and NN-based RS in their multi-

objective turbine blade design optimization problem.

Oftentimes, designers sacrifice the fidelity of the analysis model used in data

generation for RS approximation due to computational cost. This may cause decrease in

accuracy of the data and approximation. There is a simple way to improve accuracy in

such cases: correction response surfaces (CRS) that make use of low-fidelity (LF) results

with small number of high-fidelity (HF) results. Mason et al. (1994), for example,

studied design optimization of curved composite frames typical of a support structure for










semi-monocoque aircraft structures and applied two-levels of fidelity and correction of

HF solutions onto the LF ones. Vitali et al. (1997) also used two levels of fidelity and RS

technique in design of hat-stiffened panel for blended wing body aircraft. Venkataraman

et al. (1998) demonstrated effective combination of codes/models at different fidelity

levels by CRS for structural optimization of a ring-stiffened cylindrical shell of

revolution. These works were examples of CRS in component design.

Balabanov et al. (1998) used the CRS approach to improve the accuracy of RS

based wing weight equations. They used coarse FE models for thousands of HSCT

configurations, and a quadratic RS was constructed for wing bending material weight.

Then about a hundred of the configurations were optimized with a refined FE model to

construct a linear CRS. Both additive and multiplicative corrections were applied. The

linear additive correction was found slightly better for their design problem. They tried

to correct either the prediction of the RS or to correct the data themselves by the high-

fidelity optimizations. The correction reduced errors by more than half.

Another important relevant work in the sense that LF/HF coupling of numerical

simulations applied was by Knill et al. (1999). Their main contribution is that besides

CRS they used the RS fitted on LF results to identify the important polynomial terms, so

that the RS model of HF had fewer terms and required fewer analyses.

Design of experiments offers also minimum-bias criterion for reducing modeling

(bias) error. Venter and Haftka (1997) developed an algorithm implementing minimum-

bias based criterion, necessary for an irregularly shaped design space where no closed

form solution exist for minimum-bias design. They compared minimum-bias and D-

optimal designs for two problems with two and three variables. Minimum-bias design










based RS was found more accurate than D-optimal for the two problems except for the

percent maximum error. Percent maximum error was much bigger by minimum-variance

design in Balabanov et al. (1996).

Errors in RS approximations represent a part of the uncertainty in the design

process. The studies discussed above attempt to increase the accuracy and to decrease

the level of uncertainty associated with the use of RS. Another important objective of the

present work is try to characterize further the errors in RS predictions so that the

uncertainty introduced by them can be quantified.


Use of RS in Uncertainty Quantification

During the design of a new aircraft concept like HSCT, modifications and

changes in geometry and material properties are likely. Therefore, it is important to

determine how sensitive to change the design is.

Uncertainty can be studied by probabilistic or by fuzzy set models. In both cases,

design subject to uncertainty is computationally expensive. Response surface

approximations alleviate the computational burden because they allow for an inexpensive

evaluation of the effect of changes although they cause uncertainty of some degree

themselves (Papila and Haftka, 2001a). Reliability based design studies made use of RS

approximations (Schueller et al., 1989; Rajashekhar and Ellingwood, 1993; Romero and

Bankston, 1998). Xiao et al. (1999) used a reliability based MDO for shape optimization

of a transport aircraft wing, treating material properties, geometric variables and loading

conditions as random variables with assumed probability distributions. They constructed

RS approximations for the objective function and all the constraints and applied Monte

Carlo simulation for calculating reliabilities.










Qu et al. (2000) investigated the use of RS combined with efficient Monte Carlo

simulation techniques in reliability-based design optimization of angle-ply laminates

subjected to mechanical loads at cryogenic temperatures. Uncertainties for the problem

were in material properties due to short of characterization through the entire operating

temperature range, its failure modes at cryogenic temperatures, stiffness and strength

properties manufacturing dependent. Response surface approximations were used to

reduce computational cost and filter out random sampling noise in the Monte Carlo

simulation.

Fuzzy set model of uncertainty and RS approximations, and within the vertex

method (Dong and Shah, 1987) were employed by Venter and Haftka (1998b), for

calculating the possibility of failure in dropped-ply composite laminate design. Same

approach was also used by Papila and Haftka (1999) for sensitivity to uncertainty of wing

bending material weight in HSCT designs (See Chapter 5).















CHAPTER 3
RESPONSE SURFACE METHODOLOGY


Response surfaces (RS) approximate numerical or physical experimental data by

an analytical expression that is usually a low-order polynomial. Khuri and Cornell (1996,

p. 3) described the three key steps within the context of the methodology:

Pre-selecting the points-design points-where experiments are performed. This

step is called design of experiments to represent the design space so that the

points selected will yield adequate and reliable measurements/calculations of

the response of interest.

Determining a mathematical approximation model that best fits the data

collected/generated from the set of design points of the previous step. This is

performed by conducting appropriate statistical test of hypotheses concerning

the model's parameters.

Using the RS approximation for predicting the response for a given set of the

experimental factors/variables. A common application is to determine the

optimal settings of the factors that produce optimum (maximum or minimum)

value of the response.

This chapter summarizes first the design of experiments used in this dissertation.

Section starting on page 36 presents the standard least-squares fitting technique. In the

section starting on page 39, statistical tools to check the accuracy of the RS are given










including a customized lack-of-fit test (page 52). Then, section on page 58 presents

Iteratively Re-weighted Least Squares (IRLS) accommodated with the standard

weighting function.


Design of Experiments

Experimental designs possess the following properties required to fit a pth order

polynomial RS for k variables/factors:

at leastp+l levels of each factor

(k +l)(k + 2).....(k + p)
at least (k + 1)(k + 2) (k + distinct design points
p!

Montgomery and Myers (1995, p. 280) summarized important characteristics that

make choice of experimental design effective. An efficient design will

Result in a good fit of the model to the data

Give sufficient information to allow a test for lack of fit

Allow models of increasing order to be constructed sequentially

Provide an estimate of "pure" experimental error

Be insensitive (robust) to the presence of outliers in the data

Be robust to errors in control of design levels

Be cost effective

Provide a check on the homogeneous variance assumption

Provide a good distribution of prediction variance

Although not all of the above properties are required in every RS experience,

most of them must be given serious consideration on each occasion where one designs

experiments.










In many practical situations the scientist or engineer specifies strict ranges on the

design/configuration variables (experimental factors). That is region of interest and

region of operability are the same and a cuboidal. In general, factors of the experiments

or configuration variables v, in this work have different order of magnitudes and

different units of measurement. To avoid ill-conditioning and simplify the numerical

calculations used in the parameter estimates for the response surface approximation,

variables are coded by Eq. (3.1) so that the smallest and the largest limits for

configuration or control variables are assigned as -1 and +1, respectively.

v, [max(v,) + min(v, )] / 2
x = (3.1)
[max(v ) min(v )] / 2

For the application problems of cuboidal design regions in this study, different

experimental designs and their combinations were considered based on the number of

factors or configuration variables and the degree of polynomial used. As observed from

the literature for the engineering design problems studied in this work quadratic

polynomial RS approximations are commonly used (e.g. Balabanov et al. (1996, 1998

and 1999). Cubic or even higher order polynomials, on the other hand, were also applied

(e.g. Venter et al., 1996); Papila (N.) et al. 1999). In this work, quadratic polynomials

are used as an initial model and based on the accuracy achieved first order and cubic

polynomials are also studied. Three-level full factorial designs (FFD) and face-centered

central composite designs (FCCD) are typically applied for quadratic models depending

on the number of factors involved. These experimental designs are schematically shown

in Figure 2 for the case of three coded factors or configuration variables. In the cuboidal

region, it is important that the points be "pushed to the extreme" of the region and it is










important for the region to be covered in a symmetric fashion. These properties result in

attractive distribution of prediction variance (Montgomery and Myers, 1995, p. 312-313).

Three-level FFD and FCCD accomplish this.








,, < ; ". '
S A '



O -------,- -------- --- ----r -
SI I I



x X2 x X2
x1 X2 X1
Figure 2: Three-level full factorial (left) and face centered central composite (right)
experimental designs for three coded-configuration variables


With the increase in the number of factors, initial experimental designs considered

here move from the full factorial design to FCCD and with more factors added from

FCCD to a D-optimal (Khuri and Cornell, 1996; SAS, 1998) subsets of FCCD.

Moreover, the feasibility of the data points and their usefulness in terms of design

performance is checked by quick, but representative evaluations where possible.

Unreasonable designs are removed in order to reduce the number of designs from the

original FCCD to provide feasible or reasonable designs for the D-optimal selection.

When higher order (e.g. cubic) polynomials are needed, orthogonal arrays (Owen,

1994) with four or more levels or the addition of a sub-box in the original design space is

used to provide the required number of levels. Table 2 presents a summary of the

experimental designs used in this study for number of factors and degree of polynomials

employed.










Table 2: Experimental designs used in the present work
# of factors Quadratic polynomial Cubic polynomial
2 Three-level full factorial design Three-level full factorial design

3 Three-level full factorial design

+ subFCCD
5 FCCD +
orthogonal array
D-optimal subset of
10Doptmal subset of + orthogonal array
(FCCD infeasible designs)



Fitting Procedure: Standard Least Squares

The response at jh design point x,j is denoted as y, [= y(x,)] and given by Eq.

(3.2)

y, = r7 + C (3.2)

where ir [ = (x ) ] is the true mean response and e, represents other random sources of

variation such as measurement error and noise not accounted for in r7. In other words if

the experimenter has the luxury of performing the experiment many times at x,, the

average of the observations will tend to r, despite the random errors j In RS

technique the true mean response is assumed to be given in terms of coefficients A s and

shape functions f (x ) where i=1, nb as

nb
rf(x ) = f (x)= fT (3.3)
1=1

where fT ={ (x, ) (X ..) f J (x)} and p = t f,



* Bold face is used for vector and matrix representations, e.g. x = x, X2 T for two-variable case
where superscript T stands for transpose operation









The shape functions are usually assumed as monomials. For instance, for

quadratic RS in two-variables Eq. (3.4) shows the shape functions and their vector

representation

f1(x ) 1
f2(Xi) j
f = f, f3(xi) x2j (3.4)
A(x,) x
f,=f(,) f4 (X ) (34)


f6 (X) X2

The methodology (Myers and Montgomery, 1995; Khuri and Cornell, 1996)

makes assumptions, so-called ideal error assumptions (or independent and identically

distributed, i.i.d. assumptions)

The RS expression is exact and represents the true mean response

The error e, in the experiments is un-correlated, normally distributed random

noise.

This random variable has expected value of zero, E(, )= 0, and constant

variance, Var(e,) =a2 throughout the design space. Or, -N(0, '21)

where E is error vector and I is the identity matrix.

The method of least squares is used to estimate the coefficients s. The least

squares function L is the sum of squares of the difference between collected response data

yj and the assumed true mean response r77 [Eq. (3.3)]


L= y,--ff (x,) (3.5)
7=1










The function L is minimized with respect to f, s, and least-square estimators b,s

of f, s satisfy


L =-2 f(x) y bk( ) =0 i ,2,....,b (3.6)
aA k=l

This yields a system of linear equations that can be solved for b,s. Then the RS

approximation is given as

17b
j(x,)= [ bf(xj)
1=1 (3.7)
= fTb

The difference (residual) between the data y, and the estimate defined in Eq. (3.7)

is given for the th point x, as

e, = y A(x) (3.8)

In general case j(xj) may not be the true mean response and the difference from

the data may not be only the random error, therefore the residual error is denoted by e,

instead of e .

Matrix forms of Eq. (3.5), (3.6), (3.7) and (3.8) for N data points are

L = yTy 2pTXTy + PTXTX (3.9)


S= -2XTy +2XTXb = 0 (3.10)


y =Xb (3.11)

e =y Xb (3.12)










where X is the matrix whose terms in the row associated with the point x, are formed by

shape functions f (x,). For instance, for a quadratic model in two-variables with N data


points [see Eqs. (3.3) and (3.4)] X is given as

fT 1 x1 X21
fT 1 12 X22

X=
fT 1 X


JN 1 XlN X2N

The system of equations, Eq. (3.10) may

XTXb= XTy


2
Xl 1] ]X2
X12 X12 22



2

X2N XN X2N

be written as


and the coefficient vector b can now be expressed as

b = (XTX)-XTy (3.15)

If Eq. (3.14) is substituted in Eq. (3.9) the remaining error vector e, is found to

satisfy


ee, = yTy -TXTy

that is called as sum of squares of the residuals SSE.


(3.16)


Adequacy and Accuracy of RS

Adequacy and accuracy of the RS approximation are mainly affected by the

following three factors whose effects also interact with one another

Use of finite number of data points

Noise in the data

Inadequacy of the fitting model


(3.13)


(3.14)









If the true mean response is exactly represented by the RS form as assumed, the

components b,s of vector b in Eq. (3.15) are the unbiased estimates of the A s, in other

words the expected value of b is equal to P as can be shown as

E(b) = E[(XTX)- XTy]
=E[(XTX)- XT(X + E)] (3.17)
= E[(XTX)- XTX + (XTX)-' XTE]

Since E(E) = 0 and (XTX)-XTX = I (I is the identity matrix)

E(b)= P (3.18)

However, when a finite number of design points are used the b,s will not be

exactly equal 8,s due to effect of random error in the data. The variance of a random

variable Y, Var(Y), is the expected value of the squared difference between the variable

and its expected value:

Var(Y) = E[(Y E(Y))2] (3.19)

Similarly the variance-covariance matrix of a random vector Y, Var(Y) is defined as

Var(Y) = E[(Y E(Y))(Y E(Y))T ] (3.20)

Following Eq. (3.20) the variance-covariance of the coefficient estimator vector b is

derived in Eq. (3.21)

Var(b) = E{[b E(b)][b E(b)]T
= E[(XTX)-'XTy -_ ][(XTX)- XTy ]T}
= E{[(XTX)- XT(Xp + ) p][(XTX)-IXT(Xp + E) p]T
= E{(XTX)- XT X(XTX)-1 (3.21)
= (XTX)- XTE(ET)X(XTX)-I
= (XTX)-1XTVar(E)X(XTX)-1
= (XTX)- XTo2INX(XTX)-
= 02(XTX)-1









The variance-covariance matrix is symmetric and its ith diagonal (as i =1,2...n )

characterizes the variance of estimator b, that causes variance of j(x,) so-called

prediction variance. The prediction variance can be expressed as [see also Eq. (3.7)]

Var L(x,)] = Var[fb] = ff Var(b)f, (3.22)

where f, = f(x) Substitution of Eq. (3.21) into Eq. (3.22) gives prediction variance in

Eq. (3.23).

Var[I(xj)]= 2fT (XTX)-lf, (3.23)

The random noise error variance o2 is not usually available and needs also to be

estimated. Sum of squares of the residuals, SSE from Eq. (3.16) is used to develop an

estimator s2 (error mean square) for '2.


s2 =MSE SSE ee (3.24)
N-n, N-n,

Sum of squares of the residuals SSE and evidently s2 represents variation in the

data not accounted for by the fitting model. The error mean square s2 is an unbiased

estimate of the noise error variance U2 provided that fitting model exactly models the

true mean response 17. Replacing 72 with s2 estimates for the variance-covariance

matrix [Eq. (3.21)] and prediction variance [Eq. (3.23)] can be given as

Var(b) = s2(XTX)-1
Var[5(xj)], s2ff (XTX)- 1

Positive square root of the prediction variance is usually used as an estimate of the

prediction error at a design point x,, also called estimated standard error as given in Eq.

(3.26).










ee (x)= Var[(x )] sff(X TX)- f (3.26)

where estimator s is positive square root of MSE [Eq. (3.24)] often called in engineering

literature as root-mean-square-error (RMSE). The estimated standard error shows

dependence on the location of the design point. Furthermore, as Eq. (3.3) is, in general,

only an approximation to the true function, s will contain not only noise error, but also

modeling (bias) error. The estimator s is also used to calculate coefficient of variation as

given in Eq. (3.27)

s
cV = (3.27)
V
y

N
zYJ
where y = -
N

Besides s and c,, the quality of the approximation is often measured by

comparing the sum of squares of the residuals SSE (sum of squares unaccounted for by

the fitted model) and the sum of squares due to regression SSREG (sum of squares

explained by the fitted model). As SSE is given again by Eq. (3.28) the latter sum of

squares is defined by Eq. (3.29)

N
SSE = (Y, )2 (3.28)
J=1

N
SSRE = 2 (3.29)
J=1

where yj = )(x ) and y is the average of the data as used in Eq. (3.27). Their sum

shows the total variation in the data set. It is called as total sum of squares SSr and can

also be expressed as










N
SST = (y, -y)2 (3.30)
J=1

The matrix forms of the sum of squares are given in Eq. (3.31)

SSE =yTy- bTXTy

SSREG =bTXTy ( )2 (3.31)

T (1Ty)2
SS, =y y
N

where 1 is a vector of N ones.


Hypothesis Testing in RS Methodology

Statistical hypothesis testing (Stark, 2001) is formalized as making a decision

between rejecting or not rejecting a null hypothesis on the basis of a set of

observations/calculations. Two types of errors can result from any decision rule (test):

rejecting the null hypothesis when it is true (a "Type I error"), and failing to reject the

null hypothesis when it is false (a "Type II error"). For any hypothesis, it is possible to

develop various decision rules. Typically, the chance of a "Type I error" that researcher

is willing to allow is specified ahead of time. The chance that the test erroneously rejects

the null hypothesis when true, is called the significance level of the decision rule (a).

The observations are then used to calculate a test statistic. Researcher compares the test

statistic (such as F-statistic) and its statistical table value (F-distribution) associated with

the selected significance level a If the statistic is larger than the table value, the test is

considered as "significant" at level a, and null hypothesis is rejected. The most

commonly used level of significance is 0.05 by which researcher accepts 5% probability

of making a mistake by rejecting the null hypothesis.










There are certain statistical hypotheses and relevant tests to measure the

efficiency and reliability of the RS approximation model. These tests also presume the

ideal error conditions. The tests are performed on a null hypothesis Ho against an

alternative hypothesis Ha. A statistic customized for the testing is calculated and checked

against a relevant statistical table of the statistic as a function of degrees) of freedom that

describes how many redundant results/measurements exist in an overdetermined system.

One important test is the test of the significance of the fitted regression equation

as formalized by the following hypotheses:

Ho :All coefficients (excluding the intercept) = 0
(3.32)
H : At least one coefficient (other than intercept) is not zero

The test determines if there is a relationship between the response variable and a

subset of the regression variables. Rejection of Ho implies that at least one of the

variables play role on the response values. The test statistic for this test is the F-statistic

as given in Eq. (3.33).

F = SSREG(nb-1) (3.33)
SSE (N n )

Decision based on this test at a significance level a is made by comparing the F-

statistic calculated by Eq. (3.33) with the F-distribution value ., where dfn and dfd

are degrees of freedom for numerator and degrees of freedom for denominator,

respectively. For the test in Eq. (3.32) dfn=nb-1 and dfd=N-nb. The F-

distribution is the distribution of the ratio of two estimates of variance. The F-

distribution has two parameters dfn and dfd and may be denoted as F .,. The dfn is

the number of degrees of freedom that the estimate of variance used in the numerator is










based on. The dfd is the number of degrees of freedom that the estimate used in the

denominator is based on. The shape of the F-distribution F ., depends on values of


dfn and dfd. Figure 3 shows two examples for F-distribution. ., is the F-value


where the rightmost tail area under distribution F. is equal to a. For example,


F005,4,12 and F.05,10,100 are equal to 3.26 and 1.93, respectively (associated tail areas

equal to 0.05 are as shown on Figure 3).


F4,12
5% of the distribution
.- is greater than 3.26

O 1 2 3 4 5
F

SF'10,100
S\ 5 of the distribution
iCL is greater than 1.93

0 1 2
F

Figure 3: Sample F-distributions


The value F, can also be defined as the upper 100a percentile of the

distribution with degrees of freedom at numerator dfn and at denominator dfd. If the F

statistic by Eq. (3.33) is beyond the upper 100a percent point of the distribution with

dfn = nb 1 and dfd = N n,, that is if F > Fn _, _n then the null hypothesis is in

Eq. (3.32) rejected at the a level of significance. This is interpreted that the variation

accounted for by the model (through the values of b, other than intercept) is significantly

greater than the unexplained variation (Khuri and Cornell, 1996, p. 31).










In general, the test statistic calculated such as F in Eq. (3.33) is different for a null

hypothesis than the table value of the selected significance level a. The statistic

corresponds to a level in the table calledp-value. Thep-value is the smallest significance

level for which any of the tests would have rejected the null hypothesis (Stark, 2001).

Therefore, it provides a sense of significance of the evidence against the null hypothesis.

The lower the p-value, the more significant the evidence and the test. If the significance

level a is set as 0.05, the test resulting in ap-value under 0.05 would be significant, and

the null hypothesis is rejected in favor of the alternative hypothesis.

In the test of the significance of the fitted regression equation formalized in Eq.

(3.32), the p-value is determined as the rightmost tail area under the distribution Fn _,N

beyond the F statistic value by Eq. (3.33). For instance, if test statistic F is 2.50 on

FI0,00 (Figure 3), then p-value would be 0.01 (FOO10,100o =2.50). This means the

researcher can reject the null hypothesis at any significance level a larger than 0.01 and

conclude in favor of alternative hypothesis.

Coefficients of determination R2 and R2 (R2-adjusted), accompanying statistics

to F-statistic, are respectively a measure and an adjusted measure of the proportion of

total variation of the values ofy, about the mean value of the response y explained by the

fitted model (Khuri and Cornell, 1996, p. 31-32) and given as

SSR
R = REG
ST (3.34)
R21 _SSE (N-nb)
a SS /(N 1)

An R value larger than 0.9 is typically required for an adequate approximation.










Another hypothesis testing concerns the individual coefficients in the model:

Ho :p, =0
(3.35)
H, :A 0

While the test for Eq. (3.32) is performed for the comparison of the two types of

variance, the test for Eq. (3.35) is to check if the mean value for individual coefficients of

the model is different than zero. Such a test would be useful in determining the

significance of the each variable and regression coefficient in the fitting model. The test

is performed by comparing the coefficient estimates to their respective estimated standard

errors. The test statistic, t-statistic is defined by


t, = b (3.36)
S

where s, is the estimated standard deviation of ith coefficient b, (called standard error) and

is obtained as the positive square root of the estimated ith term of the diagonal (as

i= 1,2...nb) of variance-covariance matrix given in Eq. (3.21). For the alternative

hypothesis in Eq. (3.35) the test is two-sided. Absolute value of the calculated t-statistic

is compared with the value obtained from the Student's t-distribution table (denoted as

tf ) relevant to the degree of freedom, df=N -nb. Student's t-distributions of different

degree of freedom df are shown in Figure 4. The t-distribution is a symmetric, bell-

shaped probability distribution, similar to the standard normal curve. It differs from the

standard normal curve, however, in that it has an additional parameter, called degrees of

freedom, which changes its shape.

The table value t, for a level of significance a is the upper 100(a/2)
S,N-nb

percentile of the distribution td For instance, the table values of different degree of










freedom indicated on Figure 4 are values concerning level of significance a = 0.05. The

area beyond 3.18 for df-3, for instance, is equal to a 2 = 0.025.




df= C, df= =6
df= 3

a/2=0.025


-3.18 -196 196 3.18 4.00
-2A5 trit 2A5 p-value/2=0.014


Figure 4: The t-distributions of various degrees of freedom df


The statistical significance or p-value for this test is determined as two times the

area under the t-distribution tdf and beyond the absolute value of t-statistic calculated by

Eq. (3.36). For example, if one tests the null hypothesis H: i = 0 and calculates

statistic t=4.0 with df-3, from the relevant t-distribution the p-value is found 0.028 (=

2x0.014) as shown on Figure 4. This means that, there is a 2.8% chance of making an

error to reject the null and say blz0. If researcher chooses to work with a level of

significance a more than or equal to 0.028 then the test concludes that there is no

significant evidence to accept the coefficient being zero.


Standard Lack-of-Fit Test

Lack-of-fit test is used as a tool to check if the fitting model is not adequate and it

does not contain a sufficient number of terms. In order to distinguish the fitting model

and the missing terms subscripts, 1 and 2, respectively, will be used in matrix and






49


function forms in the formulation. The fitting model and the true model are represented

as in Eqs. (3.37) and (3.38), respectively.

y = Xl1P + E (3.37)

E(y) = X101 + X2,2 (3.38)

where X1 and P1 respectively denote the model previously denoted by X and P in the

second section of this chapter (starting at page 36), and X2, P2 are similar to X1 and P1,

but reflect terms present in the true response that are not included in the fitting model.

For example, for two variables with a quadratic fitting model, and a cubic true model,

response and its expectation at design point x1 = [xs x2s J are given as


Y, = A1 + A2X, + A3X2 +A 14 + A5X ~X2 + 2 + (3.39)

E(y,) = A1 + A")12 + A31 + A4 X32 + A14 5 X X2 + 2 16X2
+ + (3.40)
2 I 1 2 13 2 2 ]2 2 0 23 jly 2 24 2]

Then, the coefficient vectors (P1 and P2), matrices X1 and X2 are given in Eq. (3.41)

(3.42), and (3.43), respectively.

1 = 1, A12 A13 A14 ]1A5 A16]T (3.41)
2= iA2 g (3.41)
P2 = L[21 022 23 24 T

1 X11 X21 X X11X21 X2
11 11 21 21
12 2
1 X12 X22 X12 X12 X22 X22

X = 2 2 (3.42)
1 xlJ x2 X xJ XlX X2

X2N X 2N
X1 x2N 2 1 X2A 2N











3 2 2 3
x11 11 21 X11X21 21
3 2 2 3
X12 1222 x12 22 X22

X2 3 2 2 3 (3.43)
xJ3 x x2J lJ X2J X2J

3 2 2 3
X1N lXN 2N 2N X2N

The lack of fit test supplies statistical evidence that the fitting model is unable to

account adequately for the variation in the observed response. The test tries to isolate the

portion of the residual variation unaccounted for by the fitted model. For that purpose,

evidence is sought based on estimator of pure error variance that does not depend on the

form of the fitted model required.

An ideal scenario for obtaining such an estimate of pure experimental error

variance is to have n, replications at several design points x,. Therefore, response forth


design point among M distinct design points at its kth replication can now be represented

as


yk= y(x )k = 7(xJ )+ k (3.44)


(wherej=l, M and k=-, n,). The total number of design points is still denoted by N and

equal to

M
N = n (3.45)
J=1

For the fh design point the average yj is an estimate of the true mean response r/(x )

nj
ZYjk
y, k=l (3.46)
n
J


that is equal to the true mean response when n -> oo.










By these estimates, the sum of squares due to pure error SSpE can be calculated as

for the design points

M nj
SSPE = (Jk )2 (3.47)
j=1 k=1

It is compared to the sum of squares due to lack-of-fit, SSLOF using the difference

between the estimate y given by the model and y .


SSLOF = i ,()- 2 (3.48)


Equations (3.47) and (3.48) can also be rewritten in matrix forms

M 1 1T
SSPE nI ni nJ (3.49)
y=l n]

M 11T
SSLOF =yT[IN -XI(XTX1)- 1X]y y ni y (3.50)


where yn is the response vector of n, replications atjth design point, and 1, is a (n, x 1)


vector of ones. Note that Eq. (3.50) is the difference between total sum of squares SSE

[Eq. (3.31)] and pure error sum of squares SSPE [Eq. (3.49)].

Therefore, an F-ratio is formed by using these partitions of residual sum of square


F = LOF dLOF (3.51)
SSP / d
PE PE

where dLOF = M nb and dpE = N M are degrees of freedom associated with SSLOF

and SSpE, respectively. Lack of fit can be detected at the a level of significance if the F-

ratio exceeds the table value, Fa,M-n_,N-M where the latter quantity is the upper 100a

percentile of the central F-distribution (Khuri and Cornell, 1996, p. 38-41). Similar to










hypothesis testing for Eq. (3.32) the p-value is determined as the rightmost tail area under

the distribution FMnb,_, beyond the F statistic value by Eq. (3.51).


Lack-of-Fit with Non-Replicated Design Points

Standard lack-of-fit test described in previous section requires exact replications

at several of the design points. Unlike physical experiments, exact replications by

numerical experiments/simulations do not supply additional information since they result

in exactly the same response when repeated. It is still possible, however, to obtain an

estimator that approximates the notion of pure experimental error (Hart, 1997, p. 123).

The idea is to treat the observations at neighboring design points as "near" replicates.

Then test procedures are based on a pseudo error estimator of the error variance. Neill

and Johnson (1985) generalized the standard pure-error lack of fit test to accommodate

the case of non-replication for first order fitting model. Here, the method is extended to a

higher-order fitting model. The method considers a design point as near replicate design

point Xjk within a distance 6jk from the associated reference point x, subscriptt 'jk'

stands for kth near-replicate around xj)

xjk = X + 6jk (3.52)

where distance or disturbance vector (for a general, n-variable case) is

6jk =[ clk 2j k pjk n. jk]T (3.53)

The monomial terms at design point xjk for the approximation model are

calculated using Eq. (3.52) and (3.53). In a quadratic fitting model, for instance, they are

given as in Eq. (3.54)









XPJ = X2 p + {&k}
x = x, {22p x k + = 2x k +k } (3.54)
XpkX,k = XjX, + {Xfpj ,k + x,Ypjk + cpjkC~jk }

where, r =1, 2,..., n (number of design variables).

The design matrix X1 for the data set augmented by the near-replicate designs is

shown by Eq. (3.55)

X, = X + A (3.55)

where X1 is the design matrix as if the near-replicate designs are replaced by their

associated reference design points xi so that it represents an experimental design with

replications, and A, is the disturbance design matrix that turns X1 into X1. Entries in a

row of the disturbance matrix Al correspond to the fitting model coefficient vector P1 in

Eq. (3.56).

P1 = l1 A2 A3 "' A, "' A,, ]T (3.56)

For example the (j+k)th row of the disturbance matrix corresponding to kth near

replication of the /h design point is given as (assuming th design point is followed by its k

near replicates)

A1 j+k (A (A A)j2 (jk3 (Ajk ), (Ajk) b] (3.57)

For design points that are not near replicates the entries in the associated row are

all equal to zero. The non-zero entries in Eq. (3.57) are formed by the corresponding

monomial terms within the brackets [quadratic fitting model case, Eq. (3.54)].







54



For instance, in a two-variable problem where the data has two near replicates

(k =1,2) for design point x, and the fitting model is quadratic, design matrices,


coefficient vector and disturbance design matrix are given as in the following equations.


1 ,,1 21 11 X1 1 1 21
1 x12 22 <12 12 22 22


1 2 2
X, = xlJ X2J lJ xlj]2] 2 (3.58)
1 Xll X2]j (XlJ1)2 (xljlx2j1) (X2 1)2

1 x12 2j2 (X1-)2 (x2 12 2j2 (XC22)2


1 2 2
XIN X2N X X X X
2N N lN 2N 2N


1 x,,1 1 1 1 1 21 1
1 2 2


1 -2 2
X12 22 X2 x1222 x22
XX = 1 l x2j 1 X X2 2 (3.59)
1 2 2
1 X, X2, X- X1,X], X2]
1 2 2


1 xN x, 2N X2N X1N X2N X 2N


P1 = 11 2 A13 A14 A15 16 ] (3.60)

0 0 0 0 0 0
0 0 0 0 0 0


0 0 0 0 0 0
Am = (3.61)
0 (A 1)12 (A,1)13 (A,1)14 (Aj1)15 (A1)16

0 (A2 )12 (Aj2)13 (A2 )14 (Aj2)15 (Aj2)16


0 0 0 0 0 0


where for the kth near replication non-zero terms in A, are given by Eq. (3.62).
0 0 0 00 0 O









(A jk)12 1 { jk}
(AJk )13 {2jk}
(AJ k)l4 = {2xl Jlk + k } (3.62)
(A k )15 = {Xlj,2k + X2 1,&k + d~lk 2 2jk}

(Ak )16 = 2x2 &2Jk + 2jk

For the data with near replicates, the response vector can now be represented in

the following matrix form after substituting Eq. (3.55) into Eq. (3.37).

y = X p + A p + E (3.63)

Now a corrected response vector, y can be defined by using the fitting model to

project the near-replicate data to the reference points

y = y A i (3.64)

y = X11 + (3.65)

Equation (3.65) allows a standard lack-of-fit test since the design matrix X1 now

is with replicated design points that allow calculating the pure experimental error

variance. The corrected response vector, y, however, is not observable. Neill and

Johnson (1985) proposed to replace it with an observable response vector y as given in

Eq. (3.66) that, at least asymptotically, is comparable to y.

y = y-Alb, (3.66)

The test formalized in this approach (Neill and Johnson, 1985) is given in Eq. (3.67)

H : E(y) = (X + A)p (3.67)
H, :E(y) = (X1 + A))Pl + (X2 + A,2)p

where X2 and A2 are respectively similar to X1 and A1, but corresponding to the

missing term coefficient vector p2. Similar to standard lack-of-fit test, the test statistic in










Eq. (3.51), now denoted as F, is based on sums of square error SSPE and SSLOF [Eqs.

(3.49) and (3.50), respectively], but with y, y, and X1 replaced with y, y5 and X1,


respectively. Neill and Johnson (1985) claimed that F is asymptotically comparable to

the test statistic obtained when replication actually exists.

A five-variable problem was used in order to test the lack-of-fit test with non-

replicated data. First a quadratic function in five-variables is calculated in 126 distinct

design points (coded variables between -1 and +1) that will be used for the five-variable

HSCT wing problem (see Chapter 5 for details of the experimental design). The data is

contaminated with a random, normally distributed noise (o = 3000). Then, a face

centered composite design (FCCD) around the central design with a disturbance size of

0.1 in coded space (42 near replicate designs) was generated, and contaminated results

were calculated similarly. Figure 5 shows the clustering near-replicate designs around

the center design in 3-D case in order to visualize the procedure.


(-0.1 0.1,0.1) 0.1,0. ,0.1)

(-1 1 ,1) 1 ,1) (-0 1,-0 .1 )
(-1,- 1 i '

I I I
cV3 (0.1,0.1 -0.1)


0 CV0 .CV1_.
(-1,-1,-1)0C( 0 (1,-1,-1)



Figure 5: 3-D representation of clustering design points to generate near-replicate
data for lack-of-fit test










The left cubicle represents the whole design space with distinct points (equivalent

to 126 design points in five-variable example) and the right cubicle represents the 42

near-replicates around the center design. The lack-of-fit test for this set of data obtained

from a quadratic function should give no significant indication for the inadequacy of

fitting model. The same approach was repeated by generating data from a cubic function

while keeping the fitting model quadratic. This latter case should indicate significant

statistical evidence for the inadequacy of the fitting model.

Table 3 shows the results of the two tests. The second column presents the results

when the true function is also quadratic polynomial of five variables. For a significance

level of a = 0.05, the calculated statistic F is smaller than the relevant value from the F-

distribution table. That is, there is no significant statistical evidence for rejecting the null

hypothesis; the quadratic fitting model is the true model. The p-value of almost one

indicates that the chance of making an error by rejecting the null hypothesis-a quadratic

model is adequate-is about 100%. In contrast, results with a cubic true function indicate

that there is a statistical evidence to reject the null hypothesis, and the chance of rejecting

it erroneously (p-value) is less than 1%.


Table 3: Verification of the lack-of-fit test with non-replicated design points
Statistical parameters Quadratic true function Cubic true function
Quadratic fitting model Quadratic fitting model
M 126 126
N 168 168
nb 21 21
F 0.6176 2.1530
FO.5, nbM 1.5704 1.5704

p-value 0.9747 0.0030










Iteratively Re-weighted Least Squares

As mentioned in previous section, one of the factors affecting the accuracy of the

RS is noise in the data. While low-amplitude, random noise is filtered well by RS

approximations, data with large errors-outliers-may cause significant loss of accuracy.

An example in one-variable case is shown graphically in Figure 6 where two noisy data

set and true mean response data are compared. Data labeled as "data with outlier" is the

same as the data labeled as "noisy data", but with a larger error at the rightmost design

point. While the fit of the original "noisy data" is very close to true mean response

function, the slope of the function has changed substantially due to the outlier.


9
S true function
8 O
x noisy data
7 0 data with outlier
-fit on noisy data
6 - fit on data with outlier. *



4-

3
5--------- ". --/
5 -I t^J I




2 0


0
0 2 4 6 8 10 12
x
Figure 6: 1-D example of the effect of an outlier in the data


Outliers are atypical (by definition), infrequent observations. In the example,

because of the way in which the regression line is determined (especially the fact that it is

based on minimizing not the sum of simple distances, but the sum of squares of distances










of data points from the line), the outlier has a profound influence on the slope of the

regression line. A single outlier is capable of considerably changing the slope of the

regression line as shown in Figure 6.

Robust regression employs techniques for statistical methods, such as iteratively

re-weighted least square (IRLS) fitting (Holland and Welsch, 1977), to detect and remove

or weight down outliers. Once detected, outliers can be investigated further for possible

mistakes. In order to detect outliers, IRLS was used in the present work. Iteratively re-

weighted least square procedures start with an initial response surface fitted to data and

then assign low weights to points with large errors and refit the data using a weighted

least square procedure. The process is repeated until convergence, and weights of the

response data that do not fit the underlying model (outliers) tend to converge to small

values. This effectively eliminates these points from the fitting process. A graphical

representation of the procedure is shown in Figure 7 for a 1-D case.

1.4 5
[ Data
Data 4.5
1.2- LS
IRLS .4
1. I weighting -3.5 O i
3.5 o)
Outliers 3
0.8- CM
.2.5 )
0.6-

0.4- '1.5 -




0 0.2 0.4 0.6 0.8 1
X
Figure 7: 1-D example of IRLS procedure










In IRLS procedure, the weight w assigned to a data point can be determined by

several weighting functions (Myers and Montgomery, 1995, pp. 667-673). Table 4

presents the standard weighting functions considered in this dissertation and collaborative

research, Kim et al. (2001) focused on efficient IRLS application for results from

optimizations.


Table 4: Standard weighting functions for IRLS
Name W(ej) Range Tuning Constant

Huber's min-max 1 e/s H H=1.0
H s/e I e,/s > H
Beaton and Tukey's biweight [1-( e /B)22 eI < B B=1.0 1.9
0 e/s > B


Bi-weight function by Beaton and Tukey (1974) is preferred to min-max

weighting function by Huber (1964) because it gives zero weight to the outliers and thus

the outliers are clearly detected. The weight w assigned to a data point can be formalized

as


1 l if e,/s w = B (3.68)

10 otherwise

where e, is the residual, see Eq. (3.8), s is the estimate of noise, see Eq. (3.24), and B is

tuning constant, usually 1
used. The shape of the bi-weight function in Figure 8 clearly shows that it penalizes

outliers with zero or low weight. Points of r (= ej / s) within the supports of the bell

shaped bi-weight function are called inliers, and outliers are the points outside the range

of the supports.












1.6
Huber
1.4 Biweight
S Biased
1.2

0 1-

0.8

S 0.6

0.4 -

0.2 -
I U
0 -2 0 4

Residual, r
Figure 8: Weighting functions for IRLS
(Kim et al., 2001)


The weighting function called "Biased" in Figure 8, introduced in Kim et al.

(2001), is an alternative weighting function to standard bi-weight function that takes into

account one-sided errors due to incomplete convergence in minimization problems.

After selecting weighting function the coefficients b,s for IRLS approximation

can be found by solving Eq. (3.69).

XTWX1bi = XTWy (3.69)

where W is a diagonal weighting matrix using the weights from Eq. (3.68)

w(el) 0 .. 0
0 w(e2) "'. :
W = (3.70)

0 ... 0 w(e )_










When the weights are all one, the solution of Eq. (3.69) reduces to ordinary least

square solution. However, in general Eqs. (3.69) and (3.70) are nonlinear equations, and

the iterative IRLS procedure employs Eq.(3.71).

b -b) bz+ X 1 X X1\ '(y-Xb') (3.71)

Normally, the IRLS procedure is used only to eliminate or reduce the effect of

errors by assigning them low weights (Holland and Welsch, 1977) that basically leaves

them out and is referred as robust regression. However, the procedure is used here for the

purpose of identifying outliers and factors causing them so that the repair of erroneous

data can be sought rather than removing the erroneous points.

The JMP (SAS, 1998) statistical software was used in this work for construction

of RS approximations and IRLS procedure.















CHAPTER 4
BIAS AND NOISE ERRORS IN RESPONSE SURFACE APPROXIMATIONS


Applications making use of RS approximations may suffer in regions of design

space where the approximation may be poor. The traditional low-order fitting models

such as quadratic polynomials may not always represent the behavior of complex

engineering systems. On the other hand higher order models are generally impractical to

use in high dimensional problems and designers may have to live with lower-order

models. In this situation, information concerning the design regions where the low-order

models may suffer the most would be valuable. In this chapter a measure that can

provide such information is derived.


Mean Squared Error Criterion

As a measure of the error in the approximation, the mean squared error of

prediction MSEP defined as in Eq. (4.1) is used in this dissertation

MSEP(x) = Ei(x) 7(x)]2 (4.1)

where 77(x) and j(x) are the true mean response and the prediction by the fitted model,

respectively at x.

MSEP is by definition an expected value that would be reached if the number of

data used in approximation were unlimited. Equation (4.1) may be rewritten as given in

Eq. (4.2).









MSEP(x) = EUf(x) Ej(x)]+ [E)(x) (x)]}2
= E[(x)- ()]2 (4.2)
2Effj(x) Ej)(x)][E)(x) 7(x)]}+
E[Ej(x)- 7(x)]2

The cross-product term in the above squaring operation is

EU (x) Ej(x)][Ej(x) 7(x)]}
E (x)j)(x) )(x)77(x)-1
= (E)(x))2 + Ej(x)17(x) (4.3)
= (Ep(x))2 EF(x)77(x) (EJ(x))2 + Ep(x)77(x)
=0

since E7(x) = 7(x) and E(E.(x)) = Ej(x). Therefore

MSEP = E[i(x) Ey(x)]2 + [E(x) 7/(x)]2 (4.4)

The first term in Eq. (4.4) represents the variance error due to random noise [see

Eq. (3.19)] and the second term represents bias error due to inadequate modeling. This

error expression is usually integrated over the design space and the integral is minimized

by choosing the experimental designs that control the effect of one or both types of error

(Khuri and Cornell, 1996, pp. 207-247; Venter and Haftka, 1997). Here instead it is

attempted to characterize the error in the predictions of an RS approximation already

constructed and to determine the design regions where RS prediction may suffer due to

either or both types of error. Therefore Eq. (4.4) is used to investigate the variation of

MSEP from point-to-point.

The expectation of the predicted response at a given design point x can be

expressed as

EF(x) = fTE(bl) (4.5)









where f, is the vector of shape functions f [see Eq. (3.3)] calculated at point x. The

mean and the variation for the coefficient estimates are given as [see Eq.s (3.15) and

(3.21)]

E(b,)= (XX, 1 X1E(y) (4.6)

T -~
Var(b)-= (x )'X Var(y)X (X X, )'
2 tXX ( 1 (4.7)

where a is the standard deviation of the noise error. The first part of the mean squared

error in Eq. (4.4) is equal to the prediction variance at x [due to Eq. (3.19)].

E[-(x) Ej(2 Varli(x)
= fjVar(bl)fi (4.8)
= 2fT (XTXI) f

The second part of Eq. (4.4) is the squared error due to the inadequacy of the

model used

[Ei(x) r/(x)]2 = (Bias[(x)])2 (4.9)

Therefore mean squared error of prediction in Eq. (4.4) can be rewritten as

MSEP = Var[l(x)]+ (Bias[(x)])2 (4.10)

The form of the prediction variance, Var[4(x)], in Eq (4.8) does not change even

in the presence of inadequate modeling, since variance is only caused by random noise.

However, when bias error is present, the residual error mean square s2 is not an unbiased

estimate of a2. Using Eqs. (3.16) and (3.24) we get


* Subscripts 1 and 2 assign matrices and vectors for the fitting model and the missing terms, respectively.










2 yT(N -P)
N-p,


(4.11)


where P = X(XlTX 1 (4.12)

The accuracy of this estimate depends on factors such as the available data or

number of points and also the adequacy of the fitting model that determines whether the

estimate is unbiased or biased.

The true response at x can be written as

17(x)= = l 2 (4.13)
1(X) =f 2 0+f2P, (4.13)

where f2 are terms missing from the assumed model. Since one usually does not know

the true response, it is often assumed to be a higher order polynomial when monomials

are used as shape functions. The mean of the true response is given as

E(y)= X1 P +X2p2 (4.14)

where X2 is similar to X1, but due to the terms (shape functions) present in the true

response that are not included in the fitting model. After substitution of Eq. (4.14), Eq.

(4.6) becomes

E(bl)= 0P +Ap2 (4.15)

whereA = (XTX1)1 XX2 is called the alias matrix. Substitution of Eq. (4.15) into Eq.

(4.5) yields

Ej(x) = flr(p + Ap2) (4.16)

(Bias[(x)])2 ={fT (pl + Ap2) fT fj 2 (4.17)
= pT[ATfc f2 [fTA- fT]p2

The MSEP at a given point can now be estimated as by using s2 instead of 02









MSEP(x)= s2fT(XX1)-f1 +p Mp2 (4.18)

where M = [ATf1 f2 fTA fT] (4.19)

Note that MSEP is an expectation by definition [Eq. (4.1)], and Eq. (4.18) is its

estimate by the data available and associated s2. Using E(y) from Eq. (4.14), the

expected value of biased error mean square s2 given as (Seber, 1977)

2rP X (IN P)X2 2
E(s2) = 2 (N P)
N-pl (4.20)

since (IN P) is an idempotent matrix, that is a matrix whose square is equal to itself. It

is also positive semi-definite matrix, so E(s2) >2 provided that (IN -P)X2P2 0.

Equation (4.20) means that when bias errors are present, s2 is a biased estimate of 072.

The standard lack-of-fit test detects, when significant, the presence of the second terms in

Eqs. (4.18) and (4.20).

If this expected value, E(s2), is substituted in prediction variance contribution in

Eq. (4.18) expected value for the estimate MSEP can be expressed as


E[MSEP(x)] T (X )-'lf 02 + 2TK2 + pM2 I
()N p (4.21)
-2f1T (XTX,)- fi + p Gp2

where K= Xj (X2 XA) (4.22)

flT (XTX )-lfi
G= fT(xTx) K+M (4.23)
N -p,

The aim of this chapter is to identify points where the bias error is large. It is

assumed that the terms missing from the fitting model are known, but there is not enough









data to calculate the corresponding coefficients P2. If one can estimate the size of p2, it

is possible to formulate a constrained maximization problem for the largest magnitude of

the mean squared error predictor that may be experienced at any given design point for

the worst possible P2 of that magnitude.

max E[MSEP(x)]
P2 (4.24)
such that C 2 = C

The Lagrangian for this optimization problem can be written as

L(2, A) = E[MSEP(x)]+ A(p 2 c) (4.25)

Differentiating the Lagrangian with respect to P2

V[o2frT(XX1)-f i+ V(TGi2)+ V T2 -c) 0 (4.26)


where V = a a] Equation (4.26) yields the following
I 3/21 3f22 32 p, .

eigenvalue problem at a design point x ;

G02 +/V2 =0 or

Gp2 GP2 =0 (4.27)

for which the maximum eigenvalue /Gmax characterizes the maximum possible mean

squared and bias error associated with the assumed true model that includes shape

functions missing in the fitting model. The corresponding eigenvector defines the

coefficients of the missing shape functions that results in the largest bias error when fitted

only with the assumed model. The eigenvectors and the function experiencing the worst

possible bias error may be different point-to-point although the magnitude of the missing










coefficient vector is constrained. So the eigenvalue calculated does not reflect the true

function corresponding to the data (as the data is insufficient to calculate P2 ). It reflects

instead, the assumed form of true function with the shape function coefficients P2

(among all the possible combinations such that PIT2 = c2) causing the largest error.


Estimated Standard Prediction Error and Residual Based Error Bound

The eigenvalue measure characterizes the largest possible mean squared error at a

design point for which noise also contributes as well as the insufficient fitting model.

The measure, on the other hand, does not pay any attention to the actual data, and so does

not include actual noise in the data. The matrix G defined in Eq. (4.23) suggests that

associated eigenvalues primarily characterize the error due to missing terms (shape

functions) of the true function and the effect of the experimental design. As a result, the

characterization is qualitative only, and may be used to locate design points of possible

large approximation error.

The success of the eigenvalue measure to pinpoint the locations of high error

regions may be determined by checking its correlation with the residuals. The actual

error at a design point x, associated with the RS approximation y(x,) needs the true

mean response r7(x,) that is not always available even for data points due to noise. For

engineering problems, such as wing structural weight as studied in Chapter 5, the true

function is also not known. In order to test the use of eigenvalue measure for such

problems, another measure based on the available data, approximation and associated

estimated standard prediction error was used. This alternate measure-conservative error









bound-was also used as an approximate tool to quantify uncertainty due to the use of the

RS approximation )(x ).

Recalling MSEP expression in Eq. (4.4) and recognizing the first term as being

the prediction variance, the problem of calculating an estimate of the MSEP at any design

point stands on the second term described in Eq. (4.9) as also repeated in the following

(Bias [y(x )])2 = [E5(x,) r(x, )]2 (4.28)

Equation (4.28) cannot be calculated exactly since neither terms on right hand

side are available. To be able to determine approximately, Eq. (4.28) was replaced by

(Bias (x)])2 = [(x,) y(x )]2 (4.29)

where right hand side is square of residual e, also defined in Eq. (3.8). While y(x ) is

available at any design points, y(x ) is only available at data points, and additional

experiment/simulation is needed for any other design point. Since the alternate error

measure was also intended to be available for any design point, the right hand side was

replaced by an RS approximation fitted on available data of residual es. Two measures

of error are collectively used for the data of residual related RS,

Absolute value of residual relative to predicted response at a design point x,


eR(xJ) = e (4.30)


Equation (4.30) measures the combined effects of modeling and noise error.

Average of the absolute relative residuals in Eq. (4.30)

N
SeR (x )
eR = (4.31)
R N '









where N is number of data points. Equation (4.31) is calculated to compare with error

data obtained by Eq. (4.30) at the design points.

Since the error is not completely characterized by these two measures first the

maximum of the two was used as the residual error measure in order to be more

conservative in modeling uncertainty analysis

eRmax (X,) = max{eR (x ), eR (4.32)

Then an RS approximation eR (x) was constructed for eRmax(x,) to predict the

result of Eq. (4.29) by Eq. (4.33)

(Biask(xj) = [ R(Xj)(xj)]2 (4.33)

An estimate of error at a design point x, was then defined by square-root of an

alternate estimate of MSEP that is sum of prediction variance as first term in Eq. (4.18)

and prediction by Eq. (4.33) at x,


s = AMSEP(x,) = s2f(XXTi)- fl + R(xJ)xJ)]2 (4.34)

Equation (4.34) was treated as a conservative error or residual bound that may be

suffered in the predictions by f(x ), and can be calculated point-to-point.


Polynomial Examples

The eigenvalue measure was derived as a tool to determine the regions of

potential high bias error when the fitting model is missing some of the shape functions of

the true model. For instance, the traditional low-order polynomial fitting models are

widely used since the higher-order models may not be applicable due to insufficient data.

Therefore, the ideal case for demonstrating the use of eigenvalue estimate of bias error is










to check the measure for problems where the true function is a polynomial of a higher

degree than the fitting model used. The true function for the examples was chosen as a

cubic polynomial while the fitting model was kept as quadratic. In other words for each

example, the data were generated by a cubic polynomial and then fitted by quadratic

model. In order to see the effect of noise, data contaminated with random and normally

distributed noise were also studied. Two- and five-variable polynomials were used to

demonstrate the use of the eigenvalue measure for different dimensional problems.


Coefficient of Correlation

As a statistical indication for the success of the measure to pinpoint the high bias

error regions in the design space, coefficient of correlation between the square-root of

eigenvalues calculated and the bias error (or residual calculated as the absolute difference

between RS prediction and true response value) and its graphical representation can be

used. The most widely-used type of correlation coefficient is Pearson r, also called

simple linear correlation. The Pearson coefficient of correlation, r between two variables

zi (e.g. eigenvalues) and z2 (e.g. absolute bias error or residual) is calculated as

S(ziz2),
Z1Z2
r = n (4.35)
1 2

where z1 and z2 are the averages, and oa1 and o'2 are the standard deviations in the set

of values for variables. It is a measure of the linear relation between two variables and

can range from -1 to +1. A value of 0 represents a lack of correlation between the

variables. The value of -1 represents a perfect negative correlation while a value of +1

represents a perfect positive correlation.










The correlation coefficient r represents the linear relationship between two

variables, and if squared, the resulting value r2 is equal to coefficient of determination

[R2 in Eq. (3.34)] as if one treats one of the variables e.g. z2 being fitted as a linear

function of the other, zl. The sample plots for r are shown in Figure 9.


r=0

Z2
S




Z2 Z2a





r < 0 Zl r=-1 Zl
Z1
r>0 r=+1


z2i z2










Figure 9: Linear correlation examples
(Lowry, 2001)



The coefficient of determination r2 represents the proportion of common

variation in the two variables (i.e., the "strength" or "magnitude" of the relationship). On

the other hand the correlation is not intended to explain or explore cause and effect type

relation, and so the fit is not used to predict one from the other. In order to evaluate the

correlation between variables, it is important to know "magnitude" or "strength" as well

as the significance of the correlation that is the significance level at which one can reject

the null hypothesis that there is zero correlation, H : r = 0 and accept that calculated

non-zero correlation is not by pure chance. The t-statistic for this test is given as










t = (4.36)
(1- r2)/(N- 2)

where N is the number of data (or sample size) to check on the correlation between the

variables. It is compared with the t-table value corresponding the significance level, a

and degree of freedom (N-2) as explained in section concerning hypothesis testing (page

43). Thep-value can also be determined from the t-table. The significance orp-value in

this dissertation was calculated by a JAVA script posted on the web by Richard Lowry

(2001). High positive correlation between eigenvalues and residuals indicates that if

designer moves from a design point to another one where eigenvalue is higher, the error

due to approximation may also increase. In other words, eigenvalue measure can warn

and make the designer cautious against inaccurate regions of approximation.


Cubic Polynomial in Two Variables without Noise

First a cubic polynomial in two variables was used as the true model, and a

quadratic polynomial as the fitted model with no noise in the data.

y(x) = (x) = fTl + fT2 (4.37)

where

x = [ x, ]T
f T 3 2 2 ] 1
f X = [x x~2 x x2 x2] (4.38)
P, = A i A2 t A A4 A5 A6 i]T
02 = 21 322 )323 24 i]T

Data for a three-level design as shown in Figure 10 is assumed to be available to

form the design matrices X1 and X2 [Eq.s (3.42) and (3.43), respectively] that is, there is

not enough data points to characterize all ten coefficients.











-1,1 1,1




X1, 2


-1,-1 1,-1


Figure 10: Two-variable three-level factorial design points



Equation (4.27) depends only on the experimental design and the fitting and

assumed true models, not the response data. The eigenvalue problem in Eq. (4.27) with


P2 2 =1 was solved, at 21x21 mesh points over the design region. The maximum

eigenvalues and corresponding eigenvectors were determined. Figure 11 shows a contour

plot of the square-root of the eigenvalues. It is seen that high errors are expected at

center of the boundaries of the design region: (-1,0), (1,0), (0,-1) and (0,1), and low

values at (0,0), (-0.8,-0.8), (-0.8,0.8), (0.8,-0.8) and (0.8,0.8).
1-

2VGmax
0.8289
0.5- 0.8034
0.7778
0.7522
0.7266
S0.7011
0.6755
," C : 0.6499
0.6243
0.5987
0.5732
0.5476
-0.5- 0.5220
S0.4964
0.4709
0.4453


t-T -.5 0'.5
x1
Figure 11: Square-root of maximum eigenvalue VGmax contours from Eq. (4.27)
for a two-variable cubic polynomial, fitted with a quadratic model

(p, =l11 1,12 13 1814 15 J16]T)










Next, a specific example with P = [50 3 5 1 2 1]T was used, and sets of

02 that are the eigenvectors associated with the maximum eigenvalues obtained at nine

data points in Figure 10 were studied. Note that such an eigenvector at a design point

(with P2T2 = 1) defines the cubic polynomial that introduces the largest bias error at that

point, when fitting model is quadratic. Here, without noise, absolute true residual at

design point x,, le1- is the bias error as given in Eq. (4.39)


leT, = 7/(x,) -)(x,) (4.39)

There are four eigenvectors to investigate since the other five are negative of

either one of the four eigenvectors, and since the absolute residuals are identical when

only the sign changes. Since the eigenvalue contours consist of different third order

polynomials instead of a single one, exact agreement cannot be expected for any single

polynomial. Instead, a given polynomial is expected to have a positive correlation

between the absolute true residuals e and the square-root of the maximum eigenvalues

Umax Table 5 shows the polynomial coefficients and the associated coefficient of

correlation between eD where i denoting the polynomial/eigenvector and ma of

Eq. (4.27).

Figure 12 shows the absolute true residual plot for the polynomials studied.

Comparing Figure 11 and Figure 12, it is seen that each polynomial achieves maximal

errors in one of the regions predicted by Figure 11 and at associated design point. For

instance, Polynomial 1 is the eigenvector determined at design points (-1,-1) and (+1,+1),

where errors are high as shown in Figure 12a.








77




Table 5: Coefficient of correlation between the absolute residual e T and the

square-root of maximum eigenvalues mx from Eq. (4.27) for two-variable cubic

polynomial example without noise (j = [50 3 5 1 2 1]T)

i Corresponding design point (X1, x2) f21 022 23 024

1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 0.797

2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.797

3 (-1,0) and (+1,0) 0 0 -1 0 0.406

4 (0,-1) and (0,+1) 0 -1 0 0 0.406

Maximum of four 0.973


-0 E-




a)

(a)


e T

I 04870
04548 o'-
04225
03903
03580
0 3258
0 2935
02613 '
0 2290
01967
0 1645
S01322
01000
00355
0 00032 i 3


06623
06184
05746
0 5307
04868
04430
0 3991
0 3553
03114
0 2675
0 2237
01798
01360
00921
S00482
00044


Vq e^ T, 2

04870
04225
03903
0 3580
0 3258
0 2935
02613
0 2290
0 1967
0 1645
0 1322
00677
0 00355
0 0032


_0'5 015

(b)


eTJ 4

06623
06184
05746
05307
04868
04430
03991
03553
03114
02675
02237
01798
01360
00921
00482
00044


-1 5 ... I- -s 0 1 5'


(c) (d)

Figure 12: Absolute residual ej contours two-variable cubic polynomial

example (polynomials defined in Table 5), without noise. a) Polynomial 1;
b)Polynomial 2; c) Polynomial 3; d) Polynomial 4











Comparison of absolute residual for the polynomials and the square-root of

eigenvalues can also be shown by Figure 13. The correlations in Table 5 and Figure 13

show that even for a single polynomial the eigenvalue distribution can be used to identify

locations of possible high bias error.



0.8
Polynomial 1
0.7 Polynomial 2
A Polynomial 3
Polynomial 4 ,
0.6 ,

0.5- Ml



0.3

0.2 -

0.1 ..

0 -A -. -: ;_
0.4 0.5 0.6 0.7 0.8 0.9

VTGmax

Figure 13: Comparison between square-root of eigenvalues 2max and absolute

residual ej two-variable cubic polynomial example (polynomials defined in Table

5), without noise. Figure indicates that eigenvalues correlate well with maximal
errors. See correlation coefficients in Table 5


When the maximum of the absolute residual among the four polynomials,


e max = max(e1 -, e e e2 3 ) was used, the correlation is 0.973 as reported in


last row of Table 5. With the maximum of all four, contours given in Figure 14.b are

very similar to those of Figure 11 (also repeated as Figure 14.a). High-correlation


r=0.973 between mx and e- max can also be visualized by Figure 15.
max













wMPP"


05-








-05-


0 8289
08034
07522
07266
07011
06755
0 6499
06243
05987
05732
05476
05220
04964
S 04709
S04453


NMw


05-


-0 5-


^eT, max


06184
05746
04868
04430
03991
0 3553
03114
02675
02237
0 1798
01360
00921
00482
00044


V IV__. W,
-1t1 -05 0 05 -i1 -5 0 '5
xl (a) X(b)


Figure 14: Comparison of square-root of eigenvalues Jmax and maximum

absolute residual ej m contours (maximum of residuals among the four

polynomials defined in Table 5) two-variable cubic polynomial example, without
noise. a) Eigenvalue field; b)Maximum true residual field

0.8

0.7

0.6

0.5-








0.1 .

0
0.4 0.5 0.6 0.7 0.8 0.9

vAG max
Figure 15: Comparison between square-root of eigenvalues max and maximum

of true residual errors er, max due to the four polynomial in Table 5 two-variable

cubic polynomial example, without noise. Coefficient of correlation is 0.973



Similar contour plots (Figure 16) were also created for the conservative error


bound s, of each polynomial in Table 5 as defined in Eq. (4.34). It seems that



conservative error bound also shows high error at the design points where eigenvectors








80




were determined to define the four polynomials. For instance, Polynomial 3 is the


eigenvector determined at design points (-1,0) and (+1,0), where residual and


conservative error bound are both high compared to other design points as shown in


Figure 12c and Figure 16c.


1- -r
S1 2

S0 7800 0 7764
05- 07646 05- 07610
07492 0 7455
07338 07301
07184 07146
07029 06991
06875 0 6837
S0- 06721 0- 06682
06567 06528
06413 06373
06258 06219
0 6104 06064
-05- 05950 05909
05796 05755
05641 05600
0 5487 05446

-1 5 0 5 1 1 -1 '- 5 0 5 . .
X, X,
(a) (b)
1- S3 1- S
S S J 4

S07830 07866
05- 07657 07691
07484 07516
07311 07341
07138 07166
0 6965 0 6990
06793 06815
06620 '- 06640
06447 0 6465
06274 0 6290
06101 06115
05928 05940
-0 5- 05755 0 5764
S05582 05589
S05409 05414
0 5236 0 5239


-11 015
X, X,
(c) (d)
Figure 16: Conservative error bound s contours two-variable cubic polynomial

example (polynomials defined in Table 5), without noise. a) Polynomial 1; b)

Polynomial 2; c) Polynomial 3; d) Polynomial 4




Variation of s within each distribution is smaller compared to corresponding



eT but s, is higher than e for all design points. This can be seen better in Figure











17 that indicates s, determined a bound for the error conservatively for no-noise


example although it did not require the true function values.


0.8
Polynomial 1
0.7 Polynomial 2
A Polynomial 3
0.6 Polynomial 4

0.5

eTj, 0.4 -

0.3

0.2 '. .

0.1 -


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
S,
Figure 17: Comparison between conservative error bound and absolute residual

ej two-variable cubic polynomial example (polynomials defined in Table 5),
without noise


Cubic Polynomial in Two Variables with Noise

The two-variable polynomial example presented in the previous section was

repeated, with random, normally distributed noise E of zero mean and standard deviation

o = 0.3 in order to simulate more general problems with both noise and modeling error.

Table 6 summarizes the coefficient of correlations for the four polynomials used also in

Table 5 when noise introduced as given in Eq. (4.40).


y(x) = flT + fTp + (4.40)

Figure 18 shows the absolute true residual contours when noise is present in the

data. Comparison of Figure 12 and Figure 18 shows that the effect of the noise is to shift

slightly the location of high residual regions.








82




Table 6: Coefficient of correlations between the absolute residual e 1, and the

square-root of maximum eigenvalues mx from Eq. (4.27) for two-variable cubic

polynomial example with noise (o =0.3), (P1 = [50 3 5 1 2 ]T)

i Corresponding design point (X1, x2) f21 022 23 P24

1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 0.590

2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.370

3 (-1,0) and (+1,0) 0 0 -1 0 0.482

4 (0,-1) and (0,+1) 0 -1 0 0 0.299

Maximum of four 0.859


1- '


eTj 1

I 08118
07581
07044
06507
05970
05433
04896
04359
03822
03285
02747
S 02210
01673
0 1136
00599
0 0062


" 015
x (a)


e T 2

08988
08394
07800
07206
06611
06017
05423
04829
04235
03641
03046
02452
01858
01264
00670
00076


x (b)


* eIj
07806
07290
06774
06258
05742
05226
04710
04195
03679
03163
02647
02131
01615
01099
00583
00067


er, 4
0 7617
07112
06608
06104
05600
05095
04591
04087
03583
03078
02574
02070
01566
0 1062
00557
00053


-1 -J5 05
x (c)


x (d)


Figure 18: Absolute residual e T contours two-variable cubic polynomial

example (polynomial defined in Table 5 and Table 6), with noise (o = 0.3). a)
Polynomial 1; b) Polynomial 2; c) Polynomial 3; d) Polynomial 4


-1 5


I


-05-











Comparison curves for absolute residual and the square-root of eigenvalues for

the polynomials are shown in Figure 19. Figure 13 and Figure 19 together show that

noise caused the residuals spread more at an eigenvalue level calculated.

1
Polynomial 1
0.9 Polynomial 2
SPolynomial 3
0.8 Polynomial 4
0.7 -
0.6 .
e 1. 0 .

0.4 .
0f 4
0.3 .
O.2 I ; ... .. ,. :?
*... V ...
0.2 : .
0.1-
0 --
0.4 0.5 0.6 0.7 0.8 0.9
2G max
Figure 19: Comparison between square-root of eigenvalues max and absolute

residual e j two-variable cubic polynomial example (polynomials defined in

Table 5), with noise (o = 0.3)



With the maximum of all four absolute residuals, contours given in Figure 20.b

are very similar to eigenvalue contours of Figure 11 (also repeated as Figure 20.a). The

slight shift on the locations of maximum residual errors can also been seen in Figure

20.b. This shift is attributed to the noise in the data.


Coefficient of correlation r=0.859 between and er max is still high.


Figure 21 shows the linear correlation between those two quantities and the increased

spread due to the noise effect compared to Figure 15. In spite of the cloudy data it

reflects the benefit of eigenvalue measure since high residuals are mostly mapped

qualitatively by the magnitude of the eigenvalues.


















w


05-









-0 5-




-_1 i5 0L o 5


S0 8289
08034 05-
0 7778
0 7522

07011
0 6755
0 6499 -
0 6243
05987
05732
0 5476
05220 -05-
04964
04709
0 4453

-1 I 5 015


(a) (b)

Figure 20: Comparison of square-root of eigenvalues max and maximum

absolute residual e max contours (maximum of residuals among the four

polynomials defined in Table 6) two-variable cubic polynomial example, with
noise (o = 0.3). a) Eigenvalue field; b) Maximum residual field




1


0.9

0.8

0.7

0.6

eg 0.5
Smax 0.5

0.4

0.3

0.2

0.1

0


S'




Ii:


., ,
I

*
*3a *
,?.. '
* s1-


0.4 0.5 0.6 0.7 0.8 0.9

AG max

Figure 21: Comparison between square-root of eigenvalues max and maximum

of residual errors e-, max due to the four polynomial in Table 5 two-variable

polynomial example, with noise (o = 0.3). Coefficient of correlation is 0.859


eJ max
08990
08417
07843
07270
06697
06123
05550
04976
04403
03829
03256
02683
S2109
0 1536
0 0962
0 0389









The absolute residuals in Figure 20 are due also to noise, so they should also be

compared to the estimated standard error e,, (square root of prediction variance). Table

7 summarizes the coefficient of correlation between the estimated standard error and the

absolute residuals for four polynomials.


Table 7: Coefficients of correlation between the estimated standard error and
absolute true residuals e-,1 for cubic polynomial example with noise (o = 0.3),

(P1 =[50 3 5 1 2 1]T)
i Corresponding design point (x1, x2) i21 Pi22 23 24 S r
1 (-1,-1) and (+1,+1) 0 0.707 0.707 0 1.127 -0.043
2 (-1,+1) and (+1,-1) 0 -0.707 0.707 0 0.543 0.363
3 (-1,0) and (+1,0) 0 0 -1 0 0.793 0.020
4 (0,-1) and (0,+1) 0 -1 0 0 0.171 0.433
Maximum of four 0.126


Figure 22 shows the field of estimated standard error [square root of Eq. (4.8)]

normalized by a-. The root-mean-square-error estimator s for a is different for each

polynomial as reported in Table 7. The distribution of the estimated standard error of the

Polynomial i ee, can be pictured as Figure 22 scaled by relevant s from Table 7. The

coefficient of correlation between the estimated standard error and the maximum

eigenvalues is only 0.097 for all polynomials in this example as both distributions depend

only on the experimental design

Similar to previous example without noise, Figure 23 shows s, determining


error bound conservatively for residuals er The effect of noise on the conservative


representation of erT, by sj can be seen by the comparison of Figure 17 and Figure 23.








86




The different magnitudes of biased estimates s for a from Table 7 caused s, being



conservative at different levels. Error bound s 1 (Polynomial 1), for instance is the most



conservative if used to determine residuals for Polynomial 1 e1 .


-p.


05-


S0-


O 8956
08759
08562
08364
08167
07970
07773
07576
07379
07182
06985
06788
06591
06394
06196
05999


-05-


-1 -d ,A5
Xi
x

Figure 22: Normalized estimated standard error e, / a for a two-variable quadratic

polynomial model with experimental design of Figure 10


1.2
SPolynomial 1
D Polynomial 2
1 Polynomial 3
Polynomial 4

0.8 "



as.
.. *


0.4 -; .


0.2 -
0 -.


0 0.2 0.4 0.6 0.8 1 1.2



Figure 23: Comparison between conservative error bound sJ and absolute residual


eT two-variable cubic polynomial example (polynomials defined in Table 7),

with noise (o = 0.3)