<%BANNER%>

Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling

Permanent Link: http://ufdc.ufl.edu/UFE0041248/00001

Material Information

Title: Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling
Physical Description: 1 online resource (161 p.)
Language: english
Creator: Picheny, Victor
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: conservativeness, design, metamodel, prediction, reliability
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
Genre: Aerospace Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In most engineering fields, numerical simulators are used to model complex phenomena and obtain high-fidelity analysis. Despite the growth of computer capabilities, such simulators are limited by their computational cost. Surrogate modeling is a popular method to limit the computational expense. It consists of replacing the expensive model by a simpler model (surrogate) fitted to a few chosen simulations at a set of points called a design of experiments (DoE). By definition, a surrogate model contains uncertainties, since it is an approximation to an unknown function. A surrogate inherits uncertainties from two main sources: uncertainty in the observations (when they are noisy), and uncertainty due to finite sample. One of the major challenges in surrogate modeling consists of controlling and compensating for these uncertainties. Two classical frameworks of surrogate application are used as a discussion thread for this research: constrained optimization and reliability analysis. In this work, we propose alternatives to compensate for the surrogate model errors in order to obtain safe predictions with minimal impact on the accuracy. The methods are based on different error estimation techniques, some based on statistical assumptions and some that are non-parametric. Their efficiency are analyzed for general prediction and for the approximation of reliability measures. We also propose two contributions to the field of design of experiments in order to minimize the uncertainty of surrogate models. Firstly, we address the issue of choosing the experiments when surrogates are used for reliability assessment and constrained optimization. Secondly, we propose global sampling strategies to answer the issue of allocating limited computational resource in the context of RBDO. All methods are supported by quantitative results on simple numerical examples and engineering applications.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Victor Picheny.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Haftka, Raphael T.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0041248:00001

Permanent Link: http://ufdc.ufl.edu/UFE0041248/00001

Material Information

Title: Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling
Physical Description: 1 online resource (161 p.)
Language: english
Creator: Picheny, Victor
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: conservativeness, design, metamodel, prediction, reliability
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
Genre: Aerospace Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In most engineering fields, numerical simulators are used to model complex phenomena and obtain high-fidelity analysis. Despite the growth of computer capabilities, such simulators are limited by their computational cost. Surrogate modeling is a popular method to limit the computational expense. It consists of replacing the expensive model by a simpler model (surrogate) fitted to a few chosen simulations at a set of points called a design of experiments (DoE). By definition, a surrogate model contains uncertainties, since it is an approximation to an unknown function. A surrogate inherits uncertainties from two main sources: uncertainty in the observations (when they are noisy), and uncertainty due to finite sample. One of the major challenges in surrogate modeling consists of controlling and compensating for these uncertainties. Two classical frameworks of surrogate application are used as a discussion thread for this research: constrained optimization and reliability analysis. In this work, we propose alternatives to compensate for the surrogate model errors in order to obtain safe predictions with minimal impact on the accuracy. The methods are based on different error estimation techniques, some based on statistical assumptions and some that are non-parametric. Their efficiency are analyzed for general prediction and for the approximation of reliability measures. We also propose two contributions to the field of design of experiments in order to minimize the uncertainty of surrogate models. Firstly, we address the issue of choosing the experiments when surrogates are used for reliability assessment and constrained optimization. Secondly, we propose global sampling strategies to answer the issue of allocating limited computational resource in the context of RBDO. All methods are supported by quantitative results on simple numerical examples and engineering applications.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Victor Picheny.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Haftka, Raphael T.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0041248:00001


This item has the following downloads:


Full Text

PAGE 1

IMPROVINGACCURACYANDCOMPENSATINGFORUNCERTAINTYINSURROGATEMODELINGByVICTORPICHENYADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2009 1

PAGE 2

c2009VictorPicheny 2

PAGE 3

ACKNOWLEDGMENTS Firstofall,IwouldliketothankmyadvisorsDr.RaphaelHaftka,Dr.Nam-HoKimandDr.OlivierRoustant.Theytaughtmehowtoconductresearch,gavemepricelessadvicesandsupport,andprovidedmewithexcellentguidancewhilegivingmethefreedomtodevelopmyownideas.Ifeelveryfortunatetohaveworkedundertheirguidance.Iwouldalsoliketothankthemembersofmyadvisorycommittee,Dr.BertrandIooss,Dr.GrigoriPanasenko,Dr.JorgPeters,Dr.AlainVautrinandDr.TonySchmitz.Iamgratefulfortheirwillingnesstoserveonmycommittee,forreviewingmydissertationandforprovidingconstructivecriticismthathelpedmetoenhanceandcompletethiswork.IparticularlythankDr.IoossandDr.Schmitzfortheirworkasrapporteursofmydissertation.Iamgratefulformanypeoplethatcontributedscienticallytothisdissertation:Dr.NestorQueipo,whowelcomedmeformyinternshipattheUniversityofZulia(Venezuela)andprovidedmewiththoughtfulideasandadvices;Dr.RodolpheLeRiche,withwhomIdevelopedafruitfulcollaborationonthestudyofgearboxproblems;Dr.XavierBay,whoseencyclopedicknowledgeofmathematicsallowedustondpromisingresultsondesignoptimality;Dr.FelipeViana,forhisecientcontributiontotheconservativesurrogateproblem;andDr.DavidGinsbourger,forhiscriticalinputonthetargeteddesignswork...andhiscontagiousmotivation.Iwishtothanktoallmycolleaguesfortheirfriendshipandsupport,fromtheStructuralandMultidisciplinaryOptimizationGroupoftheUniversityofFlorida,theAppliedComputingInstituteoftheUniversityofZulia,andthe3MIdepartmentfromtheEcoledesMines:Tushar,Amit,Palani,Erdem,Jaco,Ben,Christian,Felipe,Alex,Camila;Eric,Delphine,Celine,Bertrand,David,Nicolas,Olga,Natacha,Khouloud,Ksenia,andmanyothers...IalsothankallthestaoftheUniversityofFloridaandEcoledesMines 3

PAGE 4

thatmadepossiblemyjointPhDprogram,andinparticularPamandKaren,whohelpedmemanytimes.Financialsupports,providedpartlybyNationalScienceFondation(Grant#0423280)formytimeinFloridaandbytheCETIMfoundationformytimeinFrance,aregratefullyacknowledged.Mynalthoughtsgotomyfriends,family,andNatacha. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................. 3 LISTOFTABLES ..................................... 9 LISTOFFIGURES .................................... 10 LISTOFABBREVIATIONS ............................... 13 ABSTRACT ........................................ 14 CHAPTER 1INTRODUCTION .................................. 16 2ELEMENTSOFSURROGATEMODELING ................... 22 2.1SurrogateModels ................................ 22 2.1.1NotationAndConcepts ......................... 22 2.1.2TheLinearRegressionModel ...................... 23 2.1.3TheKrigingModel ........................... 25 2.1.3.1Krigingwithnoise-freeobservations ............. 27 2.1.3.2Krigingwithnuggeteect .................. 29 2.2DesignOfExperimentStrategies ....................... 30 2.2.1ClassicalAndSpace-FillingDesigns .................. 30 2.2.2Model-OrientedDesigns ......................... 32 2.2.3AdaptiveDesigns ............................ 34 3CONSERVATIVEPREDICTIONSUSINGSURROGATEMODELING .... 36 3.1Motivation .................................... 36 3.2DesignOfConservativePredictors ....................... 37 3.2.1DenitionOfConservativePredictors ................. 37 3.2.2MetricsForConservativenessAndAccuracy ............. 38 3.2.3ConstantSafetyMarginUsingCross-ValidationTechniques ..... 40 3.2.4PointwiseSafetyMarginBasedOnErrorDistribution ........ 41 3.2.5PointwiseSafetyMarginUsingBootstrap ............... 42 3.2.5.1TheBootstrapprinciple ................... 42 3.2.5.2Generatingcondenceintervalsforregression ....... 43 3.2.6AlternativeMethods .......................... 46 3.3CaseStudies ................................... 46 3.3.1TestProblems .............................. 46 3.3.1.1TheBranin-Hoofunction .................. 46 3.3.1.2Thetorquearmmodel .................... 47 3.3.2NumericalProcedure .......................... 49 3.3.2.1Graphsofperformance .................... 49 5

PAGE 6

3.3.2.2Numericalset-upfortheBranin-Hoofunction ....... 49 3.3.2.3Numericalset-upfortheTorquearmmodel ........ 50 3.4ResultsAndDiscussion ............................. 51 3.4.1Branin-HooFunction .......................... 51 3.4.1.1Analysisofunbiasedsurrogatemodels ........... 51 3.4.1.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates ........................ 51 3.4.1.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates ........................... 55 3.4.2TorqueArmModel ........................... 58 3.4.2.1Analysisofunbiasedsurrogatemodels ........... 58 3.4.2.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates ........................ 58 3.4.2.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates ........................... 61 3.5ConcludingComments ............................. 62 4CONSERVATIVEPREDICTIONSFORRELIABILITY-BASEDDESIGN ... 63 4.1IntroductionAndScope ............................ 63 4.2EstimationOfProbabilityOfFailureFromSamples ............. 64 4.2.1Limit-StateAndProbabilityOfFailure ................ 64 4.2.2EstimationOfDistributionParameters ................ 66 4.3ConservativeEstimatesUsingConstraints .................. 68 4.4ConservativeEstimatesUsingTheBootstrapMethod ............ 70 4.5AccuracyAndConservativenessOfEstimatesForNormalDistribution .. 73 4.6EectOfSampleSizesAndProbabilityOfFailureOnEstimatesQuality 75 4.7ApplicationToACompositePanelUnderThermalLoading ........ 78 4.7.1ProblemDenition ........................... 78 4.7.2Reliability-BasedOptimizationProblem ................ 80 4.7.3ReliabilityBasedOptimizationUsingConservativeEstimates .... 82 4.7.4OptimizationResults .......................... 84 4.8ConcludingComments ............................. 86 5DESIGNOFEXPERIMENTSFORTARGETREGIONAPPROXIMATION 88 5.1MotivationAndBibliography ......................... 88 5.2ATargetedIMSECriterion .......................... 90 5.2.1TargetRegionDenedByAnIndicatorFunction ........... 90 5.2.2TargetRegionDenedByAGaussianDensity ............ 93 5.2.3Illustration ................................ 94 5.3SequentialStrategiesForSelectingExperiments ............... 95 5.4PracticalIssues ................................. 97 5.4.1SolvingTheOptimizationProblem .................. 97 5.4.2Parallelization .............................. 99 5.5ApplicationToProbabilityOfFailureEstimation .............. 99 6

PAGE 7

5.6NumericalExamples .............................. 101 5.6.1Two-DimensionalExample ....................... 101 5.6.2Six-DimensionalExample ........................ 103 5.6.3ReliabilityExample ........................... 104 5.7ConcludingRemarks .............................. 106 6OPTIMALALLOCATIONOFRESOURCEFORSURROGATEMODELING 109 6.1SimulatorsWithTunableFidelity ....................... 109 6.2ExamplesOfSimulatorsWithTunableFidelity ............... 110 6.2.1Monte-CarloBasedSimulators ..................... 110 6.2.2RepeatableExperiments ........................ 111 6.2.3FiniteElementAnalysis ........................ 112 6.3OptimalAllocationOfResource ........................ 113 6.4ApplicationToRegression ........................... 114 6.4.1ContinuousNormalizedDesigns .................... 114 6.4.2SomeImportantResultsOfOptimalDesigns ............. 116 6.4.3AnIllustrationOfAD-OptimalDesign ................ 117 6.4.4AnIterativeProcedureForConstructingD-OptimalDesigns .... 119 6.4.5ConcludingRemarks .......................... 120 6.5ApplicationToKriging ............................. 121 6.5.1ContextAndNotations ......................... 121 6.5.2AnExploratoryStudyOfTheAsymptoticProblem ......... 122 6.5.3AsymptoticPredictionVarianceAndIMSE .............. 123 6.5.3.1Generalresult ......................... 124 6.5.3.2Adirectapplication:space-llingdesigns .......... 125 6.5.4Examples ................................. 126 6.5.4.1Brownianmotion ....................... 126 6.5.4.2Orstein-Uhlenbeckprocess .................. 128 6.5.5ConcludingComments ......................... 129 7CONCLUSION .................................... 131 7.1SummaryAndLearnings ............................ 131 7.2Perspectives ................................... 133 APPENDIX AALTERNATIVESFORCONSERVATIVEPREDICTIONS ........... 135 A.1BiasedFittingEstimators ........................... 135 A.1.1BiasedFittingModels .......................... 135 A.1.2ResultsAndComparisonToOtherMethods ............. 136 A.2IndicatorKriging ................................ 137 A.2.1DescriptionOfTheModel ....................... 137 A.2.2ApplicationToTheTorqueArmAnalysis ............... 139 7

PAGE 8

BDERIVATIONOFTHEASYMPTOTICKRIGINGVARIANCE ........ 141 B.1KernelsOfFiniteDimension .......................... 141 B.2GeneralCase .................................. 146 B.3TheSpace-FillingCase ............................. 150 CSPECTRALDECOMPOSITIONOFTHEORSTEIN-UHLENBECKCOVARIANCEFUNCTION ...................................... 152 REFERENCES ....................................... 155 BIOGRAPHICALSKETCH ................................ 161 8

PAGE 9

LISTOFTABLES Table page 2-1SequentialDoEminimizingthemaximumpredictionvarianceateachstep. ... 34 3-1Rangeofthedesignvariables(cm). ......................... 47 3-2Meanvaluesofstatisticsbasedon1,024testpointsand500DoEsfortheunbiasedsurrogatesbasedon17observations. ........................ 51 3-3Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. ......... 58 4-1Comparisonofthemean,standarddeviation,andprobabilityoffailureofthethreedierentCDFestimatorsforN()]TJ /F1 11.955 Tf 9.29 0 Td[(2:33;1:02) ................. 70 4-2MeansandcondenceintervalsofdierentestimatesofP(G0)andcorrespondingvalueswhereGisthenormalrandomvariableN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02). ......... 74 4-3MechanicalpropertiesofIM600/133material. ................... 79 4-4Deterministicoptimafoundby Quetal. ( 2003 ). .................. 80 4-5Coecientsofvariationoftherandomvariables. .................. 81 4-6Variablerangeforresponsesurface. ......................... 82 4-7StatisticsofthePRSforthereliabilityindexbasedontheunbiasedandconservativedatasets. ....................................... 84 4-8StatisticsofthePRSbasedon22testpoints .................... 84 4-9Optimaldesignsofthedeterministicandprobabilisticproblemswithunbiasedandconservativedatasets. ............................. 85 5-1ProcedureoftheIMSET-basedsequentialDoEstrategy. ............. 96 5-2ProbabilityoffailureestimatesforthethreeDoEsandtheactualfunctionbasedon107MCS. ..................................... 106 6-1IMSEvaluesforthetwoKrigingmodelsandtheasymptoticmodel. ....... 128 A-1Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. ......... 139 9

PAGE 10

LISTOFFIGURES Figure page 2-1ExampleofSimpleKrigingmodel. ......................... 28 2-2ExampleofSimpleKrigingmodelwithnoisyobservations. ............ 29 2-3Examplesoffull-factorialandcentralcompositedesigns. ............. 30 2-4Anine-pointLHSdesign. .............................. 31 3-1Schematicrepresentationofbootstrapping. ..................... 43 3-2Percentileoftheerrordistributioninterpolatedfrom100valuesofu. ...... 45 3-3Branin-Hoofunction. ................................. 46 3-4InitialdesignoftheTorquearm. .......................... 47 3-5Designvariablesusedtomodifytheshape. ..................... 48 3-6VonMisesstresscontourplotattheinitialdesign. ................. 48 3-7AverageresultsforCSMandEDforKrigingbasedon17points. ......... 52 3-8AverageresultsforCSMandEDforPRSbasedon17points. .......... 52 3-9AverageresultsforCSMandEDforKrigingbasedon34points. ......... 53 3-10Targetvs.actual%cforPRSusingCSMwithcross-validationandEDbasedon17points. ..................................... 54 3-11Targetvs.actual%cforKrigingusingCSMwithcross-validationandEDbasedon17points. ..................................... 55 3-12Condenceintervals(CI)computedusingclassicalregressionandbootstrapwhentheerrorfollowsalognormaldistribution. ..................... 56 3-13AverageresultsforBSandEDforPRSbasedon17points. ........... 57 3-14Targetvs.actual%cforKrigingusingCSMandcross-validationandEDbasedon17points. ..................................... 57 3-15AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSonthetorquearmdata. ....................................... 59 3-16AverageresultsforCSMandEDforKrigingonthetorquearmdata. ...... 59 3-17Targetvs.actual%cforPRSusingCSMandcross-validationandEDonthetorquearmdata. ................................... 60 10

PAGE 11

3-18Targetvs.actual%cforKrigingusingCSMandcross-validationandEDonthetorquearmdata. ................................. 60 3-19AverageresultsforBSandEDforPRSonthetorquearmdata. ......... 61 3-20Targetvs.actual%cforPRSusingBSandEDonthetorquearmdata. .... 61 4-1CriteriatotempiricalCDFs. ............................ 68 4-2ExampleofCDFestimatorsbasedonRMSerrorforasampleofsize10generatedfromN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02) ................................. 70 4-3ConservativeestimatorsofPffrombootstrapdistribution:95thpercentile(p95)andmeanofthe10%highestvalues(CVaR). .................... 72 4-4Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforNormaldistribution. ................................. 77 4-5Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforlognormaldistribution. ................................ 77 4-6Geometryandloadingofthecryogeniclaminate. ................. 79 5-1One-dimensionalillustrationofthetargetregion. ................. 91 5-2Illustrationoftheweightsfunctions. ........................ 95 5-3Optimaldesignafter11iterations. ......................... 102 5-4EvolutionofKrigingtargetcontourlinecomparedtoactualduringthesequentialprocess. ........................................ 103 5-5ComparisonoferrordistributionfortheoptimalandLHSDoEs. ......... 105 5-6BoxplotsoferrorsfortheLHSandoptimaldesignsforthetestpointswhereresponsesareinsidethedomains[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+" ..................... 106 5-7Optimaldesignswithuniformintegrationmeasureandwithinputdistributionintegrationmeasure. ................................. 107 6-1EvolutionoftheresponseoftheFEAforthetorquearmexample,whenthecomplexityofthemodel(meshdensity)increases. ................. 113 6-2D-optimaldesignforasecond-orderpolynomialregressionandcorrespondingpredictionvariance. .................................. 118 6-3Full-factorialdesignwithhomogeneousallocationofcomputationaleortandcorrespondingpredictionvariance. .......................... 119 6-436-pointFull-factorialdesignwithhomogeneousallocationofcomputationaleortandcorrespondingpredictionvariance. .................... 119 11

PAGE 12

6-5EvolutionoftheIMSEwithrespecttothenumberofobservationsforuniformrandomdesignswithGaussianandExponentialcovariances. ........... 123 6-6IllustrationoftheMSEfunctionforaBrownianmotion(2=1,2=0:002). .. 127 6-7IllustrationoftheMSEfunctionforaGaussianprocess(GP)withexponentialcovariancefunction(2=1,=0:2and2=0:002). ............... 130 A-1AverageresultsforBiasedttingwithconstraintrelaxationandconstraintselectionontheBranin-Hoofunction. ............................. 137 A-2AverageresultsforCSMandBiasedttingwithconstraintselection. ...... 138 A-3AverageresultsforIndicatorKrigingandEDwithKriging. ............ 140 A-4Targetvs.actual%cforIK(plainblack)andKrigingwithED(mixedgray). .. 140 C-1Representationofthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2u2)sin(u). ...... 154 12

PAGE 13

LISTOFABBREVIATIONS BS BootstrapCDF CumulativeDistributionFunctionCEC ConservativetotheEmpiricalCurveCSM ConstantSafetyMarginCSP ConservativeatSamplePointsDoE DesignofExperimentsED ErrorDistributioneRMS RootMeanSquareErrorl LossinaccuracyLHS LatinHypercubeSamplingMaxUE MaximumUnconservativeErrorMaxUE% ReductionoftheMaximumUnconservativeErrorMCS Monte-CarloSimulationsMSE MeanSquareErrorPDF ProbabilityDensityFunctionPf ProbabilityoffailurePRS PolynomialResponseSurfaceRBDO Reliability-BasedDesignOptimization%c Percentageofconservativeresults 13

PAGE 14

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyIMPROVINGACCURACYANDCOMPENSATINGFORUNCERTAINTYINSURROGATEMODELINGByVictorPichenyDecember2009Chair:RaphaelT.HaftkaMajor:AerospaceEngineering Inmostengineeringelds,numericalsimulatorsareusedtomodelcomplexphenomenaandobtainhigh-delityanalysis.Despitethegrowthofcomputercapabilities,suchsimulatorsarelimitedbytheircomputationalcost.Surrogatemodelingisapopularmethodtolimitthecomputationalexpense.Itconsistsofreplacingtheexpensivemodelbyasimplermodel(surrogate)ttedtoafewchosensimulationsatasetofpointscalledadesignofexperiments(DoE). Bydenition,asurrogatemodelcontainsuncertainties,sinceitisanapproximationtoanunknownfunction.Asurrogateinheritsuncertaintiesfromtwomainsources:uncertaintyintheobservations(whentheyarenoisy),anduncertaintyduetonitesample.Oneofthemajorchallengesinsurrogatemodelingconsistsofcontrollingandcompensatingfortheseuncertainties.Twoclassicalframeworksofsurrogateapplicationareusedasadiscussionthreadforthisresearch:constrainedoptimizationandreliabilityanalysis. Inthiswork,weproposealternativestocompensateforthesurrogatemodelerrorsinordertoobtainsafepredictionswithminimalimpactontheaccuracy.Themethodsarebasedondierenterrorestimationtechniques,somebasedonstatisticalassumptionsandsomethatarenon-parametric.Theireciencyareanalyzedforgeneralpredictionandfortheapproximationofreliabilitymeasures. 14

PAGE 15

Wealsoproposetwocontributionstotheeldofdesignofexperimentsinordertominimizetheuncertaintyofsurrogatemodels.Firstly,weaddresstheissueofchoosingtheexperimentswhensurrogatesareusedforreliabilityassessmentandconstrainedoptimization.Secondly,weproposeglobalsamplingstrategiestoanswertheissueofallocatinglimitedcomputationalresourceinthecontextofRBDO. Allmethodsaresupportedbyquantitativeresultsonsimplenumericalexamplesandengineeringapplications. 15

PAGE 16

CHAPTER1INTRODUCTION Inthepastdecades,scientistshavebenetedfromthedevelopmentofnumericaltoolsforthehelpoflearning,predictionanddesign.Thegrowthofcomputationalcapabilitieshasallowedtheconsiderationofphenomenaofconstantlyincreasinglevelofcomplexity,whichresultsinbetterunderstandingandmoreecientsolutionsofreal-lifeproblems. Complexnumericalsimulatorscanbeencounteredinawiderangeofengineeringelds.Theautomotiveindustryhasdevelopednumericalmodelsforthebehaviorofcarsduringcrash-tests,whichinvolvehighlynon-linearmechanicalmodes,inordertointegrateittothedesignofcarstructures.Ingeophysics,owsimulatorsareusedforthepredictionofthebehaviorofCO2sequestrationintonaturalreservoirs,orthepredictionofoilrecoveryenhancement,basedonacomplexmappingofgroundcharacteristics. Theincreasingcomputationalpowerhasalsoallowedtheincorporationofuncertaintyinformationintosystemanalysis.Inparticular,reliabilityanalysisaimsatquantifyingthechancethatasystembehavesasitisrequired,whenasetoftheproblemparametersareuncertain[ Rackwitz ( 2000 ), Haldar&Mahadevan ( 2000 )].Instructuralanalysis,uncertaintytypicallycomesfrommaterialproperties(duetomanufacturing)andactualworkingconditions.Forobviousreasons,reliabilityassessmenthasbeenintensivelyexploredinaerospaceandnuclearengineering. However,thegaininrealismmakestheuseofnumericalmodelsextremelychallenging.Indeed,thecomplexityofthesimulatorsmakesthemhighlyexpensivecomputationally,limitingsystematiclearningordesignprocess.Also,thenumberofparameters(orinputvariables)thatexplainthephenomenonofinterestcanpotentiallybeverylarge,makingdicultthequanticationoftheirinuenceonthesimulatorresponse. Toovercomecomputationalcostlimitations,surrogatemodels,ormetamodels,havebeenfrequentlyusedonmanyapplications.Theideaofsurrogatemodelsconsistsofreplacingtheexpensivemodelbyasimplermathematicalmodel(orsurrogate)ttedtoa 16

PAGE 17

fewchosensimulationsatasetofpointscalledadesignofexperiments(DoE).Surrogatemodelsarethenusedtopredictthesimulatorresponsewithverylimitedcomputationalcost[ Box&Draper ( 1986 ), Myers&Montgomery ( 1995 ), Santneretal. ( 2003 ), Sacksetal. ( 1989a )]. Bydenition,asurrogatemodelcontainsuncertainties,sinceitisanapproximationtoanunknownfunction.Asurrogateinheritsuncertaintiesfromtwomainsources:uncertaintyintheobservations(whentheyarenoisy),anduncertaintyduetothelackofdata.Indeed,asimulatorisanapproximationtoarealphenomenon,andthecondenceonecanputinitsresponsesdependsonthequalityofthenumericalmodel.Thepropertiesofthesurrogatestronglydependontheaccuracyoftheobservations.Secondly,inmostapplications,thenumberofsimulationsrunsisseverelylimitedbythecomputationalcapabilities.Theinformationonwhichthesurrogateisttedisinsucienttoobtainanaccurateapproximationofthesimulatorbehavior.Oneofthemajorchallengesinsurrogatemodelingconsistsofcontrollingandcompensatingfortheseuncertainties. Thepresentworkconsidersasadiscussionthreadtheclassicalframeworkofreliability-baseddesignoptimization(RBDO),forwhichtheissuesassociatedwithuncertaintyinsurrogatemodelingareparticularlycrucial.RBDOisapopularwaytoaddressuncertaintyinthedesignprocessofasystem.Itconsistsofoptimizingaperformancefunctionwhileensuringaprescribedreliabilitylevel.RBDOproblemsarecomputationallychallenging,sincetheyrequirenumerouscallstoreliabilitymeasuresthataremostofthetimeexpensivetoestimate,hencebeinganaturalcandidateforsurrogateapplication[ Rajashekhar&Ellingwood ( 1993 ), Venteretal. ( 1998 ), Kurtaranetal. ( 2002 )].SurrogatemodelingcanbeusedattwolevelsoftheRBDO:(1)duringtheoptimizationprocess,byapproximatingthereliabilityconstraint,and(2)duringthereliabilityestimationitself. 17

PAGE 18

ThechallengesinusingsurrogatemodelinginRBDOarevarious.Indeed,apoorapproximationofthereliabilitylevelcanseverelyharmtheoptimizationprocess:overestimationleadstonon-optimaldesigns,andunderestimationtoweakdesigns.Reliabilityassessmentmethodsaregenerallybasedonsamplingmethods(suchasMonte-Carlosimulations)thatonlyprovideestimatesoftheactualreliabilitylevels.Quanticationoftheuncertaintyonbothreliabilityestimatesandasurrogatebasedonsuchdataisneededtolimittheriskofpoordesigns. Ramuetal. ( 2007 )showthattheuseofsurrogatesforreliabilityassessmentisparticularlychallenging,sincesmallerrorsinthemodelcouldresultinlargeerrorsinthereliabilitymeasure.Inaddition,forcomputationalreasons,thenumberofreliabilityestimatesislimitedtoasmallvalue,andecientsamplingstrategiesmustbeusedtoensureanacceptableaccuracyofthesurrogate. Itispossibletocompensateforthelackofaccuracywithextrasafetymargins.Suchapproachisoftenreferredasconservative,andaconservativenesslevelquantifythechancethatanapproximationisonthesafesidefortheanalysis.Forexample, StarnesJr&Haftka ( 1979 )replacedthelinearTaylorseriesapproximationwithatangentapproximationbiasedtobeconservativeinordertoreducethechanceofunconservativeapproximationstobucklingloads.Manyengineeringapplicationshaveadoptedconservativeestimation.Forinstance,FederalAviationAdministration(FAA)denesconservativematerialproperty(A-basisandB-basis)asthevalueofamaterialpropertyexceededby99%(forA-basis,90%forB-basis)ofthepopulationwith95%condence.FAArecommendstheuseofA-basisformaterialpropertiesandasafetyfactorof1.5ontheloads.Traditionally,safetyfactorshavebeenextensivelyusedtoaccountforuncertainties,eventhoughtheireectivenessisquestionable[ Elishako ( 2004 ), Acaretal. ( 2007 )]. Estimatingtheuncertaintyindataorthemetamodeliscrucialfortheeectivenessofmostsurrogate-basedapproaches.Uncertaintyanderrorquanticationisaclassicalthemeofsurrogatemodeling.Mostmetamodelscontainbyconstructionerrorestimates[ Cressie 18

PAGE 19

( 1993 ), Isaaks&Srivastava ( 1989 ), Box&Draper ( 1986 )].Theseestimatesarebasedonstatisticalassumptions;forinstance,linearregressionassumesnormalityandindependanceoftheerrors.Inpractice,suchassumptionscanbeviolated,makingthoseerrormeasuresquestionable.Alternatively,numericaltechniquesareavailabletoquantifytheerror[ Stine ( 1985 ), Efron ( 1982 ), Goeletal. ( 2006 )]. Uncertaintyquanticationallowscompensatingmechanisms,forinstanceinordertosettheconservativenesstoaprescribedlevel.However,littleinformationisavailableontheeectoferrorcompensationonthesurrogateapplication.Furthermore,conservativeestimatesarebiasedtobeonthesafeside,sotheconservativenesscomesatapriceofaccuracy.Onegoalofpresentworkistoproposealternativestocompensateforthesurrogatemodelerrors,basedondierenterrorestimationtechniques,anddemonstratetheireciencyforsafepredictionanddesignofengineeringsystems. Inmanyapplications,samplingstrategiescanbechosenbytheuser.Then,experimentscanbedesignedsuchthattheuncertaintyofsurrogatemodelsareminimized.Whensafepredictionisdesired,reduceduncertaintyallowstoobtainconservativenesswithaminimalimpactonaccuracy.Thechoiceandeectivenessofthesamplingstrategieshasbeenwidelyexploredanddiscussedinthesurrogateliterature[ Steinberg&Hunter ( 1984 ), Sacksetal. ( 1989b ), Fedorov&Hackl ( 1997 )].Thisworkproposestwocontributionstothiseld.Firstly,weaddresstheissueofchoosingtheexperimentswhensurrogatesareusedforreliabilityassessmentandconstrainedoptimization.Secondly,weproposeglobalsamplingstrategiestoanswertheissueofallocatinglimitedcomputationalresourceinthecontextofRBDO. Outlineofthedissertation: Chapter2reviewsseveralaspectsofsurrogatemodeling. Chapter3addressestheissueofgeneratingconservativepredictionsusingsurrogatemodels.Twoalternativesareproposed:(1)usingstatisticalinformationprovidedbythesurrogatemodeland(2)usingmodel-independenttechniques(bootstrap,cross-validation) 19

PAGE 20

toobtainstatisticalinformation.Dierentmetricsaredenedtomeasurethetrade-obetweenaccuracyandsafety.Theanalysisofthedierenttechniquesissupportedwiththehelpofananalyticalandastructuralproblemthatusesniteelementsanalysis. Chapter4considerstheapplicationofsurrogatemodelingtoreliability-baseddesign,wheresurrogatesareusedtotprobabilitydistributions.WefocusonthecasewhenlowprobabilitiesareestimatedfromasmallnumberofobservationsobtainedbyMonte-Carlosimulations(MCS).Byusingbiased-ttingtechniquesandresamplingmethods(bootstrap),wecompensatefortheuncertaintyintheanalysisbybeingontheconservativeside,withreasonableimpactontheaccuracyoftheresponse.Anapplicationtotheoptimizationofalaminatecompositewithreliabilityconstraintsisusedfordemonstration,theconstraintbeingapproximatedconservativelyusingbootstrap. InChapter5,weproposeanobjective-basedapproachtosurrogatemodeling,basedontheideathattheuncertaintymaybereducedwhereitismostuseful,insteadofglobally.Anoriginalcriterionisproposedtochoosesequentiallythedesignofexperiments,whenthesurrogateneedstobeaccurateforcertainlevelsofthesimulatorresponse.Thecriterionisatrade-obetweenthereductionofoveralluncertaintyinthesurrogate,andtheuncertaintyreductionintargetregions.Theeectivenessofthemethodisillustratedonasimplereliabilityanalysisapplication. Chapter6considerstheframeworkofsimulatorswhosedelitydependsontunablefactorsthatcontrolthecomplexityofthemodel(suchasMCS-basedsimulators,orRBDOframework).Foreachsimulationrun,theuserhastosetatrade-obetweencomputationalcostandresponseprecision.WhenaglobalcomputationalbudgetfortheDoEisgiven,onemayhavetoanswerthefollowingquestions:(1)isitbettertorunafewaccuratesimulationsoralargenumberofinaccurateones,(2)isitpossibletoimprovethesurrogatebytuningdierentdelitiesforeachrun.Answersareproposedforthesetwoquestions.Forpolynomialregression,itjoinsthewell-exploredtheoryofdesignoptimality. 20

PAGE 21

Forkriging,bothnumericalandanalyticalresultsareproposed;inparticular,asymptoticresultsaregivenwhenthenumberofsimulationrunstendstoinnity. Chapter7recapitulatesmajorconclusionsandresults,anddrawperspectivesfromthiswork. 21

PAGE 22

CHAPTER2ELEMENTSOFSURROGATEMODELING Inthischapter,weintroducetheimportantnotionsrelativetosurrogatemodeling.Wedetailtwotypesofmodels:linearregressionmodels,andKrigingmodels.Wealsodescribetheproblematicofdesignofexperiments,andpresentseveralpopularsamplingstrategies. 2.1SurrogateModels Strategiesinvolvingsurrogatemodelingarerecognizedinawiderangeofengineeringeldstoecientlyaddresscomputationallyexpensiveproblems.Theaimofthissectionistobrieypresentthenotationandmainstepsinsurrogatemodeling.Amongthenumeroustypesofsurrogatemodelsavailableintheliterature,weemphasizetwoofthemostpopularones:linearregressionandKriging.Linearregression,alsoreferredaspolynomialresponsesurfaceintheengineeringliterature,hasbeeninitiallydevelopedforstatisticalinferencebasedonphysicalexperiments.Krigingwasdevelopedbygeostatisticianstomodelspatialcorrelationsofthephysicalcharacteristicsofground.Bothmodelsarenowusedinmanyapplications,eventhoughthehypothesesonwhichthemodelsarebasedarenotnecessarilyguaranteed. Amajornotioninsurrogatemodelingisthedesignofexperiments,orsamplingstrategy,sinceitiscrucialtothequalityofthesurrogateanalysis.Inthischapter,webrieydescribethreeclassicalsamplingstrategies:space-lling,model-baseddesignsandadaptivedesigns.MoreadvancednotionsareaddressedinChapter5foradaptivedesignsandChapter6formodel-orientedoptimaldesigns. 2.1.1NotationAndConcepts Letusrstintroducesomenotation.Wedenotebyytheresponseofanumericalsimulatororfunctionthatistobestudied: y:DRd)166(!Rx7)166(!y(x)(2{1) 22

PAGE 23

wherex=fx1;:::;xdgTisad-dimensionalvectorofinputvariables,andDisthedesignspace.Inordertobuildametamodel,theresponseyisobservedatndistinctlocationsX: X=[x1;:::;xn]Yobs=[y(x1);:::;y(xn)]T=y(X)(2{2) Xiscalledthedesignofexperiments(DoE),andYobsistheobservations.Sincetheresponseyisexpensivetoevaluate,weapproximateitbyasimplemodel,calledthemetamodelorsurrogatemodel,basedonassumptionsonthenatureofyandonitsobservationsYobsatthepointsoftheDoE. Themetamodelcaninterpolatethedata(splines,Kriging)orapproximateit(linearregression,Krigingwithnuggeteect).Inthelattercase,itisassumedthatthefunctionofinterestisobservedthroughanoisyprocess,sotheobservationdiersfromthetruefunctionbyanadditiveerrorterm: yobs;i=y(xi)+"i(2{3) with"itheerror.Inmostofthemetamodelhypotheses,theerrorisconsideredasawhitenoisenormallydistributedwithzeromean.Inthissection,wewillalwaysconsiderthishypothesistrue. 2.1.2TheLinearRegressionModel Inlinearregression,theresponseismodeledasalinearcombinationofbasisfunctionsobservedwithanadditiveerrorterm: y(x)=pXj=1jfj(x) (2{4) yobs;i=pXj=1jfj(x)+"i (2{5) wherefj(x)arethebasisfunctions(forinstancepolynomial),jtheweights,and"itheerroratxi. 23

PAGE 24

Givenasetofdesignpoints,thelinearregressionmodel,inmatrixnotation,isdenedasfollow: Yobs=fT(X)+"(2{6) where:F=266666664f(x1)Tf(x2)T...f(xn)T377777775=26666666412...p377777775"=266666664"1"2..."p377777775and:f(x)T=f1(x)f2(x):::fp(x). Typically,theficanbechosenaspolynomialfunctions;inthatcasethemodelisoftencalledpolynomialresponsesurface(PRS). Sinceinpracticetheerrorisunknown,weestimatetheresponseby: ^y(x)=f(x)T^(2{7) where^isanestimateofandischosenbyminimizingthemeansquareerror(MSE)betweentheestimatesandtheactualresponsesatalldesignpoints: MSE=1 nnXi=1[^y((xi)))]TJ /F4 11.955 Tf 11.96 0 Td[(y(xi)]2(2{8) Usingthestandardlinearregression,thevalueof^thatminimizestheMSEisgivenby: ^=)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(FTF)]TJ /F5 7.97 Tf 6.59 0 Td[(1FTYobs (2{9) ^=M)]TJ /F5 7.97 Tf 6.58 0 Td[(1FTYobs (2{10) ThequantityM=FTFiscalledtheFisherinformationmatrix. Inadditiontothebestpredictor^y,thelinearregressionmodelprovidesapredictionvariance,givenby: var[^y(x)]=f(x)TM)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)(2{11) Whentheerrorisheteroskedastic,thatis,whentheerrordistributiondierfromonepointtoanother,theordinaryleastsquareestimatorofisnotappropriate.Instead,we 24

PAGE 25

estimatethecoecientsbythegeneralizedleastsquareestimator: =)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(FT)]TJ /F11 7.97 Tf 8.08 5.05 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1F)]TJ /F5 7.97 Tf 6.59 0 Td[(1FT)]TJ /F11 7.97 Tf 8.09 5.05 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1Yobs(2{12) with:)]TJ /F1 11.955 Tf 11.4 0 Td[(=[cov(y(xi);y(xj))]1i;jn.Inthatcase,theFisherinformationmatrixis: M=FT)]TJ /F11 7.97 Tf 8.09 5.04 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1F(2{13) Notethatiftheerrorsareuncorrelated,)]TJ /F1 11.955 Tf 11.98 0 Td[(reducestodiag[var("1);var("2);:::;var("n)]. Forcalculationdetails,seeforinstance Box&Draper ( 1986 ), Khuri&Cornell ( 1996 )or Myers&Montgomery ( 1995 ). 2.1.3TheKrigingModel TheKrigingmetamodelwasinitiallydevelopedinthegeostatisticframework[ Matheron ( 1969 ), Cressie ( 1993 )]topredictvaluesbasedonspatialcorrelationconsiderations.KrigingcanalsobefoundontheliteratureunderthenameofGaussianProcessregression[ Rasmussen&Williams ( 2006 )],orregressionwithspatiallycorrelatederrors[ Fedorov&Hackl ( 1997 )].ThemainhypothesisbehindtheKrigingmodelistoassumethatthetruefunctionyisonerealizationofaGaussianstochasticprocessY: y(x)=Y(x;!)(2{14) where!belongstotheunderlyingprobabilityspace.InthefollowingweusethenotationY(x)fortheprocessandy(x)foronerealization.ForUniversalKriging,Yisoftheform: Y(x)=pXj=1jfj(x)+Z(x)(2{15) wherefjarelinearlyindependentknownfunctions,andZisaGaussianprocesswithzeromeanandcovariancek(u;v). Thecovariancefunction(orkernel)kcontainsalltheinformationofspatialdependency,anddependsonparameters.Thereexistsmanytypesofcovariancefunctions;twoofthemostpopularonesaretheisotropicgaussianandexponential 25

PAGE 26

covariances: IsotropicGaussiancovariance:k(u;v)=2exp")]TJ /F9 11.955 Tf 11.29 16.86 Td[(ku)]TJ /F7 11.955 Tf 11.95 0 Td[(vk 2# (2{16) IsotropicExponentialcovariance:k(u;v)=2exp)]TJ /F9 11.955 Tf 11.29 16.85 Td[(ku)]TJ /F7 11.955 Tf 11.96 0 Td[(vk (2{17) Forthesecovariances,theparametersaretheprocessvariance2andrange.Anisotropiccovariancefunctionscanalsobedenedbyattributingadierentineachdirection: AnisotropicGaussiancovariance:k(u;v)=2exp")]TJ /F6 7.97 Tf 17.75 14.94 Td[(dXj=1juj)]TJ /F7 11.955 Tf 11.96 0 Td[(vjj j2#(2{18) InChapter3,wealsousetherationalquadraticcovariancefunction[ Rasmussen&Williams ( 2006 )]: k(u;v)= 1+ku)]TJ /F7 11.955 Tf 11.96 0 Td[(vk2 2l2!)]TJ /F6 7.97 Tf 6.59 0 Td[((2{19) with=f;lg. Inthegeostatisticliterature,threeterminologiesareused,dependingonthelinearpartconsidered: simpleKriging(SK):thelinearpartreducestoaknownconstant1 ordinaryKriging(OK):theconstant1isunknown universalKriging(UK)isthegeneralcase. Theparametersareusuallyunknownandmustbeestimatedbasedontheobservations,usingmaximumlikelihood,cross-validationorvariogramtechniquesforinstance[see Rasmussen&Williams ( 2006 ), Stein ( 1999 )or Cressie ( 1993 )].However,intheKrigingmodeltheyareconsideredasknown.Toaccountforadditionalvariabilityduetotheparameterestimation,onemayuseBayesianKrigingmodels[see Martin&Simpson ( 2004 )and Oakley&O'Hagan ( 2004 )],whichwillnotbedetailedhere. 26

PAGE 27

2.1.3.1Krigingwithnoise-freeobservations First,weconsiderthemostclassicalframework,wherethefunctionisobservedwithoutnoise: yobs;i=y(xi)(2{20) Undersuchhypothesis,thebestlinearunbiasedpredictor(BLUP)fory,knowingtheobservationsYobs,isgivenbythefollowingequation: mK(x)=E[Y(x)jY(X)=Yobs]=f(x)T^+c(x)TC)]TJ /F5 7.97 Tf 6.59 0 Td[(1Yobs)]TJ /F7 11.955 Tf 11.96 0 Td[(F^(2{21) where: f(x)=f1(x):::fp(x)Tisp1vectorofbases, ^=^1:::^pTisp1vectorofestimatesof, c(x)=cov(x;x1):::cov(x;xn)Tisn1vectorofcovariance, C=[cov(xi;xj)]1i;jnisnncovariancematrix,and F=f(x1):::f(xn)Tispnmatrixofbases. ^isthevectorofgeneralizedleastsquareestimatesof: ^=)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(FTC)]TJ /F5 7.97 Tf 6.58 0 Td[(1F)]TJ /F5 7.97 Tf 6.58 0 Td[(1FTC)]TJ /F5 7.97 Tf 6.59 0 Td[(1Yobs(2{22) Inaddition,theKrigingmodelprovidesanestimateoftheaccuracyofthemeanpredictor,theKrigingpredictionvariance: s2K(x)=k(x;x))]TJ /F7 11.955 Tf 9.3 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.58 0 Td[(1c(x)+)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(f(x)T)]TJ /F7 11.955 Tf 11.95 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.59 0 Td[(1F)]TJ /F7 11.955 Tf 12.95 -9.69 Td[(FTC)]TJ /F5 7.97 Tf 6.58 0 Td[(1F)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(f(x)T)]TJ /F7 11.955 Tf 11.96 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.58 0 Td[(1FT(2{23) NotethattheKrigingvariancedoesnotdependontheobservationsYobs,butonlyonthedesignofexperiments.Derivationdetailscanbefoundin Matheron ( 1969 ), Cressie ( 1993 ),or Rasmussen&Williams ( 2006 ).WedenotebyM(x)theGaussianprocessconditionalon 27

PAGE 28

theobservationsYobs: (M(x))x2D=(Y(x)jY(X)=Yobs)x2D=(Y(x)jobs)x2D(2{24) TheKrigingmodelprovidesthedistributionofMatapredictionpointx: M(x)N)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(mK(x);s2K(x)(2{25) Figure 2-1 showsaKrigingmodelwitharst-ordertrend(=1,simpleKriging)andveequally-spacedobservationsalongwiththe95%condenceintervals,whicharecalculatedfrommK1:96sK.Onthisexample,thecondenceintervalcontainstheactualresponse. Figure2-1. ExampleofSimpleKrigingmodel.The95%condenceintervals(CI)areequaltomK1:96sK.TheDoEconsistsofvepointsequallyspacedin[0,1]. TheKrigingmeanmKinterpolatesthefunctiony(x)atthedesignofexperimentpoints: mK(xi)=y(xi);1in(2{26) TheKrigingvarianceisnullattheobservationpointsxi,andgreaterthanzeroelsewhere: s2K(xi)=0;1inands2K(x)0;x6=xi(2{27) 28

PAGE 29

Besides,theKrigingvariancefunctionincreaseswiththedistanceofxtotheobservations. 2.1.3.2Krigingwithnuggeteect Whennoisyobservationsareconsidered,adiagonalmatrixmustbeaddedtothecovariancematrix: C=C+(2{28) with:=diag([var("1);var("2);:::;var("n)])thevariancesoftheobservations. EquationsformKands2KarethesameasinEqs. 2{21 and 2{23 butusingCinsteadofC.Fortheoreticaldetailsandbibliography,seeforinstance Ginsbourger ( 2009 )(Chap.7).ThemaindierencewiththeclassicalKrigingisthatbestpredictorisnotaninterpolatoranymore;also,theKrigingvarianceisnon-nullattheobservationpoints.Figure 2-2 showsaKrigingmodelbasedonnoisyobservations.Eachobservationhasadierentnoisevariance. Figure2-2. ExampleofSimpleKrigingmodelwithnoisyobservations.Thebarsrepresenttwotimesthestandarddeviationofthenoise.TheKrigingmeandoesnotinterpolatethedataandtheKrigingvarianceisnon-nullattheobservationpoints. 29

PAGE 30

2.2DesignOfExperimentStrategies ChoosingthesetofexperimentsXplaysacriticalroleintheaccuracyofthemetamodelandthesubsequentuseofthemetamodelforprediction,learningoroptimization.Inthissection,wedetailthreefamiliesofdesignofexperiments:classicalandspace-llingdesigns,model-oriented(oroptimal)designs,andadaptivedesigns. 2.2.1ClassicalAndSpace-FillingDesigns TherstfamilyofDoEconsistsofdesignsbasedongeometricconsiderations.Inafull-factorial(FF)design,thevariablesarediscretizedintoanitenumberoflevels,andthedesignconsistsofallthepossiblecombinationsofthediscretevariables.Two-levelFFdesigns,wheretheinputvariablesaretakenonlyattheirminimumandmaximumvalues,aretypicallyusedforscreeninginordertoidentifythemostsignicantvariablesandremovetheothers.Althoughextensivelyusedhistorically,suchtypeofDoEssuerfromseveraldrawbacksamongthefollowing: itrequiresalargenumberofobservationsinhighdimensions(aFFdesignwithqlevelsinddimensionsismadeofqdobservations),makingthemimpracticalforcomputationallyexpensiveproblems theydonotensurespace-llinginhighdimensions thenumberofobservationpointscollapseswhenprojectingonthesubspaces(whensomevariablesareremovedforinstance). Figure2-3. Examplesoffull-factorialandcentralcompositedesigns. ThereexistsclassicalalternativestoFFdesigns,suchascentral-compositedesigns(thatconsistof2dvertices,2daxialpointsandprepetitionsofcentralpoint),orfractional 30

PAGE 31

designs(thataresubsetsofFFdesigns).Seeforinstance Myers&Montgomery ( 1995 )fordetails.Figure 2-3 showsathree-levelFFdesignandacentralcompositedesignforatwo-dimensionaldomain. ApopularalternativetothegeometricaldesignsisLatinHypercubesampling(LHS)[ McKayetal. ( 2000 )].LHSisarandomDoEthatinsuresuniformityofthemarginaldistributionsoftheinputvariables. TogenerateaDoEofnpoints,eachvariablerangeisdividedintonequalintervals,forinstancefortherange[0;1]:[0;1 n];[1 n;2 n];:::;[n)]TJ /F5 7.97 Tf 6.59 0 Td[(1 n;1].Then,theDoEisconstructedbypickingnpointssothatalltheintervalsforeachvariableisrepresentedonce.Figure 2-4 showsanine-pointLHSdesignforatwo-dimensionaldomain.Inad-dimensionalspace,thetotalnumberofcombinationsisnd)]TJ /F5 7.97 Tf 6.58 0 Td[(1,whichincreasesveryrapidlywithbothsamplesizeanddimension.Thevariablevaluescanbechosendeterministicallyasthecentreoftheintervalforinstance,orrandomlyinsidetheinterval. LHScanalsobeoptimizedtoensurebetterspace-lling.Space-llingcriteriainclude: maximumminimumdistancebetweensamplingpoints(maximindesigns), maximum-entropydesigns[ Shewry&Wynn ( 1987 )], minimumdiscrepancymeasures,etc. Figure2-4. Anine-pointLHSdesign.Theprojectionsonthemarginsarenonredudantandequally-spacedifthedesignpointsarecenteredineachcell. 31

PAGE 32

Finally,low-discrepancysequencesarealsooftenusedforspace-llingstrategies,suchasSobol,Halton,Harmmersley,NiederreitterorFauresequences.See Niederreiter ( 1992 )or Sobol ( 1976 )fordetails. 2.2.2Model-OrientedDesigns Theprevioussamplingstrategieswerebuiltindependentlyofthemetamodeltobettedtotheobservations.Alternatively,whenthechoiceofthemetamodelismadeapriori,itispossibletochoosetheobservationpointsinordertomaximizethequalityofstatisticalinference.Thetheoryofoptimaldesignshasoriginallybeendevelopedintheframeworkoflinearregression,andwasextendedlatertoothermodelssuchasKrigingsincethe80's. LetbeafunctionalofinteresttominimizethatdependsonthedesignofexperimentsX.AdesignXiscalled-optimalifitachieves: X=argmin[(X)](2{29) A-andD-optimalityaimatminimizingtheuncertaintyintheparametersofthemetamodelwhenuncertaintyisduetonoisyobservations.Intheframeworkoflinearregression,D-optimaldesignsminimizethevolumeofthecondenceellipsoidofthecoecients,whileA-optimaldesignsminimizeitsperimeter(orsurfaceford>2).Formally,theA-andD-optimalitycriteriaare,respectively,thetraceanddeterminantofFisher'sinformationmatrix[ Khuri&Cornell ( 1996 )]: A(X)=trace(M(X)) (2{30) D(X)=det(M(X)) (2{31) Inlinearregression,thesecriteriaareparticularlyrelevantsinceminimizingtheuncertaintyinthecoecientsalsominimizestheuncertaintyintheprediction(accordingtotheD-Gequivalencetheorem,whichisdevelopedfurtherinChapter6). 32

PAGE 33

ForKriging,uncertaintiesincovarianceparametersandpredictionarenotsimplyrelated.Anaturalalternativeistotakeadvantageofthepredictionvarianceassociatedwiththemetamodel.ThepredictionvarianceallowsustobuildmeasuresthatreecttheoverallaccuracyoftheKriging.Twodierentcriteriaareavailable:theintegratedmeansquareerror(IMSE)andmaximummeansquareerror(MMSE)[ Santneretal. ( 2003 ), Sacksetal. ( 1989a )]: IMSE=ZDMSE(x)d(x) (2{32) MMSE=maxx2D[MSE(x)] (2{33) (x)isanintegrationmeasure(usuallyuniform)and MSE(x)=E(y(x))]TJ /F4 11.955 Tf 11.96 0 Td[(M(x))2obs](2{34) WhenthefunctiontoapproximateisarealizationofagaussianprocesswithcovariancestructureandparametersequaltotheoneschosenforKriging,theMSEcoincideswiththepredictionvariances2K.NotethattheabovecriteriaareoftencalledI-criterionandG-criterion,respectively,intheregressionframework.TheIMSEisameasureoftheaverageaccuracyofthemetamodel,whiletheMMSEmeasurestheriskoflargeerrorinprediction.Inpractice,theIMSEcriterionessentiallyreectsthespatialdistributionoftheobservations.Fordimensionshigherthantwo,theMMSEisnotveryrelevantsincetheregionsofmaximumuncertaintyarealwayssituatedontheboundaries,andMMSE-optimaldesignsconsistofobservationstakenontheedgesofthedomainonly,whichisnotecientforKriging. Optimaldesignsaremodel-dependent,inthesensethattheoptimalitycriterionisdeterminedbythechoiceofthemetamodel.Inregression,A-andD-criteriadependonthechoiceofthebasisfunctions,whileinKrigingthepredictionvariancedependsonthelineartrend,thecovariancestructure,andparametervalues.Afundamentalresult,however,isthatassumingaparticularmodelstructure(andcovarianceparametersfor 33

PAGE 34

Kriging),noneofthecriteriadependsontheresponsevaluesatthedesignpoints.TheconsequenceisthattheDoEscanbedesignedoinebeforegeneratingtheobservations.However,inpracticethisstatementhastobeenmoderated,sincetheKrigingcovarianceparametersaremostofthetimeestimatedfromtheobservations. 2.2.3AdaptiveDesigns ThepreviousDoEstrategieschooseallthepointsofthedesignbeforecomputinganyobservation.ItisalsopossibletobuildtheDoEsequentially,bychoosinganewpointasafunctionoftheotherpointsandtheircorrespondingresponsevalues.Suchapproachhasreceivedconsiderableattentionfromtheengineeringandmathematicalstatisticcommunities,foritsadvantagesofexibilityandadaptabilityoverothermethods[ Linetal. ( 2008 ), Scheidt ( 2006 ), Turneretal. ( 2003 )]. Typically,thenewpointachievesamaximumonsomecriterion.Forinstance,asequentialDoEcanbebuiltbymakingateachstepanewobservationwherethepredictionvarianceismaximal.ThealgorithmicprocedureisdetailedinTable 2-1 Table2-1. SequentialDoEminimizingthemaximumpredictionvarianceateachstep. X=fx1;:::;xngfori=1tokxnew=argmaxx2Ds2K(x)X=fX;xnewgend Sacksetal. ( 1989b )usesthisstrategyasaheuristictobuildIMSE-optimaldesignsforKriging.Theoretically,thissequentialstrategyislessecientthatthedirectminimizationoftheIMSEcriterion.Theadvantageofsequentialstrategyhereistwofold.Firstly,itiscomputational,sinceittransformsanoptimizationproblemofdimensionnd(fortheIMSEminimization)intokoptimizationsofdimensiond.Secondly,itallowsustoreevaluatethecovarianceparametersaftereachobservation.Inthesamefashion, Williamsetal. ( 2000 ), Currinetal. ( 1991 ), Santneretal. ( 2003 )useaBayesianapproachtoderivesequentialIMSEdesigns. Osio&Amon ( 1996 )proposedamultistageapproachtoenhance 34

PAGE 35

rstspace-llinginordertoaccuratelyestimatetheKrigingcovarianceparametersandthenrenetheDoEbyreducingthemodeluncertainty.Somereviewsofadaptivesamplinginengineeringdesigncanbefoundin Jinetal. ( 2002 ). Ingeneral,aparticularadvantageofsequentialstrategiesoverotherDoEsisthattheycanintegratetheinformationgivenbytherstkobservationvaluestochoosethe(k+1)thtrainingpoint,forinstancebyreevaluatingtheKrigingcovarianceparameters.Itisalsopossibletodeneresponse-dependentcriteria,withnaturallyleadstosurrogate-basedoptimization.OneofthemostfamousadaptivestrategyistheEGOalgorithm( Jonesetal. ( 1998 )),usedtoderivesequentialdesignsfortheoptimizationofdeterministicsimulationmodels.Ateachstep,thenewobservationischosentomaximizetheexpectedimprovement,afunctionalthatrepresentsacompromisebetweenexplorationofunknownregionsandlocalsearch: EI(x)=E(max[0;min(Yobs))]TJ /F4 11.955 Tf 11.95 0 Td[(M(x)])(2{35) whereM(x)istheKrigingmodelasdescribedinEq. 2{25 Kleijnen&VanBeers ( 2004 )proposedanapplication-drivenadaptivestrategyusingcriteriabasedonresponsevalues. Tu&Barton ( 1997 )usedamodiedD-optimalstrategyforboundary-focusedpolynomialregression.InChapter5,wedetailanoriginaladaptivedesignstrategyfortargetregionapproximation. 35

PAGE 36

CHAPTER3CONSERVATIVEPREDICTIONSUSINGSURROGATEMODELING 3.1Motivation Inanalyzingengineeringsystems,multiplesourcesoferror-suchasmodelingerrororinsucientdata-preventusfromtakingtheanalysisresultsatfacevalue.Conservativepredictionisasimplewaytoaccountforuncertaintiesanderrorsinasystemanalysis,byusingcalculationsorapproximationsthattendtosafelyestimatetheresponseofasystem.Traditionally,safetyfactorshavebeenextensivelyusedforthatpurpose,eventhoughtheireectivenessisquestionable[ Elishako ( 2004 ), Acaretal. ( 2007 )]. Whensurrogatemodelsareusedforpredictingcriticalquantities,verylittleisknowninpracticetoprovideconservativeestimates,andhowconservativestrategiesimpactthedesignprocess.Mostsurrogatesaredesignedsothatthereisa50%chancethatthepredictionwillbehigherthantherealvalue.Theobjectiveofthisworkistoproposealternativestomodifythetraditionalsurrogatessothatthispercentageispushedtotheconservativesidewiththeleastimpactonaccuracy. Sinceconservativesurrogatestendtooverestimatetheactualresponse,thereisatrade-obetweenaccuracyandconservativeness.Thechoiceofsuchtrade-odeterminesthebalancebetweentheriskofoverdesignandtheriskofweakdesign.Thedesignofconservativesurrogatescanbeconsideredasabi-objectiveoptimizationproblem,andresultsarepresentedintheformofParetofronts:accuracyversusconservativeness. Themostclassicalconservativestrategyistobiasthepredictionresponsebyamultiplicativeoradditiveconstant.Suchapproachesarecalledempiricalbecausethechoiceoftheconstantissomehowarbitraryandbasedonpreviousknowledgeoftheengineeringproblemconsidered.However,itisdiculttopredicthowecientitsapplicationistosurrogates,andhowitispossibletodesignthosequantities.Alternatively,itispossibletousethestatisticalknowledgefromthesurrogatetting(predictionvariance)tobuildone-sidedcondenceintervalsontheprediction. 36

PAGE 37

Forthisstudy,weconsidertwotypesofsurrogatemodels:polynomialresponsesurfaces(PRS)andKriging.Weproposethreealternativestoprovideconservativeestimates:usingcross-validationtodeneconstantmargins,usingcondenceintervalsgivenbythesurrogatemodel,andusingthebootstrapmethod.Themethodsdierinthesensethatoneassumesaparticulardistributionoftheerror,whiletheothers(cross-validationandbootstrap)doesnot.Dierentmetricsaredenedtoanalyzethetrade-obetweenaccuracyandconservativeness.Themethodsareappliedtotwotestproblems:oneanalyticalandonebasedonFiniteElementAnalysis. 3.2DesignOfConservativePredictors 3.2.1DenitionOfConservativePredictors Withoutanylossofgenerality,weassumeherethataconservativeestimatorisanestimatorthatdoesnotunderestimatetheactualvalue.Hence,aconservativeestimatorscanbeobtainedbysimplyaddingapositivesafetymargintotheunbiasedestimator: ^ySM(x)=^y(x)+Sm(3{1) Alternatively,itispossibletodeneaconservativeestimatorusingsafetyfactors,whicharemorecommonlyusedinengineering: ^ySF(x)=^y(x)Sf(3{2) Inthecontextofsurrogatemodeling,safetymarginsaremoreconvenient.Indeed,whenusingasafetyfactor,thelevelofsafetydependsontheresponsevalue,whilemostsurrogatesmodelassumethattheerrorisindependentofthemeanoftheresponse.Hence,inthisworkweuseonlysafetymarginstodeneaconservativeestimator. Thevalueofthemargindirectlyimpactsthelevelofsafetyandtheerrorintheestimation.Hence,thereisagreatincentivetondthemarginthatensureatargetedlevelofsafetywithleastimpactontheaccuracyoftheestimates.Thesafetymargincanbeconstant,ordependonthelocation,hencewrittenSm(x)(pointwisemargin).Inthe 37

PAGE 38

followingsections,weprovideseveralwaystodesignthemargin,usingparametricandnon-parametricmethods,forpointwiseandconstantmargins. 3.2.2MetricsForConservativenessAndAccuracy Asdiscussedinintroduction,conservativeestimatesarebiased,andahighlevelofconservativenesscanonlybeachievedatapriceinaccuracy.Thus,thequalityofamethodcanonlybemeasuredasatrade-obetweenconservativenessandaccuracy.Inordertoassessaglobalperformanceofthemethods,weproposetodeneaccuracyandconservativenessindexes. Themostwidelyusedmeasuretochecktheaccuracyofasurrogateistherootmeansquareerror(eRMS),denedas: eRMS=0@ZD(^y(x))]TJ /F4 11.955 Tf 11.95 0 Td[(y(x))2dx1A1 2(3{3) TheeRMScanbecomputedbyMonte-Carlointegrationatalargenumberofptesttestpoints: eRMSvuut 1 ptestptestXi=1e2i(3{4) whereei=(^yi)]TJ /F4 11.955 Tf 12.62 0 Td[(yi),^yiandyibeingthevaluesoftheconservativepredictionandactualsimulationatthei-thtestpoint,respectively. Wealsodenetherelativelossinaccuracy,inordertomeasuresimplybyhowmuchtheaccuracyisdegradedwhenhighlevelsofconservativenessaretargeted: l=eRMS eRMSjref)]TJ /F1 11.955 Tf 11.96 0 Td[(1(3{5) whereeRMSistakenatagiventargetconservativeness;andeRMSjrefistheeRMSvalueofreference.Inmostofthestudy,whichwetakeequaltothevalueofeRMSwhenthetargetconservativenessis50%. 38

PAGE 39

Therearedierentmeasuresoftheconservativenessofanapproximation.Hereweusethepercentageofconservativeerrors(i.e.positive): %c=100ZDI[^y(x))]TJ /F4 11.955 Tf 11.96 0 Td[(y(x)]dx(3{6) whereI(e)istheindicatorfunction,whichequals1ife>0and0otherwise.%ccanbeestimatedbyMonte-Carlointegration: %c100 ptestptestXi=1I(ei)(3{7) Ideally,formostsurrogatemodels,%c=50%whentheapproximationisunbiased. Thepercentageofconservativeerrorsprovidestheprobabilitytobeconservative.However,itfailstoinformbyhowmuchitisunconservativewhenpredictionsareunconservative.Thus,analternatemeasureofconservativenessisproposed;thatis,themaximumunconservativeerror(largestnegativeerror): MaxUE=max(max()]TJ /F4 11.955 Tf 9.3 0 Td[(ei);0)(3{8) Thisindexcanalsobenormalizedbytheindexoftheunbiasedestimator: MaxUE%=MaxUE MaxUEjref)]TJ /F1 11.955 Tf 11.95 0 Td[(1(3{9) whereMaxUEjrefisthevalueofreference,whichwetakeequaltothevaluefortheunbiasedestimator. AvalueofMaxUE%of50%meansthatthemaximumunconservativeerrorisreducedby50%comparedtotheBLUEestimator. Alternativemeasuresofconservativenesscanbedened,includingthemeanorthemedianoftheunconservativeerrors.However,themaximumerrordecreasesmonotonicallywhentargetconservativenessisincreased,whilemeanandmediancanincreasewhenweincreaseconservativeness,forinstancewhenwehaveinitiallyverysmallandverylargeerrors. 39

PAGE 40

3.2.3ConstantSafetyMarginUsingCross-ValidationTechniques Here,weconsiderthedesignofconservativesurrogates,whenthesamesafetymarginisappliedeverywhereonthedesigndomain.Inthefollowing,wecallsuchestimatorsCSMestimators(forConstantSafetyMargin),anddenoteit^yCSM. Intermsofthecumulativedistributionfunction(CDF)oftheerrors,Fe,thesafetymarginSmforagivenconservativeness,%c,isgivenas: Sm=F)]TJ /F5 7.97 Tf 6.58 0 Td[(1e%c 100(3{10) Theobjectiveistodesignthesafetymarginsuchthattheaboveequationisensured.Theactualerrordistributionis,inpractice,unknown.Weproposeheretoestimateitempiricallyusingcross-validationtechniquesinordertochoosethemargin. Cross-validation(XV)isaprocessofestimatingerrorsbyconstructingthesurrogatewithoutsomeofthepointsandcalculatingtheerrorsattheseleftoutpoints.Theprocessisrepeatedwithdierentsetsofleft-outpointsinordertogetstatisticallysignicantestimatesoferrors.Theprocessproceedsbydividingthesetofndatapointsintosubsets.Thesurrogateisttedtoallsubsetsexceptone,anderrorischeckedinthesubsetthatwasleftout.Thisprocessisrepeatedforallsubsetstoproduceavectorofcross-validationerrors,eXV.Usually,onlyonepointisremovedatatime(leave-one-outcross-validation),sothesizeofeXVisequalton. TheempiricalCDFFXV,denedbythenvaluesofeXV,areanapproximationofthetruedistributionFe.Now,inordertodesignthemargin,wereplaceFeinEq. 3{10 byFXV: Sm=F)]TJ /F5 7.97 Tf 6.59 0 Td[(1XV%c 100(3{11) Forinstance,whenn=100and75%conservativenessisdesired,thesafetymarginischosenequaltothe25thhighestcross-validationerror. 40

PAGE 41

3.2.4PointwiseSafetyMarginBasedOnErrorDistribution Conservativeestimatescanalsobeobtainedassumingtheerrordistributionisknownasprovidedbythesurrogateanalysis.Classicalregressionprovidesacondenceintervalforthepredictedmodel.Aunilateralcondenceintervaloflevelfortheresponsey(x)isgivenby[see Khuri&Cornell ( 1996 )]: CI=i;fT(x)^+t)]TJ /F5 7.97 Tf 6.59 0 Td[(1n)]TJ /F6 7.97 Tf 6.59 0 Td[(p)]TJ /F5 7.97 Tf 6.58 0 Td[(1(1)]TJ /F4 11.955 Tf 11.96 0 Td[()si(3{12) wheretn)]TJ /F6 7.97 Tf 6.59 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1istheStudent'sdistributionwithn)]TJ /F4 11.955 Tf 11.95 0 Td[(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1degrees-of-freedom,and:s=^p 1+fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1(X)f(x)^2=1 n)]TJ /F4 11.955 Tf 11.96 0 Td[(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1nXi=1(yi)]TJ /F1 11.955 Tf 12.74 0 Td[(^yi)2 Wedenetheconservativeestimatoroflevel(1)]TJ /F4 11.955 Tf 12.96 0 Td[()astheupperboundofthecondenceinterval: ^yED(x)=fT(xnew)^+t)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F6 7.97 Tf 6.58 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1(1)]TJ /F4 11.955 Tf 11.95 0 Td[()s(3{13) Thisconservativeestimatehastheformofamarginaddedtotheunbiasedprediction;themargindependsonthepredictionlocationandisequalto: Sm(x)=t)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F6 7.97 Tf 6.58 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1(1)]TJ /F4 11.955 Tf 11.95 0 Td[()^p 1+fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1(X)f(x)(3{14) Krigingalsoprovidescondenceintervalsfortheprediction,assumingthatthepredictionerrorisnormallydistributedwithmeanmKandvariances2K.Then,wecandenetheconservativeestimatoroflevel(1)]TJ /F4 11.955 Tf 11.96 0 Td[()as: ^yED(x)=)]TJ /F5 7.97 Tf 6.59 0 Td[(1mK(x);m2K(x)(1)]TJ /F4 11.955 Tf 11.96 0 Td[()(3{15) where)]TJ /F5 7.97 Tf 6.59 0 Td[(1mK(x);s2K(x)istheinversenormalcumulativedistributionfunctionofmeanmK(x)andvariances2K(x). 41

PAGE 42

Inthefollowing,theseestimatorsarereferredasED(errordistribution)estimate.Theyarecalledparametricestimatessincetheyassumeaparticularformofthedistributionoftheerror.Notethatcondenceintervalsgivenbykriginghaveadierentinterpretationthantheonesgivenbyregression.Indeed,inregressiontheinterpretationoftheCIisthatthereisaprobabilitythatanobservationtakenatxfallsintotheinterval.Inkriging,itmeansthat(1)]TJ /F4 11.955 Tf 12.07 0 Td[()%oftherealizationsofthegaussianprocessesconditionaltotheobservationswillstaywithintheintervalbounds. 3.2.5PointwiseSafetyMarginUsingBootstrap Errordistributionestimatorsrelyonthefactthatthehypothesisbehindthesurrogatemodelaresatisedinpractice.Inparticular,normalityoftheerrordistributionisalwaysassumed.Itisobviousthat,whensuchhypothesisisviolated,thesafetymargindesignedwitherrordistributioncanbeveryinaccurate.Inthissection,weproposetheuseofbootstraptoobtaincondenceintervalswithoutassumptionsontheerrordistribution. 3.2.5.1TheBootstrapprinciple Thebootstrapmethodcanprovideanecientwayofestimatingthedistributionofastatisticalparameterbasedonsamplesfu1;:::;ungoftherandomvariableUusingthere-samplingtechnique[ Efron ( 1982 ), Chernick ( 1999 )].Theideaistocreatemanysetsofbootstrapsamplesbyre-samplingwithreplacementfromtheoriginaldata. Thismethodonlyrequirestheinitialsetofsamples.Figure 3-1 illustratestheprocedureofthebootstrapmethod.Thesizeoftheinitialsamplesisnandthenumberofbootstrapre-samplingsisp.Eachre-samplingcanbeperformedbysamplingwithreplacementndataoutoftheninitialsamples(hence,thebootstrapsamplescontainrepeatedvaluesfromtheinitialsamplesandomitsomeoftheinitialvalues).Sincethere-samplingprocessdrawssamplesfromtheexistingsetofsamples,itdoesnotrequireadditionalsimulations.Theparameter(forinstance,themeanorstandarddeviationofU),isestimatedforeachbootstrapsamples.Sincethere-samplingprocedureallowsselectingdatawithreplacement,thestatisticalpropertiesofthere-sampleddataare 42

PAGE 43

dierentfromthatoftheoriginaldata.Then,thesetofpbootstrapestimatesbootdenesanempiricaldistributionof.Thisapproachallowsustoestimatethedistributionofanystatisticalparameterwithoutrequiringadditionaldata,andwithoutanyassumptiononthedistributionoftheparameter. Figure3-1. Schematicrepresentationofbootstrapping.Bootstrapdistributionof(histogramorCDFofthepestimates)isobtainedbymultiplere-sampling(ptimes)fromasinglesetofdata. 3.2.5.2Generatingcondenceintervalsforregression Inthissection,wepresenttheapplicationofbootstraptoregression,asproposedby Stine ( 1985 ).Calculationsandmethodologytoobtaincondenceintervalscomefromthispaper.Themainideaisthatresamplingwithreplacementisconductedontheregressionresiduals. First,theregressionmodelisestimatedusingthestandardequations.Then,theobservationsareperturbedbyresamplingtheresiduals: Y=X^+"(3{16)"isthebootstrapvectorofresiduals,obtainedbysamplingwithreplacementfromtheempiricaldistributionoftheresidualvector". 43

PAGE 44

Notethatthebootstrapmodelassumesthatthemeanofthetruefunctionisequaltothebestpredictoroftheregressionmodel,thatis,thetruefunctionisassumedtobeoftheform: y(x)=fT(x)^+"(3{17) where"followstheempiricaldistributionof". Then,themodelisbuiltbasedonthebootstrapobservations: =)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(fT(X)fT(X))]TJ /F5 7.97 Tf 6.59 0 Td[(1fT(X)Y(3{18) Usingbootstrap,weaccountforthevariabilityin^throughvariousbootstrapestimates1;2;3;:::,andalsoforthenoise"atthepredictedpoint. Thestructureofthenoiseleadstovaluablecomputationalshortcut.Indeed,theerrorEforthebootstrapmodelconsistsoftwoadditivecomponents: E="+b(3{19) with"takenfromtheinitialvectorofresidualsandbistheerrorduetouncertaintyin^: b(x)=fT(x))]TJ /F7 11.955 Tf 11.95 0 Td[(fT(x)^=fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1fT(X)" (3{20) Sinceband"areindependent,thedistributionofEistheirconvolution.Then,thedistributionFeoftheerrorcanbeapproximatedby: Fe(u)=1 NNXj=1Fn)]TJ /F4 11.955 Tf 5.47 -9.69 Td[(u)]TJ /F4 11.955 Tf 11.96 0 Td[(bj(x)(3{21) with: Fntheempiricaldistributionofthe(initial)residuals" uascalar 44

PAGE 45

bj(x)theerrorofthejthbootstrapmodelatx Nthenumberofbootstrapreplicates. Fe(u)istheprobability,computedbybootstrap,thattheerroratxexceedsu. Then,thepointwisesafetymarginp1)]TJ /F6 7.97 Tf 6.59 0 Td[(neededtoachieve(1)]TJ /F4 11.955 Tf 11.7 0 Td[()%conservativenessisdenedastheinverseofFetakenat(1)]TJ /F4 11.955 Tf 11.95 0 Td[()%: p1)]TJ /F6 7.97 Tf 6.59 0 Td[(=(1)]TJ /F4 11.955 Tf 11.95 0 Td[()(3{22) However,F)]TJ /F5 7.97 Tf 10.82 0 Td[(1ecannotbeknownanalytically.Inpractice,wechooseanitenumberofvaluesofuforwhichwecomputeFe(u)foragivenx;thenweobtaintheinverseofthedistribution(asshowninFigure 3-2 )byinterpolation. Figure3-2. Percentileoftheerrordistributioninterpolatedfrom100valuesofu. Finally,thebootstrapconservativepredictorisdenedas: ^yBS=xpred^+p1)]TJ /F6 7.97 Tf 6.59 0 Td[((3{23) Inthefollowing,theseestimatorsarecalledBS(bootstrap)estimatorsanddenotedby^yBS. 45

PAGE 46

3.2.6AlternativeMethods Inthecontextofconservativepredictions,weproposedtwootheralternatives.Therstiscalledbiasedtting:itconsistsofaddingconstraintstotheleastsquareminimizationprobleminordertobiasthesurrogatetobeononesideoftheobservations.ThesecondisIndicatorKriging(IK),whichisanon-parametricmethodthatestimatesprobabilitiesofexceedingaparticularthreshold.However,thesemethodswerefoundtobelessecientthantheothermethodstoproduceconservativepredictions.Hence,wedonotpresentithere.TheycanbefoundinAppendix A.1 and A.2 3.3CaseStudies 3.3.1TestProblems 3.3.1.1TheBranin-Hoofunction Thersttestfunctionisadeterministictwo-dimensionalfunction,whichisoftenusedtotestoptimizationmethods[ Dixon&Szego ( 1978 )].Theinputvariabledomainsare:x2[)]TJ /F1 11.955 Tf 9.29 0 Td[(510]andy2[015].TheexpressionoftheBranin-Hoofunctionisgivenas: f(x;y)=y)]TJ /F1 11.955 Tf 13.15 8.09 Td[(5:1x2 42+5x )]TJ /F1 11.955 Tf 11.96 0 Td[(62+101)]TJ /F1 11.955 Tf 16.68 8.09 Td[(1 8cos(x)+10(3{24) Therangeofthefunctionisbetweenzeroand300.Largevaluesarelocatedontheboundsofthedomain. Figure3-3. Branin-Hoofunction. 46

PAGE 47

3.3.1.2Thetorquearmmodel Thissecondexamplewasoriginallypresentedby Bennett&Botkin ( 1986 ).Itconsistsoftheshapedesignofaparticularpiecefromautomotiveindustrycalledatorquearm.Thetoquearmmodel,shownin 3-4 ,isunderahorizontalandverticalload,Fx=)]TJ /F1 11.955 Tf 9.29 0 Td[(2789NandFy=5066N,respectively,transmittedfromashaftattherighthole,whiletheleftholeisxed.ThetorquearmconsistsofamaterialwithYoung'smodulus,E=206:8GPaandPoisson'sratio,=0:29. Figure3-4. InitialdesignoftheTorquearm. Theobjectiveoftheanalysisistominimizetheweightwithaconstraintonthemaximumstress.Sevendesignvariablesaredenedtomodifytheinitialshape.Figure 3-5 andTable 3-1 showthedesignvariablesandtheirlowerandupperbounds,respectively.Thelowerandupperboundsareselectedsuchthatthetopologyofthedesignisunchanged. Table3-1. Rangeofthedesignvariables(cm). DesignvariableLowerboundUpperbound 1-2.03.52-0.22.53-2.06.04-0.20.55-0.12.06-1.52.07-0.12.0 47

PAGE 48

Figure3-5. Designvariablesusedtomodifytheshape. Thestressanalysisisperformedusingacommercialniteelementanalysis(FEA)software,ANSYS.Convergenceanalysisisperformedtondanappropriatemeshsize.Smallermeshsizeisusedfortheregionthathaspossibilityofhavingmaximumstress. Figure 3-6 showsatypicalstresscontourplotfromFEAresults.ThecontourlinesrepresentthevonMisesstress.Fortheinitialdesign,thevonMisesstressrangeisbetween10kPaand196MPa.Themaximumstressoccursattheendofthecutout.However,thelocationofmaximumstressmaychangefordierentdesigns.Infact,themaximumstressisdiscontinuousfunctionofdesignvariables.However,weassumethatitisacontinuousfunction,andtheoptimumdesignmustbeinspectedcarefully. Figure3-6. VonMisesstresscontourplotattheinitialdesign(Colorbarindicatesstressvaluesin104Pa). 48

PAGE 49

TheFEAiscomputationallyexpensive;thus,asurrogatemodelisusedtoapproximatethemaximumstressonthedesigndomain.Here,wedonotfocusontheoptimizationbutonthequalityofthesurrogateprediction. 3.3.2NumericalProcedure 3.3.2.1Graphsofperformance Wedisposeofseveraltechniquestoincreasetheconservativenessofsurrogatemodels;ourobjectiveistodetermine,throughtheexamples,ifsomeofthemarebetterthanothers.Moreover,itispossiblethatsometechniquesaremoreecientforlowlevelsofconservativeness,andotherforhighlevelsofconservativeness.Foreachmethod,thelevelofbiascanbemodiedbychangingthelevelofthecondenceinterval(1)]TJ /F4 11.955 Tf 12.12 0 Td[()(from50%to100%). Then,theindicesofperformancedenedinSection 3.2.2 canbecalculatedforeachmethodateachlevel.Inordertoanalyzeandcomparethedierentmethods,weusethefollowinggraphs: %cagainstl:therstindicatorofperformanceistheevolutionofconservativenesslevel(%c)againstthelossinaccuracy(l).ItcanbeconsideredasaParetofront,andcanbeusedtodesigntrade-osbetweenthesequantities.Also,itallowsustocomparethedierentmethodsbylookingatpartialorglobaldominations. %cagainstMaxUE%:thiscurveallowstochecktheconsistencyofthetwoconservativenessindiceslandMaxUE%when%cincreases. Targetconservativenessagainst%c:forallestimatesexceptbiasedttingmodels,thelevelofbiasisdesignedusingatargetlevelof%c.Then,itisofcrucialinteresttomeasuretheadequacytotheactual%ctotheexpectedon.Toanalyzethisperformance,wedrawtheQQ-plotofthetargetconservativeness(1)]TJ /F4 11.955 Tf 12.62 0 Td[()vs.theactualconservativeness%c. 3.3.2.2Numericalset-upfortheBranin-Hoofunction TwoDoEsareusedforthisfunction,thatconsistofthefourcornersofthedesignregionplus13and30points,respectively,generatedusingLatinHypercubeSampling(LHS)withmaximumminimumdistancecriterion;i.e.,atotalof17and34observations,respectively.Thetestpointsaregeneratedusinga3232uniformgrid(total1024points). 49

PAGE 50

TheeRMSisweightedsuchthatthepointsinsidethedomainhaveaweight1,thepointsontheedgesaweight1/2andthepointsonthecorners1/4. ThePRSisacubicpolynomial.ForKriging,anordinarykriging(OK)withrationalquadraticcovariancefunctionisused,whichisprovidedbytheGPMLToolbox[ Rasmussen&Williams ( 2006 )]. LHSarerandomdesigns,whichallowsustorepeattheprocedurealargenumberoftimes.Wepresenttheresultsastheaverageover500repetitions.Inaddition,weuseerrorbarstorepresent95%condenceintervalsonlandMaxUE%foragivenlevelof%c,andon%cforagivenleveloftargetconservativeness. TheunbiasedpolynomialresponsesurfacesarecomputedusingMatLabfunctionregress;thekrigingestimatesarecomputedusingtheGPMLtoolboxforMatLab. 3.3.2.3Numericalset-upfortheTorquearmmodel TheDoEconsistsof300pointsgeneratedfromLHSwithmaximumminimumdistancecriterion;1000testpointsaregeneratedusingLHS.Foreachpoint,theANSYScodeisrunandreturnsthemaximumstress.Theresponsevaluesareoftheorderof102MPa.ForthePRS,athirdorderpolynomialisused.ForKriging,severalcovariancefunctionsweretested,anditwasfoundthatasumofarationalquadraticcovariancefunction(Eq. 2{19 )andnuggeteectreectedaccuratelythecovariancestructure. Sincetheniteelementanalysisiscomputationallyexpensive,itisimpracticaltogeneratealargenumberofDoEsforvariabilityanalysis,aswedofortheBranin-Hoofunction.Inordertoobtainsomecondenceintervals,werandomlychoose300pointsoutofthe1300pointsgenerated(300training+1000testpoints),andusetheremaining1000astestpoints.Thisprocedureisrepeated500times.Onthegraphs,theaveragecurvecorrespondtotheoriginalDoEandtheerrorbarstothevariabilityestimationprocedure. 50

PAGE 51

3.4ResultsAndDiscussion 3.4.1Branin-HooFunction 3.4.1.1Analysisofunbiasedsurrogatemodels First,welookattheperformanceoftheunbiasedsurrogatemodelsontheBranin-Hoofunction.Table 3-2 reportstheeRMS,%candMaxUEforbothPRSandKriging. Table3-2. Meanvaluesofstatisticsbasedon1,024testpointsand500DoEsfortheunbiasedsurrogatesbasedon17observations.Numberinparenthesisarestandarddeviationsofthequantities. SurrogateeRMS%cMaxUE PRS8.1(4.6)53.1%(8.4)25.3(15.7)Kriging9.1(1.2)55.8%(5.8)22.6(8.1) Onaverage,PRSandKrigingperformsimilarlyontheBranin-Hoofunction.KrigingislesssensitivetotheDoE,sincethestandarddeviationsofeRMSandMaxUEaresmallerthanforPRS.TheMaxUEisrelativelysmallcomparedtotherangeofthefunction([0300]).Bothsurrogateshaveanaveragepercentageofconservativeerrorsslightlylargerthan50%,whichshowsthatsomehypothesisofthemodelmaybeviolated. 3.4.1.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates Now,wecompare,foreachmetamodel,constantandpointwisesafetymarginsusingerrordistribution.Figure 3-8 showstheresultscorrespondingtoPRS,andFigure 3-7 toKriging.Forbothgraphs,therangeofthesafetymarginischosenintheinterval[0;15];thetargetconservativenessischosenbetween50%and99%. ForPRS,weseeinFigure 3-8 A)thatthetwomethodsareequivalentintermsofaccuracyandvariability.95%conservativenessisobtainedforaneRMStwiceaslargeastheeRMSoftheunbiasedresponsesurface.However,thereisasubstantialdierenceforthemaximumunconservativeerror.Forthesameproportionofconservativeresults,thiserrorismorereducedwithEDthanwithCSM. ForKriging,theCSMestimatorclearlyoutperformstheEDestimatorintermsofaccuracyvs.conservativeness(Figure 3-7 A)).Thesamelevelofconservativenessis 51

PAGE 52

obtainedwithlesseectontheerror,especiallyforthe70-90%range.Moreover,thevariabilityismuchlargerforthestatisticalestimator. Figure3-7. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingbasedon17points.A)%cvs.l;B)%cvs.MaxUE%. Figure3-8. AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSbasedon17points.A)%cvs.l;B)%cvs.MaxUE%. ThelossinaccuracyoftheCSMestimatorincreasesveryslowlywith%c(Figure 3-7 A)).Upto70%conservativenessisachievedwithverylittleeectoneRMS.ThisbehaviorcanbeimputedtothenatureoftheKrigingmodel.SinceKrigingisaninterpolation,errorsareverysmallatthevicinityofthetrainingpoints.Thus,addingasmallconstantissucienttobeconservativebuthaslittleeectontheaccuracy. 52

PAGE 53

Ontheotherhand,EDestimatorappearstobemuchbettertoreducethemaximumunconservativeerror.AsshowninFigure 3-7 B),theEDcurve(grey)isalwayshigherthantheCSMone(black),whichmeans,foraequivalentproportionofconservativeresults,themaximumunconservativeerrorismorereducedwithEDthanwithCSM. ThisbehaviorisampliedwhenKrigingisbasedonalargerDoE.Figure 3-9 showsthegraphsofperformancefortheKrigingttedfrom34designpoints.TheCSMestimatorcurvehasaveryatportionupto75%conservativeness,whiletheEDestimatorcurveincreasesrapidly.Alongeratportionislogicalbecausetheregionwhereerrorsaresmall,atthevicinityoftheDOEpoints,islargersincetherearemorepoints.ThevariabilityismuchhigherfortheEDestimator.Ontheotherhand,theCSMestimatordoesnotpreventlargeunconservativeerrors:themaximumerrorremainsalmostthesameevenforlargeproportionsofconservativeresults.TheEDestimatorperformsalotbetteraccordingtothisindicator. Figure3-9. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingbasedon34points.A)%cvs.l;B)%cvs.MaxUE%. Now,wecomparethedelitytotargetconservativenessofbothCSMandEDestimators,whentheconstantmarginisdesignedwiththehelpofcross-validation.Figure 3-10 showstheresultsforPRS,andFigure 3-11 forKriging. 53

PAGE 54

Figure3-10. Targetvs.actual%cforPRSusingCSMwithcross-validation(plainblack)andED(mixedgray)basedon17points. ForPRS,bothEDandCSMestimatorsshowpoordelitytotargetconservativeness.FortheEDestimator,themeanactual%cis5%higherthanthetarget,andtheerrorbarsshowavariabilityoftheorderof5%.TheassumptionsofregressionmaybeviolatedwiththeBranin-Hoofunction;fortheunbiasedPRS,theactualconservativenessis55%where50%isexpected. TheCSMdesignedbycross-validationisinaccurateandhasverylargevariability.Thepoorperformanceofcross-validationcanbeexplainedbythesizeoftheDoE.Indeed,removingoneobservationoutof17canmodifysubstantiallythequalityofthesurrogate,socross-validationoverestimatestheerroramplitude.Then,theCSMischosenfrom17valuesonly,whichisnotenoughtoensureagoodprecision. ForKriging,inaveragetheEDestimatorshowsgooddelitysincethetrendisalmostequaltothestraightline.Forthehigherlevels,theactualconservativenessisalessthanthetarget.ThisisduetothefactthatKrigingassumenormalityoferrors,whilefortheBranin-Hoofunctionthisassumptionmaybeatacertaindegreeviolated.Whenlookingatthevariability,thecondenceintervalisabout20%,whichisverylarge.Inparticular,when97%conservativenessisexpected,theactualconservativenesscanbeaslowas70%. 54

PAGE 55

Figure3-11. Targetvs.actual%cforKrigingusingCSMwithcross-validation(plainblack)andED(mixedgray)basedon17points. AswellasforPRS,theCSMestimatorshowsbothinaccuracyandlargevariability.Inadditiontothereasonsexposedabove,cross-validationsuersherefromtheinterpolatingnatureofKriginginlowdimension.Indeed,thereareregionsofsmallerrors(inthevicinityoftrainingpoints)andregionsoflargeerrors.Sincetheobservationsarechosenaccordingtoaspace-llingDoE,trainingpointsareawayfromeachother,andcross-validationestimatestheerrorinregionsofhighuncertaintyofthemodel,whichexplainswhyhereCSMisoverconservative. 3.4.1.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates WehaveseenforPRSthatEDestimatesperformalittlebitbetterthanCSMontheBranin-Hoofunction,butshowpoordelitytotargetlevelofconservativeness,whichleadsforthisexampletounnecessarilylargemargins.Now,wecomparethisestimatortotheestimatorbasedonbootstrap. BeforeanalyzingtheresultsfortheBranin-Hoofunction,weproposetoillustratetheadvantageofbootstrapoverclassicalcondenceintervalsonatestcasewherethebootstrapissupposedtoshowbetteraccuracy.Thisisthecaseinparticularwhentheassumptionofnormalityofresidualsisviolated[ Stine ( 1985 )]. 55

PAGE 56

Theexperimentalsetupisasfollow: thefunctiontoapproximateisasecondorderpolynomialy=x)]TJ /F6 7.97 Tf 13.15 4.71 Td[(x2 2 theDoEconsistsof100observationsuniformlytakenon[0;1] thenoisefollowsalognormaldistribution,with=0and=1.Thedistributionisshiftedby)]TJ /F1 11.955 Tf 11.29 0 Td[(exp(+2=2)sothemeaniszero,anddividedbytwotoreducetheamplitude. Figure 3-12 showstheobservations,polynomialresponsesurfaceand95%condenceintervals(CI)computedusingbothclassicalformulaandbootstrap. Figure3-12. Condenceintervals(CI)computedusingclassicalregressionandbootstrapwhentheerrorfollowsalognormaldistribution.Bootstrapcatchestheasymetryofthedistributionandprovidesaccuratecondenceintervals. First,weseethattheobservationsarenotevenlydistributedaroundthetruefunction(70%arebelow).Theheavytailshapeofthedistributionofthenoiseisclearlyapparentwiththefewlargepositivenoises. Sinceitisbasedonalargenumberofobservations,theregressionbestpredictoriscorrectevenifassumptionofnormalityisviolated.However,thecondenceintervalsareinaccurate:theyaresymmetricwhilethenoiseisnot.So,(inabsolutevalues)thelowerboundisoverestimated,andtheupperboundisunderestimated.Ontheotherhand,thebootstrapcondenceintervalisaccurate:itisstronglyasymmetricandmatchesthedistributionoftheresiduals. 56

PAGE 57

Inthisexamplebootstrapisveryaccuratesinceitisbasedonalotofobservationsandtheregressionmodelisverysimple.Now,wecompareBSandCSMestimatorsontheBranin-Hoofunction.ThegraphsofperformancearegiveninFigure 3-13 ,andFigure 3-14 showsthedelityofbothestimatorstotarget%c. Figure3-13. AverageresultsforBS(plainblack)andED(mixedgrey)forPRSbasedon17points..A)%cvs.l;B)%cvs.MaxUE%. Figure3-14. Targetvs.actual%cforKrigingusingCSMandcross-validation(plainblack)andED(mixedgray)basedon17points. First,weseethattheevolutionofthelossinaccuracywiththepercentageofconservativeresultsisthesameforthetwoestimators(Figure 3-13 A)).EDisamore 57

PAGE 58

ecientthanbootstraptoreducethemaximumunconservativeerror(Figure 3-13 B)).Forthedelitytotargetconservativeness,onaverage,BSperformsalittlebitbetterthanEDbuttheirvariabilityaresimilar.Onthisexample,bootstrapdoesnotprovidesubstantiallybetterresultsthanED. 3.4.2TorqueArmModel 3.4.2.1Analysisofunbiasedsurrogatemodels First,welookattheperformancesoftheunbiasedsurrogatemodelsforthetorquearmstressanalysis.Table 3-3 reportstheeRMS,%candMaxUEforbothPRSandKriging. Table3-3. Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. SurrogateeRMS%cMaxUE PRS16.352.0%77.5Kriging14.551.6%60.7 Krigingperformsbetterintermsoferror.Bothsurrogateshaveapproximately50%negativeerrors.TheireRMSarereasonablecomparedtothemeanvalueofthestress(210MPa).However,weseethatthemaximumerrorisverylarge,especiallyforPRS.Designingthetorquearmbasedonthesesurrogateswouldresultinagreatriskofpoordesign. 3.4.2.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates Figure 3-15 showstheresultscorrespondingtoPRS,andFigure 3-16 toKriging. ForPRS,weseethesamebehaviorfortheBranin-Hoofunctionandtorquearmanalysis:CSMandEDaresimilarwhencomparingthelossinaccuracywhenthepercentageofconservativeresultsincreases,butEDreducesbetterthemaximumunconservativeerror.Here,itisimportanttonoticethatevenwhen%cishigherthan95%,theMaxUEisreducedbylessthan50%inthebestcase. 58

PAGE 59

Figure3-15. AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. Figure3-16. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. ForKriging,againCSMperformsbetterintermsoflossinaccuracyl,butreducesverypoorlytheMaxUE.EDperformsbetterregardingthisindicator,butaswellasforPRS,highlevelsof%conlycorrespondtoabout50%reductionofMaxUE. Now,welookatthedelitytotarget%c.Figure 3-17 showstheresultscorrespondingtoPRS,andFigure 3-18 toKriging. Onbothgures,weseethatusingconstantmarginandcross-validationresultsinagreatdelitytotargetconservativeness.EDestimatorsarealittlebitbiasedregardingdelity:PRSslighltyunderestimates%cforallprobabilityrange,andKriging 59

PAGE 60

overestimatesconservativenessforhighprobabilities.Theseerrorscanbeimputedtotheparametric(gaussian)formoftheerrormodel. Figure3-17. Targetvs.actual%cforPRSusingCSMandcross-validation(plainblack)andED(mixedgray)onthetorquearmdata. Figure3-18. Targetvs.actual%cforKrigingusingCSMandcross-validation(plainblack)andED(mixedgray)onthetorquearmdata. Incontrast,cross-validationdoesnotassumeanyformfortheerror.Thequalityofcross-validationmarginsarealotbetterthanintheprevioustestcas,whichcanbeexplainedbythelargernumberofobservations.Firstly,with300observations,leavingone 60

PAGE 61

observationouthasverylittleimpactontheaccuracyofthesurrogate,sotheerrorvectorgivenbycross-validationismorerealistic.Secondly,theCSMisbasedonalargenumberofvalues(300insteadof17previously),whichallowstodesignitwithbetteraccuracy. 3.4.2.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates Figure 3-19 showstheresultsofBSandEDestimates.Forthoseindicators,thetwomethodsaresimilar. Figure3-19. AverageresultsforBS(plainblack)andED(mixedgrey)forPRSonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. Figure3-20. Targetvs.actual%cforPRSusingBS(plainblack)andED(mixedgray)onthetorquearmdata. 61

PAGE 62

Figure 3-20 showsthedelityofbothestimatorstotarget%c.Here,bootstrapdoesnotperformaswellasED.Itisinterestingtonoticethantheyhaveoppositebias,sinceEDisoverconservativeandBSovercondent. 3.5ConcludingComments Inthischapter,weexploredthealternativestoobtainconservativepredictionsusingsurrogatemodeling.First,weshowedthatconservativenesscouldbeobtainedusingparametricmethods,bytakingadvantageoferrordistributionmeasuresgivenbythemodel,ornon-parametric,suchasbootstraporcross-validation.Then,aParetofrontmethodologywasintroducedtomeasurethequalityofthedierentmethods.Finally,themethodsareimplementedfortwotestproblems:oneanalyticalandonebasedonFiniteElementAnalysis.Resultsshowedthat: Theuseofsafetymarginsanderrordistributionleadtoverycomparableresultswhenthetrade-obetweenaccuracyandconservativenessareconsidered EDestimators,foranequivalentlevelofconservativenessandaccuracy,preventbetterthanCSMfromtheriskoflargeunconservativeerrors EDestimatorsarebasedonassumptionsthatmaynotbeviolatedinordertoobtainacceptabledelityofconservativenesslevel. Whenthemodelhypothesisareviolated,bootstrapoersanecientalternativetoobtainaccuratedelity Cross-validationisanecienttechniquetodesignconstantmarginswhenthenumberofobservationsishigh. Moregenerally,ithasbeenshownthatmeasuringtheconservativenessofamethodisnoteasyandcanbeveryproblem-dependant.Indeed,thechanceofbeingconservativeandtheriskoflargeunconservativeerrorsaretwomeasuresofconservativenessthatdonotbehaveidentically.Onemaychooseaconservativestrategybasedonthetrade-obetweenthesetwoquantitiesandtheglobalmeasureofaccuracy. 62

PAGE 63

CHAPTER4CONSERVATIVEPREDICTIONSFORRELIABILITY-BASEDDESIGN 4.1IntroductionAndScope Whenanengineeringsystemhasuncertaintyinitsinputparametersandoperatingconditions,thesafetyofthesystemcanbeevaluatedintermsofreliability.Manymethodshavebeenproposedtoestimatethereliabilityofasystem,suchasMonteCarlosimulation(MCS)method[ Haldar&Mahadevan ( 2000 )],FirstandSecond-orderReliabilityMethod[ Enevoldsen&Srensen ( 1994 ), Melchers ( 1987 )],importancesamplingmethod[ Engelund&Rackwitz ( 1993 )),tailmodeling( Kimetal. ( 2006 )],orinversemethods[ Qu&Haftka ( 2004 )].MCSisoftenusedtoestimatethereliabilityofthesystemthathasmanyrandominputsormultiplefailuremodes,becauseitsaccuracyisindependentofthecomplexityoftheproblem.Inthispaper,reliabilityanalysisusingMCSmethodisconsidered.Thecomparisonbetweendierentreliabilityanalysismethodsisbeyondthescopeofthischapterandcanbefoundintheliterature[ Rackwitz ( 2000 ), Leeetal. ( 2002 )]. Whenthecostofsimulationishigh,engineerscanaordtohaveonlyalimitednumberofsamples,whichisnotsucienttoestimatethereliabilitywithacceptableaccuracy[ Ben-Haim&Elishako ( 1990 ), Nealetal. ( 1992 )].Insuchacase,itisoftenrequiredtocompensateforthelackofaccuracywithconservativeapproaches.Forexample,anti-optimization[ Elishako ( 1991 ), Duetal. ( 2005 )]andpossibility-baseddesign[ Duetal. ( 2006 ), Choietal. ( 2005 )]areusedtocompensateforthelackofknowledgeintheinputdistributionbyseekingtheworstcasescenarioforagivendesign.Suchapproacheshavebeenfoundtoleadtoconservativedesigns[ Nikolaidisetal. ( 2004 )].Bayesianreliability-basedoptimization[ Youn&Wang ( 2008 )]usesBayesiantheorytoensuretoproducereliabledesignswhenonlyinsucientdataisavailablefortheinputs. Inthischapter,wefocusonthecasewhentheprobabilityoffailure,Pf,ofasystemisestimatedfromalimitednumberofsamples.Theobjectiveistondaconservativeestimate,^Pf,thatislikelytobenolowerthanthetruePf.Pfisestimatedusing 63

PAGE 64

particulartypesofsurrogatemodels,whicharedistributionfunctionmodels.Toprovidesuchestimation,twoalternativesareconsidered:therstmethodisbasedonbiasingtheprocessofttingthedistributionusedtocomputetheestimatorofPf.Thesecondistheuseofbootstrapmethod[ Efron ( 1982 ), Chernick ( 1999 )]forquantifyingtheuncertaintyinprobabilityoffailureestimations,anddeningconservativeestimatorsbasedonbootstrapping.Aswellasinthepreviouschapter,atrade-oanalysisbetweenaccuracyandthelevelofconservativenessisproposedwiththehelpofnumericalexamples. Inthenextsection,wediscusshowweusesamplingtechniquestoestimatetheprobabilityoffailure.Section 4.3 showshowtouseconstraintstoobtainconservativeestimators.Section 4.4 describeshowtousethebootstrapmethodtodeneconservativeestimator.TheaccuracyofsuchestimatorsisanalyzedusingasimplenumericalexampleinSection 4.5 ,andananalysisoftheeectsofsamplesizesandtargetprobabilityoffailureonthequalityoftheestimatesconservativeestimatorsisgiveninSection 4.6 .Finally,theconservativeapproachisappliedtoanengineeringprobleminSection 4.7 ,followedbyconcludingremarksinSection 4.8 4.2EstimationOfProbabilityOfFailureFromSamples 4.2.1Limit-StateAndProbabilityOfFailure Failureofasystemcanusuallybedeterminedthroughacriterion,calledalimit-state,G.Thelimit-stateisdenedsothatthesystemisconsideredsafeifG0andfailsotherwise.Typically,thelimit-stateofastructurecanbedenedasthedierencebetweenresponse,R,(e.g.,maximumstressorstrain)andcapacity,C,(e.g.,maximumallowablestressorstrain): G(x)=R(x))]TJ /F4 11.955 Tf 11.95 0 Td[(C(x)(4{1) Duetouncertaintiesininputparameters,thelimit-stateisrandom.Thus,onecanestimatethesafetyofthesystemintermsprobabilityoffailure,whichisdenedas Pf=Prob(G0)(4{2) 64

PAGE 65

Therearemanymethodsforcalculatingthefailureprobabilityofasystem[ Haldar&Mahadevan ( 2000 ), Enevoldsen&Srensen ( 1994 ), Melchers ( 1987 )].Someofthemusetherelationbetweeninputrandomvariablesandthelimit-state(e.g.,rstandsecond-orderreliabilitymethods)andsomeconsiderthelimit-stateasablack-boxoutput(e.g.,MCS).Whenthenumberofinputrandomvariablesislarge,andthelimit-stateiscomplexandmulti-modal,MCShasaparticularadvantageasitsaccuracyisindependentoftheproblemdimensionorcomplexityofthelimit-statefunction.MCSgeneratessamplesofthelimit-statefg1;g2;;gngandcountsthenumberoffailedsamples[ Melchers ( 1987 )].Theratiobetweenthenumbersoffailuresandthetotalnumberofsamplesapproximatestheprobabilityoffailureofthesystem: Pf^Pf=1 NnXi=1I[gi0](4{3) ThevarianceofMCSestimatesisinverselyproportionaltothesquarerootofthenumberofsamplestimestheprobabilityoffailure(forlargeN): var(^Pf)Pf(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pf) N(4{4) Thus,theaccuracyispoorwhenthenumberofsamplesissmallorwhentheprobabilityoffailureislow.Forinstance,ifaprobabilityoffailureof10)]TJ /F5 7.97 Tf 6.58 0 Td[(4isestimated(whichisatypicalvalueinreliabilitybaseddesign),106samplesareneededfor10%relativeerror. Inthischapter,wefocusonsituationswhereonlysmallsamplesareavailableregardingtheprobabilityoffailure;bysmallismeantfromtento500,forprobabilitiesoffailuregoingfrom1%to10)]TJ /F5 7.97 Tf 6.59 0 Td[(6.Insuchacase,thedirectuseofMCS(countingthenumberoffailedsamples)isunrealistic,sincetheprobabilityestimatewouldmostprobablybezero. 65

PAGE 66

Analternativeistoapproximatethecumulativedistributionfunction(CDF)FGofthelimit-statebyacontinuousfunctionandestimatetheprobabilityoffailureusing: ^Pf=1)]TJ /F4 11.955 Tf 11.95 0 Td[(FG(0)(4{5) Ingeneral,FGisunknownandisapproximatedbyaparametricdistributionF,whichisdenedbydistributionparameters,.Fcanbeconsideredasaone-dimensionalsurrogatemodelthatmustbettedfromthedatafg1;g2;;gng.Fcanbealogisticregressionmodel,ortheCDFofthenormaldistribution(hence=f;g),denedby: F(g)=1 p 2gZexp")]TJ /F1 11.955 Tf 9.29 0 Td[((u)]TJ /F4 11.955 Tf 11.95 0 Td[()2 22#du(4{6) Thesamplesoflimitstatescanbeobtainedbygeneratinginputrandomvariablesandpropagatingthroughthesystem.However,theproposedmethodcanbeappliedtothecasewheninputrandomvariablesanduncertaintypropagationareunknown.Forexample,thesamplesoflimitstatescanbeobtainedfromexperiments. Inthefollowingsection,twomethodsofestimatingdistributionparametersfromasetofsamplesarediscussed. 4.2.2EstimationOfDistributionParameters Thereexistsseveralalternativestoestimatethedistributionparameters.Thedicultyhereistodeneattingcriterion,sincetheusualassumptionsofsurrogatemodelsarenotmet(forinstance,normalityoftheresiduals).Whenpossible(forexampleforthenormaldistribution),theycanbeestimatedusingmoment-basedmethods.Here,weproposetodenecriteriabasedontheadequacyoftheparametricmodeltotheempiricalCDF. 66

PAGE 67

Considernlimit-statesamplesarrangedinincreasingorder:fg1;g2;;gng.TheempiricalCDFFnisdenedas: Fn(g)=8>>>><>>>>:0forgg1k/nforgkggk+11forggn(4{7) ItisthenpossibletoestimatetheparametersoftheCDFthatapproximatesFnbest.Twodierentapproximationmethodsarediscussedhere: minimizingtheroot-mean-square(RMS)error, minimizingtheKolmogorov-Smirnovdistance[ Kenney&Keeping ( 1954 )]. TominimizetheRMSdierencebetweentheempiricalandtheestimatedCDF,errorsarecalculatedatthesamplepoints.Inordertohaveanunbiasedestimation,thevaluesoftheempiricalCDFarechosenatthemiddleofthetwodiscretedata,as(seeFigure 4-1 A)): Fn(gk)=k)]TJ /F5 7.97 Tf 13.15 4.71 Td[(1 2 n;1kn(4{8) Thentheestimatedparameters^arechosentominimizethefollowingerror: ^=argminvuut nXk=1F(gk))]TJ ET q .478 w 284.31 -403.94 m 293.52 -403.94 l S Q BT /F4 11.955 Tf 284.31 -413.78 Td[(Fn(gk)2(4{9) TheestimatebasedonEq. 4{9 iscalledanRMSestimate. TheKolmogorov-Smirnov(K-S)distanceistheclassicalwaytotestifasetofsamplesarerepresentativeofadistribution.TheK-SdistanceisequaltothemaximumdistancebetweentwoCDFs(seeFigure 4-1 B)).TheoptimalparametersfortheK-Sdistanceare: ^=argminmax1knF(gk))]TJ /F4 11.955 Tf 13.4 8.09 Td[(k n;F(gk))]TJ /F4 11.955 Tf 13.15 8.09 Td[(k)]TJ /F1 11.955 Tf 11.95 0 Td[(1 n(4{10) Figure 4-1 A)showstheempiricalCDFfromtensamplesandthedatapointsthatareusedinEq. 4{8 .Figure 4-1 B)showstheK-SdistancebetweenanormallydistributedCDFandanempiricalCDFfrom10samples. 67

PAGE 68

Inthecasesweconsiderinthiswork,itisfoundthatthetwocriteriagiveequivalentresults.K-Sdistancelinkstoclassicalstatisticalframework,butcanbediculttominimizebecauseitisnon-smooth.TheRMSerrorminimizationappearstobemorerobust. Figure4-1. CriteriatotempiricalCDFs:A)Points(circles)chosentotanempiricalCDF(line)withRMScriterion.B)K-SdistancebetweenanempiricalCDFwithtensamplesandanormalCDF(continuousline).DataisobtainedbysamplingtenpointsfromN(0;1) s ThechoiceofthedistributionFiscriticalforaccurateestimationoftheprobabilityoffailure.Wrongassumptionontheformofthedistributioncanleadtolargebiasintheestimate,forinstance,ifthedistributionisassumedtobenormalwhileitisheavy-tailed.Statisticaltechniquesareavailabletotestifasamplebelongstoagivendistributiontype(goodness-ofttests),suchastheKolmogorov-Smirnov(foranydistribution),LillieforsorAnderson-Darlingtests(fornormaldistributions)[ Kenney&Keeping ( 1954 )].Somestatisticalsoftwarealsooersautomatedprocedurestochoosefromabenchmarkofdistributionswhichtsbestthedata. 4.3ConservativeEstimatesUsingConstraints InthespiritofthebiasedttingestimatesproposedinAppendix A.1 ,weproposetoaddconstraintstothettingprobleminordertomaketheestimatesmoreconservative. 68

PAGE 69

Aconservativeestimateoftheprobabilityoffailureshouldbeequalorhigherthantheactualone.Hence,aconservativeestimatecanbeobtainedbyconstrainingtheestimatedCDFtobelowerthanthetrueCDFwhentheparametersarefoundthroughtheoptimizationproblemsinEqs. 4{9 or 4{10 .Besides,weconsideronlysmallprobabilities,sothefailureoccursintheuppertailregionofthedistribution.Hence,theconstraintswillbeappliedtoonlytherighthalfofthedata.Whenthefailureoccursinthelowertailregion,theconstraintsshouldbeappliedonthelefthalfofthedata. TherstconservativeestimateoftheCDFisobtainedbyconstrainingtheestimatetopassbelowthesamplingdatapoints.AsecondcanbeobtainedbyconstrainingtheestimatedCDFbelowtheentireempiricalCDF.Theywillbecalled,respectively,CSP(ConservativeatSamplePoints)andCEC(ConservativetoExperimentalCDF).Thelatterismoreconservativethantheformer.Obviously,bothmethodsintroducebias,andthechoicebetweenthetwoconstraintsisamatterofbalancebetweenaccuracyandconservativeness. CSPconstraints: F(gi))]TJ /F4 11.955 Tf 14.65 8.09 Td[(i n0forn 2in(4{11) CECconstraints: F(gi))]TJ /F4 11.955 Tf 13.15 8.09 Td[(i)]TJ /F1 11.955 Tf 11.96 0 Td[(1 n0forn 2in(4{12) Toillustratetheseconservativeestimators,tensamplesaregeneratedfromarandomvariableGwhosedistributionisN(2:33;1:02).ThemeanischoseninsuchawaythattheprobabilityoffailureP(G0)isequalto1%.Figure 4-2 showstheempiricalCDFalongwiththethreeestimatesbasedonminimumRMSerror:(1)withnoconstraint,(2)withCSPconstraints,and(3)withCECconstraints.Table 4-1 showstheparametersofthethreeestimateddistributionsandthecorrespondingprobabilitiesoffailure. Theeectoftheconstraintsisclearfromthegraph.TheCSPestimatorisshifteddowntotheninthdatapoint;hence,theCDFatthetailisdecreased.TheCECestimator 69

PAGE 70

Figure4-2. ExampleofCDFestimatorsbasedonRMSerrorforasampleofsize10generatedfromN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02) Table4-1. Comparisonofthemean,standarddeviation,andprobabilityoffailureofthethreedierentCDFestimatorsforN()]TJ /F1 11.955 Tf 9.29 0 Td[(2:33;1:02).ExactvalueofPfis1%(samesampleasFigure 4-2 ). NoconstraintCSPCEC^2.292.342.21^0.840.971.31^Pf0.32%0.77%4.65% isshiftedevenfurtherdown.Sincetheconservativeestimatorsareunconstrainedonthelefthalfofthedistribution,theirCDFcurvescrosstheempiricalcurveonthatside. Forthisillustration,wechoseasamplerealizationthatisunfavorableforconservativeness.Asaconsequence,theestimatewithnoconstraintisstronglyunconservative(0.32%comparedto1.0%)eventhoughtheestimationisunbiased.TheCSPestimateisunconservative,butsubstantiallylessthantheunbiasedestimate.TheCECestimateisconservative.Inordertogeneralizetheseresultsandderivereliableconclusions,statisticalexperimentsbasedonlargenumberofsimulationswillbeperformedinSection 4.5 4.4ConservativeEstimatesUsingTheBootstrapMethod Analternativetobiasedttingistoobtaincondenceintervalsfortheprobabilityoffailureestimatesinordertodeterminethemarginneededtobeconservative.However, 70

PAGE 71

analyticalderivationofcondenceintervalsfortheprobabilityoffailureisverychallenging.Toovercomethisproblem,weproposetoobtaincondenceintervalsusingnumericalprocedures,i.e.bootstrap. Thebootstrapprincipleisdescribedinsection 3.2.5.1 .Whenonlylimitedsamplesareavailable,thebootstrapmethodcanprovideanecientwayofestimatingthedistributionofstatisticalparameter.Regardlessofitsconstruction,theprobabilityoffailureisestimatedfromsamplesofthelimit-statefg1;:::;gng.Hence,itispossibletoresamplewithreplacementfromfg1;:::;gng,andforeachsetofbootstrapre-samples,ttheCDFandcomputePf. Thestandarderrororcondenceintervalsofthestatisticalparametercanbeestimatedfromthebootstrapdistribution.However,thebootstrapmethodprovidesonlyanapproximationofthetruedistributionbecauseitdependsonthevaluesoftheinitialsamples.Inordertoobtainreliableresults,itissuggestedthatthesizeofthesamplesshouldbelargerthan100(i.e.,n)[ Efron ( 1982 )].Thenumberofbootstrapre-samplings(i.e.,p)ischosenlargeenoughsothatitdoesnotaectthequalityoftheresults(themajorsourceofuncertaintybeingtheinitialsample).Thevalueofpistypicallytakenfrom500to5,000. Forillustratingtheprocess,thefollowingcaseisconsidered:n=100andp=5;000.Thatis,100samplesofarandomvariableGaregeneratedfromthenormaldistributionN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02).Thetrueprobabilityoffailureis1%.Pretendingthatthestatisticalparameters(mean,standarddeviation,orprobabilityoffailurePf)areunknown,theseparametersaswellastheircondenceintervalswillbeestimatedusingthebootstrapmethod. Usingthegivensetof100initialsamples,5,000bootstrapre-samplingsareperformed.Similartotheconservativeestimationsintheprevioussection,thedistributiontypeisrstassumedtobenormal.Usingeachsetofbootstrapre-samples,themeanandstandarddeviationareestimated,fromwhichtheestimatedprobabilityoffailure^Pfiscalculated. 71

PAGE 72

The5,000^Pfvaluesdenetheempiricalbootstrapdistributionoftheestimator^Pf,representedinFigure 4-3 inahistogramform. Figure4-3. ConservativeestimatorsofPffrombootstrapdistribution:95thpercentile(p95)andmeanofthe10%highestvalues(CVaR). Theempiricalbootstrapdistributioncanbeusedtominimizetheriskofyieldingunconservativeestimate.Inotherwords,wewanttondaprocedurethatcalculatesthefollowingquantity: =P^PfPf(4{13) AprocedurethatsatisesEq. 4{13 iscalledan-conservativeestimatorofPf.Forexample,if=0.95isdesired,then^Pfisselectedatthe95thpercentileofthebootstrapdistributionoftheestimatedprobabilityoffailure.Thisestimatorwillbereferredto'Bootstrapp95'(seeFigure 4-3 ).Duetothenitesamplesize,however,Eq. 4{13 willbesatisedonlyapproximately. Besidesthis-conservativeestimator,themeanofthe-highestbootstrapvalues(conditionalvalue-at-riskCVaR[ Holton ( 2003 ))]isalsousedasaconservativeestimate.Here=10%isused,sotheestimatorconsistsofthemeanofthe10%highestbootstrapvalues.SinceCVaRisameanvalue,itwillbemorestablethanthe-conservativeestimator.However,itisdiculttodeterminethevalueofthatmakesEq. 4{13 satised 72

PAGE 73

precisely.Thisestimatorwillbereferredto'BootstrapCVaR90'(seeFigure 4-3 ).Notethatanybootstrappercentilehigherthan50%willbeaconservativeestimator.Averyhighorlowwillincreasethevalueof^Pfandwillyieldover-conservativeestimation. 4.5AccuracyAndConservativenessOfEstimatesForNormalDistribution ThegoalofthissectionistoevaluatetheaccuracyandtheconservativenessoftheestimatorspresentedinSections 4.3 and 4.4 ,whentheactualdistributionisknowntobenormal.StatisticalmeasuresoftheestimatorsareevaluatedbyestimatingPfalargenumberoftimes. Wealsointroduceherethereliabilityindex,whichisdenotedbyandrelatedtotheprobabilityoffailureas: =)]TJ /F1 11.955 Tf 9.3 0 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1(Pf)(4{14) whereistheCDFofthestandardnormaldistribution. ThereliabilityindexisoftenusedinsteadofPfinreliabilitybaseddesignbecausetherangeofvalues(typicallybetweenoneandve)ismoreconvenientanditsvariabilityislowerthanPf.Itisimportanttonotethatsince)]TJ /F1 11.955 Tf 9.3 0 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1isamonotonicallydecreasingfunction:alowprobabilitycorrespondstoahighreliabilityindex.Thus,aconservativeestimationofshouldnotoverestimatethetrue(sinceaconservativeestimationshouldnotunderestimatethetruePf).Inthefollowing,wepresenttheresultsforbothprobabilityoffailureandreliabilityindex. First,100samplesofGarerandomlygeneratedfromthenormaldistributionwithmean-2.33andstandarddeviation1.0.ThefailureisdenedforG0,whichcorrespondstoanactualprobabilityoffailureof1.0%.Foragivensetofsamples,dierentestimatorsareemployedtoestimatePf.Fivedierentestimatorsarecompared:theunbiasedtting,CSP,CEC,Bootstrapp95,andBootstrapCVaR90estimators.Thisprocedurewasrepeated5,000timesinordertoevaluatetheaccuracyandconservativenessofeachestimator.Fortheunbiased,CSPandCECestimators,wetestedbothRMSandKolmogorov-Smirnovdistancecriteriaandfoundthattheirperformancewascomparable 73

PAGE 74

butusingK-Sdistanceslightlyincreasesvariability.So,resultsarepresentedforRMScriteriononly. Mostoftheestimatedvalueswillexceedtheactualprobabilityoffailure,butitisdesirabletomaintainacertainlevelofaccuracy.Thus,theobjectiveistocompareeachestimatorintermsofaccuracyandconservativeness. Table 4-2 showsthestatisticalresultsfrom5,000repetitions.Resultsarepresentedintheformofthemeanvalueandthe90%symmetriccondenceinterval[5%;95%].Fortheprobabilityoffailureestimates,thelowerboundofthecondenceintervalshowstheconservativenessoftheestimator;themeanandtheupperboundshowtheaccuracyandthevariabilityoftheestimator.Ahighlowerboundmeansahighlevelofconservativeness,butahighmeanandupperboundmeanpooraccuracyandhighvariability.Forthereliabilityindexestimates,theupperboundshowstheconservativenessandthemeanandlowerboundtheaccuracy. Table4-2. MeansandcondenceintervalsofdierentestimatesofP(G0)andcorrespondingvalueswhereGisthenormalrandomvariableN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02).Thestatisticsareobtainedover5,000simulations. Estimators Pf(%) %ofcons. 90%C.I.Mean 90%C.I.Mean results Unbiased [0.37;2.1]1.02 [2.0;2.7]2.34 48CSP [0.63;3.6]1.86 [1.8;2.5]2.12 82CEC [0.95;5.5]2.97 [1.6;2.3]1.96 94Boot.p95 [0.83;3.7]2.06 [1.8;2.4]2.07 92Boot.CVaR90 [0.88;3.8]2.15 [1.8;2.4]2.05 93Actual 1.00 2.33 First,thecondenceintervaloftheunbiasedestimatorillustratestheriskofunconservativeprediction:indeed,thelowerboundis0.37%,whichmeansthereisavepercentchancetounderestimatePfbyafactorofatleast2.7(1:0=0:37=2:7).Thisresultprovidesanincentiveforndingawaytoimprovetheconservativenessoftheprobabilityestimate. 74

PAGE 75

TheCSPandCECestimatorsarebiasedontheconservativeside.Asexpected,theCECismoreconservativethantheCSP.Asaconsequence,CECismorebiasedandtheriskoflargeoverestimateisincreased.TheCECcondenceintervalshowsthatthereisavepercentchancetooverestimatePfbyatleastafactorof5.5,whilethisvalueis3.6fortheCSPestimator;ontheotherhandtheCECleadsto94percentconservativeresults,whiletheCSPestimatorleadstoonly82percentconservativeresults.ThechoicebetweentheCSPandCECestimatorswillbeachoicebetweenaccuracyandconservativeness. TheBootstrapp95estimatorachieves92percentconservativenessandtheBootstrapCVaR9093percentconservativeness.Fromtheupperboundsofbothestimations,wendthattheriskofoverestimatingPfbyatleastafactorof3.7isvepercent. Theamplitudeoferrorinthereliabilityindexismuchlowerthantheamplitudeintheprobabilityoffailure.FortheCECestimator,thelowerboundofthecondenceintervalcorrespondsto31%error([2:33)]TJ /F1 11.955 Tf 12.08 0 Td[(1:6]=2:33=0:31).Forthebootstrapestimators,thiserrorisreducedto23%.Themeanerrorsarerespectively16%and11%. Bootstrapmethodsappeartobemoreecientthanthebiasedtting(CSPandCEC)intermsofaccuracyandconservativeness.Fortheequivalentlevelofconservativeness(92-94percent),thelevelofbiasisreducedandtheriskofoverestimationsislowerwhenthebootstrapmethodisused.However,asmentionedearlier,thebootstrapmethodneedsaminimumsamplesizetobeused.Ithasbeenobservedthatwhenverysmallsamplesareavailable(10to50data),theaccuracyofthebootstrapmethoddropsdramatically.Insuchacase,theoptimizationbasedmethodsshouldbeusedinstead. 4.6EectOfSampleSizesAndProbabilityOfFailureOnEstimatesQuality Intheprevioussection,weshowedthatthebiasintheconservativeestimatecanleadtolargeoverestimationsoftheprobabilityoffailure.Themagnitudeofsuchanerrormainlydependsontwofactors:thesamplesizeandthevalueofthetrueprobability.Indeed,increasingthesamplesizewillreducethevariabilityofCDFttingand,asa 75

PAGE 76

consequence,theupperboundofthecondenceinterval.Meanwhile,inordertoestimatealowervalueoftheprobabilityoffailure,weneedtousethetailoftheCDF,whichincreasesthevariabilityoftheestimation. Controllingthelevelofuncertaintyiscrucialinoptimizationinordertoavoidover-design.Inthissection,wequantifyameasureoftheuncertaintyintheconservativeestimatesasafunctionofthesamplesizeandthevalueoftheactualPf.Suchameasurecanhelpindecidingontheappropriatesamplesizetocomputetheestimate. Bootstrapp95performedwellbasedonthepreviousexample.Thus,inthissectionweconsideronlythisestimator.Westudytwodistributioncases:standardnormaldistributionandlognormaldistributionwithparameters=0and=1(meanandstandarddeviationofthelogarithmofthevariable).Threesamplesizesareconsidered:100,200and500andsevenprobabilitiesoffailureareestimated:(1105,3105,1104,3104,1103,3103and1102)1. Foragivendistribution,samplesizeandPf,themeanand90%condenceintervalsarecalculatedusing5000repetitions.ResultsarepresentedinFigure 4.6 fornormaldistributionandinFigure 4.6 forlognormal.Theaccuracyismeasuredintermsofratiosfortheestimateoverthetrueprobabilityoffailure. Asexpected,thevariabilityof^PfincreaseswhenthesamplesizeandactualPfdecrease.Here,themostunfavorablecaseiswhenthesamplesizeisequalto100andtheactualPfisequalto105.Insuchacase,forbothdistributionsthereisavepercentchancetooverestimatePfbymorethan25timesitsactualvalue!Ontheotherhand,thecasewith500samplesleadstoaveryreasonablevariability.Forthethreesamplesizesandalltargetprobabilities,thelowerboundoftheratioisequaltoone,which 1Forthenormaldistribution,thefailuresaredenedforGgreater,respectively,than4.26,4.01,3.72,3.43,3.09,2.75and2.33;forthelognormalcase,thevaluesare71.2,54.6,41.2,30.5,22.0,15.3and10.2. 76

PAGE 77

meansthatthe95%conservativenessdonotdependsonthesefactors.Thebootstrapestimatesperformancesareremarkablysimilarforthetwodistributions,eventhoughthedistributionshapesareverydierent. Figure4-4. Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforNormaldistributionbasedon(a)100,(b)200,and(c)500samples.x-axisisthetrueprobability(lognormalscale),andy-axisistheratiobetweentheestimateandthetrueprobability.Variabilityincreaseswhentargetprobabilityorsamplesizearesmaller. Figure4-5. Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforlognormaldistributionbasedon(a)100,(b)200,and(c)500samples.Resultsarealmostidenticaltothenormaldistributioncase. Foranygivenreliabilityanalysisproblem,carefulattentionneedstobegiventotheaccuracyofprobabilityoffailureestimates.ThegraphsinFigures 4.6 and 4.6 address 77

PAGE 78

thisissue.Theyshowthecondenceintervalsandthereforedeneadequatesamplesizesneededtocomputereliableestimates.Intermsofcost-eectiveness,theguresindicatethatitmaybesmarttoallocategreaternumberofsimulationstolowprobabilitydesignsthantohighprobabilitydesigninordertogetconstantlevelofrelativeaccuracy. 4.7ApplicationToACompositePanelUnderThermalLoading Inthissection,conservativeestimatesareobtainedfortheprobabilityoffailureofacompositelaminatedpanelundermechanicalandthermalloadings.Thepanelisusedforaliquidhydrogentank.Thecryogenicoperatingtemperaturesareresponsibleforlargeresidualstrainsduetothedierentcoecientsofthermalexpansionoftheberandthematrix,whichischallengingindesign. Quetal. ( 2003 )performedthedeterministicandprobabilisticdesignoptimizationsofcompositelaminatesundercryogenictemperatures,usingresponsesurfaceapproximationsforprobabilityoffailurecalculations. Acar&Haftka ( 2005 )foundthatusingCDFestimationsforstrainsimprovestheaccuracyofprobabilityoffailurecalculation.Inthischapter,theoptimizationproblemaddressedby Quetal. ( 2003 )isconsidered.Thegeometry,materialparametersandtheloadingconditionsaretakenfromtheirpaper. 4.7.1ProblemDenition Thecompositepanelissubjecttoresultantstresscausedbyinternalpressure(Nx=8:4105N=mandNy=4:2105N=m)andthermalloadingduetotheoperatingtemperatureintherangeof20K-300K.Theobjectiveistominimizetheweightofthecompositepanelthatisasymmetricbalancedlaminatewithtwoplyangles[1;2]s(thatmeansaneight-layercomposite).Thedesignvariablesaretheplyanglesandtheplythicknesses[t1;t2].ThegeometryandloadingconditionareshowninFigure 4-6 .Thethermalloadingisdenedbyastressfreetemperatureof422K,andworkingtemperatureof300Kto20K.ThematerialusedinthelaminatescompositeisIM600/133graphite-epoxy,denedbythemechanicalpropertieslistedinTable 4-3 78

PAGE 79

Figure4-6. Geometryandloadingofthecryogeniclaminate. Table4-3. MechanicalpropertiesofIM600/133material.Quantitieswitharetemperaturedependent;thenumericalvaluesinthebracketaretherangeforTgoingfrom20to300K. ElasticpropertiesE1(GPa)147120.359E2(GPa)[148]G12(GPa)[84] Coecientsof1(K)]TJ /F5 7.97 Tf 6.59 0 Td[(1[)]TJ /F1 11.955 Tf 9.3 0 Td[(510)]TJ /F5 7.97 Tf 6.59 0 Td[(7;)]TJ /F1 11.955 Tf 9.3 0 Td[(1:510)]TJ /F5 7.97 Tf 6.59 0 Td[(7]thermalexpansion2(K)]TJ /F5 7.97 Tf 6.59 0 Td[(1[110)]TJ /F5 7.97 Tf 6.58 0 Td[(5;310)]TJ /F5 7.97 Tf 6.59 0 Td[(5] Stress-freetemperatureTzero(K)422 Failurestrains"U10.0103"L2-0.013"U20.0154U120.0138 Theminimumthicknessofeachlayeristakenas0.127mm,whichisbasedonthemanufacturingconstraintsaswellasforpreventinghydrogenleakage.Thefailureisdenedwhenthestrainvaluesoftherstplyexceedfailurestrains. 79

PAGE 80

Thedeterministicoptimizationproblemisformulatedas: Minimizet1;t2;1;2h=4(t1+t2)s:t:t1;t20:127"L1SF"1"U1"L2SF"2"U2SFj12jU12(4{15) whereSFischosenat1.4. Theanalysisofthestructuralresponseisbasedonclassicallaminationtheoryusingtemperature-dependentmaterialproperties.E2,G12,1and2arefunctionofthetemperature.Sincethedesignmustbefeasiblefortheentirerangeoftemperature,strainconstraintsareappliedat21dierenttemperatures,whichareuniformlydistributedfrom20Kto300K.Detailsontheanalysisandthetemperaturedependenceofthepropertiesaregivenin Quetal. ( 2003 ).TheirsolutionsforthedeterministicoptimizationproblemaresummarizedinTable 4-4 .Forthoseresults,t1andt2werechosenonlyasmultiplesof0.127mm.Threeoptimaarefoundwithequaltotalthicknessbutdierentplyanglesandplythicknesses. Table4-4. Deterministicoptimafoundby Quetal. ( 2003 ). 1(deg)2(deg)t1(mm)t2(mm)h(mm) 27.0427.040.2540.3812.540 028.160.1270.5082.540 25.1627.310.1270.5082.540 4.7.2Reliability-BasedOptimizationProblem Giventhematerialpropertiesandthedesignvariables,theplystrainscanbecalculatedusingClassicalLaminationTheory[ Kwon&Berner ( 1997 )].Duetothemanufacturingvariability,thematerialpropertiesandfailurestrainsareconsideredrandomvariables.Allrandomvariablesareassumedtofollowuncorrelatednormaldistributions.ThecoecientsofvariationaregiveninTable 4-5 .SinceE2,G12,1and2 80

PAGE 81

arefunctionofthetemperature,themeanvaluesoftherandomvariablesarecalculatedforagiventemperature,andthen,asetofrandomsamplesaregeneratedaccordingtotheirdistributions. Table4-5. Coecientsofvariationoftherandomvariables. E1;E2;G12;121;2Tzero "L1,"U1"L2,"U2,U12 0.0350.0350.030.060.09 Thecriticalstrainisthetransversestrainontherstply(direction2inFigure 4-6 ),theeectofotherstrainsontheprobabilityoffailurebeingnegligible.Hence,thelimit-stateisdenedasthedierencebetweenthecriticalstrainandthefailurestrain: G="2)]TJ /F4 11.955 Tf 11.95 0 Td[("U2(4{16) Then,theprobabilityoffailureisdenedinEq. 4{5 usingthedistributionFGofthelimit-state. Inordertodeterminewhichdistributiontypetsthebestthelimit-stateG,wegenerated1,000samplesateachofthethreerstoptimumdesigns.UsingaLillieforstest[ Kenney&Keeping ( 1954 )],wefoundthatallthesamplesbelongtoanormaldistribution.Hence,weassumedthatthelimit-stateGisnormallydistributedforanydesign. Onemightprefernottoassumeasingledistributiontypeoverthedesigndomainandtest,everytimeanewdesignisconsidered,severaldistributionstondtheonethattsbestthedata.Suchaprocedureisnecessaryforinstancewhenthelimit-statedistributionvariesfromnormaltoheavy-tailwithinthedesigndomain;assumingasingledistributioncanleadtolargeerrorinthereliabilityestimation.Moregenerally,onehastokeepinmindthatthemethodproposedherecanbecomehazardousifthedeterminationofthedistributiontypeisuncertain. Inourcase,sincethethreedesignsconsideredarequitedierentfromoneanotherandhavethesamelimit-statedistribution,itisreasonabletoassumethatthelimit-statedistributionisalwaysnormal,hencereducingthecomputationalburden. 81

PAGE 82

Thereliability-basedoptimizationreplacestheconstraintsonthestrainsinEq. 4{15 byaconstraintontheprobabilityoffailure.Thetargetreliabilityofthecryogenictankischosenas10)]TJ /F5 7.97 Tf 6.58 0 Td[(4.Sincetheprobabilityoffailurecanhaveavariationofseveralordersofmagnitudefromonedesigntoanother,itispreferabletosolvetheproblembasedonthereliabilityindex: Minimizet1;t2;1;2h=4(t1+t2)s:t:t1;t20:127(t1;t2;1;2))]TJ /F1 11.955 Tf 21.92 0 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(10)]TJ /F5 7.97 Tf 6.59 0 Td[(4=3:719(4{17) 4.7.3ReliabilityBasedOptimizationUsingConservativeEstimates BysolvingEq. 4{17 withasampling-basedestimateofreliability,wefacetheproblemofhavingnoiseintheconstraintevaluation,whichcanseverelyharmtheoptimizationprocess.Toaddressthisissue,wechosetotapolynomialresponsesurface(PRS)toapproximatethereliabilityindexeverywhereonaregionofthedesignspaceandsolvetheoptimizationwiththeresponsesurface.Wehavethentwolevelsofsurrogatemodeling:onesurrogatetoapproximatethereliabilityindexofagivendesign,andonesurrogatetoapproximatethereliabilityindexforallthedesignregion. ThePRSisbasedontheestimationofthereliabilityindexforaselectednumberofdesigns:f1;:::;pg.Alltheestimatesaredonebeforetheoptimizationprocess.TherangeoftheresponsesurfaceisgiveninTable 4-6 .Thenumberoftrainingpointsistakenas500;thepointsaregeneratedusingLatinhypercubesamplingtoensureagoodspace-llingproperty.Eachestimateofthereliabilityindexisbasedonthesamen=200samples,whichgivesusatotalnumberof100,000simulationstottheresponsesurface. Table4-6. Variablerangeforresponsesurface. VariablesRange 1,2(deg)[2030]t1,t2(mm)[0.1270.800] 82

PAGE 83

Withthe3,000simulationsusedtodeterminethedistributionofthelimitstate,thetotalnumberofsimulationsis103,000.ThisnumberisareasonabletotalbudgetforsolvingtheRBDOproblem:indeedthereisnofurthersimulationrunduringtheoptimization,thereliabilitybeingestimatedbythePRS.TocomparethisnumbertoclassicalMCSestimates,inordertoachievereasonablenoiseforaprobabilityoffailureof104,atleast106samplesareneededforasingleevaluation,andthereliabilityisreestimatedateachoptimizationstep.ItismorediculttocomparetoFORMandSORMmethods,butduetothelargenumberofrandomparametersintheproblem,theywouldnotbeecient. Thereliabilityindexesarecalculatedusingtwomethods:theunbiasedttingandthebootstrapconservativeestimationwith95%condence.Theestimatesaredenotedrespectively^(i)unband^(i)cons.Figure 4.6 B)showsthatwith200samples,thecondenceintervaloftheconservativeestimatorforaprobabilityof10)]TJ /F5 7.97 Tf 6.58 0 Td[(4is[10)]TJ /F5 7.97 Tf 6.58 0 Td[(4;810)]TJ /F5 7.97 Tf 6.59 0 Td[(4],whichcorrespondstoanerrorinbetween0and20%.Sincethenumberofobservationsismuchlargerthanthenumberofcoecients(500comparedto70),thenoiseislteredbythePRS. Adierentresponsesurfaceisttedtoeachsetofdata.Forboth,wefoundthatafourthorderpolynomialprovidedagoodapproximationforthereliabilityindex.Table 4-7 showsthestatisticsofeachresponsesurface.BothR2values(percentageofvarianceexplained)areveryclosetoone,andp-valuesshowthatbothmodelsareverysignicant. Inaddition,wecomputedaccurateestimatesofthereliabilityindexat22designsuniformlychoseninthedesignspaceusingseparableMonte-Carlomethod(SMC)[ Smarsloketal. ( 2008 )]with40,000samplesforeachdesign.Table 4-8 showsthestatisticsbasedonthetestpoints:therootmeansquareerror(eRMS)betweenthe22accurateresponsesandtheresponsesurface,errormeanandnumberofunconservativepredictions.Basedonthetestpoints,wecanseethattherstPRSisunbiased:theerrormeanisapproximatelyzeroandtherearetenunconservativeerrorsfortwelveconservative.On 83

PAGE 84

theotherhand,thesecondPRShaslargebiassincetheerrormeanis0.34andallthepredictionsattestpointsareconservative. Table4-7. StatisticsofthePRSforthereliabilityindexbasedontheunbiasedandconservativedatasets. DatasetsR2Fp-value n^(1)unb;:::;^(p)unbo0.96138<10)]TJ /F5 7.97 Tf 6.58 0 Td[(6n^(1)cons;:::;^(p)conso0.96136<10)]TJ /F5 7.97 Tf 6.58 0 Td[(6 Table4-8. StatisticsofthePRSbasedon22testpoints DatasetseRMSErrormeanNbofunconservativepredictions n^(1)unb;:::;^(p)unbo0.081-0.0110n^(1)cons;:::;^(p)conso0.354-0.340 Remark1 .Sincewehavetwolevelsofsurrogatemodels,itispossibletoapplyconservativestrategytoeitheroneonethem,orbothofthem,byapplyingtothePRSoneofthemethodsproposedinChapter3.Here,wefoundthatusingaconservativestrategyonlyonthereliabilityindexestimationwassucienttoguaranteesafedesign,andmoreecientthanusingunbiasedestimationofandconservativePRS. Remark2 .ThebalancebetweenthenumberofMCSbydesignnandthenumberofdesignpointspisanopenquestion.Here,wechoseasmallnumberofMCSsincewewereabletocompensateecientlyforit.Chapter7dealswiththisquestionandprovidetheoreticalanswerstochoosethisbalanceforPRSandKriging. 4.7.4OptimizationResults Wepresenttheresultsforthetwoprobabilisticoptimizationsbasedontheresponsesurfacesttedonunbiasedandconservativeestimates.Tocomparedeterministicandprobabilisticapproaches,thedeterministicoptimizationasstatedinEq. 4{15 isalsoperformedforthesamerangeoftheplyangles([2030]degrees).TheoptimizationisperformedusingMATLAB'sfunctionfminconrepeated20timeswithdierentinitialpointstoensureglobalconvergence.Theoptimaldesigns(bestoverthe20optimizations) 84

PAGE 85

aregiveninTable 4-9 .Forthesedesigns,anaccurateestimateoftheprobabilityoffailureiscomputedusingSMCwith40,000samples. Table4-9. Optimaldesignsofthedeterministicandprobabilisticproblemswithunbiasedandconservativedatasets. t1t212hfromActualPffromActualPf PRSPRS(SD)* Unbiased0.1270.50722.230.02.5363.723.6110-41.51e-4 dataset(2.28e-6) 95%cons.0.1270.58221.930.02.8353.723.9810-43.50e-5 dataset(6.3e-7) Determ.0.1270.41620.030.02.170x2.97x15.0e-4 optima(9.6e-6) Thethreeoptimaaresimilarintermsofplyangles,andforalltherstplythicknesst1isthelowerbound;thesignicantdierenceisinthesecondplythicknesst2.Bothprobabilisticdesignsareheavierthanthedeterministicoptimum.Theoptimumfoundusingunbiaseddatasetissubstantiallylighterthantheother(h=2:54comparedto2.84);however,theaccurateestimateofreliabilityshowsthattheoptimumdesignusingunbiaseddatasetviolatestheconstraint.Ontheotherhand,thedesignfoundusingconservativedatasetisconservative;theactualprobabilityis3timessmallerthanthetargetprobabilityoffailure.Thedeterministicdesignisveryunconservative,itsprobabilityoffailurebeing15timesthetarget. Thefactthatanunbiasedstrategyleadstoanunconservativedesignisnotsurprising.Indeed,optimizationisbiasedtoexploreregionswheretheerroris'favorable';thatis,wheretheconstraintisunderestimated.Usingtheconservativeapproach,thelevelofbiasissucienttoovercomethisproblem,butatthepriceofoverdesign:sincetheprobabilityoffailureisthreetimesthetarget,theactualoptimumislighterthantheonewefound. Wehaveshownthatdespiteaverylimitedcomputationalbudget(103,000MCStosolvetheRBDOproblem),itwaspossibletoobtainareasonabledesignbycompensatingthelackofinformationbytakingconservativeestimates. 85

PAGE 86

4.8ConcludingComments Theestimationoftheprobabilityoffailureofasystemiscrucialinreliabilityanalysisanddesign.Inthecontextofexpensivenumericalexperiments,orwhenalimitednumberofdatasamplesareavailable,thedirectuseofMonteCarloSimulationisnotpractical,andestimationofcontinuousdistributionsisnecessary.However,classicaltechniquesofestimationofdistributiondonotpreventdangerousunderestimatesoftheprobabilityoffailure. Severalmethodsofestimatingsafelytheprobabilityoffailurebasedonthelimitednumberofsamplesaretested,whenthesampledistributiontypeisknown.Therstmethodconstrainsdistributionttinginordertobiastheprobabilityoffailureestimate.Thesecondmethodusesthebootstraptechniquetoobtaindistributionsofprobabilityoffailureestimators,andthesedistributionsdeneconservativeestimators. Inthecaseofsamplesgeneratedfromnormaldistribution,thenumericaltestcaseshowsthatbothmethodsimprovethechanceoftheestimationtobeconservative.BootstrapbasedestimatorsoutperformedconstrainedtstotheexperimentalCDF.Thatis,forthesamecondenceintheconservativenessoftheprobabilityestimate,thepenaltyintheaccuracyoftheestimatewassubstantiallysmaller.However,optimizationbasedmethodscanbeusedwhenthesamplesizeisverysmall,wherethebootstrapmethodcannotbeused. Wealsoexploredtheinuenceofsamplesizesandtargetprobabilityoffailureonestimatesquality.Wefoundthatlargersamplesizesarerequiredtoavoidlargevariabilityinprobabilityoffailureestimateswhenthatprobabilityissmall.Theresultsindicatedthatwhensamplingatdierentpointsindesignspaceitmaybemorecosteectivetohavedierentnumberofsamplesatdierentpoints.SuchapproachisaddressedinChapter6. Finally,wehaveappliedtheconservativeestimationprocedurestoperformtheoptimizationofacompositelaminatesatcryogenictemperatures.Wecomparedthe 86

PAGE 87

optimizationresultsfoundwhereresponsesurfacesarettedtounbiasedandconservativeestimatesrespectively.Wefoundthattheunbiasedresponsesurfacesledtounsafedesigns,whiletheconservativeapproachreturnedanacceptabledesign. 87

PAGE 88

CHAPTER5DESIGNOFEXPERIMENTSFORTARGETREGIONAPPROXIMATION Theaccuracyofmetamodelsiscrucialfortheeectivenessoftheiruseinprediction,designorpropagationofuncertainty.InChapters3and4,weproposedsomealternativestocompensatefortheerrorinsurrogatemodels.Inthischapter,weaimatminimizingtheuncertaintyofthemodels.Thedesignofexperimentstrategyisoneofthemostimportantfactorsthataectthequalityofthemetamodel.Inmanypracticalproblems,thetotalnumberoffunctionevaluationsisdrasticallylimitedbycomputationalcost;hence,itisofcrucialinteresttodevelopmethodsforselectingecientlytheexperiments. Samplingstrategieshavebeenextensivelystudiedoverthepastdecades.InChapter2,wedescribedsomeofthemostpopularstrategiesthataimatoptimizingtheglobalaccuracyofthemetamodels.Thescopeofthepresentresearchistoproposeanobjective-basedapproachtodesignofexperiments,basedontheideathattheuncertaintyinsurrogatesmaybereducedwhereitismostuseful,insteadofglobally.Inparticular,weaimatndingecientstrategiesfortwoapplicationsofmetamodeling:reliabilityestimationandconstrainedoptimization. 5.1MotivationAndBibliography Mostsamplingstrategiesaredevelopedtoachievethebestglobalperformancesofthemetamodel.Suchapproachisfullyjustiedwhenthemodelisusedforpredictionoverallthedomainofinterest,inferenceofglobalbehaviorofthephenomenonstudied,orwhenallthedesignofexperimentisgeneratedatthesametimewithoutpriorinformationontheresponsebehavior,forinstancewhenseriesofphysicalexperimentsorparallelcomputationareinvolved. However,manymetamodelapplicationsfocusonthevalueofthefunctionresponse.Forinstance,whenametamodelisusedforoptimizationpurpose,itisobviousthatthemetamodelaccuracyismuchmoreimportantforlowvaluesoftheresponseratherthan 88

PAGE 89

largevalues.Hence,globalstrategiesdonotappearasthemostecientresponsestosuchapplication. Thepresentresearchfocusesonapplicationswheremetamodelsareusedinawaythattheiraccuracyiscrucialforcertainlevel(s)oftheresponsefunction.Suchsituationappearsinparticularintwopopularframeworks: Inconstrainedoptimization .Theconstraintfunctionoftenreliesonexpensivecalculations.Forinstance,atypicalstructuraloptimizationformulationistominimizeaweightfunctionsuchthatthemaximumstress,computedbyniteelementanalysis,doesnotexceedacertainvalue.Whenusingametamodeltoapproximatetheconstraint,itisofutmostimportancethattheapproximationerrorisminimalontheboundarythatseparatesthefeasibledesignsfrominfeasibleones.Substantialerrorsforvaluesfarfromtheboundary,ontheotherhand,arenotdetrimental. Inreliabilityanalysis .Weconsiderthecasewherethemetamodelisusedtoapproximatethelimitstatefunctionofasystem,G(x).Aswellasforconstraintapproximation,themetamodeldiscriminatesbetweensamplesthatarebelowandaboveacriticalvalue,inthiscasezero.Whenthelimitstateisfarfromzero,thechanceofmisclassicationisnegligibleevenifthemetamodelpoorlyapproximatethetruefunction,whiletheaccuracyiscrucialforvaluesclosetozero. Thepresentchapterpresentsamethodologytoconstructadesignofexperimentssuchthatthemetamodelaccuratelyapproximatesthevicinityofaboundaryindesignspacedenedbyatargetvalueofthefunctionapproximatedbythesurrogate.Thisproblemhasbeenaddressedbyseveralauthors. Mourelatosetal. ( 2006 )usedacombinationofglobalandlocalmetamodelstorstdetectthecriticalregionsandthenobtainalocallyaccurateapproximation. Shan&Wang ( 2004 )proposedaroughsetbasedapproachtoidentifysub-regionsofthedesignspacethatareexpectedtohaveperformancevaluesequaltoagivenlevel. Oakley ( 2004 )usesKrigingandsequentialstrategiesforuncertaintypropagationandestimationofpercentilesoftheoutputofcomputercodes. Ranjanetal. 89

PAGE 90

( 2008 )proposedamodiedversionoftheEGOalgorithmtosequentiallyexplorethedomainregionalongacontourline. Tu&Barton ( 1997 )usedamodiedD-optimalstrategyforboundary-focusedpolynomialregression. Vazquez&Piera-Martinez ( 2006 )proposedaniterativestrategytominimizetheclassicationerrorwhencomputingaprobabilityoffailurebasedonKriging. 5.2ATargetedIMSECriterion Inthischapter,weconsideronlyprobabilisticmetamodels-thatassociateadistributiontoapredictionpointinsteadofasinglevalue.Krigingisexplicitlyaprobabilisticmetamodel,butmostoftheothermodelscanhaveaprobabilisticinterpretation,suchasregressionandsplinesforinstance.However,welimitourinvestigationtothekrigingmodelhere. WegavethedenitioninSection 2.2.2 oftheIMSEcriterion,whichsumsuptheuncertaintyassociatedwiththeKrigingmodelovertheentiredomainD.However,whenoneismoreinterestedinpredictingyaccuratelyinthevicinityofacontourlineu=y)]TJ /F12 7.97 Tf 6.59 0 Td[(1(T)(Taconstant),suchacriterionisnotsuitablesinceitweightsallpointsinDaccordingtotheirKrigingvariance,whichdoesnotdependontheobservationsYobs,andhencedoesnotfavorzoneswithrespecttopropertiesconcerningtheiryvaluesbutonlyonthebasisoftheirpositionwithrespecttoDoE. WeproposetochangetheintegrationdomainfromDtoaneighborhoodofy)]TJ /F5 7.97 Tf 6.59 0 Td[(1(T).Twoapproachesareproposedthatdierbythedenitionoftheneighborhoodofthecontourline.Theneighborhoodisdenedwiththehelpofanindicatorfunctionandagaussiandistribution,respectively,whichcanbeencounteredintheliteratureunderthedenominationofdesirabilityfunctions. 5.2.1TargetRegionDenedByAnIndicatorFunction WedenearegionofinterestXT;"(parameterizedby")asthesubsetinDwhoseimageiswithintheboundsT)]TJ /F4 11.955 Tf 11.96 0 Td[("andT+": XT;"=y)]TJ /F5 7.97 Tf 6.59 0 Td[(1([T)]TJ /F4 11.955 Tf 11.95 0 Td[(";T+"])=fx2Djy(x)2[T)]TJ /F4 11.955 Tf 11.95 0 Td[(";T+"]g(5{1) 90

PAGE 91

Figure 5-1 illustratesaone-dimensionalfunctionwiththeregionofinterestbeingatT=0:8and"=0:2.Notethatthetargetregionconsistsoftwodistinctsets. Figure5-1. One-dimensionalillustrationofthetargetregion.Thelevel-setTisequalto0.8,and"to0.2.Thetargetregionconsistsoftwodistinctsets. Withtheregionofinterest,thetargetedIMSEcriterionisdenedasfollows: imseT=ZXT;"s2K(x)dx=ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][y(x)]dx(5{2) where1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][y(x)]istheindicatorfunction,equalto1wheny(x)2[T)]TJ /F4 11.955 Tf 12.02 0 Td[(";T+"]and0elsewhere. FindingadesignthatminimizesimseTwouldmakethemetamodelaccurateinthesubsetXT;",whichisexactlywhatwewant.WeightingtheIMSEcriterionoveraregionofinterestisclassicalandproposedforinstanceby Box&Draper ( 1986 ),pp.433-434.However,thenotabledierencehereisthatthisregionisunknownapriori. Now,wecanadaptthecriterioninthecontextofKrigingmodeling,whereyisarealizationofarandomprocessY(seeSection 2.1.3 ). 91

PAGE 92

Thus,imseTisdenedwithrespecttotheevent!: ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]dx=I(!)(5{3) Tocomebacktoadeterministiccriterion,weconsidertheexpectationofI(!),conditionalontheobservations(whichisthebestapproximationintheL2sense): IMSET=E[I(!)jobs]=E24ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]dxjobs35(5{4) Sincethequantityinsidetheintegralispositive,wecancommutetheexpectationandtheintegral: IMSET=ZDs2K(x)E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]jobsdx (5{5) =ZDs2K(x)E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][M(x)]dx (5{6) =ZDs2K(x)W(x)dx (5{7) AccordingtoEq. 5{7 ,thereducedcriterionistheaverageofthepredictionvarianceweightedbythefunctionW(x).Besides,W(x)issimplytheprobabilitythattheresponseisinsidetheinterval[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"].Indeed: W(x)=E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][M(x)] (5{8) =P)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(M(x)2[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"] (5{9) =P)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(Y(x)2[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"]jobs (5{10) TheGaussianprocessinterpretationofKrigingadoptedhere(seeSection 2.1.3 )allowsustoexplicitlyderivetheconditionaldistributionofM(x)=(Y(x)jobs)atanypredictionpoint: Foranyx2D:M(x)N)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(mK(x);s2K(x)(5{11) 92

PAGE 93

WecanobtainasimpleanalyticalformforW(x): W(x)=T+"ZT)]TJ /F6 7.97 Tf 6.59 0 Td[("gN(mK(x);s2K(x))(u)du(5{12) wheregN(mK(x);2K(x))(u)istheprobabilitydensityfunction(PDF)ofM(x).ByintegratingthePDFweobtain: W(x)=T+")]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x))]TJ /F1 11.955 Tf 11.96 0 Td[(T)]TJ /F4 11.955 Tf 11.96 0 Td[(")]TJ /F4 11.955 Tf 11.96 0 Td[(mK(x) sK(x)(5{13) whereistheCDFofthestandardnormaldistribution. Remark .Itispossibletodeneaweightfunctionwith"!0.Todoso,wehavetodividerstW(x)bytheconstant2".Then,weobtain: lim"!01 2"T+")]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x))]TJ /F1 11.955 Tf 11.96 0 Td[(T)]TJ /F4 11.955 Tf 11.96 0 Td[(")]TJ /F4 11.955 Tf 11.96 0 Td[(mK(x) sK(x)='T)]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x)(5{14) where'istheprobabilitydensityfunction(PDF)ofthestandardnormaldistribution. 5.2.2TargetRegionDenedByAGaussianDensity DeningtheregionofinterestasXT;isintuitiveandmakesiteasytoderivetheweightfunction.However,itmightnotalwayscorrespondexactlytoourobjective.Indeed,ifweconsideranidealcasewherethefunctionisexactlyknown,theindicatorfunctionwillyieldaweight1toapointxwhereM(x))]TJ /F4 11.955 Tf 12.49 0 Td[(T=",but0ifM(x))]TJ /F4 11.955 Tf 12.5 0 Td[(T="+10)]TJ /F5 7.97 Tf 6.58 0 Td[(9.Also,itwillnotdiscriminatebetweenapointwherethedierenceisequalto"andanotheronewherethisdierenceisequaltozero. Instead,wepreferacriterionthatcontinuouslyincreasestheimportanceofthelocationwhentheresponseapproachesthethreshold.Forinstance,wecanchooseatriangularfunction(withamaximumatT)orasigmoidfunction.Here,wechoosetousetheprobabilitydensityfunctionofanormaldistributionwhichleadstoasimpleanalyticalformoftheweightfunction.InthespiritofEq. 5{8 ,theGaussian-basedweightfunction 93

PAGE 94

isthereforedenedasfollows: W(x)=E[g"(M(x))]TJ /F4 11.955 Tf 11.96 0 Td[(T)](5{15) whereg"(u)isthePDFofN(0;2"). WhenM(x)standsfortheKrigingmodel,wecanobtainasimpleformfortheweightfunction: W(x)=+1Zg"(u)]TJ /F4 11.955 Tf 11.95 0 Td[(T)gN(mK(x);s2K(x))(u)du(5{16) Sinceg"issymmetric: W(x)=+1Zg"(T)]TJ /F4 11.955 Tf 11.95 0 Td[(u)gN(mK(x);s2K(x))(u)du(5{17) ThisintegralistheconvolutionofthetwoGaussiandensities,whichiswell-knowntobethedensityofasumofindependentGaussianvariables.Hence,weobtain: W(x)=1 p 2(2"+s2K(x))e)]TJ /F13 5.978 Tf 7.79 3.26 Td[(1 2(mk(x))]TJ /F14 5.978 Tf 5.76 0 Td[(T)2 2"+s2K(x)(5{18) Thisnewweightfunctiondependsonasingleparameterthatallowsustoselectthesizethedomainofinterestaroundthetargetlevelofthefunction.Alargevalueofwouldenhancespace-lling,sincetheweightfunctionwouldtendtoaconstantandtheweightedIMSEtoauniformIMSEcriterion.Onthecontrary,asmallvaluewouldenhancetheaccuracyofthesurrogateonanarrowregionaroundthecontourlineofinterest. 5.2.3Illustration Weconsideraone-dimensionalcase,wherethefunctionytoapproximateisarealizationofaGaussianprocesswithGaussiancovariancestructure.yisdenedon[0;1];thedesignofexperimentsconsistsofveobservationsequallyspacedinthisinterval.Thelevel-setofinterestTischosenas1.3,andboth"and"aretakenas0.2.Figure 5-2 representsthetruefunction,theKrigingmetamodelandcorrespondingweights.The 94

PAGE 95

weightfunctioninEq. 5{13 isshownas"interval",whilethatinEq. 5{18 isshownas"Gaussian"inthegure. Figure5-2. Illustrationoftheweightsfunctions.Uppergraph:truefunction,observations,Krigingmeanandcondenceintervals,andtargetregion.Lowergraph:weightfunctions.Bothweightsarelargewherethetruefunctionisinsidethetargetregion,butalsosignalingregionsofhighuncertainties(aroundx=0:65and0:85). Amongtheveobservations,oneissubstantiallyclosertoTthantheothers.Asaconsequence,theweightfunctionsarelargearoundthisobservationpoint.Fortheindicator-basedweightfunction,theweightsarenullattheobservationpoints,sinceforthisexamplenoobservationisinsidethetargetvalueinterval.FortheGaussian-basedweight,itisalsoveryclosetozero.Forbothfunctions,highweightsaregiventoregionsforwhichtheactualfunctionisinsidethetargetinterval.Bothweightfunctionsarealsonon-zerowheretheuncertaintyishigh,eveniftheKrigingmeanisfarfromT(aroundx=0:65and0:85). 5.3SequentialStrategiesForSelectingExperiments Withoutanyobservation,theweightfunctionW(x)is,assumingstationarity,aconstant(theprobabilityisthesameeverywhere).Everytimeanewobservationisperformed,theweightfunctionwillmorepreciselydiscriminatetheregionsofinterestfrom 95

PAGE 96

theothers.Hence,theproceduretobuildanoptimalDoEisnecessarilyiterative.IfweaddoneobservationatatimewecanusetheprocedureshowninTable 5-1 Table5-1. ProcedureoftheIMSET-basedsequentialDoEstrategy. CreateaninitialDoE,Xk,andgenerateobservationsYk=y(Xk)Forigoingfromonetothetotalnumberofadditionalobservationsn:FittheKrigingmodeltothedatafXk+i)]TJ /F5 7.97 Tf 6.58 0 Td[(1;Yk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1gFindanewtrainingpointxnewthatminimizesthecriterionIMSET(fXk+i)]TJ /F5 7.97 Tf 6.58 0 Td[(1;xnewg)Computethenewobservationynew=y(xnew)UpdatetheDoEandobservations:Xk+i=fXk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1;xnewgYk+i=fYk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1;ynewgEndofloop TheKrigingparameterscanbereevaluatedaftereverynewobservation,oronlyfromtheinitialDoEbeforetheiterativeprocedure.NotethattheevaluationoftheparametersiscriticaltoobtainagoodKrigingmodel.Besides,theparametersaectthecriterion:underestimationoftherangemakestheweightfunctionatterandenhancesspace-lling;onthecontrary,overestimationoftherangeleadstoaverydiscriminating(over-condent)weightfunction.However,inthenumericalexamplesusedinthiswork,wefoundthattheparameterre-evaluationhadanegligibleimpactontheeciencyofthemethod. Findingthenewtrainingpointrequiresaninneroptimizationprocedure.WhentheclassicalIMSEcriterionisconsidered,theoptimizationcanbeexpressedas: minxnew2DIMSE(Xk+1)=IMSE(fXk;xnewg)(5{19) whereIMSE(fXk;xnewg)=ZDs2K(xjfXk;xnewg)dx s2K(xjfXk;xnewg)isthevarianceatxoftheKrigingbasedonthedesignofexperimentsXaugmentedwiththetrainingpointxnew.SincetheKrigingvariancedoesnotdependontheobservation,thereisnoneedtohavey(xnew)tocomputetheIMSE. 96

PAGE 97

Incontrast,theweightedIMSEdependsontheobservationsthroughtheweightfunctionW(x).Theweightfunctioncannottakeintoaccountthenewobservation,sincetheresponseisnotavailable.Hence,whenexpressingtheweightedIMSEasafunctionofxnew,weupdateonlythevariancepartundertheintegral: IMSET(Xk;Yk;xnew)=ZDs2K(xjfXk;xnewg)W(xjXk;Yk)dx(5{20) wheres2K(xjfXk;xnewg)isthesameasinEq. 5{19 andW(xjXk;Yk)istheweightfunctionbasedontheexistingDoE.Usingthisexpression,wehavethesimpleformulationfortheinneroptimizationproblem: minxnew2DIMSET(Xk;Yk;xnew)(5{21) 5.4PracticalIssues 5.4.1SolvingTheOptimizationProblem FindingthenewobservationxnewbysolvingtheoptimizationproblemofEq. 5{21 is,inpractice,challenging.Indeed,theIMSETcriterioninEq. 5{20 mustbeevaluatedbynumericalintegration,whichiscomputationallyintensive.Besides,foranycandidatexnew,theKrigingmodelmustbereevaluatedwiththisnewobservationtoobtains2K(xjfXk;xnewg)).Thereforeweproposeheresomealternativesthatmaybeusedtoreducethecost. ApopularheuristictominimizesequentiallytheIMSEistondthepointwherethepredictionvarianceismaximum[ Sacksetal. ( 1989a ), Williamsetal. ( 2000 )],whichcanbeusedherewiththeweightedpredictionvariance.Thisstrategyhastheadvantageofsavingboththenumericalintegrationandtheinversionofanewcovariancematrix.However,thepredictionvarianceislikelytohavemany(localorglobal)maximizers,whicharenotequivalentintermsoftheIMSE.Inparticular,manyoptimaarelocatedontheboundaries,whichisveryinecientfortheIMSEminimization.Tocompensateforthisissue,onemayinarsttimegetalargenumberoflocaloptimausingadapted 97

PAGE 98

optimizationstrategies(multi-start,etc.),andinasecondtimeevaluatethoseoptimaintermsoftheweightedIMSEcriterion,andperformalocaloptimizationonthebestpoint. Avaluablecomputationalshortcutcanbeachievedintheupdateoftheinverseofthecovariancematrixwhenaddinganobservation.LetuscallCkthecovariancematrixcorrespondingtoaDoEwithkobservations.Then,thecovariancematrixoftheDoEaugmentedwiththek+1thobservationcanbewritten: Ck+1=2642cTnewcnewCk375(5{22) with: 2=k(xnew;xnew)(processvariance),and cTnew=[k(xnew;x1);:::;k(xnew;xk)]a1kvector. UsingSchur'scomplementformula,weget: Ck+1)]TJ /F5 7.97 Tf 6.58 0 Td[(1=26410)]TJ /F7 11.955 Tf 9.3 0 Td[(C)]TJ /F5 7.97 Tf 6.58 0 Td[(1kcnewIk3752641 2)]TJ /F12 7.97 Tf 6.58 0 Td[(cTnewC)]TJ /F13 5.978 Tf 5.75 0 Td[(1kcnew00C)]TJ /F5 7.97 Tf 6.59 0 Td[(1k3752641)]TJ /F7 11.955 Tf 9.3 0 Td[(C)]TJ /F5 7.97 Tf 6.58 0 Td[(1kcnew0Ik375(5{23) ThisformulaallowstocomputeCk+1)]TJ /F5 7.97 Tf 6.58 0 Td[(1fromCk)]TJ /F5 7.97 Tf 6.59 0 Td[(1withoutdoinganymatrixinversion,andcomputes2K(xjfXk;xnewg)atreasonablecost.Moredetailedcalculationsonthistopiccanbefoundin Marrel ( 2008 ). Ingeneral,thecriterionhasseverallocalminimizers.Then,itisnecessarytouseglobaloptimizationmethods,suchaspopulation-basedmethods,multi-startstrategies,etc.Inthetestproblemspresentedinthischapter,weoptimizethecriteriononanegridforlowdimensions,andusingthepopulation-basedCMA-ESalgorithm[CovarianceMatrixAdaptationEvolutionStrategies, Hansen&Kern ( 2004 )]forhigherdimensions.Experimentationshowedthatduetothenumericalintegrationprecision,thetargetedIMSEstrategybecomesinecientfordimensionshigherthanten. 98

PAGE 99

5.4.2Parallelization Itisoftenofinteresttoproducealgorithmsthatallowtocomputeseveralobservationsatatime,forinstancewhenthenumericalsimulatorcanberuninparallelonseveralcomputers.Thecriterionweproposeisfullyadaptabletoparallelstrategies.Indeed,thetargetedIMSEconsistsoftwoparts:theweightfunctionthatdependsontheresponsevalues,andthepredictionvariance,thatonlydependsontheobservationlocations.Thelaterquantitycanbeupdatedwithseveralobservationswithouttheneedofcalculatingtheresponse.Theweightfunction,ontheotherhand,remainsthesame. Letpbethenumberofobservationstobecomputedinparallel.Thecriteriontobeminimizedcanbewritten: IMSET)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(Xk;Yk;fx(1)new;:::;x(p)newg=ZDs2K)]TJ /F7 11.955 Tf 6.67 -9.68 Td[(xjXk;fx(1)new;:::;x(p)newgW(xjXk;Yk)dx(5{24) Thisstrategyislessaccuratethanthesequentialstrategy,sincetheweightfunctionisupdatedonlyeverypevaluations.However,weobservedthatforlowvaluesofp(5),theparallelizationhaslittleimpact,inparticularwhenthekriginguncertaintyisinitiallyhigh. Theparalleloptimizationproblemisofdimensiondp(wheredisthedesignspacedimension).ThedimensionofthenumericalintegrationremainsthesameasinEq. 5{20 .Theoptimizationcanbesolveddirectly,or,inordertoreducetheproblemdimension,byndingonepointatatime. 5.5ApplicationToProbabilityOfFailureEstimation Asmentionedinthebeginningofthischapter,whenapproximatingthelimit-state,itisclearthataparticulareortmustbegiventotheregionswhereitisclosetozero,sinceerrorinthatregionislikelytoaecttheprobabilityestimate.Naturally,thecriticalregioniswherethevalueoflimitstateisclosetozero. ThetargetedIMSEcriterionallowsustosampleintargetregions.However,substantialimprovementcanbegiven,bytakingintoaccountthedistributionofthe 99

PAGE 100

inputvariables.Indeed,letusconsiderthecaseoftwodistinctfailureregions,oneofitdominatingtheother(thatis,theprobabilitythattheinputfallsontotherstregionismuchlargerthantheprobabilitythatitfallsontotheother).Insteadofexploringequallythetwocriticalregions,itwillbemoreecienttospendmorecomputationaleortontheonethatwillaectmosttheprobabilityestimate.Inthesamesense,whenlearningasinglecriticalregion,itisecienttoexploreitonlywherethesamplesaremorelikelytobe. InsteadofusingtheindicatorfunctionontheKrigingmean,weusethefullKriginginformationbycomputing,ateachsamplingpoint,theprobabilitythattheresponseexceedsthethreshold: ^Pf=1 NNXi=1(i)k(0)(5{25) where(i)kdenotesthecumulativedistributionfunction(CDF)oftheKrigingmodelatxi(N(mk(xi);s2k(xi))). IftheKrigingvarianceissmall,theCDFbecomesequivalenttotheindicatorfunction,being1iftheKrigingmeanexceedsthethresholdand0otherwise.Ontheotherhand,whenthevarianceishighorthepredictedresponseclosetothethreshold,usingtheKrigingdistributionoersasmoothingeectbygivinganumberbetweenzeroandoneinsteadofaBooleannumber. Toaddresstheprobabilitydistributionofinputvariables,wemodifytheweightedIMSEcriterionbyintegratingtheweightedMSEnotwithauniformmeasure,butwiththePDFoftheinputvariables: IMSET=ZDs2K(x)W(x)d(x)=ZDs2K(x)W(x)f(x)dx(5{26) wheref(x)isthePDFofthelawwithrespecttotheLebesguemeasure. 100

PAGE 101

5.6NumericalExamples Inthissection,weevaluatetheaccuracyandeciencyofthemethodthroughnumericalexamples.Weconsiderthettingofanalyticaltestfunctionsandrealizationsofrandomprocesseswithknowncovariancestructures.Inthelattercase,thereisnomodelingerrorwhenusingaKrigingapproximation;theerrorisonlyduetothelackofinformation. 5.6.1Two-DimensionalExample Therstexampleistheapproximationofatwo-dimensionalparametricfunctionfromtheoptimizationliterature(Camelbackfunction, Chippereldetal. ( 1994 )).Theoriginalfunctionismodied(boundsaredierentandanegativeconstantisadded)andthetargetischoseninordertohavetwofailureregions,onedominatingtheother.Thetwo-dimensionaldesignspaceisgivenas[1;1]2.Theperformancefunctionisdenedas f(u;v)=4)]TJ /F1 11.955 Tf 11.96 0 Td[(2:1u2+1 3u4u2+2 3uv+16 9)]TJ /F1 11.955 Tf 9.3 0 Td[(4+16 9v2v2)]TJ /F1 11.955 Tf 11.96 0 Td[(0:7(5{27) whereu=1:2u)]TJ /F1 11.955 Tf 11.96 0 Td[(0:1andv=0:9v. Forbothnumericalintegrationandoptimization,thedesignspaceisdividedbya3232grid.Wepresenttheresultsforthefollowingconguration: TargetvalueTischosenas1.3, Gaussian-basedweightfunctionisused,withparameter"=0:2, InitialDoEconsistsofthefourcornersandthecenterofthedomain, 11pointsareaddediterativelytotheDoEasdescribedintheprevioussection. AnisotropicGaussiancovariancefunctionischosenfortheKrigingmodel.Thecovarianceparameters(processvariance2andrange)areestimatedfromtheinitial5-pointDoE,andre-estimatedaftereachnewobservation,usingtheMatLabtoolboxGPML[ Rasmussen&Williams ( 2006 )].ThenalresultsarepresentedinFigure 5-3 Figure 5-3 A)istheplotofthetruefunction,andFigure 5-3 B)isthatoftheKrigingmean.InthecontourplotinFigure 5-3 C),itisshownthattherearetwocriticalregions. 101

PAGE 102

Figure5-3. Optimaldesignafter11iterations.A)Truefunction.B)Krigingexpectation.D)Krigingvariance.C)Contourlinesofthetruefunctionatlevels[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"],whichdelimitthetargetregions;theboldlinecorrespondstothetargetcontour.Mostofthetrainingpointsarechosenclosetothetargetregion.TheKrigingvarianceisverysmallintheseregionsandlargeinnon-criticalregions. After11iterations,thesequentialstrategyusedfourpointstoexploretherstcriticalregion,threepointstoexplorethesecondregion,andfourpointsforspace-lling.AsshowninFigure 5-3 D),theKrigingvariancebecomessmallnearthecriticalregions,whileitisrelativelylargeinthenon-criticalregion. Figure 5-4 showstheevolutionofthetargetcontourlineforthekrigingexpectation,whichisagoodindicatorofthequalityofthesurrogate.Weseethatbecausetherstfouriterations(Figure 5-4 B))areusedforspace-lling,theKrigingcontourlineisverydierentfromtheactualone.Aftereightiterations(Figure 5-4 C)),thetwotargetregionsarefoundandadditionalsamplingpointsarechosenclosetotheactualcontourline.Finalstate(Figure 5-4 D))showsthatthekrigingcontourlineisclosetotheactualone. 102

PAGE 103

Figure5-4. EvolutionofKrigingtargetcontourline(mixedline)comparedtoactual(plainline)duringthesequentialprocess:A)Initial,B)afterfouriterations,C)aftereightiterations,D)nal. 5.6.2Six-DimensionalExample Inthesecondexample,weconsiderarealizationofasix-dimensionalisotropicGaussianprocesswithGaussiancovariancefunction.Here,theerrorintheKrigingmodelisonlyduetothelackofdata:thereisnoerrorinestimatingthecovarianceparameters,andasymptotically(withverylargenumberofobservations)theKrigingtsexactlytherealizationoftheGaussianprocess.Thedesignspaceis[)]TJ /F1 11.955 Tf 9.3 0 Td[(1;1]6.Inordertolimitthecomplexity(numberofnon-connectedtargetregions)ofthetargetregion,weaddalineartrendtotheGaussianprocess.Wetake2=1,=0:1and=[1:::1]. TheweightedIMSEcriterioniscomputedbyQuasiMonte-Carlointegration.TheintegrationpointsarechosenfromaSobolsequence[ Sobol ( 1976 )]toensureagoodspacellingproperty.Ateachstep,theoptimizationisperformedusingthepopulation-basedoptimizerCMA-ES[ Hansen&Kern ( 2004 )]. 103

PAGE 104

Wepresenttheresultsforthefollowingcongurations: Targetvalueischosenas2 Gaussian-basedweightfunctionisused,with"=0:05. InitialDoEconsistsof20pointschosenfromLatin-hypercubesampling(LHS) 70pointsareaddediterativelytotheDoE. Forcomparisonpurpose,wegenerateaclassicalspace-llingDoEthatconsistsof90LHSpointswithmaximumminimumdistancecriterion. First,werepresenttheerrorat10,000(uniformlydistributed)datapoints(Figure 5-5 ).Theclassicalspace-llingDoEleadstoauniformerrorbehavior,whiletheoptimalDoEleadtolargeerrorswhentheresponseisfarfromthetargetvalue,whilesmallerrorswhenitisclosetothetarget. Inordertoanalyzetheerrorinthetargetregion,wedrawtheboxplotsoftheerrorsforthetestpointswhereresponsesareinsidethedomains[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+"](Figure 5-6 A))and[)]TJ /F1 11.955 Tf 9.3 0 Td[(2";+2"](Figure 5-6 B)).Comparedtothespace-llingstrategy,theoptimaldesignreducessignicantlytheerror.Inparticular,onbothgraphsthelower-upperquartilesintervalis2.5timessmallerfortheoptimalDoE. 5.6.3ReliabilityExample ThelimitstatefunctionistakenastheCamelbackfunctionusedintheprevioussection.LetUandVbeindependentGaussianvariableswithzeromeanandstandarddeviationtakenat0.28;i.e.,U;VN)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(0;0:282.Then,thefailureisdenedwhenfbecomesgreaterthan1.3.Thus,thelimitstateisdenedas G=f(U;V))]TJ /F1 11.955 Tf 11.95 0 Td[(1:3(5{28) Forthisexample,wegeneratetwoadaptivedesigns:therstisgeneratedsequentiallyasdescribedpreviously,withuniformintegrationmeasure(Eq. 5{7 );thesecondisgeneratedusingtheinputdistributionasintegrationmeasure(Eq. 5{26 ).BothusethefourcornersandthecenterofthedomainasstartingDoEand11iterationsareperformed.For 104

PAGE 105

Figure5-5. ComparisonoferrordistributionfortwoDoEs:A)optimalDoEandB)classicalLHS.Thex-axisisthedierencebetweenthetruefunctionandthethreshold,they-axisistheerror.Fiveverticalbarsaredrawnat)]TJ /F1 11.955 Tf 9.3 0 Td[(2",)]TJ /F4 11.955 Tf 9.29 0 Td[(",0,+"and+2"forthetargetregion.TheerrorisonaveragesmallerfortheLHSdesign,buttheoptimalDoEreducessubstantiallytheerrorinthetargetregion. comparisonpurpose,a16-pointfullfactorialdesignisalsoused.ItisfoundthataSimpleKrigingmodel(UKwithoutlineartrend)withisotropicGaussiancovariancefunctionapproximateswellthefunction.ThecovarianceparametersarecomputedusingthetoolboxGPMLforalltheDoEs. Figure 5-7 drawsthetwooptimaldesignsobtainedandthefullfactorialdesigns.Bothoptimaldesignsconcentratethecomputationaleortonthefailureregionsandthecenterofthedomain.WithuniformmeasureintegrationinFigure 5-7 A),theDoEismorespace-llingthantheonebasedonthedistribution(showninFigure 5-7 D)).BytakingtheinputdistributionintoaccountinFigure 5-7 B),weseethatalltheobservationsarelocatedrelativelyclosetothecenterofthedomain.Partofeachtargetregionsisnotexplored,sinceitisfarfromthecenter. Finally,weperform107MCSonthethreemetamodelstocomputetheprobabilityoffailureestimates.107MCSarealsoperformeddirectlyonthetestfunctiontoobtainthetrueprobabilityoffailure.ResultsarereportedinTable 5-2 .Thefull-factorial 105

PAGE 106

Figure5-6. and[)]TJ /F1 11.955 Tf 9.3 0 Td[(2";+2"].]BoxplotsoferrorsfortheLHSandoptimaldesignsforthetestpointswhereresponsesareinsidethedomainsA)[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+"]andB)[)]TJ /F1 11.955 Tf 9.29 0 Td[(2";+2"].Erroratthesepointsissubstantiallysmallerfortheoptimaldesignsforbothintervals. designleadsto77%error,whilebothoptimaldesignsleadtoasmallerror.Substantialimprovementisobtainedbytakingtheinputdistributionintoaccount. Table5-2. ProbabilityoffailureestimatesforthethreeDoEsandtheactualfunctionbasedon107MCS.Thestandarddeviationofallestimatesisoftheorderof210)]TJ /F5 7.97 Tf 6.59 0 Td[(5. DoEFullFactorialOptimalwithoutinputdistributionOptimalwithinputdistributionProbabilityestimatebasedon107MCS Probabilityoffailure(%)0.170.700.770.75 Relativeerror77%7%3% 5.7ConcludingRemarks Inthischapter,wehaveproposedtoimprovetheaccuracyofsurrogatemodelsbygeneratingdesignsofexperimentsadaptedtothemodelapplication.Inparticular,wehaveaddressedtheissueofchoosingadesignofexperimentswhentheKrigingmetamodelwasusedtoapproximateafunctionaccuratelyaroundaparticularlevel-set,whichoccursinconstrainedoptimizationandreliabilityanalysis.Wedenedanoriginalcriterion,whichis 106

PAGE 107

Figure5-7. OptimaldesignwithA)uniformintegrationmeasure,B)inputdistributionintegrationmeasure;C)Fullfactorialdesignswith16points;D)Inputdistribution. basedonanexplicittrade-obetweenexplorationofregionsofinterestandreductionofuncertaintyofthemodel,andansequentialproceduretobuildtheDoEs. Throughnumericalexamples,weshowedrstthattheadaptiveDoEwaseectivefordetectingthetargetregions(wheretheresponsewasclosetothetargetlevel-set)andimprovelocallytheaccuracyofthesurrogatemodel.Then,themethodwasappliedtoareliabilityexample,anditwasfoundthatimprovinglocallytheaccuracyresultedingreatimprovementoftheaccuracyofthereliabilityestimation. However,ithasbeenfoundsomelimitationstothemethod,whichwerenotsolvedhereandrequiresfutureworktoapplythemethodtoawiderangeofproblems: 107

PAGE 108

Sinceitreliesonnumericalintegration,themethodcanbecomecomputationallyexpensiveifalargenumberofintegrationpointsisneededtocomputethecriterion.Wefoundthatfordimensionshigherthanten,thecriterionminimizationbecomescriticalwithouttheuseofcomplexandproblem-dependentnumericalprocedures,suchasdimensionreductionoradaptednumericalintegration. Secondly,itisimportanttorecallthatitisamodel-believerstrategy,sincethecriterionisentirelybasedontheKrigingmodel.Althoughsequentialstrategiesallowsomecorrectionofthemodelduringtheprocess(throughre-estimationoftheparametersforinstance),thesuccessofthemethodwillstronglydependonthecapabilityoftheKrigingmodeltottheactualresponse. Finally,itseemsthattheideaofweightedIMSEandsequentialstrategiestoconstructDoEscanbeappliedtomanyotherpurposeswherethereisaneedfortrade-obetweenexplorationofregionsofinterestandreductionoftheglobaluncertaintyinthemetamodel.Here,wedenedaweightfunctiontoaccountfortheproximitytosometargetvalue.Potentialapplicationscouldbefoundinoptimization,ordetectionofregionsofhighvariations,etc.,thedicultybeinginanadapteddenitionoftheweightfunction. 108

PAGE 109

CHAPTER6OPTIMALALLOCATIONOFRESOURCEFORSURROGATEMODELING 6.1SimulatorsWithTunableFidelity Anumericalsimulatorisanapproximationtoarealphenomenonandisbydenitionasourceoferror.Thewayasimulatorresponsefollowstherealfunctionofinterestisoftencalleddelity.Manyengineeringapplicationsusetwotypesofsimulators,alow-delitymodelandahigh-delitymodel.Atypicallow-delitymodelisbasedongreatsimplicationofthesystemgeometryandphysics(forinstancebarsorbeamsmodeling),andahigh-delitymodelistypicallyaniteelementsmodel.High-delitymodelsprovideanaccurateapproximationtothefunctionofinterest,butarecomputationallyexpensive,whilelow-delitymodelsareeasytorunbutcontainlargeerror. However,manynumericalsimulatorscanoeralargerangeofdelities,bytuningfactorsthatcontrolthecomplexityofthemodel.Forinstance,theprecisionoftheresponseofaniteelementanalysis(FEA)canbecontrolledbythemeshingdensityorelementorder.AnotherexampleiswhentheresponseisobtainedusingMonte-Carlomethods:theaccuracy(measuredbythestandarddeviationorcondenceintervals)isinverselyproportionaltothesquarerootofthenumberofMCsimulations.Forbothexamples,thegainindelityisatapriceofcomputationaltime.Thistopichasbeenaddressedonlyrecentlyinthecomputerexperimentcommunityandisusuallyfoundunderthenameofstochasticsimulators[ Kleijnen&vanBeers ( 2005 ), Boukouvalas&Cornford ( 2009 ), Ioossetal. ( 2008 ), Ginsbourger ( 2009 )]. Whenttingametamodeltosuchtypeofsimulators,thesamplingstrategymaytakeintoaccountthetunabledelity.Indeed,whenthecomputationalresourceislimited,onehastotrade-othedelityandthenumberofsimulations.Lowerdelityallowsabetterexplorationofthedesignspace,whilehigherdelityprovidesmoretrustinthesimulatorresponses.Besides,havingtunabledelityistobeconsideredasanadditionaldegreeoffreedom:byallocatingdierentcomputationaltimeforeachsimulation,thequalityof 109

PAGE 110

thesurrogatecanbeimprovedwithoutadditionalcostcomparedtoauniformdelitysituation. TheoptimalDoEformulation,aspresentedinChapter2,consistsofselectingthepointstomaximizeacriterionofqualityforametamodelconstruction.Here,wedisposeofadditionaldegreesoffreedom,sincewecanchoosethenumberoftrainingpoint,andtunethemodeldelityforeachtrainingpoint(undertheconstraintofatotalcomputationalbudget). Elfving ( 1952 )pioneeredworkintheareaofoptimalallocation.Foralinearmodelheproposedtorepeatexperimentsatsometrainingpoints,andallocateadierentnumberofrepetitionstoeachpointaccordingtoanoptimalitycriterion. Kiefer ( 1961 )generalizedthisideaofproportioningexperimentsinordertoobtainthemostaccurateregressioncoecients. Fedorov ( 1972 )proposedacontinuousversionoftheproblem,anddevelopediterativestrategiestondoptimaldesigns. Thisobjectiveofthischapteristoprovideecientstrategiesfordesigningtheexperimentsofsimulatorswithtunabledelity.First,wedescribethreeexamplesofsuchtypeofsimulatorsthatmodelengineeringsystems.Afterformalizingtheproblemofoptimalallocationofresource,wereviewsomeoftheimportantresultsofoptimaldesignsforregression.Finally,weproposeanoriginalapproachforoptimalallocationwhenKrigingmodelsareconsidered. 6.2ExamplesOfSimulatorsWithTunableFidelity 6.2.1Monte-CarloBasedSimulators Weconsiderfunctionsoftheformf(x;),thatdependsonbothdeterministicvariablesxandthedistributionofsomerandomparameters.Thefunctionf(x;)isdeterministic,butitis,generally,notknownanalytically.Hence,weestimateitusingMonte-Carlosimulations(MCS).Foragivendesignx,wewrite: ^f(x;i=1:::k)=^f(x;1;2;:::;k)(6{1) where^fistheestimatoroffand1;2;:::;karerealizationsof. 110

PAGE 111

TheestimationoftheprobabilityoffailureofasystemisatypicalexampleofaMonte-Carlosimulator.ThelimitstateG(x;)iscomputedforseveralrealizationsof,andtheprobabilityoffailureestimate^Pf(x),whichcorrespondsto^f(x;i=1:::k)inEq. 6{1 isgivenby: ^Pf(x)=1 kkXi=1I[G(x;i)0](6{2) whereIistheindicatorfunction. Monte-Carlomethodsconvergewiththesquarerootofthesamplesizek.Thatis,whenkischosensucientlylarge,theestimator^fisnormallydistributedanditsvarianceisinverselyproportionaltok: var^f(x)=Eh^f(x))]TJ /F4 11.955 Tf 11.95 0 Td[(f(x)i=cst k(6{3) Forthereliabilityexample,^Pf(x)followsabinomiallawwithparameterp=Pf(x),henceitsvarianceisgivenby: varh^Pf(x)i=Pf(x)(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pf(x)) k(6{4) 6.2.2RepeatableExperiments Repeatableexperimentsarenotinthemselvesanumericalsimulatorwithtunabledelity,butcanbeconsideredasadiscreteversionoftheproblem.Somenumericalsimulators,andmostphysicalexperiments,benetfromrepetitions.Thisisnotthecaseofaniteelementanalysisforinstance:repeatingtwotimesthesamesimulationwillgivetheexactsameanswerandwillnotprovideanyadditionalinformation.Anumericalsimulatorthatbenetsfromrepetitionhastypicallyaresponseoftheform: yobs(x)=y(x)+"(x)(6{5) wherey(x)istheactualresponseandtheerrorterm"(x)isarandomvariable.Hence: var[yobs(x)]=var["(x)](6{6) 111

PAGE 112

Now,saytheexperimentisrepeatedktimes.Assumingthatthenoiseisindependentfromoneobservationtoanother,wedeneyeqasthesingleobservationequivalenttothekobservationsyobs(j)(x),whichisthemeanofthekobservations: yeq(x)=1 kkXj=1yobs(j)(x)(6{7) Thevarianceofyeqisinverselyproportionaltothenumberofrepetitions: var[yeq(x)]=var["(x)] k(6{8) 6.2.3FiniteElementAnalysis Thequalityofaniteelementanalysismostlydependsonthemeshingchosentomodelthestructure.Themeshingcomplexityisdenedbytheelementtypeandtheelementsize,ormeshingdensity.Increasingtheelementcomplexityorthedensityresultsinagaininaccuracy,butatapriceofcomputationaltime. Figure 6-1 showsanexampleoftheevolutionofaFEAresponsewhenthecomplexityofthemodelistunedtodierentlevels.TheFEAusedisthetorquearmmodelpresentedinChapter3.TheresponseconsideredisthemaximumVonMisesstressonthestructure.Themeshdensityisatunableparameterthatdenesthecomplexityofthemeshing.Adensityofonecorrespondstoacrudemesh,whileadensityof25correspondstoaverycomplexandexpensivemodel.Theresponsefordensity25canbeconsideredastheactualvalueofthemaximumstress. Thegraphshowsthatthesimulatorerror(tothe25-mesh-densityreference)decreaseswiththemodelcomplexity.Thelowestdelitymodelhasanerrorof24%,whilefordensitiesgreaterthan15theerrorislessthan5%.Aswellasforpreviousexamples,itispossibletorelatetheerroramplitudetothecomputationaleortgivenforthesimulationrunandobtainanequationoftheform: var["(x)]=f(x;t)(6{9) 112

PAGE 113

Figure6-1. EvolutionoftheresponseoftheFEAforthetorquearmexample,whenthecomplexityofthemodel(meshdensity)increases. wherefisafunctiontodetermine,xthedesignparametersandtthecomputationaltime. However,therelationsbetweenmeshdensity,computationaleortanderroramplitudearecomplexandproblemdependent.Itdependsonthechoiceoftheelements,themeshingtechnique,theniteelementsoftwaremethodofresolutionofthesystem,etc.Hence,itisimpossibletoprovideageneralsimplerelationbetweentheresponsevarianceandthecomputationaltimeinthegeneralcase,andthisrelationhastobefoundforagivenproblem. 6.3OptimalAllocationOfResource Aclassicaldesignofexperimentsformulationconsistsofndingasamplingstrategythatmaximizesacriterionofinterest.Thedesignofexperimentsisthendenedbythesamplingpointlocations.Inourframework,thedesignofexperimentmustbedenedasthecombinationoftrainingpointsandcomputationalresourceallocatedtothatpoint.Then,adesignwithnpointsisdenedas: =8><>:x1;:::;xnt1;:::;tn9>=>;(6{10) 113

PAGE 114

Thexi'saretrainingpointlocations,andtheti'sarerealpositivenumbersthatrepresentthecomputationaltimeallocatedforeachtrainingpoint.Thesumoftheti'sisboundedbyaconstant: 0ti;nXi=1tiTtotal(6{11)Ttotalrepresentsthetotalcomputationaltimeavailableforthedesignofexperiments.Letbeafunctionalofinteresttominimizethatdependsonbothtrainingpointsandcomputationaltime.Adesigniscalledoptimalifitachieves: =argmin[()](6{12) Suchthat:nXi=1tiTtotal Findinganoptimaldesignisadicultandcomplexproblemthathasbeenwidelyaddressedintheliteratureintheframeworkofregressionwithrepeatableexperiments.Whentherelationbetweenthecomputationaltimetandthesimulatorresponsevarianceisunknownorcomplex,theoptimizationproblemcanbecomeverychallenging,andnogeneralanswerisprovidedintheliterature.Inthefollowing,wefocusonsimulatorsforwhichtheresponsevarianceisinverselyproportionaltothecomputationaltime.Thisisthecase,asseeninSection 6.2.1 ,ofreliabilityestimates. Inthetwofollowingsections,weconsidertheframeworksofregressionandkriging.Resultsforregressionarewell-knownanditssectionismostlydevotedtoreviewandbibliography.Onthecontrary,verylittleisknownforthekrigingarea,andtheapproachesweproposeareoriginal. 6.4ApplicationToRegression 6.4.1ContinuousNormalizedDesigns Inthissection,theobservationsareassumedtoberepeatable,andthatnoiseinallobservationshasthesamevariance(homoscedasticcase).WecallnthenumberoftrainingpointsandNthenumberofexperiments(countingrepetitions),Nn. 114

PAGE 115

Letribethenumberofobservationsatxi,allobservationsaredenotedyij(i=1;2;:::;n;j=1;2;:::;ri).ThedesignNconsistsofthecollectionoftrainingpointsxiandnumbersofrepetitionsri: N=8><>:x1;:::;xnr1;:::;rn9>=>;;0ri;nXi=1ri=N(6{13) Foragiventrainingpointxi,theequivalentobservation(seeSection 6.2.2 )isdenotedbyyi: yi=r)]TJ /F5 7.97 Tf 6.58 0 Td[(1iriXj=1yij(6{14) Let2bethevarianceofasingleobservation;assumingindependanceoftheobservations,thevarianceofyiis: var(yi)=r)]TJ /F5 7.97 Tf 6.58 0 Td[(1i2(6{15) Then,theFisher'sinformationmatrixbecomes: M(N)=f(X)T)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X)=f(X)Tdiag)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(2r)]TJ /F5 7.97 Tf 6.59 0 Td[(11;2r)]TJ /F5 7.97 Tf 6.59 0 Td[(12;:::;2r)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X)=)]TJ /F5 7.97 Tf 6.58 0 Td[(2f(X)T[diag(r1;r2;:::;rn)]f(X) (6{16) NormalizingbyN, M(N)=N)]TJ /F5 7.97 Tf 6.58 0 Td[(2f(X)T[diag(r1=N;r2=N;:::;rn=N)]f(X)(6{17) TheFisherinformationmatrixisproportionaltoN,andfollowing Fedorov ( 1972 ),itispossibletodene,withoutanylossofgenerality,anormalizeddesignasacollectionofsupportpointsxiandtheircorrespondingproportionsofexperimentspi(withpi=ri=N): =8><>:x1;:::;xnp1;:::;pn9>=>;;0pi;mXi=1pi=1(6{18) 115

PAGE 116

Furthermore,acontinuousnormalizeddesignisdenedwhenpicantakeanyrealvaluebetween0and1.ContinuousdesignsaregoodapproximationsofrealityforlargeN.ThecorrespondingFisher'sinformationmatrixis: M()=f(X)T[diag(p1;p2;:::;pn)]f(X)=nXi=1pif(xi)Tf(xi)(6{19) Itisimportanttonoticeherethatsuchnormalizationispossibleonlywhentheobservationnoisevarianceisinverselyproportionaltocomputationaltime.Theresultsdescribedinthefollowingsectionsonlystandwhenthisrelationistrue. 6.4.2SomeImportantResultsOfOptimalDesigns Atthispoint,weintroducethedispersionfunctionforlinearregressionforapredictionpointx: d(x;)=f(x)TM())]TJ /F5 7.97 Tf 6.58 0 Td[(1f(x)(6{20) Thedispersionfunctiondirectlyrelatestothepredictionvarianceby: var(^y)=2N)]TJ /F5 7.97 Tf 6.59 0 Td[(1d(x;)(6{21) Animportantresultindesignoptimalityisthefamousequivalencetheoremdemonstratedby Kiefer&Wolfowitz ( 1960 ).Itstatesthatthethreefollowingproblemsareequivalent: mindet(M()) (6{22) minmaxxd(x;) (6{23) ndsuchthat:maxxd(x;)=p (6{24) pbeingthenumberofregressioncoecientstoestimate. Inpractice,itmeansthatD-optimaldesignsarealsoG-optimal(andvice-versa),thatisdesignsthatminimizetheuncertaintyontheregressioncoecientsalsominimizethemaximumpredictionvariance,andfornormalizedoptimaldesignsthevalueofthemaximumpredictionvariancevalueisequaltop.However,theseresultsarerestrictedto 116

PAGE 117

continuousdesigns;whenthetotalnumberofobservationsNisnotalotlargerthanthenumberoftrainingpointsn,thepitakediscretevaluesandtheabovetheoremdoesnotholdingeneral. Anotherresultofgreatpracticalrelevanceisthatthereexistsanoptimaldesignthatcontainsnomorethann0points[see Fedorov ( 1972 )]: n0=p(p+1) 2+1(6{25) Suchresultsareveryusefulinpractice,whenalgorithmsareusedtoconstructoptimaldesigns.WiththeD-Gequivalence,wegettheintuitionthatsupportpointsmaybetakenwherethepredictionvarianceismaximal.Thelattertheoremstatesthat,forlinearregression,optimalsamplingstrategiesconsistofconcentratingthecomputationalresourceonalimitednumberof(well-chosen)trainingpoints,ratherthanusingalargenumberofpoints. 6.4.3AnIllustrationOfAD-OptimalDesign Now,weillustrateD-optimaldesignswhentheresponseisasecondorderpolynomialwithinteractionsintwodimensions: y=0+1x1+2x2+3x21+4x22+5x1x2(6{26) Theoptimaldesignconsistsofthefull-factorialdesignwithninepoints,withthefollowingweights(Fedorov,1972): 14.6%atthecorners 8.0%atthemiddlesoftheedges 9.6%atthecenter Figure 6-2 representstheoptimaldesignandcorrespondingpredictionvarianceprole.ItcanbeseenthatG-optimalityisachieved,sincethepredictionvarianceattainsitsmaximump=6(seeEq. 6{24 )onseverallocations(center,corners,andmiddlesoftheedges).Forthismodel,optimalityisachievedbyallocatingapproximatelytwotimesmore 117

PAGE 118

computationalresourcestothetrainingpointsonthecornersthantotheothertrainingpoints.Indeed,thecornersofthedomainaretheregionswheretheuncertaintyislikelytobemaximal,soitseemslogicaltocompensateforthisuncertaintybyallocatingmoreresourcesthere. Figure6-2. A)D-optimaldesignforasecond-orderpolynomialregressionandB)correspondingpredictionvariance.Theverticalbarsrepresenttheproportionofcomputationaleortgiventoeachtrainingpoint. Figure 6-3 representsthefull-factorialdesignbutwithhomogeneousresourceallocationandthecorrespondingpredictionvariance.Thepredictionvarianceproleshowsthatsuchdesignisnotoptimal,sincethevarianceismuchhigheronthecornersthanonthecenterofthedomain.Themaximumpredictionvarianceis7.25,whileitisequaltop=6fortheoptimaldesign. Finally,Figure 6-4 representsadesignwithalargenumberofpointsuniformlyspreadonthedesignregion,whichisanaturalalternativetosmalldesigns.Here,wechosea36-pointFFdesign;theobservationnoiseisveryhighsinceonly1=36ofcomputationaltimeisallocatedtoeachtrainingpoint.Suchdesigndoesnotappeartobeagoodsolution,sincethepredictionvariancehasveryhighvaluesontheedgesofthedomain.Themaximumpredictionvarianceis13.45. 118

PAGE 119

Figure6-3. A)Full-factorialdesignwithhomogeneousallocationofcomputationaleortandB)correspondingpredictionvariance. Figure6-4. A)36-pointFull-factorialdesignwithhomogeneousallocationofcomputationaleortandB)correspondingpredictionvariance. 6.4.4AnIterativeProcedureForConstructingD-OptimalDesigns Manyalgorithmicprocedureshavebeenproposedinthepasttogenerateoptimaldesigns.Here,wedetailtherst-orderalgorithmpresentedbyFedorovinhis1972book.Morerenedtechniquescanbefoundin Cook&Nachtsheim ( 1980 ) Fedorov&Hackl ( 1997 ), Wu ( 1978 ), Molchanov&Zuyev ( 2002 ). TheprincipleofFedorovalgorithmistoaddpointssequentiallytothedesign,andtotransferaproportionofexperimentsfromtheexistingpointstothenewone.Theprocedureisasfollows: 119

PAGE 120

Startingfromacurrentdesignsobtainedaftersiterations,therststepistondonepointwherethepredictionvarianceismaximal,thatis: xnew=argmax[d(x;)](6{27) Then,thenewdesigns+1is: s+1=(1)]TJ /F4 11.955 Tf 11.96 0 Td[()s+1+(xnew)(6{28) Theproportionofexperimentsforthenewpoint()ischosensuchthatdet[M(s+1)]ismaximal,showingtheformula[ Fedorov&Hackl ( 1997 ),p.47]: opt=d(x;S))]TJ /F4 11.955 Tf 11.95 0 Td[(p (d(x;S))]TJ /F1 11.955 Tf 11.95 0 Td[(1)p(6{29) Thesequencefsgiscalledoptimalinthesensethatitscriterionvalueconvergestotheoptimumvalue: lims!1(s)=()(6{30) Whenaverylargenumberofiterationsisperformed,theresultingdesigncancontainalotofpoints.Then,wecanreducethenumberofpointsby: discardingpointswithsmallweightsnotclosetoanygroupofpointswithhighweights agglomeratingpointsclosetoeachother(thenewpointisthebarycenterofthepoints,theweightisthesumoftheweights) 6.4.5ConcludingRemarks Theproblemofoptimalallocationofresourcehasbeenwidelystudiedintheregressionframework.Theremarkablepropertiesofoptimaldesignsallowproposingecientalgorithmstogeneratesuchdesigns.Theresultsaredevelopedforrepeatableexperiments,butfullyapplytoanysimulatorwithtunabledelitythathasaresponsevarianceerrorinverselyproportionaltocomputationaltime.WhenD-orG-optimalityareconsidered,optimalsamplingstrategiesconsistofconcentratingthecomputational 120

PAGE 121

resourceonalimitednumberoftrainingpoints,withnon-uniformallocationofresourcesbetweenpoints. 6.5ApplicationToKriging 6.5.1ContextAndNotations Inthissection,weconsidercontinuousdesigns,butwithweightspi'sboundedbyaconstantnotequaltoone: 0pi;nXi=1pi=1 2(6{31) Wewillseelaterthatincontrasttotheregressioncase,thequantity1 2cannotbenormalizedsinceithasannon-linearimpactontheresults.Itrepresentsthecomputationalresourceavailableforthedesign.Here,theweightsaretheinverseofthevariancesofthemeasurementerroratxi: pi=1 var["(xi)](6{32) Assumingthattheerrorvarianceisproportionaltocomputationaltime,Eqs. 6{31 and 6{32 reectsthefactthatcomputationalresourcesremainconstantforall.Ifthedesignconsistsofasinglesupportpoint,theerrorvarianceis2;ifthedesignconsistsofndistinctpointswithuniformweights,theerrorvarianceisuniform,equalto: var("i)=n2(6{33) Inthissection,wealwaysconsiderthatthevariancesareequal. Thekrigingmodelstudiedisthesimplekriging(SK),whichtrendconsistsofaknownconstant(seeSection 2.1.3 ).Then,theSKbestpredictorisgivenbytheequation: mK(x)=+k(x)TK)]TJ /F5 7.97 Tf 6.59 0 Td[(1(Y)]TJ /F4 11.955 Tf 11.95 0 Td[(1)(6{34) andtheSKpredictionvarianceis: sK2(x)=2)]TJ /F7 11.955 Tf 11.95 0 Td[(kT(x)K)]TJ /F5 7.97 Tf 6.59 0 Td[(1k(x)(6{35) 121

PAGE 122

Consideringacomputationalbudget1=2andequalvariances,thecovariancematrixKisgivenby: K=K+=(k(xi;xj))+0BBBB@n20...0n21CCCCA(6{36) Toanalyzethequalityofamodelt,weconsidertheIMSEcriterion: IMSE=ZDsK2(x)d(x)(6{37)beinganintegrationmeasure,usuallytakenuniform. 6.5.2AnExploratoryStudyOfTheAsymptoticProblem TheobjectiveofthisstudyistoquantifytheevolutionofthequalityoftheSKmodelwhenthenumberofobservationsincreases,whilethecomputationalresourceremainsthesame.Inotherwords,wewanttoseeifitisbetterusingafewaccurateobservations,ormanyinaccurateones. Anexploratorystudyisperformedrst.Weconsiderthetofatwo-dimensionalstationaryprocessesinD=[0;1]2.TherstprocesshasaGaussiancovariancefunction(smooth),thesecondhasanexponentialcovariancefunction(irregular). Wechoose2=1and2=10)]TJ /F5 7.97 Tf 6.58 0 Td[(3forbothprocesses,arangeof=3:1forthegaussiancovarianceand=2fortheexponential.Weconsiderseveralsamplesizesnfrom10to150.Foreachsamplesize,wegenerate500designs(p)n,thexibeinguniformlytakeninD,thepibeingequalton2.Then,wecomputetheIMSEcriterionforeach(p)n,usingnumericalintegrationona3232grid.Figure 6-5 showstheevolutionofthemean,90%condenceintervalandminimumIMSEcriterion(overthe500designs)withrespecttom,forbothcovariancefunctions.NotethattheIMSEcriterionisindependentoftheobservations,sotheresultsdependonlyonthecovariancestructureandparameters. WeseethatthemeanandminimumIMSEdecreasewhenthenumberofobservationsincreases.Bothquantitiesseemtoconvergetoanon-zeroconstant.Thecondence 122

PAGE 123

Figure6-5. EvolutionoftheIMSEwithrespecttothenumberofobservationsforuniformrandomdesignswithA)Gaussiancovariance,B)Exponentialcovariance. intervalwidthalsodecreases.Hence,wecanconclude,forthesetwoexamples,thatitispreferableforaconstantcomputationalbudgettohavealargenumberofpointswithlargevariancesspreadalloverthedesigndomain,ratherthanafewaccurateones. Besides,thedesignswithlargenumberofpointsaremorerobust,sincethecondenceintervaldecreaseswhenthenumberofobservationsincreases.Thisislogicalsince,withasmallnumberofpoints,havinganobservationatawronglocationhasastronginuenceontheglobalaccuracyofthemodel. Thus,thegoalistostudythebehaviorofs2K(x)andtheIMSEwhenntendstoinnity,thexibeinguniformlyspreadontothedesigndomainD.Inparticular,weaimatndinganalyticalexpressionsofbothvarianceandIMSE,whichcanbeusedforcomparisontodesignswithanitenumberofsupportpoints. 6.5.3AsymptoticPredictionVarianceAndIMSE Weassumethattheobservationsaretakenrandomlyaccordingtothemeasure.canbeeithercontinuousonthedesigndomainD,ordiscrete.Typicallyintherstcase,istheuniformmeasureonD;then,eachtrainingpointistakenrandomlywithuniformdistributiononD(whichisthecaseoftheexampleoftheprevioussection).Asymptotically,whenthenumberofobservationsntendstoinnity,thetrainingpoints 123

PAGE 124

aretakeneverywhereonD.Inversely,adiscretemeasureisdenedasthesumofDiracfunctions,withpositiveweightssuchthattheirsumequalsone: =p1x1+:::+pkxk(6{38) With:pi0,1ikandPpi=1. Eachobservationistakenrandomlyonthesetfx1;:::;xkgwithprobabilitypi.Whenntendstoinnity,thedesignofexperimentsisequivalenttoacontinuousnormalizeddesignwithtrainingpointsfx1;:::;xkgandnoisevariances2=pi. 6.5.3.1Generalresult Theorem .Theobservationsaretakenrandomlyaccordingtothemeasure.Whenthenumberofobservationsmtendstoinnity,thepredictionvarianceandIMSEtendto: s2K1(x)=2)]TJ /F11 7.97 Tf 16.36 14.94 Td[(1Xp0p2 2+p(p(x))2 (6{39) IMSE1=2)]TJ /F11 7.97 Tf 16.35 14.94 Td[(1Xp0p2 2+pZD(p(x))2d(x) (6{40) whereparetheeigenvaluesoftheHilbert-Schmidtintegraloperatorassociatedwiththecovariancefunctionk(x;y)(seeappendix B ),andparethenaturalrepresentativesoftheeigenfunctions~poftheoperator,denedby:p(x)=1 pZDk(x;y)~p(y)d(y);x2D Werecallthatthespectraldecompositionhasthefollowingproperties: k(x;y)=Pp0p~p(x)~p(y);almosteverywhere ~ppanorthonormalfamily:ZD~p(x)2d(x)=1;ZD~p(x)~q(x)d(x)=0p6=q p~p(x)=RDk(x;y)~p(y)d(y);x2supp() 124

PAGE 125

Ifandcoincide: .TheIMSE1expressionreducesto: IMSE1=2)]TJ /F11 7.97 Tf 16.35 14.95 Td[(1Xp0p2 2+p(6{41) Proof: .Twodierentproofsareproposeddependingonthenatureofthekernel.(1)Ifthekernelcanbedenedasanitesumofbasisfunctionsandcoecients:k(u;v)=21f1(u)f1(v)+:::+2lfl(u)fl(v) Then,thenumberofnon-nulleigenvaluesofthespectraldecompositionisniteandtheproofisalgebraic. (2)Thegeneralcase,towhichbelongtheusualKrigingkernels,requiresamorecomplexproof.TheproofsaregiveninAppendix B.1 andAppendix B.2 ,respectively. Remarks: .Findingthespectraldecompositionofthecovariancefunctionisaclassicalproblem.Dependingonthekernelconsidered,thedecompositioncanbewell-known,easytoderiveorinexistent(inparticular,ananalyticalformofthedecompositionoftheGaussiancovariancefunctionseemsnottoexist).Inmanycase,theseries(p)pdecreaserapidly,sotheseriesfors2K1andIMSE1arewellapproximatedbyitsrstqterms,typically:10q100. 6.5.3.2Adirectapplication:space-llingdesigns WeconsiderthecasewheretheuserwantstolearnthefunctionuniformlyonD;themeasureofintegrationfortheIMSEistheuniformmeasure.Typically,space-llingdesignsareusedforthatpurpose,sothemeasureofdistributionoftheobservationsisalsotheuniformmeasureonD.Inthatcase,theasymptoticMSEandIMSEhavesimpleform: s2K1(x)=Xp02p 2+p(p(x))2 (6{42) IMSE1=1Xp02p 2+p (6{43) 125

PAGE 126

Inthatcase,theycanbeeasilycomputednumerically.ThisvalueconstitutesthelimitoftheIMSEofaspace-llingDoE,whenthenumberofobservationstendstoinnity,andcanbeusedforastoppingcriterionwhenbuildingthedesign. Let(xn)n2Nbeaspace-llingseries,forinstanceaSoboloraNiederreitersequence,kaninitialnumberofpointsandapositivenumber.Then,wecanbuildaspace-llingdesignbyaddingpointsoftheseriestotheDoEuntilthecorrespondingIMSEiscloseenoughtotheasymptoticvalue,asweshowinthefollowingexamples. 6.5.4Examples 6.5.4.1Brownianmotion Weconsiderthettingofaone-dimensionalBrownianmotioninthespace.ThecovariancefunctionassociatedwiththeBrownianmotionis: k(x;y)=2min(x;y)(6{44) Notethatthiscovariancefunctionisnotstationary(sinceitdependsonxandyandnotonlyjx)]TJ /F4 11.955 Tf 11.95 0 Td[(yj).Howeverthekrigingmodelremainsthesame.Theeigenvaluesandeigenfunctionsassociatedwiththecovarianceare: p=2 (2p+1)p(x)=p 2sin(2p+1)x 2(6{45) Thus,theasymptoticalpredictionvarianceandIMSEare: s2K1(x)=2Xp02 1+)]TJ /F6 7.97 Tf 6.9 -4.98 Td[( 2h(2p+1) 2i2sin2(2p+1)x 2 (6{46) IMSE1=2Xp02 2+h(2p+1) 2i2 (6{47) Weillustratethisresultonanumericalexample.Wetake=0,2=1and2=210)]TJ /F5 7.97 Tf 6.59 0 Td[(3.Figure 6-6 showsonerealizationofaBrownianmotion,twoKrigingmodelsbasedonfourand19noisydata,respectively,andthepredictionvarianceforthetwomodels,andtheanalyticalfunction(usingthe50rsttermsoftheseries)foraninnite 126

PAGE 127

dataset.Thevariancesoftheobservationnoisesareequalton2,thatis,0.008and0.038,respectively. Figure6-6. IllustrationoftheMSEfunctionforaBrownianmotion(2=1,2=0:002).A)RealizationofaBrownianmotionandtheKrigingmodelbasedon4noisydata;B)Krigingbasedon19noisydata;C)predictionvariancefunctionforthetwomodels,andasymptoticvalue. Sincethenoisevarianceissmall,the4-observationbasedKriging(Figure 6-6 A))almostinterpolatesthetruefunction.However,theKrigingmeanisveryinaccuratewhenawayfromtheobservation.Asaconsequence,thepredictionvariance(Figure 6-6 C))isclosetozeroattheobservationpointsandismaximalatmid-distancesbetweentwoobservations.Sincetheresponseisknownatx=0(g(0)=0),thepredictionvarianceisequaltozero. The19-observationbasedKriging(Figure 6-6 B))isabetterapproximationtotherealfunction,eventhoughtheobservationnoiseismuchhighertosatisfytheconstraintofaconstantcomputationalbudget(seeEq. 6{31 )(indeed,someobservationpoints[black 127

PAGE 128

cross]arenotclosetothetruefunction).Whenincreasingthenumberofobservations,thepredictionvariancetendstoaconstant(exceptontheboundaries),whichislogicalsincetheinformationisuniformlyspreadintothedomain. Table 6-1 showstheIMSEvaluesforthethreemodels.Wecanseethatforaconstantcomputationalbudget,theIMSEcanbeconsiderablyreducedbyspreadingtheinformationuniformlyonthepredictiondomain.TheIMSEforthe4-pointmodelistwotimeslargerthantheasymptoticvalue.Ontheotherhand,thedierencebetweenthe19-pointmodelandtheasymptoticvalueisonly4%.Sincehavingmoreobservationsincreasesthecomputationalburden,onecandecidethataDoEwith19observationsissucientintermsofIMSE. Table6-1. IMSEvaluesforthetwoKrigingmodelsandtheasymptoticmodel. Krigingbasedon4observationsKrigingbasedon19observationsAsymptotic IMSE0.04610.02330.0224 6.5.4.2Orstein-Uhlenbeckprocess Now,weconsiderthettingofaone-dimensionalGaussianprocessinthespaceD=[0;1],withthefollowingcovariancefunction: k(x;y)=2exp)]TJ /F1 11.955 Tf 10.49 8.09 Td[(1 jx)]TJ /F4 11.955 Tf 11.96 0 Td[(yj(6{48) Theassociateddecompositionis(seeappendix C forcalculations): p(x)=Cp(pcos(px)+sin(px))(6{49) With:p=q 2)]TJ /F6 7.97 Tf 6.59 0 Td[(p 2p. Theparetherootsoftheequation: 2ucos(u)+)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2sin(u)=0(6{50) 128

PAGE 129

Suchequationdoesnothaveexplicitanalyticalsolutionsandmustbesolvednumerically.However,whenpislarge(>10),therootsarewellapproximatedby: p=(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1)(6{51) Theeigenvaluesareobtainedfromtherootsoftheequation: p=2 1+22p(6{52) TheconstantCpischosensothatpisnormalized: Cp=2p p)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1+2p2)]TJ /F9 11.955 Tf 11.95 9.68 Td[()]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1+2p2sin(p)cos(p)+2p(sin(p))2(6{53) Figure 6-7 illustratesthisresultfortheparametervalues2=1,=0:2and2=0:002.Forcomparison,amodelbasedon80dataisrepresented,whichisfoundtobeagoodapproximationtoaninnitedataset. 6.5.5ConcludingComments Inthissection,wefoundempiricallythatusingaverylargenumberofobservationswithverylowdelitywasanoptimalsolutiontominimizetheuncertaintyinthesimplekrigingmodel,whenspace-llingdesignsareused.Wefoundthatthiskindofdesignsasymptoticallyconvergetoabound.However,ithastobekeptinmindthatthissamplingstrategyisnotyetproventobeoptimal.Anywaywecanconcludethat,contrarilytoregressionwhereoptimaldesignshaveanitenumberofsupportpoints,itseemsbettertospreadasmuchaspossiblethecomputationalresourceratherthanconcentrateitonafewtrainingpoints. 129

PAGE 130

Figure6-7. IllustrationoftheMSEfunctionforaGaussianprocess(GP)withexponentialcovariancefunction(2=1,=0:2and2=0:002).A)RealizationofaGPandtheKrigingmodelbasedon80noisydata;B)MSEfunctionbasedonthe80data,andasymptoticvalue. 130

PAGE 131

CHAPTER7CONCLUSION 7.1SummaryAndLearnings Theuseofsurrogatemodelingasahelpingtoolforlearning,predictionanddesignofengineeringsystemsisanecientandcomputationallyattractivesolution.However,dealingwiththeuncertaintiesassociatedwithasurrogatemodelisachallengingbutcrucialissuetothesuccessofthesurrogate-basedstrategy. Thetwoprimaryobjectivesofthisworkwerethefollowing:(1)Proposeanddiscussalternativestocompensatefortheerroranduncertaintyinsurrogatemodelsinordertoprovidesafeestimatesthatlimittheriskofweakdesign;(2)Explorethesamplingstrategiesinordertominimizetheuncertaintyofsurrogatemodels. First,itwasshowedthatuncertaintycanbecompensatedbyaddingbiastothesurrogatemodelsinordertoincreasethechancethattheyaresafe,usingconstantorpointwisemargins,designedbytakingadvantageoferrordistributionmeasuresgivenbythemodel,orbymodel-independentmeasuresofaccuracyofthemodel.Sincetheconservativepredictionsarebiased,theproblemcanbeconsideredasamulti-objectiveoptimization,andresultsarepresentedintheformofParetofrontsbetweenaccuracyandconservativeness.Itwasfoundthattheeciencyofthedierentmethodsdependontheabilityofthesurrogatemodeltoreectthesystemresponse.Whenerrormeasuresgivenbythemodelarereasonablyaccurate,theiruseoutperformsmodel-independentmeasurestoreducelargeunsafeerrors.Ontheotherhand,model-independentmeasures,inparticularcross-validation,arefoundtobemorerobustandecientwhenthenumberofobservationsissucientlylarge(regardlessofsparsity). Then,asimplemethodologywasproposedtoestimatetheuncertaintyinareliabilitymeasurebasedonMonte-Carlosimulations.Usingthebootstrapmethod,wewereabletoquantifythelevelofbiasneededtoensureaprescribedprobabilitytobeonthesafesidefortheanalysis.Theprocedurewasappliedtothedesignofalaminatecompositeunder 131

PAGE 132

cryogenicandmechanicalstress.Itwasfoundthatttingasurrogatebasedondatasetsthatwerebiasedtocompensatefortheiruncertaintyallowedtoobtainasatisfyingdesign,evenwhentheavailableinformationwasverylimitedbycomputationalresources.Comparedtotraditionalmethodsofreliabilityestimates,equivalentlevelsofsafetywasobtainedwithareductionofcomputationalcostofseveralordersofmagnitude. Thechoiceoftheobservationsusedtotthemodel(designofexperiment)wasaddressedbytwodierentways.Inarsttime,itwasshownthatsubstantialgaincanbeobtainedbyconsideringthesurrogatemodelinanobjective-basedapproach.Inparticular,amethodologywasproposedtoconstructdesignsofexperimentsadaptedtotheframeworksofreliabilityassessmentandconstrainedoptimization,sothatthemodelaccuratelyapproximatesthevicinityofaboundaryindesignspacedenedbyatargetvalueofthefunctionapproximatedbythesurrogate.Themethodwasappliedtoareliabilityexample,anditwasfoundthatimprovinglocallytheaccuracyresultedingreatimprovementoftheaccuracyofthereliabilityestimation. Finally,aglobalapproachtouncertaintyreductioninsurrogatemodelingwasproposedtoaddresstheissueofsurrogatesbasedonsimulatorswithtunabledelity.Thissituationis,inparticular,encounteredintheframeworkofreliability-baseddesignoptimization.Solutionsdierdependingonthesurrogatemodelconsidered.Forregressionmodels,classicalresultsofdesignoptimalityshowthatsubstantialreductionofuncertaintycanbeachievedbyallocatingdierentcomputationalresources-hencedierentuncertaintylevelsintheresponses-toafewchosensimulations.Forkrigingmodels,itwasfoundthatthebestwaytoreduceuncertaintyinthemodelistospreadtheinformationasmuchaspossible,byusingverylargedatasetswithverylargelevelsofuncertainty.Thisresultwassupportedbytheoreticalasymptoticstudiesofthekrigingmodel. 132

PAGE 133

7.2Perspectives Ensuringhighlevelsofconservativenessis,aswehaveseen,atapriceofasubstantialincreaseoferror,whichleadstooverdesign.Inanengineeringcontext,suchapproachisrecommendedforpreliminaryanalysis,whichwouldbefollowedbymorerened(andexpensive)procedures.Moregenerally,theeciencyofconservativestrategiesareverydependentonthesamplingstrategiesusedtogeneratetheobservations.Possibleprolongationofthisworkmightincludeacombineduseofconservativepredictionsandsequentialstrategiesinordertolimittheriskofoverdesign. TheadapteddesignstrategyproposedinChapter5wastestedonacademicproblemsofreasonablecomplexity.Itsapplicationonreal-lifeproblemsmightleadtoadditionalproblemsanddiscussion,whichwereonlymentionedhere.Inparticular,onemightthinkofmeasuringtheinuenceofthequalityofthemetamodelontheeciencyofthemethod,thecombineduseofspace-llingandadaptedstrategiesforincreasedrobustness,ornumericalsolutionstoovercomecomputationalissues.Secondly,itseemsthattheideaofweightedIMSEandsequentialstrategiestoconstructDoEscanbeappliedtomanyotherpurposeswherethereisaneedfortrade-obetweenexplorationofregionsofinterestandreductionoftheglobaluncertaintyinthemetamodel.Here,wedenedaweightfunctiontoaccountfortheproximitytosometargetvalue.Potentialapplicationscouldbefoundinoptimization,ordetectionofregionsofhighvariations,etc.,thechallengebeinginanadapteddenitionoftheweightfunction. WediscussedinChapter6optimalstrategiesofallocationofresourceforsimulatorswithtunabledelity.Weshowedthat,inparticular,reliabilityanalysisbelongstothisframework.Itisalsopotentiallyapplicabletoawiderangeofproblems,inparticularFiniteElementAnalysisorComputationalFluidDynamicscodes.Futureresearchmayaimatrelatingthisworktostudiesofconvergenceofnumericalmodels(dependingonmeshing,complexityofelements,etc.),inordertoestimatetherelationbetweencomputationaltimeanderrorandproposeoptimalstrategiesfortheseproblems.In 133

PAGE 134

addition,weproposedinthisworksomelinkstofunctionalanalysisandspectraltheorywhenconsideringoptimalallocationfortheKrigingmodel.Itseemsthatthiswayoersmanypotentialprolongationsforthetheoryofdesignofexperiments,whichmightleadtoexactlyoptimalstrategies,ornumericalmethodstogeneratedesigns. 134

PAGE 135

APPENDIXAALTERNATIVESFORCONSERVATIVEPREDICTIONS Inthisappendix,wedetailtwoalternativestoproduceconservativepredictionsusingsurrogatemodeling. A.1BiasedFittingEstimators A.1.1BiasedFittingModels Weproposeheretoincludebiasduringthettingprocessinordertoproduceconservativepredictions.Todoso,weconstrainthepredictedresponsetobeononesideoftheDOEresponses;thatis,theerrorsbetweenpredictionandactualresponsearepositiveatDOEpoints.Suchmethoddonotapplytointerpolationtechniquessincetheerrorisbydenitionnullatdatapoints.Forpolynomialregression,theconservativecoecients^consarethesolutionofthefollowingconstrainedoptimizationmodel: Minimize^consMSE=1 nnXi=1[^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)]2s.t.^y(xi))]TJ /F4 11.955 Tf 11.96 0 Td[(y(xi)>0;i=1;:::;n(A{1) Then,thebiasedttingconservativeestimateisgivenby: ^yBF(x)=fT(x)^cons(A{2) Inordertocontrolthelevelofbias,weproposetwoalternatives.Therstisconstraintrelaxation,thatallowsagivenamountofconstraintviolationdenotedas: Minimize^consMSEs.t.^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)+>0;i=1;:::;n(A{3) Apositivewillreducethebiasinthetting.canalsobechosennegativeinordertobemoreconservativethanwithnoconstraintrelaxation. Thesecondalternativeistoreducethenumberofconstraints,thatis,toconstraintheerrorstobepositiveonlyataselectednumberofpoints(constraintselection).The 135

PAGE 136

constraintsselectedarethoseapriorieasiertobesatised,thatis,wheretheerrorfromtheunbiasedtisminimal. Theproceduretoselectthesepointsisasfollow: Computetheunbiasedestimatesusingclassicalregression Computetheerrorsandsortthembyascendingorder Selectthepointscorrespondingtotheksmallesterrors Solvethefollowingoptimizationproblem: Minimize^consMSEs.t.^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)+>0;i=1;:::;k(A{4) wherethexiaresortedinascendingorderaccordingtoerrorsand1kn. Inthefollowing,thetwoabove-referencedbiased-ttingalternativesareentitledconstraintrelaxationandconstraintselection,respectively. Remark .Thebiasedttingstrategyrequiresthesolvingofaconstrainedoptimizationproblemtoobtainthecoecients^insteadofusingananalyticalfunction.However,thisoptimizationproblemisquitesimpletosolve,sincethecriterionisconvex(leastsquares),andtheconstraintsarelinear. A.1.2ResultsAndComparisonToOtherMethods Here,wecomparethetwobiasedttingstrategiestotheconstantsafetymargin(CSM)strategyontheBranin-Hoofunction,intheexperimentalsetupdescribedinSection 3.3.2.2 .First,wecomparethetwostrategiesforbiasedttingestimates.Therangeoftheconstraintrelaxationischosenas[-3;15];theproportionofselectedconstraintsischosenbetween0and1.ThegraphsofperformanceareshowninFigure A-1 Bothmethodsseemtoprovidesimilarresults;thedierenceforhighpercentilesontheleftgureishereduetonumericalnoise.Usingconstraintselectiondoesnotallow 136

PAGE 137

FigureA-1. AverageresultsforBiasedttingwithconstraintrelaxation(plainblack)andconstraintselection(mixedgrey)ontheBranin-Hoofunction.A)%cvs.lalpha;B)%cvs.MaxUE%. obtainingveryconservativeestimates:indeed,usingallthe17constraintsleadstoa85%conservativeness.Ontheotherhand,withconstraintrelaxation,usinganegativeshiftallowstobemoreconservative.Ontherightgraph,weseethatahighproportionofconservativeestimatesdoesnotpreventforhavinglargeunconservativeerrors:forinstance,fora90%conservativeness,themaximumunconservativeerrorisreducedby40%only. Now,wecomparebiasedttingandCSMstrategies.Figure A-2 showsthegraphsofperformancefortheCSMestimatorandthebiasedttingestimatorwithconstraintrelaxation. Thetwoestimatorshavesimilartrends,butbiasedttingresultswithhighervariability.Indeed,thismethodisalotmoresensitivetotheDOE,sinceasingleconstraintcanhavealargeinuenceontheshapeoftheresponse. A.2IndicatorKriging A.2.1DescriptionOfTheModel IndicatorKriging(IK)isasurrogate-basedalternativetoprovideconservativeestimates.Insteadofestimatingthevalueofresponse,IKestimatestheprobabilitythattheresponseexceedsagivenvalue(cut-o).Inotherwords,IKprovidesanestimateof 137

PAGE 138

FigureA-2. AverageresultsforCSM(plainblack)andBiasedttingwithconstraintselection(mixedgrey).A)%cvs.lalpha;B)%cvs.MaxUE%. theconditionalcumulativedistributionfunction(CCDF)ataparticularcut-oc.ThekeyideaofIKistocodetheobservedresponsesintoprobabilitiesofexceedingthecut-o.Whentheresponseisdeterministic(withoutobservationnoise),theprobabilityiseither0or1.Theindicatorcodingatasampledlocationxiiswritten: Ii=I(c;xijfy1;y2;:::;yng)=P(y(xi)>c)=8><>:1ify(xi)>c0otherwise(A{5) Atanunsampledlocationx,theprobabilityisestimatedbytheKrigingpredictionbasedontheindicatordata: ^P(y(x)>c)=^yIK(x)(A{6) where^yIKistheKrigingestimatebasedonfI1;I2;:::;Inginsteadoffy1;y2;:::;yng. Foragivensetofcut-osfc1;c2;:::;cqgandpredictionlocationx,weobtainacorrespondingsetofprobabilitiesfP1;P2;:::;Pqg.WeusethesediscreteprobabilitiestotacontinuousapproximationoftheCCDFoftheresponseatandbuildcondenceintervals.IKisoftenqualiedasa'non-parametric'approachsinceitdoesnotrelyonapre-specieddistributionmodel.Notethatitisanexpensiveprocedurecomparedtoaclassicalkrigingmodelsinceitmayrequirealargenumberofkrigingmodels. 138

PAGE 139

Post-processingisnecessarytotransformtheIKsetofvaluesintoausablediscreteCDF.Indeed,thereisnoconstraintduringtheproceduretohavevaluesonlyinside[0;1]orthatCDFestimatesvarymonotonicallywithcut-os.WeusehereoneofthemethodsproposedintheGSLIBuser'sguide[ Deutsch&Journel ( 1997 )].First,valuesoutoftheinterval[0,1]arereplacedby0or1.Then,theoriginalIK-derivedpercentilesareperturbedbyrunninganoptimizationthatminimizestheperturbationwhileensuringallorderrelations. The(1)]TJ /F4 11.955 Tf 12.4 0 Td[()%conservativeestimatoristhe(1)]TJ /F4 11.955 Tf 12.4 0 Td[()thpercentileofthisdistribution,givenbytheinverseoftheCDF.Thispercentileisinterpolatedfromthesetofprobability(likeforthebootstrapmethod).Here,weuselinearinterpolation. A.2.2ApplicationToTheTorqueArmAnalysis Sinceitrequiresalargenumberofpoints,IKisappliedtotheanalysisofthetorquearmonly.Arationalquadraticfunctionisusedforcovariance;theparametersofthefunctionarere-estimatedforeachcut-ousingthetoolboxGPML.Atotalof100cutosareusedforthedistributionestimation. Inthissection,wecompareIKtotheregularKrigingwitherrordistribution(ED).First,wecomparetheperformancesoftheunbiasedmetamodels:withoutmarginforKrigingandwithatargetof50%conservativenessforIK.Table A-1 reportstheireRMS,%candMaxUEbasedonthe1,000testpoints. TableA-1. Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. SurrogateeRMS%cMaxUE IK22.945.6%228.9Kriging14.551.6%60.7 Forunbiasedpredictions,IKhasanerroralotlargerthanKriging.TheeRMSistwotimeslargerandtheMaxUEalmostthreetimes.Here,itappearsthattheIKprocedure,withconversionintobooleannumber,CDFnormalizationandinterpolation,resultsinsubstantiallossinaccuracy. 139

PAGE 140

InordertohaveafaircomparisonbetweenIKandKriging,wecompareunscaledquantities,eRMSandMaxUE,insteadoflandMaxUE%.Figure A-3 showsthegraphsofperformanceforthetwoconservativeestimators. FigureA-3. AverageresultsforIndicatorKriging(IK,plainblack)andEDwithKriging(UK,mixedgrey).A)%cvs.l;B)%cvs.MaxUE%. Ontheleftgraph(Figure A-3 A)),weseethattherateofincreaseofeRMSoftheIKisverysmallupto%c=70%.However,itisnotenoughtocompensatefortheinitialvalueofeRMSandbecompetitivewiththeUKestimator.Figure A-4 showsthedelitytotargetconservativenessforbothestimators.DespiteverylargeerrorscomparedtoclassicalKriging,IndicatorKrigingisfoundtobeveryaccurateregardingdelity. FigureA-4. Targetvs.actual%cforIK(plainblack)andKrigingwithED(mixedgray). 140

PAGE 141

APPENDIXBDERIVATIONOFTHEASYMPTOTICKRIGINGVARIANCE ThisappendixshowsthederivationoftheasymptoticformulaofthekrigingvariancedescribedinChapter6. Onallthesecalculations,weassumethattheobservationsaredistributedaccordingtotheprobability.IfistheuniformmeasureonthedesignregionD,itmeansthattheobservationsareuniformlyspreadonD.IfisaDiracfunction,alltheobservationsaretakenonasinglelocation. Weproposetwodierentproofs,dependingonthenatureofthekernel.Forkernelsofnitedimension,theproofisalgebraic.Thegeneralcaseismorecomplex,andtheproofisformal. B.1KernelsOfFiniteDimension Thecovariancekernelisassumedtobeoftheformofanitesumofbasisfunctionsandcoecients: k(u;v)=f(u))]TJ /F4 11.955 Tf 8.08 0 Td[(f(v)T(B{1) With:f(u)=[f1(u);:::;fp(u)]. )]TJ /F1 11.955 Tf 11.98 0 Td[(canbediagonalforinstance,hencethekernelisoftheform: k(u;v)=21f1(u)f1(v)+:::+2pfp(u)fp(v)(B{2) Forsimplicationpurpose,wewillassumeinthefollowingthat)]TJ /F1 11.955 Tf 11.98 0 Td[(isalwaysdiagonal. Werecalltheequationofthekrigingpredictionvariance,basedonmobservationswithhomogeneousnuggeteectequaltom2: sK2(x)=k(x;x))]TJ /F7 11.955 Tf 11.96 0 Td[(kT(x)(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1k(x)(B{3) 141

PAGE 142

With:K+=(k(xi;xj))1i;jm+0BBBB@m20...0m21CCCCA Then,withcovariancefunctionasdenedinEq. B{1 ,thekrigingvariancecanbeexpressedas: s2K(x)=f(u))]TJ /F4 11.955 Tf 8.08 0 Td[(f(u)T)]TJ /F4 11.955 Tf 11.96 0 Td[(f(x))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Im+f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T(B{4) withtheconvention:f(X)=266664f1(x1):::fp(x1)...f1(xm):::fp(xm)377775: Then,byfactorizationweobtain: s2K(x)=f(x))]TJ /F2 11.955 Tf 10.73 0 Td[()]TJ /F7 11.955 Tf 11.96 0 Td[()]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Im+f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(X))]TJ /F9 11.955 Tf 8.09 16.86 Td[(f(x)T(B{5) Sothepredictionvarianceisoftheform: s2K(x)=f(x)M)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(x)T(B{6) with:M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[()]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Imf(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1isofdimensionpp. Usingtheequality[ Fedorov&Hackl ( 1997 ),p.107] )]TJ /F7 11.955 Tf 5.48 -9.69 Td[(A+BBT)]TJ /F5 7.97 Tf 6.59 0 Td[(1=A)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F7 11.955 Tf 11.95 0 Td[(A)]TJ /F5 7.97 Tf 6.58 0 Td[(1B)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(I+BTA)]TJ /F5 7.97 Tf 6.59 0 Td[(1B)]TJ /F5 7.97 Tf 6.59 0 Td[(1BTA)]TJ /F5 7.97 Tf 6.59 0 Td[(1;(B{7) withA=1 m2)]TJ /F1 11.955 Tf 11.98 0 Td[(andB=f(X)T,M)]TJ /F5 7.97 Tf 6.58 0 Td[(1canbesimpliedto: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=1 m2f(X)Tf(X)+)]TJ /F11 7.97 Tf 8.08 4.93 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1(B{8) 142

PAGE 143

Thepredictionvarianceisthen: s2K(x)=f(x)1 m2f(X)Tf(X)+)]TJ /F11 7.97 Tf 8.08 4.94 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)T(B{9) Now,wewanttoconsiderthecasewhenm!+1.Wehavebydenition: 1 mf(X)Tf(X)= 1 mmXi=1fk(xi)fl(xi)!1k;lp(B{10) Whenm!+1,wegetbythestronglawoflargenumbers: 1 mf(X)Tf(X)!hfk;fli1k;lp(B{11) where:hfk;fli=ZDfk(x)fl(x)d(x) WedenetheGrammatrixGsuchthat: G=hfk;fli1k;lp(B{12) Bycontinuity,weobtain: s2K(x)!s2K1(x)=f(x)G 2+)]TJ /F11 7.97 Tf 8.08 4.94 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)T(B{13) Equation B{13 showsthat,evenifthenumberofobservationstendstoinnity,thepredictionvariancecanbeexpressedasafunctionoftheinverseofappmatrix.Hence,thecomplexityofthecalculationsisdrivenbythedimensionofthekernelinsteadofthenumberofobservations,whichiscounter-intuitiveregardingtheoriginalformofthepredictionvariance(Eq. 2{23 ). Now,weintroducetheHilbert-SchmidtintegraloperatorTk;associatedwiththekernelkandmeasure: Tk;:L2()!L2()g7!Tk;g:x7!RDk(x;y)g(y)d(y)(B{14) 143

PAGE 144

ByusingEq. B{2 : Tk;g(x)=ZDk(x;y)g(y)d(y)=pXk=12k24ZDfk(y)g(y)35fk(x)=pXk=12khfk;gifk(x) (B{15) Equation B{15 showsthattheimageofTk;isgeneratedbypfunctionsfk(x).Hence,Tk;isanoperatorofniterankr,rpanditadmitsaspectraldecomposition1;2;:::rand12:::rsuchthat: (j)janorthonormalfamily:ZD(j(x))2d(x)=1andZDi(x)j(x)d(x)=0;i6=j Tk;j=jj Remark: .Therankrdependsonboth(fi)iand.Inparticular,whenisaweightedsumofDiracfunctions,L2()isofdimensionq=Card(supp()).TheimageofTk;beingavectorsubspaceofL2(),itsdimensionrisboundedbyq.Fortheextremecasewhere=x(singleDiracfunction),wehaver=1,evenifthecovarianceisdenedbyalargenumberpofbasisfunctions.Inpractice,mostofthetimer=min(p;q). Now: jj(x)=Zk(x;y)j(y)d(y)=pXk=12khfk;jifk(x) (B{16) WedenetheprmatrixA=(ajk)1jp;1krsuchthat:ajk=hfk;ji 144

PAGE 145

Then: jj(x)=pXk=12kakjfk(x) (B{17) (x)T=ATf(x)T (B{18) With:=diag([1;:::;r]). WemultiplybothsidesofEq. B{18 by(x),andintegratewithrespecttothemeasure: Z(x)T(x)d(x)=ZATf(x)T(x)d(x) Since(j)jisanorthonormalbasis,theleft-hand-sidereducesto.Then,weobtaintheequality: =ATA(B{19) Similarly,bymultiplyingbothsidesofEq. B{18 byf(x)andintegrating,weobtain: AT=ATGG=AAT (B{20) Now,usingtheequalityofEq. B{7 weobtain: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=1 n2ATA+)]TJ /F11 7.97 Tf 8.09 4.94 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[(A2Ir+ATA)]TJ /F5 7.97 Tf 6.59 0 Td[(1AT)]TJ /F1 11.955 Tf -172.16 -26.89 Td[(=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[(A2Ir+)]TJ /F5 7.97 Tf 6.58 0 Td[(1AT)]TJ ET BT /F1 11.955 Tf 433.05 -481.93 Td[((B{21) Then: s2K1(x)=f(x))]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T)]TJ /F4 11.955 Tf 11.96 0 Td[(f(x)A2Ir+)]TJ /F5 7.97 Tf 6.59 0 Td[(1AT)]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T=k(x;x))]TJ /F10 11.955 Tf 11.95 0 Td[((x)2Ir+)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)T (B{22) 145

PAGE 146

Now: 2Ir+)]TJ /F5 7.97 Tf 6.58 0 Td[(1=diag1 2+k=diagk2 2+k (B{23) Finally: s2K1(x)=k(x;x))]TJ /F6 7.97 Tf 18.56 14.94 Td[(rXk=12k 2k+2k(x)2(B{24) B.2GeneralCase Thedemonstrationisorganizedasfollow:rst,theclassicalkrigingequationsaregivenforanitenumberofobservationsm.Aftertransformations,thekrigingvarianceisexpressedwhenmtendstoinnityusingthelawofgreatnumbers.Then,thetheoryofHilbert-Schmidtoperatorsandtheirspectraldecompositionareintroduced.Finally,thekrigingvarianceexpressionissimpliedusingthespectraldecompositiontoobtaintheformula. First,werecalltheequationofthekrigingpredictionvariance,basedonmobservationswithhomogeneousnuggeteectequaltom2: sK2(x)=2)]TJ /F7 11.955 Tf 11.96 0 Td[(kT(x)(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1k(x)(B{25) With:K+=(k(xi;xj))1i;jm+0BBBB@m20...0m21CCCCA Wedecompose(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1inTaylorseries,using:(I+H))]TJ /F5 7.97 Tf 6.59 0 Td[(1=I)]TJ /F7 11.955 Tf 11.96 0 Td[(H+H2)]TJ /F7 11.955 Tf 11.95 0 Td[(H3+:::=1Xp=0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pHp Herewehave:K+=m2I+K m2 146

PAGE 147

Hence: (K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1=1 m21Xp=0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pKp (m2)p(B{26) NotethatthisdecompositioniscorrectwhenthespectralradiusofKissmallerthanm2.However,ithasbeenfoundempiricallythattheresultsofthisproofstandwhenthisasumptionisnotmet.Yet,wehavenotfoundhowtorelaxthishypothesis. UsingthedecompositionofEq. B{26 ,thepredictionvarianceinEq. B{25 becomes:sK2(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)p1 (m2)p+1kT(x)Kpk(x) Weexpandthequadraticformontheright-handside: kT(x)Kpk(x)=Xi;jk(x;xi)(Kp)ijk(xj;x)=Xi1;:::;ip+1k(xi;xi1)k(xi1;xi2):::k)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(xip;xip+1k)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(xip+1;x Whenm!+1,weobtain,bythelawoflargenumbers:1 mp+1Xi1;:::;ip+1k(x;xi1)k(xi1;xi2):::k)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(xip;xip+1k)]TJ /F7 11.955 Tf 5.47 -9.68 Td[(xip+1;x!ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1) Thus:sK21(x)=2)]TJ /F9 11.955 Tf 9.29 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)p1 (2)p+1ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1) Letusintroducethenotation: p+2(x;x)=ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1)(B{27) Then: sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pp+2(x;x) (2)p+1(B{28) 147

PAGE 148

Atthispoint,weintroducetheHilbert-SchmidtintegraloperatorTk;associatedwiththekernelk: Tk;:L2(D)!L2(D)f7!Tk;f:x7!RDk(x;y)f(y)d(y)(B{29) Mercer'stheoremstatesthatthekernelfunctionkcanberepresentedonL2()inanorthogonalbasis(n)nofL2()consistingoftheeigenfunctionsofTk;suchthatthecorrespondingsequenceofeigenvaluesfngnisnonnegative[ Riez&Nagy ( 1952 ),p.242]: k(u;v)=Xn0nn(u)n(v);u;v2supp()(B{30) With: (n)nanorthonormalbasis:ZD(n(x))2d(x)=1andZDfi(x)fj(x)d(x)=0;i6=j n0;123::: Tk;n=nn ThepthpowerofTk;(Tk;appliedptimes)hasthesameeigenfunctionsanditseigenvaluesarenp.Itsassociatedkernelis(forp>1):p(u;v)=ZDp)]TJ /F13 5.978 Tf 5.76 0 Td[(1k(u;u1)k(u1;u2):::k(up)]TJ /F5 7.97 Tf 6.59 0 Td[(2;up)]TJ /F5 7.97 Tf 6.59 0 Td[(1)k(up)]TJ /F5 7.97 Tf 6.59 0 Td[(1;v)d(u1):::d(up)]TJ /F5 7.97 Tf 6.58 0 Td[(1) Forinstance,forp=2: T2k;=Tk;(Tk;f)(x)=ZDk(x;u1)Tk;f(u1)d(u1)=ZDk(x;u1)24ZDk(y;u2)f(u2)d(u2)35d(u1)=ZZD2k(x;u1)k(u1;u2)f(u2)d(u2)d(u1) 148

PAGE 149

=ZD24ZDk(x;u1)k(u1;u2)d(u1)35f(u2)d(u2)=ZD2(x;u2)f(u2)d(u2) Foru;v2supp(),wehavethespectraldecompositionofthekernelp: p(u;v)=Xn0npn(u)n(v)(B{31) Now,fromEq. B{27 : p+2(x;x)=ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1)=ZD2k(x;u1)24ZDp)]TJ /F13 5.978 Tf 5.75 0 Td[(1k(u1;u2):::k(up;up+1)d(u2):::d(up)35k(up+1;x)d(u1)d(up+1)=ZD2k(x;u1)[p(u1;up+1)]k(up+1;x)d(u1)d(up+1) Thekernelpisevaluatedonthesupportof,wecanreplaceitbyitsspectralrepresentation(Eq. B{31 ): p+2(x;x)=ZD2k(x;u1)"Xn0npn(u1)n(up+1)#k(up+1;x)d(u1)d(up+1)=Xn0npZD2k(x;u1)n(u1)n(up+1)k(up+1;x)d(u1)d(up+1)=Xn024np0@ZDk(x;u1)n(u1)d(u1)1A0@ZDn(up+1)k(up+1;x)d(up+1)1A35 Bydenition,wehave:Tk;n(x)=nn(x)andZDk(x;y)n(y)d(y)=nn(x) Finally:p+2(x;x)=Xn0[np(nn(x))(nn(x))] 149

PAGE 150

p+2(x;x)=Xn0np+2(n(x))2(B{32) Then,weputEq. B{32 intoEq. B{28 :sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.29 0 Td[(1)p1 (2)p+1Xn0np+2(n(x))2sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xn0n(n(x))2Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pnp+1 (2)p+1 ThesecondsumcorrespondstoaTaylorextension:Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pnp+1 (2)p+1=Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pn 2p+1=1)]TJ /F1 11.955 Tf 26.57 8.09 Td[(1 1+n 2 Hence:sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n 1)]TJ /F1 11.955 Tf 26.57 8.09 Td[(1 1+n 2!(n(x))2 sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xn0n2 2+n(n(x))2(B{33) B.3TheSpace-FillingCase Inthespace-llingcase,istheuniformmeasureonD.Whenuniformaccuracyoverthedesigndomainiswanted,theintegrationmeasureusedfortheIMSEisalsouniform:IMSE=ZDsK2(x)d(x) Inthecasewhenthetwomeasurescoincide,theIMSEcanbeexpressedinsimpleform.Foranyxinsupp(),wehave:Xn0n(n(x))2=2 Here,supp()=D,sothisequalityiscorrectforanyx2D.Then: sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n 1)]TJ /F1 11.955 Tf 26.58 8.09 Td[(1 1+n 2!(n(x))2 150

PAGE 151

=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n(n(x))2+Xn0 n 1+n 2!(n(x))2==Xn02n 2+n(n(x))2 (B{34) Besides:ZD(n(x))2d(x)=1 WhenintegratingthepredictionvariancetocomputetheIMSE,weobtain: IMSE1=ZDsK21(x)d(x)=Xn02n 2+nZD(n(x))2d(x)=1Xn02n 2+n (B{35) 151

PAGE 152

APPENDIXCSPECTRALDECOMPOSITIONOFTHEORSTEIN-UHLENBECKCOVARIANCEFUNCTION Weconsiderthefollowingkernel:k(x;y)=2exp)]TJ /F1 11.955 Tf 10.49 8.09 Td[(1 jx)]TJ /F4 11.955 Tf 11.96 0 Td[(yj Aneigenfunctionfwithcorrespondingeigenvalueforthatkernelveries:Kf(y)=2yZ0exp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(y)]TJ /F4 11.955 Tf 11.95 0 Td[(x f(x)dx+21Zyexp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(x)]TJ /F4 11.955 Tf 11.95 0 Td[(y f(x)dx=f(y) Bydierentiatingthisequationtwotimeswithrespecttoywend:Kf"(y)=1 2Kf(y))]TJ /F1 11.955 Tf 13.15 8.09 Td[(22 f(y) However:Kf(x)=f(x)Kf"(x)=f"(x) Thus:f"(x)= 2f(x))]TJ /F1 11.955 Tf 13.16 8.09 Td[(22 f(x)f"(x)=)]TJ /F9 11.955 Tf 11.29 16.85 Td[(22)]TJ /F4 11.955 Tf 11.95 0 Td[( 2f(x) Wededucethatfisoftheform: f(x)=Acos(x)+Bsin(x)(C{1) And: =r 22)]TJ /F4 11.955 Tf 11.96 0 Td[( 2(C{2) Now,wereplacefbyitstrigonometricformofEq. C{1 intheintegralequation,andby:=22 1+22 152

PAGE 153

Wehave:Kf(x))]TJ /F4 11.955 Tf 9.3 0 Td[(f(x)=21Z0exp)]TJ 10.5 8.09 Td[(jx)]TJ /F4 11.955 Tf 11.95 0 Td[(yj [Acos(x)+Bsin(x)]dx)]TJ /F1 11.955 Tf 19.55 8.09 Td[(22 1+22[Acos(x)+Bsin(x)] Aftersimplications,weobtain:Kf(x))]TJ /F4 11.955 Tf 11.96 0 Td[(f(x)=2 1+2e)]TJ /F14 5.978 Tf 7.78 3.26 Td[(x +ex With:8><>:=)]TJ /F4 11.955 Tf 9.29 0 Td[(A+B=e)]TJ /F13 5.978 Tf 7.82 3.26 Td[(1 [A(sin())]TJ /F1 11.955 Tf 11.95 0 Td[(cos()))]TJ /F4 11.955 Tf 11.95 0 Td[(B(sin()+cos())] Thisexpressionisnullforanyxonlyifbothandareequaltozero: 1+2e)]TJ /F14 5.978 Tf 7.78 3.26 Td[(x +ex =0,8><>:=0=0 Wededucefrom=0that:A=B So,theeigenfunctionisoftheform:f(x)=C(cos(x)+sin(x)) Cisanormalizationconstantsothat1R0f2(x)=1. Besides,from=0weobtain:(sin())]TJ /F1 11.955 Tf 11.96 0 Td[(cos()))]TJ /F1 11.955 Tf 11.96 0 Td[((sin()+cos())=0 2cos()+)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(22sin()=0(C{3) Therootsofthisequationdenealltheeigenvaluesandeigenfunctionsoftheproblem.However,thisequationdoesnothaveanalyticalsolutionandmustbefoundnumerically. 153

PAGE 154

Figure C-1 representsthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2u2)sin(u).Weseethatg(u)ispseudo-periodic.Whenuisrelativelylarge(2u2)]TJ /F1 11.955 Tf 12.46 0 Td[(12u),therootsarewellapproximatedby: u=n;n2N(C{4) Finally,thespectraldecompositionassociatedwiththeexponentialcovarianceis: fp(x)=Cp(pcos(px)+sin(px))p=22 1+22p(C{5) With:p=s 22)]TJ /F4 11.955 Tf 11.96 0 Td[(p 2pCp=2p p)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1+2p2)]TJ /F9 11.955 Tf 11.95 9.69 Td[()]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1+2p2sin(p)cos(p)+2p(sin(p))2 Theparetherootsoftheequation:2ucos(u)+)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2sin(u)=0 FigureC-1. Representationofthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2)sin(u).Therootsofthisequationaretheeigenvaluesoftheexponentialcovariancekernel. 154

PAGE 155

REFERENCES Acar,E.,&Haftka,R.(2005).TheEectsofuncertaintycontrolmeasuresontheweightsavingfromcompositelaminatesatcryogenictemperatures.InProceedingsofASME-IDECT-2005,September,LongBeach,California. Acar,E.,Kale,A.,&Haftka,R.(2007).ComparingEectivenessofMeasuresThatImproveAircraftStructuralSafety.JournalofAerospaceEngineering,20,186. Ben-Haim,Y.,&Elishako,I.(1990).Convexmodelsofuncertaintyinappliedmechanics.North-Holland. Bennett,J.,&Botkin,M.(1986).Theoptimumshape:Automatedstructuraldesign.PlenumPubCorp. Boukouvalas,A.,&Cornford,D.(2009).LearningheteroscedasticGaussianprocessesforcomplexdatasets.Tech.rep.,Technicalreport,NeuralComputingResearchGroup,AstonUniversity. Box,G.,&Draper,N.(1986).Empiricalmodel-buildingandresponsesurface.WileySeriesinProbabilityandMathematicalStatistics. Chernick,M.(1999).Bootstrapmethods:Apractitioner'sguide.WileyNewYork. Chippereld,A.,Fleming,P.,Pohlheim,H.,&Fonseca,C.(1994).Geneticalgorithmtoolboxusersguide.ACSEResearchReport,512. Choi,K.,Du,L.,Youn,B.,&Gorsich,D.(2005).Possibility-baseddesignoptimizationmethodfordesignproblemswithbothstatisticalandfuzzyinputdata.6thWorldCongressesofStructuralandMultidisciplinaryOptimization,RiodeJaneiro,Brazil. Cook,R.,&Nachtsheim,C.(1980).AcomparisonofalgorithmsforconstructingexactD-optimaldesigns.Technometrics,(pp.315{324). Cressie,N.(1993).Statisticsforspatialdata:WileySeriesinProbabilityandMathematicalStatistics.Findthisarticleonline. Currin,C.,Mitchell,T.,Morris,M.,&Ylvisaker,D.(1991).Bayesianpredictionofdeterministicfunctions,withapplicationstothedesignandanalysisofcomputerexperiments.JournaloftheAmericanStatisticalAssociation,(pp.953{963). Deutsch,C.,&Journel,A.(1997).GSLIB:Geostatisticalsoftwarelibraryanduser'sguide.Appliedgeostatistics,2. Dixon,L.,&Szego,G.(1978).TowardsGlobalOptimisation2.North-Holland. Du,L.,Choi,K.,Youn,B.,&Gorsich,D.(2006).Possibility-baseddesignoptimizationmethodfordesignproblemswithbothstatisticalandfuzzyinputdata.JournalofMechanicalDesign,128,928. 155

PAGE 156

Du,X.,Sudjianto,A.,&Huang,B.(2005).Reliability-baseddesignwiththemixtureofrandomandintervalvariables.Journalofmechanicaldesign,127(6),1068{1076. Efron,B.(1982).Thejackknife,thebootstrapandotherresamplingplans.InCBMS-NSFregionalconferenceseriesinappliedmathematics,vol.38,(p.92).Siam. Elfving,G.(1952).Optimumallocationinlinearregressiontheory.TheAnnalsofMathematicalStatistics,(pp.255{262). Elishako,I.(1991).Anideaoftheuncertaintytriangle.TheShockandVibrationDigest,(p.1). Elishako,I.(2004).Safetyfactorsandreliability:friendsorfoes?.KluwerAcademicPub. Enevoldsen,I.,&Srensen,J.(1994).Reliability-basedoptimizationinstructuralengineering.StructuralSafety,15(3),169{196. Engelund,S.,&Rackwitz,R.(1993).Abenchmarkstudyonimportancesamplingtechniquesinstructuralreliability.StructuralSafety,12(4),255{276. Fedorov,V.(1972).Theoryofoptimalexperiments.NewYork:Academicpress. Fedorov,V.,&Hackl,P.(1997).Model-orienteddesignofexperiments.Springer. Ginsbourger,D.(2009).MetamodelesMultiplespourl'Approximationetl'OptimisationdeFonctionsNumeriquesMultivariables.Ph.D.thesis,EcoleNationaleSuperieuredesMinesdeSaintEtienne. Goel,T.,Haftka,R.,Papila,M.,&Shyy,W.(2006).Generalizedpointwisebiaserrorboundsforresponsesurfaceapproximations.Internationaljournalfornumericalmethodsinengineering,65(12). Haldar,A.,&Mahadevan,S.(2000).Probability,reliabilityandstatisticalmethodsinengineeringdesign.WileyNewYork. Hansen,N.,&Kern,S.(2004).EvaluatingtheCMAevolutionstrategyonmultimodaltestfunctions.LectureNotesinComputerScience,(pp.282{291). Holton,G.(2003).Value-at-risk:theoryandpractice.AcademicPress. Iooss,B.,Ribatet,M.,&Marrel,A.(2008).GlobalSensitivityAnalysisofStochasticComputerModelswithjointmetamodels.ArxivpreprintarXiv:0802.0443. Isaaks,E.,&Srivastava,R.(1989).Anintroductiontoappliedgeostatistics. Jin,R.,Chen,W.,&Sudjianto,A.(2002).Onsequentialsamplingforglobalmetamodelinginengineeringdesign.InProceedingsofDETC,vol.2.Citeseer. 156

PAGE 157

Jones,D.,Schonlau,M.,&Welch,W.(1998).Ecientglobaloptimizationofexpensiveblack-boxfunctions.JournalofGlobalOptimization,13(4),455{492. Kenney,J.,&Keeping,E.(1954).Mathematicsofstatistics,partone.D.VanNostrandCompany,Inc.,Princeton,NJ. Khuri,A.,&Cornell,J.(1996).Responsesurfaces:designsandanalyses.CRC. Kiefer,J.(1961).Optimumdesignsinregressionproblems,II.TheAnnalsofMathematicalStatistics,(pp.298{325). Kiefer,J.,&Wolfowitz,J.(1960).Theequivalenceoftwoextremumproblems.Canad.J.Math,12,363{366. Kim,N.,Ramu,P.,&N.V.,Q.(2006).TailModelinginReliability-BasedDesignOptimizationforHighlySafeStructuralSystems.InProceedingsof47thAIAA/ASME/ASCE/AHS/ASCStructures,StructuralDynamicsandMaterialsConference. Kleijnen,J.,&VanBeers,W.(2004).Application-drivensequentialdesignsforsimulationexperiments:Krigingmetamodelling.JournaloftheOperationalResearchSociety,55(8),876{883. Kleijnen,J.,&vanBeers,W.(2005).RobustnessofKrigingwheninterpolatinginrandomsimulationwithheterogeneousvariances:someexperiments.EuropeanJournalofOperationalResearch,165(3),826{834. Kurtaran,H.,Eskandarian,A.,Marzougui,D.,&Bedewi,N.(2002).Crashworthinessdesignoptimizationusingsuccessiveresponsesurfaceapproximations.Computationalmechanics,29(4),409{421. Kwon,Y.,&Berner,J.(1997).Matrixdamageofbrouscomposites:eectsofthermalresidualstressesandlayersequences.ComputersandStructures,64(1-4),375{382. Lee,J.,Yang,Y.,&Ruy,W.(2002).Acomparativestudyonreliability-indexandtarget-performance-basedprobabilisticstructuraldesignoptimization.ComputersandStructures,80(3-4),257{269. Lin,Y.,Luo,D.,Bailey,T.,Khire,R.,Wang,J.,&Simpson,T.W.(2008).ModelValidationandErrorModelingtoSupportSequentialSampling.InProceedingsoftheASMEDesignEngineeringTechnicalConferences-DesignAutomationConference,NewYork,NY,DETC2008/DAC-49336. Marrel,A.(2008).Miseenoeuvreetutilisationdumetamodeleprocessusgaussienpourl'analysedesensibilitedemodelesnumeriques:Applicationauncodedetransporthydrogeologique.Ph.D.thesis,INSAToulouse. 157

PAGE 158

Martin,J.,&Simpson,T.(2004).AMonteCarlosimulationoftheKrigingmodel.In10thAIAA/ISSMOSymposiumonMultidisciplinaryAnalysisandOptimization,(pp.2004{4483). Matheron,G.(1969).Lekrigeageuniversel.Cahiersducentredemorphologiemathema-tique,1. McKay,M.,Beckman,R.,&Conover,W.(2000).Acomparisonofthreemethodsforselectingvaluesofinputvariablesintheanalysisofoutputfromacomputercode.Technometrics,(pp.55{61). Melchers,R.(1987).Structuralreliability:analysisandprediction.EllisHorwood. Molchanov,I.,&Zuyev,S.(2002).Steepestdescentalgorithmsinaspaceofmeasures.StatisticsandComputing,12(2),115{123. Mourelatos,Z.,Kuczera,R.,&Latcha,M.(2006).AnEcientMonteCarloReliabilityAnalysisusingGlobalandLocalMetamodels.InProceedingsof11thAIAA/ISSMOmultidisciplinaryanalysisandoptimizationconference,September,Portsmouth,VA. Myers,R.,&Montgomery,D.(1995).Responsesurfacemethodology:processandproductoptimizationusingdesignedexperiments.JohnWiley&Sons,NewYork. Neal,D.,Matthews,W.,&Vangel,M.(1992).UncertaintyinObtainingHighReliabilityfroStress-StrengthModels.InProceedingsofthe9thDOD-/NASA/FAAConferenceonFibriousCompositesinStructuralDesign,vol.1,(pp.503{521). Niederreiter,H.(1992).Randomnumbergenerationandquasi-MonteCarlomethods.SocietyforIndustrialMathematics. Nikolaidis,E.,Chen,S.,Cudney,H.,Haftka,R.,&Rosca,R.(2004).Comparisonofprobabilityandpossibilityfordesignagainstcatastrophicfailureunderuncertainty.JournalofMechanicalDesign,126,386. Oakley,J.(2004).Estimatingpercentilesofuncertaincomputercodeoutputs.Appliedstatistics,(pp.83{93). Oakley,J.,&O'Hagan,A.(2004).Probabilisticsensitivityanalysisofcomplexmodels:aBayesianapproach.JournaloftheRoyalStatisticalSociety.SeriesB,StatisticalMethodology,(pp.751{769). Osio,I.,&Amon,C.(1996).AnengineeringdesignmethodologywithmultistageBayesiansurrogatesandoptimalsampling.ResearchinEngineeringDesign,8(4),189{206. Qu,X.,&Haftka,R.(2004).Reliability-baseddesignoptimizationusingprobabilisticsuciencyfactor.StructuralandMultidisciplinaryOptimization,27(5),314{325. 158

PAGE 159

Qu,X.,Haftka,R.,Venkataraman,S.,&Johnson,T.(2003).Deterministicandreliability-basedoptimizationofcompositelaminatesforcryogenicenvironments.AIAAjournal,41(10),2029{2036. Rackwitz,R.(2000).Reliabilityanalysispast,presentandfuture.In8thASCESpecialityConferenceonProbabilisticMechanicsandStructuralReliability. Rajashekhar,R.,&Ellingwood,B.(1993).Anewlookattheresponsesurfaceapproachforreliabilityanalysis.StructuralSafety,(pp.205{220). Ramu,P.,Kim,N.,&Haftka,R.(2007).ErrorAmplicationinFailureProbabilityEstimatesofSmallErrorsinResponseSurfaceApproximations.SAESP,2119,37. Ranjan,P.,Bingham,D.,&Michailidis,G.(2008).Sequentialexperimentdesignforcontourestimationfromcomplexcomputercodes.Technometrics,50(4),527{541. Rasmussen,C.,&Williams,C.(2006).Gaussianprocessesformachinelearning.Springer. Riez,F.,&Nagy,B.(1952).Leconsd`AnalyseFonctionnelle. Sacks,J.,Schiller,S.,&Welch,W.(1989a).Designsforcomputerexperiments.Techno-metrics,(pp.41{47). Sacks,J.,Welch,W.,Mitchell,T.,&Wynn,H.(1989b).Designandanalysisofcomputerexperiments.Statisticalscience,(pp.409{423). Santner,T.,Williams,B.,&Notz,W.(2003).Thedesignandanalysisofcomputerexperiments.Springer. Scheidt,C.(2006).Analysestatistiqued'experiencessimulees:Modelisationadaptativedereponsesnonregulieresparkrigeageetplansd'experiences,Applicationalaquanti-cationdesincertitudeseningenieriedesreservoirspetroliers.Ph.D.thesis,UniversiteLouisPasteurStrasbourg. Shan,S.,&Wang,G.(2004).Spaceexplorationandglobaloptimizationforcomputationallyintensivedesignproblems:aroughsetbasedapproach.StructuralandMultidisciplinaryOptimization,28(6),427{441. Shewry,M.,&Wynn,H.(1987).Maximumentropysampling.JournalofAppliedStatistics,14(2),165{170. Smarslok,B.,Alexander,D.,Haftka,R.,Carraro,L.,&Ginsbourger,D.(2008).SeparableMonteCarloSimulationAppliedtoLaminatedCompositePlatesReliability.In4thMultidisciplinaryDesignOptimizationSpecialistConference.AmericanInstituteofAeronauticsandAstronautics. Sobol,I.(1976).Uniformlydistributedsequenceswithanadditionaluniformproperty.Zh.Vychisl.Mat.Mat.Fiz,16,1332{1337. 159

PAGE 160

StarnesJr,J.,&Haftka,R.(1979).Preliminarydesignofcompositewingsforbuckling,strength,anddisplacementconstraints.JournalofAircraft,16(8),564{570. Stein,M.(1999).Interpolationofspatialdata:Sometheoryforkriging.Springer. Steinberg,D.,&Hunter,W.(1984).Experimentaldesign:reviewandcomment.Techno-metrics,(pp.71{97). Stine,R.(1985).Bootstrappredictionintervalsforregression.JournaloftheAmericanStatisticalAssociation,(pp.1026{1031). Tu,C.,&Barton,R.(1997).Productionyieldestimationbythemetamodelmethodwithaboundary-focusedexperimentdesign.InASMEDesignEngineeringTechnicalConference.AmericanSocietyOfMechanicalEngineers. Turner,C.,Campbell,M.,&Crawford,R.(2003).Genericsequentialsamplingformeta-modelapproximations.InASME2003DesignEngineeringTechnicalConferencesandComputersandInformationinEngineeringConference,(pp.2{6). Vazquez,E.,&Piera-Martinez,M.(2006).EstimationofthevolumeofanexcursionsetofaGaussianprocessusingintrinsickriging.URLhttp://arxiv.org/abs/math.ST/0611273. Venter,G.,Haftka,R.,&Starnes,J.(1998).Constructionofresponsesurfaceapproximationsfordesignoptimization.AIAAjournal,36(12),2242{2249. Williams,B.,Santner,T.,&Notz,W.(2000).Sequentialdesignofcomputerexperimentstominimizeintegratedresponsefunctions.StatisticaSinica,10(4),1133{1152. Wu,C.(1978).Somealgorithmicaspectsofthetheoryofoptimaldesigns.TheAnnalsofStatistics,(pp.1286{1301). Youn,B.,&Wang,P.(2008).Bayesianreliability-baseddesignoptimizationusingeigenvectordimensionreduction(EDR)method.StructuralandMultidisciplinaryOptimization,36(2),107{123. 160

PAGE 161

BIOGRAPHICALSKETCH VictorPichenywasborninParis,Francein1983.AftertwoyearsofpreparatoryclassesatthelyceeEugeneLivetinNantes,hejoinedtheEcoledesMinesdeSaintEtienneforhisgraduateeducation.HereceivedinSeptember2005anengineeringdiplomaandamasterofscienceinappliedmathematics.HepursuedhiseducationbyajointprogrambetweentheEcoledesMinesdeSaintEtienneandtheUniversityofFloridatoworktowardaPh.Dinaerospaceengineeringandappliedmathematics,underthetutelageofProf.RaphaelHaftkaandProf.Nam-HoKim.DuringhisPh.Dstudy,hedidaninternshipattheAppliedComputingInstituteoftheUniversityofMaracaiboinVenezuela.Hisresearchinterestsincludedesignandoptimizationmethods,reliabilityanalysis,surrogatemodeling,anddesignofexperiments. 161