Citation
Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling

Material Information

Title:
Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling
Creator:
Picheny, Victor
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (161 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Aerospace Engineering
Mechanical and Aerospace Engineering
Committee Chair:
Haftka, Raphael T.
Committee Members:
Kim, Nam Ho
Schmitz, Tony L.
Peters, Jorg
Roustant, Olivier
Panasenko, Grigori
Vautrin, Alain
Iooss, Bertrand
Graduation Date:
12/18/2009

Subjects

Subjects / Keywords:
Covariance ( jstor )
Cumulative distribution functions ( jstor )
Design optimization ( jstor )
Estimate reliability ( jstor )
Estimators ( jstor )
Experiment design ( jstor )
Kriging ( jstor )
Modeling ( jstor )
Safety factors ( jstor )
Simulations ( jstor )
Mechanical and Aerospace Engineering -- Dissertations, Academic -- UF
conservativeness, design, metamodel, prediction, reliability
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Aerospace Engineering thesis, Ph.D.

Notes

Abstract:
In most engineering fields, numerical simulators are used to model complex phenomena and obtain high-fidelity analysis. Despite the growth of computer capabilities, such simulators are limited by their computational cost. Surrogate modeling is a popular method to limit the computational expense. It consists of replacing the expensive model by a simpler model (surrogate) fitted to a few chosen simulations at a set of points called a design of experiments (DoE). By definition, a surrogate model contains uncertainties, since it is an approximation to an unknown function. A surrogate inherits uncertainties from two main sources: uncertainty in the observations (when they are noisy), and uncertainty due to finite sample. One of the major challenges in surrogate modeling consists of controlling and compensating for these uncertainties. Two classical frameworks of surrogate application are used as a discussion thread for this research: constrained optimization and reliability analysis. In this work, we propose alternatives to compensate for the surrogate model errors in order to obtain safe predictions with minimal impact on the accuracy. The methods are based on different error estimation techniques, some based on statistical assumptions and some that are non-parametric. Their efficiency are analyzed for general prediction and for the approximation of reliability measures. We also propose two contributions to the field of design of experiments in order to minimize the uncertainty of surrogate models. Firstly, we address the issue of choosing the experiments when surrogates are used for reliability assessment and constrained optimization. Secondly, we propose global sampling strategies to answer the issue of allocating limited computational resource in the context of RBDO. All methods are supported by quantitative results on simple numerical examples and engineering applications. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ph.D.)--University of Florida, 2009.
Local:
Adviser: Haftka, Raphael T.
Statement of Responsibility:
by Victor Picheny.

Record Information

Source Institution:
UFRGP
Rights Management:
Copyright Picheny, Victor. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Classification:
LD1780 2009 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

IMPROVINGACCURACYANDCOMPENSATINGFORUNCERTAINTYINSURROGATEMODELINGByVICTORPICHENYADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2009 1

PAGE 2

c2009VictorPicheny 2

PAGE 3

ACKNOWLEDGMENTS Firstofall,IwouldliketothankmyadvisorsDr.RaphaelHaftka,Dr.Nam-HoKimandDr.OlivierRoustant.Theytaughtmehowtoconductresearch,gavemepricelessadvicesandsupport,andprovidedmewithexcellentguidancewhilegivingmethefreedomtodevelopmyownideas.Ifeelveryfortunatetohaveworkedundertheirguidance.Iwouldalsoliketothankthemembersofmyadvisorycommittee,Dr.BertrandIooss,Dr.GrigoriPanasenko,Dr.JorgPeters,Dr.AlainVautrinandDr.TonySchmitz.Iamgratefulfortheirwillingnesstoserveonmycommittee,forreviewingmydissertationandforprovidingconstructivecriticismthathelpedmetoenhanceandcompletethiswork.IparticularlythankDr.IoossandDr.Schmitzfortheirworkasrapporteursofmydissertation.Iamgratefulformanypeoplethatcontributedscienticallytothisdissertation:Dr.NestorQueipo,whowelcomedmeformyinternshipattheUniversityofZulia(Venezuela)andprovidedmewiththoughtfulideasandadvices;Dr.RodolpheLeRiche,withwhomIdevelopedafruitfulcollaborationonthestudyofgearboxproblems;Dr.XavierBay,whoseencyclopedicknowledgeofmathematicsallowedustondpromisingresultsondesignoptimality;Dr.FelipeViana,forhisecientcontributiontotheconservativesurrogateproblem;andDr.DavidGinsbourger,forhiscriticalinputonthetargeteddesignswork...andhiscontagiousmotivation.Iwishtothanktoallmycolleaguesfortheirfriendshipandsupport,fromtheStructuralandMultidisciplinaryOptimizationGroupoftheUniversityofFlorida,theAppliedComputingInstituteoftheUniversityofZulia,andthe3MIdepartmentfromtheEcoledesMines:Tushar,Amit,Palani,Erdem,Jaco,Ben,Christian,Felipe,Alex,Camila;Eric,Delphine,Celine,Bertrand,David,Nicolas,Olga,Natacha,Khouloud,Ksenia,andmanyothers...IalsothankallthestaoftheUniversityofFloridaandEcoledesMines 3

PAGE 4

thatmadepossiblemyjointPhDprogram,andinparticularPamandKaren,whohelpedmemanytimes.Financialsupports,providedpartlybyNationalScienceFondation(Grant#0423280)formytimeinFloridaandbytheCETIMfoundationformytimeinFrance,aregratefullyacknowledged.Mynalthoughtsgotomyfriends,family,andNatacha. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................. 3 LISTOFTABLES ..................................... 9 LISTOFFIGURES .................................... 10 LISTOFABBREVIATIONS ............................... 13 ABSTRACT ........................................ 14 CHAPTER 1INTRODUCTION .................................. 16 2ELEMENTSOFSURROGATEMODELING ................... 22 2.1SurrogateModels ................................ 22 2.1.1NotationAndConcepts ......................... 22 2.1.2TheLinearRegressionModel ...................... 23 2.1.3TheKrigingModel ........................... 25 2.1.3.1Krigingwithnoise-freeobservations ............. 27 2.1.3.2Krigingwithnuggeteect .................. 29 2.2DesignOfExperimentStrategies ....................... 30 2.2.1ClassicalAndSpace-FillingDesigns .................. 30 2.2.2Model-OrientedDesigns ......................... 32 2.2.3AdaptiveDesigns ............................ 34 3CONSERVATIVEPREDICTIONSUSINGSURROGATEMODELING .... 36 3.1Motivation .................................... 36 3.2DesignOfConservativePredictors ....................... 37 3.2.1DenitionOfConservativePredictors ................. 37 3.2.2MetricsForConservativenessAndAccuracy ............. 38 3.2.3ConstantSafetyMarginUsingCross-ValidationTechniques ..... 40 3.2.4PointwiseSafetyMarginBasedOnErrorDistribution ........ 41 3.2.5PointwiseSafetyMarginUsingBootstrap ............... 42 3.2.5.1TheBootstrapprinciple ................... 42 3.2.5.2Generatingcondenceintervalsforregression ....... 43 3.2.6AlternativeMethods .......................... 46 3.3CaseStudies ................................... 46 3.3.1TestProblems .............................. 46 3.3.1.1TheBranin-Hoofunction .................. 46 3.3.1.2Thetorquearmmodel .................... 47 3.3.2NumericalProcedure .......................... 49 3.3.2.1Graphsofperformance .................... 49 5

PAGE 6

3.3.2.2Numericalset-upfortheBranin-Hoofunction ....... 49 3.3.2.3Numericalset-upfortheTorquearmmodel ........ 50 3.4ResultsAndDiscussion ............................. 51 3.4.1Branin-HooFunction .......................... 51 3.4.1.1Analysisofunbiasedsurrogatemodels ........... 51 3.4.1.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates ........................ 51 3.4.1.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates ........................... 55 3.4.2TorqueArmModel ........................... 58 3.4.2.1Analysisofunbiasedsurrogatemodels ........... 58 3.4.2.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates ........................ 58 3.4.2.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates ........................... 61 3.5ConcludingComments ............................. 62 4CONSERVATIVEPREDICTIONSFORRELIABILITY-BASEDDESIGN ... 63 4.1IntroductionAndScope ............................ 63 4.2EstimationOfProbabilityOfFailureFromSamples ............. 64 4.2.1Limit-StateAndProbabilityOfFailure ................ 64 4.2.2EstimationOfDistributionParameters ................ 66 4.3ConservativeEstimatesUsingConstraints .................. 68 4.4ConservativeEstimatesUsingTheBootstrapMethod ............ 70 4.5AccuracyAndConservativenessOfEstimatesForNormalDistribution .. 73 4.6EectOfSampleSizesAndProbabilityOfFailureOnEstimatesQuality 75 4.7ApplicationToACompositePanelUnderThermalLoading ........ 78 4.7.1ProblemDenition ........................... 78 4.7.2Reliability-BasedOptimizationProblem ................ 80 4.7.3ReliabilityBasedOptimizationUsingConservativeEstimates .... 82 4.7.4OptimizationResults .......................... 84 4.8ConcludingComments ............................. 86 5DESIGNOFEXPERIMENTSFORTARGETREGIONAPPROXIMATION 88 5.1MotivationAndBibliography ......................... 88 5.2ATargetedIMSECriterion .......................... 90 5.2.1TargetRegionDenedByAnIndicatorFunction ........... 90 5.2.2TargetRegionDenedByAGaussianDensity ............ 93 5.2.3Illustration ................................ 94 5.3SequentialStrategiesForSelectingExperiments ............... 95 5.4PracticalIssues ................................. 97 5.4.1SolvingTheOptimizationProblem .................. 97 5.4.2Parallelization .............................. 99 5.5ApplicationToProbabilityOfFailureEstimation .............. 99 6

PAGE 7

5.6NumericalExamples .............................. 101 5.6.1Two-DimensionalExample ....................... 101 5.6.2Six-DimensionalExample ........................ 103 5.6.3ReliabilityExample ........................... 104 5.7ConcludingRemarks .............................. 106 6OPTIMALALLOCATIONOFRESOURCEFORSURROGATEMODELING 109 6.1SimulatorsWithTunableFidelity ....................... 109 6.2ExamplesOfSimulatorsWithTunableFidelity ............... 110 6.2.1Monte-CarloBasedSimulators ..................... 110 6.2.2RepeatableExperiments ........................ 111 6.2.3FiniteElementAnalysis ........................ 112 6.3OptimalAllocationOfResource ........................ 113 6.4ApplicationToRegression ........................... 114 6.4.1ContinuousNormalizedDesigns .................... 114 6.4.2SomeImportantResultsOfOptimalDesigns ............. 116 6.4.3AnIllustrationOfAD-OptimalDesign ................ 117 6.4.4AnIterativeProcedureForConstructingD-OptimalDesigns .... 119 6.4.5ConcludingRemarks .......................... 120 6.5ApplicationToKriging ............................. 121 6.5.1ContextAndNotations ......................... 121 6.5.2AnExploratoryStudyOfTheAsymptoticProblem ......... 122 6.5.3AsymptoticPredictionVarianceAndIMSE .............. 123 6.5.3.1Generalresult ......................... 124 6.5.3.2Adirectapplication:space-llingdesigns .......... 125 6.5.4Examples ................................. 126 6.5.4.1Brownianmotion ....................... 126 6.5.4.2Orstein-Uhlenbeckprocess .................. 128 6.5.5ConcludingComments ......................... 129 7CONCLUSION .................................... 131 7.1SummaryAndLearnings ............................ 131 7.2Perspectives ................................... 133 APPENDIX AALTERNATIVESFORCONSERVATIVEPREDICTIONS ........... 135 A.1BiasedFittingEstimators ........................... 135 A.1.1BiasedFittingModels .......................... 135 A.1.2ResultsAndComparisonToOtherMethods ............. 136 A.2IndicatorKriging ................................ 137 A.2.1DescriptionOfTheModel ....................... 137 A.2.2ApplicationToTheTorqueArmAnalysis ............... 139 7

PAGE 8

BDERIVATIONOFTHEASYMPTOTICKRIGINGVARIANCE ........ 141 B.1KernelsOfFiniteDimension .......................... 141 B.2GeneralCase .................................. 146 B.3TheSpace-FillingCase ............................. 150 CSPECTRALDECOMPOSITIONOFTHEORSTEIN-UHLENBECKCOVARIANCEFUNCTION ...................................... 152 REFERENCES ....................................... 155 BIOGRAPHICALSKETCH ................................ 161 8

PAGE 9

LISTOFTABLES Table page 2-1SequentialDoEminimizingthemaximumpredictionvarianceateachstep. ... 34 3-1Rangeofthedesignvariables(cm). ......................... 47 3-2Meanvaluesofstatisticsbasedon1,024testpointsand500DoEsfortheunbiasedsurrogatesbasedon17observations. ........................ 51 3-3Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. ......... 58 4-1Comparisonofthemean,standarddeviation,andprobabilityoffailureofthethreedierentCDFestimatorsforN()]TJ /F1 11.955 Tf 9.29 0 Td[(2:33;1:02) ................. 70 4-2MeansandcondenceintervalsofdierentestimatesofP(G0)andcorrespondingvalueswhereGisthenormalrandomvariableN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02). ......... 74 4-3MechanicalpropertiesofIM600/133material. ................... 79 4-4Deterministicoptimafoundby Quetal. ( 2003 ). .................. 80 4-5Coecientsofvariationoftherandomvariables. .................. 81 4-6Variablerangeforresponsesurface. ......................... 82 4-7StatisticsofthePRSforthereliabilityindexbasedontheunbiasedandconservativedatasets. ....................................... 84 4-8StatisticsofthePRSbasedon22testpoints .................... 84 4-9Optimaldesignsofthedeterministicandprobabilisticproblemswithunbiasedandconservativedatasets. ............................. 85 5-1ProcedureoftheIMSET-basedsequentialDoEstrategy. ............. 96 5-2ProbabilityoffailureestimatesforthethreeDoEsandtheactualfunctionbasedon107MCS. ..................................... 106 6-1IMSEvaluesforthetwoKrigingmodelsandtheasymptoticmodel. ....... 128 A-1Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. ......... 139 9

PAGE 10

LISTOFFIGURES Figure page 2-1ExampleofSimpleKrigingmodel. ......................... 28 2-2ExampleofSimpleKrigingmodelwithnoisyobservations. ............ 29 2-3Examplesoffull-factorialandcentralcompositedesigns. ............. 30 2-4Anine-pointLHSdesign. .............................. 31 3-1Schematicrepresentationofbootstrapping. ..................... 43 3-2Percentileoftheerrordistributioninterpolatedfrom100valuesofu. ...... 45 3-3Branin-Hoofunction. ................................. 46 3-4InitialdesignoftheTorquearm. .......................... 47 3-5Designvariablesusedtomodifytheshape. ..................... 48 3-6VonMisesstresscontourplotattheinitialdesign. ................. 48 3-7AverageresultsforCSMandEDforKrigingbasedon17points. ......... 52 3-8AverageresultsforCSMandEDforPRSbasedon17points. .......... 52 3-9AverageresultsforCSMandEDforKrigingbasedon34points. ......... 53 3-10Targetvs.actual%cforPRSusingCSMwithcross-validationandEDbasedon17points. ..................................... 54 3-11Targetvs.actual%cforKrigingusingCSMwithcross-validationandEDbasedon17points. ..................................... 55 3-12Condenceintervals(CI)computedusingclassicalregressionandbootstrapwhentheerrorfollowsalognormaldistribution. ..................... 56 3-13AverageresultsforBSandEDforPRSbasedon17points. ........... 57 3-14Targetvs.actual%cforKrigingusingCSMandcross-validationandEDbasedon17points. ..................................... 57 3-15AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSonthetorquearmdata. ....................................... 59 3-16AverageresultsforCSMandEDforKrigingonthetorquearmdata. ...... 59 3-17Targetvs.actual%cforPRSusingCSMandcross-validationandEDonthetorquearmdata. ................................... 60 10

PAGE 11

3-18Targetvs.actual%cforKrigingusingCSMandcross-validationandEDonthetorquearmdata. ................................. 60 3-19AverageresultsforBSandEDforPRSonthetorquearmdata. ......... 61 3-20Targetvs.actual%cforPRSusingBSandEDonthetorquearmdata. .... 61 4-1CriteriatotempiricalCDFs. ............................ 68 4-2ExampleofCDFestimatorsbasedonRMSerrorforasampleofsize10generatedfromN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02) ................................. 70 4-3ConservativeestimatorsofPffrombootstrapdistribution:95thpercentile(p95)andmeanofthe10%highestvalues(CVaR). .................... 72 4-4Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforNormaldistribution. ................................. 77 4-5Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforlognormaldistribution. ................................ 77 4-6Geometryandloadingofthecryogeniclaminate. ................. 79 5-1One-dimensionalillustrationofthetargetregion. ................. 91 5-2Illustrationoftheweightsfunctions. ........................ 95 5-3Optimaldesignafter11iterations. ......................... 102 5-4EvolutionofKrigingtargetcontourlinecomparedtoactualduringthesequentialprocess. ........................................ 103 5-5ComparisonoferrordistributionfortheoptimalandLHSDoEs. ......... 105 5-6BoxplotsoferrorsfortheLHSandoptimaldesignsforthetestpointswhereresponsesareinsidethedomains[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+" ..................... 106 5-7Optimaldesignswithuniformintegrationmeasureandwithinputdistributionintegrationmeasure. ................................. 107 6-1EvolutionoftheresponseoftheFEAforthetorquearmexample,whenthecomplexityofthemodel(meshdensity)increases. ................. 113 6-2D-optimaldesignforasecond-orderpolynomialregressionandcorrespondingpredictionvariance. .................................. 118 6-3Full-factorialdesignwithhomogeneousallocationofcomputationaleortandcorrespondingpredictionvariance. .......................... 119 6-436-pointFull-factorialdesignwithhomogeneousallocationofcomputationaleortandcorrespondingpredictionvariance. .................... 119 11

PAGE 12

6-5EvolutionoftheIMSEwithrespecttothenumberofobservationsforuniformrandomdesignswithGaussianandExponentialcovariances. ........... 123 6-6IllustrationoftheMSEfunctionforaBrownianmotion(2=1,2=0:002). .. 127 6-7IllustrationoftheMSEfunctionforaGaussianprocess(GP)withexponentialcovariancefunction(2=1,=0:2and2=0:002). ............... 130 A-1AverageresultsforBiasedttingwithconstraintrelaxationandconstraintselectionontheBranin-Hoofunction. ............................. 137 A-2AverageresultsforCSMandBiasedttingwithconstraintselection. ...... 138 A-3AverageresultsforIndicatorKrigingandEDwithKriging. ............ 140 A-4Targetvs.actual%cforIK(plainblack)andKrigingwithED(mixedgray). .. 140 C-1Representationofthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2u2)sin(u). ...... 154 12

PAGE 13

LISTOFABBREVIATIONS BS BootstrapCDF CumulativeDistributionFunctionCEC ConservativetotheEmpiricalCurveCSM ConstantSafetyMarginCSP ConservativeatSamplePointsDoE DesignofExperimentsED ErrorDistributioneRMS RootMeanSquareErrorl LossinaccuracyLHS LatinHypercubeSamplingMaxUE MaximumUnconservativeErrorMaxUE% ReductionoftheMaximumUnconservativeErrorMCS Monte-CarloSimulationsMSE MeanSquareErrorPDF ProbabilityDensityFunctionPf ProbabilityoffailurePRS PolynomialResponseSurfaceRBDO Reliability-BasedDesignOptimization%c Percentageofconservativeresults 13

PAGE 14

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyIMPROVINGACCURACYANDCOMPENSATINGFORUNCERTAINTYINSURROGATEMODELINGByVictorPichenyDecember2009Chair:RaphaelT.HaftkaMajor:AerospaceEngineering Inmostengineeringelds,numericalsimulatorsareusedtomodelcomplexphenomenaandobtainhigh-delityanalysis.Despitethegrowthofcomputercapabilities,suchsimulatorsarelimitedbytheircomputationalcost.Surrogatemodelingisapopularmethodtolimitthecomputationalexpense.Itconsistsofreplacingtheexpensivemodelbyasimplermodel(surrogate)ttedtoafewchosensimulationsatasetofpointscalledadesignofexperiments(DoE). Bydenition,asurrogatemodelcontainsuncertainties,sinceitisanapproximationtoanunknownfunction.Asurrogateinheritsuncertaintiesfromtwomainsources:uncertaintyintheobservations(whentheyarenoisy),anduncertaintyduetonitesample.Oneofthemajorchallengesinsurrogatemodelingconsistsofcontrollingandcompensatingfortheseuncertainties.Twoclassicalframeworksofsurrogateapplicationareusedasadiscussionthreadforthisresearch:constrainedoptimizationandreliabilityanalysis. Inthiswork,weproposealternativestocompensateforthesurrogatemodelerrorsinordertoobtainsafepredictionswithminimalimpactontheaccuracy.Themethodsarebasedondierenterrorestimationtechniques,somebasedonstatisticalassumptionsandsomethatarenon-parametric.Theireciencyareanalyzedforgeneralpredictionandfortheapproximationofreliabilitymeasures. 14

PAGE 15

Wealsoproposetwocontributionstotheeldofdesignofexperimentsinordertominimizetheuncertaintyofsurrogatemodels.Firstly,weaddresstheissueofchoosingtheexperimentswhensurrogatesareusedforreliabilityassessmentandconstrainedoptimization.Secondly,weproposeglobalsamplingstrategiestoanswertheissueofallocatinglimitedcomputationalresourceinthecontextofRBDO. Allmethodsaresupportedbyquantitativeresultsonsimplenumericalexamplesandengineeringapplications. 15

PAGE 16

CHAPTER1INTRODUCTION Inthepastdecades,scientistshavebenetedfromthedevelopmentofnumericaltoolsforthehelpoflearning,predictionanddesign.Thegrowthofcomputationalcapabilitieshasallowedtheconsiderationofphenomenaofconstantlyincreasinglevelofcomplexity,whichresultsinbetterunderstandingandmoreecientsolutionsofreal-lifeproblems. Complexnumericalsimulatorscanbeencounteredinawiderangeofengineeringelds.Theautomotiveindustryhasdevelopednumericalmodelsforthebehaviorofcarsduringcrash-tests,whichinvolvehighlynon-linearmechanicalmodes,inordertointegrateittothedesignofcarstructures.Ingeophysics,owsimulatorsareusedforthepredictionofthebehaviorofCO2sequestrationintonaturalreservoirs,orthepredictionofoilrecoveryenhancement,basedonacomplexmappingofgroundcharacteristics. Theincreasingcomputationalpowerhasalsoallowedtheincorporationofuncertaintyinformationintosystemanalysis.Inparticular,reliabilityanalysisaimsatquantifyingthechancethatasystembehavesasitisrequired,whenasetoftheproblemparametersareuncertain[ Rackwitz ( 2000 ), Haldar&Mahadevan ( 2000 )].Instructuralanalysis,uncertaintytypicallycomesfrommaterialproperties(duetomanufacturing)andactualworkingconditions.Forobviousreasons,reliabilityassessmenthasbeenintensivelyexploredinaerospaceandnuclearengineering. However,thegaininrealismmakestheuseofnumericalmodelsextremelychallenging.Indeed,thecomplexityofthesimulatorsmakesthemhighlyexpensivecomputationally,limitingsystematiclearningordesignprocess.Also,thenumberofparameters(orinputvariables)thatexplainthephenomenonofinterestcanpotentiallybeverylarge,makingdicultthequanticationoftheirinuenceonthesimulatorresponse. Toovercomecomputationalcostlimitations,surrogatemodels,ormetamodels,havebeenfrequentlyusedonmanyapplications.Theideaofsurrogatemodelsconsistsofreplacingtheexpensivemodelbyasimplermathematicalmodel(orsurrogate)ttedtoa 16

PAGE 17

fewchosensimulationsatasetofpointscalledadesignofexperiments(DoE).Surrogatemodelsarethenusedtopredictthesimulatorresponsewithverylimitedcomputationalcost[ Box&Draper ( 1986 ), Myers&Montgomery ( 1995 ), Santneretal. ( 2003 ), Sacksetal. ( 1989a )]. Bydenition,asurrogatemodelcontainsuncertainties,sinceitisanapproximationtoanunknownfunction.Asurrogateinheritsuncertaintiesfromtwomainsources:uncertaintyintheobservations(whentheyarenoisy),anduncertaintyduetothelackofdata.Indeed,asimulatorisanapproximationtoarealphenomenon,andthecondenceonecanputinitsresponsesdependsonthequalityofthenumericalmodel.Thepropertiesofthesurrogatestronglydependontheaccuracyoftheobservations.Secondly,inmostapplications,thenumberofsimulationsrunsisseverelylimitedbythecomputationalcapabilities.Theinformationonwhichthesurrogateisttedisinsucienttoobtainanaccurateapproximationofthesimulatorbehavior.Oneofthemajorchallengesinsurrogatemodelingconsistsofcontrollingandcompensatingfortheseuncertainties. Thepresentworkconsidersasadiscussionthreadtheclassicalframeworkofreliability-baseddesignoptimization(RBDO),forwhichtheissuesassociatedwithuncertaintyinsurrogatemodelingareparticularlycrucial.RBDOisapopularwaytoaddressuncertaintyinthedesignprocessofasystem.Itconsistsofoptimizingaperformancefunctionwhileensuringaprescribedreliabilitylevel.RBDOproblemsarecomputationallychallenging,sincetheyrequirenumerouscallstoreliabilitymeasuresthataremostofthetimeexpensivetoestimate,hencebeinganaturalcandidateforsurrogateapplication[ Rajashekhar&Ellingwood ( 1993 ), Venteretal. ( 1998 ), Kurtaranetal. ( 2002 )].SurrogatemodelingcanbeusedattwolevelsoftheRBDO:(1)duringtheoptimizationprocess,byapproximatingthereliabilityconstraint,and(2)duringthereliabilityestimationitself. 17

PAGE 18

ThechallengesinusingsurrogatemodelinginRBDOarevarious.Indeed,apoorapproximationofthereliabilitylevelcanseverelyharmtheoptimizationprocess:overestimationleadstonon-optimaldesigns,andunderestimationtoweakdesigns.Reliabilityassessmentmethodsaregenerallybasedonsamplingmethods(suchasMonte-Carlosimulations)thatonlyprovideestimatesoftheactualreliabilitylevels.Quanticationoftheuncertaintyonbothreliabilityestimatesandasurrogatebasedonsuchdataisneededtolimittheriskofpoordesigns. Ramuetal. ( 2007 )showthattheuseofsurrogatesforreliabilityassessmentisparticularlychallenging,sincesmallerrorsinthemodelcouldresultinlargeerrorsinthereliabilitymeasure.Inaddition,forcomputationalreasons,thenumberofreliabilityestimatesislimitedtoasmallvalue,andecientsamplingstrategiesmustbeusedtoensureanacceptableaccuracyofthesurrogate. Itispossibletocompensateforthelackofaccuracywithextrasafetymargins.Suchapproachisoftenreferredasconservative,andaconservativenesslevelquantifythechancethatanapproximationisonthesafesidefortheanalysis.Forexample, StarnesJr&Haftka ( 1979 )replacedthelinearTaylorseriesapproximationwithatangentapproximationbiasedtobeconservativeinordertoreducethechanceofunconservativeapproximationstobucklingloads.Manyengineeringapplicationshaveadoptedconservativeestimation.Forinstance,FederalAviationAdministration(FAA)denesconservativematerialproperty(A-basisandB-basis)asthevalueofamaterialpropertyexceededby99%(forA-basis,90%forB-basis)ofthepopulationwith95%condence.FAArecommendstheuseofA-basisformaterialpropertiesandasafetyfactorof1.5ontheloads.Traditionally,safetyfactorshavebeenextensivelyusedtoaccountforuncertainties,eventhoughtheireectivenessisquestionable[ Elishako ( 2004 ), Acaretal. ( 2007 )]. Estimatingtheuncertaintyindataorthemetamodeliscrucialfortheeectivenessofmostsurrogate-basedapproaches.Uncertaintyanderrorquanticationisaclassicalthemeofsurrogatemodeling.Mostmetamodelscontainbyconstructionerrorestimates[ Cressie 18

PAGE 19

( 1993 ), Isaaks&Srivastava ( 1989 ), Box&Draper ( 1986 )].Theseestimatesarebasedonstatisticalassumptions;forinstance,linearregressionassumesnormalityandindependanceoftheerrors.Inpractice,suchassumptionscanbeviolated,makingthoseerrormeasuresquestionable.Alternatively,numericaltechniquesareavailabletoquantifytheerror[ Stine ( 1985 ), Efron ( 1982 ), Goeletal. ( 2006 )]. Uncertaintyquanticationallowscompensatingmechanisms,forinstanceinordertosettheconservativenesstoaprescribedlevel.However,littleinformationisavailableontheeectoferrorcompensationonthesurrogateapplication.Furthermore,conservativeestimatesarebiasedtobeonthesafeside,sotheconservativenesscomesatapriceofaccuracy.Onegoalofpresentworkistoproposealternativestocompensateforthesurrogatemodelerrors,basedondierenterrorestimationtechniques,anddemonstratetheireciencyforsafepredictionanddesignofengineeringsystems. Inmanyapplications,samplingstrategiescanbechosenbytheuser.Then,experimentscanbedesignedsuchthattheuncertaintyofsurrogatemodelsareminimized.Whensafepredictionisdesired,reduceduncertaintyallowstoobtainconservativenesswithaminimalimpactonaccuracy.Thechoiceandeectivenessofthesamplingstrategieshasbeenwidelyexploredanddiscussedinthesurrogateliterature[ Steinberg&Hunter ( 1984 ), Sacksetal. ( 1989b ), Fedorov&Hackl ( 1997 )].Thisworkproposestwocontributionstothiseld.Firstly,weaddresstheissueofchoosingtheexperimentswhensurrogatesareusedforreliabilityassessmentandconstrainedoptimization.Secondly,weproposeglobalsamplingstrategiestoanswertheissueofallocatinglimitedcomputationalresourceinthecontextofRBDO. Outlineofthedissertation: Chapter2reviewsseveralaspectsofsurrogatemodeling. Chapter3addressestheissueofgeneratingconservativepredictionsusingsurrogatemodels.Twoalternativesareproposed:(1)usingstatisticalinformationprovidedbythesurrogatemodeland(2)usingmodel-independenttechniques(bootstrap,cross-validation) 19

PAGE 20

toobtainstatisticalinformation.Dierentmetricsaredenedtomeasurethetrade-obetweenaccuracyandsafety.Theanalysisofthedierenttechniquesissupportedwiththehelpofananalyticalandastructuralproblemthatusesniteelementsanalysis. Chapter4considerstheapplicationofsurrogatemodelingtoreliability-baseddesign,wheresurrogatesareusedtotprobabilitydistributions.WefocusonthecasewhenlowprobabilitiesareestimatedfromasmallnumberofobservationsobtainedbyMonte-Carlosimulations(MCS).Byusingbiased-ttingtechniquesandresamplingmethods(bootstrap),wecompensatefortheuncertaintyintheanalysisbybeingontheconservativeside,withreasonableimpactontheaccuracyoftheresponse.Anapplicationtotheoptimizationofalaminatecompositewithreliabilityconstraintsisusedfordemonstration,theconstraintbeingapproximatedconservativelyusingbootstrap. InChapter5,weproposeanobjective-basedapproachtosurrogatemodeling,basedontheideathattheuncertaintymaybereducedwhereitismostuseful,insteadofglobally.Anoriginalcriterionisproposedtochoosesequentiallythedesignofexperiments,whenthesurrogateneedstobeaccurateforcertainlevelsofthesimulatorresponse.Thecriterionisatrade-obetweenthereductionofoveralluncertaintyinthesurrogate,andtheuncertaintyreductionintargetregions.Theeectivenessofthemethodisillustratedonasimplereliabilityanalysisapplication. Chapter6considerstheframeworkofsimulatorswhosedelitydependsontunablefactorsthatcontrolthecomplexityofthemodel(suchasMCS-basedsimulators,orRBDOframework).Foreachsimulationrun,theuserhastosetatrade-obetweencomputationalcostandresponseprecision.WhenaglobalcomputationalbudgetfortheDoEisgiven,onemayhavetoanswerthefollowingquestions:(1)isitbettertorunafewaccuratesimulationsoralargenumberofinaccurateones,(2)isitpossibletoimprovethesurrogatebytuningdierentdelitiesforeachrun.Answersareproposedforthesetwoquestions.Forpolynomialregression,itjoinsthewell-exploredtheoryofdesignoptimality. 20

PAGE 21

Forkriging,bothnumericalandanalyticalresultsareproposed;inparticular,asymptoticresultsaregivenwhenthenumberofsimulationrunstendstoinnity. Chapter7recapitulatesmajorconclusionsandresults,anddrawperspectivesfromthiswork. 21

PAGE 22

CHAPTER2ELEMENTSOFSURROGATEMODELING Inthischapter,weintroducetheimportantnotionsrelativetosurrogatemodeling.Wedetailtwotypesofmodels:linearregressionmodels,andKrigingmodels.Wealsodescribetheproblematicofdesignofexperiments,andpresentseveralpopularsamplingstrategies. 2.1SurrogateModels Strategiesinvolvingsurrogatemodelingarerecognizedinawiderangeofengineeringeldstoecientlyaddresscomputationallyexpensiveproblems.Theaimofthissectionistobrieypresentthenotationandmainstepsinsurrogatemodeling.Amongthenumeroustypesofsurrogatemodelsavailableintheliterature,weemphasizetwoofthemostpopularones:linearregressionandKriging.Linearregression,alsoreferredaspolynomialresponsesurfaceintheengineeringliterature,hasbeeninitiallydevelopedforstatisticalinferencebasedonphysicalexperiments.Krigingwasdevelopedbygeostatisticianstomodelspatialcorrelationsofthephysicalcharacteristicsofground.Bothmodelsarenowusedinmanyapplications,eventhoughthehypothesesonwhichthemodelsarebasedarenotnecessarilyguaranteed. Amajornotioninsurrogatemodelingisthedesignofexperiments,orsamplingstrategy,sinceitiscrucialtothequalityofthesurrogateanalysis.Inthischapter,webrieydescribethreeclassicalsamplingstrategies:space-lling,model-baseddesignsandadaptivedesigns.MoreadvancednotionsareaddressedinChapter5foradaptivedesignsandChapter6formodel-orientedoptimaldesigns. 2.1.1NotationAndConcepts Letusrstintroducesomenotation.Wedenotebyytheresponseofanumericalsimulatororfunctionthatistobestudied: y:DRd)166(!Rx7)166(!y(x)(2{1) 22

PAGE 23

wherex=fx1;:::;xdgTisad-dimensionalvectorofinputvariables,andDisthedesignspace.Inordertobuildametamodel,theresponseyisobservedatndistinctlocationsX: X=[x1;:::;xn]Yobs=[y(x1);:::;y(xn)]T=y(X)(2{2) Xiscalledthedesignofexperiments(DoE),andYobsistheobservations.Sincetheresponseyisexpensivetoevaluate,weapproximateitbyasimplemodel,calledthemetamodelorsurrogatemodel,basedonassumptionsonthenatureofyandonitsobservationsYobsatthepointsoftheDoE. Themetamodelcaninterpolatethedata(splines,Kriging)orapproximateit(linearregression,Krigingwithnuggeteect).Inthelattercase,itisassumedthatthefunctionofinterestisobservedthroughanoisyprocess,sotheobservationdiersfromthetruefunctionbyanadditiveerrorterm: yobs;i=y(xi)+"i(2{3) with"itheerror.Inmostofthemetamodelhypotheses,theerrorisconsideredasawhitenoisenormallydistributedwithzeromean.Inthissection,wewillalwaysconsiderthishypothesistrue. 2.1.2TheLinearRegressionModel Inlinearregression,theresponseismodeledasalinearcombinationofbasisfunctionsobservedwithanadditiveerrorterm: y(x)=pXj=1jfj(x) (2{4) yobs;i=pXj=1jfj(x)+"i (2{5) wherefj(x)arethebasisfunctions(forinstancepolynomial),jtheweights,and"itheerroratxi. 23

PAGE 24

Givenasetofdesignpoints,thelinearregressionmodel,inmatrixnotation,isdenedasfollow: Yobs=fT(X)+"(2{6) where:F=266666664f(x1)Tf(x2)T...f(xn)T377777775=26666666412...p377777775"=266666664"1"2..."p377777775and:f(x)T=f1(x)f2(x):::fp(x). Typically,theficanbechosenaspolynomialfunctions;inthatcasethemodelisoftencalledpolynomialresponsesurface(PRS). Sinceinpracticetheerrorisunknown,weestimatetheresponseby: ^y(x)=f(x)T^(2{7) where^isanestimateofandischosenbyminimizingthemeansquareerror(MSE)betweentheestimatesandtheactualresponsesatalldesignpoints: MSE=1 nnXi=1[^y((xi)))]TJ /F4 11.955 Tf 11.96 0 Td[(y(xi)]2(2{8) Usingthestandardlinearregression,thevalueof^thatminimizestheMSEisgivenby: ^=)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(FTF)]TJ /F5 7.97 Tf 6.59 0 Td[(1FTYobs (2{9) ^=M)]TJ /F5 7.97 Tf 6.58 0 Td[(1FTYobs (2{10) ThequantityM=FTFiscalledtheFisherinformationmatrix. Inadditiontothebestpredictor^y,thelinearregressionmodelprovidesapredictionvariance,givenby: var[^y(x)]=f(x)TM)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)(2{11) Whentheerrorisheteroskedastic,thatis,whentheerrordistributiondierfromonepointtoanother,theordinaryleastsquareestimatorofisnotappropriate.Instead,we 24

PAGE 25

estimatethecoecientsbythegeneralizedleastsquareestimator: =)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(FT)]TJ /F11 7.97 Tf 8.08 5.05 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1F)]TJ /F5 7.97 Tf 6.59 0 Td[(1FT)]TJ /F11 7.97 Tf 8.09 5.05 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1Yobs(2{12) with:)]TJ /F1 11.955 Tf 11.4 0 Td[(=[cov(y(xi);y(xj))]1i;jn.Inthatcase,theFisherinformationmatrixis: M=FT)]TJ /F11 7.97 Tf 8.09 5.04 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1F(2{13) Notethatiftheerrorsareuncorrelated,)]TJ /F1 11.955 Tf 11.98 0 Td[(reducestodiag[var("1);var("2);:::;var("n)]. Forcalculationdetails,seeforinstance Box&Draper ( 1986 ), Khuri&Cornell ( 1996 )or Myers&Montgomery ( 1995 ). 2.1.3TheKrigingModel TheKrigingmetamodelwasinitiallydevelopedinthegeostatisticframework[ Matheron ( 1969 ), Cressie ( 1993 )]topredictvaluesbasedonspatialcorrelationconsiderations.KrigingcanalsobefoundontheliteratureunderthenameofGaussianProcessregression[ Rasmussen&Williams ( 2006 )],orregressionwithspatiallycorrelatederrors[ Fedorov&Hackl ( 1997 )].ThemainhypothesisbehindtheKrigingmodelistoassumethatthetruefunctionyisonerealizationofaGaussianstochasticprocessY: y(x)=Y(x;!)(2{14) where!belongstotheunderlyingprobabilityspace.InthefollowingweusethenotationY(x)fortheprocessandy(x)foronerealization.ForUniversalKriging,Yisoftheform: Y(x)=pXj=1jfj(x)+Z(x)(2{15) wherefjarelinearlyindependentknownfunctions,andZisaGaussianprocesswithzeromeanandcovariancek(u;v). Thecovariancefunction(orkernel)kcontainsalltheinformationofspatialdependency,anddependsonparameters.Thereexistsmanytypesofcovariancefunctions;twoofthemostpopularonesaretheisotropicgaussianandexponential 25

PAGE 26

covariances: IsotropicGaussiancovariance:k(u;v)=2exp")]TJ /F9 11.955 Tf 11.29 16.86 Td[(ku)]TJ /F7 11.955 Tf 11.95 0 Td[(vk 2# (2{16) IsotropicExponentialcovariance:k(u;v)=2exp)]TJ /F9 11.955 Tf 11.29 16.85 Td[(ku)]TJ /F7 11.955 Tf 11.96 0 Td[(vk (2{17) Forthesecovariances,theparametersaretheprocessvariance2andrange.Anisotropiccovariancefunctionscanalsobedenedbyattributingadierentineachdirection: AnisotropicGaussiancovariance:k(u;v)=2exp")]TJ /F6 7.97 Tf 17.75 14.94 Td[(dXj=1juj)]TJ /F7 11.955 Tf 11.96 0 Td[(vjj j2#(2{18) InChapter3,wealsousetherationalquadraticcovariancefunction[ Rasmussen&Williams ( 2006 )]: k(u;v)= 1+ku)]TJ /F7 11.955 Tf 11.96 0 Td[(vk2 2l2!)]TJ /F6 7.97 Tf 6.59 0 Td[((2{19) with=f;lg. Inthegeostatisticliterature,threeterminologiesareused,dependingonthelinearpartconsidered: simpleKriging(SK):thelinearpartreducestoaknownconstant1 ordinaryKriging(OK):theconstant1isunknown universalKriging(UK)isthegeneralcase. Theparametersareusuallyunknownandmustbeestimatedbasedontheobservations,usingmaximumlikelihood,cross-validationorvariogramtechniquesforinstance[see Rasmussen&Williams ( 2006 ), Stein ( 1999 )or Cressie ( 1993 )].However,intheKrigingmodeltheyareconsideredasknown.Toaccountforadditionalvariabilityduetotheparameterestimation,onemayuseBayesianKrigingmodels[see Martin&Simpson ( 2004 )and Oakley&O'Hagan ( 2004 )],whichwillnotbedetailedhere. 26

PAGE 27

2.1.3.1Krigingwithnoise-freeobservations First,weconsiderthemostclassicalframework,wherethefunctionisobservedwithoutnoise: yobs;i=y(xi)(2{20) Undersuchhypothesis,thebestlinearunbiasedpredictor(BLUP)fory,knowingtheobservationsYobs,isgivenbythefollowingequation: mK(x)=E[Y(x)jY(X)=Yobs]=f(x)T^+c(x)TC)]TJ /F5 7.97 Tf 6.59 0 Td[(1Yobs)]TJ /F7 11.955 Tf 11.96 0 Td[(F^(2{21) where: f(x)=f1(x):::fp(x)Tisp1vectorofbases, ^=^1:::^pTisp1vectorofestimatesof, c(x)=cov(x;x1):::cov(x;xn)Tisn1vectorofcovariance, C=[cov(xi;xj)]1i;jnisnncovariancematrix,and F=f(x1):::f(xn)Tispnmatrixofbases. ^isthevectorofgeneralizedleastsquareestimatesof: ^=)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(FTC)]TJ /F5 7.97 Tf 6.58 0 Td[(1F)]TJ /F5 7.97 Tf 6.58 0 Td[(1FTC)]TJ /F5 7.97 Tf 6.59 0 Td[(1Yobs(2{22) Inaddition,theKrigingmodelprovidesanestimateoftheaccuracyofthemeanpredictor,theKrigingpredictionvariance: s2K(x)=k(x;x))]TJ /F7 11.955 Tf 9.3 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.58 0 Td[(1c(x)+)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(f(x)T)]TJ /F7 11.955 Tf 11.95 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.59 0 Td[(1F)]TJ /F7 11.955 Tf 12.95 -9.69 Td[(FTC)]TJ /F5 7.97 Tf 6.58 0 Td[(1F)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(f(x)T)]TJ /F7 11.955 Tf 11.96 0 Td[(c(x)TC)]TJ /F5 7.97 Tf 6.58 0 Td[(1FT(2{23) NotethattheKrigingvariancedoesnotdependontheobservationsYobs,butonlyonthedesignofexperiments.Derivationdetailscanbefoundin Matheron ( 1969 ), Cressie ( 1993 ),or Rasmussen&Williams ( 2006 ).WedenotebyM(x)theGaussianprocessconditionalon 27

PAGE 28

theobservationsYobs: (M(x))x2D=(Y(x)jY(X)=Yobs)x2D=(Y(x)jobs)x2D(2{24) TheKrigingmodelprovidesthedistributionofMatapredictionpointx: M(x)N)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(mK(x);s2K(x)(2{25) Figure 2-1 showsaKrigingmodelwitharst-ordertrend(=1,simpleKriging)andveequally-spacedobservationsalongwiththe95%condenceintervals,whicharecalculatedfrommK1:96sK.Onthisexample,thecondenceintervalcontainstheactualresponse. Figure2-1. ExampleofSimpleKrigingmodel.The95%condenceintervals(CI)areequaltomK1:96sK.TheDoEconsistsofvepointsequallyspacedin[0,1]. TheKrigingmeanmKinterpolatesthefunctiony(x)atthedesignofexperimentpoints: mK(xi)=y(xi);1in(2{26) TheKrigingvarianceisnullattheobservationpointsxi,andgreaterthanzeroelsewhere: s2K(xi)=0;1inands2K(x)0;x6=xi(2{27) 28

PAGE 29

Besides,theKrigingvariancefunctionincreaseswiththedistanceofxtotheobservations. 2.1.3.2Krigingwithnuggeteect Whennoisyobservationsareconsidered,adiagonalmatrixmustbeaddedtothecovariancematrix: C=C+(2{28) with:=diag([var("1);var("2);:::;var("n)])thevariancesoftheobservations. EquationsformKands2KarethesameasinEqs. 2{21 and 2{23 butusingCinsteadofC.Fortheoreticaldetailsandbibliography,seeforinstance Ginsbourger ( 2009 )(Chap.7).ThemaindierencewiththeclassicalKrigingisthatbestpredictorisnotaninterpolatoranymore;also,theKrigingvarianceisnon-nullattheobservationpoints.Figure 2-2 showsaKrigingmodelbasedonnoisyobservations.Eachobservationhasadierentnoisevariance. Figure2-2. ExampleofSimpleKrigingmodelwithnoisyobservations.Thebarsrepresenttwotimesthestandarddeviationofthenoise.TheKrigingmeandoesnotinterpolatethedataandtheKrigingvarianceisnon-nullattheobservationpoints. 29

PAGE 30

2.2DesignOfExperimentStrategies ChoosingthesetofexperimentsXplaysacriticalroleintheaccuracyofthemetamodelandthesubsequentuseofthemetamodelforprediction,learningoroptimization.Inthissection,wedetailthreefamiliesofdesignofexperiments:classicalandspace-llingdesigns,model-oriented(oroptimal)designs,andadaptivedesigns. 2.2.1ClassicalAndSpace-FillingDesigns TherstfamilyofDoEconsistsofdesignsbasedongeometricconsiderations.Inafull-factorial(FF)design,thevariablesarediscretizedintoanitenumberoflevels,andthedesignconsistsofallthepossiblecombinationsofthediscretevariables.Two-levelFFdesigns,wheretheinputvariablesaretakenonlyattheirminimumandmaximumvalues,aretypicallyusedforscreeninginordertoidentifythemostsignicantvariablesandremovetheothers.Althoughextensivelyusedhistorically,suchtypeofDoEssuerfromseveraldrawbacksamongthefollowing: itrequiresalargenumberofobservationsinhighdimensions(aFFdesignwithqlevelsinddimensionsismadeofqdobservations),makingthemimpracticalforcomputationallyexpensiveproblems theydonotensurespace-llinginhighdimensions thenumberofobservationpointscollapseswhenprojectingonthesubspaces(whensomevariablesareremovedforinstance). Figure2-3. Examplesoffull-factorialandcentralcompositedesigns. ThereexistsclassicalalternativestoFFdesigns,suchascentral-compositedesigns(thatconsistof2dvertices,2daxialpointsandprepetitionsofcentralpoint),orfractional 30

PAGE 31

designs(thataresubsetsofFFdesigns).Seeforinstance Myers&Montgomery ( 1995 )fordetails.Figure 2-3 showsathree-levelFFdesignandacentralcompositedesignforatwo-dimensionaldomain. ApopularalternativetothegeometricaldesignsisLatinHypercubesampling(LHS)[ McKayetal. ( 2000 )].LHSisarandomDoEthatinsuresuniformityofthemarginaldistributionsoftheinputvariables. TogenerateaDoEofnpoints,eachvariablerangeisdividedintonequalintervals,forinstancefortherange[0;1]:[0;1 n];[1 n;2 n];:::;[n)]TJ /F5 7.97 Tf 6.59 0 Td[(1 n;1].Then,theDoEisconstructedbypickingnpointssothatalltheintervalsforeachvariableisrepresentedonce.Figure 2-4 showsanine-pointLHSdesignforatwo-dimensionaldomain.Inad-dimensionalspace,thetotalnumberofcombinationsisnd)]TJ /F5 7.97 Tf 6.58 0 Td[(1,whichincreasesveryrapidlywithbothsamplesizeanddimension.Thevariablevaluescanbechosendeterministicallyasthecentreoftheintervalforinstance,orrandomlyinsidetheinterval. LHScanalsobeoptimizedtoensurebetterspace-lling.Space-llingcriteriainclude: maximumminimumdistancebetweensamplingpoints(maximindesigns), maximum-entropydesigns[ Shewry&Wynn ( 1987 )], minimumdiscrepancymeasures,etc. Figure2-4. Anine-pointLHSdesign.Theprojectionsonthemarginsarenonredudantandequally-spacedifthedesignpointsarecenteredineachcell. 31

PAGE 32

Finally,low-discrepancysequencesarealsooftenusedforspace-llingstrategies,suchasSobol,Halton,Harmmersley,NiederreitterorFauresequences.See Niederreiter ( 1992 )or Sobol ( 1976 )fordetails. 2.2.2Model-OrientedDesigns Theprevioussamplingstrategieswerebuiltindependentlyofthemetamodeltobettedtotheobservations.Alternatively,whenthechoiceofthemetamodelismadeapriori,itispossibletochoosetheobservationpointsinordertomaximizethequalityofstatisticalinference.Thetheoryofoptimaldesignshasoriginallybeendevelopedintheframeworkoflinearregression,andwasextendedlatertoothermodelssuchasKrigingsincethe80's. LetbeafunctionalofinteresttominimizethatdependsonthedesignofexperimentsX.AdesignXiscalled-optimalifitachieves: X=argmin[(X)](2{29) A-andD-optimalityaimatminimizingtheuncertaintyintheparametersofthemetamodelwhenuncertaintyisduetonoisyobservations.Intheframeworkoflinearregression,D-optimaldesignsminimizethevolumeofthecondenceellipsoidofthecoecients,whileA-optimaldesignsminimizeitsperimeter(orsurfaceford>2).Formally,theA-andD-optimalitycriteriaare,respectively,thetraceanddeterminantofFisher'sinformationmatrix[ Khuri&Cornell ( 1996 )]: A(X)=trace(M(X)) (2{30) D(X)=det(M(X)) (2{31) Inlinearregression,thesecriteriaareparticularlyrelevantsinceminimizingtheuncertaintyinthecoecientsalsominimizestheuncertaintyintheprediction(accordingtotheD-Gequivalencetheorem,whichisdevelopedfurtherinChapter6). 32

PAGE 33

ForKriging,uncertaintiesincovarianceparametersandpredictionarenotsimplyrelated.Anaturalalternativeistotakeadvantageofthepredictionvarianceassociatedwiththemetamodel.ThepredictionvarianceallowsustobuildmeasuresthatreecttheoverallaccuracyoftheKriging.Twodierentcriteriaareavailable:theintegratedmeansquareerror(IMSE)andmaximummeansquareerror(MMSE)[ Santneretal. ( 2003 ), Sacksetal. ( 1989a )]: IMSE=ZDMSE(x)d(x) (2{32) MMSE=maxx2D[MSE(x)] (2{33) (x)isanintegrationmeasure(usuallyuniform)and MSE(x)=E(y(x))]TJ /F4 11.955 Tf 11.96 0 Td[(M(x))2obs](2{34) WhenthefunctiontoapproximateisarealizationofagaussianprocesswithcovariancestructureandparametersequaltotheoneschosenforKriging,theMSEcoincideswiththepredictionvariances2K.NotethattheabovecriteriaareoftencalledI-criterionandG-criterion,respectively,intheregressionframework.TheIMSEisameasureoftheaverageaccuracyofthemetamodel,whiletheMMSEmeasurestheriskoflargeerrorinprediction.Inpractice,theIMSEcriterionessentiallyreectsthespatialdistributionoftheobservations.Fordimensionshigherthantwo,theMMSEisnotveryrelevantsincetheregionsofmaximumuncertaintyarealwayssituatedontheboundaries,andMMSE-optimaldesignsconsistofobservationstakenontheedgesofthedomainonly,whichisnotecientforKriging. Optimaldesignsaremodel-dependent,inthesensethattheoptimalitycriterionisdeterminedbythechoiceofthemetamodel.Inregression,A-andD-criteriadependonthechoiceofthebasisfunctions,whileinKrigingthepredictionvariancedependsonthelineartrend,thecovariancestructure,andparametervalues.Afundamentalresult,however,isthatassumingaparticularmodelstructure(andcovarianceparametersfor 33

PAGE 34

Kriging),noneofthecriteriadependsontheresponsevaluesatthedesignpoints.TheconsequenceisthattheDoEscanbedesignedoinebeforegeneratingtheobservations.However,inpracticethisstatementhastobeenmoderated,sincetheKrigingcovarianceparametersaremostofthetimeestimatedfromtheobservations. 2.2.3AdaptiveDesigns ThepreviousDoEstrategieschooseallthepointsofthedesignbeforecomputinganyobservation.ItisalsopossibletobuildtheDoEsequentially,bychoosinganewpointasafunctionoftheotherpointsandtheircorrespondingresponsevalues.Suchapproachhasreceivedconsiderableattentionfromtheengineeringandmathematicalstatisticcommunities,foritsadvantagesofexibilityandadaptabilityoverothermethods[ Linetal. ( 2008 ), Scheidt ( 2006 ), Turneretal. ( 2003 )]. Typically,thenewpointachievesamaximumonsomecriterion.Forinstance,asequentialDoEcanbebuiltbymakingateachstepanewobservationwherethepredictionvarianceismaximal.ThealgorithmicprocedureisdetailedinTable 2-1 Table2-1. SequentialDoEminimizingthemaximumpredictionvarianceateachstep. X=fx1;:::;xngfori=1tokxnew=argmaxx2Ds2K(x)X=fX;xnewgend Sacksetal. ( 1989b )usesthisstrategyasaheuristictobuildIMSE-optimaldesignsforKriging.Theoretically,thissequentialstrategyislessecientthatthedirectminimizationoftheIMSEcriterion.Theadvantageofsequentialstrategyhereistwofold.Firstly,itiscomputational,sinceittransformsanoptimizationproblemofdimensionnd(fortheIMSEminimization)intokoptimizationsofdimensiond.Secondly,itallowsustoreevaluatethecovarianceparametersaftereachobservation.Inthesamefashion, Williamsetal. ( 2000 ), Currinetal. ( 1991 ), Santneretal. ( 2003 )useaBayesianapproachtoderivesequentialIMSEdesigns. Osio&Amon ( 1996 )proposedamultistageapproachtoenhance 34

PAGE 35

rstspace-llinginordertoaccuratelyestimatetheKrigingcovarianceparametersandthenrenetheDoEbyreducingthemodeluncertainty.Somereviewsofadaptivesamplinginengineeringdesigncanbefoundin Jinetal. ( 2002 ). Ingeneral,aparticularadvantageofsequentialstrategiesoverotherDoEsisthattheycanintegratetheinformationgivenbytherstkobservationvaluestochoosethe(k+1)thtrainingpoint,forinstancebyreevaluatingtheKrigingcovarianceparameters.Itisalsopossibletodeneresponse-dependentcriteria,withnaturallyleadstosurrogate-basedoptimization.OneofthemostfamousadaptivestrategyistheEGOalgorithm( Jonesetal. ( 1998 )),usedtoderivesequentialdesignsfortheoptimizationofdeterministicsimulationmodels.Ateachstep,thenewobservationischosentomaximizetheexpectedimprovement,afunctionalthatrepresentsacompromisebetweenexplorationofunknownregionsandlocalsearch: EI(x)=E(max[0;min(Yobs))]TJ /F4 11.955 Tf 11.95 0 Td[(M(x)])(2{35) whereM(x)istheKrigingmodelasdescribedinEq. 2{25 Kleijnen&VanBeers ( 2004 )proposedanapplication-drivenadaptivestrategyusingcriteriabasedonresponsevalues. Tu&Barton ( 1997 )usedamodiedD-optimalstrategyforboundary-focusedpolynomialregression.InChapter5,wedetailanoriginaladaptivedesignstrategyfortargetregionapproximation. 35

PAGE 36

CHAPTER3CONSERVATIVEPREDICTIONSUSINGSURROGATEMODELING 3.1Motivation Inanalyzingengineeringsystems,multiplesourcesoferror-suchasmodelingerrororinsucientdata-preventusfromtakingtheanalysisresultsatfacevalue.Conservativepredictionisasimplewaytoaccountforuncertaintiesanderrorsinasystemanalysis,byusingcalculationsorapproximationsthattendtosafelyestimatetheresponseofasystem.Traditionally,safetyfactorshavebeenextensivelyusedforthatpurpose,eventhoughtheireectivenessisquestionable[ Elishako ( 2004 ), Acaretal. ( 2007 )]. Whensurrogatemodelsareusedforpredictingcriticalquantities,verylittleisknowninpracticetoprovideconservativeestimates,andhowconservativestrategiesimpactthedesignprocess.Mostsurrogatesaredesignedsothatthereisa50%chancethatthepredictionwillbehigherthantherealvalue.Theobjectiveofthisworkistoproposealternativestomodifythetraditionalsurrogatessothatthispercentageispushedtotheconservativesidewiththeleastimpactonaccuracy. Sinceconservativesurrogatestendtooverestimatetheactualresponse,thereisatrade-obetweenaccuracyandconservativeness.Thechoiceofsuchtrade-odeterminesthebalancebetweentheriskofoverdesignandtheriskofweakdesign.Thedesignofconservativesurrogatescanbeconsideredasabi-objectiveoptimizationproblem,andresultsarepresentedintheformofParetofronts:accuracyversusconservativeness. Themostclassicalconservativestrategyistobiasthepredictionresponsebyamultiplicativeoradditiveconstant.Suchapproachesarecalledempiricalbecausethechoiceoftheconstantissomehowarbitraryandbasedonpreviousknowledgeoftheengineeringproblemconsidered.However,itisdiculttopredicthowecientitsapplicationistosurrogates,andhowitispossibletodesignthosequantities.Alternatively,itispossibletousethestatisticalknowledgefromthesurrogatetting(predictionvariance)tobuildone-sidedcondenceintervalsontheprediction. 36

PAGE 37

Forthisstudy,weconsidertwotypesofsurrogatemodels:polynomialresponsesurfaces(PRS)andKriging.Weproposethreealternativestoprovideconservativeestimates:usingcross-validationtodeneconstantmargins,usingcondenceintervalsgivenbythesurrogatemodel,andusingthebootstrapmethod.Themethodsdierinthesensethatoneassumesaparticulardistributionoftheerror,whiletheothers(cross-validationandbootstrap)doesnot.Dierentmetricsaredenedtoanalyzethetrade-obetweenaccuracyandconservativeness.Themethodsareappliedtotwotestproblems:oneanalyticalandonebasedonFiniteElementAnalysis. 3.2DesignOfConservativePredictors 3.2.1DenitionOfConservativePredictors Withoutanylossofgenerality,weassumeherethataconservativeestimatorisanestimatorthatdoesnotunderestimatetheactualvalue.Hence,aconservativeestimatorscanbeobtainedbysimplyaddingapositivesafetymargintotheunbiasedestimator: ^ySM(x)=^y(x)+Sm(3{1) Alternatively,itispossibletodeneaconservativeestimatorusingsafetyfactors,whicharemorecommonlyusedinengineering: ^ySF(x)=^y(x)Sf(3{2) Inthecontextofsurrogatemodeling,safetymarginsaremoreconvenient.Indeed,whenusingasafetyfactor,thelevelofsafetydependsontheresponsevalue,whilemostsurrogatesmodelassumethattheerrorisindependentofthemeanoftheresponse.Hence,inthisworkweuseonlysafetymarginstodeneaconservativeestimator. Thevalueofthemargindirectlyimpactsthelevelofsafetyandtheerrorintheestimation.Hence,thereisagreatincentivetondthemarginthatensureatargetedlevelofsafetywithleastimpactontheaccuracyoftheestimates.Thesafetymargincanbeconstant,ordependonthelocation,hencewrittenSm(x)(pointwisemargin).Inthe 37

PAGE 38

followingsections,weprovideseveralwaystodesignthemargin,usingparametricandnon-parametricmethods,forpointwiseandconstantmargins. 3.2.2MetricsForConservativenessAndAccuracy Asdiscussedinintroduction,conservativeestimatesarebiased,andahighlevelofconservativenesscanonlybeachievedatapriceinaccuracy.Thus,thequalityofamethodcanonlybemeasuredasatrade-obetweenconservativenessandaccuracy.Inordertoassessaglobalperformanceofthemethods,weproposetodeneaccuracyandconservativenessindexes. Themostwidelyusedmeasuretochecktheaccuracyofasurrogateistherootmeansquareerror(eRMS),denedas: eRMS=0@ZD(^y(x))]TJ /F4 11.955 Tf 11.95 0 Td[(y(x))2dx1A1 2(3{3) TheeRMScanbecomputedbyMonte-Carlointegrationatalargenumberofptesttestpoints: eRMSvuut 1 ptestptestXi=1e2i(3{4) whereei=(^yi)]TJ /F4 11.955 Tf 12.62 0 Td[(yi),^yiandyibeingthevaluesoftheconservativepredictionandactualsimulationatthei-thtestpoint,respectively. Wealsodenetherelativelossinaccuracy,inordertomeasuresimplybyhowmuchtheaccuracyisdegradedwhenhighlevelsofconservativenessaretargeted: l=eRMS eRMSjref)]TJ /F1 11.955 Tf 11.96 0 Td[(1(3{5) whereeRMSistakenatagiventargetconservativeness;andeRMSjrefistheeRMSvalueofreference.Inmostofthestudy,whichwetakeequaltothevalueofeRMSwhenthetargetconservativenessis50%. 38

PAGE 39

Therearedierentmeasuresoftheconservativenessofanapproximation.Hereweusethepercentageofconservativeerrors(i.e.positive): %c=100ZDI[^y(x))]TJ /F4 11.955 Tf 11.96 0 Td[(y(x)]dx(3{6) whereI(e)istheindicatorfunction,whichequals1ife>0and0otherwise.%ccanbeestimatedbyMonte-Carlointegration: %c100 ptestptestXi=1I(ei)(3{7) Ideally,formostsurrogatemodels,%c=50%whentheapproximationisunbiased. Thepercentageofconservativeerrorsprovidestheprobabilitytobeconservative.However,itfailstoinformbyhowmuchitisunconservativewhenpredictionsareunconservative.Thus,analternatemeasureofconservativenessisproposed;thatis,themaximumunconservativeerror(largestnegativeerror): MaxUE=max(max()]TJ /F4 11.955 Tf 9.3 0 Td[(ei);0)(3{8) Thisindexcanalsobenormalizedbytheindexoftheunbiasedestimator: MaxUE%=MaxUE MaxUEjref)]TJ /F1 11.955 Tf 11.95 0 Td[(1(3{9) whereMaxUEjrefisthevalueofreference,whichwetakeequaltothevaluefortheunbiasedestimator. AvalueofMaxUE%of50%meansthatthemaximumunconservativeerrorisreducedby50%comparedtotheBLUEestimator. Alternativemeasuresofconservativenesscanbedened,includingthemeanorthemedianoftheunconservativeerrors.However,themaximumerrordecreasesmonotonicallywhentargetconservativenessisincreased,whilemeanandmediancanincreasewhenweincreaseconservativeness,forinstancewhenwehaveinitiallyverysmallandverylargeerrors. 39

PAGE 40

3.2.3ConstantSafetyMarginUsingCross-ValidationTechniques Here,weconsiderthedesignofconservativesurrogates,whenthesamesafetymarginisappliedeverywhereonthedesigndomain.Inthefollowing,wecallsuchestimatorsCSMestimators(forConstantSafetyMargin),anddenoteit^yCSM. Intermsofthecumulativedistributionfunction(CDF)oftheerrors,Fe,thesafetymarginSmforagivenconservativeness,%c,isgivenas: Sm=F)]TJ /F5 7.97 Tf 6.58 0 Td[(1e%c 100(3{10) Theobjectiveistodesignthesafetymarginsuchthattheaboveequationisensured.Theactualerrordistributionis,inpractice,unknown.Weproposeheretoestimateitempiricallyusingcross-validationtechniquesinordertochoosethemargin. Cross-validation(XV)isaprocessofestimatingerrorsbyconstructingthesurrogatewithoutsomeofthepointsandcalculatingtheerrorsattheseleftoutpoints.Theprocessisrepeatedwithdierentsetsofleft-outpointsinordertogetstatisticallysignicantestimatesoferrors.Theprocessproceedsbydividingthesetofndatapointsintosubsets.Thesurrogateisttedtoallsubsetsexceptone,anderrorischeckedinthesubsetthatwasleftout.Thisprocessisrepeatedforallsubsetstoproduceavectorofcross-validationerrors,eXV.Usually,onlyonepointisremovedatatime(leave-one-outcross-validation),sothesizeofeXVisequalton. TheempiricalCDFFXV,denedbythenvaluesofeXV,areanapproximationofthetruedistributionFe.Now,inordertodesignthemargin,wereplaceFeinEq. 3{10 byFXV: Sm=F)]TJ /F5 7.97 Tf 6.59 0 Td[(1XV%c 100(3{11) Forinstance,whenn=100and75%conservativenessisdesired,thesafetymarginischosenequaltothe25thhighestcross-validationerror. 40

PAGE 41

3.2.4PointwiseSafetyMarginBasedOnErrorDistribution Conservativeestimatescanalsobeobtainedassumingtheerrordistributionisknownasprovidedbythesurrogateanalysis.Classicalregressionprovidesacondenceintervalforthepredictedmodel.Aunilateralcondenceintervaloflevelfortheresponsey(x)isgivenby[see Khuri&Cornell ( 1996 )]: CI=i;fT(x)^+t)]TJ /F5 7.97 Tf 6.59 0 Td[(1n)]TJ /F6 7.97 Tf 6.59 0 Td[(p)]TJ /F5 7.97 Tf 6.58 0 Td[(1(1)]TJ /F4 11.955 Tf 11.96 0 Td[()si(3{12) wheretn)]TJ /F6 7.97 Tf 6.59 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1istheStudent'sdistributionwithn)]TJ /F4 11.955 Tf 11.95 0 Td[(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1degrees-of-freedom,and:s=^p 1+fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1(X)f(x)^2=1 n)]TJ /F4 11.955 Tf 11.96 0 Td[(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1nXi=1(yi)]TJ /F1 11.955 Tf 12.74 0 Td[(^yi)2 Wedenetheconservativeestimatoroflevel(1)]TJ /F4 11.955 Tf 12.96 0 Td[()astheupperboundofthecondenceinterval: ^yED(x)=fT(xnew)^+t)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F6 7.97 Tf 6.58 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1(1)]TJ /F4 11.955 Tf 11.95 0 Td[()s(3{13) Thisconservativeestimatehastheformofamarginaddedtotheunbiasedprediction;themargindependsonthepredictionlocationandisequalto: Sm(x)=t)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F6 7.97 Tf 6.58 0 Td[(p)]TJ /F5 7.97 Tf 6.59 0 Td[(1(1)]TJ /F4 11.955 Tf 11.95 0 Td[()^p 1+fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1(X)f(x)(3{14) Krigingalsoprovidescondenceintervalsfortheprediction,assumingthatthepredictionerrorisnormallydistributedwithmeanmKandvariances2K.Then,wecandenetheconservativeestimatoroflevel(1)]TJ /F4 11.955 Tf 11.96 0 Td[()as: ^yED(x)=)]TJ /F5 7.97 Tf 6.59 0 Td[(1mK(x);m2K(x)(1)]TJ /F4 11.955 Tf 11.96 0 Td[()(3{15) where)]TJ /F5 7.97 Tf 6.59 0 Td[(1mK(x);s2K(x)istheinversenormalcumulativedistributionfunctionofmeanmK(x)andvariances2K(x). 41

PAGE 42

Inthefollowing,theseestimatorsarereferredasED(errordistribution)estimate.Theyarecalledparametricestimatessincetheyassumeaparticularformofthedistributionoftheerror.Notethatcondenceintervalsgivenbykriginghaveadierentinterpretationthantheonesgivenbyregression.Indeed,inregressiontheinterpretationoftheCIisthatthereisaprobabilitythatanobservationtakenatxfallsintotheinterval.Inkriging,itmeansthat(1)]TJ /F4 11.955 Tf 12.07 0 Td[()%oftherealizationsofthegaussianprocessesconditionaltotheobservationswillstaywithintheintervalbounds. 3.2.5PointwiseSafetyMarginUsingBootstrap Errordistributionestimatorsrelyonthefactthatthehypothesisbehindthesurrogatemodelaresatisedinpractice.Inparticular,normalityoftheerrordistributionisalwaysassumed.Itisobviousthat,whensuchhypothesisisviolated,thesafetymargindesignedwitherrordistributioncanbeveryinaccurate.Inthissection,weproposetheuseofbootstraptoobtaincondenceintervalswithoutassumptionsontheerrordistribution. 3.2.5.1TheBootstrapprinciple Thebootstrapmethodcanprovideanecientwayofestimatingthedistributionofastatisticalparameterbasedonsamplesfu1;:::;ungoftherandomvariableUusingthere-samplingtechnique[ Efron ( 1982 ), Chernick ( 1999 )].Theideaistocreatemanysetsofbootstrapsamplesbyre-samplingwithreplacementfromtheoriginaldata. Thismethodonlyrequirestheinitialsetofsamples.Figure 3-1 illustratestheprocedureofthebootstrapmethod.Thesizeoftheinitialsamplesisnandthenumberofbootstrapre-samplingsisp.Eachre-samplingcanbeperformedbysamplingwithreplacementndataoutoftheninitialsamples(hence,thebootstrapsamplescontainrepeatedvaluesfromtheinitialsamplesandomitsomeoftheinitialvalues).Sincethere-samplingprocessdrawssamplesfromtheexistingsetofsamples,itdoesnotrequireadditionalsimulations.Theparameter(forinstance,themeanorstandarddeviationofU),isestimatedforeachbootstrapsamples.Sincethere-samplingprocedureallowsselectingdatawithreplacement,thestatisticalpropertiesofthere-sampleddataare 42

PAGE 43

dierentfromthatoftheoriginaldata.Then,thesetofpbootstrapestimatesbootdenesanempiricaldistributionof.Thisapproachallowsustoestimatethedistributionofanystatisticalparameterwithoutrequiringadditionaldata,andwithoutanyassumptiononthedistributionoftheparameter. Figure3-1. Schematicrepresentationofbootstrapping.Bootstrapdistributionof(histogramorCDFofthepestimates)isobtainedbymultiplere-sampling(ptimes)fromasinglesetofdata. 3.2.5.2Generatingcondenceintervalsforregression Inthissection,wepresenttheapplicationofbootstraptoregression,asproposedby Stine ( 1985 ).Calculationsandmethodologytoobtaincondenceintervalscomefromthispaper.Themainideaisthatresamplingwithreplacementisconductedontheregressionresiduals. First,theregressionmodelisestimatedusingthestandardequations.Then,theobservationsareperturbedbyresamplingtheresiduals: Y=X^+"(3{16)"isthebootstrapvectorofresiduals,obtainedbysamplingwithreplacementfromtheempiricaldistributionoftheresidualvector". 43

PAGE 44

Notethatthebootstrapmodelassumesthatthemeanofthetruefunctionisequaltothebestpredictoroftheregressionmodel,thatis,thetruefunctionisassumedtobeoftheform: y(x)=fT(x)^+"(3{17) where"followstheempiricaldistributionof". Then,themodelisbuiltbasedonthebootstrapobservations: =)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(fT(X)fT(X))]TJ /F5 7.97 Tf 6.59 0 Td[(1fT(X)Y(3{18) Usingbootstrap,weaccountforthevariabilityin^throughvariousbootstrapestimates1;2;3;:::,andalsoforthenoise"atthepredictedpoint. Thestructureofthenoiseleadstovaluablecomputationalshortcut.Indeed,theerrorEforthebootstrapmodelconsistsoftwoadditivecomponents: E="+b(3{19) with"takenfromtheinitialvectorofresidualsandbistheerrorduetouncertaintyin^: b(x)=fT(x))]TJ /F7 11.955 Tf 11.95 0 Td[(fT(x)^=fT(x)M)]TJ /F5 7.97 Tf 6.59 0 Td[(1fT(X)" (3{20) Sinceband"areindependent,thedistributionofEistheirconvolution.Then,thedistributionFeoftheerrorcanbeapproximatedby: Fe(u)=1 NNXj=1Fn)]TJ /F4 11.955 Tf 5.47 -9.69 Td[(u)]TJ /F4 11.955 Tf 11.96 0 Td[(bj(x)(3{21) with: Fntheempiricaldistributionofthe(initial)residuals" uascalar 44

PAGE 45

bj(x)theerrorofthejthbootstrapmodelatx Nthenumberofbootstrapreplicates. Fe(u)istheprobability,computedbybootstrap,thattheerroratxexceedsu. Then,thepointwisesafetymarginp1)]TJ /F6 7.97 Tf 6.59 0 Td[(neededtoachieve(1)]TJ /F4 11.955 Tf 11.7 0 Td[()%conservativenessisdenedastheinverseofFetakenat(1)]TJ /F4 11.955 Tf 11.95 0 Td[()%: p1)]TJ /F6 7.97 Tf 6.59 0 Td[(=(1)]TJ /F4 11.955 Tf 11.95 0 Td[()(3{22) However,F)]TJ /F5 7.97 Tf 10.82 0 Td[(1ecannotbeknownanalytically.Inpractice,wechooseanitenumberofvaluesofuforwhichwecomputeFe(u)foragivenx;thenweobtaintheinverseofthedistribution(asshowninFigure 3-2 )byinterpolation. Figure3-2. Percentileoftheerrordistributioninterpolatedfrom100valuesofu. Finally,thebootstrapconservativepredictorisdenedas: ^yBS=xpred^+p1)]TJ /F6 7.97 Tf 6.59 0 Td[((3{23) Inthefollowing,theseestimatorsarecalledBS(bootstrap)estimatorsanddenotedby^yBS. 45

PAGE 46

3.2.6AlternativeMethods Inthecontextofconservativepredictions,weproposedtwootheralternatives.Therstiscalledbiasedtting:itconsistsofaddingconstraintstotheleastsquareminimizationprobleminordertobiasthesurrogatetobeononesideoftheobservations.ThesecondisIndicatorKriging(IK),whichisanon-parametricmethodthatestimatesprobabilitiesofexceedingaparticularthreshold.However,thesemethodswerefoundtobelessecientthantheothermethodstoproduceconservativepredictions.Hence,wedonotpresentithere.TheycanbefoundinAppendix A.1 and A.2 3.3CaseStudies 3.3.1TestProblems 3.3.1.1TheBranin-Hoofunction Thersttestfunctionisadeterministictwo-dimensionalfunction,whichisoftenusedtotestoptimizationmethods[ Dixon&Szego ( 1978 )].Theinputvariabledomainsare:x2[)]TJ /F1 11.955 Tf 9.29 0 Td[(510]andy2[015].TheexpressionoftheBranin-Hoofunctionisgivenas: f(x;y)=y)]TJ /F1 11.955 Tf 13.15 8.09 Td[(5:1x2 42+5x )]TJ /F1 11.955 Tf 11.96 0 Td[(62+101)]TJ /F1 11.955 Tf 16.68 8.09 Td[(1 8cos(x)+10(3{24) Therangeofthefunctionisbetweenzeroand300.Largevaluesarelocatedontheboundsofthedomain. Figure3-3. Branin-Hoofunction. 46

PAGE 47

3.3.1.2Thetorquearmmodel Thissecondexamplewasoriginallypresentedby Bennett&Botkin ( 1986 ).Itconsistsoftheshapedesignofaparticularpiecefromautomotiveindustrycalledatorquearm.Thetoquearmmodel,shownin 3-4 ,isunderahorizontalandverticalload,Fx=)]TJ /F1 11.955 Tf 9.29 0 Td[(2789NandFy=5066N,respectively,transmittedfromashaftattherighthole,whiletheleftholeisxed.ThetorquearmconsistsofamaterialwithYoung'smodulus,E=206:8GPaandPoisson'sratio,=0:29. Figure3-4. InitialdesignoftheTorquearm. Theobjectiveoftheanalysisistominimizetheweightwithaconstraintonthemaximumstress.Sevendesignvariablesaredenedtomodifytheinitialshape.Figure 3-5 andTable 3-1 showthedesignvariablesandtheirlowerandupperbounds,respectively.Thelowerandupperboundsareselectedsuchthatthetopologyofthedesignisunchanged. Table3-1. Rangeofthedesignvariables(cm). DesignvariableLowerboundUpperbound 1-2.03.52-0.22.53-2.06.04-0.20.55-0.12.06-1.52.07-0.12.0 47

PAGE 48

Figure3-5. Designvariablesusedtomodifytheshape. Thestressanalysisisperformedusingacommercialniteelementanalysis(FEA)software,ANSYS.Convergenceanalysisisperformedtondanappropriatemeshsize.Smallermeshsizeisusedfortheregionthathaspossibilityofhavingmaximumstress. Figure 3-6 showsatypicalstresscontourplotfromFEAresults.ThecontourlinesrepresentthevonMisesstress.Fortheinitialdesign,thevonMisesstressrangeisbetween10kPaand196MPa.Themaximumstressoccursattheendofthecutout.However,thelocationofmaximumstressmaychangefordierentdesigns.Infact,themaximumstressisdiscontinuousfunctionofdesignvariables.However,weassumethatitisacontinuousfunction,andtheoptimumdesignmustbeinspectedcarefully. Figure3-6. VonMisesstresscontourplotattheinitialdesign(Colorbarindicatesstressvaluesin104Pa). 48

PAGE 49

TheFEAiscomputationallyexpensive;thus,asurrogatemodelisusedtoapproximatethemaximumstressonthedesigndomain.Here,wedonotfocusontheoptimizationbutonthequalityofthesurrogateprediction. 3.3.2NumericalProcedure 3.3.2.1Graphsofperformance Wedisposeofseveraltechniquestoincreasetheconservativenessofsurrogatemodels;ourobjectiveistodetermine,throughtheexamples,ifsomeofthemarebetterthanothers.Moreover,itispossiblethatsometechniquesaremoreecientforlowlevelsofconservativeness,andotherforhighlevelsofconservativeness.Foreachmethod,thelevelofbiascanbemodiedbychangingthelevelofthecondenceinterval(1)]TJ /F4 11.955 Tf 12.12 0 Td[()(from50%to100%). Then,theindicesofperformancedenedinSection 3.2.2 canbecalculatedforeachmethodateachlevel.Inordertoanalyzeandcomparethedierentmethods,weusethefollowinggraphs: %cagainstl:therstindicatorofperformanceistheevolutionofconservativenesslevel(%c)againstthelossinaccuracy(l).ItcanbeconsideredasaParetofront,andcanbeusedtodesigntrade-osbetweenthesequantities.Also,itallowsustocomparethedierentmethodsbylookingatpartialorglobaldominations. %cagainstMaxUE%:thiscurveallowstochecktheconsistencyofthetwoconservativenessindiceslandMaxUE%when%cincreases. Targetconservativenessagainst%c:forallestimatesexceptbiasedttingmodels,thelevelofbiasisdesignedusingatargetlevelof%c.Then,itisofcrucialinteresttomeasuretheadequacytotheactual%ctotheexpectedon.Toanalyzethisperformance,wedrawtheQQ-plotofthetargetconservativeness(1)]TJ /F4 11.955 Tf 12.62 0 Td[()vs.theactualconservativeness%c. 3.3.2.2Numericalset-upfortheBranin-Hoofunction TwoDoEsareusedforthisfunction,thatconsistofthefourcornersofthedesignregionplus13and30points,respectively,generatedusingLatinHypercubeSampling(LHS)withmaximumminimumdistancecriterion;i.e.,atotalof17and34observations,respectively.Thetestpointsaregeneratedusinga3232uniformgrid(total1024points). 49

PAGE 50

TheeRMSisweightedsuchthatthepointsinsidethedomainhaveaweight1,thepointsontheedgesaweight1/2andthepointsonthecorners1/4. ThePRSisacubicpolynomial.ForKriging,anordinarykriging(OK)withrationalquadraticcovariancefunctionisused,whichisprovidedbytheGPMLToolbox[ Rasmussen&Williams ( 2006 )]. LHSarerandomdesigns,whichallowsustorepeattheprocedurealargenumberoftimes.Wepresenttheresultsastheaverageover500repetitions.Inaddition,weuseerrorbarstorepresent95%condenceintervalsonlandMaxUE%foragivenlevelof%c,andon%cforagivenleveloftargetconservativeness. TheunbiasedpolynomialresponsesurfacesarecomputedusingMatLabfunctionregress;thekrigingestimatesarecomputedusingtheGPMLtoolboxforMatLab. 3.3.2.3Numericalset-upfortheTorquearmmodel TheDoEconsistsof300pointsgeneratedfromLHSwithmaximumminimumdistancecriterion;1000testpointsaregeneratedusingLHS.Foreachpoint,theANSYScodeisrunandreturnsthemaximumstress.Theresponsevaluesareoftheorderof102MPa.ForthePRS,athirdorderpolynomialisused.ForKriging,severalcovariancefunctionsweretested,anditwasfoundthatasumofarationalquadraticcovariancefunction(Eq. 2{19 )andnuggeteectreectedaccuratelythecovariancestructure. Sincetheniteelementanalysisiscomputationallyexpensive,itisimpracticaltogeneratealargenumberofDoEsforvariabilityanalysis,aswedofortheBranin-Hoofunction.Inordertoobtainsomecondenceintervals,werandomlychoose300pointsoutofthe1300pointsgenerated(300training+1000testpoints),andusetheremaining1000astestpoints.Thisprocedureisrepeated500times.Onthegraphs,theaveragecurvecorrespondtotheoriginalDoEandtheerrorbarstothevariabilityestimationprocedure. 50

PAGE 51

3.4ResultsAndDiscussion 3.4.1Branin-HooFunction 3.4.1.1Analysisofunbiasedsurrogatemodels First,welookattheperformanceoftheunbiasedsurrogatemodelsontheBranin-Hoofunction.Table 3-2 reportstheeRMS,%candMaxUEforbothPRSandKriging. Table3-2. Meanvaluesofstatisticsbasedon1,024testpointsand500DoEsfortheunbiasedsurrogatesbasedon17observations.Numberinparenthesisarestandarddeviationsofthequantities. SurrogateeRMS%cMaxUE PRS8.1(4.6)53.1%(8.4)25.3(15.7)Kriging9.1(1.2)55.8%(5.8)22.6(8.1) Onaverage,PRSandKrigingperformsimilarlyontheBranin-Hoofunction.KrigingislesssensitivetotheDoE,sincethestandarddeviationsofeRMSandMaxUEaresmallerthanforPRS.TheMaxUEisrelativelysmallcomparedtotherangeofthefunction([0300]).Bothsurrogateshaveanaveragepercentageofconservativeerrorsslightlylargerthan50%,whichshowsthatsomehypothesisofthemodelmaybeviolated. 3.4.1.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates Now,wecompare,foreachmetamodel,constantandpointwisesafetymarginsusingerrordistribution.Figure 3-8 showstheresultscorrespondingtoPRS,andFigure 3-7 toKriging.Forbothgraphs,therangeofthesafetymarginischosenintheinterval[0;15];thetargetconservativenessischosenbetween50%and99%. ForPRS,weseeinFigure 3-8 A)thatthetwomethodsareequivalentintermsofaccuracyandvariability.95%conservativenessisobtainedforaneRMStwiceaslargeastheeRMSoftheunbiasedresponsesurface.However,thereisasubstantialdierenceforthemaximumunconservativeerror.Forthesameproportionofconservativeresults,thiserrorismorereducedwithEDthanwithCSM. ForKriging,theCSMestimatorclearlyoutperformstheEDestimatorintermsofaccuracyvs.conservativeness(Figure 3-7 A)).Thesamelevelofconservativenessis 51

PAGE 52

obtainedwithlesseectontheerror,especiallyforthe70-90%range.Moreover,thevariabilityismuchlargerforthestatisticalestimator. Figure3-7. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingbasedon17points.A)%cvs.l;B)%cvs.MaxUE%. Figure3-8. AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSbasedon17points.A)%cvs.l;B)%cvs.MaxUE%. ThelossinaccuracyoftheCSMestimatorincreasesveryslowlywith%c(Figure 3-7 A)).Upto70%conservativenessisachievedwithverylittleeectoneRMS.ThisbehaviorcanbeimputedtothenatureoftheKrigingmodel.SinceKrigingisaninterpolation,errorsareverysmallatthevicinityofthetrainingpoints.Thus,addingasmallconstantissucienttobeconservativebuthaslittleeectontheaccuracy. 52

PAGE 53

Ontheotherhand,EDestimatorappearstobemuchbettertoreducethemaximumunconservativeerror.AsshowninFigure 3-7 B),theEDcurve(grey)isalwayshigherthantheCSMone(black),whichmeans,foraequivalentproportionofconservativeresults,themaximumunconservativeerrorismorereducedwithEDthanwithCSM. ThisbehaviorisampliedwhenKrigingisbasedonalargerDoE.Figure 3-9 showsthegraphsofperformancefortheKrigingttedfrom34designpoints.TheCSMestimatorcurvehasaveryatportionupto75%conservativeness,whiletheEDestimatorcurveincreasesrapidly.Alongeratportionislogicalbecausetheregionwhereerrorsaresmall,atthevicinityoftheDOEpoints,islargersincetherearemorepoints.ThevariabilityismuchhigherfortheEDestimator.Ontheotherhand,theCSMestimatordoesnotpreventlargeunconservativeerrors:themaximumerrorremainsalmostthesameevenforlargeproportionsofconservativeresults.TheEDestimatorperformsalotbetteraccordingtothisindicator. Figure3-9. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingbasedon34points.A)%cvs.l;B)%cvs.MaxUE%. Now,wecomparethedelitytotargetconservativenessofbothCSMandEDestimators,whentheconstantmarginisdesignedwiththehelpofcross-validation.Figure 3-10 showstheresultsforPRS,andFigure 3-11 forKriging. 53

PAGE 54

Figure3-10. Targetvs.actual%cforPRSusingCSMwithcross-validation(plainblack)andED(mixedgray)basedon17points. ForPRS,bothEDandCSMestimatorsshowpoordelitytotargetconservativeness.FortheEDestimator,themeanactual%cis5%higherthanthetarget,andtheerrorbarsshowavariabilityoftheorderof5%.TheassumptionsofregressionmaybeviolatedwiththeBranin-Hoofunction;fortheunbiasedPRS,theactualconservativenessis55%where50%isexpected. TheCSMdesignedbycross-validationisinaccurateandhasverylargevariability.Thepoorperformanceofcross-validationcanbeexplainedbythesizeoftheDoE.Indeed,removingoneobservationoutof17canmodifysubstantiallythequalityofthesurrogate,socross-validationoverestimatestheerroramplitude.Then,theCSMischosenfrom17valuesonly,whichisnotenoughtoensureagoodprecision. ForKriging,inaveragetheEDestimatorshowsgooddelitysincethetrendisalmostequaltothestraightline.Forthehigherlevels,theactualconservativenessisalessthanthetarget.ThisisduetothefactthatKrigingassumenormalityoferrors,whilefortheBranin-Hoofunctionthisassumptionmaybeatacertaindegreeviolated.Whenlookingatthevariability,thecondenceintervalisabout20%,whichisverylarge.Inparticular,when97%conservativenessisexpected,theactualconservativenesscanbeaslowas70%. 54

PAGE 55

Figure3-11. Targetvs.actual%cforKrigingusingCSMwithcross-validation(plainblack)andED(mixedgray)basedon17points. AswellasforPRS,theCSMestimatorshowsbothinaccuracyandlargevariability.Inadditiontothereasonsexposedabove,cross-validationsuersherefromtheinterpolatingnatureofKriginginlowdimension.Indeed,thereareregionsofsmallerrors(inthevicinityoftrainingpoints)andregionsoflargeerrors.Sincetheobservationsarechosenaccordingtoaspace-llingDoE,trainingpointsareawayfromeachother,andcross-validationestimatestheerrorinregionsofhighuncertaintyofthemodel,whichexplainswhyhereCSMisoverconservative. 3.4.1.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates WehaveseenforPRSthatEDestimatesperformalittlebitbetterthanCSMontheBranin-Hoofunction,butshowpoordelitytotargetlevelofconservativeness,whichleadsforthisexampletounnecessarilylargemargins.Now,wecomparethisestimatortotheestimatorbasedonbootstrap. BeforeanalyzingtheresultsfortheBranin-Hoofunction,weproposetoillustratetheadvantageofbootstrapoverclassicalcondenceintervalsonatestcasewherethebootstrapissupposedtoshowbetteraccuracy.Thisisthecaseinparticularwhentheassumptionofnormalityofresidualsisviolated[ Stine ( 1985 )]. 55

PAGE 56

Theexperimentalsetupisasfollow: thefunctiontoapproximateisasecondorderpolynomialy=x)]TJ /F6 7.97 Tf 13.15 4.71 Td[(x2 2 theDoEconsistsof100observationsuniformlytakenon[0;1] thenoisefollowsalognormaldistribution,with=0and=1.Thedistributionisshiftedby)]TJ /F1 11.955 Tf 11.29 0 Td[(exp(+2=2)sothemeaniszero,anddividedbytwotoreducetheamplitude. Figure 3-12 showstheobservations,polynomialresponsesurfaceand95%condenceintervals(CI)computedusingbothclassicalformulaandbootstrap. Figure3-12. Condenceintervals(CI)computedusingclassicalregressionandbootstrapwhentheerrorfollowsalognormaldistribution.Bootstrapcatchestheasymetryofthedistributionandprovidesaccuratecondenceintervals. First,weseethattheobservationsarenotevenlydistributedaroundthetruefunction(70%arebelow).Theheavytailshapeofthedistributionofthenoiseisclearlyapparentwiththefewlargepositivenoises. Sinceitisbasedonalargenumberofobservations,theregressionbestpredictoriscorrectevenifassumptionofnormalityisviolated.However,thecondenceintervalsareinaccurate:theyaresymmetricwhilethenoiseisnot.So,(inabsolutevalues)thelowerboundisoverestimated,andtheupperboundisunderestimated.Ontheotherhand,thebootstrapcondenceintervalisaccurate:itisstronglyasymmetricandmatchesthedistributionoftheresiduals. 56

PAGE 57

Inthisexamplebootstrapisveryaccuratesinceitisbasedonalotofobservationsandtheregressionmodelisverysimple.Now,wecompareBSandCSMestimatorsontheBranin-Hoofunction.ThegraphsofperformancearegiveninFigure 3-13 ,andFigure 3-14 showsthedelityofbothestimatorstotarget%c. Figure3-13. AverageresultsforBS(plainblack)andED(mixedgrey)forPRSbasedon17points..A)%cvs.l;B)%cvs.MaxUE%. Figure3-14. Targetvs.actual%cforKrigingusingCSMandcross-validation(plainblack)andED(mixedgray)basedon17points. First,weseethattheevolutionofthelossinaccuracywiththepercentageofconservativeresultsisthesameforthetwoestimators(Figure 3-13 A)).EDisamore 57

PAGE 58

ecientthanbootstraptoreducethemaximumunconservativeerror(Figure 3-13 B)).Forthedelitytotargetconservativeness,onaverage,BSperformsalittlebitbetterthanEDbuttheirvariabilityaresimilar.Onthisexample,bootstrapdoesnotprovidesubstantiallybetterresultsthanED. 3.4.2TorqueArmModel 3.4.2.1Analysisofunbiasedsurrogatemodels First,welookattheperformancesoftheunbiasedsurrogatemodelsforthetorquearmstressanalysis.Table 3-3 reportstheeRMS,%candMaxUEforbothPRSandKriging. Table3-3. Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. SurrogateeRMS%cMaxUE PRS16.352.0%77.5Kriging14.551.6%60.7 Krigingperformsbetterintermsoferror.Bothsurrogateshaveapproximately50%negativeerrors.TheireRMSarereasonablecomparedtothemeanvalueofthestress(210MPa).However,weseethatthemaximumerrorisverylarge,especiallyforPRS.Designingthetorquearmbasedonthesesurrogateswouldresultinagreatriskofpoordesign. 3.4.2.2ComparingConstantSafetyMargin(CSM)andErrorDistribution(ED)estimates Figure 3-15 showstheresultscorrespondingtoPRS,andFigure 3-16 toKriging. ForPRS,weseethesamebehaviorfortheBranin-Hoofunctionandtorquearmanalysis:CSMandEDaresimilarwhencomparingthelossinaccuracywhenthepercentageofconservativeresultsincreases,butEDreducesbetterthemaximumunconservativeerror.Here,itisimportanttonoticethatevenwhen%cishigherthan95%,theMaxUEisreducedbylessthan50%inthebestcase. 58

PAGE 59

Figure3-15. AverageresultsforCSM(plainblack)andED(mixedgrey)forPRSonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. Figure3-16. AverageresultsforCSM(plainblack)andED(mixedgrey)forKrigingonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. ForKriging,againCSMperformsbetterintermsoflossinaccuracyl,butreducesverypoorlytheMaxUE.EDperformsbetterregardingthisindicator,butaswellasforPRS,highlevelsof%conlycorrespondtoabout50%reductionofMaxUE. Now,welookatthedelitytotarget%c.Figure 3-17 showstheresultscorrespondingtoPRS,andFigure 3-18 toKriging. Onbothgures,weseethatusingconstantmarginandcross-validationresultsinagreatdelitytotargetconservativeness.EDestimatorsarealittlebitbiasedregardingdelity:PRSslighltyunderestimates%cforallprobabilityrange,andKriging 59

PAGE 60

overestimatesconservativenessforhighprobabilities.Theseerrorscanbeimputedtotheparametric(gaussian)formoftheerrormodel. Figure3-17. Targetvs.actual%cforPRSusingCSMandcross-validation(plainblack)andED(mixedgray)onthetorquearmdata. Figure3-18. Targetvs.actual%cforKrigingusingCSMandcross-validation(plainblack)andED(mixedgray)onthetorquearmdata. Incontrast,cross-validationdoesnotassumeanyformfortheerror.Thequalityofcross-validationmarginsarealotbetterthanintheprevioustestcas,whichcanbeexplainedbythelargernumberofobservations.Firstly,with300observations,leavingone 60

PAGE 61

observationouthasverylittleimpactontheaccuracyofthesurrogate,sotheerrorvectorgivenbycross-validationismorerealistic.Secondly,theCSMisbasedonalargenumberofvalues(300insteadof17previously),whichallowstodesignitwithbetteraccuracy. 3.4.2.3ComparingBootstrap(BS)andErrorDistribution(ED)estimates Figure 3-19 showstheresultsofBSandEDestimates.Forthoseindicators,thetwomethodsaresimilar. Figure3-19. AverageresultsforBS(plainblack)andED(mixedgrey)forPRSonthetorquearmdata.A)%cvs.l;B)%cvs.MaxUE%. Figure3-20. Targetvs.actual%cforPRSusingBS(plainblack)andED(mixedgray)onthetorquearmdata. 61

PAGE 62

Figure 3-20 showsthedelityofbothestimatorstotarget%c.Here,bootstrapdoesnotperformaswellasED.Itisinterestingtonoticethantheyhaveoppositebias,sinceEDisoverconservativeandBSovercondent. 3.5ConcludingComments Inthischapter,weexploredthealternativestoobtainconservativepredictionsusingsurrogatemodeling.First,weshowedthatconservativenesscouldbeobtainedusingparametricmethods,bytakingadvantageoferrordistributionmeasuresgivenbythemodel,ornon-parametric,suchasbootstraporcross-validation.Then,aParetofrontmethodologywasintroducedtomeasurethequalityofthedierentmethods.Finally,themethodsareimplementedfortwotestproblems:oneanalyticalandonebasedonFiniteElementAnalysis.Resultsshowedthat: Theuseofsafetymarginsanderrordistributionleadtoverycomparableresultswhenthetrade-obetweenaccuracyandconservativenessareconsidered EDestimators,foranequivalentlevelofconservativenessandaccuracy,preventbetterthanCSMfromtheriskoflargeunconservativeerrors EDestimatorsarebasedonassumptionsthatmaynotbeviolatedinordertoobtainacceptabledelityofconservativenesslevel. Whenthemodelhypothesisareviolated,bootstrapoersanecientalternativetoobtainaccuratedelity Cross-validationisanecienttechniquetodesignconstantmarginswhenthenumberofobservationsishigh. Moregenerally,ithasbeenshownthatmeasuringtheconservativenessofamethodisnoteasyandcanbeveryproblem-dependant.Indeed,thechanceofbeingconservativeandtheriskoflargeunconservativeerrorsaretwomeasuresofconservativenessthatdonotbehaveidentically.Onemaychooseaconservativestrategybasedonthetrade-obetweenthesetwoquantitiesandtheglobalmeasureofaccuracy. 62

PAGE 63

CHAPTER4CONSERVATIVEPREDICTIONSFORRELIABILITY-BASEDDESIGN 4.1IntroductionAndScope Whenanengineeringsystemhasuncertaintyinitsinputparametersandoperatingconditions,thesafetyofthesystemcanbeevaluatedintermsofreliability.Manymethodshavebeenproposedtoestimatethereliabilityofasystem,suchasMonteCarlosimulation(MCS)method[ Haldar&Mahadevan ( 2000 )],FirstandSecond-orderReliabilityMethod[ Enevoldsen&Srensen ( 1994 ), Melchers ( 1987 )],importancesamplingmethod[ Engelund&Rackwitz ( 1993 )),tailmodeling( Kimetal. ( 2006 )],orinversemethods[ Qu&Haftka ( 2004 )].MCSisoftenusedtoestimatethereliabilityofthesystemthathasmanyrandominputsormultiplefailuremodes,becauseitsaccuracyisindependentofthecomplexityoftheproblem.Inthispaper,reliabilityanalysisusingMCSmethodisconsidered.Thecomparisonbetweendierentreliabilityanalysismethodsisbeyondthescopeofthischapterandcanbefoundintheliterature[ Rackwitz ( 2000 ), Leeetal. ( 2002 )]. Whenthecostofsimulationishigh,engineerscanaordtohaveonlyalimitednumberofsamples,whichisnotsucienttoestimatethereliabilitywithacceptableaccuracy[ Ben-Haim&Elishako ( 1990 ), Nealetal. ( 1992 )].Insuchacase,itisoftenrequiredtocompensateforthelackofaccuracywithconservativeapproaches.Forexample,anti-optimization[ Elishako ( 1991 ), Duetal. ( 2005 )]andpossibility-baseddesign[ Duetal. ( 2006 ), Choietal. ( 2005 )]areusedtocompensateforthelackofknowledgeintheinputdistributionbyseekingtheworstcasescenarioforagivendesign.Suchapproacheshavebeenfoundtoleadtoconservativedesigns[ Nikolaidisetal. ( 2004 )].Bayesianreliability-basedoptimization[ Youn&Wang ( 2008 )]usesBayesiantheorytoensuretoproducereliabledesignswhenonlyinsucientdataisavailablefortheinputs. Inthischapter,wefocusonthecasewhentheprobabilityoffailure,Pf,ofasystemisestimatedfromalimitednumberofsamples.Theobjectiveistondaconservativeestimate,^Pf,thatislikelytobenolowerthanthetruePf.Pfisestimatedusing 63

PAGE 64

particulartypesofsurrogatemodels,whicharedistributionfunctionmodels.Toprovidesuchestimation,twoalternativesareconsidered:therstmethodisbasedonbiasingtheprocessofttingthedistributionusedtocomputetheestimatorofPf.Thesecondistheuseofbootstrapmethod[ Efron ( 1982 ), Chernick ( 1999 )]forquantifyingtheuncertaintyinprobabilityoffailureestimations,anddeningconservativeestimatorsbasedonbootstrapping.Aswellasinthepreviouschapter,atrade-oanalysisbetweenaccuracyandthelevelofconservativenessisproposedwiththehelpofnumericalexamples. Inthenextsection,wediscusshowweusesamplingtechniquestoestimatetheprobabilityoffailure.Section 4.3 showshowtouseconstraintstoobtainconservativeestimators.Section 4.4 describeshowtousethebootstrapmethodtodeneconservativeestimator.TheaccuracyofsuchestimatorsisanalyzedusingasimplenumericalexampleinSection 4.5 ,andananalysisoftheeectsofsamplesizesandtargetprobabilityoffailureonthequalityoftheestimatesconservativeestimatorsisgiveninSection 4.6 .Finally,theconservativeapproachisappliedtoanengineeringprobleminSection 4.7 ,followedbyconcludingremarksinSection 4.8 4.2EstimationOfProbabilityOfFailureFromSamples 4.2.1Limit-StateAndProbabilityOfFailure Failureofasystemcanusuallybedeterminedthroughacriterion,calledalimit-state,G.Thelimit-stateisdenedsothatthesystemisconsideredsafeifG0andfailsotherwise.Typically,thelimit-stateofastructurecanbedenedasthedierencebetweenresponse,R,(e.g.,maximumstressorstrain)andcapacity,C,(e.g.,maximumallowablestressorstrain): G(x)=R(x))]TJ /F4 11.955 Tf 11.95 0 Td[(C(x)(4{1) Duetouncertaintiesininputparameters,thelimit-stateisrandom.Thus,onecanestimatethesafetyofthesystemintermsprobabilityoffailure,whichisdenedas Pf=Prob(G0)(4{2) 64

PAGE 65

Therearemanymethodsforcalculatingthefailureprobabilityofasystem[ Haldar&Mahadevan ( 2000 ), Enevoldsen&Srensen ( 1994 ), Melchers ( 1987 )].Someofthemusetherelationbetweeninputrandomvariablesandthelimit-state(e.g.,rstandsecond-orderreliabilitymethods)andsomeconsiderthelimit-stateasablack-boxoutput(e.g.,MCS).Whenthenumberofinputrandomvariablesislarge,andthelimit-stateiscomplexandmulti-modal,MCShasaparticularadvantageasitsaccuracyisindependentoftheproblemdimensionorcomplexityofthelimit-statefunction.MCSgeneratessamplesofthelimit-statefg1;g2;;gngandcountsthenumberoffailedsamples[ Melchers ( 1987 )].Theratiobetweenthenumbersoffailuresandthetotalnumberofsamplesapproximatestheprobabilityoffailureofthesystem: Pf^Pf=1 NnXi=1I[gi0](4{3) ThevarianceofMCSestimatesisinverselyproportionaltothesquarerootofthenumberofsamplestimestheprobabilityoffailure(forlargeN): var(^Pf)Pf(1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pf) N(4{4) Thus,theaccuracyispoorwhenthenumberofsamplesissmallorwhentheprobabilityoffailureislow.Forinstance,ifaprobabilityoffailureof10)]TJ /F5 7.97 Tf 6.58 0 Td[(4isestimated(whichisatypicalvalueinreliabilitybaseddesign),106samplesareneededfor10%relativeerror. Inthischapter,wefocusonsituationswhereonlysmallsamplesareavailableregardingtheprobabilityoffailure;bysmallismeantfromtento500,forprobabilitiesoffailuregoingfrom1%to10)]TJ /F5 7.97 Tf 6.59 0 Td[(6.Insuchacase,thedirectuseofMCS(countingthenumberoffailedsamples)isunrealistic,sincetheprobabilityestimatewouldmostprobablybezero. 65

PAGE 66

Analternativeistoapproximatethecumulativedistributionfunction(CDF)FGofthelimit-statebyacontinuousfunctionandestimatetheprobabilityoffailureusing: ^Pf=1)]TJ /F4 11.955 Tf 11.95 0 Td[(FG(0)(4{5) Ingeneral,FGisunknownandisapproximatedbyaparametricdistributionF,whichisdenedbydistributionparameters,.Fcanbeconsideredasaone-dimensionalsurrogatemodelthatmustbettedfromthedatafg1;g2;;gng.Fcanbealogisticregressionmodel,ortheCDFofthenormaldistribution(hence=f;g),denedby: F(g)=1 p 2gZexp")]TJ /F1 11.955 Tf 9.29 0 Td[((u)]TJ /F4 11.955 Tf 11.95 0 Td[()2 22#du(4{6) Thesamplesoflimitstatescanbeobtainedbygeneratinginputrandomvariablesandpropagatingthroughthesystem.However,theproposedmethodcanbeappliedtothecasewheninputrandomvariablesanduncertaintypropagationareunknown.Forexample,thesamplesoflimitstatescanbeobtainedfromexperiments. Inthefollowingsection,twomethodsofestimatingdistributionparametersfromasetofsamplesarediscussed. 4.2.2EstimationOfDistributionParameters Thereexistsseveralalternativestoestimatethedistributionparameters.Thedicultyhereistodeneattingcriterion,sincetheusualassumptionsofsurrogatemodelsarenotmet(forinstance,normalityoftheresiduals).Whenpossible(forexampleforthenormaldistribution),theycanbeestimatedusingmoment-basedmethods.Here,weproposetodenecriteriabasedontheadequacyoftheparametricmodeltotheempiricalCDF. 66

PAGE 67

Considernlimit-statesamplesarrangedinincreasingorder:fg1;g2;;gng.TheempiricalCDFFnisdenedas: Fn(g)=8>>>><>>>>:0forgg1k/nforgkggk+11forggn(4{7) ItisthenpossibletoestimatetheparametersoftheCDFthatapproximatesFnbest.Twodierentapproximationmethodsarediscussedhere: minimizingtheroot-mean-square(RMS)error, minimizingtheKolmogorov-Smirnovdistance[ Kenney&Keeping ( 1954 )]. TominimizetheRMSdierencebetweentheempiricalandtheestimatedCDF,errorsarecalculatedatthesamplepoints.Inordertohaveanunbiasedestimation,thevaluesoftheempiricalCDFarechosenatthemiddleofthetwodiscretedata,as(seeFigure 4-1 A)): Fn(gk)=k)]TJ /F5 7.97 Tf 13.15 4.71 Td[(1 2 n;1kn(4{8) Thentheestimatedparameters^arechosentominimizethefollowingerror: ^=argminvuut nXk=1F(gk))]TJ ET q .478 w 284.31 -403.94 m 293.52 -403.94 l S Q BT /F4 11.955 Tf 284.31 -413.78 Td[(Fn(gk)2(4{9) TheestimatebasedonEq. 4{9 iscalledanRMSestimate. TheKolmogorov-Smirnov(K-S)distanceistheclassicalwaytotestifasetofsamplesarerepresentativeofadistribution.TheK-SdistanceisequaltothemaximumdistancebetweentwoCDFs(seeFigure 4-1 B)).TheoptimalparametersfortheK-Sdistanceare: ^=argminmax1knF(gk))]TJ /F4 11.955 Tf 13.4 8.09 Td[(k n;F(gk))]TJ /F4 11.955 Tf 13.15 8.09 Td[(k)]TJ /F1 11.955 Tf 11.95 0 Td[(1 n(4{10) Figure 4-1 A)showstheempiricalCDFfromtensamplesandthedatapointsthatareusedinEq. 4{8 .Figure 4-1 B)showstheK-SdistancebetweenanormallydistributedCDFandanempiricalCDFfrom10samples. 67

PAGE 68

Inthecasesweconsiderinthiswork,itisfoundthatthetwocriteriagiveequivalentresults.K-Sdistancelinkstoclassicalstatisticalframework,butcanbediculttominimizebecauseitisnon-smooth.TheRMSerrorminimizationappearstobemorerobust. Figure4-1. CriteriatotempiricalCDFs:A)Points(circles)chosentotanempiricalCDF(line)withRMScriterion.B)K-SdistancebetweenanempiricalCDFwithtensamplesandanormalCDF(continuousline).DataisobtainedbysamplingtenpointsfromN(0;1) s ThechoiceofthedistributionFiscriticalforaccurateestimationoftheprobabilityoffailure.Wrongassumptionontheformofthedistributioncanleadtolargebiasintheestimate,forinstance,ifthedistributionisassumedtobenormalwhileitisheavy-tailed.Statisticaltechniquesareavailabletotestifasamplebelongstoagivendistributiontype(goodness-ofttests),suchastheKolmogorov-Smirnov(foranydistribution),LillieforsorAnderson-Darlingtests(fornormaldistributions)[ Kenney&Keeping ( 1954 )].Somestatisticalsoftwarealsooersautomatedprocedurestochoosefromabenchmarkofdistributionswhichtsbestthedata. 4.3ConservativeEstimatesUsingConstraints InthespiritofthebiasedttingestimatesproposedinAppendix A.1 ,weproposetoaddconstraintstothettingprobleminordertomaketheestimatesmoreconservative. 68

PAGE 69

Aconservativeestimateoftheprobabilityoffailureshouldbeequalorhigherthantheactualone.Hence,aconservativeestimatecanbeobtainedbyconstrainingtheestimatedCDFtobelowerthanthetrueCDFwhentheparametersarefoundthroughtheoptimizationproblemsinEqs. 4{9 or 4{10 .Besides,weconsideronlysmallprobabilities,sothefailureoccursintheuppertailregionofthedistribution.Hence,theconstraintswillbeappliedtoonlytherighthalfofthedata.Whenthefailureoccursinthelowertailregion,theconstraintsshouldbeappliedonthelefthalfofthedata. TherstconservativeestimateoftheCDFisobtainedbyconstrainingtheestimatetopassbelowthesamplingdatapoints.AsecondcanbeobtainedbyconstrainingtheestimatedCDFbelowtheentireempiricalCDF.Theywillbecalled,respectively,CSP(ConservativeatSamplePoints)andCEC(ConservativetoExperimentalCDF).Thelatterismoreconservativethantheformer.Obviously,bothmethodsintroducebias,andthechoicebetweenthetwoconstraintsisamatterofbalancebetweenaccuracyandconservativeness. CSPconstraints: F(gi))]TJ /F4 11.955 Tf 14.65 8.09 Td[(i n0forn 2in(4{11) CECconstraints: F(gi))]TJ /F4 11.955 Tf 13.15 8.09 Td[(i)]TJ /F1 11.955 Tf 11.96 0 Td[(1 n0forn 2in(4{12) Toillustratetheseconservativeestimators,tensamplesaregeneratedfromarandomvariableGwhosedistributionisN(2:33;1:02).ThemeanischoseninsuchawaythattheprobabilityoffailureP(G0)isequalto1%.Figure 4-2 showstheempiricalCDFalongwiththethreeestimatesbasedonminimumRMSerror:(1)withnoconstraint,(2)withCSPconstraints,and(3)withCECconstraints.Table 4-1 showstheparametersofthethreeestimateddistributionsandthecorrespondingprobabilitiesoffailure. Theeectoftheconstraintsisclearfromthegraph.TheCSPestimatorisshifteddowntotheninthdatapoint;hence,theCDFatthetailisdecreased.TheCECestimator 69

PAGE 70

Figure4-2. ExampleofCDFestimatorsbasedonRMSerrorforasampleofsize10generatedfromN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02) Table4-1. Comparisonofthemean,standarddeviation,andprobabilityoffailureofthethreedierentCDFestimatorsforN()]TJ /F1 11.955 Tf 9.29 0 Td[(2:33;1:02).ExactvalueofPfis1%(samesampleasFigure 4-2 ). NoconstraintCSPCEC^2.292.342.21^0.840.971.31^Pf0.32%0.77%4.65% isshiftedevenfurtherdown.Sincetheconservativeestimatorsareunconstrainedonthelefthalfofthedistribution,theirCDFcurvescrosstheempiricalcurveonthatside. Forthisillustration,wechoseasamplerealizationthatisunfavorableforconservativeness.Asaconsequence,theestimatewithnoconstraintisstronglyunconservative(0.32%comparedto1.0%)eventhoughtheestimationisunbiased.TheCSPestimateisunconservative,butsubstantiallylessthantheunbiasedestimate.TheCECestimateisconservative.Inordertogeneralizetheseresultsandderivereliableconclusions,statisticalexperimentsbasedonlargenumberofsimulationswillbeperformedinSection 4.5 4.4ConservativeEstimatesUsingTheBootstrapMethod Analternativetobiasedttingistoobtaincondenceintervalsfortheprobabilityoffailureestimatesinordertodeterminethemarginneededtobeconservative.However, 70

PAGE 71

analyticalderivationofcondenceintervalsfortheprobabilityoffailureisverychallenging.Toovercomethisproblem,weproposetoobtaincondenceintervalsusingnumericalprocedures,i.e.bootstrap. Thebootstrapprincipleisdescribedinsection 3.2.5.1 .Whenonlylimitedsamplesareavailable,thebootstrapmethodcanprovideanecientwayofestimatingthedistributionofstatisticalparameter.Regardlessofitsconstruction,theprobabilityoffailureisestimatedfromsamplesofthelimit-statefg1;:::;gng.Hence,itispossibletoresamplewithreplacementfromfg1;:::;gng,andforeachsetofbootstrapre-samples,ttheCDFandcomputePf. Thestandarderrororcondenceintervalsofthestatisticalparametercanbeestimatedfromthebootstrapdistribution.However,thebootstrapmethodprovidesonlyanapproximationofthetruedistributionbecauseitdependsonthevaluesoftheinitialsamples.Inordertoobtainreliableresults,itissuggestedthatthesizeofthesamplesshouldbelargerthan100(i.e.,n)[ Efron ( 1982 )].Thenumberofbootstrapre-samplings(i.e.,p)ischosenlargeenoughsothatitdoesnotaectthequalityoftheresults(themajorsourceofuncertaintybeingtheinitialsample).Thevalueofpistypicallytakenfrom500to5,000. Forillustratingtheprocess,thefollowingcaseisconsidered:n=100andp=5;000.Thatis,100samplesofarandomvariableGaregeneratedfromthenormaldistributionN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02).Thetrueprobabilityoffailureis1%.Pretendingthatthestatisticalparameters(mean,standarddeviation,orprobabilityoffailurePf)areunknown,theseparametersaswellastheircondenceintervalswillbeestimatedusingthebootstrapmethod. Usingthegivensetof100initialsamples,5,000bootstrapre-samplingsareperformed.Similartotheconservativeestimationsintheprevioussection,thedistributiontypeisrstassumedtobenormal.Usingeachsetofbootstrapre-samples,themeanandstandarddeviationareestimated,fromwhichtheestimatedprobabilityoffailure^Pfiscalculated. 71

PAGE 72

The5,000^Pfvaluesdenetheempiricalbootstrapdistributionoftheestimator^Pf,representedinFigure 4-3 inahistogramform. Figure4-3. ConservativeestimatorsofPffrombootstrapdistribution:95thpercentile(p95)andmeanofthe10%highestvalues(CVaR). Theempiricalbootstrapdistributioncanbeusedtominimizetheriskofyieldingunconservativeestimate.Inotherwords,wewanttondaprocedurethatcalculatesthefollowingquantity: =P^PfPf(4{13) AprocedurethatsatisesEq. 4{13 iscalledan-conservativeestimatorofPf.Forexample,if=0.95isdesired,then^Pfisselectedatthe95thpercentileofthebootstrapdistributionoftheestimatedprobabilityoffailure.Thisestimatorwillbereferredto'Bootstrapp95'(seeFigure 4-3 ).Duetothenitesamplesize,however,Eq. 4{13 willbesatisedonlyapproximately. Besidesthis-conservativeestimator,themeanofthe-highestbootstrapvalues(conditionalvalue-at-riskCVaR[ Holton ( 2003 ))]isalsousedasaconservativeestimate.Here=10%isused,sotheestimatorconsistsofthemeanofthe10%highestbootstrapvalues.SinceCVaRisameanvalue,itwillbemorestablethanthe-conservativeestimator.However,itisdiculttodeterminethevalueofthatmakesEq. 4{13 satised 72

PAGE 73

precisely.Thisestimatorwillbereferredto'BootstrapCVaR90'(seeFigure 4-3 ).Notethatanybootstrappercentilehigherthan50%willbeaconservativeestimator.Averyhighorlowwillincreasethevalueof^Pfandwillyieldover-conservativeestimation. 4.5AccuracyAndConservativenessOfEstimatesForNormalDistribution ThegoalofthissectionistoevaluatetheaccuracyandtheconservativenessoftheestimatorspresentedinSections 4.3 and 4.4 ,whentheactualdistributionisknowntobenormal.StatisticalmeasuresoftheestimatorsareevaluatedbyestimatingPfalargenumberoftimes. Wealsointroduceherethereliabilityindex,whichisdenotedbyandrelatedtotheprobabilityoffailureas: =)]TJ /F1 11.955 Tf 9.3 0 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1(Pf)(4{14) whereistheCDFofthestandardnormaldistribution. ThereliabilityindexisoftenusedinsteadofPfinreliabilitybaseddesignbecausetherangeofvalues(typicallybetweenoneandve)ismoreconvenientanditsvariabilityislowerthanPf.Itisimportanttonotethatsince)]TJ /F1 11.955 Tf 9.3 0 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1isamonotonicallydecreasingfunction:alowprobabilitycorrespondstoahighreliabilityindex.Thus,aconservativeestimationofshouldnotoverestimatethetrue(sinceaconservativeestimationshouldnotunderestimatethetruePf).Inthefollowing,wepresenttheresultsforbothprobabilityoffailureandreliabilityindex. First,100samplesofGarerandomlygeneratedfromthenormaldistributionwithmean-2.33andstandarddeviation1.0.ThefailureisdenedforG0,whichcorrespondstoanactualprobabilityoffailureof1.0%.Foragivensetofsamples,dierentestimatorsareemployedtoestimatePf.Fivedierentestimatorsarecompared:theunbiasedtting,CSP,CEC,Bootstrapp95,andBootstrapCVaR90estimators.Thisprocedurewasrepeated5,000timesinordertoevaluatetheaccuracyandconservativenessofeachestimator.Fortheunbiased,CSPandCECestimators,wetestedbothRMSandKolmogorov-Smirnovdistancecriteriaandfoundthattheirperformancewascomparable 73

PAGE 74

butusingK-Sdistanceslightlyincreasesvariability.So,resultsarepresentedforRMScriteriononly. Mostoftheestimatedvalueswillexceedtheactualprobabilityoffailure,butitisdesirabletomaintainacertainlevelofaccuracy.Thus,theobjectiveistocompareeachestimatorintermsofaccuracyandconservativeness. Table 4-2 showsthestatisticalresultsfrom5,000repetitions.Resultsarepresentedintheformofthemeanvalueandthe90%symmetriccondenceinterval[5%;95%].Fortheprobabilityoffailureestimates,thelowerboundofthecondenceintervalshowstheconservativenessoftheestimator;themeanandtheupperboundshowtheaccuracyandthevariabilityoftheestimator.Ahighlowerboundmeansahighlevelofconservativeness,butahighmeanandupperboundmeanpooraccuracyandhighvariability.Forthereliabilityindexestimates,theupperboundshowstheconservativenessandthemeanandlowerboundtheaccuracy. Table4-2. MeansandcondenceintervalsofdierentestimatesofP(G0)andcorrespondingvalueswhereGisthenormalrandomvariableN()]TJ /F1 11.955 Tf 9.3 0 Td[(2:33;1:02).Thestatisticsareobtainedover5,000simulations. Estimators Pf(%) %ofcons. 90%C.I.Mean 90%C.I.Mean results Unbiased [0.37;2.1]1.02 [2.0;2.7]2.34 48CSP [0.63;3.6]1.86 [1.8;2.5]2.12 82CEC [0.95;5.5]2.97 [1.6;2.3]1.96 94Boot.p95 [0.83;3.7]2.06 [1.8;2.4]2.07 92Boot.CVaR90 [0.88;3.8]2.15 [1.8;2.4]2.05 93Actual 1.00 2.33 First,thecondenceintervaloftheunbiasedestimatorillustratestheriskofunconservativeprediction:indeed,thelowerboundis0.37%,whichmeansthereisavepercentchancetounderestimatePfbyafactorofatleast2.7(1:0=0:37=2:7).Thisresultprovidesanincentiveforndingawaytoimprovetheconservativenessoftheprobabilityestimate. 74

PAGE 75

TheCSPandCECestimatorsarebiasedontheconservativeside.Asexpected,theCECismoreconservativethantheCSP.Asaconsequence,CECismorebiasedandtheriskoflargeoverestimateisincreased.TheCECcondenceintervalshowsthatthereisavepercentchancetooverestimatePfbyatleastafactorof5.5,whilethisvalueis3.6fortheCSPestimator;ontheotherhandtheCECleadsto94percentconservativeresults,whiletheCSPestimatorleadstoonly82percentconservativeresults.ThechoicebetweentheCSPandCECestimatorswillbeachoicebetweenaccuracyandconservativeness. TheBootstrapp95estimatorachieves92percentconservativenessandtheBootstrapCVaR9093percentconservativeness.Fromtheupperboundsofbothestimations,wendthattheriskofoverestimatingPfbyatleastafactorof3.7isvepercent. Theamplitudeoferrorinthereliabilityindexismuchlowerthantheamplitudeintheprobabilityoffailure.FortheCECestimator,thelowerboundofthecondenceintervalcorrespondsto31%error([2:33)]TJ /F1 11.955 Tf 12.08 0 Td[(1:6]=2:33=0:31).Forthebootstrapestimators,thiserrorisreducedto23%.Themeanerrorsarerespectively16%and11%. Bootstrapmethodsappeartobemoreecientthanthebiasedtting(CSPandCEC)intermsofaccuracyandconservativeness.Fortheequivalentlevelofconservativeness(92-94percent),thelevelofbiasisreducedandtheriskofoverestimationsislowerwhenthebootstrapmethodisused.However,asmentionedearlier,thebootstrapmethodneedsaminimumsamplesizetobeused.Ithasbeenobservedthatwhenverysmallsamplesareavailable(10to50data),theaccuracyofthebootstrapmethoddropsdramatically.Insuchacase,theoptimizationbasedmethodsshouldbeusedinstead. 4.6EectOfSampleSizesAndProbabilityOfFailureOnEstimatesQuality Intheprevioussection,weshowedthatthebiasintheconservativeestimatecanleadtolargeoverestimationsoftheprobabilityoffailure.Themagnitudeofsuchanerrormainlydependsontwofactors:thesamplesizeandthevalueofthetrueprobability.Indeed,increasingthesamplesizewillreducethevariabilityofCDFttingand,asa 75

PAGE 76

consequence,theupperboundofthecondenceinterval.Meanwhile,inordertoestimatealowervalueoftheprobabilityoffailure,weneedtousethetailoftheCDF,whichincreasesthevariabilityoftheestimation. Controllingthelevelofuncertaintyiscrucialinoptimizationinordertoavoidover-design.Inthissection,wequantifyameasureoftheuncertaintyintheconservativeestimatesasafunctionofthesamplesizeandthevalueoftheactualPf.Suchameasurecanhelpindecidingontheappropriatesamplesizetocomputetheestimate. Bootstrapp95performedwellbasedonthepreviousexample.Thus,inthissectionweconsideronlythisestimator.Westudytwodistributioncases:standardnormaldistributionandlognormaldistributionwithparameters=0and=1(meanandstandarddeviationofthelogarithmofthevariable).Threesamplesizesareconsidered:100,200and500andsevenprobabilitiesoffailureareestimated:(1105,3105,1104,3104,1103,3103and1102)1. Foragivendistribution,samplesizeandPf,themeanand90%condenceintervalsarecalculatedusing5000repetitions.ResultsarepresentedinFigure 4.6 fornormaldistributionandinFigure 4.6 forlognormal.Theaccuracyismeasuredintermsofratiosfortheestimateoverthetrueprobabilityoffailure. Asexpected,thevariabilityof^PfincreaseswhenthesamplesizeandactualPfdecrease.Here,themostunfavorablecaseiswhenthesamplesizeisequalto100andtheactualPfisequalto105.Insuchacase,forbothdistributionsthereisavepercentchancetooverestimatePfbymorethan25timesitsactualvalue!Ontheotherhand,thecasewith500samplesleadstoaveryreasonablevariability.Forthethreesamplesizesandalltargetprobabilities,thelowerboundoftheratioisequaltoone,which 1Forthenormaldistribution,thefailuresaredenedforGgreater,respectively,than4.26,4.01,3.72,3.43,3.09,2.75and2.33;forthelognormalcase,thevaluesare71.2,54.6,41.2,30.5,22.0,15.3and10.2. 76

PAGE 77

meansthatthe95%conservativenessdonotdependsonthesefactors.Thebootstrapestimatesperformancesareremarkablysimilarforthetwodistributions,eventhoughthedistributionshapesareverydierent. Figure4-4. Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforNormaldistributionbasedon(a)100,(b)200,and(c)500samples.x-axisisthetrueprobability(lognormalscale),andy-axisistheratiobetweentheestimateandthetrueprobability.Variabilityincreaseswhentargetprobabilityorsamplesizearesmaller. Figure4-5. Meanandcondenceintervalsofthebootstrapp95conservativeestimatorsforlognormaldistributionbasedon(a)100,(b)200,and(c)500samples.Resultsarealmostidenticaltothenormaldistributioncase. Foranygivenreliabilityanalysisproblem,carefulattentionneedstobegiventotheaccuracyofprobabilityoffailureestimates.ThegraphsinFigures 4.6 and 4.6 address 77

PAGE 78

thisissue.Theyshowthecondenceintervalsandthereforedeneadequatesamplesizesneededtocomputereliableestimates.Intermsofcost-eectiveness,theguresindicatethatitmaybesmarttoallocategreaternumberofsimulationstolowprobabilitydesignsthantohighprobabilitydesigninordertogetconstantlevelofrelativeaccuracy. 4.7ApplicationToACompositePanelUnderThermalLoading Inthissection,conservativeestimatesareobtainedfortheprobabilityoffailureofacompositelaminatedpanelundermechanicalandthermalloadings.Thepanelisusedforaliquidhydrogentank.Thecryogenicoperatingtemperaturesareresponsibleforlargeresidualstrainsduetothedierentcoecientsofthermalexpansionoftheberandthematrix,whichischallengingindesign. Quetal. ( 2003 )performedthedeterministicandprobabilisticdesignoptimizationsofcompositelaminatesundercryogenictemperatures,usingresponsesurfaceapproximationsforprobabilityoffailurecalculations. Acar&Haftka ( 2005 )foundthatusingCDFestimationsforstrainsimprovestheaccuracyofprobabilityoffailurecalculation.Inthischapter,theoptimizationproblemaddressedby Quetal. ( 2003 )isconsidered.Thegeometry,materialparametersandtheloadingconditionsaretakenfromtheirpaper. 4.7.1ProblemDenition Thecompositepanelissubjecttoresultantstresscausedbyinternalpressure(Nx=8:4105N=mandNy=4:2105N=m)andthermalloadingduetotheoperatingtemperatureintherangeof20K-300K.Theobjectiveistominimizetheweightofthecompositepanelthatisasymmetricbalancedlaminatewithtwoplyangles[1;2]s(thatmeansaneight-layercomposite).Thedesignvariablesaretheplyanglesandtheplythicknesses[t1;t2].ThegeometryandloadingconditionareshowninFigure 4-6 .Thethermalloadingisdenedbyastressfreetemperatureof422K,andworkingtemperatureof300Kto20K.ThematerialusedinthelaminatescompositeisIM600/133graphite-epoxy,denedbythemechanicalpropertieslistedinTable 4-3 78

PAGE 79

Figure4-6. Geometryandloadingofthecryogeniclaminate. Table4-3. MechanicalpropertiesofIM600/133material.Quantitieswitharetemperaturedependent;thenumericalvaluesinthebracketaretherangeforTgoingfrom20to300K. ElasticpropertiesE1(GPa)147120.359E2(GPa)[148]G12(GPa)[84] Coecientsof1(K)]TJ /F5 7.97 Tf 6.59 0 Td[(1[)]TJ /F1 11.955 Tf 9.3 0 Td[(510)]TJ /F5 7.97 Tf 6.59 0 Td[(7;)]TJ /F1 11.955 Tf 9.3 0 Td[(1:510)]TJ /F5 7.97 Tf 6.59 0 Td[(7]thermalexpansion2(K)]TJ /F5 7.97 Tf 6.59 0 Td[(1[110)]TJ /F5 7.97 Tf 6.58 0 Td[(5;310)]TJ /F5 7.97 Tf 6.59 0 Td[(5] Stress-freetemperatureTzero(K)422 Failurestrains"U10.0103"L2-0.013"U20.0154U120.0138 Theminimumthicknessofeachlayeristakenas0.127mm,whichisbasedonthemanufacturingconstraintsaswellasforpreventinghydrogenleakage.Thefailureisdenedwhenthestrainvaluesoftherstplyexceedfailurestrains. 79

PAGE 80

Thedeterministicoptimizationproblemisformulatedas: Minimizet1;t2;1;2h=4(t1+t2)s:t:t1;t20:127"L1SF"1"U1"L2SF"2"U2SFj12jU12(4{15) whereSFischosenat1.4. Theanalysisofthestructuralresponseisbasedonclassicallaminationtheoryusingtemperature-dependentmaterialproperties.E2,G12,1and2arefunctionofthetemperature.Sincethedesignmustbefeasiblefortheentirerangeoftemperature,strainconstraintsareappliedat21dierenttemperatures,whichareuniformlydistributedfrom20Kto300K.Detailsontheanalysisandthetemperaturedependenceofthepropertiesaregivenin Quetal. ( 2003 ).TheirsolutionsforthedeterministicoptimizationproblemaresummarizedinTable 4-4 .Forthoseresults,t1andt2werechosenonlyasmultiplesof0.127mm.Threeoptimaarefoundwithequaltotalthicknessbutdierentplyanglesandplythicknesses. Table4-4. Deterministicoptimafoundby Quetal. ( 2003 ). 1(deg)2(deg)t1(mm)t2(mm)h(mm) 27.0427.040.2540.3812.540 028.160.1270.5082.540 25.1627.310.1270.5082.540 4.7.2Reliability-BasedOptimizationProblem Giventhematerialpropertiesandthedesignvariables,theplystrainscanbecalculatedusingClassicalLaminationTheory[ Kwon&Berner ( 1997 )].Duetothemanufacturingvariability,thematerialpropertiesandfailurestrainsareconsideredrandomvariables.Allrandomvariablesareassumedtofollowuncorrelatednormaldistributions.ThecoecientsofvariationaregiveninTable 4-5 .SinceE2,G12,1and2 80

PAGE 81

arefunctionofthetemperature,themeanvaluesoftherandomvariablesarecalculatedforagiventemperature,andthen,asetofrandomsamplesaregeneratedaccordingtotheirdistributions. Table4-5. Coecientsofvariationoftherandomvariables. E1;E2;G12;121;2Tzero "L1,"U1"L2,"U2,U12 0.0350.0350.030.060.09 Thecriticalstrainisthetransversestrainontherstply(direction2inFigure 4-6 ),theeectofotherstrainsontheprobabilityoffailurebeingnegligible.Hence,thelimit-stateisdenedasthedierencebetweenthecriticalstrainandthefailurestrain: G="2)]TJ /F4 11.955 Tf 11.95 0 Td[("U2(4{16) Then,theprobabilityoffailureisdenedinEq. 4{5 usingthedistributionFGofthelimit-state. Inordertodeterminewhichdistributiontypetsthebestthelimit-stateG,wegenerated1,000samplesateachofthethreerstoptimumdesigns.UsingaLillieforstest[ Kenney&Keeping ( 1954 )],wefoundthatallthesamplesbelongtoanormaldistribution.Hence,weassumedthatthelimit-stateGisnormallydistributedforanydesign. Onemightprefernottoassumeasingledistributiontypeoverthedesigndomainandtest,everytimeanewdesignisconsidered,severaldistributionstondtheonethattsbestthedata.Suchaprocedureisnecessaryforinstancewhenthelimit-statedistributionvariesfromnormaltoheavy-tailwithinthedesigndomain;assumingasingledistributioncanleadtolargeerrorinthereliabilityestimation.Moregenerally,onehastokeepinmindthatthemethodproposedherecanbecomehazardousifthedeterminationofthedistributiontypeisuncertain. Inourcase,sincethethreedesignsconsideredarequitedierentfromoneanotherandhavethesamelimit-statedistribution,itisreasonabletoassumethatthelimit-statedistributionisalwaysnormal,hencereducingthecomputationalburden. 81

PAGE 82

Thereliability-basedoptimizationreplacestheconstraintsonthestrainsinEq. 4{15 byaconstraintontheprobabilityoffailure.Thetargetreliabilityofthecryogenictankischosenas10)]TJ /F5 7.97 Tf 6.58 0 Td[(4.Sincetheprobabilityoffailurecanhaveavariationofseveralordersofmagnitudefromonedesigntoanother,itispreferabletosolvetheproblembasedonthereliabilityindex: Minimizet1;t2;1;2h=4(t1+t2)s:t:t1;t20:127(t1;t2;1;2))]TJ /F1 11.955 Tf 21.92 0 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(10)]TJ /F5 7.97 Tf 6.59 0 Td[(4=3:719(4{17) 4.7.3ReliabilityBasedOptimizationUsingConservativeEstimates BysolvingEq. 4{17 withasampling-basedestimateofreliability,wefacetheproblemofhavingnoiseintheconstraintevaluation,whichcanseverelyharmtheoptimizationprocess.Toaddressthisissue,wechosetotapolynomialresponsesurface(PRS)toapproximatethereliabilityindexeverywhereonaregionofthedesignspaceandsolvetheoptimizationwiththeresponsesurface.Wehavethentwolevelsofsurrogatemodeling:onesurrogatetoapproximatethereliabilityindexofagivendesign,andonesurrogatetoapproximatethereliabilityindexforallthedesignregion. ThePRSisbasedontheestimationofthereliabilityindexforaselectednumberofdesigns:f1;:::;pg.Alltheestimatesaredonebeforetheoptimizationprocess.TherangeoftheresponsesurfaceisgiveninTable 4-6 .Thenumberoftrainingpointsistakenas500;thepointsaregeneratedusingLatinhypercubesamplingtoensureagoodspace-llingproperty.Eachestimateofthereliabilityindexisbasedonthesamen=200samples,whichgivesusatotalnumberof100,000simulationstottheresponsesurface. Table4-6. Variablerangeforresponsesurface. VariablesRange 1,2(deg)[2030]t1,t2(mm)[0.1270.800] 82

PAGE 83

Withthe3,000simulationsusedtodeterminethedistributionofthelimitstate,thetotalnumberofsimulationsis103,000.ThisnumberisareasonabletotalbudgetforsolvingtheRBDOproblem:indeedthereisnofurthersimulationrunduringtheoptimization,thereliabilitybeingestimatedbythePRS.TocomparethisnumbertoclassicalMCSestimates,inordertoachievereasonablenoiseforaprobabilityoffailureof104,atleast106samplesareneededforasingleevaluation,andthereliabilityisreestimatedateachoptimizationstep.ItismorediculttocomparetoFORMandSORMmethods,butduetothelargenumberofrandomparametersintheproblem,theywouldnotbeecient. Thereliabilityindexesarecalculatedusingtwomethods:theunbiasedttingandthebootstrapconservativeestimationwith95%condence.Theestimatesaredenotedrespectively^(i)unband^(i)cons.Figure 4.6 B)showsthatwith200samples,thecondenceintervaloftheconservativeestimatorforaprobabilityof10)]TJ /F5 7.97 Tf 6.58 0 Td[(4is[10)]TJ /F5 7.97 Tf 6.58 0 Td[(4;810)]TJ /F5 7.97 Tf 6.59 0 Td[(4],whichcorrespondstoanerrorinbetween0and20%.Sincethenumberofobservationsismuchlargerthanthenumberofcoecients(500comparedto70),thenoiseislteredbythePRS. Adierentresponsesurfaceisttedtoeachsetofdata.Forboth,wefoundthatafourthorderpolynomialprovidedagoodapproximationforthereliabilityindex.Table 4-7 showsthestatisticsofeachresponsesurface.BothR2values(percentageofvarianceexplained)areveryclosetoone,andp-valuesshowthatbothmodelsareverysignicant. Inaddition,wecomputedaccurateestimatesofthereliabilityindexat22designsuniformlychoseninthedesignspaceusingseparableMonte-Carlomethod(SMC)[ Smarsloketal. ( 2008 )]with40,000samplesforeachdesign.Table 4-8 showsthestatisticsbasedonthetestpoints:therootmeansquareerror(eRMS)betweenthe22accurateresponsesandtheresponsesurface,errormeanandnumberofunconservativepredictions.Basedonthetestpoints,wecanseethattherstPRSisunbiased:theerrormeanisapproximatelyzeroandtherearetenunconservativeerrorsfortwelveconservative.On 83

PAGE 84

theotherhand,thesecondPRShaslargebiassincetheerrormeanis0.34andallthepredictionsattestpointsareconservative. Table4-7. StatisticsofthePRSforthereliabilityindexbasedontheunbiasedandconservativedatasets. DatasetsR2Fp-value n^(1)unb;:::;^(p)unbo0.96138<10)]TJ /F5 7.97 Tf 6.58 0 Td[(6n^(1)cons;:::;^(p)conso0.96136<10)]TJ /F5 7.97 Tf 6.58 0 Td[(6 Table4-8. StatisticsofthePRSbasedon22testpoints DatasetseRMSErrormeanNbofunconservativepredictions n^(1)unb;:::;^(p)unbo0.081-0.0110n^(1)cons;:::;^(p)conso0.354-0.340 Remark1 .Sincewehavetwolevelsofsurrogatemodels,itispossibletoapplyconservativestrategytoeitheroneonethem,orbothofthem,byapplyingtothePRSoneofthemethodsproposedinChapter3.Here,wefoundthatusingaconservativestrategyonlyonthereliabilityindexestimationwassucienttoguaranteesafedesign,andmoreecientthanusingunbiasedestimationofandconservativePRS. Remark2 .ThebalancebetweenthenumberofMCSbydesignnandthenumberofdesignpointspisanopenquestion.Here,wechoseasmallnumberofMCSsincewewereabletocompensateecientlyforit.Chapter7dealswiththisquestionandprovidetheoreticalanswerstochoosethisbalanceforPRSandKriging. 4.7.4OptimizationResults Wepresenttheresultsforthetwoprobabilisticoptimizationsbasedontheresponsesurfacesttedonunbiasedandconservativeestimates.Tocomparedeterministicandprobabilisticapproaches,thedeterministicoptimizationasstatedinEq. 4{15 isalsoperformedforthesamerangeoftheplyangles([2030]degrees).TheoptimizationisperformedusingMATLAB'sfunctionfminconrepeated20timeswithdierentinitialpointstoensureglobalconvergence.Theoptimaldesigns(bestoverthe20optimizations) 84

PAGE 85

aregiveninTable 4-9 .Forthesedesigns,anaccurateestimateoftheprobabilityoffailureiscomputedusingSMCwith40,000samples. Table4-9. Optimaldesignsofthedeterministicandprobabilisticproblemswithunbiasedandconservativedatasets. t1t212hfromActualPffromActualPf PRSPRS(SD)* Unbiased0.1270.50722.230.02.5363.723.6110-41.51e-4 dataset(2.28e-6) 95%cons.0.1270.58221.930.02.8353.723.9810-43.50e-5 dataset(6.3e-7) Determ.0.1270.41620.030.02.170x2.97x15.0e-4 optima(9.6e-6) Thethreeoptimaaresimilarintermsofplyangles,andforalltherstplythicknesst1isthelowerbound;thesignicantdierenceisinthesecondplythicknesst2.Bothprobabilisticdesignsareheavierthanthedeterministicoptimum.Theoptimumfoundusingunbiaseddatasetissubstantiallylighterthantheother(h=2:54comparedto2.84);however,theaccurateestimateofreliabilityshowsthattheoptimumdesignusingunbiaseddatasetviolatestheconstraint.Ontheotherhand,thedesignfoundusingconservativedatasetisconservative;theactualprobabilityis3timessmallerthanthetargetprobabilityoffailure.Thedeterministicdesignisveryunconservative,itsprobabilityoffailurebeing15timesthetarget. Thefactthatanunbiasedstrategyleadstoanunconservativedesignisnotsurprising.Indeed,optimizationisbiasedtoexploreregionswheretheerroris'favorable';thatis,wheretheconstraintisunderestimated.Usingtheconservativeapproach,thelevelofbiasissucienttoovercomethisproblem,butatthepriceofoverdesign:sincetheprobabilityoffailureisthreetimesthetarget,theactualoptimumislighterthantheonewefound. Wehaveshownthatdespiteaverylimitedcomputationalbudget(103,000MCStosolvetheRBDOproblem),itwaspossibletoobtainareasonabledesignbycompensatingthelackofinformationbytakingconservativeestimates. 85

PAGE 86

4.8ConcludingComments Theestimationoftheprobabilityoffailureofasystemiscrucialinreliabilityanalysisanddesign.Inthecontextofexpensivenumericalexperiments,orwhenalimitednumberofdatasamplesareavailable,thedirectuseofMonteCarloSimulationisnotpractical,andestimationofcontinuousdistributionsisnecessary.However,classicaltechniquesofestimationofdistributiondonotpreventdangerousunderestimatesoftheprobabilityoffailure. Severalmethodsofestimatingsafelytheprobabilityoffailurebasedonthelimitednumberofsamplesaretested,whenthesampledistributiontypeisknown.Therstmethodconstrainsdistributionttinginordertobiastheprobabilityoffailureestimate.Thesecondmethodusesthebootstraptechniquetoobtaindistributionsofprobabilityoffailureestimators,andthesedistributionsdeneconservativeestimators. Inthecaseofsamplesgeneratedfromnormaldistribution,thenumericaltestcaseshowsthatbothmethodsimprovethechanceoftheestimationtobeconservative.BootstrapbasedestimatorsoutperformedconstrainedtstotheexperimentalCDF.Thatis,forthesamecondenceintheconservativenessoftheprobabilityestimate,thepenaltyintheaccuracyoftheestimatewassubstantiallysmaller.However,optimizationbasedmethodscanbeusedwhenthesamplesizeisverysmall,wherethebootstrapmethodcannotbeused. Wealsoexploredtheinuenceofsamplesizesandtargetprobabilityoffailureonestimatesquality.Wefoundthatlargersamplesizesarerequiredtoavoidlargevariabilityinprobabilityoffailureestimateswhenthatprobabilityissmall.Theresultsindicatedthatwhensamplingatdierentpointsindesignspaceitmaybemorecosteectivetohavedierentnumberofsamplesatdierentpoints.SuchapproachisaddressedinChapter6. Finally,wehaveappliedtheconservativeestimationprocedurestoperformtheoptimizationofacompositelaminatesatcryogenictemperatures.Wecomparedthe 86

PAGE 87

optimizationresultsfoundwhereresponsesurfacesarettedtounbiasedandconservativeestimatesrespectively.Wefoundthattheunbiasedresponsesurfacesledtounsafedesigns,whiletheconservativeapproachreturnedanacceptabledesign. 87

PAGE 88

CHAPTER5DESIGNOFEXPERIMENTSFORTARGETREGIONAPPROXIMATION Theaccuracyofmetamodelsiscrucialfortheeectivenessoftheiruseinprediction,designorpropagationofuncertainty.InChapters3and4,weproposedsomealternativestocompensatefortheerrorinsurrogatemodels.Inthischapter,weaimatminimizingtheuncertaintyofthemodels.Thedesignofexperimentstrategyisoneofthemostimportantfactorsthataectthequalityofthemetamodel.Inmanypracticalproblems,thetotalnumberoffunctionevaluationsisdrasticallylimitedbycomputationalcost;hence,itisofcrucialinteresttodevelopmethodsforselectingecientlytheexperiments. Samplingstrategieshavebeenextensivelystudiedoverthepastdecades.InChapter2,wedescribedsomeofthemostpopularstrategiesthataimatoptimizingtheglobalaccuracyofthemetamodels.Thescopeofthepresentresearchistoproposeanobjective-basedapproachtodesignofexperiments,basedontheideathattheuncertaintyinsurrogatesmaybereducedwhereitismostuseful,insteadofglobally.Inparticular,weaimatndingecientstrategiesfortwoapplicationsofmetamodeling:reliabilityestimationandconstrainedoptimization. 5.1MotivationAndBibliography Mostsamplingstrategiesaredevelopedtoachievethebestglobalperformancesofthemetamodel.Suchapproachisfullyjustiedwhenthemodelisusedforpredictionoverallthedomainofinterest,inferenceofglobalbehaviorofthephenomenonstudied,orwhenallthedesignofexperimentisgeneratedatthesametimewithoutpriorinformationontheresponsebehavior,forinstancewhenseriesofphysicalexperimentsorparallelcomputationareinvolved. However,manymetamodelapplicationsfocusonthevalueofthefunctionresponse.Forinstance,whenametamodelisusedforoptimizationpurpose,itisobviousthatthemetamodelaccuracyismuchmoreimportantforlowvaluesoftheresponseratherthan 88

PAGE 89

largevalues.Hence,globalstrategiesdonotappearasthemostecientresponsestosuchapplication. Thepresentresearchfocusesonapplicationswheremetamodelsareusedinawaythattheiraccuracyiscrucialforcertainlevel(s)oftheresponsefunction.Suchsituationappearsinparticularintwopopularframeworks: Inconstrainedoptimization .Theconstraintfunctionoftenreliesonexpensivecalculations.Forinstance,atypicalstructuraloptimizationformulationistominimizeaweightfunctionsuchthatthemaximumstress,computedbyniteelementanalysis,doesnotexceedacertainvalue.Whenusingametamodeltoapproximatetheconstraint,itisofutmostimportancethattheapproximationerrorisminimalontheboundarythatseparatesthefeasibledesignsfrominfeasibleones.Substantialerrorsforvaluesfarfromtheboundary,ontheotherhand,arenotdetrimental. Inreliabilityanalysis .Weconsiderthecasewherethemetamodelisusedtoapproximatethelimitstatefunctionofasystem,G(x).Aswellasforconstraintapproximation,themetamodeldiscriminatesbetweensamplesthatarebelowandaboveacriticalvalue,inthiscasezero.Whenthelimitstateisfarfromzero,thechanceofmisclassicationisnegligibleevenifthemetamodelpoorlyapproximatethetruefunction,whiletheaccuracyiscrucialforvaluesclosetozero. Thepresentchapterpresentsamethodologytoconstructadesignofexperimentssuchthatthemetamodelaccuratelyapproximatesthevicinityofaboundaryindesignspacedenedbyatargetvalueofthefunctionapproximatedbythesurrogate.Thisproblemhasbeenaddressedbyseveralauthors. Mourelatosetal. ( 2006 )usedacombinationofglobalandlocalmetamodelstorstdetectthecriticalregionsandthenobtainalocallyaccurateapproximation. Shan&Wang ( 2004 )proposedaroughsetbasedapproachtoidentifysub-regionsofthedesignspacethatareexpectedtohaveperformancevaluesequaltoagivenlevel. Oakley ( 2004 )usesKrigingandsequentialstrategiesforuncertaintypropagationandestimationofpercentilesoftheoutputofcomputercodes. Ranjanetal. 89

PAGE 90

( 2008 )proposedamodiedversionoftheEGOalgorithmtosequentiallyexplorethedomainregionalongacontourline. Tu&Barton ( 1997 )usedamodiedD-optimalstrategyforboundary-focusedpolynomialregression. Vazquez&Piera-Martinez ( 2006 )proposedaniterativestrategytominimizetheclassicationerrorwhencomputingaprobabilityoffailurebasedonKriging. 5.2ATargetedIMSECriterion Inthischapter,weconsideronlyprobabilisticmetamodels-thatassociateadistributiontoapredictionpointinsteadofasinglevalue.Krigingisexplicitlyaprobabilisticmetamodel,butmostoftheothermodelscanhaveaprobabilisticinterpretation,suchasregressionandsplinesforinstance.However,welimitourinvestigationtothekrigingmodelhere. WegavethedenitioninSection 2.2.2 oftheIMSEcriterion,whichsumsuptheuncertaintyassociatedwiththeKrigingmodelovertheentiredomainD.However,whenoneismoreinterestedinpredictingyaccuratelyinthevicinityofacontourlineu=y)]TJ /F12 7.97 Tf 6.59 0 Td[(1(T)(Taconstant),suchacriterionisnotsuitablesinceitweightsallpointsinDaccordingtotheirKrigingvariance,whichdoesnotdependontheobservationsYobs,andhencedoesnotfavorzoneswithrespecttopropertiesconcerningtheiryvaluesbutonlyonthebasisoftheirpositionwithrespecttoDoE. WeproposetochangetheintegrationdomainfromDtoaneighborhoodofy)]TJ /F5 7.97 Tf 6.59 0 Td[(1(T).Twoapproachesareproposedthatdierbythedenitionoftheneighborhoodofthecontourline.Theneighborhoodisdenedwiththehelpofanindicatorfunctionandagaussiandistribution,respectively,whichcanbeencounteredintheliteratureunderthedenominationofdesirabilityfunctions. 5.2.1TargetRegionDenedByAnIndicatorFunction WedenearegionofinterestXT;"(parameterizedby")asthesubsetinDwhoseimageiswithintheboundsT)]TJ /F4 11.955 Tf 11.96 0 Td[("andT+": XT;"=y)]TJ /F5 7.97 Tf 6.59 0 Td[(1([T)]TJ /F4 11.955 Tf 11.95 0 Td[(";T+"])=fx2Djy(x)2[T)]TJ /F4 11.955 Tf 11.95 0 Td[(";T+"]g(5{1) 90

PAGE 91

Figure 5-1 illustratesaone-dimensionalfunctionwiththeregionofinterestbeingatT=0:8and"=0:2.Notethatthetargetregionconsistsoftwodistinctsets. Figure5-1. One-dimensionalillustrationofthetargetregion.Thelevel-setTisequalto0.8,and"to0.2.Thetargetregionconsistsoftwodistinctsets. Withtheregionofinterest,thetargetedIMSEcriterionisdenedasfollows: imseT=ZXT;"s2K(x)dx=ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][y(x)]dx(5{2) where1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][y(x)]istheindicatorfunction,equalto1wheny(x)2[T)]TJ /F4 11.955 Tf 12.02 0 Td[(";T+"]and0elsewhere. FindingadesignthatminimizesimseTwouldmakethemetamodelaccurateinthesubsetXT;",whichisexactlywhatwewant.WeightingtheIMSEcriterionoveraregionofinterestisclassicalandproposedforinstanceby Box&Draper ( 1986 ),pp.433-434.However,thenotabledierencehereisthatthisregionisunknownapriori. Now,wecanadaptthecriterioninthecontextofKrigingmodeling,whereyisarealizationofarandomprocessY(seeSection 2.1.3 ). 91

PAGE 92

Thus,imseTisdenedwithrespecttotheevent!: ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]dx=I(!)(5{3) Tocomebacktoadeterministiccriterion,weconsidertheexpectationofI(!),conditionalontheobservations(whichisthebestapproximationintheL2sense): IMSET=E[I(!)jobs]=E24ZDs2K(x)1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]dxjobs35(5{4) Sincethequantityinsidetheintegralispositive,wecancommutetheexpectationandtheintegral: IMSET=ZDs2K(x)E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][Y(x;!)]jobsdx (5{5) =ZDs2K(x)E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][M(x)]dx (5{6) =ZDs2K(x)W(x)dx (5{7) AccordingtoEq. 5{7 ,thereducedcriterionistheaverageofthepredictionvarianceweightedbythefunctionW(x).Besides,W(x)issimplytheprobabilitythattheresponseisinsidetheinterval[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"].Indeed: W(x)=E1[T)]TJ /F6 7.97 Tf 6.59 0 Td[(";T+"][M(x)] (5{8) =P)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(M(x)2[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"] (5{9) =P)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(Y(x)2[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"]jobs (5{10) TheGaussianprocessinterpretationofKrigingadoptedhere(seeSection 2.1.3 )allowsustoexplicitlyderivetheconditionaldistributionofM(x)=(Y(x)jobs)atanypredictionpoint: Foranyx2D:M(x)N)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(mK(x);s2K(x)(5{11) 92

PAGE 93

WecanobtainasimpleanalyticalformforW(x): W(x)=T+"ZT)]TJ /F6 7.97 Tf 6.59 0 Td[("gN(mK(x);s2K(x))(u)du(5{12) wheregN(mK(x);2K(x))(u)istheprobabilitydensityfunction(PDF)ofM(x).ByintegratingthePDFweobtain: W(x)=T+")]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x))]TJ /F1 11.955 Tf 11.96 0 Td[(T)]TJ /F4 11.955 Tf 11.96 0 Td[(")]TJ /F4 11.955 Tf 11.96 0 Td[(mK(x) sK(x)(5{13) whereistheCDFofthestandardnormaldistribution. Remark .Itispossibletodeneaweightfunctionwith"!0.Todoso,wehavetodividerstW(x)bytheconstant2".Then,weobtain: lim"!01 2"T+")]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x))]TJ /F1 11.955 Tf 11.96 0 Td[(T)]TJ /F4 11.955 Tf 11.96 0 Td[(")]TJ /F4 11.955 Tf 11.96 0 Td[(mK(x) sK(x)='T)]TJ /F4 11.955 Tf 11.95 0 Td[(mK(x) sK(x)(5{14) where'istheprobabilitydensityfunction(PDF)ofthestandardnormaldistribution. 5.2.2TargetRegionDenedByAGaussianDensity DeningtheregionofinterestasXT;isintuitiveandmakesiteasytoderivetheweightfunction.However,itmightnotalwayscorrespondexactlytoourobjective.Indeed,ifweconsideranidealcasewherethefunctionisexactlyknown,theindicatorfunctionwillyieldaweight1toapointxwhereM(x))]TJ /F4 11.955 Tf 12.49 0 Td[(T=",but0ifM(x))]TJ /F4 11.955 Tf 12.5 0 Td[(T="+10)]TJ /F5 7.97 Tf 6.58 0 Td[(9.Also,itwillnotdiscriminatebetweenapointwherethedierenceisequalto"andanotheronewherethisdierenceisequaltozero. Instead,wepreferacriterionthatcontinuouslyincreasestheimportanceofthelocationwhentheresponseapproachesthethreshold.Forinstance,wecanchooseatriangularfunction(withamaximumatT)orasigmoidfunction.Here,wechoosetousetheprobabilitydensityfunctionofanormaldistributionwhichleadstoasimpleanalyticalformoftheweightfunction.InthespiritofEq. 5{8 ,theGaussian-basedweightfunction 93

PAGE 94

isthereforedenedasfollows: W(x)=E[g"(M(x))]TJ /F4 11.955 Tf 11.96 0 Td[(T)](5{15) whereg"(u)isthePDFofN(0;2"). WhenM(x)standsfortheKrigingmodel,wecanobtainasimpleformfortheweightfunction: W(x)=+1Zg"(u)]TJ /F4 11.955 Tf 11.95 0 Td[(T)gN(mK(x);s2K(x))(u)du(5{16) Sinceg"issymmetric: W(x)=+1Zg"(T)]TJ /F4 11.955 Tf 11.95 0 Td[(u)gN(mK(x);s2K(x))(u)du(5{17) ThisintegralistheconvolutionofthetwoGaussiandensities,whichiswell-knowntobethedensityofasumofindependentGaussianvariables.Hence,weobtain: W(x)=1 p 2(2"+s2K(x))e)]TJ /F13 5.978 Tf 7.79 3.26 Td[(1 2(mk(x))]TJ /F14 5.978 Tf 5.76 0 Td[(T)2 2"+s2K(x)(5{18) Thisnewweightfunctiondependsonasingleparameterthatallowsustoselectthesizethedomainofinterestaroundthetargetlevelofthefunction.Alargevalueofwouldenhancespace-lling,sincetheweightfunctionwouldtendtoaconstantandtheweightedIMSEtoauniformIMSEcriterion.Onthecontrary,asmallvaluewouldenhancetheaccuracyofthesurrogateonanarrowregionaroundthecontourlineofinterest. 5.2.3Illustration Weconsideraone-dimensionalcase,wherethefunctionytoapproximateisarealizationofaGaussianprocesswithGaussiancovariancestructure.yisdenedon[0;1];thedesignofexperimentsconsistsofveobservationsequallyspacedinthisinterval.Thelevel-setofinterestTischosenas1.3,andboth"and"aretakenas0.2.Figure 5-2 representsthetruefunction,theKrigingmetamodelandcorrespondingweights.The 94

PAGE 95

weightfunctioninEq. 5{13 isshownas"interval",whilethatinEq. 5{18 isshownas"Gaussian"inthegure. Figure5-2. Illustrationoftheweightsfunctions.Uppergraph:truefunction,observations,Krigingmeanandcondenceintervals,andtargetregion.Lowergraph:weightfunctions.Bothweightsarelargewherethetruefunctionisinsidethetargetregion,butalsosignalingregionsofhighuncertainties(aroundx=0:65and0:85). Amongtheveobservations,oneissubstantiallyclosertoTthantheothers.Asaconsequence,theweightfunctionsarelargearoundthisobservationpoint.Fortheindicator-basedweightfunction,theweightsarenullattheobservationpoints,sinceforthisexamplenoobservationisinsidethetargetvalueinterval.FortheGaussian-basedweight,itisalsoveryclosetozero.Forbothfunctions,highweightsaregiventoregionsforwhichtheactualfunctionisinsidethetargetinterval.Bothweightfunctionsarealsonon-zerowheretheuncertaintyishigh,eveniftheKrigingmeanisfarfromT(aroundx=0:65and0:85). 5.3SequentialStrategiesForSelectingExperiments Withoutanyobservation,theweightfunctionW(x)is,assumingstationarity,aconstant(theprobabilityisthesameeverywhere).Everytimeanewobservationisperformed,theweightfunctionwillmorepreciselydiscriminatetheregionsofinterestfrom 95

PAGE 96

theothers.Hence,theproceduretobuildanoptimalDoEisnecessarilyiterative.IfweaddoneobservationatatimewecanusetheprocedureshowninTable 5-1 Table5-1. ProcedureoftheIMSET-basedsequentialDoEstrategy. CreateaninitialDoE,Xk,andgenerateobservationsYk=y(Xk)Forigoingfromonetothetotalnumberofadditionalobservationsn:FittheKrigingmodeltothedatafXk+i)]TJ /F5 7.97 Tf 6.58 0 Td[(1;Yk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1gFindanewtrainingpointxnewthatminimizesthecriterionIMSET(fXk+i)]TJ /F5 7.97 Tf 6.58 0 Td[(1;xnewg)Computethenewobservationynew=y(xnew)UpdatetheDoEandobservations:Xk+i=fXk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1;xnewgYk+i=fYk+i)]TJ /F5 7.97 Tf 6.59 0 Td[(1;ynewgEndofloop TheKrigingparameterscanbereevaluatedaftereverynewobservation,oronlyfromtheinitialDoEbeforetheiterativeprocedure.NotethattheevaluationoftheparametersiscriticaltoobtainagoodKrigingmodel.Besides,theparametersaectthecriterion:underestimationoftherangemakestheweightfunctionatterandenhancesspace-lling;onthecontrary,overestimationoftherangeleadstoaverydiscriminating(over-condent)weightfunction.However,inthenumericalexamplesusedinthiswork,wefoundthattheparameterre-evaluationhadanegligibleimpactontheeciencyofthemethod. Findingthenewtrainingpointrequiresaninneroptimizationprocedure.WhentheclassicalIMSEcriterionisconsidered,theoptimizationcanbeexpressedas: minxnew2DIMSE(Xk+1)=IMSE(fXk;xnewg)(5{19) whereIMSE(fXk;xnewg)=ZDs2K(xjfXk;xnewg)dx s2K(xjfXk;xnewg)isthevarianceatxoftheKrigingbasedonthedesignofexperimentsXaugmentedwiththetrainingpointxnew.SincetheKrigingvariancedoesnotdependontheobservation,thereisnoneedtohavey(xnew)tocomputetheIMSE. 96

PAGE 97

Incontrast,theweightedIMSEdependsontheobservationsthroughtheweightfunctionW(x).Theweightfunctioncannottakeintoaccountthenewobservation,sincetheresponseisnotavailable.Hence,whenexpressingtheweightedIMSEasafunctionofxnew,weupdateonlythevariancepartundertheintegral: IMSET(Xk;Yk;xnew)=ZDs2K(xjfXk;xnewg)W(xjXk;Yk)dx(5{20) wheres2K(xjfXk;xnewg)isthesameasinEq. 5{19 andW(xjXk;Yk)istheweightfunctionbasedontheexistingDoE.Usingthisexpression,wehavethesimpleformulationfortheinneroptimizationproblem: minxnew2DIMSET(Xk;Yk;xnew)(5{21) 5.4PracticalIssues 5.4.1SolvingTheOptimizationProblem FindingthenewobservationxnewbysolvingtheoptimizationproblemofEq. 5{21 is,inpractice,challenging.Indeed,theIMSETcriterioninEq. 5{20 mustbeevaluatedbynumericalintegration,whichiscomputationallyintensive.Besides,foranycandidatexnew,theKrigingmodelmustbereevaluatedwiththisnewobservationtoobtains2K(xjfXk;xnewg)).Thereforeweproposeheresomealternativesthatmaybeusedtoreducethecost. ApopularheuristictominimizesequentiallytheIMSEistondthepointwherethepredictionvarianceismaximum[ Sacksetal. ( 1989a ), Williamsetal. ( 2000 )],whichcanbeusedherewiththeweightedpredictionvariance.Thisstrategyhastheadvantageofsavingboththenumericalintegrationandtheinversionofanewcovariancematrix.However,thepredictionvarianceislikelytohavemany(localorglobal)maximizers,whicharenotequivalentintermsoftheIMSE.Inparticular,manyoptimaarelocatedontheboundaries,whichisveryinecientfortheIMSEminimization.Tocompensateforthisissue,onemayinarsttimegetalargenumberoflocaloptimausingadapted 97

PAGE 98

optimizationstrategies(multi-start,etc.),andinasecondtimeevaluatethoseoptimaintermsoftheweightedIMSEcriterion,andperformalocaloptimizationonthebestpoint. Avaluablecomputationalshortcutcanbeachievedintheupdateoftheinverseofthecovariancematrixwhenaddinganobservation.LetuscallCkthecovariancematrixcorrespondingtoaDoEwithkobservations.Then,thecovariancematrixoftheDoEaugmentedwiththek+1thobservationcanbewritten: Ck+1=2642cTnewcnewCk375(5{22) with: 2=k(xnew;xnew)(processvariance),and cTnew=[k(xnew;x1);:::;k(xnew;xk)]a1kvector. UsingSchur'scomplementformula,weget: Ck+1)]TJ /F5 7.97 Tf 6.58 0 Td[(1=26410)]TJ /F7 11.955 Tf 9.3 0 Td[(C)]TJ /F5 7.97 Tf 6.58 0 Td[(1kcnewIk3752641 2)]TJ /F12 7.97 Tf 6.58 0 Td[(cTnewC)]TJ /F13 5.978 Tf 5.75 0 Td[(1kcnew00C)]TJ /F5 7.97 Tf 6.59 0 Td[(1k3752641)]TJ /F7 11.955 Tf 9.3 0 Td[(C)]TJ /F5 7.97 Tf 6.58 0 Td[(1kcnew0Ik375(5{23) ThisformulaallowstocomputeCk+1)]TJ /F5 7.97 Tf 6.58 0 Td[(1fromCk)]TJ /F5 7.97 Tf 6.59 0 Td[(1withoutdoinganymatrixinversion,andcomputes2K(xjfXk;xnewg)atreasonablecost.Moredetailedcalculationsonthistopiccanbefoundin Marrel ( 2008 ). Ingeneral,thecriterionhasseverallocalminimizers.Then,itisnecessarytouseglobaloptimizationmethods,suchaspopulation-basedmethods,multi-startstrategies,etc.Inthetestproblemspresentedinthischapter,weoptimizethecriteriononanegridforlowdimensions,andusingthepopulation-basedCMA-ESalgorithm[CovarianceMatrixAdaptationEvolutionStrategies, Hansen&Kern ( 2004 )]forhigherdimensions.Experimentationshowedthatduetothenumericalintegrationprecision,thetargetedIMSEstrategybecomesinecientfordimensionshigherthanten. 98

PAGE 99

5.4.2Parallelization Itisoftenofinteresttoproducealgorithmsthatallowtocomputeseveralobservationsatatime,forinstancewhenthenumericalsimulatorcanberuninparallelonseveralcomputers.Thecriterionweproposeisfullyadaptabletoparallelstrategies.Indeed,thetargetedIMSEconsistsoftwoparts:theweightfunctionthatdependsontheresponsevalues,andthepredictionvariance,thatonlydependsontheobservationlocations.Thelaterquantitycanbeupdatedwithseveralobservationswithouttheneedofcalculatingtheresponse.Theweightfunction,ontheotherhand,remainsthesame. Letpbethenumberofobservationstobecomputedinparallel.Thecriteriontobeminimizedcanbewritten: IMSET)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(Xk;Yk;fx(1)new;:::;x(p)newg=ZDs2K)]TJ /F7 11.955 Tf 6.67 -9.68 Td[(xjXk;fx(1)new;:::;x(p)newgW(xjXk;Yk)dx(5{24) Thisstrategyislessaccuratethanthesequentialstrategy,sincetheweightfunctionisupdatedonlyeverypevaluations.However,weobservedthatforlowvaluesofp(5),theparallelizationhaslittleimpact,inparticularwhenthekriginguncertaintyisinitiallyhigh. Theparalleloptimizationproblemisofdimensiondp(wheredisthedesignspacedimension).ThedimensionofthenumericalintegrationremainsthesameasinEq. 5{20 .Theoptimizationcanbesolveddirectly,or,inordertoreducetheproblemdimension,byndingonepointatatime. 5.5ApplicationToProbabilityOfFailureEstimation Asmentionedinthebeginningofthischapter,whenapproximatingthelimit-state,itisclearthataparticulareortmustbegiventotheregionswhereitisclosetozero,sinceerrorinthatregionislikelytoaecttheprobabilityestimate.Naturally,thecriticalregioniswherethevalueoflimitstateisclosetozero. ThetargetedIMSEcriterionallowsustosampleintargetregions.However,substantialimprovementcanbegiven,bytakingintoaccountthedistributionofthe 99

PAGE 100

inputvariables.Indeed,letusconsiderthecaseoftwodistinctfailureregions,oneofitdominatingtheother(thatis,theprobabilitythattheinputfallsontotherstregionismuchlargerthantheprobabilitythatitfallsontotheother).Insteadofexploringequallythetwocriticalregions,itwillbemoreecienttospendmorecomputationaleortontheonethatwillaectmosttheprobabilityestimate.Inthesamesense,whenlearningasinglecriticalregion,itisecienttoexploreitonlywherethesamplesaremorelikelytobe. InsteadofusingtheindicatorfunctionontheKrigingmean,weusethefullKriginginformationbycomputing,ateachsamplingpoint,theprobabilitythattheresponseexceedsthethreshold: ^Pf=1 NNXi=1(i)k(0)(5{25) where(i)kdenotesthecumulativedistributionfunction(CDF)oftheKrigingmodelatxi(N(mk(xi);s2k(xi))). IftheKrigingvarianceissmall,theCDFbecomesequivalenttotheindicatorfunction,being1iftheKrigingmeanexceedsthethresholdand0otherwise.Ontheotherhand,whenthevarianceishighorthepredictedresponseclosetothethreshold,usingtheKrigingdistributionoersasmoothingeectbygivinganumberbetweenzeroandoneinsteadofaBooleannumber. Toaddresstheprobabilitydistributionofinputvariables,wemodifytheweightedIMSEcriterionbyintegratingtheweightedMSEnotwithauniformmeasure,butwiththePDFoftheinputvariables: IMSET=ZDs2K(x)W(x)d(x)=ZDs2K(x)W(x)f(x)dx(5{26) wheref(x)isthePDFofthelawwithrespecttotheLebesguemeasure. 100

PAGE 101

5.6NumericalExamples Inthissection,weevaluatetheaccuracyandeciencyofthemethodthroughnumericalexamples.Weconsiderthettingofanalyticaltestfunctionsandrealizationsofrandomprocesseswithknowncovariancestructures.Inthelattercase,thereisnomodelingerrorwhenusingaKrigingapproximation;theerrorisonlyduetothelackofinformation. 5.6.1Two-DimensionalExample Therstexampleistheapproximationofatwo-dimensionalparametricfunctionfromtheoptimizationliterature(Camelbackfunction, Chippereldetal. ( 1994 )).Theoriginalfunctionismodied(boundsaredierentandanegativeconstantisadded)andthetargetischoseninordertohavetwofailureregions,onedominatingtheother.Thetwo-dimensionaldesignspaceisgivenas[1;1]2.Theperformancefunctionisdenedas f(u;v)=4)]TJ /F1 11.955 Tf 11.96 0 Td[(2:1u2+1 3u4u2+2 3uv+16 9)]TJ /F1 11.955 Tf 9.3 0 Td[(4+16 9v2v2)]TJ /F1 11.955 Tf 11.96 0 Td[(0:7(5{27) whereu=1:2u)]TJ /F1 11.955 Tf 11.96 0 Td[(0:1andv=0:9v. Forbothnumericalintegrationandoptimization,thedesignspaceisdividedbya3232grid.Wepresenttheresultsforthefollowingconguration: TargetvalueTischosenas1.3, Gaussian-basedweightfunctionisused,withparameter"=0:2, InitialDoEconsistsofthefourcornersandthecenterofthedomain, 11pointsareaddediterativelytotheDoEasdescribedintheprevioussection. AnisotropicGaussiancovariancefunctionischosenfortheKrigingmodel.Thecovarianceparameters(processvariance2andrange)areestimatedfromtheinitial5-pointDoE,andre-estimatedaftereachnewobservation,usingtheMatLabtoolboxGPML[ Rasmussen&Williams ( 2006 )].ThenalresultsarepresentedinFigure 5-3 Figure 5-3 A)istheplotofthetruefunction,andFigure 5-3 B)isthatoftheKrigingmean.InthecontourplotinFigure 5-3 C),itisshownthattherearetwocriticalregions. 101

PAGE 102

Figure5-3. Optimaldesignafter11iterations.A)Truefunction.B)Krigingexpectation.D)Krigingvariance.C)Contourlinesofthetruefunctionatlevels[T)]TJ /F4 11.955 Tf 11.96 0 Td[(";T+"],whichdelimitthetargetregions;theboldlinecorrespondstothetargetcontour.Mostofthetrainingpointsarechosenclosetothetargetregion.TheKrigingvarianceisverysmallintheseregionsandlargeinnon-criticalregions. After11iterations,thesequentialstrategyusedfourpointstoexploretherstcriticalregion,threepointstoexplorethesecondregion,andfourpointsforspace-lling.AsshowninFigure 5-3 D),theKrigingvariancebecomessmallnearthecriticalregions,whileitisrelativelylargeinthenon-criticalregion. Figure 5-4 showstheevolutionofthetargetcontourlineforthekrigingexpectation,whichisagoodindicatorofthequalityofthesurrogate.Weseethatbecausetherstfouriterations(Figure 5-4 B))areusedforspace-lling,theKrigingcontourlineisverydierentfromtheactualone.Aftereightiterations(Figure 5-4 C)),thetwotargetregionsarefoundandadditionalsamplingpointsarechosenclosetotheactualcontourline.Finalstate(Figure 5-4 D))showsthatthekrigingcontourlineisclosetotheactualone. 102

PAGE 103

Figure5-4. EvolutionofKrigingtargetcontourline(mixedline)comparedtoactual(plainline)duringthesequentialprocess:A)Initial,B)afterfouriterations,C)aftereightiterations,D)nal. 5.6.2Six-DimensionalExample Inthesecondexample,weconsiderarealizationofasix-dimensionalisotropicGaussianprocesswithGaussiancovariancefunction.Here,theerrorintheKrigingmodelisonlyduetothelackofdata:thereisnoerrorinestimatingthecovarianceparameters,andasymptotically(withverylargenumberofobservations)theKrigingtsexactlytherealizationoftheGaussianprocess.Thedesignspaceis[)]TJ /F1 11.955 Tf 9.3 0 Td[(1;1]6.Inordertolimitthecomplexity(numberofnon-connectedtargetregions)ofthetargetregion,weaddalineartrendtotheGaussianprocess.Wetake2=1,=0:1and=[1:::1]. TheweightedIMSEcriterioniscomputedbyQuasiMonte-Carlointegration.TheintegrationpointsarechosenfromaSobolsequence[ Sobol ( 1976 )]toensureagoodspacellingproperty.Ateachstep,theoptimizationisperformedusingthepopulation-basedoptimizerCMA-ES[ Hansen&Kern ( 2004 )]. 103

PAGE 104

Wepresenttheresultsforthefollowingcongurations: Targetvalueischosenas2 Gaussian-basedweightfunctionisused,with"=0:05. InitialDoEconsistsof20pointschosenfromLatin-hypercubesampling(LHS) 70pointsareaddediterativelytotheDoE. Forcomparisonpurpose,wegenerateaclassicalspace-llingDoEthatconsistsof90LHSpointswithmaximumminimumdistancecriterion. First,werepresenttheerrorat10,000(uniformlydistributed)datapoints(Figure 5-5 ).Theclassicalspace-llingDoEleadstoauniformerrorbehavior,whiletheoptimalDoEleadtolargeerrorswhentheresponseisfarfromthetargetvalue,whilesmallerrorswhenitisclosetothetarget. Inordertoanalyzetheerrorinthetargetregion,wedrawtheboxplotsoftheerrorsforthetestpointswhereresponsesareinsidethedomains[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+"](Figure 5-6 A))and[)]TJ /F1 11.955 Tf 9.3 0 Td[(2";+2"](Figure 5-6 B)).Comparedtothespace-llingstrategy,theoptimaldesignreducessignicantlytheerror.Inparticular,onbothgraphsthelower-upperquartilesintervalis2.5timessmallerfortheoptimalDoE. 5.6.3ReliabilityExample ThelimitstatefunctionistakenastheCamelbackfunctionusedintheprevioussection.LetUandVbeindependentGaussianvariableswithzeromeanandstandarddeviationtakenat0.28;i.e.,U;VN)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(0;0:282.Then,thefailureisdenedwhenfbecomesgreaterthan1.3.Thus,thelimitstateisdenedas G=f(U;V))]TJ /F1 11.955 Tf 11.95 0 Td[(1:3(5{28) Forthisexample,wegeneratetwoadaptivedesigns:therstisgeneratedsequentiallyasdescribedpreviously,withuniformintegrationmeasure(Eq. 5{7 );thesecondisgeneratedusingtheinputdistributionasintegrationmeasure(Eq. 5{26 ).BothusethefourcornersandthecenterofthedomainasstartingDoEand11iterationsareperformed.For 104

PAGE 105

Figure5-5. ComparisonoferrordistributionfortwoDoEs:A)optimalDoEandB)classicalLHS.Thex-axisisthedierencebetweenthetruefunctionandthethreshold,they-axisistheerror.Fiveverticalbarsaredrawnat)]TJ /F1 11.955 Tf 9.3 0 Td[(2",)]TJ /F4 11.955 Tf 9.29 0 Td[(",0,+"and+2"forthetargetregion.TheerrorisonaveragesmallerfortheLHSdesign,buttheoptimalDoEreducessubstantiallytheerrorinthetargetregion. comparisonpurpose,a16-pointfullfactorialdesignisalsoused.ItisfoundthataSimpleKrigingmodel(UKwithoutlineartrend)withisotropicGaussiancovariancefunctionapproximateswellthefunction.ThecovarianceparametersarecomputedusingthetoolboxGPMLforalltheDoEs. Figure 5-7 drawsthetwooptimaldesignsobtainedandthefullfactorialdesigns.Bothoptimaldesignsconcentratethecomputationaleortonthefailureregionsandthecenterofthedomain.WithuniformmeasureintegrationinFigure 5-7 A),theDoEismorespace-llingthantheonebasedonthedistribution(showninFigure 5-7 D)).BytakingtheinputdistributionintoaccountinFigure 5-7 B),weseethatalltheobservationsarelocatedrelativelyclosetothecenterofthedomain.Partofeachtargetregionsisnotexplored,sinceitisfarfromthecenter. Finally,weperform107MCSonthethreemetamodelstocomputetheprobabilityoffailureestimates.107MCSarealsoperformeddirectlyonthetestfunctiontoobtainthetrueprobabilityoffailure.ResultsarereportedinTable 5-2 .Thefull-factorial 105

PAGE 106

Figure5-6. and[)]TJ /F1 11.955 Tf 9.3 0 Td[(2";+2"].]BoxplotsoferrorsfortheLHSandoptimaldesignsforthetestpointswhereresponsesareinsidethedomainsA)[)]TJ /F4 11.955 Tf 9.3 0 Td[(";+"]andB)[)]TJ /F1 11.955 Tf 9.29 0 Td[(2";+2"].Erroratthesepointsissubstantiallysmallerfortheoptimaldesignsforbothintervals. designleadsto77%error,whilebothoptimaldesignsleadtoasmallerror.Substantialimprovementisobtainedbytakingtheinputdistributionintoaccount. Table5-2. ProbabilityoffailureestimatesforthethreeDoEsandtheactualfunctionbasedon107MCS.Thestandarddeviationofallestimatesisoftheorderof210)]TJ /F5 7.97 Tf 6.59 0 Td[(5. DoEFullFactorialOptimalwithoutinputdistributionOptimalwithinputdistributionProbabilityestimatebasedon107MCS Probabilityoffailure(%)0.170.700.770.75 Relativeerror77%7%3% 5.7ConcludingRemarks Inthischapter,wehaveproposedtoimprovetheaccuracyofsurrogatemodelsbygeneratingdesignsofexperimentsadaptedtothemodelapplication.Inparticular,wehaveaddressedtheissueofchoosingadesignofexperimentswhentheKrigingmetamodelwasusedtoapproximateafunctionaccuratelyaroundaparticularlevel-set,whichoccursinconstrainedoptimizationandreliabilityanalysis.Wedenedanoriginalcriterion,whichis 106

PAGE 107

Figure5-7. OptimaldesignwithA)uniformintegrationmeasure,B)inputdistributionintegrationmeasure;C)Fullfactorialdesignswith16points;D)Inputdistribution. basedonanexplicittrade-obetweenexplorationofregionsofinterestandreductionofuncertaintyofthemodel,andansequentialproceduretobuildtheDoEs. Throughnumericalexamples,weshowedrstthattheadaptiveDoEwaseectivefordetectingthetargetregions(wheretheresponsewasclosetothetargetlevel-set)andimprovelocallytheaccuracyofthesurrogatemodel.Then,themethodwasappliedtoareliabilityexample,anditwasfoundthatimprovinglocallytheaccuracyresultedingreatimprovementoftheaccuracyofthereliabilityestimation. However,ithasbeenfoundsomelimitationstothemethod,whichwerenotsolvedhereandrequiresfutureworktoapplythemethodtoawiderangeofproblems: 107

PAGE 108

Sinceitreliesonnumericalintegration,themethodcanbecomecomputationallyexpensiveifalargenumberofintegrationpointsisneededtocomputethecriterion.Wefoundthatfordimensionshigherthanten,thecriterionminimizationbecomescriticalwithouttheuseofcomplexandproblem-dependentnumericalprocedures,suchasdimensionreductionoradaptednumericalintegration. Secondly,itisimportanttorecallthatitisamodel-believerstrategy,sincethecriterionisentirelybasedontheKrigingmodel.Althoughsequentialstrategiesallowsomecorrectionofthemodelduringtheprocess(throughre-estimationoftheparametersforinstance),thesuccessofthemethodwillstronglydependonthecapabilityoftheKrigingmodeltottheactualresponse. Finally,itseemsthattheideaofweightedIMSEandsequentialstrategiestoconstructDoEscanbeappliedtomanyotherpurposeswherethereisaneedfortrade-obetweenexplorationofregionsofinterestandreductionoftheglobaluncertaintyinthemetamodel.Here,wedenedaweightfunctiontoaccountfortheproximitytosometargetvalue.Potentialapplicationscouldbefoundinoptimization,ordetectionofregionsofhighvariations,etc.,thedicultybeinginanadapteddenitionoftheweightfunction. 108

PAGE 109

CHAPTER6OPTIMALALLOCATIONOFRESOURCEFORSURROGATEMODELING 6.1SimulatorsWithTunableFidelity Anumericalsimulatorisanapproximationtoarealphenomenonandisbydenitionasourceoferror.Thewayasimulatorresponsefollowstherealfunctionofinterestisoftencalleddelity.Manyengineeringapplicationsusetwotypesofsimulators,alow-delitymodelandahigh-delitymodel.Atypicallow-delitymodelisbasedongreatsimplicationofthesystemgeometryandphysics(forinstancebarsorbeamsmodeling),andahigh-delitymodelistypicallyaniteelementsmodel.High-delitymodelsprovideanaccurateapproximationtothefunctionofinterest,butarecomputationallyexpensive,whilelow-delitymodelsareeasytorunbutcontainlargeerror. However,manynumericalsimulatorscanoeralargerangeofdelities,bytuningfactorsthatcontrolthecomplexityofthemodel.Forinstance,theprecisionoftheresponseofaniteelementanalysis(FEA)canbecontrolledbythemeshingdensityorelementorder.AnotherexampleiswhentheresponseisobtainedusingMonte-Carlomethods:theaccuracy(measuredbythestandarddeviationorcondenceintervals)isinverselyproportionaltothesquarerootofthenumberofMCsimulations.Forbothexamples,thegainindelityisatapriceofcomputationaltime.Thistopichasbeenaddressedonlyrecentlyinthecomputerexperimentcommunityandisusuallyfoundunderthenameofstochasticsimulators[ Kleijnen&vanBeers ( 2005 ), Boukouvalas&Cornford ( 2009 ), Ioossetal. ( 2008 ), Ginsbourger ( 2009 )]. Whenttingametamodeltosuchtypeofsimulators,thesamplingstrategymaytakeintoaccountthetunabledelity.Indeed,whenthecomputationalresourceislimited,onehastotrade-othedelityandthenumberofsimulations.Lowerdelityallowsabetterexplorationofthedesignspace,whilehigherdelityprovidesmoretrustinthesimulatorresponses.Besides,havingtunabledelityistobeconsideredasanadditionaldegreeoffreedom:byallocatingdierentcomputationaltimeforeachsimulation,thequalityof 109

PAGE 110

thesurrogatecanbeimprovedwithoutadditionalcostcomparedtoauniformdelitysituation. TheoptimalDoEformulation,aspresentedinChapter2,consistsofselectingthepointstomaximizeacriterionofqualityforametamodelconstruction.Here,wedisposeofadditionaldegreesoffreedom,sincewecanchoosethenumberoftrainingpoint,andtunethemodeldelityforeachtrainingpoint(undertheconstraintofatotalcomputationalbudget). Elfving ( 1952 )pioneeredworkintheareaofoptimalallocation.Foralinearmodelheproposedtorepeatexperimentsatsometrainingpoints,andallocateadierentnumberofrepetitionstoeachpointaccordingtoanoptimalitycriterion. Kiefer ( 1961 )generalizedthisideaofproportioningexperimentsinordertoobtainthemostaccurateregressioncoecients. Fedorov ( 1972 )proposedacontinuousversionoftheproblem,anddevelopediterativestrategiestondoptimaldesigns. Thisobjectiveofthischapteristoprovideecientstrategiesfordesigningtheexperimentsofsimulatorswithtunabledelity.First,wedescribethreeexamplesofsuchtypeofsimulatorsthatmodelengineeringsystems.Afterformalizingtheproblemofoptimalallocationofresource,wereviewsomeoftheimportantresultsofoptimaldesignsforregression.Finally,weproposeanoriginalapproachforoptimalallocationwhenKrigingmodelsareconsidered. 6.2ExamplesOfSimulatorsWithTunableFidelity 6.2.1Monte-CarloBasedSimulators Weconsiderfunctionsoftheformf(x;),thatdependsonbothdeterministicvariablesxandthedistributionofsomerandomparameters.Thefunctionf(x;)isdeterministic,butitis,generally,notknownanalytically.Hence,weestimateitusingMonte-Carlosimulations(MCS).Foragivendesignx,wewrite: ^f(x;i=1:::k)=^f(x;1;2;:::;k)(6{1) where^fistheestimatoroffand1;2;:::;karerealizationsof. 110

PAGE 111

TheestimationoftheprobabilityoffailureofasystemisatypicalexampleofaMonte-Carlosimulator.ThelimitstateG(x;)iscomputedforseveralrealizationsof,andtheprobabilityoffailureestimate^Pf(x),whichcorrespondsto^f(x;i=1:::k)inEq. 6{1 isgivenby: ^Pf(x)=1 kkXi=1I[G(x;i)0](6{2) whereIistheindicatorfunction. Monte-Carlomethodsconvergewiththesquarerootofthesamplesizek.Thatis,whenkischosensucientlylarge,theestimator^fisnormallydistributedanditsvarianceisinverselyproportionaltok: var^f(x)=Eh^f(x))]TJ /F4 11.955 Tf 11.95 0 Td[(f(x)i=cst k(6{3) Forthereliabilityexample,^Pf(x)followsabinomiallawwithparameterp=Pf(x),henceitsvarianceisgivenby: varh^Pf(x)i=Pf(x)(1)]TJ /F4 11.955 Tf 11.95 0 Td[(Pf(x)) k(6{4) 6.2.2RepeatableExperiments Repeatableexperimentsarenotinthemselvesanumericalsimulatorwithtunabledelity,butcanbeconsideredasadiscreteversionoftheproblem.Somenumericalsimulators,andmostphysicalexperiments,benetfromrepetitions.Thisisnotthecaseofaniteelementanalysisforinstance:repeatingtwotimesthesamesimulationwillgivetheexactsameanswerandwillnotprovideanyadditionalinformation.Anumericalsimulatorthatbenetsfromrepetitionhastypicallyaresponseoftheform: yobs(x)=y(x)+"(x)(6{5) wherey(x)istheactualresponseandtheerrorterm"(x)isarandomvariable.Hence: var[yobs(x)]=var["(x)](6{6) 111

PAGE 112

Now,saytheexperimentisrepeatedktimes.Assumingthatthenoiseisindependentfromoneobservationtoanother,wedeneyeqasthesingleobservationequivalenttothekobservationsyobs(j)(x),whichisthemeanofthekobservations: yeq(x)=1 kkXj=1yobs(j)(x)(6{7) Thevarianceofyeqisinverselyproportionaltothenumberofrepetitions: var[yeq(x)]=var["(x)] k(6{8) 6.2.3FiniteElementAnalysis Thequalityofaniteelementanalysismostlydependsonthemeshingchosentomodelthestructure.Themeshingcomplexityisdenedbytheelementtypeandtheelementsize,ormeshingdensity.Increasingtheelementcomplexityorthedensityresultsinagaininaccuracy,butatapriceofcomputationaltime. Figure 6-1 showsanexampleoftheevolutionofaFEAresponsewhenthecomplexityofthemodelistunedtodierentlevels.TheFEAusedisthetorquearmmodelpresentedinChapter3.TheresponseconsideredisthemaximumVonMisesstressonthestructure.Themeshdensityisatunableparameterthatdenesthecomplexityofthemeshing.Adensityofonecorrespondstoacrudemesh,whileadensityof25correspondstoaverycomplexandexpensivemodel.Theresponsefordensity25canbeconsideredastheactualvalueofthemaximumstress. Thegraphshowsthatthesimulatorerror(tothe25-mesh-densityreference)decreaseswiththemodelcomplexity.Thelowestdelitymodelhasanerrorof24%,whilefordensitiesgreaterthan15theerrorislessthan5%.Aswellasforpreviousexamples,itispossibletorelatetheerroramplitudetothecomputationaleortgivenforthesimulationrunandobtainanequationoftheform: var["(x)]=f(x;t)(6{9) 112

PAGE 113

Figure6-1. EvolutionoftheresponseoftheFEAforthetorquearmexample,whenthecomplexityofthemodel(meshdensity)increases. wherefisafunctiontodetermine,xthedesignparametersandtthecomputationaltime. However,therelationsbetweenmeshdensity,computationaleortanderroramplitudearecomplexandproblemdependent.Itdependsonthechoiceoftheelements,themeshingtechnique,theniteelementsoftwaremethodofresolutionofthesystem,etc.Hence,itisimpossibletoprovideageneralsimplerelationbetweentheresponsevarianceandthecomputationaltimeinthegeneralcase,andthisrelationhastobefoundforagivenproblem. 6.3OptimalAllocationOfResource Aclassicaldesignofexperimentsformulationconsistsofndingasamplingstrategythatmaximizesacriterionofinterest.Thedesignofexperimentsisthendenedbythesamplingpointlocations.Inourframework,thedesignofexperimentmustbedenedasthecombinationoftrainingpointsandcomputationalresourceallocatedtothatpoint.Then,adesignwithnpointsisdenedas: =8><>:x1;:::;xnt1;:::;tn9>=>;(6{10) 113

PAGE 114

Thexi'saretrainingpointlocations,andtheti'sarerealpositivenumbersthatrepresentthecomputationaltimeallocatedforeachtrainingpoint.Thesumoftheti'sisboundedbyaconstant: 0ti;nXi=1tiTtotal(6{11)Ttotalrepresentsthetotalcomputationaltimeavailableforthedesignofexperiments.Letbeafunctionalofinteresttominimizethatdependsonbothtrainingpointsandcomputationaltime.Adesigniscalledoptimalifitachieves: =argmin[()](6{12) Suchthat:nXi=1tiTtotal Findinganoptimaldesignisadicultandcomplexproblemthathasbeenwidelyaddressedintheliteratureintheframeworkofregressionwithrepeatableexperiments.Whentherelationbetweenthecomputationaltimetandthesimulatorresponsevarianceisunknownorcomplex,theoptimizationproblemcanbecomeverychallenging,andnogeneralanswerisprovidedintheliterature.Inthefollowing,wefocusonsimulatorsforwhichtheresponsevarianceisinverselyproportionaltothecomputationaltime.Thisisthecase,asseeninSection 6.2.1 ,ofreliabilityestimates. Inthetwofollowingsections,weconsidertheframeworksofregressionandkriging.Resultsforregressionarewell-knownanditssectionismostlydevotedtoreviewandbibliography.Onthecontrary,verylittleisknownforthekrigingarea,andtheapproachesweproposeareoriginal. 6.4ApplicationToRegression 6.4.1ContinuousNormalizedDesigns Inthissection,theobservationsareassumedtoberepeatable,andthatnoiseinallobservationshasthesamevariance(homoscedasticcase).WecallnthenumberoftrainingpointsandNthenumberofexperiments(countingrepetitions),Nn. 114

PAGE 115

Letribethenumberofobservationsatxi,allobservationsaredenotedyij(i=1;2;:::;n;j=1;2;:::;ri).ThedesignNconsistsofthecollectionoftrainingpointsxiandnumbersofrepetitionsri: N=8><>:x1;:::;xnr1;:::;rn9>=>;;0ri;nXi=1ri=N(6{13) Foragiventrainingpointxi,theequivalentobservation(seeSection 6.2.2 )isdenotedbyyi: yi=r)]TJ /F5 7.97 Tf 6.58 0 Td[(1iriXj=1yij(6{14) Let2bethevarianceofasingleobservation;assumingindependanceoftheobservations,thevarianceofyiis: var(yi)=r)]TJ /F5 7.97 Tf 6.58 0 Td[(1i2(6{15) Then,theFisher'sinformationmatrixbecomes: M(N)=f(X)T)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X)=f(X)Tdiag)]TJ /F4 11.955 Tf 5.48 -9.68 Td[(2r)]TJ /F5 7.97 Tf 6.59 0 Td[(11;2r)]TJ /F5 7.97 Tf 6.59 0 Td[(12;:::;2r)]TJ /F5 7.97 Tf 6.58 0 Td[(1n)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X)=)]TJ /F5 7.97 Tf 6.58 0 Td[(2f(X)T[diag(r1;r2;:::;rn)]f(X) (6{16) NormalizingbyN, M(N)=N)]TJ /F5 7.97 Tf 6.58 0 Td[(2f(X)T[diag(r1=N;r2=N;:::;rn=N)]f(X)(6{17) TheFisherinformationmatrixisproportionaltoN,andfollowing Fedorov ( 1972 ),itispossibletodene,withoutanylossofgenerality,anormalizeddesignasacollectionofsupportpointsxiandtheircorrespondingproportionsofexperimentspi(withpi=ri=N): =8><>:x1;:::;xnp1;:::;pn9>=>;;0pi;mXi=1pi=1(6{18) 115

PAGE 116

Furthermore,acontinuousnormalizeddesignisdenedwhenpicantakeanyrealvaluebetween0and1.ContinuousdesignsaregoodapproximationsofrealityforlargeN.ThecorrespondingFisher'sinformationmatrixis: M()=f(X)T[diag(p1;p2;:::;pn)]f(X)=nXi=1pif(xi)Tf(xi)(6{19) Itisimportanttonoticeherethatsuchnormalizationispossibleonlywhentheobservationnoisevarianceisinverselyproportionaltocomputationaltime.Theresultsdescribedinthefollowingsectionsonlystandwhenthisrelationistrue. 6.4.2SomeImportantResultsOfOptimalDesigns Atthispoint,weintroducethedispersionfunctionforlinearregressionforapredictionpointx: d(x;)=f(x)TM())]TJ /F5 7.97 Tf 6.58 0 Td[(1f(x)(6{20) Thedispersionfunctiondirectlyrelatestothepredictionvarianceby: var(^y)=2N)]TJ /F5 7.97 Tf 6.59 0 Td[(1d(x;)(6{21) Animportantresultindesignoptimalityisthefamousequivalencetheoremdemonstratedby Kiefer&Wolfowitz ( 1960 ).Itstatesthatthethreefollowingproblemsareequivalent: mindet(M()) (6{22) minmaxxd(x;) (6{23) ndsuchthat:maxxd(x;)=p (6{24) pbeingthenumberofregressioncoecientstoestimate. Inpractice,itmeansthatD-optimaldesignsarealsoG-optimal(andvice-versa),thatisdesignsthatminimizetheuncertaintyontheregressioncoecientsalsominimizethemaximumpredictionvariance,andfornormalizedoptimaldesignsthevalueofthemaximumpredictionvariancevalueisequaltop.However,theseresultsarerestrictedto 116

PAGE 117

continuousdesigns;whenthetotalnumberofobservationsNisnotalotlargerthanthenumberoftrainingpointsn,thepitakediscretevaluesandtheabovetheoremdoesnotholdingeneral. Anotherresultofgreatpracticalrelevanceisthatthereexistsanoptimaldesignthatcontainsnomorethann0points[see Fedorov ( 1972 )]: n0=p(p+1) 2+1(6{25) Suchresultsareveryusefulinpractice,whenalgorithmsareusedtoconstructoptimaldesigns.WiththeD-Gequivalence,wegettheintuitionthatsupportpointsmaybetakenwherethepredictionvarianceismaximal.Thelattertheoremstatesthat,forlinearregression,optimalsamplingstrategiesconsistofconcentratingthecomputationalresourceonalimitednumberof(well-chosen)trainingpoints,ratherthanusingalargenumberofpoints. 6.4.3AnIllustrationOfAD-OptimalDesign Now,weillustrateD-optimaldesignswhentheresponseisasecondorderpolynomialwithinteractionsintwodimensions: y=0+1x1+2x2+3x21+4x22+5x1x2(6{26) Theoptimaldesignconsistsofthefull-factorialdesignwithninepoints,withthefollowingweights(Fedorov,1972): 14.6%atthecorners 8.0%atthemiddlesoftheedges 9.6%atthecenter Figure 6-2 representstheoptimaldesignandcorrespondingpredictionvarianceprole.ItcanbeseenthatG-optimalityisachieved,sincethepredictionvarianceattainsitsmaximump=6(seeEq. 6{24 )onseverallocations(center,corners,andmiddlesoftheedges).Forthismodel,optimalityisachievedbyallocatingapproximatelytwotimesmore 117

PAGE 118

computationalresourcestothetrainingpointsonthecornersthantotheothertrainingpoints.Indeed,thecornersofthedomainaretheregionswheretheuncertaintyislikelytobemaximal,soitseemslogicaltocompensateforthisuncertaintybyallocatingmoreresourcesthere. Figure6-2. A)D-optimaldesignforasecond-orderpolynomialregressionandB)correspondingpredictionvariance.Theverticalbarsrepresenttheproportionofcomputationaleortgiventoeachtrainingpoint. Figure 6-3 representsthefull-factorialdesignbutwithhomogeneousresourceallocationandthecorrespondingpredictionvariance.Thepredictionvarianceproleshowsthatsuchdesignisnotoptimal,sincethevarianceismuchhigheronthecornersthanonthecenterofthedomain.Themaximumpredictionvarianceis7.25,whileitisequaltop=6fortheoptimaldesign. Finally,Figure 6-4 representsadesignwithalargenumberofpointsuniformlyspreadonthedesignregion,whichisanaturalalternativetosmalldesigns.Here,wechosea36-pointFFdesign;theobservationnoiseisveryhighsinceonly1=36ofcomputationaltimeisallocatedtoeachtrainingpoint.Suchdesigndoesnotappeartobeagoodsolution,sincethepredictionvariancehasveryhighvaluesontheedgesofthedomain.Themaximumpredictionvarianceis13.45. 118

PAGE 119

Figure6-3. A)Full-factorialdesignwithhomogeneousallocationofcomputationaleortandB)correspondingpredictionvariance. Figure6-4. A)36-pointFull-factorialdesignwithhomogeneousallocationofcomputationaleortandB)correspondingpredictionvariance. 6.4.4AnIterativeProcedureForConstructingD-OptimalDesigns Manyalgorithmicprocedureshavebeenproposedinthepasttogenerateoptimaldesigns.Here,wedetailtherst-orderalgorithmpresentedbyFedorovinhis1972book.Morerenedtechniquescanbefoundin Cook&Nachtsheim ( 1980 ) Fedorov&Hackl ( 1997 ), Wu ( 1978 ), Molchanov&Zuyev ( 2002 ). TheprincipleofFedorovalgorithmistoaddpointssequentiallytothedesign,andtotransferaproportionofexperimentsfromtheexistingpointstothenewone.Theprocedureisasfollows: 119

PAGE 120

Startingfromacurrentdesignsobtainedaftersiterations,therststepistondonepointwherethepredictionvarianceismaximal,thatis: xnew=argmax[d(x;)](6{27) Then,thenewdesigns+1is: s+1=(1)]TJ /F4 11.955 Tf 11.96 0 Td[()s+1+(xnew)(6{28) Theproportionofexperimentsforthenewpoint()ischosensuchthatdet[M(s+1)]ismaximal,showingtheformula[ Fedorov&Hackl ( 1997 ),p.47]: opt=d(x;S))]TJ /F4 11.955 Tf 11.95 0 Td[(p (d(x;S))]TJ /F1 11.955 Tf 11.95 0 Td[(1)p(6{29) Thesequencefsgiscalledoptimalinthesensethatitscriterionvalueconvergestotheoptimumvalue: lims!1(s)=()(6{30) Whenaverylargenumberofiterationsisperformed,theresultingdesigncancontainalotofpoints.Then,wecanreducethenumberofpointsby: discardingpointswithsmallweightsnotclosetoanygroupofpointswithhighweights agglomeratingpointsclosetoeachother(thenewpointisthebarycenterofthepoints,theweightisthesumoftheweights) 6.4.5ConcludingRemarks Theproblemofoptimalallocationofresourcehasbeenwidelystudiedintheregressionframework.Theremarkablepropertiesofoptimaldesignsallowproposingecientalgorithmstogeneratesuchdesigns.Theresultsaredevelopedforrepeatableexperiments,butfullyapplytoanysimulatorwithtunabledelitythathasaresponsevarianceerrorinverselyproportionaltocomputationaltime.WhenD-orG-optimalityareconsidered,optimalsamplingstrategiesconsistofconcentratingthecomputational 120

PAGE 121

resourceonalimitednumberoftrainingpoints,withnon-uniformallocationofresourcesbetweenpoints. 6.5ApplicationToKriging 6.5.1ContextAndNotations Inthissection,weconsidercontinuousdesigns,butwithweightspi'sboundedbyaconstantnotequaltoone: 0pi;nXi=1pi=1 2(6{31) Wewillseelaterthatincontrasttotheregressioncase,thequantity1 2cannotbenormalizedsinceithasannon-linearimpactontheresults.Itrepresentsthecomputationalresourceavailableforthedesign.Here,theweightsaretheinverseofthevariancesofthemeasurementerroratxi: pi=1 var["(xi)](6{32) Assumingthattheerrorvarianceisproportionaltocomputationaltime,Eqs. 6{31 and 6{32 reectsthefactthatcomputationalresourcesremainconstantforall.Ifthedesignconsistsofasinglesupportpoint,theerrorvarianceis2;ifthedesignconsistsofndistinctpointswithuniformweights,theerrorvarianceisuniform,equalto: var("i)=n2(6{33) Inthissection,wealwaysconsiderthatthevariancesareequal. Thekrigingmodelstudiedisthesimplekriging(SK),whichtrendconsistsofaknownconstant(seeSection 2.1.3 ).Then,theSKbestpredictorisgivenbytheequation: mK(x)=+k(x)TK)]TJ /F5 7.97 Tf 6.59 0 Td[(1(Y)]TJ /F4 11.955 Tf 11.95 0 Td[(1)(6{34) andtheSKpredictionvarianceis: sK2(x)=2)]TJ /F7 11.955 Tf 11.95 0 Td[(kT(x)K)]TJ /F5 7.97 Tf 6.59 0 Td[(1k(x)(6{35) 121

PAGE 122

Consideringacomputationalbudget1=2andequalvariances,thecovariancematrixKisgivenby: K=K+=(k(xi;xj))+0BBBB@n20...0n21CCCCA(6{36) Toanalyzethequalityofamodelt,weconsidertheIMSEcriterion: IMSE=ZDsK2(x)d(x)(6{37)beinganintegrationmeasure,usuallytakenuniform. 6.5.2AnExploratoryStudyOfTheAsymptoticProblem TheobjectiveofthisstudyistoquantifytheevolutionofthequalityoftheSKmodelwhenthenumberofobservationsincreases,whilethecomputationalresourceremainsthesame.Inotherwords,wewanttoseeifitisbetterusingafewaccurateobservations,ormanyinaccurateones. Anexploratorystudyisperformedrst.Weconsiderthetofatwo-dimensionalstationaryprocessesinD=[0;1]2.TherstprocesshasaGaussiancovariancefunction(smooth),thesecondhasanexponentialcovariancefunction(irregular). Wechoose2=1and2=10)]TJ /F5 7.97 Tf 6.58 0 Td[(3forbothprocesses,arangeof=3:1forthegaussiancovarianceand=2fortheexponential.Weconsiderseveralsamplesizesnfrom10to150.Foreachsamplesize,wegenerate500designs(p)n,thexibeinguniformlytakeninD,thepibeingequalton2.Then,wecomputetheIMSEcriterionforeach(p)n,usingnumericalintegrationona3232grid.Figure 6-5 showstheevolutionofthemean,90%condenceintervalandminimumIMSEcriterion(overthe500designs)withrespecttom,forbothcovariancefunctions.NotethattheIMSEcriterionisindependentoftheobservations,sotheresultsdependonlyonthecovariancestructureandparameters. WeseethatthemeanandminimumIMSEdecreasewhenthenumberofobservationsincreases.Bothquantitiesseemtoconvergetoanon-zeroconstant.Thecondence 122

PAGE 123

Figure6-5. EvolutionoftheIMSEwithrespecttothenumberofobservationsforuniformrandomdesignswithA)Gaussiancovariance,B)Exponentialcovariance. intervalwidthalsodecreases.Hence,wecanconclude,forthesetwoexamples,thatitispreferableforaconstantcomputationalbudgettohavealargenumberofpointswithlargevariancesspreadalloverthedesigndomain,ratherthanafewaccurateones. Besides,thedesignswithlargenumberofpointsaremorerobust,sincethecondenceintervaldecreaseswhenthenumberofobservationsincreases.Thisislogicalsince,withasmallnumberofpoints,havinganobservationatawronglocationhasastronginuenceontheglobalaccuracyofthemodel. Thus,thegoalistostudythebehaviorofs2K(x)andtheIMSEwhenntendstoinnity,thexibeinguniformlyspreadontothedesigndomainD.Inparticular,weaimatndinganalyticalexpressionsofbothvarianceandIMSE,whichcanbeusedforcomparisontodesignswithanitenumberofsupportpoints. 6.5.3AsymptoticPredictionVarianceAndIMSE Weassumethattheobservationsaretakenrandomlyaccordingtothemeasure.canbeeithercontinuousonthedesigndomainD,ordiscrete.Typicallyintherstcase,istheuniformmeasureonD;then,eachtrainingpointistakenrandomlywithuniformdistributiononD(whichisthecaseoftheexampleoftheprevioussection).Asymptotically,whenthenumberofobservationsntendstoinnity,thetrainingpoints 123

PAGE 124

aretakeneverywhereonD.Inversely,adiscretemeasureisdenedasthesumofDiracfunctions,withpositiveweightssuchthattheirsumequalsone: =p1x1+:::+pkxk(6{38) With:pi0,1ikandPpi=1. Eachobservationistakenrandomlyonthesetfx1;:::;xkgwithprobabilitypi.Whenntendstoinnity,thedesignofexperimentsisequivalenttoacontinuousnormalizeddesignwithtrainingpointsfx1;:::;xkgandnoisevariances2=pi. 6.5.3.1Generalresult Theorem .Theobservationsaretakenrandomlyaccordingtothemeasure.Whenthenumberofobservationsmtendstoinnity,thepredictionvarianceandIMSEtendto: s2K1(x)=2)]TJ /F11 7.97 Tf 16.36 14.94 Td[(1Xp0p2 2+p(p(x))2 (6{39) IMSE1=2)]TJ /F11 7.97 Tf 16.35 14.94 Td[(1Xp0p2 2+pZD(p(x))2d(x) (6{40) whereparetheeigenvaluesoftheHilbert-Schmidtintegraloperatorassociatedwiththecovariancefunctionk(x;y)(seeappendix B ),andparethenaturalrepresentativesoftheeigenfunctions~poftheoperator,denedby:p(x)=1 pZDk(x;y)~p(y)d(y);x2D Werecallthatthespectraldecompositionhasthefollowingproperties: k(x;y)=Pp0p~p(x)~p(y);almosteverywhere ~ppanorthonormalfamily:ZD~p(x)2d(x)=1;ZD~p(x)~q(x)d(x)=0p6=q p~p(x)=RDk(x;y)~p(y)d(y);x2supp() 124

PAGE 125

Ifandcoincide: .TheIMSE1expressionreducesto: IMSE1=2)]TJ /F11 7.97 Tf 16.35 14.95 Td[(1Xp0p2 2+p(6{41) Proof: .Twodierentproofsareproposeddependingonthenatureofthekernel.(1)Ifthekernelcanbedenedasanitesumofbasisfunctionsandcoecients:k(u;v)=21f1(u)f1(v)+:::+2lfl(u)fl(v) Then,thenumberofnon-nulleigenvaluesofthespectraldecompositionisniteandtheproofisalgebraic. (2)Thegeneralcase,towhichbelongtheusualKrigingkernels,requiresamorecomplexproof.TheproofsaregiveninAppendix B.1 andAppendix B.2 ,respectively. Remarks: .Findingthespectraldecompositionofthecovariancefunctionisaclassicalproblem.Dependingonthekernelconsidered,thedecompositioncanbewell-known,easytoderiveorinexistent(inparticular,ananalyticalformofthedecompositionoftheGaussiancovariancefunctionseemsnottoexist).Inmanycase,theseries(p)pdecreaserapidly,sotheseriesfors2K1andIMSE1arewellapproximatedbyitsrstqterms,typically:10q100. 6.5.3.2Adirectapplication:space-llingdesigns WeconsiderthecasewheretheuserwantstolearnthefunctionuniformlyonD;themeasureofintegrationfortheIMSEistheuniformmeasure.Typically,space-llingdesignsareusedforthatpurpose,sothemeasureofdistributionoftheobservationsisalsotheuniformmeasureonD.Inthatcase,theasymptoticMSEandIMSEhavesimpleform: s2K1(x)=Xp02p 2+p(p(x))2 (6{42) IMSE1=1Xp02p 2+p (6{43) 125

PAGE 126

Inthatcase,theycanbeeasilycomputednumerically.ThisvalueconstitutesthelimitoftheIMSEofaspace-llingDoE,whenthenumberofobservationstendstoinnity,andcanbeusedforastoppingcriterionwhenbuildingthedesign. Let(xn)n2Nbeaspace-llingseries,forinstanceaSoboloraNiederreitersequence,kaninitialnumberofpointsandapositivenumber.Then,wecanbuildaspace-llingdesignbyaddingpointsoftheseriestotheDoEuntilthecorrespondingIMSEiscloseenoughtotheasymptoticvalue,asweshowinthefollowingexamples. 6.5.4Examples 6.5.4.1Brownianmotion Weconsiderthettingofaone-dimensionalBrownianmotioninthespace.ThecovariancefunctionassociatedwiththeBrownianmotionis: k(x;y)=2min(x;y)(6{44) Notethatthiscovariancefunctionisnotstationary(sinceitdependsonxandyandnotonlyjx)]TJ /F4 11.955 Tf 11.95 0 Td[(yj).Howeverthekrigingmodelremainsthesame.Theeigenvaluesandeigenfunctionsassociatedwiththecovarianceare: p=2 (2p+1)p(x)=p 2sin(2p+1)x 2(6{45) Thus,theasymptoticalpredictionvarianceandIMSEare: s2K1(x)=2Xp02 1+)]TJ /F6 7.97 Tf 6.9 -4.98 Td[( 2h(2p+1) 2i2sin2(2p+1)x 2 (6{46) IMSE1=2Xp02 2+h(2p+1) 2i2 (6{47) Weillustratethisresultonanumericalexample.Wetake=0,2=1and2=210)]TJ /F5 7.97 Tf 6.59 0 Td[(3.Figure 6-6 showsonerealizationofaBrownianmotion,twoKrigingmodelsbasedonfourand19noisydata,respectively,andthepredictionvarianceforthetwomodels,andtheanalyticalfunction(usingthe50rsttermsoftheseries)foraninnite 126

PAGE 127

dataset.Thevariancesoftheobservationnoisesareequalton2,thatis,0.008and0.038,respectively. Figure6-6. IllustrationoftheMSEfunctionforaBrownianmotion(2=1,2=0:002).A)RealizationofaBrownianmotionandtheKrigingmodelbasedon4noisydata;B)Krigingbasedon19noisydata;C)predictionvariancefunctionforthetwomodels,andasymptoticvalue. Sincethenoisevarianceissmall,the4-observationbasedKriging(Figure 6-6 A))almostinterpolatesthetruefunction.However,theKrigingmeanisveryinaccuratewhenawayfromtheobservation.Asaconsequence,thepredictionvariance(Figure 6-6 C))isclosetozeroattheobservationpointsandismaximalatmid-distancesbetweentwoobservations.Sincetheresponseisknownatx=0(g(0)=0),thepredictionvarianceisequaltozero. The19-observationbasedKriging(Figure 6-6 B))isabetterapproximationtotherealfunction,eventhoughtheobservationnoiseismuchhighertosatisfytheconstraintofaconstantcomputationalbudget(seeEq. 6{31 )(indeed,someobservationpoints[black 127

PAGE 128

cross]arenotclosetothetruefunction).Whenincreasingthenumberofobservations,thepredictionvariancetendstoaconstant(exceptontheboundaries),whichislogicalsincetheinformationisuniformlyspreadintothedomain. Table 6-1 showstheIMSEvaluesforthethreemodels.Wecanseethatforaconstantcomputationalbudget,theIMSEcanbeconsiderablyreducedbyspreadingtheinformationuniformlyonthepredictiondomain.TheIMSEforthe4-pointmodelistwotimeslargerthantheasymptoticvalue.Ontheotherhand,thedierencebetweenthe19-pointmodelandtheasymptoticvalueisonly4%.Sincehavingmoreobservationsincreasesthecomputationalburden,onecandecidethataDoEwith19observationsissucientintermsofIMSE. Table6-1. IMSEvaluesforthetwoKrigingmodelsandtheasymptoticmodel. Krigingbasedon4observationsKrigingbasedon19observationsAsymptotic IMSE0.04610.02330.0224 6.5.4.2Orstein-Uhlenbeckprocess Now,weconsiderthettingofaone-dimensionalGaussianprocessinthespaceD=[0;1],withthefollowingcovariancefunction: k(x;y)=2exp)]TJ /F1 11.955 Tf 10.49 8.09 Td[(1 jx)]TJ /F4 11.955 Tf 11.96 0 Td[(yj(6{48) Theassociateddecompositionis(seeappendix C forcalculations): p(x)=Cp(pcos(px)+sin(px))(6{49) With:p=q 2)]TJ /F6 7.97 Tf 6.59 0 Td[(p 2p. Theparetherootsoftheequation: 2ucos(u)+)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2sin(u)=0(6{50) 128

PAGE 129

Suchequationdoesnothaveexplicitanalyticalsolutionsandmustbesolvednumerically.However,whenpislarge(>10),therootsarewellapproximatedby: p=(p)]TJ /F1 11.955 Tf 11.96 0 Td[(1)(6{51) Theeigenvaluesareobtainedfromtherootsoftheequation: p=2 1+22p(6{52) TheconstantCpischosensothatpisnormalized: Cp=2p p)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1+2p2)]TJ /F9 11.955 Tf 11.95 9.68 Td[()]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1+2p2sin(p)cos(p)+2p(sin(p))2(6{53) Figure 6-7 illustratesthisresultfortheparametervalues2=1,=0:2and2=0:002.Forcomparison,amodelbasedon80dataisrepresented,whichisfoundtobeagoodapproximationtoaninnitedataset. 6.5.5ConcludingComments Inthissection,wefoundempiricallythatusingaverylargenumberofobservationswithverylowdelitywasanoptimalsolutiontominimizetheuncertaintyinthesimplekrigingmodel,whenspace-llingdesignsareused.Wefoundthatthiskindofdesignsasymptoticallyconvergetoabound.However,ithastobekeptinmindthatthissamplingstrategyisnotyetproventobeoptimal.Anywaywecanconcludethat,contrarilytoregressionwhereoptimaldesignshaveanitenumberofsupportpoints,itseemsbettertospreadasmuchaspossiblethecomputationalresourceratherthanconcentrateitonafewtrainingpoints. 129

PAGE 130

Figure6-7. IllustrationoftheMSEfunctionforaGaussianprocess(GP)withexponentialcovariancefunction(2=1,=0:2and2=0:002).A)RealizationofaGPandtheKrigingmodelbasedon80noisydata;B)MSEfunctionbasedonthe80data,andasymptoticvalue. 130

PAGE 131

CHAPTER7CONCLUSION 7.1SummaryAndLearnings Theuseofsurrogatemodelingasahelpingtoolforlearning,predictionanddesignofengineeringsystemsisanecientandcomputationallyattractivesolution.However,dealingwiththeuncertaintiesassociatedwithasurrogatemodelisachallengingbutcrucialissuetothesuccessofthesurrogate-basedstrategy. Thetwoprimaryobjectivesofthisworkwerethefollowing:(1)Proposeanddiscussalternativestocompensatefortheerroranduncertaintyinsurrogatemodelsinordertoprovidesafeestimatesthatlimittheriskofweakdesign;(2)Explorethesamplingstrategiesinordertominimizetheuncertaintyofsurrogatemodels. First,itwasshowedthatuncertaintycanbecompensatedbyaddingbiastothesurrogatemodelsinordertoincreasethechancethattheyaresafe,usingconstantorpointwisemargins,designedbytakingadvantageoferrordistributionmeasuresgivenbythemodel,orbymodel-independentmeasuresofaccuracyofthemodel.Sincetheconservativepredictionsarebiased,theproblemcanbeconsideredasamulti-objectiveoptimization,andresultsarepresentedintheformofParetofrontsbetweenaccuracyandconservativeness.Itwasfoundthattheeciencyofthedierentmethodsdependontheabilityofthesurrogatemodeltoreectthesystemresponse.Whenerrormeasuresgivenbythemodelarereasonablyaccurate,theiruseoutperformsmodel-independentmeasurestoreducelargeunsafeerrors.Ontheotherhand,model-independentmeasures,inparticularcross-validation,arefoundtobemorerobustandecientwhenthenumberofobservationsissucientlylarge(regardlessofsparsity). Then,asimplemethodologywasproposedtoestimatetheuncertaintyinareliabilitymeasurebasedonMonte-Carlosimulations.Usingthebootstrapmethod,wewereabletoquantifythelevelofbiasneededtoensureaprescribedprobabilitytobeonthesafesidefortheanalysis.Theprocedurewasappliedtothedesignofalaminatecompositeunder 131

PAGE 132

cryogenicandmechanicalstress.Itwasfoundthatttingasurrogatebasedondatasetsthatwerebiasedtocompensatefortheiruncertaintyallowedtoobtainasatisfyingdesign,evenwhentheavailableinformationwasverylimitedbycomputationalresources.Comparedtotraditionalmethodsofreliabilityestimates,equivalentlevelsofsafetywasobtainedwithareductionofcomputationalcostofseveralordersofmagnitude. Thechoiceoftheobservationsusedtotthemodel(designofexperiment)wasaddressedbytwodierentways.Inarsttime,itwasshownthatsubstantialgaincanbeobtainedbyconsideringthesurrogatemodelinanobjective-basedapproach.Inparticular,amethodologywasproposedtoconstructdesignsofexperimentsadaptedtotheframeworksofreliabilityassessmentandconstrainedoptimization,sothatthemodelaccuratelyapproximatesthevicinityofaboundaryindesignspacedenedbyatargetvalueofthefunctionapproximatedbythesurrogate.Themethodwasappliedtoareliabilityexample,anditwasfoundthatimprovinglocallytheaccuracyresultedingreatimprovementoftheaccuracyofthereliabilityestimation. Finally,aglobalapproachtouncertaintyreductioninsurrogatemodelingwasproposedtoaddresstheissueofsurrogatesbasedonsimulatorswithtunabledelity.Thissituationis,inparticular,encounteredintheframeworkofreliability-baseddesignoptimization.Solutionsdierdependingonthesurrogatemodelconsidered.Forregressionmodels,classicalresultsofdesignoptimalityshowthatsubstantialreductionofuncertaintycanbeachievedbyallocatingdierentcomputationalresources-hencedierentuncertaintylevelsintheresponses-toafewchosensimulations.Forkrigingmodels,itwasfoundthatthebestwaytoreduceuncertaintyinthemodelistospreadtheinformationasmuchaspossible,byusingverylargedatasetswithverylargelevelsofuncertainty.Thisresultwassupportedbytheoreticalasymptoticstudiesofthekrigingmodel. 132

PAGE 133

7.2Perspectives Ensuringhighlevelsofconservativenessis,aswehaveseen,atapriceofasubstantialincreaseoferror,whichleadstooverdesign.Inanengineeringcontext,suchapproachisrecommendedforpreliminaryanalysis,whichwouldbefollowedbymorerened(andexpensive)procedures.Moregenerally,theeciencyofconservativestrategiesareverydependentonthesamplingstrategiesusedtogeneratetheobservations.Possibleprolongationofthisworkmightincludeacombineduseofconservativepredictionsandsequentialstrategiesinordertolimittheriskofoverdesign. TheadapteddesignstrategyproposedinChapter5wastestedonacademicproblemsofreasonablecomplexity.Itsapplicationonreal-lifeproblemsmightleadtoadditionalproblemsanddiscussion,whichwereonlymentionedhere.Inparticular,onemightthinkofmeasuringtheinuenceofthequalityofthemetamodelontheeciencyofthemethod,thecombineduseofspace-llingandadaptedstrategiesforincreasedrobustness,ornumericalsolutionstoovercomecomputationalissues.Secondly,itseemsthattheideaofweightedIMSEandsequentialstrategiestoconstructDoEscanbeappliedtomanyotherpurposeswherethereisaneedfortrade-obetweenexplorationofregionsofinterestandreductionoftheglobaluncertaintyinthemetamodel.Here,wedenedaweightfunctiontoaccountfortheproximitytosometargetvalue.Potentialapplicationscouldbefoundinoptimization,ordetectionofregionsofhighvariations,etc.,thechallengebeinginanadapteddenitionoftheweightfunction. WediscussedinChapter6optimalstrategiesofallocationofresourceforsimulatorswithtunabledelity.Weshowedthat,inparticular,reliabilityanalysisbelongstothisframework.Itisalsopotentiallyapplicabletoawiderangeofproblems,inparticularFiniteElementAnalysisorComputationalFluidDynamicscodes.Futureresearchmayaimatrelatingthisworktostudiesofconvergenceofnumericalmodels(dependingonmeshing,complexityofelements,etc.),inordertoestimatetherelationbetweencomputationaltimeanderrorandproposeoptimalstrategiesfortheseproblems.In 133

PAGE 134

addition,weproposedinthisworksomelinkstofunctionalanalysisandspectraltheorywhenconsideringoptimalallocationfortheKrigingmodel.Itseemsthatthiswayoersmanypotentialprolongationsforthetheoryofdesignofexperiments,whichmightleadtoexactlyoptimalstrategies,ornumericalmethodstogeneratedesigns. 134

PAGE 135

APPENDIXAALTERNATIVESFORCONSERVATIVEPREDICTIONS Inthisappendix,wedetailtwoalternativestoproduceconservativepredictionsusingsurrogatemodeling. A.1BiasedFittingEstimators A.1.1BiasedFittingModels Weproposeheretoincludebiasduringthettingprocessinordertoproduceconservativepredictions.Todoso,weconstrainthepredictedresponsetobeononesideoftheDOEresponses;thatis,theerrorsbetweenpredictionandactualresponsearepositiveatDOEpoints.Suchmethoddonotapplytointerpolationtechniquessincetheerrorisbydenitionnullatdatapoints.Forpolynomialregression,theconservativecoecients^consarethesolutionofthefollowingconstrainedoptimizationmodel: Minimize^consMSE=1 nnXi=1[^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)]2s.t.^y(xi))]TJ /F4 11.955 Tf 11.96 0 Td[(y(xi)>0;i=1;:::;n(A{1) Then,thebiasedttingconservativeestimateisgivenby: ^yBF(x)=fT(x)^cons(A{2) Inordertocontrolthelevelofbias,weproposetwoalternatives.Therstisconstraintrelaxation,thatallowsagivenamountofconstraintviolationdenotedas: Minimize^consMSEs.t.^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)+>0;i=1;:::;n(A{3) Apositivewillreducethebiasinthetting.canalsobechosennegativeinordertobemoreconservativethanwithnoconstraintrelaxation. Thesecondalternativeistoreducethenumberofconstraints,thatis,toconstraintheerrorstobepositiveonlyataselectednumberofpoints(constraintselection).The 135

PAGE 136

constraintsselectedarethoseapriorieasiertobesatised,thatis,wheretheerrorfromtheunbiasedtisminimal. Theproceduretoselectthesepointsisasfollow: Computetheunbiasedestimatesusingclassicalregression Computetheerrorsandsortthembyascendingorder Selectthepointscorrespondingtotheksmallesterrors Solvethefollowingoptimizationproblem: Minimize^consMSEs.t.^y(xi))]TJ /F4 11.955 Tf 11.95 0 Td[(y(xi)+>0;i=1;:::;k(A{4) wherethexiaresortedinascendingorderaccordingtoerrorsand1kn. Inthefollowing,thetwoabove-referencedbiased-ttingalternativesareentitledconstraintrelaxationandconstraintselection,respectively. Remark .Thebiasedttingstrategyrequiresthesolvingofaconstrainedoptimizationproblemtoobtainthecoecients^insteadofusingananalyticalfunction.However,thisoptimizationproblemisquitesimpletosolve,sincethecriterionisconvex(leastsquares),andtheconstraintsarelinear. A.1.2ResultsAndComparisonToOtherMethods Here,wecomparethetwobiasedttingstrategiestotheconstantsafetymargin(CSM)strategyontheBranin-Hoofunction,intheexperimentalsetupdescribedinSection 3.3.2.2 .First,wecomparethetwostrategiesforbiasedttingestimates.Therangeoftheconstraintrelaxationischosenas[-3;15];theproportionofselectedconstraintsischosenbetween0and1.ThegraphsofperformanceareshowninFigure A-1 Bothmethodsseemtoprovidesimilarresults;thedierenceforhighpercentilesontheleftgureishereduetonumericalnoise.Usingconstraintselectiondoesnotallow 136

PAGE 137

FigureA-1. AverageresultsforBiasedttingwithconstraintrelaxation(plainblack)andconstraintselection(mixedgrey)ontheBranin-Hoofunction.A)%cvs.lalpha;B)%cvs.MaxUE%. obtainingveryconservativeestimates:indeed,usingallthe17constraintsleadstoa85%conservativeness.Ontheotherhand,withconstraintrelaxation,usinganegativeshiftallowstobemoreconservative.Ontherightgraph,weseethatahighproportionofconservativeestimatesdoesnotpreventforhavinglargeunconservativeerrors:forinstance,fora90%conservativeness,themaximumunconservativeerrorisreducedby40%only. Now,wecomparebiasedttingandCSMstrategies.Figure A-2 showsthegraphsofperformancefortheCSMestimatorandthebiasedttingestimatorwithconstraintrelaxation. Thetwoestimatorshavesimilartrends,butbiasedttingresultswithhighervariability.Indeed,thismethodisalotmoresensitivetotheDOE,sinceasingleconstraintcanhavealargeinuenceontheshapeoftheresponse. A.2IndicatorKriging A.2.1DescriptionOfTheModel IndicatorKriging(IK)isasurrogate-basedalternativetoprovideconservativeestimates.Insteadofestimatingthevalueofresponse,IKestimatestheprobabilitythattheresponseexceedsagivenvalue(cut-o).Inotherwords,IKprovidesanestimateof 137

PAGE 138

FigureA-2. AverageresultsforCSM(plainblack)andBiasedttingwithconstraintselection(mixedgrey).A)%cvs.lalpha;B)%cvs.MaxUE%. theconditionalcumulativedistributionfunction(CCDF)ataparticularcut-oc.ThekeyideaofIKistocodetheobservedresponsesintoprobabilitiesofexceedingthecut-o.Whentheresponseisdeterministic(withoutobservationnoise),theprobabilityiseither0or1.Theindicatorcodingatasampledlocationxiiswritten: Ii=I(c;xijfy1;y2;:::;yng)=P(y(xi)>c)=8><>:1ify(xi)>c0otherwise(A{5) Atanunsampledlocationx,theprobabilityisestimatedbytheKrigingpredictionbasedontheindicatordata: ^P(y(x)>c)=^yIK(x)(A{6) where^yIKistheKrigingestimatebasedonfI1;I2;:::;Inginsteadoffy1;y2;:::;yng. Foragivensetofcut-osfc1;c2;:::;cqgandpredictionlocationx,weobtainacorrespondingsetofprobabilitiesfP1;P2;:::;Pqg.WeusethesediscreteprobabilitiestotacontinuousapproximationoftheCCDFoftheresponseatandbuildcondenceintervals.IKisoftenqualiedasa'non-parametric'approachsinceitdoesnotrelyonapre-specieddistributionmodel.Notethatitisanexpensiveprocedurecomparedtoaclassicalkrigingmodelsinceitmayrequirealargenumberofkrigingmodels. 138

PAGE 139

Post-processingisnecessarytotransformtheIKsetofvaluesintoausablediscreteCDF.Indeed,thereisnoconstraintduringtheproceduretohavevaluesonlyinside[0;1]orthatCDFestimatesvarymonotonicallywithcut-os.WeusehereoneofthemethodsproposedintheGSLIBuser'sguide[ Deutsch&Journel ( 1997 )].First,valuesoutoftheinterval[0,1]arereplacedby0or1.Then,theoriginalIK-derivedpercentilesareperturbedbyrunninganoptimizationthatminimizestheperturbationwhileensuringallorderrelations. The(1)]TJ /F4 11.955 Tf 12.4 0 Td[()%conservativeestimatoristhe(1)]TJ /F4 11.955 Tf 12.4 0 Td[()thpercentileofthisdistribution,givenbytheinverseoftheCDF.Thispercentileisinterpolatedfromthesetofprobability(likeforthebootstrapmethod).Here,weuselinearinterpolation. A.2.2ApplicationToTheTorqueArmAnalysis Sinceitrequiresalargenumberofpoints,IKisappliedtotheanalysisofthetorquearmonly.Arationalquadraticfunctionisusedforcovariance;theparametersofthefunctionarere-estimatedforeachcut-ousingthetoolboxGPML.Atotalof100cutosareusedforthedistributionestimation. Inthissection,wecompareIKtotheregularKrigingwitherrordistribution(ED).First,wecomparetheperformancesoftheunbiasedmetamodels:withoutmarginforKrigingandwithatargetof50%conservativenessforIK.Table A-1 reportstheireRMS,%candMaxUEbasedonthe1,000testpoints. TableA-1. Statisticsbasedon1,000testpointsfortheunbiasedsurrogates. SurrogateeRMS%cMaxUE IK22.945.6%228.9Kriging14.551.6%60.7 Forunbiasedpredictions,IKhasanerroralotlargerthanKriging.TheeRMSistwotimeslargerandtheMaxUEalmostthreetimes.Here,itappearsthattheIKprocedure,withconversionintobooleannumber,CDFnormalizationandinterpolation,resultsinsubstantiallossinaccuracy. 139

PAGE 140

InordertohaveafaircomparisonbetweenIKandKriging,wecompareunscaledquantities,eRMSandMaxUE,insteadoflandMaxUE%.Figure A-3 showsthegraphsofperformanceforthetwoconservativeestimators. FigureA-3. AverageresultsforIndicatorKriging(IK,plainblack)andEDwithKriging(UK,mixedgrey).A)%cvs.l;B)%cvs.MaxUE%. Ontheleftgraph(Figure A-3 A)),weseethattherateofincreaseofeRMSoftheIKisverysmallupto%c=70%.However,itisnotenoughtocompensatefortheinitialvalueofeRMSandbecompetitivewiththeUKestimator.Figure A-4 showsthedelitytotargetconservativenessforbothestimators.DespiteverylargeerrorscomparedtoclassicalKriging,IndicatorKrigingisfoundtobeveryaccurateregardingdelity. FigureA-4. Targetvs.actual%cforIK(plainblack)andKrigingwithED(mixedgray). 140

PAGE 141

APPENDIXBDERIVATIONOFTHEASYMPTOTICKRIGINGVARIANCE ThisappendixshowsthederivationoftheasymptoticformulaofthekrigingvariancedescribedinChapter6. Onallthesecalculations,weassumethattheobservationsaredistributedaccordingtotheprobability.IfistheuniformmeasureonthedesignregionD,itmeansthattheobservationsareuniformlyspreadonD.IfisaDiracfunction,alltheobservationsaretakenonasinglelocation. Weproposetwodierentproofs,dependingonthenatureofthekernel.Forkernelsofnitedimension,theproofisalgebraic.Thegeneralcaseismorecomplex,andtheproofisformal. B.1KernelsOfFiniteDimension Thecovariancekernelisassumedtobeoftheformofanitesumofbasisfunctionsandcoecients: k(u;v)=f(u))]TJ /F4 11.955 Tf 8.08 0 Td[(f(v)T(B{1) With:f(u)=[f1(u);:::;fp(u)]. )]TJ /F1 11.955 Tf 11.98 0 Td[(canbediagonalforinstance,hencethekernelisoftheform: k(u;v)=21f1(u)f1(v)+:::+2pfp(u)fp(v)(B{2) Forsimplicationpurpose,wewillassumeinthefollowingthat)]TJ /F1 11.955 Tf 11.98 0 Td[(isalwaysdiagonal. Werecalltheequationofthekrigingpredictionvariance,basedonmobservationswithhomogeneousnuggeteectequaltom2: sK2(x)=k(x;x))]TJ /F7 11.955 Tf 11.96 0 Td[(kT(x)(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1k(x)(B{3) 141

PAGE 142

With:K+=(k(xi;xj))1i;jm+0BBBB@m20...0m21CCCCA Then,withcovariancefunctionasdenedinEq. B{1 ,thekrigingvariancecanbeexpressedas: s2K(x)=f(u))]TJ /F4 11.955 Tf 8.08 0 Td[(f(u)T)]TJ /F4 11.955 Tf 11.96 0 Td[(f(x))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Im+f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T(B{4) withtheconvention:f(X)=266664f1(x1):::fp(x1)...f1(xm):::fp(xm)377775: Then,byfactorizationweobtain: s2K(x)=f(x))]TJ /F2 11.955 Tf 10.73 0 Td[()]TJ /F7 11.955 Tf 11.96 0 Td[()]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Im+f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(X))]TJ /F9 11.955 Tf 8.09 16.86 Td[(f(x)T(B{5) Sothepredictionvarianceisoftheform: s2K(x)=f(x)M)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(x)T(B{6) with:M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[()]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Thm2Imf(X))]TJ /F4 11.955 Tf 8.08 0 Td[(f(X)Ti)]TJ /F5 7.97 Tf 6.58 0 Td[(1f(X))]TJ /F4 11.955 Tf 8.08 0 Td[(: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1isofdimensionpp. Usingtheequality[ Fedorov&Hackl ( 1997 ),p.107] )]TJ /F7 11.955 Tf 5.48 -9.69 Td[(A+BBT)]TJ /F5 7.97 Tf 6.59 0 Td[(1=A)]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F7 11.955 Tf 11.95 0 Td[(A)]TJ /F5 7.97 Tf 6.58 0 Td[(1B)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(I+BTA)]TJ /F5 7.97 Tf 6.59 0 Td[(1B)]TJ /F5 7.97 Tf 6.59 0 Td[(1BTA)]TJ /F5 7.97 Tf 6.59 0 Td[(1;(B{7) withA=1 m2)]TJ /F1 11.955 Tf 11.98 0 Td[(andB=f(X)T,M)]TJ /F5 7.97 Tf 6.58 0 Td[(1canbesimpliedto: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=1 m2f(X)Tf(X)+)]TJ /F11 7.97 Tf 8.08 4.93 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1(B{8) 142

PAGE 143

Thepredictionvarianceisthen: s2K(x)=f(x)1 m2f(X)Tf(X)+)]TJ /F11 7.97 Tf 8.08 4.94 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)T(B{9) Now,wewanttoconsiderthecasewhenm!+1.Wehavebydenition: 1 mf(X)Tf(X)= 1 mmXi=1fk(xi)fl(xi)!1k;lp(B{10) Whenm!+1,wegetbythestronglawoflargenumbers: 1 mf(X)Tf(X)!hfk;fli1k;lp(B{11) where:hfk;fli=ZDfk(x)fl(x)d(x) WedenetheGrammatrixGsuchthat: G=hfk;fli1k;lp(B{12) Bycontinuity,weobtain: s2K(x)!s2K1(x)=f(x)G 2+)]TJ /F11 7.97 Tf 8.08 4.94 Td[()]TJ /F5 7.97 Tf 6.59 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1f(x)T(B{13) Equation B{13 showsthat,evenifthenumberofobservationstendstoinnity,thepredictionvariancecanbeexpressedasafunctionoftheinverseofappmatrix.Hence,thecomplexityofthecalculationsisdrivenbythedimensionofthekernelinsteadofthenumberofobservations,whichiscounter-intuitiveregardingtheoriginalformofthepredictionvariance(Eq. 2{23 ). Now,weintroducetheHilbert-SchmidtintegraloperatorTk;associatedwiththekernelkandmeasure: Tk;:L2()!L2()g7!Tk;g:x7!RDk(x;y)g(y)d(y)(B{14) 143

PAGE 144

ByusingEq. B{2 : Tk;g(x)=ZDk(x;y)g(y)d(y)=pXk=12k24ZDfk(y)g(y)35fk(x)=pXk=12khfk;gifk(x) (B{15) Equation B{15 showsthattheimageofTk;isgeneratedbypfunctionsfk(x).Hence,Tk;isanoperatorofniterankr,rpanditadmitsaspectraldecomposition1;2;:::rand12:::rsuchthat: (j)janorthonormalfamily:ZD(j(x))2d(x)=1andZDi(x)j(x)d(x)=0;i6=j Tk;j=jj Remark: .Therankrdependsonboth(fi)iand.Inparticular,whenisaweightedsumofDiracfunctions,L2()isofdimensionq=Card(supp()).TheimageofTk;beingavectorsubspaceofL2(),itsdimensionrisboundedbyq.Fortheextremecasewhere=x(singleDiracfunction),wehaver=1,evenifthecovarianceisdenedbyalargenumberpofbasisfunctions.Inpractice,mostofthetimer=min(p;q). Now: jj(x)=Zk(x;y)j(y)d(y)=pXk=12khfk;jifk(x) (B{16) WedenetheprmatrixA=(ajk)1jp;1krsuchthat:ajk=hfk;ji 144

PAGE 145

Then: jj(x)=pXk=12kakjfk(x) (B{17) (x)T=ATf(x)T (B{18) With:=diag([1;:::;r]). WemultiplybothsidesofEq. B{18 by(x),andintegratewithrespecttothemeasure: Z(x)T(x)d(x)=ZATf(x)T(x)d(x) Since(j)jisanorthonormalbasis,theleft-hand-sidereducesto.Then,weobtaintheequality: =ATA(B{19) Similarly,bymultiplyingbothsidesofEq. B{18 byf(x)andintegrating,weobtain: AT=ATGG=AAT (B{20) Now,usingtheequalityofEq. B{7 weobtain: M)]TJ /F5 7.97 Tf 6.59 0 Td[(1=1 n2ATA+)]TJ /F11 7.97 Tf 8.09 4.94 Td[()]TJ /F5 7.97 Tf 6.58 0 Td[(1)]TJ /F5 7.97 Tf 6.59 0 Td[(1=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[(A2Ir+ATA)]TJ /F5 7.97 Tf 6.59 0 Td[(1AT)]TJ /F1 11.955 Tf -172.16 -26.89 Td[(=)]TJ /F2 11.955 Tf 10.74 0 Td[()]TJ /F7 11.955 Tf 11.95 0 Td[(A2Ir+)]TJ /F5 7.97 Tf 6.58 0 Td[(1AT)]TJ ET BT /F1 11.955 Tf 433.05 -481.93 Td[((B{21) Then: s2K1(x)=f(x))]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T)]TJ /F4 11.955 Tf 11.96 0 Td[(f(x)A2Ir+)]TJ /F5 7.97 Tf 6.59 0 Td[(1AT)]TJ /F4 11.955 Tf 8.08 0 Td[(f(x)T=k(x;x))]TJ /F10 11.955 Tf 11.95 0 Td[((x)2Ir+)]TJ /F5 7.97 Tf 6.59 0 Td[(1(x)T (B{22) 145

PAGE 146

Now: 2Ir+)]TJ /F5 7.97 Tf 6.58 0 Td[(1=diag1 2+k=diagk2 2+k (B{23) Finally: s2K1(x)=k(x;x))]TJ /F6 7.97 Tf 18.56 14.94 Td[(rXk=12k 2k+2k(x)2(B{24) B.2GeneralCase Thedemonstrationisorganizedasfollow:rst,theclassicalkrigingequationsaregivenforanitenumberofobservationsm.Aftertransformations,thekrigingvarianceisexpressedwhenmtendstoinnityusingthelawofgreatnumbers.Then,thetheoryofHilbert-Schmidtoperatorsandtheirspectraldecompositionareintroduced.Finally,thekrigingvarianceexpressionissimpliedusingthespectraldecompositiontoobtaintheformula. First,werecalltheequationofthekrigingpredictionvariance,basedonmobservationswithhomogeneousnuggeteectequaltom2: sK2(x)=2)]TJ /F7 11.955 Tf 11.96 0 Td[(kT(x)(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1k(x)(B{25) With:K+=(k(xi;xj))1i;jm+0BBBB@m20...0m21CCCCA Wedecompose(K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1inTaylorseries,using:(I+H))]TJ /F5 7.97 Tf 6.59 0 Td[(1=I)]TJ /F7 11.955 Tf 11.96 0 Td[(H+H2)]TJ /F7 11.955 Tf 11.95 0 Td[(H3+:::=1Xp=0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pHp Herewehave:K+=m2I+K m2 146

PAGE 147

Hence: (K+))]TJ /F5 7.97 Tf 6.58 0 Td[(1=1 m21Xp=0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pKp (m2)p(B{26) NotethatthisdecompositioniscorrectwhenthespectralradiusofKissmallerthanm2.However,ithasbeenfoundempiricallythattheresultsofthisproofstandwhenthisasumptionisnotmet.Yet,wehavenotfoundhowtorelaxthishypothesis. UsingthedecompositionofEq. B{26 ,thepredictionvarianceinEq. B{25 becomes:sK2(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)p1 (m2)p+1kT(x)Kpk(x) Weexpandthequadraticformontheright-handside: kT(x)Kpk(x)=Xi;jk(x;xi)(Kp)ijk(xj;x)=Xi1;:::;ip+1k(xi;xi1)k(xi1;xi2):::k)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(xip;xip+1k)]TJ /F7 11.955 Tf 5.48 -9.69 Td[(xip+1;x Whenm!+1,weobtain,bythelawoflargenumbers:1 mp+1Xi1;:::;ip+1k(x;xi1)k(xi1;xi2):::k)]TJ /F7 11.955 Tf 5.48 -9.68 Td[(xip;xip+1k)]TJ /F7 11.955 Tf 5.47 -9.68 Td[(xip+1;x!ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1) Thus:sK21(x)=2)]TJ /F9 11.955 Tf 9.29 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)p1 (2)p+1ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1) Letusintroducethenotation: p+2(x;x)=ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1)(B{27) Then: sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pp+2(x;x) (2)p+1(B{28) 147

PAGE 148

Atthispoint,weintroducetheHilbert-SchmidtintegraloperatorTk;associatedwiththekernelk: Tk;:L2(D)!L2(D)f7!Tk;f:x7!RDk(x;y)f(y)d(y)(B{29) Mercer'stheoremstatesthatthekernelfunctionkcanberepresentedonL2()inanorthogonalbasis(n)nofL2()consistingoftheeigenfunctionsofTk;suchthatthecorrespondingsequenceofeigenvaluesfngnisnonnegative[ Riez&Nagy ( 1952 ),p.242]: k(u;v)=Xn0nn(u)n(v);u;v2supp()(B{30) With: (n)nanorthonormalbasis:ZD(n(x))2d(x)=1andZDfi(x)fj(x)d(x)=0;i6=j n0;123::: Tk;n=nn ThepthpowerofTk;(Tk;appliedptimes)hasthesameeigenfunctionsanditseigenvaluesarenp.Itsassociatedkernelis(forp>1):p(u;v)=ZDp)]TJ /F13 5.978 Tf 5.76 0 Td[(1k(u;u1)k(u1;u2):::k(up)]TJ /F5 7.97 Tf 6.59 0 Td[(2;up)]TJ /F5 7.97 Tf 6.59 0 Td[(1)k(up)]TJ /F5 7.97 Tf 6.59 0 Td[(1;v)d(u1):::d(up)]TJ /F5 7.97 Tf 6.58 0 Td[(1) Forinstance,forp=2: T2k;=Tk;(Tk;f)(x)=ZDk(x;u1)Tk;f(u1)d(u1)=ZDk(x;u1)24ZDk(y;u2)f(u2)d(u2)35d(u1)=ZZD2k(x;u1)k(u1;u2)f(u2)d(u2)d(u1) 148

PAGE 149

=ZD24ZDk(x;u1)k(u1;u2)d(u1)35f(u2)d(u2)=ZD2(x;u2)f(u2)d(u2) Foru;v2supp(),wehavethespectraldecompositionofthekernelp: p(u;v)=Xn0npn(u)n(v)(B{31) Now,fromEq. B{27 : p+2(x;x)=ZDp+1k(x;u1)k(u1;u2):::k(up;up+1)k(up+1;x)d(u1):::d(up+1)=ZD2k(x;u1)24ZDp)]TJ /F13 5.978 Tf 5.75 0 Td[(1k(u1;u2):::k(up;up+1)d(u2):::d(up)35k(up+1;x)d(u1)d(up+1)=ZD2k(x;u1)[p(u1;up+1)]k(up+1;x)d(u1)d(up+1) Thekernelpisevaluatedonthesupportof,wecanreplaceitbyitsspectralrepresentation(Eq. B{31 ): p+2(x;x)=ZD2k(x;u1)"Xn0npn(u1)n(up+1)#k(up+1;x)d(u1)d(up+1)=Xn0npZD2k(x;u1)n(u1)n(up+1)k(up+1;x)d(u1)d(up+1)=Xn024np0@ZDk(x;u1)n(u1)d(u1)1A0@ZDn(up+1)k(up+1;x)d(up+1)1A35 Bydenition,wehave:Tk;n(x)=nn(x)andZDk(x;y)n(y)d(y)=nn(x) Finally:p+2(x;x)=Xn0[np(nn(x))(nn(x))] 149

PAGE 150

p+2(x;x)=Xn0np+2(n(x))2(B{32) Then,weputEq. B{32 intoEq. B{28 :sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xp0()]TJ /F1 11.955 Tf 9.29 0 Td[(1)p1 (2)p+1Xn0np+2(n(x))2sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xn0n(n(x))2Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pnp+1 (2)p+1 ThesecondsumcorrespondstoaTaylorextension:Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pnp+1 (2)p+1=Xp0()]TJ /F1 11.955 Tf 9.3 0 Td[(1)pn 2p+1=1)]TJ /F1 11.955 Tf 26.57 8.09 Td[(1 1+n 2 Hence:sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n 1)]TJ /F1 11.955 Tf 26.57 8.09 Td[(1 1+n 2!(n(x))2 sK21(x)=2)]TJ /F9 11.955 Tf 11.95 11.36 Td[(Xn0n2 2+n(n(x))2(B{33) B.3TheSpace-FillingCase Inthespace-llingcase,istheuniformmeasureonD.Whenuniformaccuracyoverthedesigndomainiswanted,theintegrationmeasureusedfortheIMSEisalsouniform:IMSE=ZDsK2(x)d(x) Inthecasewhenthetwomeasurescoincide,theIMSEcanbeexpressedinsimpleform.Foranyxinsupp(),wehave:Xn0n(n(x))2=2 Here,supp()=D,sothisequalityiscorrectforanyx2D.Then: sK21(x)=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n 1)]TJ /F1 11.955 Tf 26.58 8.09 Td[(1 1+n 2!(n(x))2 150

PAGE 151

=2)]TJ /F9 11.955 Tf 11.96 11.36 Td[(Xn0n(n(x))2+Xn0 n 1+n 2!(n(x))2==Xn02n 2+n(n(x))2 (B{34) Besides:ZD(n(x))2d(x)=1 WhenintegratingthepredictionvariancetocomputetheIMSE,weobtain: IMSE1=ZDsK21(x)d(x)=Xn02n 2+nZD(n(x))2d(x)=1Xn02n 2+n (B{35) 151

PAGE 152

APPENDIXCSPECTRALDECOMPOSITIONOFTHEORSTEIN-UHLENBECKCOVARIANCEFUNCTION Weconsiderthefollowingkernel:k(x;y)=2exp)]TJ /F1 11.955 Tf 10.49 8.09 Td[(1 jx)]TJ /F4 11.955 Tf 11.96 0 Td[(yj Aneigenfunctionfwithcorrespondingeigenvalueforthatkernelveries:Kf(y)=2yZ0exp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(y)]TJ /F4 11.955 Tf 11.95 0 Td[(x f(x)dx+21Zyexp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(x)]TJ /F4 11.955 Tf 11.95 0 Td[(y f(x)dx=f(y) Bydierentiatingthisequationtwotimeswithrespecttoywend:Kf"(y)=1 2Kf(y))]TJ /F1 11.955 Tf 13.15 8.09 Td[(22 f(y) However:Kf(x)=f(x)Kf"(x)=f"(x) Thus:f"(x)= 2f(x))]TJ /F1 11.955 Tf 13.16 8.09 Td[(22 f(x)f"(x)=)]TJ /F9 11.955 Tf 11.29 16.85 Td[(22)]TJ /F4 11.955 Tf 11.95 0 Td[( 2f(x) Wededucethatfisoftheform: f(x)=Acos(x)+Bsin(x)(C{1) And: =r 22)]TJ /F4 11.955 Tf 11.96 0 Td[( 2(C{2) Now,wereplacefbyitstrigonometricformofEq. C{1 intheintegralequation,andby:=22 1+22 152

PAGE 153

Wehave:Kf(x))]TJ /F4 11.955 Tf 9.3 0 Td[(f(x)=21Z0exp)]TJ 10.5 8.09 Td[(jx)]TJ /F4 11.955 Tf 11.95 0 Td[(yj [Acos(x)+Bsin(x)]dx)]TJ /F1 11.955 Tf 19.55 8.09 Td[(22 1+22[Acos(x)+Bsin(x)] Aftersimplications,weobtain:Kf(x))]TJ /F4 11.955 Tf 11.96 0 Td[(f(x)=2 1+2e)]TJ /F14 5.978 Tf 7.78 3.26 Td[(x +ex With:8><>:=)]TJ /F4 11.955 Tf 9.29 0 Td[(A+B=e)]TJ /F13 5.978 Tf 7.82 3.26 Td[(1 [A(sin())]TJ /F1 11.955 Tf 11.95 0 Td[(cos()))]TJ /F4 11.955 Tf 11.95 0 Td[(B(sin()+cos())] Thisexpressionisnullforanyxonlyifbothandareequaltozero: 1+2e)]TJ /F14 5.978 Tf 7.78 3.26 Td[(x +ex =0,8><>:=0=0 Wededucefrom=0that:A=B So,theeigenfunctionisoftheform:f(x)=C(cos(x)+sin(x)) Cisanormalizationconstantsothat1R0f2(x)=1. Besides,from=0weobtain:(sin())]TJ /F1 11.955 Tf 11.96 0 Td[(cos()))]TJ /F1 11.955 Tf 11.96 0 Td[((sin()+cos())=0 2cos()+)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(22sin()=0(C{3) Therootsofthisequationdenealltheeigenvaluesandeigenfunctionsoftheproblem.However,thisequationdoesnothaveanalyticalsolutionandmustbefoundnumerically. 153

PAGE 154

Figure C-1 representsthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2u2)sin(u).Weseethatg(u)ispseudo-periodic.Whenuisrelativelylarge(2u2)]TJ /F1 11.955 Tf 12.46 0 Td[(12u),therootsarewellapproximatedby: u=n;n2N(C{4) Finally,thespectraldecompositionassociatedwiththeexponentialcovarianceis: fp(x)=Cp(pcos(px)+sin(px))p=22 1+22p(C{5) With:p=s 22)]TJ /F4 11.955 Tf 11.96 0 Td[(p 2pCp=2p p)]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1+2p2)]TJ /F9 11.955 Tf 11.95 9.69 Td[()]TJ /F1 11.955 Tf 5.48 -9.69 Td[(1+2p2sin(p)cos(p)+2p(sin(p))2 Theparetherootsoftheequation:2ucos(u)+)]TJ /F1 11.955 Tf 5.48 -9.68 Td[(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2sin(u)=0 FigureC-1. Representationofthefunctiong(u)=2ucos(u)+(1)]TJ /F4 11.955 Tf 11.96 0 Td[(2u2)sin(u).Therootsofthisequationaretheeigenvaluesoftheexponentialcovariancekernel. 154

PAGE 155

REFERENCES Acar,E.,&Haftka,R.(2005).TheEectsofuncertaintycontrolmeasuresontheweightsavingfromcompositelaminatesatcryogenictemperatures.InProceedingsofASME-IDECT-2005,September,LongBeach,California. Acar,E.,Kale,A.,&Haftka,R.(2007).ComparingEectivenessofMeasuresThatImproveAircraftStructuralSafety.JournalofAerospaceEngineering,20,186. Ben-Haim,Y.,&Elishako,I.(1990).Convexmodelsofuncertaintyinappliedmechanics.North-Holland. Bennett,J.,&Botkin,M.(1986).Theoptimumshape:Automatedstructuraldesign.PlenumPubCorp. Boukouvalas,A.,&Cornford,D.(2009).LearningheteroscedasticGaussianprocessesforcomplexdatasets.Tech.rep.,Technicalreport,NeuralComputingResearchGroup,AstonUniversity. Box,G.,&Draper,N.(1986).Empiricalmodel-buildingandresponsesurface.WileySeriesinProbabilityandMathematicalStatistics. Chernick,M.(1999).Bootstrapmethods:Apractitioner'sguide.WileyNewYork. Chippereld,A.,Fleming,P.,Pohlheim,H.,&Fonseca,C.(1994).Geneticalgorithmtoolboxusersguide.ACSEResearchReport,512. Choi,K.,Du,L.,Youn,B.,&Gorsich,D.(2005).Possibility-baseddesignoptimizationmethodfordesignproblemswithbothstatisticalandfuzzyinputdata.6thWorldCongressesofStructuralandMultidisciplinaryOptimization,RiodeJaneiro,Brazil. Cook,R.,&Nachtsheim,C.(1980).AcomparisonofalgorithmsforconstructingexactD-optimaldesigns.Technometrics,(pp.315{324). Cressie,N.(1993).Statisticsforspatialdata:WileySeriesinProbabilityandMathematicalStatistics.Findthisarticleonline. Currin,C.,Mitchell,T.,Morris,M.,&Ylvisaker,D.(1991).Bayesianpredictionofdeterministicfunctions,withapplicationstothedesignandanalysisofcomputerexperiments.JournaloftheAmericanStatisticalAssociation,(pp.953{963). Deutsch,C.,&Journel,A.(1997).GSLIB:Geostatisticalsoftwarelibraryanduser'sguide.Appliedgeostatistics,2. Dixon,L.,&Szego,G.(1978).TowardsGlobalOptimisation2.North-Holland. Du,L.,Choi,K.,Youn,B.,&Gorsich,D.(2006).Possibility-baseddesignoptimizationmethodfordesignproblemswithbothstatisticalandfuzzyinputdata.JournalofMechanicalDesign,128,928. 155

PAGE 156

Du,X.,Sudjianto,A.,&Huang,B.(2005).Reliability-baseddesignwiththemixtureofrandomandintervalvariables.Journalofmechanicaldesign,127(6),1068{1076. Efron,B.(1982).Thejackknife,thebootstrapandotherresamplingplans.InCBMS-NSFregionalconferenceseriesinappliedmathematics,vol.38,(p.92).Siam. Elfving,G.(1952).Optimumallocationinlinearregressiontheory.TheAnnalsofMathematicalStatistics,(pp.255{262). Elishako,I.(1991).Anideaoftheuncertaintytriangle.TheShockandVibrationDigest,(p.1). Elishako,I.(2004).Safetyfactorsandreliability:friendsorfoes?.KluwerAcademicPub. Enevoldsen,I.,&Srensen,J.(1994).Reliability-basedoptimizationinstructuralengineering.StructuralSafety,15(3),169{196. Engelund,S.,&Rackwitz,R.(1993).Abenchmarkstudyonimportancesamplingtechniquesinstructuralreliability.StructuralSafety,12(4),255{276. Fedorov,V.(1972).Theoryofoptimalexperiments.NewYork:Academicpress. Fedorov,V.,&Hackl,P.(1997).Model-orienteddesignofexperiments.Springer. Ginsbourger,D.(2009).MetamodelesMultiplespourl'Approximationetl'OptimisationdeFonctionsNumeriquesMultivariables.Ph.D.thesis,EcoleNationaleSuperieuredesMinesdeSaintEtienne. Goel,T.,Haftka,R.,Papila,M.,&Shyy,W.(2006).Generalizedpointwisebiaserrorboundsforresponsesurfaceapproximations.Internationaljournalfornumericalmethodsinengineering,65(12). Haldar,A.,&Mahadevan,S.(2000).Probability,reliabilityandstatisticalmethodsinengineeringdesign.WileyNewYork. Hansen,N.,&Kern,S.(2004).EvaluatingtheCMAevolutionstrategyonmultimodaltestfunctions.LectureNotesinComputerScience,(pp.282{291). Holton,G.(2003).Value-at-risk:theoryandpractice.AcademicPress. Iooss,B.,Ribatet,M.,&Marrel,A.(2008).GlobalSensitivityAnalysisofStochasticComputerModelswithjointmetamodels.ArxivpreprintarXiv:0802.0443. Isaaks,E.,&Srivastava,R.(1989).Anintroductiontoappliedgeostatistics. Jin,R.,Chen,W.,&Sudjianto,A.(2002).Onsequentialsamplingforglobalmetamodelinginengineeringdesign.InProceedingsofDETC,vol.2.Citeseer. 156

PAGE 157

Jones,D.,Schonlau,M.,&Welch,W.(1998).Ecientglobaloptimizationofexpensiveblack-boxfunctions.JournalofGlobalOptimization,13(4),455{492. Kenney,J.,&Keeping,E.(1954).Mathematicsofstatistics,partone.D.VanNostrandCompany,Inc.,Princeton,NJ. Khuri,A.,&Cornell,J.(1996).Responsesurfaces:designsandanalyses.CRC. Kiefer,J.(1961).Optimumdesignsinregressionproblems,II.TheAnnalsofMathematicalStatistics,(pp.298{325). Kiefer,J.,&Wolfowitz,J.(1960).Theequivalenceoftwoextremumproblems.Canad.J.Math,12,363{366. Kim,N.,Ramu,P.,&N.V.,Q.(2006).TailModelinginReliability-BasedDesignOptimizationforHighlySafeStructuralSystems.InProceedingsof47thAIAA/ASME/ASCE/AHS/ASCStructures,StructuralDynamicsandMaterialsConference. Kleijnen,J.,&VanBeers,W.(2004).Application-drivensequentialdesignsforsimulationexperiments:Krigingmetamodelling.JournaloftheOperationalResearchSociety,55(8),876{883. Kleijnen,J.,&vanBeers,W.(2005).RobustnessofKrigingwheninterpolatinginrandomsimulationwithheterogeneousvariances:someexperiments.EuropeanJournalofOperationalResearch,165(3),826{834. Kurtaran,H.,Eskandarian,A.,Marzougui,D.,&Bedewi,N.(2002).Crashworthinessdesignoptimizationusingsuccessiveresponsesurfaceapproximations.Computationalmechanics,29(4),409{421. Kwon,Y.,&Berner,J.(1997).Matrixdamageofbrouscomposites:eectsofthermalresidualstressesandlayersequences.ComputersandStructures,64(1-4),375{382. Lee,J.,Yang,Y.,&Ruy,W.(2002).Acomparativestudyonreliability-indexandtarget-performance-basedprobabilisticstructuraldesignoptimization.ComputersandStructures,80(3-4),257{269. Lin,Y.,Luo,D.,Bailey,T.,Khire,R.,Wang,J.,&Simpson,T.W.(2008).ModelValidationandErrorModelingtoSupportSequentialSampling.InProceedingsoftheASMEDesignEngineeringTechnicalConferences-DesignAutomationConference,NewYork,NY,DETC2008/DAC-49336. Marrel,A.(2008).Miseenoeuvreetutilisationdumetamodeleprocessusgaussienpourl'analysedesensibilitedemodelesnumeriques:Applicationauncodedetransporthydrogeologique.Ph.D.thesis,INSAToulouse. 157

PAGE 158

Martin,J.,&Simpson,T.(2004).AMonteCarlosimulationoftheKrigingmodel.In10thAIAA/ISSMOSymposiumonMultidisciplinaryAnalysisandOptimization,(pp.2004{4483). Matheron,G.(1969).Lekrigeageuniversel.Cahiersducentredemorphologiemathema-tique,1. McKay,M.,Beckman,R.,&Conover,W.(2000).Acomparisonofthreemethodsforselectingvaluesofinputvariablesintheanalysisofoutputfromacomputercode.Technometrics,(pp.55{61). Melchers,R.(1987).Structuralreliability:analysisandprediction.EllisHorwood. Molchanov,I.,&Zuyev,S.(2002).Steepestdescentalgorithmsinaspaceofmeasures.StatisticsandComputing,12(2),115{123. Mourelatos,Z.,Kuczera,R.,&Latcha,M.(2006).AnEcientMonteCarloReliabilityAnalysisusingGlobalandLocalMetamodels.InProceedingsof11thAIAA/ISSMOmultidisciplinaryanalysisandoptimizationconference,September,Portsmouth,VA. Myers,R.,&Montgomery,D.(1995).Responsesurfacemethodology:processandproductoptimizationusingdesignedexperiments.JohnWiley&Sons,NewYork. Neal,D.,Matthews,W.,&Vangel,M.(1992).UncertaintyinObtainingHighReliabilityfroStress-StrengthModels.InProceedingsofthe9thDOD-/NASA/FAAConferenceonFibriousCompositesinStructuralDesign,vol.1,(pp.503{521). Niederreiter,H.(1992).Randomnumbergenerationandquasi-MonteCarlomethods.SocietyforIndustrialMathematics. Nikolaidis,E.,Chen,S.,Cudney,H.,Haftka,R.,&Rosca,R.(2004).Comparisonofprobabilityandpossibilityfordesignagainstcatastrophicfailureunderuncertainty.JournalofMechanicalDesign,126,386. Oakley,J.(2004).Estimatingpercentilesofuncertaincomputercodeoutputs.Appliedstatistics,(pp.83{93). Oakley,J.,&O'Hagan,A.(2004).Probabilisticsensitivityanalysisofcomplexmodels:aBayesianapproach.JournaloftheRoyalStatisticalSociety.SeriesB,StatisticalMethodology,(pp.751{769). Osio,I.,&Amon,C.(1996).AnengineeringdesignmethodologywithmultistageBayesiansurrogatesandoptimalsampling.ResearchinEngineeringDesign,8(4),189{206. Qu,X.,&Haftka,R.(2004).Reliability-baseddesignoptimizationusingprobabilisticsuciencyfactor.StructuralandMultidisciplinaryOptimization,27(5),314{325. 158

PAGE 159

Qu,X.,Haftka,R.,Venkataraman,S.,&Johnson,T.(2003).Deterministicandreliability-basedoptimizationofcompositelaminatesforcryogenicenvironments.AIAAjournal,41(10),2029{2036. Rackwitz,R.(2000).Reliabilityanalysispast,presentandfuture.In8thASCESpecialityConferenceonProbabilisticMechanicsandStructuralReliability. Rajashekhar,R.,&Ellingwood,B.(1993).Anewlookattheresponsesurfaceapproachforreliabilityanalysis.StructuralSafety,(pp.205{220). Ramu,P.,Kim,N.,&Haftka,R.(2007).ErrorAmplicationinFailureProbabilityEstimatesofSmallErrorsinResponseSurfaceApproximations.SAESP,2119,37. Ranjan,P.,Bingham,D.,&Michailidis,G.(2008).Sequentialexperimentdesignforcontourestimationfromcomplexcomputercodes.Technometrics,50(4),527{541. Rasmussen,C.,&Williams,C.(2006).Gaussianprocessesformachinelearning.Springer. Riez,F.,&Nagy,B.(1952).Leconsd`AnalyseFonctionnelle. Sacks,J.,Schiller,S.,&Welch,W.(1989a).Designsforcomputerexperiments.Techno-metrics,(pp.41{47). Sacks,J.,Welch,W.,Mitchell,T.,&Wynn,H.(1989b).Designandanalysisofcomputerexperiments.Statisticalscience,(pp.409{423). Santner,T.,Williams,B.,&Notz,W.(2003).Thedesignandanalysisofcomputerexperiments.Springer. Scheidt,C.(2006).Analysestatistiqued'experiencessimulees:Modelisationadaptativedereponsesnonregulieresparkrigeageetplansd'experiences,Applicationalaquanti-cationdesincertitudeseningenieriedesreservoirspetroliers.Ph.D.thesis,UniversiteLouisPasteurStrasbourg. Shan,S.,&Wang,G.(2004).Spaceexplorationandglobaloptimizationforcomputationallyintensivedesignproblems:aroughsetbasedapproach.StructuralandMultidisciplinaryOptimization,28(6),427{441. Shewry,M.,&Wynn,H.(1987).Maximumentropysampling.JournalofAppliedStatistics,14(2),165{170. Smarslok,B.,Alexander,D.,Haftka,R.,Carraro,L.,&Ginsbourger,D.(2008).SeparableMonteCarloSimulationAppliedtoLaminatedCompositePlatesReliability.In4thMultidisciplinaryDesignOptimizationSpecialistConference.AmericanInstituteofAeronauticsandAstronautics. Sobol,I.(1976).Uniformlydistributedsequenceswithanadditionaluniformproperty.Zh.Vychisl.Mat.Mat.Fiz,16,1332{1337. 159

PAGE 160

StarnesJr,J.,&Haftka,R.(1979).Preliminarydesignofcompositewingsforbuckling,strength,anddisplacementconstraints.JournalofAircraft,16(8),564{570. Stein,M.(1999).Interpolationofspatialdata:Sometheoryforkriging.Springer. Steinberg,D.,&Hunter,W.(1984).Experimentaldesign:reviewandcomment.Techno-metrics,(pp.71{97). Stine,R.(1985).Bootstrappredictionintervalsforregression.JournaloftheAmericanStatisticalAssociation,(pp.1026{1031). Tu,C.,&Barton,R.(1997).Productionyieldestimationbythemetamodelmethodwithaboundary-focusedexperimentdesign.InASMEDesignEngineeringTechnicalConference.AmericanSocietyOfMechanicalEngineers. Turner,C.,Campbell,M.,&Crawford,R.(2003).Genericsequentialsamplingformeta-modelapproximations.InASME2003DesignEngineeringTechnicalConferencesandComputersandInformationinEngineeringConference,(pp.2{6). Vazquez,E.,&Piera-Martinez,M.(2006).EstimationofthevolumeofanexcursionsetofaGaussianprocessusingintrinsickriging.URLhttp://arxiv.org/abs/math.ST/0611273. Venter,G.,Haftka,R.,&Starnes,J.(1998).Constructionofresponsesurfaceapproximationsfordesignoptimization.AIAAjournal,36(12),2242{2249. Williams,B.,Santner,T.,&Notz,W.(2000).Sequentialdesignofcomputerexperimentstominimizeintegratedresponsefunctions.StatisticaSinica,10(4),1133{1152. Wu,C.(1978).Somealgorithmicaspectsofthetheoryofoptimaldesigns.TheAnnalsofStatistics,(pp.1286{1301). Youn,B.,&Wang,P.(2008).Bayesianreliability-baseddesignoptimizationusingeigenvectordimensionreduction(EDR)method.StructuralandMultidisciplinaryOptimization,36(2),107{123. 160

PAGE 161

BIOGRAPHICALSKETCH VictorPichenywasborninParis,Francein1983.AftertwoyearsofpreparatoryclassesatthelyceeEugeneLivetinNantes,hejoinedtheEcoledesMinesdeSaintEtienneforhisgraduateeducation.HereceivedinSeptember2005anengineeringdiplomaandamasterofscienceinappliedmathematics.HepursuedhiseducationbyajointprogrambetweentheEcoledesMinesdeSaintEtienneandtheUniversityofFloridatoworktowardaPh.Dinaerospaceengineeringandappliedmathematics,underthetutelageofProf.RaphaelHaftkaandProf.Nam-HoKim.DuringhisPh.Dstudy,hedidaninternshipattheAppliedComputingInstituteoftheUniversityofMaracaiboinVenezuela.Hisresearchinterestsincludedesignandoptimizationmethods,reliabilityanalysis,surrogatemodeling,anddesignofexperiments. 161