Citation
Convergence Analysis of Gibbs Samplers for Bayesian Regression Models

Material Information

Title:
Convergence Analysis of Gibbs Samplers for Bayesian Regression Models
Creator:
Choi, Hee Min
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (70 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Statistics
Committee Chair:
HOBERT,JAMES P
Committee Co-Chair:
KHARE,KSHITIJ
Committee Members:
SU,ZHIHUA
GILLAND,DAVID R
Graduation Date:
8/9/2014

Subjects

Subjects / Keywords:
Density ( jstor )
Eigenvalues ( jstor )
Ergodic theory ( jstor )
Estimators ( jstor )
Logistic regression ( jstor )
Markov chains ( jstor )
Regression analysis ( jstor )
Sandwiches ( jstor )
Simulations ( jstor )
Statistics ( jstor )
Statistics -- Dissertations, Academic -- UF
bayesian -- convergence -- mcmc -- regression
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Statistics thesis, Ph.D.

Notes

Abstract:
We study the convergence rates of the Markov chains underlying Markov chain Monte Carlo (MCMC) algorithms associated with two widely used Bayesian regression models. First, we analyze the data augmentation (DA) algorithm in Polson, Scott and Windle (2013) for exploring the intractable posterior density that results when the logistic likelihood is combined with normal prior. It is established that the corresponding Markov chain is uniformly ergodic. Second, we analyze the Markov chains underlying DA and Haar PX DA algorithms for the intractable posterior density that results when the standard default prior is placed on the parameters in a linear regression model with Laplace errors. In particular, we establish that the Markov chains underlying the two algorithms are geometrically ergodic. It is also shown that the Haar PX DA algorithm converges at least as fast as the DA algorithm. The convergence rate results developed in this dissertation are important because they are key sufficient conditions for the honest MCMC estimation for the posterior expectations associated with the two Bayesian regression models. By honest MCMC estimation, we mean the MCMC estimates are reported along with an asymptotically valid standard error, and there is an informed decision strategy for determining how long the MCMC simulation should be run. To describe how to carry out an honest estimation procedure, we use a real data set on shuttle O-ring thermal-distress for the Bayesian logistic regression model. We discuss a slightly different topic in the final chapter. When using the MCMC method for exploring an intractable density pi, one is free to choose any MCMC algorithm that has invariant density pi. Obviously, different algorithms have different performance in terms of convergence and efficiency. We carefully address this issue when we use particular MCMC algorithms, general PX DA and Haar PX DA algorithms. In particular, we establish that general Haar PX DA is at least as good as general PX DA in the efficiency ordering and operator norm sense. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ph.D.)--University of Florida, 2014.
Local:
Adviser: HOBERT,JAMES P.
Local:
Co-adviser: KHARE,KSHITIJ.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2015-08-31
Statement of Responsibility:
by Hee Min Choi.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Embargo Date:
8/31/2015
Resource Identifier:
968786170 ( OCLC )
Classification:
LD1780 2014 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

CONVERGENCEANALYSISOFGIBBSSAMPLERSFORBAYESIANREGRESSIONMODELSByHEEMINCHOIADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2014

PAGE 2

c2014HeeMinChoi 2

PAGE 3

Tomyparents 3

PAGE 4

ACKNOWLEDGMENTS Iwouldliketoexpressmydeepestappreciationtomyadvisor,ProfessorJimHobert.Ihavebeenveryfortunatetohaveanadvisorwhocontinuouslyandambitiouslyconveyedaspiritofadventurewhileresearchingandanenthusiasmforteaching.Hissupportandgenuineencouragementhavehelpedmeendurethroughdifculttimes,andthisdissertationwouldnothavebeenpossiblewithouthisguidanceandpatience.IwouldalsoliketothankthefacultyandgraduatestudentsatUF.Ihavebeenveryfortunatetohavestudiedundersuchexcellentprofessorsduringmytimehere.IwishtothankProfessorsKshitijKhare,ZhihuaSu,andDavidGillandforagreeingtoserveonmycommittee.IamespeciallygratefultoProfessorsHaniDoss,MalayGhosh,andMeganMockoforwritingrecommendationlettersforme.AspecialthanksalsogoestoYeunjiJung,TrangNguyen,DanRaghinaru,YeonheePark,JiyounMyung,ChanyoungKim,XiaoWu,AbhishekSaha,SubhadipPal,ChanminKim,SyedRahman,JooyunHwang,TavisAbrahamsen,MelissaCrow,JohnnyWu,ZheChenandmanyotherUFgraduatestudentsfortheirfriendshipandencouragement.IamduallyindebtedtomyfriendsbackinKorea,especially,HyunjooMinandEunbiLee.Ithankthemforalltheemotionalsupport,entertainment,andprayers.Lastly,andmostimportantly,mydeepestappreciationgoestomyparents;wordscannotexpresshowgratefulIamfortheirlove,prayers,andunconditionalsupport.Idedicatethisdissertationtothem. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 7 LISTOFFIGURES ..................................... 8 ABSTRACT ......................................... 9 CHAPTER 1INTRODUCTION ................................... 11 1.1BayesianModelsandMarkovChainMonteCarlo .............. 11 1.2ClassicalandMarkovChainMonteCarloMethods ............. 13 2BACKGROUNDONMARKOVCHAINSANDMCMC ............... 17 2.1SomeMarkovChainTheory .......................... 17 2.2HonestMCMCEstimation ........................... 22 2.3DataAugmentationandSandwichAlgorithms ................ 24 3BAYESIANLOGISTICREGRESSION ....................... 29 3.1Model ...................................... 29 3.2Polson,ScottandWindle'sAlgorithm ..................... 29 3.3UniformErgodicity ............................... 32 3.4HonestEstimationofPosteriorExpectations ................. 36 4BAYESIANLINEARREGRESSIONWITHLAPLACEERRORS ......... 41 4.1Model ...................................... 41 4.2DAandHaarPX-DAAlgorithms ........................ 42 4.3Trace-ClassResult ............................... 47 5THEORETICALCOMPARISONOFPX-DAandHaarPX-DAALGORITHMS . 50 5.1PX-DAandHaarPX-DAAlgorithms ..................... 50 5.2GeneralPX-DAandHaarPX-DAAlgorithms ................ 51 5.2.1HobertandMarchev'sGroupStructure ................ 51 5.2.2ConstructingGeneralPX-DAandHaarPX-DAAlgorithms ..... 52 5.2.3ComparingGeneralPX-DAandHaarPX-DAAlgorithms ...... 54 APPENDIX AEXISTENCEOFMOMENTGENERATINGFUNCTION ............. 57 BEXTENSIONTOBAYESIANQUANTILEREGRESSION ............. 60 5

PAGE 6

CPOSTERIORPROPRIETY ............................. 63 REFERENCES ....................................... 66 BIOGRAPHICALSKETCH ................................ 70 6

PAGE 7

LISTOFTABLES Table page 3-1O-ringthermal-distressdata ............................. 39 3-2Preliminaryresultswhichareusedtodeterminemarginoferrors ........ 39 3-3Resultsbasedonm=138,000iterations ..................... 39 3-4Resultsbasedonm=3,000,000iterations .................... 39 7

PAGE 8

LISTOFFIGURES Figure page 3-1Traceplot:O-ringthermaldistressdataexample ................. 40 8

PAGE 9

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyCONVERGENCEANALYSISOFGIBBSSAMPLERSFORBAYESIANREGRESSIONMODELSByHeeMinChoiAugust2014Chair:JamesP.HobertMajor:StatisticsWestudytheconvergenceratesoftheMarkovchainsunderlyingMarkovchainMonteCarlo(MCMC)algorithmsassociatedwithtwowidelyusedBayesianregressionmodels.First,weanalyze Polson,Scott,&Windle 's( 2013 )dataaugmentation(DA)algorithmforexploringtheintractableposteriordensitythatresultswhenthelogisticlikelihoodiscombinedwithnormalprior.ItisestablishedthatthecorrespondingMarkovchainisuniformlyergodic.Second,weanalyzetheMarkovchainsunderlyingDAandHaarPX-DAalgorithmsfortheintractableposteriordensitythatresultswhenthestandarddefaultpriorisplacedontheparametersinalinearregressionmodelwithLaplaceerrors.Inparticular,weestablishthattheMarkovchainsunderlyingthetwoalgorithmsaregeometricallyergodic.ItisalsoshownthattheHaarPX-DAalgorithmconvergesatleastasfastastheDAalgorithm.TheconvergencerateresultsdevelopedinthisdissertationareimportantbecausetheyarekeysufcientconditionsforthehonestMCMCestimationfortheposteriorexpectationsassociatedwiththetwoBayesianregressionmodels.ByhonestMCMCestimation,wemeantheMCMCestimatesarereportedalongwithanasymptoticallyvalidstandarderror,andthereisaninformeddecisionstrategyfordetermininghowlongtheMCMCsimulationshouldberun.Todescribehowtocarryoutanhonestestimationprocedure,weusearealdatasetonshuttleO-ringthermal-distressfortheBayesianlogisticregressionmodel. 9

PAGE 10

Wediscussaslightlydifferenttopicinthenalchapter.WhenusingtheMCMCmethodforexploringanintractabledensity,oneisfreetochooseanyMCMCalgorithmthathasinvariantdensity.Obviously,differentalgorithmshavedifferentperformanceintermsofconvergenceandefciency.WecarefullyaddressthisissuewhenweuseparticularMCMCalgorithms,generalPX-DAandHaarPX-DAalgorithms.Inparticular,weestablishthatgeneralHaarPX-DAisatleastasgoodasgeneralPX-DAintheefciencyorderingandoperatornormsense. 10

PAGE 11

CHAPTER1INTRODUCTION 1.1BayesianModelsandMarkovChainMonteCarloInBayesianstatisticalanalysis,makinginferencesaboutunknownparametersoftenrequiresthecalculationofintegralswithrespecttotheposteriordensity.Thedensityistypicallyhighdimensionalandcomplicated,sotheclosedformcalculationofposteriorexpectationsisgenerallyimpossible.Apopularsimulation-basedmethodtoapproximatetheseexpectationsistheMonteCarlo(MC)method.Thebasicideaofthe(classical)MCmethodinvolvesgeneratingindependentandidenticallydistributed(iid)samplesfromtheposteriorandusingsampleaveragestoestimateposteriormeans.However,inmanysituations,directsimulationfromisimpossibleorprohibitivelydifcult.Insuchcases,onemaystillresorttotheincreasinglypopularMarkovchainMonteCarlo(MCMC)methodinwhichtheiidsamplesarereplacedbysamplesfromaMarkovchainthathasasaninvariantdensity.ThemaingoalofthisdissertationistoanalyzeMCMCalgorithmsthatallowonetoestimateintractableposteriorexpectationsassociatedwithtwoBayesianregressionmodels,oneinvolvinglogisticregressionandtheotherlinearregressionwithLaplaceerrors( Choi&Hobert , 2013a , b ).Tobespecic,considerabinaryregressionset-upinwhichY1,...,YnareindependentBernoullirandomvariablessuchthatPr(Yi=1j)=F(xTi),wherexiisap1vectorofknowncovariatesassociatedwithYi,isap1vectorofunknownregressioncoefcients,andF:R!(0,1)isthestandardlogisticdistributionfunction,thatis,F(t)=et=(1+et).ThejointmassfunctionofY1,...,YnisgivenbynYi=1Pr(Yi=yij)=nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.96 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.58 0 Td[(yiIf0,1g(yi).WeconsideraBayesianmodelwithaproperNp(b,B)prioron.Letusdenotethepriordensityby().Thentheposteriordensityofgiventhedata,y:=(y1,...,yn)T,is 11

PAGE 12

denedas(jy)=() c(y)nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.95 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.58 0 Td[(yi,wherec(y)isthenormalizingconstant;thatis,c(y)=ZRp()nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.95 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.59 0 Td[(yid.Thisposteriordensityisintractableinthesensethatexpectationswithrespecttotheposterior(jy)cannotbecomputedinclosedform.Moreover,theMonteCarlomethodbasedoniidsamplesfrom(jy)isproblematicwhenthedimension,p,islarge.WeanalyzetheMarkovchainunderlyingthedataaugmentationalgorithmdevelopedin Polsonetal. ( 2013 )forexploring(jy).Inparticular,weestablishthattheMarkovchainisuniformlyergodic.Wenowdescribethesecondmodel.LetfYigni=1beindependentrandomvariablessuchthatYi=xTi+i,wherexi2RpisavectorofknowncovariatesassociatedwithYi,2Rpisavectorofunknownregressioncoefcients,and2(0,1)isanunknownscaleparameter.Theerrors,figni=1,areassumedtobeiidfromtheLaplacedistributionwithscaleequaltotwo,sothecommondensityisd()=e)]TJ /F14 5.978 Tf 7.79 4.03 Td[(jj 2=4.Thejointdensityofthedataisgivenbyf(yj,2)=1 4nnexp)]TJ /F4 11.955 Tf 16.7 8.09 Td[(1 2nXi=1yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi.WeconsideraBayesianmodelwithanimproperprioron(,2)thattakestheform(,2)=(2))]TJ /F7 7.97 Tf 6.59 0 Td[((a+1)=2IR+(2),whereR+=(0,1)andaisahyper-parameter.Ofcourse,wheneveronedealswithanimproperprior,itmustbeestablishedthatthecorrespondingposteriorisproper.WeprovidenecessaryandsufcientconditionsforproprietyinChapter 4 .Assumethattheposterioriswell-dened.Thentheposterior 12

PAGE 13

densityis(,2jy)=f(yj,2)(,2) c0(y),wherec0(y)isthenormalizingconstant;thatis,c0(y)=ZR+ZRpf(yj,2)(,2)dd2.Asinthepreviousmodel,posteriorexpectationswithrespecttotheposterior(,2jy)arenotavailableinclosed-form,anditisnotpossibletomakeiiddrawsfrom(,2jy).WeanalyzetheMarkovchainsunderlyingadataaugmentationalgorithm(whichisconstructedusingascalemixturenormalrepresentationoftheLaplacedistribution)andtheHaarPX-DAalgorithmforexploring(,2jy).Inparticular,weshowthatbothMarkovchainsaregeometricallyergodic.Also,itisestablishedthattheHaarPX-DAalgorithmconvergesatleastasfastasthedataaugmentationalgorithm.TheconvergencerateresultsdevelopedinthisdissertationareimportantbecausetheyarekeysufcientconditionsforthehonestMCMCestimation( Jones&Hobert , 2001 )ofposteriorexpectationsassociatedwiththetwoBayesianregressionmodels.ByhonestMCMCestimation,wemeantheMCMCestimatesarereportedalongwithasymptoticallyvalidstandarderrors,andthereisaninformeddecisionstrategyfordetermininghowlongtheMCMCsimulationshouldberun. 1.2ClassicalandMarkovChainMonteCarloMethodsInthissection,weprovideabriefreviewofthetheorybehindtheclassicalMCandMCMCmethods.Letbeadensitywithrespecttoa-nitemeasureonatopologicalspaceX,andsupposeweareinterestedinestimatingtheintractableintegralEg:=ZXg(x)(x)(dx),wheregisa-integrablefunction. 13

PAGE 14

IntheclassicalMCsetting,alargeiidsample,say=f(i)gm)]TJ /F7 7.97 Tf 6.59 0 Td[(1i=0,isobtainedfrom,andEgisestimatedbythesampleaverage^gm:=1 mm)]TJ /F7 7.97 Tf 6.59 0 Td[(1Xi=0g((i)).Theestimator^gmisunbiasedforEg,andthestronglawoflargenumbersimpliesthat^gm!Egalmostsurelyasm!1.Theestimator^gmusedinpracticeisbasedonanitenumberofiidelements,sothereisanassociatedMCerror,^gm)]TJ /F3 11.955 Tf 12.35 0 Td[(Eg.WecanneverknowthisMCerrorbecausewedonotknowthevalueofEg.However,wecanstudytheasymptoticdistributionoftheestimator^gm.Indeed,ifEg2<1,thereisacentrallimittheorem(CLT)for^gm;thatis,asm!1,wehavep m(^gm)]TJ /F3 11.955 Tf 11.96 0 Td[(Eg)d!N(0,Vg),whereVg=Eg2)]TJ /F4 11.955 Tf 11.95 0 Td[((Eg)2<1.Thesamplevarianceoftheg((i))'s,^Vg,m:=1 m)]TJ /F4 11.955 Tf 11.96 0 Td[(1m)]TJ /F7 7.97 Tf 6.59 0 Td[(1Xi=0)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(g((i)))]TJ /F4 11.955 Tf 12.2 0 Td[(^gm2,isstronglyconsistentforVg.So,using^Vg,m,anasymptoticallyvalidMonteCarlostandarderror(MCSE)for^gmcanbecalculatedbyq ^Vg,m=m.WiththisMCSE,wecanconstructanasymptoticallyvalidcondenceinterval(CI)forEg,^gmzs ^Vg,m m,wherezistheappropriatecriticalvalueofthestandardnormaldistribution.ThisCIallowsustodetermineasufcientMCsamplesize.Forinstance,wechoosemforwhichthehalf-widthoftheCIissufcientlysmall.TheMCMCprocedureissimilarlydeveloped;however,therearesomeimportantdifferences.Asmentionedearlier,intheMCMCmethod,theiiddrawsarereplaced 14

PAGE 15

withdrawsfromaMarkovchain,=f(m)g1m=0,withinvariantdistribution.IntheMCMCsetting,theergodicaveragegm:=1 mm)]TJ /F7 7.97 Tf 6.58 0 Td[(1Xi=0g((i))isusedtoestimateEg.Undersimpleregularityconditions,gmisstronglyconsistentforEg( Meyn&Tweedie , 1993 ,Chapter17),butgmisusuallynotunbiased.Also,unlikeintheMCmethod,verifyingthataCLTholdsfortheergodicaveragegmishardbecauseofdependenceamongMarkovchaindraws.Thatis,thenitesecondmomentcondition,Eg2<1,doesnotinsureaCLTforgm.Indeed,establishingtheexistenceofaCLTtypicallyentailstwosteps.TherststepistoprovethattheMarkovchainisgeometricallyergodic;thatis,thechainconvergestoatageometricrate.Thesecondstepistoshownitesecond(orsometimeshigher)momentcondition(see,e.g., Chan&Geyer , 1994 ; Roberts&Rosenthal , 2004 ).Inpracticallyrelevantsituations,establishinggeometricergodicityofrequiresdifculttheoreticalanalysisrelativetocheckingthemomentcondition.Thestandardapproachtothistheoreticalanalysiscanbedividedintotwocategories.OneapproachisbasedontheprobabilisticconvergenceanalysisoftheMarkovchain,andtheotherapproachexploitsresultsfromtheoperatortheory(seeSection 2.1 ).EvenwhenthereisaCLT,ndingasimple,consistentestimatoroftheasymptoticvarianceisdifcultduetothedependenceintheMarkovchain.Indeed,itrequiresspecializedtechniquessuchasregenerativesimulation,batchmeans,orspectralvariancemethods.Toapplythesemethods,wehavetoverifythetwoconditionsforaCLTplusextraconditionsthataremethoddependent(see,e.g., Flegal&Jones , 2010 ; Hobertetal. , 2002 ; Jonesetal. , 2006 ).Duetothesedifculties,insettingswhereMCMCisused,thereisacultureofrarelyreportingvalidstandarderrors( Flegaletal. , 2008 ). 15

PAGE 16

NotethatwhenusingtheMCMCmethod,oneisfreetochooseanyMCMCalgorithmthathasinvariantdensity.Obviously,differentalgorithmshavedifferentperformanceintermsofconvergenceandefciency.WediscussthisissuewhenweuseparticularMCMCalgorithms,generalPX-DAandHaarPX-DAalgorithms,inthenalchapter.Wenowprovideabriefoverviewoftheremainingfourchaptersofthisdissertation.Chapter 2 containsbasicMarkovchainandoperatortheorythatisusedinanalysisofMarkovchains.MarkovchainCLTsandmethodsforestimatingtheasymptoticvariancesarealsodiscussed.Further,westudydataaugmentationandsandwichalgorithms,twospecicMCMCalgorithmsthatarewidelyused.InChapter 3 ,theMarkovchainunderlying Polson,Scott,&Windle 's( 2013 )dataaugmentationalgorithmforBayesianlogisticregressionisstudied.Inparticular,basedontheprobabilisticconvergencerateanalysis,weestablishuniformergodicityofthechain.AnhonestMCMCprocedureisalsodescribedusingarealdatasetonshuttleO-ringthermal-distress.InChapter 4 ,usingresultsfromtheoperatortheory,weestablishgeometricergodicityoftheMarkovchainsunderlyingdataaugmentationandHaarPX-DAalgorithmsforBayesianlinearregressionwithLaplaceerrors.WealsoshowthattheHaarPX-DAalgorithmconvergesatleastasfastasthedataaugmentationalgorithm.Aslightlydifferenttopicisdiscussedinthenalchapter.WecomparegeneralizationsofPX-DAandHaarPX-DAalgorithms.Indeed,weestablishthatgeneralHaarPX-DAisatleastasgoodasgeneralPX-DAintheefciencyorderingandoperatornormsense. 16

PAGE 17

CHAPTER2BACKGROUNDONMARKOVCHAINSANDMCMC 2.1SomeMarkovChainTheoryLet=f(m)g1m=0denoteatime-homogeneousdiscrete-timeMarkovchainonatopologicalspaceXequippedwithBorel-algebraB(X).LetPm(x,A)bethem-stepMarkovtransitionfunction(Mtf)associatedwithform2N:=f1,2,3,...g.Thenforx2X,A2B(X),m2Nandl2f0,1,2,...g,Pm(x,A)=Pr((l+m)2Aj(l)=x).Asusual,wedenoteP1asP.Form2f2,3,4,...g,PmisdenediterativelybyPm(x,A)=ZXPm)]TJ /F7 7.97 Tf 6.59 0 Td[(1(x0,A)P(x,dx0).Aprobabilitymeasureon(X,B(X))iscalledaninvariantprobabilitymeasureof(orP)if,forallA2B(X),(A)=ZXP(x,A)(dx).Ifitisinvariant,wecanshowbyinductionthatRXPm(x,A)(dx)=(A)forallA2B(X)andallm2N.TheMarkovchain(or,equivalently,theMtfP)issaidtobereversiblewithrespecttoaprobabilitymeasureon(X,B(X))if,forallx,x02X, (dx)P(x,dx0)=(dx0)P(x0,dx).(2)Infact,allMarkovchainsanalyzedinthisdissertationarereversible.Itiseasytoseethatifisreversiblewithrespecttoaprobabilitymeasure,thenhasasaninvariantprobabilitymeasure.Indeed,integratingbothsidesof( 2 )withrespecttoxyieldsZX(dx)P(x,dx0)=(dx0),whichshowsthatisaninvariantprobabilitymeasureoftheMarkovchain. 17

PAGE 18

Wesayis-irreducibleforsomenontrivial-nitemeasureon(X,B(X))if,forallx2XandallA2B(X)with(A)>0,thereexistsapositiveintegerm=m(x,A)suchthatPm(x,A)>0.Ifis-irreducibleforsomemeasure,thenthereexistsaprobabilitymeasure ,calledamaximalirreducibilitymeasuresuchthat(i)is -irreducibleand(ii)foranyothermeasure0,theMarkovchainis0irreducibleifandonlyif0isabsolutelycontinuouswithrespectto ,whichwedenoteby0 .Themaximalirreducibilitymeasureisuniqueuptoequivalence;thatis,if 1and 2arebothmaximalirreducibilitymeasures,then 1 2and 2 1.Thechainiscalled -irreducibleifitis-irreducibleforsomemeasure,and isamaximalirreducibilitymeasure.The -irreducibleMarkovchainhasperioddifXcanbepartitionedintodisjointsetsE0,E1,...,Ed2B(X)suchthatE0isa -nullset,P(x,Ei+1)=1forallx2Ei,i=1,2,...,d)]TJ /F4 11.955 Tf 12.41 0 Td[(1,andP(x,E1)=1forallx2Ed.Ifd2,thenthechainiscalledperiodic,otherwiseitiscalledaperiodic.Wesaythatthe -irreduciblechainisHarrisrecurrentif,foranyx2Xandany -positivesetA,Pr((m)2Ai.o.j(0)=x)=1.Inotherwords,isHarrisrecurrentif,foranyinitialvaluex2Xandany -positivesetA,thechainentersthesetAinnitelyoftenwithprobabilityone.Finally,theMarkovchainthathasaninvariantprobabilitymeasureiscalledHarrisergodicifitis -irreducible,aperiodicandHarrisrecurrent.WenowstateasimplesufcientconditionforHarrisergodicitythatiseasytocheckandholdsforallMarkovchainsstudiedinChapters 3 and 4 andmanyotherMarkovchainsthatareusedinpractice.Inordertostatethecondition,werstneedadenition.Supposethatisa-nitemeasureon(X,B(X))andthatk:XX![0,1)isa 18

PAGE 19

functionsuchthat,forallx2XandallA2B(X),P(x,A)=ZAk(x0jx)(dx0).ThenthefunctionkiscalledaMarkovtransitiondensity(Mtd)ofwithrespectto.Hereistheresult,whoseproofcanbefoundin Tan&Hobert ( 2009 ). Lemma1. SupposeisaMarkovchainwithMtdkandinvariantprobabilitymeasure.Ifk(j)isstrictlypositiveonXX,thenisHarrisergodic.IfisHarrisergodicwithinvariantprobabilitymeasure,thenthetotalvariationdistancebetweentheprobabilitymeasuresPm(x,)and()decreasesto0asmgetslargeforeachxedx2X(see,e.g., Meyn&Tweedie , 1993 ).Insymbols,asm!1,kPm(x,))]TJ /F4 11.955 Tf 11.95 0 Td[(()kTV:=supA2B(X)jPm(x,A))]TJ /F4 11.955 Tf 11.95 0 Td[((A)j#0.Notethatthisdoesnottellusanythingabouttherateofconvergence.TheHarrisergodicMarkovchainiscalledgeometricallyergodicifthereexistanonnegativefunctionVdenedonX,andaconstant2[0,1)suchthat,forallx2Xandallm2N,kPm(x,))]TJ /F4 11.955 Tf 11.95 0 Td[(()kTVV(x)m.Moreover,ifthefunctionVisboundedabove,thenthechainissaidtobeuniformlyergodic.Now,wediscussonestandardmethodofestablishinguniformergodicity.Theideaistoconstructaminorizationconditionthatholdsontheentirestatespace.Wesaythatsatisesa(m0-step)minorizationconditiononEXifthereexistaprobabilitymeasureQon(X,B(X)),apositiveintegerm0,and>0suchthatPm0(x,A)Q(A)forallx2EandallA2B(X).Hereistheresult,whichcanbeshownbyastandardcouplingargument(see,e.g., Rosenthal , 1995 ). 19

PAGE 20

Theorem2.1. LetdenoteaHarrisergodicMarkovchainon(X,B(X))withinvariantprobabilitymeasureandm-stepMtfPm,andletkbeaMtdofwithrespecttoa-nitemeasure.SupposethatthereexistadensityqonXwithrespecttoand>0suchthat,forallx,x02X,k(xjx0)q(x).Thenforallx2Xandallm2N,kPm(x,))]TJ /F4 11.955 Tf 11.95 0 Td[(()kTV(1)]TJ /F5 11.955 Tf 11.96 0 Td[()m.OthermethodsofprovinguniformergodicitycanbefoundinTheorem16.0.2of Meyn&Tweedie ( 1993 ).Intheremainderofthissection,wegiveabriefreviewoftheoperatortheoryusedintheanalysisofreversibleMarkovchains.(See Hobertetal. ( 2011 )foranintroduction.)Assumethatisreversiblewithrespecttoaprobabilitymeasure.LetL20()bethevectorspaceofreal-valued,measurable,mean-zerofunctionsonXthataresquare-integrablewithrespectto,i.e.,L20()=g:X!R:ZXjg(x)j2(dx)<1andZXg(x)(dx)=0.ThisisaHilbertspacewheretheinnerproductofg,h2L20()isdenedashg,hi=ZXg(x)h(x)(dx),andthecorrespondingnormiskgk=hg,gi1=2.LetP:L20()!L20()denotetheoperatorthatmapsg2L20()to(Pg)(x):=ZXg(x0)P(x,dx0).Ofcourse,(Pg)(x)isjusttheexpectedvalueofg((m+1))giventhat(m)=x.Itiseasytoshow,usingreversibility( 2 ),thatforallg,h2L20(),hPg,hi=hg,Phi;thatis,Pis 20

PAGE 21

self-adjoint.TheoperatornormofPisdenedaskPk=supkPgk kgk:g2L20(),g6=0.AsimpleapplicationofJensen'sinequalityshowsthatkPk2[0,1].Infact,kPkprovidesagreatdealofinformationabouttheconvergencebehaviorofthecorrespondingMarkovchain.Forinstance,ifthereversibleMarkovchainisHarrisergodic,thenisgeometricallyergodicifandonlyifkPk<1( Roberts&Rosenthal , 1997 ).Moreover,inthegeometricallyergodiccase,kPkcanbeviewedastheasymptoticrateofconvergenceof,withsmallervalueassociatedwithfasterconvergence(see,e.g., Rosenthal , 2003 ).WenowdenethespectrumofP,whichiscloselyrelatedtothenormkPk.LetIdenotetheidentityoperatoronL20().ThespectrumofP,Sp(P),isthesetofallrealnumberssuchthatP)]TJ /F5 11.955 Tf 12.18 0 Td[(Iisnotinvertible.Therefore,2Sp(P)ifandonlyifatleastoneofthefollowingtwostatementsistrue: (i) TherangeofP)]TJ /F5 11.955 Tf 11.96 0 Td[(IisnotallofL20(). (ii) TheoperatorP)]TJ /F5 11.955 Tf 11.95 0 Td[(Iisnotone-to-one.Inparticular,if(ii)holds;thatis,thereexistsanonzerog2L20()suchthatPg=g,thenissaidtobeaneigenvalueofP(see,e.g., Rudin , 2005 ,p.104).Notethatthecollectionofeigenvaluesdoesnotnecessarilyexhaustthespectrum.ItfollowsfromstandardoperatortheorythatSp(P)[kPk,kPk][)]TJ /F4 11.955 Tf 9.29 0 Td[(1,1](see,e.g., Retherford , 1993 ,p.50).Typically,whenthecardinalityofthestatespaceXisinnite,Sp(P)canbequitecomplexandmaycontainanuncountablenumberofpoints.However,iftheself-adjointoperatorPispositiveandcompact,thenSp(P)hasaparticularlysimpleform.Werstgivedenitions.WecallPapositiveoperatorifhPg,gi0forallg2L20().ItcanbeshownthatifPispositive,thenSp(P)[0,kPk][0,1](see,e.g., Rudin , 2005 ,p.330).PissaidtobecompactifPmapstheunitball,fg2L20():kgk1g,intoasetin 21

PAGE 22

L20()whoseclosureiscompact.Now,wedescribethestructureofSp(P)incasewherePispositiveandcompact(see,e.g., Retherford , 1993 ,p.61). Theorem2.2. Supposethattheself-adjointMarkovoperatorPispositiveandcompact.ThenPhasonlyacountablenumberofdistincteigenvalues,andalleigenvaluesarein[0,1].Letfng1n=1denotetheordered,distinct,nonzeroeigenvaluesofP.Thenthespectrum,Sp(P),consistssolelyoftheeigenvaluesfng1n=1andthepointf0g.Moreover,Sp(P)hasatmostonelimitpoint,namely,f0g.Finally,thenormofPisequaltoitslargesteigenvalue;thatis,kPk=1.NotethatTheorem 2.2 canbeusedtoanalyzethecorrespondingMarkovchain.Tobespecic,assumeisHarrisergodic,andsupposetheassociatedself-adjointoperatorPispositiveandcompact.Then,since1cannotbeaneigenvalueforaHarrisergodicMarkovchain(seee.g., Chan&Geyer , 1994 ,Lemma5),Theorem 2.2 impliesthat1=kPk<1;thatis,isgeometricallyergodic. 2.2HonestMCMCEstimationLetdenoteaprobabilitydensitywithrespecttoa-nitemeasureon(X,B(X)),andford>0,letLd()bethevectorspaceofreal-valued,measurablefunctionsgonXsuchthatEjgjd<1.SupposeweareinterestedinestimatingtheintractableintegralEg=ZXg(x)(x)(dx),whereg2L1().IftheMarkovchain=f(m)g1m=0withinvariantdensityisHarrisergodic,thenTheorem17.0.1of Meyn&Tweedie ( 1993 )impliesthat,asm!1,gm=1 mm)]TJ /F7 7.97 Tf 6.58 0 Td[(1Xi=0g((i))!Egwithprobabilityonenomatterwhatthedistributionof(0).However,asmentionedinSection 1.2 ,HarrisergodicityisnotenoughtoguaranteetheexistenceofaCLTforgm.Thefollowingtheoremisawell-knownresultonsufcientconditionsfortheexistenceofaCLT(see Ibragimov&Linnik , 1971 ,Theorem18.5.3). 22

PAGE 23

Theorem2.3. SupposetheMarkovchainwithinvariantdensityisHarrisergodic.Ifisgeometricallyergodic,andthereexists>0suchthatg2L2+(),thenforanyinitialdistributionof(0),asm!1,p m(gm)]TJ /F3 11.955 Tf 11.96 0 Td[(Eg)d!N(0,2g),where2g=Var)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(g)]TJ /F4 11.955 Tf 5.48 -9.68 Td[((0)+21Xm=1Cov)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(g)]TJ /F4 11.955 Tf 5.48 -9.68 Td[((0),g)]TJ /F4 11.955 Tf 5.48 -9.68 Td[((m)and(m)1m=0isthestationaryversionof;thatis,(0). Remark1. InTheorem 2.3 ,ifwealsohavetheconditionthatthechainisreversible,thentheCLTholdsundertheweakerassumptionofnitesecondmoment;thatis,g2L2()( Roberts&Rosenthal , 1997 ,Corollary3). Remark2. Ifisuniformlyergodic,thentheaboveCLTholdsforg2L2()( Cogburn , 1972 ,Corollary4.2). Remark3. Theasymptoticvariance2gintheCLTdependsonthefunctiongbeingintegratedandtheMtfof.Here,thedependencyofMtfissuppressedinthenotation.UnliketheCLTintheclassicalMCmethod(whichwasdiscussedinSection 1.2 ),ndingasimple,consistentestimatoroftheasymptoticvariance2gischallengingduetotheinherentserialcorrelationintheMarkovchain.Hence,estimating2grequiresspecializedtechniquessuchasnon-overlappingbatchmeans(BM),overlappingbatchmeans(OBM)orspectralvariance(SV)methods.StrongconsistencyoftheBMestimatorsisstudiedby Jonesetal. ( 2006 ).Recently, Flegal&Jones ( 2010 )providesufcientconditionsforthestrongconsistencyofOBMandSVmethods.Consistencyofallofthesemethodsrequiresatleastgeometricergodicityofthechainandaone-stepminorizationcondition.Inaddition,theconsistencyresultsforBMandOBMrequireg2L2+();however,thoseforSVestimatorsrequirethestrongercondition,g2L4+().See Flegal&Jones ( 2010 )forpreciseformulasandempiricalnite-samplepropertiesoftheestimators. 23

PAGE 24

ConsistentestimationoftheasymptoticvarianceallowsustoconstructanasymptoticallyvalidCIforEg.Givenaconsistentestimator^2g,m,avalidstandarderrorisq ^2g,m=mandCIisgmqs ^2g,m m,whereqisanappropriatequantile.OnceweareabletocomputeanasymptoticallyvalidCI,wemustdecidewhentoterminatethesimulation.Acommonstrategyistousethexed-widthruleinwhichthesimulationisallowedtocontinueuntiladesiredCIwidthisattained.Thexed-widthmethodrequiresstoppingthesimulationthersttime qs ^2g,m m+p(m)4,(2)wherep()isafunctiononthesetofpositiveintegerssuchthatp(m)=o(m)]TJ /F7 7.97 Tf 6.59 0 Td[(1=2),and4isthedesiredhalf-widthoftheCI(see,e.g., Jonesetal. , 2006 ).Here,theroleofthefunctionp()istoensurethatthesimulationisnotstoppedprematurelyduetoapoorestimateof2g.Oneexampleofsuchfunctionsisp(m)=4I(0,m0)(m)+1 m,wherem0issomepre-speciedsamplesize.Theproceduresbasedon( 2 )areasymptoticallyvalidinthatifthegoalistoobtaina100(1)]TJ /F5 11.955 Tf 11.95 0 Td[()%CIwidth24,thenPr(Eg2Int[T(4)])!1)]TJ /F5 11.955 Tf 11.95 0 Td[(as4!0,whereT(4)isthersttimethat( 2 )issatised,andInt[T(4)]istheintervalatthistime( Glynn&Whitt , 1992 ).Theregenerativemethod(RS)canalsobeusedtoformasymptoticallyvalidcondenceintervalsforEg.RScanbeappliedunderthesameconditionsastheBMandOBMestimators( Hobertetal. , 2002 ).However,RSisbasedonadifferentCLT,anditisdifculttoapplywhentheclosedformexpressionofMtdisnotavailable,whichisthecasefor Polson,Scott,&Windle 's( 2013 )algorithm(Section 3.2 ). 24

PAGE 25

2.3DataAugmentationandSandwichAlgorithmsInthissection,wedescribedataaugmentationandsandwichalgorithms,twospecicMCMCalgorithmsthatarewidelyused.Supposeisanintractabledensitywithrespecttoa-nitemeasureonXthatwewouldliketoexplore.(Weusetodenotethecorrespondingmeasure.)AssumethatYisatopologicalspaceequippedwithBorel-algebraB(Y)andthatisa-nitemeasureonY.Tobuildadataaugmentation(DA)algorithm,onemustndajointdensity,say,f:XY![0,1)(withrespectto)suchthatRYf(x,y)(dy)=(x)andsamplingfromtheassociatedconditionaldensitiesfXjYandfYjXisstraightforward.TheDAalgorithm( Tanner&Wong , 1987 )isbasedontheMtdgivenby k(x0jx)=ZYfXjY(x0jy)fYjX(yjx)(dy).(2)Sincek(x0jx)(x)=k(xjx0)(x0)forallx,x2X,itiseasytoseethattheMarkovchaindrivenbytheDAalgorithm,whichwedenoteby=f(m)g1m=0,isreversiblewithrespect.Ofcourse,theDAchaincanbesimulatedbydrawingalternatelyfromthetwoconditionaldensities.Ifthecurrentstateis(m)=x,then(m+1)issimulatedintwosteps:drawYfYjX(jx),calltheresulty,andthendraw(m+1)fXjY(jy).WenowdiscusssomeimportantpropertiesoftheoperatorsassociatedwithDAalgorithms.SupposeanoperatorKonL20()correspondstotheDAchainthatisHarrisergodic.ThenitiseasytoseethatKisself-adjointandpositive( Liuetal. , 1994 ).Furthermore,ifksatisesthefollowinginequality:ZXk(xjx)(dx)<1,thenKistrace-class(see,e.g., Buja , 1990 ; Diaconisetal. , 2008 ).Here,theself-adjoint,positiveoperatorKiscalledtrace-classifitiscompactandthesumofitseigenvaluesisnite(see,e.g., Conway , 1990 ,p.267).TheDAalgorithmisconsideredausefulalgorithmthatsometimessuffersfromslowconvergence.Itssandwichvariant(thesandwichalgorithm)isaviablealternativetoDA 25

PAGE 26

thatoftenconvergesmuchfasterwhilerequiringroughlythesamecomputationaleffortperiteration(see,e.g., Hobertetal. , 2011 ; vanDyk&Meng , 2001 ).LetfYdenotetheymarginaldensityoff.TheMtdofthesandwichalgorithmisgivenby kQ(x0jx)=ZYZYfXjY(x0jy0)Q(y,dy0)fYjX(yjx)(dy),(2)whereQ(y,dy0)isaMtfon(Y,B(Y))thatisreversiblewithrespecttofY(y)(dy).(GeneralmethodsforbuildingQ(y,dy0)canbefoundin Hobert&Marchev ( 2008 ); Liu&Wu ( 1999 ).)ItiseasytoseethatkQ(x0jx)(x)issymmetricin(x,x0),sotheMarkovchain,~=f~(m)g1m=0,underlyingthesandwichalgorithmisreversiblewithrespectto.Thesimulationofthesandwichalgorithmisaccomplishedusingathree-stepprocedureinwhichtherstandthirdstepsareexactlythetwostepsusedtosimulatetheDAalgorithm,andthemiddlestepinvolvesamoveaccordingtoQ(y,dy0).Precisely,ifthecurrentstateofthenewchainis~(m)=x,then~(m+1)canbesimulatedasfollows.DrawYfYjX(jx),calltheobservedvaluey,thendrawY0Q(y,),calltheresulty0,andnallydraw~(m+1)fXjY(jy0).Whatmakesthefastconvergencepossibleisthefactthata(typicallylow-dimensional)perturbationontheYspace(introducedbythemiddlestepdraw)canleadtoasubstantialimprovementinmixing.Insomecases,kQcanbereexpressedastheMtdofaDAchain;thatis,thereexistsajointdensityf(x,y)onXYwithrespecttowhosexmarginalissuchthatkQ(x0jx)=ZYZYfXjY(x0jy0)Q(y,dy0)fYjX(yjx)(dy)=ZYfXjY(x0jy)fYjX(yjx)(dy).Itisshownin Hobert&Marchev ( 2008 )thatifkQcanbeexpressedasaDAalgorithm,thenitisatleastasgoodaskintermsofperformanceintheCLTandintheoperatornormsense.Inordertoformallystatethisresult,weneedadenition.AssumethatMCMCwillbeusedtoestimatethenite,intractableexpectationEg=RXg(x)(x)(dx).SupposewehaveavailabletwoMtfsPandQwithinvariantdensitythatareHarrisergodic.AssumethatthereexistsaCLT(fortheergodicaverage).Recallthatthe 26

PAGE 27

asymptoticvariance2gintheCLTdependsonthefunctiongandtheMtf,soletusdenote2gfortwoMtfsby2g(P)and2g(Q).(IfthereisnoCLT,thenwesimplywrite2g=1.)WesayPisatleastasgoodasQintheefciencyordering,writtenPEQ,if2g(P)2g(Q)foreveryg2L2()( Mira&Geyer , 1999 ).Hereistheresult. Theorem2.4. SupposethatQisaMtfonYthathasfYasaninvariantdensity.LetkandkQbeasin( 2 )and( 2 ),anddenotetheassociatedoperatorsonL20()byKandKQ,respectively.AssumethattheMarkovchainsdrivenbykandkQareHarrisergodic.IfkQisreversiblewithrespectto,thenkQEk.If,inaddition,kQisaDAalgorithm,thenkKQkkKk. Remark4. Theorem 2.4 canbeappliedtocomparetwoDAchains.Indeed,ifwehaveavailabletwodifferentDAalgorithmsthathavethesameinvariantdensity,andonealgorithmadmitsasandwichrepresentationwithrespecttotheother,thenTheo-rem 2.4 allowsustocomparethetwoalgorithms.ThisideaisactuallyusedtoprovethecomparisonresultinSection 5.2.3 .Weendthissectionbydescribingapopularspecialcaseofthesandwichalgorithm,namely,theHaarPX-DAalgorithm( Liu&Wu , 1999 ).InHaarPX-DA,themiddlestepMtf,whichwedenotebyR(y,dy0),isconstructedusingagroupactionandtheHaarmeasureonthatgroup.Tobespecic,supposethatGisatopologicalgroupwithidentityelemente.Supposefurtherthatthereisaone-to-oneanddifferentiablefunction(groupaction)tgonYsuchthat(i)te(y)=yforally2Yand(ii)tg1g2(y)=tg1(tg2(y))forallg1,g22Gandally2Y.AssumethatGisunimodularwithHaarmeasure%.LetJg(y)denotetheJacobianofthetransformationz=t)]TJ /F7 7.97 Tf 6.59 0 Td[(1g(y).TheMtfR(y,dy0)isassociatedwithanoperatorRonL20(fY)denedby(Rh)(y)=ZGh(tg(y))fY(tg(y))jJg(y)j m(y)%(dg),whereweassumem(y)=RGfY(tg(y))jJg(y)j%(dg)ispositive,niteforally2Y.Thatis,theMtfR(y,dy0)correspondstothemovey!y0=tg(y),wheregisarandom 27

PAGE 28

elementfromGwhosedensity(withrespectto%)isfY(tg(y))jJg(y)j=m(y).ItisshowninProposition3of Hobert&Marchev ( 2008 )thatR(y,dy0)isreversiblewithrespecttofY(y)(dy).Also,Theorem4of Hobert&Marchev ( 2008 )impliesthattheHaarPX-DAalgorithmcanbeexpressedasaDAalgorithm.(SeeChapter 5 forbroaderdiscussionontheHaarPX-DAalgorithm.) 28

PAGE 29

CHAPTER3BAYESIANLOGISTICREGRESSION 3.1ModelConsiderabinaryregressionset-upinwhichY1,...,YnareindependentBernoullirandomvariablessuchthatPr(Yi=1j)=F(xTi),wherexiisap1vectorofknowncovariatesassociatedwithYi,isap1vectorofunknownregressioncoefcients,andF:R!(0,1)isthestandardlogisticdistributionfunction,thatis,F(t)=et=(1+et).ThejointmassfunctionofY1,...,YnisgivenbynYi=1Pr(Yi=yij)=nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.96 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.58 0 Td[(yiIf0,1g(yi).ABayesianversionofthemodelrequiresapriordistributionfortheunknownregressionparameter,.WeputaproperNp(b,B)prioron.If()isthedensityofNp(b,B),thentheposteriordensityofgiventhedata,y=(y1,...,yn)T,isdenedas(jy)=() c(y)nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.95 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.58 0 Td[(yi,wherec(y)isthenormalizingconstant;thatis,c(y)=ZRp()nYi=1F(xTi)yi1)]TJ /F3 11.955 Tf 11.95 0 Td[(F(xTi)1)]TJ /F8 7.97 Tf 6.59 0 Td[(yid.Thisposteriordensityisintractableinthesensethatexpectationswithrespecttotheposterior(jy),whicharerequiredforBayesianinference,cannotbecomputedinclosedform.Moreover,theMonteCarlomethodbasedoniidsamplesfrom(jy)isproblematicwhenthedimension,p,islarge.ThesedifcultieshaveledtothedevelopmentofMarkovchainMonteCarloalgorithmsforexploring(jy)(see,e.g., Fruhwirth-Schnatter&Fruhwirth , 2010 ; Holmes&Held , 2006 ; Marchev , 2011 ; Polsonetal. , 2013 ).Inthenextsection,wedescribethemethodof Polsonetal. ( 2013 ). 29

PAGE 30

3.2Polson,ScottandWindle'sAlgorithmBeforewedescribethealgorithm,weneedtointroducewhat Polsonetal. ( 2013 )(hereafter,PS&W)callthePolya-Gammadistribution.LetfEkg1k=1beasequenceofiidExp(1)randomvariablesanddeneU=2 21Xk=1Ek (2k)]TJ /F4 11.955 Tf 11.96 0 Td[(1)2.Itiswell-known(see,e.g., Bianeetal. , 2001 )thattherandomvariableUhasdensity h(u)=1Xk=0()]TJ /F4 11.955 Tf 9.3 0 Td[(1)k(2k+1) p 2u3e)]TJ /F16 5.978 Tf 7.78 3.86 Td[((2k+1)2 8uI(0,1)(u),(3)andthatitsLaplacetransformisgivenbyEe)]TJ /F8 7.97 Tf 6.59 0 Td[(tU=cosh)]TJ /F7 7.97 Tf 6.58 0 Td[(1)]TJ 5.48 1.12 Td[(p t=2.PS&WcreatethePolya-Gammafamilyofdensitiesthroughanexponentialtiltingofthedensityh.Indeed,consideraparametricfamilyofdensities,indexedbyc0,thattakestheformg(x;c)=cosh(c=2)e)]TJ /F11 5.978 Tf 7.78 3.26 Td[(c2x 2h(x).Ofcourse,whenc=0,werecovertheoriginaldensity.Arandomvariablewithdensityg(x;c)issaidtohaveaPG(1,c)distribution.Here,weintroduceabitmorenotation.Asusual,letXdenotethenpmatrixwhoseithrowisxTi,andletR+=(0,1).Forxedw2Rn+,dene(w)=)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(XT(w)X+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F7 7.97 Tf 6.58 0 Td[(1and~m(w)=(w))]TJ /F3 11.955 Tf 5.48 -9.68 Td[(XT(y)]TJ /F7 7.97 Tf 13.84 4.71 Td[(1 21n)+B)]TJ /F7 7.97 Tf 6.58 0 Td[(1b,where(w)isthenndiagonalmatrixwhoseithdiagonalelementiswi,and1nisann1vectorof1's.WenowstatePS&W'sDAalgorithm.Let=f(m)g1m=0beaMarkovchainwithstatespaceRpwhosedynamicsaredened(implicitly)throughthefollowingtwo-stepprocedureformovingfromthecurrentstate,(m)=,to(m+1). 1. DrawW1,...,WnindependentlywithWiPG(1,jxTij)andcalltheobservedvaluew=(w1,...,wn)T 30

PAGE 31

2. Draw(m+1)Np)]TJ /F4 11.955 Tf 7.33 -9.69 Td[(~m(w),(w)HighlyefcientmethodsofsimulatingPolya-GammarandomvariablesareprovidedbyPS&W.WenowdescribealatentdataformulationthatleadstoPS&W'sDAalgorithm.Thisdevelopmentisdifferent,andwebelievesomewhatmoretransparent,thanthatgivenbyPS&W.Conditionalon,letf(Yi,Wi)gni=1beindependentrandompairssuchthatYiandWiarealsoindependent,withYiBernoulli)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(exTi=(1+exTi)andWiPG(1,jxTij).LetW=(W1,...,Wn)Tanddenoteitsdensitybyf(wj).Combiningthislatentdatamodelwiththeprior,(),yieldstheaugmentedposteriordensitydenedas(,wjy)=Qni=1Pr(Yi=yij)f(wj)() c(y).Clearly,ZRn+(,wjy)dw=(jy),whichisourtargetposteriordensity.PS&W'sDAalgorithmisbasedontheaugmentedposteriordensity(,wjy).Indeed,PS&W'sDAalgorithmalternatesbetweendrawsfromtwoconditionaldensities(jw,y)and(wj,y).TheconditionalindependenceofYiandWiimpliesthat(wj,y)=f(wj).Therefore,wecandrawfrom(wj,y)bymakingnindependentdrawsfromthePolya-Gammadistribution.Theotherconditionaldensityismultivariatenormal.Toseethis,notethat(jw,y)/nYi=1Pr(Yi=yij)f(wj)()="nYi=1)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(exTiyi 1+exTi#"nYi=1coshjxTij 2e)]TJ /F16 5.978 Tf 7.78 5.34 Td[((xTi)2wi 2h(wi)#()/()nYi=1)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(exTiyi 1+exTicoshjxTij 2e)]TJ /F16 5.978 Tf 7.78 5.34 Td[((xTi)2wi 2=2)]TJ /F8 7.97 Tf 6.58 0 Td[(n()nYi=1expyixTi)]TJ /F3 11.955 Tf 13.15 8.09 Td[(xTi 2)]TJ /F4 11.955 Tf 13.15 8.09 Td[((xTi)2wi 2/exp)]TJ /F4 11.955 Tf 10.5 8.09 Td[(1 2T)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2TXTy)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 21n+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1b, 31

PAGE 32

wherethelastequalityfollowsfromthefactthatcosh(z)=(1+e2z)=(2ez).AroutineBayesianregression-typecalculationthenrevealsthatjw,yNp(~m(w),(w)),where~m(w)and(w)aredenedabove.TheMtdoftheDAchain,=f(m)g1m=0,isk)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(0=ZRn+(jw,y)(wj0,y)dw.Byconstruction,(jy)isaninvariantdensityforthisMtd(seeSection 2.3 ).Notethatk:RpRp!(0,1);thatis,kisstrictlypositive.ItfollowsimmediatelyfromLemma 1 thattheMarkovchainisHarrisergodic. 3.3UniformErgodicityInthissection,weestablishtheuniformergodicityofPS&W'sMarkovchain.Inordertointroducetheresult,weneedabitmorenotation.Let(z;0,V0)denotethemultivariatenormaldensitywithmean0andcovariancematrixV0evaluatedatthepointz.RecallthattheprioronisNp(b,B),anddenes=B1 2XTy)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 21n+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b.Hereistheresult. Proposition3.1. TheMtdofsatisesthefollowingminorizationconditionk)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(0(;m,),wherem=)]TJ /F7 7.97 Tf 6.68 -4.98 Td[(1 2XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F7 7.97 Tf 6.59 0 Td[(1B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2s,=)]TJ /F7 7.97 Tf 6.68 -4.98 Td[(1 2XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F7 7.97 Tf 6.58 0 Td[(1,and=jj1=2 jBj1=2exp1 2mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1me)]TJ /F11 5.978 Tf 7.79 3.26 Td[(n 42)]TJ /F8 7.97 Tf 6.59 0 Td[(nexp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(1 2sTs.Hence,isuniformlyergodic.ThefollowinglemmaswillbeusedintheproofofProposition 3.1 . Lemma2. IfAisappsymmetricnonnegativedenitematrix,thenalloftheeigenval-uesof(I+A))]TJ /F7 7.97 Tf 6.59 0 Td[(1arein(0,1],andI)]TJ /F4 11.955 Tf 11.95 0 Td[((I+A))]TJ /F7 7.97 Tf 6.59 0 Td[(1isalsononnegativedenite. 32

PAGE 33

Proof. WritethespectraldecompositionofthenonnegativedenitematrixAasOTDO,soOisanorthogonalmatrixandDisadiagonalmatrixcontainingtheeigenvaluesofA,whichwedenotebyfdigpi=1.Then(I+A))]TJ /F7 7.97 Tf 6.59 0 Td[(1=OT(I+D))]TJ /F7 7.97 Tf 6.59 0 Td[(1O.Notethatthediagonalelementsofthediagonalmatrix(I+D))]TJ /F7 7.97 Tf 6.58 0 Td[(1aregivenbyf(1+di))]TJ /F7 7.97 Tf 6.58 0 Td[(1gpi=1,wheredi'sarenonnegative.Thisshowsthattheeigenvaluesof(I+A))]TJ /F7 7.97 Tf 6.59 0 Td[(1aref(1+di))]TJ /F7 7.97 Tf 6.58 0 Td[(1gpi=1,andtheyarein(0,1].Therefore,forallx2Rp,xT)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(I)]TJ /F4 11.955 Tf 11.95 0 Td[((I+A))]TJ /F7 7.97 Tf 6.59 0 Td[(1x=xTOT)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(I)]TJ /F4 11.955 Tf 11.96 0 Td[((I+D))]TJ /F7 7.97 Tf 6.59 0 Td[(1Ox0,whichcompletestheproof. Lemma3. Fora,b2R,cosh(a+b)2cosh(a)cosh(b). Proof. Straightforwardcalculationsrevealthat2cosh(a)cosh(b)=2ea+e)]TJ /F8 7.97 Tf 6.59 0 Td[(a 2eb+e)]TJ /F8 7.97 Tf 6.58 0 Td[(b 2=ea+b+e)]TJ /F7 7.97 Tf 6.59 0 Td[((a+b)+ea)]TJ /F8 7.97 Tf 6.59 0 Td[(b+e)]TJ /F7 7.97 Tf 6.59 0 Td[((a)]TJ /F8 7.97 Tf 6.58 0 Td[(b) 2=cosh(a+b)+cosh(a)]TJ /F3 11.955 Tf 11.95 0 Td[(b).Sincecosh(a)]TJ /F3 11.955 Tf 11.96 0 Td[(b)0,theresultfollows. WearenowreadytoproveProposition 3.1 . ProofofProposition 3.1 . Recallthat=(w)=)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(XT(w)X+B)]TJ /F7 7.97 Tf 6.58 0 Td[(1)]TJ /F7 7.97 Tf 6.59 0 Td[(1and~m=~m(w)=(w))]TJ /F3 11.955 Tf 5.48 -9.69 Td[(XT(y)]TJ /F7 7.97 Tf 13.15 4.71 Td[(1 21n)+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1b.Webeginbyshowingthatjj)]TJ /F16 5.978 Tf 7.78 3.25 Td[(1 2jBj)]TJ /F16 5.978 Tf 7.78 3.25 Td[(1 2.Indeed,jj=(XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1))]TJ /F7 7.97 Tf 6.59 0 Td[(1=)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2B1 2XTXB1 2B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2)]TJ /F7 7.97 Tf 6.59 0 Td[(1=jBj(~XT~X+I))]TJ /F7 7.97 Tf 6.59 0 Td[(1,where~X=XB1 2.Now,since~XT~Xisnonnegativedenite,Lemma 2 impliesthatjjjBj,andtheresultfollows.Next,weshowthat~mT)]TJ /F7 7.97 Tf 6.58 0 Td[(1~msTs.Letting~y=y)]TJ /F7 7.97 Tf 10.83 4.71 Td[(1 21n,wehave~mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1~m=(XT~y+B)]TJ /F7 7.97 Tf 6.58 0 Td[(1b)T(XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1))]TJ /F7 7.97 Tf 6.59 0 Td[(1(XT~y+B)]TJ /F7 7.97 Tf 6.58 0 Td[(1b) 33

PAGE 34

=(XT~y+B)]TJ /F7 7.97 Tf 6.58 0 Td[(1b)TB1 2(~XT~X+I))]TJ /F7 7.97 Tf 6.59 0 Td[(1B1 2(XT~y+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1b)=(~XT~y+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b)T(~XT~X+I))]TJ /F7 7.97 Tf 6.59 0 Td[(1(~XT~y+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b)(~XT~y+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b)T(~XT~y+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b)=sTs,wheretheinequalityfollowsfromLemma 2 .Usingthesetwoinequalities,wehave(jw,y)=(2))]TJ /F11 5.978 Tf 7.78 3.53 Td[(p 2jj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2()]TJ /F4 11.955 Tf 13.8 0 Td[(~m)T)]TJ /F7 7.97 Tf 6.58 0 Td[(1()]TJ /F4 11.955 Tf 13.81 0 Td[(~m)=(2))]TJ /F11 5.978 Tf 7.78 3.52 Td[(p 2jj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2T(XTX))]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2~mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1~m+~mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1(2))]TJ /F11 5.978 Tf 7.78 3.53 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2T(XTX))]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2sTs+~mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1=(2))]TJ /F11 5.978 Tf 7.78 3.53 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2nXi=1wi(xTi)2)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.16 8.09 Td[(1 2sTs+sTB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2=(2))]TJ /F11 5.978 Tf 7.78 3.53 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2sTs+sTB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2"nYi=1exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[((xTi)2 2wi#.Nowsince(wj,y)=nYi=1coshjxTij 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[((xTi)2 2wih(wi),itfollowsthat(jw,y)(wj0,y)(2))]TJ /F11 5.978 Tf 7.78 3.52 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.16 8.09 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2sTs+sTB)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2"nYi=1coshjxTi0j 2exp)]TJ /F10 11.955 Tf 11.95 16.85 Td[((xTi)2+(xTi0)2 2wih(wi)#.Recallthatk(j0)=RRn+(jw,y)(wj0,y)dw.Tothisend,notethatZR+exp)]TJ /F10 11.955 Tf 11.96 16.86 Td[((xTi)2+(xTi0)2 2wih(wi)dwi=(cosh p (xTi)2+(xTi0)2 2!))]TJ /F7 7.97 Tf 6.58 0 Td[(1(cosh jxTij 2+jxTi0j 2!))]TJ /F7 7.97 Tf 6.59 0 Td[(1 34

PAGE 35

(2cosh jxTij 2!cosh jxTi0j 2!))]TJ /F7 7.97 Tf 6.58 0 Td[(1,wheretherstinequalityisduetothefactthatp a+bp a+p bfornonnegativea,b,andthesecondinequalityisbyLemma 3 .ItfollowsthatZRn+"nYi=1coshjxTi0j 2exp)]TJ /F4 11.955 Tf 10.49 8.09 Td[((xTi)2+(xTi0)2 2wih(wi)#dw2)]TJ /F8 7.97 Tf 6.59 0 Td[(n"nYi=1coshjxTij 2#)]TJ /F7 7.97 Tf 6.58 0 Td[(1.Moreover,"nYi=1coshjxTij 2#)]TJ /F7 7.97 Tf 6.58 0 Td[(1"nYi=1expjxTij 2#)]TJ /F7 7.97 Tf 6.59 0 Td[(1"nYi=1exp(xTi)2+1 4#)]TJ /F7 7.97 Tf 6.59 0 Td[(1=exp()]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2nXi=1(xTi)2 2+1 2)=e)]TJ /F11 5.978 Tf 7.78 3.26 Td[(n 4exp()]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TXTX 2),wherethesecondinequalityholdsbecausejaj(a2+1)=2foranyreala.Puttingallofthistogether,wehavek(j0)=ZRn+(jw,y)(wj0,y)dw(2))]TJ /F11 5.978 Tf 7.79 3.53 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 2TB)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 2sTs+sTB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 22)]TJ /F8 7.97 Tf 6.58 0 Td[(ne)]TJ /F11 5.978 Tf 7.79 3.26 Td[(n 4exp()]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 2TXTX 2)=(2))]TJ /F11 5.978 Tf 7.79 3.53 Td[(p 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2TXTX 2+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1+sTB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 22)]TJ /F8 7.97 Tf 6.59 0 Td[(ne)]TJ /F11 5.978 Tf 7.78 3.26 Td[(n 4)]TJ /F11 5.978 Tf 7.79 3.26 Td[(sTs 2=(2))]TJ /F11 5.978 Tf 7.79 3.53 Td[(p 2jj)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2expn)]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 2()]TJ /F3 11.955 Tf 11.96 0 Td[(m)T)]TJ /F7 7.97 Tf 6.59 0 Td[(1()]TJ /F3 11.955 Tf 11.95 0 Td[(m)ojj1 2jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 22)]TJ /F8 7.97 Tf 6.59 0 Td[(ne)]TJ /F11 5.978 Tf 7.78 3.26 Td[(n 4)]TJ /F11 5.978 Tf 7.78 3.26 Td[(sTs 2+mT)]TJ /F16 5.978 Tf 5.76 0 Td[(1m 2=(;m,),andthereforeisuniformlyergodicbyTheorem 2.1 . Remark5. ItisworthpointingoutthatourproofcircumventstheneedtodealdirectlywiththeunwieldydensityofthePG(1,0)distribution,whichisgivenin( 3 ). 35

PAGE 36

3.4HonestEstimationofPosteriorExpectationsInthissection,wedescribehowtoconstructasymptoticallyvalidcondenceintervalsforintractableposteriorexpectationsofregressionparametersofthelogisticregressionmodel.Forillustration,weusespaceshuttleO-ringthermal-distressdatafrom Dalaletal. ( 1989 )presentedinTable 3-1 .Thedataconsistof(yi,xi)fori=1,...,23,wherexiisthejointtemperature(F)ofithightatlaunch,andyiisanindicatorforO-ringsthatexperiencedsomethermaldistress,say“incident”.WeconsideraBayesianversionofthefollowingmodel,Pr(Yi=1j0,1)=FL(0+1xi),withafairlydiffuseprior(0,1)N2(0,1000I).RecallthatwecanconstructasymptoticallyvalidcondenceintervalsforposteriorexpectationsusingthemethodsdescribedinSection 2.2 providedthattheMarkovchainisgeometricallyergodic,anappropriatemomentconditionholdsforthefunctionofinterest,andaone-stepminorizationconditionholdsforthechain.RecallalsothatBM,OBM,andRSmethodsrequireanite“2+”momentconditionwhiletheSVmethodrequiresthestrongerconditionofnite“4+”moment.WenotethatProposition 3.1 impliesuniformergodicityandaone-stepminorizationconditionofPS&W'sDAchainforthisproblem.Moreover,itfollowsfromProposition A.1 (inAppendix A )that,fori=0,1andalla>0,Ejgija<1,wheregi()=i.Therefore,allofthemethodsexceptforRSdiscussedinSection 2.2 canbeusedinthisexample.Again,RSisdifculttoapplybecausetheMtdofPS&W'salgorithmisnotavailableinclosedform.Here,weadopttheSVmethodwiththeTukey-Hanning(TH)window.ThepreciseformoftheTHestimatorof2gis^2g,m=tm)]TJ /F7 7.97 Tf 6.59 0 Td[(1Xs=)]TJ /F7 7.97 Tf 6.59 0 Td[((tm)]TJ /F7 7.97 Tf 6.58 0 Td[(1)wm(s)^m(s), 36

PAGE 37

wheretmiscalledthetruncationpoint,thelagwindowwm(s)isgivenbywm(s)=1 2+1 2cosjsj tmI()]TJ /F8 7.97 Tf 6.59 0 Td[(tm,tm)(s),and^m(s)=^m()]TJ /F3 11.955 Tf 9.3 0 Td[(s):=1 mm)]TJ /F8 7.97 Tf 6.59 0 Td[(sXt=1)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(g((t)))]TJ /F4 11.955 Tf 12.2 0 Td[(gm)]TJ /F3 11.955 Tf 12.95 -9.68 Td[(g((t+s)))]TJ /F4 11.955 Tf 12.19 0 Td[(gm.Wenowspecifyatruncationpoint,soconsiderusingtm=bmcforsome>0.AnapplicationofTheorem1andLemma7of Flegal&Jones ( 2010 )yieldsthatifEjgj4+1+2<1forsome1,2>0,and2 4+1<<1 2,thentheTHestimatorwithtm=bmcisstronglyconsistentfor2g.Toestimatetheasymptoticvariancescorrespondingtofunctionsfgig1i=0,wewilluseTHestimatorswithtruncationpointtm=bm1=3c.TheseestimatorscanbeshowntobestronglyconsistentsinceEjgija<1fori=0,1andalla>0.Infact,adefaultmethodsuggestedby Flegal&Jones ( 2010 )isTHwithtm=bm1=2c.However,thisestimatormightbeinconsistentandiscomputationallydemanding.Supposethatwewanttoconstructapproximate95%BonferoniCIsforfEgig1i=0withhalf-widths,f4ig1i=0,usingthexed-widthmethoddiscussedinSection 2.2 .Supposefurtherthatwewant4itobeabitlessthan0.5%ofapreliminaryestimateofEgi.Tochoosef4ig1i=0,wesimulatePS&W'salgorithmfor1,000,000iterationsusingthedefaultBayesianlogisticregressionimplementedbylogitcommandinR-packageBayesLogit.Wethenset(40,41)=(0.0875,0.00135)usingthepreliminaryMCMCestimatesoftheposteriorexpectationswhicharegiveninTable 3-2 .Eventhoughburn-inisnotrequiredforxed-widthmethods,wethrewawaytherst1,000,000drawsasburn-in.(ThisseemedtoimprovetheperformanceoftheTH 37

PAGE 38

estimators.)WethenranthePS&W'salgorithmuntilthersttime 2.241s ^2gi,m m+4iI(0,10000)(m)+1 m4i(3)foralli2f0,1g.Toreducethecomputationaleffort,weonlychecked( 3 )every3,000iterationsstartingat12,000.Thestoppingpointwasm=138,000iterations.(Table 3-3 containsasummaryofoursimulationresults.)Wenotethatwhenusingxed-widthmethods,thereisnoguaranteetheMarkovchainisrunlongenoughsothattheestimatorsf^2gi,mg1i=0provideagoodestimateofthecorrespondingasymptoticvariances.Forinstance,Figure 3-1 showsthat^2g1,mstartstostabilizeabitafter1,000,000thiteration.Welookedatthesimilartraceplotcorrespondingtotheotherfunctionofinterest,andtheseplotssuggestthatallestimatorshavestabilizedbythe2,000,000thiteration.SeeTable 3-4 forasummaryoftheresultsbasedon3,000,000thiteration.Infact,itmaybepossibletondsomeeffectiveapproaches(toestimateintractableposteriorexpectations)otherthanPS&W'salgorithmforthisexample.Theseincludeiidsamplingbasedonarejectionsampler,independenceMetropolissamplingandLaplaceapproximationmethods(seee.g., Fruhwirth-Schnatteretal. , 2009 ; Zellner&Rossi , 1984 ).However,numericalevidencesuggeststhatPS&W'salgorithmisthebestcurrentmethod.Indeed,numericalexperiments(inPS&W)supportthatPS&W'salgorithmgenerallyworksbetterthanindependenceMetropolissamplerandLaplaceapproximation.Moreover,as Liu ( 1996 )pointedout,independenceMetropolissamplingissuperiortorejectionsamplinginefciencyandeaseofcomputation. 38

PAGE 39

Table3-1. O-ringthermal-distressdata FlightIncidentTemperatureFlightIncidentTemperature 106641-G078217051-A067306951-C153506851-D067606751-B075707251-G070807351-F081907051-I07641-B15751-J07941-C16361-A17541-D17061-B07661-C158 Table3-2. Preliminaryresultswhichareusedtodeterminemarginoferrors Parameter01 Estimate17.537)]TJ /F4 11.955 Tf 9.3 0 Td[(0.270 Table3-3. Resultsbasedonm=138,000iterations ParameterEstimate^&2g,mp ^&2g,m=m95%BonferroniCI 017.510207.2800.0388(17.424,17.597)1-0.2690.04610.000587(-0.271,-0.268) Table3-4. Resultsbasedonm=3,000,000iterations ParameterEstimate^&2g,mp ^&2g,m=m95%BonferroniCI 017.516218.4310.00853(17.4969,17.5351)1-0.2690.04850.000127(-0.2693,-0.2687) 39

PAGE 40

Figure3-1. ThetopplotshowsthebehavioroftheestimatorofE1andthecorrespondingapproximate95%BonferroniCI.ThesolidlinerepresentstheestimatorandthedashedlinesdenotetheupperandlowerboundsoftheCI.ThemiddleplotshowsthebehavioroftheTHestimatoroftheasymptoticvariance,^&2g1,m.Thebottomplotshowsthetrajectoryofthemarginoferror2.241q ^2g1,m=m. 40

PAGE 41

CHAPTER4BAYESIANLINEARREGRESSIONWITHLAPLACEERRORS 4.1ModelLetfYigni=1beindependentrandomvariablessuchthat Yi=xTi+i,(4)wherexi2RpisavectorofknowncovariatesassociatedwithYi,2Rpisavectorofunknownregressioncoefcients,and2(0,1)isanunknownscaleparameter.Theerrors,figni=1,areassumedtobeiidfromtheLaplacedistributionwithscaleequaltotwo,sothecommondensityisd()=e)]TJ /F14 5.978 Tf 7.78 4.03 Td[(jj 2=4.(ThereasonforthepeculiarscalingismadeclearinAppendix B .)TheLaplacedistributionisoftenusedasaheavy-tailedalternativetotheGaussiandistributionwhenthedatacontainoutliers(see,e.g., West , 1984 ).AsmentionedinSection 1.1 ,weconsideraBayesianmodelwithanimproperprioron(,2)thattakestheform(,2)=(2))]TJ /F7 7.97 Tf 6.59 0 Td[((a+1)=2IR+(2),whereR+=(0,1)andaisahyper-parameter.Thestandarddefaultpriorcanberecoveredbytakinga=1(see,e.g., Kass&Wasserman , 1996 ).Ofcourse,wheneveronedealswithanimproperprior,itmustbeestablishedthatthecorrespondingposteriorisproper.Thejointdensityofthedataisgivenbyf(yj,2)=1 4nnexp)]TJ /F4 11.955 Tf 16.7 8.09 Td[(1 2nXi=1yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi,wherey=(y1,...,yn).Bydenition,theposteriordensityisproperifc0(y)=ZR+ZRpf(yj,2)(,2)dd2<1.Asusual,letXdenotethenpmatrixwhoseithrowisxTi,andalsoletC(X)denotethecolumnspaceofX.Wenowstatetheproprietyresult,whichisproveninAppendix C . Proposition4.1. Theposteriorisproper(thatis,c0(y)<1)ifandonlyifXhasfullcolumnrank,a>)]TJ /F3 11.955 Tf 9.3 0 Td[(n+p+1,andy=2C(X). 41

PAGE 42

Fernandez&Steel ( 2000 )showthatwhena=1,theposteriorisproperifandonlyifXhasfullcolumnrankandy=2C(X).Wecandemonstratethatthisresultisaspecialcaseoftheaboveproposition.Indeed,whena=1,itfollowsfromProposition 4.1 thattheposteriorisproperifandonlyifthen(p+1)augmentedmatrix(X:y)hasfullcolumnrankandn>p.Notethatif(X:y)hasfullcolumnrank,thennp+1,andthustheresultfollows. 4.2DAandHaarPX-DAAlgorithmsAssumenowthatc0(y)<1sothattheposteriordensity,whichwedenoteby(,2jy),iswell-dened.Recallthattheposteriorisintractableinthesensethatposteriorexpectationscannotbecomputedinclosedform.Also,itisnotpossibletomakeiiddrawsfrom(,2jy).WenowdescribeapairofMCMCalgorithmsforthisproblem(mentionedinSection 1.1 ).OneisaDAalgorithmthatisbasedonarepresentationoftheLaplacedensityasascalemixtureofnormalswithrespecttotheinversegammadistribution.TheotheristheHaarPX-DAalgorithmthatrequiresoneadditionalsimulationstepateachiteration.Inordertoformallystatethealgorithms,wemustintroduceabitmorenotation.WhenwewriteWIG(,),wemeanthatWhasdensityproportionaltow)]TJ /F17 7.97 Tf 6.59 0 Td[()]TJ /F7 7.97 Tf 6.58 0 Td[(1e)]TJ /F17 7.97 Tf 6.59 0 Td[(=wIR+(w),whereandarestrictlypositiveparameters.Similarly,whenwewriteWInverseGaussian(,),wemeanthatWhasdensitygivenbyr 2w3exp)]TJ /F5 11.955 Tf 13.15 8.09 Td[((w)]TJ /F5 11.955 Tf 11.96 0 Td[()2 22wIR+(w),whereandarestrictlypositiveparameters.Givenz2Rn+,letQbeannndiagonalmatrixwhoseithdiagonalelementisz)]TJ /F7 7.97 Tf 6.59 0 Td[(1i.Also,dene=(XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X))]TJ /F7 7.97 Tf 6.58 0 Td[(1and=(XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X))]TJ /F7 7.97 Tf 6.59 0 Td[(1XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y.WenowstatetheDAalgorithm.Let=f)]TJ /F5 11.955 Tf 5.48 -9.69 Td[((m),(2)(m)g1m=0beaMarkovchainwithstatespaceX=RpR+whosedynamicsaredened(implicitly)throughthefollowingthree-stepprocedureformovingfromthecurrentstate,)]TJ /F5 11.955 Tf 5.48 -9.68 Td[((m),(2)(m)=(,2),to)]TJ /F5 11.955 Tf 5.48 -9.68 Td[((m+1),(2)(m+1). 42

PAGE 43

1. DrawZ1,...,ZnindependentlywithZi(InverseGaussian 2jyi)]TJ /F8 7.97 Tf 6.58 0 Td[(xTij,1 4ifjyi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTij>0IG)]TJ /F7 7.97 Tf 6.68 -4.97 Td[(1 2,1 8ifjyi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTij=0andcalltheobservedvaluez=(z1,...,zn)T 2. Draw(2)(m+1)IGn)]TJ /F8 7.97 Tf 6.59 0 Td[(p+a)]TJ /F7 7.97 Tf 6.59 0 Td[(1 2,yTQ)]TJ /F16 5.978 Tf 5.75 0 Td[(1y)]TJ /F17 7.97 Tf 6.59 0 Td[(T)]TJ /F16 5.978 Tf 5.75 0 Td[(1 2 3. Draw(m+1)Np)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(,(2)(m+1)WenowderivetheDAalgorithm.Whena=1,thisalgorithmisaspecialcaseofthatin Roy&Hobert ( 2010 ).Whilethederivationforthea6=1caseissimilartothea=1case,wefeelthatitisworthwhiletogivethedetailshereforcompleteness.Webeginbyintroducingonelatentrandomvariableforeachobservation.Indeed,letf(Yi,Zi)gni=1beindependentrandompairssuchthatYijZi=ziN(xTi,2=zi)and,marginally,ZiIG(1,1=8).AstraightforwardcalculationshowsthatthemarginaldensityofYiis(4))]TJ /F7 7.97 Tf 6.59 0 Td[(1ejyi)]TJ /F8 7.97 Tf 6.58 0 Td[(xTij=(2),whichispreciselythedensityofYiundertheoriginalmodel.Therefore,ifweletz=(z1,...,zn)Tandwedenotethejointdensityoff(Yi,Zi)gni=1byf(y,z;,2),then ZRn+f(y,z;,2)dz=f(yj,2),(4)showingthatfZigni=1areindeedlatentvariables.WenowusethelatentdatatoconstructtheDAalgorithm.Combiningthelatentdatamodelwiththeprior,(,2),yieldstheaugmentedposteriordensitydenedas(,2,zjy)=f(y,z;,2)(,2) c0(y).Itfollowsimmediatelyfrom( 4 )thatZRn+(,2,zjy)dz=(,2jy), 43

PAGE 44

whichisourtargetposteriordensity.TheDAalgorithmisbasedontheaugmentedposteriordensity(,2,zjy).Indeed,theDAalgorithmalternatesbetweendrawsfromtwoconditionaldensities(,2jz,y)and(zj,2,y).Therstofthethreestepsdrawsfrom(zj,2,y).Indeed,(zj,2,y)/(,2,zjy)/f(y,z;,2)(,2),whichisgivenby "nYi=1z1 2i (2)1 2(2)1 2exp)]TJ /F3 11.955 Tf 13.15 8.09 Td[(zi 2(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2 2#"nYi=11 8z2ie)]TJ /F16 5.978 Tf 10.46 3.25 Td[(1 8zi#"(2))]TJ /F11 5.978 Tf 7.78 3.26 Td[(a+1 2#.(4)Thus,conditionalon(,2,y),fZigni=1areindependent,andtheconditionaldensityofzigiven(,2,y)isproportionaltoz)]TJ /F16 5.978 Tf 7.78 3.26 Td[(3 2iexp)]TJ /F3 11.955 Tf 13.15 8.09 Td[(zi 2(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2 2)]TJ /F4 11.955 Tf 17.43 8.09 Td[(1 8zi.Whenjyi)]TJ /F3 11.955 Tf 9.71 0 Td[(xTij>0,thisisan(unnormalized)inverseGaussiandensitywith= 2jyi)]TJ /F8 7.97 Tf 6.58 0 Td[(xTijand=1=4.Ontheotherhand,ifjyi)]TJ /F3 11.955 Tf 12.33 0 Td[(xTij=0,thenitisan(unnormalized)inversegammadensitywith=1=2and=1=8.Hence,ineithercase,theconditionaldensityofzigiven(,2,y)takestheform 1 81 2z)]TJ /F16 5.978 Tf 7.78 3.25 Td[(3 2iexp)]TJ /F3 11.955 Tf 13.15 8.09 Td[(zi(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi)2 22+jyi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTij 2)]TJ /F4 11.955 Tf 17.43 8.09 Td[(1 8zi.(4)Notethatthesecondandthirdstepsofthealgorithmresultinadrawfrom(,2jz,y).Infact,thesecondstepyieldsadrawfrom(2jz,y),whilethethirdstepresultsinadrawfrom(j2,z,y).Asafunctionof,( 4 )isproportionaltoexp)]TJ /F4 11.955 Tf 19.16 8.09 Td[(1 22nXi=1zi(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2.However,nXi=1zi(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi)2=(y)]TJ /F3 11.955 Tf 11.96 0 Td[(X)TQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1(y)]TJ /F3 11.955 Tf 11.96 0 Td[(X)=T(XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X))]TJ /F4 11.955 Tf 11.96 0 Td[(2TXTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y+yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y=()]TJ /F5 11.955 Tf 11.96 0 Td[()T)]TJ /F7 7.97 Tf 6.59 0 Td[(1()]TJ /F5 11.955 Tf 11.96 0 Td[()+[yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.58 0 Td[(1]. (4) 44

PAGE 45

Thus,j2,z,yNp)]TJ /F5 11.955 Tf 5.48 -9.69 Td[(,2.Finally,itisclearthattheconditionaldensityof2given(z,y)isproportionaltoRRpf(y,z;,2)(,2)d,anditfollowsfrom( 4 )thatthisquantityisitselfproportionalto(2))]TJ /F16 5.978 Tf 7.78 3.86 Td[((n)]TJ /F11 5.978 Tf 5.76 0 Td[(p+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1) 2)]TJ /F7 7.97 Tf 6.58 0 Td[(1exp)]TJ /F10 11.955 Tf 13.15 18.53 Td[()]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1 22,whichisthekerneloftheIGdistributionfromthethirdstepoftheDAalgorithm.Here,weestablishthaty=2C(X)impliesyTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1>0.Notethat(y)]TJ /F3 11.955 Tf 11.96 0 Td[(X)TQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1(y)]TJ /F3 11.955 Tf 11.96 0 Td[(X)=yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F4 11.955 Tf 11.96 0 Td[(2yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X+T(XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X)=yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F4 11.955 Tf 11.96 0 Td[(2yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X(XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X))]TJ /F7 7.97 Tf 6.59 0 Td[(1XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y+yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X(XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X))]TJ /F7 7.97 Tf 6.58 0 Td[(1XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y=yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F3 11.955 Tf 11.96 0 Td[(yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X(XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X))]TJ /F7 7.97 Tf 6.59 0 Td[(1XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y=yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1.Sincey=2C(X),wehavey)]TJ /F3 11.955 Tf 12.16 0 Td[(X6=0.Therefore,itfollowsfrompositivedenitenessofQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1that(y)]TJ /F3 11.955 Tf 11.95 0 Td[(X)TQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1(y)]TJ /F3 11.955 Tf 11.95 0 Td[(X)=yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.95 0 Td[(T)]TJ /F7 7.97 Tf 6.58 0 Td[(1>0.TheMtdoftheDAchain,=f)]TJ /F5 11.955 Tf 5.48 -9.68 Td[((m),(2)(m)g1m=0,isk)]TJ /F4 11.955 Tf 6.95 -7.03 Td[(^,^2,2=ZRn+(^,^2jz,y)(zj,2,y)dz.Byconstruction,(,2jy)isaninvariantdensityforthisMtd(seeSection 2.3 ).Moreover,becausekisstrictlypositive,itfollowsfromLemma 1 thatthecorrespondingMarkovchain,,isHarrisergodic.WenowdescribetheHaarPX-DAalgorithm,whichisasandwichalgorithm.Let~=f(~(m),(~2)(m))g1m=0beasecondMarkovchainonXwhosedynamicsaredened(implicitly)throughthefollowingfour-stepprocedureformovingfromthecurrentstate,(~(m),(~2)(m))=(,2),to(~(m+1),(~2)(m+1)). 45

PAGE 46

1. Draw^Z1,...,^Znindependentlywith^Zi(InverseGaussian 2jyi)]TJ /F8 7.97 Tf 6.58 0 Td[(xTij,1 4ifjyi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTij>0IG)]TJ /F7 7.97 Tf 6.68 -4.97 Td[(1 2,1 8ifjyi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTij=0andcalltheobservedvalue^z=(^z1,...,^zn)T 2. DrawgIG2n+a)]TJ /F4 11.955 Tf 11.96 0 Td[(1 2,nXi=11 8^zi,andsetz=(g^z1,...,g^zn)T 3. Draw(~2)(m+1)IGn)]TJ /F3 11.955 Tf 11.95 0 Td[(p+a)]TJ /F4 11.955 Tf 11.96 0 Td[(1 2,yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1 2 4. Draw~(m+1)Np)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(,(~2)(m+1)TheMtdoftheHaarPX-DAchain,~=f(~(m),(~2)(m))g1m=0,canbewrittenas~k)]TJ /F4 11.955 Tf 6.95 -7.03 Td[(~,~2,2=ZRn+ZRn+(~,~2jz,y)R(^z,dz)(^zj,2,y)d^z,whereR(^z,dz)istheMtfinducedbythesecondstepthattakes^z!z=(g^z1,...,g^zn)T.TheMtfR(^z,dz)isconstructedusingagroupactionandtheHaarmeasureonthatgroup,asdescribedinSection 2.3 .Tobespecic,letGbethemultiplicativegroupR+wheregroupcompositionisdenedasmultiplication.Thenthe(unimodular)HaarmeasureonGis%(dg)=dg=gwheredgdenotesLebesguemeasureonR+.AllowGtoactonRn+throughcomponent-wisemultiplication;thatis,tg(z)=gz=(gz1,...,gzn)T.Therandomvariablegisdrawnfromadensity(withrespecttoLebesguemeasure)thatisproportionalto(gzjy)gn)]TJ /F7 7.97 Tf 6.59 0 Td[(1IR+(g).Notethat(zjy)=ZR+ZRp(,2,zjy)dd2/)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F3 11.955 Tf 11.95 0 Td[(yTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X(XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1X))]TJ /F7 7.97 Tf 6.58 0 Td[(1XTQ)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)]TJ /F16 5.978 Tf 7.78 3.86 Td[((n)]TJ /F11 5.978 Tf 5.76 0 Td[(p+a)]TJ /F16 5.978 Tf 5.75 0 Td[(1) 2 XTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1X1 2"nYi=1z)]TJ /F16 5.978 Tf 7.78 3.26 Td[(3 2ie)]TJ /F16 5.978 Tf 10.46 3.26 Td[(1 8zi#. 46

PAGE 47

Hence,(gzjy)gn)]TJ /F7 7.97 Tf 6.59 0 Td[(1IR+(g)/g)]TJ /F16 5.978 Tf 7.78 3.26 Td[(2n+a+1 2exp)]TJ /F4 11.955 Tf 13.4 8.08 Td[(1 gnXi=11 8ziIR+(g).So,wemustdrawgIG2n+a)]TJ /F4 11.955 Tf 11.95 0 Td[(1 2,nXi=11 8zi.Byconstruction,(,2jy)isaninvariantdensityof~(seeSection 2.3 ).Again,it'seasytoseethattheMarkovchain~isHarrisergodic. 4.3Trace-ClassResultInthissection,westudythetheoreticalpropertiesoftheMarkovchains,and~,underlyingtheDAandHaarPX-DAalgorithms.Inordertodescribetheresult,wemustintroducetheoperatorsassociatedwiththeMarkovchainsand~.Asbefore,letL20()bethespaceofreal-valued,measurablefunctionswithdomainXthataresquareintegrableandhavemeanzerowithrespecttotheposterior.LetKand~KdenotetheoperatorsonL20()denedbyMtdskand~k,respectively.Hereistheresult. Theorem4.1. TheMarkovoperatorsKand~Karebothtrace-class.Moreover,lettingfig1i=1andf~ig1i=1denotetheorderedeigenvaluesofKand~K,respectively,wehavethat0~ii<1foralli2N,and~i
PAGE 48

nYi=1"1 81 2z)]TJ /F16 5.978 Tf 7.79 3.26 Td[(3 2iexp)]TJ /F3 11.955 Tf 13.15 8.09 Td[(zi(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi)2 22+1 12zi9z2i(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2 2+1)]TJ /F4 11.955 Tf 17.43 8.09 Td[(1 8zi#="nYi=11 81 2z)]TJ /F16 5.978 Tf 7.78 3.26 Td[(3 2iexp)]TJ /F4 11.955 Tf 20.57 8.09 Td[(1 24zi#exp1 42nXi=1zi(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi)2="nYi=11 81 2z)]TJ /F16 5.978 Tf 7.78 3.26 Td[(3 2iexp)]TJ /F4 11.955 Tf 20.57 8.09 Td[(1 24zi#exp()]TJ /F5 11.955 Tf 11.95 0 Td[()T)]TJ /F7 7.97 Tf 6.59 -.01 Td[(1()]TJ /F5 11.955 Tf 11.95 0 Td[() 42expyTQ)]TJ /F7 7.97 Tf 6.59 -.01 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 -.01 Td[(1 42, (4)wheretheinequalityholdsbecausejxj(x2+1)=2,andthelastequalityisdueto( 4 ).Nownotethat(j2,z,y)exp()]TJ /F5 11.955 Tf 11.95 0 Td[()T)]TJ /F7 7.97 Tf 6.59 0 Td[(1()]TJ /F5 11.955 Tf 11.95 0 Td[() 42=(2))]TJ /F11 5.978 Tf 7.78 3.53 Td[(p 22)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2exp)]TJ /F4 11.955 Tf 10.93 8.08 Td[(()]TJ /F5 11.955 Tf 11.96 0 Td[()T)]TJ /F7 7.97 Tf 6.58 0 Td[(1()]TJ /F5 11.955 Tf 11.96 0 Td[() 42,sothat ZRp(j2,z,y)exp()]TJ /F5 11.955 Tf 11.95 0 Td[()T)]TJ /F7 7.97 Tf 6.59 0 Td[(1()]TJ /F5 11.955 Tf 11.95 0 Td[() 42d=2p 2.(4)Similarly,(2jz,y)expyTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1 42=)]TJ /F8 7.97 Tf 6.68 -4.65 Td[(yTQ)]TJ /F16 5.978 Tf 5.75 0 Td[(1y)]TJ /F17 7.97 Tf 6.59 0 Td[(T)]TJ /F16 5.978 Tf 5.75 0 Td[(1 2n)]TJ /F11 5.978 Tf 5.75 0 Td[(p+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1 2 )]TJ /F10 11.955 Tf 6.78 9.69 Td[()]TJ /F8 7.97 Tf 6.67 -4.65 Td[(n)]TJ /F8 7.97 Tf 6.58 0 Td[(p+a)]TJ /F7 7.97 Tf 6.59 0 Td[(1 2)]TJ /F5 11.955 Tf 5.48 -9.69 Td[(2)]TJ /F16 5.978 Tf 7.78 3.86 Td[((n)]TJ /F11 5.978 Tf 5.76 0 Td[(p+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1) 2)]TJ /F7 7.97 Tf 6.58 0 Td[(1exp)]TJ /F10 11.955 Tf 13.15 18.53 Td[()]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.96 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1 42,sothatZR+(2jz,y)expyTQ)]TJ /F7 7.97 Tf 6.59 0 Td[(1y)]TJ /F5 11.955 Tf 11.95 0 Td[(T)]TJ /F7 7.97 Tf 6.59 0 Td[(1 42d2=2(n)]TJ /F11 5.978 Tf 5.75 0 Td[(p+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1) 2. (4)Also,nYi=1"ZR+1 81 2z)]TJ /F16 5.978 Tf 7.78 3.26 Td[(3 2iexp)]TJ /F4 11.955 Tf 20.57 8.09 Td[(1 24zidzi#=3n 2. (4)Finally,combining( 4 ),( 4 ),( 4 )and( 4 ),wehaveZRpZR+k)]TJ /F5 11.955 Tf 5.47 -9.69 Td[(,2,2d2d=ZRpZR+ZRn+(,2jz,y)(zj,2,y)dzd2d2n+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1 23n 2.Hence,( 4 )issatised. 48

PAGE 49

Now,because~KisdenedbytheHaarPX-DAMarkovchain~,Theorem1of Khare&Hobert ( 2011 )immediatelyimpliesthat~Kistrace-class,andthat0~ii<1foralli2N.Thefactthatthereisatleastonei2Nsuchthat0~i
PAGE 50

CHAPTER5THEORETICALCOMPARISONOFPX-DAANDHAARPX-DAALGORITHMS 5.1PX-DAandHaarPX-DAAlgorithmsLetX,Y,andbeasinSection 2.3 .SupposeisanintractabledensitywithrespecttoonXthatwewouldliketoexplore.ConsideraDAalgorithmbasedonthejointdensityf:XY![0,1)withrespecttowhich,ofcourse,mustsatisfy ZYf(x,y)(dy)=(x).(5)RecallthattheMarkovchainunderlyingtheDAalgorithmhasMtdk(xjx0)=ZYfXjY(xjy)fYjX(yjx0)(dy),andthattheDAchaincanbesimulatedbydrawingalternatelyfromfXjYandfYjX,theconditionaldensitiesassociatedwithf.ThePX-DAalgorithm( Liu&Wu , 1999 )isamodiedversionoftheDAalgorithm.Thebasicideaistouseftocreateanentirefamilyofjointdensitiessuchthateachmemberofthisfamilysatises( 5 ).ThisallowsfortheconstructionofaclassofviableDAalgorithms.Tobespecic,consideraclassoffunctionstg:Y!Yforg2Gsuchthat,foreachxedg,tg(y)isone-to-oneanddifferentiableiny.WeareassumingherethatGisatopologicalgroupwithidentityelemente.Assumefurtherthat(i)te(y)=yforally2Yand(ii)tg1g2(y)=tg1(tg2(y))forallg1,g22Gandally2Y.Supposethatr(jx)isaconditionalprobabilitydensityonGgivenx(2X)withrespectto(unimodular)Haarmeasure%onG.Now,deneaprobabilitydensity~f(r):XYG![0,1)as~f(r)(x,y,g)=f(x,tg(y))jJg(y)jr(gjx),whereJg(z)istheJacobianofthetransformationz=t)]TJ /F7 7.97 Tf 6.58 0 Td[(1g(y).Let~f(r)(x,y)=RG~f(r)(x,y,g)%(dg),andnotethatZY~f(r)(x,y)(dy)=ZGZYf(x,tg(y))jJg(y)jr(gjx)(dy)%(dg) 50

PAGE 51

=ZG(x)r(gjx)%(dg)=(x).ThePX-DAalgorithmissimplyanalternativeDAalgorithmbasedontheMtdgivenbykr(xjx0)=ZY~f(r)XjY(xjy)~f(r)YjX(yjx0)(dy),where~f(r)XjYand~f(r)YjXareconditionaldensitiesassociatedwith~f(r)(x,y).RecallfromSection 2.3 that Liu&Wu ( 1999 )alsoproposetheHaarPX-DAalgorithmbasedontheMtd~k(xjx0)=ZYZYfXjY(xjy0)R(y,dy0)fYjX(yjx0)(dy).Here,theMtfR(y,dy0)correspondstotheoperator(Rh)(y)=ZGh(tg(y))fY(tg(y))jJg(y)j m(y)%(dg),wherem(y)=RGfY(tg(y))jJg(y)j%(dg). Liu&Wu ( 1999 )(hereafter,L&W)provideageneralresultcomparingthePX-DAandHaarPX-DAalgorithms.Theorem5ofL&WestablishesthattheHaarPX-DAalgorithmisatleastasgoodintheoperatornormsenseaseveryPX-DAalgorithminthespecialcasewhereX,YandGareEuclideanspacesandthegroupGisunimodular.Akeyassumptioninshowingtheresultistheexistenceofacross-sectionandacorrespondingdiffeomorphism.Unfortunately,asL&Wpointout,across-sectionandanassociateddiffeomorphismdonotnecessarilyexistingeneral.Inthenextsection,wecomparegeneralPX-DAandHaarPX-DAalgorithms.Infact, Hobert&Marchev ( 2008 )(hereafter,H&M)generalizeTheorem5ofL&Winthecasewherer(jx)isfreeofx,andwewillshowthatH&M'sresultalsoholdswhenr(jx)doesdependonx. 51

PAGE 52

5.2GeneralPX-DAandHaarPX-DAAlgorithms 5.2.1HobertandMarchev'sGroupStructureLetusallowthegroupGtoacttopologicallyontheleftofY;thatis,thereisacontinuousfunctionF:GY!YsuchthatF(e,y)=yforally2YandF(g1g2,y)=F(g1,F(g2,y))forallg1,g22Gandally2Y.(NotethatF(g,y)correspondstotg(y)fromSection 2.3 .)Asistypicallydone,wewilldenotethevalueofFat(g,y)bygyso,inthisnotation,thetwoconditionsarewritteney=yand(g1g2)y=g1(g2y).Assumethatthereexistsafunctionj:GY!(0,1)suchthat: (i) j(g)]TJ /F7 7.97 Tf 6.59 0 Td[(1,y)=1 j(g,y)forallg2Gandally2Y, (ii) j(g1g2,y)=j(g1,g2y)j(g2,y)forallg1,g22Gandally2Y,and (iii) Forallg2Gandallintegrablefunctionsh:Y!R, ZYh(gy)j(g,y)(dy)=ZYh(y)(dy).(5)AsinL&W,supposethatYRn,isLebesguemeasureonY,andforeachxedg2G,F(g,):Y!Yisdifferentiable.Thenifwetakej(g,y)tobetheJacobianofthetransformationy7!F(g,y),thethreepropertieslistedabovecanbeeasilyveriedfromcalculus. 5.2.2ConstructingGeneralPX-DAandHaarPX-DAAlgorithmsWeconstructageneralversionofPX-DAalgorithmunderthegroupstructuredescribedinSection 5.2.1 .Deneaprobabilitydensity~f(r):XY![0,1)as~f(r)(x,y)=ZGf(x,gy)j(g,y)r(x,dg),wherer(x,)isaconditionalprobabilitymeasureonGgivenx2X.Notethatr(gjx)%(dg)inSection 5.1 isaspecialcaseofr(x,dg).NotealsothatH&Mconsiderthecasewherer(x,dg)doesnotdependonx.By( 5 ),thexmarginalof~f(r)is,and 52

PAGE 53

theymarginaldensityismr(y):=ZXZGf(x,gy)j(g,y)r(x,dg)(dx),whereweassumemr(y)ispositive,niteforally2Y.Theassociatedconditionaldensitiesare~f(r)XjY(xjy)=~f(r)(x,y) mr(y)=RGf(x,g0y)j(g0,y)r(x,dg0) mr(y)and~f(r)YjX(yjx)=RGfYjX(gyjx)j(g,y)r(x,dg).OurgeneralPX-DAisaDAalgorithmbasedon~f(r)withMtdgivenbykr(xjx0)=ZY~f(r)XjY(xjy)~f(r)YjX(yjx0)(dy).Wenotethatifr(x,)isfreeofx,thenwerecoverH&M'sgeneralPX-DAchain.WenowdescribeH&M'sgeneralHaarPX-DAalgorithm.UndertheassumptionsinSection 5.2.1 ,thereexistsaleft-Haarmeasure,%l,onG,whichisanontrivialmeasuresatisfying ZGh(~gg)%l(dg)=ZGh(g)%l(dg)(5)forall~g2Gandallintegrablefunctionsh:G!R.Thismeasureisuniqueuptoamultiplicativeconstant.Moreover,thereexistsamultiplier,4,calledthe(right)modularfunctionofthegroup,whichrelatestheleft-Haarandright-Haarmeasures,%land%r,onGsuchthat%r(dg)=4(g)]TJ /F7 7.97 Tf 6.59 0 Td[(1)%l(dg).(Afunction:G!(0,1)iscalledamultiplierifiscontinuousand(g1g2)=(g1)(g2)forallg1,g22G.)Here,theright-Haarmeasuresatisestheobviousanalogueof( 5 ).Groupsforwhich41;thatis,forwhichright-Haarandleft-Haarmeasuresareequivalent,arecalledunimodular.WenowstatetwousefulformulasfromH&Mthatwillbeusedinthenextsection.If~g2Gandh:G!Risanintegrablefunction,then ZGh(g~g)]TJ /F7 7.97 Tf 6.59 0 Td[(1)%l(dg)=4(~g)ZGh(g)%l(dg).(5) 53

PAGE 54

Asbefore,letfYbetheymarginaloff,andassumethatm(y):=RGfY(gy)j(g,y)%l(dg)ispositive,niteforally2Y.Astraightforwardapplicationof( 5 )showsthat,fory2Y, m(gy)=j(g)]TJ /F7 7.97 Tf 6.58 0 Td[(1,y)4(g)]TJ /F7 7.97 Tf 6.59 0 Td[(1)m(y).(5)LetRbeanoperatoronL20(fY)denedby(Rh)(y)=ZGh(gy)fY(gy)j(g,y) m(y)%l(dg).ThenthecorrespondingMarkovchainonYevolvesasfollows.Ifthecurrentstateisy,thenthedistributionofthenextstateisthatofgywheregisarandomelementdrawnfromthedensity(withrespectto%l)fY(gy)j(g,y)=m(y).WedenoteitsMtfbyR(y,dy0).ItisshowninProposition3ofH&MthatR(y,dy0)isreversiblewithrespecttofY(y)(dy).TheMtdofH&M'sgeneralHaarPX-DAis~k(xjx0)=ZYZYfXjY(xjy0)R(y,dy0)fYjX(yjx0)(dy).Moreover,Theorem4ofH&Mimplies~kisitselfaDAalgorithm. 5.2.3ComparingGeneralPX-DAandHaarPX-DAAlgorithmsInthissection,weestablishthat~kisatleastasgoodaskrintheefciencyorderingandintheoperatornormsense.Infact,Theorem4ofH&MimplythattheirgeneralHaarPX-DAalgorithmisatleastasgoodastheirgeneralPX-DAalgorithmintheefciencyorderingandoperatornormsense.SinceH&M'sgeneralPX-DAisaspecialcaseofourgeneralPX-DA,ourresultimprovesupontheirresult.WhileourproofisthesameasH&M's,wefeelthatitisworthwhiletogivethedetailshereforcompleteness.Inordertointroducetheresult,weneedabitmorenotation.LetKrand~KdenotetheoperatorsonL20()associatedwithkrand~k,respectively.Hereistheresult. Theorem5.1. Letr(x,)beaconditionalprobabilitymeasureonGgivenx(2X).Supposemr(y)andm(y)arepositiveandniteforally2Y.IftheMarkovchainsdrivenbykrand~kareHarrisergodic,then~kEkrandk~KkkKrk. 54

PAGE 55

Proof. RecallthatkrisaDAalgorithmwithrespecttothejointdensity~f(r)(x,y)=ZGf(x,g0y)j(g0,y)r(x,dg0)andthattheymarginaldensityof~f(r)ismr(y)=ZXZGf(x,g0y)j(g0,y)r(x,dg0)(dx).Let~RbetheMtfonYwithinvariantdensitymr(y)thatisconstructedaccordingtotherecipeinSection 5.2.2 ;thatis,~Riswhatwewouldhaveendedupwithhadweusedmr(y)inplaceoffY(y).Wewillshowthat~k(xjx0)=ZYZY~f(r)XjY(xjy0)~R(y,dy0)~f(r)YjX(yjx0)(dy);thatis,~kisasandwichvariantofkr.First,ifwesubstitutemr(y)forfY(y)inthedenitionofm(y),wehaveZGmr(gy)j(g,y)%l(dg)=ZGZXZGf(x,g0gy)j(g0,gy)r(x,dg0)(dx)j(g,y)%l(dg)=ZXZGZGf(x,g0gy)j(g0g,y)%l(dg)r(x,dg0)(dx)=ZXZGZGf(x,gy)j(g,y)%l(dg)r(x,dg0)(dx)=ZGZXf(x,gy)j(g,y)(dx)%l(dg)=ZGfY(gy)j(g,y)%l(dg)=m(y).Hence,thefunctionm(y)isthesamewhetherweusefYormr.Now,usingthedenitionof~Randthecalculationabove,wehaveZY~f(r)XjY(xjy0)~R(y,dy0)=ZG~f(r)XjY(xjg00y)mr(g00y)j(g00,y) m(y)%l(dg00).Thus,ZYZY~f(r)XjY(xjy0)~R(y,dy0)~f(r)YjX(yjx0)(dy) 55

PAGE 56

=ZYZGRGf(x,g0g00y)j(g0,g00y)r(x,dg0) mr(g00y)mr(g00y)j(g00,y) m(y)%l(dg00)ZGfYjX(gyjx0)j(g,y)r(x0,dg)(dy)=ZGZGZGZYf(x,g0g00y)j(g0,g00y)j(g00,y)fYjX(gyjx0)j(g,y) m(y)(dy)%l(dg00)r(x,dg0)r(x0,dg)=ZGZGZGZYf(x,g0g00g)]TJ /F7 7.97 Tf 6.58 0 Td[(1gy)j(g0g00,g)]TJ /F7 7.97 Tf 6.59 0 Td[(1gy)fYjX(gyjx0)j(g,y) m(g)]TJ /F7 7.97 Tf 6.59 0 Td[(1gy)(dy)%l(dg00)r(x,dg0)r(x0,dg)=ZGZGZGZYf(x,g0g00g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)j(g0g00,g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)fYjX(yjx0) m(g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)(dy)%l(dg00)r(x,dg0)r(x0,dg)=ZYZGZGZGf(x,g0g00g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)j(g0g00,g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)fYjX(yjx0) j(g,y)4(g)m(y)%l(dg00)r(x,dg0)r(x0,dg)(dy)=ZYZGZGZGf(x,g0g00g)]TJ /F7 7.97 Tf 6.58 0 Td[(1y)j(g0g00g)]TJ /F7 7.97 Tf 6.58 0 Td[(1,y)4(g)]TJ /F7 7.97 Tf 6.59 0 Td[(1)fYjX(yjx0) m(y)%l(dg00)r(x,dg0)r(x0,dg)(dy)=ZYZGZGZGf(x,g0g00y)j(g0g00,y)fYjX(yjx0) m(y)%l(dg00)r(x,dg0)r(x0,dg)(dy)=ZYZGZGZGf(x,g00y)j(g00,y)fYjX(yjx0) m(y)%l(dg00)r(x,dg0)r(x0,dg)(dy)=ZYZGfXjY(xjg00y)fY(g00y)j(g00,y) m(y)%l(dg00)fYjX(yjx0)(dy)=ZYZYfXjY(xjy0)R(y,dy0)fYjX(yjx0)(dy)=~k(xjx0),wherethethirdandfourthequalitiesfollowfrompropertiesofj,thefthisdueto( 5 ),thesixthfollowsfrompropertiesofjand4,theseventhisaconsequenceof( 5 ),theeighthisduetotheleft-invarianceof%l,theninthfollowsfromthefactthatrisaprobabilitymeasure,andthepenultimateequalityisduetothedenitionofR.AnapplicationofTheorem 2.4 yieldstheresult. 56

PAGE 57

APPENDIXAEXISTENCEOFMOMENTGENERATINGFUNCTIONInthissection,weestablishthattheposteriordistributionoftheBayesianlogisticregressionmodel(Section 3.1 )hasamomentgeneratingfunction(mgf). PropositionA.1. Foranyxedt2Rp,ZRpeTt(jy)d<1.Hence,themomentgeneratingfunctionoftheposteriordistributionexists. Remark6. Chen&Shao ( 2000 )developresultsforBayesianlogisticregressionwithaat(improper)prioron.Inparticular,theseauthorsprovideconditionsonXandythatguaranteetheexistenceofthemgfoftheposteriorofgiventhedata.NotethatourresultholdsforallXandy. Remark7. Proposition A.1 impliesthatifg(1,...,p)=aiforsomei2f1,2,...,pganda>0,thengisintegrablewithrespecttotheposterior. ProofofProposition A.1 . RecallthatZRn+(,wjy)dw=(jy),where(,wjy)=Qni=1Pr(Yi=yij)f(wj)() c(y).Hence,itsufcestoshowthatZRn+ZRpeTt(,wjy)ddw<1.StraightforwardcalculationssimilartothosedoneinSection 3.2 showthat(,wjy)=1 c(y)"nYi=1)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(exTiyi 1+exTi#"nYi=1coshjxTij 2e)]TJ /F16 5.978 Tf 7.79 5.33 Td[((xTi)2wi 2h(wi)#()=1 2nc(y)()"nYi=1expyixTi)]TJ /F3 11.955 Tf 13.15 8.08 Td[(xTi 2)]TJ /F4 11.955 Tf 13.15 8.08 Td[((xTi)2wi 2h(wi)# 57

PAGE 58

=jBj)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2 2nc(y)exp)]TJ /F4 11.955 Tf 10.5 8.09 Td[(1 2T)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F4 11.955 Tf 11.95 0 Td[(2TXTy)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 21n+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1bexp)]TJ /F4 11.955 Tf 10.49 8.09 Td[(1 2bTB)]TJ /F7 7.97 Tf 6.58 0 Td[(1b"nYi=1h(wi)#=(;~m,) 2nc(y)jj1 2 jBj1 2exp)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2bTB)]TJ /F7 7.97 Tf 6.58 0 Td[(1bexp1 2~mT)]TJ /F7 7.97 Tf 6.58 0 Td[(1~mnYi=1h(wi).Then,sincejjjBjand~mT)]TJ /F7 7.97 Tf 6.59 0 Td[(1~msTs,wehave(,wjy)(;~m,) 2nc(y)exp)]TJ /F4 11.955 Tf 13.16 8.09 Td[(1 2bTB)]TJ /F7 7.97 Tf 6.59 0 Td[(1bexp1 2sTsnYi=1h(wi).Now,usingtheformulaforthemultivariatenormalmgf,itsufcestoshowthatZRn+ZRpeTt(;~m,)nYi=1h(wi)ddw=ZRn+exp~mTt+1 2tTtnYi=1h(wi)dw<1.Weestablishthisbydemonstratingthat~mTt+1 2tTtisuniformlyboundedinw.ForamatrixA,denekAk=supkxk=1kAxk.Ofcourse,ifAisnonnegativedenite,thenkAkisequaltothelargesteigenvalueofA.Now,usingCauchy-Schwartzandpropertiesofthenorm,wehavej~mTtj2k~mk2ktk2=kB1 2B)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2~mk2ktk2kB1 2k2kB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2~mk2ktk2.NowsinceB)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2~m=)]TJ /F4 11.955 Tf 7.17 -7.02 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.58 0 Td[(1)]TJ /F4 11.955 Tf 7.17 -7.02 Td[(~XT(y)]TJ /F7 7.97 Tf 13.15 4.71 Td[(1 21n)+B)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2b,wehavekB)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2~mk2=)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.58 0 Td[(1~XTy)]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 21n+B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b22)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.59 0 Td[(1~XTy)]TJ /F4 11.955 Tf 13.15 8.08 Td[(1 21n2+2)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.59 0 Td[(1B)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2b22)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.59 0 Td[(12~XTy)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 21n2+2)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.58 0 Td[(12B)]TJ /F16 5.978 Tf 7.79 3.26 Td[(1 2b22~XTy)]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 21n2+2B)]TJ /F16 5.978 Tf 7.78 3.26 Td[(1 2b2,wheretherstinequalityisduetothefactthatka+bk22kak2+2kbk2foranyvectorsaandb,andthethirdinequalityisduetothefactthat)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.59 0 Td[(121(Lemma 2 ). 58

PAGE 59

Hence,j~mTtjisuniformlyboundedinw.Finally,anotherapplicationofLemma 2 yieldstTt=tT)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(XTX+B)]TJ /F7 7.97 Tf 6.59 0 Td[(1)]TJ /F7 7.97 Tf 6.58 0 Td[(1t=tTB1 2)]TJ /F4 11.955 Tf 7.17 -7.03 Td[(~XT~X+I)]TJ /F7 7.97 Tf 6.59 0 Td[(1B1 2ttTBt,sotTtisalsouniformlyboundedinw,andtheresultfollows. 59

PAGE 60

APPENDIXBEXTENSIONTOBAYESIANQUANTILEREGRESSIONInthissection,weconsideranalternativeversionofmodel( 4 )inwhichtheLaplaceerrorsarereplacedbyerrorsfromtheasymmetricLaplacedensitygivenbyd(;r)=r(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r)he(1)]TJ /F8 7.97 Tf 6.59 0 Td[(r)IR)]TJ /F4 11.955 Tf 6.75 2.87 Td[(()+e)]TJ /F8 7.97 Tf 6.58 0 Td[(rIR+()i,wherer2(0,1)isxedandR)]TJ /F4 11.955 Tf 10.84 1.8 Td[(:=(,0].Thisdensityhasrthquantileequaltozero,andwhenr=1=2,werecovertheLaplaceerrors.(ThiscorrespondenceisthereasonforthepeculiarscalinginSection 4.1 )ThejointdensityofthedataisnowgivenbyfX,r(yj,2)=rn(1)]TJ /F3 11.955 Tf 11.95 0 Td[(r)n nexp)]TJ /F4 11.955 Tf 13.55 8.09 Td[(1 nXi=1)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTihr)]TJ /F3 11.955 Tf 11.96 0 Td[(IR)]TJ /F10 11.955 Tf 6.75 12.55 Td[()]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTii.Asnotedby Yu&Moyeed ( 2001 ),themaximumlikelihoodestimatorofunderthisfullyparametricmodelisgivenbyargmin2RpnXi=1r)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(Yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi,wherer(u)=ur)]TJ /F3 11.955 Tf 13.41 0 Td[(IR)]TJ /F4 11.955 Tf 6.75 2.86 Td[((u).Ofcourse,thisalsohappenstobethestandardnonparametricestimatorof(r)whentheconditionalquantilefunctionofYgivenX=xtakestheformQ(rjX=x)=xT(r)(see,e.g., Koenker , 2005 ).Forthisreason,thefullyparametricmodelwiththeasymmetricLaplaceerrorsissometimesusedasthebasisofaBayesianquantileregressionmodel(see,e.g., Kozumi&Kobayashi , 2011 ; Yu&Moyeed , 2001 ; Yuan&Yin , 2010 ).Inparticular, Kozumi&Kobayashi ( 2011 )putaproperprioron(,2),anddevelopedathree-variableGibbssamplertoexploretheresultingintractableposteriordensity.(See Khare&Hobert ( 2012 )foratheoreticalanalysisofthatGibbssampler.)If,insteadoftheproperpriorusedby Kozumi&Kobayashi ( 2011 ),weusetheimproperprior(,2)=(2))]TJ /F7 7.97 Tf 6.59 0 Td[((a+1)=2IR+(2),thentheposteriorisproperifmr(y,X,a):=ZR+ZRpfX,r(yj,2)(,2)dd2<1. 60

PAGE 61

ThefollowingresultallowsustoreuseProposition 4.1 toestablishnecessaryandsufcientconditionsforposteriorproprietyunderthemoregeneralmodelwithasymmetricLaplaceerrors. Lemma4. FixX,yandr2(1=2,1).Forall(,2)2RpR+,(4r)n(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r)nfX,1=2(yj,2)fX,r(yj,2)(4r)n(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r)nfX,1=2(yj,2),whereX=2rX,y=2ry,X=2(1)]TJ /F3 11.955 Tf 12.17 0 Td[(r)Xandy=2(1)]TJ /F3 11.955 Tf 12.17 0 Td[(r)y.Ontheotherhand,ifr2(0,1=2),thenthesameinequalitiesholdwhen(X,y)and(X,y)arereversed. Proof. Weprovidedetailsforonlyoneofthefourinequalities,astheothersarecompletelyanalogous.Ifr2(1=2,1),thenr>1)]TJ /F3 11.955 Tf 11.96 0 Td[(r,andforeachi2f1,2,...,ng,exp)]TJ /F4 11.955 Tf 13.56 8.09 Td[(1 )]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTihr)]TJ /F3 11.955 Tf 11.96 0 Td[(IR)]TJ /F10 11.955 Tf 6.75 12.56 Td[()]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTii=exp1 22(1)]TJ /F3 11.955 Tf 11.95 0 Td[(r))]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTiIR)]TJ /F10 11.955 Tf 6.76 12.55 Td[()]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi+exp)]TJ /F4 11.955 Tf 16.69 8.09 Td[(1 22r)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTiIR+)]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTiexp1 22(1)]TJ /F3 11.955 Tf 11.95 0 Td[(r))]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTiIR)]TJ /F10 11.955 Tf 6.76 12.55 Td[()]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi+exp)]TJ /F4 11.955 Tf 16.69 8.08 Td[(1 22(1)]TJ /F3 11.955 Tf 11.95 0 Td[(r))]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTiIR+)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi=exp)]TJ /F4 11.955 Tf 16.69 8.09 Td[(1 22(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r))]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTi.Therefore,fX,r(yj,2)=rn(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r)n nnYi=1exp)]TJ /F4 11.955 Tf 13.55 8.09 Td[(1 )]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTihr)]TJ /F3 11.955 Tf 11.96 0 Td[(IR)]TJ /F10 11.955 Tf 6.75 12.55 Td[()]TJ /F3 11.955 Tf 5.48 -9.69 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTii4nrn(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r)n n1 4nnYi=1exp)]TJ /F4 11.955 Tf 16.7 8.08 Td[(1 22(1)]TJ /F3 11.955 Tf 11.96 0 Td[(r))]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi=(4r)n(1)]TJ /F3 11.955 Tf 11.95 0 Td[(r)nfX,1=2(yj,2). Ofcourse,multiplicationofXbyapositiveconstantdoesnotalteritsrank.Furthermore,y=2C(X)isequivalenttoy=2C(X)andtoy=2C(X).Hence,it 61

PAGE 62

followsimmediatelyfromLemma 4 thatProposition 4.1 remainsvalidwhenwereplacetheLaplaceerrorsbyasymmetricLaplaceerrors.WestatethisasanotherProposition. PropositionB.1. Foranyxedr2(0,1),mr(y,X,a)<1ifandonlyifXhasfullcolumnrank,a>)]TJ /F3 11.955 Tf 9.3 0 Td[(n+p+1andy=2C(X).Lastly,wenotethat Yu&Moyeed ( 2001 )consideredarestrictionofthemodeldescribedaboveinwhichthescaleparameter,,isknownandequalto1,andtheprioronisat.Inthiscase,theposteriorisproperwhenmr(y,X):=ZRpfX,r(yj,1)d<1. Yu&Moyeed ( 2001 )claimedintheirTheorem1thattheposteriordensityisalwaysproper.Thisisactuallyfalse.Infact,argumentssimilartothoseusedintheAppendix C canbeusedinconjunctionwithLemma 4 (with=1)toshowthatmr(y,X)<1ifandonlyifXhasfullcolumnrank. 62

PAGE 63

APPENDIXCPOSTERIORPROPRIETYInthissection,weprovideaproofofProposition 4.1 .Beforepresentingtheproof,westateausefulequationthatwillbeusedacoupleoftimeswithintheproof.LetkkdenotetheusualEuclideannorm,andletM)]TJ /F1 11.955 Tf 10.41 -4.33 Td[(denoteageneralizedinverseofthematrixM.Setting^=(XTX))]TJ /F3 11.955 Tf 7.09 -4.34 Td[(XTyandP=X(XTX))]TJ /F3 11.955 Tf 7.08 -4.34 Td[(XT,wehavenXi=1(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2=(y)]TJ /F3 11.955 Tf 11.95 0 Td[(X^+X^)]TJ /F3 11.955 Tf 11.96 0 Td[(X)T(y)]TJ /F3 11.955 Tf 11.96 0 Td[(X^+X^)]TJ /F3 11.955 Tf 11.95 0 Td[(X)=ky)]TJ /F3 11.955 Tf 11.96 0 Td[(X^k2+2(y)]TJ /F3 11.955 Tf 11.95 0 Td[(X^)T(X^)]TJ /F3 11.955 Tf 11.96 0 Td[(X)+()]TJ /F4 11.955 Tf 13.43 2.65 Td[(^)T(XTX)()]TJ /F4 11.955 Tf 13.43 2.65 Td[(^)=k(I)]TJ /F3 11.955 Tf 11.95 0 Td[(P)yk2+2yT(I)]TJ /F3 11.955 Tf 11.96 0 Td[(P)X(^)]TJ /F5 11.955 Tf 11.96 0 Td[()+()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^)T(XTX)()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^)=()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^)T(XTX)()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^)+k(I)]TJ /F3 11.955 Tf 11.95 0 Td[(P)yk2.Ofcourse,ifrank(X)=p,then(XTX))]TJ /F4 11.955 Tf 10.41 -4.34 Td[(=(XTX))]TJ /F7 7.97 Tf 6.59 0 Td[(1.Webeginwithnecessity.Observethatf(yj,2)(,2)=c0(2))]TJ /F11 5.978 Tf 7.78 3.25 Td[(n 2exp()]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2nXi=1jyi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTij )(2))]TJ /F16 5.978 Tf 7.79 3.85 Td[((a+1) 2c0exp()]TJ /F4 11.955 Tf 13.15 8.09 Td[(1 2nXi=11 2(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi)2 2+1)(2))]TJ /F16 5.978 Tf 7.78 3.86 Td[((a+n+1) 2=c1exp()]TJ /F4 11.955 Tf 19.16 8.09 Td[(1 42()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^)T(XTX)()]TJ /F4 11.955 Tf 13.43 2.66 Td[(^))exp()]TJ 13.15 8.09 Td[(k(I)]TJ /F3 11.955 Tf 11.95 0 Td[(P)yk2 42)(2))]TJ /F16 5.978 Tf 7.79 3.85 Td[((a+n+1) 2, (C)wherec0andc1areconstants(inand2)andtheinequalityfollowsfromthefactthatjxj(x2+1)=2.Nowbythespectraldecomposition,wecanwriteXTX=OTDO,whereOisorthogonalandDisdiagonalwithitselementsequaltotheeigenvaluesofXTX.IfXisnotoffullcolumnrank,thenXTXhasatleastoneeigenvalueequaltozero.Thenbyachangeofvariableswithu=O()]TJ /F4 11.955 Tf 14.09 2.66 Td[(^),itfollowsthattheintegralof 63

PAGE 64

theright-handsideof( C )withrespecttodiverges.Hence,theposteriorisimproperwheneverrank(X))]TJ /F3 11.955 Tf 9.3 0 Td[(n+p+1andk(I)]TJ /F3 11.955 Tf 12 0 Td[(P)yk>0.Hence,theproofofnecessityiscomplete.Nowforthesufciencypart.AroutinecalculationrevealsthatZR+(2))]TJ /F16 5.978 Tf 7.78 3.86 Td[((n+a+1) 2exp()]TJ /F4 11.955 Tf 12.71 8.09 Td[(1 2nXi=1jyi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTij )d2=2a+n\(a+n)]TJ /F4 11.955 Tf 11.5 0 Td[(1)"nXi=1jyi)]TJ /F3 11.955 Tf 11.51 0 Td[(xTij#)]TJ /F7 7.97 Tf 6.59 0 Td[((n+a)]TJ /F7 7.97 Tf 6.58 0 Td[(1),anditfollowsthat,c0(y)=c02a+n\(a+n)]TJ /F4 11.955 Tf 11.95 0 Td[(1)ZRpnXi=1jyi)]TJ /F3 11.955 Tf 11.95 0 Td[(xTij)]TJ /F7 7.97 Tf 6.59 0 Td[((n+a)]TJ /F7 7.97 Tf 6.59 0 Td[(1)d=c3ZRpnXi=1jyi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTij2)]TJ /F16 5.978 Tf 7.78 3.86 Td[((n+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1) 2dc3ZRpnXi=1)]TJ /F3 11.955 Tf 5.48 -9.68 Td[(yi)]TJ /F3 11.955 Tf 11.96 0 Td[(xTi2)]TJ /F16 5.978 Tf 7.78 3.85 Td[((n+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1) 2d=c3ZRp()]TJ /F4 11.955 Tf 13.42 2.66 Td[(^)T(XTX)()]TJ /F4 11.955 Tf 13.42 2.66 Td[(^)+k(I)]TJ /F3 11.955 Tf 11.96 0 Td[(P)yk2)]TJ /F16 5.978 Tf 7.78 3.86 Td[((n)]TJ /F11 5.978 Tf 5.75 0 Td[(p+a)]TJ /F16 5.978 Tf 5.76 0 Td[(1)+p 2d=c3 (I)]TJ /F3 11.955 Tf 11.95 0 Td[(P)yn+a)]TJ /F7 7.97 Tf 6.59 0 Td[(1ZRp1+1 ()]TJ /F4 11.955 Tf 13.43 2.65 Td[(^)T)]TJ /F7 7.97 Tf 6.59 0 Td[(1()]TJ /F4 11.955 Tf 13.43 2.65 Td[(^))]TJ /F15 5.978 Tf 7.78 3.52 Td[(+p 2d, (C)where=n)]TJ /F3 11.955 Tf 12.05 0 Td[(p+a)]TJ /F4 11.955 Tf 12.05 0 Td[(1and)]TJ /F7 7.97 Tf 6.59 0 Td[(1=(n)]TJ /F3 11.955 Tf 12.06 0 Td[(p+a)]TJ /F4 11.955 Tf 12.05 0 Td[(1)(XTX)(I)]TJ /F3 11.955 Tf 11.95 0 Td[(P)y2.Buttheintegrandin( C )isthekernelofap-variateStudent'stdensitywithdegreesoffreedom,and 64

PAGE 65

locationandscale(matrix)equalto^and,respectively.Hence,c0(y)<1andtheproofiscomplete. 65

PAGE 66

REFERENCES Biane,P.,Pitman,J.,&Yor,M.(2001).ProbabilitylawsrelatedtotheJacobithetaandRiemannzetafunctions,andBrownianexcursions.BulletinoftheAmericanMathematicalSociety,38,435. Buja,A.(1990).Remarksonfunctionalcanonicalvariates,alternatingleastsquaresmethodsandACE.TheAnnalsofStatistics,18,1032. Chan,K.S.,&Geyer,C.J.(1994).Discussionof“Markovchainsforexploringposteriordistributions,”byL.Tierney.TheAnnalsofStatistics,22,1747. Chen,M.-H.,&Shao,Q.-M.(2000).Proprietyofposteriordistributionfordichotomousquantalresponsemodels.InProceedingsoftheAmericanMathematicalSociety,vol.129,(pp.293). Choi,H.M.,&Hobert,J.P.(2013a).AnalysisofMCMCalgorithmsforBayesianlinearregressionwithLaplaceerrors.JournalofMultivariateAnalysis,117,32. Choi,H.M.,&Hobert,J.P.(2013b).ThePolya-GammaGibbssamplerforBayesianlogisticregressionisuniformlyergodic.ElectronicJournalofStatistics,7,2054. Cogburn,R.(1972).ThecentrallimittheoremforMarkovprocesses.InProceedingsoftheBerkeleySymposiumonMathematicalStatisticsandProbability,vol.2,(pp.485). Conway,J.B.(1990).ACourseinFunctionalAnalysis.NewYork:Springer,2nded. Dalal,S.R.,Fowlkes,E.B.,&Hoadley,B.(1989).Riskanalysisofthespaceshuttle:Pre-challengerpredictionoffailure.JournaloftheAmericanStatisticalAssociation,84,945. Diaconis,P.,Khare,K.,&Saloff-coste,L.(2008).Gibbssampling,exponentialfamiliesandorthogonalpolynomials.StatisticalSciences,23,151. Fernandez,C.,&Steel,M.F.J.(2000).Bayesianregressionanalysiswithscalemixturesofnormals.EconometricTheory,16,80. Flegal,J.M.,Haran,M.,&Jones,G.L.(2008).MarkovchainMonteCarlo:Canwetrustthethirdsignicantgure?StatisticalScience,23,250. Flegal,J.M.,&Jones,G.L.(2010).BatchmeansandspectralvarianceestimatorsinMarkovchainMonteCarlo.TheAnnalsofStatistics,38,1034. Fruhwirth-Schnatter,S.,&Fruhwirth,R.(2010).DataaugmentationandMCMCforbinaryandmultinomiallogitmodels.InStatisticalModellingandRegressionStructures,(pp.111).Springer-Verlag. 66

PAGE 67

Fruhwirth-Schnatter,S.,Fruhwirth,R.,Held,L.,&Rue,H.(2009).Improvedauxiliarymixturesamplingforhierarchicalmodelsofnon-Gaussiandata.StatisticsandComputing,19,479. Glynn,P.W.,&Whitt,W.(1992).Theasymptoticvalidityofsequentialstoppingrulesforstochasticsimulations.TheAnnalsofAppliedProbability,2,180. Hobert,J.P.,Jones,G.L.,Presnell,B.,&Rosenthal,J.S.(2002).OntheapplicabilityofregenerativesimulationinMarkovchainMonteCarlo.Biometrika,89,731. Hobert,J.P.,&Marchev,D.(2008).Atheoreticalcomparisonofthedataaugmentation,marginalaugmentationandPX-DAalgorithms.TheAnnalsofStatistics,36,532. Hobert,J.P.,Roy,V.,&Robert,C.P.(2011).ImprovingtheconvergencepropertiesofthedataaugmentationalgorithmwithanapplicationtoBayesianmixturemodelling.StatisticalScience,26,332. Holmes,C.,&Held,L.(2006).Bayesianauxiliaryvariablemodelsforbinaryandmultinomialregression.BayesianAnalysis,1,145. Ibragimov,I.A.,&Linnik,Y.V.(1971).IndependentandStationarySequencesofRandomVariables.Groningen:Wolters-Noordhoff. Jones,G.L.,Haran,M.,Caffo,B.S.,&Neath,R.(2006).Fixed-widthoutputanalysisforMarkovchainMonteCarlo.JournaloftheAmericanStatisticalAssociation,101,1537. Jones,G.L.,&Hobert,J.P.(2001).HonestexplorationofintractableprobabilitydistributionsviaMarkovchainMonteCarlo.StatisticalScience,16,312. Kass,R.E.,&Wasserman,L.(1996).Theselectionofpriordistributionsbyformalrules.JournaloftheAmericanStatisticalAssociation,91,1343. Khare,K.,&Hobert,J.P.(2011).Aspectralanalyticcomparisonoftrace-classdataaugmentationalgorithmsandtheirsandwichvariants.TheAnnalsofStatistics,39,2585. Khare,K.,&Hobert,J.P.(2012).GeometricergodicityoftheGibbssamplerforBayesianquantileregression.JournalofMultivariateAnalysis,112,108. Koenker,R.(2005).QuantileRegression.EconometricSocietyMonographs.CambridgeUniversityPress. Kozumi,H.,&Kobayashi,G.(2011).GibbssamplingmethodsforBayesianquantileregression.JournalofStatisticalComputationandSimulation,81,1565. Liu,J.S.(1996).Metropolizedindependentsamplingwithcomparisonstorejectionsamplingandimportancesampling.StatisticsandComputing,6,113. 67

PAGE 68

Liu,J.S.,Wong,W.H.,&Kong,A.(1994).CovariancestructureoftheGibbssamplerwithapplicationstocomparisonsofestimatorsandaugmentationschemes.Biometrika,81,27. Liu,J.S.,&Wu,Y.N.(1999).Parameterexpansionfordataaugmentation.JournaloftheAmericanStatisticalAssociation,94,1264. Marchev,D.(2011).MarkovchainMonteCarloalgorithmsfortheBayesianlogisticregressionmodel.InProceedingsoftheAnnualInternationalConferenceonOpera-tionsResearchandStatistics,(pp.154). Meyn,S.P.,&Tweedie,R.L.(1993).MarkovChainsandStochasticStability.London:Springer-Verlag. Mira,A.,&Geyer,C.J.(1999).OrderingMonteCarloMarkovchains.Tech.rep.,SchoolofStatistics,UniversityofMinnesota. Polson,N.G.,Scott,J.G.,&Windle,J.(2013).BayesianinferenceforlogisticmodelsusingPolya-Gammalatentvariables.JournaloftheAmericanStatisticalAssociation,108,1339. Retherford,J.R.(1993).HilbertSpace:CompactOperatorsandtheTraceTheorem.CambridgeUniversityPress. Roberts,G.O.,&Rosenthal,J.S.(1997).GeometricergodicityandhybrindMarkovchains.ElectronicCommunicatiosninProbability,2,13. Roberts,G.O.,&Rosenthal,J.S.(2004).GeneralstatespaceMarkovchainsandMCMCalgorithms.ProbabilitySurveys,1,20. Rosenthal,J.S.(1995).MinorizationconditionsandconvergenceratesforMarkovchainMonteCarlo.JournaloftheAmericanStatisticalAssociation,90,558. Rosenthal,J.S.(2003).Asymptoticvarianceandconvergenceratesofnearly-periodicMCMCalgorithms.JournaloftheAmericanStatisticalAssociation,98,169. Roy,V.,&Hobert,J.P.(2010).OnMonteCarlomethodsforBayesianmultivariateregressionmodelswithheavy-tailederrors.JournalofMultivariateAnalysis,101,1190. Rudin,W.(2005).FunctionalAnalysis.McGraw-Hill,2nded. Tan,A.,&Hobert,J.P.(2009).BlockGibbssamplingforBayesianrandomeffectsmodelswithimproperpriors:convergenceandregeneration.JournalofComputa-tionalandGraphicalStatistics,18,861. Tanner,M.A.,&Wong,W.H.(1987).Thecalculationofposteriordistributionsbydataaugmentation(withdiscussion).JournaloftheAmericanStatisticalAssociation,82,528. 68

PAGE 69

vanDyk,D.A.,&Meng,X.-L.(2001).Theartofdataaugmentation(withdiscussion).JournalofComputationalandGraphicalStatistics,10,1. West,M.(1984).OutliermodelsandpriordistributionsinBayesianlinearregression.JournaloftheRoyalStatisticalSociety,SeriesB,46,431. Yu,K.,&Moyeed,R.A.(2001).Bayesianquantileregression.Statistics&ProbabilityLetters,54,437. Yuan,Y.,&Yin,G.(2010).Bayesianquantileregressionforlongitudinalstudieswithnonignorablemissingdata.Biometrics,66,105. Zellner,A.,&Rossi,P.E.(1984).Bayesiananalysisofdichotomousquantalresponsemodels.JournalofEconometrics,25,365. 69

PAGE 70

BIOGRAPHICALSKETCH HeeMinChoiisborninSeoul,Korea.In2004,shewasadmittedtoYonseiUniversitywherein2009,sheearnedabachelor'sdegreeinindustrialengineeringandmathematics.Thatsameyear,shejoinedtheDepartmentofStatisticsattheUniversityofFlorida,receivingherdoctoraldegreein2014.Aftergraduation,HeeMinChoiwilljointheDepartmentofStatisticsattheUniversityofCalifornia,DavisasaWaldAssistantProfessor. 70