<%BANNER%>

Convergence Analysis of Block Gibbs Samplers for Bayesian General Linear Mixed Models

Permanent Link: http://ufdc.ufl.edu/UFE0043966/00001

Material Information

Title: Convergence Analysis of Block Gibbs Samplers for Bayesian General Linear Mixed Models
Physical Description: 1 online resource (97 p.)
Language: english
Creator: Roman, Jorge C
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2012

Subjects

Subjects / Keywords: chains -- ergodicity -- geometric -- gibbs -- markov -- sampler
Statistics -- Dissertations, Academic -- UF
Genre: Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: We consider two widely applicable Bayesian versions of the general linear mixed model (GLMM). These Bayesian GLMMs are created by adopting a proper and an improper (conditionally) conjugate prior density for the unknown parameters. The posterior densities for these models, which are often high-dimensional, are intractable in the sense that the integrals required for making inferences cannot be computed in closed form. However, in each case, there is a simple Markov chain Monte Carlo (MCMC) algorithm, the two-variable Gibbs sampler (TVGS), that can be employed to explore the intractable posterior density. We study the convergence rates of the Markov chains underlying these TVGSs and, in each case, we obtain an easily-checked sufficient condition for the geometric ergodicity of the TVGS Markov chain. The sufficient condition for geometric ergodicity found in the proper prior case is very mild and nearly always holds in practice. The result in the improper prior case is close to the best one possible in the sense that the sufficient condition is only slightly stronger than what is required to ensure posterior propriety. The theory developed in this dissertation is extremely important from a practical standpoint because it guarantees the existence of central limit theorems that allow for the computation of valid asymptotic standard errors for the estimates computed using the TVGS. In particular, we provide sufficient conditions for the honest estimation of the posterior expectations of the model parameters. By honest estimation, we mean that estimates are reported along with an asymptotically valid standard error (or a confidence interval) and there is a coherent strategy for deciding when to stop the MCMC simulation. To illustrate how to carry out an honest MCMC procedure, we use a real data set concerning health maintenance organizations. Finally, we use the results developed herein to provide a refinement of Tan and Hobert's (2009) main result.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Jorge C Roman.
Thesis: Thesis (Ph.D.)--University of Florida, 2012.
Local: Adviser: Hobert, James P.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2014-05-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2012
System ID: UFE0043966:00001

Permanent Link: http://ufdc.ufl.edu/UFE0043966/00001

Material Information

Title: Convergence Analysis of Block Gibbs Samplers for Bayesian General Linear Mixed Models
Physical Description: 1 online resource (97 p.)
Language: english
Creator: Roman, Jorge C
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2012

Subjects

Subjects / Keywords: chains -- ergodicity -- geometric -- gibbs -- markov -- sampler
Statistics -- Dissertations, Academic -- UF
Genre: Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: We consider two widely applicable Bayesian versions of the general linear mixed model (GLMM). These Bayesian GLMMs are created by adopting a proper and an improper (conditionally) conjugate prior density for the unknown parameters. The posterior densities for these models, which are often high-dimensional, are intractable in the sense that the integrals required for making inferences cannot be computed in closed form. However, in each case, there is a simple Markov chain Monte Carlo (MCMC) algorithm, the two-variable Gibbs sampler (TVGS), that can be employed to explore the intractable posterior density. We study the convergence rates of the Markov chains underlying these TVGSs and, in each case, we obtain an easily-checked sufficient condition for the geometric ergodicity of the TVGS Markov chain. The sufficient condition for geometric ergodicity found in the proper prior case is very mild and nearly always holds in practice. The result in the improper prior case is close to the best one possible in the sense that the sufficient condition is only slightly stronger than what is required to ensure posterior propriety. The theory developed in this dissertation is extremely important from a practical standpoint because it guarantees the existence of central limit theorems that allow for the computation of valid asymptotic standard errors for the estimates computed using the TVGS. In particular, we provide sufficient conditions for the honest estimation of the posterior expectations of the model parameters. By honest estimation, we mean that estimates are reported along with an asymptotically valid standard error (or a confidence interval) and there is a coherent strategy for deciding when to stop the MCMC simulation. To illustrate how to carry out an honest MCMC procedure, we use a real data set concerning health maintenance organizations. Finally, we use the results developed herein to provide a refinement of Tan and Hobert's (2009) main result.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Jorge C Roman.
Thesis: Thesis (Ph.D.)--University of Florida, 2012.
Local: Adviser: Hobert, James P.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2014-05-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2012
System ID: UFE0043966:00001


This item has the following downloads:


Full Text

PAGE 1

CONVERGENCEANALYSISOFBLOCKGIBBSSAMPLERSFORBAYESIAN GENERALLINEARMIXEDMODELS By JORGECARLOSROM AN ADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF DOCTOROFPHILOSOPHY UNIVERSITYOFFLORIDA 2012

PAGE 2

c 2012JorgeCarlosRom an 2

PAGE 3

ToAmy&Jody 3

PAGE 4

ACKNOWLEDGMENTS Firstly,IwanttothankmyadvisorDr.JamesHobert.Ifeelveryluckytohavehad theopportunitytobehisPh.D.student.Ithankhimforhispatience,forallthetimehe hasspentdiscussingideaswithme,andforallthecommentsandsuggestionshehas madethathavehelpedmegrowasaresearcherandwriter. Iwouldalsoliketothankeveryoneonmysupervisorycommittee.Iwouldliketo expressmyappreciationtoDr.HaniDossforhisexcellentteaching.Hetaughtmethree coursesallofwhichwereveryinteresting,challenging,andfun.IamthankfultoDr. KshitijKhareforhishelpandforgivingveryinterestingandhelpfulstudentseminarson Markovchains.IthankDr.ScottMcCulloughforhistimeandforteachingagreatcourse onMathematicalAnalysis. Ithankthefaculty,thegraduatestudents,andthestaffmembersattheDepartment ofStatisticsfortheirgeneralsupport.IspeciallythankDr.AndrewRosalsky,myMaster's advisor,forhisamazingteachingandmentoring.Iadmiretheenergyandpassionhe showswhenheteaches.IwouldalsoliketothankMar aRipolforherhelpandadvice. IthankShibasishDasgupta,DougSparks,JeremyGaskins,RebeccaSteorts,Anestis Touloumis,AlexanderKirpich,OleksandrSavenkov,ArkenduChatterjee,HeeMinChoi, andmanyotherUFgraduatestudentsfortheirhelp,laughter,andinformalforeign languagelessons. IwishtothankallmyfriendsinPuertoRico,specially,JodyMaysonet,Elsimarie Rosa,GlorielFlores,andMarianaTav arez.Icannotputintowordsthejoytheyhave broughttomylife.Lastly,IwishtothankmywifeAmyNonformakingmylifebetterin somanydifferentways,andforhearingmetalkaboutMarkovchainsandBayesian statistics. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS..................................4 LISTOFTABLES......................................7 LISTOFFIGURES.....................................8 ABSTRACT.........................................9 CHAPTER 1INTRODUCTION...................................11 1.1IntractableIntegralsandMarkovChainMonteCarloAlgorithms......11 1.2TheModels...................................14 1.3TheMCMCAlgorithms.............................17 2ONGEOMETRICERGODICITYANDHONESTMCMCPROCEDURES....22 2.1NotionsofConvergence............................22 2.2TheRoleofGeometricErgodicityinHonestMCMCProcedures......28 3CONVERGENCEANALYSISOFTHETVGS:PROPERPRIORS........32 3.1TheTVGSMarkovChain...........................32 3.2GeometricErgodicityoftheTVGSMarkovChain..............34 3.3HobertandGeyer'sResult......................44 3.4HonestEstimationofthePosteriorExpectations..............47 4CONVERGENCEANALYSISOFTHETVGS:IMPROPERPRIORS......54 4.1TheTVGSMarkovChain...........................54 4.2GeometricErgodicityoftheTVGSMarkovChain..............57 4.3TanandHobert'sResult........................66 4.4HonestEstimationofthePosteriorExpectations..............76 APPENDIX AMATRIXINEQUALITIES...............................79 BAPROOFOFCONTINUITY............................81 CUPPERBOUNDS..................................82 DFINITEMOMENTCONDITIONS..........................87 D.1ProperPriorCase...............................87 D.2ImproperPriorCase..............................92 5

PAGE 6

REFERENCES.......................................94 BIOGRAPHICALSKETCH................................97 6

PAGE 7

LISTOFTABLES Table page 3-1MIXEDprocedureestimates............................51 3-2PreliminaryMCMCestimates.............................52 3-3SimulationresultsfortheGLMMwithproperpriors: n = 116,000.........52 3-4SimulationresultsfortheGLMMwithproperpriors: n = 2,000,000........52 4-1SimulationresultsfortheGLMMwithimproperpriors: n = 720,000.......77 4-2SimulationresultsfortheGLMMwithimproperpriors: n = 2,000,000......77 7

PAGE 8

LISTOFFIGURES Figure page 3-1Traceplots:properpriorexample..........................53 4-1Traceplots:improperpriorexample.........................78 8

PAGE 9

AbstractofDissertationPresentedtotheGraduateSchool oftheUniversityofFloridainPartialFulllmentofthe RequirementsfortheDegreeofDoctorofPhilosophy CONVERGENCEANALYSISOFBLOCKGIBBSSAMPLERSFORBAYESIAN GENERALLINEARMIXEDMODELS By JorgeCarlosRom an May2012 Chair:JamesP.Hobert Major:Statistics WeconsidertwowidelyapplicableBayesianversionsofthegenerallinearmixed modelGLMM.TheseBayesianGLMMsarecreatedbyadoptingaproperandan improperconditionallyconjugatepriordensityfortheunknownparameters.The posteriordensitiesforthesemodels,whichareoftenhigh-dimensional,areintractable inthesensethattheintegralsrequiredformakinginferencescannotbecomputed inclosedform.However,ineachcase,thereisasimpleMarkovchainMonteCarlo MCMCalgorithm,thetwo-variableGibbssamplerTVGS,thatcanbeemployed toexploretheintractableposteriordensity.Westudytheconvergenceratesofthe MarkovchainsunderlyingtheseTVGSsand,ineachcase,weobtainaneasily-checked sufcientconditionforthegeometricergodicityoftheTVGSMarkovchain.Thesufcient conditionforgeometricergodicityfoundintheproperpriorcaseisverymildandnearly alwaysholdsinpractice.Theresultintheimproperpriorcaseisclosetothebestone possibleinthesensethatthesufcientconditionisonlyslightlystrongerthanwhatis requiredtoensureposteriorpropriety. Thetheorydevelopedinthisdissertationisextremelyimportantfromapractical standpointbecauseitguaranteestheexistenceofcentrallimittheoremsthatallowfor thecomputationofvalidasymptoticstandarderrorsfortheestimatescomputedusing theTVGS.Inparticular,weprovidesufcientconditionsforthehonestestimationof theposteriorexpectationsofthemodelparameters.Byhonestestimation,wemean 9

PAGE 10

thatestimatesarereportedalongwithanasymptoticallyvalidstandarderrorora condenceintervalandthereisacoherentstrategyfordecidingwhentostopthe MCMCsimulation.ToillustratehowtocarryoutanhonestMCMCprocedure,weusea realdatasetconcerninghealthmaintenanceorganizations.Finally,weusetheresults developedhereintoprovidearenementofTanandHobert's2009mainresult. 10

PAGE 11

CHAPTER1 INTRODUCTION 1.1IntractableIntegralsandMarkovChainMonteCarloAlgorithms Oneproblemthatfrequentlyarisesacrossmanydifferentdisciplinesistheneed toapproximateanintractableintegralwithrespecttoaprobabilitydistribution .A popularalternativetonumericalintegrationandanalyticalapproximationmethodsisthe MonteCarloMCmethodwhichusescomputersimulationstoestimatetheintegral.In theclassicalMCmethod,onegeneratesindependentandidenticallydistributediid samplesfrom andthenusessampleaveragestoestimatetheintegral.However,in manysituations, isacomplexhigh-dimensionalprobabilitydistributionandobtaining iidsamplesfromitiseitherimpossibleorimpractical.Whenthishappens,onemaystill beabletousetheincreasinglypopularMarkovchainMonteCarloMCMCmethod inwhichtheiiddrawsarereplacedbyaMarkovchainthathas asitsstationary distribution. TheMCMCmethodhasnumerousapplicationsinBayesianstatistics,anareain whichmakinginferenceabouttheunknownparametersofteninvolveshigh-dimensional integrationthatcannotbedoneinclosedform.Themaingoalofthisdissertationisto provideadetailedexplanationabouthowtodevelopanhonestMCMCprocedurefor twopopularBayesianversionsofthegenerallinearmixedmodelGLMM.Byhonest MCMC,wemeanthatestimatesofquantitiesofinterestarereportedalongwitha valid measureoftheiruncertaintyandthereisacoherentstrategyfordecidingwhentostop thesimulation.BeforeweintroducetheBayesianGLMMs,wegiveabriefreviewof thetheorybehindtheMCandMCMCmethodsandexplainhowthesemethodscanbe utilizedinthecontextofBayesianstatistics. Let denoteaposteriordensitywithrespecttoameasure onatopologicalspace X andsupposewewouldliketoestimatetheintractableposteriorexpectation E g := Z X g x x dx 11

PAGE 12

where g isa -integrablefunctionofinterest.Ifiidrandomelements, f X n g 1 n =0 ,canbe drawndirectlyfrom ,thenwecanestimate E g usingthesampleaverage ^ g n = 1 n n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X i =0 g X i ThisestimatorisunbiasedandthestronglawoflargenumbersSLLNimpliesthat,with probability1, ^ g n E g as n !1 Inotherwords, ^ g n isunbiasedandstronglyconsistentfor E g Inpractice,wecanonlydrawanitenumberofrandomelements,sothereisan associatedMCerror, ^ g n )]TJ/F39 11.9552 Tf 12.513 0 Td [(E g .Unfortunately,wecannotknowtheexactvalueofthis errorbecausewedonotknowthevalueof E g .However,wecanapproximatethe samplingdistributionoftheMCerrorprovidedthataCentralLimitTheoremCLTholds for f ^ g n g 1 n =0 .Thatis,if p n ^ g n )]TJ/F39 11.9552 Tf 11.955 0 Td [(E g d N V g as n !1 where V g = E g 2 )]TJ/F22 11.9552 Tf 11.955 0 Td [( E g 2 < 1 AsufcientconditionfortheexistenceoftheCLT1issimply E g 2 < 1 .Once weknowthataCLTholds,wecanestimatetheasymptoticvariance V g using ^ V n := 1 n )]TJ/F22 11.9552 Tf 11.955 0 Td [(1 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X i =0 )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(g X i )]TJ/F22 11.9552 Tf 12.198 0 Td [(^ g n 2 whichisstronglyconsistentfor V g andcalculateastandarderror, q ^ V n = n .Withthis information,wecanformanasymptoticallyvalidcondenceintervalCIfor E g and thisallowsustodecidewhentostopthesimulation.Acommonwayofstoppingthe simulationissimplytocontinuetheprocedureuntiltheCIissufcientlysmall. TheMCMCprocedurefollowssimilarsteps,however,therearesomeimportant differenceswhichwenowdescribe.RecallthatintheMCMCmethod,theiiddraws 12

PAGE 13

f X n g 1 n =0 arereplacedwithaMarkovchain f n g 1 n =0 withstationarydistribution ,the probabilitydistributioncorrespondingtothedensity .InanMCMCprocedure,we estimate E g withtheergodicaverage g n := 1 n n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X i =0 g i which,undersimpleregularityconditions,isstronglyconsistentfor E g Meynand Tweedie,1993,Chapter17,but,unlike ^ g n ,itisnotnecessarilyunbiased. AnotherdifferencebetweentheMCandMCMCmethodsisthatverifyingthataCLT for g n holdsismuchharderduetothedependencyamongtherandomvariablesinthe Markovchain.Thatis,themomentcondition E g 2 < 1 isnolongersufcientforthe existenceofaCLTfor g n .ThestandardmethodofestablishingtheexistenceofaCLT for g n entailstwosteps.TherststepistoprovethattheMarkovchainis geometrically ergodic ;thatis,thechainconvergestothetargetdistribution atageometricrate. Thesecondistoestablishthemomentcondition: E j g j 2+ < 1 ,forsome > 0 Accomplishingtherststepisgenerallyverychallenginganddependsonlyonthe Markovchain.Indeed,verifyingthataMarkovchainisgeometricallyergodicusually boilsdowntoestablishinga driftcondition which,inpracticallyrelevantsituations,tends torequirealotoftrialanderroranddifculttheoreticalanalysis.Ontheotherhand, accomplishingthesecondstepwhichdependsonthefunctionofinterest g andthe targetdensity tendstobemucheasierrelativetostepone. Finally,theexpressionfortheasymptoticvarianceinMarkovchainCLTsismuch morecomplicatedthanthatoftheclassicalCLTforiidrandomvariables.Itaccounts fortheserialdependenceintheMarkovchainandconsistentestimationofitrequires specializedmethodssuchasregenerativesimulation,batchmeans,orspectralmethods see,e.g.,FlegalandJones2010;Hobertetal.2002;Jonesetal.2006.Applying thesemethodsrequiresthetwostepsmentionedaboveplusthevericationofextra conditionsthataremethoddependentseeSection2.2.Duetothesedifculties, 13

PAGE 14

MCMCusersgenerallydo not reportvalidstandarderrorsnorCIsalongwiththepoint estimatesof E g Flegaletal.,2008.Inthenextsection,wedescribethetwoBayesian GLMMsforwhichwewilldevelopanMCMCprocedurethatallowstheusertoreport,not onlypointestimatesofposteriorexpectations,butalsoasymptoticallyvalidCIsforthese quantities. 1.2TheModels TheGLMMtakestheform Y = X + Zu + e where Y isan N 1 datavector, X and Z areknownmatriceswithdimensions N p and N q ,respectively, isanunknown p 1 regressioncoefcient, u isarandomvector whoseelementsrepresentthevariouslevelsoftherandomfactorsinthemodel,and e N N 2 e I .Therandomvectors e and u areassumedtobeindependent.Suppose thereare r randomfactorsinthemodel.Then u and Z arepartitionedaccordinglyas u = )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(u T 1 u T 2 u T r T and Z = Z 1 Z 2 Z r ,where u i is q i 1 Z i is N q i ,and q 1 + + q r = q .Then Zu = r X i =1 Z i u i anditisassumedthat u N q u ,where u = r i =1 2 u i I q i .Forbackgroundonthis model,whichissometimescalledthe variancecomponentsmodel ,seeSearleetal. 1992. BeforewedescribeourrstBayesianGLMM,wedenesomenotation.Itis convenienttoworkwiththeprecisionparameters e =1 = 2 e u 1 =1 = 2 u 1 ,..., u r =1 = 2 u r Withthisinmind,let = )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( e u 1 u r T denotethevectorofprecisionparameters andlet u := )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u ;thatis, u isa q q diagonalmatrixwiththerst q 1 diagonalentries equalto u 1 ,thenext q 2 diagonalentriesequalto u 2 ,andsoon. 14

PAGE 15

Attherststageofthemodel,wehave Y j u N N )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(X + Zu )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e I Given u 1 ,..., u r ,therandomvectors u ,and e areassumedtobemutually independent.Thesecondstagespeciestheirdistribution: N p 0 u j u 1 ,..., u r N q )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [(0, )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u and e Gamma a e b e Finally,thethirdstageofthemodelspeciesthedistributionof u 1 ,..., u r ,whichare independentwithmarginalsgivenby u i Gamma a i b i for i =1,..., r Let f G g ; a b denotethedensityofagammarandomvariable;thatis, f G g ; a b = b a \050 a g a )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e )]TJ/F40 7.9701 Tf 6.586 0 Td [(bg I R + g where a b > 0 and R + :=, 1 .Thepriordensityof and isgivenby p ; 0 ~ a ~ b = f N ; 0 f G e ; a e b e r Y i =1 f G u i ; a i b i # where 0 ~ a = a e a 1 a r T ,and ~ b = b e b 1 b r T areknownhyper-parameters. Itisassumedthatthesehyper-parametersarechosensothattheposteriorisproper. Dene = T u T T and W = XZ ,sothat W = X + Zu .Also,let y denotethe observeddata.Theposteriordensitycorrespondingtotheprior p isgivenby j y = j y m y where isanunnormalizeddensitygivenby j y = f N y ; W )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e I f N )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(u ;0, )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u p ; 0 ~ a ~ b 15

PAGE 16

and m y isthemarginaldensityofthedata,whichservesasanormalizingconstant andisgivenby m y = Z R r +1 + Z R p + q j y d d < 1 Ofcourse, ,and m alldependon X Z 1 ,..., Z r ,andthehyper-parameters,butthis dependencyissuppressedinthenotation. Theproperconditionallyconjugateprior p describedaboveisapopularchoice. Oneobviousreasonforthisisthattheresultingposteriorhasconditionaldensities withstandardforms,andthisfacilitatestheuseofMCMCalgorithmssuchasthe Gibbssampler.However,insituationswherethereislittlepriorinformation,the hyper-parameters ~ a and ~ b of p areoftensettoextremevaluesasthisisthoughtto yieldanon-informativeprior.Unfortunately,theseextremeproperpriorsapproximate improperpriorsthatcorrespondtoimproperposteriors,andthisresultsinvarious formsofinstability.Thisproblemhasledseveralauthors,includingDaniels1999and Gelman2006,todiscouragetheuseofsuchextremeproperpriors,andtorecommend alternativedefaultpriorsthatareimproper,butleadtoproperposteriors.Forthisreason, wealsoconsidertheimproperpriordensitygivenby, ~ p ; ~ a ~ b = a e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e e )]TJ/F40 7.9701 Tf 6.586 0 Td [(b e e r Y i =1 a i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i e )]TJ/F40 7.9701 Tf 6.587 0 Td [(b i u i # whichwhenwrittenintermsofthevarianceparametersgeneralizestheimproperprior consideredinHobertandCasella1996.Bytaking ~ b tobe0,wecanrecovertheprior ofHobertandCasella1996andbyfurthertaking ~ a tobe )]TJ/F22 11.9552 Tf 12.608 0 Td [(0.5 )]TJ/F22 11.9552 Tf 12.608 0 Td [(0.5 )]TJ/F22 11.9552 Tf 31.858 0 Td [(0.5 wecanrecoveroneofthepriorsrecommendedbyGelman2006.Asopposedtothe properpriorcase,wedonotassumethatthecomponentsof ~ a and ~ b arepositive.Note thattheright-handsideof1doesnotdependon ;thatis,weareusingaso-called atpriorfor .Consequently,evenifalltheelementsof a and b arestrictlypositive,so thateveryvariancecomponentgetsaproperprior,theoverallpriorremainsimproper. 16

PAGE 17

Let ~ denotetheposteriordensitycorrespondingtotheprior ~ p .Bydenition, ~ j y isproperif ~ m y := Z R r +1 + Z R p + q ~ j y d d < 1 where ~ j y = f N y ; W )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e I f N )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(u ;0, )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u ~ p ; ~ a ~ b Wewillstatearesultthatprovidessufcientconditionsfortheproprietyoftheposterior inSection1.3whereitisusedinacomparison. Wenowexplainexactlyinwhatsense and ~ areintractable.Let E g and E ~ g denotetheposteriorexpectationsofafunction g : R r +1 + R p + q R withrespectto and ~ ,respectively.Ofcourse, E g = Z R r +1 + Z R p + q g j y d d = 1 m y Z R r +1 + Z R p + q g j y d d and E ~ g canbeexpressedinasimilarfashionusing ~ and ~ m y .Recalltheexpression of m y and ~ m y givenin1and1,respectively.Theinnerintegralsin m y and ~ m y canactuallybeevaluatedinclosedformbuttheouterintegralsdonothave aknownclosedformsolution.Thus m y and ~ m y are r +1 -dimensionalintegrals anditfollowsthat,ingeneral, E g and E ~ g willbearatiooftwointractableintegrals,one havingdimension p + q + r +1 andtheotherhavingdimension r +1 .Inthenextsection, wedescribesimpleMCMCalgorithmsthatwillallowustoeffectivelyapproximatethe quantities E g and E ~ g 1.3TheMCMCAlgorithms Thetwo-variableGibbssamplerTVGSisasimplebutveryusefulMCMC algorithmthatisapplicableintheposterioranalysisofmanycomplexBayesianmodels. TherearetwosimpleTVGSsthatcanbeusedtoapproximatetheintractableposterior expectationscorrespondingtothedensities and ~ .SeeChapters3and4forformal denitionsoftheseMCMCalgorithms.BothoftheseTVGSssimulateaMarkovchain 17

PAGE 18

oftheform, f n n g 1 n =0 ,thatliveson X= R r +1 + R p + q ,andhasthecorresponding posteriordensity or ~ asitsinvariantdensity.Ifthecurrentstateofthechainis n n ,thenthenextstate, n +1 n +1 ,issimulatedintwosteps.Indeed,wedraw n +1 fromtheconditionalposteriordensityof given = n ,whichisaproductof r +1 univariategammadensities,andthenwedraw n +1 fromtheconditionalposterior densityof given = n +1 ,whichisa p + q -dimensionalmultivariatenormaldensity. TheseTVGSsaresometimesreferredtoas block Gibbssamplerssincethey block theprecisionparameters e u 1 ,..., u r intothevector ,andthevectors and u intoa singlevector = T u T T BecausetheTVGSMarkovchainsdescribedhereinareHarrisergodicsee Section2.1fordetailsandadenition,wecanusethemtoconstructconsistent estimatesofintractableposteriorexpectations.Unfortunately,Harrisergodicity, apropertythatensuresthattheMarkovchainconvergestothetargetposterior distribution,doesnotimplytheexistenceofaCLT.RecallfromSection1.1thata CLTholdsifthechainisgeometricallyergodicanda + momentconditionholds. Let = f n n g 1 n =0 betheunderlyingMarkovchainoftheTVGSusedtoexplore .Hereisourrstresultwhichprovideseasily-checkedconditionsunderwhich is geometricallyergodic. Proposition1.1. TheMarkovchain ,isgeometricallyergodicif 1. X hasfullcolumnrank, 2. a e > 1 2 rank Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(N +2 ,and 3. 1 2 )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(q )]TJ/F47 11.9552 Tf 11.956 0 Td [(rank Z +1 < min a 1 + q 1 2 ,..., q r + q r 2 TheconditionsofProposition1.1arequiteweakandwillalmostalwaysholdinpractice. Notealsothattheseconditionsdonotinvolvetheobserveddata, y Proposition1.1isarenementofthemainresultinJohnsonandJones2010. Themostimportantdifferencebetweenourresultandtheirsisthatwedonotassume that X T Z =0 .Ofcourse,itisrarelythecaseinpracticethat X T Z =0 .Moreover,this conditiongreatlysimpliesthedriftcalculationsthatarerequiredtoestablishgeometric 18

PAGE 19

convergence.Indeed,when X T Z =0 and u areconditionallyindependentgiven .Anothertwomajordifferencesbetweenourresultandtheirsarethatweconsider thecasewhenthereare r > 1 variancecomponentsassociatedwith u andwedonot assumethat Z hasfullcolumnrank. Beforewediscussourconvergenceresultsintheimproperpriorcase,westatea resultthatprovidessufcientandnearlynecessaryconditionsfortheproprietyofthe posterior.TheresultfollowsimmediatelyfromthemainresultinSunetal.2001. Theorem1.1. [Sun,Tsutakawa&He]Assumethatrank X = p andlet t = rank )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Z T I )]TJ/F39 11.9552 Tf 11.955 0 Td [(X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z .Ifthefollowingfourconditionshold,then ~ m y < 1 A Foreach i 2f 1,2,..., r g ,oneofthefollowingholds: A1 a i < b i =0; A2 b i > 0 B Foreach i 2f 1,2,..., r g q i +2 a i > q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t C N +2 a e > p )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 P r i =1 a i I ,0 a i D 2 b e + k I )]TJ/F39 11.9552 Tf 11.955 0 Td [(W W T W )]TJ/F39 11.9552 Tf 7.085 -4.339 Td [(W T y k 2 > 0 Let ~ denotetheTVGSMarkovchainfortheimproperpriorcasedescribedearlier. Proposition4.1providesconditionsunderwhichtheMarkovchain ~ isgeometrically ergodic.Checkingtheseconditionsmayrequiresomenumericalwork.Thefollowing corollarytoProposition4.1isweaker,buteasiertoapply. Corollary1. Assumethatrank X = p .Ifthefollowingfourconditionshold,thenthe Markovchain ~ isgeometricallyergodic. A Foreach i 2f 1,2,..., r g ,oneofthefollowingholds: A1 a i < b i =0; A2 b i > 0 B 0 Foreach i 2f 1,2,..., r g q i +2 a i > q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t +2 C 0 N +2 a e > p + t +2 D 2 b e + k I )]TJ/F39 11.9552 Tf 11.956 0 Td [(W W T W )]TJ/F39 11.9552 Tf 7.085 -4.338 Td [(W T y k 2 > 0 Incaseswheretheposteriordensityisimproper,itissometimesstillpossibletorun theTVGS,butthecorrespondingMarkovchainscannotbegeometricallyergodicsee Chapter4.Therefore,thebestwecouldhopeforisthattheTVGSMarkovchain ~ 19

PAGE 20

isgeometricallyergodicwhenevertheposterior ~ isproper.Withthisinmind,note thattheconditionsofCorollary1areveryclosetotheconditionsforproprietygivenin Theorem1.1.Infact,theformerimplythelatter.Toseethis,assumethat A B 0 C 0 and D allhold.Then,obviously, B holds,andallthatremainsistoshowthat C holds.Thiswouldfollowimmediatelyifwecouldestablishthat t )]TJ/F22 11.9552 Tf 21.917 0 Td [(2 r X i =1 a i I ,0 a i Weconsidertwocases.First,if P r i =1 I ,0 a i =0 ,then )]TJ/F22 11.9552 Tf 9.299 0 Td [(2 P r i =1 a i I ,0 a i =0 and 1holdssince t isnon-negative.Ontheotherhand,if P r i =1 I ,0 a i > 0 ,then thereisatleastone i 2f 1,2,..., r g suchthat a i < 0 ,and B 0 impliesthat r X i =1 q i +2 a i I ,0 a i > q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t Thisinequalitycombinedwiththefactthat q = q 1 + + q r yields t > q )]TJ/F40 7.9701 Tf 18.666 14.944 Td [(r X i =1 q i +2 a i I ,0 a i )]TJ/F22 11.9552 Tf 21.918 0 Td [(2 r X i =1 a i I ,0 a i so1holds,andthiscompletestheargument.Thestrongsimilaritybetween Theorem1.1andCorollary1mightleadthereadertobelievethattheproofsofour resultsrelysomehowonTheorem1.1.Thisisnotthecase.Infact,inourconvergence rateanalysis,wedonotassumethat ~ isproper. AsidefromourworkandthatofJohnsonandJones's2010,thereareonlyafew articlesonBayesianlinearmodelswithnormalerrorsinwhichgeometricergodicity oftheGibbssamplerisestablished.TheseareHobertandGeyer1998,Jonesand Hobert2004,andPapaspiliopoulosandRoberts2008,whichconsiderlinearmodels with proper priors,andTanandHobert2009whichprovidestheonlyotherexisting resultsforlinearmodelswith improper priors.Ourmodelsaremuchmoregeneralthan theone-wayrandomeffectsmodelsconsideredinHobertandGeyer1998,Jones andHobert2004,andTanandHobert2009.Finally,themodelsconsideredby 20

PAGE 21

PapaspiliopoulosandRoberts2008aremuchsimplerthanoursbecausetheyassume thatvariancecomponentsareknown. Theconvergenceanalysesintheproperandimpropercasesaresimilarinthe sensethatgeometricergodicityisestablishedviadriftfunctionsinbothcases.However, thedriftfunctionsareverydifferentfromeachother,andtheanalysisrequiredinthe improperpriorcasesissubstantiallymoredemanding.Moreover,thereisaninteresting technicalissuethatarisesintheimproperpriorcases.AnoversightbyTanandHobert 2009regardingthistechnicalityledtoanerrorintheirmainresult.However,itis showninChapter4thattheirproofiseasilyrepairedusingtheresultsdevelopedherein. Furthermore,ourresultscanbeusedtoimproveTanandHobert's2009resultby removingunnecessaryconditionsthatcanfailwhenthedataaretoounbalanced. Therestofthedissertationisorganizedasfollows.Chapter2containsbasic backgroundongeneralstatespaceMarkovchaintheoryandresultsforestablishing geometricergodicityinthecontextoftheTVGS.MarkovchainCLTsandmethods forestimatingtheasymptoticvariancesarealsodiscussed.InChapter3,weperforma convergenceanalysisoftheTVGSMarkovchain andprovideaproofofProposition1.1. Further,weprovideresultsthatcanbeusedtoestablishmomentconditionsthatlead tothevalidestimationoftheposteriorexpectationsofthecanonicalparametersinthe model.Inaddition,anhonestMCMCprocedureisillustratedusingarealdataseton healthmaintenanceorganizations.Chapter4followsthesamestructureasChapter3.A sufcientconditionforthegeometricergodicityof ~ isgivenandaninterestingtechnical issuerelatedtotheuseofimproperpriorsisdiscussed.Also,acorrectedproofforTan andHobert's2009mainresultisgiven. 21

PAGE 22

CHAPTER2 ONGEOMETRICERGODICITYANDHONESTMCMCPROCEDURES 2.1NotionsofConvergence Let f n g 1 n =0 beatime-homogeneousMarkovchainonatopologicalspace X withBorel -algebra B X andsupposethatthechainevolvesaccordingtotheMarkov transitionfunctionMtf K :X B X [0,1] ;thatis, K x A =P m +1 2 A j m = x x 2 X, A 2B X forall m 2f 0,1,2,... g .Also,let K 1 x A = K x A .For n 2 ,wedenethe n -stepMtf inductivelyas K n x A = Z X K n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 y A K x dy x 2 X, A 2B X Todescribethenotionsofconvergenceandthetoolsthatwillbeusedinourconvergence analyses,werstgiveaseriesofbasicdenitions. Wesaythat is -irreducibleforsomenon-trivial -nitemeasure on X, B X if forall x 2 X and A 2B X with A > 0 ,thereexistsapositiveinteger n = n x A suchthat K n x A > 0 .Thatis,theMarkovchainis -irreducibleifall -positivesets canbereachedfromanygivenstate x 2 X .If is -irreducibleforsomemeasure thenthereexistsaprobabilitymeasure ,calledthe maximalirreducibilitymeasure such thati is -irreducible,andiiforanyothermeasure 0 ,thechain is 0 irreducible ifandonlyif 0 isabsolutelycontinuouswithrespectto whichwedenoteby 0 Clearly,themaximalirreducibilitymeasureisuniqueuptoequivalence;i.e.,if 1 and 2 arebothmaximalirreducibilitymeasures,then 2 1 and 1 2 whichwedenote by 1 2 .TheMarkovchain willbecalled -irreducibleifitis -irreducibleforsome measure and isamaximalirreducibilitymeasure. 22

PAGE 23

Supposethat isa -nitemeasureon X, B X and k :X X [0, 1 isafunction suchthatforany x 2 X and A 2B X K x A = Z A k x 0 j x dx 0 Thefunction k iscalledaMarkovtransitiondensityMtdfor withrespectto .Hereis asimpleresultonmaximalirreducibilitymeasures. Lemma1. Supposethat isa -irreducibleMarkovchainwithMtd k withrespectto If k j isstrictlypositiveon X X ,then Proof. Since k isstrictlypositive,wehave K x A = R A k x 0 j x dx 0 > 0 forall x 2 X whenever A > 0 .Thisshowsthat is -irreducible.Now,bydenition,wemusthave .Toshowthat ,suppose A =0 ,thenbyinduction,wehave K n x A =0 forall x 2 X andall n 2 N whichimpliesthat A =0 .Thiscompletestheproof. A -irreducibleMarkovchainhasperiod d ifthestatespace X canbepartitioned intodisjointsets E 1 ,..., E d N 2B X suchthat N isa -nullset, K x E i +1 =1 forall x 2 E i i =1,2,..., d )]TJ/F22 11.9552 Tf 11.987 0 Td [(1 ,and K x E 1 =1 forall x 2 E d .If d 2 thenthechainiscalled periodic,otherwiseitiscalledaperiodic.A -irreducibleMarkovchainisHarrisrecurrent ifforanystartingvalue x 2 X andany -positiveset A ,wehave P n 2 A i.o. j 0 = x =1. Inotherwords,a -irreducibleMarkovchainisHarrisrecurrentifforanystartingvalue x 2 X andany -positiveset A ,thechainenterstheset A innitelyoftenwithprobability one.A -nitemeasure on X, B X issaidtobeinvariantif A = Z X K x A dx forall A 2B X AMarkovchainiscalledpositiveifitis -irreducibleandadmitsaninvariantprobability measure.Finally,wesaythat is Harrisergodic if isa -irreducible,aperiodic, positiveHarrisrecurrentMarkovchain. 23

PAGE 24

ThefollowingresultprovidessufcientconditionsforHarrisergodicity. Lemma2. Suppose isaMarkovchainwithMtd k andinvariantprobabilitymeasure If k j isjointlymeasurableandstrictlypositiveon X X ,then isHarrisergodic. Proof. Since k isstrictlypositive,thechaincangettoany -positivesetfromanystate x 2 X in one step.Thisimpliesthat isboth -irreducibleandaperiodic.Finally,that isHarrisrecurrentfollowsfromTheorem2inAsmussenandGlynn2011.Hence, is Harrisergodic. Asshownabove,provingLemma2comesdowntoestablishingHarrisrecurrence. Todothis,weusedAsmussenandGlynn's2011resultwhichtakesadvantageof theexistenceofaninvariantprobabilitymeasure.ForotherresultsthatleadtoHarris recurrence,seeNummelin1984andMeynandTweedie1993. Harrisergodicityimpliesthat,forany x 2 X k K n x )]TJ/F22 11.9552 Tf 11.955 0 Td [( k TV :=sup A 2B X j K n x A )]TJ/F22 11.9552 Tf 11.955 0 Td [( A j# 0 as n !1 seeMeynandTweedie1993Theorem13.0.1andProposition13.3.2.Thistellsus thatthetotalvariationdistancebetweenthemeasures K n x and decreasesto zeroasthenumberofsteps, n ,increases,butitdoes not tellusanythingaboutthe rate ofconvergence.AMarkovchainwithstationarydistribution iscalled geometrically ergodic ifthereexistsaconstant 0 < 1 andfunction M :X [0, 1 ,suchthatfor any x 2 X k K n x )]TJ/F22 11.9552 Tf 11.955 0 Td [( k TV M x n forall n 2 N Thatis,aMarkovchainisgeometricallyergodicifthetotalvariationdistancebetween themeasures K n x and decreasestozeroata geometric rateasthenumberof steps, n ,increases. Inordertogivearesultthatprovidessufcientconditionsforgeometricergodicity weneedtwomoredenitions.If K O isalowersemi-continuousfunctionon X for 24

PAGE 25

allopensets O 2B X then iscalleda weakFellerchain .Thefollowingresultgives sufcientconditionsforaMarkovchaintobeFeller. Lemma3. Let beaMarkovchainwithMtd k withrespectto .If k x j isalower semi-continuousfunctionon X forallxed x 2 X ,then isaFellerchain. Proof. Let K betheMtfof andlet O 2B X beanopenset.Wewillverifythat K O isalowersemi-continuousfunctionon X .Fix x 2 X andlet f x m g 1 m =1 beasequencein X thatconvergesto x .Then liminf m !1 K x m O =liminf m !1 Z O k x 0 j x m dx 0 Z O liminf m !1 k x 0 j x m dx 0 Z O k x 0 j x dx 0 = K x O wheretherstinequalityfollowsfromFatou'sLemmaandthesecondinequalityfollows fromthelowersemi-continuityof k x 0 j .Hence, K O islowersemi-continuouson X Afunction v :X [0, 1 issaidtobe unboundedoffcompactsets iftheset f x 2 X: v x d g iscompactforall d 2 R Thefollowingresultwillbeourmain theoreticaltoolforestablishinggeometricergodicity.Wewillprovideaproofofitusinga seriesofresultsinMeynandTweedie1993. Lemma4. Let = f n g 1 n =0 beaMarkovchainonatopologicalspace X, B X .If i isa -irreducibleFellerchain iithesupportofthemaximalirreducibilitymeasure hasnonemptyinterior iiithereexist 0 < 1 L < 1 ,andafunction v :X [0, 1 thatisunboundedoff compactsetssuchthat Kv x := Z X v x 0 K x dx 0 v x + L forall x 2 X, then 25

PAGE 26

1. isHarrisrecurrent 2.thereexistsauniqueuptoconstantmultiplesinvariantmeasure If isalsoaperiodic,thentheinvariantmeasure isniteand isgeometrically ergodic. Theinequality2iscalledadriftconditionandthefunction v iscalledadriftfunction. Remark1. Theleft-handsideof 2 isoftenwrittenas E [ v 1 j 0 = x ] Remark2. ThehypothesisofLemma4doesnotincludetheexistenceofaninvariant measure.Theexistenceofsuchmeasureispartoftheconclusion. ProofofLemma4. Let V x = v x +1 andnotethat V isalsounboundedofcompact setsin X .Moreover,2impliesthat V isalsoadriftfunctionandsatisesthedrift condition KV x V x + L 0 where L 0 = L +1 )]TJ/F25 11.9552 Tf 11.07 0 Td [( .Since is -irreducible,Fellerandthesupportof hasnon-empty interior,MeynandTweedie's1993Theorem6.0.1impliesthatallcompactsetsin X are petitesets for .Therefore,thedriftfunction V is unboundedoffpetitesets Meynand Tweedie,1993,p.191. Now,MeynandTweedie's1993Lemma15.2.8impliesthatthereexistsaconstant > 0 andapetiteset C suchthat V x := KV x )]TJ/F39 11.9552 Tf 11.955 0 Td [(V x )]TJ/F25 11.9552 Tf 21.918 0 Td [( V x + I C x L 0 forall x 2 X. Thussince is -irreducibleand V isunboundedofpetitesetswith V x 0 for all x 2 C c ,itfollowsfromMeynandTweedie's1993Theorem9.1.8that isHarris recurrent.Thisestablishestherststatement.Thesecondstatementnowfollowsfrom MeynandTweedie's1993Theorem10.0.1. Supposethat isalsoaperiodic.AnapplicationofMeynandTweedie's1993 Theorem15.0.1andTheorem10.0.1showsthattheinvariantmeasure isnite. Moreover,anotherapplicationofMeynandTweedie's1993Theorem15.0.1anda 26

PAGE 27

straightforwardmanipulationshowthatthereexistconstants 0 < 1 and c 0 < 1 such thatforany x 2 X k K n x )]TJ/F22 11.9552 Tf 11.955 0 Td [( k TV c 0 V x n forall n 2 N Hence,weconcludethat isgeometricallyergodic. Lemma4isaqualitativeresultbecauseitcanonlyguaranteetheexistenceofa function M x andaconstant suchthat2holds.Inotherwords,itdoesnotprovide uswith M and thatsatisfy2.Indeed,itdoesnotevenprovideuswithboundsfor thesethings.However,asshownintheproofofLemma4,thefunction M x canbe takentobeproportionaltothedriftfunction V x = v x +1 .Therefore,intheabsence ofadditionalinformationabouttheconvergencerate,itseemsreasonabletostartthe chainfromthestate x thatminimizes v x ThefollowingresultfollowsimmediatelyfromRosenthal's1995Theorem12and canbeusedtoderiveupperboundsontheleft-handsideof2providedthatadrift conditionandanassociated minorizationcondition areestablished.Wesaythata one-stepminorizationconditionholdsfor ifthereexistafunction s :X R + such that E s := Z X s x dx > 0, andaprobabilitymeasure Q on X, B X suchthat K x A s x Q A forall x 2 X and A 2B X Theorem2.1. Let = f n g 1 n =0 bea -irreducibleandaperiodicMarkovchainwith invariantprobabilitymeasure on X, B X .Supposethat 1.thereexist 0 < 1 L < 1 ,andafunction V :X [0, 1 suchthat KV x V x + L forall x 2 X, 27

PAGE 28

2.thechainsatisestheminorizationcondition 2 with s x = I C x where > 0 C = f x 2 X: V x d g ,and d > 2 b 1 )]TJ/F26 7.9701 Tf 6.587 0 Td [( Thenforany 0 < r < 1 and x 2 X k P n x )]TJ/F22 11.9552 Tf 11.955 0 Td [( k TV )]TJ/F25 11.9552 Tf 11.956 0 Td [( rn + )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [( )]TJ/F40 7.9701 Tf 6.586 0 Td [(r A r n 1+ b 1 )]TJ/F25 11.9552 Tf 11.956 0 Td [( + V x where )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 1+2 b + d 1+ d < 1 and A =1+2 d + b Remark3. UnlikeLemma4,thedriftfunctioninTheorem2.1isnotrequiredtobe unboundedoffcompactsets. Remark4. Whenthisresultisapplied,theuserhastospecifythevaluesof d and r Thevalueof r shouldbechosensothat )]TJ/F23 7.9701 Tf 6.586 0 Td [( )]TJ/F40 7.9701 Tf 6.587 0 Td [(r A r < 1 ,otherwisetheboundmaynot decreasein n Whenapplicable,Rosenthal's1995Theorem12canbeusedtocalculatehow longittakesfortheMarkovchaintogetsufcientlycloseto .Thatis,onecanactually calculatehowmuch burnin isnecessaryinMCMCapplications.Thistheoremhasbeen successfullyusedinseveralarticlesincludingJonesandHobert2004andMarchev andHobert2004.However,inourexperiencewithTVGSsforBayesianGLMMs,we havenotbeenabletoestablishdriftandminorizationconditionsthatleadtouseful Rosenthal-typebounds.SeealsoRobertsandTweedie1999,2001forasimilarresult thathasthepotentialtoyieldtighterupperbounds.Forfurtherdetailsongeometric ergodicity,seeMeynandTweedie1993,Chapter15. Inthenextsection,wediscusstherolethatgeometricergodicityplaysinthe asymptoticbehaviorofMCMCestimators. 2.2TheRoleofGeometricErgodicityinHonestMCMCProcedures Let beaprobabilitydensitywithrespecttoa -nitemeasure onatopological space X .Also,for d > 0 ,let L d denotethesetoffunctions g :X R suchthat 28

PAGE 29

E j g j d < 1 .Supposeweareinterestedinestimatingtheintractableintegral E g = Z X g x x dx where g 2 L 1 .If = f g 1 n =0 isaHarrisergodicchainwithinvariantdensity then MeynandTweedie's1993Theorem17.0.1guaranteesthat,withprobability1, g n = 1 n n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X i =0 g i E g as n !1 nomatterwhatthedistributionof 0 .Thatis, g n isstronglyconsistentfor E g .However, asmentionedinChapter1,Harrisergodicitydoes not implyexistenceofaCLTfor g n Thefollowingtheoremisawellknownresultthatprovidessufcientconditionsfor theexistenceofaCLTfor g n .SeeTheorem18.5.3ofIbragimovandLinnik1971, Theorem2ofChanandGeyer1994,andTheorem24ofRobertsandRosenthal 2004.Foragivenprobabilitydensity ,wewilluse todenoteitscorresponding probabilitymeasure. Theorem2.2. Suppose isaHarrisergodicMarkovchainwithinvariantdensity .If isgeometricallyergodic,and g 2 L 2+ forsome > 0 ,thenforanyinitialdistribution of 0 p n g n )]TJ/F39 11.9552 Tf 11.956 0 Td [(E g d N & 2 g where & 2 g = Var g 0 0 +2 1 X n =1 Cov g 0 0 g 0 n and f 0 n g 1 n =0 isthestationaryversionof ;thatis, 0 0 AsopposedtotheCLTusedintheclassicalMCmethodwhichwasdescribedin Chapter1,thereisnoobviousestimatorfortheasymptoticvarianceinTheorem2.2. Findingaconsistentestimatorfor & 2 g hasbeenachallengingtaskduetothedependency amongthevariablesinthechain.However,therehasbeensomerecentprogresson thisproblem.Indeed,Jonesetal.2006andBednorzandLatuszy nski2007provide 29

PAGE 30

sufcientconditionsforthestrongconsistencyofBatchMeansBMestimatorsof & 2 g Moreover,therecentarticleFlegalandJones2010providessufcientconditionsfor thestrongconsistencyoftheOverlappingBatchMeansOBMandSpectralVariance SVestimators.Alloftheseconsistencyresultsrequire tobegeometricallyergodic andtosatisfyaone-stepminorizationcondition.Also,theconsistencyresultsforBM andOBMestimatorsrequire g 2 L 2+ whereasthoseforSVestimatorsrequirethe strongercondition, g 2 L 4+ .SeeFlegalandJones2010forexplicitformulasof theBM,OBM,andSVestimatorsaswellasasimulationstudyoftheirnitesample properties. Consistentestimationoftheasymptoticvarianceallowsustoconstructasymptotically validcondenceintervalsfor E g .Indeed,supposewehaveaconsistentestimatorof & 2 g callit ^ & 2 g n .ThenavalidMCMCstandarderrorisgivenby q ^ & 2 g n n andanasymptotically validCIfor E g isgivenby g n q ? r ^ & 2 g n n where q ? isanappropriatequantile.Onceweareabletocomputeasymptoticallyvalid CIs,wemustdecidewhentostopthesimulation.Asimplestrategyforthisisthe xedtime ruleinwhichthesimulationisrunforapre-determinednumberofiterations.When usingthisstrategy,itispossiblethattheresultingCIisdeemedtoowideinwhichcase thesimulationisallowedtocontinue.Anotherstrategyisthe xed-width ruleinwhich thesimulationisallowedtocontinueuntiladesiredCIwidthisachievedofcourse,here thetotalsimulationeffortisrandom.Thexed-widthmethodsrequireterminatingthe simulationthersttime q ? r ^ & 2 g n n + p n where p n isafunctiononthepositiveintegerssuchthat p n = o n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 2 and isthedesiredhalf-widthoftheCIseeJonesetal.2006.Theseproceduresare asymptoticallyvalidinthesensethatifourgoalistoobtaina 100 )]TJ/F25 11.9552 Tf 12.103 0 Td [( % CIwithwidth 30

PAGE 31

2 then Pr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(E g 2 Int [ T ] 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( as 0, where T isthersttimethat2issatisedandInt [ T ] istheintervalatthis time.Thefunction p isusedtoensurethatthesimulationisnotterminatedprematurely duetoapoorestimateof & 2 g .Examplesofsuchfunctionare p n = I n n 0 and p n = I n n 0 + n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 where I istheusualindicatorfunctionand n 0 isapre-specied samplesize. Thereisanothermethodthatyieldsasymptoticallyvalidcondenceintervalsfor E g ,calledtheRegenerativeSimulationRSmethodHobertetal.,2002;Mykland etal.,1995.SeealsoJonesetal.2006foraRSxed-widthprocedure.This methodisbasedonadifferentCLTandcanbeappliedunderthesameconditions astheBMandOBMestimators.Thatis,itrequirestheconstructionofaminorization condition,thechaintobegeometricallyergodic,and g 2 L 2+ .Onemajordifference betweentheRSmethodandalltheothermethodspreviouslymentionedisthatthe requiredminorizationconditionaffects only theperformanceoftheRSestimator oftheasymptoticvariance.Inotherwords,theRSestimatormightsufferfroma poorminorizationconditionwhereastheotherestimatorsdonot.However,inour experiencewithBayesianGLMMs,theRSmethodhasworkedwellanditsperformance iscomparabletothatoftheothermethods. 31

PAGE 32

CHAPTER3 CONVERGENCEANALYSISOFTHETVGS:PROPERPRIORS 3.1TheTVGSMarkovChain Inthischapter,weanalyzetheTVGSfortheBayesianGLMMwithproperpriorsthat weredescribedinChapter1.Specically,weprovideaproofofProposition1.1,aresult thatprovidesconditionsunderwhichtheTVGSMarkovchainisgeometricallyergodic. Inaddition,weuserealdataconcerninghealthmaintenanceorganizationstoillustrate anhonestMCMCprocedurethatyieldsestimatesandasymptoticallyvalidCIsforthe posteriorexpectationsofthecanonicalparametersinthemodel. RecallfromChapter1theproperpriordensity p ; 0 ~ a ~ b = f N ; 0 f G e ; a e b e r Y i =1 f G u i ; a i b i # andthecorrespondingposteriordensity j y = j y m y where = T u T T isanunnormalizeddensitygivenby j y = f N y ; W )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e I f N )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(u ;0, )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u p ; 0 ~ a ~ b and m y isthemarginaldensityofthedata.RecallalsotheTVGSMarkovchain = f n n g 1 n =0 whichhasstatespace R r +1 + R p + q .TodescribeitsMtd,let 1 j 2 j denotetheposteriorconditionaldensityof given and 2 j 1 j denotetheposterior conditionaldensityof given .Thedependencyonthedataissuppressedinthe notation.TheMtdof isgivenby k j 0 0 = 2 j 1 j 1 j 2 j 0 Thatis,ifthecurrentstateis n n = 0 0 ,thenthedensityofthenextstate, n +1 n +1 ,is3. 32

PAGE 33

Let kk denotetheFrobeniusmatrixnormandrecallthat W = XZ .Routine manipulationoftheposteriordensity, ,showsthat 1 j 2 j = f G e ; a e + N 2 b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 2 r Y i =1 f G u i ; a i + q i 2 b i + k u i k 2 2 # Thatis,conditionalon and y ,theprecisionparametersareindependent,each followingagammadistribution. Anotherstraightforwardmanipulationof showsthat 2 j 1 j isamultivariate normaldensity.Todenethemeanandcovariancematrix,weneedtointroducea bitmorenotation.Dene T = e X T X + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 M = I )]TJ/F25 11.9552 Tf 12.993 0 Td [( e XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ,and Q = e Z T M Z + u .Themeanofthemultivariatenormalis E j = 2 6 4 T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e X T y + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 2 e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T M y )]TJ/F39 11.9552 Tf 11.955 0 Td [(XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T M y )]TJ/F39 11.9552 Tf 11.955 0 Td [(XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 3 7 5 andthecovariancematrixis Var j = 2 6 4 T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + 2 e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 9.298 0 Td [( e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 9.298 0 Td [( e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 3 7 5 RecalltheresultbyJohnsonandJones2010andnotethesubstantialsimplication thatoccurswhen X T Z =0 ThetwomarginalsequencesoftheTVGS, f n g 1 n =0 and f n g 1 n =0 ,arethemselves Markovchainswithstatespaces R r +1 + and R p + q ,respectivelyLiuetal.,1994.Their Mtdsaregivenby k 1 j 0 = Z R p + q 1 j 2 j 2 j 1 j 0 d and k 2 j 0 = Z R r +1 + 2 j 1 j 1 j 2 j 0 d respectively.ItiseasytoseethattheMtds k j k 1 j ,and k 2 j arealljointly measurableandstrictlypositiveintheirrespectivedomains.Moreover,theposterior 33

PAGE 34

density ,andthemarginaldensitiesof and areinvariantfor f n n g 1 n =0 f n g 1 n =0 and f n g 1 n =0 ,respectively.Ofcourse,themarginalposteriordensitiesof and are givenby R R p + q j y d and R R r +1 + j y d ,respectively.Hence,anapplicationof Lemma2showsthatallthreeMarkovchainsareHarrisergodic. Afactthatwewillexploitisthatthesethreechainsallconvergeatthesamerate Diaconisetal.,2008;RobertsandRosenthal,2001.Consequently,wecanprovethat theTVGSchainisgeometricallyergodicbyprovingthateitherofthemarginalchains convergesatageometricrate.Inthenextsection,weproveProposition1.1indirectlyby showingthatthe -chainisgeometricallyergodic. 3.2GeometricErgodicityoftheTVGSMarkovChain Wewillestablishthegeometricergodicityofthe -chainundertheconditionsin Proposition1.1byverifyingthatthethreeconditionsinLemma4hold.First,recallthat wehavealreadyshownthatthe -chainisHarrisergodicand,therefore, -irreducible. Itiseasytoseethat 1 j 2 j isacontinuousfunctionon R p + q foreachxed 2 R r +1 + .ThistogetherwithanapplicationofFatou'sLemmashowsthat k 2 j islower semi-continuouson R p + q foranyxed 2 R p + q .Hence,byLemma3,wehavethatthe -chainisFeller.Moreover,Lemma1impliesthatthemaximalirreducibilitymeasureof the -chainisequivalenttotheLebesguemeasureon R p + q ameasurewhosesupport hasanon-emptyinterior.Therefore,itremainstoshowthatadriftconditionholdsfor the -chain. Ourcandidatedriftfunctionisgivenby v = k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(W k 2 + k u k 2 where > 0 isaconstanttobedetermined.Wewillshowthatif X hasfullcolumnrank then v is unboundedoffcompactsets ;thatis,forevery d 2 R ,theset S d = n 2 R p + q : k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 + k u k 2 d o 34

PAGE 35

iscompact.Indeed,if d issuchthat S d = ; ,then S d isclearlycompact.Soassume that S d isnon-empty.Let u ij denotethe j thcomponentof u i ,where i =1,..., r and j =1,..., q i .Since v iscontinuous, S d isclosed,soitsufcestoshowthat i and u ij areboundedforall i =1,..., r andall j =1,..., q i .Since k u k 2 = P r i =1 P q i j =1 u 2 ij !1 as j u ij j!1 ,wehaveallthe u ij contained.Foragiven u ,let y u := y )]TJ/F39 11.9552 Tf 13.205 0 Td [(Zu and ^ u := X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T y u .Notethatsinceallthe j u ij j arecontained,theelementsof y u and ^ u arealsobounded.Nowsince X hasfullcolumnrankwemusthavethat X T X ispositivedeniteandthus min X T X ,thesmallesteigenvalueof X T X ,ispositive. Finally,thefactthat k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 d impliesthat min X T X )]TJ/F22 11.9552 Tf 13.428 2.657 Td [(^ u T )]TJ/F22 11.9552 Tf 13.428 2.657 Td [(^ u k y u )]TJ/F39 11.9552 Tf 11.955 0 Td [(X ^ u k 2 + )]TJ/F22 11.9552 Tf 13.428 2.657 Td [(^ u T X T X )]TJ/F22 11.9552 Tf 13.428 2.657 Td [(^ u k y u )]TJ/F39 11.9552 Tf 11.955 0 Td [(X k 2 d Thus,forany 2 S d ,wehaveallthe j u ij j containedand j i )]TJ/F22 11.9552 Tf 11.956 0 Td [( ^ u i j d min X T X Thisimpliesthat S d isbounded.Hence, v isunboundedoffcompactsets. Aswehavealreadyexplained,thegeometricergodicityoftheTVGSwillfollowonce thefollowingresultisestablished. Proposition3.1. UnderthethreeconditionsofProposition1.1,thereexista 2 [0,1 andaniteconstant L suchthat,forevery 0 2 R p + q E k 2 )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(v j 0 v 0 + L BeforeembarkingonourproofofProposition3.1,wewillbrieydescribetheimportant stepsofourproof. First,notethattheleft-handsideof3satises E k 2 v j 0 = E E v j 0 = E E k y )]TJ/F39 11.9552 Tf 11.234 0 Td [(W k 2 j j 0 + E E )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k u k 2 j 0 35

PAGE 36

Now,supposeweareabletoobtainfunctionalupperboundson E v j oftheform h ~ c = c e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + c u r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i + const where ~ c = c e c u T isavectorofconstantsthatmaydependon ,and const denotes agenericconstant.UnderthethreeconditionsofProposition1.1,wehave min n a e + N 2 a 1 + q 1 2 ,..., a r + a r 2 o > 1, whichimpliesthat E )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e 0 and E )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i 0 i 2f 1,2,..., r g areallnite.Moreover,itis easytoshowthat E )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e 0 = 2 b e + k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(W 0 k 2 2 a e + N )]TJ/F22 11.9552 Tf 11.956 0 Td [(2 and,foreach i 2f 1,2,..., r g E )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i 0 = 2 b i + k u 0 i k 2 2 a i + q i )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 Withthis,wecanobtain E )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(h ~ c j 0 C e k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W 0 k 2 + C u k u 0 k 2 + const where C e = c e 2 a e + N )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 and C u = c u 2min a 1 + q 1 2 ,..., a r + q r 2 )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 Combiningwhatwehavesofar,weobtain E k 2 v j 0 = E E v j 0 E )]TJ/F39 11.9552 Tf 5.48 -9.683 Td [(h ~ c j 0 v 0 + L forsomeniteconstant L ,where =max f C e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 C u g .Hence,establishingthedrift condition3comesdowntoasearchforfunctionalupperboundson E v j [asin 3]thatleadto < 1 forsome > 0 36

PAGE 37

Recallfrom3that,inordertoobtainfunctionalupperboundson E v j of theform h ~ c ,wemustcarefullyanalyzetheterms E )]TJ/F21 11.9552 Tf 5.479 -9.684 Td [(k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(W k 2 = tr W Var j W T + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(WE j k 2 and E )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k u k 2 j = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + k E u j k 2 whicharecomplicatedfunctionsof .Ourstrategywillconsistoftwomainsteps:ito ndfunctionalupperbounds[oftheform h ~ c ]onthetracetermsin3and3, andiitoshowthat,asfunctionsof k y )]TJ/F39 11.9552 Tf 11.982 0 Td [(WE j k 2 and k E u j k 2 [whichappearin 3and3,respectively]arebounded.ThisapproachwasalsotakenbyJohnson andJones2010,whoworkedundertheassumption X T Z =0 andconsideredasimilar driftfunction, v JJ = k y )]TJ/F39 11.9552 Tf 11.993 0 Td [(W k 2 + k u k 2 .However,itshouldbepointedoutthatJohnson andJones's2010proofofgeometricergodicityinvolvesapplicationsoftheirLemma A.2.whichis,unfortunately,false.Itisnothardtoprovideacounterexampleforit. Hence,theirargumentforgeometricergodicityisincorrect.Ourtechniquesforobtaining upperboundson3and3areverydifferentfromtheirs.Inparticular,weexploit singularvalueandspectraldecompositionsofkeymatrices,andthisallowsusto establishaseriesofmatrixinequalitiesthatleadtoveryweakconditionsforgeometric ergodicity. Wenowgiveaseriesofpreliminaryresultsthatwillbeusedinourproofof Proposition3.1.Recallthatif C isanon-negativedenitematrixthentr C 0 Also,if A and B aresymmetricmatricesofthesamedimensionsuchthat B )]TJ/F39 11.9552 Tf 12.698 0 Td [(A is non-negativedeniteinsymbols, A B ,thentr A tr B .Furthermore,if A and B arepositivedenitematrices,then A B ifandonlyif B )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 A )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Dene ~ X = X 1 = 2 .Wewillusethefollowingdecompositionseveraltimesinwhat follows.Since ~ X hasrank p ,itcanbewrittenas ~ X = USV T ,where U and V are orthogonalmatricesofdimension N and p ,respectively,and S isthe N p matrixgiven 37

PAGE 38

by S = 0 B @ S 0 1 C A where S isa p p diagonalmatrixwhosediagonalelements, f s i g p i =1 ,arethestrictly positivesingularvaluesof ~ X .Ofcourse,thematrixof 0 sunderneath S hasdimension N )]TJ/F39 11.9552 Tf 11.955 0 Td [(p p .Hereisourrstresult. Lemma5. Ifrank X = p ,then XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T = UR U T where R isan N N diagonal matrixwhosediagonalelements, f r i g N i =1 ,aregivenby r i = 8 > < > : s 2 i e s 2 i +1 i 2f 1,2,..., p g 0 i 2f p +1, p +2,..., N g Proof. Wehave XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T = X 1 = 2 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 2 T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = 2 1 = 2 X T = ~ X )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( e ~ X T ~ X + I )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ X T = USV T e VS T SV T + VV T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 VS T U T = US e S T S + I )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 S T U T = US e S 2 + I )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 S T U T = UR U T Recallthat M = I )]TJ/F25 11.9552 Tf 12.13 0 Td [( e XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T .Thefollowingresultprovidesadecompositionfor M Lemma6. Ifrank X = p ,then M = UH U T where H isan N N diagonalmatrix whosediagonalelements, f h i g N i =1 ,aredenedas h i = 8 > < > : 1 e s 2 i +1 i 2f 1,2,..., p g 1 i 2f p +1, p +2,..., N g 38

PAGE 39

Furthermore, e s 2 max +1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 I M I ,where s max denotesthelargestsingularvalueof ~ X Proof. ThedecompositionfollowsimmediatelyfromLemma5since M = I )]TJ/F25 11.9552 Tf 11.955 0 Td [( e XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T = UU T )]TJ/F25 11.9552 Tf 11.955 0 Td [( e UR U T = U )]TJ/F39 11.9552 Tf 5.479 -9.683 Td [(I )]TJ/F25 11.9552 Tf 11.956 0 Td [( e R U T Nowforeach i =1,2,..., N 0 < e s 2 max +1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 h i 1 ,anditfollowsthat e s 2 max +1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 I = U e s 2 max +1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 U T UH U T UU T = I Let A + denotetheMoore-Penroseinverseofthematrix A .Thefollowingresultwill allowustoprovideveryweakconditionsforgeometricergodicityandwillbeusedto establishnitemomentconditionsinAppendixD.1.Lemma13whichisstatedand proveninAppendixAwillbeusedintheproof. Lemma7. Ifrank X = p ,thenforall 2 R r +1 + 1.tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( Z T Z + + )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(q )]TJ/F47 11.9552 Tf 11.956 0 Td [(rank Z )]TJ 12.952 -0.717 Td [(P r i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i + s 2 max tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( Z T Z + 2.tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e rank Z + s 2 max rank Z 3.tr XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T )]TJ/F47 11.9552 Tf 11.955 0 Td [(tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T I + M tr X X T where s max denotesthelargestsingularvalueof X 1 = 2 Proof. Statements1.and2.followimmediatelyfromLemma13since min 1 i r u i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i Now,byLemma6,wehave 0 U I )]TJ/F39 11.9552 Tf 11.955 0 Td [(H 2 U T = I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M 2 = I + M I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M 39

PAGE 40

Since A := I + M I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M and Q bothhavesquareroots,wehave tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T I + M = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(A 1 = 2 A 1 = 2 ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 2 Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 2 Z T = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = 2 Z T A 1 = 2 A 1 = 2 ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = 2 0. Finally,itiseasytoseethat T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 whichimpliesthattr XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T tr X X T Hence3.isestablished. Hereisourlastpreliminaryresult. Lemma8. Ifrank X = p ,thenthefunctions k E j k k E u j k ,and k y )]TJ/F39 11.9552 Tf 12.052 0 Td [(WE j k areallbounded. ThefollowingresultfromKhareandHobert2011willbeusedinourproof. Lemma9. Fix n 2f 2,3,... g and m 2 N ,andlet t 1 ,..., t n bevectorsin R m .Then C m n t 1 ; t 2 ,..., t n :=sup c 2 R n + t T 1 t 1 t T 1 + n X i =2 c i t i t T i + c 1 I )]TJ/F23 7.9701 Tf 6.587 0 Td [(2 t 1 isnite. ProofofLemma8. Recall E j fromSection3.1.Bythetriangleinequality,wehave k E j kk T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e X T y + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k + k 2 e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T M y k + k 2 e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k Now,usingpropertiesoftheFrobeniusmatrixnorm,wehave k T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e X T y + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 kk e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 kk X T y k + k T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 kk )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 k k 2 e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T M y kk e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 kk X T Z kk e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T M y k and k 2 e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ZQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 kk e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 kk X T Z kk e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k Similarly, k E u j kk e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T M y k + k e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k 40

PAGE 41

Also, k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(WE j kk y k + k W k p k E j k 2 + k E u j k 2 Thus,itsufcestoshowthatthefollowingfunctionsareallbounded: k T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k k e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k k XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 k k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T M y k ,and k e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k Since T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ,wehave k T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k 2 = tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 2 )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(tr T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 2 )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(tr 2 Usingthefactthattr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(A 2 tr 2 A forpositivedenite A see,e.g.,FuzhenandZhang, 1999,Theorem6.5.3andthefactthat 0 e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ,wehave k e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k 2 = tr e T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 2 )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(tr e T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 2 )]TJ/F20 11.9552 Tf 5.48 -9.684 Td [(tr X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 2 Now, k XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 kk X kk T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 kk )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 kk X k tr k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k Dene ~ Z = U T Z and ~ y = U T y .Let ~ z i denotethe i thcolumnof ~ Z T ,andlet ~ y i denotethe i thcomponentof ~ y .UsingLemma6,wehave k e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T M y k = k )]TJ/F39 11.9552 Tf 5.479 -9.683 Td [(Z T M Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T M y k = k )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Z T UH U T Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T UH U T y k = N X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ z i h i ~ y i N X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i h i ~ y i = N X i =1 N X j =1 ~ z j ~ z T j h j + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ z i h i ~ y i = N X i =1 j ~ y i j ~ z i ~ z T i + X j 6 = i ~ z j ~ z T j h j h i + h )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i 41

PAGE 42

Thus,itsufcestoshowthatforeach i 2f 1,2,..., N g C i := ~ z i ~ z T i + X j 6 = i ~ z j ~ z T j h j h i + h )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i isbounded.Nowdene = P r i =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i andnotethat C 2 i =~ z T i ~ z i ~ z T i + X j 6 = i ~ z j ~ z T j h j h i + h )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(2 ~ z i =~ z T i ~ z i ~ z T i + X j 6 = i ~ z j ~ z T j h j h i + h )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F25 11.9552 Tf 11.955 0 Td [( )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 I + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 h i e I )]TJ/F23 7.9701 Tf 6.587 0 Td [(2 ~ z i sup c 2 R N + k + t T i t i t T i + X j 6 = i c j t j t T j + N + q X j = N +1 c j t j t T j + c i I )]TJ/F23 7.9701 Tf 6.587 0 Td [(2 t i where,for j =1,2,..., N t j =~ z j ,andfor j 2f N +1,..., N + q g ,the t j arethestandard orthonormalbasisvectorsin R q ;thatis, t N + l hasaoneinthe l thpositionandzeros everywhereelse.AnapplicationofLemma9showsthat C 2 i isbounded.Hence, k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T M y k isbounded. Dene y = U T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 ,andlet y i denotethe i thcomponentof y .Then usingLemma5wehave k e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 0 k = k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k = k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T UR U T X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 0 k = k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ Z T R y k 42

PAGE 43

RecallLemmas5and6.When i 2f 1,2,..., p g h i = r i = s )]TJ/F23 7.9701 Tf 6.586 0 Td [(2 i ,andwhen i 2 f p +1, p +2,..., N g r i =0 .Hence, k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k = k e Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ Z T R y k = k )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ Z T R y k = N X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i r i y i = p X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i r i y i p X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T H ~ Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ z i r i y i = p X i =1 N X j =1 ~ z j ~ z T j h j + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i r i y i = p X i =1 j y i j ~ z i ~ z T i h i r i + X j 6 = i ~ z j ~ z T j h j r i + r )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i = p X i =1 j y i j s i ~ z i ~ z T i s )]TJ/F23 7.9701 Tf 6.586 0 Td [(2 i + X j 6 = i ~ z j ~ z T j h j r i + r )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(s )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i ~ z i AnotherapplicationofLemma9showsthat,foreach i 2f 1,2,..., p g ~ z i ~ z T i s )]TJ/F23 7.9701 Tf 6.586 0 Td [(2 i + X j 6 = i ~ z j ~ z T j h j r i + r )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(s )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i ~ z i isbounded.So k e Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k isbounded. WearenowreadytoproveProposition3.1. ProofofProposition3.1. RecallthatthematrixVar j involvesthematrices T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = e X T X + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 M = I )]TJ/F25 11.9552 Tf 9.825 0 Td [( e XT )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ,and Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = e Z T M Z + u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 .Astraightforward manipulationshowsthat tr W Var j W T = tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T + tr XT )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T )]TJ/F20 11.9552 Tf 11.955 0 Td [(tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( I )]TJ/F39 11.9552 Tf 11.955 0 Td [(M ZQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T I + M 43

PAGE 44

ItfollowsimmediatelyfromLemma7that tr W Var j W T rank Z )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + const and tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + + )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(q )]TJ/F20 11.9552 Tf 11.955 0 Td [(rank Z r X i =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i + const Combining3-3withLemma8,wehave E v j rank Z + tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(q )]TJ/F20 11.9552 Tf 11.955 0 Td [(rank Z r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i + const Hence,ourupperboundon E v j hastheform h ~ c with c e = rank Z + tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + and c u = q )]TJ/F20 11.9552 Tf 11.955 0 Td [(rank Z Now,recallfrom3,thattheresulting takestheform =max f C e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 C u g UndertheconditionsofProposition1.1, C u < 1 andrank Z < 2 a e + N )]TJ/F22 11.9552 Tf 11.262 0 Td [(2 .Thus,forany > 0 := tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + 2 a e + N )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 )]TJ/F20 11.9552 Tf 11.955 0 Td [(rank Z > 0, wehave C e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = rank Z 2 a e + N )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( Z T Z + 2 a e + N )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 < 1. Thereforethereexists > 0 with =max f C e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 C u g < 1 ,andthiscompletesthe proof. 3.3HobertandGeyer'sResult AnimportantspecialcaseofourGLMMistheone-wayrandomeffectsmodelgiven by Y ij = 0 + u i + ij where i =1,..., q j =1,..., m i ,the u i sareiidN )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u andthe ij s,whichare independentofthe u i s,areiidN )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e .Thisisthenon-centeredparameterization 44

PAGE 45

NCPoftheone-wayrandomeffectsmodel.InthecenteredparameterizationCP,the parameter 0 doesnotappearinthemodelequation,butratherasthemeanofthe u i s. Considerthetwointractableposteriordensitiesthatresultwhenthecenteredand non-centeredversionsoftheone-wayrandomeffectsmodelarecombinedwiththe samesetofproperconjugatepriorsfor 0 e and u .TherearetwodifferentTVGSs associatedwiththesetwoposteriordensities.HobertandGeyer1998analyzedthe TVGSMarkovchainassociatedwiththeCPmodel,callit CP ,andshowedthatitis geometricallyergodicaslongas m i 2 forall i =1,2,..., q .Ontheotherhand,our Proposition1.1showsthattheTVGSMarkovchainfortheNCPmodel,callit NCP ,is alsogeometricallyergodicunderthesameconditiononthe m i s. Anaturalquestiontoaskis:do CP and NCP convergeat exactly thesamerate?It iswellknownthatminorreparameterizationssuchasNCPvsCPcanhaveimportant consequencesinthecontextofMCMCsee,e.g.Gelfandetal.,1995;Papaspiliopoulos etal.,2007;YuandMeng,2011.Therefore,itisnaturaltoexpectthat CP and NCP convergeatadifferentrate.However,aswenowexplain,thisisnotthecase.Wewill provethat CP and NCP convergeexactlyatthesameratebyexploitingthefactthat thethreeunderlyingMarkovchainsofaTVGSallconvergeatthesamerateandthe factthattheposteriordensitiesintheCPandNCPcasesarerelatedbyadifferentiable one-to-onetransformationthatdoes not involvetheprecisionparameters.Indeed,the transformationtakestheform # = g )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 := 0 u 1 + 0 ,..., u q + 0 T where = 0 u 1 ,..., u q T .Moreover,theJacobianofthetransformation,whichwe denoteby J # ,canbeshowntobeequaltoone.Thefactthatweareworkingwith TVGSsforone-waymodelsand J # =1 isnotimportant.Togiveahintofhowour resultscanbegeneralizedtomoregeneralTVGSsandanydifferentiableone-to-one transformation,wewillcontinuetokeeptrackof J # 45

PAGE 46

Wewilluse fortheposteriordensitycorrespondingtotheNCPone-way model.TheposteriordensityoftheCPone-waymodelcanbewrittenas $ # := g # j J # j Also,theMtdof CP = f ~ n # n g 1 n =0 isgivenby k # j 0 # 0 = $ 2 j 1 # j $ 1 j 2 j # 0 Hereisourresult. Lemma10. TheMarkovchains CP and NCP haveexactlythesameconvergencerate. Proof. Recallthat NCP = f n n g 1 n =0 and CP = f ~ n # n .Itsufcestoshowthatthe marginalchains f n g 1 n =0 and f ~ n g 1 n =0 convergeexactlyatthesamerate.Let k 1 denote theMtdof f ~ n g 1 n =0 whichisgivenby k 1 j 0 = Z R p + q $ 1 j 2 j # $ 2 j 1 # j 0 d # Wewillshowthat k 1 j 0 = k 1 j 0 forall 0 2 R r +1 + R r +1 + .Let 1 and $ 1 denotethe -marginalsof and $ ,respectively.Itiseasytoseethatthesemarginals areactuallythesame.Indeed, 1 = Z R p + q d = Z R p + q g # j J # j d # = Z R p + q $ # d # = $ 1 forall 2 R r +1 + .Now,applyingthetransformation # = g )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ,wecanwrite k 1 j 0 = Z R p + q 1 j 2 j 2 j 1 j 0 d = Z R p + q 1 j 2 j g # 2 j 1 g # j 0 j J # j d # Thus,itsufcestoshowthat $ 1 j 2 j # = 1 j 2 j g # and $ 2 j 1 # j = 2 j 1 g # j j J # j 46

PAGE 47

forall 2 R r +1 + andalmostall # 2 R p + q .Notethatforall 2 R r +1 + andfor # 2 R p + q ,we have $ 2 j 1 # j = $ # $ 1 = g # j j J # j 1 = 2 j 1 g # j j J # j Finally,letting 2 and $ 2 denotethe -marginals,wehavethatforall # 2 R p + q $ 1 j 2 j # = $ # $ 2 # = g # j J # j 2 g # j J # j = 1 j 2 j g # Hence, k 1 j 0 = k 1 j 0 forall 0 2 R r +1 + R r +1 + .Thiscompletestheproof. Remark5. NotethattheargumentintheproofofLemma10usesfactsparticular toTVGSsandnotforGibbssamplerswithmorethantwocomponents.Also,itis importanttonotethatweappliedatransformationtoonlyoneofthecomponentsofthe TVGS. Remark6. AnargumentalmostidenticaltotheoneusedintheproofofLemma10 showsthatanydifferentiableone-to-onetransformationthatisappliedtoonlyoneof thecomponentsoftheTVGSwillnotaffecttheconvergencerateoftheresultingTVGS. Hence,itcanalsobesaidthattheprecisionvsvariancereparametrizationdoesnot affecttheconvergencerateoftheTVGSs. Remark7. Thereisnothingspecialaboutthetargetdensitiesconsideredherethe posteriors and $ .Thusaverygeneralresultthatholdseveninnon-Bayesian situationscanalsobeestablished. SinceasimilarresulttoLemma10canbegivenintheimproperpriorcase,wewill beabletousetheresultsdevelopedhereintoimproveTanandHobert's2009result forthecenteredone-wayrandomeffectsmodelwithimproperpriors.SeeSection4.3for details. 3.4HonestEstimationofthePosteriorExpectations Inthissection,weillustratehowtoprovideasymptoticallyvalidcondenceintervals fortheintractableposteriorexpectationsofthecanonicalparametersoftheGLMM.To dothis,wewilluseUSgovernmentdataconcerninghealthmaintenanceorganizations 47

PAGE 48

HMOsHodges,1998.SeealsoJohnsonandJones2010.Thedatacontain informationon341HMOsin45differentjurisdictionsstates,DistrictofColumbia, GuamandPuertoRico,andweregatheredtostudythecostoftransferringmilitary retireesandtheirdependentsfromaDefenseDepartmenthealthplantohealth plansforgovernmentemployees.Let y ij denotetheindividualmonthlypremiumof the j thHMOplaninthe i thjurisdiction,where i =1,2,...,45 and j =1,2,..., n i .Of course, P 45 i =1 n i =341 .Also,let x i 1 bethecenteredandscaledaverageexpenses peradmissioninthe i thjurisdiction,andlet x i 2 beanindicatorfortheregionofNew England.Hodges1998modeledthesedatausingaBayesianversionofthefollowing linearmixedmodel Y ij = 0 + 1 x i 1 + 2 x i 2 + u i + ij wherethe ij sareiidN 2 e andthe u i sareiidN 2 u .Notethatmodel3isa specialcaseoftheGLMMfromChapter1inwhich N =341 r =1 q =45 X is 341 3 = 0 1 2 T Z is 341 45 withtheusualcellmeansstructure,and u is 45 1 WeconsideraBayesianhierarchicalversionofmodel3withthreestages.The rststageis Y j u e u N 341 )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(X + Zu )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e I and,atthesecondstage,wehave N 3 ,1000 I u j u N 45 )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [(0, )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u I and e Gamma a e b e and,nally,thethirdstageis u Gamma a 1 b 1 Toselectthehyper-parametersofthepriordistributionof e and u ,weconducted apreliminaryfrequentistanalysis.WeusedtheMIXEDprocedureinSAStot model3,andTable3-1containsasummaryoftheresults.TheMIXEDprocedure withoptionCOVTESTproducestheREMLestimatesofthevariancecomponents, 48

PAGE 49

togetherwiththecorrespondingasymptoticstandarderrorssee,e.g.,Littelletal., 1996.Thehyperparameters a e b e werechosenbymatchingthepriormeanand varianceof 2 e tothecorrespondingfrequentistestimatesinTable3-1.Specically,we set E 2 e = E )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e = b e a e )]TJ/F22 11.9552 Tf 11.956 0 Td [(1 =494.74 and Var 2 e = Var )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e = b 2 e a e )]TJ/F22 11.9552 Tf 11.955 0 Td [(1 2 a e )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 =41.01 2 Thisyields a e =147.57 and b e =72,515.1 .Thesamemomentmatchingprocedure wasusedfortheprioron u ,andtheresultwas a 1 =6.546 and b 1 =547.87 .Note thatthismodelisaspecialcaseoftheBayesianGLMMwithproperpriorsdescribedin Chapter1.WenotethatHodges1998usedanimproperpriorinhisanalysis. RecallfromSection2.2thatwecancomputeasymptotically valid condence intervalsfortheposteriorexpectationsusingthemethodsdescribedthereinprovided thattheTVGSMarkovchainisgeometricallyergodic,amomentconditionholdsforthe functionofinterest,andaminorizationconditionholdsforthechain.Recallalsothatthe methodsBM,OBM,andRSrequirea 2+ momentconditionwhiletheSVmethods requirethestronger 4+ momentcondition.Now,notethatthethreeconditionsof Proposition1.1aresatised,sotheTVGSMarkovchainforthisproblemisgeometrically ergodic.AnapplicationofourPropositionD.1foundinAppendixD.1impliesthat, E j g i j 8 < 1 for i =1,2,3,4 and E j g 5 j 6.5 < 1 ,where g 1 = 0 g 2 = 1 g 3 = 2 g 4 = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e and g 5 = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u Keepinmindthat 2 e = )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e and 2 u = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u .Morever,resultsinHobertetal.2006imply thataone-stepminorizationconditionholdsforourTVGS.Hence,anyofthemethods discussedinSection2.2canbeusedinthisexample. 49

PAGE 50

HereweusetheTukey-HanningTHestimatorwhichisanSVestimatorof & 2 g andis givenby ^ & 2 g n = t n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X s = )]TJ/F23 7.9701 Tf 6.586 0 Td [( t n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 w n s ^ s s where t n iscalledthetruncationpoint,the lagwindow w n s ,isgivenby w n s = 1 2 + 1 2 cos j s j t n I )]TJ/F40 7.9701 Tf 6.586 0 Td [(t n t n s and ^ n s =^ n )]TJ/F39 11.9552 Tf 9.298 0 Td [(s := 1 n n )]TJ/F40 7.9701 Tf 6.586 0 Td [(s X t =1 )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(g t )]TJ/F22 11.9552 Tf 12.198 0 Td [( g n )]TJ/F39 11.9552 Tf 10.959 -9.684 Td [(g t + s )]TJ/F22 11.9552 Tf 12.198 0 Td [( g n Notethatwemustspecifyatruncationpointsoconsiderusinga t n =[ n ] forsome 0 << 1 .AnapplicationofFlegalandJones's2010Theorem1andLemma7shows thatif E j g j 4+ 1 + 2 < 1 forsome 1 2 > 0 ,and 2 4+ 1 << 1 2 thentheTHestimatorwith t n =[ n ] isstronglyconsistentfor & 2 g .Toestimatethe asymptoticvariancescorrespondingtothefunctions f g i g 5 i =1 ,wewilluseTHestimators with t n =[ n 1 = 3 ] whichcaneasilybeshowntobestronglyconsistentsince E j g i j 8 < 1 for i =1,2,3,4 and E j g 5 j 6.5 < 1 .WenotethattheTHestimatorwithtruncation point t n =[ n 1 = 2 ] wasrecommendedbyFlegalandJones2010asadefaultmethod. However,itisnotclearthatthisestimatorisstronglyconsistentfor & 2 g .Althoughone coulduse t n =[ n 0.499 ] whichdoesleadtoaconsistentestimator,wewilladopt t n =[ n 1 = 3 ] becausetheresultingestimatorseemstobemorestableinourexample andislesscomputer-intensive. Supposewewouldliketoconstructapproximate95%BonferoniCIsfor f E g i g 5 i =1 withhalf-widthsormarginoferrors, f i g 5 i =1 ,usingthexed-widthmethodology describedinSection2.2.Supposefurtherthatwewant i tobeabitlessthan0.5% ofthepreliminaryestimateof E g i .Tochoose := 1 2 3 4 5 T ,weranthe 50

PAGE 51

TVGSfor1,000,000iterationsusingtheSASMIXEDprocedureestimatesof and u as startingpoints.Then,usingtheresultingpreliminaryMCMCestimatesoftheposterior meanswhicharegiveninTable3-2,weset =.810.020.162.470.47 T Althoughburn-inisnotrequiredforthexed-widthmethods,wethrewawaythe 1,000,000preliminarydrawsandusedthemasburn-in.Thisseemedtoimprovethe performanceoftheTHestimator.WethenrantheTVGSuntilthersttime 2.58 r ^ & 2 g i n n + i I ,10000 n + 1 n i forall i 2f 1,2,3,4,5 g .Toreducethecomputationaleffort,weonlychecked3 every2000iterationsstartingat10,000.Thesimulationwasstoppedafter n = 116,000 iterations.Table3-3containsasummaryoftheresultsofoursimulation. Onenoteofcautionwhenusingxed-widthmethodsisthatthereisnoguarantee thatthechainwillberunlongenoughsothattheestimators f ^ 2 g i n g 5 i =1 provideagood estimateofthecorrespondingasymptoticvariances.Forexample,Figure3-1showsthat ^ 2 g 5 n startstostabilizeabitafterthe500,000thiteration.Welookedatsimilartraceplots correspondingtotheotherfunctionsofinterestandthesesuggestthatallestimators havestabilizedbythe2,000,000thiteration.SeeTable3-4forasummaryoftheresults basedon n = 2,000,000iterations. Table3-1.ThistablesummarizestheoutputoftheMIXEDprocedureusedtotthe model3fortheHMOdata.Foreachmodelparameter,itcontainsa frequentistestimatealongwithastandarderror. ParameterEstimateStandardError 0 163.682.19 1 4.832.31 2 32.147.85 2 e 494.7441.01 2 u 98.7846.33 51

PAGE 52

Table3-2.Thistablecontainspreliminaryresultsthatwereusedtodeterminethe desiredhalf-widthsrequiredforthexed-widthprocedure.Foreachmodel parameter,thetablecontainsanMCMCestimateofthecorresponding posteriormean, E g Parameter 0 1 2 2 e 2 u Estimate163.684.7932.16495.8994.85 Table3-3.ThistablecontainsasummaryofourMCMCprocedurefortheBayesian GLMMwithproperpriorsusedintheHMOdataexample.Foreachmodel parameter,itcontainsestimatesoftheposteriorexpectationandthe correspondingasymptoticvariance,aswellasthestandarderroranda95% asymptoticBonferronicondenceinterval.Theresultsarebasedon n = 116,000iterations. ParameterEstimate ^ & 2 g n q ^ & 2 g n n 95% BonferroniCI 0 163.6754.5180.006.659,163.691 1 4.8035.2370.007.786,4.821 2 32.13655.2870.022.080,32.193 2 e 495.875952.0900.091.641,496.108 2 u 94.8953834.7640.182.425,95.364 Table3-4.Resultsbasedon n = 2,000,000iterations. ParameterEstimate ^ & 2 g n q ^ & 2 g n n 95% BonferroniCI 0 163.6774.5820.002.673,163.681 1 4.7955.2110.002.790,4.799 2 32.15557.1250.005.141,32.169 2 e 495.852993.8940.022.795,495.910 2 u 94.9263903.7360.044.812,95.040 52

PAGE 53

Figure3-1.Thetopplotshowsthebehavioroftheestimatorof E 2 u andthe correspondingapproximate95%BonferroniCIasthenumberofiterations increases.Thesolidlinerepresentstheestimatorandthedashedlines denotetheupperandlowerendpointsoftheCI.Themiddleplotshowsthe behavioroftheTHestimatoroftheasymptoticvariance, ^ & 2 g 5 n .Thebottom plotshowsthetrajectoryofthemarginoferror 2.58 q ^ & 2 g 5 n n 53

PAGE 54

CHAPTER4 CONVERGENCEANALYSISOFTHETVGS:IMPROPERPRIORS 4.1TheTVGSMarkovChain Inthischapter,weanalyzetheTVGSfortheBayesianGLMMcorrespondingtothe improper priorsthatweredescribedinChapter1.Recalltheimproperprior ~ p ; ~ a ~ b = a e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e e b e e r Y i =1 a i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i e )]TJ/F40 7.9701 Tf 6.586 0 Td [(b i u i # andtheunnormalizedposteriordensity ~ j y = f N y ; W )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e I f N )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(u ;0, )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u ~ p ; ~ a ~ b giveninChapter1.Inwhatfollows,wedonotassumethattheresultingposterior densityisproper;thatis,wewillnotrequirethat ~ m y = Z R r +1 + Z R p + q ~ j y d d < 1 ToformallydenetheTVGS,supposethat Z R p + q ~ j y d < 1 forall outsideasetofmeasurezeroin R r +1 + ,andthat Z R r +1 + ~ j y d < 1 forall outsideasetofmeasurezeroin R p + q .Thesetwointegrabilityconditionsare necessary,butnotsufcient,forposteriorpropriety.Whentheyhold,wecandene conditionaldensitiesasusual.Whentheposteriorisproper,theseconditionalsarethe usualonesbasedon ~ j y ,butwhentheposteriorisimproper,theyareincompatible conditionaldensities;thatis,thereisnoproperjointdensitythatgeneratesthem. Ofcourse,thereisnothingstoppingusfromrunningtheTVGSwhentheconditional 54

PAGE 55

densitiesareincompatible.However,ifthisisdone,thentheTVGSMarkovchainwillbe unstable.Wewillmakethismorepreciselaterinthissection. Wenowdescribeaminimalsetofconditionsunderwhichtheintegrabilityconditions aresatised.Let ^ = W T W )]TJ/F39 11.9552 Tf 7.084 -4.339 Td [(W T y ,andassumethat 2 b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W ^ k 2 > 0. Notethatif N > p + q then k Y )]TJ/F39 11.9552 Tf 10.745 0 Td [(W ^ k 2 > 0 withprobabilityoneunderthedatagenerating model.Also,itfollowsfrom4that,forall 2 R p + q 2 b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 =2 b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W ^ k 2 + k W )]TJ/F39 11.9552 Tf 11.955 0 Td [(W ^ k 2 2 b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W ^ k 2 > 0. Dene ~ s =min q 1 +2 a 1 q 2 +2 a 2 ,... q r +2 a r N +2 a e ,andassumethat ~ s > 0. Finally,assumethat b i 0 8 i 2f 1,2,..., r g andthat rank X = p Under4,4,4,and4,theintegrabilityconditions,4and4, hold,sotheconditionaldensitiesarewelldened.Routinemanipulationof ~ j y showsthat ~ 2 j 1 j isamultivariatenormaldensitywithmeanvector E ~ j = 2 6 4 X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(I )]TJ/F25 11.9552 Tf 11.956 0 Td [( e Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? y e ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T P ? y 3 7 5 andcovariancematrix Var ~ j = 2 6 4 e X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F22 11.9552 Tf 9.299 0 Td [( X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F22 11.9552 Tf 11.286 2.657 Td [(~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 3 7 5 55

PAGE 56

where P ? = I )]TJ/F39 11.9552 Tf 11.955 0 Td [(X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T and ~ Q = e Z T P ? Z + u Duetothepossibleexistenceofaproblematicsetofmeasurezero,therearesome technicalitiesthatarisewhenderivinganexpressionfor ~ 1 j 2 j .Dene A = i 2 f 1,2,..., r g : b i =0 andrecallthat f G x ; a b = b a \050 a x a )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(bx I R + x for a b > 0 .If A isempty,then ~ 1 j 2 j iswelldenedforevery 2 R p + q ,anditisthe followingproductof r +1 gammadensities ~ 1 j 2 j = f G e ; N 2 + a e b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 2 r Y i =1 f G u i ; q i 2 + a i b i + k u i k 2 2 Ontheotherhand,if A isnonempty,then Z R r +1 + ~ j y d = 1 whenever 2N := 2 R p + q : Q i 2 A k u i k =0 .Thefactthat ~ 1 j 2 j isnot denedwhen 2N isirrelevantfromasimulationstandpointbecausetheprobability thatadrawform ~ 2 j 1 j fallsin N iszero.However,inordertocarefullystudythe convergencepropertiesoftheTVGSMarkovchain,itsMtdmustbedenedfor every 2 R p + q .Therefore,wedenetheproblematicconditionaldensityasfollows: ~ 1 j 2 j = 8 > < > : f G e ; N 2 + a e b e + k y )]TJ/F40 7.9701 Tf 6.586 0 Td [(W k 2 2 Q r i =1 f G u i ; q i 2 + a i b i + k u i k 2 2 if = 2N f G e ;1,1 Q r i =1 f G u i ;1,1 if 2N Thisdenitioncanbeusedinallcasesifwesimplydene N tobe ; whenever A is empty. TheMtdoftheTVGSMarkovchain ~ = f n n g 1 n =0 ,isgivenby ~ k j 0 0 =~ 2 j 1 j ~ 1 j 2 j 0 56

PAGE 57

Itisstraightforwardtocheckthat ~ j y isanunnormalizedinvariantdensityfor ~ .Also,since ~ k j 0 0 isstrictlypositiveon R r +1 + R p + q R r +1 + R p + q ,it iseasytoseethat ~ is -irreducible.Thus ~ ispositiverecurrentifandonlyifthe posteriorisproperMeynandTweedie,1993,Chapter10.Therefore ~ cannotbe geometricallyergodicwhentheposteriorisimproperbecauseageometricallyergodic chainisnecessarilypositiverecurrent. RecallfromChapter3thatthetwomarginalsequencesofaTVGS,arethemselves Markovchains.The -chainliveson R r +1 + andhasMtdgivenby ~ k 1 j 0 = Z R p + q ~ 1 j 2 j ~ 2 j 1 j 0 d andinvariantdensity R R p + q ~ j y d .Similarly,the -chainliveson R p + q andhasMtd ~ k 2 j 0 = Z R r +1 + ~ 2 j 1 j ~ 1 j 2 j 0 d andinvariantdensity R R r +1 + ~ j y d .Itiseasytocheckthatthetwomarginalchains arealso -irreducible.Thustheyarepositiverecurrentifandonlyiftheposterior isproper.Moreover,whentheposteriorisproper,onecanapplyLemma2which showsthat ~ anditstwomarginalchainsareHarrisergodic.AsinChapter3,wewill usethefactthat,whenthesethreechainsareallHarrisergodic,eitherallthreeare geometricallyergodicornoneofthemis. 4.2GeometricErgodicityoftheTVGSMarkovChain Beforewestateourconvergenceresult,wedenesomenotation.Writethe spectraldecompositionofthenon-negativedenitematrix Z T P ? Z as O T DO ,so O isa q -dimensionalorthogonalmatrix,and D isadiagonalmatrixcontainingtheeigenvalues of Z T P ? Z ,whichwedenoteby f d i g q i =1 .Dene D ? tobea q q diagonalmatrixwhose diagonalelements, f d ? i g q i =1 ,aregivenby d ? i = 8 > < > : 1 d i =0 0 d i 6 =0. 57

PAGE 58

Also,for i 2f 1,..., r g ,dene R i tobethe q i q matrixof0sand1'ssuchthat R i u = u i Inotherwords, R i isthematrixthat extracts u i from u .Finally,for ~ s > 0 ,considerthe function :,1] ,~ s = 2 R + givenby s :=2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s max p + t s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(N 2 + a e )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(N 2 + a e r X i =1 tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i O T D ? OR T i s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i where t = rank )]TJ/F39 11.9552 Tf 5.48 -9.683 Td [(Z T P ? Z .Hereisourresultfor ~ Proposition4.1. Assumethat 2 b e + k y )]TJ/F39 11.9552 Tf 9.489 0 Td [(W ^ k 2 > 0 ~ s > 0 b i 0 foreach i 2f 1,2,..., r g andrank X = p sothat ~ iswelldened.Ifthefollowingtwoconditionshold,then ~ is geometricallyergodic. 1.Foreach i 2f 1,2,..., r g ,oneofthefollowingholds: i a i < b i =0; ii b i > 0. 2.Thereexistsan s 2 ,1] ,~ s = 2 suchthat s < 1 Remark8. Bythemselves, 4 4 4 ,and 4 donotimplythattheposterior densityisproper.Ofcourse,ifconditions1.and2.inProposition4.1holdaswell,then thechainisgeometricallyergodicandtheposteriorisnecessarilyproper. Remark9. Anumericalsearchcouldbeemployedtocheckthesecondconditionof Proposition4.1.Indeed,onecouldevaluatethefunction s atallpointsonanegrid ofvaluesintheinterval ,1] ,~ s = 2 .Thegoal,ofcourse,wouldbetondavalueof s atwhich s < 1 .Also,recallfromChapter1thatCorollary1providesanalternative setofsufcientconditionsthatarehardertosatisfy,buteasiertocheck. Remark10. When r =1 ,tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i O T D ? OR T i reducesto q )]TJ/F39 11.9552 Tf 11.989 0 Td [(t whichisthenumberofzero eigenvaluesof Z T P ? Z .Thus,inthiscase, s reducesto 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s max p + t s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(N 2 + a e )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(N 2 + a e q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 1 2 + a 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 1 2 + a 1 WewillproveProposition4.1indirectlybyprovingthatthe -chainsatisesthe conditionsofLemma4whentheconditionsofProposition4.1hold.BecauseitsMtd isstrictlypositiveon R r +1 + R r +1 + ,the -chainis -irreducibleandaperiodic,andits 58

PAGE 59

maximalirreducibilitymeasureisequivalenttoLebesguemeasureon R r +1 + .Since ~ 2 j 1 j iscontinuouson R r +1 + forallxed 2 R p + q seeAppendixBforaproof,it followsfromLemma3thatthe -chainisFeller.Thusallthatislefttodoistoestablish adriftconditionforthe -chain.Notethatsince ~ 1 := R R p + q ~ j y d isan unnormalizedinvariantdensityforthe -chain,Lemma4canbeusedtoshowthat theinvariantmeasure 1 A := R A ~ 1 d isnite.Thus,onceweestablishadrift condition,itwillfollowfromLemma4thattheposteriordensity ~ isproper. Considerthedriftfunction ~ v : R r +1 + R + denedas ~ v = )]TJ/F40 7.9701 Tf 6.587 0 Td [(s e + r X i =1 u i )]TJ/F40 7.9701 Tf 6.587 0 Td [(s + c e + r X i =1 c u i and > 0 s 2 ,1] ,and c 2 ,1 = 2 arexedconstantstobedetermined.Toshow that ~ v isunboundedoffcompactsets,wewilldemonstratethat,forevery d 2 R ,the set S d = 2 R r +1 + :~ v d iscompact.Let d besuchthat S d isnon-emptyotherwise S d istriviallycompact,which meansthat d and d = mustbelargerthan1.Since ~ v isacontinuousfunction, S d is closedin R r +1 + .Nowconsiderthefollowingset: T d = d = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = s d = 1 = c d )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = s d 1 = c d )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = s d 1 = c Theset T d iscompactin R r +1 ,andhencein R r +1 + .Since S d T d S d isaclosedsubset ofacompactsetin R r +1 + ,soitiscompactin R r +1 + .Hence,thedriftfunctionisunbounded offcompactsets. OurproofofProposition4.1willbecompleteonceweestablishthefollowingresult. Proposition4.2. Assumethat 2 b e + k y )]TJ/F39 11.9552 Tf 9.489 0 Td [(W ^ k 2 > 0 ~ s > 0 b i 0 foreach i 2f 1,2,..., r g andrank X = p sothat ~ iswelldened.UnderthetwoconditionsofProposition4.1, 59

PAGE 60

thereexista 2 [0,1 andaniteconstant L suchthat,forevery 0 2 R r +1 + E ~ k 1 )]TJ/F22 11.9552 Tf 5.768 -9.684 Td [(~ v j 0 ~ v 0 + L Therefore,underthetwoconditionsofProposition4.1,the -chainisgeometrically ergodic. Proof. Byconditioningon anditerating,wecanexpress E ~ k 1 ~ v j 0 as E ~ E ~ )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F40 7.9701 Tf 6.587 0 Td [(s e + E ~ r X i =1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s u i + E ~ )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( c e + E ~ r X i =1 c u i 0 Wenowdevelopupperboundsforeachofthefourtermsinsidethesquarebracketsin 4.Fix s 2 S :=,1] ,~ s = 2 ,anddene G 0 s =2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(N 2 + a e )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(N 2 + a e and,foreach i 2f 1,2,..., r g ,dene G i s =2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i Notethat,since s 2 ,1] x 1 + x 2 s x s 1 + x s 2 whenever x 1 x 2 0 .Thus, E ~ )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( )]TJ/F40 7.9701 Tf 6.586 0 Td [(s e =2 s G 0 s b e + k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(W k 2 2 s 2 s G 0 s b s e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 2 s = G 0 s )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 s + const where const denotesagenericconstant.Similarly, E ~ )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F40 7.9701 Tf 6.587 0 Td [(s u i =2 s G i s b i + k u i k 2 2 s G i s )]TJ/F21 11.9552 Tf 5.479 -9.684 Td [(k u i k 2 s + const 60

PAGE 61

Now,forany c > 0 ,wehave E ~ )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( c e =2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c G 0 )]TJ/F39 11.9552 Tf 9.299 0 Td [(c b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c G 0 )]TJ/F39 11.9552 Tf 9.299 0 Td [(c b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W ^ k 2 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c = const and,foreach i 2f 1,2,..., r g E ~ )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( c u i =2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c G i )]TJ/F39 11.9552 Tf 9.298 0 Td [(c b i + k u i k 2 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c G i )]TJ/F39 11.9552 Tf 9.299 0 Td [(c h )]TJ/F21 11.9552 Tf 5.479 -9.683 Td [(k u i k 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c I f 0 g b i + b i )]TJ/F40 7.9701 Tf 6.586 0 Td [(c I 1 b i i Let A = f i : a i < b i =0 g ,andnotethat E ~ P r i =1 c u i canbeboundedabovebya constantif A isempty.Thus,weconsiderthecaseinwhich A isemptyseparatelyfrom thecasewhere A 6 = ; .Webeginwiththelatter,whichisthemoredifcultcase. CaseI : A isnon-empty.CombiningthefourboundsaboveandapplyingJensen's inequalitytwice,wehave E ~ k 1 ~ v j 0 G 0 s h E ~ )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 0 i s + r X i =1 G i s h E ~ )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k u i k 2 0 i s + X i 2 A G i )]TJ/F39 11.9552 Tf 9.298 0 Td [(c E ~ h k u i k )]TJ/F23 7.9701 Tf 6.586 0 Td [(2 c 0 i + const AppendixCcontainsaproofofthefollowinginequalities: E ~ k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 0 p + t 0 e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + const and,foreach i 2f 1,2,..., r g E ~ k u i k 2 0 i 0 e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + i r X j =1 0 u j )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + const where i = tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i Z T P ? Z + R T i i = tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i O T D ? OR T i ,and A + denotesthe Moore-Penroseinverseofthematrix A .Itfollowsimmediatelyfrom4and4 that h E ~ )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 0 i s p + t s 0 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(s + const 61

PAGE 62

and h E ~ )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k u i k 2 0 i s s i 0 e )]TJ/F40 7.9701 Tf 6.586 0 Td [(s + s i r X j =1 0 u j )]TJ/F40 7.9701 Tf 6.586 0 Td [(s + const ItisalsoshowninAppendixCthat,forany c 2 ,1 = 2 ,andforeach i 2f 1,2,..., r g ,we have E ~ h k u i k )]TJ/F23 7.9701 Tf 6.587 0 Td [(2 c 0 i 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c )]TJ/F30 11.9552 Tf 6.775 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 h d c max )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 0 e c + )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 0 u i c i where d max denotesthelargesteigenvalueof Z T P ? Z .Combining4-4with 4,wehave E ~ k 1 ~ v j 0 1 s + 2 s 0 e )]TJ/F40 7.9701 Tf 6.586 0 Td [(s + 3 s r X j =1 0 u j )]TJ/F40 7.9701 Tf 6.586 0 Td [(s + 4 c )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 0 e c + 5 c X j 2 A )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 0 u j c + const where 1 s := G 0 s p + t s 2 s := r X i =1 s i G i s 3 s := r X i =1 s i G i s 4 c :=2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c d c max X i 2 A G i )]TJ/F39 11.9552 Tf 9.298 0 Td [(c \050 q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q i 2 and 5 c :=2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c max i 2 A G i )]TJ/F39 11.9552 Tf 9.298 0 Td [(c \050 q i 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(c \050 q i 2 Hence, E ~ k 1 ~ v j 0 s c ~ v 0 + L where s c =max 1 s + 2 s 3 s 4 c 5 c ToconcludetheproofofCaseI,weshowthatthereexistsatriple s c 2 R + S ,1 = 2 suchthat s c < 1 .Webeginbydemonstratingthat,if c issmallenough, then 5 c < 1 .Dene ~ a = )]TJ/F22 11.9552 Tf 11.291 0 Td [(max i 2 A a i .Also,set C =,1 = 2 ,~ a .Fix c 2 C and notethat 5 c =max i 2 A \050 q i 2 + a i + c \050 q i 2 + a i \050 q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q i 2 62

PAGE 63

Forany i 2 A c + a i < 0 ,andsince ~ s > 0 ,itfollowsthat 0 < q i 2 + a i < q i 2 + a i + c < q i 2 But, \050 x )]TJ/F39 11.9552 Tf 11.955 0 Td [(z = \050 x isdecreasingin x for x > z > 0 ,sowehave, \050 q i 2 + a i \050 q i 2 + a i + c = \050 q i 2 + a i + c )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q i 2 + a i + c > \050 q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q i 2 anditfollowsimmediatelythat 5 c < 1 whenever c 2 C .Condition2.inProposition4.1 impliesthatthereexistsan s ? 2 S suchthat 1 s ? < 1 and 3 s ? < 1 .Let c ? beany pointin C ,andchoose ? tobeanynumberlargerthan max 2 s ? 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 s ? 4 c ? Asimplecalculationshowsthat ? s ? c ? < 1 ,andthiscompletestheproofforCaseI. CaseII : A = ; .Sincewenolongerhavetoworkwith E ~ )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( c u i ,thebound4 becomes E ~ k 1 ~ v j 0 1 s + 2 s 0 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(s + 3 s r X j =1 0 u j )]TJ/F40 7.9701 Tf 6.587 0 Td [(s + const andthereisnorestrictionon c otherthan c > 0 .Hence, E ~ k 1 ~ v j 0 s ~ v 0 + L where s =max 1 s + 2 s 3 s Allthatremainsistoshowthatthereexistsa s 2 R + S suchthat s < 1 .As inCaseI,Condition2.inProposition4.1impliesthatthereexistsan s ? 2 S suchthat 1 s ? < 1 and 3 s ? < 1 .Let ? beanynumberlargerthan 2 s ? 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 s ? 63

PAGE 64

Asimplecalculationshowsthat ? s ? < 1 ,andthiscompletestheproofforCaseII. Theproofisnowcomplete. WenowprovideaproofofCorollary1. ProofofCorollary1. Itsufcestoshowthat,together,conditions B 0 and C 0 of Corollary1implythesecondconditionofProposition4.1.Clearly, B 0 and C 0 imply that ~ s = 2 > 1 ,so ,1] ,~ s = 2=,1] .Take s ? =1 .Condition C 0 implies 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s ? p + t s ? )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(N 2 + a e )]TJ/F39 11.9552 Tf 11.955 0 Td [(s ? )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(N 2 + a e = p + t N +2 a e )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 < 1. Now,sincetr D ? isthenumberofzeroeigenvaluesof Z T P ? Z ,wehave r X i =1 tr )]TJ/F39 11.9552 Tf 5.48 -9.683 Td [(R i O T D ? OR T i = tr O T D ? O r X i =1 R T i R i = tr )]TJ/F39 11.9552 Tf 5.479 -9.683 Td [(O T D ? OI = tr D ? = q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t Hence, 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s ? r X i =1 )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i )]TJ/F39 11.9552 Tf 11.955 0 Td [(s ? )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 + a i tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i O T D ? OR T i s ? = r X i =1 tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i O T D ? OR T i q i +2 a i )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 P r i =1 tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i O T D ? OR T i min j 2f 1,2,..., r g f q j +2 a j )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 g = q )]TJ/F39 11.9552 Tf 11.956 0 Td [(t min j 2f 1,2,..., r g f q j +2 a j )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 g < 1, wherethelastinequalityfollowsfromcondition B 0 Wechosetoworkwiththe -chainratherthanthe -chainbecausethereisan importanttechnicaldifferencebetweenthetwochainsthatstemsfromthefactthat, whenthenullset N isnon-empty, ~ 1 j 2 j isnotcontinuouson R p + q foreachxed Recallthat ~ 1 j 2 j = 8 > < > : f G e ; N 2 + a e b e + k y )]TJ/F40 7.9701 Tf 6.586 0 Td [(W k 2 2 Q r i =1 f G u i ; q i 2 + a i b i + k u i k 2 2 if = 2N f G e ;1,1 Q r i =1 f G u i ;1,1 if 2N 64

PAGE 65

Also,recallthattheMtdofthe -chainisgivenby ~ k 1 j 0 = Z R p + q ~ 1 j 2 j ~ 2 j 1 j 0 d Sincetheset N hasmeasurezero,thearbitrarypartof ~ 1 j 2 j doesnotcontribute anythingto ~ k 1 .However,thesamecannotbesaidforthe -chain,whoseMtdisgivenby ~ k 2 j 0 = Z R r +1 + ~ 2 j 1 j ~ 1 j 2 j 0 d Thisdifferencebetween ~ k 1 and ~ k 2 isimportantandshouldbekeptinmindwhenone triestoapplytopologicalresultsfromMarkovchaintheory,suchasthoseinChapter 6ofMeynandTweedie1993.Forexample,inourproofthatthe -chainisaFeller chain,weusedthefactthat ~ 2 j 1 j iscontinuouson R r +1 + foreachxed .Ontheother hand,since ~ 1 j 2 j isnotcontinuouson R p + q ,wecannotnotusethesameargumentto provethatthe -chainisFeller.Ofcourse,ifthechainisnotknowntobeFeller,thenour Lemma4cannotbeusedtoestablishthegeometricergodicityofthe -chain. Itispossibletoavoidtheproblemdescribedabovebyremovingtheset N fromthe statespaceofthe -chain.Inthiscase,wearenolongerrequiredtodene ~ 1 j 2 j for 2N ,andsince ~ 1 j 2 j iscontinuousforxed 2 R r +1 + on R p + q nN ,the Fellerargumentforthe -chainwillgothrough.Ontheotherhand,thenewstate spacehasholesinit,andthiscouldcomplicatethesearchforadriftfunctionthat isunboundedoffcompactsets.Forexample,consideratoydriftfunctiongivenby ~ v x = x 2 .Straightforwardargumentsshowthat ~ v x isunboundedoffcompactsets whenthestatespaceis R ,butnotwhenthestatespaceis R nf 0 g .However,ifoneadds theterm 1 = x 2 to ~ v x thenthemodiedfunction, v x = x 2 +1 = x 2 ,isunboundedoff compactsetswhenthestatespaceis R nf 0 g .TanandHobert2009overlookedaset ofmeasurezerosimilartoour N ,andthisoversightledtoanerrorintheproofoftheir Proposition3.Inthenextsection,weprovideacorrectedproofofthisresultaswellasa renementofit. 65

PAGE 66

4.3TanandHobert'sResult TanandHobert2009hereafter,T&Hconsideredthecenteredversionofthe one-wayrandomeffectsmodel,which,intheirnotation,is Y ij = i + ij where i =1,..., q j =1,..., m i ,the i sareiidN 2 andthe ij s,whichare independentofthe i s,areiidN 2 e .Theseauthorsconsideredaparametricfamilyof improperpriordensitiesgivenby a b )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 2 e = )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [( a +1 )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 e )]TJ/F23 7.9701 Tf 6.586 0 Td [( b +1 where a and b areknownhyper-parameters.Let 2 = )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 2 e and = )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 1 ,..., q andassumethat SSE = P q i =1 P m i j =1 y ij )]TJETq1 0 0 1 290.647 451.748 cm[]0 d 0 J 0.478 w 0 0 m 6.854 0 l SQBT/F39 11.9552 Tf 290.647 444.428 Td [(y i 2 ,theerrorsumofsquares,ispositive.Of course, y i denotesthe i thgroupmean, m )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i P m i j =1 y ij .ResultsinSunetal.2001imply thattheposteriordensityof 2 isproperifandonlyif a < 0, a + q 2 > 0, and M +2 a +2 b )]TJ/F22 11.9552 Tf 11.955 0 Td [(1 > 0, where M = P q i =1 m i isthetotalnumberofobservations. T&HstudiedtheTVGSMarkovchain, f n 2 n g 1 n =0 whichdrawsthecomponents of 2 frominvertedgammadistributionsand,giventhevalueof 2 ,itthendraws from amultivariatenormaldistribution.SeeTanandHobert2009forthespecicformulas. T&Hpresentedanargumentforthefollowingresult. Proposition4.3. Assumethattheproprietyconditionsgivenin4hold.Then f n 2 n g 1 n =0 isgeometricallyergodicif 1. M +2 b q +3 2. q min P q i =1 m i m i +1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 m M < 2 exp )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 2 + a where m =max f m 1 ,..., m q g and x denotesthedigammafunction. 66

PAGE 67

T&Hanalyzedthe -chain, f n g 1 n =0 ,whichwasdenedtohavestatespace R q +1 andMtd ^ k )]TJ/F22 11.9552 Tf 6.487 -7.027 Td [(~ j = Z R 2 + )]TJ/F22 11.9552 Tf 6.487 -7.027 Td [(~ j 2 y )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 j y d 2 Theseauthorsexpressedtheconditionaldensity )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 j y asaproductoftwoinverted gammadensities,oneofwhichisthesourceofaproblemsimilartotheonediscussed inSection4.2.Toseethis,let f IG x ; c d denotethedensityofaninvertedgamma randomvariablewithshapeparameter c > 0 andscaleparameter d > 0 ;thatis, f IG x ; c d = 8 > < > : d c \050 c x c +1 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(d = x x > 0 0 x 0. Theproblematicconditionaldensity )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( 2 j y wasexpressedasaninvertedgamma withshape q = 2+ a andscale 1 2 q X i =1 i )]TJ/F25 11.9552 Tf 11.955 0 Td [( 2 T&Hoverlookedthefactthatthisdensityisnotdenedontheset N = 2 R q +1 : = 1 = = q Thus,T&H'sMtd, ^ k ,isnotwell-denedfor 2N ,and,asaresult,theirargument showingthatthe -chainisFellerasachainon R q +1 isincorrect. WecanrepairT&H'sproofbyredeningthestatespaceofthe chaintobe R q +1 nN ,provingthatthe -chainisgeometricallyergodicon R q +1 nN usingLemma4, andthenshowingthatthischainmustalsobegeometricallyergodicon R q +1 .Indeed, forxed 2 )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 j y isacontinuousfunctionof on R q +1 nN ,soanapplication ofFatou'sLemmaandourLemma3showsthatthe -chainisFelleronthenewstate space.Now,thedriftfunctionthatisusedinT&H'sproofofProposition3takestheform w TH := w 1 s + w 2 s = q X i =1 i )]TJ/F25 11.9552 Tf 11.955 0 Td [( 2 # s + q X i =1 m i y i )]TJ/F25 11.9552 Tf 11.956 0 Td [( i 2 # s 67

PAGE 68

where > 0 and s 2 ,1] .Thisfunctionisunboundedoffcompactsetswhenthestate spaceis R q +1 ,butnotwhenthestatespaceis R q +1 nN .Toremedythisproblem,we addthefollowingtermtothedriftfunction w new := w 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c = q X i =1 i )]TJ/F25 11.9552 Tf 11.955 0 Td [( 2 # )]TJ/F40 7.9701 Tf 6.586 0 Td [(c where c > 0 .Sincethisfunctionblowsupas approaches N ,itcanbeshownthatthe modieddriftfunctionisunboundedoffcompactsetsonthenewstatespace, R q +1 nN Theargumentissimilartotheoneusedforthedriftfunction v inSection4.2. Assumethattheproprietyconditonsin4hold.Itisstraightforwardtoshow usingLemma2thattherestricted -chainisHarrisErgodic.Also,anapplicationof Lemma1showsthatthesupportofthemaximalirreducibilitymeasureoftherestricted -chainisnon-empty.Hence,inordertoobtainthegeometricergodicityoftherestricted -chain,weestablishthefollowingdriftconditionandthenapplyourLemma4. Proposition4.4. UndertheconditionsinProposition4.3,thereexista 2 [0,1 anda niteconstant L suchthat,forevery 2 R q +1 nN E )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(w ~ j w + L ThefollowingresultwillbeusedintheproofofProposition4.4. Lemma11. Let c 2 ,1 = 2 .Then E [ w new j 2 ] )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(c \050 q 2 + )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(c m c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 forany 2 2 R 2 + Proof. Dene t = q X i =1 m i 2 e + 2 m i and,for i 2f 1,2,..., q g h i = 2 e 2 e + 2 m i 68

PAGE 69

FromTanandHobert2009wehave, Var i j 2 = h i 2 + h 2 i t Cov i j j 2 = h i h j t Cov i j 2 = h i t andVar j 2 = 1 t Usingthiswehave, Var i )]TJ/F25 11.9552 Tf 11.955 0 Td [( j 2 = 2 h i + )]TJ/F39 11.9552 Tf 11.955 0 Td [(h i 2 t andCov i )]TJ/F25 11.9552 Tf 11.956 0 Td [( j )]TJ/F25 11.9552 Tf 11.956 0 Td [( j 2 = )]TJ/F39 11.9552 Tf 11.955 0 Td [(h i )]TJ/F39 11.9552 Tf 11.955 0 Td [(h j t Thuswecanwrite Var )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 j 2 = 2 H + 1 t hh T where H isa q q diagonalmatrixwhose i thdiagonalentryisequalto h i and h = )]TJ/F39 11.9552 Tf 11.955 0 Td [(h 1 ,...,1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(h q T .Itiseasytoseethat, )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 I q 2 H 2 H + 1 t hh T = Var )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 j 2 Hence, )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Var )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 j 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 m I q Now,given 2 ,wehave )]TJ/F25 11.9552 Tf 11.956 0 Td [( 1 T )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Var )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 j 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 2 q where isthenon-centralityparameter.Finally, E 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 T )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 c 2 = )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 m c E 2 4 0 @ 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 T 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + 2 e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 m )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 1 A c 2 3 5 )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 m c E 1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 T Var )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 j 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 c 2 )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 m c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 )]TJ/F25 11.9552 Tf 5.48 -9.683 Td [( 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 + )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( 2 e )]TJ/F40 7.9701 Tf 6.586 0 Td [(c m c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 69

PAGE 70

wheretherstinequalityfollowsfrom4andthesecondinequalityfollowsfrom 4andLemma16.Thelastinequalityfollowssince c 2 ,1 = 2 ProofofProposition4.4. First,notethatwecanexpress E w ~ j as E w TH ~ + E w new ~ Let S :=,1] ,~ s = 2 where ~ s =min f M +2 b q +2 a g FromTanandHobert2009,wehavethatconditions1.and2.inProposition4.3imply thatthereexists > 0 s 2 S TH 2 [0,1 ,and L TH < 1 suchthat E w TH ~ TH w TH + L TH forevery 2 R q +1 nN .Ourgoalisnowtoshowthat a < 0 in4and SSE > 0 imply thatthereexists c 2 ,1 = 2 new 2 [0,1 ,and L new < 1 suchthat E w new ~ new w new + L new forevery 2 R q +1 nN .Thisisbecauseif4holdsthenthedesireddriftconditionis established.Indeed,if4holdsthen E w ~ j TH w TH + L TH + new w new + L new w + L where =max f TH new g < 1 and L = L TH + L new < 1 Now,byLemma11wehave E [ w new ~ j 2 ] )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(c \050 q 2 + )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 e )]TJ/F40 7.9701 Tf 6.587 0 Td [(c m c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 70

PAGE 71

Since SSE > 0 ,wehave E h E w new ~ 2 i E h )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c i 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(c \050 q 2 + E h )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( 2 e )]TJ/F40 7.9701 Tf 6.586 0 Td [(c i m c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 \050 a + q 2 + c \050 a + q 2 2 c 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 w new + \050 b + M 2 + c \050 b + M 2 2 c m c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 SSE )]TJ/F40 7.9701 Tf 6.586 0 Td [(c = new c w new + const where new c = \050 q 2 + a + c \050 q 2 + a \050 q 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c \050 q 2 Now,let C =,1 = 2 )]TJ/F39 11.9552 Tf 9.299 0 Td [(a 6 = ; andargueasintheproofofProposition4.2toshow that new c < 1 whenever c 2 C .Hence,thereexists > 0 s 2 ,1] ,and c 2 C such that4holds.Thiscompletestheproof. Wenowusethefactthattherestricted -chainisgeometricallyergodictoshowthat theoriginal -chainisalsogeometricallyergodic.Redene 2 j y sothattheoriginal -chainiswell-denedontheentirespace, R q +1 .Fromnowon,welet 2 j y = 8 > < > : f IG 2 e ; M 2 + b P q i =1 m i y i )]TJ/F26 7.9701 Tf 6.587 0 Td [( i 2 + SSE 2 f IG 2 ; q 2 + a P q i =1 i )]TJ/F26 7.9701 Tf 6.586 0 Td [( 2 2 if = 2N f IG 2 e ;1,1 f IG 2 ;1,1 if 2N Itisstraightforwardtocheckthat f n 2 n g 1 n =0 f n g 1 n =0 ,and f 2 n g 1 n =0 areallHarris ergodicontheoriginalstatespaces.Now,let and ~ denotetheoriginaland therestricted -chain,respectively.Also,let K and ~ K denotetheMtfsof and ~ respectively,andlet X= R q +1 and ~ X= R q +1 nN .NotethattheMtf ~ K satises ~ K x B = K x B forany x 2 ~ X and B 2B ~ X = f ~ X A : A 2B X g .Wehavethefollowingresult. Lemma12. UndertheconditionsinProposition4.3,theoriginalMarkovchain, ,is geometricallyergodic. 71

PAGE 72

Proof. Let and ~ denotetheLebesguemeasureson X and ~ X ,respectively.Recall thatsincetheirMtdsarestrictlypositiveintheirrespectivedomains, and ~ are -irreducibleand ~ -irreducible,respectively.Also,letting and ~ denotethemaximal irreducibilitymeasuresoftheoriginalandtherestrictedchain,respectively,wehaveby Lemma1that and ~ ~ .Recallalsothat and ~ arebothaperiodicbutonly ~ hasbeenproventobeFeller. Now,sincewehaveestablishedadriftconditionfortherestrictedchainusing Lemma4,partiofMeynandTweedie's1993Theorem15.0.1saysthatthereexists a -petiteset C 2B ~ X seebelowforadenitionandanumber ~ K 1 C suchthat ~ C > 0 and j ~ K n x C )]TJ/F22 11.9552 Tf 13.735 2.656 Td [(~ K 1 C j < M C n C forall x 2 C .Itcanbeshownbyinduction,that K n x B = ~ K n x B ~ X, forany x 2 ~ X and B 2B X .Thus,for x 2 C K n x C = ~ K n x C .Hence, j K n x C )]TJ/F22 11.9552 Tf 13.735 2.657 Td [(~ K 1 C j < M C n C forall x 2 C .Also,since ~ C > 0 ,wemusthave C =~ C > 0 whichimpliesthat C > 0 .Therefore,ifwecanshowthat C isalsopetitefortheoriginalchain,thenit willfollowfromTheorem15.0.1thattheoriginalchainisgeometricallyergodic. Theset C isa -petitesetfortherestrictedchain.Thismeansthat isanontrivial measureon B ~ X suchthat,forall x 2 C andall B 2B ~ X ,wehave 1 X n =0 ~ K n x B a n B where a n isamassfunctionon f 0,1,2,... g .Astandardargumentshowsthattheset function on B X givenby = ~ X iswelldened.Moreover, isameasureon 72

PAGE 73

B X .Now,forany x 2 C andany B 2B X ,wehave 1 X n =0 K n x B a n = 1 X n =0 ~ K n x B ~ X a n B ~ X = B wheretherstequalityfollowsfrom4.Thisimpliesthat C is -petiteforthe originalchain. WenowexplainhowT&H'sresultcanbeimprovedusingtheresultsdevelopedin Sections3.3and4.2.ConsidertheNCPversionoftheone-waymodelwhichisgivenby Y ij = + u i + ij where i =1,..., q j =1,..., m i .AsintheCPversion,the u i sareiidN 2 andthe ij s,whichareindependentofthe u i s,areiidN 2 e .Also,considerthe twointractableposteriordensitiesthatresultwhentheCPandNCPversionsof theone-wayrandomeffectsmodelarecombinedwiththesameimproperprior a b )]TJ/F25 11.9552 Tf 5.479 -9.683 Td [( 2 2 e .Itiseasytoseethatthesetwoposteriorsarerelatedbyaone-to-one transformationthatinvolves i u i = i )]TJ/F25 11.9552 Tf 12.889 0 Td [( .NotealsothatiftheNCPone-way modeliswrittenintermsoftheprecisioncomponents e =1 = 2 e and =1 = 2 ,then itisaspecialcaseoftheGLMMwithimproperpriorsgiveninChapter1.Recallfrom Section3.3,thatreparametrizationssuchasvariancevsprecisionandCPvsNCPdo notaffecttheconvergenceratesoftheTVGSsconsideredinthisdissertation.Therefore, T&H'sTVGSMarkovchainisgeometricallyergodicundersametheconditionsin Proposition4.1. 73

PAGE 74

Proposition4.5. Let ~ s =min f M +2 b q +2 a g andassumethattheproprietyconditions givenin 4 hold.Ifthereexistsan s 2 ,1] ,~ s = 2 suchthat 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s max q s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 2 + a )]TJ/F39 11.9552 Tf 11.956 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 2 + a < 1, thenT&H'sTVGSMarkovchainisgeometricallyergodic. Remark11. Notethat,asopposedtotheconditionsfoundinT&H'sresult,theconditionsinProposition4.5donotdependexplicitlyonthegroupsizes, f m i g q i =1 Remark12. ToobtaintheNCPone-waymodelfromourGLMM,take X tobea N 1 vectorofonesand Z tobetheusual N q cell-meansmatrix.Notethat,inthiscase, rank Z T P ? Z = q )]TJ/F22 11.9552 Tf 11.955 0 Td [(1 CalculationssimilartothosefoundinTanandHobert's2009AppendixB.2canbe usedtoestablishthefollowingcorollarytoProposition4.1. Corollary2. Assumethattheproprietyconditionsgivenin4hold.ThenT&H's TVGSMarkovchain, f n 2 n g 1 n =0 ,isgeometricallyergodicif 1. M +2 b q +2 2. 1 < 2 exp )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 2 + a where x denotesthedigammafunction. Proof. WewillshowthatthetwoconditionsinCorollary2implythatthereexists s 2 S :=,1 ,~ s = 2 suchthat ~ 1 s :=2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s q s )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(M 2 + b )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b and ~ 3 s :=2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 2 + a )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 2 + a are both lessthan1.Notethat ~ 1 s isactuallyafunctionof s q ,and M 2 + b sowrite ~ 1 s q M 2 + b = )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(M 2 + b q 2 s Also,recallthat,foranyxed s > 0 \050 x )]TJ/F39 11.9552 Tf 11.972 0 Td [(s = \050 x isdecreasingin x for x > s .Usingthis andthefactthat T x := \050 x + s \050 x x + s )]TJ/F22 11.9552 Tf 11.955 0 Td [(1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(s for x > 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s 74

PAGE 75

isstrictlydecreasingin x with lim x !1 T x =1 aproofofthesetwopropertiesof T x canbefoundinTanandHobert's2009AppendixB.2.1,wehavethatif M +2 b q +2 then ~ 1 s q M 2 + b = )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(M 2 + b q 2 s )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q +2 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q +2 2 q 2 s = T q +2 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 < 1. Thelastinequalityfollowsfromthepropertiesof T x andthefactthat,forany s 2 ,1 ,wehave q +2 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s > 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(s Sofarwehaveshownthat M +2 b q +2 impliesthat ~ 1 s < 1 forall s 2 S .Thus, itremainstoshowthat 1 < 2 exp q 2 + a impliesthatthereexists s 2 S suchthat ~ 3 s q a =2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 2 + a )]TJ/F39 11.9552 Tf 11.955 0 Td [(s )]TJ/F30 11.9552 Tf 6.775 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 2 + a < 1. Notethatthereexists s 2 S suchthat ~ 3 s q a < 1 ifandonlyif 1 < A q a ,where A q a :=2sup s 2 S )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(q 2 + a )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.649 Td [(q 2 + a )]TJ/F39 11.9552 Tf 11.955 0 Td [(s 1 = s Now,fromTanandHobert's2009AppendixB.2.2,weobtainthat 2 exp q 2 + a A q a andsinceweareassumingthat 1 < 2 exp q 2 + a weconcludethatthereexists s 2 S suchthat ~ 3 s q a < 1 .Thiscompletesthe proof. 75

PAGE 76

Finally,thefactthat,foranygivencombinationofgroupsizes f m i g q i =1 1 q min 8 < : q X i =1 m i m i +1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 m M 9 = ; showsthattheconditionsinCorollary2areweakerthanthoseinT&H'sresult. 4.4HonestEstimationofthePosteriorExpectations HonestestimationofposteriorexpectationsinthecontextoftheBayesianGLMM withimproperpriorsrequiresthesamestepsdescribedinSection3.4.Itinvolvesnding ageometricallyMarkovchain,verifyingthataone-stepminorizationconditionholdsfor thechain,andestablishingcertainmomentconditionsforthefunctionsofinterest.We willexplainhowthesestepscanbeaccomplishedintheimproperpriorcaseandreport oursimulationresultsfortheHMOdatathatweredescribedinSection3.4. ConsiderusingthesameGLMMasinSection3.4butthistimeadoptanimproper prior.Recallthatthemodelequationintermsoftheprecisionparameters )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e =1 2 e and )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u =1 2 u isgivenby Y ij = 0 + 1 x i 1 + 2 x i 2 + u i + ij wherethe ij sareiidN )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e andthe u i sareiidN )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u .Hereweadopttheimproper prior ~ p ; ~ a ~ b ,where ~ a =, )]TJ/F22 11.9552 Tf 9.298 0 Td [(1 = 2 T ~ b =0 T ;thatis, a e = b e = b u =0 and a u = )]TJ/F22 11.9552 Tf 9.299 0 Td [(1 = 2 .Ofcourse,thismodelisaspecialcaseoftheBayesianGLMMwith improperpriorsdiscussedinChapter1. ItiseasytocheckthattheconditionsofCorollary1aresatised,sotheTVGS Markovchainthatcanbeemployedinthiscaseisgeometricallyergodic.Anargument similartotheonegivenbyHobertetal.2006showsthataone-stepminorization conditionholdsinthiscase.Further,PropositionsD.4andD.5implythat,fortheHMO dataexample,thefunctions g 1 = 0 g 2 = 1 g 3 = 2 g 4 = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e and g 5 = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u 76

PAGE 77

allsatisfythemomentconditionsthatguaranteethattheTHestimatorsofthe asymptoticvariancewithtruncationpoint t n =[ n 1 = 3 ] arestronglyconsistent.Thus,as intheproperpriorexamplegiveninSection3.4,wecanusetheseestimatorstogether withthexed-widthruletoobtainBonferroniCIsfortheposteriorexpectations f E ~ g i g 5 i =1 NotethatanyoftheothermethodsforestimatingtheasymptoticvarianceoftheCLT giveninSection2.2canalsobeemployedhere.Wecarriedoutthesamexed-width proceduredescribedinSection3.4withtheonlydifferencebeingthatthevectorofthe desiredmarginoferrorsisnow =.810.020.162.50.5 T .Afterchoosing witha preliminaryrunof1,000,000iterations,theresultingdrawswerethenthrownawayand usedasburn-in.Thexed-widthprocedurewasstoppedafter n = 720,000iterations. Fortheresultsofthisprocedure,seeTable4-1.SeealsoTable4-2forresultsbasedon alongerrunof n = 2,000,000iterations. Table4-1.ThistablecontainsasummaryofourMCMCprocedurefortheBayesian GLMMwithimproperpriorsusedintheHMOdataexample.Foreachmodel parameter,itcontainsestimatesoftheposteriorexpectationandthe correspondingasymptoticvariance,aswellasthestandarderroranda95% asymptoticBonferronicondenceinterval.Theresultsarebasedon n = 720,000observations. ParameterEstimate ^ & 2 g n q ^ & 2 g n n 95%BonferroniCI 0 163.6695.1420.003.662,163.676 1 4.8216.2190.003.814,4.829 2 32.12864.3600.009.103,32.152 2 e 499.7044311.0780.077.504,499.904 2 u 110.29827020.6890.194.798,110.798 Table4-2.Resultsbasedon n = 2,000,000observations. ParameterEstimate ^ & 2 g n q ^ & 2 g n n 95%BonferroniCI 0 163.6695.1210.002.665,163.674 1 4.8216.2340.002.816,4.825 2 32.13565.1020.006.120,32.150 2 e 499.6424185.6600.046.524,499.760 2 u 110.30026735.0120.116.002,110.598 77

PAGE 78

Figure4-1.Thetopplotshowsthebehavioroftheestimatorof E ~ 2 u andthe correspondingapproximate95%BonferroniCIasthenumberofiterations increases.Thesolidlinerepresentstheestimatorandthedashedlines denotetheupperandlowerendpointsoftheCI.Themiddleplotshowsthe behavioroftheTHestimatoroftheasymptoticvariance, ^ & 2 g 5 n .Thebottom plotsshowsthetrajectoryofthemarginoferror 2.58 q ^ & 2 g 5 n n 78

PAGE 79

APPENDIXA MATRIXINEQUALITIES Lemma13. Suppose Q isan n n matrixoftheform Q = A T A +, where isapositiveconstant, A isanon-null m n matrixand isan n n diagonal matrixwithpositivediagonalelements, f i g n i =1 .Let O T DO bethespectraldecompositionof A T A ,so O isan n -dimensionalorthogonalmatrix,and D isadiagonalmatrix whosediagonalelements, f d i g q i =1 ,aretheeigenvaluesof A T A .Also,let D ? denotethe n -dimensionaldiagonalmatrixwhosediagonalelements, f d ? i g q i =1 ,aregivenby d ? i = 8 > < > : 1 d i =0 0 d i 6 =0. Then 1. Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 A T A + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + O T D ? O )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 min 2.tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( A T A + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + n )]TJ/F47 11.9552 Tf 11.955 0 Td [(rank A )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 min 3.tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(AQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 A T rank A )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 where A T A + denotestheMoore-Penroseinverseof A T A and min =min 1 i n f i g Remark13. Notethatrank A = rank A T A = rank O T DO = rank D .Thusrank A equalsthenumberofnon-zeroeigenvaluesof A T A Proof. Itisclearthat A T A + I n min A T A +. Thisyields Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = A T A + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 A T A + I n min )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = O T D + I n min )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 O A 79

PAGE 80

Nowlet D + beadiagonalmatrixwhosediagonalelements, f d + i g q i =1 ,aregivenby d + i = 8 > < > : d )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i d i 6 =0 0 d i =0. Notethat,foreach i 2f 1,2,..., r g ,wehave 1 d i + min d + i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + I f 0 g d i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 min Thisshowsthat D + I n min )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 D + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + D ? )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 min TogetherwithA,thisleadsto Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 O T D + I n min )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 O O T )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(D + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + D ? )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 min O = A T A + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + O T D ? O )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 min whichprovestherststatement.Thesecondstatementfollowsbytakingtracesboth sidesandthefactthattr D ? equalsthenumberofzeroeigenvaluesof A T A .Pre-and post-multiplyingtherststatementby A and A T ,respectively,andthentakingtraces yields tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(AQ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 A T tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(A A T A + A T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(AO T D ? OA T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 min A Since A T A A T A + isidempotent,wehave tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(A A T A + A T = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(A T A A T A + = rank )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(A T A A T A + = rank A T A = rank A Furthermore, tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(AO T D ? OA T = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(O T D ? OA T A = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(O T D ? OO T DO = tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(O T D ? DO =0, wherethelastlinefollowsfromthefactthat D ? D =0 .ItnowfollowsfromAthat tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(AQ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 A T rank A )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 andthethirdstatementhasbeenestablished. 80

PAGE 81

APPENDIXB APROOFOFCONTINUITY Hereweshowthat,thefunction ~ 1 j 2 j iscontinuouson R r +1 + forallxed 2 R p + q Recallthat ~ 2 j 1 j isamultivariatenormaldensitywithmeanvector E ~ j = 2 6 4 X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(I )]TJ/F25 11.9552 Tf 11.956 0 Td [( e Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? y e ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T P ? y 3 7 5 andcovariancematrix Var ~ j = 2 6 4 e X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F22 11.9552 Tf 9.299 0 Td [( X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F22 11.9552 Tf 11.286 2.657 Td [(~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 3 7 5 where P ? = I )]TJ/F39 11.9552 Tf 11.955 0 Td [(X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T and ~ Q = e Z T P ? Z + u .Also,let V = Var ~ j Itiseasytoseethat ~ 1 j 2 j y = j V )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 j 1 = 2 p + q 2 exp )]TJ/F22 11.9552 Tf 10.494 8.087 Td [(1 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(E ~ j T V )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(E ~ j where V )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = 2 6 4 e X T X e X T Z e Z T X e Z T Z + u 3 7 5 Notethattheelementsof V )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 areallcontinuousfunctionsof .Also,since X hasfull columnrank,wehave j V )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 j = p e j X T X j ~ Q .Now,itsufcestoshowthat j V )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 j andthe elementsof E ~ j areallcontinuousfunctionsof .Indeed,sincetheelementsof ~ Q arecontinuousfunctionsof ,itfollowsfromthedenitionofthedeterminantthat ~ Q is alsoacontinuousfunctionof .Moreover,theelementsoftheadjointof ~ Q ,adj ~ Q ,are alsocontinuousfunctionsof .Finally,since ~ Q isnon-singularforall 2 R r +1 + ,wehave that ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = 1 ~ Q adj ~ Q andthustheelementsof ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 arecontinuousfunctionsaswell.Hence,thedeterminant j V )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 j andtheelementsof E ~ j areallcontinuousfunctionsof 81

PAGE 82

APPENDIXC UPPERBOUNDS Hereweestablishtheinequalties4,4,and4.Recallthat = T u T T and W = XZ .Astraightforwardcalculationthatusesthemultivariate normaldensity ~ 2 j 1 j showsthat E ~ )]TJ/F21 11.9552 Tf 5.48 -9.683 Td [(k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 = tr W Var ~ j W T + k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(WE ~ j k 2 C and E ~ )]TJ/F21 11.9552 Tf 5.479 -9.684 Td [(k u i k 2 j = E ~ )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k R i u k 2 j = tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i + k E ~ )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i u j k 2 C Thefollowingresultwillallowustoboundthetermstr W Var ~ j W T ,tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 R T i whichappearinCandC,and E )]TJ/F21 11.9552 Tf 10.461 -9.684 Td [(k u i k 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c whichappearsin4. Lemma14. Thefollowinginequalitiesholdforall 2 R r +1 + : 1. ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + O T D ? O )]TJ 7.472 -0.718 Td [(P r j =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u j 2.tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(P ? Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? rank Z T P ? Z )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e 3. R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max + u i I q i ,forall i 2f 1,2..., r g Proof. RecallfromSection4.2that O T DO isthespectraldecompositionof Z T P ? Z = ZP ? T P ? Z ,andthat D ? isabinarydiagonalmatrixwhose i thdiagonalelementis1if andonlyifthe i thdiagonalelementof D is0.AnapplicationofLemma13establishes 1.and2..RecallfromSection4.2that d max isthelargesteigenvalueof Z T P ? Z ,andthat R i isthe q i q matrixof0sand1'ssuchthat R i u = u i .Now,x i 2f 1,2,..., r g andnote that ~ Q = e Z T P ? Z + u e d max I q + u Itfollowsthat R i )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( e d max I q + u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i R i ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 R T i 82

PAGE 83

andsincethesetwomatricesarebothpositivedenite,wehave R i ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 R i )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max I q + u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( e d max + u i I q i andthisprovesthatthethirdstatementistrue. Now, tr W Var ~ j W T = )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e tr P + tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(PZ ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Z ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T P + tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T = p )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e )]TJ/F20 11.9552 Tf 11.955 0 Td [(tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P + tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Z ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T = p )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + tr )]TJ/F39 11.9552 Tf 5.48 -9.683 Td [(Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? = p )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(P ? Z ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? p )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + rank Z T P ? Z )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e = p + t )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e C wheretheinequalityisanapplicationofLemma14.AnotherapplicationofLemma14, showsthat tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i Z T P ? Z + R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(R i O T D ? OR T i r X j =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u j = i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + i r X j =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u j C Lemma15. Assumethatrank X = p .Thefunctions k E ~ j k k E ~ u j k and k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(WE ~ j k areallbounded. Proof. Webeginbyshowingthat k E ~ u j k = k e ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? y k isboundedon R r +1 + Let ~ z i and y i denotethe i thcolumnof ~ Z T = P ? Z T andthe i thcomponentof y 83

PAGE 84

respectively.Then, k E ~ u j k = )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Z T P ? Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Z T P ? y = N X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.027 Td [(~ Z T ~ Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i y i N X i =1 )]TJ/F22 11.9552 Tf 6.885 -7.028 Td [(~ Z T ~ Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ z i y i = N X i =1 N X j =1 ~ z j ~ z T j + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i y i = N X i =1 j y i j ~ z i ~ z T i + X j 2f 1,2,..., N gnf i g ~ z j ~ z T j + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ z i AstraightforwardmanipulationandanapplicationofLemma9showthatforeach i 2f 1,2,..., N g ~ C i := ~ z i ~ z T i + X j 2f 1,2,..., N gnf i g ~ z j ~ z T j + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e u )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ~ z i isbounded.AsimilarargumentwasgivenintheproofofLemma8.Hence, E ~ u j is bounded.Toendtheproof,notethat k E ~ j k 2 )]TJ/F21 11.9552 Tf 5.48 -9.684 Td [(k X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T y k + k X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T Z kk E ~ u j k 2 const and k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(WE ~ j kk y k + k W k p k E ~ j k 2 + k E ~ u j k 2 BycombiningCandC,andapplyingLemma15,wehave E ~ k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 p + t )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + const AnotherconsequenceofLemma15isthat E ~ R i u j k R i kk E ~ u j k const C 84

PAGE 85

Theinequality E ~ k u i k 2 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + i r X j =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u j + const nowfollowsbycombiningC,CandC. Thefollowingresultwillallowustoprovideanupperboundon E )]TJ/F21 11.9552 Tf 10.461 -9.683 Td [(k u i k 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c .Let 2 k w denotethenon-centralchi-squaredistributionwith k degreesoffreedomand non-centralityparameter w Lemma16. If U 2 k w and 2 k = 2 ,then E [ U )]TJ/F26 7.9701 Tf 6.587 0 Td [( ] 2 )]TJ/F26 7.9701 Tf 6.587 0 Td [( )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(k 2 )]TJ/F25 11.9552 Tf 11.955 0 Td [( )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(k 2 Proof. Since \050 x )]TJ/F25 11.9552 Tf 11.955 0 Td [( = \050 x isdecreasingfor x >> 0 ,wehave E [ U )]TJ/F26 7.9701 Tf 6.586 0 Td [( ]= 1 X i =0 w i e )]TJ/F40 7.9701 Tf 6.586 0 Td [(w i Z R + u )]TJ/F26 7.9701 Tf 6.586 0 Td [( 1 )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(k 2 + i 2 k 2 + i u k 2 + i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e )]TJ/F40 5.9776 Tf 7.782 3.259 Td [(u 2 # du =2 )]TJ/F26 7.9701 Tf 6.587 0 Td [( 1 X i =0 w i e )]TJ/F40 7.9701 Tf 6.587 0 Td [(w i )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(k 2 + i )]TJ/F25 11.9552 Tf 11.955 0 Td [( )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(k 2 + i 2 )]TJ/F26 7.9701 Tf 6.587 0 Td [( )]TJ/F30 11.9552 Tf 6.775 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.976 Td [(k 2 )]TJ/F25 11.9552 Tf 11.955 0 Td [( )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.977 Td [(k 2 Fix i 2f 1,2,..., r g .Given R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = 2 u i hasamultivariatenormaldistribution withidentitycovariancematrix.Itfollowsthat,conditionalon ,thedistributionof u T i R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i is 2 q i w ,forsome w whoseexpressionwewillnotrequire.Itfollows fromLemma16that,aslongas c 2 ,1 = 2 ,wehave E ~ h u T i R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i )]TJ/F40 7.9701 Tf 6.586 0 Td [(c i 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F40 7.9701 Tf 6.675 -4.571 Td [(q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 C 85

PAGE 86

Finally, E h )]TJ/F21 11.9552 Tf 5.479 -9.684 Td [(k u i k 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c i = )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max + u i c E h u T i )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max + u i I q i u i i )]TJ/F40 7.9701 Tf 6.587 0 Td [(c )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max + u i c E h u T i R i ~ Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 R T i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i )]TJ/F40 7.9701 Tf 6.586 0 Td [(c i )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( e d max + u i c 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(c )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(c )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(c )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(c )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.572 Td [(q i 2 )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(d c max c e + c u i wheretherstinequalityfollowsfromLemma14,thesecondfollowsfromC,andthe thirdfollowssince c 2 ,1 = 2 86

PAGE 87

APPENDIXD FINITEMOMENTCONDITIONS D.1ProperPriorCase PropositionD.1. Ifrank X = p ,thenforanypositiveinteger k : 1. E j i j k < 1 forall i 2f 1,2,..., p g 2. E j e j k < 1 and E j u j j k < 1 forall j 2f 1,2,..., r g Proof. Let k beapositiveinteger.Fix i 2f 1,2,..., p g .Itsufcestoshowthat E j i j 2 k < 1 .Toaccomplishthis,weshowthat E j i j 2 k j isboundedabovebyaconstant thatdoesnotdependon andtheniterateexpectations.Let i = E i j and i = p Var i j .UsingtheBinomial-Theorem,wehave E j i j 2 k j = E h i )]TJ/F22 11.9552 Tf 11.956 0 Td [( i + i 2 k j i = 2 k X j =0 2 k j E h )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( i )]TJ/F22 11.9552 Tf 11.955 0 Td [( i j j i 2 k )]TJ/F40 7.9701 Tf 6.587 0 Td [(j i Now, E )]TJ/F25 11.9552 Tf 10.461 -9.684 Td [( i )]TJ/F22 11.9552 Tf 11.955 0 Td [( i j j iszerowhen j isodd.When j iseven,wehave E h )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( i )]TJ/F22 11.9552 Tf 11.956 0 Td [( i j j i = j i 2 j 2 )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F40 7.9701 Tf 6.675 -4.65 Td [(j +1 2 )]TJ/F30 11.9552 Tf 6.774 9.684 Td [()]TJ/F23 7.9701 Tf 6.675 -4.977 Td [(1 2 Thus,itsufcestoshowthat 2 i and 2 i areboundedabovebyaconstant.By Lemma8,wehavethat 2 i k E j k 2 const Dene ~ T = e Z T Z + u and ~ M = I )]TJ/F25 11.9552 Tf 12.167 0 Td [( e Z ~ T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T .Itispossibletowritethecovariance matrixVar j as Var j = 2 6 4 ~ S )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 9.298 0 Td [( e ~ S )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z ~ T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F25 11.9552 Tf 9.299 0 Td [( e ~ T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T X ~ S )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ~ T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 + 2 e ~ T )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T X ~ S )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z ~ T )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 3 7 5 87

PAGE 88

where ~ S = e X T ~ MX + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 .Aslightgeneralizationoftheargumentsusedintheproofof Lemmas5and6showsthat 0 e X T ~ MX .Therefore, ~ S )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 whichimpliesthat 2 i tr )]TJ/F20 11.9552 Tf 5.479 -9.684 Td [(Var j = tr )]TJ/F22 11.9552 Tf 6.362 -7.027 Td [(~ S )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 tr Thisestablishes1.Theproofof2.issimple.Indeed, E [ j e j k ]= E E [ k e j ] = E \050 a e + N 2 + k b e + 1 2 k y )]TJ/F39 11.9552 Tf 11.956 0 Td [(W k 2 k \050 a e + N 2 # \050 a e + N 2 + k b k e \050 a e + N 2 < 1 andsimilarargumentcanbeusedfor u j j =1,..., r Thefollowingresultgivesconditionsunderwhichthemomentsofthecomponents of u arenite. PropositionD.2. Assumethatrank X = p andlet k beapositiveinteger.If E j )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e j k < 1 and E j )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i j k < 1 forall i 2f 1,2,..., r g then E j u jl j 2 k < 1 forall j 2f 1,2,..., r g and l 2f 1,2,..., q j g Proof. Dene u jl = E u jl j and u jl = p Var u jl j .ByLemma8,wehavethat u 2 jl k E u j k 2 const D andbyLemma7,wehave u 2 jl tr )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k e )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + k u r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i + s 2 max tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + where k e = tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + and k u = )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(q )]TJ/F20 11.9552 Tf 11.955 0 Td [(rank Z .Recallthewellknowninequality n X i =1 x i b n b )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 n X i =1 x b i 88

PAGE 89

where x i 0 for i =1,2,..., n ,and n and b arepositiveintegers.Hence,foranypositive integer b ,wehave )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( u 2 jl b k e )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + k u r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i + s 2 max tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T Z + b 3 b )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k b e )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e b + k b u r X i =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i b + const 3 b )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k b e )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e b +3 b )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k b u r b )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 r X i =1 )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i b + const whichimpliesthat E h )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( u 2 jl b i 3 b )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k b e E h )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e b i +3 b )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k b u r b )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 r X i =1 E h )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i b i + const D Now,Dimpliesthatthereexistsaconstant C ? suchthat max n 2f 0,1,2,..., k g u 2 k )]TJ/F40 7.9701 Tf 6.587 0 Td [(n jl C ? Finally,usingDanditeratingexpectations,wehave E j u jl j 2 k = E 2 k X n =0 2 k n E )]TJ/F39 11.9552 Tf 10.46 -9.684 Td [(u jl )]TJ/F22 11.9552 Tf 11.955 0 Td [( u jl n j u 2 k )]TJ/F40 7.9701 Tf 6.587 0 Td [(n jl # = E k X n =0 2 k 2 n E h )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(u jl )]TJ/F22 11.9552 Tf 11.955 0 Td [( u jl 2 n j i u 2 k )]TJ/F23 7.9701 Tf 6.587 0 Td [(2 n jl # C ? E k X n =0 2 k 2 n E h )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(u jl )]TJ/F22 11.9552 Tf 11.955 0 Td [( u jl 2 n j i # C ? k X n =0 2 k 2 n 2 n )]TJ/F30 11.9552 Tf 6.775 9.684 Td [()]TJ/F23 7.9701 Tf 6.675 -4.976 Td [(2 n +1 2 )]TJ/F30 11.9552 Tf 6.774 9.683 Td [()]TJ/F23 7.9701 Tf 6.675 -4.976 Td [(1 2 E h )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( u 2 jl n i C ? k X n =0 2 k 2 n 3 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k n e E h )]TJ/F25 11.9552 Tf 5.479 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e n i +3 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k n u r n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 r X i =1 E h )]TJ/F25 11.9552 Tf 5.48 -9.684 Td [( )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i n i + const < 1 wherethesecondequalityfollowsformthefactthat E )]TJ/F39 11.9552 Tf 10.461 -9.683 Td [(u jl )]TJ/F22 11.9552 Tf 11.955 0 Td [( u jl n j =0 forallodd n .Thiscompletestheproof. 89

PAGE 90

RecallfromChapter1thatthevariancecomponentsoftheGLMMaregivenby 2 e = )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e and 2 u i = )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i ,for i 2f 1,2,..., r g PropositionD.3. Let k > 0 1.If a e + N 2 > k then E j )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e j k < 1 2.For i 2f 1,2,..., r g if a i > k then E j )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u i j k < 1 Remark14. PropositionD.3canbeusedinconjunctionwithPropositionD.2toestablish momentconditionsforthecomponentsof u Proof. Suppose a e + N 2 > k andnotethat E [ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e k ]= 1 m y Z R p + q Z R r +1 + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e k f N y ; W )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e I f N )]TJ/F39 11.9552 Tf 5.48 -9.683 Td [(u ;0, )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u f G e ; a e b e r Y i =1 f G u i ; a i b i d # f N ; 0 d D AfterevaluatingtheinnerintegralinD,wehave E [ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e k ]= c y Z R p + q b e + k y )]TJ/F39 11.9552 Tf 11.955 0 Td [(W k 2 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [( a e + N 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(k r Y i =1 b i + k u i k 2 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [( a i + q i 2 f N ; 0 d where c y isaconstant.Now, E [ )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e k ] c y b a e + N 2 )]TJ/F40 7.9701 Tf 6.587 0 Td [(k e Z R p f N ; 0 d r Y i =1 Z R q i )]TJ/F39 11.9552 Tf 5.479 -9.683 Td [(b i + u T i u i 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [( a i + q i 2 du i c 0 y r Y i =1 Z R q i )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(b i + a i u T i u i a i 2 )]TJ/F23 7.9701 Tf 6.586 0 Td [( a i + q i 2 du i c 0 y r Y i =1 )]TJ/F22 11.9552 Tf 7.472 -9.684 Td [(min f a i b i g )]TJ/F23 5.9776 Tf 7.782 4.623 Td [(2 a i + q i 2 r Y i =1 Z R q i 1+ u T i u i 2 a i )]TJ/F23 5.9776 Tf 7.782 4.623 Td [(2 a i + q i 2 du i < 1 where c 0 y isaconstant.Thatthelastintegralisnitefollowsfromtheformofthe multivariateStudentdensityfunction.Thisestablishes1.Toestablish2.,wefollow 90

PAGE 91

similarstepsandobtain E [ )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u j k ] c y b a e + N 2 e Y i 6 = j Z R q i )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(b i + u T i u i 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [( a i + q i 2 du i Z R q j )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(b j + u T j u j 2 )]TJ/F23 7.9701 Tf 6.587 0 Td [([ a j )]TJ/F40 7.9701 Tf 6.586 0 Td [(k + q j 2 ] du j < 1 whenever a j > k .Theproofisnowcomplete. 91

PAGE 92

D.2ImproperPriorCase PropositionD.4. Let k beapositiveintegerandassumethatrank X = p .Assume furtherthattheproprietyconditionsinTheorem1.1hold.If E ~ j )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e j k < 1 and E ~ j )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u i j k < 1 forall i 2f 1,2,..., r g ,then E ~ j i j 2 k < 1 forall i 2f 1,2,..., p g and E ~ j u jl j 2 k < 1 forall j 2f 1,2,..., r g andall l 2f 1,2,..., q j g Proof. TheargumentusedhereissimilartotheoneusedintheproofofPropositionD.2. Wewillonlyprovidethedetailsofwhatisdifferentandleavetheremainingdetailstothe reader.Indeed,anapplicationofLemma15showsthat )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(E ~ i j 2 k E ~ j k 2 const and )]TJ/F39 11.9552 Tf 5.479 -9.684 Td [(E ~ u jl j 2 k E ~ u j k 2 const ByLemma14,wehave Var ~ i j tr )]TJ/F22 11.9552 Tf 5.48 -9.684 Td [( X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X T Z ~ Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Z T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 tr )]TJ/F22 11.9552 Tf 5.48 -9.683 Td [( X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + tr X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T Z )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T P ? Z + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 e + O T P ? D O )]TJ/F40 7.9701 Tf 14.182 5.26 Td [(r X j =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u j Z T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 = tr X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 + X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T Z Z T P ? Z + Z T X X T X )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + tr X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X T ZO T P ? D OZ T X X T X )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 r X j =1 )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 u j and Var ~ u jl j tr )]TJ/F22 11.9552 Tf 5.479 -9.684 Td [( Z T P ? Z + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 e + tr )]TJ/F39 11.9552 Tf 5.48 -9.684 Td [(O T P ? D O r X j =1 )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 u j Thefollowingresultprovideseasily-checkedconditionsfor E ~ j e j k < 1 and E ~ j u j j k < 1 j 2f 1,2,..., r g PropositionD.5. Let k > 0 andassumethatrank X = p .Assumefurtherthatthe proprietyconditionsinTheorem1.1hold. 92

PAGE 93

1.If N +2 a e + k > p )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 r X i =1 a i I ,0 a i holdsthen E ~ j e j k < 1 2.Let j 2f 1,2,..., r g .Ifthefollowingconditionshold,then E ~ j u j j k < 1 a a j + k < b j =0 or b j > 0 b q j +2 a j + k > q )]TJ/F39 11.9552 Tf 11.955 0 Td [(t c N +2 a e > p )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 a j + k I )]TJ/F40 7.9701 Tf 6.586 0 Td [(k a j )]TJ/F22 11.9552 Tf 11.955 0 Td [(2 P i 6 = j a j I ,0 a i Proof. Notethat E ~ j e j k = 1 ~ m y Z R r +1 + Z R p + q k e ~ j y d d = 1 ~ m y Z R r +1 + Z R p + q ~ k j y d d where ~ k isthesameas ~ exceptthat a e isreplacedby ~ a e = a e + k .Toestablish1.,we applyTheorem1.1twice.Theproofof2.issimilar. 93

PAGE 94

REFERENCES A SMUSSEN ,S.andG LYNN ,P.W..AnewproofofconvergenceofMCMCviathe ergodictheorem. Statistics&ProbabilityLetters 81 1482. B EDNORZ ,W.andL ATUSZY NSKI .AfewremarksonFixed-WidthOutputanalysis forMarkovChainMonteCarlobyJonesetal. JournaloftheAmericanStatistical Association 102 1485. C HAN ,K.S.andG EYER ,C.J..DiscussionofMarkovchainsforexploring posteriordistributions. AnnalsofStatistics 22 1747. D ANIELS ,M.J..Apriorforthevarianceinhierarchicalmodels. CanadianJournal ofStatistics 27 569. D IACONIS ,P.,K HARE ,K.andS ALOFF -C OSTE ,L..Gibbssampling,exponential familiesandorthogonalpolynomialswithdiscussion. StatisticalScience 151. F LEGAL ,J.M.,H ARAN ,M.andJ ONES ,G.L..MarkovchainMonteCarlo:Can wetrustthethirdsignicantgure? StatisticalScience 23 250. F LEGAL ,J.M.andJ ONES ,G.L..Batchmeansandspectralvarianceestimators inMarkovchainMonteCarlo. as 38 1034. F UZHEN andZ HANG MatrixTheory:BasicResultsandTechniques .Springer Verlag,NewYork. G ELFAND ,A.E.,S AHU ,S.K.andC ARLIN ,B.P..Efcientparametrisationsfor normallinearmixedmodels. Biometrika 82 479. G ELMAN ,A..Priordistributionsforvarianceparametersinhierarchicalmodels. BayesianAnalysis 1 515. H OBERT ,J.P.andC ASELLA ,G..TheeffectofimproperpriorsonGibbs samplinginhierarchicallinearmixedmodels. JournaloftheAmericanStatistical Association 91 1461. H OBERT ,J.P.andG EYER ,C.J..GeometricergodicityofGibbsandblockGibbs samplersforahierarchicalrandomeffectsmodel. JournalofMultivariateAnalysis 67 414. H OBERT ,J.P.,J ONES ,G.L.,P RESNELL ,B.andR OSENTHAL ,J.S..Onthe applicabilityofregenerativesimulationinMarkovchainMonteCarlo. Biometrika 89 731. H OBERT ,J.P.,J ONES ,G.L.andR OBERT ,C.P..UsingaMarkovchainto constructatractableapproximationofanintractableprobabilitydistribution. ScandinavianJournalofStatistics 33 37. 94

PAGE 95

H ODGES ,J.S..Somealgebraandgeometryforhierarchicalmodels,applied todiagnosticswithdiscussion. JournaloftheRoyalStatisticalSociety, SeriesB 60 497. I BRAGIMOV ,I.A.andL INNIK ,Y.V.. IndependentandStationarySequencesof RandomVariables .Wolters-Noordhoff,Groningen. J OHNSON ,A.A.andJ ONES ,G.L..GibbssamplingforaBayesianhierarchical generallinearmodel. ElectronicJournalofStatistics 4 313. J ONES ,G.L.,H ARAN ,M.,C AFFO ,B.S.andN EATH ,R..Fixed-widthoutput analysisforMarkovchainMonteCarlo. JournaloftheAmericanStatisticalAssociation 101 1537. J ONES ,G.L.andH OBERT ,J.P..SufcientburninforGibbssamplersfora hierarchicalrandomeffectsmodel. AnnalsofStatistics 32 784. K HARE ,K.andH OBERT ,J.P..Aspectralanalyticcomparisonoftrace-classdata augmentationalgorithmsandtheirsandwichvariants. AnnalsofStatistics toappear L ITTELL ,R.C.,M ILLIKEN ,G.A.,S TROUP ,W.W.andW OLFINGER ,R.D.. SAS SystemforMixedModels .SASInstituteInc.,Cary,NC. L IU ,J.S.,W ONG ,W.H.andK ONG ,A..CovariancestructureoftheGibbs samplerwithapplicationstocomparisonsofestimatorsandaugmentationschemes. Biometrika 81 27. M ARCHEV ,D.andH OBERT ,J.P..GeometricergodicityofvanDykandMeng's algorithmforthemultivariateStudent'stmodel. JournaloftheAmericanStatistical Association 99 228. M EYN ,S.P.andT WEEDIE ,R.L.. MarkovChainsandStochasticStability SpringerVerlag,London. M YKLAND ,P.,T IERNEY ,L.andY U ,B..RegenerationinMarkovchainsamplers. JournaloftheAmericanStatisticalAssociation 90 233. N UMMELIN ,E.. GeneralIrreducibleMarkovChainsandNon-negativeOperators CambridgeUniversityPress,London. P APASPILIOPOULOS ,O.andR OBERTS ,G.O..StabilityoftheGibbssamplerfor Bayesianhierarchicalmodels. AnnalsofStatistics 36 95. P APASPILIOPOULOS ,O.,R OBERTS ,G.O.andS K OLD ,M..Ageneralframework fortheparametrizationofhierarchicalmodels. StatisticalScience 22 59. R OBERTS ,G.O.andR OSENTHAL ,J.S..Markovchainsandde-initializing processes. ScandinavianJournalofStatistics 28 489. 95

PAGE 96

R OBERTS ,G.O.andR OSENTHAL ,J.S..GeneralstatespaceMarkovchains andMCMCalgorithms. ProbabilitySurveys 1 20. R OBERTS ,G.O.andT WEEDIE ,R.L..Boundsonregenerationtimesand convergenceratesforMarkovchains. StochasticProcessesandtheirApplications 80 211. R OBERTS ,G.O.andT WEEDIE ,R.L..Corrigendumtoboundsonregeneration timesandconvergenceratesforMarkovchains. StochasticProcessesandtheir Applications 91 337. R OSENTHAL ,J.S..MinorizationConditionsandConvergenceratesforMarkov ChainMonteCarlo. JournaloftheAmericanStatisticalAssociation 90 558. S EARLE ,S.R.,C ASELLA ,G.andM C C ULLOCH ,C.E.. VarianceComponents JohnWiley&Sons,NewYork. S UN ,D.,T SUTAKAWA ,R.andH E ,Z..Proprietyofposteriorswithimproperpriors inhierarchicallinearmixedmodels. StatisticaSinica 11 77. T AN ,A.andH OBERT ,J.P..BlockGibbssamplingforBayesianrandomeffects modelswithimproperpriors:Convergenceandregeneration. JournalofComputationalandGraphicalStatistics 18 861. Y U ,Y.andM ENG ,X.-L..Tocenterornottocenter:Thatisnotthequestion-An ancillarity-sufciencyinterweavingstrategyASISforboostingMCMCefciencywith discussion. JournalofComputationalandGraphicalStatistics 20 531. 96

PAGE 97

BIOGRAPHICALSKETCH JorgeCarlosRom anwasborninMayag uez,PuertoRico.In2003,hewasadmitted toUniversidaddePuertoRico-RecintodeR oPiedras.In2007,heearnedabachelor's degreeinmathematicsandjoinedtheDepartmentofStatisticsattheUniversityof Florida.Hethenreceivedhismaster'sdegreein2009andhisdoctoraldegreein2012. Aftergraduation,JorgeCarloswilljointheDepartmentofMathematicsatVanderbilt Universityasanon-tenuretrackassistantprofessor. 97