Asymptotic Variance Evaluations in Discrete Markov Chains

MISSING IMAGE

Material Information

Title:
Asymptotic Variance Evaluations in Discrete Markov Chains
Physical Description:
1 online resource (92 p.)
Language:
english
Creator:
Mukherjee,Nabanita
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Statistics
Committee Chair:
Casella, George
Committee Members:
Ghosh, Malay
Khare, Kshitij
Triplett, Eric W

Subjects

Subjects / Keywords:
asymptotic -- binomial -- carlo -- case -- chain -- contingency -- control -- correlated -- countable -- gibbs -- hastings -- markov -- metropolis -- monte -- peskun -- sampler -- space -- state -- table -- variance
Statistics -- Dissertations, Academic -- UF
Genre:
Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Markov chain Monte Carlo (MCMC) methods have become widely used in various statistical applications as well as in theoretical approaches to statistical computing.The motivation behind performing this computer based simulation method is due to the possible intractable nature of the distribution of the quantity of interest. There are many Markov chains that preserve the same stationary distribution. Thus for MCMC simulation purposes any one of the chains could be used. So, orderings defined on Markov state spaces with a specified stationary distribution guide us in choosing one Markov chain over another. One such ordering in Markov transition kernels in terms of off-diagonal elements of transition matrices was first introduced by Peskun in 1973. We provide different methods of constructing a ?better? (minimum asymptotic variance) Markov chain starting from a given chain, for discrete state space Markov chains. In constructing a better chain, preserving stationarity is very delicate, a Metropolis-Hastings algorithm step is used to preserve the right stationary distribution. Most importantly, we propose an optimal algorithm in the case of finite state Markov chains, which will sequentially move the mass from the diagonals to the off-diagonals of the transition matrix, preserving stationarity and hence an asymptotically efficient Markov chain in terms of Peskun ordering is obtained. Among other techniques, we use a matrix majorization technique and Metropolis within Gibbs sampling technique to construct a better Markov chain. We also provide some examples of the application of these results. We also use the Metropolis within Gibbs technique on a two stage Gibbs sampler, by replacing a conditional step by a Metropolis step and show that Metropolis with Gibbs chain is asymptotically more efficient than the ordinary two stage Gibbs sampler. We extend the methods for improving asymptotic variance to countable state space Markov chains and apply the result in generating contingency tables with fixed margins as a random walk. Finally, we also use the random walk approach in generating tables for a matched case-control data set, where a Metropolis step is used to preserve the correct stationary distribution.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Nabanita Mukherjee.
Thesis:
Thesis (Ph.D.)--University of Florida, 2011.
Local:
Adviser: Casella, George.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-08-31

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043149:00001


This item is only available as the following downloads:


Full Text

PAGE 1

ASYMPTOTICVARIANCEEVALUATIONSINDISCRETEMARKOVCHAINS By NABANITAMUKHERJEE ADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOL OFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENT OFTHEREQUIREMENTSFORTHEDEGREEOF DOCTOROFPHILOSOPHY UNIVERSITYOFFLORIDA 2011

PAGE 2

c 2011NabanitaMukherjee 2

PAGE 3

Tomyparents, ShilaandNandaGopalMukherjee 3

PAGE 4

ACKNOWLEDGMENTS IhadthegoodfortunetobeastudentintheDepartmentofStatisticsatthe UniversityofFlorida.Inthisdepartment,Icameinclosecontactwithsomeofthe preeminentstatisticiansofthedayandlearntalotfromthem.Ideeplyacknowledgethe tremendoushelp,encouragementandendlesssupportthatIreceivedfrommyadvisor Prof.GeorgeCasellathroughoutthehighsandlowsofdoingmyresearchwork.Henot onlytaughtmeStatisticsortheartofwritingpapersorsolvingproblems-heintroduced metothespiritofdiscoveryandthejoyoflearning,somethingthatwillstaywithme foreverandwouldmotivatemetomoveforwardinlife. SpecialthanksgoestoProf.KshitijKhare,amemberofcommittee,forhis tremendoussupport,guidanceandencouragement.Iwillneverforgetwhathehas doneformeandjustThanksissurelynotenough. Mydoctoralcommitteemembersdeserverecognitionandthanks.Iamthankful toProf.MalayGhoshandProf.EricTriplettfortheirguidanceandencouragement. However,thelistdoesnotendheresinceeachandeverymemberofthefacultyopened upnewdoorsformethroughwhichknowledgeowedandenrichedmealongtheway. Myendlessgratitudetoeachandeveryoneofthem.Ialsothankthestaffmembersof theDepartmentofStatisticsfortheirhelpineverypossibleway. Iamthankfultomyfriends,Dhiman,EugeniaandShibasishforbeingthereforme always.IhavespentsomequalitytimewiththeminGainesville. Finally,Icouldnotimagineembarkingonthisjourneywithouttheloveandsupport ofmyfamily.Myunendinggratitudetomyparentswhosesacrice,unconditionallove andblessingarealwayswithme,guidingmealongtheway.Mygratitudetowardsmy husband,Sounakforhisloveandsupport.Iamthankfultohimforwaitingallthese yearsforme.MysincerestgratitudetowardsmysistersChaitaliandArpitaandmy botherDeepak.Thereisoneperson,whoisalwaysthereforme,mybrother,Tapabrata, whoismybestestfriend.IjustwanttosaytohimthatIlovehimalot.Lastbutnot 4

PAGE 5

theleast,mynephewandniece,AmanandMithi,theyaremystressrelief.Iamvery fortunatetohavesuchagreatfamily.Mylifeismeaninglesswithoutthem. 5

PAGE 6

TABLEOFCONTENTS page ACKNOWLEDGMENTS..................................4 LISTOFTABLES......................................8 LISTOFFIGURES.....................................9 ABSTRACT.........................................10 CHAPTER 1INTRODUCTION...................................12 1.1Motivation....................................12 1.2Background...................................14 2FINITESTATESPACEMARKOVCHAINS.....................17 2.1Setup......................................17 2.2AlternateProofsofPeskunOrdering.....................19 2.2.1DiagonallyDominatedMatricesandPeskunOrdering.......20 2.2.2OntheApproachofTierney......................22 2.3ConstructionofImprovedTransitionMatrices................24 2.3.1DecreaseinDiagonals........................24 2.3.2ApplyingtheMetropolis-HastingsAlgorithm.............26 2.3.3OptimalTransitionMatrix........................29 2.3.4OptimalAlgorithm...........................31 2.3.5Majorization...............................37 2.3.6AnotherConstruction..........................40 3METROPOLISWITHINGIBBSSAMPLER....................42 3.1ImportanceofMetropoliswithinGibbsSampler...............42 3.2TwoStageGibbsSampler...........................43 3.3MetropoliswithintheRandomScanGibbsSampler.............46 3.4MetropoliswithintheTwoStageGibbsSampler...............48 4COUNTABLESTATESPACEMARKOVCHAINS.................56 4.1Setup......................................56 4.2ExpressionoftheLimitingVariance......................58 4.3SequentialConstruction............................62 4.4Examples....................................63 4.5GenerationofContingencyTables......................65 4.6TestforIndependence.............................69 6

PAGE 7

5APPLICATIONS...................................72 5.1BackgroundonType1DiabetesStudy....................72 5.2Motivation....................................72 5.2.1DataDescription............................73 5.2.2ImportantFeaturesoftheData....................73 5.2.3Analysis.................................74 5.2.4Fisher'sCombining...........................74 5.2.5Shortcomings..............................75 5.3ImprovedAnalysis...............................75 5.4Case-ControlDependencyandItsDistribution...............75 6CONCLUSIONSANDFUTURERESEARCH...................82 APPENDIX ATHEOREMSFROMCHAPTER2..........................85 A.1 ProofofTheorem2.2 .............................85 A.2 ProofofTheorem2.6 .............................86 BSOMECONCEPTSNEEDEDFORCHAPTER4.................87 B.1Denitions....................................87 B.2Theorems....................................87 B.3SizeoftheStateSpace............................88 REFERENCES.......................................89 BIOGRAPHICALSKETCH................................92 7

PAGE 8

LISTOFTABLES Table page 2-1Asymptoticvariancesatthethreestepsofthe optimalalgorithm ........37 3-1 m n multinomialsampling.............................49 3-2 3 3 multinomialsampling.............................52 4-1 2 4 contingencytable...............................66 4-2TransitionprobabilityforMarkovchains,MC1andMC2, a isthestayingstate, a 1 a 2 and a 3 arethetransitionstatesfrom a ...................68 4-3 2 rows4 columnscontingencytable........................68 4-4P-valuesandasymptoticvariances.........................68 4-5Contingencytablewith 2 rowsand 4 columns...................70 4-6P-valuesandasymptoticvariancesforaMarkovchainwith 2 and 4 moves..70 5-1 2 2 contingencytable...............................76 5-2Case-controlpair1withtotalsamplesize 27 ...................78 5-3Case-controlpair2withtotalsamplesize 25 ...................78 5-4P-valuesfordifferentvaluesof ..........................78 5-5Case-controlpairwithtotalsamplesize 48 ....................80 5-6P-valuesfordifferentvaluesof ,fromMonteCarloandMHmethods......80 5-7Case-controlpair1..................................81 5-8Case-controlpair2..................................81 5-9P-valuesfordifferentvaluesof fromMonteCarloandMHmethodsforTable 5-7and5-8......................................81 8

PAGE 9

LISTOFFIGURES Figure page 4-1ContingencytableswithxedmarginsasofTable4-1..............67 4-2SixothermovesinthecontingencyTable4-1...................67 9

PAGE 10

AbstractofDissertationPresentedtotheGraduateSchool oftheUniversityofFloridainPartialFulllmentofthe RequirementsfortheDegreeofDoctorofPhilosophy ASYMPTOTICVARIANCEEVALUATIONSINDISCRETEMARKOVCHAINS By NabanitaMukherjee August2011 Chair:GeorgeCasella Major:Statistics MarkovchainMonteCarloMCMCmethodshavebecomewidelyusedinvarious statisticalapplicationsaswellasintheoreticalapproachestostatisticalcomputing. Themotivationbehindperformingthiscomputerbasedsimulationmethodisduetothe possibleintractablenatureofthedistributionofthequantityofinterest.Therearemany Markovchainsthatpreservethesamestationarydistribution.ThusforMCMCsimulation purposesanyoneofthechainscouldbeused.So,orderingsdenedonMarkovstate spaceswithaspeciedstationarydistributionguideusinchoosingoneMarkovchain overanother.OnesuchorderinginMarkovtransitionkernelsintermsofoff-diagonal elementsoftransitionmatriceswasrstintroducedbyPeskunin1973. Weprovidedifferentmethodsofconstructingabetterminimumasymptotic varianceMarkovchainstartingfromagivenchain,fordiscretestatespaceMarkov chains.Inconstructingabetterchain,preservingstationarityisverydelicate,a Metropolis-Hastingsalgorithmstepisusedtopreservetherightstationarydistribution. Mostimportantly,weproposean optimalalgorithm inthecaseofnitestateMarkov chains,whichwillsequentiallymovethemassfromthediagonalstotheoff-diagonals ofthetransitionmatrix,preservingstationarityandhenceanasymptoticallyefcient MarkovchainintermsofPeskunorderingisobtained.Amongothertechniques,we useamatrixmajorizationtechniqueandMetropoliswithinGibbssamplingtechniqueto 10

PAGE 11

constructabetterMarkovchain.Wealsoprovidesomeexamplesoftheapplicationof theseresults. WealsousetheMetropoliswithinGibbstechniqueonatwostageGibbssampler, byreplacingaconditionalstepbyaMetropolisstepandshowthatMetropoliswithGibbs chainisasymptoticallymoreefcientthantheordinarytwostageGibbssampler. Weextendthemethodsforimprovingasymptoticvariancetocountablestatespace Markovchainsandapplytheresultingeneratingcontingencytableswithxedmargins asarandomwalk.Finally,wealsousetherandomwalkapproachingeneratingtables foramatchedcase-controldataset,whereaMetropolisstepisusedtopreservethe correctstationarydistribution. 11

PAGE 12

CHAPTER1 INTRODUCTION 1.1Motivation MarkovchainMonteCarloMCMCmethodshavebecomewidelyusedinvarious statisticalapplicationsaswellasintheoreticalapproachestostatisticalcomputing. Themotivationbehindperformingthiscomputerbasedsimulationmethodisdue tothepossibleintractablenatureofthedistributionofthequantityofinterest.For example,supposeweareinterestedinndingtheexpectedvalueofafunction f ,of arandomvariable X with astheprobabilitydistribution, E f X = .Tosimulate directlyfrom isoftennotfeasiblebecauseinmanycases isknownonlyuptoa normalizingconstant,orisdifculttosimulatefrom.MCMCmethodscouldbeusedto getanestimateof inwhichanirreducibleMarkovchain X 1 X 2 ,.. isrunwith asthe stationaryandlimitingdistribution.Theergodicaverage ^ n = 1 n P n i =1 f X i givesan estimateof AMarkovchainisdeterminedbyitstransitionmatrixorkernelincontinuousstate spaceswhichisdenedas P x A =P X n 2 A j X n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = x foreveryset A .Innite statespacemodels,MCMCmethodsareusedtogetanestimate ^ n whenthestate spaceislarge.ForanirreducibleMarkovchainwithnitestatespace,theMCMC estimatesobtainedarestronglyconsistentandasymptoticallynormallydistributed,and sotheefciencyoftheMCMCmethodscanbeevaluatedonthebasisoftheasymptotic varianceoftheestimateobtained.Theperformanceofatransitionmatrixusedfor MCMCsimulationcanalsobeevaluatedonthebasisofitsspeedofconvergence tostationarity.Buthere,wemainlyfocusourattentionontheperformanceofthe asymptoticvarianceof ^ n Foragiven ,therearemanyMarkovchainsthatpreservethesamestationary distribution.ThusforMCMCsimulationpurposesanyoneofthechainscouldbeused. So,orderingsdenedonMarkovstatespaceswithaspeciedstationarydistribution 12

PAGE 13

guideusinchoosingoneMarkovchainoveranother.Thischoicecanbemadeintwo ways,eitherbydetermininganewMCMCalgorithmwiththesamestationarydistribution orbyimprovingtheexistingchain.InChapter2,ourfocusisondifferentmethodsof constructingabetterMarkovchain.Weproposedifferentmethodsofconstructinga betterMarkovchainfromagivenchain.Thealgorithmproposedforgettingtheoptimal transitionmatrix,intermsofPeskunordering,doesnotrequiretheknowledgeofthe normalizingconstantof Asinthemethodofconstructingabetterchain,preserving stationarityisverydelicate,theMetropolis-HastingsMHalgorithmcomestotherescue inpreservingthestationarity. TheapplicationoftheMetropolis-HastingsalgorithmwithinaGibbssampler isbasedonreplacingasinglestepinaGibbssamplerbyaMetropolisstepwhich preservestherightstationarydistributionoftheMarkovchain.Liu1996showedthat theMetropolizedversionoftheGibbssamplerisasymptoticallymoreefcientthan theregularrandom-scanGibbssampler.Itistobenotedthattheproposaldistribution consideredbyLiurequiresexplicitknowledgeofthestationarydistribution,whichin practicemaybeavailableonlyuptoanormalizingconstant. Theintuitionbehindthe MetropolizedGibbssampler byLiuissamethatofPeskun originalideaofformingabetterMarkovchain,whichhassmallerprobabilityofstaying ataparticularstateandhencewillenablethechaintoexplorethestatespacemore rapidly,producingbetterestimates.InChapter3,weinvestigate,inatwostageGibbs samplerhowdoestheasymptoticvarianceoftheoverallMarkovchainbehave,by replacingaconditionalstepbyaMetropolisstep.WeshowfromMuller1991that thefullMarkovchainpreservestherightstationarydistribution.Also,weshowthat undertheproperchoiceoftheproposaldistribution,byreplacingaconditionalstepin atwostageGibbssampler,themarginalchainisasymptoticallymoreefcientthanthe ordinarytwostageGibbssampler. 13

PAGE 14

First,inthemethodsforconstructinganasymptoticallyefcientMarkovchains, weconsidernitestatespaceMarkovchains.Next,weextendthemethodsto countablyinnitestatespaceMarkovchainsinChapter4.Weprovideexpression fortheasymptoticvarianceoftheergodicaverageofaMarkovchain.Weprovide examples,wherethemethodsofconstructingabetterMarkovchaincanbeused.Most importantly,weimproveuponavariationofaDiaconis-GangollirandomwalkBezakova etal.2009;DiaconisandGangolli1995. InChapter5,wehavedatafromamatchedcase-controlstudyoftype1diabetes inchildren.Thesearematchedcase-controls,hencethepairsaredependent.Also, thepairsaremeasuredacrosstime-points,whichbringscorrelationacrosstime.Here atrst,wegeneratethecontingencytablesbytakingintoaccountthedependency betweencaseandcontrolpairforanygiventimepointbyusingarandomwalk approach.ThetherightstationarydistributionispreservedbyaMetropolis-Hastings algorithm. 1.2Background OrderingsinMarkovtransitionkernelsintermsofoff-diagonalelementsoftransition matriceswererstintroducedbyPeskun1973.Theauthorgaveasufcientcondition underwhichasymptoticefciencyofregular,reversibletransitionmatricescanbe compared,andshowedthattheMetropolis-Hastingsalgorithmdominatesaclass ofreversiblechainswithrespectto allwiththesameproposalandacceptance probability.ThePeskunorderingsuggeststhattheasymptoticvarianceof ^ n can bereducedbyappropriatelytransferringweightfromthediagonalelementstothe off-diagonalelementsofatransitionmatrix.Theintuitionbehindtheorderingisevident, Markovchainswithsmallerdiagonalelementshavesmallerprobabilityofremaining inanygivenstate,thustheyexplorethestatespacemoreefciently.Thissuggests animprovementinthesamplingofallstateswhichresultsintheimprovementin theefciencyof ^ n FollowingPeskun,Frigessietal.1992constructedtheoptimal 14

PAGE 15

transitionmatrixrelativetoPeskunbuttheconstructionrequirestheexactknowledge of while,inmostMCMCapplications, isknownonlyuptoanormalizingconstant, whichisindeedthebenetofusingMCMCmethods. Tierney1998extendedthePeskunorderingfromnitestatesspacestogeneral statespaces.TheauthorgaveasimpleandelegantproofofPeskunorderingina moregeneralframeworkandappliedPeskunorderingincomparingtheimprovement inefciencyoftwoapproachestousingmixturesoftransitionkernels.Mira2001 gavenecessaryandsufcientconditionsfortheimprovementinefciencyofMarkov chainsinnitestatespacesforbothreversibleaswellasfornon-reversiblechains. MiraandGeyer1999alsoextendedtheconditionsfortheimprovementinefciency tocontinuousstatesspaces.Miraalsointroducedsomeotherconceptsintheordering Markovchaintheory,thelikecovarianceordering,whichisweaker,inthesensethat itworksforspecialformsof f Inthecontextofnon-reversibleMarkovchains,Mira introducedtheconceptofsouthwestordering.Thissouthwestorderingguarantees orderinginthelag-onecovariancesoftransitionmatrices.Theonlydifferencebetween thisandthecovarianceorderingisthatheretheorderingonthelag-onecovariances hastoholdonlyforthespecicfunctionofinterest f ,whichisassumedtobemonotone, notforallfunctions f TheapplicationoftheMetropolis-HastingsalgorithmwithinaGibbssamplerwas rstintroducedbyMuller1991.AnotherimportantpaperinthistopicisMuller1993. ThemethodisusedwhenaconditionaldistributioninaGibbsstepcannotbeeasily simulatedfrom.Then,inthatsituation,theconditionalGibbsstepisreplacedbya MetropolisstepwhichpreservestherightstationarydistributionoftheMarkovchain. Liu1996considersrandomscanGibbssamplersandshowthattheMetropolized versionoftheGibbssamplerisasymptoticallymoreefcientthantheregularrandom scanGibbssampler.Liuproposesaproposaldistributionwhichworksundertheexplicit knowledgeoftheconditionaldistribution,whichinpracticemaynotbeknown. 15

PAGE 16

Generatingcontingencytablewithxedmargins,withaMarkovchain,though havingnitestatespace,canbeconsideredasacountablyinnitestatessincethe statespaceisenormous.Togeneratecontingencytableswithxedmargins,thereare differentapproaches,amongthemsomeprominentonesare: 1.Generatingthecontingencytablesasarandomwalk.Thismethodisdiscussed inseveralpapersbyDiaconisandGangolli1995andDiaconisandSturmfels 1998. 2.Generatingthecontingencytablesasamultinomialsampling.Thismethodis discussedinButlerandSutton1998. Togeneratecontingencytableswithxedmarginstakinginaccountthedependency inpairwisecasecontrolstructures,itistobenotedthatthestationarydistributionfora particularpairisacorrelatedbinomialdistributionBiswasandHwang2002.Thetables aregeneratedwitharandomwalkasaproposaldistributionthatconditionsonone margin,thenaMetropolisstepisusedtopreservethecorrectstationarydistribution. 16

PAGE 17

CHAPTER2 FINITESTATESPACEMARKOVCHAINS 2.1Setup InaMarkovchainwithadiscretestatespace S ,supposeweareinterestedin ndingtheestimateoftheexpectationofsomenonconstantfunction f denedonthe states S = f 1,2,..., m g Innotation, =E f = m X i =1 f i i where = 1 2 ,..., m isaprobabilitydistributionsuchthat i > 0 forall i The Markovchainofinteresthereisaregular 1 chain,i.e.,withnotransientstates,orin otherwordsthechainisirreduciblewithperiod 1 i.e.,aperiodic.Let P = f p i j g bea transitionprobabilitymatrixoftheMarkovchain.Atransitionmatrix P isregularifand onlyifforsome n P n hasnozeroentries.Thus,nomatterwheretheprocessstarts, afterasufcientlapseoftimeitispossibletobeinanystate. Foraniteirreducible,aperiodicMarkovchaindeterminedbythetransitionmatrix P withstationarydistribution ,thepowers P n approachaprobabilitymatrix ,where eachrowof isthesameprobabilityvector, .Thus,foraregularMarkovchainthe stationarydistribution isthelimitingdistribution.Thematrix willbecalledthe `limitingmatrix' for P .Theinversematrix Z = f I )]TJ/F22 11.9552 Tf 12.082 0 Td [( P )]TJ/F48 11.9552 Tf 12.082 0 Td [( g )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 iscalledthe `fundamental matrix' forthesame P Let f bethe 1 m columnvector f = f f ,..., f m 0 and D =diag f i g 1 i m FromKemenyandSnell1969,theasymptoticexpressionforthevarianceofthe estimate, ^ n = P n i =1 1 n f X i whichisindependentoftheinitialdistributionoftheinitial 1 ForaniteMarkovchain,regularisequivalenttoergodic. 17

PAGE 18

state X 1 isgivenas v f , P =lim n !1 n Var n X i =1 1 n f X i # = f 0 f D Z + D Z 0 )]TJ/F48 11.9552 Tf 11.955 0 Td [(D )]TJ/F48 11.9552 Tf 11.955 0 Td [(D g f = f 0 f 2 D Z )]TJ/F48 11.9552 Tf 11.955 0 Td [(D )]TJ/F48 11.9552 Tf 11.955 0 Td [(D g f = f 0 D f 2 Z )]TJ/F48 11.9552 Tf 11.955 0 Td [(I )]TJ/F48 11.9552 Tf 11.955 0 Td [( g f ThePeskunorderingofasymptoticvariancesconsiderstransitionmatricesthatare reversible.AMarkovchainwithinitialdistribution andtransitionmatrix P issaidtobe areversibleifandonlyifitsatisesthedetailedbalancedcondition,i.e., i p i j = j p j i issatised.Ifthedetailedbalancerelationholds,then X i i p i j = X i j p j i = j forall j 2S soforreversibleMarkovchains, isalwaysastationarydistribution.Reversibilityof P withrespectto ensuresthat D P issymmetric,whichguaranteesthatitseigenvalues arereal.Moreover, D Z isalsosymmetric. Let Q = f q i j g beanothertransitionmatrixwiththesamestationarydistribution Moreover,toprovethetheoremofPeskunordering, Q mustsatisfythereversibility condition2.LetusnowdiscussthetheoremofPeskunorderingwhichisgiven below. Theorem2.1 Peskunordering. Supposethateachoftheirreducibletransitionmatrices P and Q satisfythereversibilitycondition2forthesamestationarydistribution .Ifeachoftheoff-diagonalelementsof P isgreaterthanorequaltothecorresponding off-diagonalelementsof Q ,innotation, P Q ,thenfortheestimate ^ n = P n i =1 1 n f X i v f , P v f , Q 18

PAGE 19

Fortheproofoftheabovetheoremofordering,Peskundifferentiatedthevarianceterm v f , P withrespecttotheoff-diagonalelementsof P ,andfoundthatthederivativeis lessthanzero,whichimpliesthat v f , P isadecreasingfunctionintheoff-diagonal elements.AlternativeandsimplerproofsofthePeskunorderingareavailable[Section 2.2].Tierney1998gaveonesuchproof.Moreover,wealsoproposeanalternative proofofPeskunorderingbyconsideringthediagonaldominancestructureof D P )]TJ/F48 11.9552 Tf 12.006 0 Td [(Q inthePeskunsenerio.BeforegoingtothealternateproofsofthePeskunordering,for thesakeofcompletenessandclaritywereproveTheorem2.2fromMira2001.To provethetheoremfromMira2001,weneedthedenitionbelow: Denition2.1. Let P Q betransitionmatricesbothwithstationarydistribution ,then P isatleastasuniformlyefcientas Q if v f , P v f , Q Theorem2.2 Mira2001. Supposethateachoftheirreducibletransitionmatrices P and Q satisesthereversibilitycondition2forthesamestationarydistribution Thenthefollowingstatementsareequivalent: 1. Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P isapositivesemi-denitematrix; 2. Cov f Q Cov f P 8 f ; 3. P isuniformlymoreefcientthan Q TheproofoftheTheorem2.2isgiveninAppendixA.1.Now,fromTheorem2.2, wenoticethatunderPeskun'sconditioni.e,wheneachoftheoff-diagonalelements of P isgreaterthanorequaltothecorrespondingoff-diagonalelementsof Q ,toshow theorderingintheasymptoticvarianceofMarkovchains,itisenoughtoshowthat D Q )]TJ/F48 11.9552 Tf 12.093 0 Td [(P ispositivesemi-denite.Keepingthisinmind,inthenextsectionweprovide someofthealternateproofsofPeskunordering. 2.2AlternateProofsofPeskunOrdering Asdiscussedearlier,theoriginalproofofthePeskunorderingconsidered differentiationofthevarianceterm v f , P withrespecttotheoff-diagonalelementsof P takingintoaccountthereversibilityoftheMarkovchaincorrespondingtothetransition 19

PAGE 20

matrix P Inthissection,twoalternateproofsofthePeskunorderingaregiven.Therst approachtakesintoaccountthediagonaldominantstructureof D Q )]TJ/F48 11.9552 Tf 12.749 0 Td [(P andthe secondapproachissimilartotheapproachofTierney1998whichisaveryelegant proof. 2.2.1DiagonallyDominatedMatricesandPeskunOrdering Inthissectionwewilldiscussthediagonaldominatedstructureof D Q )]TJ/F48 11.9552 Tf 12.047 0 Td [(P under thePeskunscenarioandforthatweneedsomedenitionsandtheoremsfrommatrix analysis,whichareasfollowsSaad2003: Denition2.2 Diagonallydominatedmatrix. Let A beasquarematrixofordernwith entries a ij whicharerealorcomplex.Then, A issaidtobe diagonallydominant if j a ii j n X j 6 = i =1 j a ij j i =1,2,..., n Denition2.3 Gershgorindiscs. Let A beasquarematrixofordernwithentries a ij whicharerealorcomplex.Aroundeveryelement a ii onthediagonalofthematrix, wedrawacirclewithradiusequaltothesumofthenormsoftheotherelementsonthe samerow P j 6 = i j a ij j .SuchcirclesarecalledGershgorindiscs. Denition2.4 Hermitianmatrix. AHermitianmatrixorself-adjointmatrix A isa squarematrixwithcomplexentriesandisequaltoitsownconjugatetranspose,i.e.,the elementinthe i th rowand j th columnisequaltothecomplexconjugateoftheelementin the j th rowand i th column,forallindices i and j ,i.e., a ij = a ji ThefollowingtwotheoremsareneededforthealternativeproofofPeskunordering: Theorem2.3 Gershgorin'scircletheorem. Let A beasquarematrixofordern withentries a ij whicharerealorcomplex.Theneveryeigenvalueof A liesinoneofits Gershgorindiscs. Proof. Let beaneigenvalueof A and x = x j beitscorrespondingeigenvector. Choose i suchthat j x i j =max j j x j j .Since x cannotbe 0 j x i j > 0 .Now,considering the i th componentof Ax = x ,wehave )]TJ/F39 11.9552 Tf 12.156 0 Td [(a ii x i = P j 6 = i a ij x j .Takingthenormonboth 20

PAGE 21

sidesgives, j )]TJ/F39 11.9552 Tf 11.955 0 Td [(a ii j = j X j 6 = i a ij x j = x i jj X j 6 = i a ij j wherethelastinequalityisvalidbecause x j x i 1 for i 6 = j Theorem2.4. AnHermitiandiagonallydominatedmatrixwithrealnon-negativediagonal entriesispositivesemi-denite. Proof. Let A beanHermitiandiagonallydominantmatrixwithrealnonnegativediagonal entries;thenitseigenvaluesarereal.Now,byGershgorin'scircletheorem,Theorem2.3, foreacheigenvalueanindex i existssuchthat: 2 [ a ii )]TJ/F30 11.9552 Tf 11.955 11.358 Td [(X j 6 = i j a ij j a ii + X j 6 = i j a ij j ], whichimplies,bydenitionofdiagonallydominance, 0. Now,usingthepropertyofdiagonaldominance,analternativeproofforPeskunordering isgivenbelow: AlternateproofofTheorem2.1. Firstnotethatreversibilityof P and Q ensuresthe symmetryof D Q and D P whichguaranteesthattheeigenvaluesof H = D Q )]TJ/F48 11.9552 Tf 12.32 0 Td [(P arereal.Let h ij bethe i j th entryof H Now,for h ij 0 ,i.e,anoff-diagonalelement of P isbiggerthan Q ,whichimpliesthatdiagonalelementsof H arepositive.Inthat case, h ii = P j 6 = i j h ij j ,whichshowsthat H isadiagonallydominatedmatrix.Hence,by Gershgorin'scircletheorem,Theorem2.3,foreacheigenvalueof H anindex i exists suchthattheeigenvaluewilllieintheinterval [ h ii )]TJ/F30 11.9552 Tf 12.487 8.966 Td [(P j 6 = i j h ij j h ii + P i 6 = j j h ij j ] .Inother words,anyeigenvalueof H willlieintheinterval [0,max i h ii + P i 6 = j j h ij j ] .Therefore, H is apositivesemi-denitematrixbyTheorem2.4,whichimpliesPeskunordering. 21

PAGE 22

2.2.2OntheApproachofTierney AnotheralternativeproofofPeskunorderingcanbeobtainedbyconsideringthe followingidentity: Letxandybetwovariables.Thenwehave, xy = x + y 2 )]TJ/F22 11.9552 Tf 11.955 0 Td [( x )]TJ/F39 11.9552 Tf 11.955 0 Td [(y 2 4 Theorem2.5. Suppose P and Q aretwotransitionmatricesbothwithstationary distribution .Let f = f 1 f 2 ,... f m 0 and H = D Q )]TJ/F48 11.9552 Tf 11.956 0 Td [(P whichhasentries f h ij g then f 0 D Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P f = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # = 1 2 m X i =1 m X j =1 h ij f i + f j 2 # Proof. Atrstnotethat P m j =1 h ij =0 and P m i =1 h ij =0 ,since P and Q aretransition matrices,theirrowsumis 1 andhavestationarydistribution whichgivestheidentity j = P m i =1 i p i j .Toprovetherstequalityof2,letusconsider f 0 D Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P f = f 0 Hf = m X i =1 m X j =1 h ij f i f j = m X i =1 m X j =1 h ij f i + f j 2 )]TJ/F22 11.9552 Tf 11.955 0 Td [( f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 4 = 1 4 m X i =1 m X j =1 h ij f 2 i + m X j =1 m X i =1 h ij f 2 j +2 m X i =1 m X j =1 f i f j h ij # )]TJ/F22 11.9552 Tf 20.456 8.088 Td [(1 4 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 = 1 2 m X i =1 m X j =1 f i f j h ij # )]TJ/F22 11.9552 Tf 13.15 8.088 Td [(1 4 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 = )]TJ/F22 11.9552 Tf 10.494 8.087 Td [(1 2 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # wherethethirdequalityisobtainedbyconsideringtheidentity2andthefthequality isobtainedbyconsideringthefactthat P m j =1 h ij =0 and P m i =1 h ij =0. 22

PAGE 23

Toprovethesecondequalityconditionof2,letusproceedfrom2 f 0 D Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P f = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 m X i =1 m X j =1 h ij f i + f j 2 )]TJ/F22 11.9552 Tf 11.955 0 Td [(4 f i f j # = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 m X i =1 m X j =1 h ij f i + f j 2 # + 2 m X i =1 m X j =1 f i f j # Takingtheterm, 2 P m i =1 P m j =1 f i f j fromtheright-hand-sideof2totheleft-hand-side, wegetthesecondequalityconditionof2. Boththerepresentationsof f 0 D Q )]TJ/F48 11.9552 Tf 12.033 0 Td [(P f inTheorem2.5areveryhelpfulinproving somemajorresults.Therstequalityof2isusedtoprovideanalternateproof ofPeskunorderingandthesecondequalityisusedintheconstructionofabetter Markovchain,discussedinSection2.3.Tierney1998usedtherstrepresentationin provingthatoff-diagonaldominanceimpliesvariancedominanceinthecontinuouscase. ConsideringtherstrepresentationofTheorem2.5,anotherproofofPeskunorderingis discussedbelow: AlternateproofofTheorem2.1. ConsideringtherstequalityofTheorem2.5, f 0 D Q )]TJ/F48 11.9552 Tf 11.956 0 Td [(P f = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # Now,forthequadraticform f 0 D Q )]TJ/F48 11.9552 Tf 12.678 0 Td [(P f tobepositivesemi-denite,wemusthave )]TJ/F39 11.9552 Tf 9.298 0 Td [(h ij 0 whichfollowssinceevery p i j q i j i 6 = j .ThereforePeskunordering holdsandhence,theasymptoticefciencyoftheMarkovchaindeterminedbythe transitionmatrix P isgreaterthanthatof Q Remark. Peskungavetheproofoforderingintheasymptoticvarianceintermsof off-diagonalentries,whichworksonlyforreversibletransitionmatrices.Butfrom 2,itisobservedthatonly )]TJ/F39 11.9552 Tf 9.299 0 Td [(h ij 0 isenoughtoshowthat D Q )]TJ/F48 11.9552 Tf 12.621 0 Td [(P ispositive semi-denite.IfweseetheproofofTheorem2.2,itistobenotedthattherstparti.e., 23

PAGE 24

Q )]TJ/F48 11.9552 Tf 12.611 0 Td [(P positivesemi-denite Cov f Q Cov f P isdonewithouttakinginto considerationthereversibilityassumptionof P and Q Thus,thecovarianceordering holdsfornon-reversibleMarkovchains.Inprovingthevarianceordering,theL owner orderingisneeded,whichonlyworksforsymmetricmatrices.Thisimpliesreversibilityof P and Q isneededinthePeskunordering. 2.3ConstructionofImprovedTransitionMatrices Wehavementionedintheprevioussectionthatthesecondrepresentationof Theorem2.5canbeusedtoconstructabettertransitionmatrixintermsofefciency. Herewehaveshownthatforareversibletransitionmatrix P ,movingmassfromthe diagonalelementstotheoff-diagonalelementsgivesabetterMarkovchaininterms ofefciency,aproofofwhichisgiveninthissection.Thisnotiongivesusanintuition abouttheoptimalwayofmovingmassfromthediagonalelementstotheoff-diagonal elementsofareversibletransitionmatrix, P whichgivesbetteraMarkovchainin termsofefciency,preservingstationarity.TheMetropolis-HastingsMHalgorithmis employedintheconstructionofbetterMarkovchains.Moreover,wealsoprovidean optimalalgorithmbyapplyingtheMHalgorithmateachsteptoobtainatransitionmatrix whichisoptimalinthesensethatitcannotbeimprovedintermsofPeskunordering. 2.3.1DecreaseinDiagonals ToconstructabetterMarkovchain,thesecondrepresentationofTheorem2.5is usedwhichisdiscussedindetailinthissection.Let P = f p i j g and Q = f q i j g be reversibletransitionmatricesbothwithstationarydistribution .Consideringthesecond equalityofTheorem2.5, f 0 D Q )]TJ/F48 11.9552 Tf 11.956 0 Td [(P f = 1 2 m X i =1 m X j =1 i f i + f j 2 f q i j )]TJ/F39 11.9552 Tf 11.956 0 Td [(p i j g # Now,supposeweget P from Q bytransferringmassfromdiagonalselementsto off-diagonalelements,thenweexpect P tobeabettertransitionmatrix,intermsof efciency.Toillustratetheabovediscussion,considerthefollowingtheorem. 24

PAGE 25

Theorem2.6. Thequantity X i j f i + f j 2 i p i j =4 X i f 2 i i p i i + X j 6 = i f i + f j 2 i p i j isincreasingineach p k k k =1,2,..., m TheproofofTheorem2.6isgiveninAppendixA.2.Theorem2.6willthenimplythatif adiagonalelementdecreases,wehaveabetterMarkovchainandhenceweconclude thattransitionmatricescanbeimprovedintermsofefciencyonlybymovingmassfrom diagonalelementstooff-diagonalelements.NotethatTheorem2.6onlyallowsusto improveagiventransitionmatrix P butwecannotcompareanytwoarbitrarytransition matrices P and Q ,inwhicheverydiagonalentryof P iscomponent-wisebiggerthan Q whichthefollowingexampleillustrates. Example2.1. Considerthetransitionmatrices Q = 0 B B B B @ 1 = 21 = 41 = 4 1 = 41 = 21 = 4 1 = 41 = 41 = 2 1 C C C C A and P = 0 B B B B @ 11 = 403 = 1017 = 40 3 = 101 = 21 = 5 17 = 401 = 53 = 8 1 C C C C A bothwithstationarydistribution = f 1 = 3,1 = 3,1 = 3 g Nowconsiderthedifference, Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P = 0 B B B B @ 3 = 40 )]TJ/F22 11.9552 Tf 9.298 0 Td [(1 = 60 )]TJ/F22 11.9552 Tf 9.299 0 Td [(7 = 120 )]TJ/F22 11.9552 Tf 9.299 0 Td [(1 = 6001 = 60 )]TJ/F22 11.9552 Tf 9.298 0 Td [(7 = 1201 = 601 = 24 1 C C C C A Theeigenvaluesof D Q )]TJ/F48 11.9552 Tf 12.409 0 Td [(P are f 0.123419, )]TJ/F22 11.9552 Tf 9.299 0 Td [(0.00675208,0 g showingthe Q )]TJ/F48 11.9552 Tf 12.409 0 Td [(P is notpositivesemi-denite.Butnotethateverydiagonalentryof P iscomponent-wise smallerthan Q butthePeskunconditionabouttheoff-diagonalsisnotsatised.So,for anytwoarbitraryregulartransitionmatricessatisfyingreversibility,andpossessingsame stationarydistribution weconcludethathavingonlydiagonalentriessmallerisnot enoughtoimprovetheefciencyofaMarkovchain. 25

PAGE 26

2.3.2ApplyingtheMetropolis-HastingsAlgorithm AlltheresultsinSection2.2gavetheconclusion D Q )]TJ/F48 11.9552 Tf 11.41 0 Td [(P 0. Thisonlyresultsin thevarianceinequalityif P and Q havethesamestationarydistribution,whichmustbe veriedbeforecomparingvarianceoftheMarkovchains.Intheproblemofconstructing abetterMarkovchainfromagivenchain,preservingstationarityateachstepisensured bytheMetropolis-Hastingsalgorithm,byconstructingaMetropolistransitionmatrixwith thesamestationarydistributionasthatofgiventransitionmatrix. TheMetropolis-HastingsMHalgorithmwasrstproposedinthepaperby Metropolisetal.1953.Hastings1970extendedthealgorithmtoamoregeneral setup.TheMHalgorithmisamethodofconstructingaregularreversibletransition matrixwithaspeciedstationarydistribution Themethodtakesaproposaltransition matrixgivenbyanabsolutelyarbitrarystochasticmatrix G andanacceptancefunction in [0,1] andthentransformsintoastochasticmatrix Q thatisreversibleandstationary withrespectto BilleraandDiaconis2001notedthatiftheproposalchain G is irreducible,then Q isirreducible. Improvingefciencybymetropolization .Let G = f g i j g betheproposal transitionmatrixwithsomearbitrarystationarydistribution .Wecanconstructa transitionmatrix Q = f q i j g withstationarydistribution bytheMHalgorithm: Theacceptanceprobabilityis i j =min j g j i i g i j ,1 ThetransitionmatrixfortheMetropolis-Hastingschainisgivenby q i j = i j g i j +1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(r i i j where, i j = 1 i = j 0 i 6 = j and r i = X j 0 i j 0 g i j 0 = X j 0 min j 0 g j 0 i i g i j 0 ,1 g i j 0 Diagonalelementsof Q : q i i = g i i + )]TJ/F39 11.9552 Tf 11.955 0 Td [(r i i =1,2,..., m 26

PAGE 27

Off-diagonalelementsof Q : q i j =min j g j i i g i j ,1 g i j i 6 = j =1,2,..., m Thealgorithmdependsonlyontheratios j = i and g j i = g i j andistherefore, independentofnormalizingconstants.The j = i partencouragesthechaintomove toareasofhigh probability.The g j i = g i j partexpressesreluctancetoenter blackholes.Thus,ifthechainiscurrentlyatstate i ,acandidatepoint j ischosen withprobability g i j Thismovetostate j isacceptedwithacceptanceprobability i j andwiththeremainingprobabilitythechainremainswhereitis.Thequantity 1 )]TJ/F39 11.9552 Tf 13.043 0 Td [(r i istheoverallprobabilitythatthechainremainsat i Moreover,ifweknow thatthereisareversibletransitionmatrix P with asthestationarydistribution,then p i j = p j i = j = i .So,inthatcaseratio j = i canbereplacedby p i j = p j i Now,weknowthatpreservingstationarityoftheMetropolis-Hastingsalgorithm isneededincomparingtwodifferentMarkovchains.HereweusetheMHalgorithm intheconstructionofabetterMarkovchain,inasensethattheresultingchainwillbe asymptoticallymoreefcientthanthegivenchain.Thefollowingtheoremillustratesthe statement. Theorem2.7. Suppose P = f p i j g isanirreducible,reversibletransitionmatrix withstationarydistribution and G = f g i j g anytransitionmatrixwithanarbitrary stationarydistributionsuchthat G P .Also,suppose Q = f q i j g isthetransition matrixobtainedbyapplyingtheMetropolis-Hastingsalgorithmto P with G asthe proposaldistribution.Then Q isasymptoticallymoreefcientthan P Proof. Wehave G P ,i.e., g i j p i j i 6 = j ,implies g i i p i i .Since P is irreducible,thisimplies G isirreducible.Alsonotethat G isirreducibleguaranteesthe irreducibilityof Q BilleraandDiaconis2001. Consideroff-diagonalentriesof G and P g i j p i j = i g i j i p i j 27

PAGE 28

Also, g j i p j i = j g j i j p j i = j g j i i p i j = j g j i i p i j wherethethirdinequalityisobtainedbyassumingreversibilityof P Now,off-diagonal entriesof Q are q i j =min j g j i i g i j ,1 g i j =min j g j i i g i j Combining2and2,weget q i j p i j i 6 = j ,i.e., Q hasbigger off-diagonalentriesthan P .Hence,fromPeskun'stheoremwehave Q isasymptotically moreefcientthan P Example2.2. Considerthe 3 3 reversibletransitionmatrix P = 0 B B B B @ 29 = 481 = 3235 = 96 1 = 247 = 123 = 8 7 = 249 = 4029 = 60 1 C C C C A withstationarydistribution = f 1 = 3,1 = 4,5 = 12 g Letusconsidertheproposaltransition matrix G = 0 B B B B @ 01 = 32 = 3 1 = 302 = 3 8 = 157 = 150 1 C C C C A suchthateveryoff-diagonalentriesof G isgreaterthan P .With G astheproposal distribution,theMetropolis-Hastingstransitionmatrix Q andhence D P )]TJ/F48 11.9552 Tf 12.922 0 Td [(Q are obtainedas Q = 0 B B B B @ 1 = 121 = 42 = 3 1 = 302 = 3 8 = 152 = 51 = 15 1 C C C C A D P )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q = 0 B B B B @ 25 = 144 )]TJ/F22 11.9552 Tf 9.299 0 Td [(7 = 96 )]TJ/F22 11.9552 Tf 9.298 0 Td [(29 = 288 )]TJ/F22 11.9552 Tf 9.299 0 Td [(7 = 967 = 48 )]TJ/F22 11.9552 Tf 9.298 0 Td [(7 = 96 )]TJ/F22 11.9552 Tf 9.299 0 Td [(29 = 288 )]TJ/F22 11.9552 Tf 9.299 0 Td [(7 = 9625 = 144 1 C C C C A 28

PAGE 29

Notethattheoff-diagonalentriesof Q arebiggerthanthecorrespondingoff-diagonal entriesof P .Moreover,theeigenvaluesof D P )]TJ/F48 11.9552 Tf 12.391 0 Td [(Q are f 7 = 288,7 = 32,0 g Hence,by Peskun'stheorem Q isasymptotiallymoreefcientthan P Corollary2.7.1. Suppose P = f p i j g isanirreducible,reversibletransitionmatrixwith stationarydistribution and E = f e i j g beamatrixsuchthat e i j 0, i 6 = j and P j e i j =0, suchthat p i i )]TJ/F39 11.9552 Tf 12.047 0 Td [(e i i > 0. ThenbyTheorem2.7,Metropolizationgives atransitionmatrix Q withproposaltransitionmatrix G = P + E Q isasymptoticallymore efcientthan P 2.3.3OptimalTransitionMatrix Let P = f p i j g beareversibleregulartransitionmatrixwithstationarydistribution .AsmentionedinSection2.3.1,theefciencyintermsofvariancesof P canbe improvedbytransferringmassfromthediagonalentries p i i 'stotheoff-diagonal entries p i j 's.Foraparticularrow i ,let ij bethefractionof p i i thatwouldbe addedto p i j 'sfor j =1,2,..., m .Alsoletusassumethat 0 ij 1 .Thetransfer ofmassfromdiagonalstotheoff-diagonalsisdoneinsuchawaythattheresulting matrixpreservesthepropertiesofatransitionmatrix.Forthistohold,wemusthave P m j 6 = i =1 ij =1. TocomparetwodifferenttransitionmatricesinPeskun'ssense,thematrices shouldhavethesamestationarydistributionandbereversible.Thetwomaindifculties intransferringmassfromthediagonalstotheoff-diagonalsarethattheresulting transitionmatrix a maynotpreservethesamestationarydistribution,and b may notbereversible.TheMetropolis-Hastingsalgorithmsolvesthesetwoproblemsvery well.Buthereweproposeanalgorithmwhichalsosolvesthesetwodifcultiesof transferringmassfromdiagonalstooff-diagonals,sothattransitionmatricesobtained arecomparableinPeskun'ssenseandimprovementintermsofinefciencyisobtained whencomparedtoagiventransitionmatrix P .Themotivationforthe optimalalgorithm isobtainedfromtheMetropolis-Hastingsalgorithm. 29

PAGE 30

Letusconsideraproposaltransitionmatrix G suchthat g i i =0, g i j = p i j + ij p i i where 0 ij 1 and P m j 6 = i =1 ij =1. Itistobenotedthatthechoice of ij ismadeinsuchawaysothat G isindeedatransitionmatrix.ThentheMetropolis transitionmatrix Q hasoff-diagonalentriesas q i j =min j g j i i g i j = 1 i min j g j i i g i j = 1 i min [ j f p j i + ji p j j g i f p i j + ij p i i g ] = p i j + 1 i min [ j ji p j j i ij p i i ] wherethelastequalityisobtainedbyreversibilityassumptionof P ,i.e., i p i j = j p j i Thediagonalentriesareobtainedas q i i = p i i + )]TJ/F39 11.9552 Tf 11.955 0 Td [(r i = p i i +[1 )]TJ/F30 11.9552 Tf 11.956 11.358 Td [(X j 0 f p i j 0 + 1 i min [ ij 0 j 0 p j 0 j 0 ij 0 i p i i ] g ] = p i i )]TJ/F30 11.9552 Tf 11.955 11.358 Td [(X j 0 1 i min [ j 0 i j 0 p j 0 j 0 ij 0 i p i i ] Letusassume ij = ji Theassumptionof ij = ji ismadesothatthereversibility assumptionissatisedandmoresothisaddsequalproportionofmassto i j th and j i th entryof D P sothattherecannotbeanyfurtherincreaseinmassthatcanbe madetothe i j th and j i th entryof D P separately.Inaway,wecansaythatthe assumptionmakesanoptimaldistributionofmasstotheoff-diagonals,whichisevident inthe optimalalgorithm discussedinSection2.3.4. Therefore, q i j = p i j + 1 i ij min [ j p j j i p i i ] i 6 = j 30

PAGE 31

and q i i = p i i )]TJ/F30 11.9552 Tf 11.955 11.357 Td [(X j 0 ij 0 1 i min [ j 0 p j 0 j 0 i p i i ] TheformofEquations2and2motivatetoproposeanalgorithmwhich movemassfromadiagonalentrytotheoff-diagonalentriessothattheresultingmatrix obtainedatthenalstepissuchthatitcannotbeimprovedinPeskun'ssense.Forthe optimalalgorithm,weassume 0 < ij < 1 and P m j 6 = i =1 ij =1. Itistobenotedthatateverystepofthealgorithm, i p i i 'sareorderedinsucha waythattheresultingdiagonalentriesaregreaterthanorequaltozero.Also,wehave toensureateachstepofthealgorithmthattheresultingmatrixisatransitionmatrixand isreversible. Thealgorithmusedtoreachaoptimaltransitionmatrixisexplainedindetailbelow. 2.3.4OptimalAlgorithm Let P bethegiventransitionmatrixwithstationarydistribution whichwewantto improveandlet P k = f p k i j g betheoptimaltransitionmatrixobtainedattheendof Step k Step1:Arrange i p i i 's, i =1,2,..., m inthenon-decreasingorder.Wecanassume withoutlossofgeneralitybyrelabelling 1 p ,1 2 p ,2 .... m p m m Let p i j = p i j + 1 i ij 1 p ,1, i 6 = j =1,2,..., m p i i = p i i 1 )]TJ/F25 11.9552 Tf 13.151 8.088 Td [( 1 i p ,1 p i i i =1,2,..., m Since, 1 p ,1 isthesmallest,therefore,from2and2,wehave p ,1= p ,1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p ,1=0. p j = p j + 1 j p ,1, j =2,..., m 31

PAGE 32

p i ,1= p i ,1+ 1 i 1 i p ,1, i =2,..., m Notethat p i i = p i i )]TJ/F26 7.9701 Tf 13.826 4.884 Td [( 1 i p ,1 isgreaterthan0,becauseoftheorderingsin i p i i 's.Aftertherstiteration,thetransitionmatrixisoftheform 2 6 6 6 6 6 6 6 4 0 p ,2+ 12 p ,1... p m + 1 m p ,1 p ,1+ 1 2 12 p ,1 p ,2 )]TJ/F26 7.9701 Tf 13.151 4.884 Td [( 1 2 p ,1... p m + 1 2 2 m p ,1 . . . . . . p m ,1+ 1 m 1 m p ,1 p m ,2+ 1 m 2 m p ,1... p m m )]TJ/F26 7.9701 Tf 14.124 4.883 Td [( 1 m p ,1 3 7 7 7 7 7 7 7 5 Thus P obtainedattheendoftherststepofthealgorithmisoptimalinthe distributionofmassesoftherstrowandcolumn,i.e.,wehave P P Now, if 1 p ,1= 2 p ,2=....= m p m m thenoptimalityisreached.Else,if 1 p ,1 < 2 p ,2, thenwegotostep2. Step2:Next,westartwiththe ,2 th elementof P andconsiderthe m )]TJ/F22 11.9552 Tf 11.196 0 Td [(1 m )]TJ/F22 11.9552 Tf 11.196 0 Td [(1 sub-matrixof P ,notconsideringrstrowandrstcolumnof P Let ij i 6 = j = 2,3,.. m bethefractionofweightgiventothe i j th entryinthesub-matrixsuchthat ij = ji and P j 0 6 = i ij 0 =1, i =2,3,.., m Let i p i i = i p i i )]TJ/F25 11.9552 Tf 11.955 0 Td [( 1 p ,1, i =2,3,.., m whicharealsoordered,i.e., 2 p ,2 3 p ,3 ... m p m m Let p i j = p i j + 1 i ij 2 p ,2, i 6 = j =2,..., m p i i = p i i 1 )]TJ/F25 11.9552 Tf 13.151 8.088 Td [( 2 i p ,2 p i i i =2,..., m Since, 2 p ,2 isthesmallestdiagonalelementinthe m )]TJ/F22 11.9552 Tf 11.31 0 Td [(1 m )]TJ/F22 11.9552 Tf 11.309 0 Td [(1 sub-matrixof P ,thereforefrom2and2,wehavetheentriesof P as p ,2= p ,2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p ,2=0 32

PAGE 33

p j = p j + 2 j p ,2, j =3,4,..., m p i ,2= p i ,2+ 2 i p ,2, i =3,4,..., m Alsonotethat p i i = p i i )]TJ/F26 7.9701 Tf 13.871 4.883 Td [( 2 i p ,2, i =3,4,.., m isgreaterthanzero, becauseoftheorderingsof i p i i 's.Thus, P isthenewsecondstageoptimal matrixinasensethatthemassdistributionofrsttworowsandcolumnsareoptimalin Peskun'ssense. Thus,wehaveattheendofthesecondstage P P P Now,if 2 p ,2=....= m p m m thenoptimalityisreached.Else,if 2 p ,2 < 3 p ,3, thenwecontinuerepeatingStep2tillwegetanoptimal transitionmatrix, P opt whichcannotbeimprovedinPeskun'ssense. Notethefollowing: 1Aftertherststepofthealgorithm,wehaveshownthatdiagonalentriesaregreater thanzerowhichfollowsduetotheorderingin i p i i 's.Since P j 6 = i ij =1 andweare addingafractionofthemassofthediagonalstotheoff-diagonals,itfollowsthatthe resultingsumoftherowsofthetransitionmatrixis 1 .Hence, P obtainedattheend oftherststepisindeedatransitionmatrix.Thisargumentworksforeverystepofthe algorithm,whichguaranteesthatthematrixobtainedateachstepofthealgorithmisa transitionmatrix. 2Ateachstep,weensurethatthetransitionmatrixobtainedisreversible.Thiscanbe explainedbyconsideringtheelementsofthetransitionmatrixateachstep.Aftertheend ofStep k ,theoff-diagonalentriesareoftheform p k )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 i j + k i k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 ij p k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k k weobservethat i [ p k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 i j + k i k )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ij p k )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k k ]= j [ p k )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 j i + k j k )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 ij p k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 k k ] 33

PAGE 34

, i p k i j = j p k j i Hence, P k isreversiblewithrespectto atStep k Remark. 1.For 1 p ,1 < 2 p ,2 < .... < m p m m theabovealgorithmneeds m )]TJ/F22 11.9552 Tf 12.291 0 Td [(1 steps,togettheoptimaltransitionmatrix P opt 2.For 1 p ,1= 2 p ,2=....= m p m m theabovealgorithmneeds 1 step,to gettheoptimaltransitionmatrix P opt 3.Foranytworows i and i +1 ,if i p i i = i +1 p i +1, i +1, theimprovementin termsofoptimalmassallocationfor i th and i +1 th rowsaredoneinasinglestep. Since, q i i = p i i )]TJ/F30 11.9552 Tf 12.403 8.966 Td [(P j 0 ij 0 p i i =0 and q i +1, i +1= p i +1, i +1 )]TJ/F30 11.9552 Tf -413.638 -5.479 Td [(P j 0 i +1 j 0 p i +1, i +1=0. 4.If m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 p m )]TJ/F22 11.9552 Tf 11.955 0 Td [(1, m )]TJ/F22 11.9552 Tf 11.955 0 Td [(1= m p m m the P opt obtainedhasalldiagonalentries 0. 5.If m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 p m )]TJ/F22 11.9552 Tf 12.164 0 Td [(1, m )]TJ/F22 11.9552 Tf 12.164 0 Td [(1 < m p m m thenthelastnon-zerodiagonalentryof P opt is p m m )]TJ/F26 7.9701 Tf 13.151 5.714 Td [( m )]TJ/F23 5.9776 Tf 5.756 0 Td [(1 m p m )]TJ/F22 11.9552 Tf 11.955 0 Td [(1, m )]TJ/F22 11.9552 Tf 11.955 0 Td [(1. 6.Theminimumnumberofstepsneededtoreach P opt dependsonthepositionof theequalitiesandinequalities. Forexample,consider m =3. Therearefourpossibleorderings: 1 p ,1= 2 p ,2= 3 p ,3, 1 p ,1= 2 p ,2 < 3 p ,3, 1 p ,1 < 2 p ,2= 3 p ,3 and 1 p ,1 < 2 p ,2 < 3 p ,3. Inthersttwocases,thenumberof stepsneededis 1 andinthelasttwocases,thenumberofstepsneededis 2. 7.Notethat P opt obtainedisnotunique.Forvariouschoicesof ij P opt isobtained. Thematricesobtainedatthenalstageofthealgorithmarenotcomparableinthe Peskun'ssense.Thesecanbeexplainedwiththehelpoftheresultbelow: Theorem2.8. Iftwotransitionmatriceswiththesamestationarydistributionhavethe samediagonalentries,thensuchmatricesarenotcomparableinPeskun'ssense. Proof. Let P and Q betwotransitionmatricesbothwithstationarydistribution Ifthe diagonalentriesaresame,thenatleastoneoftheoffdiagonalentriesof P willbeless thanthatof Q sincetherowsumsareequal.Let H = D Q )]TJ/F48 11.9552 Tf 12.59 0 Td [(P suchthat H has entries f h ij g Letusconsider h 12 > 0 f 1 =1 and f 2 = )]TJ/F22 11.9552 Tf 9.299 0 Td [(1, and f i =0, i =3,4,.. m then f 0 D Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P f = )]TJ/F22 11.9552 Tf 10.494 8.087 Td [(1 2 m X i =1 m X j =1 h ij f i )]TJ/F39 11.9552 Tf 11.956 0 Td [(f j 2 # = )]TJ/F39 11.9552 Tf 9.298 0 Td [(h 12 < 0, 34

PAGE 35

whichshowsthatthe f 0 D Q )]TJ/F48 11.9552 Tf 11.665 0 Td [(P f isnotpositivesemi-deniteandhencecompletesthe proof. TwoImportantfeaturesofthe optimalalgorithm .1ForareversibleMarkov chain,the optimalalgorithm appliedtothetransitionmatrixdoesnotrequirethe knowledgeofthestationarydistribution.Thisisbecauseofthedetailedbalance condition2,whichgivestheratio i j = p j i p i j neededforthealgorithm.2The optimalalgorithm canalsobeappliedtothetransition matricesfornon-reversibleMarkovchains,butinthatcase,wecomparethesematrices intermsoftheircovariancesi.e.,covarianceordering. Letusillustratetheabove optimalalgorithm withthehelpofanexamplebelow. Example2.3. ConsideragaintheExample2.2.Thereversibletransitionmatrixwith stationarydistribution = f 1 = 3,1 = 4,5 = 12 g is P = 0 B B B B @ 29 = 481 = 3235 = 96 1 = 247 = 123 = 8 7 = 249 = 4029 = 60 1 C C C C A Wehave 1 p ,1=29 = 144 2 p ,2=7 = 48 and 3 p ,3=29 = 144 Thestepsofthealgorithmareasfollows: Step1:Wehavethefollowingordering 2 p ,2 < 1 p ,1= 3 p ,3. Relabelingthestates, as and as ,wegetthetransitionmatrix P as P = 0 B B B B @ 7 = 121 = 243 = 8 1 = 3229 = 4835 = 96 9 = 407 = 2429 = 60 1 C C C C A 35

PAGE 36

withstationarydistribution = f 1 =1 = 4, 2 =1 = 3, 3 =5 = 12 g Wechoose ij =1 = 2, for j 6 = i =1,2,3. Theelementsof P are p ,1= p ,1 )]TJ/F25 11.9552 Tf 14.038 8.088 Td [( 1 1 p ,1=0. p ,2= p ,2+ 12 p ,1=1 = 3. p ,3= p ,3+ 13 p ,1=2 = 3 p ,1= p ,1+ 1 2 12 p ,1=1 = 4. p ,2= p ,2 )]TJ/F25 11.9552 Tf 13.151 8.087 Td [( 1 2 p ,1=1 = 6. p ,3= p ,3+ 1 2 23 p ,1=7 = 12. p ,1= p ,1+ 1 3 13 p ,1=2 = 5. p ,2= p ,2+ 1 3 23 p ,1=2 = 15. p ,3= p ,3 )]TJ/F25 11.9552 Tf 13.151 8.088 Td [( 1 3 p ,1=2 = 15. Thus,attheendoftherststep, P isobtainedas P = 0 B B B B @ 01 = 32 = 3 1 = 41 = 67 = 12 2 = 57 = 152 = 15 1 C C C C A Step2:Next,westartwiththe ,2 th elementof P andconsiderthe 2 2 sub-matrix of P ,andrepeatstep1withthesubmatrix.Thus,attheendofthesecondstep, P opt isobtainedas P opt = 0 B B B B @ 01 = 32 = 3 1 = 403 = 4 2 = 53 = 50 1 C C C C A FromExample2.3,itistobenotedthatchoice ij isnotunique.Hence,wecan havedifferent P opt fordifferent ij 's.Thenumberofstepsneededtoreach P opt is 2 ,in thiscase.Thecomputedvalueofvariancescorrespondingtotransitionmatrices P P 36

PAGE 37

and P opt arecalculatedfromtheexpressionoftheasymptoticvariancein2taking f =, )]TJ/F22 11.9552 Tf 9.299 0 Td [(1,0 .ThevariancescomputedaregiveninTable2-1: Table2-1.Asymptoticvariancesatthethreestepsofthe optimalalgorithm StepTransitionmatrixAsymptoticvariance 1 P 2.01674 2 P 0.381173 3 P opt 0.313763 Thus,weobservefromTable2-1that v f , P > v f , P > v f , P opt 2.3.5Majorization Inlinearalgebraanditsapplication,vectormajorizationisamuchstudiedconcept. If a b 2 R m Wesay a majorizes b denotedby a b if P k i =1 a [ i ] P k i =1 b [ i ] k = 1,2,... m and P m i =1 a i = P m i =1 b i ,where a [ i ] and b [ i ] denotethe i th largestcomponent of a and b respectively.Severalgeneralizationsofthisconcepthavebeenintroduced andonesuchdirectionistodenemajorizationformatrices.InthebookbyMarshall andOlkin1979,onesuchdenitionofmatrixmajorizationisgiven,whichsaysthat A majorizes B ,i,e., A B ifthereisadoublystochasticmatrix X ,suchthat AX = B Asquarematrix X isdoublystochasticiftheentriesofthematrixarenonnegativereal numbers,witheachrowandcolumnsummingto1.Thisismotivatedbythetheoremof Hardy-LittlewoodandPolya,whichsaysthatfor a b 2 R m a b ifandonlyif aX = b Dahl1999gaveadenitionofmatrixmajorizationwhichisafurthergeneralizationof thedenitiongivenbyMarshallandOlkin,considering X tobearow-stochasticmatrix. Arowstochasticmatrixisamatrixeachofwhoserowsconsistsofnonnegativereal numbers,witheachrowsummingto1. Denition2.5 Rowmajorization. A majorizes B i.e., A B ifandonlyifthereexists arowstochasticmatrix X ,suchthat AX = B ThoughDahl'soriginaldentiondoesnotneedthematricestobesquarebutinthis sectionsincewewillbedealingwithtransitionmatrices,wemustconsider A and B to besquarematrices.Notethattocomparevariancescorrespondingtodifferenttransition 37

PAGE 38

matriceswiththesamestationarydistribution,therow-stochasticmatrix X involved inthemajorizationshouldbeatransitionmatrixwiththesamestationarydistribution aseachofthemajorizedtransitionmatrices.Moreover,ifthereversibilityconditionis satisedbybothby A and X ,itisnottypicallytruethattheirproduct AX willalsobe reversible.AcommonexampleofthisiswhenGibbssamplingupdatesareappliedto eachcomponentofastateinsomedeterministicorder. Beforegoingtotheactualtheoremconnectingmajorizationandasymptotic efciencyorderingintransitionmatrices,weneedthefollowingresult. Result2.1. Suppose P isatransitionmatrixwithstationarydistribution .Then I )]TJ/F48 11.9552 Tf 12.077 0 Td [(P is apositivesemi-denitematrix. Proof. Taking Q = I inTheorem2.5,wehave f 0 D I )]TJ/F48 11.9552 Tf 11.955 0 Td [(P f = 1 2 m X i =1 m X j =1 i p i j f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # 0. Hence, I )]TJ/F48 11.9552 Tf 11.955 0 Td [(P isapositivesemi-denitematrix. Theorem2.9. Supposethateachoftheregulartransitionmatrices P and Q have thesamestationarydistribution with P beingreversibleandpositivesemi-denite. Suppose PQ isregularandreversible,then PQ isatleastasuniformlyefcientas P Proof. First,letusobservethefollowing:1 PQ isalsoatransitionmatrix. 1 0 PQ = 1 0 Q = 1 0 .2 PQ ishasthesamestationarydistribution : PQ = Q = .Since PQ isregular,i.e., PQ n hasnozeroentries.Thisregularityensuresthatthestationary distribution isthelimitingdistributionof PQ Now,wewillshowthat f 0 D P )]TJ/F48 11.9552 Tf 12.145 0 Td [(PQ f ispositivesemi-deniteandbyTheorem2.2 thiswillimply PQ isatleastasuniformlyefcientas P Since P isreversible, D P isasymmetricmatrixandpositivesemi-denite, thereforewecanwrite D P = BB 0 forsomenon-negativedenitematrix B Thus, 38

PAGE 39

forany f ,thequadraticform f 0 D P )]TJ/F48 11.9552 Tf 11.955 0 Td [(PQ f canbewrittenas f 0 D P )]TJ/F48 11.9552 Tf 11.955 0 Td [(PQ f = f 0 D P I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q f = f 0 BB 0 I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q f Weknowthateigenvaluesof BB 0 I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q arethesameastheeigenvaluesof B 0 I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q B Thus,forany y y 0 B 0 I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q By = x 0 I )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q x where By = x ,whichisnecessarilypositivesemi-denitebyResult2.1. Remark. IntheTheorem2.9,ifwemakefurtherassumptionregarding Q tobepositive semi-denite.Then PQ isasymptoticallymoreefcientthanboth P and Q Thefollowingexampleillustratestheabovetheorem. Example2.4. Considerthetransitionmatrices P = 0 B B B B @ 1 = 21 = 41 = 4 1 = 41 = 21 = 4 1 = 41 = 41 = 2 1 C C C C A and Q = 0 B B B B @ 2 = 53 = 103 = 10 3 = 102 = 53 = 10 3 = 103 = 102 = 5 1 C C C C A bothwithstationarydistribution = f 1 = 3,1 = 3,1 = 3 g Notethat P and Q areboth regularandreversibletransitionmatrices.Moreover, P haseigenvalues f 1,1 = 4,1 = 4 g whichshows P ispositivedenite.Now, PQ = 0 B B B B @ 7 = 2013 = 4013 = 40 13 = 407 = 2013 = 40 13 = 4013 = 407 = 20 1 C C C C A whichisregularandhasthesamestationarydistribution Also, P )]TJ/F48 11.9552 Tf 13.522 0 Td [(PQ has eigenvalues f 9 = 40,9 = 40,0 g Hence,byTheorem2.9, PQ isasymptoticallymoreefcient than P 39

PAGE 40

2.3.6AnotherConstruction Theorem2.10. Supposethatairreducible,reversibletransitionmatrix P withstationary distribution hasallnon-zerodiagonalentries.Then Q = P )]TJ/F39 11.9552 Tf 11.955 0 Td [(k I 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k with p i i > k and p i j < 1 )]TJ/F39 11.9552 Tf 11.956 0 Td [(k ,isasymptoticallymoreefcientthan P Proof. Firstnotethefollowingpropertiesof Q :1 Q isatransitionmatrix. 1 0 Q = 1 0 P )]TJ/F39 11.9552 Tf 11.955 0 Td [(k I 1 )]TJ/F39 11.9552 Tf 11.956 0 Td [(k = 1 0 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k = 1 0 2 q i j isgreaterthanzerowhere p i j isgreaterthanzero,therefore, Q also representsanergodicchain.Thus,thenewchain Q isalsoaregularMarkovchain. 3 Q hasstationarydistribution Q = P )]TJ/F39 11.9552 Tf 11.955 0 Td [(k I 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k = )]TJ/F39 11.9552 Tf 11.955 0 Td [(k 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k = Consider, f 0 D P )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q f = f 0 D P )]TJ/F48 11.9552 Tf 13.15 8.088 Td [(P )]TJ/F39 11.9552 Tf 11.955 0 Td [(k I 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k f = f 0 D [ P )]TJ/F39 11.9552 Tf 11.955 0 Td [(k )]TJ/F48 11.9552 Tf 11.955 0 Td [(P + k I ] 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k f = k 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(k f 0 D I )]TJ/F48 11.9552 Tf 11.956 0 Td [(P f whichispositivesemi-denitebyResult2.1.Hence,byTheorem2.2, Q isasymptotically moreefcientthan P Example2.5. Considerthetransitionmatrix P = 0 B B B B @ 0.50.250.25 0.250.50.25 0.250.250.5 1 C C C C A 40

PAGE 41

withstationarydistribution = f 1 = 3,1 = 3,1 = 3 g Inthisexample,letustake k =0.5. Usingtheaboveconstruction,thetransitionmatrix Q isobtainedas Q = 0 B B B B @ 00.50.5 0.500.5 0.50.50 1 C C C C A whichhasalsostationarydistribution = f 1 = 3,1 = 3,1 = 3 g andisasymptoticallymore efcientthan P 41

PAGE 42

CHAPTER3 METROPOLISWITHINGIBBSSAMPLER 3.1ImportanceofMetropoliswithinGibbsSampler TheapplicationoftheMetropolis-HastingsalgorithmwithinaGibbssamplerwas introducedbyMuller1991.AnotherimportantpaperinthisdirectionisMuller1993. ThemethodisusedwhenaconditionaldistributioninaGibbsstepcannotbeeasily simulatedfrom.Then,inthatsituation,theconditionalGibbsstepisreplacedbya Metropolisstepwhichmakesoursimulationprocedureeasier.Letuselaborateonthis. Suppose X = X 1 X 2 ,..., X m isamultivariaterandomvariablewithstationary distribution x .Ouraimistosamplefromthedistribution x bytheGibbssampling procedurewhichcreatesatransitionfrom X t = X t 1 X t 2 ,..., X t m to X t +1 = X t +1 1 X t +1 2 ,..., X t +1 m .OneofthemostpopulartechniquesinthetheoryofMCMCis thesystematicscanGibbssampler,whichisdescribedas:Let X t bethecurrentstate ofthechain,then X t +1 isobtainedasfollows: Atrst, X t +1 1 isobtainedasadrawfromtheconditionaldensity, 1 x 1 j x t 2 x t 3 ,..., x t m Next,weobtain X t +1 2 asadrawfromtheconditionaldensity, 2 x 2 j x t +1 1 x t 3 ,..., x t m andcontinueduntil X t +1 m isdrawnfromtheconditionaldensity m x m j x t +1 1 x t +1 2 ,..., x t +1 m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Therefore,thetransitiondistributionofthesystematicscanGibbssamplerisgivenby K x t x t +1 = 1 x t +1 1 j x t 2 x t 3 ,..., x t m 2 x t +1 2 j x t +1 1 x t 3 ,..., x t m ... m x t +1 m j x t +1 1 x t +1 2 ,..., x t +1 m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 42

PAGE 43

Let x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i bethevector x withthe i th componentdeleted.Atthe i th stepofthesystematic scanGibbssampler,if i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i isdifculttosimulatefrom,thenMetropoliswithinGibbs MiGprocedurereplacestheconditionalstepasfollows: Let x t t +1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(i = x t +1 1 x t +1 2 ,... x t +1 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 x t i ,..., x t m where x t i isthe i th componentof x atthe t th iterationoftheMarkovchain.Let g i betheproposaldistributionatthe i th step. Thenthemovefrom x t i to x t +1 i occurswithprobability K x t i x t +1 i j x t t +1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(i =min i x t +1 i j x t t +1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(i g i x t i x t +1 i j x t t +1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(i g i x t +1 i x t i j x t t +1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(i i x t i j x t t +1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(i ,1 g i x t i x t +1 i j x t t +1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(i TheMiGalgorithmisperfectlyvalidsinceitpreservedthecorrectstationarydistribution x .Toprovethisfact,weconsideratwostageGibbssamplersincetheproofforthe multi-stageGibbssamplerexactlyfollowsfromtwostageGibbssetup.Moreover,inthis chapter,ouraimistoinvestigatethemarginaldistributionofatwostageGibbssampler andseehowtheasymptoticvarianceoftheergodicaverageofanestimatorbehaves whenaconditionalstepinatwostageGibbssamplerisreplacedbyaMetropolisstep. 3.2TwoStageGibbsSampler Let X and Y betworandomvariablesdenedonthediscretespaces X and Y respectively.ConsiderthefollowingtwostageGibbssampler, X j Y x j y and Y j X y j x withupdatesdoneintheorder y x y 0 x 0 Thefullchain X Y X 0 Y 0 hastransitionkernel, K G x y x 0 y 0 = x 0 j y 0 y 0 j x 43

PAGE 44

andstationarydistribution x y .AMetropolisstepinaGibbssamplerreplacesany conditionalstepbyaMetropolis-Hastingsalgorithmstep.IntheaboveGibbssampler, insteadofgenerating X 0 from X j Y ,wegenerate X 0 fromtheproposaldensity g .Let g x x 0 j y denotetheconditionalprobabilityofmovingfrom x to x 0 given y Thisgivesthe MiGtransitionkernelas K MiG x x 0 j y 0 =min x 0 j y 0 g x x 0 j y 0 g x 0 x j y 0 x j y 0 ,1 g x x 0 j y 0 + )]TJ/F39 11.9552 Tf 11.955 0 Td [(r x j y 0 x x 0 j y 0 where x x 0 j y 0 isaDiracdeltameasure,whichis 1 when x = x 0 and r x j y 0 = X x 0 min x 0 j y 0 g x x 0 j y 0 g x 0 x j y 0 x j y 0 ,1 g x x 0 j y 0 Alsonotethat,hereweareusing g x x 0 j y 0 whichisimportant,sinceweneedsome transitionprobabilityinthe X spaceconditioningon y 0 Toshowthat K MiG y y 0 j x satisesdetailedbalancecondition,i.e.,toshow K MiG x x 0 j y 0 x j y 0 = K MiG x 0 x j y 0 x 0 j y 0 wenotethatfor x 6 = x 0 K MiG x x 0 j y 0 x j y 0 =min x 0 j y 0 g x x 0 j y 0 g x 0 x j y 0 x j y 0 ,1 g x x 0 j y 0 x j y 0 =min f x 0 j y 0 g x 0 x j y 0 g x x 0 j y 0 x j y 0 g =min x j y 0 g x 0 x j y 0 g x x 0 j y 0 x 0 j y 0 ,1 g x x 0 j y 0 x 0 j y 0 = K MiG x 0 x j y 0 x 0 j y 0 andfor x = x 0 trivially )]TJ/F39 11.9552 Tf 11.955 0 Td [(r x j y 0 x x 0 j y 0 x j y 0 = )]TJ/F39 11.9552 Tf 11.955 0 Td [(r x 0 j y 0 x 0 x j y 0 x 0 j y 0 whichtogetherestablishthedetailedbalanceconditionoftheMetropolis-Hastingschain with x j y asthestationarydistribution. 44

PAGE 45

Now,wehavethenewGibbstransitionkernelas K MiG x y x 0 y 0 = K MiG x x 0 j y 0 y 0 j x andthestationarydistributionofthischainis x y since X x 2X X y 2Y K MiG x y x 0 y 0 x y = X x 2X X y 2Y K MiG x x 0 j y 0 y 0 j x x y = X x 2X K MiG x x 0 j y 0 y 0 j x x = X x 2X K MiG x x 0 j y 0 x y 0 = X x 2X K MiG x x 0 j y 0 x j y 0 y 0 = y 0 X x 2X K MiG x x 0 j y 0 x j y 0 = y 0 x 0 j y 0 = x 0 y 0 ThusweshowthattheMiGchaininthisformalwayshasthecorrectstationary distributionnomatterwhat g x x 0 j y 0 is. Note: Itistobenotedthattheproofforthecontinuouscaseissameasthatforthe discretecase,whichfollowsbyreplacingsummationswithintegrals. Considerthetransitionkernelofthemarginal X chainwhichisgivenby K G x x 0 = X y 2Y x 0 j y 0 y 0 j x Weknowthatthefullchain X Y generatedbythetwostageGibbssamplerisnot necessarilytimereversible,butthemarginalchainsare.Similarly,incaseoftheMiG chain,thefullchain X Y isnotnecessarilytimereversiblebutthemarginalchains arereversible,whichisshownbelow.ThemarginalchaininaMetropoliswithinGibbs samplerisgivenby K MiG x x 0 = X y 2Y K MiG x x 0 j y 0 y 0 j x 45

PAGE 46

Toshow, K MiG x x 0 x = K MiG x 0 x x 0 For x 6 = x 0 K MiG x x 0 x = X y 2Y K MiG x x 0 j y 0 y 0 j x x = X y 2Y K MiG x x 0 j y 0 x j y 0 y 0 = X y 2Y K MiG x 0 x j y 0 x 0 j y 0 y 0 = X y 2Y K MiG x 0 x j y 0 y 0 j x 0 x 0 = K MiG x 0 x x 0 whichuses K MiG x x 0 j y 0 tobereversibleand x y = x j y y = y j x x Thus wehaveshownthemarginalchainof X inMetropoliswithinGibbsalgorithmisalso reversible. 3.3MetropoliswithintheRandomScanGibbsSampler Liu1996usesaMetropoliswithinaGibbstechniqueincaseofarandomscan GibbssamplerwhenthestatespaceoftheMarkovchainisdiscreteandnite.The intuitionbehindthis MetropolizedGibbssampler isthesameasthatofPeskun'soriginal ideaofconstructinganasymptoticallyefcientMarkovchain,bywhichtheMarkovchain constructedhasbiggeroff-diagonalentriescomparedtotheordinaryGibbssamplerand hencehassmallerprobabilityofstayingataparticularstate.Thisenablesthechainto explorethestatespacemorerapidly.Liu,inhispaper,showedthatthisMetropolized versionoftherandomscanGibbssamplerisasymptoticallymoreefcientthanthe regularrandomscanGibbssampler.ItistobenotedthatthePeskunorderingamong asymptoticvariancesworksforreversibleMarkovchainsandthisconditionissatised byarandomscanGibbssampler. 46

PAGE 47

Suppose X = X 1 X 2 ,..., X m where X i takes m i possiblevalueswith x asthe stationarydistribution.InarandomscanGibbssampler,acoordinate i ischosenat eachstepwithprobabilityvector p = p 1 p 2 ,..., p m Whenupdatingthe i th coordinate withthecurrentvalue X i = x i ,therandomscanGibbssamplerreplacesthevalueof x i by x 0 i drawnfromthefullconditionaldistribution x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i Instead,intheMetropolized GibbssamplerofLiu, x 0 i isdrawnfromtheproposaldistribution, g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i = 8 > < > : x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i = [1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i ] if x i 6 = x 0 i 0 if x i = x 0 i whichhastheacceptanceprobability, x i x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i =min 1, [1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i ] [1 )]TJ/F25 11.9552 Tf 11.955 0 Td [( x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i ] Liushowedthattheoff-diagonalentriesofthetransitionmatrix,say P MiG obtainedby thisMetropolizedGibbsarelargerthantheoff-diagonalentriesoftransitionmatrix P RS obtainedbytherandomscanGibbssampler.Notethattheproposalconsideredby Liurequiresexactknowledgeofthestationarydistribution,whichmaynotbeexplicitly availableinmostpracticalscenarios. Instead,ifweconsidertheproposaldistributiontobesuchthatfor x i 6 = x 0 i g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i and g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i then P MiG P RS inPeskun'ssense. Theorem3.1. Supposetheproposaldistribution g issuchthatfor x i 6 = x 0 i g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i and g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i then P MiG P RS inPeskun'ssense. 47

PAGE 48

Proof. Let p MiG x i x 0 i denotetheoff-diagonalentriesof P RS Then p MiG x i x 0 i = x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i =min 1, g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i ] g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i ] g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i =min g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i ] x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i ] x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i whicharetheoff-diagonalentriesof P RS andtheaboveinequalityholdssinceby construction g x i x 0 i jj x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i and g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i i.e., g x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i Hence, P MiG P RS inPeskun'ssense. Choiceof g .Oneobviouschoiceof g willbesimilartothechoiceof g madein Chapter2,whichisgivenby g x i x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i = 8 > < > : x 0 i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i + x i x 0 i x i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i if x i 6 = x 0 i 0 if x i = x 0 i where x i x 0 i x i x 0 i 1 isthefractionofmassof x i j x )]TJ/F40 7.9701 Tf 6.587 0 Td [(i thatistransferredto x 0 i j x )]TJ/F40 7.9701 Tf 6.586 0 Td [(i 'ssuchthat P x i 6 = x 0 i x i x 0 i =1. 3.4MetropoliswithintheTwoStageGibbsSampler ThetwostageGibbssamplerfordiscreteandniterandomvariables, X and Y canbeexplainedintermsofamultinomialsamplingscheme.Let X and Y beeach marginallymultinomialrandomvariableswithjointdistributionasgiveninTable3-1 FromCasellaandGeorge1992,extendingthe 2 2 Bernoullivariablesto m n multinomialvariables,thetheoryfollowsexactlyinthesameway.Forthisdistribution, 48

PAGE 49

Table3-1. m n multinomialsampling X n Y 12 ... n Total 1 p 11 p 12 ... p 1 n r 1 2 p 21 p 22 ... p 2 n r 2 . ............. . mp m 1 p m 2 ... p mn r m Total c 1 c 2 ... c n 1 themarginaldistributionsof X and Y aregivenby x = r x x =1,2,..., m and y = c y y =1,2,..., n respectively where r x x =1,2,..., m and c y y =1,2,..., n aretherowandcolumnsumsofTable3-1 respectively.Letusdenethefollowingmatrices, P = 0 B B B B B B B @ p 11 p 12 p 1 n p 21 p 22 p 2 n . . . . . . p m 1 p m 2 p mn 1 C C C C C C C A D r x =diag r 1 r 2 ,..., r m and D c y =diag c 1 c 2 ,..., c n Theconditionaldistributionof X j Y = y and Y j X = x canbecalculatedasfollows. x j y = p yx c y x =1,2,..., m y j x = p xy r x y =1,2,..., n Let A m n Y j X and A n m X j Y denotetheconditionalprobabilitymatricesof Y j X and X j Y respectively.Inmatrixnotation, A Y j X canbewrittenas A Y j X = 0 B B B B B B B @ p 11 r 1 p 12 r 1 p 1 n r 1 p 21 r 2 p 22 r 2 p 2 n r 2 . . . . . . p m 1 r m p m 2 r m p mn r m 1 C C C C C C C A = D )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 r x P 49

PAGE 50

andsimilarly A X j Y canbewrittenas A X j Y = 0 B B B B B B B @ p 11 c 1 p 21 c 1 p m 1 c 1 p 12 c 2 p 22 c 2 p m 2 c 2 . . . . . . p 1 n c n p 2 n c n p mn c n 1 C C C C C C C A = D )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 c y P 0 Thematrices A Y j X and A X j Y maybethoughtofastransitionmatricesgivingthe probabilitiesofgettingfromstate X tostate Y Notethefollowing: 1. A Y j X 1 = D )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 r x P1 = 1 2. [ r 1 r 2 ,..., r m ] A Y j X =[ r 1 r 2 ,..., r m ] D )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 r x P =[ c 1 c 2 ,..., c n ]. Similarpropertiesholdfor A X j Y .Thetransitionprobabilityforthe X sequenceisgivenby A X j X = A Y j X A X j Y = D )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 r x PD )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 c y P 0 Notethat A X j X isareversibletransitiontransitionmatrix,since PD )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 c y P 0 issymmetric. Also [ r 1 r 2 ,..., r m ] isthestationarydistributionofthetransitionmatrix A X j X since [ r 1 r 2 ,..., r m ] A X j X =[ r 1 r 2 ,..., r m ]. Nowwewanttoreplacetheconditionalstep X j Y byaMetropolis-Hastingsstep.Letthe marginaltransitionmatrixforthe X chainintheMiGbedenotedby A X j X = A Y j X A X j Y where A X j Y istheconditionaltransitionmatrixof X j Y withthestepreplacedbya Metropolisstep.So,nowouraimistocomparethetransitionmatrices A X j X and A X j X in termsoftheirasymptoticefciency. 50

PAGE 51

Now,theGibbskernelforthetransition X Y X 0 Y 0 withupdatesgiventhe order y x y 0 x 0 is K G x y x 0 y 0 = p y 0 x 0 c y 0 p xy 0 r x Let g x x 0 j y = g x x 0 j y beproposaldistributionfor X chainforagiven Y = y .IntheMiG version,wheretheconditionalstep X j Y isreplacedbyaMetropolisstep,thekernelfor thetransition X Y X 0 Y 0 isgivenby K MiG x y x 0 y 0 = min p y 0 x 0 g x 0 x j y 0 p y 0 x g x x 0 j y 0 ,1 g x x 0 j y 0 + )]TJ/F39 11.9552 Tf 11.955 0 Td [(r x j y 0 x x 0 j y 0 p xy 0 r x where r x j y 0 = n X y 0 =1 min p y 0 x 0 g x 0 x j y 0 p y 0 x g x x 0 j y 0 ,1 g x x 0 j y 0 and x x 0 j y 0 isDiracdeltamassat x Next,wewanttocomparetheoff-diagonalentriesofthemarginal X chainofboth thetransitionkernels3and3givenby,for x 6 = x 0 K G x x 0 = n X y 0 =1 p y 0 x 0 c y 0 p xy 0 r x K MiG x x 0 = n X y 0 =1 min p y 0 x 0 g x 0 x j y 0 p y 0 x g x x 0 j y 0 ,1 g x x 0 j y 0 p xy 0 r x Wehaveshownbeforethattheboththemarginalschainsabovearereversibleandthey bothhavethesamestationarydistribution,whichisthemarginaldistributionof x y soitwillbeperfectlyvalidtocomparetheasymptoticvariancesofthemarginalchains. Peskunorderingintermsofoff-diagonalentriesoftwotransitionmatricesallowsusto comparetheirasymptoticvariances. If K MiG x x 0 )]TJ/F39 11.9552 Tf 11.956 0 Td [(K G x x 0 0. = n X y 0 =1 min p y 0 x 0 g x 0 x j y 0 p y 0 x g x x 0 j y 0 ,1 g x x 0 j y 0 p xy 0 r x )]TJ/F40 7.9701 Tf 18.925 14.944 Td [(n X y 0 =1 p y 0 x 0 c y 0 p xy 0 r x 0. = n X y 0 =1 p xy 0 r x min p y 0 x 0 g x 0 x j y 0 p y 0 x g x x 0 j y 0 ,1 g x x 0 j y 0 )]TJ/F39 11.9552 Tf 13.151 8.088 Td [(p y 0 x 0 c y 0 0. 51

PAGE 52

= n X y 0 =1 p xy 0 r x min p y 0 x 0 g x 0 x j y 0 p y 0 x )]TJ/F39 11.9552 Tf 13.151 8.088 Td [(p y 0 x 0 c y 0 g x x 0 j y 0 )]TJ/F39 11.9552 Tf 13.151 8.088 Td [(p y 0 x 0 c y 0 0. = n X y 0 =1 p xy 0 r x min p y 0 x 0 g x 0 x j y 0 p y 0 x )]TJ/F22 11.9552 Tf 16.795 8.087 Td [(1 c y 0 g x x 0 j y 0 )]TJ/F39 11.9552 Tf 13.151 8.087 Td [(p y 0 x 0 c y 0 0. Forequation3tohold,asufcientconditionon g is g x 0 x j y 0 p y 0 x )]TJ/F22 11.9552 Tf 16.794 8.088 Td [(1 c y 0 0= g x 0 x j y 0 p y 0 x c y 0 and g x x 0 j y 0 )]TJ/F39 11.9552 Tf 13.15 8.088 Td [(p y 0 x 0 c y 0 0= g x x 0 j y 0 p y 0 x 0 c y 0 So,thechoiceofsuchaproposaldistributionfor X j Y wouldensurethatinthecaseof MiG,thetransitionmatrixcorrespondingtothemarginal X chainhasbiggeroff-diagonal elementsthanthetransitionmatrixcorrespondingtothemarginal X chaininatwo stageGibbssampler.Hence,aproperchoiceofproposaldistributionensuresthata MetropolisstepwithaGibbssamplerisasymptoticallymoreefcientthanthetwostage Gibbssampler.Theexamplebelowillustratesthisphenomenon. Example3.1. Considerthe 3 2 multinomialsampling.Suppose X and Y areeach marginallymultinomialrandomvariableswithjointdistributiongiveninTable3-2.The Table3-2. 3 3 multinomialsampling X n Y 123 Total 10.100.100.100.3 20.200.050.050.3 30.100.200.100.4 Total 0.400.350.251.0 conditionaldistributionof X j Y and Y j X arerespectivelygivenbelow: A X j Y = 0 B B B B @ 1 = 41 = 21 = 4 2 = 71 = 74 = 7 2 = 51 = 52 = 5 1 C C C C A A Y j X = 0 B B B B @ 1 = 31 = 31 = 3 2 = 31 = 61 = 6 1 = 42 = 41 = 4 1 C C C C A 52

PAGE 53

Next,wereplace X j Y stepbyaMetropolisstep.For X j Y =1, wehavethe stationarydistribution f 1 = 4,1 = 2,1 = 4 g andtakingtheproposaltransitionmatrix G = 0 B B B B @ 05 = 83 = 8 1 = 201 = 2 3 = 85 = 80 1 C C C C A whichgivestheMetropolizedtransitionmatrixas P MiG = 0 B B B B @ 05 = 83 = 8 5 = 163 = 85 = 16 3 = 85 = 80 1 C C C C A For X j Y =2, wehavethestationarydistribution f 2 = 7,1 = 7,4 = 7 g andtakingtheproposal transitionmatrix G = 0 B B B B @ 02 = 75 = 7 5 = 1409 = 14 4 = 73 = 70 1 C C C C A whichgivestheMetropolizedtransitionmatrixas P MiG = 0 B B B B @ 3 = 285 = 285 = 7 5 = 1409 = 14 5 = 149 = 5627 = 56 1 C C C C A For X j Y =3, wehavethestationarydistribution f 2 = 5,1 = 5,2 = 5 g andtakingtheproposal transitionmatrix G = 0 B B B B @ 02 = 53 = 5 1 = 201 = 2 3 = 52 = 50 1 C C C C A 53

PAGE 54

whichgivestheMetropolizedtransitionmatrixas P MiG = 0 B B B B @ 3 = 201 = 43 = 5 1 = 201 = 2 3 = 51 = 43 = 20 1 C C C C A Now,theentriesofthematrix A X j X areobtainedas P X =1 j X =1= 3 X k =1 P Y = k j X =1P X =1 j Y = k =3 = 35. P X =2 j X =1= 3 X k =1 P Y = k j X =1P X =2 j Y = k =59 = 168. P X =3 j X =1= 3 X k =1 P Y = k j X =1P X =3 j Y = k =473 = 840. P X =1 j X =2= 3 X k =1 P Y = k j X =2P X =1 j Y = k =59 = 168. P X =3 j X =2= 3 X k =1 P Y = k j X =2P X =2 j Y = k =1 = 4. P X =3 j X =2= 3 X k =1 P Y = k j X =2P X =3 j Y = k =67 = 168. P X =1 j X =3= 3 X k =1 P Y = k j X =3P X =1 j Y = k =473 = 1120. P X =2 j X =3= 3 X k =1 P Y = k j X =3P X =2 j Y = k =67 = 224. P X =3 j X =3= 3 X k =1 P Y = k j X =3P X =3 j Y = k =39 = 140. Thus,thematrix A X j X isobtainedas A X j X = 0 B B B B @ 3 = 3559 = 168473 = 840 59 = 1681 = 467 = 168 473 = 112067 = 22439 = 140 1 C C C C A 54

PAGE 55

Also,thematrix A X j X isobtainedas A X j X = A Y j X A X j Y = 0 B B B B @ 131 = 42059 = 21057 = 140 59 = 21041 = 10523 = 70 171 = 56069 = 280251 = 560 1 C C C C A Ithasbeenveriedthatboththematrices A X j X and A X j X havethesamestationary distribution c x = f 3 = 10,3 = 10,4 = 10 g Letusdenote, D =diag c x Then,thedifference betweenthematrices A X j X and A X j X isobtainedas D f A X j X )]TJ/F48 11.9552 Tf 11.955 0 Td [(A X j X g = 0 B B B B @ )]TJ/F22 11.9552 Tf 9.298 0 Td [(19 = 28059 = 2800131 = 2800 59 = 2800 )]TJ/F22 11.9552 Tf 9.298 0 Td [(59 = 140059 = 2800 131 = 280059 = 2800 )]TJ/F22 11.9552 Tf 9.298 0 Td [(19 = 280 1 C C C C A whichclearlyshowsthattheoff-diagonalentriesof A X j X isgreaterthanthecorresponding off-diagonalentriesof A X j X .Hence,byPeskunorderingtheMarkovchaincorresponding tothetransitionmatrixwiththeMetropolisstepisasymptoticallymoreefcientthatthe ordinaryGibbschain.Forafunction, f =, )]TJ/F22 11.9552 Tf 9.299 0 Td [(1,0, theasymptoticvariancecomputed areasfollows: v f , A X j X =0.424149 < v f , A X j X =0.684848, whichfurther supportsourargument. 55

PAGE 56

CHAPTER4 COUNTABLESTATESPACEMARKOVCHAINS 4.1Setup Markovchainswithacountably-innitestatespacemorebriey,countablestate spaceMarkovchainsexhibitsometypesofbehaviornotpossibleforMarkovchains withnitestatespaces.InChapter2,fornite-stateMarkovchains,wedenedthe limitingdistributionofatransitionmatrix P asaprobabilityvector thatsatises P = Inthecaseofcountable-stateMarkovchains,let C denotethestatespaceoftheMarkov chains.Inthiscase,aMarkovtransitionkernel P isaninnitematrix,whichwecalla Markovoperator.Herewedene f i ; i 0 g inthesameway,asasetofnumbersthat satisfy i = X j 2C j p j i forall i ; i 0 forall i ; X j 2C p i j =1. Let f X n g n 0 beaMarkovchaindenedon C withtransitionkernel P where p i j isthe one-steptransitionprobabilitydenedas p i j =P X 1 = j j X 0 = i Supposethataset ofnumbers f i ; i 0 g satisfying5-7ischosenastheinitialprobabilitydistributionfora Markovchain,i.e.,if P X 0 = i = i forall i .Then P X 1 = i = P j 2C j p j i = i forall i ,and,byinduction, P X n = i = i forall i andall n 0 .Thefactthat P X n = i = i forall i motivatesthedenitionofthelimitingdistributionofMarkovchainsincountable statespaces. Let T ii betheexpectedrecurrencetimeofstate i eitherniteorinniteandifstate i isaperiodic,thissaysthat lim n !1 P X n = i j X 0 = i =1 = T ii .Therefore, 1 = T ii isalimiting probabilityforstate i ,bothinatime-averageandalimitingensemble-averagesense. AnirreducibleMarkovchainisaMarkovchaininwhichallpairsofstatescommunicate. InnitestatespaceMarkovchains,irreducibilityimpliesasingleclassofrecurrent states,whereasforcountablyinnitestatespaceMarkovchains,anirreduciblechainis asingleclassthatcanbetransient,null-recurrent,orpositive-recurrent.ForaMarkov 56

PAGE 57

chainconsistingofasingleclassofstateswithallstatespositive-recurrent,thereisa uniquestationarydistribution, f i ; i 0 g suchthat i =1 = T ii > 0 forall i InaMarkovchainwithacountablyinnitestatespace C ,supposeweareinterested inndingtheestimateoftheexpectationofsomenonconstantfunction f denedon thestatesof C Thusinnotation,wewanttoestimate =E [ f X ]= X i 2C f i i Theabovequantityisestimatedby ^ n = P n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 k =0 1 n f X k ,inwhichanirreducible,aperiodic Markovchainisrunsoastoobtainaconsistentestimateof Wewillalsoassumethatthefunction f isdenedontheHilbertspace, l 2 which isdenedasseealsoappendixB.2 l 2 = f f : C! R : X i 2C f 2 i i < 1g equippedwithinnerproduct h f g i l 2 = E [ f X g X ]= X i 2C f i g i i TheMarkovoperator, P operateson l 2 by Pf i = X j 2C p i j f j Supposeweareinterestedinndingtheformoftheasymptoticvarianceof ^ n HereweeitherassumethattheconditionsneededfortheexistenceofaCentralLimit TheoremholdforMarkovchainsconsideredinthissection[Theorem6.64,Robert andCasella2004].FromKipnisandVaradhan1986,wecanalsoconsiderMarkov chainswhichareergodic,positiverecurrentandsatisfytheconditionofreversibility, whichalsoguaranteestheexistenceofCLT.Beforegoingtotheactualderivationfor theexpressionoftheasymptoticvariance,whichissimilartonitestatespaceMarkov 57

PAGE 58

chains,forcountablyinnitestatespaceMarkovchains,letusdenethefollowing quantitiesneededforthederivation: Diagonaloperator, D istheinnitediagonalmatrixwith i th diagonalentry i Limitingoperator, istheinnitelimitingmatrixwhereeachrowis f i ; i 0 g Fundamentaloperator, Z = I )]TJ/F39 11.9552 Tf 12.368 0 Td [(P + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 whoseexistenceisguaranteedbyan ergodicandpositiverecurrentMarkovchain. 4.2ExpressionoftheLimitingVariance Theorem4.1. Suppose f X n g n 0 beanergodicandpositiverecurrentMarkovchainon C withMarkovoperator P andstationarydistribution .Suppose f isafunctiondened onthestatesof C andalsodenedontheHilbertspace l 2 .Thentheasymptotic varianceexpressionoftheestimate ^ n isgivenby v f , P =lim n !1 n Var n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X k =0 1 n f X k # = h D f 2 Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(I )]TJ/F22 11.9552 Tf 11.955 0 Td [( g f f i Proof. WeprovethistheorembyadaptingtheprooffromKemenyandSnell1969fora nitestatespaceMarkovchainandextendtoacountablestatespacecase. Wewillprovetheresultbyassumingthatthe isthestationarydistributionandthe limitingdistributionoftheMarkovchain f X n g n 0 Letusdenote n Var n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X k =0 1 n f X k # := v n f , P v n f , P = 1 n Cov n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X k =0 f X k n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X ` =0 f X ` # = 1 n E n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X k =0 f X k )]TJ/F39 11.9552 Tf 11.956 0 Td [(n X i 2C i f i n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X ` =0 f X ` )]TJ/F39 11.9552 Tf 11.955 0 Td [(n X i 2C i f i Theaboveequalityisobtainedfrom Cov XY =E[ X )]TJ/F22 11.9552 Tf 12.552 0 Td [(E X ][ Y )]TJ/F22 11.9552 Tf 12.552 0 Td [(E Y ] forrandom variables X and Y 58

PAGE 59

Then v n f , P = 1 n n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X k =0 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X ` =0 E [ f X k f X ` ] )]TJ/F40 7.9701 Tf 12.796 14.944 Td [(n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X k =0 E [ f X k ] X i 2C i f i )]TJ/F40 7.9701 Tf 20.102 14.944 Td [(n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X ` =0 E [ f X ` ] X i 2C i f i + n X i j 2C i j f i f j = 1 n n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X k =0 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X ` =0 E [ f X k f X ` ] )]TJ/F30 11.9552 Tf 12.627 11.357 Td [(X i j 2C i j f i f j Now, E [ f X k f X ` ]= 8 > > > > < > > > > : P i j 2C i p ` )]TJ/F40 7.9701 Tf 6.587 0 Td [(k i j f i f j k <` P i j 2C j p k )]TJ/F26 7.9701 Tf 6.586 0 Td [(` j i f i f j `< k P i j 2C i i j f i f j k = ` where i j istheKroneckerdeltafunctionwhichequals 1 for i = j and 0 otherwise. Thus,weget v n f , P = 1 n X i j 2C i f i f j X k <` p ` )]TJ/F40 7.9701 Tf 6.586 0 Td [(k i j )]TJ/F25 11.9552 Tf 11.955 0 Td [( j + 1 n X i j 2C j f i f j X k <` p k )]TJ/F26 7.9701 Tf 6.587 0 Td [(` j i )]TJ/F25 11.9552 Tf 11.955 0 Td [( i + X i j 2C i f i f j i j )]TJ/F25 11.9552 Tf 11.955 0 Td [( j Collectingtermswiththesame d = j ` )]TJ/F39 11.9552 Tf 11.955 0 Td [(k j ,wehave v n f , P = X i j 2C i f i f j n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =1 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n p d i j )]TJ/F25 11.9552 Tf 11.955 0 Td [( j + 1 n X i j 2C j f i f j n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X d =1 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n p d j i )]TJ/F25 11.9552 Tf 11.955 0 Td [( i + X i j 2C i f i f j i j )]TJ/F25 11.9552 Tf 11.956 0 Td [( j = 2 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =1 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n D P d )]TJ/F22 11.9552 Tf 11.955 0 Td [( f f + + h D I )]TJ/F22 11.9552 Tf 11.955 0 Td [( f f i 59

PAGE 60

Theaboveequalityisobtainedbyexchangingsummationsandusingthenotation, P ij 2C a ij f i f j = < Af f > foranoperator A andforallfunctions f Now, P )]TJ/F22 11.9552 Tf 11.955 0 Td [( d = d X i =0 )]TJ/F22 11.9552 Tf 9.299 0 Td [(1 d )]TJ/F40 7.9701 Tf 6.586 0 Td [(i P i n )]TJ/F40 7.9701 Tf 6.587 0 Td [(i = P d + d )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X i =0 )]TJ/F22 11.9552 Tf 9.299 0 Td [(1 d )]TJ/F40 7.9701 Tf 6.586 0 Td [(i P i n )]TJ/F40 7.9701 Tf 6.587 0 Td [(i = P d + d )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X i =0 )]TJ/F22 11.9552 Tf 9.298 0 Td [(1 d )]TJ/F40 7.9701 Tf 6.587 0 Td [(i = P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(. Now,usingTheorem1ofSchempp1970AppendixB.1and4,wehave lim n !1 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =0 P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(=lim n !1 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =0 P )]TJ/F22 11.9552 Tf 11.955 0 Td [( d = I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = Z Also,using4,wehave lim n !1 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =0 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(=lim n !1 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X d =0 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n P )]TJ/F22 11.9552 Tf 11.955 0 Td [( d ThenbyCesaro-summability, lim n !1 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X d =0 P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(=lim n !1 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X d =0 n )]TJ/F39 11.9552 Tf 11.956 0 Td [(d n P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(= I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 = Z whichgives lim n !1 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 X d =1 n )]TJ/F39 11.9552 Tf 11.956 0 Td [(d n P d )]TJ/F22 11.9552 Tf 11.955 0 Td [(=lim n !1 n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 X d =1 n )]TJ/F39 11.9552 Tf 11.955 0 Td [(d n P )]TJ/F22 11.9552 Tf 11.955 0 Td [( d = Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(I Finally,theexpressionoftheasymptoticvarianceisobtainedas v f , P = h 2 D Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(I f f i + h D I )]TJ/F22 11.9552 Tf 11.955 0 Td [( f f i = h D f 2 Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(I )]TJ/F22 11.9552 Tf 11.955 0 Td [( g f f i ItistobenotedthatTheorem4.1reducestosamethequadraticformexpression 2asgivenbyKemenyandSnell1969fornitestatespaceMarkovchains.Also, 60

PAGE 61

notethatTierney1998provideanexpressionoftheasymptoticvarianceonageneral statespaceMarkovchain,whichisbasedonthespectraltheory. NowforcomparingasymptoticvarianceofMarkovchains,similartoMira2001, wefurtherassumethattheMarkovoperator, P isreversible,i.e.,when i P i j = j P j i forall i j 2C ,orequivalently,theoperator P isself-adjoint,i.e.,when h Pf g i l 2 = < f Pg > l 2 Theorem4.2. Suppose P and Q aretwoself-adjointreversibleMarkovoperatorson C bothwithstationarydistribution Thenthefollowingareequivalent: 1. Q )]TJ/F39 11.9552 Tf 11.955 0 Td [(P isanon-negativeoperator. 2. v Q , f v P , f Proof. Wehave,fromtheexpression4 v f , P = h D f 2 Z )]TJ/F39 11.9552 Tf 11.955 0 Td [(I )]TJ/F22 11.9552 Tf 11.955 0 Td [( g f f i Then,wehave v Q , f v P , f D f 2 I )]TJ/F39 11.9552 Tf 11.955 0 Td [(Q )]TJ/F22 11.9552 Tf 11.955 0 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(I )]TJ/F22 11.9552 Tf 11.955 0 Td [( g f f D f 2 I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P )]TJ/F22 11.9552 Tf 11.955 0 Td [( )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(I )]TJ/F22 11.9552 Tf 11.955 0 Td [( g f f Deletingcommon I and onbothsidesoftheaboveequation,weget v Q , f v P , f D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(Q + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 f f D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 f f Notethat,since P and Q areself-adjointoperatorswhichimplies I )]TJ/F39 11.9552 Tf 12.934 0 Td [(P + and I )]TJ/F39 11.9552 Tf 12.126 0 Td [(Q + arealsoself-adjoint,whichgives I )]TJ/F39 11.9552 Tf 12.126 0 Td [(P + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 and I )]TJ/F39 11.9552 Tf 12.126 0 Td [(Q + )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 arealso self-adjoint.FromFuruta1997AppendixB.2,wehave,if D I )]TJ/F39 11.9552 Tf 11.956 0 Td [(Q + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 > 0, then D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(Q + > 0. 61

PAGE 62

Therefore,nallywehave v Q , f v P , f h D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(Q + f f ih D I )]TJ/F39 11.9552 Tf 11.955 0 Td [(P + f f i ,h D Q )]TJ/F39 11.9552 Tf 11.955 0 Td [(P f f i 0, i.e., Q )]TJ/F39 11.9552 Tf 11.955 0 Td [(P isanon-negativeoperator. SimilartoTheorem2.5ofChapter2fornitestatespaceMarkovchains,the expressionsinthecaseofcountableinnitestatespaceMarkovchainsareobtained below. Theorem4.3. Suppose P and Q aretwoMarkovoperatorbothwithstationarydistribution .Let H = D Q )]TJ/F39 11.9552 Tf 12.634 0 Td [(P bethedifferenceoperatorwith i j th entryas f h ij g then h D Q )]TJ/F39 11.9552 Tf 11.955 0 Td [(P f f i = )]TJ/F22 11.9552 Tf 10.494 8.088 Td [(1 2 X i j 2C h ij f i )]TJ/F39 11.9552 Tf 11.955 0 Td [(f j 2 # = 1 2 X i j 2C h ij f i + f j 2 # Fromtherstexpressionofequation4,weobservethat p i j )]TJ/F39 11.9552 Tf 12.562 0 Td [(q i j 0, whichgives Q )]TJ/F39 11.9552 Tf 12.683 0 Td [(P anon-negativeoperator.Hence,Peskunorderingextendstothe countableinnitestatespaceMarkovchain.Itistobenotedthattheconstructionof asymptoticallyefcienttransitionkernelsisobtainedinasimilarmanneraswehave showninthecaseofnitestateMarkovchains.Thoughthetotalorderingproposedin thecaseoftheoptimalalgorithmdoesnotworkformostoftheMarkovoperatorson C thealgorithmcanbeappliedtoanynitesetofstatesforacountablyinnitestate Markovchain.Theexamplesbelowshowsomeoftheapplicationsinconstructinga betterMarkovchain. 4.3SequentialConstruction ForanyMarkovoperator P onacountablyinnitestatespace C theconstruction ofanasymptoticallyefcientMarkovoperatorisdoneinthefollowingway:Chooseany 62

PAGE 63

2 2 arrayaroundadiagonal P Withoutlossofgeneralityassumeittobe [ p ,1, p ,2, p ,1, p ,2]. Next,compare 1 p ,1 and 2 p ,2. If 1 p ,1 2 p ,2, thenchangethearrayto 0, p ,2+ p ,1, p ,1+ 1 2 p ,1, p ,2 )]TJ/F25 11.9552 Tf 13.151 8.088 Td [( 1 2 p ,1 whichisastationarypreservingmasstransfer.Now,if 1 p ,1 2 p ,2, then ipstate 1 and 2 anddothesame.Next,comparethenexttwoconsecutivestates withnon-zerodiagonalsanddothesamemasstransfer.Thiswayofmasstransferis convenientinthecasewhenthestatespaceincountablyinnite,sinceinmostofthe casesherewedonothavethetotalorderinthestates,i.e., 1 p ,1 2 p ,2 3 p ,3 ... Buttheexamplebelowissuchacase,wherewehaveatotalorderingin thestates. Theorem4.4. AMarkovoperator, P S obtainedthroughsequentialconstructionis asymptoticallymoreefcientthantheoriginalMarkovoperator P 4.4Examples Example4.1. Consideranexampleincountablestatespace, C = f 0,1,2,... g withthe Markovoperator, P = 0 B B B B B B B @ 1 2 1 2 2 1 2 3 1 2 1 2 2 1 2 3 1 2 1 2 2 1 2 3 . . . . . . 1 C C C C C C C A Itistobenotedthatfortheoperator P abovehasthestationarydistribution = f 1 2 1 2 2 1 2 3 ... g Itisobvioustoseethattheorderingsinthestatespaceis 1 2 > 1 2 2 > 1 2 3 > ... 63

PAGE 64

Inthiscasetheoptimalalgorithmforthenitestatespacescanbeextendedtothe countablestatespaceforthisspeciedexamplewhichisdescribedbelow: Step1.Let ij bethefractionof p i i thatwouldbeaddedto p i j 'ssuchthat 0 < ij < 1 and P 1 i =1 ij =1. FollowingexactlyfromnitestatespaceMarkovchains,attheend ofStep 1 ,wehavezeroastherstelementofthenew P whichpreservesthecorrect stationarydistribution. Step2.Nextweproceedbynotconsideringtherstrowandrstcolumnofthenew P andfollowsimilarprocedureasinStep1andcontinue. Example4.2 Birth-deathchain Abirth-deathMarkovchainisaMarkovchaininwhich thestatespaceisthesetofnonnegativeintegers;forall i 0 ,thetransitionprobabilities satisfy p i i +1 > 0 and p i +1, i > 0 ,andforall j i )]TJ/F39 11.9552 Tf 12.279 0 Td [(j j > 1 p i j =0 .Atransition fromstate i to i +1 isregardedasabirthandonefrom i +1 to i asadeath.Thus therestrictiononthetransitionprobabilitiesmeansthatonlyonebirthordeathcan occurinoneunitoftime.Manyapplicationsofbirth-deathprocessesariseinqueueing theory,wherethestateisthenumberofcustomers,birthsarecustomerarrivals,and deathsarecustomerdepartures.Therestrictiontoonlyonearrivalordepartureata timeseemsratherpeculiar,butusuallysuchachainisanelysampledapproximationto acontinuous-timeprocess,andthetimeincrementsarethensmallenoughthatmultiple arrivalsordeparturesinatimeincrementareunlikelyandcanbeignoredinthelimit. Wedenote p i i +1 by p i and p i i )]TJ/F22 11.9552 Tf 12.775 0 Td [(1 by q i Thus p i i =1 )]TJ/F39 11.9552 Tf 12.775 0 Td [(p i )]TJ/F39 11.9552 Tf 12.775 0 Td [(q i .Forthis Markovchainifwemake p i i small,wecanmaketheMarkovchainasymptotically moreefcient.Inthebirth-deathchain,thereisaaneasywaytondthesteady-state probabilities,whichisobtainedbyconsideringtheequation i p i = i +1 q i +1 Denoting i = p i = q i +1 wegetthelimitingprobabilitydistributionsas i = 0 i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Y j =0 j ; 0 = 1 1+ P 1 i =1 Q i )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 j =0 j 64

PAGE 65

If P 1 i =1 Q i )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 j =0 j < 1 then 0 ispositiveandallthestatesarepositive-recurrent.So, itistobenotedthatthemasstransferof p i i shouldbedoneinsuchawaythat theconditionsneededfortheexistenceofCLTissatised,whichwillthengivean asymptoticallyefcientMarkovchain. 4.5GenerationofContingencyTables ContingencytablegenerationasarandomwalkwasintroducedbyDiaconisand Gangolli1995.AnotherprominentpaperisthisareaisDiaconisandSturmfels1998, whichdiscussmanyvariationsandextensionsofthisalgorithm. Considergeneratingcontingencytableswithxedrowandcolumnsums.Letthere be m rowsand n columnsandtherowandcolumnsumsvectors, r = r 1 r 2 ,..., r m and c = c 1 c 2 ,..., c n suchthat N = P m i =1 r i = P n j =1 c j Thesetofcontingencytable, C r,c is thesetof m n non-negativeintegermatrices T = f t ij g m n thatsatisfythegivenrow andcolumnsums.Alsoitistobenotedthatonlytableswhichsatisfythenon-negativity ineachcellofthetable T areconsideredintheset C r,c .Innotation, C r,c = t ij 0, i =1,2,.., m j =1,2,.., n ,: m X i =1 t ij = c j n X j =1 t ij = r i N = m X i =1 r i = n X j =1 c j AvariationofDiaconis-GangolliMarkovchaingiveninBezakovaetal.2009is describedasfollows: Given, T 2C r,c thenextmovetotable T 0 isdoneinthefollowingway: Withprobability1/2stayinthecurrenttable,i.e., T 0 = T Otherwise,choose i 1 < i 2 uniformlyatrandomfrom 1,2,..., m and j 1 < j 2 uniformly atrandomfrom 1,2,..., n Withprobability1/2,let e =1, else e = )]TJ/F22 11.9552 Tf 9.299 0 Td [(1. Let E = f e ij g m n beamatrixwithonlyfournon-zeroentries: e i 1 j 1 = e i 2 j 2 = e and e i 1 j 2 = e i 2 j 1 = )]TJ/F39 11.9552 Tf 9.299 0 Td [(e If T + E hasallnon-negativeentries,then T = T 0 thenew contingencytable. ItistobenotedthatthisMarkovchainissymmetricandhasstationarydistribution whichisuniformover C r,c .Alsonotethatthereisnoperiodicityproblemssincethechain haspositiveholdingproperty.Anobviousimprovementintermsofasymptoticefciency, 65

PAGE 66

canbeobtainedasfollows:Given, T 2C r,c thenextmovetotable T 0 isdoneinthe followingway: Withprobability1/3stayinthecurrenttable,i.e., T 0 = T Otherwise,choose i 1 < i 2 uniformlyatrandomfrom 1,2,..., m and j 1 < j 2 uniformly atrandomfrom 1,2,..., n Withprobability2/3,let e =1, else e = )]TJ/F22 11.9552 Tf 9.299 0 Td [(1. Let E = f e ij g m n beamatrixwithonlyfournon-zeroentries: e i 1 j 1 = e i 2 j 2 = e and e i 1 j 2 = e i 2 j 1 = )]TJ/F39 11.9552 Tf 9.299 0 Td [(e If T + E hasallnon-negativeentries,then T = T 0 thenew contingencytable. Thustheonlynon-zeroentriesintheoff-diagonalofthetransitionmatrixintherstcase are 1 = 4 wheresthecorrespondingnon-zeroentriesintheimprovedrandomwalkare equalto 1 = 3 .Hence,fromPeskun'sordering,wecansaythelatercaseisasymptotically moreefcientthantheformer. Next,wesuggestadifferentwayofimprovingthevariationoftherandomwalkby Diaconis-Gangolli,whichwillimprovetheasymptoticefciencyoftheaboveMarkov chain.Weexplaintheprocedurewiththehelpofthisexample. Example4.3. Considerthefollowing 2 4 contingencytable, Table4-1. 2 4 contingencytable Col1Col2Col3Col4 Row11111 Row21111 Now,wegetthetablesinFigure4-1,withsamerowsandcolumnssumasofTable4-1 in C r,c TherandomwalkproposedbyDiaconis-Gangollihasonlytwomoves 2 6 4 +-+ 3 7 5 := a 1 or, 2 6 4 -+ +3 7 5 := a 2 eachwithprobability 1 2 onTable4-1.Itistobeobservedthatthisrandomwalkcan generateallthetablesmentionedaboveexceptthelastsixofFigure4-1,whichare obtainedwiththemovesgiveninFigure4-2. 66

PAGE 67

0 2 1 1 2 0 1 1 2 0 1 1 0 2 1 1 1 0 2 1 1 2 0 1 1 2 0 1 1 0 2 1 1 1 2 0 1 1 0 2 1 1 0 2 1 1 2 0 2 1 0 1 0 1 2 1 0 1 2 1 2 1 0 1 0 1 1 2 2 1 1 0 2 1 1 0 0 1 1 2 1 0 1 2 1 2 1 0 1 2 1 0 1 0 1 2 0 2 0 2 2 0 2 0 2 0 2 0 0 2 0 2 0 0 2 2 2 2 0 0 2 2 0 0 0 0 2 2 0 2 2 0 2 0 0 2 2 0 0 2 0 2 2 0 Figure4-1.ContingencytableswithxedmarginsasofTable4-1 +-+-+-+ -+-+ +-+++---++ --++ ++-+--+ -++-+++--+ Figure4-2.SixothermovesinthecontingencyTable4-1 Now,withthersttwomoves[assuggestedbyDiaconisandGangolli1995],we addonemoremovefromtheabovesixmoves,say, 2 6 4 +-+-+-+ 3 7 5 := a 3 So,wehaveMarkovchain1MC1withmoves a 1 and a 2 andwehaveMarkovchain 2MC2withmoves a 1 a 2 and a 3 Let a denotethestayingstateofbothMC1and MC2.FromTable4-2,weobservethattheaddingatransitionstatetoMC1,increases thecorrespondingoff-diagonalentriesinMC2.Hence,byPeskun'sordering,MC2is asymptoticallymoreefcientthanMC1.Weobservethattheproposalwith 2 moveshas zeroprobabilityfor a 3 ,buttheproposalwith 3 moveshasnon-zeroprobabilityfor a 3 67

PAGE 68

Table4-2.TransitionprobabilityforMarkovchains,MC1andMC2, a isthestayingstate, a 1 a 2 and a 3 arethetransitionstatesfrom a MC1MC2 Movespossible aa 1 a 2 a 3 aa 1 a 2 a 3 a 1 a 2 a 3 1/21/41/4001/31/31/3 a 1 a 2 1/21/41/401/31/31/30 a 2 a 3 3/401/401/301/31/3 a 1 a 3 3/41/4001/31/301/3 ThenwecansaythatMetropolizationwith 3 movesisasymptoticallymoreefcientthan theMetropolizationwith 2 moves,whichfollowsfromtheTheorem4.5. Example4.4. Considera 2 4 contingencytableinTable4-3.Intheexample,we Table4-3. 2 rows4 columnscontingencytable Col1Col2Col3Col4 Row13131 Row21313 addedanothersymmetricmoveto a 3 whichis 2 6 4 -+-+ +-+3 7 5 := a 4 ForthiscontingencytableFisher'sexacttestgivesthep-valueas 0.2261215 .The p-valueswith 2 movesand 4 movesalongwiththeirasymptoticvariancesbasedon 10,000simulationsrepeated10timesarelistedintheTable4-4. Table4-4.P-valuesandasymptoticvariances p-valueasymptoticvariance 2 moves 0.22297.2469 10 )]TJ/F23 7.9701 Tf 6.587 0 Td [(5 4 moves 0.22214.5991 10 )]TJ/F23 7.9701 Tf 6.587 0 Td [(5 FromTable4-4,weseethattheasymptoticvarianceoftheMarkovchainwith 2 movesisgreaterthanthatof 4 moves. Theorem4.5. Suppose G 1 = f g 1 i j g and G 2 = f g 2 i j g aretwoproposaldistributions suchthat g 1 i j g 2 i j 8 i 6 = j .ThenMetropolizing G 1 and G 2 withstationary 68

PAGE 69

distribution ,givestransitiondistribution Q 1 = f q 1 i j g and Q 2 = f q 2 i j g .Then Q 1 is asymptoticallymoreefcientthan Q 2 Proof. Given g 1 i j g 2 i j and g 1 j i g 2 j i = j g 1 j i i j g 2 j i i whichgives min j g 1 j i i g 1 i j i min j g 2 j i i g 2 i j i Hence, q 1 i j q 2 i j whichshowsthat Q 1 isasymptoticallymoreefcientthan Q 2 Remark. Itistobenotedthatthemethoddiscussedearlier,ofincludingmore moves,canbegeneralizedfurther,keepinginmindthemoves a 1 and a 2 shouldhave probabilitiesnotlessthan 1 = 4. Otherwise,thematricescannotbecomparedinPeskun's sense. 4.6TestforIndependence Enumerationof C r c isusedindeterminingtheindependenceassumptionin two-waycontingencytables.Theindependencehypothesisisspeciedas H 0 : p ij = p i p j 8 i and j where p i = P j p ij p j = P i p ij AstandardtestofindependenceusestheChi-square statistics 2 = X ij t ij )]TJ/F39 11.9552 Tf 11.955 0 Td [(r i c j = N 2 r i c j = N Nowconsiderthefollowingexample, DiaconisandEfron1985giveanapproximationforevaluatingthesizeofthestate space C r c Thisgives 2.963 10 3 ,whichisobtainedfromtheformulagiveninAppendix 69

PAGE 70

Table4-5.Contingencytablewith 2 rowsand 4 columns col1col2col3col4total row12341524 row21520302085 total17233435109 B.3,asanapproximationofthenumberoftableswiththesamemarginsasinTable 4 )]TJ/F22 11.9552 Tf 11.955 0 Td [(5. InTable 4 )]TJ/F22 11.9552 Tf 11.955 0 Td [(5, 2 =13.0537, whichhasp-value= 0.004998 ,whichimpliesthe hypothesisofindependenceinrejected.Ifahypothesisisstronglyrejected,wehave littleguidancetodeterminetheactualdistributionfromwhichthedataaregenerated. Moreover,itisverycommontohaveahugeobservedChi-squarestatisticsvaluefor two-waycontingencytables,sostatisticianshavesuggestedothermethodsofcalibrating thechi-squarestatistics.Forexample,DiaconisandEfron1985assumedthatthe underlyingprobabilities p ij wereunknownandputauniformprioronthem.Thesetof allpossible m n tablesisasimplexin mn dimensions, S mn = f p ij : p ij 0, P p ij =1 g Underthisassumption,thetables t ij generatedareuniform,i.e.,theyhaveanequal chanceofoccurring.ThisledDiaconisandEfron1985tosuggest calibratingthe distributionof 2 undertheuniformdistributionasanantagonisticalternativetothe modelofindependence. Forcalibrationofteststatisticsinthesetofpossibletables,itiscustomarytox ontherowandcolumnmarginsofatwo-waycontingencytable.Thisgiverisetothe followingcombinatorialproblemDiaconisandEfron1985:for r and c partitionsof N ndtheproportionoftablesin C r c withChi-squarevaluesmallerthan t as t varies. Wecompute P 2 13.0537 fortherandomwalkapproachwith 2 movesand 4 movesbasedon 100,000 simulations,repeated 10 times,whichisgiventheTable 4-6Eachp-valueobtainedinthetableisthemedianofthe 10 p-valuesbasedon Table4-6.P-valuesandasymptoticvariancesforaMarkovchainwith 2 and 4 moves p-valueasymptoticvariance 2 moves 0.319672.0398 10 )]TJ/F23 7.9701 Tf 6.587 0 Td [(4 4 moves 0.305310.7194 10 )]TJ/F23 7.9701 Tf 6.587 0 Td [(4 70

PAGE 71

100,000 simulationseach.Since,thep-valueisapproximately 0.3 ,weconcludethat thedataarecompatiblewiththehypothesisofauniformdistribution.Itistobenoted thattheasymptoticvariancewith 2 movesislargerthantheasymptoticvariancewith 4 moves.ThisisinaccordancewithPeskun'stheorem. Also,notethatundertheclassicalmodelofindependence,thedistributionofthe stationarydistributionisthehypergeometricdistributionon C r c whichis H = n Y j =1 c j t 1 j ... t mj N r 1 r 2 ... r m DiaconisandGangolli1995suggestanextraMetropolissteptogeneratefrom H The MarkovchainobtainedaftertheMetropolisstepwiththerandomwalkastheproposal distributiongivesaconnected,aperiodicandreversiblechainwithstationarydistribution H 71

PAGE 72

CHAPTER5 APPLICATIONS 5.1BackgroundonType1DiabetesStudy Inrecentyears,therehasbeenincreasinginterestinthediscoveryofcorrelations betweenbacterialcommunitiesinthehumangutanddisease.Severalstudieshave shownacorrelationbetweengutbacteriacommunitiesandthedevelopmentof autoimmunediseasesBrugmanetal.2006;Dicksvedetal.2008;Tjellstrometal. 2005;Wenetal.2008.Theserecentresultsencouragetheuseofhumanstool samplestosearchforbacteriathatarecorrelatedwiththeonsetoftype1diabetes T1Dinchildren.BacterialDNAfromthestoolsamplesofcontrolandcasechildrenare extracted,then16SRNAamplicationfollowedby454pyrosequencingisdonetoobtain theOTUsOperationalTaxonomicUnit.Anystrandofasequenceofnucleotidesthat matchesanyknownsequenceinthegenelibraryistermedanOTU. Thedataobtainedconsistofmatchedcase-controlpairsofchildrenwithT1Dover differenttimepoints.WehaveperformedouranalysisonthedatabytheChi-squaretest followedbyFisher'scombining.ThisisdiscussedinthepaperbyGiongoetal.2011. Now,wearetryingtoimproveontheanalysisbytakingintoaccountthedependent structureofthedata.Themethodweemployisbasedongeneratingcontingencytables withxedmargins,takingthedependentstructureofthedataandthenperformingthe Chi-squaretest.Wediscussindetail,inthesubsequentsections,thedatastructureand Fisher'scombininganalysis,itsshortcomings,followedbythemethodweproposefor theimprovedanalysis. 5.2Motivation Goalistoanswerthequestion:Dohumanintestinalmicrobesplayaroleinthe developmentofautoimmunitythatoftenleadstotype1diabetesT1D?Priortostudy inhumansseveralstudieshaveshownthatgutbacteriaplayaroleindiabetesinmurine 72

PAGE 73

models.Also,inthestudyithasbeenfoundthatsomespecicbacteriaarecorrelated withtheonsetofdiabetesinaratmodel. HumanstoolsamplesforsuchananalysishavebeencollectedbytheDiabetes PredictionandPreventionstudyDIPPinFinland.DIPPhasbeencollectingstool samplesfromchildren,basedontheirgenotype,since1994.Atbirth,theHLA-DQ genotypesofbabiesaredetermined.ThoseinfantswhopossessspecicHLA genotypesareconsideredtobeathighriskforautoimmunityandprogressiontoward T1Dearlyinlife. 5.2.1DataDescription Whenchildrenenterthestudy,stoolsamplesarecollectedeverythreemonths.The samplesusedinthisworkcamefromatotalofeightFinnishchildren,eachrepresented bythreestoolsamplescollectedatthreetimepoints,foratotalof 24 separatesamples. Thecasechildrenalldevelopedautoimmunityandeventuallytype1diabetesover time.Autoimmunitywasdiagnosedbytheappearanceofatleasttwoautoantibodies. Eachofthesecasesismatchedwiththreesamplesfromachildofthesameageand HLA-DQgenotypewhodidnotbecomeautoimmuneduringthestudy.DNAextraction, 16SrRNAamplication,andpyrosequencingwereperformedinwhichanaverageof 15,709sequenceswereobtainedforeachofthe 24 samples.Thesequenceswere groupedintoOTUswhichwerethenprocessedandanalyzedtodeterminedifferencesat alltaxonomicandcommunitylevelsusingPANGEAGiongoetal.2010a. 5.2.2ImportantFeaturesoftheData Thefollowingaresomeimportantfeaturesofthecase-controldatathatshouldbe takenintoaccountwhenperformingtheanalysis: Somecellcountsareaslowaszeroandsomeareashighas 12000 Notall 4 case-controlpairscorrespondingtoeachtimepointhavenon-zero marginals. 73

PAGE 74

Foragiventimepointthecase-controlpairsareindependentbutthereis dependencyinthecase-controlpairsacrosstimepointsbecausethesame childrenareobservedineachtimepoint. 5.2.3Analysis Todeterminewhetherspecicclustersofbacteriadifferbetweenenvironments caseandcontrol,foragiventimepoint,basedona 2 2 contingencytable,a Chi-squaretestbasedon50,000MonteCarloiterationsisperformedforeachmatched case-controlpaircorrespondingtoeachOTUandap-valueisobtained.TheChi-square testperformedisanexacttestoranapproximatetestdependsonthecountsineach cellofthe 2 2 contingencytable.Wehavefoundthatforsomecase-controlpairs,the 2 2 contingencytablehaszerocounts,forthosetablesaChi-squaretestisnotvalid, soweeliminatethosetablesfromouranalysis.Foragiventimepoint,thep-values obtainedfromdifferentsubjectsarecombinedbythemethodofFishercombining Goutisetal.1996.Finally,thep-valuesobtainedfordifferenttimepointsareagain combinedbyFisher'smethodtogetanoverallp-value. 5.2.4Fisher'sCombining Fisher'scombiningmethodgoesbackto1932,whichisstillaverypopular techniqueofcombiningevidencefromdifferentsourcesbycombiningp-values.The p-valuesarecombinedusingan evidentialstatistic whichisdenedasafunctionofthe datathattakesvaluesin [0,1], andsmallvaluesindicatethatalternativehypothesisis trueandlargevaluesindicatenullhypothesisistrue. Givenasetofp-values p 1 p 2 ,... p k wecancombineandderivean evidential statistic s = s p 1 p 2 ,..., p k Weconsiderarandomvariable U i whoseobservedvalue istheactualp-value, p i Fisherproposedtheteststatistictobe Q k i =1 U i Weknowthat U i Uniform ,1 under H 0 Therefore, S = )]TJ/F22 11.9552 Tf 9.298 0 Td [(2 P i log U i 2 2 k and s = )]TJ/F22 11.9552 Tf 9.299 0 Td [(2 P i log p i is theobservedvalueoftheteststatistic S whichisusedtogettheoverallp-value. 74

PAGE 75

5.2.5Shortcomings Asmentionedintheprevioussection,Fisher'scombiningofp-valuesworkswhen theindividualexperimentsperformedareindependent.Itistobetobenotedthatinour casethep-valuescorrespondingtodifferenttimepointsarenotindependentbecause thesamecase-controlpairsareobservedforthreedifferenttimepoints.Butforany giventimepointthecase-controlpairsareindependent.Thus,combiningp-valuesby Fisher'smethodforanygiventimepointisperfectlyvalidbutcombiningp-valuesacross timepointsmaynotbevalid,sincewedonotknowthepreciseeffectofthedependence. Butalsonotethatthecomparisonswithintheexperimentarevalid,asweperformthe sametestonallOTUs. 5.3ImprovedAnalysis ThemodiedrandomwalkapproachdiscussedinChapter4isusedtogenerate contingencytableswithxedrowandcolumnmargins,whichprovidebetterestimatein termsofasymptoticefciency.Inthischapter,ouraimistogeneratecontingencytables withonemarginxed,takingintoaccountaspeciccorrelationstructure.Correlation structureisimportantsincethedataisamatchedcase-controlstudy,whichisagain measuredovertime. Wealsoproposeanewrandomwalkapproachtogeneratetables,whichcondition ononesetofmarginsinthiscasethesearecolumnstotals,whichisrelevanttothe type1diabetesdata.ThecorrectstationaryispreservedbytheMetropolis-Hastings algorithm.Atrst,wegeneratecontingencytableswithxedmarginscorresponding toeachOTU,takingintoaccountthepairwisecasecontrolstructureforanygiventime pointwithstationarydistributionascorrelatedbinomialmodel,whichisdiscussedbelow. 5.4Case-ControlDependencyandItsDistribution Thecase-controldatastructuredescribedabovehasinherentdependencyis becauseofthematchthatisbeingmadeonthebasisoftheageofthechildren andgenotype.Foranygiventime,thepairsareindependent.Letusrstobtainthe 75

PAGE 76

distributionofthedependentpairs.Considerthefollowing 2 2 contingencytableforthe k th pair, k =1,2,..., K Table5-1. 2 2 contingencytable CaseControl Success Y k 1 Y k 2 Failure n 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(Y k 1 n 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(Y k 2 Total n 1 n 2 where Y k 1 denotesthesuccesscountsoftheBinomialrandomvariableforthecases forthe k th pairand Y k 2 denotesthesuccesscountsoftheBinomialrandomvariables forthecontrolsofthe k th pair.Let p 1 betheprobabilityofsuccessofacaseand p 2 betheprobabilityofsuccessforacontrol.Notethatmarginallyboth Y k 1 and Y k 2 are Binomial n 1 p 1 andBinomial n 2 p 2 respectively.Thejointdistributionof Y k 1 and Y k 2 is givenbyBiswasandHwang2002, f y k 1 y k 2 =P Y k 1 = y k 1 Y k 2 = y k 2 = n 1 y k 1 p y k 1 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 1 n 1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(y k 1 f y k 2 j y k 1 where f y k 2 j y k 1 =+ )]TJ/F40 7.9701 Tf 6.587 0 Td [(n 1 X j 1 j 2 j 3 2X y 1 j 1 n 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(y k 1 j 2 n 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(n 1 j 3 f p 2 + p 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 1 + g j 1 f 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 2 )]TJ/F25 11.9552 Tf 11.955 0 Td [( p 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 1 g y k 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(j 1 f p 2 + p 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 1 g j 2 f 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 2 )]TJ/F25 11.9552 Tf 11.955 0 Td [( p 2 )]TJ/F39 11.9552 Tf 11.956 0 Td [(p 1 + g n 1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(y k 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(j 1 p j 3 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 2 n 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(n 1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(j 3 with X = f j 1 j 2 j 3 : j 1 + j 2 + j 3 = y 2 ; j 1 =0,1,..., y k 2 ; j 2 =0,1,..., n 1 )]TJ/F39 11.9552 Tf 12.874 0 Td [(y k 2 ; j 3 = 0,1,..., n 2 )]TJ/F39 11.9552 Tf 10.644 0 Td [(n 1 g Thus f y k 2 y k 1 givesjointdistributionofthecorrelatedbinomialvariables withcorrelationgivenby = r m M 1+ s p 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 1 p 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p 2 where m =min n 1 n 2 and M =max n 1 n 2 76

PAGE 77

Hereweareinterestedinndingthedistributionunderthenull H 0 : p 1 = p 2 = p Hencethedistributionof f y k 2 j y k 1 reducesto f y k 2 j y k 1 =+ )]TJ/F40 7.9701 Tf 6.586 0 Td [(n 1 X j 1 j 2 j 3 2X y 1 j 1 n 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(y k 1 j 2 n 2 )]TJ/F39 11.9552 Tf 11.955 0 Td [(n 1 j 3 f p + g j 1 f 1 )]TJ/F39 11.9552 Tf 11.956 0 Td [(p g n 2 )]TJ/F40 7.9701 Tf 6.586 0 Td [(n 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(j 3 + y k 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(j 1 p j 2 + j 3 f 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p + g n 1 )]TJ/F40 7.9701 Tf 6.586 0 Td [(y k 1 )]TJ/F40 7.9701 Tf 6.587 0 Td [(j 1 Let y 1 = y 1 1 y 2 1 ,..., y K 2 and y 2 = y 1 2 y 2 2 ,..., y K 2 .Thusthejointdistributionofallthepairs withinaparticulartimepointisgivenby f y 1 y 2 = K Y k =1 f y k 1 y k 2 ThisisthestationarydistributionoftheMarkovchainwhenthecase-controlpairsare correlatedforanygiventime-point.Thoughthisexpressionseemsverycomplicated,the samplescanbegeneratedveryeasilyfromthisdistribution,whichisdiscussedbelow. Generatingcontingencytables .Let Y 1 j denotesthesuccesscountsofthe Bernoullirandomvariableforthecasesand Y 2 j denotesthesuccesscountsofthe Bernoullirandomvariablesforthecontrols.WewanttogeneratetheBernoullivariables, under H 0 : p 1 = p 2 = p suchthatthetotalnumberofcasesandcontrolsare n 1 and n 2 respectively.Assume n 2 n 1 1.Generate Y 1 j for j =1,2,..., n 1 from Bernoulli p 2.Given Y 1 j ,generate Y 2 j from Bernoulli p + Y 1 j 1+ forsome andfor j =1,2,...,min m 1 n 1 3.If n 2 > min n 1 n 2 ,thengenerate Y 2 j from Bernoulli p for j = m 1 +1,..., n 2 Notethattheunconditionaldistributionof Y 2 j inStep2isalso Bernoulli p under H 0 Now,weprovideanexamplebelow,whichshowstheeffectofcorrelationonthep-value ofatest. 77

PAGE 78

Example5.1. Considertwo 2 2 contingencytablesforpair1andpair2giveninTable 5-2and5-3. Table5-2.Case-controlpair1withtotalsamplesize 27 Pair1Case1Control1 Success76 Failure59 Total1215 Table5-3.Case-controlpair2withtotalsamplesize 25 Pair2Case2Control2 Success65 Failure410 Total1015 LetPair1denotethepairCase1,Control1andPair2denotethepairCase2, Control2.Wegeneratethetablesfromthecorrelatedbinomialdistribution5.We thencomputethe 2 statisticvaluefor 10,000 MonteCarlosimulationsandcomputethe p-valuesundervariousdegreesofcorrelation, underthehypothesis H 0 : p 1 = p 2 = 1 = 2, whichislistedinthetablebelow: Table5-4.P-valuesfordifferentvaluesof Pair1Pair2 0.00.38610.2237 0.20.31910.1771 0.50.21730.0822 0.80.05710.0065 FromtheTable5-4,weobservethat,asthedegreeofcorrelationbetweenthe pairsincreases,thep-valuesdecreases,whichimpliesthatthetestbecomesmoreand moresignicant.Totestthenullhypothesis, H 0 : p 1 = p 2 = p ifweknowthatthereis signicantdependencebetweenthepairs,thentheconclusiondrawnfromthetestis morepowerfulthantheFisher'sexacttest.So,apropercorrelationstructureneedsto bespeciedinordertogetavalidconclusionfromatest.Fisher'sexacttestgivesthe p-valueforPair1equalto 0.4373 andp-valueforPair2equalto 0.2489 ,whichshowsthe Fisher'stestisconservativethanthecorrelatedBinomialtest. 78

PAGE 79

Metropolis-Hastingsalgorithm .Wecangeneratethecontingencytablesbya randomwalkmethoddiscussedinChapter4,ifwewanttotestwhethercase-control isindependentofthesuccessandfailure.TherandomwalkapproachinChapter4 conditionsonallmargins,whichistheproposaldistributionfortheMetropolis-Hastings algorithm.Weconsidermovessothattheproposaldistributionissymmetric.TheMH algorithmisusedinpreservingthecorrectstationarydistributionofindependence Section4.6. Butforthetype1diabetesdata,itismorerelevanttoconditiononthecolumn totals,sinceweknowthattheresultobtainedbyconditioningonallmarginscanbevery differentfromconditioningononesetofmarginsCasellaandMoreno2009.Similarto therandomwalkapproachinChapter4,wedeviseanewapproachwhichxesonlythe columntotals. Forourstudyweconsider 2 n contingencytables.Lettherebe 2 rowsand n columnsandcolumnsumsvectors c = c 1 c 2 ,..., c n suchthat N = P n j =1 c j Thesetof contingencytable, C c isthesetof 2 n non-negativeintegermatrices T = f t ij g 2 n that satisfythegivencolumnsum.Also,itistobenotedthatonlytableswhichsatisfythe non-negativityineachcellofthetable T areconsideredintheset C c .Innotation, C c = t ij 0, i =1,2, j =1,2,.., n : 2 X i =1 t ij = c j N = n X j =1 c j Togeneratefrom f in5,weproposearandomwalkasfollows:Let t beatable whichsatisestheconstraintsin C c Modify t bychoosingacolumnatrandom.Thendo steps [+ )]TJ/F22 11.9552 Tf 9.298 0 Td [(] or [ )]TJ/F22 11.9552 Tf 12.622 0 Td [(+] onthetworowswithprobability 1 = 2 each.Thisdoesnotchange thecolumnsums.Ifthemodicationforcesnegativeentries,westayatthecurrenttable. Anotherobviouschoiceoftheproposaldistributionistheproductoftwoindependent binomialdistributions.BytheMetropolis-Hastingsalgorithm,wepreservethestationary distribution f ,whichgivesareversibleMarkovchain.TheMetropolisstepsare describedasfollows: 79

PAGE 80

Giventhecurrentstate, y 1 y 2 Togotothenextstep, y 0 1 y 0 2 ,wefollowthe procedurebelow: 1.Generate y 0 1 y 0 2 fromtheproposaldistribution g y 0 1 y 0 2 j y 1 y 2 2.Calculate f y 1 y 2 and f y 0 1 y 0 2 3.Generate U Uniform[0,1]. Accept y 0 1 y 0 2 if U < min f y 0 1 y 0 2 g y 1 y 2 j y 0 1 y 0 2 f y 1 y 2 g y 0 1 y 0 2 j y 1 y 2 ,1 Weprovideasimpleillustrationoftheabovetechniquewiththehelpofanexample below. Example5.2. Considera 2 2 contingencytableForthisexample,Fisher'sexact Table5-5.Case-controlpairwithtotalsamplesize 48 CaseControl Success57 Failure1517 Total2024 testgivesap-valueof 1. Wesummarizethep-valuesfortestinghypothesis H 0 : p 1 = p 2 =1 = 2 forthetablesgeneratedfromtheactualdistributionMonteCarlo generationandfromtheMetropolis-HastingsMHstepbasedon 10,000 simulations fordifferentdegreesofcorrelation.Wealsoobservethatinthisexample,asthe Table5-6.P-valuesfordifferentvaluesof ,fromMonteCarloandMHmethods MonteCarloMHrandomwalkMHIndependentbinomials 0.00.88920.98480.8862 0.20.87920.95420.8730 0.50.86150.94670.8579 0.80.85310.92030.8483 correlationincreasesthep-valuedecreases.So,weconcludethatthedegreeof correlationmustbetakenintoaccountwhendoinginferencebasedonahypothesistest. Nextweprovideapplicationofthetechniquesmentionedearliertothetype1 diabetescase-controldata. 80

PAGE 81

Example5.3. Considertwo 2 2 contingencytablesTable5-7and5-8,eachdescribing apairofcaseandcontrol. Table5-7.Case-controlpair1 Pair1Case1Control1 Success39 Failure1045310498 Total1045610507 Table5-8.Case-controlpair2 Pair2Case2Control2 Success4122 Failure91019231 Total91429253 Hereweareinterestedintestingthenullhypothesis H 0 : p 1 = p 2 Thecomputed 2 valuesforTable5-7and5-8are 2.9726 and 5.381 respectively,andtheircorresponding p-valuesfromtheFisher'sexacttestare 0.1455 and 0.01649 .Wesummarizethe p-valuesforthetables5-7and5-8generatedfromthecorrelatedBinomialdistribution MonteCarlomethodandfromtheMetropolis-HastingsMHtakingproposaltobe independentbinomialsandstationarydistributiontobecorrelatedBinomialdistribution methodsbasedon 10,000 simulationsfordifferentdegreesofcorrelation, Table5-9.P-valuesfordifferentvaluesof fromMonteCarloandMHmethodsforTable 5-7and5-8. Pair1Pair1Pair2Pair2 MonteCarloMHMonteCarloMH 0.00.08940.08820.01450.0111 0.20.05850.06040.00610.0008 0.50.01790.02770.00030.0005 0.80.00180.00930.00000.0000 Wealsoobservethatinthisexample,asthecorrelationincreasesthep-value decreases.So,weconcludethatthedegreeofcorrelationmustbetakenintoaccount whendoinginferencebasedonahypothesistest. 81

PAGE 82

CHAPTER6 CONCLUSIONSANDFUTURERESEARCH InChapter2,wemainlyfocusourattentiononthenitestateMarkovchains, developingvariousmethodstoconstructabetterMarkovchainfromagivenchain intermsofincreasingtheasymptoticefciency.Inthisregard,ourmostprominent approachisthedevelopmentofan optimalalgorithm whichsequentiallymovesmass fromthediagonalstotheoff-diagonals,preservingthestationarydistributionofthe Markovchain.WealsoproposeaconstructionmethodthatusesMetropolis-Hastings algorithm.Fromthe optimalalgorithm ,weobtain P opt atthenalstep,whichcannot beimprovedinPeskun'ssense.Weplantoinvestigateonafunction, f suchthat asymptoticvarianceobtainedatthenalstepisaminimum. InSection2.9,wewouldalsoliketoconstructafunction, f and Q suchthat PQ is asymptoticallymoreefcientthan P Tierney1998extendtheresultofPeskunorderingfromdiscretestatespacesto continuousstatespaces.Wearetryingtoextendthisideainthecaseofcontinuous stateMarkovchains.Next,weemphasizeontheconstructionofabettertransition kernelintermsofasymptoticvariance,extendingtheapproachofTheorem2.7,by applyingtheMetropolis-Hastingsalgorithm.Tillnow,wemainlyfocusourattentionon theorderingofasymptoticvariancesforreversibleMarkovchains.Fornon-reversible Markovchains,wemainlyestablishcovarianceordering,withour optimalalgorithm Nowwewilltrytoexploremoretheareaconcerningnon-reversibleMarkovchains Diaconisetal.2000;MiraandGeyer1999andestablishorderingintheasymptotic variancesfortheseMarkovchains. InChapter3,weuseMetropoliswithinGibbstechniqueforadiscreteandnite statespaceMarkovchainandweshowthatthemarginalchaininthetwostageGibbs samplerisasymptoticallymoreefcientthantheordinarytwostageGibbssampler. 82

PAGE 83

WeplantoextendtheMetropoliswithinGibbstechniqueincaseofcountablestate spaceMarkovchainsandtothecontinuousstatespaceMarkovchains.Weplanto investigatethewellknownBeta-Binomialdistribution,whereonepartisdiscreteand oneiscontinuous,intermsofitsasymptoticefciencyusingtheMetropoliswithinGibbs technique. InChapter4,weextendthemethodsofconstructinganasymptoticallyefcient MarkovchainsfromnitestatespaceMarkovchainsChapter2tocountablyinnite statespaceMarkovchains.Weprovideexpressionfortheasymptoticvarianceofan ergodicaverageofaMarkovchain.Animportantapplicationoftheseresultsisinthe generationofcontingencytablesbyarandomwalkapproachDiaconisandGangolli 1995. InChapter5,forthetype1diabetesdata,thecorrelatedBinomialmodeltakescare ofthecorrelationbetweencaseandcontrolpair.WeobtainthattheFisher'sexacttest isconservativethanthecorrelatedtest.Next,weplantomodelthedependencyacross timepointsandthengeneratethecontingencytablesandperformtheChi-squaretest ontheentire 2 K table.Now,totakeintoaccountthecorrelationbetweentime-points, wewouldliketoextendtheideatothecorrelationbetweentwobivariatecorrelated Binomialmodel.Thismodelwilltakecareofthecorrelationbetweencase-controlpair andtime-points.Wethenuseindependentbinomialrandomvariablesasaproposal distributionandcorrectthestationarydistributionbyaMetropolis-Hastingsstep. Weplantofocusourattentionincomparingtheasymptoticvariancesbetweenthe estimatesobtainedfromtheMonteCarlosimulationtothatgeneratedfrombythe Metropolis-Hastingsalgorithm.WehopetoarguethecomparisonbythePeskun ordering. Inthecase-controlstudy,tillnow,weconsidertwocategories,successandfailure. Infuture,weplantoextendtheideatomorethantwocategories. 83

PAGE 84

Wearealsoworkingonttingageneralizedlinearmixedmodelforababydiet datafromGermany,whichisalsoacase-controlstudyacrosstimepointslikeFinland studyChapter5butwithcovariates,whichmaybetimedependent.Weuseaarcsine transformationontheresponsevariable,sincetheresponsevariablesareproportions. Inthisstudy,thecorrelationbetweenmatchedcase-controlpairistakenintoaccountby specifyingarandomeffectterminthemodelandthecorrelationacrosstimepointsare takenintoaccountbyspecifyinganARcorrelationstructureinthemodel. 84

PAGE 85

APPENDIXA THEOREMSFROMCHAPTER2 A.1ProofofTheorem2.2 Proof. Part1.Toshow1 2. Q )]TJ/F48 11.9552 Tf 12.801 0 Td [(P isapositivesemidenitematrix h 0 Q )]TJ/F48 11.9552 Tf 12.801 0 Td [(P h 0 ,forall h 2 R m .Let h = D 1 = 2 g = g 0 D 1 = 2 Q )]TJ/F48 11.9552 Tf 12.56 0 Td [(P D 1 = 2 g 0 .Theeigenvaluesof D 1 = 2 Q )]TJ/F48 11.9552 Tf 12.56 0 Td [(P D 1 = 2 are equaltotheeigenvaluesof D Q )]TJ/F48 11.9552 Tf 12.314 0 Td [(P Therefore,forall f 2 R m f 0 D Qf f 0 D Pf Cov f Q Cov f P Part2.Toshow1 3. Wehavetheexpressionof v f , P from2as v f , P = f 0 D f 2 I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(P )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 )]TJ/F48 11.9552 Tf 11.956 0 Td [(I )]TJ/F48 11.9552 Tf 11.955 0 Td [( g f Weknowthat Q )]TJ/F48 11.9552 Tf 11.956 0 Td [(P isapositivesemidenitematrix h 0 Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P h 0 ,forall h 2 R m h 0 h + h 0 h + h 0 Q )]TJ/F48 11.9552 Tf 11.955 0 Td [(P h h 0 h + h 0 h ,forall h 2 R m h 0 I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(P h h 0 I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q h I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(P I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q ItiswellknownfromL ownerordering,Lowner1934thatforthepositivedenite Hermitianmatrices A and B thedifference B )]TJ/F48 11.9552 Tf 12.505 0 Td [(A isnonnegativedeniteifandonlyif A )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 )]TJ/F48 11.9552 Tf 11.955 0 Td [(B )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 isnonnegativedenite,orstatedotherwise: A B ifandonlyif B )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 A )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 Since, P and Q arereversibletransitionmatrices,therefore D I + )]TJ/F48 11.9552 Tf 12.746 0 Td [(P and D I + )]TJ/F48 11.9552 Tf 11.145 0 Td [(Q aresymmetricmatrices.Therefore, I + )]TJ/F48 11.9552 Tf 11.145 0 Td [(P )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 D )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 I + )]TJ/F48 11.9552 Tf 11.145 0 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 D )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 byL ownerordering,Lowner1934.Thus, I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(P )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Forall x 2 R m x 0 I + )]TJ/F48 11.9552 Tf 13.064 0 Td [(P )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 x x 0 I + )]TJ/F48 11.9552 Tf 13.064 0 Td [(Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 x .Let g = D 1 = 2 x ,then g 0 D 1 = 2 I + )]TJ/F48 11.9552 Tf 11.998 0 Td [(P )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 D 1 = 2 g g 0 D 1 = 2 I + )]TJ/F48 11.9552 Tf 11.998 0 Td [(Q )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 D 1 = 2 g .Theeigenvaluesof D 1 = 2 I + )]TJ/F48 11.9552 Tf -457.177 -23.908 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 D 1 = 2 areequaltotheeigenvaluesof D I + )]TJ/F48 11.9552 Tf 12.491 0 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 Therefore,forall f 2 R m f 0 D I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(P )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 f f 0 D I + )]TJ/F48 11.9552 Tf 11.955 0 Td [(Q )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 f v f , P v f , Q 85

PAGE 86

A.2ProofofTheorem2.6 Proof. Since P j p i j =1 ,increasing p k k meansthatatleastoneofthe p k j will decrease.Wewillonlyletonedecreaseatatime;thisdoesnotlosegeneralityaswe canalwaysincreasethe p k j sequentially.Nowwewilldifferentiatewithrespectto p k k andallowtheoff-diagonalelement p k ` tomove.Theonlyotherelementsof P thatareinvolvedare p ` k and p ` ` ,andwecanwrite p k ` =1 )]TJ/F39 11.9552 Tf 11.956 0 Td [(p k k )]TJ/F30 11.9552 Tf 11.955 8.966 Td [(P j 6 = k ` p k j @ p k ` @ p k k = )]TJ/F22 11.9552 Tf 9.298 0 Td [(1. p ` k = k ` p k ` @ p ` k @ p k k = )]TJ/F26 7.9701 Tf 10.494 5.112 Td [( k ` p ` ` =1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(p ` k )]TJ/F30 11.9552 Tf 11.955 8.966 Td [(P j 6 = k ` p ` j @ p ` ` @ p k k = k ` Thenwehave, @ @ p kk 4 X i f 2 i i p i i + X j 6 = i f i + f j 2 i p i j # =4 f 2 k k +4 f 2 ` ` k ` )]TJ/F22 11.9552 Tf 11.955 0 Td [( f k + f ` 2 k )]TJ/F22 11.9552 Tf 11.955 0 Td [( f ` + f k 2 ` k ` =2 k 2 f 2 k +2 x 2 ` )]TJ/F22 11.9552 Tf 11.956 0 Td [( f k + f ` 2 =2 k f k )]TJ/F39 11.9552 Tf 11.956 0 Td [(f ` 2 0. Sothefunction2isincreasingineachofitsdiagonalelement, p k k 86

PAGE 87

APPENDIXB SOMECONCEPTSNEEDEDFORCHAPTER4 B.1Denitions DenitionB.1 Inner-productspace Acomplexvectorspace H issaidtobeainnerproductspace,ifforeverypair x and y in H thereisacomplexnumber h x y i called theinnerproductof x and y suchthat a h x y i = h y x i b h x + y z i = h x z i + h y z i c h x x i 0 forall x 2H d h x x i =0 ifandonlyif x =0. Arealvectorspace H isaninner-productspace,ifforeach x y 2H thereexistsa realnumber h x y i satisfyingconditionsa-d.Ofcourseconditionareducesto h x y i = h y x i DenitionB.2 Hilbertspace AHilbertspace H isaninner-productspacewhichis complete,i.e.,aninner-productspaceinwhicheveryCauchysequence f x n g converges innormtosomeelement x 2H DenitionB.3 Positiveoperator Anoperator A issaidtobe positive denotedby A > 0 if h Ax x i > 0 forall x 2H B.2Theorems TheoremB.1 Schempp,1970 Let H beacomplexHilbertspace, A 2H anormal operatorand 1 = 2 Sp A .Supposethatthegeneralsummationmethod S sumsupthe geometricseries P n 0 z n onthespectrum Sp A suchthat S X n 0 z n = X n 0 v n z = )]TJ/F39 11.9552 Tf 11.955 0 Td [(z )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 forallpoints z 2 Sp A .Thenwehave X n 0 v n A = I )]TJ/F39 11.9552 Tf 11.955 0 Td [(A )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 87

PAGE 88

inthe Banachalgebra L H TheoremB.2 Furuta,1997 Suppose A and B areboundedlinearoperatorsona Hilbertspace H .Also,suppose A and B areselfadjointsandalso A B > 0 see denitionB.3.Then B )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 A )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 > 0 B.3SizeoftheStateSpace DiaconisandEfron1985gaveanapproximationstocomputethesizeofthestate space C rc whichworkswellwhennumberofrowsand/orcolumnsaresmallbut N is large.Let w = 1 1+ mn = 2 N k = n +1 c P i r i 2 )]TJ/F22 11.9552 Tf 13.253 8.088 Td [(1 n r i = 1 )]TJ/F39 11.9552 Tf 11.956 0 Td [(w m + wr i N c j = 1 )]TJ/F39 11.9552 Tf 11.955 0 Td [(w n + wc j N ThenDiaconisandEfronsuggest jC rc j 2 N + mn 2 m )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 n )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 m Y i =1 r i n )]TJ/F23 7.9701 Tf 6.587 0 Td [(1 n Y j =1 c j k )]TJ/F23 7.9701 Tf 6.586 0 Td [(1 \050 nk \050 n m \050 k n 88

PAGE 89

REFERENCES B EZAKOVA ,I.,B HATNAGAR ,N.andR ANDALL ,D..OntheDiaconis-Gangolli Markovchainforsamplingcontingencytableswithcell-boundedentries.In Computing andCombinatoricsNiagaraFalls,NY .Springer-Verlag,307. B ILLERA ,L.andD IACONIS ,P..Ageometricalinterpretationofthe Metropolis-Hastingsalgorithm. StatisticalScience 16 335. B ISWAS ,A.andH WANG ,J..Anewbivariatebinomialdistribution. Statisticsand ProbabilityLetters 60 231. B RUGMAN ,S.,K LATTER ,F.andV ISSER ,J..Antibiotictreatmentpartiallyprotects againsttype1diabetesinthebio-breedingdiabetespronerat.isthegutorainvolved inthedevelopmentoftype1diabetes? Diabetologia 49 2105. B UTLER ,R.W.andS UTTON ,R.K..Saddlepointapproximationformultivariate cumulativedistributionfunctionsandprobabilitycomputationsinsamplingtheoryand outliertesting. JournaloftheAmericanStatisticalAssociation 93 596. C ASELLA ,G.andG EORGE ,E..ExplainingtheGibbssampler. TheAmerican Statistician 46 167. C ASELLA ,G.andM ORENO ,E..Assessingrobustnessofintrinsictestsof independenceintwo-waycontingencytables. JournaloftheAmericanStatistical Association 104 1261. D AHL ,G..Matrixmajorization. LinearAlgebraandItsApplications 288 53. D IACONIS ,P.andE FRON ,B..Testingforindependenceinatwo-waytable:New interpretationsofthechi-squarestatistic. TheAnnalsofStatistics 13 845. D IACONIS ,P.andG ANGOLLI ,A..Rectangulararrayswithxedmargins.In DiscreteProbabilityandAlgorithmsMinneapolis,MN .Springer-Verlag,NewYork, 15. D IACONIS ,P.,H OLMES ,S.andN EAL ,R.M..AnalysisofanonreversibleMarkov chainsampler. TheAnnalsofAppliedProbability 10 726. D IACONIS ,P.andS TURMFELS ,B..Algebraicalgorithmsforsamplingfrom conditionaldistributions. TheAnnalsofStatistics 72 363. D ICKSVED ,J.,H ALFVARSON ,J.andR OSENQUIST ,M..Molecularanalysisofthe gutmicrobiotaofidenticaltwinswithCrohn'sdisease. ISME 2 716. F RIGESSI ,A.,H WANG ,C.andY OUNES ,L..Optimalspectralstructureof reversiblestochasticmatrices,MonteCarlomethodsandthesimulationofMarkov randomelds. TheAnnalsofAppliedProbability 2 610. 89

PAGE 90

F URUTA ,T..EquivaluencerelationsamongReid,Lowner-HeinzandHeinzKato inequalitiesandextensionsoftheseinequalities. IntegralEquationsandOperation Theory 29 1. G IONGO ,A.,C RABB ,D.B.,D AVIS -R ICHARDSON ,A.G.,C HAULIAC ,D.,M OBBERLEY J.M.,G ANO ,K.,M UKHERJEE ,N.,C ASELLA ,G.,R OESCH ,L.F.,W ALTS ,B.,R IVA A.,K ING ,G.andW.T RIPLETT ,E.a.Pangea:Pipelineforanalysisofnext generationamplicons. TheISMEJournal 4 852. G IONGO ,A.,G ANO ,K.,C RABB ,D.B.,M UKHERJEE ,N.,N OVELO ,L.L.,G.C ASELLA J.C.D.,I LONEN ,J.,K NIP ,M.,H YOTY ,H.,V EIJOLA ,R.,S IMELL ,T.,S IMELL ,O., N EU ,J.,W ASSERFALL ,C.H.,S CHATZ ,D.,A TKINSON ,M.A.andT RIPLETT ,E.W. .Towarddeningtheautoimmunemicrobiomefortype1diabetes. TheISME Journal 5 82. G OUTIS ,C.,C ASELLA ,G.andW ELLS ,M..Assessingevidenceinmultiple hypotheses. JournaloftheAmericanStatisticalAssociation 91 1268. H ASTINGS ,W.K..MonteCarlosamplingmethodsusingMarkovChainsand theirapplications. Biometrika 57 97. K EMENY ,J.andS NELL ,J.. FiniteMarkovChains .Springer-Verlag,NewYork. K IPNIS ,C.andV ARADHAN ,S.R.S..Centrallimittheoremforadditive functionalsofreversibleMarkovprocessesandapplicationstosimpleexclusions. CommunicationsinMathematicalPhysics 104 1. L IU ,J..Peskun'stheoremandamodieddiscrete-stateGibbssampler. Biometrika 83 681. L OWNER ,K..UbermonotoneMatrixfunktionen. Math.Z 38 177. M ARSHALL ,A.andO LKIN ,I.. Inequalities:TheoryofMajorizationandIts Applications .AcademicPress. M ETROPOLIS ,N.,R OSENBLUTH ,A.W.,R OSENBLUTH ,M.N.,T ELLER ,A.H.and T ELLER ,E..Equationofstatecalculationsbyfastcomputingmachines. JournalofChemicalPhysics 21 1087. M IRA ,A..OrderingandimprovingtheperformanceofMonteCarloMarkov chains. StatisticalScience 16 340. M IRA ,A.andG EYER ,C.J..OrderingMonteCarloMarkovchains.Tech.rep., 632,SchoolofStatistics,Univ.Minnesota. M ULLER ,P..AgenericapproachtoposteriorintegrationandGibbssampling. Tech.rep.,Dept.ofStatistics,PurdueUniversity. 90

PAGE 91

M ULLER ,P..AlternativestotheGibbssamplingscheme.Tech.rep.,Instituteof StatisticsandDecisionSciences,DukeUniversity. P ESKUN ,P..OptimumMonteCarlosamplingusingMarkovchains. Biometrika 60 607. R OBERT ,C.andC ASELLA ,G.. MonteCarloStatisticalMethods .Springer-Verlag. S AAD ,Y.. IterativeMethodsforSparseLinearSystems .Siam. S CHEMPP ,W..IterativesolutionoflinearoperatorequationsinHilbertspaceand optimaleulermethods. ArchivderMathematik XXI 390. T IERNEY ,L..AnoteonMetropolis-Hastingskernelsforgeneralstatespaces. TheAnnalsofAppliedProbability 8 1. T JELLSTROM ,B.,S TENHAMMAR ,L.andH OGBERG ,L..Gutmicroora associatedcharacteristicsinchildrenwithceliacdisease. AmJGastroenterol 100 2784. W EN ,L.,L EY ,R.,V OLCHKOV ,P.andS TRANGES ,P..Innateimmunityand intestinalmicrobiotainthedevelopmentoftype1diabetes. Nature 455 1109. 91

PAGE 92

BIOGRAPHICALSKETCH NabanitaMukherjeeobtainedherBachelorofSciencedegreeinStatisticsfromSt. Xavier'sCollege,CalcuttaIndiain2003.ShedidherM.Stat.fromtheIndianStatistical Institute,Indiain2005.ShejoinedtheDepartmentofStatisticsattheUniversityof FloridainAugust2005forpursuingaPh.D.inStatistics.BesidesbeingaPh.D.student, shealsoworkedasaTeachingAssistantthere,entrustedwithteachingStatisticsin theundergraduateandthegraduatelevel.Shealsohadtheopportunitytoworkasa TeachingInstructorinanintroductorylevelcourseinStatistics.Moreover,sheworked asaResearchAssistantwithProf.EricTriplettandhisgroupfromtheDepartmentof CellScienceandMicrobiologyattheUniversityofFlorida.Herexperienceasasummer internatNovartisOncology,NewJerseyin2008wasrichandexhilarating.Sheearned theappreciationofall,shecameincontactwith,atNovartis.Shegraduatedfromthe UniversityofFloridawithaPh.D.degreeinStatisticsinthesummerof2011.Afterher Ph.D,shejoinedtheUniversityofPennsylvaniaasapostdoctoralfellow. 92