Citation
Bayesian Methodology for Models with Multivariate (Longitudinal) Outcomes

Material Information

Title:
Bayesian Methodology for Models with Multivariate (Longitudinal) Outcomes
Creator:
LIU, XUEFENG ( Author, Primary )
Copyright Date:
2008

Subjects

Subjects / Keywords:
Correlations ( jstor )
Covariance ( jstor )
Markov chains ( jstor )
Matrices ( jstor )
Mining ( jstor )
Modeling ( jstor )
Parametric models ( jstor )
Quit rates ( jstor )
Statistical models ( jstor )
Statistics ( jstor )

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
Copyright Xuefeng Liu. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Embargo Date:
8/31/2016
Resource Identifier:
658219655 ( OCLC )

Downloads

This item is only available as the following downloads:


Full Text

PAGE 4

IhavebeenluckytostudyattheUniversityofFloridawhereIhavelearnedmuchfromsomeofthemostdistinguishedstatisticiansintheworld.IhavebeenevenluckiertoworkcloselywithprofessorMichaelDaniels,bothmyadvisorandmyfriend.Hisinspiration,insight,patience,andconcernhelpedmethroughthesevaluableyearsintheUniversityofFlorida.IwishtoacknowledgemydebttoprofessorRandyCarterforhisprovidingmearesearchassistantshipfor3years.ProfessorsMalayGhosh,MyronChang,TrevorParkandXueliLiuhavehelpedinmanycrucialways.Also,IfeelfortunatetohaveprofessorAndreKhuriasmyfriend,whoseloveofstatisticsandbroadknowledgehavedirectedmetothinkharderanddeeper.Mostdeeply,Igivemyheartfeltthankstomyfamily.Theloveofmywifeandchildrenareessentialtome.Mywife'ssupportandmykids'smilehavealwaysbeenagreatimpetustome.Iwouldliketodedicatethispieceofworktothem. iv

PAGE 5

page ACKNOWLEDGMENTS ............................. iv LISTOFTABLES ................................. vii LISTOFFIGURES ................................ viii ABSTRACT .................................... ix CHAPTER 1INTRODUCTION .............................. 1 1.1OverviewofDissertation ........................ 1 1.2ReviewofJointModels ......................... 2 1.3StochasticSearchVariableSelectionApproaches ........... 5 1.4OverviewofMarkovChainMonteCarloMethodology ........ 7 1.4.1MarkovChainMonteCarloSimulation ............ 7 1.4.2TheMetropolis-HastingsAlgorithm .............. 9 1.4.3TheGibbsSampler ....................... 11 1.4.4DataAugmentationandAuxiliaryVariables ......... 13 1.4.5ParameterExpansion ...................... 16 1.4.6GeneralFrameworkforCharacterizingMissingData ..... 18 2PARAMETEREXPANSIONANDRE-PARAMETRIZATIONFORSIMULATINGACORRELATIONMATRIX:ANEFFICIENTPX-RPMHALGORITHM .......................... 22 2.1Introduction ............................... 22 2.2TwoCommonMethodstoSimulatetheCorrelationMatrix ..... 24 2.2.1TheGriddy-GibbsSampler ................... 24 2.2.2RandomWalkMetropolis-HastingsAlgorithms ........ 27 2.3MotivatingExamplesfortheNewAlgorithm ............. 28 2.3.1MultivariateProbitModels ................... 28 2.3.2MultivariateRegressionModels ................ 29 2.4ThePX-RPMHAlgorithm ....................... 30 2.4.1OverviewofAlgorithm ..................... 30 2.4.2ThePX-RPMHAlgorithmforMVPModels ......... 32 2.4.3ThePX-RPMHAlgorithmforMVRModelI ......... 34 2.5AGeneralizationofthePX-RPMHAlgorithm ............ 37 v

PAGE 6

........... 37 2.5.2TheGPX-RPMHAlgorithmforMVRModelII ....... 40 2.6ApplicationofthePX-RPMHAlgorithmtoOtherModels ..... 41 2.6.1ScaleMixtureofMultivariateNormalLinkModels ...... 41 2.6.2JointModelsforMixedLongitudinalProcesses ........ 44 2.7ASimulationStudy ........................... 47 2.8ARealDataExample ......................... 50 2.9Discussion ................................ 53 3JOINTMODELSFORTHEASSOCIATIONOFLONGITUDINALBINARYANDCONTINUOUSPROCESSES ............... 54 3.1Introduction ............................... 54 3.2MotivatingExample .......................... 55 3.3JointModelsandPriors ........................ 56 3.3.1JointModels ........................... 56 3.3.2PriorsforParametersinJointModels ............. 58 3.4PosteriorSampling ........................... 60 3.4.1TheMCMCSamplingAlgorithm ............... 60 3.4.2TechnicalIssues ......................... 62 3.5ModelSelection ............................. 68 3.5.1TheBayesFactor ........................ 68 3.5.2TheDevianceInformationCriterion .............. 75 3.6Example ................................. 76 3.6.1ModelComparison ....................... 77 3.6.2CheckingtheGoodnessofFit ................. 78 3.6.3InferenceontheQuitRatesandAssociation ......... 82 3.7ConclusionandDiscussion ....................... 84 4CONCLUSIONANDFUTURERESEARCH ............... 86 4.1StatisticalComputingandAlgorithmDevelopment ......... 86 4.2ModellingtheAssociationbetweenMixedLongitudinalProcesses ............. 86 4.3LatentClassModelsforPatternsofAssociationinEachClass ... 89 APPENDIX AAPROOFOFTHEOREMSANDPROPOSITIONS ........... 91 BDETAILSONSAMPLINGALGORITHMS ................ 108 CDETAILSONGIBBSSAMPLERANDCOMPUTATIONOFMARGINALLIKELIHOODS ................................ 112 REFERENCES ................................... 124 BIOGRAPHICALSKETCH ............................ 131 vi

PAGE 7

Table page 2{1Posteriormeansand95%credibleintervalsforthecorrelations,variances,andregressionparametersforthegrowthhormonedataexample. .... 51 3{1Modelsunderconsideration ......................... 77 3{2EstimatesofthemarginallikelihoodsandBayesfactorscomparedtoM5 3{3EstimatesoftheDIC ............................. 78 3{4Posteriorpredictivep-values ......................... 78 3{5Posteriormeansand95%credibleintervals(CIs)forthequitrates .... 81 3{6Posteriormeansoftheassociationmatriceswith95%CIsforeachtreatment .............................. 83 3{7Posteriormeansofpairwisecorrelationswith95%CIsforeachtreatment 84 vii

PAGE 8

Figure page 2{1Traceplotsof5componentsofRoverthelast2000iterationsof50,000iterationsforthePX-RPMHalgorithmandRW-MHalgorithm. ..... 48 2{2Traceplotsof3componentsofRoverthelast2000iterationsof10,000iterationsforthePX-RPMHalgorithmandRW-MHalgorithm. ..... 52 3{1Histogramsofdierencesbetweentherealizedandpredictedquitrates. . 79 viii

PAGE 9

Clinicaltrialsfrequentlycollectseveraloutcomesovertime.Asaresult,modelsfor(multivariate)longitudinaloutcomeshavereceivedmuchattentionintherecentliterature.Inthisdissertationwhichhastwoparts,weproposenewmethodologyformodellingsuchdata.TherstpartaddressesthegeneralproblemofsamplingacorrelationmatrixusingMarkovchainMonteCarloalgorithms.Thesecondpartproposesjointmodelsforabivariatelongitudinalprocessthatrequiresthesamplingofacorrelationmatrix. Thecorrelationmatrix(denotedbyR)playsanimportantroleinmanystatisticalmodels.Unfortunately,samplingthecorrelationmatrixinMCMCalgorithmscanbeproblematic.Inthisdissertation,weproposeanecienttwo-stageparameterexpandedreparametrizationandMetropolis-Hastings(PX-RPMH)algorithmforsimulatingR.ThetheoryofthePX-RPMHalgorithmanddetailsonitsimplementationareshownindetail.Usingthisalgorithm,wedrawallelementsofRsimultaneouslybyrstdrawingacovariancematrixfromaninverseWishartdistribution,andthentranslatingitbacktoacorrelationmatrixthroughareductionfunctionandacceptingitbasedonaMetropolis-Hastings ix

PAGE 10

Inthesecondpartofthisdissertation,weproposejointmodelsfortheassociationoflongitudinalbinaryandcontinuousprocesses.Themodelsareparameterizedsuchthatthedependencebetweenthelongitudinalbinaryandcontinuousprocessesischaracterizedbyunconstrainedregressioncoecients.Bayesianvariableselectiontechniquesareusedtoparsimoniouslymodelthesecoecientsforeachtreatment.AnMCMCsamplingalgorithmisdevelopedforsamplingfromtheposteriordistribution,usingdataaugmentationstepstohandlemissingdata.SeveraltechnicalissuesareresolvedinordertoimplementtheMCMCalgorithmmoreeciently.Formodelcomparisonandassessingmodelt,weadaptseveralstandardmeasuresandprovidecomputationaldetails.Themodelsareusedfortheanalysisofasmokingcessationclinicaltrialinwhichaquestionofinterestwastheeectofthe(exercise)treatmentontherelationshipbetweensmokingcessation(longitudinalbinaryprocess)andweightgain(longitudinalcontinuousprocess). x

PAGE 11

InChapter 2 ,wedevelopanewalgorithm,calledtheparameterexpandedreparametrizationandMetropolis-Hastings(PX-RPMH)algorithm,forsamplingacorrelationmatrix(R)basedontheideaofparameterexpansion.ThetheoryofthePX-RPMHalgorithmanddetailsonitsimplementationareshownindetail.Thesealgorithmsarederivedforseveralcorrelateddatamodels,includingmultivariateprobitmodels(ChibandGreenberg,1998),multivariateregressionmodels(ManlyandRayner,1987),scalemixtureofmultivariatenormallinkmodels(ChenandDey,1998,2000)andjointlongitudinalmodels(LiuandDaniels,2005). InChapter 3 ,wedevelopjointmodelsformodellingtheassociationbetweenlongitudinalbinaryandcontinuousprocesses.Weparameterizethemodelsuchthattheassociationbetweenprocesseshasasensibleinterpretationandiseasyto 1

PAGE 12

model.Stochasticsearchvariableselectiontechniquesareusedtoparsimoniouslymodeltheassociationmatrix.MCMCsamplingtechniquesaredevelopedforBayesianinference.SeveraltechnicalissuesareresolvedinordertoimplementtheMCMCalgorithmeciently.Formodelcomparisonandassessingmodelst,weadaptseveralmeasuresandprovidecomputationaldetails. InChapter 4 ,wediscusssomeopenquestionsandpossiblefutureareasforresearch. CatalanoandRyan(1992)utilizedalatentvariabletoderiveajointdistributionofdiscreteandcontinuousoutcomesandthenchoseageneralizedestimatingequation(GEE)approachtoestimateparametersinthemodels.Theyassumedthatthebinaryoutcomehadsomecorrespondingunobservedlatentvariableandthenassumedthatthecontinuousoutcomeandthelatentoutcomefollowedajointnormaldistribution.LetYi=(Yi1;;Yini)andWi=(Wi1;;Wini)beni1vectorsforthelatentandcontinuousvariablesforgroupi.Theirmodelwasgivenby0B@YiWi1CA=Xi+i;

PAGE 13

whereXi=0B@X1i00X2i1CAisthedesignmatrix,=0B@121CAisthevectorofregressioncoecientsandi=0B@1i2i1CAistherandomerrorwithanormaldistributionwith mean0andcovariancematrixVar(i)=0B@21[(11)I+1J]12[(12)I+12J]12[(12)I+12J]22[(12)I+2J]1CA: Theadvantageoftheirlatentvariableformulationisthatitleadstoasetofappealingcovariatesintheconditionalmodel.Furthermore,thecoecientsofthesecondmodelaredirectlyrelatedtothevarianceandcorrelationtermsfromtheunderlyinglatentvariablemodel.Becauseofthereparametrization,someparametersintheoriginalmodelwerenotestimable.FitzmauriceandLaird

PAGE 14

(1995)consideredasimilartwo-stagemodelbutreversedtheconditioningorder,specifyingamarginallogitmodelforthebinaryoutcomeandaconditionalmodelforthecontinuousoutcomegiventhebinaryoutcome.GueorguievaandAgresti(2001)proposedacorrelatedprobitmodelforjointlymodellingclusteredbinaryandcontinuousresponses,anddevelopedaMonteCarloexpectation-conditionalmaximizationalgorithmforestimation.TheyformulatedthefollowingmixedmodelsYi=X1i1+Z1ib1i+1iWi=X2i2+Z2ib2i+2i; Themainquestionofinterestinthesestudieswastheeectofsometreatmentortherapyonthemeanoftheresponsevector.Theassociationbetweentwo

PAGE 15

outcomeswasnotofmaininterest.InChapter 3 ,wewillproposesimilarmodelsthataddressthisassociation. GeorgeandMcCulloch(1993)developedSSVStoselectpromisingsubsetsofasetofcovariatesX1;;Xpforfurtherconsideration.TheybasedthisprocedureonembeddingtheentireregressionsetupinahierarchicalBayesiannormalmixturemodel,wherelatentvariableswereusedtoidentifysubsetchoices.Thepromisingsubsetsofpredictorswereidentiedasthosewithhigherposteriorprobability.SSVSproceededbyusingGibbssamplingtoindirectlysamplefromthismultinomialposteriordistributiononthesetofpossiblesubsetchoices.Thosesubsetswithhigherprobabilitycouldthenbeidentiedbytheirmorefrequentappearanceinthesample.Inthisway,SSVSavoidscalculatingtheposteriorprobabilitiesofallsubsets,whichisnotcomputationallypractical.Next,wedescribethebasicideaofSSVSusinganexamplefromGeorgeandMcCulloch(1993). Fortheregressionsituation,considerthefollowingregressionmodel whereYisan1vectorofdependentvariables,X=(X1;;Xp)isthenpdesignmatrix,=(1;;p)isp1vectorofregressioncoecients,and2istheresidualvariance.Forthemodel( 1-1 ),selectingasubsetofpredictorsis

PAGE 16

equivalenttosettingto0thosei'scorrespondingtothenon-selectedpredictors.Toextractinformationrelevanttovariableselection,weconsider( 1-1 )aspartofalargerhierarchicalmodel.Thekeyfeatureofthishierarchicalmodelisthateachcomponentofcanbemodelledashavingcomefromamixtureoftwonormaldistributionswithdierentvariances.Byintroducingthelatentvector=(1;;p),werepresentthenormalmixtureby and whereRisthepriorcorrelationmatrix,D=diag(a11;;app)withai=1ifi=0andai=ciifi=1,andpimaybethoughtofasthepriorprobabilitythatXishouldbeincludedinthemodel.Ddeterminesthescalingofthepriorcovariancematrixinsuchawaythatifweset1;;psmallandc1;;cplarge,thoseiforwhichi=0willtendtobeclusteredaround0,whereasthoseiforwhichi=1willtendtobedispersed.Forthepurposeofmodelselection,theBernoullimodel( 1-3 )isthestatistician'spriorprobabilitythatcorrectlyidentiesthoseithatshouldhaveanon-0estimateinthenalmodel.Themarginalprioron,p()=Pp(j)p(),isanitemixtureofmultivariatenormalpriors. Themainreasonforembeddingthenormallinearmodel( 1-1 )inthehierarchicalmixturemodelistoobtainthemarginalposteriordistributionf(jY)/f(Yj)f(),whichcontainsinformationrelevanttovariableselection.BasedonthedataY,theposteriorf(jY)updatesthepriorprobabilitiesoneachofthe2ppossiblevaluesof.Identifyingeachwithasubmodelvia(i=1)()(Xiisincluded),thosewithhigherposteriorprobability

PAGE 17

SSVSiscontrolledbyseveraltuningparameters(e.g.,ciandi,i=1;;p)thatcanbeprespeciedbytheusers.Withdierentparametersettings,userscanaddresstheparticulargoalsofvariableselection,whichinclude,forexample,thesearchforaparsimoniousmodelthatdoesnotdrasticallyincreasetheerroroftheapproximationortheeliminationofensemblesofvariablesthatareunimportantcomparedtotheirsamplinguncertainty.AdistinguishingfeatureoftheSSVSisthatitallowsuserstoletthepracticalimportanceofavariableinuenceitsselection,ratherthanjustitsstatisticalsignicance.WewillusetheideasofSSVStomodeltheassociationmatrixinthejointmodelsproposedinChapter 3 . 1.4.1MarkovChainMonteCarloSimulation Webeginbyreviewingsomenecessarydetailsondiscretetime,continuousstate-spaceMarkovchains.ThetransitionkernelP(x;A)forx2RdandA2B,whereBistheBorel-eldonRd,isaconditionaldistributionfunctionthatrepresentstheprobabilityofmovingfromxtoapointinthesetA.ItispossiblethatthechaincanmakeatransitionfromthepointxtoxwhichmeansP(x;fxg)isnotnecessarilyzero.AnimportantissueinMarkovchaintheoryisto

PAGE 18

determinetheconditionsunderwhichthereexistsaninvariantdistributionandconditionsunderwhichiterationsofthetransitionkernelconvergetotheinvariantdistribution.Theinvariantdistributionsatises(dy)=ZP(x;dy)(x)dx; whereisthedensitywithrespecttoLebesguemeasureof.MCMCmethodsturnthetheoryaround.Theyassumethattheinvariantdensityortargetdensity(denotedby),fromwhichsamplesaredrawn,isknownuptoanormalizingconstant,butthetransitionkernelisunknown.Togeneratesamplesfrom,weneedtondandutilizeatransitionkernelP(x;A)whosenthiterateconvergestoforlargen.Theprocessisstartedatanarbitraryxanditeratedalargenumberoftimes.Thedistributionoftheobservationsgeneratedfromthesimulationapproachesthetargetdistributionaftermanyiterations.OncewendanappropriateP(x;A),MCMCmethodscanproceed. SupposethatthetransitionkernelcanbeexpressedasP(x;dy)=p(x;y)dy+r(x)x(dy); wherep(x;y)isafunctionwithp(x;x)=0,x(dy)=1ifx2dyand0otherwise,andr(x)=1RRdp(x;y)dyistheprobabilitythatthechainremainsatx.Fromthepossibilitythatr(x)=0,itshouldbeclearthattheintegralofp(x;y)overyisnotnecessarily1.Ifthefunctionp(x;y)isreversible,i.e.,(x)p(x;y)=(y)p(y;x);

PAGE 19

Combining( 1-4 )and( 1-6 ),weknowthatistheinvariantdistributionforP.Hence,reversibilityofp(x;y)isasucientconditionthatP(t)(x;A)intheformof( 1-5 )convergestoast!1. Thus,afterasucientlylongburn-in(therstmvaluesgeneratedtoallowtheMarkovchaintoreachitsstationarydistribution),pointsfXt;t=m+1;;Kgwillbedependentsamples(approximately)from.NowwecanusetheoutputfromtheMarkovchaintoestimatetheexpectationofE(f(x)),whereXhasadistribution.Afterdiscardingtheburn-insamples,weobtainanestimatorofE(f(x))givenby^E(f(x))=1 wherex(t)isthetthiterateafterburn-ingeneratedfromP(t).Equation( 1-7 )iscalledergodicaverage.Convergencetotherequiredexpectationisensuredbytheergodictheorem(RobertsandSmith,1994;RobertsandTweedie,1996).Obviously,thelargerthesizeKofthesamplewedraw,themorepreciseistheestimate. 1-7 )showshowtouseanappropriatelyconstructedMarkovchaintoestimateE(f(x)).ThenextstepistochooseP(x;A)sothatitsinvariantdistributionisthedistributionofinterest(inoursetting,theposteriordistribution).WestatethetransitionkernelduetoHastings(1970),whichisageneralizationofthemethodrstproposedbyMetropolisetal:(1953).FortheMetropolis-Hastings(M-H)algorithm,givencurrentstatex,thenextstateischosenbyrstsamplingacandidateyfromaproposaldistributionq(x;y)whichmaydependonthecurrentstate.Thecandidatevalueyisthenacceptedwith

PAGE 20

probability(x;y)givenby(x;y)=min1;(y)q(x;y) Theproposaldensitycanhaveanyformandthestationarydistributionofthechainwillbegivensomeregularityconditions.Weshowthisnext. ThetransitionkernelfortheMetropolis-HastingsalgorithmisPMH(x;dy)=q(x;y)(x;y)dy+x(dy)1ZRdq(x;y)(x;y)dy: Thersttermin( 1-9 )arisesfromacceptanceofacandidateyandthesecondonearisesfromrejection.Usingthefactthat(x)q(x;y)(x;y)=(y)q(y;x)(y;x); 1-8 ),wehavethereversibilityequation(x)PMH(y;x)=(y)PMH(x;y): Integratingbothsidesof( 1-10 )withrespecttox,weobtainZ(x)PMH(y;x)dx=(y): Theleftsideof( 1-11 )isthemarginaldistributionofygiventhatxcomesfrom.Therefore( 1-11 )meansthatifxisfrom,thenywillbealsofrom.Thus,onceasamplefromthestationarydistributionhasbeenobtained,allsubsequentsampleswillbealsofromthatdistribution.Inaddition,itiseasytoshowintheMetropolis-HastingsalgorithmthatP(t)MH(x;dy)!ast!1basedon( 1-6 ). WenowsummarizetheM-Halgorithmas 1)Sett=0andtheinitialvaluex(0); 2)Atiterationt+1,generateacandidatevalueyfromq(x(t);y);

PAGE 21

3)SamplearandomvariableufromU(0,1).Thenx(t+1)=8>><>>:ywithprobability(x(t);y)x(t)withprobability1(x(t);y); 1-8 ). 4)Repeat2)and3).

PAGE 22

Thesamplercanbegeneralizedtomorethantwovariablesandthecasewherethevariablescanbeavectororamatrix.Inparticular,thevalueofthekthvariable(scalar,vectorormatrix)isdrawnfromthedistributionp(kj(k))where(k)denotesavectorcontainingallofthevariablesbutk(k=1;;T).Thus,duringthetthiterationofthesample,toobtainthevalueof(t)kwedrawfromthe

PAGE 23

followingfulldistributions(t)1p((t)1j(t1)2;;(t1)k;(t1)k+1;;(t1)T)......(t)kp((t)kj(t)1;;(t)k1;(t1)k+1;;(t1)T)(t)k+1p((t)k+1j(t)1;;(t)k;(t1)k+2;;(t1)T)......(t)Tp((t)Tj(t)1;;(t)k;(t)k+1;;(t)T1): Thistechniquewaspopularizedforconstructingdeterministicmode-ndingalgorithmsbyDempster,LairdandRubin(1977)intheirseminalarticleontheEMalgorithm,butthetermdataaugmentation(DA)originatedwithTannerandWong's(1987)paper.TheDAalgorithmstartswiththeconstructionoftheso-calleddataaugmentation,Z,whicharelinkedtotheobserveddataviaamany-to-onemappingM:Z!Yobs.AdataaugmentationschemeisamodelforZ,p(Zj),thatsatisesthefollowingconstraintZM(Z)=Yobsp(Zj)(dZ)=p(Yobsj):

PAGE 24

Thatis,themarginaldistributionofYobsimpliedbyp(Zj)mustbetheoriginalmodelp(Yobsj).Thenecessityofthisrequirementisobviousbecausep(Zj)isintroducedpurelyforcomputationalpurposesandthusshouldnotalterouroriginalmodel.TheutilityoftheDAalgorithmstemsfromthefactthatwithanappropriatechoiceofp(Zj),samplingfrombothp(jZ)andp(ZjYobs;)ismucheasierthansamplingdirectlyfromp(jYobs).Consequently,startingwithaninitialvalue,(0)2,wecanformaMarkovchainf((t);Z(t);t1gbyiterativelydrawingZ(t+1)and(t+1)fromp(Zj(t);Yobs)andp(jZ(t+1)),respectively.Thisissimplyatwo-stepversionofthemoregeneralGibssampler(GemanandGeman,1984),andthusunderthestandardregularityconditionsfortheGibbssampler(Roberts,1996;Tierney,1994&1996),thelimitingdistributionof((t);Z(t))isgivenbyp(;ZjYobs)andsample(t)isfromp(jYobs). Besidessimplifyingtheimplementationofthesamplingalgorithms,anotherdesirablepropertyofthedataaugmentationschemewouldbefortheresultingMarkovchainstomixmorequickly,thusreducingtherequiredcomputationtime.HeremixingreferstoeciencywithwhichtheMCMCalgorithmsamplesaparameterorasetofparameters.Infact,insomecasesdataaugmentationschemeshavebeenintroducedsolelytoimprovemixing.ThisisthecasewiththewellknownSwendsenandWang(1987)algorithmforsimulatingfromtheIsing(1925)andPotts(1952)models.TheiralgorithmisaspecialcaseofwhatNeal(1997)termedslicesampling,ageneralclassofwhichcanbeformulatedasfollows.Supposewewanttosimulatefromadensityf(x),whichcanbewrittenasf(x)/(x)QTk=1lk(x).Wethencanintroduceanauxiliaryvariableu=(u1;;uk)2(0;1)Tsuchthatthejointdensityofxandu(withrespecttoLebesguemeasure)isgivenbyf(x;u)/(x)TYk=1Ifuklk(x)g;

PAGE 25

whereIfgistheindicatorfunction.Itisclearthatthemarginaldensityofximpliedby( 1-13 )isf(x).TheGibbssamplercanthenbeimplementedby1)simulatingufromf(ujx),whichamountstoindependentlysimulatingukfromUniform(0,lk(x)),k=1;;T,and2)simulatingxfromf(xju),whichis(x)truncatedtotheregionTTk=1fx:lk(x)ukg;whenxismultidimensional,furtherGibbsstepsmaybeneededtosamplefromf(xju).Insomeapplications,suchastheIsingmodelwherexisalattice,(x)isasimpledistributionwithindependencestructuresamongthecomponentsofx,andthereforeiseasytosample.ThefactorQTk=1lk(x),however,reectsthedependencestructureamongthecomponentsofx.ThisdependenceisresponsiblefortheslowmixingwhenoneimplementstheGibbssamplerortheMetropolis-Hastingsalgorithmdirectlyonf(x)/(x)QTk=1lk(x).TheuseoftheauxiliaryvariableueectivelyeliminatessuchinteractionsandthusreducesthestrongautocorrelationintheMCMCdraws. ThesuccessoftheSwendsen-Wangalgorithmhasstimulatedmuchinterestinthegeneraluseofthemethodofauxiliaryvariablesinthephysicsliterature.Inthestatisticalliterature,therealsohasbeengrowinggeneralinterestinthismethod,startingfromtheoverviewarticleofBesagandGreen(1993);importantmethodologicaland/ortheoreticalpapersincludeDamien,WakeeldandWalker(1999),Higdon(1998),Neal(1997),andRobertsandRosenthal(1997).Itisworthwhiletonotethatthestatisticalliteratureonauxiliaryvariableshasgrownlargelyindependentlyoftheliteratureondataaugmentation,despitethefactthatthetwomethodsareidenticalintheirgeneralform.Thegeneralformofauxiliaryvariables,ofwhich( 1-13 )isaspecialcase,istoembedthetargetdistribution(ordensity)f(x)intof(x;u),whereuisanauxiliaryvariableofarbitrarydimension.Thisisthesameasin( 1-12 )ifweexpressitintheequivalentformZM(Z)=Yobsp(;ZjYobs)(dZ)=p(jYobs);

PAGE 26

andidentifywithx,Zwithu,andp(jYobs)withf(),where\"canbeeitherxorfx;ug.Inotherwords,wecaneitherviewthemethodofauxiliaryvariablesasdataaugmentationwithoutanyobserveddata(orequivalentlybyxingtheobserveddatatobeconstant),orviewdataaugmentationasintroducingauxiliaryvariablesintop(jYobs). Toillustratethebasicideasbehindparameterexpansion,weconsiderthefollowingmodelfromLiuandWu(1999) whereisanunknownparameterandWisaknownconstant.Inthismodelyistheobserveddataandqismissing.Hencetheobserved-datamodelisyjN(;1+W).Withaatprioron,theDAalgorithmiteratesbetweentheconditionaldistributionsqj;yN(q 1+W1)andjy;qN(yq;1)forsimulatingtheposteriordistributionofgiveny.TheconvergencerateoftheDAalgorithmcanbecomputedasr0=1 1+W1.Toavoidthepotentiallyslowrate,weexpandthemodel( 1-15 )byintroducingasyj;;qN(+q;1)andqj;N(;W):

PAGE 27

Thismodelpreservestheobserved-datamodelyj;(;1+W)buthasanexpansionparameteridentiableonlyfromthecompletedata(y;q).UnderthepriorN(0;b),theparameterexpandeddataaugmentation(PX-DA)algorithmis:1)Drawj;yN(0;b)andqj;;yN(+y 1+W1);2)Drawjy;qN(bq b+W;1 1+W1.ThusthenewalgorithmisneverslowerthantheordinaryDAalgorithmfor( 1-15 ).ifweconsiderthecasethat=a,thenthecorrespondingre-parametrizationisyj;;qN((1a)+q;1)andqj;N(;W): Liuetal:(1998)introducedthetechniqueofparameterexpansionandproposedtheparameterexpanded(PX)-EMalgorithm.PX-EMexpandsthecomplete-datamodelf(Ycomj)(YcomdenotesYobsplusaugmenteddata)furthertoobtaintheparameter-expandedcomplete-datamodelf(Ycomj=(x;x))(xistheexpansionparameter)whilepreservingtheobserved-datamodelinthesensethatthereisamany-to-oneontomapping,calledthereductionfunction,=R()suchthatf(Yobsj)=f(Yobsj=R()).Thus,theexpansionparameterisunidentiablefromtheobserveddata.Afterparameterexpansion,PX-EMissimplyanEMalgorithmappliedtotheparameter-expandedcomplete-datamodelwiththeM-stepadjustedby^=R(^x;^x).Asaresult,PX-EMsharesthesimplicityandstabilityoftheEMalgorithm.ForfurtherdiscussiononPX-EM,includingitslimitations,seeLiuetal:(1998). SincethedevelopmentofPX-EMformaximumlikelihoodestimation,therehavebeenconsiderableeortsinstudyingitsstochasticversionforBayesiananalysis(MengandvanDyk,1999;LiuandWu,1999;Liu,2003).Toconstruct

PAGE 28

aparameterexpanded(PX)-DAalgorithm(orthemarginalDAalgorithm),bothMengandvanDyk(1999)andLiuandWu(1999)startedwiththesameframeworkwheretheexpandedparameterspaceconsistsoftheoriginalparameterandtheexpansionparameter,thatis,=(;x),andaproperpriordistributionf(x)ontheexpandedparameterx.Consider,forexample,theschemeinMengandvanDyk(1999)andLiuandWu(1999)withthefollowingtwosteps ThenrepeatStep1andStep2formanytimes.ThePX-DAalgorithmusuallyconvergesfasterthanthecorrespondingDAalgorithm. LetY=fYijg(i=1;;c;j=1;;ni)denotethecompletedata.WedecomposeYasY=(Yobs;Ymis),whereYobsdenotestheobservedvaluesandYmisdenotesthemissingvalues.DenoteLasthelikelihoodfunctionanddenethe

PAGE 29

missing-dataindicatormatrixM=fMijgsuchthatMij=8>><>>:1ifYijmissing0ifYijobserved: Thatisf(Y;Mj;)=f(Yj)f(MjY;);(;)2;; Thedistributionoftheobserveddata(Yobs;M)isobtainedbyintegratingYmisoutofthejointdensityofY=(Yobs;Ymis)andM;thatis, IfweassumethatthedataareMARandandareseparable(theparameterspacesofandarenotoverlapping),thisfurthersimpliesto

PAGE 30

Theobserveddatalikelihoodofandisanyfunctionofandproportionalto( 1-17 ) BayesinferenceunderthecompletemodelforYandMisobtainedbycombiningthefulllikelihood( 1-18 )withajointpriordistribution(;)forand(;jYobs;M)/(;)Lfull(;jYobs;M): Ifweassumethattheprobabilityofnon-responsedependsonmissingvalues,thenthedropoutisnotignorable(MNAR)andweneedtomodelthemissingdatamechanism.OnestrategytomodeltherelationshipbetweenYandMispatternmixturemodeling,wherethejointdistributionf(Y;Mj;)isfactoredasf(YjM;)f(Mj).Thelikelihoodofand(dierentfrompreviousand)becomesL(;jYobs;M)/f(Mj)f(YobsjM;):

PAGE 31

Inthiscase,bothandareofinterestsincebothcharacterizethefulldataresponsemodel,f(Yj;)=XMf(Y;Mj;)=XMf(Yj;M)f(Mj): 3 .

PAGE 32

Barnard,McCullochandMeng(2000)proposedusingtheGriddyGibbs(GG)sampler(RitterandTanner,1992)todrawthecomponentsofacorrelationmatrixoneatatimeinthecontextofahierarchicalshrinkagemodelforthemarginalvariancesofacovariancematrix.AlthoughtheGGsamplerissimpletoimplement,itisnotcomputationallyecient.ChibandGreenberg(1998)sampledthecorrelationmatrixinthemultivariateprobitmodelusingarandom 22

PAGE 33

walkMetropolis-Hastings(RW-MH)algorithm.ThisalgorithmiscomputationallymoreecientthanGGSamplersamplerasitsamplesmorethanonecomponentatatime.However,adisadvantageisthatthevaluessampledfromtheproposaldensityarenotguaranteedtobepositivedenite.Inaddition,ithastheproblemofpotentiallyslowmixingassociatedwithmostRW-MHalgorithms.OtherapproacheshavebeendiscussedinNandramandChen(1994),Nobile(1998)andWong,CarterandKohn(2003). Parameterexpansionhasbeenexploredbyseveralauthorsforsimulatingacorrelationmatrix.Liu(2001)consideredsimulatingacorrelationmatrixthroughreparameterizationinthemultivariateprobitregressionmodel.ThismethodisrestrictedtoaparticularpriorforR.Zhang,BoscardinandBelin(2004)proposedaparameter-extendedMetropolis-Hastingsalgorithm(PX-MH)forsamplingRinBayesianmodelswithcorrelatedlatentvariables.TheseminalideaintheirmethodisthatinsteadofamarginalpriorforR,theyspeciedajointpriorforRandD(unidentiedmarginalvariances)derivedfromsomeinverseWishartdistributionof=DRDinmodelestimation.Thensampling(R;D)jointlywasaccomplishedthroughaMetropolisHastingsalgorithmbyrstdrawingfromaprespeciedWishartdistributionwithdegreesoffreedombeingthesamplesizeandthescalematrixbeingthecurrentvalueof.Usingthismethod,allcomponentsofRaredrawnatonetime.However,thisalgorithmissimilartorandomwalkalgorithmssincetheyaresamplingfromacandidatedistribution'centered'atthepreviousvalue.Thoughtheysampleallthecomponentsatonce,therandomwalkaspectcanslowdownmixingandconvergence. Therestofthechapterisorganizedasfollows.InSection 2.2 ,weintroducetwocommonmethodsforsamplingR:TheGriddy-Gibbs(GG)samplerandtherandomwalkMetropolis-Hastings(RW-MH)algorithm.TwomotivatingexamplesareintroducedinSection 2.3 .InSection 2.4 ,weproposeatwo-stageparameter

PAGE 34

expandedreparametrizationandMetropolis-Hastings(PXRP-MH)algorithmforsimulatingthecorrelationmatrixfromitsfullconditionaldistribution.Thetheoryofthisalgorithmisinducedandsomesucientconditionstoimplementitaregiven.ThisalgorithmisthenderivedbothforMVPmodelsandforMVRmodelswithacommoncorrelationmatrix.Section 2.5 furthergeneralizesthePXRP-MHalgorithm.ApplicationofthenewalgorithmtoseveralothermodelstosamplefromtheposteriorofacorrelationmatrixiscontainedinSection 2.6 .Section 2.7 describestheresultsofasimulationstudytoexploretheperformanceofthealgorithm.ArealdataexampleisgiveninSection 2.8 . 2.2.1TheGriddy-GibbsSampler sampler(new)12fromf(r(new)12jr(old)13;;r(old)1;T;r(old)23;;r(old)T1;T;Y;(R))......sampler(new)1;T1fromf(r(new)1;T1jr(new)12;;r(new)1;T2;r(old)23;;r(old)T1;T;Y;(R))sampler(new)23fromf(r(new)23jr(new)12;;r(new)1;T1;r(old)24;;r(old)T1;T;Y;(R))......sampler(new)T1;Tfromf(r(new)T1;Tjr(new)12;;r(new)1;T1;r(new)23;;r(new)T1;T2;Y;(R)):(2-1) whererij(i
PAGE 35

TheGGsamplerwasrstintroducedbyRitterandTanner(1992).Itisarelativelysimpleapproachinthesenseofbeingeasytocodeandimplement.Inourcase,itsbasicideaistoformasimpleapproximationtotheinversecdfbasedontheevaluationoff(rijjr(ij);(R);i
PAGE 36

xed.Itprovidesaconvenientwaytodeterminewhatpointsshouldberemovedinstep2.BasedonLemma 2 ,wecanapplythefollowingpropositiontoidentifythepositivedeniteregionforR(s)intherststep. 2(f(1)4f(0))andc2=1 2(f(1)4f(0)).Letrmin=minc1c2p CaseIc0+2c2>0andc21+c22c20 CaseIIc0+2c2>0andc21+c22c20 CaseIVc0+2c2=0 2-1 )ateachofdiscretizedpointsoneatatime.2)ThecomponentsofRaredrawnoneatatime.TwoissuescanmaketheGGsamplercomputationallyinecientwhennislarge.

PAGE 37

Iftheincrementvectorhisassumedtohaveasymmetricdistribution,thentheacceptanceratemaybefurthersimpliedanddoesnotdependontheproposaldensity.Totraversethecorrelationspaceandproducewell-mixingoutput,weneedtochoosethevarianceoftheincrementsuchthattheacceptancerate(apercentageoftimesamovetoanewpointismade)isroughly21-50%intherange(Roberts,GelmanandGilks,1997;RobertsandRosenthal,2001).IfthedimensionofRisrelativelylarge,asevidencedinthesimulationstudiesinSection 2.7 ,itisbettertopartitionrintoseveralblocksandtoapplytheRW-MHalgorithmtoeachoftheblocks. TheRW-MHalgorithmdrawsallcomponentsofRatonceandhenceismorecomputationallyecientthantheGGSamplerforsamplingacorrelation

PAGE 38

matrix.EventhoughtheRW-MHalgorithmovercomesthedisadvantagesoftheGGsampler,itstillhasamajordisadvantagethattheproducedMarkovchainofRmaymixandconvergeslowly,duetotherandomwalkaspectofthealgorithm. 2exp(1 2nXi=1(YiXi)01(YiXi))dYi;(2-2) whereAijistheinterval(0;1)ifQij=1andtheinterval(;0]ifQij=0.AsnotedbyChibandGreenberg(1998),theparameters(;)arenotidentiablefromtheobserved-datamodel.Toensureidentiabilityofthemodelparameters,itiscommontorestrictthecovariancematrixtobeacorrelationmatrixR. Tosamplefromtheposteriordistributionof(;R),acombinationofGibbssampling(GelfandandSmith,1990)anddataaugmentation(TannerandWong,

PAGE 39

1987)istypicallyused.Thiscanbeconceptualizedastwosteps:animputation(I)stepandaposterior(P)stepasfollows whereXijisaTpdesignmatrix,isap1vectorofregressioncoecientsandiisthecovariancematrixofYij.Thisisamultivariateregressionmodelwiththesameeectofcovariatesacrossresponses. Analternativespecicationofmodel( 2-3 )isthemodelwithcovariateeectsallowedtodieracrossresponsesinwhichXijisaTpTdesignmatrix.WereexpresstheMVRmodelinthissettingasYijN(0xij;i); wherexijisthep1vectorofcovariatesandisthepTmatrixwhoseelementsaretakencolumnwisefrom. WhentherearenotenoughdatatoestimateaseparatecorrelationmatrixRiforeachgroupin( 2-3 )and( 2-4 ),wemightconsiderthemodelwithacommoncorrelationmatrixinwhichicanbedecomposedasViRVi,withVi=diag(i1;;iT)andikthestandarddeviationofthek-thresponseingroupi(i=1;;c;k=1;;T).Thus,variancesareallowedtodieracross

PAGE 40

groups,butthecorrelationmatrixisassumedconstant.Forconvenience,wereferto( 2-3 )withacommonRasMVRModelIand( 2-4 )withacommonRasMVRModelII.SamplingtechniquessimilartothatinMVPmodelswillbeusedtosamplefromtheposteriordistributionof(();R;Vi). TheparametersinMVPmodelsand(();Vi)inMVRModelscanbesampledfromtheirfullconditionalsusingthestraightforwardtechniquesasthoseinotherstandardMCMCalgorithms.Wewillnotprovidedetailsinthispaper.However,itiswellknownthatitisdiculttosimulatefromtheposteriordistributionsofunknowncorrelationmatricesR(ChibandGreenberg,1998;ManlyandRayner,1987;Pourahmadi,DanielsandPark,2005).Inwhatfollows,weshowthatthePX-RPMHalgorithmisasimplewaytosampleacorrelationmatrixingeneralandspecicallyinthesemodels. ThePXRPstagehasfoursteps.Step1istondtheexpansionparameters.ThescaleparametersofacorrelationmatrixRarexedtobeoneandtheo-diagonalelementsarerestrictedtoaconvexregionV[1;1]T(T1) 2.The

PAGE 41

rstconstraintmakesdrawingRmoredicultthandrawingacovariancematrix.ThisdicultycanbeovercomebytransformingRintoalessconstrainedcovariancematrix,byborrowingthescalefromsome(matrixvalued)expansionparameter(D).ThecorrelationmatrixRcanbeconvertedintoacovariancematrixsuchthattheconditionaldistributionofhasastandarddistribution(e.g.inverseWishartdistribution).Theexpansionoftheparameterspacewiththescaleparameters,however,mustbedonetopreservetheobserved-datamodelinthefollowingsense:thereisamany-to-onemapping,calledthereductionfunction,=P(),suchthatf(Yobsj)=f(Yobsj),whereYobsistheobserveddata,istheparametervectorintheoriginalmodelandistheparametervectorintheexpandedmodel.Forexample,inMVPmodels,Yobs=Q,=(;R),=(;)andP()=(;D1D1).Werefertothisstepas`expansionparametermining'or`XPmining'. Oncewendtheappropriatesetofparametersforparameterexpansion,weneedtondacandidatetransformationT,whichmapsthestandardparameterspacetotheconstrainedexpandedparameterspace.Underthistransformation,thejointposteriordistributionindexedbythestandardparameterscanbeconvertedintotheoneindexedbytheexpandedparameters.ThestandardparameterspacehaslowerdimensionthantheexpandedparameterspacebecauseRherehasfewerparametersthan.Constraintsontheexpandedspaceareneededsinceourcandidatetransformationneedstobeaone-to-onemapping.Theseconstraintsvarywiththecandidatetransformationanddependonthespecicmodelsordesigns.Werefertothisstepas`candidatetransformationsmining'or`CTmining'.CTminingcomprisesthesecondstepofthePXRPstage. Inthethirdstep,wederiveacandidatepriorforR.Weintroducethisprior,basedonthecandidatetransformationinstep2sothattheconditionaldistributionofRcanbetransformedintotheparameterexpandedconditionaldistributionof

PAGE 42

fromwhichiseasytosample.ThispriorforRisnotprespeciedorxed.ItisderivedbasedontheCTinstep2,thelikelihoodfunctionandthepriorsforparameters,exceptR,toobtainthecandidatedensityofderivedbelow,fromwhichitiseasytosample.Typically,thecandidatepriorchoseninthisstepisdierentfromthepriorforRweprespecifyforposteriorinference.Todistinguishthem,wecallthelatterthetargetpriorforR.Werefertothisstepas`candidatepriormining'or`CPmining'. Thenalstepistoderivetheparameterexpandedcandidatedensity(PXCD)ofbasedontheCTinstep2andtheCPinstep3.ThisdensitywillbeusedastheproposaldensitytosampleR.ItisusuallyaninverseWishartdistributionwithdegreesoffreedomandscalematricesvaryingindierentmodelsettings.Werefertothisstepasparameterexpandedcandidatedensityderiving(PXCDderiving). OnceweobtainthePXCDof,wegotothesecondstage,i.e,thePXMHstage.Inthisstage,werstsimulate(R;D)fromthePXCDofandthenaccordingtoaM-Hacceptanceprobability,wedecidewhethertokeepthecandidateR. 2.3.1 .ForBayesianinference,wespecifyindependentpriorsforandRas()/expf1 2(0)01(0)g whereRklistheelementinthekthrowandlthcolumnofR.WewillweakenthisindependenceassumptioninSection4. Forthismodel,thePX-RPMHalgorithmisderivedasfollows,

PAGE 43

XPmining:Theexpandedparametervectoris=(;)where=DRDandthereductionfunctionisP()=(;D1D1)withexpansionparameterDsatisfyingR=D1D1. CTmining:BasedonP()fromtheXPmining,wendthefollowingone-to-onemappingfromfYi;RgtofYi;gforcreatingdrawsofthecorrelationmatrix, whereisaTTpositivedenitematrixandPni=1Y2it=1foranyt=1;;T.Notethatin( 2-7 )isxedandnotinvolvedinthetransformation.Given,thestepthatdrawsYiimplicitlydrawsYiandDbecausePni=1(Yitx0it)2=D1ttPni=1Y2it=D1tt,whereDttisthetthelementofDandx0itisthetthrowofXi.Thespacefor(Yi;)ishigherdimensionalthanthatfor(Yi;R)sinceRhasfewerparametersthan.Theconstraints,Pni=1Y2it=1foranyt=1;;T,areneeded,tomakethecandidatetransformationaone-to-onemapping. CPmining:Basedon( 2-2 )and( 2-7 ),toobtainthePXCDof,werstneedtondthecandidatepriorforR,givenby(R)/jRjT+1 2IfRkl:jRklj1andRispos.def.g; whereTisthedimensionofR.Wenotethat( 2-8 )isdierentfrom( 2-6 ). PXCDderiving:Theparameterexpandedcandidatedensityofisgiveninthefollowingproposition. 2-5 )and( 2-6 )forandR,respectively,thenfromlikelihoodfunction( 2-2 ),transformation( 2-7 )andcandidateprior( 2-8 ),weobtain(jY;)/jj+T+1 2expn1 2tr(S1)o;

PAGE 44

Proposition 2 givesthePXCDoftouseastheproposaldensityinthePXMHstage.Inthisstage,werstsimulatefrom( 2-9 )andthenobtainthecorrelationmatrixRthroughthereductionfunctionP(;)=(;D1D1).Second,wekeepthecandidateRwithprobability(theacceptancerateintheM-Halgorithm).TheentirePX-RPMHalgorithmforMVPmodelsisgiveninthefollowingtheorem. 2-5 )and( 2-6 )aspriorsforandR,respectively,thenundertransformation( 2-7 )andcandidateprior( 2-8 ),simulatingRisequivalenttosimulatingrstfromtheinverseWishartdistribution( 2-9 ),andthentranslatingitbacktoRthroughR=D1D1in( 2-7 )andacceptingthecandidateRusingaMetropolis-Hastingsstepwithsomeacceptancerate,where=min1;expT+1 2(logjRjlogjR(k)j)atiterationk+1. 2.3.2 .WespecifythepriorsforandRasin( 2-5 )and( 2-6 )andthepriorforVias(diag(Vi))/TYt=1(2it)(rit+1)expit wherediag(Vi)isthevectorofthediagonalelementsofVi.

PAGE 45

WederivethePX-RPMHalgorithmasfollows.InthePXRPstage,XPminingissimilartothatinMVPmodels.InCTmining,weconsiderthefollowingone-to-onemappingT1:fYij;Rg!fij;g whereisxedandnotinvolvedinT1,andijisaT1vectorsatisfyingPci=1Pnij=12ijt=1(t=1;;T).Notethatgiven,thestepthatdrawsYijimplicitlydrawsijandDbecausePci=1Pnij=1(Yijtx0ijt)2=D2ttPci=1Pnij=1(2ijt)=D2ttforanyt=1;;T,whereDttisthet-thdiagonalelementofDandxijtisthevectorofcovariatesassociatedwiththetthresponseofsubjectjingroupi.Basedon( 2-3 )and( 2-11 ),thecandidatepriorforRisthesameas( 2-8 )giveninMVPmodels.ThefollowingpropositiongivesthePXCDof 2-5 ),( 2-6 ),and( 2-10 )for,RandVi,respectively,thenunderT1( 2-11 )andcandidateprior( 2-8 ),theposteriordistributionofgivenijandVi(i=1;;c;j=1;;ni)isaninverseWishartwithdegreesoffreedom1andscalecovariancematrixS1,i.e.,(j;;V1;;Vc)/jj1+T+1 2expn1 2tr(S11)o; 2-5 ),( 2-6 ),and( 2-10 )for,RandVi,re-spectively,thenundertransformation( 2-11 )andcandidateprior( 2-8 ),simulatingRisequivalenttosimulatingrstfromtheinverseWishartdistribution( 2-12 ),

PAGE 46

2-11 )andacceptingthecandidateRthroughaMetropolis-Hastingsstepwithacceptancerate,where=min1;expT+1 2(logjRjlogjR(k)j)atiterationk+1.

PAGE 47

2-6 )andthecandidateprior( 2-8 )forR. 2.5.1OverviewoftheGPX-RPMHAlgorithm (R)/jRja+1 2exp1 2tr(W0R1)IfRkl:jRklj1andRispos.def.g;(2-13)

PAGE 48

whereaisaconstantandW0=PP0isaTTpositivesemidenitetuningparametermatrix,whereisadiagonalmatrixcontainingtheeigenvaluesofW0andPisanorthogonalmatrixwitheigenvectorsascolumns.Equation( 2-13 )isamoregeneralversionof( 2-8 ).WhenW0=0,itreducesto(R)/jRja+1 2IfRkl:jRklj1andRispos.def.g.Notethatwhena1,thereducedpriorisproper,butwhena>1,itisimproper.Theadvantageofthemoregeneralspecicationin( 2-13 )isthatwecantuneW0ifnecessarytoincreasetheacceptancerateintheM-Hstep.ThiswillbehelpfulifthetargetpriorforRisinformative(Zhangetal:(2004)). BeforewegothroughthedetailsofthePX-RPMHinthemoregeneralsetting,weintroducesomeadditionalnotations.LetYobsbetheobserveddata,Ymisbethemissingdata/latentdataandY=(Yobs;Ymis)bethecompletedata.Supposethatthereexistsageneralcomplete-datamodelMwithdistributionf(Yj).WeexpandittobeMindexedbytheaugmented,whichpreservestheobserveddatamodelinthesensethatf(Yobsj)=f(Yobsj),where=((R);R),=((R);),(R)containsalltheparametersexceptRintheoriginalmodeland(R)isthetransformedversionof(R)intheexpandedmodel. DeneL()tobethelikelihood.SupposethatTisatransformationwhichisaone-to-onemappingfrom(Yi;(R);R)to(Yi;(R);),where=DRD,DisisadiagonalmatrixofexpandedparametersandfYi:i=1;;ngsatisfyingPni=1Yi=1.LetY=(Y1;;Yn)andY=(Y1;;Yn).HeretheYijcanbebinary(e.g.MVPmodels),ordinal,mixedcategoricalorcontinuous(e.g.MVRModelIandII)(i=1;;n;j=1;;T).Thenundermildconditions,wehavethefollowingtheorem.

PAGE 49

2expn1 2tr(S1)o; 2-13 ),thensimulatingRisequivalenttosimulatingrstfromtheaboveinverseWishartdistribution,andthentranslatingitbacktoRthroughR=D1D1underTandcorrectingRthroughaMetropolisHastingsstepwithsomeacceptanceprobability,where=min1;CP(k)(R);R(k)TP((R);R) 3 ,condition( 2-14 )guaranteesthatthejointposteriordistributionisproperwhenweuseimpropertargetpriorsforsomeparametersinBayesianmodelanalysis.Ifalltargetpriorsareproper(e.g.thoseinSection 2.4.2 ),( 2-14 )isalwaystrue.Condition( 2-15 )givesthegeneralizedPXCDderivedthroughXPmining,CTminingandCPmining.ItiseasytoseethatTheorem 3 isadirectextensionofTheorem 1 fromMVPmodelswithproperindependentpriorstothemoregeneralmodelsettingwithproperorimproperdependentpriorsandthemoregeneralcandidateprior( 2-13 )forR. ThegeneralizedparameterexpandedreparametrizationandMetropolis-Hastings(GPX-RPMH)algorithmisformulatedinAppendixII.BeforeimplementingtheGPX-RPMHalgorithm,weneedtoverifythatthejointposteriordistributionisintegrable.Forexample,inthemultivariateprobitmodel,ifwespecifyaatprior(improper)on,itbecomesnecessarytoverifythat(;RjQ)isproper.

PAGE 50

Fordetails,seeChenandShao(1999)forproprietyofposteriordistributionsformultivariatecategoricalresponsemodelandDaniels(inpress)forsomeresultsonproprietyoftheposteriorforlinearregressionwithcorrelatedand/orheterogeneouserrors. 2.3.2 .Forposteriorinference,aconditionalpriorusedbyLiuandSun(2000)forv,whichisthepT-dimensionalvectorobtainedbystackingtheprowsof,isgivenby 20v(1R)1vg;(2-16) whereisappdiagonalmatrixwithnonnegativetuningparameters1;;pasdiagonalelements.ThepriorsforViandRarethesameasthoseinSection 2.4.3 .Thus,(or)andRareaprioridependent.Forthismodel,weimplementthePX-RPMHalgorithmasfollows.XPminingissimilartothatinMVRModelI.InCTminingandCPmining,weconsiderthetransformationT2:fYij;;R;Pg!fij;;;Pgusingthecandidateprior( 2-8 )forR, wherethematrixofregressioncoecientsandthetuningparameterin( 2-8 )areinvolvedinthecandidatetransformation,dierentfromthecasein( 2-11 ).ThisisduetothepriordependencyofandRandtheuseofthegeneralprior( 2-8 )).Herewederiveatobe2TtoobtainaninverseWishartdistributionfor.ThefollowingpropositiongivesthePXCDofintheMVRModelII

PAGE 51

2-16 ),( 2-6 ),and( 2-10 )aspriorsfor,RandVi,respectively,thenunderT2givenin( 2-17 ),theposteriordistributionofgivenij,andPisaninverseWishartwithdegreesoffreedom2andthescalecovariancematrixS2,i.e.,(j;;P;V1;;Vc)/jj2+T+1 2expn1 2tr(S21)o; 2-17 ). 2-18 )astheproposaldensity,wecanimplementtheMetropolisHastingsstep,i.e.,thePXMHstageofouralgorithmsimilarlytothatinSection 2.4.2 and 2.4.3 . 2.6.1ScaleMixtureofMultivariateNormalLinkModels

PAGE 52

a)ThelatentvariablesYi1;;YiqfollowthemultivariatenormaldistributionwithmeanXiandcovariancematrix(i)fori=1;;n;thatis whereXi=diag(xi1;;xiq),(i)isapositivefunctionofaone-dimensionalpositive-valuedscalemixingvariableiand(i)isamixingdistribution. b)GivenYij,Qijj;i;;Yij=LjXl=1lIfj;l1Yij
PAGE 53

fromfYi;R;Pgtofi;;Pg wherePisdenedthesameasin( 2-13 )andiisaq1vectorsatisfyingPni=12ij=1(j=1;;q).Thetuningparametersin( 2-13 )areinvolvedinthecandidatetransformation,dieringfromthecasesconsideredpreviously.DrawingYiimplicitlydrawsiandDbecausePni=1(YijXij)2=D2jjPni=1(2ij)=D2jj,whereDjj(j=1;;q)isthe(j;j)thdiagonalelementofD.Thus,drawingRandDisequivalenttodrawingfromtheposteriordistribution(ji;i)inProposition 5 below. PXCDderiving:BasedontheXP,CTandCPminingsteps,wehavethefollowingproposition 2-21 )andT( 2-22 ),theposteriordistributionofgivenisaninverseWishartwithdegreesoffreedomandscalecovariancematrixS,i.e.,(j;i)/jj+q+1 2expn1 2tr(S1)o; 2-13 )andijandParedenedin( 2-22 ). 2trW0RR(k)exp2T+1 2(logjRjlogjR(k)j);(2-24) wheretr(A)denotesthetraceofmatrixA.

PAGE 54

WeneedtonotethatinProposition 5 ,thedegreesoffreedomandthescalematrixin( 2-23 )andtheacceptanceratein( 2-24 )dependonthetuningparametermatrixW0from( 2-13 ).WemayadjusttheacceptanceratebychangingthevalueofW0. AsnotedbyChenandShao(1999),theclassofSMMVNlinksisquiterich.Taking(i)=1,theSMMVNlinkmodelreducestothemultivariateprobit(MVP).Whenwetake(i)=1=and(i)/=21iexpf1 2ig,theSMMVNlinkbecomesamultivariatet-link(MVT)withdegreesoffreedom.ThemultivariateCauchy(MVC)andtheMVParetwospecialcasesofMVTwith=1and!1,respectively.Themultivariatelogit,whichcanbeapproximatedwithMVT(AlbertandChib,1993;O'BrienandDunson,2004),isaspecialcaseoftheSMMVN-linkbytaking(i)=42i,whereiisdistributedas(i)/P1k=1(1)k+1k2iexpf2k22ig.ThisshowsthatouralgorithmcanbeappliedtosimulateRinanymodelabove,usingthesamecandidatetransformationandcandidateprior.Wejustneedtochange(i)in( 2-19 )appropriately. 2.6.1 when(i)=1.Soweonlydiscusscase1and2.Withoutlossofgenerality,weassumethatlongitudinalprocesseshavethesameequally-spacedtimepoints.

PAGE 55

DenotethejointvectorofresponsesforthelongitudinalbinaryorordinalprocessandthelongitudinalcontinuousprocesstobeQTij=(Qij;1;;Qij;T)andWTij=(Wij;1;;Wij;T),respectively.DeneavectoroflatentvariablesunderlyingthebinaryvectorQijtobeYTij=(Yij;1;;Yij;T).Weassumethatthemodelsallowcovariateeectstodieracrosstimepoints.Thejointdistributionoflongitudinalbinaryorordinalandcontinuousprocessesovertimecanbemodelledusing and wherexijisap1covariatevector,isapTmatrix,=t0=0t;Lt1
PAGE 56

CTandCPmining:Bychoosingthecandidateprior( 2-13 )forR,wendthefollowingtransformation:fUij;;R;Pg!fij;;;Pg, wherePci=1Pnij=1Uij;t=1foranyt=1;;2T.Wenotethatthematrixofregressioncoecientsandthetuningparameterin( 2-13 )areinvolvedinthecandidatetransformation.ThisisduetothepriordependencyofandRandtheuseofthegeneralprior( 2-13 ).Herewederiveatobe4TtoobtainaninverseWishartdistributionfor.ThefollowingpropositiongivesthePXCDofinthesemodels 2-27 ),( 2-6 ),and( 2-10 )aspriorsfor,RandVi,respectively,thenunderthetransformationgivenin( 2-28 ),theposteriordistributionofgivenij,andPisaninverseWishartwithdegreesoffreedomandscalecovariancematrixS,i.e.,(j;;P;V1;;Vc)/jj+T+1 2expn1 2tr(S1)o; 2-28 ). 2-29 )astheproposaldensity,weimplementtheMetropolis-Hastingsstepasfollows:1)Samplefrom( 2-29 )andthenobtainthecorrelationmatrixRthroughD1D1;2)KeepthecandidateRwithprobabilitygivenby=min1;1 2trW0RR(k)exp4T+1 2(logjRjlogjR(k)j):

PAGE 57

Theobserveddata,Qi,werespeciedasQij=1ifYij>0and0otherwise.DeneY=(Y1;;Y200)andQ=(Q1;;Q200).WeusedtheDAalgorithmfromSection 2.3.1 tosamplefromtheposterior.Somemoredetailsofthefullconditionaldistributionsaregivennext.YjQ;;Risatruncatedmultivariatenormalwithdensity 2Pni=1(YiXi)0R1(YiXi) 2Pni=1(YiXi)0R1(YiXi)dY;(2-30)

PAGE 58

Figure2{1. Traceplotsof5componentsofRoverthelast2000iterationsof50,000iterationsforthePX-RPMHalgorithmandRW-MHalgorithm.

PAGE 59

whereCisthetruncationregiondenedasC=fY:Yij0ifQij=1andYij<0ifQij=0g.jY;Rhasanormaldistribution 2exp1 2()01();(2-31) where=Pni=1X0iR1Xi1Pni=1X0iR1Yiand=Pni=1X0iR1Xi1.ThefullconditionaldistributionforRis 2trhnXi=1YiXiYiXi0R1ioIfRjk:jRjkj1andRispos.def.g:(2-32) WesampledfromthefullconditionaldistributionofRusingthePX-RPMHalgorithm,theGriddy-GibbssamplingalgorithmandtheRW-MHalgorithm.FortheRW-MHalgorithm,wechoseastheproposaldensityanormaldistributionwithmeanr(k)andcovarianceW,wherer(k)isthevectorofthecomponentsofR(k)fromthecurrentiterationkdenedasr(k)=(R(k)12;;R(k)(T1)T)(T=7),WistheinverseofthenegativeoftheHessianmatrixoflogL(RjY;^),whereYisthetruevaluesfromthesimulateddata(unknowninarealdataset),^istheMLEof,andisatuningconstant.Letrbethecandidatevaluefromtheproposaldensityq(rjr(k);Y;).TheRW-MHalgorithm,atiterationk+1,isgivenby TheMetropolis-Hastingsstepwasappliedtor=(r1;r2;r3;r4)infourblockssinceRhas21elements,wherer1,r2andr3eachconsistofsixcomponentsandr4ofthreecomponentsinarow-wiseexpansionofR.Wechose=1:5fortherstthreeblocksand=2:0forthefourthblocksuchthattheacceptancerate(apercentage

PAGE 60

oftimesamovetoanewpointismade)wasequalto0.21(Roberts,GelmanandGilks,1997;RobertsandRosenthal,2001). TocomparetherunningtimeofthePX-RPMHalgorithm,theGriddy-GibbssamplerandtheRW-MHalgorithm,wegenerated5000samplesfrom( 2-32 ).FortheGriddy-Gibbssampler,forsimplicity,wediscretizedthedomainof(rijjr(ij);Y;)uniformlyusingthempointsr1;;rm(setm=20).Thesamplingrequired991secondsfortheGriddy-Gibbssampler;fortheRW-MHalgorithm,therunningtimewas11seconds;forthePX-RPMH,therunningtimewas6seconds,whichismorethan150timesfasterthantheGriddy-Gibbssampler. Second,weexaminedgraphicallythemixingofthePX-RPMHalgorithmandtherandomwalkMetropolis-Hastingsalgorithm.WeusedtheDAalgorithmtoiterativelygenerateY,andRfrom( 2-30 ),( 2-31 )and( 2-32 ),respectively.Achainof50,000iterationswasgeneratedandonlythelast2000sampleswereretained.Werandomlychose5componentsofR:R15,R27,R35,R47andR56andexaminedthetraceplots(seeFigure1).TherstcolumnisfromthePX-RPMHalgorithmandthesecondisfromtheRW-MHalgorithm.ItisclearthatthePX-RPMHalgorithmismixingbetterthantheRW-MHalgorithm. 4)Exerciseplusgrowthhormone.Theplaceboandgrowthhormonetreatmentswereadministereddailyviainjections.Variousmusclestrengthmeasureswererecordedatbaseline(0months),6monthsand12months.Weareinterestedintheoutcomeofmeanquadricepsstrength.Thedropoutratesinthefourtreatment

PAGE 61

Table2{1. Posteriormeansand95%credibleintervalsforthecorrelations,variances,andregressionparametersforthegrowthhormonedataexample. Groups(i)Rdiag(Vi) 514.03(384,742)68.4(61.4,75.2)2-589.(448,822)66.0(58.6,73.7)492.(357,672)65.1(58.0,72.0) 606(465,775)65.6(58.1,73.0)3-640(507,818)81.0(73.0,88.7)465(350,601)72.4(65.6,79.2) 689(517,932)65.2(57.4,73.2)4-628(477,868)62.1(54.5,69.8)554(407,782)62.6(55.0,70.4) groupswere11/41,13/41,9/40and16/38,respectively.Wewillconductouranalysisunderanassumptionofrandomdropout(MAR)forillustrationofourmethod. LetYij=(Yij1;Yij2;Yij3)denotethevectoroflongitudinalresponsesforsubjectjingroupi(i=1;;4;j=1;;ni).Xijisspeciedsuchthateachcomponentofcorrespondstoatime/treatmentspecicmean.WeusethemodeldescribedinSection2.2,YijN(Xij;i),withi=ViRVi.Wespecifypriorsfor,ViandRasfollows,()/expf1 2(0)01(0)g(diag(Vi))/TYt=1(2it)(rit+1)expit

PAGE 62

Figure2{2. Traceplotsof3componentsofRoverthelast2000iterationsof10,000iterationsforthePX-RPMHalgorithmandRW-MHalgorithm.

PAGE 63

Detailsonsampling(Ymis;;Vi)canbefoundinAppendixB.ThefullconditionaldistributionofRisgivenby(RjY;)/jRjni 2trh4Xi=1niXj=1ViYijXijYijXij0ViR1ioIfRjk:jRjkj1andRispos.def.g: 2.4.3 .Forcomparison,wealsosampleRusingtheRW-MHalgorithmasdescribedinSection 2.2.2 . Weranachainof10,000iterationsusingthetwodierentmethodstosampleRandexaminedtraceplotsofthelast2000samples(seeFigure 2{2 ).ItisclearthatthePX-RPMHalgorithmismixingbetterthantheRW-MHalgorithmaswasthecasewiththesimulationsinSection4.Table 2{1 containstheposteriormeanand95%credibleintervalsfortheelementsofthecommoncorrelationmatrix(R),thegroup-specicvariances(Vi),andthemeanparametersbasedonthechainrunusingthePX-RPMHalgorithm.TheeasewithwhichthismodelcanbetusingthePX-RPMHalgorithmshouldencourageitsconsideration. Theexpansionparametersplayanimportantroleinouralgorithm.IfthesameexpansionparametersalsomaketheDAalgorithmconvergefaster,thenwemayunifytheDAalgorithmandthePX-RPMHalgorithmandmaketheMCMCalgorithmmoreecient.WhethertheexpansionparametersusedinouralgorithmacceleratetheDAalgorithmneedsfurtherstudy.

PAGE 64

Thischapterisorganizedasfollows.WewillintroduceaclinicaltrialasanexampletomotivatethisresearchinSection 3.2 .InSection 3.3 ,wewillpresentjointmodelsandhierarchicalpriorstoparsimoniouslymodeltheassociationoflongitudinalbinaryandcontinuousprocesses.MCMCsamplingtechniquestoestimatetheposteriordistributionofparameterswillbedescribledinSection 3.4 .Severaltechnicalissueswillalsobeaddressed.WewillderiveseveralmeasuresformodelcomparisoninSection 3.5 .InSection 3.6 ,wewillpresenttheresultsfromtheanalysisoftheclinicaltrialdata. 54

PAGE 66

IntheCTQIIclinicaltrial,theprimaryoutcomewasquitstatus(alongitudinalbinaryoutcome),butanotheroutcome,weightgain(alongitudinalcontinuousoutcome),wasalsomeasured.Theinvestigatorswereinterestedintwoquestions:1)Doesmoderate-intensityexercisehavesignicanteectsonsmokingcessation?InthestudyoftheCTQItrial,theinvestigatorsfoundsignicantdierencesbetweenvigorousactivityandcontactcontrolgroupthrough12monthsoffollowup;2)Whatisthemechanismforexerciseasanadjuncttherapytosmokingcessation?Inthischapter,ourmaininterestisthesecondquestion.Bymodellingtheassociationbetweenquitstatusandweightgain,wewillseewhetherexercisecanweakentheassociationofthesetwooutcomes. Forstatisticalanalysis,weremovedtheobservationsatweek1and2fromdatasincethequitratesinthesetwoweekswereverylow(0:00%and1:10%).Althoughparticipantswereencouragedtoperformmakeupsforsessionsmissed,therestillexistedintermittentmissingvaluesinquitstatusand/orweightgain.Inaddition,alargenumberofsubjectsdroppedoutbeforetheendoftheexperiment.WewilladdresstheseissuesinSection 3.6 . 3.3.1JointModels

PAGE 67

normalspecication(Y0ij;W0ij)0N(Xij;i),Vij=0B@YijWij1CA=Xij+i; whereXijisthedesignmatrix,isthevectorofregressioncoecients,andiN(0;i)withi=0B@i;11i;12i;21i;221CA.Usingthisprobitformulationforthebinaryprocess,wehaveQij;t=IfYij;t>0g.ToestimatetheassociationbetweenQijandWij,weneedmodelsfori;12asafunctionoftreatmentand/orothersubjectspeciccovariatesthatmightaectthisrelation.However,bothi;12andtheentirecovariancematrixiarediculttomodelduetopositivedenitenessconstraints(DanielsandKass,2001;PourahmadiandDaniels,2002).Toaddressthisproblem,wefactorthejointdistributionofYijandWijintotwocomponents:amarginalmodelforYijandacorrelatedregressionmodelforWijgivenYij.LetXij=0B@X1ij00X2ij1CAand=0B@121CA,thenthenewmodelsareYij=X1ij1+1i whereBi=i;211i;11isthematrixthatreectsassociationbetweenYijandWij,1iN(0;i;11),2iN(0;i;22)andi;22=i;22i;211i;11i;12.Itiseasytoseethat( 3-2 )isacorrelatedprobitmodeland( 3-3 )isastandardcorrelatedregressionmodel,conditionalonthelatentvariableYij.Foridentiability,itiscommontorestricti;11tobeacorrelationmatrixRi;11in( 3-2 )(ChibandGreenberg,1998).Intherestofthisproposal,wewilluseRi;11insteadofi;11asnotationforthecovariancematrixin( 3-2 ).Theadvantageofthislatentvariableformulationisthatthecoecientsofthesecondmodelaredirectlyrelatedtothevarianceandcorrelationtermsfromtheunderlyinglatentvariablemodel.Thisfactorization

PAGE 68

providesaconvenientparametrizationtoexaminetheassociationbetweenYijandWijsincethecomponentsoftheBimatrixareunconstrained. Besidebeingunconstrained,theassociationmatrixBiinmodel( 3-3 )hasaconvenientinterpretation.ThetthrowofBireectstheassociationofthecontinuousprocessatweektwiththebinaryprocessatallweeks(t=1;;T).Inparticular,itcorrespondstotheregressionmodelwij;tjYij=x02ij;t2+b0i;t(YijX1ij1)+2i;t,whereb0i;t=(bi;t1;;bi;tT)isthetthrowofBi.SincethecovariancematrixofYijisacorrelationmatrix,thecomponentsofBicanbethoughtasthestandardizedregressioncoecients.ThispropertyofBiwillfacilitatecomparisonamongcomponentsofBi. 3.3.1 .DenotebitobethevectorobtainedbystingingtherowsofBi(i=1;;m).Wewritethejointpriorinourmodelsas:(;Ri;11;b1;;bm;i;22)=mYi=1()(Ri;11)(biji;22)(i;22): Wespecifyaatpriorfor;thatis()/1: Ajointuniformprior(derivedbyBarnardetal:(2000))isputonRi;11as(Ri;11)/Ifri;jk:jri;jkj1andRi;11ispositivedeniteg; whereri;jkistheelementofthejthrowandkthcolumninRi;11.ThispriorispropersinceRi;112[1;1]T(T1) 2hasacompactdomain.Fori;22,weuseeitheran

PAGE 69

inverseWishartpriororaatprior, 4TYi=1+1i 2exp1 2trS122or(i;22)/Ifi;222(;1)T(T+1) 2andi;22ispositivedeniteg:(3-6) Wenowprovidedetailsontheprior(biji;22).SincethecomponentsofBiaretheregressioncoecientsofWijonYij,weexpectmanyofthecomponentsofBitobezeros.ThisprovidesagoodwaytoparsimoniouslymodeltheassociationmatricesBi.Todothis,wespecifyahierarchicalpriordistributionthatessentiallyallowsthecomponentsofBitobezero.WeborrowideasfromGeorgeandMcCulloch(1993,1997),andSmithandKohn(2002)here.ThekeyfeatureofthehierarchicalprioristhateachcomponentofBiismodelledashavingcomefromamixtureoftwonormaldistributionswithdierentvariances.Introducingthelatentindicatori;tl,werepresentournormalmixturebybi;tlji;tl(1i;tl)N(0;0i;tl)+i;tlN(0;1i;tl) (3-7)i;tljpiBernoulli(pjtlj1=a0+1i) (3-8)piBeta(ri;i); wherebi;tlistheregressioncoecientofwij;tonyij;l,a0isatuningparameter,and(0i;tl;1i;tl;ri;i)arethecorrespondinghyper-parameters(l;t=1;;T).Wheni;tl=0,bi;tlN(0;0i;tl)andwheni;tl=1,bi;tlN(0;1i;tl).Theideaisthatif0i;tl(>0)issetsmall,thenbi;tlwouldbesosmallthatitcouldbe\safely"estimatedby0wheni;tl=0;if1i;tlissetlarge(1i;tl>>0i;tl),thenanon-0estimateofbi;tlwouldbelargewheni;tl=1.Basedonthisinterpretation,pjtlj1=a0+1iinprior( 3-8 )canbethoughtofasthepriorprobabilitythatbi;tlwill

PAGE 70

requireanon-0estimate,andjtlj1=a0impliesthatbi;tlismorelikelytobe0asjtlj1=a0getsbigger.ThismeansthatthecomponentsofBibecomesmalleraprioriastheymoveawayfromthemaindiagonal. Toaccountforpossiblecorrelationbetweenbi;tl,wecanreplace( 3-7 )withbijiN(0;DiRiDi); wherebiisacolumnvectorobtainedbystingingtherowsofBiin( 3-3 )andiisthecorrespondingindicatorvector.RiisthepriorcorrelationmatrixandDiisadiagonalmatrixwithdiagonalelementsbeing0i;tlifi;tl=0and1i;tlifi;tl=1.Typically,theratioof1i;tland0i;tlisxedand0i;tlistreatedasatuningparameter.Howtochoose0i;tland1i;tlwillbediscussedinSection 3.4.2 . Analternativetoxing0i;tlistoputaprioron0iandestimateitbasedondata.Inthiscase,wereplace( 3-7 )withthecorrespondingprior(biji;0i)(2)T2 2jHi(0)j1 2exp1 2(bibi0)0H1i(0)(bibi0); whereriandiaretheshapeandscaleparametersofthedistribution,respectively.Howtoestimate20iwillalsobedetailedinthenextsection. 3.4.1TheMCMCSamplingAlgorithm

PAGE 71

thatwillbeusedtoimputethelatentdataandthemissingvalues.LetQobsbethevectorofobservedbinaryoutcomes,Qmisbethevectorofmissingbinaryoutcomes,WobsbethevectorofobservedcontinuousoutcomesandWmisbethevectorofmissingcontinuousoutcomes.DenoteYobstobethelatentvectorassociatedwithQobsandYmistobethelatentvectorrelatedtoQmis.DeneYtobe(Yobs;Ymis),Qtobe(Qobs;Qmis),Wtobe(Wobs;Wmis)andtobetheparametervector(;Ri;11;Bi;i;pi;i;22)or=(;Ri;11;Bi;0i;i;pi;22).Weusethegenericnotationfforthedistributionofresponses,forthepriorandposteriordistributionsofrelatedparameters,andLforthelikelihoodfunction.OuralgorithmwillconsistofaDAimputationstepandaposteriorsamplingstepasfollows: (1)DAIStep(dataaugmentationimputationstep):SamplelatentdataYandmissingvaluesWmisfromf(Y;Wmisj;Qobs;Wobs).Todothis,wefactorthisdistributionas (2)PSstep(posteriorsamplingstep):Generatefromf(jY;W)usingGibbssampling. TheDAI-stepinvolvessamplingYobsj;Qobs;Wobs(truncatedmultivariatenormal),Ymisj;Yobs;Wobs(multivariatenormal),andWmisj;Wobs;Y(multivariatenormal).Oncewesamplethelatentdataandthemissingvalues,weneedtosamplefromthefullconditionaldistributionsofthecomponentsof.ThiscanbecompletedbyusingtheGibbssampler.AllthefullconditionaldistributionsfortheGibbssamplerarederivedinAppendixC. Werepeatedlysamplelatentdataandmissingvaluesfrom( 3-12 ),andparametersfromtheirfullconditionaldistributions(seeAppendixC),separately,untiltheMarkovchainofparametersconverges.WerunthechainforGadditionaliterationsaftertheburn-inperiodforinference.

PAGE 72

Forsimplicity,assumethatYisaT1vectorandYN(;)IU1,whereU12CTisatruncationregionwithpositiveLebesguemeasure.IfwepartitionY,andasY=0B@ytY(t)1CA,=0B@t(t)1CAand=0B@tt202221CA,thenwehave wherewehave HereU1t=fyt2C:(yt;y(t))2U1g.Geweke(1991)andRobert(1995)proposedaGibbssamplingalgorithmtosampleY.ThekerneloftheMarkovChainy(k)=(y(k)1;;y(k)T)inthisalgorithmwasobtainedbysuccessivelygeneratingthecomponentsofyfromtheirfullconditionaldistributions(y(k)tjy(k)1;;y(k)t1;y(k1)t+1;;y(k1)T).AdisadvantageofthisalgorithmisthatitrequiresrepeatedlycomputingtheTmeansandvariancesgivenin( 3-14 ).ThefollowingpropositionprovidesasimplewaytosamplefromtheTMVNDwithouttheneedtocompute( 3-14 )eachtime.

PAGE 73

3-14 )andtheunivariatetruncationintervalsU2tcanbeeasilyderived.Forexample,inourmodel,YTN(;R)IU1withU1=fy2CT:S1y0g,whereS1isandiagonalmatrixwithS1(t;t)=1ifQt=0and-1ifQt=1(t=1;;T).BythetransformationZ=P1Y,wehaveZTN(;ITT)IU2,where=P1,U2=fz2CT:S2z0gandS2=S1P. So,atiterationk,z(k)tjz(k)1;;z(k)t1;z(k1)t+1;;z(k1)TN(t;1)ISt,wheretisthetthelementofandSt=fzt2C:S2z0g.LetS2(t)bethematrixfs1;;st1;st+1;;sTg0andz(t)denotethevectorfz1;;zt1;zt+1;;zTg0.TheU2tisthengivenbyU2t=fzt2C:stztS2(t)z(t)g: TheMarkovChainy(k)=(y(k)1;;y(k)T)inourimplementationoftheGibbssamplercanbeobtainedbyrstgeneratingallcomponentsofzonebyonefrom(z(k)tjz(k)1;;z(k)t1;z(k1)t+1;;z(k1)T)(t=1;;T),andthentranslatingbacktoy(k)byy(k)=Pz(k). 2 .Inthis

PAGE 74

algorithm,thedicultyofsimulatingRcanbeovercomebycreatingan`expanded'modelinwhichRcanbetransformedtoalessconstrainedcovariancematrixbyborrowingthescaleparametersfromanexpansionparametermatrix. Inthejointmodels,dene(R)tobetheparametervectornotincludingRandDtobetheexpansionparameter.ThediagonalmatrixDistheexpansionparameterthatweneedtotransformRintoalessconstrainedcovariancematrix=DRD.Considerthefollowingone-to-onemappingfromfYij;R;Bi;HigtofYij;;Bi;Hig8>>>>>>>>>><>>>>>>>>>>:Yij=X1ij+D1YijR=D1D1Bi=BiDHi=diag(D;;D)Hidiag(D;;D)(i=1;;m;j=1;;ni); whereHi=DiRiDiisdenedin( 3-10 )andPmi=1Pnij=1Y2ijt=1foranyt=1;;T.Notethatin( 3-16 )isxedandnotinvolvedinthetransformation.Given,thestepthatdrawsYijimplicitlydrawsYijandDbecausePmi=1Pnij=1(Yijtx01ijt)2=D1ttPmi=1Pnij=1Y2ijt=D1tt,whereDttisthetthelementofDandx01ijtisthetthrowofX1ij.Thespacefor(Yij;)ishigherdimensionalthanthatfor(Yij;R)sinceRhasfewerparametersthan.Theconstraints,Pmi=1Pnij=1Y2ijt=1foranyt=1;;T,areneeded,tomakethecandidatetransformationaone-to-onemapping.ByspecifyingthefollowingcandidatepriorforR,givenby(R)/jRj3T1 2IfRkl:jRklj1andRispos.def.g; whereTisthedimensionofR,wecanderiveaparameterexpandedcandidatedensity(PXCD)forbasedonthefollowingproposition

PAGE 75

3.3.2 ,thenfromthelikelihoodfunctionforcompletedatain( 3-2 ),transformation( 3-16 )andcandidateprior( 3-17 ),weobtain(jY;)/jj+T+1 2expn1 2tr(S1)o; 8 givesthePXCDoftouseastheproposaldensityinthePXMHstage.Inthisstage,werstsimulatefrom( 3-18 )andthenobtainthecorrelationmatrixRthroughthereductionfunctionP(;)=(;D1D1).Second,wekeepthecandidateRwithprobability(theacceptancerateintheM-Halgorithm).SamplingRbasedonthePX-RPMHalgorithmforjointmodelsisgiveninthefollowingtheorem. 3.3.2 for(R)andR,thenundertransformation( 3-16 )andcandidateprior( 3-17 ),simulatingRisequivalenttosimulatingrstfromtheinverseWishartdistribution( 3-18 ),andthentranslatingitbacktoRthroughR=D1D1in( 3-16 )andacceptingthecandidateRusingaMetropolisHastingsstepwithsomeacceptancerate,where=min1;exp3T1 2(logjRjlogjR(k)j)atiterationk+1. 4 providesasimplewaytosimulatethecorrelationmatrixinmultivariateprobitmodels.InChapter 2 ,weshowedthatthePX-RPMHismoreecientthanothermethods,suchastheGriddy-Gibbssampler(RitterandTanner,1992)andrandomwalkMetropolis-Hastingsalgorithms(RobertsandSmith,1995).

PAGE 76

3-7 )canbeviewedashowtochoose0i;tlandci;tl. Thechoiceofci;tlin( 3-7 )and( 3-10 )shouldbesuchthatifbi;tlcomesfromN(0;ci;tl0i;tl),thenanon-0estimateofbi;tlshouldbeincludedinthenalmodel.FromasubjectivistBayesianstandpoint,onewouldliketochooseci;tllargeenoughtogivesupporttovaluesofbi;tlthataresignicantlydierentfrom0,butnotsolargethatunrealisticvaluesofbi;tlaresupported.UsingtheideafromGeorgeandMcCulloch(1993,1997),wechooseci;tlbyconsideringtheintersectionpoints!(ci;tl)i;tloftwonormaldensitiesN(0;0i;tl)andN(0;ci;tl0i;tl),where!(ci;tl)=q Thechoiceof0i;tlin( 3-7 )and( 3-10 )shouldbesuchthatifbi;tlcomesfromN(0;0i;tl),thenbi;tlshouldbeexcludedfromthejointmodels.Wenotethat0i;tlcapturesthepracticalsignicanceofbi;tlwhichdiersfromthestatisticalsignicanceofbi;tlcapturedbystandarddeviation^bi;tloftheestimator^bi;tl.AgainborrowingtheideafromGeorgeandMcCulloch(1993),anapproachtoselecting0i;tlmaybeobtainedbyconsideringtheintersectionpointti;tlbi;tlofthenormaldensities(^bi;tljbi;tl;i;tl=0)N(0;2bi;tl+20i;tl)and(^bi;tljbi;tl;i;tl=1)N(0;2bi;tl+ci;tl20i;tl),inwhichti;tlisdeterminedby Thisti;tlmaybethoughtasthethresholdatwhichbi;tlshouldbeincludedinthejointmodel.Notethatti;tldependsonlyontheratiobi;tl=0i;tlandci;tl.Itcan

PAGE 77

belargeorsmallbyadjustingtheratiobi;tl=0i;tlandci;tl.Largeti;tlfavormoreparsimoniousmodels. 3-2 )and( 3-3 ),thelikelihoodfunctionisL(;Ri;11;Bi;i;22jQ;W)=mYi=1niYj=1Z[Yij2Yij]f(Yijj;Ri;11)f(WijjYij;;Bi;i;22)dYij; 3-7 ),( 3-8 )and( 3-9 )for(Bi;i;pi),thejointposteriordistributionisgivenas where(;Ri;11;Bi;i;pi;i;22)=()(Ri;11)(Biji)(ijpi)(pi)(i;22).( 3-20 )isintegrableifandonlyif=ZL(;Ri;11;Bi;i;22jQ;W)(Biji)(ijpi)(pi)ddR11dBididpidi;22<1: Thefollowingtheoremgivesconditionsfortobenite. 3-2 ).WriteX1ij=(x1ij;;x1ij)0andx1ij;t=hij;tx1ij,wherex01ijistheanyrowofX1ij.LetI[S]denotetheindicatorfunctionsuchthatI[S]=1ifSistrueand0otherwise.Assumethatthefollowingconditionsaresatisedforanyi(i=1;;c) (T1)ni>T2,whereniisthenumberofsubjectsin(treatment)groupi;

PAGE 78

(T3)X1iisoffullrank,whereX1i=(X01i1;;X01ini)0; 3-21 ). 3-2 )and( 3-3 ).Forexample,wemightassumethatRi;11=R11andi;22=22whentherearenotenoughdatatoestimatealltheparameters.ThesealternativemodelsmaybecomparedbyusingtheBayesfactor(KassandRaftery,1995)orthedevianceinformationcriterion(DIC)(Spiegelhalteretal:,2002).

PAGE 79

reexpressedasBF(M1;M2)=m(OjM1) wherethemarginallikelihoodsinthenumeratoranddenominatorcanbecomputedusingthemethodoutlinedinthenextsection. 3.4.1 .ThemarginallikelihoodofmodelMkisdenedasm(Qobs;WobsjMk)=Zf(Qobs;WobsjMk;)(jMk)d; where(jMk)isthejointpriorand(jMk;Qobs;Wobs)isthejointposteriordistribution.Notethatthemarginallikelihoodin( 3-23 )isfreeof.Wemayxatapointofhighposteriordensitysuchasthejointposteriormean.Takingthelogofbothsidesof( 3-23 )andreplacingwith=(;Ri;11;Bi;i;Pi;22),wehave logm(Qobs;WobsjMk)=logf(Qobs;WobsjMk;)+log(jMk)log(jMk;Qobs;Wobs):(3-24)

PAGE 80

WenowsuppressthedependenceonMkanddiscusstheestimationofthethreetermsin( 3-24 ).Evaluationofthejointprior()isstraightforward.Tocomputetherstterm,thelikelihoodfunctionf(Qobs;Wobsj),weneedtocomputef(Qij;obs;Wij;obsj),whichcanbeexpressedas 2exp(1 2nXi=1(Vij;obsXij)01ij;obs(Vij;obsXij))dYij;obs;(3-25) whereVij;obs=(Yij;obs;Wij;obs),dij=dim(Vij;obs)andAdij;;A1denotetruncatedintervalofeachcomponentofVij;obs(i=1;;m;j=1;;ni).WecalculatethisintegralbyMonteCarlointegrationgiven. Finally,weconsidercomputationof(jQobs;Wobs),whichistheordinateoftheposteriordistributionat=.Itcanbere-writtenas where(jY;W)isanevaluationat=ofthejointposteriordistributiongivencompletedata(Y;W).Supposethat(Y;Wmis)isasampleofsizeMfromitsdistributiongiven=.Thenusing( 3-26 ),anestimatorof(jQobs;Wobs)isgivenby^(jQobs;Wobs)=1 Now,weneedtoevaluatetheterminthesumin( 3-27 ),(jY(i);W(i)mis;Wobs)foreachiterationitoobtaintheestimateof(jQobs;Wobs).Forsimplicity,inthefollowingwewrite(jY(i);W(i)mis;Wobs)as(jY;W).Tocompute(jY;W)

PAGE 81

givencompletedata,wedecomposeitas ThenwecanusereducedMarkovchainMonteCarlotechniques(Chib,1995)toestimateeachtermin( 3-28 ).Detailsonestimatingeachterm(exceptfor(Ri;11jY;W;Bi;i;Pi;22))canbefoundinAppendixC.TheycanbecomputeddirectlyfromtheGibsandMetropolis-HastingsoutputsusingideasinChib(1995)andChibandJeliazkov(2001).However,tocompute(Ri;11jY;W;Bi;i;Pi;22),weneedtoextendsomepreviousresults. UsingtheideainChibandJeliazkov(2001),wecanestimatethistermas ^(Ri;11jY;W;Bi;i;Pi;22)=A5 whereA5=G15G5Xg5=12(R(g5)i;11;Ri;11jY;W;(g5);Bi;i;Pi;22)q2(R(g5)i;11;Ri;11jY;W;(g5);Bi;i;Pi;22); A3-2 )(seeAppendixC),2isthecorrespondingmoveprobability,and(R(g5)i;11;(g5))and(R(h2)i;11;(h2))aretheMCMCsamplesfrom(Ri;11;jY;W;Bi;i;Pi;22)and(jY;W;Ri;11;Bi;i;Pi;22)q2(Ri;11;Ri;11jY;W;;Bi;i;Pi;22),respectively.SinceweuseaPX-RPMHalgorithmtosampleRi;11,q2usuallydoesnothaveaclosedform.Hereq2isgivenbyq2(R(g5)i;11;Ri;11jY;W;(g5);Bi;i;Pi;22)=Z(D;Ri;11jY;W;(g5);Bi;i;Pi;22)dD;

PAGE 82

where(D;Ri;11jY;W;(g5);Bi;i;Pi;22)(g5=1;;G5)isthejointdensityof(D;Ri;11)withRi;11=Ri;11.Theintegralin( 3-30 )canbecomputedbyimportancesampling.Thekeyistochooseanappropriatecandidatedensityfunctionh(DjY;W)tomakeevaluationof( 3-30 )accurate.Wewantacandidatedensityh(DjY;W)fromwhichitiseasytosampleandisagoodapproximationof(D;Ri;11jY;W;(g5);Bi;i;Pi;22)(i.e.,theratioofh(DjY;W)and(D;Ri;11jY;W;(g5);Bi;i;Pi;22)isapproximatelyaconstant).HerewechoosetheproductoftheindividualfullconditionaldistributionsofDtt(t=1;;T)ash(DjY;W).Rewritingthedistribution(D;Ri;11jY;W;(g5);Bi;i;Pi;22)as(D;RjY;W;),wehave whereD(tt)containsallcomponentsofDexceptDtt.Thechoiceofh(DjY;W)in( 3-31 )isnotaregulardistributionfromwhichsamplesareeasytodraw.Detailsonhowtosamplefrom( 3-31 )aregiveninAppendixC.Onceweobtainasamplefrom( 3-31 ),wecanestimate( 3-30 )as^q2(Ri;11jY;)=KXk=1(D(k);RjY;) WeusethesamplesfromthereducedMarkovchainMonteCarlorunstoestimateeachtermin( 3-28 ).Sincethiscanbecomputationallyintensive,weintroduceapotentiallysimplermethodtoestimatethemarginallikelihoodinthenextsection.

PAGE 83

samplingalgorithmthatisusedtogeneratetheMCMCsampletobeknown.UsingthenotationinSection 3.4.1 andsuppressingmodelnotationMk,weexpressthejointposteriordistributionas(;Y;WmisjYobs;Wobs)=f(Qobs;Wobsj;Y;Wmis)f(Y;Wmisj)() wheref(Y;Wmisj)isthedensityof(Y;Wmis)giventheparametervector=(;Ri;11;Bi;i;pi;22)andm(Qobs;Wobs)isthemarginallikelihoodinthejointmodels.f(Qobs;Wobsj;Y;Wmis)f(Y;Wmisj)isthejointlikelihoodforthecompletedata(Y;W).Assumethath(jY;Wmis)isafunctionsuchthatRh(jY;Wmis)d=1.Thenanewidentitycanbederivedas whereisanxedpointofwithhighposteriordensityandtheexpectationistakenwithrespectto(;Y;WmisjQobs;Wobs).Takinglogarithmsonbothsidesof( 3-33 ),wehave logm(Qobs;Wobs)=logf(Qobs;Wobsj)logEh(jY;Wmis) Itiseasytoshowthatwhenwechooseh(jY;Wmis)=(jY;W),thentheidentitygivenin( 3-34 )reducestothekeyidentity( 3-24 )wederivedinthelastsubsection.

PAGE 84

Suppose((g);Y(g);W(g)mis)(g=1;;G)isanMCMCoutputfromthejointposterior(;Y;WmisjYobs;Wobs)in( 3-32 ).Thenusing( 3-34 ),anestimatoroflogm(Qobs;Wobs)isgivenby log^m(Qobs;Wobs)=logf(Qobs;Wobsj)log"1 Itiseasytoshowthattheestimatorgivenin( 3-35 )isasimulationconsistentestimatoroflogm(Qobs;Wobs). Next,wediscussthechoiceofh(jY;Wmis).Whentheprior()isproper,acrudechoiceofhissimplyh(jY;Wmis)=().Withthischoice,( 3-35 )reducesto log^m(Qobs;Wobs)=logf(Qobs;Wobsj)log"1 Althoughthischoiceofh(jY;Wmis)yieldsasimulationconsistentestimatoroflogm(Qobs;Wobs),thischoicemaynotbeoptimalinthesensethatthevarianceoftheestimatorin( 3-36 )isnotminimal.Toobtainanoptimalchoiceofh,wecanrewrite( 3-35 )as log^m(Qobs;Wobs)=logf(Qobs;Wobsj)log"1 Thentheoptimalchoiceofh(jY;Wmis)is

PAGE 85

Withthischoice,( 3-37 )reducesto log^m(Qobs;Wobs)=logf(Qobs;Wobsj)log"1 Itisobviousthatinestimationoflogm(Qobs;Wobs),( 3-39 )isequivalentto( 3-24 ).Tosupporttheoptimalityofthechoicein( 3-38 ),wereferthereadertoTheorem1inChen(2005). Spiegelhalteretal:(2002)proposedadevianceinformationcriterionDIC,denedasaclassicalestimateoft,plusapenalty(twicetheeectivenumberofparameters);thatis,DIC=D( andpDisgivenbypD=2EjQobs;Wobs(logf(Qobs;Wobsj))+2logf(Qobs;Wobsj

PAGE 86

modelcomparison,wecanwritethekerneloftheDICas DIC=4EjQobs;Wobs(logf(Qobs;Wobsj))+2logf(Qobs;Wobsj ThesecondtermcanbecalculatedusingtheprocedureinSection 3.5.1 .Next,weaddressthecomputationoftherstterm.EjQobs;Wobs(logf(Qobs;Wobsj))canbeexpressedas Assumethat(g)isasampledvaluefromtheposteriordistribution(jQobs;Wobs).Thenweestimate( 3-41 )as^EjQobs;Wobs(logf(Qobs;Wobsj))=1 logf(Qobs;Wobsj(g))=logZAQf(Y;Wobsj(g))dY=logZAQf(Y;Wobsj(g)) whereAQisthetruncationregionofY,andf(Y;Wobsj 3-42 )canbeestimatedaslog^Ef(Y;Wobsj(g))

PAGE 87

Table3{1. Modelsunderconsideration Settings ModelsBiPriorforBiRi11i22 Footnote:NormalpriorbiN(0;106I). Table3{2. EstimatesofthemarginallikelihoodsandBayesfactorscomparedtoM5 -1413.2-1415.4-1884.7-1401.3BF 7:8410238:6910241:33102271:151017 Forthemeanofthetwolongitudinalresponsesasafunctionofcovariates,wesetin( 3-1 )asthevectorofmeansateachtimepointacrosstreatments;thus,thetthrowofthedesignmatrixisavectorof0'switha1inthetthslot. 3{1 givesthedetailsofallthemodelsconsidered.

PAGE 88

Table3{3. EstimatesoftheDIC Items 2841.52843.03775.82834.6 Items Table3{4. Posteriorpredictivep-valuesforquitrate(QR)andaverageweightgain(AWG)bytimeandtreatments Weeks TreatmentsItems123456 NoexerciseQR0.6250.5410.3130.2170.5430.436AWG0.4810.4210.5230.4200.5370.465 ExerciseQR0.4520.4360.4520.2980.4050.261AWG0.4020.3550.4190.6030.3050.408 ForeachofthemodelsinTable 3{1 ,wecomputethemarginallikelihoodusingthemethodologyderivedinSection 3.5.1 .Table 3{2 givesthemarginallikelihoods(ML)andBayesfactors(BF)forallthemodels.FromTable 3{2 ,wecanseethatthemarginallikelihoodformodelM5islargestandtheoneformodelM3issmallest.Themodelsusingshrinkagepriorsforassociationmatricestbetterthanthoseusingcommonnormalpriors.AccordingtoBayesfactors,modelM5tstheCTQIIclinicaltrialdata`best'amongtheninemodelsconsidered. WealsocomputedtheDICbasedonthemethodderivedinSection 3.5.2 .TheDICforthenineconsideredmodelsarelistedinTable 3{3 .M5againtsbestamongthesemodels.TheconclusionbasedontheDICisconsistentwiththatusingBayesfactor.

PAGE 89

Figure3{1. Histogramsofdierencesbetweentherealizedandpredictedquitrates.

PAGE 90

Histogramsofdierencesbetweentherealizedandpredictedquitrates(continued).

PAGE 91

Table3{5. Posteriormeansand95%credibleintervals(CIs)forthequitrates(QR:quitrate;CIL:lowerbound;CIU:upperbound) NE Items123456 QR0.3940.4960.5860.5300.5300.458CIL0.3380.4380.5280.4710.4720.401CIU0.4520.5550.6450.5880.5880.517 Items123456 QR0.4500.3740.4360.4240.4600.378CIL0.3950.3210.3800.3690.4040.325CIU0.5070.4290.4920.4810.5170.433 FortheCTQIIdata,T()denotestheweeklyquitratesorweeklyaverageweightgainsandthepvaluecanbeinterpretedastheprobabilitythatthereplicatedquitratesoraverageweightgainislargerthanthecorrespondingrealizedquitratesorweightgain.Notethatin( 3-43 ),wehaveusedthepropertyoftheposteriorpredictivedistributionthat(vrepcomj;vobs)=(vrepcomj).

PAGE 92

WesamplefromtheposteriorpredictivedistributionusingtheoutputfromtheMCMCalgorithm.Foreachsampleofdrawsfromthejointposteriordistribution,wedrawonevrepcomfromthepredictivedistribution.TheposteriorpredictivecheckisthecomparisonbetweentherealizedtestquantitiesT(v(g)com;(g))(g=1;;G)andthepredictivetestquantitiesT(vrep(g)com;(g)).Theestimatedp-valueistheproportionoftheseGsimulationsforwhichthepredictivetestquantityequalsorexceedsitsrealizedteststatisticvalue. WealsocalculatedthedierencebetweentherealizedandpredictivestatisticsforeachofGsimulatedpairs.Figure 3{1 showsthedistributionoftheGsimulatedpairsateachweek.Thesearemostlycenteredaroundzero. Table 3{4 givesthep-valuesforquitratesandaverageweightgainsateachweekacrosstreatments.Wecanseethattherearenoextremep-values(<0:05or>0:95)withmostaround0.5.ThesecheckssuggestthejointmodelsttheCTQIIclinicaltrialdatawell. 3.4.1 untilconvergenceandbasedinferenceonthelast10,000cyclesafterburn-in. Theposteriormeansand95%credibleintervals(CIs)foraregiveninTable 3{5 .Thequitratesovertimeintheexercisetreatmentareslightlylowerthanthoseinthenoexercisetreatment.The95%credibleintervalforthedierenceofquitratesbetweentwotreatmentsforthenalweekis(-0.012,0.148),marginallysignicant.Thissuggeststhattheexercisetreatmentdoesnothavepositiveeectonsmokingcessation. Thepi(i=1;2)denedin( A3-6 )canbeviewedasasummarymeasureoftheoverallmagnitudeoftheassociationbetweenweightgainandquitstatusacrosstreatments.Theestimatesofpiarep1=0:253(noexercise)andp2=0:175

PAGE 93

Table3{6. Posteriormeansoftheassociationmatriceswith95%CIsforeachtreatment Weight123456 NE10.39-0.60-2.26--3.06(-0.12,0.90)(-1.18,-0.01)(1.42,3.10)(-3.94,-2.18)2--0.40--0.74-(-0.25,1.05)(-1,48,0.00)3------4----0.32--(-1.00,0.37)5-0.87--0.96--0.51(-1.44,-0.30)(0.14,1.78)(-1.24,0.22)6---2.16-1.59-0.01(1.06,3.26)(-2.67,-0.51)(-0.81,0.79) E1-----0.67(0.35)-(-1.36,0.02)2-0.75(0.29)----0.92(0.43)(0.18,1.32)(-1.76,-0.08)3------4------5-0.90(0.30)---0.71(0.32)--(-1.49,-0.31)(-1.34,-0.08)6--1.39(0.45)0.84(0.47)---(-2.27,-0.51)(-0.08,1.76) (NE:noexercisetreatment;E:exercisetreatment) \-"meansthatthecorrespondingelementissmall,denedasP(=1jQobs;Wobs)<0:1 TheposteriormeansoftheassociationmatricesacrosstreatmentsaregiveninTable 3{6 .WehaveremovedfromthetablethoseelementsoftheBimatrixwhoseprobabilitiesofthecorrespondingindicatorsbeingequalto1islessthan0.1.Wecanseetheweakeningoftheassociationsbetweenweightgainandsmokingcessationbynotingthepresenceofmorezerosundertheexercisetreatment. Table 3{7 showsposteriormeansofthepairwisecorrelationswith95%credibleintervals.Fromthistable,wecanseethatweightgainandsmokingcessationappeartohavealaggedcorrelationstructure,andexercisemayweakenthepairwisecorrelations.Inthenon-exercisetreatment,thecorrelationsbetweenweightgainandquitstatusatweek5and6areallnegative.Thismeansthatpeoplewhogainweightearlyinthetrialaremorelikelytobesmokingattheend.Thesecorrelationsareessentiallyzerosundertheexercisearm.

PAGE 94

Table3{7. Posteriormeansofpairwisecorrelationswith95%CIsforeachtreatment Weight123456 NE1---0.14--0.24-0.29(-0.25,-0.02)(-0.35,-0.13)(-0.40,-0.18)2----0.12-0.17-0.16(-0.24,-0.00)(-0.29,-0.05)(-0.28,-0.04)3------4------5------60.110.120.160.20--(0.00,0.22)(0.01,0.23)(0.04,0.27)(0.07,0.32) E1------2------0.12(-0.24,-0.00)3------40.12-----(0.00,0.23)5------6--0.14-0.120.15(0.02,0.26)(0.01,0.23)(0.03,0.26) (NE:noexercisetreatment;E:exercisetreatment) \-"meansthatthecorrespondingelementisnotsignicant; inparenthesisisthe95%CI. 3{7 )betweenquitstatusandweightgainwouldbelikelywithbettercompliance(andthiswouldnotimpacttheresultsinthenon-exercisetreatmentforwhichnoncompliancewasnotanissue).Inaddition,thesmallsamplesizewiththe

PAGE 95

highdropoutrateandlowcomplianceresultedinlowoverallpowertodetecttheassociation(orlackthereof)betweenthetwoprocesses. Themethodologyinthischaptercanbeappliedtoanalysisofotherdatasetswheretherearetwoprocesses,andtheprimaryinterestistheassociationbetweenthetwoprocesses.Alsoourmethodologycanbedirectlyextendedtootherlongitudinalcases,suchasmodellingtheassociationoflongitudinalordinalandcontinuousprocesses(detailsinthenextchapter)ortwocontinuousprocesses.

PAGE 96

Wehavedevelopedanewalgorithm(thePX-RPMHalgorithm)forsimulatinganunstructuredcorrelationmatrixanddemonstrateditsimprovedperformanceoverotherstandardmethodsforsamplingacorrelationmatrix.Wealsohavedevelopedanewclassofjointmodelsfortheassociationoflongitudinalbinaryandcontinuousprocesses.ThesemodelswereappliedtoanalyzetheCTQIIclinicaltrialdata.ThestudyshowedthatthejointmodelsttheCTQIIclinicaltrialdatawellandcouldbeusedtounderstandthejointevolutionofsmokingstatusandweightgain.Theresultsshowedthat1)moderate-intensityexerciseappearedtonotbeeectiveonsmokingcessation.2)exercisemightweakentheassociationbetweensmokingstatusandweightgain. betweenMixedLongitudinalProcesses 3 ,wedevelopedmethodologyformodellingtheassociationbetweenlongitudinalbinaryandcontinuousprocesses.Ourmethodscanbeeasilyextendedtoothercasesoftwolongitudinalprocesses,suchasmodellingtheassociationbetweenlongitudinalordinalandcontinuousprocessesandtheassociationbetweenlongitudinalbinaryandordinalprocesses.Also,wecanusethismethodologyto 86

PAGE 97

modeltheassociationamongmorethantwoprocesses.Forbrevity,wehereonlyintroducetheextensionofmodellingtheassociationbetweenlongitudinalordinalandcontinuousprocesses. WeassumethatthedatavectorVi=(W0i;Q0i)0fori=1;;NconsistsofcontinuouscomponentsWi=(Wi1;;WiT1)0withlengthT1andordinalcomponentsQi=(Qi1;;QiT2)0withlengthT2(T1+T2=T).SupposeWifollowsamultivariatenormaldistributionwithmeanequaltoXi1,whereXi1isaT1bykdesignmatrixandisthenormalregressionparametercolumnvectorwithkdimensions,andcovariancematrixequaltoW.WeuseathemultivariateprobitmodelforQi(ChibandGreenberg,1998);thatis,weassumethereisanunderlyingnormallatentvariableYiforeachQi,fori=1;;N.TheelementQijtakesvaluesonthediscreteset0;1;;Jj1,andQij=lifandonlyifthelatentvariableYij2(j;l1;j;l]wherej;larethesetofcutpoints,forj=1;;T2andl=0;;Jj1.Wesetj;0=andj;Jj1=1fornotationalsimplicity,andj;1=0foridentiabilityofthecut-points.ThevectorQifollowsamultivariateprobitmodelifYiN(Xi2;RY)whereXi2istheT2bykdesignmatrixforthelinearmodelandRYisaT2byT2correlationmatrix.TomodeltheWiandtheYisimultaneously,weassumethatgiventhelatentvectorYi,thevector(W0i;Y0i)0N(Xi;C)whereXiisverticallypartitionedasXi1andXi2,andCisthecovariancematrixofthevector(W0i;Y0i)0. SimilarlytoChapter 3 ,wedecomposethejointmodelforamixtureoflongitudinalordinalandcontinuousprocessesintotwoparts:AmarginalmultivariateprobitmodelforYiandaconditionalmultivariateregressionmodelforWi.TheestimationprocedureusedinChapter 3 canbeusedforinferenceinthismodelexceptfortheinferenceoftheclassofcutpointparametersandtheimputationoflatentdata.LetY=(Y1;;YN);W=(W1;;WN)andQ=(Q1;;QN).ThelatentvariableYicanbeimputedbycyclingthefullconditionals

PAGE 98

withintervaltruncatednormaldistributions.ThetruncationischosensothatYijisconstrainedtoliebetweenthetwocutpointscompatiblewithQij.p(Yijj;C;;Wi;Qi;Yik;k6=j)/IijN(Yijj^ij;^ij); 3 .Wewillexplorethismodelinfuturework.

PAGE 99

3 next. ConsideralatentclassmodelwithLclasses.DeneStobetheunivariateunobservablelatentvariableofinterestwithvaluess2f1;;LgandYij=(Yij1;;YijT)tobethejointvectorofresponsesforthelongitudinalprocessforsubjectjingroupi.LetP(S=s)betheprobabilitythatasingleindividualisinclasssfors2f1;;Lgandf(YijjS=s)betheconditionaldistributionofthelongitudinalprocessgiventhelatentclasss.ThenthemarginaldistributionofYijcanbewrittenasf(Yij)=LXs=1P(S=s)f(YijjS=s): Thelatentclassmodelsin( 4-1 )canbegeneralizedtoallowmultivariatelongitudinalresponses.Basedonthejointmodels,thegeneralizationisinprinciplestraightforward:thejointdistributionofthemultivariateprocessfYij;WijgwillfollowaseparatemultivariatenormaldistributionateachlevelofdiscretelatentvariableS.Thejointmodels( 3-2 )and( 3-3 )tinSection 3.3.1 canbeusedwithineachlatentclass.Foridentiability,wemightassumethatassociationmatrixBibetweenYijandWijandregressioncoecientvectordependonS,butthecorrelationmatrixRi;11in( 3-2 )andconditionalcovariancematrixi;22in( 3-3 )are

PAGE 100

thesameacrossS.Givenlatentclassess,wehaveYijj(S=s)=X1ij1(s)+1i where1(s)and2(s)areregressioncoecientswithinlatentclasss,Bi(s)istheassociationmatrixofYijandWijwhichvaryacrosslatentclasses,and1iN(0;Ri;11)and2iN(0;i;22). WecanusesimilarcomputationalalgorithmsdescribedinChapter 3 tosamplefromtheposteriordistributionofparameters,conditionalonclassmembership.SamplingthelatentclassindicatorsandrelatedparameterscanproceedusingalgorithmsasoutlinedinGarretandZeger(2000).Thesemodelswillalsobefurtherdevelopedinfuturework.

PAGE 101

1 whereRq(s)meansitdoesnotdependons.Thersthalfof( A1-1 )holdssinceitisnotafunctionofs.Hence,thepositivedenitenessofRisequivalenttojRk(s)j>0orfk(s)>0.ThisshowsLemma 1 holdsfori=k.BypermutingrowsandcolumnsofR(s)asnecessary,thelemmafollowswhen1ik1. 2 whereCkk(s)isafunctionofsandd1isaconstantnotdependingons.SinceR=C0C,wehavekXj=1C2jk=1andk1Xj=1Cj;k1Cjk=Rk1;k: 91

PAGE 102

Basedon( A1-3 ),wederiveCk1;kandC2kkasCk1;k=sd2 A1-2 )and( A1-3 ),Lemma 2 holds. 1 2 ,wehavef(s)=b2s2+b1s+b0; 1 ,andb0isanconstant,notdependingons.Bydenitionsofc0,c1andc2,itisstraightforwardtoshowthat8>>>>>><>>>>>>:b2=1=2(2c2+2c1+3c0)b1=c2c1b0=1=2c0: FromLemma 1 ,weknowthatthevaluesofcomponentsofthecorrelationmatrixsatisfyingthepositivedenitenessconsistsofthoses'ssuchthatf(s)>0,i.e.,b2s2+b1s+b0>0: Let=b214b2b0,r1=(b1+p A1-4 )intotheseequalities,wehave8>>>>>><>>>>>>:=c21+c22c20r1=(c1c2+p Thus,rmin=minfr1;r2gandrmax=maxfr1;r2g.CaseI,CaseII,CaseIIIandCaseIVaredeterminedbythesignofb2and,respectively.By( A1-5 ),( A1-6 )

PAGE 103

andsimplealgebra,thepositivedenitenessintervalcanbeeasilyobtainedforeachcase. 2 2-7 ).Thefollowingvedeterminantsareneeded@(Y1;;Yn)0 2)TTYj=1(2jj)3 2/jD1j3@(Y1;;Yn)0

PAGE 104

=jD1jT+20B@In1ND100I31CA0B@P1P21CA+=jD1jT+2jD1jm1=jD1jn+T+1; 2-8 ),thejointdistributionofYandRgivenisp(Y;Rj)/jRjn+T+1 2exp(1 2nXi=1(YiXi)0R1(YiXi)): Throughtheone-to-onemappingT:(Y;R)!(Y;),wehavep(Y;j)/jRjn+T+1 2Jexp(1 2nXi=1Y0i1Yi)=jRjn+T+1 2jD1jn+T+1exp(1 2nXi=1Y0i1Yi)=jjn+T+1 2exp(1 2nXi=1Y0i1Yi): So,(jY;)/p(Y;j)/jj+T+1 2exp1 2S1; 1 2-7 ),and( A1-7 )and( A1-8 )intheproofofProposition 2 ,drawingYandRgivenisequivalenttodrawingYandrst,andthentranslatingbacktoYandRthroughT.R=D1D1willbeusedasthecandidatevalue.ItisacceptedintheMetropolis-Hastingsstepwithprobability.Thecanbederivedasfollows.Let1denotethepriororfullconditionaldistributionofRand2denotethe

PAGE 105

correspondingcandidatepriororproposaldensity.Then1(RjY;)/1(R)f(Yj;R) and2(RjY;)/2(R)f(Yj;R); 2-6 ),( 2-8 )and( 2-2 ),respectively.Theprobabilityofacceptanceatiterationk+1is=min1;1(RjY;)2(R(k)jY;) 2 2)=min(1;jR(k)j jRjT+1 2)=min1;expT+1 2(logjRjlogjR(k)j): 3 2 ,theJacobiancanbeshowntobeJ=jD1jPci=1ni+T+1,whereniisthenumberofindividualsingroupiandTisthedimensionofR.Usingprior( 2-8 ),thejointdistributionofYandRgivenandViisp(Y;Rj;V1;;Vc)/jRjPci=1ni+T+1 2exp(1 2cXi=1niXj=1(YijXij)0V1iR1V1i(YijXij)): Denetobe=(011;;0cnc)0,whereij=D(YijXij)givenin( 2-11 ).Throughthetransformation( 2-11 ),wehavep(;j;V1;;Vc)/jRjPci=1ni+T+1 2Jexp(1 2cXi=1niXj=10ijV1i1V1iij)=jRjPci=1ni+T+1 2jD1jPci=1ni+T+1exp(1 2cXi=1niXj=10ijV1i1V1iij)=jjPci=1ni+T+1 2exp(1 2cXi=1niXj=10ijV1i1V1iij):

PAGE 106

Hence,wehave(j;;V1;;Vc)/p(;j;V1;;Vc)/jj1+T+1 2exp1 2S11; 2 2-11 ),and( A1-9 )and( A1-10 )intheproofofProposition 3 ,drawingYandRgivenandVi(i=1;;c)isequivalenttodrawingandrst,andthentranslatingbacktoYandRthroughT1.R=D1D1willbeusedasthecandidatevalue.ItisacceptedintheMetropolisHastingsstepwithprobability.ThecanbederivedsimilarlytothatintheProofofTheorem 1 . 3 1 and 2 . 4 2-17 )isJ=jD1jPci=1ni+p+T+1,whereniisthenumberofindividualsingroupiandpandTaredimensionsofandR,respectively.Usingprior( 2-16 )and( 2-13 ),thejointdistributionofY,,RandPgivenVi(i=1;;c)isp(Y;;R;PjV1;;Vc)/jRjPci=1ni+p+T+1 2exp(1 2cXi=1niXj=1(YijXij)0V1iR1V1i(YijXij))expf1 20v(1R)1vgjRj2T+1 2exp1 2tr(PPR1); 2-17 ),wehavep(;;;PjV1;;Vc)/jRjPci=1ni+p+T+1 2Jexp(1 2cXi=1niXj=10ijV1i1V1iij)exp(1 2pXl=1ll0l)exp1 2PP0

PAGE 107

=jRjPci=1ni+T+1 2jD1jPci=1ni+T+1exp(1 2cXi=1niXj=10ijV1i1V1iij)exp(1 2pXl=1ll0l)exp1 2PP0=jjPci=1ni+T+1 2exp(1 2"cXi=1niXj=10ijV1i1V1iij+pXl=1ll0l+PP0#); 2-17 ).Hence,(j;;P;V1;;Vc)/p(;;;PjV1;;Vc)/jj2+T+1 2exp1 2S21; 5 2-22 )isproportionaltoJ=jD1jn+q+1.ThejointdistributionofWandRgivenothersinSMMVNlinkmodelisp(W;Rj)/jRjn 2nXi=1(WiXi)0((i)R)1(WiXi))jRjq+1 2: 2-22 ),p(;j)/JjRjn+q+1 2exp(1 2nXi=10i((i))1i): 2exp1 2tr(S1); 6 2-28 )isproportionaltoJ=jD1jPci=1ni+p+2T+1,whereniisthenumberofindividualsingroupi,andpand2TaredimensionsofandR,respectively.Usingprior( 2-27 )

PAGE 108

and( 2-13 ),thejointdistributionofU,,RandPgivenEi(i=1;;c)isp(Y;;R;PjE1;;Ec)/jRjPci=1ni+p 2cXi=1niXj=1(UijXij)0E1iR1E1i(UijXij))expf1 20v(1R)1vgjRj2T+1 2exp1 2tr(PP0R1); 2-28 ),wehavep(;;;PjE1;;Ec)/jRjPci=1ni+p+2T+1 2Jexp(1 2cXi=1niXj=10ijE1i1E1iij)exp(1 2pXl=1l0l1l)exp1 2trPP01=jRjPci=1ni+p+2T+1 2jD1jPci=1ni+p+2T+1exp(1 2cXi=1niXj=10ijE1i1E1iij)exp(1 2pXl=1l0l1l)exp1 2trPP01=jjPci=1ni+p+2T+1 2exp(1 2"cXi=1niXj=10ijE1i1E1iij+pXl=1l0l1l+trPP01!#); 2-28 ).Hence,(j;;P;E1;;Ec)/p(;;;PjE1;;Ec)/jj+T+1 2exp1 2S1; 7

PAGE 109

3-14 )isequivalenttorstgeneratingZbyGibbssamplingfrom(z(k)tjz(k)1;;z(k)t1;z(k1)t+1;;z(k1)T)withmeantandvariancet,wheretandtarethet-thelementsofP1and,respectively. 8 3-16 ),wederivetheJacobian.Thefollowingdeterminantsareneeded@(r21;;rT;T1)0 2)TTYj=1(2jj)3 2/jD1j3;@(Y1;;Yn)0

PAGE 111

3-17 ),thejointdistributionofY,R,BandHgivenis 2exp(1 2mXi=1niXj=1(YijXij)0R1(YijXij))exp(1 2mXi=1niXj=1(ZijCijbi)0122(ZijCijbi))mYi=1jHij1 2exp(1 2mXi=1b0iH1ibi); whereY=(Y11;;Ymnm)0,B=(B1;;Bm)andH=(H1;;Hm).Zij,Cij,Hiandbiaredenedthesameasbefore.Throughtheone-to-onemappingT:(Y;R;B;H)!(Y;;B;H),wehave 2JjDj2Texp(1 2mXi=1niXj=1Y0i1Yi)exp(1 2mXi=1niXj=1ZijBiYij0122ZijBiYij)

PAGE 112

2exp(1 2mXi=1b0iH1ibi): 2exp1 2S1; 4 3-16 ),and( A1-11 )and( A1-12 )intheproofofProposition 8 ,drawingY,R,BandHgivenisequivalenttodrawingY,,BandHrst,andthentranslatingbacktoY,R,BandHthroughT.R=D1D1willbeusedasthecandidatevalue.ItisacceptedintheMetropolisHastingsstepwithprobability.Thecanbederivedasfollows.Let1denotethepriororfullconditionaldistributionofRand2denotethecorrespondingcandidatepriororproposaldensity.Then1(RjY;)/1(R)f(Yj;R) and2(RjY;)/2(R)f(Yj;R); 3-5 ),( 3-17 )and( 3-2 ),respectively.Theprobabilityofacceptanceatiterationk+1is=min1;1(RjY;)2(R(k)jY;) 2 2)=min(1;jR(k)j jRj3T+1 2)=min1;exp3T1 2(logjRjlogjR(k)j):

PAGE 113

BeforeweproveTheorem 5 ,werstpresentthreeimportantlemmaswhichwillbeusedinthisproof.ThersttwolemmascomefromMarshallandOlkin(1979)andthethirdoneisfromChenandShao(1999).Forthedetailsontheproofoftheselemmas,wecanrefertothereferences. 5 byusingtheabovelemmasandextendingsomeresultsfromChenandShao(1999)andDaniels(inpress).

PAGE 114

5 2cXiniXj=1ZijX2ij20122ZijX2ij2d2=(2)Pci=1niT 2cXiniXj=1Z0ij122Zijexp1 2cXi=1niXj=102X02ij122X2ij2202X02ij122Zijd2=(2)Pci=1niT 2cXi=1niXj=1Z0ij122Zijexp1 2cXi=1niXj=1X02ij122Zij0cXi=1niXj=1X02ij122X2ij1cXi=1niXj=1X02ij122Zij=(2)Pci=1niT 2cXi=1niXj=1ZijX2ij^20122ZijX2ij^2=(2)Pci=1niT 2trhcXi=1niXj=1ZijX2ij^2ZijX2ij^20122i=(2)Pci=1niT 2trhS122i; where^2=Pci=1Pnij=1X02ij122X2ij1Pci=1Pnij=1X02ij122ZijandS=Pci=1Pnij=1ZijX2ij^2ZijX2ij^20.NotethathereSisthefunctionofi;22;1andB. ByLemma 4 ,trS122TXk=1pk+1(S)k(122)min(S)TXk=1k(122)TXk=1k(122);

PAGE 115

where=min122;2;Bmin(S).From(T1)and(T2),wehavePci=1ni>TandSispositivedenitewith0
PAGE 116

By( 3-21 )and( A1-15 ),M1ZhZ[Y2YQ]f(Yj1;Ri;11)d1dR11dYi(Bj)(jp)(p)dBddw=M1ZZ[Y2YQ]f(Yj1;Ri;11)d1dR11dYZ(Bj)(jp)(p)dBddw: LetYij=(Yij1;;YijT)beindependentrandomvariablessuchthatYijjRi;11N(0;Ri;11); thatis,givenRi;11,Yijisnormallydistributedwithmean0andvarianceRi;11.LetS=Qci=1Qnij=1Sijdenotetheset[Y2YQ],andputSij=Sij1Sij2SijT; whereL(1;Ri;11jY)isthelikelihoodfunctionassociatedwithmodel(1.a).Wecanrewriteitas

PAGE 117

=EfI[Yijt+x1ijt12Sijt;1tT;1iPci=1ni]g: [Yijt+xij1t12Sijt;1tT;1icXi=1ni] (A1-18) =\1tT[Yijt+x1ijt1<0;Yijt=0;1icXi=1ni][[Yijt+x1ijt10;Yijt=1;1icXi=1ni][hijtx1ijt1hijtYijt;1tT;1icXi=1ni]=[X1ij1Y]; Combining A1-16 , A1-17 and A1-18 ,wehaveI=ZEfI[X1ij1Y]gd1dR11: 5

PAGE 118

2.5.1 (1)PXRPstage Step1XPminingFindtheexpansionparametersnecessarytochangeRintointheexpandedmodelM. 3 .CalculateitsJacobianvalue. 2-13 )forRandthecandidatepriorsforotherparameterswhichdependonthecandidatetransformationinstep2. 2-15 ). (2)PXMHstageAtiterationk+1 108

PAGE 119

2.8 24XiniXj=1(Yij;mis(c)ij;mis)0(c)1i;mis(Yij;mis(c)ij;mis)); where(c)ij;misand(c)i;misaretheconditionalmeanandcovarianceofYij;misgivenYij;obsforsubjectjingroupi.Theycanbeexpressedas(c)ij;mis=ij;mis+i121i22(Yij;obsij;obs)and(c)i;mis=i11i121i220i12: A2-1 ),weobtaincompletedataY,Y=(Y0mis;Y0obs)0.Thefullconditionaldistributionfor,jY;Vi;Risnormallydistributedas(jY;Vi;R)/exp1 2()01();

PAGE 120

whereait=it+1 2CttPnij=12ijtandbit=PTl=t+1Ctl1il(Pnij=1ijtijl).HereCtlisthe(t;l)thelementofR1.ThecomponentsofVicanbesampledonebyonefrom( A2-2 ),usingtheMetropolis-Hastings(M-H)algorithm.TheproposaldensityintheM-Halgorithmcanbederivedasfollows.Equation( A2-2 )canberearrangedas(1 2itexpait 22ritit: 2itoisa22nidensitywith2nidegreesoffreedomandhenceforlargeni,itapproachesanormaldistribution;expnait 2oisanormal;2ritittendstobe1sincewespecifythehyper-parameterritassmall.Hence( A2-2 )isapproximatelyanormaldistributionwithmean^hitandvariance^hit.Next,wederivethisapproximatenormalastheproposaldensityofit. Lethit=1 4a(b+p 2f00(^hit)(hit^hit)+:

PAGE 121

Inorderfor( A2-3 )totthelogofthenormaldensitywhichisjustaquadratic,namely1 2^2hit(hit^hit),set^2hit=1 2^2hit(hit^hit)2: Wegeneratetheproposalvalueofhitfrom( A2-4 )andacceptitaccordingtotheprobabilitywhichisfoundbysubstituting( A2-2 )and( A2-4 )into( 1-8 )in Chapter 1 .

PAGE 122

3.4.1 whereandVaremeanandvariance,respectively,fromanormalpriorfor.ThecorrelationmatrixRi;11hasthefullconditionaldistribution 2expn1 2trhmXi=1niXj=1YijX1ij1YijX1ij10R1i;11ioIfrtl:jrtlj1andRi;11ispos.def.g; wherertl(t;l=1;;T)istheelementofthetthrowandlthcolumninRi;11.ThisisaconstrainedinverseWishartwithdegreesoffreedomPmi=1niT1andscalematrixPmi=1Pnij=1YijX1ij1YijX1ij10.HowtosampleRi;11fromthisdistributionwasaddressedinSection 3.4.2 .Tosamplebi(i=1;;m),werstconsiderthecasethattheratioof1i;tland0i;tlisxedand0i;tlistreatedas 112

PAGE 123

atuningconstant.Thefullconditionaldistributionofbigivenothersis whereZij=WijX2ij2andCijistheTT2matrixwhoseelementsarefunctionsofYijand1.ThetthrowofCijisC0ij;t=(00ij;1;;00ij;(t1);(YijX1ij1)0t;00ij;(t+1);;00ij;T).TheothernotationwasdenedasinSection 3.3.1 .Tosampletheindicatorvectori,itiseasytoshowthatthedistributionofeachcomponenti;tlofiisBernoulliwithprobability(i;tl=1jbi;pi;i;22;(i;tl))=ai1 where(i;tl)denotesavectorconsistingofallelementsofiexcepti;tl,ai0=(biji;22;(i;tl);i;tl=0)(i;22j(i;tl);i;tl=0)((i;tl);i;tl=0jpi) andai1=(biji;22;(i;tl);i;tl=1)(i;22j(i;tl);i;tl=1)((i;tl);i;tl=1jpi): 3-8 )oni,andwhenthepriorparametersfori;22donotdependoni,( A3-4 )canbeobtainedmoresimplybyai0=(biji;22;(i;jk);i;jk=0)((i;jk);i;jk=0jpi) andai1=(biji;22;(i;jk);i;jk=1)((i;jk);i;jk=1jpi): 3-10 ),thedependenceon(i;jk)throughout( A3-4 )maybeeliminated.Then(i;jk=1jbi;pi;i;22;(i;jk))=(i;jk=1jbi;pi;i;22),

PAGE 124

( A3-4 )canbefurthersimpliedasai0=(biji;22;i;jk=0)(i;jk=0jpi) andai1=(biji;22;i;jk=1)(i;jk=1jpi): 3-11 ),wehavebijZi;Ci;0i;i;i;22NniXj=1C0ij1i;22Cij+H1i(0)1niXj=1Cij1i;22Zij+H1i(0)bi0;niXj=1C0ij1i;22Cij+H1i(0)1!: 2jTiRTij1 2exp1 2(bibi0)0(TiRiTi)1(bibi0)(20i)(ri+1)expi 2+1)exp1 2(bibi0)0(TiRiTi)1(bibi0)+i; 2+1)exp8<:1 2X(Ai)d2(Ai)+1 2c2iX(Bi)d2(Bi)+i359=;;

PAGE 125

2P(Ai)d2(Ai)+1 2c2iP(Bi)d2(Bi)+i.Fori,wehave(ijZi;Ci;bi;Pi;i;22)=(ijbi;Pi;i;22)/jHi(0)j1 2exp1 2bibi00H1i(0)bibi0rii 2trSi1i;22i: and(i;jk=1;bi;0i;(i;jk);Pi;i;22)=(bij0i;i;jk=1;(i;jk);Pi;i;22)(0iji;jk=1;(i;jk);Pi;i;22)(i;22ji;jk=1;(i;jk);Pi)(i;jk=1j(i;jk);Pi)(Pi)=(bij0i;i;jk=1;(i;jk))(0iji;jk=1;(i;jk))(i;22ji;jk=1;(i;jk))(i;jk=1j(i;jk);Pi)(Pi): 1+ai0

PAGE 126

whereai0=(bij0i;i;jk=0;(i;jk))(0iji;jk=0;(i;jk))(i;22ji;jk=0;(i;jk))(i;jk=0jPi) andai1=(bij0i;i;jk=1;(i;jk))(0iji;jk=1;(i;jk))(i;22ji;jk=1;(i;jk))(i;jk=1jPi): 3-8 )oni,andwhenthepriorparametersfori;22donotdependoni,( A3-5 )canbeobtainedmoresimplybyai0=(bij0i;i;jk=0;(i;jk))(0iji;jk=0;(i;jk))(i;jk=0jPi) andai1=(bij0i;i;jk=1;(i;jk))(0iji;jk=1;(i;jk))(i;jk=1jPi): A3-5 )canbefurthersimpliedasai0=(bij0i;i;jk=0;(i;jk))(i;jk=0jPi) andai1=(bij0i;i;jk=1;(i;jk))(i;jk=1jPi): 3.3.2 ),thedependenceon(i;jk)throughout( A3-5 )maybeeliminated.Inthiscase,ai0=(bi;jkj0i;i;jk=0)(i;jk=0jPi) andai1=(bi;jkj0i;i;jk=1)(i;jk=1jPi):

PAGE 127

Inthenalcase,wehave(i;jk=1jbi;0i;(i;jk);Pi;i;22)=(i;jk=1jbi;jk;0i;Pi)=1 1+ciexpnb2i;jk(c2i1) 220ic2io(P(jjj0j+1)1 Finally,i;22issampledfrom 22jZi;Ci;bi;iIW(;S1) (A3-7) with=mXi=1ni+andS=hmXi=1niXj=1ZijCijbiZijCijbi0+Si; 3.5.1 3-28 ). (1)(jY;W;Ri;11;Bi;i;Pi;22):Givencompletedata(Y;W),weevaluatethenormaldensity( A3-1 )with=andi=i=0B@Ri;11Ri;11BiBiRi;1122+BiRi;11Bi1CA.ThisevaluationdoesnotdependoniandPi.

PAGE 128

(2)(BijY;W;i;Pi;22):Thisconditionalordinateisestimatedas(BijY;W;i;Pi;22)=Z(BijY;W;i;Pi;22;;Ri;11)(jY;W;i;Pi;22;Ri;11)(Ri;11jY;W;i;Pi;22)ddRi;11; A3-3 )withBi=Bi,i=iandPi=Pi.AkeypointisthatthisintegralcanbeestimatedaccuratelybydrawingalargesampleofandRi;11fromthejointdensity(;Ri;11jY;W;i;Pi;22).These(;Ri;11)variatesareproducedfromareducedMarkovchainMonteCarlorunconsistingof[jY;W;Bi;i;Pi;22;Ri;11];[Ri;11jY;W;Bi;i;Pi;22;];[BijY;W;i;Pi;22;;Ri;11]: GivenasampleofG1valueson(;Ri;11)fromtheaboverun,anestimateof(BijY;W;i;Pi;22)isavailableas^(BijY;W;i;Pi;22)=1 A3-4 )withi=iandPi=Pi,and((g2);R(g2)i;11;B(g2)i)isasampleofsizeG2fromthefollowingreducedMCMCrun,conditioningon(Y;W;i;Pi;22)[jY;W;i;Pi;22;Ri;11;Bi];[Ri;11jY;W;i;Pi;22;;Bi];[BijY;W;i;Pi;22;;Ri;11];[ijY;W;Pi;22;;Ri;11;Bi]:

PAGE 129

(4)(22jY;W):BasedonasampleofsizeG3producedfromthecorrespondingreducedMCMCoutputsimilarto( A3-8 )and( A3-9 ),(22jY;W)isestimatedas^(22jY;W)=1 A3-7 )withi;22=22and(g3);R(g3)i;11;B(g3)i;(g3)i;P(g3)iistheoutputatiterationgfromthereducedMCMCrun. Thecommonthemeintheestimationof(BijY;W;i;Pi;22),(ijY;W;Pi;22)and(22jY;W)isthatthefullconditionaldensities,includingtheirnormalizingconstants,arefullyknown.TheyareestimatedthroughRao-BlackwellizationandthereducedMCMCruns.Inthecasethatfullconditionaldistributionsarenotofstandardformandhaveintractablenormalizingconstant,suchas( A3-2 )and( A3-6 )inwhichposteriorsamplingisconductedviatheMetropolis-HastingsAlgorithm,theabovemethoddoesnotwork.Wenowaddressthisproblemrelatedtocomputing(PijY;W;22)fromtheMetropolis-Hastingsoutputs. (5)(PijY;W;22):Thefullconditionaldensityin( A3-6 )issampledbytheM-Halgorithm.Letq1(Pi;P0ijY;W;;Ri;11;Bi;i;i22)betheproposaldensityforthetransitionfromPitoP0i.ThenthemoveprobabilityintheM-Halgorithmis1(Pi;P0ijY;W;;Ri;11;Bi;i;i22)=min(1;f(Y;WjP0i;;Ri;11;Bi;i;i22)(P0i;;Ri;11;Bi;i;i22)

PAGE 130

ByexploitingthelocalreversibilityoftheM-HstepforPiandanalogousargumentspresentedinChibandJeliazkov(2001),weobtain(PijY;W;22)=E1f1(Pi;PijY;W;;Ri;11;Bi;i;i22)g 3-31 ).Samplingfrom( 3-31 )isnotstraightforward.WeproposingusingMetropolisHastingsalgorithmstoattackit.Notethateachtermontherightsideof( 3-31 )canbeeasilyderivedthroughthejointdistributionh(D;RjY;W;),whichisobtainedbytransformingtheparameterexpandedcandidatedensity( 3-18 )usingone-to-onemapping:=DRi;11D!(D;Ri;11).TheJacobianofthistransformationisQTt=1(D2tt)T1 2,whereDttisthe(t;t)thelementofD. From( 3-18 ),(D;RjY;W;)canbederivedas (A3-10)

PAGE 131

=c10TYt=1(D2tt)T1 2(D2tt)+T+1 2jRj+T+1 2etr1 2SD1R1D1=c10jRj+T+1 2jD2j+2 2etr1 2SD1R1D1; 4TYk=11 2(+1k): A3-10 ),thedistribution(C;RjY;)canbederivedas 2jCjetr1 2SCR1C: Bysomesimplecalculus,itiseasytoshowthat( 3-30 )isequivalentto ( 3-31 )becomes From( A3-13 ),weseethatwecandrawasamplefromh(CjY;W)bycyclingdrawingeachcomponentofCfrom(Ctt;RjY;;C(tt)).TheMetropolis-Hastingsalgorithmprovidesagoodwaytodrawasamplefrom(Ctt;RjY;;C(tt))(t=1;;T)oncewendagoodproposaldensitywhichapproximatesit.Wewillshowbelowthath(CjY;W)in( A3-13 )transformedfromh(DjY;W)in( 3-31 )makethisapproximationbecomeeasier.( A3-12 )canbeestimatedas^q2(Ri;11jY;)=KXk=1(C(k);RjY;)

PAGE 132

extendingtheideainWongetal:,from( A3-11 ),wederive wherea=1 2Sttandb=1 2(PTk=1StkrtkCkkSttrttCtt).HereStkandrtkarethe(t;k)thelementsofSandR1in( A3-11 ),respectively.Equation( A3-14 )canberewrittenas 2Ctt)exp(aC2tt2bCtt+1 2Ctt): Uptoanormalizingconstant,C(n4T+1)1ttexp(1 2Ctt)isclosetoa2n4T+1densitywithn4T+1degreesoffreedomandhenceforlargen,ittendstobeanormaldistribution.Thedensityexp(aC2tt2bCtt+1 2Ctt)isalsonormal.Hence,(Ctt;RjY;;C(tt))isapproximatelyanormaldistributionsinceitisaproductofnormals.UsingtheargumentsinSection 2.8 ofChapter 2 ,wederivetheM-Halgorithmproposaldensityas 2^2C(Ctt^Ctt)2; where^Ctt=1 2ab+p A3-14 )andapproximatedensity( A3-16 )into( 1-8 ).

PAGE 133

Albert,J.H.andChib,S.(1993).Bayesiananalysisofbianryandpolychotomousresponsedata.JournaloftheAmericanStatisticalAssociation88,669-679. Ashford,J.R.andSowden,R.R.(1970).Multivariateprobitanalysis.Biometrics26,535-546. Baker,D.andTaylor,H.(1997).InequalityinhealthandhealthserviceuseformothersofyoungchildreninSouthWestEnglandSurveyteamoftheAvonLongitudinalStudyofPregnancyandChildrenTeam.JournalofEpidemiologyandCommunityHealth51,74-79. Barnard,J.,McCulloch,R.andMeng,X.L.(2000).Modelingcovariancematricesintermsofstandarddeviationsandcorrelationswithapplicationtoshrinkage.StatisticaSinica10,1281-1311. Berry,J.A.MandLinoG.S.(2000).MasteringDataMining:TheArtandScienceofCustomerRelationshipManagement.NewYork:JohnWileyandSons. Besag,J.andGreen,P.J.(1993).SpatialstatisticsandBayesiancomputation.JournaloftheRoyalStatisticalSociety,SeriesB55,25-37. Boik,R.J.(2002).Spectralmodelsforcovariancematrices.Biometrika89,159-182. Carlin,B.PandChib,S.(1995).BayesianmodelchoiceviaMarkovchainMonteCarlomethods.JournaloftheRoyalStatisticalSociety,SerialB57,473-484. Catalano,P.J.andRyan,L.M.(1992).Bivariatelatentvariablemodelsforclustereddiscreteandcontinuousoutcomes.JournaloftheAmericanSta-tisticalAssociation87,651-658. Chen,M.-H.(2005).ComputingmarginallikelihoodsfromasingleMCMCoutput.StatisticaNeerlandica59,16-29. Chen,M.-H.andDey,D.K.(1998).Bayesianmodellingofcorrelatedbinaryresponsesviascalemixtureofmultivariatenormallinkfunctions.Sankhya,SerialA60,322-343. Chen,M.-H.andDey,D.K.(2000).Auniedbayesiananalysisforcorrelatedordinaldatamodels.BrazilianJournalofProbabilityandStatistics,14,87-111. Chen,M.-H.andShao,Q.-M.(1999).Propertiesofpriorandposteriordistributionsformultivariatecategoricalresponsedatamodels.JournalofMultivariateAnalysis71,277-296. 123

PAGE 134

Chib,S.(1995).MarginallikelihoodfromtheGibbsoutput.JournaloftheAmericanStatisticalAssociation90,1313-1321. Chib,S.andGreenberg,E.(1998).Bayesiananalysisofmultivariateprobitmodels.Biometrika85,347-361. Chib,S.andJeliazkov,I.(2001).MarginallikelihoodfromtheMetropolis-Hastingsoutput.JournaloftheAmericanStatisticalAssociation96,270-281. Chipman,H.(1996).Bayesianvariableselectionwithrelatedpredictors.CanadianJournalofStatistics24,17-36. Cox,D.R.andWermuth,N.(1992).Responsemodelsformixedbinaryandquantitativevariables.Biometrika79,441-461. Damien,P.,Wakeeld,J.andWalker,S.(1999).GibbssamplingforBayesiannon-conjugateandhierarchicalmodelsusingauxiliaryvariables.JournaloftheRoyalStatisticalSociety,SeriesB61,331-344. Daniels,M.J.(inpress).Bayesianmodellingofseveralcovariancematricesandsomeresultsonproprietyoftheposteriorforlinearregressionwithcorrelatedand/orheterogeneouserrors.JournalofMultivariateAnalysis Daniels,M.J.andKass,R.E.(2001).Shrinkageestimatorsforcovariancematrices.Biometrics57,1173-1184. Dempster,A.P.,Laird,N.M.andRubin,D.B.(1977).MaximumlikelihoodestimationfromincompletedataviatheEMalgorithm(withdiscussion).JournaloftheRoyalStatisticalSociety,SeriesB39,1-38. Dunson,D.B.(2000).Bayesianlatentvariablemodelsforclusteredmixedoutcomes.JournaloftheRoyalStatisticalSociety,SeriesB62,355-366. Eaton,W.W.,McCutcheon,A.,Dryman,A.andSorenson,A.(1989).Latentclassanalysisofanxietyanddepression.SociologicalMethodsandResearch18,104-125. Eaves,L.J.,Silberg,J.L.,Hewitt,J.K.,Rutter,M.,Meyer,J.M.,Neale,M.C.andPickles,A.(1993).Analyzingtwinresemblanceinmulti-symptomdata:geneticapplicationsofalatentclassmodelforsymptomsofconductdisorderinjuvenileboys.BehaviroalGenetics23,5-19. Fitzmaurice,G.M.andLaird,N.M.(1995).Regressionmodelsforabivariatediscreteandcontinuousoutcomewithclustering.JournaloftheAmericanStatisticalAssociation90,845-852.

PAGE 135

Flury,B.N.(1984).Commonprincipalcomponentsinkgroups.JournaloftheAmericanStatisticalAssociation79,892-898. Flury,B.N.(1986).Asymptotictheoryforcommonprincipalcomponentanalysis.AnnalsofStatistics14,418-430. Flury,B.N.(1988).CommonPrincipalComponentsandRelatedMultivariateModels.NewYork:JohnWileyandSons. Garrett,E.S.andZeger,S.L.(2000).Latentclassmodeldignosis.Biometrics56,1055-1067. Gelfand,A.E.andSmith,A.F.M.(1990).Sampling-basedapproachestocalculatingmarginaldensities.JournaloftheAmericanStatisticalAssociation85,398-409. Geman,S.andGeman,D.(1984).Stochasticrelaxation,GibbsdistributionsandtheBayesianrestorationofimages.IEEETransactionsonPatternAnalysisandMachineIntelligence6,721-741. George,E.I.andMcCulloch,R.E.(1993).VariableselectionviaGibbssampling.JournaloftheAmericanStatisticalAssociation88,881-889. George,E.I.andMcCulloch,R.E.(1997).ApproachesforBayesianvariableselection.StatisticaSinica7,339-373 Geweke,J.(1991).Ecientsimulationfromthemultivariatenormalandstudentt-distributionssubjecttolinearconstraints.ComputerSciencesandStatistics,Proceedingsofthe23rdSymposiumInterface,Seattle,p50-64. Gilks,W.,Richardson,S.andSpiegelhalter,D.(1996).MarkovchainMonteCarloinpractice.NewYork:ChapmanandHall. Gueorguieva,R.V.andAgresti,A.(2001).Acorrelatedprobitmodelforjointmodellingofclusteredbinaryandcontinuousresponses.JournaloftheAmericanStatisticalAssociation96,1102-1112. Gustafson,P.,MacNab,Y.C.andWen,S.(2004).ThevalueofderivativesandrandomwalksuppressioninMarkovchainMonteCarloalgorithms.StatisticsandComputing14,23-37. Hastings,W.K.(1970).MonteCarlosamplingmethodsusingMarkovchainsandtheirapplications.Biometrika57,97-109. Heagerty,P.J.(2002).Marginalizedtransitionmodelsandlikelihoodinferenceforlongitudinalcategoricaldata.Biometrics58,342-351. Higdon,D.M.(1998).AuxiliaryvariablemethodsforMarkovchainMonteCarlowithapplications.JournaloftheAmericanStatisticalAssociation93,585-595.

PAGE 136

Hoeting,J.,Raftery,A.E.andMadigan,D.(1996).Amethodforsimultaneousvariableselectionandoutlieridenticationinlinearregression.ComputationalStatisticsandDataAnalysis22,251-270. Ising,E.(1925).Beitragzurtheoriedesferromagnetismus.ZeitschriftfurPhysik31,253-258. Jereys,H.(1935).Sometestsofsignicance,treatedbythetheoryofprobability.ProceedingsoftheCambridgePhilosophySociety31,201-222. Kass,R.E.andRaftery,A.E.(1995).Bayesfactors.JournaloftheAmericanGeriatricsSociety90,773-795. Kendler,K.S.,Karkowski,L.M.,Prescott,C.A.andPedersen,N.L.(1998).LatentclassanalysisoftemperanceboardregistrationsinSwedishmale-maletwinpairsborn1902to1949:searchingforsubtypeofalcoholism.PsychologicalMedicine28,803-813. Kendler,K.S.,Karkowski,L.M.andWalsh,D.(1998).Thestructureofpsychosis:latentclassanalysisofprobandsfromtheRoscommonFamilyStudy.ArchivesofGeneralPsychiatry55,492-499. Kessler,R.C.,Stein,M.B.andBerglund,P.(1998).SocialphobiasubtypesintheNationalComorbiditySurvey.AmericanJournalofPsychiatry155,613-619. Liu,C.(2001).Discussionon\TheArtofDataAugmentation"byvanDykandMeng.JournalofComputationalandGraphicalStatistics10,75-81. Liu,C.(2003).Alternatingsubspace-spanningresamplingtoaccelerateMarkovchainMonteCarlosimulation.JournaloftheAmericanGeriatricsSociety98,110-117. Liu,C.,Rubin,D.B.andWu,Y.N.(1998).ParameterexpansiontoaccelerateEM:thePX-EMalgorithm.Biometrika85,755-770. Liu,C.andSun,D.X.(2000).Analysisofintervalcensoreddatafromfractionatedexperimentsusingcovarianceadjustments.Technometrics42,353-365. Liu,J.S.andWu,Y.(1999).Parameterexpansionfordataaugmentation.JournaloftheAmericanStatisticalAssociation94,1264-1274. Liu,X.F.andDaniels,M.J.(2005).JointModelsfortheassociationoflongitudinalbinaryandcontinuousprocesses(manuscript).DepartmentofStatistics,UniversityofFlorida,Gainesville. Liu,X.F.andDaniels,M.J.(2006).ANewAlgorithmforSimulatingaCorrelationMatrixBasedonParameterExpansionandRe-parameterization.ToappearinJournalofComputationalandGraphicalStatistics. Manly,B.F.J.andRayner,J.C.W.(1987).Thecomparisonofsamplecovariancematricesusinglikelihoodratiotests.Biometrika74,841-847.

PAGE 137

Marshall,A.W.andOlkin,I.(1979).Inequalities-TheoryofMajorizationandItsApplications.AcademicPress. Melton,B.,Liang,K.Y.andPulver,A.E.(1994).Extendedlatentclassapproachofthestudyoffamilial/sporadicformsofdisease:itsapplicationofthestudyofheterogeneityofschizophrenia.GeneticEpidemiology11,311-327. Meng,X.L.andvanDyk,D.A.(1997).TheEMalgorithm{anoldfolksongsungtoafastnewtune(withdiscussion).JournaloftheRoyalStatisticalSociety,SerialB59,511-567. Meng,X.L.andvanDyk,D.A.(1999).Seekingecientdataaugmentationschemesviaconditionalandmarginalaugmentation.Biometrika86,301-320. Metropolis,N.,Rosenbluth,A.W.,Rsenbluth,M.N.,Teller,A.H.andTeller,E.(1953).Equationsofstatecalculationsbyfastcomputingmachines.JournalofChemicalPhysics21,1087-1092. Nandram,B.andChenM.-H.(1994).AcceleratingGibbssamplerconvergenceinthegeneralizedlinearmodelsviaareparameterization.JournalofStatisticalComputationandSimulation81,27-40. Neal,R.M.(1997).MarkovchainMonteCarlomethodsbasedonslicingthedensityfunction.TechnicalreportNo.9722.DepartmentofStatistics,UniversityofToronto. Nestadt,G.,Hanfelt,J.,Liang,K.Y.,Lamacz,M.,Wolyniec,A.andPulver,A.E.(1994).Anevaluationofthestructureofschizophreniaspectrumpersonalitydisorders.JournalofPersonalityDisorders8,288-298. Nobile,A.(1998).AhybridMarkovchainfortheBayesiananalysisofthemultinomialprobitmodel.StatisticsandComputing8,229-242. O'Brian,S.M.andDuson,D.B.(2004)Bayesianmultivariatelogisticregression.Biometrics60,739-746. Potts,R.B.(1952).Somegeneralizedorder-disordertransformations.ProceedingsoftheCambridgePhilosophicSociety48,106-109. Pourahmadi,M.andDaniels,M.J.(2002).Dynamicconditionallylinearmixedmodelsforlongitudinaldata.Biometrics58,225-231. Pourahmadi,M.,Daniels,M.J.andPark,T.(inpress).SimultaneouslymodellingoftheCholeskydecompositionofseveralcovariancematrices.JournalofMultivariateAnalysis. Regan,M.M.andCatalano,P.J.(1999).Likelihoodmodelsforclusteredbinaryandcontinuousoutcomes:applicationtodevelopmentaltoxicology.Biometrics55,760-768.

PAGE 138

Ritter,C.andTanner,M.A.(1992).FacilitatingtheGibbsSampler:theGibbsstopperandtheGriddy-Gibbssampler.JournaloftheAmericanStatisticalAssociation87,861-868. Robert,C.P.(1995).Simulationoftruncatednormalvariables.StatisticsandComputing5,121-125. Roberts,G.O.(1996).Markovchainconceptsrelatedtosamplingalgorithm.InMarkovChainMonteCarloinPractice.eds.W.R.Gilks,S.RichardsonandD.J.Spiegelhalter.London:ChapmanandHall,p45-58. Roberts,G.O.,Gelman,A.,andGilks,W.R.(1997).WeakconvergenceandoptimalscalingofrandomwalkMetropolisHastingsalgorithms.TheAnnalsofAppliedProbability7,110-120. Roberts,G.O.andRosenthal,J.S.(1997).ConvergenceofslicesamplerMarkovchains.Technicalreport.CambridgeUniversity,StatisticalLaboratory. Roberts,G.O.andRosenthal,J.S.(2001).OptimalscalingforvariousMetropolis-Hastingsalgorithms.StatisticalScience16,351-367. Roberts,G.O.andSmith,A.F.M.(1994).SomesimpleconditionsfortheconvergenceoftheGibbsamplerandMetropolis-Hastingsalgorithms.Stochas-ticprocessesanditsApplications49,207-216. Roberts,G.O.andTweedie,R.L.(1996).GeometricconvergenceandcentrallimittheoremsformultidimensionalHastingsandMetropolisalgorithms.Biometrika83,95-110. Roche,K.B.,Miglioretti,D.L.,Zeger,S.L.andRathouz,P.J.(1997).Latentvariableregressionformultiplediscreteoutcomes.JournaloftheAmericanStatisticalAssociation92,1375-1386. Roy,J.andLin,X.(2000).Latentvariablemodelsforlongitudinaldatawithmultiplecontinuousoutcomes.Biometrics56,1047-1054. Sammel,M.D.,Ryan,L.M.andLegler,J.M.(1997).Latentvariablemodelsformixeddiscreteandcontimuousoutcomes.JournaloftheRoyalStatisticalSociety,SeriesB59,667-678. Shaw,G.M.andLammer,E.J.(1999).Maternalpericonceptionalalcoholconsumptionandriskfororofacialclefts.JournalofPediatrics134,298-303. Smith,A.andRoberts,G.(1993).BayesiancomputationviatheGibbssamplerandrelatedMarkovchainMonteCarlomethods(withdiscussion).JournaloftheRoyalStatisticalSociety,SeriesB55,3-24. Smith,M.andKohn,R.(2002).Parsimoniouscovariancematrixestimationforlongitudinaldata.JournaloftheAmericanStatisticalAssociation97,1141-1153.

PAGE 139

Spiegelhalter,D.J.,Best,N.G.,Carlin,B.P.andvanderLinde,A.(2002).Bayesianmeasuresofmodelcomplexityandt(withdiscussion).JournaloftheRoyalStatisticalSociety,SeriesB64,583-639. SwendsenR.H.andWang,J.S.(1987).non-universalcriticaldynamicsinMonteCarlosimulation.physicalReviewLetters58,86-88. Szatmari,P.,Volkmar,F.andWalter,S.(1995).Evaluationofdiagnosticcriteriaforautismusinglatentclassmodels.JournaloftheAcademyofChildandAdolescentPsychiatry34,216-222. Tanner,M.A.andWong,W.H.(1987).Thecalculationofposteriordistributionbydataaugmentation(withdiscussion).JournaloftheAmericanStatisticalAssociation82,528-550. Tierney,L.(1994).Markovchainsforexploringposteriordistributions(withdiscussion).AnnalsofStatistics22,1701-1786. Tierney,L.(1994).Introductiontogeneralstate-spaceMarkovchaintheory.InMarkovChainMonteCarloinPractice.eds.W.R.Gilks,S.RichardsonandD.J.Spiegelhalter.London:ChapmanandHall,p59-74. vanDyk,D.A.andMeng,X.L.(2001).Theartofdataaugmentation(withdiscussion).JournalofComputationalandGraphicalStatistics10,1-111. vanPutten,W.L.,deVries,W.,Reinders,P.,Levering,W.,vanderLinden,R.,Tanke,H.J.,Bolhuis,R.L.andGratama,J.W.(1993).Quanticationofuorescencepropertiesoflymphocytesinperipheralbloodmononuclearcellsuspensionsusingalatentclassmodel.Cytometry14,86-96. Wakeeld,J.C.andBennett,J.E.(1996).TheBayesianmodellingofcovariatesforpopulationpharmacokineticmodels.JournaloftheAmericanStatisticalAssociation91,917-927. Wong,F.,Carter,C.K.andKohn,R.(2003).Ecientestimationofcovarianceselectionmodels.Biometrika90,809-830. Zhang,X.,Boscardin,W.J.andBelin,T.R.(2004).SamplingcorrelationmatricesinBayesianmodelswithcorrelatedlatentvariables(manuscript).DepartmentofBiostatistics,UniversityofCalifornia,LosAngels.

PAGE 140

XuefengLiureceivedtheMasterofScienceinquantitativegenetics(classicalstatisticalgenetics)in1993fromYangzhouUniversityinChina.HeworkedasanassistantprofessorforoverveyearsinLaiyangAgriculturalUniversity.In1999,hewasawardedtheNationalScienceScholarshipfromChinaScholarshipCouncilandvisitedtheDepartmentofStatisticsattheUniversityofCalifornia,Davis.In2000,hetransferredasagraduatestudenttotheDepartmentofStatisticsattheUniversityofFlorida.HeearnedhisMasterofStatisticsfromtheUniversityofFloridainWinter,2002.BeingadmittedtothePh.D.program,Liualsoworkedthreeyearsasapart-timegraduatestudentintheMaternalChildrenHealthandEducationResearchandDataCenter,CollegeofMedicine,attheUniversityofFlorida.HegraduatedinAugust,2006. 130