Citation
Bayesian Estimation and Model Selection for Single and Multiple Graphical Models

Material Information

Title:
Bayesian Estimation and Model Selection for Single and Multiple Graphical Models
Creator:
Jalali, Peyman
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (87 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Statistics
Committee Chair:
Michailidis,George
Committee Co-Chair:
Khare,Kshitij
Committee Members:
Ghosh,Malay
Huo,Zhiguang
Graduation Date:
5/3/2019

Subjects

Subjects / Keywords:
bayesian-graphical-models -- joint-estimation-of-graphical-models -- metabolomics-data
Statistics -- Dissertations, Academic -- UF
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Statistics thesis, Ph.D.

Notes

Abstract:
Undirected Graphical Models represent a family of canonical statistical models for reconstructing interactions amongst a set of entities from multi-dimensional data profiles. They have numerous applications in biology involving Omics and neuroimaging data, in social sciences for voting records and econ/financial data, in text mining, etc. Recently, the problem of joint estimation of multiple graphical models from high dimensional data has also received much attention in the statistics and machine learning literature, due to its importance in diverse fields including molecular biology, neuroscience and the social sciences. In the first part of this dissertation, we will develop two Bayesian methodologies, using spike and slab and continuous shrinkage priors coupled with a pseudo-likelihood that enables fast computations, for estimating a single high-dimensional graphical model. We will introduce efficient Gibbs samplers and illustrate the efficiency of the models by comparing with the state of the art models. The second part develops a Bayesian approach that decomposes the model parameters across multiple underlying graphical models into shared components across subsets of models and edges, and idiosyncratic ones. Further, it leverages a novel multivariate prior distribution, coupled with the same pseudo-likelihood as above through a robust and efficient Gibbs sampling scheme. We establish strong posterior consistency for model selection, as well as estimation of model parameters under high dimensional scaling with the number of variables growing exponentially with the sample size. The efficacy of the proposed approach is illustrated on both synthetic and real data. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ph.D.)--University of Florida, 2019.
Local:
Adviser: Michailidis,George.
Local:
Co-adviser: Khare,Kshitij.
Statement of Responsibility:
by Peyman Jalali.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
LD1780 2019 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

BAYESIANESTIMATIONANDMODELSELECTIONFORSINGLEANDMULTIPLEGRAPHICALMODELSByPEYMANJALALIADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2019

PAGE 2

c2019PeymanJalali

PAGE 3

Idedicatethistomyparentsandmythreebrothers.

PAGE 4

ACKNOWLEDGMENTSIwouldliketoexpressmydeepestgratitudetomyadvisers,Dr.GeorgeMichailidisandDr.KshitijKharefortheirpatience,motivationandimmenseknowledgethroughoutmyPhDstudies.Workingundertheirjointsupervisionhasbeenanexceptionalopportunityformetogrowtoabetterperson,personallyandprofessionally.Thisdissertationwouldhavebeenimpossiblewithouttheiramazingmentorship.Dr.Michailidishasbeenalwaysencouragingmetoworkoninterestingandchallengingproblemsandalwayshelpedmebyprovidingamazinginsightandaskinghardquestion.Inadditiontobeinganoutstandingresearcher,heisanamazingleaderandittrulyhasbeenanhonortoworkunderhissupervision.Dr.Khare'ssupportthroughoutmyresearchhasbeenessentialfromthebeginningtoend.HiscriticalwayofthinkingandoutstandingabilityinsolvingdicultproblemswasakeyelementinmysuccessduringmyPhDresearch.Hehasalwaysbeenapproachableandeasilyaccessible.AttimesthatIwasstuckatdicultproblems,heextendedhissupportandtaughtmeingeniouswaysoftacklingchallengingproblems.Iamforeverindebtedtohimforhisamazingmentorship.IalsowouldliketothankmyPhDcommitteemembersDr.MalayGhoshandDr.CalebHuo,fortheirinsightfulcommentsandquestions.IamespeciallythankfultoDr.Ghosh,whoalsowasmyteacherforthreePhDlevelcourses;sittinginhisclassesasastudentwasoneofmygreatesthonorsduringmyPhDstudies.IamgratefulforthehelpsandsupportIhavereceivedfromthefacultyandstaatthedepartmentofstatisticsandinformaticsinstituteattheuniversityofFlorida.IwouldliketothankDr.JimHobert,who,inadditiontobeinganoutstandingteacher,isanamazingsourceofsupportforallstudentsatthedepartment.Iamforevergratefulforallhehasdonetohelpmegrow,overthepast5years.IamalsogratefulforDr.SophiaSu,Dr.AndrewRosalsky,Dr.HaniDoss,andDr.LarryWinner,forhelpingmedeveloparmfoundationinstatistics.ThankyoutoTinaGreenly,ChristineMiron,AletheaGeiger,andFloraMarynak,forhelpingmewithadministrativetasks. 4

PAGE 5

IwouldliketothankmyfriendsandfellowgraduatestudentsRayBai,SyedRahman,HunterMerrill,IsaacDuerr,QianQin,EthanAlt,andTuoChenforbeingmygreatfriends;Iamgratefulforallthememorablemomentswithyouguys.Inallyexpressmyloveformyamazingparentsandmythreewonderfulbrothersfortheirunconditionalsupport.IwouldnotbewhereIamwithouttheloveIhavealwaysreceivedfromyou.Iloveyou! 5

PAGE 6

TABLEOFCONTENTS page ACKNOWLEDGMENTS ................................... 4 LISTOFTABLES ...................................... 8 LISTOFFIGURES ..................................... 9 ABSTRACT ......................................... 10 CHAPTER 1INTRODUCTION ................................... 12 1.1TheGaussianGraphicalModelProblem ..................... 12 1.2TheProblemofJoinEstimationofMultipleGraphicalModels ......... 13 1.3ContributionsofThisWork ........................... 15 2BAYESIANFRAMEWORKFORESTIMATIONOFASINGLEGRAPHICALMODEL 16 2.1SpikeandSlabPrior ............................... 16 2.2ContinuousShrinkagePrior ........................... 19 2.3BayesianInferenceandSparsitySelection .................... 25 2.4SelectionofTheShrinkageParameter ...................... 25 2.5SimulationStudy ................................. 27 3BAYESIANFRAMEWORKFORJOINTESTMATIONOFMULTPLEGRAPHICALMODELS ....................................... 30 3.1ModelFormulation ................................ 30 3.2SubsetSpecic(S2)PriorDistribution ..................... 32 3.3TheBayesianJointNetworkSelector(BJNS) .................. 35 3.3.1GibbsSamplingSchemeforBJNS .................... 38 3.3.2SelectionofShrinkageSarameters .................... 41 3.3.3ProcedureforSparsitySelection ..................... 43 4THEORATICALGARANTEESFORBJNS ...................... 44 5NUMERICALANALYSIS ............................... 47 5.1Simulation1:fourgroups(K=4) ....................... 48 5.2SimulationStudy2:ComparisonwithExistingMethods ............ 53 5.2.1ComparisonwithExistingMethods ................... 54 5.2.2AComputationalStrategytoGainEciencyandReduceCost ..... 55 5.2.3ComputationalCostofBJNS ...................... 59 5.3MetabolomicsData ............................... 59 APPENDIX 6

PAGE 7

ATHESTRUCTUREOF ............................... 65 BPROOFSOFTHEOREMS1AND2 ......................... 67 REFERENCES ........................................ 84 BIOGRAPHICALSKETCH ................................. 87 7

PAGE 8

LISTOFTABLES Table page 2-1Percentageofspecicity(SP%),sensitivity(SE%)andMatthewsCorrelationCoecient(MC%)forthedierentmodelsanddierentmethodsbasedon50replications. .. 29 5-1Summaryresultsforsimulationi ........................... 51 5-2Summaryresultsforsimulationii ........................... 52 5-3ComparingBJNSwithexistingmethods ....................... 56 5-4Eciencygainedbythecomputationalstrategy ................... 59 5-5AccuracyandcostofBJNSforvaryingvaluesofp .................. 59 5-6Summeryresultsformetabolomicsdata ........................ 61 8

PAGE 9

LISTOFFIGURES Figure page 5-1Heatmapsoftheprecisionmatricesofthefourgroups ................ 49 5-2ConvergenceandthedistributionofduringtheMCMCsamplings ......... 53 5-3Heatmapoftheprecisionmatrices .......................... 55 5-4Barplotoftheedgecountofthepairwisejointmatrices ............... 58 5-5Barplotoftheedgecountofthematricesinthesecondstep ............ 58 5-6Metabolomicsdata:networkofcommonedges .................... 61 5-7Metabolomicsdata:heatmapofcommonedges ................... 62 5-8Metabolomicsdata:pairwisematrices ......................... 63 5-9Metabolomicsdata:individualmatrices ........................ 64 9

PAGE 10

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyBAYESIANESTIMATIONANDMODELSELECTIONFORSINGLEANDMULTIPLEGRAPHICALMODELSByPeymanJalaliMay2019Chair:GeorgeMichailidisCochair:KshitijKhareMajor:StatisticsUndirectedGraphicalModelsrepresentafamilyofcanonicalstatisticalmodelsforreconstructinginteractionsamongstasetofentitiesfrommulti-dimensionaldataproles.TheyhavenumerousapplicationsinbiologyinvolvingOmicsandneuroimagingdata,insocialsciencesforvotingrecordsandecon/nancialdata,intextmining,etc.Recently,theproblemofjointestimationofmultiplegraphicalmodelsfromhighdimensionaldatahasalsoreceivedmuchattentioninthestatisticsandmachinelearningliterature,duetoitsimportanceindiverseeldsincludingmolecularbiology,neuroscienceandthesocialsciences.Intherstpartofthisdissertation,wewilldeveloptwoBayesianmethodologies,usingspikeandslabandcontinuousshrinkagepriorscoupledwithapseudo-likelihoodthatenablesfastcomputations,forestimatingasinglehigh-dimensionalgraphicalmodel.WewillintroduceecientGibbssamplersandillustratetheeciencyofthemodelsbycomparingwiththestateoftheartmodels.ThesecondpartdevelopsaBayesianapproachthatdecomposesthemodelparametersacrossmultipleunderlyinggraphicalmodelsintosharedcomponentsacrosssubsetsofmodelsandedges,andidiosyncraticones.Further,itleveragesanovelmultivariatepriordistribution,coupledwiththesamepseudo-likelihoodasabovethrougharobustandecientGibbssamplingscheme.Weestablishstrongposteriorconsistencyformodelselection,aswellasestimationofmodelparametersunderhighdimensionalscalingwiththenumberofvariables 10

PAGE 11

growingexponentiallywiththesamplesize.Theecacyoftheproposedapproachisillustratedonbothsyntheticandrealdata. 11

PAGE 12

CHAPTER1INTRODUCTION 1.1TheGaussianGraphicalModelProblemSupposewehaveobservationsyi:=(yi1;:::;yip)0;i=1;:::nwhichareindependentandidenticallydistributedfromNp(0;)]TJ /F3 7.97 Tf 6.58 0 Td[(1).DenotethesamplecovariancematrixbyS,andthesamplecorrespondingtojthvariablebyy:j=(y1j;:::;ynj);j=1;:::;p.InaGaussiangraphicalmodel,weassumethattheinversecovariancematrixissparse.Inthefrequentistsetting,oneofthestandardmethodstoachieveasparseestimateofistominimizeanobjectivefunction,comprisedofthe(negative)log-likelihoodandan`1-penaltytermfortheo-diagonalentriesof,overthespaceofpositivedenitematrices exp )]TJ /F4 11.955 Tf 10.49 8.09 Td[(n 2(tr(S))]TJ /F1 11.955 Tf 11.96 0 Td[(logdet+ nXX1jkpj!jkj)!;(1-1)thisapproachanditsvariantsareknownasthegraphicallassowhichhasbeenwelladdressedintheliterature, YuanandLin ( 2007 ), Friedmanetal. ( 2008 ), Banerjeeetal. ( 2008 ),and Wangetal. ( 2012 ).ThegraphicallassousestheGaussianlikelihood,whichmaynotbesuitablewhenthedataisnotGaussian,i.e.itisnotrobusttomodelmisspecication.Moreover,duetotheconstraintofpositivedeniteness,theiterationsoftheoptimizationalgorithmsforgraphicallassoortheMCMCalgorithmsforit'sBayesiancounterpartsinvolvesinversionof(p)]TJ /F5 11.955 Tf 10.8 0 Td[(1)(p)]TJ /F5 11.955 Tf 10.8 0 Td[(1)matrices,whichcanbecomputationallyveryintensivewhenpislarge.Toaddressthisproblemsinthefrequentistsetting,severalworks(see Pengetal. ( 2009 ), Khareetal. ( 2015 ))haveconsideredreplacingthelogGaussianlikelihoodbythelogpseudo-likelihood.Thepseudo-likelihood(inthiscontext)ischosentobetheproductofconditionaldensitiesofeachvariable(givenalltheothervariables),andisproportionalto exp0@)]TJ /F8 7.97 Tf 17.8 15.43 Td[(pXj=1!jjnXi=1 yij+Xk6=j!jk !jjyik!2+n 2pXj=1log!ii1A:(1-2) 12

PAGE 13

Notethat)]TJ /F8 7.97 Tf 10.49 6.28 Td[(!jk !jjistheregressioncoecientofyk:whenweregressyj:onallothervariables,and1 !jjistheresidualvariance.Thisistrueforanyunderlyingdistributionfory.Hencethe(negative)logpseudo-likelihoodcanbeinterpretedasthecombinedsquarederrorlossassociatedwithalltheseregression(correspondingtoallpvariables)eveninnon-Gaussiansettings.Owingtothisproperty, Pengetal. ( 2009 )developedtheSPACEalgorithmwhichobtainsasparseestimatorforbyminimizingalossfunctionconsistingofthe(negative)logpseudo-likelihoodandan`1penaltytermforo-diagonalentriesof.However,thisobjectivefunctionisnotjointlyconvex,whichcanleadtoseriousconvergenceissues.Toaddressthisproblem, Khareetal. ( 2015 )developedtheCONCORDalgorithm,whichconsidersamodiedversionofthelogpseudo-likelihoodwhichisjointlyconvexin.Theobjectivefunctionforthismethodisgivenby Qcon()=)]TJ /F4 11.955 Tf 9.3 0 Td[(npXj=1log!jj+1 2pXj=1nXi=1 !jjyij+Xk6=j!jkyik!2+XX1j
PAGE 14

componenttozero,thecorrespondingedgeisabsentacrossallmodels,whereasifthecommoncomponentisnotzero,edgescanbeabsentbecausethepenaltysetstheidiosyncraticonetozeroforselectedmodels.Anothersetofapproachesaimstoachieveacertainamountof\fusing"acrossallmodelsunderconsideration,thusfocusingbothofthepresenceofcommonedges,aswellastheirabsenceacrossallmodelssimultaneously.Examplesofsuchapproachesinclude Danaheretal. ( 2014 )thatemployedagrouplassoand/orafusedlassopenaltyoneachedgeparameteracrossallmodelsand Caietal. ( 2016 )thatusedamixed`1=`1normforthesametask.Variantsoftheaboveapproacheswithmodicationstothepenaltieshavealsobeenexplored Zhuetal. ( 2014 ), MajumdarandMichailidis ( 2018 ).However,inmanyapplicationsettings,sharedconnectivitypatternsacrossmodelsoccursonlyforasubsetofedges,whiletheremainingonesexhibitdierentconnectivitypatternsineachmodel.Inothersettings,subsetsofedgessharecommonconnectivitypatternsacrossonlyasubsetofmodels.Inbothinstances,thepreviouslymentionedapproacheswillexhibitaratherpoorperformanceindiscoveringthesemorecomplexpatterns.Toaddressthisissue, MaandMichailidis ( 2016 )proposedasupervisedapproachbasedonfusingthroughagrouplassopenalty,whereinthevariousconnectivitypatternsacrosssubsetsofedgesandsubsetsofmodelsareaprioriknown.Analternativesupervisedapproach SaegusaandShojaie ( 2016 )employedasimilaritygraphpenaltyforfusingacrossmodels,coupledwithan`1penaltyforobtainingsparsemodelestimates.Thesimilaritygraphisassumedtobeaprioriknown.ABayesianvariantofthelatterapproachwasintroducedin Petersonetal. ( 2015 ),whereinaMarkovrandomeldpriordistributionwasusedtocapturemodelsimilarity,followedbyaspike-and-slabpriordistributionontheedgemodelparameters.AnotherBayesianapproachwasrecentlydevelopedin Tanetal. ( 2017 ).Similarto Petersonetal. ( 2015 ),itusesG-Wishartpriordistributionsonthegroup-wiseprecisionmatricesgiventhesparsitypatterns/networksineachgroup,andthenemploysamultiplicativemodelbasedhierarchicalpriordistributiononthesenetworkstoinducesimilarity/dependence. 14

PAGE 15

Mostofthefrequentistapproachesreviewedabovecomewithperformanceguaranteesintheformofhighprobabilityerrorsboundsforthemodelparametersasafunctionofthenumberofvariables(nodes),sparsitylevelandsamplesize.Somerecentworkhasfocusedonconstructingcondenceintervalsforthedierenceinthemagnitudeoftheedgeparameteracrosstwomodelsthatareestimatedseparatelyusingan`1penalization.Ontheotherhand,theoreticalguaranteesbasedonhigh-dimensionalposteriorconsistencyresultsarenotavailablefortheBayesianapproachesmentionedabove.Also,theseapproachescansuerfromcomputationalscalability/eciencyissuesinmoderate/highdimensionalsettings,sayinthepresenceofp>30nodes/variables. 1.3ContributionsofThisWorkInthisdissertation,ourrstgoalistoutilizebothspikeandslabandcontinuousshrinkagepriorsalongwiththepseudo-likelihoodinEq. 1-3 toconstructfastandecientBayesianapproachestoestimateasinglegraphicalmodel.Justasin Pengetal. ( 2009 ), Khareetal. ( 2015 ),wewillrespectthesymmetryofandwillintroduceappropriateestimatorsforeachframework,basedonthepropertiesoftheunderlyingprior.ThesecondandmoreimportantobjectiveofthisdissertationistodevelopascalableapproachtojointlyestimatemultiplerelatedGaussiangraphicalmodelsthatexhibitcomplexedgeconnectivitypatternsacrossmodelsfordierentsubsetsofedges.Tothatend,weintroduceanovelSubsetSpecic(S2)priorthatforeachedgeaimsinselectingthesubsetofmodelsitiscommonto.WecoupleitwiththeGaussianpseudo-likelihoodusedin Khareetal. ( 2015 )forestimatingasingleGaussiangraphicalmodel,thatleadstoaneasytoimplementandscalableGibbssamplingschemeforestimatingtheposteriordistribution.Finally,weestablishstrongposteriormodelselectionconsistencyresultsthatcanbeleveragedforconstructionofcredibleintervalsfortheedgeparameters.Intuitively,theproposedframeworkachievestheobjectivessetforthinthe MaandMichailidis ( 2016 )work,withoutrequiringapriorispecicationofthesharededgeconnectivitypatterns;thus,theapproachcanbeconsideredasfullyunsupervised. 15

PAGE 16

CHAPTER2BAYESIANFRAMEWORKFORESTIMATIONOFASINGLEGRAPHICALMODELInthischapter,weintroducetwonewfullyBayesiangraphicalmodelingmethods,usingtwofamilyofwellknownpriors,spikeandslabandcontinuousshrinkagepriors,coupledwiththeCONCORDpseudo-likelihoodgiveninEq. 1-3 .ThereisanextensiveliteratureonBayesianmethodsbasedonpseudo-likelihood,see VenturaandRacugno ( 2016 )foranextensivereview.Whilethepseudo-likelihoodisnotaprobabilitydensityanymore,itcanstillberegardedasaweightfunction,andaslongastheproductofthepseudo-likelihoodandthepriordensityisintegrableovertheparameterspace,onecanconstructaposteriordistributionanddoBayesianinference. 2.1SpikeandSlabPriorForaBayesiananalysis,onestandardwayistochooseaspikeandslabprior(rstpioneeredby MitchellandBeauchamp ( 1988 )),whichisamixtureofapointmassatzeroandanappropriatecontinuousprobabilitydistributioneverywhereelse, !jk(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q)IS0(!jk)+q(!jk)IS1(!jk);(2-1)whereqisthepriorinclusionprobability,sothatevery!jkisnonzerowithprobabilityq,Iistheindicatorfunction,andS0andS1correspondtotheeventsf!jk=0gandf!jk6=0g,respectively.Inthissetting,weconsider(:)tobeanormaldistributionwithN(0;1=),andwelet(1)]TJ /F4 11.955 Tf 12.02 0 Td[(q)/p p 2,whichisthevalueofthedensityN(0;1=)at!jk=0.Hence,weconsiderthefollowingindependentpriorsforeveryodiagonalcoordinate!jk, !jkp p 2 1+p p 2IS0(!jk)+1 1+p p 2N(0;1 )IS1(!jk);1j
PAGE 17

Weputanindependentmixtureprioroneacho-diagonalelementandanindependentExponential()prioroneachdiagonalelementoftoconstructanovelpseudo-likelihoodbasedBayesianGraphicalmodelingmethodology,calledBayesianSpikeandSlabConcord(BSSC),whichascanbeseenfromthemixturepriorsinEq. 2-3 ,isabletoproduceexactzerosbyallocatingpositiveposteriorprobabilitiestotheeventsf!jk=0g.Sinceourmaingoalissparsityselection,similarto Pengetal. ( 2009 ), Khareetal. ( 2015 ), Leppa-Ahoetal. ( 2017 )wewillnotimposethepositivedenitenessconstraintontheentriesof.LetM+pdenotethespaceofallppmatriceswithpositivediagonalentries.PosteriorcomputationandGibbssamplingschemeforforBSSC.Usingthepseudo-likelihoodinEq. 1-2 ,theBSSCposteriordensityfunctiononM+pisproportionalto, exp npXj=1log!jj)]TJ /F4 11.955 Tf 13.15 8.08 Td[(n 2tr)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(2S)]TJ /F4 11.955 Tf 11.96 0 Td[(XX1j
PAGE 18

!jkof,giventheremainingelementsdenotedby)]TJ /F3 7.97 Tf 6.59 0 Td[((jk)tointroduceanentrywiseGibbssamplerthatcangenerateapproximatesamplesfromtheposteriordensityinEq. 2-4 .Inordertocomputetheconditionalposteriordensityoftheo-diagonalelements!jk,for1jkp,werstnotethat ntr)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(2S=nXi=1k(yi:)k2=nXi=1pXj=1 pXk=1!jkyik!2=pXj=1k pXk=1!jky:k!k2;(2-7)thus,inviewofEq. 2-4 , f(!jkj)]TJ /F3 7.97 Tf 6.58 0 Td[((jk))/expn)]TJ /F4 11.955 Tf 10.5 8.09 Td[(n 2)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(ajk!2jk+2bjk!jkoIS0SS1(!jk)=IS0(!jk)+cjkp najk p 2exp()]TJ /F4 11.955 Tf 10.49 8.09 Td[(najk 2!jk+bjk ajk2)IS1(!jk)(2-8)with, ajk=sjj+skk+ n;bjk=0)]TJ /F8 7.97 Tf 6.59 0 Td[(jkS)]TJ /F8 7.97 Tf 6.58 0 Td[(jj+0)]TJ /F8 7.97 Tf 6.59 0 Td[(kjS)]TJ /F8 7.97 Tf 6.59 0 Td[(kk;cjk=p 2 p najkexpnb2jk 2ajk:(2-9)now,bylettingpjk=cjk 1+cjk,wecanwrite (!jkj)]TJ /F3 7.97 Tf 6.59 0 Td[((jk))(1)]TJ /F4 11.955 Tf 11.96 0 Td[(pjk)IS0(!jk)+pjkN()]TJ /F4 11.955 Tf 11.08 8.08 Td[(bjk ajk;1 najk)IS1(!jk);1j
PAGE 19

onecanuseadiscretizationtechniquetogeneratesamplesfromit.However,wehaveobservedinoursimulationsthatthedensityinEq. 2-11 hasahighpickatit'smode.Asaresult,onecansimplyapproximateitusingadegeneratedensitywithapointmassatit'smode,giveninEq. 2-12 .Thisapproximationallowsfasterimplementationofouralgorithmwithoutmuchsacricingonit'saccuracy.Therefore,usingthedistributionsinEq. 2-10 ,andEq. 2-11 ,webuildanentrywiseGibbssampler,oneiterationofwhich,giventhecurrentvalueof,isdescribedinalgorithm 2.1 . Algorithm2.1. procedureBSSC(S).Inputthedata forj=1;:::;p)]TJ /F5 11.955 Tf 11.95 0 Td[(1do fork=j+1;:::;pdo !jk(1)]TJ /F4 11.955 Tf 11.95 0 Td[(pjk)IS0(!jk)+pjkN()]TJ /F8 7.97 Tf 10.94 6.27 Td[(bjk ajk;1 ajk)IS1(!jk) endfor endfor forj=1;:::;pdo !jj )]TJ /F3 7.97 Tf 6.59 0 Td[((+n0)]TJ /F11 5.978 Tf 5.75 0 Td[(jjS)]TJ /F11 5.978 Tf 5.75 0 Td[(jj)+p (+n0)]TJ /F11 5.978 Tf 5.76 0 Td[(jjS)]TJ /F11 5.978 Tf 5.75 0 Td[(jj)2+4n2skii 2nskii endfor return.Return endprocedure 2.2ContinuousShrinkagePriorContinuousshrinkagepriorsareanotherpopularalternativetothemixtureprior.Thistypeofpriorshaveapeakatzeroandhavetailsdecayingatanappropriaterate,whichservesthepurposeofshrinking.Continuousshrinkagepriordistributionsareoftenascalemixtureofnormals,suchasLaplace,half-Cauchyetc.(see PolsonandScott ( 2010 ), Bhattacharyaetal. ( 2015 )andthereferencestherein).Inthecontextoflinearregression,theBayesianlassoof ParkandCasella ( 2008 ),basedontheinterpretationofthewell-knownlassoestimatoroftheregressioncoecientsastheposteriormodeinaBayesianmodelwhichputsindependentLaplacepriorsontheindividualcoecients,hasgainedpopularityinrecentyears. 19

PAGE 20

TheBayesianGraphicallassowasproposedby Wangetal. ( 2012 ),asaBayesianadaptationofthegraphicallasso.ABayesianmodel,whichputsindependentLaplacepriorsontheo-diagonalentriesofandindependentexponentialpriorsonthediagonalentriesof(restrictedtobeginpositivedenite).Theposteriordensityforinthissettingisproportionaltoexp )]TJ /F4 11.955 Tf 10.49 8.09 Td[(n 2(tr(S))]TJ /F1 11.955 Tf 11.96 0 Td[(logdet+ nXX1jkpj!jkj)!:ItfollowsthatthegraphicallassoestimatoristheposteriormodeofthisBayesianmodel.TheBayesiangraphicallassointerpretationimmediatelyyieldscredibleregionsforthegraphicallassoestimateof.Suchestimatesofuncertaintyarenotreadilyavailableinthefrequentistsetting.Alternatively,somepractitionersalsodeterminesparsityinbasedonwhetherzeroiscontainedinthecredibleintervalfortherespectiveo-diagonalentries;WeusethesameshrinkagepriorstodevelopaBayesianprocedure,calledBayesianContinuousShrinkageConcord(BCSC),fortheCONCORDalgorithm.Similartothepreviousframework,wewillnotimposethepositivedenitenessconstraintontheentriesof,assparsityselectionisthemaingoalofthismethod,aswell.PosteriorcomputationandGibbssamplingforBCSC.InviewofEq. 1-2 ,thepseudo-likelihoodbasedposteriordensityonM+p,usingcontinuouspriors,isproportionalto exp npXj=1log!jj)]TJ /F4 11.955 Tf 13.15 8.08 Td[(n 2tr)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(2S)]TJ /F4 11.955 Tf 11.96 0 Td[(XX1jkpj!jkj!=exp )]TJ /F4 11.955 Tf 9.3 0 Td[(Qcon())]TJ /F4 11.955 Tf 11.96 0 Td[(pXj=1!jj!:(2-13)TheposteriormodeisaslightlymodiedversionoftheCONCORDestimator(duetotheadditionalterm)]TJ /F4 11.955 Tf 9.3 0 Td[(pPj=1!jj).WecanusetheposteriorinEq. 2-13 toconstructcredibleregionsforthisestimate.Theposteriordensityisclearlyintractableinthesensethatclosedformcomputationsordirecti.i.d.samplingfromthisdensityarenotfeasible.However,usingastandardrepresentationoftheLaplaceasascalemixtureofnormals,wecanderiveablock 20

PAGE 21

GibbssamplingalgorithmtogenerateapproximatesamplesfromthedensityinEq. 2-13 .Notethat Z101 p 2xexp)]TJ /F4 11.955 Tf 10.52 8.09 Td[(!2 2xexp)]TJ /F4 11.955 Tf 10.5 8.09 Td[(2x 2dx=constexp()]TJ /F4 11.955 Tf 9.3 0 Td[(j!j):(2-14)Basedonthisobservation,weintroduceappsymmetricmatrixTwithzerodiagonalentriesasanaugmentedvariable,andconstructajointdensityon(;T)whichisproportionalto exp0@npXj=1log!jj)]TJ /F20 10.909 Tf 12.11 7.38 Td[(n 2tr)]TJ /F22 10.909 Tf 5 -8.84 Td[(2S)]TJ /F10 10.909 Tf 12.63 10.36 Td[(XX1j
PAGE 22

with, j=[y:j+1;:::;y:p]n(p)]TJ /F8 7.97 Tf 6.59 0 Td[(j);Dj=diagtjj+1 1+tjj+1ky:jk2;:::;tjp 1+tjpky:jk2(p)]TJ /F8 7.97 Tf 6.59 0 Td[(j)(p)]TJ /F8 7.97 Tf 6.58 0 Td[(j);al=pXi6=j=1!liy:i(l=j+1;:::;p);andbj=jXi=1!jiy:i:(2-18)Therefore, !j+j)]TJ /F8 7.97 Tf 6.59 0 Td[(j+;T;f!jjgpj=1Np)]TJ /F8 7.97 Tf 6.58 0 Td[(j(j;j);j=1;:::;p)]TJ /F5 11.955 Tf 11.95 0 Td[(1;(2-19)where,jandjaredenedaccordingtoEq. 2-17 .DirectsamplingfromthemultivariatenormaldensitiesinEq. 2-19 hascomplexityoforderO(p)]TJ /F4 11.955 Tf 12.61 0 Td[(j)3.However,usingthealgorithmintroducedin Bhattacharyaetal. ( 2016 ),wecanreducethecomplexityofgeneratingsamplefromEq. 2-19 toO(n2(p)]TJ /F4 11.955 Tf 11.6 0 Td[(j)).Thealgorithm1in Bhattacharyaetal. ( 2016 )oersafastwaytodrawexactsamplesfromaclassofmultivariatenormaldistributionsNp(;),with )]TJ /F3 7.97 Tf 6.59 0 Td[(1=0+D)]TJ /F3 7.97 Tf 6.59 0 Td[(1;)]TJ /F3 7.97 Tf 6.59 0 Td[(1=0;(2-20)whereDisappsymmetricpositivedenitematrix(inourcaseadiagonalmatrix),isannpmatrixandisann1vector.TheyshowedthatifEq. 2-20 holdsforsomeappropriatematrices,D,andavector,onecanusealgorithm 2.2 ,whichhascomplexityO(n2p),togenerateexactsamplesfromthemultivariatenormaldistributionwithmean,andcovariancematrix(Bhattacharyaetal(2014),proposition2.1 Bhattacharyaetal. ( 2016 )). Algorithm2.2. procedure uNp(0;D). 22

PAGE 23

Nn(0;I). v=u+. (D0+I)w=()]TJ /F6 11.955 Tf 11.95 0 Td[(v). =u+D0w. endprocedureInthecaseofthemultivariatenormaldensitiesinEq. 2-19 ,theadditionalcomputationaladvantagecomesfromthefactthatDjsarediagonalmatrices.Hence,instep2oftheabovealgorithm,wesimplygenerateindependentunivariatenormalrandomvariables.Also,wenotethatsincewecannotensuretheexistenceofthevectorsj,withj)]TJ /F3 7.97 Tf 6.58 0 Td[(1j=0jj;j=1;:::;p)]TJ /F5 11.955 Tf 12.21 0 Td[(1,(becausetheselinearsystemsdonotnecessarilyhavesolutions),werstgeneratesamplesfromNp)]TJ /F8 7.97 Tf 6.58 0 Td[(j(0;j)andshifttheresultingsamplesbyj,whichinviewofEq. 2-17 canbeeasilycalculatedasj=)]TJ /F6 11.955 Tf 5.48 -9.68 Td[()]TJ /F3 7.97 Tf 6.59 0 Td[(1j)]TJ /F3 7.97 Tf 6.59 0 Td[(1)]TJ /F6 11.955 Tf 5.48 -9.68 Td[()]TJ /F3 7.97 Tf 6.58 0 Td[(1jj.ConsideringtheseremarksaboutthedensitiesinEq. 2-19 ,thestructureof)]TJ /F16 7.97 Tf 6.59 0 Td[(1jgiveninEq. 2-17 ,andusingtheSherman-Morrison-Woodburyidentity,weareabletomodifythealgorithm 2.2 intoanewprocedurewhichgeneratessamples,withmuchsmallercomputationalcost,fromdensitiesNp)]TJ /F8 7.97 Tf 6.59 0 Td[(j(j;j)inEq. 2-19 .Theresultingprocedureisdescribedinalgorithm 2.3 ,anditiseasytoseethatithascomplexityO(min(n;p)]TJ /F4 11.955 Tf 12.98 0 Td[(j))3,whichisasignicantreductioninthecomputationalburden,particularlyforthen=p)]TJ /F4 11.955 Tf 11.96 0 Td[(j)then Aj=)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0jj+D)]TJ /F3 7.97 Tf 6.58 0 Td[(1j)]TJ /F3 7.97 Tf 6.59 0 Td[(1. wj=(In)]TJ /F6 11.955 Tf 11.96 0 Td[(jAj0j)()]TJ /F4 11.955 Tf 9.3 0 Td[(vj). j=Aj)]TJ /F6 11.955 Tf 5.48 -9.68 Td[()]TJ /F3 7.97 Tf 6.59 0 Td[(1jj endif 23

PAGE 24

if(n0;(2-21)whichisthekernelofthedensityofrandomvariable1 X,whenXInverse-Gaussian j!jkj;2:Finally,lookingatEq. 2-15 ,theconditionalindependenceoftheentriesoff!jjgpj=1givenf!jkg1j
PAGE 25

!j+Np)]TJ /F8 7.97 Tf 6.59 0 Td[(j)]TJ /F23 11.955 Tf 5.48 -9.69 Td[(j;)]TJ /F3 7.97 Tf 6.58 0 Td[(1j.UseAlgorithm 2.3 endfor forj=1;:::;p)]TJ /F5 11.955 Tf 11.95 0 Td[(1do fork=j+1;:::;pdo xjkInverse-Gaussian j!jkj;2 updatetjk 1 xjk endfor endfor forj=1;:::;pdo !jj )]TJ /F3 7.97 Tf 6.59 0 Td[((+n0)]TJ /F11 5.978 Tf 5.75 0 Td[(jjS)]TJ /F11 5.978 Tf 5.75 0 Td[(jj)+p (+n0)]TJ /F11 5.978 Tf 5.76 0 Td[(jjS)]TJ /F11 5.978 Tf 5.75 0 Td[(jj)2+4n2skii 2nskii endfor return.Return endprocedure 2.3BayesianInferenceandSparsitySelectionInthecaseofBSSC,wenotethattheconditionalposteriorprobabilitydensityoftheodiagonalelementsofisamixturedensitywithpositiveprobabilityallocatedtotheeventsf!jk=0g.ThispropertyofBSSCallowsanimmediatemodeldeterminationinthesensethatonecansimplyuseamajorityvotingcriteriatodecidewhetherornotanodiagonalcoordinateofispresent.However,inthecaseofBCSC,sinceourcontinuousshrinkagepriorsallocatezeroprobabilitytotheeventsf!jk=0g,naturallytheposteriorprobabilityofsucheventsisalsoequaltozero.Thus,todeterminethezeros,weconstruct95%crediblebondsforeach!jk,aftera6decimalroundingoftheresults.Coordinateswhosecredibleboundscontainzero,areestimatedtobeequaltozero. 2.4SelectionofTheShrinkageParameterSelectingappropriatevaluesfortheshrinkageparameterisacrucialtaskingraphicalmodeling.Infrequentistframeworks,thereareseveralwaystochoose.Forexample,one 25

PAGE 26

canestimatethisparameterusingamaximumlikelihoodestimationprocedure,orusecrossvalidationandselectthevalueofforwhichthe\error"isminimum.IntheBayesiansettings,canbegeneratedfromanappropriatehyper-prior.Inthecontextofregressionanalysis,onecaneasilycalculatethefullconditionaldistributionof,andusetheresultingconditionaldistributionintheMCMCimplementationtoselectthevalueof( ParkandCasella ( 2008 ), Kyungetal. ( 2010 ), Hans ( 2009 )).However,inthecontextofgraphicalmodels,thisfullconditionaldistributioninvolvesanintractablenormalizingconstant,whichappearsinthecalculationsandisduetotheconstraintofpositivedeniteness.TheCONCORDlikelihoodhowever,doesnotimposeanyconstraintofpositivedenitenesson,allowingfordevelopingfullyBayesiangraphicalmodelingframeworks,usingstandardpriors.Inourcase,weputindependentgammapriorsoneachshrinkageparameterjk.Hence,wecanwritethefullyBayesianversionofBSSCas p(y1:;:::;yn:j)/exp()]TJ /F4 11.955 Tf 9.3 0 Td[(npXj=1log!jj+n 2tr)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(2S);pjfjkgjk/Yj
PAGE 27

thecaseofBSSC,onecaneasilyseethatjkjGamma(r+0:5;o:5!2jk+s):andinthecaseofBCSC,jkjGamma(1+r;j!jkj+s);Thus,BSSCandBCSCtendtoshrinkanycoordinate!jk,onaveragebyE(jkj)=r+0:5 o:5!2jk+sandE(jkj)=1+r j!jkj+s,respectively.Thatis,bothmethodsselecttheshrinkageparameterjkbasedonthecurrentvalueoftheentriesof.Theselectionofthehyper-parametersrandsisacrucialtaskandcansignicantlyaecttheperformanceofourmodel.But,followingthesuggestionof Wangetal. ( 2012 ),weletr=10)]TJ /F3 7.97 Tf 6.59 0 Td[(2,ands=10)]TJ /F3 7.97 Tf 6.58 0 Td[(6. 2.5SimulationStudyInordertotesttheperformanceofourmethod,asimulationexperimentwasdesignedinwhichwecomparedBCSCandBSSCwith4othermethods,Glasso( Friedmanetal. ( 2008 )),theadaptivegraphicallassoandthegraphicalSCAD( Fanetal. ( 2009 ))andtheadaptiveBayesiangraphicallasso( Wangetal. ( 2012 )).Inthissimulationstudyweconsidered6dierentmodels: Model1:AnAR(1)modelswithij=0:7ji)]TJ /F8 7.97 Tf 6.58 0 Td[(jj. Model2:AnAR(2)modelwith!ii=1,!i;i)]TJ /F3 7.97 Tf 6.58 0 Td[(1=!i)]TJ /F3 7.97 Tf 6.58 0 Td[(1;i=0:5and!i;i)]TJ /F3 7.97 Tf 6.59 0 Td[(2=!i)]TJ /F3 7.97 Tf 6.58 0 Td[(2;i=0:25. Model3:ABlockmodelwithii=1,ij=0:5for1i6=jp=2;ij=0:5forp=2+1i6=j10andij=0otherwise. Model4:Astarmodelwitheverynodeconnectedtotherstnode,with!ii=1,!1;i=!i;1=0:1and!ij=0otherwise. Model5:Acirclemodelwith!ii=2,!i;i)]TJ /F3 7.97 Tf 6.59 0 Td[(1=!i)]TJ /F3 7.97 Tf 6.58 0 Td[(1;i=1and!1p=!p1=0:9. Model6:Afullmodelwithwith!ii=2and!ij=1fori6=j.Foreachmodel,weconsideredtwosettingsp=30;n=50andp=100;n=200.Inordertotoassesstheperformanceofthedierentmodels,weusedthreewellknownaccuracy 27

PAGE 28

measures,specicity(SP),sensitivity(SE)andMatthewsCorrelationCoecients(MCC)denedas( Fanetal. ( 2009 )): SP=TN TN+FP;SE=TP TP+FNMCC=TPTN)]TJ /F1 11.955 Tf 11.96 0 Td[(FPFN p (TP+FP)(TP+FN)(TN+FP)(TN+FN)(2-25)where,TP,TN,FPandFNrepresentthenumberoftruepositives,truenegatives,falsepositivesandfalsenegatives,respectively.Largervaluesofanyoftheabovemetricsindicatesabetterclassicationruleproducedbytheunderlyingmethod.Inordertoachieveamorerobustassessment,werepeatedeachexperiment50timesandcalculatedaverageoftheresults,whichisgivenintable 2-1 .Thetoptomethodineverycasearehighlighted.Ascanbeseen,exceptintheBlockcase,theoverallperformanceofBSSCandBCSCaresignicantlyhigherthantheothermethods.GlassoandadaptiveglassotendtohavehighsensitivityscoresespeciallywithdensestructuressuchasBlockandCircle.Betweenourtwomodels,BSSCtendtohaveahigherMCCscoreandthatisduetothepropertiesofit'spriorthatallowsproducingexactzeros. 28

PAGE 29

Table2-1. Percentageofspecicity(SP%),sensitivity(SE%)andMatthewsCorrelationCoecient(MC%)forthedierentmodelsanddierentmethodsbasedon50replications. AR(1)AR(2)BlockStarCircleSPSEMCSPSEMCSPSEMCSPSEMCSPSEMC(%)(%)(%)(%)(%)(%)(%)(%)(%)(%)(%)(%)(%)(%)(%) p=30n=50Glas66100346582327867469128166310032Adap78100447581428049319117057510041Scad8293457876437436109603038610053Bada96100779043339334349414079510076BCSC998585976667981221973841999995BSSC99100989958701002032993452968475p=100n=200Glas771002575973189443795100547310023Adap90100388596419031279397448610033Scad9810074909650822256794184210012Bada96100559490569320199185369710062BCSC99100909797789905109667479910093BSSC10010099100969799122597725410010096 29

PAGE 30

CHAPTER3BAYESIANFRAMEWORKFORJOINTESTMATIONOFMULTPLEGRAPHICALMODELS 3.1ModelFormulationInthischapter,wedevelopanovelBayesianapproachtojointestimationofmultiplegraphicalmodels.SupposewehavedatafromKaprioridenedgroups.Foreachgroupk(k=1;2;:::;K),letYk:=yki:nki=1denotep-dimensionali.i.dobservationsfromamultivariatenormaldistribution,withmean0andcovariancematrix)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(k)]TJ /F3 7.97 Tf 6.59 0 Td[(1,whichisspecictogroupk.Basedonthediscussionintheintroductorysection,theKprecisionmatriceskKk=1cansharecommonpatternsacrosssubsetsoftheKmodels,asdelineatednext.Ourgoalistoaccountforthesesharedstructures.LetP(K)denotethepowersetoff1;;Kgandfork=1;:::;K,dene#kasfollows: #k=fr2P(K)nf0g:k2rg;k=1;:::;K:(3-1)Itiseasytocheckthateach#kisthecollectionofsubsetswhichcontaink,andhasPK)]TJ /F3 7.97 Tf 6.59 0 Td[(1i=00B@K)]TJ /F5 11.955 Tf 11.96 0 Td[(1i1CA=2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1members.Denotebyrthematrixthatcontainscommonpatternsamongstprecisionmatricesfjgj2r.Specically,foranysingletonsetr=fkg,thematrixrcontainsapatternthatisuniquetogroupk,whileforanyothersetrcontainingmorethanasingleelement,rcapturesedgeconnectivitypatterns(andtheirmagnitudes)thatarecommonacrossallmembersinr.Forexample,123:=f1;2;3gcontainssharedstructuresin1,2,and3.Therefore,eachprecisionmatrixkcanbedecomposedas k=Xr2#kr;k=1;:::;K;(3-2)wherePr2#kraccountsforallthestructuresinkwhichareeitheruniquetogroupk(i.e.k)oraresharedbetweengroupkandsomecombinationofothergroups(i.e.Pr2#knfkgr).Wefurtherassumethatk2M+pfork=1;2;:::;K,whereM+pdenotesthespaceofallppmatriceswithpositivediagonalentries.Finally,thediagonalentriesofeveryjointmatrix 30

PAGE 31

r,withr2[Kk=1(#knfkg)aresettozero;inotherwords,thediagonalsentriesofkarecontainedinthecorrrespondingk.Toillustratethenotation,considerthecaseofK=3groups.Then,eachprecisionmatrixisdecomposedas1=1+12+13+1232=2+12+23+1233=3+13+23+123wherethe1,2,and3matricescontaingroupspecicpatterns,the12,13,23matricescontainpatternssharedacrosspairsofgroupsandnallymatrix123containspatternssharedacrossallgroups.Identiabilityconsiderations.AmomentofreectionshowsthatthemodeldecompositioninEq. 3-2 isnotunique.Forexample,foranyarbitrarymatrixX,themodelinEq. 3-2 isequivalenttok=Pr2#krwithr=r+Xandk=k)]TJ /F3 7.97 Tf 27.22 4.71 Td[(1 2K)]TJ /F13 5.978 Tf 5.75 0 Td[(1)]TJ /F3 7.97 Tf 6.58 0 Td[(1X.Hence,withoutimposingappropriateidentiabilityconstraints,meaningfulestimationofthemodelparametersisnotfeasible.Inordertoaddressthisissue,werstrewritetheelement-wiserepresentationofmodelinEq. 3-2 : !kij=Xr2#k rij;1i
PAGE 32

sets2[Kk=1(#knfkg),whileallsubsetsofswillbeallocatedazerovalue.Further,themagnitudeofall!kij2swillbethesame.Asanexample,condiserthecaseofK=3groupsandanedge(i;j)sharedamongstallthreegroups.Inthiscase,theedgewillbeallocatedtothe123componentandnottoanysubsetofit,suchas12or13.Hence123ijwillbenon-zero,but12ij=13ij=23ij=1ij=2ij=3ij=0:Next,weconstructanovelpriordistributionthatrespectstheintroducedidentiabilityconstraint. 3.2SubsetSpecic(S2)PriorDistributionForanygenericsymmetricppmatrixA,denea =(a12;a13;:::;ap)]TJ /F3 7.97 Tf 6.59 0 Td[(1p);A=(a11;:::;app);whereduetothesymmetricnatureofA,thevectora containsallitso-diagonalelements,whileAthediagonalsones.Inparticular, risthevectorizedversionoftheo-diagonalelementsofr.Usingtheabovenotation,denetobethevectorobtainedbycombiningthevectors r;r2P(K)nf0g.Toillustrate,forthecaseofK=3groups,isgivenby =( 123; 23; 13; 12; 3; 2; 1)0:(3-5)InviewofEq. 3-4 ,itcanbeeasilyseethatthatisarearrangementofthevector(12;13;:::;p)]TJ /F3 7.97 Tf 6.59 0 Td[(1p)0.Thus,accordingtothelocationofthezerocoordinatesinij(2Kpossibilities),thereare2Kp(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 2possiblesparsitypatternsacrosstheKgroupsthatcanhave.Let`beagenericsparsitypatternforanddenotethesetofallthe2Kp(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2sparsitypatternsbyL.Toillustrate,consideracasewithK=2groupsandp=3variables.Inthiscase,eachmatrixhas3o-diagonaledges(fij:1i
PAGE 33

inbothgroups.Inthiscaseisgivenby=\000 1212;0;0;)]TJ /F5 11.955 Tf 5.48 -9.69 Td[(0; 213;0;(0;0;0)0;andthesparsitypatternextractedfromisasfollows`=((1;0;0);(0;1;0);(0;0;0))0:Foreverysparsitypattern`,letd`bethedensity(numberofnon-zeroentries)of`,andM`bethespacewherevaries,whenrestrictedtofollowthesparsitypattern`.WespecifythehierarchicalpriordistributionS2asfollows (j`)=j``j1 2 (2)d` 2exp)]TJ /F6 11.955 Tf 10.5 8.09 Td[(0 2I(2M`);(3-6) (`)/8>><>>:qd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`d`;qd`2(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q2)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`d`>;(3-7)whereisadiagonalmatrixwhoseentriesdeterminetheamountofshrinkageimposedonthecorrespondingelementsin,``isasub-matrixofobtainedafterremovingtherowsandcolumnscorrespondingtothezerosin2M`,andq1andq2areedgeinclusionprobabilities,respectively,forthecaseofsparse(d`)anddense(d`>).Laterinourtheoreticalanalysis,wespecifyvaluesforq1,q2,andthethreshold.Let`bethevectorcontainingthenon-zerocoordinatesof2M`.Then,thepriorinEq. 3-6 correspondstoputtinganindependentnormalprioroneachentryof`. 33

PAGE 34

UsingthepriordistributionpositedinEq. 3-6 andEq. 3-7 ,wederivethemarginalpriordistributionon,asfollows ()=X`2L(j`)(`)=X`2L(j``j1 2 (2)d` 2exp)]TJ /F6 11.955 Tf 10.49 8.09 Td[(0 2I(2M`))(`)=X`2Lc`j``j1 2 (2)d` 2exp)]TJ /F6 11.955 Tf 10.5 8.09 Td[(0```` 2I(`2Rd`);(3-8)where (`)=qd`1(1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`Ifd`g P`2Lhqd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`Ifd`gi:(3-9)Inotherwords,()canberegardedasamixtureof2Kp(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 2multivariatenormaldistributionsofdimensionsd`thatisobtainedafterprojectingalargerdimension(p(p)]TJ /F5 11.955 Tf 12.11 0 Td[(1)2K)multivariatenormaldistributionintotheunionofallthesubspacesM`;namelyS`2LM`.NotethattheS2priorinducessparsityon,whichwillbehelpfulformodelselectionpurposes.Further,thepriorrespectstheidentiabilityconstraintbyforcingatleast(2K)]TJ /F3 7.97 Tf 6.58 0 Td[(1)]TJ /F5 11.955 Tf 12.37 0 Td[(1)p(p)]TJ /F5 11.955 Tf 12.36 0 Td[(1)parameterstobeexactlyequaltozero.Inadditiontoforcingsparsity,thediagonalentriesof``enforceshrinkagetothecorrespondingelementsin`.WeshalllaterdiscusstheselectionoftheseshrinkageparameterslaterinSection 3.3.2 .Notethatthevectoronlyincorporatestheo-diagonalentriesofmatrices.Regardingthediagonalentries,foreveryk2f1;:::;Kg,weletkbethevectorcomprisingofthediagonalelementsofthematrixkanddenetobethevectorofalldiagonalvectorsK,i.e. =(1;:::;K):(3-10)WeassignanindependentExponential()prioroneachcoordinateof(diagonalelementofthematricesk,k=1;:::;K),i.e, ()/exp()]TJ /F4 11.955 Tf 9.3 0 Td[(10)IRKp+():(3-11) 34

PAGE 35

TheselectionofthehyperparameterisalsodiscussedinSection 3.3.2 .Sincethediagonalentriesofeveryjointmatrixr,withr2[Kk=1(#knfkg)aresettozero,thespecicationofthepriorisnowcomplete. 3.3TheBayesianJointNetworkSelector(BJNS)Estimationofthemodelparameters(;)isbasedusingapseudo-likelihoodapproach.Thepseudo-likelihood,whichisbasedontheregressioninterpretationoftheentriesof,canberegardedasaweightfunctionandaslongastheproductofthepseudo-likelihoodandthepriordensityisintegrableovertheparameterspace,onecanconstructa(pseudo)posteriordistributionandcarryoutBayesianinference.Themainadvantageofusingapseudo-likelihood,asopposedtoafullGaussianlikelihood,isthatitallowsforaneasytoimplementsamplingschemefromtheposteriordistributionandinadditionprovidesmorerobustresultsunderdeviationsfromtheGaussianassumption,asillustratedbyworkinthefrequentistdomain Khareetal. ( 2015 ); Pengetal. ( 2009 ).Notethatthepseudo-likelihoodapproachdoesnotrespectthepositivedeniteconstraintontheprecisionmatricesunderconsideration,butsinceourprimarygoalisestimatingtheskeletonoftheunderlyinggraphsthismitigatesthisissue.Further,accurateestimationofthemagnitudeoftheestimatededgescanbeaccomplishedthrougharettingstepofthemodelparametersrestrictedtotheskeleton,asshownin MaandMichailidis ( 2016 ).Wewillalsoestablishhigh-dimensionalsparsityselectionandestimationconsistencyforourprocedurelaterinSection??.LetSkdenotethesamplecovariancematrixoftheobservationsinthekthgroup.Basedontheabovediscussion,weemploytheCONCORDpseudo-likelihoodintroducedin Khareetal. ( 2015 ),exp(npXj=1log!kjj)]TJ /F4 11.955 Tf 13.15 8.09 Td[(n 2trh)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(k2Ski);k=1;:::;K; 35

PAGE 36

andthemodelspecicationinEq. 3-2 toconstructthejointpseudo-likelihoodfunctionforKprecisionmatrices,asfollows, KYk=1exp8<:npXj=1log kjj)]TJ /F4 11.955 Tf 13.15 8.09 Td[(n 2tr24 Xr2#kr!2Sk359=;:(3-12)SincewehaveparametrizedtheS2priorintermsof(;),wewillrewritetheabovepseudo-likelihoodfunctionintermsof(;),aswell.Somestraightforwardalgebrashowsthat tr24 Xr2#kr!2Sk35=1 nKXk=1nXi=1" Xr2#kr!yki:#0" Xr2#kr!yki:#=1 nKXk=1pXj=1nXi=1"Xr2#krj:yki:#2=1 nKXk=1pXj=1nXi=1Xr2#krj:yki:2+2 nKXk=1pXj=1nXi=1Xr2#kXs2#kr6=srj:yki:sj:yki:;(3-13)whereyki:nki=1denotep-dimensionalobservationsforgroupk.Next,forany1jp,1kK,andr2#k 1 nnXi=1Xr2#krj:yki:2=1 nXr2#knXi=1 pXl=1 rjlykil!2=Xr2#kpXl=1)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( rjl2skll+2Xr2#kpXl=1pXm=1l6=m)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( rjl rjmsklm:(3-14)Similarly,forany1jp,1kK,and(r6=s)2#k, 36

PAGE 37

1 nnXi=1Xr2#kXs2#kr6=srj:yki:sj:yki:=Xr2#kpXl=1)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( rjl sjlskll+2Xr2#kXs2#kr6=spXl=1pXm=1l6=m)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( rjl sjmsklm:(3-15)Thus,bycombiningEq. 3-13 ,Eq. 3-14 ,andEq. 3-15 ,wehavethat tr24 Xr2#kr!2Sk35=000B@AA0D1CA0B@1CA;(3-16)where,isap(p)]TJ /F3 7.97 Tf 6.59 0 Td[(1)(2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1) 2p(p)]TJ /F3 7.97 Tf 6.59 0 Td[(1)(2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1) 2symmetricmatrixwhoseentriesareeitherzerooralinearcombinationofskij1kK1i<>:n10log())]TJ /F4 11.955 Tf 13.15 8.08 Td[(n 2264000B@AA0D1CA0B@1CA3759>=>;exp)]TJ /F6 11.955 Tf 10.49 8.09 Td[(0 2X`2L(j``j1 2 (2)d` 2I(2M`)hqd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`gioexp()]TJ /F4 11.955 Tf 9.3 0 Td[(10):(3-17)Moreover,theconditionalposteriordistributionofgivenisgivenby fj;Yg/exp)]TJ /F5 11.955 Tf 10.5 8.09 Td[(1 2[0(n+)+2n0a]X`2L(j``j1 2 (2)d` 2I(2M`)hqd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`gi);(3-18) 37

PAGE 38

whilethatofgivenby fj;Yg/KYk=1pYi=1)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( kiinexp()]TJ /F4 11.955 Tf 10.5 8.09 Td[(n 2skii)]TJ /F4 11.955 Tf 5.48 -9.68 Td[( kii2)]TJ /F10 11.955 Tf 11.96 20.44 Td[( +nXj6=i!kijskij! kii);(3-19)where!kij=Pr2#k rij,for1i
PAGE 39

distributionofijgiven)]TJ /F3 7.97 Tf 6.59 0 Td[((ij)andisgivenbyijj)]TJ /F3 7.97 Tf 6.58 0 Td[((ij);;Y/exp)]TJ /F5 11.955 Tf 10.49 8.09 Td[(1 2h0ij(n+)(ij)(ij)ij+2n0ij)]TJ /F23 11.955 Tf 5.48 -9.69 Td[(aij+(ij)()]TJ /F3 7.97 Tf 6.59 0 Td[((ij)))]TJ /F3 7.97 Tf 6.58 0 Td[((ij)iI2K)]TJ /F13 5.978 Tf 5.75 0 Td[(1Sl=0Ml(ij):Notethatsinceijhasatmostonenon-zeroelement,allcrossproductsin0ij(n+)(ij)(ij)ijareequaltozero,i.e.0ij(n+)(ij)(ij)ij=2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1Xl=12l;ijh(n+)(ij)(ij)ill=2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1Xl=12l;ijfn(ij)(ij)ll+l;ijgwhere(ij)(ij)llisthelthdiagonalelementofmatrix(ij)(ij),forl=1;:::;2K.Hence,denotingtheunivariatenormalprobabilitydensityfunctionby,wegetijj)]TJ /F3 7.97 Tf 6.59 0 Td[((ij);;Y/exp8<:)]TJ /F5 11.955 Tf 10.5 8.09 Td[(1 22K)]TJ /F3 7.97 Tf 6.58 0 Td[(1Xl=1)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(2l;ijfn(ij)(ij)ll+l;ijg+2nl;ijaij+(ij)()]TJ /F3 7.97 Tf 6.59 0 Td[((ij)))]TJ /F3 7.97 Tf 6.59 0 Td[((ij)l9=;I2K)]TJ /F13 5.978 Tf 5.76 0 Td[(1Sl=0Ml(ij)=1+2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1Xl=1cl;ijfl;ij;)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(l;ij;2l;ijgIR(l;ij);1i
PAGE 40

Theabovedensityisamixtureofunivariatenormaldensitiesandthecostofgeneratingsamplesfromthisdensityiscomparabletothatofgeneratingfromaunivariatenormaldistribution.InviewofEq. 3-19 ,onecanalsoeasilyseethatconditionalon,thediagonalentries kii(1kK,1ip)areaposterioriindependentandtheirconditionalposteriordensitygivenisasfollows, kiij;Y/expnnlog)]TJ /F4 11.955 Tf 5.48 -9.69 Td[( kii)]TJ /F4 11.955 Tf 13.15 8.09 Td[(n 2skii)]TJ /F4 11.955 Tf 5.48 -9.69 Td[( kii2)]TJ /F4 11.955 Tf 11.96 0 Td[(bki kiio;(3-22)wherebki=+nPj6=i Pr2#k rij!skij,for1i
PAGE 41

x sample(1;S;probs/fpt:t2Sg) returnx.Outputx endprocedureHowever,wehaveobservedinoursimulationsthatthedensityinEq. 3-22 hasahighpickatit'smode.Asaresult,onecansimplyapproximateitusingadegeneratedensitywithapointmassat)]TJ /F8 7.97 Tf 6.59 0 Td[(bki+p (bki)2+4n2skii 2nskii.Thisapproximationallowsfasterimplementationofouralgorithmwithoutmuchsacricingonit'saccuracy. 3.3.2SelectionofShrinkageSarametersLetandbegenericelementsofandandletandbetheircorrespondingshrinkageparameters.Selectingappropriatevaluesforthelatterisanimportanttask.InotherBayesiananalysisofhighdimensionalmodels,shrinkageparametersareusuallygeneratedbasedonanappropriatepriordistribution;see ParkandCasella ( 2008 ); Kyungetal. ( 2010 ); Hans ( 2009 ))forregressionanalysisand Wangetal. ( 2012 )forgraphicalmodels.Weassignindependentgammapriorsoneachshrinkageparameteror;specically,;Gamma(r;s),forsomehyper-parametersrands.Theamountofshrinkageimposedoneachelementandcanbecalculatedbyconsideringtheposteriordistributionofandgiven(;).Straightforwardalgebrashowsthatj(;)Gamma(r+0:5;0:52+s);j(;)Gamma(r+1;jj+s):Thus,weshrink,andonaveragebyEfj(;)g=r+0:5 0:52+sandEfj(;)g=r+1 jj+s,respectively.Thatis,ourapproachselectstheshrinkageparametersandbasedonthecurrentvaluesofandinawaythatlarger(smaller)entriesareregularizedmore(less).Theselectionofthehyper-parametersrandsisalsoanimportanttaskandcansignicantlyaectperformance.Basedonnumericalevidencefromsyntheticdata,wesetr=10)]TJ /F3 7.97 Tf 6.59 0 Td[(2ands=10)]TJ /F3 7.97 Tf 6.59 0 Td[(6;asimilarsuggestionisalsomadein Wangetal. ( 2012 ). 41

PAGE 42

Finally,theconstructionoftheGibbssamplerproceedsasfollows:matricesfkgKk=1areinitializedastheidentitymatrix,whilefrgfr:r2P(K);&jrj>1gatzero.Then,ineachiterationoftheMarkovChainMonteCarlochain,weupdatethevectorsijinEq. 3-4 andthediagonalentries kii,oneatatime,usingtheirfullconditionalposteriordensitiesgiveninEq. 3-21 andEq. 3-22 ,respectively.Algorithm 3.2 describesoneiterationoftheresultingGibbsSampler. Algorithm3.2(htp). 1: procedureBJNS.InputY;; 2: fori=1;:::;p)]TJ /F5 11.955 Tf 11.95 0 Td[(1do 3: forj=i+1;:::;pdo 4: forl=1;:::;2K)]TJ /F5 11.955 Tf 11.96 0 Td[(1do 5: Gamma(r+0:5;0:5(2l;ij+s)) 6: )]TJ /F8 7.97 Tf 25.77 8.19 Td[(n(aij+(ij)()]TJ /F13 5.978 Tf 5.76 0 Td[((ij)))]TJ /F13 5.978 Tf 5.76 0 Td[((ij))l n[(ij)(ij)]ll+ 7: 2 1 n[(ij)(ij)]ll+ 8: cl p 22expn2 22o 9: endfor 10: ij 0(2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1)1 11: l sample)]TJ /F5 11.955 Tf 5.48 -9.68 Td[(1;0;1;:::;2K)]TJ /F5 11.955 Tf 11.96 0 Td[(1;probs/1;c1;ij:::;c2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1;ij 12: ifl6=0then 13: l;ij N(;2) 14: endif 15: endfor 16: fork=1;:::;Kdo 17: Gamma(r+1;j kiij+s) 18: b +nPj6=i Pr2#k rij!skij 19: kii )]TJ /F8 7.97 Tf 6.59 0 Td[(b+p b2+4n2skii 2nskii 20: endfor 21: endfor 22: fork=1;:::;Kdo 42

PAGE 43

23: Gamma(r+1;j kppj+s) 24: b +nPj6=p Pr2#k rpj!skpj 25: Update kppusingAlgorithm 3.1 26: endfor 27: return;.Returnthesetofupdatedparameters 28: endprocedure Remark1. NotethatalthoughBJNSisacompletelyunsupervisedapproach,availablepriorknowledgeonsharedpatternsacrossthegroupscanbeeasilyincorporatedbyremov-ingredundnantcomponentsr.Further,priorinformationcouldbeincorporatedthroughappropriatespecicationoftheedgeselectionprobabilitiesq1;q2. Remark2. TheGibbssamplerdescribedinAlgorithm 3.2 doesnotinvolveanymatrixinversionwhichiscriticalforitscomputationaleciency. 3.3.3ProcedureforSparsitySelectionNotethattheconditionalposteriorprobabilitydensityoftheo-diagonalelementsofijisamixturedensitythatputsallofitsmassontheeventsfij:jijj1g,wherejijjisthenumberofnon-zerocoordinatesofij.ThispropertyofBJNSallowsformodelselection,inthesensethatineveryiterationoftheGibbssampleronecancheckwhetherij=0orwhichelementofij(therecouldbeatmostonenon-zeroelement)isnon-zero.Finally,intheendoftheprocedure,wechoosetheeventwiththehighestfrequencyduringsampling.Inaddition,credibleintervalscanbeconstructed,byusingtheempiricalquantilesofthevaluescorrespondingtothefrequencydistributionduringsampling. 43

PAGE 44

CHAPTER4THEORATICALGARANTEESFORBJNSHighdimensionalsparsityselectionconsistencyandconvergencerates.Next,weestablishtheoreticalpropertiesforBJNS.Letnr;0;r2SKk=1#kobethetruevaluesofthematricesinEq. 3-2 ,sothatk;0=Pr2#kr;0correspondstothetruedecompositionofeachprecisionmatrixk=1;:::;K.UsingasimilarorderingasinEq. 3-5 ,wedene0tobethevectorizedversionoftheo-diagonalelementsofthetruematricesnr;0;r2SKk=1#ko.Thefollowingassumptionsaremadetoobtaintheresults. Assumption1. (AccurateDiagonalestimates)Thereexistestimatesn^ kiio1kK1ip,suchthatforany>0,thereexistsaconstantC>0,suchthat max1ip1kKk^ kii)]TJ /F4 11.955 Tf 11.95 0 Td[( kiikC r logp n!;(4-1)withprobabilityatleast1)]TJ /F4 11.955 Tf 11.95 0 Td[(O(n)]TJ /F8 7.97 Tf 6.59 0 Td[().Notethatourmainobjectiveistoaccuratelyestimatetheo-diagonalentriesofallthematricespresentinthemodelinEq. 3-2 .Hence,ascommonlydoneforpseudo-Likelihoodbasedhigh-dimensionalconsistencyproofs Khareetal. ( 2015 ); Pengetal. ( 2009 ),weassumetheexistenceofaccurateestimatesforthediagonalelementsthroughAssumption 4-1 .OnewaytogettheaccurateestimatesofthediagonalentriesisdiscussedinLemma4of Khareetal. ( 2015 ).DenotetheresultingestimatesofthevectorsinEq. 3-10 andainEq. A-3 by^and^a,respectively.Wenowconsidertheaccuracyoftheestimatesoftheo-diagonalentriesobtainedafterrunningtheBJNSprocedurewiththediagonalentriesxedat^. Assumption2. dtq logp n!0;asn!1:Thisassumptionessentiallystatesthatthenumberofvariablesphastogrowslowerthane(n d2t).Similarassumptionshavebeenmadeinotherhighdimensionalcovarianceestimationmethodse.g.(Banerjee,etall.2014),(Banerjee,etall.2015),(Bickel,etal.2008),and(Xiang,etal.2015). 44

PAGE 45

Assumption3. Thereexistsc>0,independentofnandKsuchthatEexp)]TJ /F23 11.955 Tf 5.48 -9.69 Td[(0yki:exp(c0):Whilebuildingourmethod,weassumedthatthedataineachgroupcomesfromamultivariatenormaldistribution.TheaboveassumptionallowsfordeviationsfromGaussianity.Hence,Theorem 1 belowwillshowthattheBJNSprocedureisrobust(intermsofconsistency)undermodelmisspecication,aslongasthetailsaresub-Gaussian. Assumption4. (Boundedeigenvalues).Thereexists~"0>0,independentofnandK,suchthatforall1kK,~"0eigmin)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(keigmax)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(k1 ~"0:Thisisastandardassumptioninhighdimensionalanalysistoobtainconsistencyresults,seeforexample BuhlmannandVanDeGeer ( 2011 ).Henceforth,welet"0=c2~"0 K,a1="30 768K,a2=8c2 K"0,a3=16c2 . Assumption5. (SignalStrength).Letsnbethesmallestnon-zeroentry(inmagnitude)inthevector0.Weassume1 2logn+a2dtlogp a1ns2n!0.Thisisagainastandardassumption.Similarassumptionsontheappropriatesignalsizecanbefoundin Khareetal. ( 2015 ); Pengetal. ( 2009 ). Assumption6. (Decayrateoftheedgeprobabilities).Letq1=p)]TJ /F8 7.97 Tf 6.59 0 Td[(a2dt,andq2=p)]TJ /F8 7.97 Tf 6.59 0 Td[(a3n.Thiscanbeinterpretedasaprioripenalizingmatriceswithtoomanynon-zeroentries.Wehavefasterrateq2forthecaseofsuperdensematrices.Similarassumptionsarecommonintheliterature,seeforexample Narisettyetal. ( 2014 )and Caoetal. ( 2016 ).Wenowestablishthemainposteriorconsistencyresult.Inparticular,weshowthattheposteriormassassignedtothetruemodelconvergencetooneinprobability(underthetruemodel). 45

PAGE 46

Theorem1. (StrongSelectionConsistency)BasedonthejointposteriordistributiongiveninEq. 3-17 ,andunderAssumptions1-6,thefollowingholds, n2Stj^;YoP0)471(!1;asn!1:(4-2)OurnextresultestablishesestimationconsistencyoftheBJNSprocedurefor,andalsoprovidesacorrespondingrateofconvergence. Theorem2. (EstimationConsistencyRate)LetRnbethemaximumvalue(inmagnitude)inthevector0,andassumethatRncannotgrowataratefasterthanp nlogp.Then,basedonthejointposteriordistributiongiveninEq. 3-17 ,andunderAssumptions1-6,thereexistsalargeenoughconstantG(notdependingonn),suchthatthefollowingholds, E0"P k)]TJ /F6 11.955 Tf 11.95 0 Td[(0k2>Gr dtlogp nj^;Y!#!0asn!1:(4-3)Theproofsoftheaboveresultsareprovidedinsupplementalsection B . 46

PAGE 47

CHAPTER5NUMERICALANALYSISInthischapter,wepresentthreesimulationstudiestoevaluatetheperformanceofBJNS.Intherststudy,weillustratetheperformanceofBJNSintwoscenarioswithfourprecisionmatrices,each.Inthesecondsimulation,wecomparetheperformanceofBJNSwithothermethodologies,suchasGlasso,wheretheGraphicallassoby Friedmanetal. ( 2008 )isappliedtoeachgraphicalmodelseparately,jointestimationby Guoetal. ( 2011 ),denotedbyJEM-G,theGroupGraphicalLassodenotedbyGGLby Danaheretal. ( 2014 ),andtheJointStructuralEstimationMethoddenotedbyJSEM,by MaandMichailidis ( 2016 ).Inthethirdsimulation,wedemonstratethenumericalscalabilityofBJNS,whenthenumberofprecisionmatricesKisrelativelylarge.Throughout,foranyKprecisionmatricesfkgKk=1ofdimensionspp,wegeneratedataasfollows,yki:Np0;)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(k)]TJ /F3 7.97 Tf 6.59 0 Td[(1;i=1;:::;nk;k=1;:::;K:Thepositivedenitenessofkisguaranteedbysettingthediagonalelementstobejeigmin)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(kj+0:1.Thecovariancematrixkisdeterminedbyk=)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(k)]TJ /F3 7.97 Tf 6.59 0 Td[(1.WeassessthemodelperformanceusingSP,SE,andMCCasdenedinEq 2-25 .RecallthatBJNSestimatesthevectorsijsuchthattheydonothavemorethanonenon-zerocoordinate.Thatis,regardlessofthenumberofnetworksK,BJNSestimatesatmostp(p)]TJ /F5 11.955 Tf 12.7 0 Td[(1)=2o-diagonalparametersandenforcesatleast(2K)]TJ /F3 7.97 Tf 6.59 0 Td[(1)]TJ /F5 11.955 Tf 12.69 0 Td[(1)p(p)]TJ /F5 11.955 Tf 12.7 0 Td[(1)otherso-diagonalstobezero.ThenaldecisionforthesparsityofeachvectorijinEq. 3-4 ,ismadebasedonmajorityvoting.Foreveryexperiment,webaseourinferenceon2000samplesthataregeneratedfromtheMCMCchainafterremoving2000burn-insamples.Weensuretherobustnessofthepresentedresults,byrepeatingeachexperiment100timesandtakingtheaverageoftheaccuracymeasuresacrosstherepetitions.Finally,wewouldliketopointoutthatthecomputationalsoftwareforBJNSiswrittenusingtheRcppandRcppArmadillolibraries( EddelbuettelandSanderson ( 2014 ), Eddelbuettel 47

PAGE 48

andFrancois ( 2011 ))whichaddstootheradvantagesofBJNS,includingit'sfastconvergenceandthefactthatitdoesnotinvolvematrixinversion,tomakeitacomputationallyfriendlyalgorithm. 5.1Simulation1:fourgroups(K=4)WeconsidertwochallengingscenariostoexaminetheperformanceofBJNSforthecaseofsimultaneouslyestimatingfourinversecovariancematrices(K=4). 1. Fourprecisionmatrices1,2,3,and4,withdierentdegreesofsharedstructures.Welettherstmatrix1tobeanAR(2)modelwith!1jj=1,forj=1;:::;p;!1jj+1=!1j+1j=0:5,forj=1;:::;p)]TJ /F5 11.955 Tf 12.05 0 Td[(1;and!1jj+2=!1j+2j=0:25,forj=1;:::;p)]TJ /F5 11.955 Tf 12.04 0 Td[(2.Toconstruct2werandomlyreplacep 4nonzeroedgesfrom1withzerosandreplacep 4zeroedges,atrandom,withnumbersgeneratedfrom[)]TJ /F5 11.955 Tf 9.3 0 Td[(0:6;)]TJ /F5 11.955 Tf 9.3 0 Td[(0:4][[0:4;0:6].Weconstruct3byrandomlyremovingp 2edgessharedbetween1and2andthenusing[)]TJ /F5 11.955 Tf 9.3 0 Td[(0:6;)]TJ /F5 11.955 Tf 9.3 0 Td[(0:4][[0:4;0:6],werandomlyaddp 2otheredgesthatarenotpresentinnonof1or2.Finally,weconstruct4byremovingtheremaining2p)]TJ /F5 11.955 Tf 12.34 0 Td[(3)]TJ /F3 7.97 Tf 13.54 5.25 Td[(3p 4edgesthatarecommonin1and2andrandomlyadd2p)]TJ /F5 11.955 Tf 12.2 0 Td[(3)]TJ /F3 7.97 Tf 13.4 5.25 Td[(3p 4edgesthatarenotpresentinanyoftheothergraphs.Theresultingmatrix4hasnothingincommonwiththeotherprecisionmatrices.Therefore,thetruerelationbetweenthefournetworksisasfollows, 1=1+12+1232=2+12+1233=3+1234=4;(5-1)where,1,2,3,and4accountfortheedgesthatareuniquetotheircorrespondinggroupsand12and123containstheedgesthatarecommonbetweenthefourgroups.Aheatmapplotofthetrueprecisionmatriceswithp=50isgivengure 1 . 2. Inmostrealdataapplications,theunderlyingnetworkstendtofollowsparsitypatternsthatarecompletelyrandomanddonotnecessarilyfollowacertainstructure.Forthisreason,wewouldliketoexaminetheperformanceofBJNSunderunstructuredsettings.Wewillspecicallyconsiderestimatingfourprecisionmatriceswithcompleterandomsparsitypatternswithsignalsgeneratedfrom[)]TJ /F5 11.955 Tf 9.3 0 Td[(0:6;)]TJ /F5 11.955 Tf 9.3 0 Td[(0:4][[0:4;0:6].Wetakethelevelofsparsityineachmatrixtobe95%,halfofwhichisuniquetothematricesandtheotherhalfissharedbetweenallfourofthem.Thetruerelationshipbetweenthenetworksisasfollows, k=k+1234;k=1;2;3;4:(5-2)where,1,2,3,and4accountfortheedgesthatareuniquetotheircorrespondinggroupsand1234containstheedgesthatarecommonbetweenallofthefourgroups. 48

PAGE 49

Figure5-1. Heatmapsoftheprecisionmatricesofthefourgroups Foreachoftheabovesettings,werunthefollowingcombinationsofpandn: p=200,andn=50;100;150;200;250;and;300, p=500,andn=300;400;and;500.Theestimationsarebasedonthefulldecompositionmodel,whichforthecaseofK=4isgivenas, 1=1+12+13+14+123+124+134+1234;2=2+12+23+24+123+124+234+1234;3=3+13+23+34+123+134+234+1234;4=4+14+24+34+124+134+234+1234:(5-3)Tables 5-1 and 5-2 summarizetheaverageoftheaccuracymeasuresforbothsettings,across100repetitions.Ascanbeseen,thevaluesoftheaccuracymeasurestendtobemuchhigherforthejointeects(namely12and123intable 5-1 and1234intable 5-2 ),whichimplies 49

PAGE 50

thatBJNSisborrowingstrengthacrossthedistinctsamples,toprovidemorerobustestimatesofthejointedges.Inaddition,highvaluesofspecicity,regardlessofthesamplesize,showsasurprisinglylowfalsepositiverate.AnotherkeystrengthofBJNSisit'suniquecapabilityinselectingtherightmodelfromtheabovefullrepresentation.AsdescribedintheGibbssamplerinsection4.1,modelselectiontakesplaceatthelevelofijs(for1i
PAGE 51

Table5-1. Summaryresultsforsimulationi p=200p=200p=200p=200p=200p=200p=500p=500p=500n=50n=100n=150n=200n=250n=300n=300n=400n=500 MC%60(0.020)79(0.016)86(0.012)88(0.011)89(0.010)89(0.009)95(0.004)96(0.005)96(0.004)1SP%99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%67(0.021)90(0.014)97(0.009)99(0.004)100(0.002)100(0.001)100(0.001)100(0.000)100(0.000)MC%51(0.021)69(0.016)78(0.013)81(0.014)80(0.011)82(0.012)86(0.007)87(0.006)88(0.006)2SP%99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%59(0.024)82(0.018)92(0.014)95(0.011)95(0.010)97(0.010)97(0.005)98(0.004)99(0.003)MC%43(0.021)64(0.019)73(0.015)79(0.015)80(0.014)80(0.014)84(0.007)86(0.007)86(0.007)3SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%51(0.024)78(0.022)89(0.016)95(0.015)96(0.011)96(0.010)97(0.006)98(0.005)99(0.004)MC%19(0.020)47(0.020)54(0.020)72(0.015)75(0.016)81(0.012)80(0.009)85(0.006)88(0.007)4SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%21(0.020)53(0.024)61(0.023)84(0.016)87(0.014)95(0.010)89(0.009)96(0.005)99(0.003)MC%30(0.048)52(0.047)66(0.051)75(0.043)76(0.040)81(0.03787(0.020)91(0.017)93(0.015)1SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%33(0.056)57(0.058)71(0.067)82(0.047)85(0.044)89(0.040)86(0.028)91(0.021)94(0.020)MC%31(0.051)56(0.043)71(0.035)75(0.037)74(0.033)76(0.035)78(0.021)79(0.018)79(0.019)2SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%37(0.067)69(0.067)87(0.041)92(0.040)94(0.038)97(0.024)95(0.020)98(0.013)99(0.009)MC%25(0.041)58(0.040)73(0.033)82(0.025)84(0.023)84(0.022)85(0.013)87(0.013)88(0.012)3SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%23(0.038)60(0.046)79(0.039)92(0.028)94(0.024)96(0.022)94(0.013)98(0.009)99(0.007)MC%24(0.027)56(0.023)64(0.021)81(0.017)84(0.013)90(0.011)87(0.008)93(0.006)95(0.005)4SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%14(0.017)42(0.024)52(0.022)76(0.020)80(0.017)90(0.015)84(0.010)93(0.008)97(0.005)MC%36(0.041)57(0.034)70(0.021)78(0.029)80(0.030)83(0.026)86(0.015)90(0.014)93(0.012)12SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%34(0.043)59(0.040)74(0.038)82(0.036)83(0.039)88(0.029)87(0.019)92(0.014)95(0.014)MC%50(0.029)71(0.022)80(0.020)88(0.018)89(0.014)90(0.013)92(0.008)95(0.006)96(0.005)123SP%100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%34(0.028)60(0.028)72(0.029)83(0.027)85(0.020)88(0.022)88(0.013)93(0.009)95(0.009) 51

PAGE 52

Table5-2. Summaryresultsforsimulationii p=200p=200p=200p=200p=200p=200p=500p=500p=500n=50n=100n=150n=200n=250n=300n=300n=400n=500 MC%17(0.009)36(0.012)46(0.012)56(0.011)60(0.011)68(0.009)61(0.004)67(0.004)72(0.004)1SP%88(0.004)94(0.003)95(0.002)95(0.002)96(0.002)96(0.00299(0.000)99(0.000)99(0.000)SE%39(0.015)53(0.013)63(0.013)75(0.013)81(0.011)90(0.009)47(0.004)56(0.004)63(0.004)MC%18(0.011)36(0.011)49(0.013)55(0.010)59(0.013)66(0.011)59(0.004)65(0.004)70(0.004)2SP%88(0.004)94(0.003)95(0.002)95(0.002)96(0.002)96(0.00299(0.000)99(0.000)99(0.000)SE%39(0.016)53(0.013)67(0.013)74(0.011)78(0.014)85(0.011)46(0.004)54(0.004)61(0.004)MC%18(0.009)39(0.011)48(0.012)56(0.011)63(0.010)64()0.01061(0.005)67(0.005)72(0.004)3SP%88(0.004)94(0.003)95(0.002)95(0.002)96(0.002)96(0.00299(0.000)99(0.000)99(0.000)SE%39(0.014)57(0.013)66(0.014)75(0.013)84(0.011)85(0.010)48(0.005)56(0.004)63(0.004)MC%17(0.009)36(0.011)50(0.013)58(0.010)61(0.012)67(0.009)61(0.005)68(0.008)72(0.004)4SP%88(0.004)94(0.002)95(0.002)95(0.002)96(0.002)96(0.00199(0.000)99(0.000)99(0.000)SE%38(0.013)53(0.014)69(0.013)78(0.010)82(0.012)89(0.010)47(0.005)56(0.004)63(0.004)MC%7(0.013)20(0.020)32(0.021)47(0.023)55(0.017)67(0.015)30(0.008)40(0.008)48(0.008)1SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%9(0.012)17(0.018)26(0.019)41(0.022)49(0.019)64(0.019)11(0.005)18(0.006)26(0.007)MC%7(0.012)20(0.018)37(0.021)46(0.019)52(0.019)65(0.020)29(0.008)38(0.008)46(0.008)2SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%9(0.011)17(0.015)31(0.019)40(0.020)46(0.020)62(0.021)11(0.004)17(0.006)24(0.007)MC%8(0.013)24(0.021)37(0.021)47(0.021)58(0.019)62(0.019)30(0.008)40(0.009)49(0.006)3SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%9(0.013)20(0.019)29(0.021)40(0.020)53(0.023)59(0.022)11(0.005)18(0.006)26(0.006)MC%7(0.014)21(0.019)38(0.024)50(0.022)56(0.022)66(0.017)30(0.009)40(0.008)49(0.008)4SP%98(0.001)99(0.001)99(0.001)99(0.001)99(0.001)99(0.001)100(0.000)100(0.000)100(0.000)SE%9(0.013)18(0.017)32(0.020)43(0.023)50(0.022)63(0.021)11(0.005)18(0.006)26(0.007)MC%21(0.020)48(0.019)63(0.022)73(0.015)76(0.014)85(0.011)68(0.007)76(0.006)82(0.005)1234SP%99(0.001)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)100(0.000)SE%16(0.015)36(0.019)52(0.026)66(0.019)73(0.019)82(0.016)49(0.009)61(0.009)69(0.008) 52

PAGE 53

OnecanalsousethemodelselectionpropertiesofBJNStostudytheconvergencepropertiesofourMCMC.Wedosobytrackingtheratioofcorrectlyselectedijs,whichwedenoteby.By"correctlyselected\wemeanthesparsitypatternofij(thatithasatmostonenon-zerocoordinate),iscorrectlyidentiedbyBJNS.Formally,welet=#ijsthatarecorrectlyselected p(p)]TJ /F3 7.97 Tf 6.59 0 Td[(1) 2Inadditiontoaccuracyassessment,helpsstudyingthenumberofiterationsthatonaverageittakesfortheGibbssamplertoconverge.Figure 5-2 describesthetraceplotandthehistogramofduringthe4000iterationsoftheabovetwosimulationsforthep=200,n=300settings.TheMCMCtraceplotsofshowthattheGibbssamplerconvergesfairlyquickly.Moreover,thehistogramsofindicatehighproportionofcorrectlyselectedijswhichisourmaingoalinthispaper. Figure5-2. ConvergenceandthedistributionofduringtheMCMCsamplings 5.2SimulationStudy2:ComparisonwithExistingMethodsInthissection,wecomparetheperformanceofBJNSwithGlasso,JEM-G,GGLandJSEM,intwosettingswith6networks.WealsodemonstrateastrategytoscaleupBJNS 53

PAGE 54

forthecaseofevenlargernumberofnetworks.TheproposedstrategyresultsinasignicantboostinbothaccuracyandthecomputationalcostofBJNS.IntheendofthissectionwewillalsoillustratethecomputationalcostofBJNSwhenestimating6networks,formoderatetolargenumberofvariablesp. 5.2.1ComparisonwithExistingMethodsWeconsiderascenariowithK=6graphseachconsistsofp=200variables(seeFigure 5-3 ),wherewerstgeneratetheadjacencymatricescorrespondingtovedistinctp-dimensionalnetworkssothattheadjacencymatricesineachcolumnoftheplot 5-3 arethesame.Next,wereplacetheconnectivitystructureofthebottomrightdiagonalblockofsizep=2byp=2ineachadjacencymatrixwiththatofanothertwodistinctp=2-dimensionalnetworks,sothatgraphicalmodelsineachcolumnexhibitthesameconnectivitypatternexceptinthebottomrightdiagonalblockoftheiradjacencymatrices.Notethatbyreplacingtheconnectivitystructureamongthesecondhalfofthenodes,therelationshipsbetweenthersthalfandthesecondhalfofthenodesarealsoaltered.Insummary,thesesparsitypatternsillustratehowdierentsubsetsoftheedgesetsacrossmultiplegraphicalmodelscanbesimilar,aswellasexhibitdierencesintheirtopologies.Allnetworksaresettobe92%sparse,withcompleterandompatterns,andtheproportionofsharednon-edgesamongallgraphicalmodelsisabout60%.Giventheadjacencymatrices,wethenconstructedtheinversecovariancematriceswiththenonzeroo-diagonalentriesineachkbeinguniformlygeneratedfromthe[)]TJ /F5 11.955 Tf 9.3 0 Td[(0:6;)]TJ /F5 11.955 Tf 9.3 0 Td[(0:4][[0:4;0:6]interval.ToimplementJESM,wesupplythesparsitypatternsdenedaccordingtothepatterningure 5-3 .Wealsostudytheeectofmiss-specication,denotedby,inthesparsitypatternsbyaddinganadditional=4%,sparsitytothenetworks.Adding=4%miss-specicationcorrespondstohaving60%oftheinformationinthesparsitypatternsbeingcorrectforJSEM.Ateachlevelofpatternmiss-specication,wegeneratednk=200;300independentsamplesforeachk=1;:::;Kandexaminedthenitesampleperformanceofdierentmethodsinidentifyingthetruegraphsandestimatingtheprecisionmatricesattheoptimalchoiceof 54

PAGE 55

tuningparameters.Table 5-3 showthedeviancemeasuresbetweentheestimatedandthetrueprecisionmatricesbasedon50replicationsforvaryinglevelsofsamplesizeandpatternmiss-specication.When=0(nomiss-specications),JSEM,whichbenetsfromknowing100%oftheinformationaboutthesparsitypatterns,achievesagoodbalancebetweenfalsepositiveandfalsenegativeandyieldsthehighestMCCscore.BJNSisalsoverycompetitiveandit'soverallperformanceissignicantlybetterthanalltheunsupervisedmethods.GlassoandGGLtendtoperformwellincontrollingfalsenegativesandJEM-GisconstantlyamongthebesttwomodelsintermsofSPscore.With=0:04andsparsitylevelof12%,naturallytheoverallperformanceofallmethodsdecrease.However,JSEMsuersmoreasitonlyhas60%ofalltheinformationinthesparsitypatterns.BJNSthoughprovesit'sadvantagebyscoringhighestMCCscoreandachievingbestoverallperformance.Inthiscase,GlassoalsoappearstobeverycompetitiveinoverallperformancewithconstantlyachievinghighestSEscore. Figure5-3. imageplotsoftherandomsparseadjacencymatricesfromallgraphicalmodels.Graphsinthesamerowsharethesamesparsitypatternatthebottomrightblock,whereasgraphsinthesamecolumnsharethesamepatternatremaininglocations 5.2.2AComputationalStrategytoGainEciencyandReduceCostAspresentedinAlgorithm 3.2 ,theGibbssamplerupdatesallthep(p)]TJ /F5 11.955 Tf 12.35 0 Td[(1)=2vectorsijbasedontheirfullconditionaldistributions.Althoughtheconditionalposteriordistributionof 55

PAGE 56

Table5-3. Summaryofmodelcomparisonsbasedonaverageofdeviancemeasuresacrossthe6networks,forthecaseofrandomsparsitypatterns GlassoJEM-GGGLJSEMBJNS n=200 MC%47(0.009)50(0.010)47(0.009)61(0.009)57(0.010)0SP%94(0.003)97(0.001)93(0.003)99(0.001)97(0.001)SE%60(0.010)48(0.010)61(0.010)46(0.009)59(0.010) MC%40(0.008)35(0.010)35(0.008)32(0.009)40(0.010)0.04SP%93(0.003)97(0.001)91(0.003)99(0.001)96(0.002)SE%46(0.009)30(0.008)46(0.008)18(0.008)39(0.009) n=300 MC%54(0.009)60(0.010)54(0.008)73(0.008)70(0.010)0SP%93(0.003)97(0.001)92(0.002)99(0.001)97(0.001)SE%72(0.009)61(0.010)75(0.009)63(0.008)77(0.010) MC%47(0.009)43(0.009)42(0.008)42(0.008)51(0.010)0.04SP%93(0.003)97(0.001)90(0.003)99(0.001)95(0.002)SE%58(0.009)39(0.007)58(0.008)29(0.007)53(0.010) ijsisamixtureofunivariatenormaldensities,all2Kmixtureprobabilitiescl;ijgiveninEq. 3-20 stillneedtobecalculated,whichinturninvolvesmatrix-vectormultiplications.HencewithincreasingK,thecomputationcomplexityofthefulldecompositioninEq. 3-2 growsquickly.Next,wediscussastrategythatstartsbyexaminingallpairwisedecompositionstoidentifyinactivepairwisecomponentsandthehighermatricesrofsuchpairwisecomponents.Intherststep,)]TJ /F8 7.97 Tf 5.48 -4.38 Td[(K2pairwisejointmodelsareconsidered;namely,foranypair1k1
PAGE 57

matrices.Wethenfurtherreducethemodelifanymatrixcomponentseemtobeinactive(hassignicantlysmallernumberofedges)andnallyrunBJNSonelasttimewiththeresultingreducedmodel.TheproposedpurelycomputationalstrategyisillustratedonthesettinggiveninFigure 5-3 ,whichinvolvesK=6groupswithp=200variablesandnk=200&300samplespergroups.Notethatthefulldecompositionwouldinvolver-tupleinteractioncomponents,r=2;;6,whichrenderscomputationsexpensive,sinceeachvectorijwouldhavelength2K)]TJ /F5 11.955 Tf 12.13 0 Td[(1=63.Butinrealitymostofther-tuplesareinactive.Forinstance,inthecaseof 5-3 ,thetruematrixdecompositionofthenetworksisasfollows: 1=1+12+135;2=2+12+246;3=3+34+135;4=4+34+246;5=5+56+135;6=6+56+246:(5-6)Therefore,only5outofthe63componentsinthefullmodelarenon-zero(active).HenceitisencouragingtotryandlearnthetruedecompositionasaninitializationstageandsubsequentlyrunBJNSonareducedlibrarywhich,asweshallillustrate,resultsinsignicantimprovementsoftheresults.Forthecurrentsetting,followingtheabovestrategy,westartbyrststudyingthe)]TJ /F3 7.97 Tf 5.48 -4.38 Td[(62pairwiseinteractioncomponents(onecandothisstepusingparallelcomputing).Figure 5-4 showsthenumberofedgesinthepairwisematricesk1k2,1k1
PAGE 58

Figure5-4. barplotoftheedgecountofthepairwisejointmatrices Doingsoresultsinareducedmodelwithcomponents,12,13,15,24,26,34,35,46,56,135,and246.Thus,inthesecondstep,werunBJNSwithadecompositionthatisbasedontheabovematrices;theedgecountofthesematricesareshowninplot 5-5 . Figure5-5. barplotoftheedgecountofthematricesinthesecondstep Fromtheaboveplot,itisclearthatmatrices13,15,24,26,35,and46areredundantandshouldberemovedfromthemodel.ThisfurthermodelreductionachievesthetruedecompositiongiveninEq. 5-6 .TheresultsofBJNSrunningontheresultingreducedmodelcanbereadofromtheTable 5-4 .Ascanbeseen,BJNSoutperformsJSEM,whichhasbeensuppliedwithcompleteinformationaboutthesparsitypatterns. 58

PAGE 59

Table5-4. ComparionsbetweenJSEMandBJNSwhenemployingthestepwisecomputationalstrategy;theresultsarebasedon50replications JSEMBJNSJSEMBJNSn=200n=200n=300n=300 MC%61(0.009)69(0.011)73(0.008)83(0.007)SP%99(0.001)99(0.001)99(0.001)100(0.001)SE%46(0.009)56(0.010)63(0.008)76(0.010) 5.2.3ComputationalCostofBJNSLastly,aspresentedinTable 5-5 ,weinvestigatethecomputationalcostassociatedwiththeabovestrategy,acrossvaryingvaluesofpandn.Eachexperimentwasrepeated5timesandallcomputationsweredonesequentiallyusingoneprocessor(CPU).Notethataround60%ofthetimeineachexperimentwasspentontherststepwhichisinvestigatingallthe)]TJ /F3 7.97 Tf 5.48 -4.37 Td[(62pairwisemodels.Since,thepairwisemodelsareranindependently,onecanuseparallelcomputingandreducethecomputationaltimeintable 5-5 potentiallyby50%.Finally,ascanbeseeninthelastrowofthetable,thememoryusageofBJNSisnotnecessarilylargeandthatisduetothefactthatthealgorithmdoesnotinvolveanymatrixinversionorgenerationfrommultivariatedistributions. Table5-5. AccuracyandcostofBJNSforvaryingvaluesofp p=200p=500p=700p=1000n=300n=750n=1050n=1500 MC%83848585SP%100100100100SE%76787878hours2.9h38.9h83.3h214.5hGigaBytes0.25gb0.5gb0.6gb0.9gb 5.3MetabolomicsDataInthissection,weemploytheproposedmethodologytoobtainnetworksacrossfourgroupsofpatientsthatparticipatedintheIntegrativeHumanMicrobiomeProject.ThedataweredownloadedfromtheMetabolomicsWorkbenchwww.metabolomicsworkbench.org(StudyIDST000923)andcorrespondtomeasurementsof428primaryandsecondarymetabolitesandlipidsfromstoolsamplesof542subjects,partitionedinthefollowinggroups:inammatory 59

PAGE 60

boweldisease(IBD)patients(malesn1=202andfemalesn2=208)andnon-IBDcontrols(malesn3=72andfemalesn4=70),Groups1-4,respectively.Sincetherearetwofactorsinthestudydesign,thefollowingmodelwasttedtothedata.1=1+12+13+1234;3=3+13+34+1234;2=2+12+24+1234;4=4+24+34+1234:TheresultsareshowninthenextTable(set1:289lipidsinredandset2:139metabolitesinblue,set12:interactionedgesbetweenset1andset2),forboththenalestimatesoftheprecisionmatrices,aswellasthecomponentsintheproposeddecomposition.Itisinterestingtonotethatalargenumberofedgesaresharedacrossallgroups,indicatingcommonpatterns.Further,thecomponentsharedbetweenmaleandfemaleIBDpatientshasafairlylargenumberofedges,indicatingthatthediseasestatusexhibitscommonalitiesacrossbothmalesandfemales.Figures 5-6 and 5-7 shownetworkandheatmapsmapofthecommoncomponentsharedacrossallgroups.Theprimaryandsecondarymetabolitesaredepictedinred,whilethelipidsinblue.Notsurprisingly,primarymetabolites(thoseinvolvedincellulargrowth,developmentandreproduction)formafairlystronglyconnectednetwork.Ontheotherhand,theconnectivitybetweenlipids(whosefunctionsincludestoringenergy,signalingandactingasstructuralcomponentsofcellmembranes)tothemetabolitesisnotparticularlystrong.Ontheotherhand,dierentfairlystronglyconnectedsubnetworksamongstlipidsarepresent,includingdicylglycerols(DAG)withtricylglycerols(TAG)thataremainconstituentsofanimalandvegetablefat(upperrightcorneroftheplot)andvariousphospholipids(upperleftcorneroftheplot).Ingeneral,theresultsrevealinterestingpatternsthatcanbeusedtounderstandprogressionofIBDdisease. 60

PAGE 61

Table5-6. Numberofedgesineachmatrix 12341234121324341234 set194193079779947291813161818846652set2264269243244252134141613214set12154155124124101155341313997 Figure5-6. Networkplotoftheedgessharedbetweenthefourgroups(1234);namesofthemetabolitesandthelipidsappearinredandblue,respectively 61

PAGE 62

Figure5-7. Heatmapplotoftheedgessharedbetweenthefourgroups(1234) 62

PAGE 63

A B C DFigure5-8. A:12(MaleandFemaleIBD),B:13(MaleIBDandnon-IBD),C:24(FemaleIBDandnon-IBD),D:34(MaleandFemalenon-IBD) 63

PAGE 64

A B C DFigure5-9. A:1(MaleIBD),B:2(FemaleIBD),C:3(Malenon-IBD),D:4(Femalenon-IBD) 64

PAGE 65

APPENDIXATHESTRUCTUREOFThematrixisasfollows =0BBBBBBBBBBBBBBBBB@B1+B2+B3B2+B3B1+B3B1+B2B3B2B1B2+B3B2+B3B3B2B3B20B1+B3B3B1+B3B1B30B1B1+B2B2B1B1+B20B2B1B3B3B30B300B2B20B20B20B10B1B100B11CCCCCCCCCCCCCCCCCA;(A-1)where,Bksarep(p)]TJ /F3 7.97 Tf 6.59 0 Td[(1) 2p(p)]TJ /F3 7.97 Tf 6.59 0 Td[(1) 2symmetricmatrices.TounderstandthestructureofthematricesBks,weindexit'srowsandcolumnsas(12;13;:::;p)]TJ /F5 11.955 Tf 11.96 0 Td[(1p).Then, Bk(ab;cd)=8>>>>>>><>>>>>>>:skaa+skbbifa=b&c=d;skacifb=d&a6=c;skbdifa=c&b6=d;0ifa6=b&c6=d;for1a
PAGE 66

Forfurtherillustration,whenp=5,Bkisasfollows,0BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB@sk11+sk22sk23sk24sk25sk13sk14sk15000sk23sk11+sk33sk34sk35sk1200sk14sk150sk24sk34sk11+sk44sk450sk120sk130sk15sk25sk35sk45sk11+sk5500sk120sk13sk14sk13sk1200sk22+sk33sk34sk35sk24sk250sk140sk120sk34sk22+sk44sk45sk230sk25sk1500sk12sk35sk45sk22+sk550sk23sk240sk14sk130sk24sk230sk33+sk44sk45sk350sk150sk13sk250sk23sk45sk33+sk55sk3400sk15sk140sk25sk24sk35sk34sk44+sk551CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCA;Also,thevectoraforthespecialcaseofK=3,isgivenas a=0BBBBBBBBBBBBBBBBB@a1+a2+a3a2+a3a1+a3a1+a2a3a2a11CCCCCCCCCCCCCCCCCA;(A-3)withakbeingthefollowingvector,ak=(sk12( k11+ k22);:::;sk1p( k11+ kpp);:::;skp)]TJ /F3 7.97 Tf 6.59 0 Td[(1p( kp)]TJ /F3 7.97 Tf 6.59 0 Td[(1p)]TJ /F3 7.97 Tf 6.59 0 Td[(1+ kpp))0;k=1;2;3: 66

PAGE 67

APPENDIXBPROOFSOFTHEOREMS1AND2ByAssumption3andHanson-Wrightinequalityfrom Rudelsonetal. ( 2013 ),thereexistsac>0,independentofnandK,suchthatP(maxi;j;kkskij)]TJ /F4 11.955 Tf 11.95 0 Td[(kijk
PAGE 68

Letx2Rp(p+1) 2bethesymmetricversionofyobtainedbyremovingall!k;ijwithi>j.Moreprecisely,x=)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(!k11;!k12;!k22;:::;!k1p;:::;!kpp0:Let~Pbethep2p(p+1) 2matrixsuchthateveryentryof~Piseitherzeroorone,exactlyoneentryineachrowof~Pisequalto1,andy=~Px.Now,dene! k=)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(!k12;!k13;:::;!kp)]TJ /F3 7.97 Tf 6.59 0 Td[(1p0and k=)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(!k11;!k22;:::;!kpp0andlet~Qbethep(p+1) 2p(p+1) 2permutationmatrixforwhichx=Q0B@! k k1CA:Let~kbeap2p2blockdiagonalmatrixwithpdiagonalblocks,theithblockisequalto~k;i:=PiSkPi0.Itfollowsthattrh)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(k2Ski=pXi=1k:i0Skk:i=pXi=1k:i0Pi0PiSkPi0Pik:i=pXi=1k:i0Pi0PiSkPi0Pik:i=y0~ky=x0~P0~k~Px=! k0; 0kQ0~P0~k~PQ0B@! k k1CA:TherealsoexistappropriatematricesAkandDksuchthattrh)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(k2Ski=! k0; 0k0B@BkAkAkDk1CA0B@! k k1CA;therefore,wemusthaveQ0~P0~k~PQ=0B@BkAkAkDk1CA:Now,since~PQisorthogonal,weconcludethattheeigenvaluesof~kand0B@BkAkAkDk1CAarethesame.Moreover,thediagonalblocksof~kallhavethesameeigenvaluesasSk,andBk 68

PAGE 69

canberegardedasaprincipalsubmatrixof~k,hence,wehavethateigmin)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(Sk=eigmin~keigmin)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(Bkeigmax)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(Bkeigmax~k=eigmax)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(Sk: Lemma2. Let`2Lbeanysparsitypattern/modelwithd`<"0 4cq n logp,thenthesubmatrix``of,obtainedbytakingoutalltherowsandcolumnscorrespondingtothezerocoordinatesin2M`,ispositivedenite.Specically, 3K"0 4eigmin(``)eigmax(``)3K 2"0;8`2L:(B-4) Proof. Foreaseofexposition,weshowthisresultholdsforthecaseofK=3.Theproofforageneralcasewillfollowexactlyfromthesameargument.Letxbead`1vectorinRd`andpartitionxasx=x0 `;123;x0 `;23;x0 `;13;x0 `;12;x0 `;3;x0 `;2;x0 `;1;then,bymakingsimilarpartitionsoneachblockof``(seeinEq. A-1 ),wehavethatx0``x=x0 `;123;x0 `;13;x0 `;12;x0 `;10B1x0 `;123;x0 `;13;x0 `;12;x0 `;1+x0 `;123;x0 `;23;x0 `;12;x0 `;20B2x0 `;123;x0 `;23;x0 `;12;x0 `;2+x0 `;123;x0 `;23;x0 `;13;x0 `;30B3x0 `;123;x0 `;23;x0 `;13;x0 `;3;where,B1=0BBBBBBBB@B1 `;123 `;123B1 `;123 `;13B1 `;123 `;12B1 `;123 `;1B1 `;13 `;123B1 `;13 `;13B1 `;13 `;12B1 `;13 `;1B1 `;12 `;123B1 `;12 `;13B1 `;12 `;12B1 `;12 `;1B1 `;1 `;123B1 `;1 `;13B1 `;1 `;12B1 `;1 `;11CCCCCCCCA:LetBk;0denotethepopulationversionofBK.Since,wearerestrictedtoC1;n\C2;n,kBK)]TJ /F6 11.955 Tf 11.96 0 Td[(Bk;0kcdkq logp n,hence 69

PAGE 70

eigmin()``KXk=1eigmin)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(Bk=KXk=1infjxj=1x0BkxKXk=1infjxj=1x0Bk;0x)]TJ /F5 11.955 Tf 15.42 0 Td[(infjxj=1x0)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(Bk)]TJ /F6 11.955 Tf 11.95 0 Td[(Bk;0xKXk=1infjxj=1x0Bk;0x)]TJ /F8 7.97 Tf 16.74 14.94 Td[(KXk=1kBk)]TJ /F6 11.955 Tf 11.96 0 Td[(Bk;0k2KXk=1infjxj=1x0Bk;0x)]TJ /F4 11.955 Tf 11.96 0 Td[(Kd`cr logp nhence,byLemma 2 ,eigmin()``K"0)]TJ /F4 11.955 Tf 11.96 0 Td[(Kcd`r logp nK "0)]TJ /F4 11.955 Tf 11.95 0 Td[(cnr logp n!=3K"0 4:Similarlyonecanshowthateigmax(``)3K 2"0: ByLemma 2 ,thevalueofthethresholdnwhichweusedinbuildingourhierarchicalpriorinEq. 3-7 isgivenasn="0 4cq n logp.HencebyAssumption 2 ,wecanwritedtn,foranysucientlylargen. Lemma3. Let,,andabeaccordingtoEq. A-1 ,Eq. A-3 ,andlet0bethetruevalueofinEq. 3-5 .Thenforlargeenoughn,thereexistsaconstantc0suchthat k0+^akmaxc0r logp n:(B-5) Proof. Notethatbythetriangularinequality, k0+^akmaxk0+akmax+k^a)]TJ /F23 11.955 Tf 11.96 0 Td[(akmax;(B-6)where,^aistheestimateofaprovidedbyAssumption1. 70

PAGE 71

Now,InviewofEq. A-1 ,Eq. A-3 ,andEq. 3-5 ,onecaneasilycheckthat0+a=0BBBBBBBBBBBBBBBBB@B1! 1;0+a1+B2! 2;0+a2+B3! 3;0+a3B2! 2;0+a2+B3! 3;0+a3B1! 1;0+a1+B3! 3;0+a3B1! 1;0+a1+B2! 2;0+a2B3! 3;0+a3B2! 2;0+a2B1! 1;0+a11CCCCCCCCCCCCCCCCCA;where! 1;0=)]TJ /F23 11.955 Tf 5.47 -9.68 Td[( 1;0+ 12;0+ 13;0+ 123;0,! 2;0=)]TJ /F23 11.955 Tf 5.48 -9.68 Td[( 2;0+ 12;0+ 23;0+ 123;0,and! 3;0=)]TJ /F23 11.955 Tf 5.48 -9.68 Td[( 3;0+ 13;0+ 23;0+ 123;0.Furthermore,Bk! k;0+ak=0BBBBBBB@k;0:10Sk:2+k;0:20Sk:1k;0:10Sk:3+k;0:30Sk:1...k;0:p)]TJ /F3 7.97 Tf 6.59 0 Td[(10Sk:p+k;0:p0Sk:p)]TJ /F3 7.97 Tf 6.58 0 Td[(11CCCCCCCA;k=1;2;3:Now,werewrite0+aas,0+a=0BBBBBBBBBBBBBBBBB@Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 20Ip(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 20Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 2Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2000Ip(p)]TJ /F13 5.978 Tf 5.76 0 Td[(1) 2000Ip(p)]TJ /F13 5.978 Tf 5.75 0 Td[(1) 21CCCCCCCCCCCCCCCCCA0BBBB@B3! 3;0+a3B2! 2;0+a2B1! 1;0+a11CCCCA:Thenormofthematrixintherighthandsideoftheaboveequationisequaltop K(2K)]TJ /F5 11.955 Tf 11.95 0 Td[(1),hence,byrestrictingtotheeventC1;n\C2;n,wehavethat 71

PAGE 72

k0+akmaxvuut KXk=1kBk! k;0+akk2maxvuut Kmax1kK1i
PAGE 73

PR(`;t)=Pn`j^;Yo Pntj^;Yo=hqd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`gi hqdt1(1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(dtIfdtg+qdt1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(dtIfdtgij``j1 2 j`tj1 2j(n+)ttj1 2 j(n+)``j1 2expnn2 2^a`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``^a`o expn2 2^at(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1tt^at:(B-10) Proof. WenotethatPn`j^;Yo=Pn2M`j^;Yo=ZM`j^;Yd;hence,inviewofEq. 3-18 ,Pn`j^;Yo=C0hqd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d`Ifd`g+qd`1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(d`Ifd`gij``j1 2 j(n+)``j1 2expn2 2^a`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``^a`;wherethelastequalityisachievedusingthepropertiesofthemultivariatenormaldistribution. Inthenextseriesoflemmas,wewillshowthatforanysparsitypattern`2L,theposteriorprobabilityratioPR(`;t)isapproachingzero,asngoesto1.Specically,weconsiderfourcasesofundertted(`t),overtted(t`withd`n),andnon-inclusive(t6`and`6t). Lemma5. Suppose`tthen,underAssumptions1-6, PR(`;t)!0;asn!1:(B-11) Proof. ByAssumption 2 ,dt<,henced`
PAGE 74

PR(`;t)=k``k1 2 kttk1 2q1 1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1d`)]TJ /F8 7.97 Tf 6.58 0 Td[(dtk(n+)ttk1 2 k(n+)``k1 2expnn2 2^a0`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``^a`o expn2 2^a0t(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1tt^at=k``k1 2 kttk1 2q1 1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1d`)]TJ /F8 7.97 Tf 6.58 0 Td[(dtk(n+)ttk1 2 k(n+)``k1 2exp)]TJ /F4 11.955 Tf 10.49 8.08 Td[(n2 2^a`c)]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``^a`0(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1tj`^a`c)]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``^a`;thatis,PR(`;t)k``k1 2 kttk1 2q1 1)]TJ /F4 11.955 Tf 11.95 0 Td[(q1d`)]TJ /F8 7.97 Tf 6.58 0 Td[(dtk(n+)ttk1 2 k(n+)``k1 2exp()]TJ /F4 11.955 Tf 10.5 8.15 Td[(n2k^a`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``^a`k2 2eigmax(n+)tt);Now,bythetriangularinequality, k^a`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``^a`kka`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``a`k)-222(k(^a`c)]TJ /F23 11.955 Tf 11.95 0 Td[(a`c))]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``(^a`)]TJ /F23 11.955 Tf 11.96 0 Td[(a`)k=k)]TJ /F2 11.955 Tf 5.47 -9.69 Td[(0+a`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``)]TJ /F2 11.955 Tf 5.48 -9.69 Td[(0+a`k)-222(k(^a`c)]TJ /F23 11.955 Tf 11.95 0 Td[(a`c))]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``(^a`)]TJ /F23 11.955 Tf 11.96 0 Td[(a`)kk)]TJ /F6 11.955 Tf 5.47 -9.68 Td[(0`c)]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0`k)-222(k)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0+a`c)]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0+a`k)-222(k(^a`c)]TJ /F23 11.955 Tf 11.95 0 Td[(a`c))]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``(^a`)]TJ /F23 11.955 Tf 11.96 0 Td[(a`)k:(B-12)Now,byappropriatelypartitioning,wecanwrite(0)`c=`c`0`+`c`c0`cand(0)`=``0`+``c0`c.Hence,forlargeenoughn, 74

PAGE 75

k)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0`k=k1 n(n+)tj`0`c)]TJ /F6 11.955 Tf 11.96 0 Td[(`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1````0`kk1 n(n+)tj`0`ck)-222(k`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1````0`kk1 n(n+)tj`0`ck)]TJ /F1 11.955 Tf 21.79 8.09 Td[(eigmin(`c`)k``0`k eigmin(n+)``k1 n(n+)tj`0`ck)]TJ /F5 11.955 Tf 21.79 8.09 Td[(2k``0`k n"201 2k1 n(n+)tj`0`ck1 21 neigmin(n+)ttsnp (dt)]TJ /F4 11.955 Tf 11.95 0 Td[(d`)1 21 nneigmin()ttsnp (dt)]TJ /F4 11.955 Tf 11.96 0 Td[(d`)1 2"0snp (dt)]TJ /F4 11.955 Tf 11.95 0 Td[(d`)(B-13)MovingontothesecondtermintherighthandsideofEq. B-12 , k)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0+a`c)]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0+a`kk)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0+a`ck+kn`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0+a`kk)]TJ /F6 11.955 Tf 5.48 -9.68 Td[(0+a`ck+neigmax(`c`)k(0+a)`k eigmin(n+)``k)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(0+a`ck+2k(0+a)`k "20c0r logp np dt)]TJ /F4 11.955 Tf 11.96 0 Td[(d`+2p d` "20;(B-14)wherethelastequalitywasachievedbyLemma 3 .Further,regardingthethirdterminrighthandsideofEq. B-12 wecanwrite k(^a`c)]TJ /F23 11.955 Tf 11.96 0 Td[(a`c))]TJ /F4 11.955 Tf 11.95 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``(^a`)]TJ /F23 11.955 Tf 11.95 0 Td[(a`)kk^a`c)]TJ /F23 11.955 Tf 11.95 0 Td[(a`ck+neigmax(`c`)k(^a`)]TJ /F23 11.955 Tf 11.96 0 Td[(a`)k eigmin(n+)``3CK2 "0r logp np dt)]TJ /F4 11.955 Tf 11.95 0 Td[(d`+2p d` "20;(B-15) 75

PAGE 76

hence,bycombiningEq. B-12 ,Eq. B-13 ,Eq. B-14 ,andEq. B-15 ,forsucientlylargen,wehavethatk^a`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1``^a`k1 2"0snp (dt)]TJ /F4 11.955 Tf 11.96 0 Td[(d`))]TJ /F4 11.955 Tf 11.95 0 Td[(c0r logp np dt)]TJ /F4 11.955 Tf 11.95 0 Td[(d`+2p d` "20)]TJ /F5 11.955 Tf 13.15 8.09 Td[(3CK2 "0r logp np dt)]TJ /F4 11.955 Tf 11.96 0 Td[(d`+2p d` "201 2"0snp (dt)]TJ /F4 11.955 Tf 11.96 0 Td[(d`))]TJ /F10 11.955 Tf 11.95 16.85 Td[(c0+3CK2 "0r logp np dt)]TJ /F4 11.955 Tf 11.95 0 Td[(d`+2p d` "201 2"0sn)]TJ /F10 11.955 Tf 11.96 16.86 Td[(c0+3CK2 "0r logp n2p dt "20;inviewofAssumption 5 ,1 2"0sn c0+3CK2 "0p logp n2p dt "20!1,asn!1,hence,foralllargen,wecanwrite,k^a`c)]TJ /F4 11.955 Tf 11.96 0 Td[(n`c`(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``^a`k1 4"0snNow,onceagainbyLemma 3 PR(`;t)k``k1 2 kttk1 2(2q1)d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtndt)]TJ /F11 5.978 Tf 5.75 0 Td[(d` 2exp)]TJ /F4 11.955 Tf 10.49 8.78 Td[(n21 64"20s2n 6Kn")]TJ /F3 7.97 Tf 6.59 0 Td[(10=k``k1 2 kttk1 22d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtp n q1exp)]TJ /F4 11.955 Tf 10.51 8.09 Td[(n"30s2n 384Kdt)]TJ /F8 7.97 Tf 6.59 0 Td[(d`=k``k1 2 kttk1 22d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtp n q1exp)]TJ /F5 11.955 Tf 9.3 0 Td[(2a1ns2ndt)]TJ /F8 7.97 Tf 6.58 0 Td[(d`byAssumption 5 ,foralllargen,PR(`;t)k``k1 2 kttk1 22d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtp n q1expf)]TJ /F5 11.955 Tf 17.27 0 Td[(logn)]TJ /F5 11.955 Tf 11.95 0 Td[(2a2dtlogpgdt)]TJ /F8 7.97 Tf 6.59 0 Td[(d`=k``k1 2 kttk1 22d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtp)]TJ /F3 7.97 Tf 6.59 0 Td[(2a2dt p nq1dt)]TJ /F8 7.97 Tf 6.59 0 Td[(d`=k``k1 2 kttk1 22d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtp)]TJ /F8 7.97 Tf 6.59 0 Td[(a2dt p ndt)]TJ /F8 7.97 Tf 6.59 0 Td[(d`!0asn!1: 76

PAGE 77

Lemma6. Suppose`t,andd`2dt, qd`)]TJ /F8 7.97 Tf 6.58 0 Td[(dt1p2c2d` K"0qd` 21p2c2d` K"0p)]TJ /F13 5.978 Tf 5.75 0 Td[(8c2 K"0p4c2 K"0d` 2p)]TJ /F13 5.978 Tf 5.75 0 Td[(4c2 K"0d` 2p)]TJ /F13 5.978 Tf 5.76 0 Td[(4c2 K"0d`)]TJ /F11 5.978 Tf 5.76 0 Td[(dt 2(B-17) 77

PAGE 78

and,inthecaseofdtnthen,underAssumptions1-6,PR(`;t)!0;asn!1: Proof. When`t,PR(`;t)k``k1 2 kttk1 2qd`2(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q2)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(d` qdt1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(dt1 k(n+)`jtk1 2exp1 2(n+)``0`+n^a`0(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1``(n+)``0`+n^a`;similartotheargumentinLemma 6 ,eachentryofthevector(n+)``0`+n^a`,inabsolutevalue,issmallerthan2ncq logp n.Now,sinceisnon-negativedenite(notethatinthecaseofd`>n,isnotnecessarilypositivedenite)wehavethateigmin(n+)``eigmin()=kkmin,henceforlargeenoughnPR(`;t)k``k1 2 kttk1 2q2 1)]TJ /F4 11.955 Tf 11.95 0 Td[(q2d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtqdt2(1)]TJ /F4 11.955 Tf 11.95 0 Td[(q2)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(dt qdt1(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q1)(p2))]TJ /F8 7.97 Tf 6.58 0 Td[(dtexp4c2n2d`logp nkkmin;now,sincethefunctionqdt(1)]TJ /F4 11.955 Tf 11.96 0 Td[(q)(p2))]TJ /F8 7.97 Tf 6.59 0 Td[(dtisgloballymaximizedat^q=dt (p2)andq2
PAGE 79

hence,PR(`;t)k``k1 2 kttk1 2(2q2)d`)]TJ /F8 7.97 Tf 6.59 0 Td[(dtexp4c2nd`logp kkmin;sinced`>,andbyAssumption 2 dt< 2,wehavethat,d`)]TJ /F4 11.955 Tf 11.95 0 Td[(dtd` 2;hence,PR(`;t)k``k1 2 kttk1 2(2q2)d` 2exp4c2nd`logp kkmin2d` 2k``k1 2 kttk1 2q2exp8c2nlogp kkmind` 22d` 2k``k1 2 kttk1 2p)]TJ /F13 5.978 Tf 11.06 3.26 Td[(16nc2 kkminp8nc2 kkmind` 22d` 2k``k1 2 kttk1 2p)]TJ /F13 5.978 Tf 12.89 3.26 Td[(8nc2 kkmind` 22d` 2k``k1 2 kttk1 2p)]TJ /F13 5.978 Tf 12.89 3.25 Td[(8nc2 kkmind`)]TJ /F11 5.978 Tf 5.75 0 Td[(dt 2!0;asn!1: Now,let, fn=max`2L(2k``k1 2 kttk1 2;23d` 2k``k1 2 kttk1 2;233d` 2k``k1 2 kttk1 2)max8><>:0@p)]TJ /F13 5.978 Tf 7.78 4.27 Td[(8c2dt K"0 p n1A;p)]TJ /F13 5.978 Tf 12.89 3.26 Td[(8nc2 kkmin1 2;0@p)]TJ /F13 5.978 Tf 5.75 0 Td[(4c2 K"0 nK"01A1 29>=>;:(B-19) Lemma8. Let`2Lsuchthat`6t,t6`,and`6=t,thenunderAssumptions1-6,forsucientlylargen,PR(`;t)!0;asn!1: 79

PAGE 80

Proof. Denotethesharedpartbetween`andtbyh=`\t.Then,PR(`;t)=PR(`;h)PR(h;t);Since,h`andht,byLemma 5 , 6 ,and 7 ,wehavethatPR(`;t)fd`)]TJ /F8 7.97 Tf 6.59 0 Td[(dhnfdt)]TJ /F8 7.97 Tf 6.59 0 Td[(dhn=fd`+dt)]TJ /F3 7.97 Tf 6.59 0 Td[(2dhn!0;asn!1: Corollary1. Let`2Lsuchthat`6t,t6`,and`6=t.DenotethetotalnumberofdisagreementsbyD(`;t).Then,underAssumptions1-6,forsucientlylargen,PR(`;t)fD(`;t)n;8`2L: Proof. TheproofisstraightforwardapplicationofLemmas 5 , 6 , 7 and 8 . ProofofTheorem 1 . 1)]TJ /F4 11.955 Tf 11.96 0 Td[(Pn2Mtj^;Yo Pn2Mtj^;Yo=X`6=tPR(`;t)=X`6=t(p2)Xj=1PR(`;t)IfD(`;t)=jg(p2)Xj=1)]TJ /F8 7.97 Tf 5.48 -4.38 Td[(p2jfjn(p2)Xj=1p2jfjnp2Xj=1)]TJ /F4 11.955 Tf 5.48 -9.69 Td[(p2fnjp2fn 1)]TJ /F4 11.955 Tf 11.96 0 Td[(p2fn!0;asn!1:(B-20)Thelasttwoinequalitiesfollowfromthefactthatp2fn<1andp2fn!0.WhichfollowsfromEq. B-19 andchoiceof"0=c2 K~"0. 80

PAGE 81

ProofofTheorem 2 . Forsimplicityinnotation,letn=q dtlogp n.FirstnotethatforanyconstantG,E0hPk)]TJ /F6 11.955 Tf 11.96 0 Td[(0k2>Gnj^;Yi=X`2LE0hPk)]TJ /F6 11.955 Tf 11.96 0 Td[(0k2>Gnj`;^;YP`j^;YiE0hPk)]TJ /F6 11.955 Tf 11.95 0 Td[(0k2>Gnjt;^;Yi+E0"X`6=tP`j^;Y#:ByTheorem 1 ,itissucienttoproveE0hPk)]TJ /F6 11.955 Tf 11.96 0 Td[(0k2>Gnjt;^;Yi!0asn!1.FirstnotethatPk)]TJ /F6 11.955 Tf 11.95 0 Td[(0k2>Gnjt;^;Y=Pkt)]TJ /F6 11.955 Tf 11.95 0 Td[(0tk2>Gnjt;^;Y;now,fromEq. 3-18 ,itiseasytoseethat tjt;^;YMVNM;(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1tt;(B-21)withM=)]TJ /F4 11.955 Tf 9.3 0 Td[(n(n+))]TJ /F3 7.97 Tf 6.58 0 Td[(1ttat.Hence, E0hPk)]TJ /F6 11.955 Tf 11.95 0 Td[(0k2>Gnj^;YiE0Pkt)]TJ /F31 11.955 Tf 11.95 0 Td[(Mtk2>Gn 2jt;^;Y+E0PkMt)]TJ /F6 11.955 Tf 11.95 0 Td[(0tk2>Gn 2jt;^;Y:(B-22)Now,Pkt)]TJ /F31 11.955 Tf 11.95 0 Td[(Mtk2>Gn 2jt;^;YPmax1lddtj(t)]TJ /F31 11.955 Tf 11.96 0 Td[(Mt)lj>Gn 2p dtjt;^;YdtXl=1Pj(t)]TJ /F31 11.955 Tf 11.95 0 Td[(Mt)lj>Gn 2p dtjt;^;Y; 81

PAGE 82

denotingthelthelementof(t)]TJ /F31 11.955 Tf 11.96 0 Td[(Mt)by(t)]TJ /F31 11.955 Tf 11.95 0 Td[(Mt)landinviewofEq. B-21 ,(t)]TJ /F31 11.955 Tf 11.96 0 Td[(Mt)lN(0;l);wherebyLemma 2 ,leigmax(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1tt=1 eigmin(ntt)2 nK"0.Therefore, Pkt)]TJ /F31 11.955 Tf 11.95 0 Td[(Mtk2>Gn 2jt;^;YdtXl=1Pj(t)]TJ /F31 11.955 Tf 11.95 0 Td[(Mt)lj>Gn 2p dtjt;^;YdtPj(t)]TJ /F31 11.955 Tf 11.96 0 Td[(Mt)1j>Gn 2p dtjt;^;YdtPjZj>p nK"0Gn 2p 2dtdte)]TJ /F11 5.978 Tf 7.78 4.52 Td[(nK"0G22n 8dt;(B-23)wherethelastinequalitywasachievedusingtheMillsratioinequality.MovingontothesecondterminEq. B-21 ,andrecallingthatwearerestrictedtotheevenC1;n\C2;n,foralllargenwehavethatkMt)]TJ /F6 11.955 Tf 11.96 0 Td[(0tk2=kn(n+))]TJ /F3 7.97 Tf 6.59 0 Td[(1ttat+0tk2=k+1 n)]TJ /F3 7.97 Tf 6.59 0 Td[(1ttat+0tk21 eigmin)]TJ /F6 11.955 Tf 5.48 -9.69 Td[(+1 nttkat++1 ntt0tk24 3K"0ktt0t+at+1 ntt0tk24 3K"0p dtktt0t+atkmax+k1 ntt0tk24 3K"0 c0r dtlogp n+p dtRnkkmax n!4 3K"0 c0r dtlogp n+p dtp nlogpkkmax n!=4(c0+kkmax) 3K"0r dtlogp n; 82

PAGE 83

whereweusedLemma 2 togetthesecondinequality,usedLemma 3 withc0=2cp K+3CK2 "0togetthefourthinequalityandthelastinequalityisadirectapplicationoftheassumption.Hence,bytakingG=4(c0+kkmax) 3K"0,wehavethat PkMt)]TJ /F6 11.955 Tf 11.96 0 Td[(0tk2>Gn 2jt;^;Y=PkMt)]TJ /F6 11.955 Tf 11.96 0 Td[(0tk2>Gn 21)]TJ /F4 11.955 Tf 11.95 0 Td[(P(C1;n\C2;n)!0:(B-24)Thus,bycombiningEq. B-21 ,Eq. B-22 ,Eq. B-23 ,andEq. B-24 ,wehavethat E0hPk)]TJ /F6 11.955 Tf 11.95 0 Td[(0k2>Gnj^;Yidte)]TJ /F11 5.978 Tf 7.79 4.52 Td[(nK"0G22n 8dt+0=dte)]TJ /F13 5.978 Tf 7.78 5.03 Td[(32c20logp 9K"0!0asn!1;(B-25)whichcompletestheproof. 83

PAGE 84

REFERENCES Banerjee,Onureena,Ghaoui,LaurentEl,anddAspremont,Alexandre.\Modelselectionthroughsparsemaximumlikelihoodestimationformultivariategaussianorbinarydata."JournalofMachinelearningresearch9(2008).Mar:485{516. Bhattacharya,Anirban,Chakraborty,Antik,andMallick,BaniK.\FastsamplingwithGaussianscalemixturepriorsinhigh-dimensionalregression."Biometrika(2016):asw042. Bhattacharya,Anirban,Pati,Debdeep,Pillai,NateshS,andDunson,DavidB.\Dirichlet{Laplacepriorsforoptimalshrinkage."JournaloftheAmericanStatisticalAssociation110(2015).512:1479{1490. Buhlmann,PeterandVanDeGeer,Sara.Statisticsforhigh-dimensionaldata:methods,theoryandapplications.SpringerScience&BusinessMedia,2011. Cai,TTony,Li,Hongzhe,Liu,Weidong,andXie,Jichun.\Jointestimationofmultiplehigh-dimensionalprecisionmatrices."StatisticaSinica26(2016).2:445. Cao,Xuan,Khare,Kshitij,andGhosh,Malay.\Posteriorgraphselectionandestimationconsistencyforhigh-dimensionalbayesiandagmodels."arXivpreprintarXiv:1611.01205(2016). Danaher,Patrick,Wang,Pei,andWitten,DanielaM.\Thejointgraphicallassoforinversecovarianceestimationacrossmultipleclasses."JournaloftheRoyalStatisticalSociety:SeriesB(StatisticalMethodology)76(2014).2:373{397. Eddelbuettel,DirkandFrancois,Romain.\Rcpp:SeamlessRandC++Integration."JournalofStatisticalSoftware40(2011).8:1{18.URL http://www.jstatsoft.org/v40/i08/ Eddelbuettel,DirkandSanderson,Conrad.\RcppArmadillo:AcceleratingRwithhigh-performanceC++linearalgebra."ComputationalStatisticsandDataAnalysis71(2014):1054{1063.URL http://dx.doi.org/10.1016/j.csda.2013.02.005 Fan,Jianqing,Feng,Yang,andWu,Yichao.\NetworkexplorationviatheadaptiveLASSOandSCADpenalties."Theannalsofappliedstatistics3(2009).2:521. Friedman,Jerome,Hastie,Trevor,andTibshirani,Robert.\Sparseinversecovarianceestimationwiththegraphicallasso."Biostatistics9(2008).3:432{441. Guo,Jian,Levina,Elizaveta,Michailidis,George,andZhu,Ji.\Jointestimationofmultiplegraphicalmodels."Biometrika98(2011).1:1{15. Hans,Chris.\Bayesianlassoregression."Biometrika96(2009).4:835{845. 84

PAGE 85

Khare,Kshitij,Oh,Sang-Yun,andRajaratnam,Bala.\Aconvexpseudolikelihoodframeworkforhighdimensionalpartialcorrelationestimationwithconvergenceguarantees."JournaloftheRoyalStatisticalSociety:SeriesB(StatisticalMethodology)77(2015).4:803{825. Kling,Teresia,Johansson,Patrik,Sanchez,Jose,Marinescu,VoichitaD,Jornsten,Rebecka,andNelander,Sven.\Ecientexplorationofpan-cancernetworksbygeneralizedcovarianceselectionandinteractivewebcontent."Nucleicacidsresearch43(2015).15:e98{e98. Kyung,Minjung,Gill,Je,Ghosh,Malay,Casella,George,etal.\Penalizedregression,standarderrors,andBayesianlassos."BayesianAnalysis5(2010).2:369{411. Leppa-Aho,Janne,Pensar,Johan,Roos,Teemu,andCorander,Jukka.\LearningGaussiangraphicalmodelswithfractionalmarginalpseudo-likelihood."InternationalJournalofApproximateReasoning83(2017):21{42. Ma,JingandMichailidis,George.\Jointstructuralestimationofmultiplegraphicalmodels."TheJournalofMachineLearningResearch17(2016).1:5777{5824. Majumdar,SubhabrataandMichailidis,George.\JointEstimationandInferenceforDataIntegrationProblemsbasedonMultipleMulti-layeredGaussianGraphicalModels."arXivpreprintarXiv:1803.03348(2018). Mitchell,TobyJandBeauchamp,JohnJ.\Bayesianvariableselectioninlinearregression."JournaloftheAmericanStatisticalAssociation83(1988).404:1023{1032. Narisetty,NaveenNaidu,He,Xuming,etal.\Bayesianvariableselectionwithshrinkinganddiusingpriors."TheAnnalsofStatistics42(2014).2:789{817. Park,TrevorandCasella,George.\Thebayesianlasso."JournaloftheAmericanStatisticalAssociation103(2008).482:681{686. Peng,Jie,Wang,Pei,Zhou,Nengfeng,andZhu,Ji.\Partialcorrelationestimationbyjointsparseregressionmodels."JournaloftheAmericanStatisticalAssociation104(2009).486:735{746. Peterson,Christine,Stingo,FrancescoC,andVannucci,Marina.\BayesianinferenceofmultipleGaussiangraphicalmodels."JournaloftheAmericanStatisticalAssociation110(2015).509:159{174. Pierson,Emma,Koller,Daphne,Battle,Alexis,Mostafavi,Sara,Consortium,GTEx,etal.\Sharingandspecicityofco-expressionnetworksacross35humantissues."PLoScomputa-tionalbiology11(2015).5:e1004220. Polson,NicholasGandScott,JamesG.\Shrinkglobally,actlocally:SparseBayesianregularizationandprediction."Bayesianstatistics9(2010):501{538. Rudelson,Mark,Vershynin,Roman,etal.\Hanson-Wrightinequalityandsub-gaussianconcentration."ElectronicCommunicationsinProbability18(2013). 85

PAGE 86

Saegusa,TakumiandShojaie,Ali.\Jointestimationofprecisionmatricesinheterogeneouspopulations."Electronicjournalofstatistics10(2016).1:1341. Tan,LindaSL,Jasra,Ajay,DeIorio,Maria,Ebbels,TimothyMD,etal.\BayesianinferenceformultipleGaussiangraphicalmodelswithapplicationtometabolicassociationnetworks."TheAnnalsofAppliedStatistics11(2017).4:2222{2251. Ventura,LauraandRacugno,Walter.\Pseudo-likelihoodsforBayesianinference."TopicsonMethodologicalandAppliedStatisticalInference.Springer,2016.205{220. Wang,Haoetal.\Bayesiangraphicallassomodelsandecientposteriorcomputation."BayesianAnalysis7(2012).4:867{886. Yuan,MingandLin,Yi.\ModelselectionandestimationintheGaussiangraphicalmodel."Biometrika94(2007).1:19{35. Zhu,Yunzhang,Shen,Xiaotong,andPan,Wei.\Structuralpursuitovermultipleundirectedgraphs."JournaloftheAmericanStatisticalAssociation109(2014).508:1683{1696. 86

PAGE 87

BIOGRAPHICALSKETCHPeymanJalaliearnedhisbachelor'sinstatisticsin2011fromUniversityofKurdistanandmaster'sinmathematicalstatisticsin2013fromShahidBeheshtiUniversityinIran.Afterhismasterprogram,heworkedatthesecurityandexchangeorganizationofIranfor11months.HejoinedtheDepartmentofStatisticsattheUniversityofFloridainAugustof2014,whereheearnedhisdoctorateinstatisticsinMayof2019. 87