<%BANNER%>

On Some New Contributions toward Objective Priors

Permanent Link: http://ufdc.ufl.edu/UFE0024909/00001

Material Information

Title: On Some New Contributions toward Objective Priors
Physical Description: 1 online resource (87 p.)
Language: english
Creator: Liu, Ruitao
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: asymptotic, bayesian, divergence, jeffreys, likelihood, moment, posterior, prior, shrinkage
Statistics -- Dissertations, Academic -- UF
Genre: Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Bayesian methods are increasingly used in recent years in the theory and practice of statistics. One key component in Bayesian inference is the selection of priors. Generally speaking, there are two classes of priors: subjective priors and objective priors. With enough historical data, it is often possible to elicit a suitable subjective prior. However, for most real situations, elicitation of priors is quite difficult and time consuming. Therefore one needs to use 'objective' priors. In this dissertation, I have made some new contributions to objective priors. The first one is a generalization of Bernardo's 'Reference Prior' in the absence of nuisance parameters Bernardo (1979) and also of the results from Clarke (1997) and Sun and Ghosh, Mergel and Liu (2009). For the regular multiparameter family of distributions without any nuisance parameters, I have found objective priors by approximately maximizing a class of divergences between the prior and the posterior. This class includes the Kullback-Leibler, Bhattacharyya-Hellinger and Chisquare divergence. A full characterization of optimal priors for every member in this divergence class is provided. It turns out that Jeffreys' prior maximizes this distance in the interior of this class of divergence measures. On the boundary, the prior turns out to be different from Jefferys' prior for some common families of distributions. Also, outside the boundary, Jeffreys' prior turned out to be the minimizer rather than maximizer of the distance, and there does not exist any prior which maximizes the distance between the posterior and the prior. Also, I have made an extension of the above work by considering the prior selection with presence of nuisance parameters. Under the same class of divergence measures as the one in the first part, I have provided explicit expressions of optimal priors for almost every member of the class of divergences except for chi-square divergence. In the case of chi-square divergence, I have showed that the objective prior should be the solution to a set of partial differential equations. The final part of my work is a new criterion for objective priors which I will refer to as the `moment matching criterion'. The moment matching priors are obtained by matching the posterior mean with the maximum likelihood estimator up to a high order of approximation. A complete characterization of such priors in the one or multi-parameter case is provided. In the process, many new priors are derived.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Ruitao Liu.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Ghosh, Malay.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024909:00001

Permanent Link: http://ufdc.ufl.edu/UFE0024909/00001

Material Information

Title: On Some New Contributions toward Objective Priors
Physical Description: 1 online resource (87 p.)
Language: english
Creator: Liu, Ruitao
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: asymptotic, bayesian, divergence, jeffreys, likelihood, moment, posterior, prior, shrinkage
Statistics -- Dissertations, Academic -- UF
Genre: Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Bayesian methods are increasingly used in recent years in the theory and practice of statistics. One key component in Bayesian inference is the selection of priors. Generally speaking, there are two classes of priors: subjective priors and objective priors. With enough historical data, it is often possible to elicit a suitable subjective prior. However, for most real situations, elicitation of priors is quite difficult and time consuming. Therefore one needs to use 'objective' priors. In this dissertation, I have made some new contributions to objective priors. The first one is a generalization of Bernardo's 'Reference Prior' in the absence of nuisance parameters Bernardo (1979) and also of the results from Clarke (1997) and Sun and Ghosh, Mergel and Liu (2009). For the regular multiparameter family of distributions without any nuisance parameters, I have found objective priors by approximately maximizing a class of divergences between the prior and the posterior. This class includes the Kullback-Leibler, Bhattacharyya-Hellinger and Chisquare divergence. A full characterization of optimal priors for every member in this divergence class is provided. It turns out that Jeffreys' prior maximizes this distance in the interior of this class of divergence measures. On the boundary, the prior turns out to be different from Jefferys' prior for some common families of distributions. Also, outside the boundary, Jeffreys' prior turned out to be the minimizer rather than maximizer of the distance, and there does not exist any prior which maximizes the distance between the posterior and the prior. Also, I have made an extension of the above work by considering the prior selection with presence of nuisance parameters. Under the same class of divergence measures as the one in the first part, I have provided explicit expressions of optimal priors for almost every member of the class of divergences except for chi-square divergence. In the case of chi-square divergence, I have showed that the objective prior should be the solution to a set of partial differential equations. The final part of my work is a new criterion for objective priors which I will refer to as the `moment matching criterion'. The moment matching priors are obtained by matching the posterior mean with the maximum likelihood estimator up to a high order of approximation. A complete characterization of such priors in the one or multi-parameter case is provided. In the process, many new priors are derived.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Ruitao Liu.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Ghosh, Malay.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024909:00001


This item has the following downloads:


Full Text

PAGE 1

1

PAGE 2

2

PAGE 3

3

PAGE 4

Thisdissertationwouldnothavebeenpossiblewithoutthehelp,support,andguidanceofalotofpeople.Firstly,IwouldliketosincerelythankmyadvisorDr.MalayGhoshforhisguidance,understanding,patience.Hisguidancehasprovidedmewiththefoundationforbecomingastatistician.IwouldalsoliketothankmycommitteemembersDr.RonaldRandles,Dr.AndrewRosalskyandDr.CynthiaGarvanfortheirencouragementandhelpfuladvice.Theirhelp,commentsandsuggestionshavemadeabigimpactonthisdissertation.ManypeopleonthefacultyandstaoftheDepartmentofStatisticsassistedandencouragedmeinvariouswaysduringmycourseofstudies.IamespeciallygratefultoDr.AlanAgresti,Dr.MichaelDaniels,Dr.HaniDoss,Dr.JamesHobert,Dr.AndrreKhuri,Dr.RamonLittell,Dr.TrevorPark,Dr.BrettPresnell,Dr.ClydeSchooleld,Dr.LindaYoungforallthattheyhavetaughtme.IwasalsogreatlyinspiredbyDr.GeorgeCasella,forwhomIwasateachingassistantfortwoyears.Ithankmyparentsfortheirfaithandcondenceinmeandencouragingmetopursuemychildhooddreams.ItwasundertheirwatchfuleyesthatIgainedsomuchabilitytotacklechallengesheadon.Finally,IwouldliketothankmygirlfriendAixin.Hersupport,encouragementandlovemadethetoughgraduatestudysomucheasier.Itisreallyfuntohaveacademicornon-academicdiscussionswithherathome,inoceoronthetenniscourt. 4

PAGE 5

page ACKNOWLEDGMENTS ................................. 4 .......................................... 7 ABSTRACT ........................................ 7 CHAPTER 1Introduction ...................................... 9 1.1Background ................................... 9 1.2InvariantPriors ................................. 10 1.2.1Jereys'Prior .............................. 10 1.2.2LeftandRightInvariantPriors .................... 11 1.3PriorswithMaximalMissingInformation ................... 12 1.3.1Bernardo'sReferencePrior ....................... 12 1.3.1.1Referencepriorintheabsenceofnuisanceparameters ... 13 1.3.1.2Referencepriorinthepresenceofnuisanceparameters .. 13 1.3.2PriorsUndertheExpectedChi-squareDistance ........... 16 1.3.2.1Thechi-squareDistance ................... 16 1.3.2.2Priorsundertheexpectedchi-squaredistance ....... 17 1.3.3DivergencePriorsforOne-parameterModels ............. 18 1.4ProbabilityMatchingPriors .......................... 18 1.5TwoBasicTools ................................ 22 1.5.1AsymptoticExpansionofthePosteriorsDensity ........... 22 1.5.2ShrinkageArgument .......................... 25 2DivergencePriorsforMultiparameterModels:WithoutNuisanceParameters .. 28 2.1TheExpectedDivergenceBetweenthePriorandthePosterior ....... 28 2.2Jereys'Prior. .................................. 36 2.3Priorsfor=1. ............................... 38 3DivergencePriorsforMultiparameterModels:inthePresenceofNuisanceParameters 43 3.1DerivationofPriors ............................... 44 3.2DivergencePriorsfor6=1 ......................... 46 3.3DivergencePriorsfor=1 ......................... 51 4MomentMatchingPriors ............................... 68 4.1PriorsfortheRegularOne-parameterFamilyofDistributions ....... 69 4.2Multi-parameterExtension ........................... 73 4.3MatchingofHigherMoments ......................... 80 5Summary ....................................... 82 5

PAGE 6

....................................... 84 BIOGRAPHICALSKETCH ................................ 87 6

PAGE 7

Bayesianmethodsareincreasinglyusedinrecentyearsinthetheoryandpracticeofstatistics.OnekeycomponentinBayesianinferenceistheselectionofpriors. Generallyspeaking,therearetwoclassesofpriors:subjectivepriorsandobjectivepriors.Withenoughhistoricaldata,itisoftenpossibletoelicitasuitablesubjectiveprior.However,formostrealsituations,elicitationofpriorsisquitedicultandtimeconsuming.Thereforeoneneedstouse"objective"priors. Inthisdissertation,Ihavemadesomenewcontributionsto`objective'priors.TherstoneisageneralizationofBernardo's`ReferencePrior'intheabsenceofnuisanceparameters( Bernardo 1979 )andalsooftheresultsfrom ClarkeandSun ( 1997 )and Ghoshetal. ( 2009 ).Fortheregularmultiparameterfamilyofdistributionswithoutanynuisanceparameters,Ihavefound`objective'priorsbyapproximatelymaximizingaclassofdivergencesbetweenthepriorandtheposterior.ThisclassincludestheKullback-Leibler,Bhattacharyya-HellingerandChisquaredivergence.Afullcharacterizationofoptimalpriorsforeverymemberinthisdivergenceclassisprovided.ItturnsoutthatJereys'priormaximizesthisdistanceintheinteriorofthisclassofdivergencemeasures.Ontheboundary,thepriorturnsouttobedierentfromJeerys'priorforsomecommonfamiliesofdistributions.Also,outsidetheboundary,Jereys'priorturnedouttobetheminimizerratherthanmaximizerofthedistance,andtheredoesnotexistanypriorwhichmaximizesthedistancebetweentheposteriorandtheprior. 7

PAGE 8

ThenalpartofmyworkisanewcriterionforobjectivepriorswhichIwillrefertoasthe`momentmatchingcriterion'.Themomentmatchingpriorsareobtainedbymatchingtheposteriormeanwiththemaximumlikelihoodestimatoruptoahighorderofapproximation.Acompletecharacterizationofsuchpriorsintheone-ormulti-parametercaseisprovided.Intheprocess,manynewpriorsarederived. 8

PAGE 9

Ideally,ifonehasenoughhistoricaldata,itispossibletoelicitanappropriatepriorwhichreectsone'spreviousknowledgeandbeliefaboutunknownparameters.Thisisasubjectiveprior.Butthechoiceofsubjectivepriorsisdicult,especiallywhenthereisnotenoughhistoricalinformationabout:Hence,thiskindofpriorshasnotbeenextensivelystudied.Inpractice,itiscommontousetheso-called`objective'priorswhicharealsoreferredtoas`noninformativepriors'or`defaultpriors'.Thosepriorsaredeterminedbysomeobjectiveorstructuralrules.Inthepastdecades,manystatisticiansworkedonthistopic.Consequently,numerousmethodsforselectingobjectivepriorshavebeenproposed.Inourassessment,sofar,therearethreemajorcriteriatodetermineobjectivepriors. 9

PAGE 10

Bayes ( 1763 )whousedtheuniformpriorforthebinomialproportionp.Acommonargumentforusingthispriorisbasedontheprincipleofignoranceorprincipleofinsucientreason.Thisprinciplesaysthatifoneknowsnothingabout,thenthereisnoreasontoputmoreweighttoonepointthananother.Uniformpriorseemstoaverygoodchoicebutithasbeencriticizedduetoitslackofinvarianceunderone-to-onereparameterization.Forexample,auniformpriorfor,thepopulationstandarddeviation,doesnotresultinaunformpriorforthepopulationvariance2.So,theusageofuniformpriorcouldbringdicultiesandconfusionsinstatisticalinference,especiallywhenpeoplearenotsurewhichparametrizationispreferred. Jereys ( 1946 )recommendedapriorwhichisinvariantunderone-to-onereparametrization.Supposeisaparametervectorandtheotherparametervectorisequaltog();whereg()isanyone-to-onefunctionwithrstderivatives.Then,bychange-of-variablesformula,theobjectivepriors()and()shouldbelinkedbytheequation()=(g())@ Amongmanyinvariantpriors,themostpopularoneisJereys'generalrulepriorproposedbyJereysin1946.LetI()denotetheFisherinformationmatrix.ThatisI()=E@2logf(Xj)

PAGE 11

ConsiderafamilyofdistributionsF=ff(xj)g;2A.Abeingtheparameterspace.ForagivengroupoftransformationsG,thefamilyFissaidtobeinvariantunderthegroupG,ifforeveryg2Gand2A,thereexistaunique02AsuchthatY=g(X)hasdensityf(yj0).Forconvenience,wewrite0=g().Obviously,foranygiveng,thetransformation!g()isatransformationofAintoitself.ItcanbecheckedthatG=fg:g2GgisagroupoftransformationsonA.Accordingtothegeneraltheoryoflocallycompactgroups,thereexisttwomeasureslandronGwhicharecalledleftinvariantandrightinvariantHaarmeasuresrespectively.Herelisleftinvariantinthesensethatforanyg2GandBG,l(gB)=l(B) andrisrightinvariantinthesensethatforanyg2GandBG,r(Bg)=r(B):

PAGE 12

Hartigan ( 1964 )and Ghoshetal. ( 2006 ). Lindley ( 1956 ),andwassubsequentlyhighlypopularizedby Bernardo ( 1979 ), BergerandBernardo ( 1989 1992a b ), ClarkeandSun ( 1997 1999 ), Ghoshetal. ( 2009 ),amongothers. Intuitively,theposteriorcontainsextrainformationcomingfromthedata.Hence,thedivergencebetweenthepriorandtheposteriorcanbethoughtasameasureoftheamountoftheinformationcontainedonlyinthedata(notintheprior).Inotherwords,thedivergenceisthemeasureofthemissinginformation.Therefore,bymaximizingthedivergence,onecangetthepriorwhichchangesthemostuponreceiptofthedata.Hence,thispriorissortof\noninformative". Lindley ( 1956 ), Bernardo ( 1979 )suggestedaexpectedKullback-Leiblerdivergencebetweenpriorandposterior,namely, wherem(x)isthemarginaldensityofxm(x)=Z()f(xj)d: 12

PAGE 13

1{1 ),giventhedatax,theinnerintegralisjustthewell-knownKullback-Leiblerdistancebetweentheposteriordensity(jx)andthepriordensity().Here,theKullback-Leiblerdistancecanbetreatedasthegainininformationprovidedbythedatax.Therefore,KL(())istheexpectedgainininformation,wheretheexpectationiswithrespecttothemarginaldensitym(x). Bernardo ( 1979 )suggestedusinganasymptoticmaximization(ngoestoinnity).Later, BergerandBernardo ( 1989 )showedthatifonemaximizesthisdivergenceforxedn,thismayleadtoadiscretepriorwithnitelymanyjumps{afarcryfromadiuseprior. Todotheasymptoticmaximization,onerstxesanincreasingsequenceofcompactd-dimensionalrectanglesKiwhoseunionisRd(assumingthattheparameterspaceisd-dimensionalEuclidianspace).ForaxedKi,oneconsiderspriorsisupportedonKi,andletn!1. Bernardo ( 1979 ),somewhatheuristically,foundthatthemaximizershouldbei()=cijI()j1=2onKi; ClarkeandBarron ( 1990 1994 ).ForxedKi,theyshowedthatKL(i())=d 2ZKii()logjI()jd+ZKii()log(1=i())d+o(1):

PAGE 14

( 1979 )alsoproposedaproceduretondobjectivepriorsinthepresenceofnuisanceparameters.Let=(1;2)Tbetheparametervector,andtheFisherinformationmatrixI()=0B@I11()I12()I21()I22()1CA; Bernardo ( 1979 )suggestedsettingtheconditionaldensity(2j1)equaltotheconditionalJereys'prior,namely(2j1)=c(1)p First,wexanincreasingsequenceofcompactrectanglesK1iK2iwhoseunionisthewholeparameterspace,andleti(2j1)betheconditionalJeerys'priorrestrictedtoK2iandi(1)apriorsupportedonK1i.Thatis,i(2j1)=ci(1)p

PAGE 15

2d2d1ZK1ii(1)logi(1)d1; 2logjI11()ji(2j1)d2: 2(1{2) isthereferenceprioronK1iK2i. 221001 1{2 ),onegetsi(1;2)=di1 NeymanandScott 1948 ).Thedataconsistofnpairsofobservations:XijN(i;2);i=1;;n,j=1;2:Ifweconsideralltheparametersofequalimportance.Then,fromresultinsection( 1.3.1.1 ),onegetstheprior(1;;n;)/(n+1).Sotheposteriormeaniss2=(2n2),wheres2=Pni=1P2j=1(xijxi)2andxi=(xi1+xi2)=2.Thisisaninconsistentestimatorof2.

PAGE 16

1{2 )leadstheprior(1;;n;)/1:Thisgivesaposteriormeanofs2=(n2)whichisconsistent. ClarkeandSun ( 1997 1999 )consideredthefollowingexpectedchi-squaredistancebetweenthepriordensityandtheposteriordensity2(())=ZZ((jx)())2 whereOiisthenumberofobservationsincategoryiandEi=npi. Wecangeneralizethischi-squarestatistictothecaseofcontinuousrandomvariables.SupposeXisacontinuousrandomvariablewithcumulativedistributionfunctionFanddensityfunctionp(x).Nowselect0=a0<:::
PAGE 17

( 1997 )provedthat2(())=ZRr 41300() 4n()I()+0()2 8n+3 Formultiparameterexponentialfamilyofdistributions,theyconjecturedthatthepriorshouldalsobethethereciprocalofJereys'prior. Forthecasewherenuisanceparametersarepresent,theygaveabriefdiscussionandleftitasanopenquestion. Unfortunately,theresultof ClarkeandSun ( 1997 )intheone-parametercaseisnotquitecorrect.Fortheregularone-parameterexponentialfamilyofdistributions,thepriorshouldbethefourthrootoftheFisherinformationnumber. 17

PAGE 18

Ghoshetal. ( 2009 )consideredthefollowinggeneraldivergencebetweenpriorandposteriorR()=1RR()1(jx)dm(x)(dx) Amari ( 1982 ), CressieandRead ( 1984 )). Actually,thisisafamilyofdivergencewithindexparameter.When!0,thisistheKullback-Leiblerdivergence.Also,=1=2givestheBhattacharyya-Hellinger( Bhattacharyya ( 1943 ); Hellinger ( 1909 ))distance,and=1amountstothechi-squaredistance.Underthegeneraldivergence,theyfoundthatwiththeexceptionofthechi-squaredivergence,Jereys'priorturnsouttobetheuniqueoptimizingprior.Forchi-squaredistance,theresultingpriorturnsouttobethepositivefourthrootratherthanthepositivesquarerootoftheFisherinformationnumber. Ghoshetal. ( 2009 ),underchi-squaredivergence,thedesiredpriorischi()/exph1 4RI0() 4().Onthecontrary,forKullback-Leiblerdivergence,thepriorisJereys'priorKL()/I1 2().Thus,inparticular,fortheBinomial(n,)problem,onegetschi()/1 4(1)1 4whichisaBeta(3 4,3 4)distributionquitedierentfromJereys'Beta(1 2,1 2)prior.Similarly,forthePoisson()case,onegetschi()/1 4,againdierentfromJereys'KL()/1 2prior.However,fortheN(;1)situation,chi()=c(>0),aconstant,whichisthesameasJereys'prior. ( 1963 )and Peers ( 1965 )proposedadierentcriteriontondobjectiveprior.ItisbasedonmatchingtheposteriorcoverageprobabilityofaBayesian 18

PAGE 19

Aprobabilitymatchingdenitionisasfollows.LetX1;;Xnbei.i.dwithcommondensityf(xj)(isarealnumber).AssumealltheregularityconditionsneededforexpansionoftheposterioraroundMLE^n.For0<<1,choose1dependingontheprior()suchthatP[1jX1;;Xn]=1+OP(n) forsome>0.IfnowP[1j]=1+OP(n),thensomeorderofprobabilitymatchingisachieved.If=1,wecallarstorderprobabilitymatchingprior.If=3=2,wecallasecondorderprobabilitymatchingprior. Peers ( 1965 )showedthataprior()isrstorderprobabilitymatchingifandonlyifitsatisesthepartialdierentialequation d()=I1=2()=0:(1{4) Theuniquesolutionis()/I1=2()whichistheJereys'prior. WelchandPeers ( 1963 )showedthataprior()issecondorderprobabilitymatchingifandonlyifitsatisesequation( 1{4 )andI3=2()L1;1;1isaconstantfreefrom,whereL1;1;1=Edlogf(X1j) (12)3:

PAGE 20

WelchandPeers ( 1963 )andlatermorerigorouslyby DattaandGhosh ( 1995 ),writingI1=Ijk,therstorderprobabilitymatchingequationis @j()Ij1(I11)1 2=0:(1{5) MukerjeeandGhosh ( 1997 )showedthatthepriorissecondorderprobabilitymatchingifandonlyifitsatises,inadditionto( 1{5 ),thepartialdierentialequation 1 3dXj=1dXr=1dXs=1dXu=1@ @u()jrLjrs3su+sudXj=1dXr=1@2 whereLjrs=E@3logf(X1j) 1{5 )and( 1{6 )andis,therefore,secondorderprobabilitymatching.Onecanalsoverifythatthesamepriorenjoysthesecondordermatchingpropertywhen2istheinterestparameterand1isthenuisanceparameter.

PAGE 21

( 1989 )consideredaspecialcasewhere1isorthogonalto2;;pintheFisheriansense,i.e.Ij1=0forj=2;3;;p.Then( 1{5 )and( 1{6 )simplifyto @1()I1 211=0;(1{7) and @u()I111IsuL11s+1 3@ @1()I211IsuL111@2 respectively.So,apriorisrstorderprobabilitymatchingifandonlyifitisoftheform wherehisarbitrary.Furthermore,byequation( 1{8 ),aprioroftheform( 1{9 ),issecondorderprobabilitymatchingifandonlyif @uh(2;;p)I1=211IsuL11s+1 6h(2;;p)@ @1I3=211L1;1;1=0;(1{10) whereL1;1;1=E@logf(X1j) 1{9 )isasecondordermatchingprior.Otherwise,asecondordermatchingpriordoesnotexist. MukerjeeandGhosh 1997 )Considerthebivariatenormaldistributionwithmeans1,2,variance21,22andcorrelationcoecient.Theinterestliesintheregressioncoecient2=1.Reparameterizeas 1{11 )isanorthogonalparametriza-tionandthatI11=3=2.Henceby( 1{9 ),rstordermatchingisachievedifandonlyif 21

PAGE 22

1{10 ),onecancheckthat()issecondordermatchingifandonlyif Withreferencetotheoriginal(1;2;1;2;)parametrization,priorsoftheform 1{14 )getstransformedto()=(1+213)ts 1{12 )and( 1{13 ),itfollowsthataprioroftheform( 1{14 )isrstorderprobabilitymatchingifandonlyift=1 2s+1,anditissecondordermatchingifandonlyifinadditiont=1. CoxandReid ( 1987 ),onecanalwayschoosethe(p1)-dimensionalnuisanceparameter(2;;p)whichareorthogonalto1.Also,asshowedin MukerjeeandGhosh ( 1997 ),thematchingpriorsareinvariantwithrespecttothechoiceof(2;;p)wheninterestliesin1. Johnson ( 1970 ).TheresultgoesbeyondthatofBernsteinandVon-MisesTheoremwhichprovidestheasymptoticnormalityoftheposteriordensity. Webeginwiththeone-parametercase.LetX1;X2;;Xnjbeindependentandidenticallydistributedwithcommonpdff(xj),where2,someintervalinthereal 22

PAGE 23

Johnson ( 1970 )and BickelandGhosh ( 1990 ). Let^ndenotetheMLEof.TheBernstein-VonMisesTheoremassertsthatundersomeregularityconditions,theposteriordistributionofTn=p Johnson ( 1970 ); Ghoshetal. ( 1982 ))givesanasymptoticexpansionoftheposterioruptoacertainorder. Johnson ( 1970 )and BickelandGhosh ( 1990 ).ThenanasymptoticexpansionfortheposteriornofSnisgivenby 2a3s3 2^In00(^n) 1 isasfollow.Suppose=(1;;p)istheparametervectorand^istheMLEof.Leth=(h1;;hp)T=n1=2(^).Then 23

PAGE 24

2R1(h)+1 6R3(h)+n11 2[R2(h)W1]+1 6[R1(h)R3(h)W2]+1 24[R4(h)W3]+1 72[R23(h)W4]+oP(n1) wherep(h;C1)isthep-variatenormaldensitywith0meanvectoranddispersionmatrixC1,andR1(h)=^jhj=^;R2(h)=^jrhjhr=^;R3(h)=ajrshjhrhs;R4(h)=ajrsuhjhrhshu;W1=^jrcjr=^;W2=3ajrs^ucjrcsu=^;W3=3ajrsucjrcsu;W4=ajrsauvw(9cjrcsucvw+6cjucrvcsw); 5 involvesonlyderivativesoflog-likelihood,andtypicallyismuchsimplertoevaluate,especiallyinthemulti-parametercase,whereanEdgeworthexpansioninvolvesseveralmixedcumulants. 24

PAGE 25

DattaandMukerjee ( 2004 ).Considerapossiblyvector-valuedrandomvariableXwithadensityfunctionf(xj)wheretheparameterbelongstoRporsomeopensubsetofit.WearegoingtondanexpressionfortheE[q(X;)],whereqisameasurablefunction.Thisexpectationisknowntoexistandissupposedtobecontinuousforall.ThefollowingstepsdescribetheshrinkageargumentfortheevaluationofE[q(X;)],whereq(X;)isameasurablefunctionwithrespecttothejointmeasureofXandinducedby(). Now,weillustratetherationalebehindtheabovesteps. Notethattheposteriordensityofundertheprior()isgivenbyf(Xj)()=N(X),whereN(X)=Zf(Xj)()d:

PAGE 26

Notethatexplicitspecicationoftheprior()isnotneededinSteps1-3.Whenexecuteduptothedesiredorderofapproximation,undersuitableassumptions,thesestepscanleadtosignicantreductioninthealgebraunderlyinghigherorderfrequentistasymptotics. Inthisstudy,Irevisittheproblemofselectionofobjectivepriorsandmakesomenewcontributions. Inchapter2,Igeneralizetheresultfrom Ghoshetal. ( 2009 )totheregularmultiparameterfamilyofdistributionswithoutnuisanceparameters.Theobjectivepriorsarefoundbymaximizingaverygeneralclassofdivergencemeasuresbetweenthepriorandtheposterior.ThisclassofdivergencemeasuresincludestheKullback-Leibler,Bhattacharyya-Hellingerandchi-squaredivergence.Ihavebeenabletocharacterizeoptimalpriorsforeverymemberinthisdivergenceclass.ItturnsoutthatJereys'generalruleprioristheuniqueoptimalpriorwithintheinteriorofthisdivergenceclass.However,itisnolongeroptimal 26

PAGE 27

Ghoshetal. ( 2009 )intheone-parametercase. Inchapter3,withthepresenceofnuisanceparameters,underthesameclassofdivergencemeasures,Icancharacterizeoptimalpriorsforeverymemberinthisclassbyusingthetwo-stepprocedureproposedby Bernardo ( 1979 ).ForKullback-Leiblerdivergencewhichwasstudiedby Bernardo ( 1979 ),Ireachthesamepriorashefound.Underthechi-squaredivergence,Ihaveshownthattheobjectivepriorshouldbethesolutiontoasetofpartialdierentialequations.Ialsoconsideraspecialcasewhentheparameterofinterestisonedimensional.Inthiscase,theclosedformsoftheoptimalpriorsareprovided. Inchapter4,Iintroduceanewcriterionforobjectivepriors,themomentmatchingcriterion,whichrequiresthematchingoftheposteriormeanwiththemaximumlikelihoodestimatoruptoahighorderofapproximation.Theclassofpriorsischaracterizedwhentheparameterofinterestisreal-valuedaswellaswhenitismultidimensional.Onesurprisingndingisthatevenintheone-parametercasewithoutanynuisanceparameters,theproposedapproachcanleadtopriorsotherthanJereys'prior. Somenalremarksaremadeinchapter5. 27

PAGE 28

Weconsiderinthischapterselectionofpriorsfortheregularmultiparameterfamilyofdistributionswithoutanynuisanceparameters.Thisfamilyincludes,butisnotlimitedtotheregularmultiparameterexponentialfamily.Thisworkisacontinuationofthepreviousworkof Ghoshetal. ( 2009 ).Theseauthorsconsideredselectionofpriorsfortheregularone-parameterfamilyofdistributions,includingbutnotlimitedtotheone-parameterexponentialfamily.Theselectioncriterionwasmaximizationofthedistancebetweenthepriorandposteriorunderaclassofexpecteddivergencemeasuresconsideredamongothersin Amari ( 1982 )and CressieandRead ( 1984 )inothercontexts.ThisclassincludesasspecialcasestheKullbeck-Leibler,Hellingerandchi-squaredistances. InSection 2.1 ,fortheregularmultiparameterfamilyofdistributionswithoutanynuisanceparameters,wehaveprovidedageneralexpressionfortheexpectedgeneraldivergencemeasurebetweenthepriorandtheposterior.Section 2.2 isdevotedtothederivationofoptimalpriorintheinteriorofthedivergenceclassandnon-existenceofoptimalpriorsoutsidetheboundaryofthisclass.Section 2.3 providesacharacterizationofoptimalpriorsontheboundaryofthedivergenceclassfollowedbysomeexamples. WriteLn()=nYi=1p(xij) andxn=(x1;:::;xn):Theposterior(jxn)isthengivenby 28

PAGE 29

where(dxn)=((dx1);:::;(dxn)): whichistheKLdivergencebetweenthepriorandtheposteriorconsideredforexamplein Lindley ( 1956 ), Bernardo ( 1979 ), ClarkeandBarron ( 1990 1994 ),and GhoshandMukerjee ( 1992 ). FromtherelationLn()()=(jxn)m(xn);onecanreexpressR()givenin( 2{2 )asR()=1RR+1()(jxn)Ln()(dxn)d whereE(j)denotestheconditionalexpectationgiven: 29

PAGE 30

where,jjjj=maxfj1j;;jpjg (A3)E24sup02N()logp(xj0)35!E[logp(xj)] as!0 (A4)Thepriordensity()isthreetimescontinuouslydierentiableinaneighborhoodofand()>0. Beforestatingthemaintheoremofthissection,weneedafewmorenotations.Let^n=(^n1;;^np)TdenotetheMLEof.Also,letrln()andr2ln()denotethegradientandHessianofln()andlet^In=r2ln(^n).Further,letajrs=@3ln()=@j@r@sj=^n;ajrsu=@4ln()=@j@r@s@uj=^n;Ajrs=E@3ln()=@j@r@sj;Ajrsu=E@4ln()=@j@r@s@uj;j()=@=@j;jr()=@2=(@j@r);1j;r;s;up: 30

PAGE 31

2(1)p 2jI()j@jI1()j 2X1j;rpj()r() DattaandMukerjee ( 2004 ).Tothisend,webeginwithanasymptoticexpansionoftheposterior(jXn)( DattaandMukerjee 2004 )as 2pXj=1p 6X1j;r;spn3 2(j^nj)(r^nr)(s^ns)ajrs+n11 2X1j;rpn(j^nj)(r^nr)Ijr(^n)jr(^n) 6X1j;r;s;upn2(j^nj)(r^nr)(s^ns)(u^nu)Ijr(^n)Isu(^n)Ijs(^n)Iru(^n)Iju(^n)Irs(^n)ajrsu(^n) 2)#;(2{5) wherekinvolvesfunctionsofand^nbutnotoritsderivatives. 31

PAGE 32

n1=2pXj=1p 6X1j;r;spn3 2(j^nj)(r^nr)(s^ns)ajrs+(+1) 2nX1j;rpn(j^nj)(r^nr)j(^n)r(^n)=2(^n)+1 3X1j;r;s;upn2(j^nj)(r^nr)(s^ns)(u^nu)ajrsu(^n)=(^n) 2)#;(2{6) whereonceagainkdoesnotinvolveoritsderivatives.NowfromStep1ofJ.K.Ghosh,foranyarbitrarythricedierentiablepriorvanishingoutsideacompactset,anexpressionsimilarto( 2{1 )yields (jXn)=(2)p 2pXj=1p 6X1j;r;spn3 2(j^nj)(r^nr)(s^ns)ajrs+OP(n1)#;(2{7) wherej()istherstderivativeof()withrespecttojandtheOP(n1)termsinvolvesecondderivativesof.Nowfrom( 2{6 )and( 2{7 ),omittingtermswhichintegrate 32

PAGE 33

2(^n)T^In(^n)"1 nX1j;r;pn(j^nj)(r^nr)j(^n)r(^n) 2nX1j;rpn(j^nj)(r^nr)j(^n)r(^n)=2(^n) 2)#d;(2{8) wherek(;^n;)doesnotinvolveoritsderivatives,butdoesinvolveanditsderivatives. Frompropertiesofthemultivariatenormaldistributionandnotingthatajrsissymmetricinitsarguments,onegetsfrom( 2{8 ), 33

PAGE 34

n(1)X1j;r;pj(^n) (^n)Ijr(^n)+(+1) 2nX1j;rpj(^n)r(^n) 2)#:(2{9) Writingtherighthandsideof( 2{9 )asn(^)+Op(n3 2),since^n=Op(n1 2)(P),itfollowsfrom( 2{9 )thatn(^n)n()=Op(n3 2)(P),where n(1)X1j;r;pj() ()Ijr()+(+1) 2nX1j;rpj()r() wherek(;)doesnotinvolveoritsderivatives. Step2ofJ.K.Ghoshinvolvesintegratingn()withrespectto()andStep3involvesmaking()degenerateataftercompletionoftheintegration.Inthepresentcontext,in 34

PAGE 35

drj() drjI()j+jI()j drIjr()()d:(2{11) Nowcombining( 2{10 )and( 2{11 )onegetsaftersomesimplication,Zn()()d=(2)p drjI()j n(1)X1j;rpZj() drIjr()

PAGE 36

drjI()j=jI()j2d drjI1()jn()=(2)p @rjI1()jIjr()j() n(1)Xj;r@ @rIjr()j() 2)#: Thenexttwosectionswillfocusonderivationofoptimalpriorsunderthegivendivergenceloss. 2{4 ),neglectingtheOP(n1)term,theselectionofaprioramountstominimizationof1

PAGE 37

Similarly,when1<<0,(1)<0andnowtheproblemreduces(neglectingonceagaintheOP(n1)term)tomaximizationofZ+1()jI()j ClarkeandBarron ( 1990 1994 ),onegetsR0()=p np=2(1+)p HenceitsucestomaximizeZjI()j1=2=()()d

PAGE 38

2: (AM)c1 1;when2AM; (AM)cd=M: Accordingly,theonlyinectionpriorisJereys'priorwhichistheminimizerandnotthemaximizer. ClarkeandSun ( 1997 )fortheoneparameterexponentialfamilyandin Ghoshetal. ( 2009 )forthegeneralone-parameterfamilyofdistributions.Here+1()=1sothattherstordertermappearinginTheorem 2 willnotsuceinndingthepriorandthecoecientofn1isneededinndingtheoptimal.Tothisend,since=1sothat(1)=2, 38

PAGE 39

2 ,neglectingalltermswhichdonotinvolveoritsderivatives,itsucestomaximizeuptothesecondorderapproximation, @rjI1()jIjr()1 2@Ijr() 4Xj;rjr() 2Xj;rj()r() 4Xj;r;sAjrsu() Writingjr()=@ @rj() 2{13 )simpliesto @rjI1()jIjr()1 2@Ijr() 4Xj;rj()r() 4Xj;r@ @rj() 4Xj;r;sAjrsu() Lety()=(y1();;yp())=1() 2{14 )canbeexpressedas soweneedtondy()tomaximizetheaboveintegral.From Giaquinta ( 1983 ),themaximizershouldsatisfytheEuler-Lagrangeequations: @yi()dXj=1@ @j@F @(@yj=@j)=0i=1;;p:(2{16) 39

PAGE 40

2dXj=1dXr=1dXs=1AjrsIjr()Isi()+1 2dXj=11 2jI()j@jI1()j Inmatrixnotations,( 2{17 )is: 2I1()0BBBB@A1...Ap1CCCCA+1 4I1()0BBBB@B1...Bp1CCCCA+1 2I1()0BBBB@C1...Cp1CCCCA;(2{18) whereAi=dXj=1dXr=1AjriIjr()i=1;;p;Bi=jI1()j@jI()j 2{18 )hastheform: where@() 4Bi+1 2Ci1 2Ai;i=1;;p. Thefollowingexamplesillustratehowtondoptimalpriorsforsomecommonfamiliesofdistributions.

PAGE 41

4jI1()j0BBBB@@jI()j 4. )wherep(x)=p(x).Writingh(x)=logp(x)andnotingthath0(x)=h0(x),h00(x)=h00(x)andh000(x)=h000(x),onegetsE@2logf @2;=2Zh00(x)p(x)dx;E@2logf @@;=0;E@2logf @2;=21+2Zxh0(x)p(x)dx+Zx2h00(x)p(x)dx;E@3logf @3;=E@3logf @@2;=0;E@3logf @2@;=32Zh00(x)p(x)dx+Zxh000(x)p(x)dx @3;=32+6Zxh0(x)p(x)dx+6Zx2h00(x)p(x)dx+Zx3h000(x)p(x)dx 2{19 ),thepriorisfoundbysolvingtheequations@log=@=0and@log=@=c ,wherec=2Rh00(x)p(x)dx+Rxh000(x)p(x)dx

PAGE 42

2sothat(;)/7 2. 12#=1=2 22 @2;= 3;E@2logf @@;=0;E@2logf @2;=1 22E@3logf @3;=6 4;E@3logf @@2;=0;E@3logf @2@;=1 @3;=1 2{19 ),theprior(;)shouldsatisfytheseequations:@log(;) 4;@log(;) 4: 41 4:

PAGE 43

Inthepreviouschapter,forthecaseofmultivariateparameters,weconsiderthatalltheparametersareofequalimportance.Thedivergencepriorforthewholeparametervectorisderivedbymaximizingthegeneralexpecteddivergencebetweenthepriorsandthecorrespondingposteriors. Inpractice,oftentherearecertainparametersofinterest,whiletheothersarenuisanceparameters.Forexample,inthestandardlinearmodel,onemodelstheresponsevariableYasY=X+,wherehasamultivariatenormaldistributionwithmeanvector0eanddispersionmatrix2Ien,whereIenisnnidentitymatrix.Inthiscontext,theparametervectorconsistsoftwocomponents{regressioncoecientandvariancecomponent2.Peopleusuallyhavemoreinterestintheregressioncoecientandtreatthevariancecomponent2asanuisanceparameter. InthisChapter,weareinterestedindevelopingdivergencepriorsfortheparametersofinterestinthepresenceofnuisanceparameters. LetT=(1;;d1;d1+1;;d)=(T1;T2)betheddimensionalparametervector,where1isd1dimensionalnuisanceparametervectorand2isd2dimensionalparametervectorofinterest(d1+d2=d).Weapplythefollowingtwostepprocedureproposedby Bernardo ( 1979 )tondthedivergencepriorsfortheparametersofinterest. 43

PAGE 44

InStep1,for(1j2), Bernardo ( 1979 )recommendsusingtheconditionalJereys'priorc(2)p InStep2,withthechoiceof(1j2),werstndanasymptoticexpansionofR(p(2))andthenobtainthepriorsbymaximizingthatexpansion. DerivationofdivergencepriorsinthepresenceofnuisanceparametersismuchmorecomplicatedthantheoneintheChapter2.InSection 3.1 ,wegiveageneralschemeofderivingtheasymptoticexpansionofexpectedgeneraldivergence.InSections( 3.2 )and( 3.3 ),byusingtheasymptoticexpansionwithdierentorderoftheremainderterms,weconsiderthepriorselectionfortwocaseswhen6=1and=1separately. Asintheprevioussection,bytherelationfn(xnj2)p(2)=p(2jxn)m(xn);onecanreexpressR(p(2))as wherefn(xnj2)isthejointdensityfunctionofxn=(x1;;xn)given2. Observethatalthough(x1;;xn)given2arenotindependent,wecanstilluseshrinkageargumenttondtheasymptoticexpressiontoEp(2jXn)2. 44

PAGE 45

Byusingtheaboveprocedureandequation( 3{2 ),onecangetanasymptoticapproximationtoR(p(2)).Furthermore,thedivergencepriorsareobtainedbymaximizingtheapproximation. Inthenexttwosections,accordingtodierentvaluesof,wederivetwoapproximationstoR(p(2))andcallthemtherstorderapproximationandthesecondorderapproximation 45

PAGE 46

3.2 ).Insection( 3.3 ),wewilldiscussthederivationofdivergencepriorwhen=1thatistheonlycasewhichneedsthesecondorderapproximation. Inthissection,weconsiderthepriorselectionforgeneralexpecteddivergencewith6=1.Tobeginwith,wederivetherstorderapproximationtoEp(2jXn)2.Then,inviewof( 3{2 ),wegettherstorderapproximationtoR(p(2)).Finally,wediscussthedivergencepriorsaccordingtodierentvaluesofsuchthat6=1. First,byassumingthesameregularityconditionsofTheoreminpreviouschapter,onegetsthefollowingtheoremwhichgivestherstorderexpansiontoEp(2jXn)2. 3 LethT=(h1;;hd1;hd1+1;hd)=(hT1;hT2)=p DattaandMukerjee ( 2004 )onegets 2R1(h)+1 6R3(h)+o(n1);(3{4) whereCistheobservedFisherinformationmatrixandR1(h);R3(h)aredenedinChapter2.LetNd2(h2j;)denotethedensityfunctionofmultivariatenormal 46

PAGE 47

InStep0,forpriorp(2),onegets 2L1(h2)+1 6L3(h2)+o(n1);(3{5) whereL1(h2)=ZR1(h)fd1(1j2)d1;L3(h2)=ZR3(h)fd1(1j2)d1: n+1 2b22 2L1(h2)+1 6L3(h2)+o(n1):(3{6) 47

PAGE 48

3{5 )and( 3{6 ),foranyarbitrarythricedierentiablepriorp(2)vanishingoutsideacompactset,onegets 2L1(h2)+1 6L3(h2)L1(h2) whereL1(h2)=ZR1(h)fd1(1j2)d1: 2;Q=(qjr)dd=0B@C11+ SinceG(xn)canbewrittenasafunctionof^and^n=op(n1)(P).Then,byusingTaylorexpansion,onegets 48

PAGE 49

2:Moreover,attheendofStep2,onegets InStep3,integrating(2)withrespecttop(2)andallowingp(2)weaklyconvergetothedegeneratedensityoftrue2,weobtainthenalasymptoticexpansionforEp(2jXn)2as 2ZjI22j Thisprovesthetheorem. When<1and6=1,wecanobtainthedivergencepriorsbymaximizingtherstorderapproximationtothegeneralexpecteddivergenceR(p(2)).Theapproximationisderivedbyneglectingtheo(n1)terminTheorem( 3 ).Thatis: nd2 2Z(2) where(2)=ZI22()

PAGE 50

3 oralternativelyfrom Bernardo ( 1979 ),onegetstherstorderapproximationofthegeneralexpecteddivergenceR0(p(2)):R0(p(2))Kn+Zp(2)log(2) 50

PAGE 51

nd2=2(1+)d2 2R(2) HenceitsucestomaximizeZf(2)=p(2)gp(2)d2 Itremainstoconsiderthecase=1;thechisquaredistanceasconsideredin ClarkeandSun ( 1997 )fortheoneparameterexponentialfamilyandin Ghoshetal. ( 2009 )forthegeneralone-parameterfamilyofdistributions.Herep+1(2)=1sothattherstordertermappearinginTheorem 3 willnotsuceinndingthepriorp(2).WecanmimictheTheorem 3 togetthesecondorderexpansiontoE[p(2jXn)j2].HereisthenewTheorem: 51

PAGE 52

2"ZK()(1j2)d1+1 2IjrK()r(1j2)d1ipj(2) 2dXj=d1+1dXr=1@ @rZIjrK()(1j2)d1pj(2) 6X1j;r;sddXu=d1+1hZAjrs(qojrqosu+qojuqors+qojsqoru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrs(IjrIsu+IjuIrs+IjsIru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrskjrsu()K()(1j2)d1ipu(2) 2dXj=d1+1dXr=d1+1hZIjrK()(1j2)d1ipj(2)pr(2) 2I12[I22]1I21I12=2I21=2I22=21CA;IjristhejrthelementoftheFisherinformationmatrix,kjrsu()involvesp(2)anditsderivatives.ButS(2)isonlyafunctionof2:AjrsandAjrsuaredenedinChapter2. 4

PAGE 53

DattaandMukerjee ( 2004 )onegets 2R1(h)+1 6R3(h)+n11 2[R2(h)W1]+1 6[R1(h)R3(h)W2]+1 24[R4(h)W3]+1 72[R23(h)W4]+o(n1):(3{15) Itiseasytoestablishtherelationd(h;C1)=Nd2(h2j0;C22)fd1(1j2);wherefd1(1j2)=Nd1(h1jC12[C22]1h2;C11C12[C22]1C21). So,inStep0,forpriorp(2),onegets 2L1(h2)+1 6L3(h2)+n11 2[L2(h2)W1]+1 6[L1(h2)L3(h2)W2]+1 24[L4(h2)W3]+1 72[L23(h2)W4]+o(n1);(3{16) whereL1(h2)=ZR1(h)fd1(1j2)d1;L2(h2)=ZR2(h)fd1(1j2)d1L3(h2)=ZR3(h)fd1(1j2)d1;L13(h2)=ZR1(h)R3(h)fd1(1j2)d1L4(h2)=ZR4(h)fd1(1j2)d1;L33(h2)=ZR23(h)fd1(1j2)d1:

PAGE 54

3{16 ),foranyarbitrarythricedierentiablepriorp(2)vanishingoutsideacompactset,onegets 2L1(h2)+1 3L3(h2)+L1(h2)+n11 2[L2(h2) 6[L13(h2) 12[L4(h2)W3]+1 36[L33(h2)W4]+1 2[L2(h2)W1]+1 6[L13(h2)W2]+[L1(h2)+1 6L3(h2)][L1(h2)+1 6L3(h2)]+oP(n1);(3{17) whereL1(h2)=ZR1(h)fd1(1j2)d1;L2(h2)=ZR2(h)fd1(1j2)d1L13(h2)=ZR1(h)R3(h)fd1(1j2)d1 2jC22j1 22d2 2;Q=(qjr)dd=0B@C111 2C12[C22]1C21C12=2C21=2C22=21CA:

PAGE 56

4(cjrcsu+cjscru+cjucrs) Later,duetoitscomplexity,weonlyevaluatekjrsu(^)insomespecialcases. Withalltheintegralswementionedaboveandtherelation=h=p 2Zp(h2jxn)p(h2jxn)dh2=nd2 2K(^)"1+1 2"X1j;rd^jr 6"X1j;r;s;ud^u 24hb4(^)W3i+1 72hb33(^)W4i+1 2"X1j;rd^jr 6"X1j;r;s;ud^u 24hb4(^)W3i+1 72hb33(^)W4i+"X1j;rd^j^rcjr 6X1j;r;s;udajrs^u 6X1j;r;s;udajrs^u 36#)+oP(n1)#:(3{18) 56

PAGE 57

2K()"1+1 2"X1j;rdjr 6"X1j;r;s;udu 12M1()+1 36M2()+1 2"X1j;rdjr 6"X1j;r;s;udu 6X1j;r;s;udAjrsu 6X1j;r;s;udAjrsu 36#)+o(n1)i: 57

PAGE 58

2"ZK()(1j2)d1+1 2X1j;rdZjr 6X1j;r;s;udZu 6X1j;r;s;udZu 12ZM1()K()(1j2)d1+1 36ZM2()K()(1j2)d1+1 2X1j;rdZjr 6X1j;r;s;udZu 6X1j;r;s;udZu 6X1j;r;s;udZAjrsu 6X1j;r;s;udZAjrsu 36K()(1j2)d1)+o(n1)#: 58

PAGE 59

3{20 ),wehave 2"ZK()(1j2)d1+1 12ZM1()K()(1j2)d1+1 36ZM2()K()(1j2)d1+1 2X1j;rdZjr 6X1j;r;s;udZu 6X1j;r;s;udZu 6X1j;r;s;udZAjrsu 36K()(1j2)d1o+o(n1)#p(2)d2 +Z1 2"1 6X1j;r;s;udZu 6X1j;r;s;udZu 6X1j;r;s;udZAjrsu 2Z1 2"X1j;rdZjrIjr 2"1 2X1j;rdZjr Forgivenindexesj;r(1
PAGE 60

Becauseweareonlyinterestedinthetermsdependingonp(2)anditsderivatives,wedividetermsin( 3{21 { 3{24 )whichinvolveanditsderivativesintotwoparts.Forexample,ZAjrsu Also,withthechoiceofp(2)whichvaluesontheboundaryofparameterspaceiszero,onecanprovethatforanytwicedierentiablefunctionof2,sayF(2),ZF(2)pj(2)d2=Z@F(2) 3{22 ),( 3{23 ),( 3{24 )andcombineallthetermswhichdonotinvolvep(2)anditsderivativesintooneterm(S(2)),togetthefollowingexpression: 60

PAGE 61

2"ZK()(1j2)d1+1 2dXj=d1+1dXr=1@ @rZIjrK()(1j2)d1pj(2) 6X1j;r;sddXu=d1+1hZAjrs(qojrqosu+qojuqors+qojsqoru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrs(IjrIsu+IjuIrs+IjsIru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrskjrsu()K()(1j2)d1ipu(2) 2dXj=d1+1dXr=d1+1hZIjrK()(1j2)d1ipj(2)pr(2) AttheendofStep3,allowingp(2)weaklyconvergetothedegeneratedensityoftrue2,weobtainthenalasymptoticexpansionofE[p(2jXn)j2]: 61

PAGE 62

2"ZK()(1j2)d1+1 2IjrK()r(1j2)d1ipj(2) 2dXj=d1+1dXr=1@ @rZIjrK()(1j2)d1pj(2) 6X1j;r;sddXu=d1+1hZAjrs(qojrqosu+qojuqors+qojsqoru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrs(IjrIsu+IjuIrs+IjsIru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrskjrsu()K()(1j2)d1ipu(2) 2dXj=d1+1dXr=d1+1hZIjrK()(1j2)d1ipj(2)pr(2) Thisprovesthetheorem. Tothisend,since=1sothat(1)=2,neglectingalltermswhichdonotinvolvep(2)oritsderivativesandusingtherelationpjr(2) @rpj(2) 62

PAGE 63

2nd2 2Z"ZK()(1j2)d1+1 2dXj=d1+1dXr=1@ @rZIjrK()(1j2)d1pj(2) 6X1j;r;sddXu=d1+1hZAjrs(qojrqosu+qojuqors+qojsqoru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrs(IjrIsu+IjuIrs+IjsIru)K()(1j2)d1ipu(2) 6X1j;r;sddXu=d1+1hZAjrskjrsu()K()(1j2)d1ipu(2) 2Ijr 2(qojrIjr)oK()(1j2)d1ipj(2)pr(2) Lety(2)=(yd1+1(2);;yd(2))=pd1+1() 3{27 )canbeexpressedas soweneedndy(2)tomaximizetheaboveintegral.From Giaquinta ( 1983 ),themaximizershouldsatisfytheEuler-Lagrangeequations: @yi(2)dXj=d1+1@ @j@F @(@yj=@j)=0i=d1+1;;d:(3{29) 63

PAGE 64

2X1j;r;sdZAjrsmojrIsiK()(1j2)d1+1 2dXr=d1+1@ @rZIirK()(1j2)d1;i=d1+1;;d: Heremojr=8><>:nojrwhenj;r2H1Ijrother; Bysolvingthesepartialdierentialequations,onegetsthedivergencepriors.Usually,withmultidimensionalparameterofinterest,theseequationsaresocomplicatedthatitisimpossibletogiveageneralsolutionandsometimes,thereisnosolutiontotheequations. Inthefollowing,wefocusonaspecialcasethatparameterofinterestisone-dimensional.Inthiscase,insteadofseveralpartialdierentialequations,weonlyneedtosolveonedierentialequationandeasilygetageneralformofthedivergencepriors. Whenparameterofinterest2isonedimensional,thatis2=d,thentheEuler-Lagrangeequationbecomes: 2(1jd)d1p0(d)=1 2X1j;r;sdZAjrsmojrIsd(Idd)1 2(1jd)d1+1 2@ @dZ(Idd)1 2(1jd)d1: Bysolving( 3{31 ),onegetsthedivergencepriorp(d)whichisproportionalto 2expZT(d)dd;(3{32) 64

PAGE 65

2(1jd)d1;T(d)=1 2P1j;r;sdRAjrsmojrIsd(Idd)1 2(1jd)d1 2(1jd)d1: )wherep(x)=p(x).Writingh(x)=logp(x)andnotingthath0(x)=h0(x),h00(x)=h00(x)andh000(x)=h000(x),onegetsE@2logf @2;=2Zh00(x)p(x)dx;E@2logf @@;=0;E@2logf @2;=21+2Zxh0(x)p(x)dx+Zx2h00(x)p(x)dx;E@3logf @3;=E@3logf @@2;=0;E@3logf @2@;=32Zh00(x)p(x)dx+Zxh000(x)p(x)dx @3;=32+6Zxh0(x)p(x)dx+6Zx2h00(x)p(x)dx+Zx3h000(x)p(x)dx 3{32 ),thepriorshouldhavethefollowingform:p()/Z(j)d1 2: 3{32 ),thepriorshouldhavethefollowingform:p()/1 2+2Rh00(x)p(x)dx+Rxh000(x)p(x)dx 2.

PAGE 66

xx1 (): 3,onegetsQ()=Z1 2(j)d;T()=4 3{32 ),thepriorshouldhavethefollowingform:p()/7 2Z1 2(j)d: Bernardo ( 1979 )),thedivergencepriorisproportionalto7 2. dlog()andallthemjr=0exceptmdd=Idd.Hence,withAddd=u00(),onegetsQ()=[u0()]1 2;T()=u00() 3{32 ),thepriorshouldhavethefollowingform:p()/[u0()]3 4: 2exp(x)2 4,onegetsQ()=3 2Z1 2(j)d;T()=6

PAGE 67

3{32 ),thepriorshouldhavethefollowingform:p()/21 4Z1 2(j)d: 11 ,when(j)isindependentof,thedivergencepriorispropor-tionalto21 4. 2;T()=2 3{32 ),thepriorshouldhavethefollowingform:p()/3 2:

PAGE 68

Inthischapter,weintroduceanewmatchingcriterionwhichwewillrefertoasthe`momentmatchingcriterion'.Foraregularfamilyofdistributions,theclassicarticleofBernsteinandVon-Mises(seee.g. Ferguson ( 1967 )or Ghoshetal. ( 2006 ))provedtheasymptoticnormalityoftheposteriorofaparametervectorcenteredaroundthemaximumlikelihoodestimatorortheposteriormode.Moregeneralresultsinvolvingtheasymptoticexpansionoftheposteriordensityaredueto Johnson ( 1970 )and Ghoshetal. ( 1982 ).Aconvenientsourceforthisistherecentbookby Ghoshetal. ( 2006 ). Herewemakeuseofthisasymptoticexpansiontondpriorswhichcanprovidehigherordermatchingofthemomentsoftheposteriormeanandthemaximumlikelihoodestimator.Forsimplicityofexposition,weshallprimarilyconneourselvestopriorswhichachievethematchingoftherstmoment,although,itiseasytoseehowhigherordermomentmatchingisequallypossible. Themotivationformomentmatchingpriorsstemsprimarilyfromtwoconsiderations.First,thesepriorsleadtoposteriormeanswhichsharetheasymptoticoptimalityoftheMLE'suptoahighorder.Inparticular,ifoneisinterestedinasymptoticbiasorMSEreductionoftheMLE'sthroughsomeadjustment,thesameadjustmentappliesdirectlytotheposteriormeans.Inthisway,itispossibletoachieveBayes-frequentistsynthesisofpointestimates.Wewilldiscussthisissuemoreintheconcludingsection.ThesecondimportantaspectofthesepriorsisthattheyprovidenewviablealternativestoJereys'priorevenforreal-valuedparametersintheabsenceofnuisanceparameters. InSection 4.1 ofthischapter,fortheregularone-parameterfamilyofdistributions,wecharacterizetheclassofpriorswhichachievesrstordermomentmatching.Examplesaregiventoillustratethemaintheorem.Extensionsoftheseresultstothemulti-parametercasearegivenin 4.2 ,andonceagainanecessaryandsucientconditionfortheexistenceofsuchpriorsisfound.Oneexampleisprovidedtoshownon-existenceofsuchpriors 68

PAGE 69

4.3 Johnson ( 1970 )and BickelandGhosh ( 1990 ). Let^ndenotethemaximumlikelihoodestimatorof.TheBernstein-VonMisesTheoremassertsthatundersomeregularityconditions,theposteriordistributionofTn=p Johnson ( 1970 ); Ghoshetal. ( 1982 ))givesanasymptoticexpansionoftheposterioruptoacertainorder. Johnson ( 1970 )and BickelandGhosh ( 1990 ).ThenanasymptoticexpansionfortheposteriornofSnisgivenby 2a3s3 2^In00(^n) 69

PAGE 70

5 involvesonlyderivativesoflog-likelihood,andtypicallyismuchsimplertoevaluate,especiallyinthemultiparametercase,whileanEdgeworthexpansioninvolvesseveralmixedcumulants. ThefollowingsimpletheoremwhichfollowsfromTheorem 5 isthestartingpointofthepresentresearch.TheformalproofmakesuseofthemomentsoftheN(0;1)distribution.Arigorousproofisgivenin Ghoshetal. ( 1982 ).Theresultisalsostatedin Ghosh ( 1994 )and Ghoshetal. ( 2006 ).Wehavecorrectedaminortypointhesesources. 5 .Then ^In0(^n) Nextobservethatbythelawsoflargenumbersandconsistencyofthemaximumlikelihoodestimator,conditionalon, 2I2()+1 whereg3()=E@3logf(X1j) Withthechoice0() 2I(),i.e. 2Zg3(t) onegets^Bn;^n=oP(n1).WeshalldenotethispriorbyM(). WenowprovidearesultwhichprovidesageneralexpressionforthematchingpriorM(),whereisaone-to-onefunctionof. d3 2. 4{4 ),onegetsM()=exp1 2Zg3()

PAGE 71

d2:(4{5) NextusetheBartlettidentity( Bartlett 1953 ) 0=g3()+3Ed2logf d2dlogf d+Edlogf d3(4{6) and d2dlogf d:(4{7) From( 4{5 ){( 4{7 ), 2I0()+1 2Edlogf d3=3 2"I0()d d3+2I()d dd2 d2#+1 2Edlogf d3d d3: Sinceg3()=3 2I0()+1 2Edlogf d3;( 4{8 )leadsto d33I()d dd2 d2:(4{9) Hence,from( 4{9 ),M()=exp1 2Zg3() 2Zg3() ddexp2643 2ZI()d dd2 d2 d2d375=M()exp3 2Zd2=d2 2logd d=M()d d3 2:2 71

PAGE 72

2RI0()=I()d]=I1=2()whichisJereys'prior.Ontheotherhand,forthepopulationmean=0()whichisastrictlyincreasingfunctionof(since00()=V(Xj)>0),byTheorem 7 ,M()=M()d d3 2=I1=2()1 d2=00()(00())2=(00())1:Here,insteadofJefreys'prior,namelythesquarerootoftheFisherinformation,wegettheFisherinformationitself.Inparticular,forthebinomialproportionp,thisleadstotheHaldanepriorH(p)/p1(1p)1. dx(p0(x)=p(x))dx=c2(say)andg3()=Rd2 2):Jereys'priorinthiscasecontinuestobeJ()=1: 72

PAGE 73

2c2: 2.Fortheexponentialdistributionwithscaleparameter,h0(x)=1andh00(x)=h000(x)=0sothatc2=1c3=4.Hence,M=2.Forthedoubleexponentialdistributionwithmedian0andscaleparameter,h0(x)=sgn(x)andh00(x)=h000(x)=0foralmostallx.Hence,c2=1;c3=4andonceagainM=2.FortheCauchydistributionwithlocationparameter0andscaleparameter,c2=1=2andc3=3=2.ThisleadstoM=3 2. DattaandMukerjee 2004 )underthepriorisn(tjX1;;Xn)=(2)d 2tTIn(^)t1+n1 2nR1(t)+1 6R3(t)o+OP(n1);

PAGE 74

@j=^;R3(t)=n1nXi=1dXj=1dXr=1dXs=1tjtrts@3logf(Xij) 2E(TnjX1;;Xn)=n1Un+1 2Vn+oP(n1); 2tTIn(^)tdt=dXk=1@log() 2V;whereU=(U1;;Ud)T; @kIjkandV=(V1;;Vd)T;Vj=dXk=1dXr=1dXs=1Ijk()Irs()E@3logf(Xj)=@k@r@s: 2V=0;therebyleadingtoI1()rlog=1 2I1()b;whereb=(b1;;bd)T; bk=dXr=1dXs=1E@3logf(Xj) 74

PAGE 75

2bwhenitexists. Werstillustratetheabovegeneralresultwithafewexamples. @@T=I() @j=1 2dXr=1dXs=1@Irs() Anderson ( 1986 )(p.598),therighthandsideof( 4{11 )equalsto1 2@logjI()j NextobservethatE(Xnj)=@ @=(say),andisone-to-onefunctionofsinced @@T=I()whichispositivedenite.Also,I()=I1().Furthermore,E@3logf(xj) 4{10 )becomestobk=2dXr=1dXs=1@Irs() Then,bysolvingthedierentialequation@log @k=@logjI()j

PAGE 76

Jorgensen ( 1997 ).Thisgeneraltwo-parameterclassofprobabilitydensityfunctionsisgivenbyf(xj;)=a()c(x)exp[t(x;)]: @=@t @;@logf @=u()+t(x;); @2=@2t @2;@2logf @@=@t @;@2logf @2=u0(): @j;=0;theFisherinformationmatrixI(;)=Diag[I(;);u0()];whereI(;)=Eh@2t @2j;i:ThusandareorthogonalinthesenseofCoxandReid(1987).Further,@3logf @3=@3t @3;@3logf @2@=@2t @2;@3logf @@2=0;@3logf @3=u00():Now,Eh@3logf @2@j;i=Eh@2t @2j;i=1I(;):Thusthemomentmatchingprior(whenitexists)mustsatisfythedierentialequations(i)@log @=1 2Eh@3t @3j;i @2j;i @=1 2u00() 2Z8<:E@3t @3j; @2j;9=;35(4{12) 76

PAGE 77

2; 2. GarvanandGhosh ( 1997 ).Theyalsostudiedtheproperdispersionmodelbuttheirfocusisonprobabilitymatchingpriors.Whenistheparameterofinterestandisthenuisanceparameter,therstorderprobabilitymatchingprioris:(;)=E1 2@2t(x;) 2g(): xx1 (): dlog()isthedigammafunction.Thusu0()=11():Moreover,t(x;)=x +logx sothat@t @=x 21 @2=2x 3+1 @3=6x 42 @2j;=1 @3j;=4

PAGE 78

4{12 )simpliestoexp(2=).Thenthemomentmatchingprior(;)is(;)=2[0()1]1 2: 2exp(x)2 2x 22 2;u0()=1 22andu00()=1 @=x 31 @2=3x 4+2 @3=12x 56 @2j;=1 @3j;=6 4{12 )simpliestoexp(3log)=3:Therefore,themomentmatchingprior(;)=31=2: 2I0();I0()=1 2Zexp(cosx)dx: @=sin(x);@2t @2=cos(x);@3t @3=sin(x);sothatE@3t @3j;=E@t @j;=0:Furtheru()=I00() dhI00() dI00() 2=d2 2: )wherep(x)=p(x).Writingh(x)=logp(x)andnotingthath0(x)=h0(x),h00(x)=h00(x)andh000(x)=h000(x),onegetsE@2logf @2;=2Zh00(x)p(x)dx;E@2logf @@;=0;E@2logf @2;=21+2Zxh0(x)p(x)dx+Zx2h00(x)p(x)dx;E@3logf @3;=E@3logf @@2;=0;

PAGE 79

@2@;=32Zh00(x)p(x)dx+Zxh000(x)p(x)dx @3;=32+6Zxh0(x)p(x)dx+6Zx2h00(x)p(x)dx+Zx3h000(x)p(x)dx: 2c. Asspecialcases,recallthatfortheN(;2)distribution,h0(x)=x,h00(x)=1andh000(x)=0.Hence,c=2+266 121=7sothat(;)/7 2.FortheCauchydistributionwithlocationparameterandscaleparameter,recallthath0(x)=2x 1+x2+4x2 2b:(4{13) Ifthereisnosolutiontotheseequations,thenmomentmatchingpriorsdonotexist.Hereisanexamplefornon-existenceofsuchpriors. 79

PAGE 80

x1 @2;=d2log() @@;=1 @2;= 2;E@3logf @3;=d3log() @2@;=0;E@3logf @@2;=1 @3;=4 3: 4{13 ),themomentmatchingpriorshouldsatisfythefollowingequations:@log @=T1();@log @=1 6 ,onegets(^n;^n)m=nma3 ^In0(^n) 80

PAGE 81

4{4 )alsoleadsthesameprior()=exp1 2Zg3(t) 81

PAGE 82

Inthisdissertation,Irevisittheproblemofselectionofobjectivepriorsandmakesomenewcontributions.Igeneralizetheresultfrom Ghoshetal. ( 2009 )totheregularmultiparameterfamilyofdistributionswithandwithoutnuisanceparameters. Withoutnuisanceparameters,theobjectivepriorsarefoundbymaximizingaverygeneralclassofdivergencemeasuresbetweenthepriorandtheposterior.ThisclassofdivergencemeasuresincludestheKullback-Leibler,Bhattacharyya-Hellingerandchi-squaredivergence.Itisshownthat,intheinteriorofthisgeneralclassofdivergencemeasures,Jereys'prioristhedesiredprior.Ontheboundary,Ihavegivenanexpressionoftheoptimalpriorsandshowedthat,forsomecommonfamiliesofdistributions,theoptimalpriorsmaybedierentfromJereys'prior.Furthermore,outsidetheboundary,Jereys'priorturnsouttobetheminimizerratherthanmaximizerofthedistance,andtheredoesnotexistanypriorwhichmaximizesthedivergencebetweentheposteriorandtheprior. Withthepresenceofnuisanceparameters,underthesameclassofdivergencemeasures,Icancharacterizeoptimalpriorsforeverymemberinthisclassbyusingthetwo-stepprocedureproposedby Bernardo ( 1979 ).Theexplicitexpressionsoftheoptimalpriorsundereverydivergencemeasure(exceptforchi-squaredivergence)aregiven.Especially,forKullback-Leiblerdivergencewhichwasstudiedby Bernardo ( 1979 ),Ireachthesamepriorashefound.Underthechi-squaredivergence,Ihaveshownthattheobjectivepriorshouldbethesolutiontoasetofpartialdierentialequations.Ialsoconsideraspecialcasewhentheparameterofinterestisonedimensional.Inthiscase,theclosedformsoftheoptimalpriorsareprovidedandalsoseveralexamplesaregiventoillustratehowtogetthesepriorsinpractice. Inthenalpart,Iintroduceanewcriterionforobjectivepriors,themomentmatchingcriterion,whichrequiresthematchingoftheposteriormeanwiththemaximumlikelihoodestimatoruptoahighorderofapproximation.Theclassofpriorsischaracterized 82

PAGE 83

83

PAGE 84

84

PAGE 85

85

PAGE 86

86

PAGE 87

Ruitaowasbornin1975inBeijing,China.Heearnedhisbachelor'sdegreefromBeijingUniversityofTechnologyin1999,majoringincomputationalmathematics.Oneyearlater,heenrolledasagraduatestudentatBeijingUniversity,wherehereceivedthedegreeofMasterofScienceinprobabilityandstatisticsin2003.HeenrolledinthePh.D.programintheDepartmentofStatisticsattheUniversityofFloridainthefallof2004andreceivedhisPh.D.fromtheUniversityofFloridainthesummerof2009.Upongraduation,hewilljointheUniversityofIowaasavisitingassistantprofessorintheDepartmentofStatisticsandActuarialScience. 87