The Stereotype Model and the Existence of Maximum Likelihood Estimates in Categorical Regression Models

MISSING IMAGE

Material Information

Title:
The Stereotype Model and the Existence of Maximum Likelihood Estimates in Categorical Regression Models
Physical Description:
1 online resource (167 p.)
Language:
english
Creator:
Ozrazgat Baslanti,T
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Statistics
Committee Chair:
Agresti, Alan G
Committee Co-Chair:
Daniels, Michael J
Committee Members:
Brumback, Babette
Henretta, John C

Subjects

Subjects / Keywords:
categorical -- estimates -- existence -- likelihood -- logistic -- maximum -- mle -- model -- ordinal -- partial -- stereotype
Statistics -- Dissertations, Academic -- UF
Genre:
Statistics thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
In this dissertation, we focus on two main topics: the stereotype model and the existence of maximum likelihood estimates in categorical regression models. Despite the many appealing features, the stereotype model has not been widely used because of the problems in estimation due to nonlinearity in the parameters. However, it can be easily fit with current software. We compare standard confidence interval methods for the odds ratios. We also introduce a partial stereotype model, which is more parsimonious than the multinomial logistic model, but yet, more flexible than the stereotype model. We determine identifiability constraints needed and examine some properties of the partial stereotype model. We show how the standard confidence interval methods extend to the partial stereotype model. Simulation studies are done to examine the approximate distribution of the LRT statistic used to compare the fit of the partial stereotype model with the stereotype model and the multinomial logistic model. Another simulation study is done to compare the performance of the LRT and AIC in choosing the correct model among these types of models. The problems in fitting stereotype and partial stereotype models for some cases led a study on the existence of maximum likelihood estimates for categorical regression models in general. We summarize the existing literature, generalize some results, and examine connections between the different approaches.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by T Ozrazgat Baslanti.
Thesis:
Thesis (Ph.D.)--University of Florida, 2011.
Local:
Adviser: Agresti, Alan G.
Local:
Co-adviser: Daniels, Michael J.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-08-31

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043178:00001


This item is only available as the following downloads:


Full Text

PAGE 2

2

PAGE 3

3

PAGE 4

Ideeplyacknowledgethetremendoushelp,encouragementandendlesssupportthatIreceivedfrommyadvisors,Prof.AlanAgrestiandProf.MichaelDaniels.IwouldalsoliketothankProfessorsBabetteBrumbackandJohnHenrettaforservingonmysupervisorycommitteeandofferingmehelpfulcomments.IoweadebtofgratitudetoallofmyteachersattheDepartmentofStatisticsforprovidingawell-roundededucationandtostudentsandstafffortheirkindnessandsupport.Specialthanksgotomymomandmymother-in-lawfortheirunwaveringsupportandhelp.Iamthankfultoallofmyfamilymembersandfriendsforbeingaconstantsourceofencouragementduringthisjourney.Finally,Iwouldliketothankmyhusband,Ugurforalwaysbeingthereforme. 4

PAGE 5

page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 8 LISTOFFIGURES ..................................... 11 ABSTRACT ......................................... 12 CHAPTER 1THESTEREOTYPEMODEL ............................ 13 1.1LogisticModelsforMultinomialResponses ................. 13 1.1.1Baseline-CategoryLogitModels ................... 13 1.1.2CumulativeLogitModels ........................ 14 1.1.3Adjacent-CategoriesLogitModels .................. 16 1.1.4Continuation-RatioLogitModels ................... 16 1.2StereotypeModel ................................ 17 1.2.1ConnectionBetweentheStereotypeModelandOtherLogisticModels .................................. 19 1.2.2DimensionalityandIndistinguishability ................ 21 1.2.3InferencewiththeStereotypeModel ................. 23 1.2.4SubsequentResearchontheStereotypeModel ........... 24 1.2.5MethodsofEstimatingtheModelParameters ............ 26 1.2.6ComparisonofEstimatesfromDifferentMethods .......... 30 1.2.6.1Example1:Handeczemaprognosis ............ 30 1.2.6.2Example2:Backpainprognosis .............. 32 1.3OutlineofDissertation ............................. 36 2CONFIDENCEINTERVALSFORTHESTEREOTYPEMODEL ......... 38 2.1BasicIntroductionofThreeCondenceIntervals .............. 38 2.1.1WaldCondenceInterval ....................... 39 2.1.2LikelihoodRatioTest(LRT)-BasedCondenceInterval ....... 39 2.1.3ScoreCondenceInterval ....................... 39 2.2SimulationStudy ................................ 41 2.2.1DesignofConditions .......................... 42 2.2.2EvaluationCriteriaforCondenceIntervals ............. 43 2.2.2.1Coverageprobability ..................... 44 2.2.2.2Expectedlength ....................... 45 2.2.3ComparisonofMethods ........................ 45 2.2.4ProblematicCases ........................... 47 2.3Summary .................................... 57 5

PAGE 6

........................ 58 3.1IntroductionofaPartialStereotypeModel .................. 58 3.1.1IdentiabilityConstraintsandtheNumberofParameters ...... 59 3.1.2AlternativeWayofWritingthePartialStereotypeModel ...... 62 3.1.3OddsRatiosforthePartialStereotypeModel ............ 65 3.1.4SomePropertiesofthePartialStereotypeModel .......... 66 3.1.5CondenceIntervalsforthePartialStereotypeModel ........ 68 3.1.5.1Waldcondenceinterval .................. 68 3.1.5.2Likelihoodratiotest(LRT)-basedcondenceinterval ... 69 3.1.5.3Scorecondenceinterval .................. 69 3.2ComparisonoftheFitofthePartialStereotypeModeltotheOtherModels 71 3.2.1ModelSelectionCriteria ........................ 71 3.2.2SimulationStudyontheDistributionoftheLikelihoodRatioTestStatisticsUnderH0WhenThereisNoLackofIdentiability .... 72 3.2.2.1Simulationstudyontestingindependenceforthestereotypemodel ............................. 74 3.2.2.2Simulationstudyontestingindependenceforthepartialstereotypemodel ...................... 76 3.2.2.3Simulationstudyontestingindependenceforthemultinomiallogisticmodel ......................... 77 3.2.2.4Simulationstudyoncomparisonoftsofthestereotypemodelandthepartialstereotypemodel .......... 78 3.2.2.5Simulationstudyoncomparisonoftsofthestereotypemodelandthemultinomiallogisticmodel ......... 79 3.2.2.6Simulationstudyoncomparisonoftsofthepartialstereotypemodelandthemultinomiallogisticmodel ......... 80 3.2.2.7SimulationstudyonproportionsofchoosingnullmodelusingLRTstatistics ..................... 81 3.2.2.8SimulationstudyonproportionsofchoosingnullmodelusingAIC ........................... 82 3.2.3SimulationStudyontheDistributionoftheLikelihoodRatioTestStatisticsUnderH0WhenThereisLackofIdentiability ...... 83 3.2.4ExampleforthePartialStereotypeModel .............. 86 3.3Summary .................................... 89 4EXISTENCEOFTHEMAXIMUMLIKELIHOODESTIMATESFORCATEGORICALREGRESSIONMODELS .............................. 91 4.1Introduction ................................... 91 4.2ExistenceofMaximumLikelihoodEstimatesforDichotomousResponseModels ..................................... 92 4.3ExistenceofMaximumLikelihoodEstimatesforLogisticRegressionModelsUsingBaseline-CategoryLogits ........................ 105 4.3.1TheMultinomialLogisticModel .................... 105 4.3.2MultinomialChoiceModels ...................... 109 6

PAGE 7

......................... 112 4.3.4TheAdjacent-CategoriesLogitModel ................ 113 4.4ExistenceofMaximumLikelihoodEstimatesforCategoricalRegressionModelsUsingCumulativeLinks ........................ 114 4.4.1NecessaryandSufcientConditionsfortheExistenceofMaximumLikelihoodEstimates .......................... 114 4.4.2GeneralizationsandModicationsoftheTheoremsfortheExistenceofMaximumLikelihoodEstimates ................... 119 4.4.2.1ModicationofTse'stheorem ................ 119 4.4.2.2GeneralizationofChenandShao'stheorems ....... 122 4.5SeparationDetection .............................. 128 4.6Examples .................................... 130 5SUMMARYANDFUTURERESEARCH ...................... 135 APPENDIX ADERIVATIONSOFTHEELEMENTSOFINFORMATIONMATRIX ....... 140 A.1TheElementsoftheInformationMatrixfortheStereotypeModel ..... 140 A.2TheElementsoftheInformationMatrixforthePartialStereotypeModel 144 BPROOFS ....................................... 149 REFERENCES ....................................... 163 BIOGRAPHICALSKETCH ................................ 166 7

PAGE 8

Table page 1-1Handeczemadatafromaprospectivecohortstudyofapprenticesinthecarindustry ........................................ 31 1-2Comparisonofestimatesobtainedforstereotypemodelttedtohandeczemadata .......................................... 33 1-3Countsforeachcategoryofbackpain ....................... 34 1-4Comparisonofsstimatesobtainedforstereotypemodelttedtoback-paindata 35 1-5Comparisonofestimatesforstereotypemodelttedtoback-paindatawhenindistinguishablecategoriesarecombined ..................... 35 2-1CellprobabilitiesusedtosimulatedatasetswhenL=3 .............. 43 2-2CellprobabilitiesusedtosimulatedatasetswhenL=5 .............. 43 2-3Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhen1=1 ......................................... 48 2-4Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhen2=1 ......................................... 49 2-5ComparisonofthemeanandmedianlengthsofCIsforlargesamplesizeswhen1=1 ...................................... 50 2-6ComparisonofthemeanandmedianlengthsofCIsforsmallsamplesizeswhen1=1 ...................................... 51 2-7ComparisonofthemeanandmedianlengthsofCIsforlargesamplesizeswhen2=1 ...................................... 52 2-8ComparisonofthemeanandmedianlengthsofCIsforsmallsamplesizeswhen2=1 ...................................... 53 2-9Theaverageofcoverageprobabilitiesandtheaverageofabsolutedifferencesbetweenthecoverageprobabilityandthenominalvalue(0.95)forlargeandsmallsamplesizes .................................. 54 2-10CellProbabilitiesUsedtoSimulateDatasetsWhenL=3 ............ 55 2-11Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhenthetruevaluefor2=0 ............................... 56 3-1Comparisonofnumberofparametersforthemultinomiallogistic,thestereotypeandthepartialstereotypemodel .......................... 65 8

PAGE 9

........ 74 3-3Cellprobabilitiesusedtosimulatedatasetsforpartialstereotypemodels .... 74 3-4ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthestereotypemodel ............ 76 3-5ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthepartialstereotypemodel ........ 76 3-6ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthemultinomiallogisticmodel ....... 77 3-7ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthepartialstereotypemodel ......................................... 78 3-8ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthemultinomiallogisticmodel ......................................... 79 3-9ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthepartialstereotypemodelandthemultinomiallogisticmodel ..................................... 80 3-10ProportionofchoosingnullmodelusingLRT(=0.05) ............. 82 3-11ProportionofchoosingnullmodelusingLRT(=0.1) .............. 82 3-12ProportionofchoosingnullmodelusingAIC .................... 83 3-13Cellprobabilitiesusedtosimulatedatasetsforstereotypemodels ........ 84 3-14Stereotype,partialstereotypeandmultinomiallogisticmodelswithsmallestAICvalues ...................................... 87 3-15MLestimatesforthepartialstereotypemodel,ttedtorespiratoryillnessdata,withtheminimumAIC ................................ 87 4-1Dataexhibitingcompleteseparationwhenthereisasinglequantitativeexplanatoryvariable ................................. 94 4-2Dataexhibitingquasi-completeseparationwhenthereisasinglequantitativeexplanatoryvariable ................................. 94 4-3Dataexhibitingcompleteseparationwhenthereisabinaryexplanatoryvariable 96 4-4Dataexhibitingcompleteandquasi-completeseparationwhenthereisaqualitativeexplanatoryvariable .......................... 97 9

PAGE 10

................................. 97 4-6MarginaltablesforthedatagiveninTable 4-5 ................... 97 4-7PartialtablesforthedatagiveninTable 4-5 (foreachlevelofx1) ........ 97 4-8PartialtablesforthedatagiveninTable 4-5 (foreachlevelofx2) ........ 97 4-9Haberman'smethodofdetectingexistenceofMLEwhentherearetwobinaryexplanatoryvariablesandabinaryresponsevariable ............... 98 4-10AnapplicationofHaberman'smethodwhenMLEdoesnotexist ........ 98 4-11AnapplicationofHaberman'smethodwhenMLEexists ............. 98 4-12DatathatisusedtoillustrateChenandShao'smethod .............. 104 4-13DatagiveninTable11 ................................ 104 4-14Examplesofcompleteseparation .......................... 107 4-15Examplesofquasi-completeseparation ...................... 108 4-16Examplesofpartialquasi-completeseparation .................. 109 4-17Anexampleofquasi-completeseparationtoillustrateTse'stheorem ...... 119 4-18Theexampleofquasi-completeseparationusedtoillustrateTse'stheorem .. 121 4-19Anexamplewheretherearenoconcordantpairs ................. 121 4-20AnexampleforwhichMLEExistsforthecumulativelogitmodel ......... 124 4-21TheexampleforwhichtheMLEexistsforthecumulativelogitmodel ...... 127 4-22TheexampleforwhichtheMLEdoesnotexistforthecumulativelogitmodel 127 4-23DataforExample1 .................................. 130 4-24DataforExample2 .................................. 132 4-25CountdataforExample2 .............................. 133 4-26DataforExample3 .................................. 133 10

PAGE 11

Figure page 3-1ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthestereotypemodel .................. 75 3-2ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthepartialstereotypemodel ............... 77 3-3ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthemultinomiallogisticmodel .............. 78 3-4ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthepartialstereotypemodel .... 79 3-5ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthemultinomiallogisticmodel ... 80 3-6ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthepartialstereotypemodelandthemultinomiallogisticmodel ......................................... 81 3-7ComparisonoftheSMandthePSMwhenthereislackofidentiability ..... 84 3-8ComparisonoftheSMandtheMLMwhenthereislackofidentiability .... 85 3-9ComparisonofthePSMandtheMLMwhenthereislackofidentiability .... 85 4-1Theloglikelihoodasafunctionoftheslopeforthecompleteseparationexample 95 4-2Theloglikelihoodasafunctionoftheslopeforthequasi-completeseparationexample ........................................ 95 4-3Anexampleofcompleteseparation ........................ 100 4-4Anexampleofquasi-completeseparation ..................... 100 4-5Anexampleofoverlap ................................ 100 4-6Representationofthevepossibledatacongurations .............. 108 11

PAGE 12

12

PAGE 13

Agresti 2002 ).Wethenreviewthelesspopularstereotypemodelinmoredetail. logPs(x) (1) whereL=0andL=0.Theregressioncoefcientofxkissk,whichisthelogoddsratiocomparingcategorysversuscategoryLforeachone-unitincreaseinxk.Themultinomiallogisticmodelinducesthefollowingresponseprobabilities:Ps(xi)=P(Y=sjxi)=exp(s+0sxi) Greenland ( 1994 ),isthatthemodelcoefcientsareeasytointerpret.Anotheradvantageisthatthemodelispreservedbyoutcome-stratied(case-control)sampling,wherethecasesoroutcomelevelsareidentiedrstandtheirexposurestatusisdeterminedlateron.Thismodelassumesthatthepredictorvariableshavedifferenteffectsatdifferentoutcomelevels.Adisadvantage 13

PAGE 14

1+exp(s+0xi),s=1,...,L1.SinceP(Yisjxi)increasesinsforeachxedvalueofexplanatoryvariablesandthelogitisanincreasingfunctionofthisprobability,theinterceptparameters,fsg,arealsoincreasingins.Astheeffectsofx,,arethesameforthelogitsofthedifferentcumulativeprobabilities,therearefewerparameterstobeestimatedcomparedtothebaseline-categorylogitmodelthathasseparateparameters,s,dependingonthecategory.Thecumulativeoddsratiofortwodifferentcovariatevectorsxandxequalsexp(0(xx)),whichisindependentoftheresponsecategory.Sincethelogcumulativeoddsratioisproportionaltothedistancebetweenxandxwithoutdependingontheresponsecategory,thismodelisoftenreferredtoastheproportionaloddsmodel.However,thisassumptionmaynotbeappropriateinsomecasesandneedstobechecked( PetersonandHarrell 1990 ). 14

PAGE 15

PetersonandHarrell ( 1990 ),isamodelthatfallsbetweentheproportionaloddsversionandthegeneralizedversionofthecumulativelogitmodel.Itiswrittenas logitP(Yisjxi)=s+0xi+0swi,s=1,...,L1. (1) wheres=(s1,s2,...,sq)isaq1,q
PAGE 16

logPs(xi) (1) whereuis=(Ls)xi.Sinceadjacent-categorieslogitmodelsusepairsofadjacentcategories,theregressioncoefcientsaredescribedwithlocallogoddsratiosthatusecellsinadjacentrowsandadjacentcolumns. 16

PAGE 17

( 1984 )describedtwomajortypesofordinalcategoricalvariables.Eitherthevariableisdirectlyrelatedtoasingle,underlyingcontinuousvariableoritisgeneratedbyanassessorwhomakesthejudgmentofthegradeoftheorderedvariableafterprocessinganindeterminateamountofinformation.Groupinginherentlycontinuousvariablessuchasincomeandweightwouldgivethersttypeofvariable,wheresuchvariablescouldbecalled`groupedcontinuous'.Asanexampleforthesecondtypeofvariable,Andersongavetheexampleof`extentofpainreliefaftertreatment'beingworse,same,slightimprovement,moderateimprovement,markedimprovementorcompleterelief. Kuss ( 2006 )gavetheexampleofhandeczema,whereeachhandeczemaisdifferentintermsofobjectivecriteriasuchassize,erythema,dryness,anddegreeofscaling.Also,subjectivecriteriasuchasdegreeofitchingordegreeofannoyanceandhencetheseverityofthediseaseisevaluatedsubjectively.Andersonnotedthatalthoughthereisasingle,unobservable,continuousvariablerelatedtothisorderedscale,thedoctormakingtheassessmentwilluseseveralpiecesofinformationinmakinghisjudgmentontheobservedcategorybasedonhisorherconceptualstereotypesofmildly,moderate,andseverelyillpatients.Andersonreferredtothesecondtypeofvariableasan`assessed'orderedcategoricalvariable.Inthesekindsofcases,althoughthecategoriescanbeassumedtobeordered,thestructureof 17

PAGE 18

Anderson ( 1984 )proposedamodelthatmightbemoreappropriatethanothermodelsandnameditthestereotypemodel.Thestereotyperegressionmodelisobtainedbytakingfsgtobeparallelinthemultinomiallogisticmodel( 1 ),thatisbytakings='s,andisdenedas logPs(xi) (1) whereL=0and'L=0.Since's=('s=C)(C)foranynonzeroconstantC,theseparametersarenotidentiableunlessanotherconstraintisimposed.Andersontook'1=1,toavoididentiabilitydifculties.Iftherstcategoryischosenasthereferencecategory,thenwelet1=0,'1=0,and'L=1.Anotherpossibilityistosetoneoftheparameters,say1,equalto1.Thus,thestereotypemodelhas(L1)+p+(L2)parameters,anditismoreparsimoniousthanthemultinomiallogisticmodel.Theestimatesobtainedusingonesetofidentiabilityconstraintscanbeeasilytransformedtoestimatesthatwouldbeobtainedunderadifferentsetofidentiabilityconstraints,usingtheequivalenceoflogoddsratios.Differentidentiabilityconstraintsareoftenbetterfordifferentdatasetsintermsofgettingfasterconvergence.Responseprobabilitiesinducedby( 1 )arePs(x)=P(Y=sjx)=exp(s+'s0x) 18

PAGE 19

Anderson ( 1984 )alsoproposedanorderedstereotypemodelwithorderedscores:1='1'2...'L=0.Oddsratiosarealsomonotonewhen'sparametersaremonotone.Inthiscase,thelogoddsratiosfortwocategoriesgrowslargerinabsolutevalueascategoriesgetfartherapart. 1 )aslogPs(x) 19

PAGE 20

Goodman ( 1979 ).TheRCmodelisaspecialcaseofthesaturatedloglinearmodelforIJtwo-waytables.Thesaturatedmodeliswrittenaslogij=+Xi+Yj+XYij.TheRCmodelisobtainedasfollows:logij=+Xi+Yj+ij,bytakingXYij=ij,whererowscoresfigandcolumnscoresfjgareparameters.Thismodelisnotloglinearsincethepredictorisamultiplicativefunctionofparametersiandj.TheRCmodelcanbegeneralizedbyreplacingthetermijbyXYij=MXk=1kikjk,whereM=min(I1,J1).Whenk=0fork>M,themodeliscalledtheRC(M)model.Thelinear-by-linearassociationmodelisaspecialcaseoftheRCmodel,wherefigandfjgarexedmonotonescoresinsteadofparameters.TheRCmodelsimpliestotheRmodelifthecolumnscoresarexedandtotheCmodeliftherowscoresarexed.TreatingYastheresponsevariableandXastheexplanatoryvariable,thebaseline-categorylogitsarelogP(Y=sjX=i) Agresti ( 2010 )showedtheRCmodelhastheform logP(Y=sjX=i) (1) whens=YsYL,i=i,and's=sL,andthus,itisthespecialcaseofthestereotypemodelsincetheright-handsideofthisequationhastheform:s+'s0x, 20

PAGE 21

1 )isaone-dimensionalstereotypemodel.Ifmorethanonelinearfunctionisrequiredtodescribetherelationship,thenitismulti-dimensional.Andersonwrotethetwo-dimensionalmodelbytakings='s+s,s=1,...,L.Anm-dimensionalstereotypemodelcanbewritteninasimilarwaybytakings='s11+'s22+...+'smm. Lunt ( 2005 )gaveanexample,whereatwo-dimensionalstereotypemodelisappropriate,asfollows:`Considerthatadiseaseoutcomehasfourlevels:none,mild,moderateandsevere,andonesetofvariablesdeterminesusceptibilitytoadisease,butdifferentvariablesareimportantindeterminingtheseverityofthedisease.Inthiscase,twolinearcombinationswouldbeused:onecombinationtodeterminetheprobabilityofbeingclassedasmild,moderateorsevereratherthannone,andtheothertodeterminehowfaralongtheprogressionfrommildthroughmoderatetoseverethesubjectislikelytohavepassed.'Hementionedthattheproportionaloddsmodelisnotappropriateinthiscase,sinceitusesasinglecombinationtodeterminetheprogressionfromnonetosevere,andhencecannotcapturethefactthatdifferentvariablesareimportantatdifferentstagesofthedisease.Ontheotherhand,themultinomiallogisticmodelassumesthattheeffectofeachvariableisdifferentateachstageofthediseasehavingaseparatecoefcients,s=1,...,L.AndersonstatedthatiftheresponsevariableisformedofLcategoriesand 21

PAGE 22

Lunt ( 2005 )statedthatastereotypemodelwiththemaximumdimensionpossibleisequivalenttoamultinomiallogisticmodelintermsofpredictedprobabilitiesandthelikelihood,buttheactualparametersttedandtheirinterpretationsdiffer.AnotherconceptAndersonemphasizedisindistinguishability.Ifxisnotpredictivebetweentwocategories,thenthatpairofcategoriesisindistinguishablewithrespecttox.So,whenthestereotypemodelholdswith's='t,thenthecategoriessandtaresaidtobeindistinguishable.Sincethelogoddsratiocomparingvaluesxj1andxj2ofanexplanatoryvariablexj,intermsofobservingtheresponseincategorysort,isgivenby('s't)j(xj1xj2),thelogoddsratiois0when's='t.Also,Ps(x)=Pt(x)=exp(st)isindependentofxandprobabilityratiosareequal,namely,Ps(x)=Ps(x0)=Pt(x)=Pt(x0)forsomexandx0,whenthepairofcategoriessandtisindistinguishable.OnecantestwhethercategorysandcategorytareindistinguishablewithrespecttoxbytestingH0:s=tforthemultinomiallogisticmodel.ThissimpliestoH0:'s='tfortheone-dimensionalstereotypemodel.Thecategoriesarelikelytobeindistinguishableiff'lgvaluesforthosecategoriesaresimilar.Simplicationscanbeobtainedbycombiningcategoriesthatareindistinguishable,andonecangetamoreparsimoniousmodelinthiscase.Andersonstatedthatiftheestimatesforf'lgparametersarenotmonotoneandtheirstandarderrorsarehigh,thissuggestsquestionsaboutdistinguishability.Inthatcase,itmighthelptocombinecategoriesthatareindistinguishabletogetmonotonicityforf'lgparameters.Andersonnoted 22

PAGE 23

Anderson ( 1984 ),McCullaghandKentstatedthattheasymptoticnulldistributionofthelikelihoodratiostatisticfortestingindependenceisthesameasthedistributionofthelargesteigenvaluefromaWishartmatrix,whichisaresultgivenby Haberman ( 1981 )forthespecialcaseofGoodman'sRCmodel.LetW(r1,s1)bether1byr1centralWishartmatrixwiths1degreesoffreedom.HabermanwrotetheelementsofWas (1) whereij,1ir1,1js1,areindependentN(0,1)randomvariables.HabermanstatedthatifwedenotethemaximumeigenvalueofW(r1,s1)byF(r1,s1),thenF(r1,s1)hasachi-squareddistributionwithr1degreesoffreedomifs=2,andwiths1degreesoffreedomifr=2.However,F(r1,s1)doesnothaveachi-squareddistributionotherwise.ThedistributionofthelikelihoodratiostatisticfortestingindependenceforthestereotypemodelisthesameasthedistributionofthelargesteigenvaluefromaWishartmatrixsinceRCmodelisaspecialcaseofthestereotypemodelasshownin( 1 )whentheexplanatoryvariableisnominal.However,thelikelihoodratioteststatisticfortestingindependencehasanapproximatechi-squareddistributionwithp(L1)degreesoffreedomwhenthealternativemodelisthed-dimensionalmodel,whichisequivalenttothemultinomiallogisticmodel,whered=min(p,L1). 23

PAGE 24

3 .Itisalsopossibletotesttheindistinguishabilityofasubsetofresponsecategoriesusingtheordinarymethodssincethelikelihoodratioteststatistichasanapproximatechi-squareddistributioninthiscase. AnanthandKleinbaum ( 1997 )reviewedmodelsforordinalresponsesandtallthemodelsexceptforthestereotypemodelduetothelackofsoftware. DiPrete ( 1990 )usedthestereotypemodelinsocialmobilityanalysissincehedidnotndthelinearregressionandloglinearmodelsthathavebeenusedinstudyofsocialmobilitysufcient. HendrickxandGanzeboom ( 1998 )usedthestereotypemodeltoanalyzeoccupationalstatusattainment. HoltbruggeandSchumacher ( 1991 )usedthestereotypemodelinbiomedicalresearchtocomparetwochemotherapeuticstrategieswithrespecttotumourresponseinaclinicaltrialonthetreatmentoflungcancer. GuisanandHarrell ( 2000 )appliedthestereotypemodeltoecologicaldataand Lalletal. ( 2002 )useditintheanalysisofqualityoflife. Greenland ( 1994 )comparedthestereotypemodelwiththecumulativelogitmodelofproportionaloddsformandthecontinuation-ratiomodel.Thecumulativelogitmodelisinvariantunderreversalofthecodingoftheoutcomevariable,andtheregressioncoefcientsarecollapsinginvariant.However,Greenlandclaimedthatcollapsinginvarianceisnotareasontopreferthecumulativelogitmodeloverothers,sinceinvariancedoesnotapplytotheestimatesorinferences,butonlytotheunderlyingmodelform.Also,GreenlandnotedthattheassumptionthatYisderivedfromaone-dimensionalcontinuumisneededforthatargumenttohold.Healsostatedthatthe 24

PAGE 25

GuisanandHarrell ( 2000 )comparedthecumulativelogitmodelofproportionaloddsform,thecontinuation-ratiomodelofproportionaloddsformandthestereotypemodel.Theymentionedwaystodeterminetheappropriatemodel.Otherthangoodness-of-ttesting,empiricalevaluations,andgraphicalchecksofassumptions,theystatedthattheresponseinthetruepopulationbeingatruediscreteordinalresponseoracontinuummightinuencetheuseofstereotypemodelsoverproportionaloddsmodels.Theyalsostatedthat`ifthetruepopulationresponseisconsideredcontinuous,onemaythenwonderifanobservationneedsto`passthrough'onecategorytoreachthenext,inwhichcasethecontinuation-ratiomodelmaybepreferredtothecumulativelogitmodelofproportionaloddsform(Harrelletal.,1998).Anotherpossibledistinctionistoconsiderifthemodelisinvarianttothegroupingofadjacentresponsecategoriesorifitisinvarianttoreversalsofthecategories(exceptforthesignoftheregressioncoefcientsandintercepts),inwhichcasethecumulativelogitmodelofproportionaloddsformwouldbepreferredtothecontinuation-ratiomodelofproportionaloddsform(McCullagh&Nelder,1989).Iftheresponseisadiscretizedversionofacontinuous 25

PAGE 26

Kuss ( 2003 )extendedthestereotypemodeltoaccountforcorrelatedresponsesbyasimplenonlinearparameterrestrictiononthemultinomiallogisticmodelwithrandomeffectsandnamedittherandomeffectsstereotyperegressionmodel. Ahnetal. ( 2009 )emphasizedmanyappealingfeaturesofthestereotypemodel.Theymentionedthatthestereotypemodelgivesunbiasedandvalidinferenceunderoutcome-stratiedsamplingandallowsforthestratum-specicnuisanceparameterstobeeliminatedbyusingaconditionallikelihood.TheypresentedBayesianinferenceforthestereotyperegressionmodelandshowedanapplicationwithacase-controlstudyofprostatecancer. Ahnetal. ( 2011 )illustratedhowtohandlemissingdatainmatchedcase-controlstudiesunderastereotyperegressionmodel.TheypresentedbothaMonteCarlobasedfullyBayesianapproachandexpectation/conditionalmaximizationalgorithmfortheestimationofmodelparametersinthepresenceofanonignorablemissingnessmechanism.Theyillustratedtheirmethodsbyusingdatafromanongoingmatchedcase-controlstudyofcolorectalcancer. Agresti ( 2010 )containsasectionthatdiscussesdifferentaspectsofthestereotypemodel. Anderson ( 1984 )proposedtotthestereotypemodelusingthemethodofmaximumlikelihood,wherethelikelihoodfunctionismaximizediteratively.Andersonsuggestedusingaquasi-NewtonalgorithmwhichisimplementedintheNAGlibraryorusingGreen's(1982)approachofusingiterativelyreweightedleastsquaresapproachasanalternative. HoltbruggeandSchumacher ( 1991 )usedtheequivalenceofmaximum 26

PAGE 27

Greenland ( 1994 )statedthatthemodelparameterscanbeestimatediterativelyalternatingbetweentheestimationoff'sgandparameters,keepingtheothergroupofparametersxed.First,thef'sgarexedandtheestimateofcanbeobtainedeasilysincethemodelislinearinthatcase,andabaseline-categorylogitmodelcanbeusedtotit.Then,f'sgparametersareestimatedbyxingatitsestimate.Fixingf'sgattheestimatesfromrstcycle,wecangettheestimatesof.Thisprocesscontinuesuntilconvergence.Greenlandnotedthatwhenstartingvaluesforscoresarederivedfromthemultinomiallogisticmodel,thementionedapproachconvergedrapidlyandreliably.However,sincethismethodignoresthesamplingvariabilityduetoestimationoftheothergroupofparameters,standarderrorsoftheestimatesthatthisestimationyieldsareincorrect,andhencetheyshouldnotbeusedforinferences.HerecommendedusingMonteCarlosimulationfromthettedmodelforcomputingp-valuesandcondenceintervals.Lalletal.(2002)usedGreenland'srecommendationofusingMonteCarlotechniquesbybootstrappingthecorrectstandarderrors. HendrickxandGanzeboom ( 1998 )andHendrickx(2005)introducedtheSASmacro`MCL'andtheSTATAmacro`MCLEST'thatusedthemultinomialconditionallogitmodelandperformedaseriesofgeneralizedlinearmodelts,inwhichf'sgandparametersarealternativelyheldxedwhiletheotherisestimated,asGreenlandproposed.Lalletal.(2002)usedthismacrofortheanalysisofqualityoflife. Lunt ( 2001 )introducedtheSTATAmodule`SOREG'fortheestimationofthemodel.HestatedthatthestereotypemodelisttedusingatechniqueduetoBoxandTidwell(1962).Insteadofttinganon-linearfunction,aniterativelyttingalgorithmisusedontransformedfunction.Luntnotedthattheconvergenceoftheiterationprocesswasnotguaranteed.HementionedsomeadvantagesofSOREGoverMCLEST,themacroHendrickxwroteusingthemultinomialconditionallogitmodel(MCL).Oneof 27

PAGE 28

YeeandHastie ( 2003 )extendedthereduced-rankideatothevectorgeneralizedlinearmodelsclass,whichwouldbecalledasreduced-rankvectorgeneralizedlinearmodels(RR-VGLMs)andshowedthatthestereotypemodelisinthisclass.TheystatedthatYee'sS-Pluspackage,VGAM(Yee,2005),canbeusedforestimation.VGAMtsvectorgeneralizedlinearandadditivemodels(VGLMs/VGAMs),aswellasreduced-rankVGLMs(RR-VGLMs).Itisageneralprogramformaximumlikelihoodestimation,whichisperformedbyiterativelyreweightedleastsquaresusingeitherNewton-RaphsonorFisher-scoring.ThepackageVGAMcantregressionmodels,suchasmultinomiallogitmodel,stereotypemodel(reduced-rankmultinomiallogitmodel),adjacentcategoriesmodel,cumulativelogitmodelofproportionaloddsform,proportionalhazardsmodelandcontinuation-ratiomodel,toanydatawithcategoricalresponse.Konig(2005)usedFeldmann'sfamilyofcanonicaldiscriminantmodelsofwhichthestereotypemodelisaspecialcaseandintroduced`CANLOG',aSAS/IMLmacroforperformingmaximumlikelihoodestimation.InthediscussionofAnderson'spaper,McCullaghwrotethathepreferredtocallstereotypemodelsascanonicalregressionmodels,whichhavethepropertythattheorderofthecategoriesneednotbespeciedinadvance,andhencecouldbeusedincaseswheretheorderisambiguous.McCullaghalsomentionedthatdimensionalityisthenumberofcanonicalroots. 28

PAGE 29

( 2006 )usedgeneralizedleastsquares(GLS)anditerativelyreweightedgeneralizedleastsquares(IRGLS)methodstogettheestimatesbyusingtheMODELprocedureofSAS,whichrequirestartingpoints,togettheestimatesusingthesetwomethods.Hesuggestedtotakestartingvaluesfromttingthemultinomiallogisticmodel:thestartingvaluesfortheinterceptsdirectlyfromtheinterceptsofthemultinomiallogisticmodel,thestartingvaluesfortheparametersfromL,andthestartingvaluesforf'jgbyjustusingtherelationj='jbetweenthemultinomiallogisticmodelandthestereotypemodelandaveragingthepstartingvaluesforf'jg. Kuss ( 2006 )alsousedtheNLMIXEDprocedureofSAStogetmaximumlikelihoodestimates.TheNLMIXEDproceduretsnonlinearmixedmodels,thatis,modelsinwhichbothxedandrandomeffectsenternonlinearlybymaximizinganapproximationtothelikelihoodintegratedovertherandomeffects.ThisprocedureusedwithouttheRANDOMstatementbecomesamoduleforstandardnonlinearregression.Avarietyofalternativeoptimizationtechniquesareavailabletocarryoutthemaximization.AparticularoptimizercanbechosenwiththeTECH=nameoptioninthePROCNLMIXEDstatement.Sincethetechniquesareiterative,theyrequiretherepeatedcomputationofthefunctionvalue(optimizationcriterion),thegradientvector(rst-orderpartialderivatives)andforsometechniques,the(approximate)Hessianmatrix(second-orderpartialderivatives).Thedefaultisadualquasi-Newtonalgorithm.The(dual)quasi-Newtonmethod(QUANEW)usesthegradient,anditdoesnotneedtocomputesecond-orderderivatives,sincetheyareapproximated.Thedualquasi-NewtonmethodapproximatestheHessianmatrix,whiletheoriginalquasi-NewtonmethodgetsanapproximationoftheinverseofHessian.Itworkswellformediumtomoderatelylargeoptimizationproblems,wheretheobjectivefunctionandthegradientaremuchfastertocomputethantheHessian;but,ingeneral,itrequiresmoreiterationsthantheothertechniques,whichcomputesecond-orderderivatives.The(dual)quasi-Newtonmethod(QUANEW)isthedefaultoptimizationalgorithm,becauseitprovidesanappropriate 29

PAGE 30

SAS 2008 )TheRpackage,GNM,whichwasdevelopedbyHeatherL.TurnerandDavidFirth,canbeusedfortheestimationofgeneralizednonlinearmodels.Theystatedthat`Modelparametersareestimatedviaaniterativeweightedleastsquaresalgorithm,usingtheMoore-Penrosepseudoinversetohandletherank-decientdesignmatrix.'IntheGNMpackagedocument( TurnerandFirth 2005 ),itisstatedthat`GNMprovidesfunctionstotgeneralizednonlinearmodelsbymaximumlikelihood.' TurnerandFirth ( 2007 )statedthatmultinomialresponsemodelsmaybetwithGNMbyusingtheequivalence,betweenmultinomialand(conditional)Poissonlikelihoods.Theywrotethestereotypemodelasfollows:logis=i+s+'sXrrxir,whereisrepresentsPoissonmeans.TurnerandFirthrstmanipulatedthedatabyusingafunction,expandCategoricalwhichexpandsthedataframeandre-expressesthecategoricaldataascounts,andthentheytthelog-linearmodel.Theystatedthat`Thecategory-specicmultipliers,f'sgaremadeidentiablebyconstrainingboththeirlocationandscale.Thelocationmaybeconstrainedbysettingoneofthecategory-specicmultiplierstozeroeitherspecifyingtheconstraintinthecalltoGNM,ormaybeappliedbyusinggetContrastsafterthemodelhasbeenestimated.Toconstrainthescale,oneoftheslopescanbexedtobeequalto1byspecifyingthe 30

PAGE 31

1.2.5 .Wecomparetheestimatesandthestandarderrorsobtainedbythesemethodsusingtwodatasets. ( 2006 )illustratedthestereotypemodelwithadatasetfromaprospectivecohortstudyofapprenticesinthecarindustry,whichwasmadetoassessendogenousandexogenousriskfactorsfordevelopingoccupationalhandeczema.Theresponsevariable,theseverityofhandeczema(HE),wasrecordedonathree-levelscale:1=NoHE,2=LightHE,3=SevereHE.Kussstatedthattheassessmentofseveritywashighlysubjectivesinceitwasdeterminedintermsofobjectivecriterialikesize,erythema,dryness,degreeofscalingandalsosubjectivecriteriasuchasitchingordegreeofannoyance.Threebinarycovariatesarewhethertheapprenticesworkedmorethan3hoursofdailywetworkornot(WetWork),theexistenceofananamnestichandeczema(PreviousHE)andtheexistenceofananamnesticexuraleczema(PreviousFE).While1640apprenticesdidnothaveanyskinproblems,103ofthemhadlighthandeczemaand167ofthemhadseverehandeczema.ThedataaresummarizedinTable 1-1 .Table 1-2 comparesestimatesandstandarderrorsobtainedfromvariousprograms.TheestimatesfromGLSandIRGLSmethodaresimilar,howevertheyareslightlydifferentfromtheestimatesobtainedbytheotherestimationmethods:MCL,CANLOG,NLMIXED,GNM,andVGAM.Theestimatesandthestandarderrorsaresimilarforthesevemethods,exceptforMCL,whichgivessmallerstandarderrorssincef'sgandparametersarealternativelyheldxedtoestimatetheotherone.ResidualdeviancesforGLSandIRGLSmethodsare1827.72,whereasitis1827.41foralltheothermethods.ThisshowsthattheestimatesobtainedbyGLSandIRGLSmethods 31

PAGE 32

Handeczemadatafromaprospectivecohortstudyofapprenticesinthecarindustry WetWorkPreviousHEPreviousFENoHELightHESevereHETotal 000140265103157000129411440101551131011766191001802135236101310411041051110011Total16401031671910 arenotthemaximumlikelihoodestimates.Theoddsratiosonegetsusingthesevemethodsaresimilar.Theexistenceofriskfactorsincreasestheriskfordevelopingahandeczemasinceallthef'sgandparametersarepositive.Forexample,theestimatedoddsofhavinglighthandeczemainsteadofseverehandeczemafortheonesthathadaprevioushandeczemaisaboutexp(0.85(1.031))=exp(0.05)=1.05timesthattheestimatedoddsfortheonesthatdidnothaveaprevioushandeczema.Kussnotedthatparameterestimatesseemtobecounterintuitive.Thisisbecause'2isestimatedtobegreaterthan1.However,testresultsshowthat'2isnotsignicantlydifferentthan1,andhencelighthandeczemaandseverehandeczemacategoriesareindistinguishable.So,onecangetmorereasonableresultsifthosetwocategoriesarecombined,inwhichcaseoneneedsonlyordinarybinarylogisticregressionmodels.Tosumup,wehaveobservedthatGLSandIRGLSgivedifferentestimatesthanMCL,CANLOG,NLMIXED,GNMandVGAM,whichgivesimilarestimates,butthestandarderrorsobtainedbythesemethodsdifferedslightlyduetouseofdifferenttechniques.TheestimatedstandarderrorsMCLmethodgivearesmallerthanotherssinceonegroupofparametersiskeptxedwhileestimatingother.OneofthestandarderrorsobtainedbyCANLOGwasobservedtobehigherthancorrespondingstandarderrorsforothermethods.Next,wecompareestimatesandstandarderrorsobtained 32

PAGE 33

Comparisonofestimatesobtainedforstereotypemodelttedtohandeczemadata ParmsGLS*IRGLS*MCL*CANLOG*NLMIXED*GNMVGAM *RetrievedfromKuss(2006)

PAGE 34

( 1984 )analyzedback-paindatafromastudywith101patientsusingthestereotypemodel.Thepatientssufferingfrombackpainweregivenatreatment,andthreeweeksafterthetreatment,theirprogresswasassessedona6-pointscale:worse(1),same(2),slightimprovement(3),moderateimprovement(4),markedimprovement(5)orcompleterelief(6).Threecovariatesarex1=lengthofpreviousattack(1=short,2=long),x2=painchange(1=gettingbetter,2=same,3=worse),andx3=lordosis,aninwardcurvatureofaportionofthevertebralcolumn(1=absent/decreasing,2=present/increasing).ThedataaresummarizedinTable 1-3 Table1-3. Countsforeachcategoryofbackpain BackPain*x1x2x3123456Total 1110100247112000130412102306415122010201413100002241320011305211003012621201003042210345622022214430113231221520122322023007Total51418202816101 WewillcomparetheestimatesreportedbyAndersontotheestimatesobtainedbystandardsoftware:theNLMIXEDprocedureofSASandtheGNMandVGAMpackagesofR.TheestimatesaregiveninTable 1-4 .Thestandarderrorsarealsoprovidedinparentheses.Theestimatesreportedbythesemethodsaresimilar,whereasthe 34

PAGE 35

Table1-4. Comparisonofsstimatesobtainedforstereotypemodelttedtoback-paindata UnstandardizedPredictorsStandardizedPredictorsNLMIXEDGNMVGAMAnderson*GNMVGAM *RetrievedfromAnderson(1984) Sincetheestimatesforf'sgarenotmonotoneandtheirstandarderrorsarehigh,Andersoncombinedcategories2,3and4intoonecategory,andcategories5and6intoanothercategory.Therefore,thescoreparametersaregroupedintothreevaluesandnewscoreparametersaredenotedby'01,'02and'03.Andersonreportedparameterestimatesforthestereotypemodelwiththreegroupsofcategories.Usingthesamegrouping,thestereotypemodelwastusingNLMIXED,GNM,andVGAM.WeagaincompareestimatesgivenbyAndersontotheestimatesVGAMandGNMgive,ifstandardizedexplanatoryvariablesareusedandindistinguishablecategoriesare 35

PAGE 36

1-5 .Again,weseethattheestimatesaresimilar,buttherearesomedifferencesinthestandarderrors. Table1-5. Comparisonofestimatesforstereotypemodelttedtoback-paindatawhenindistinguishablecategoriesarecombined UnstandardizedPredictorsStandardizedPredictorsNLMIXEDGNMVGAMAnderson*GNMVGAM *RetrievedfromAnderson(1984) Theoddsratiosobtainedwithdifferentmethodsaresimilarsincetheestimatesaresimilar.However,thestandarderrorsdifferslightlyamongdifferentmethodsduetouseofdifferenttechniques.WewillusetheestimatesandthestandarderrorsreportedbyGNMtocalculatetheoddratios.Forexample,theestimatedoddsoftheresponseworseinsteadofmarkedimprovementorcompletereliefforpatientswithlordosis(presentorincreasing)areexp(2.17)=8.76timestheestimatedoddsforpatientswithoutlordosisorwithadecreasinglordosis,controllingforlengthofpreviousattackandpainchange.The95%Waldcondenceintervalforthiseffectisexp[2.171.96(1.11)]=(0.99,77.14),whichisaverywideinterval.Finally,weshowthattheoddsratiosobtainedbystandardprogramsareequivalenttotheoneswecomputeusingtheestimatesreportedbyAndersonwhostandardizedtheexplanatoryvariables.Forexample,thestandardizedvalueforx3=1is0.76andthestandardizedvalueforx3=2is1.31.Therefore,thedifferencebetweentwolevelsoflordosisis2.07.Therefore,theestimatedoddsoftheresponseworseinsteadofmakedimprovementorcompletereliefforpatientswithlordosisareexp(1.05(2.07))=exp(2.17)=8.76timestheestimatedoddsforpatientswithoutlordosisorwithadecreasinglordosis,controllingforlengthofpreviousattackandpainchange. 36

PAGE 37

2 ,wecomputecondenceintervalsfortheoddsratiosforthestereotypemodelusingthreecondenceintervalmethods:theWaldcondenceinterval,thelikelihoodratiotest(LRT)-basedcondenceinterval,andthescorecondenceinterval.Wecomparethecoverageprobabilitiesofthecondenceintervalsfortheoddsratiosintermsofachievingerrorratesclosetothenominallevelandtheirlengthswithbothsmallandlargesamplesizes.InChapter 3 ,weintroduceapartialstereotypemodelwhichisnestedbetweenthestereotypemodelandthemultinomiallogisticmodel.Wedetermineidentiabilityconstraints,investigatepropertiesofthemodel,andcomparethetofthepartialstereotypemodelwithalternativemodelstoaclinicaltrialdataset.Wealsoshowhowthestandardcondenceintervalmethodsextendtothepartialstereotypemodel.Asimulationstudyisconductedtochecktheasymptoticnulldistributionofthelikelihoodratioteststatistictocomparethetsofthestereotypemodel,thepartialstereotypemodel,andthemultinomiallogisticmodel.PerformanceofthelikelihoodratiotestandAICarecomparedinchoosingthenullmodelwhenthesemodelsarecompared.Wehaveencounteredproblemsinttingstereotypeandpartialstereotypemodelsforsomecases.Inordertocomparedifferentmethodsforcondenceintervals,wehadtocreatedatasetsforwhichthemaximumlikelihoodestimatesexistforthestereotypemodel.Ingeneral,itisimportanttoknowifthemaximumlikelihoodestimatesexistornotforagivendatasetassomeoftheprogramscannotrecognizewhentheydo 37

PAGE 38

4 ,wereviewtheliteratureontheexistenceofmaximumlikelihoodestimatesforthemultinomiallogisticmodel,themultinomialchoicemodel,andthecumulativelinkmodelofproportionaloddsform.Wegeneralizesomeresultsandexamineconnectionsbetweenthedifferentapproaches.Chapter 5 concludesthedissertationwithasummaryofourcontributionsandadiscussionoffuturework. 38

PAGE 39

logP(Yi=s) 2 ),thecoefcientoftheexplanatoryvariablexkis'sk,whichisthelogoddsratioforcategoriessand1foreachone-unitincreaseinxk.Thatis,whenxk=u+1,theoddsofbeingincategorysinsteadofcategory1areexp('sk)timestheoddswhenxk=u.Asnotedabove,onemoreidentiabilityconstraintisrequiredotherthan1=0and'1=0.Iftheconstraintthatk=1isusedforidentiabilitywhenttingthestereotypemodel,thelogoddsofbeingincategorysinsteadofcategory1becomes's.Sincewearemainlyinterestedinthelogoddsratios,wewillobtainandcomparethecondenceintervalsusingdifferentmethodsforthecategory-specicscores,namelyf'sgparameters. 39

PAGE 40

40

PAGE 41

A.1 41

PAGE 42

42

PAGE 43

43

PAGE 44

2-1 Table2-1. CellprobabilitiesusedtosimulatedatasetswhenL=3 10.4490.3500.20220.2850.3660.34830.1550.3290.51640.0740.2580.668 WedothesameforL=5choosing1=0,2=0.3,3=0.7,4=0.5,5=0.1,'1=0,'2=0.2,'3=0.4,'4=0.7,'5=1,1=1,and2=0.5.ThecellprobabilitiesforthiscasearegiveninTable 2-2 Table2-2. CellprobabilitiesusedtosimulatedatasetswhenL=5 10.2550.2550.2810.1470.06320.1750.2140.2890.2040.11830.1100.1640.2700.2570.20040.0620.1130.2270.2920.307 Weproducecondenceintervalsforthecategory-specicscores,namelyf'sgparameters,undertheconstraints1=1and2=1.Weusetwocriteriatocomparethethreemethods:coverageprobabilitiesandcondenceintervallengths. 44

PAGE 45

45

PAGE 46

46

PAGE 47

2-3 showsthecoverageprobabilitiesundertheconstraint1=1andTable 2-4 showsthecoverageprobabilitiesundertheconstraint2=1.Thecoverageprobabilitiesaresimilarandcloseto0.95fordifferentmethodswhenthesamplesizesarelarge,howevertheyvarymoreforsmallersamplesizes.ThecoverageprobabilitiesfortheWaldCIaremostlybelow0.95,andtheyvary,muchmorethantheothermethods,dependingonthesamplesizes.ThecoverageprobabilitiesfortheLRT-basedandthescorecondenceintervalsarecloserto0.95.TheLRT-basedCIsaretypicallyslightlybelow0.95.ThescoreCIhasthehighestcoverageprobability,slightlyabove0.95,mostofthetime.ThemeanandthemedianlengthsofthecondenceintervalsaregiveninTable 2-5 ,Table 2-6 ,Table 2-7 ,andTable 2-8 .Thelengthsofthecondenceintervalsdonotdiffermuchamongthethreemethodsforeachcase.Thecondenceintervalsarenarrowerfor 47

PAGE 48

2-9 .ItshowsthatabsolutedifferencesbetweenthecoverageprobabilityandthenominalvaluearesmallerfortheLRT-basedandthescoremethods.Therefore,thesemethodsseemtoperformbetterthantheWaldmethod.HowevertheWaldmethodalsoworkswellwhenthesamplesizesarelarge,andonecanalwayseasilyobtaintheWaldCIsevenwhentheothermethodsfailtogiveacondenceintervalwithnumericendpoints. 48

PAGE 49

Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhen1=1 WaldCI(ni=100)0.9480.9480.9480.9490.9460.9410.9420.945LRT-basedCI(ni=100)TotalNumberofNAs000000NumberofNAsineachendpoint(0,0)(0,0)(0,0)(0,0)(0,0)(0,0)Coverageprob.(Method1)0.9510.9470.9490.9480.9470.9450.9430.946Coverageprob.(Method2)0.9510.9470.9490.9480.9470.9450.9430.946Coverageprob.(Method3)0.9510.9470.9490.9480.9470.9450.9430.946ScoreCI(ni=100)TotalNumberofNAs000016NumberofNAsineachendpoint(0,0)(0,0)(0,0)(0,0)(1,0)(6,0)Coverageprob.(Method1)0.9530.9480.9510.9510.9500.9490.9460.949Coverageprob.(Method2)0.9530.9480.9510.9510.9500.9490.9460.949Coverageprob.(Method3)0.9530.9480.9510.9510.9500.9490.9450.949WaldCI(ni=25)0.9420.9520.9470.9450.9250.9140.9240.927LRT-basedCI(ni=25)TotalNumberofNAs14283936NumberofNAsineachendpoint(1,1)(4,4)(2,2)(8,8)(39,35)(36,34)Coverageprob.(Method1)0.9390.9430.9410.9370.9400.9380.9420.939Coverageprob.(Method2)0.9390.9430.9410.9370.9400.9350.9380.938Coverageprob.(Method3)0.9390.9430.9410.9370.9390.9340.9380.937ScoreCI(ni=25)TotalNumberofNAs215631038145NumberofNAsineachendpoint(2,2)(156,2)(3,1)(10,1)(37,8)(145,8)Coverageprob.(Method1)0.9500.9520.9510.9520.9530.9530.9580.954Coverageprob.(Method2)0.9500.9520.9510.9520.9530.9510.9570.953Coverageprob.(Method3)0.9500.9370.9440.9510.9520.9500.9440.949

PAGE 50

Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhen2=1 WaldCI(ni=100)0.9330.9460.9400.9370.9190.9300.9360.931LRT-basedCI(ni=100)TotalNumberofNAs00001447NumberofNAsineachendpoint(0,0)(5,5)(0,0)(0,0)(14,14)(47,47)Coverageprob.(Method1)0.9470.9470.9470.9470.9470.9490.9480.948Coverageprob.(Method2)0.9470.9460.9470.9470.9470.9480.9440.947Coverageprob.(Method3)0.9470.9460.9470.9470.9470.9480.9440.946ScoreCI(ni=100)TotalNumberofNAs136132011667NumberofNAsineachendpoint(136,0)(32,2)(0,0)(1,0)(16,13)(67,64)Coverageprob.(Method1)0.9530.9480.9510.9510.9510.9530.9560.953Coverageprob.(Method2)0.9480.9480.9480.9510.9510.9520.9500.951Coverageprob.(Method3)0.9400.9450.9430.9510.9510.9510.9500.951WaldCI(ni=25)0.9200.9670.9430.9570.9100.9080.9280.926LRT-basedCI(ni=25)TotalNumberofNAs142415742199433NumberofNAsineachendpoint(14,14)(241,241)(57,55)(41,39)(199,194)(433,428)Coverageprob.(Method1)0.9540.9620.9580.9460.9500.9450.9430.946Coverageprob.(Method2)0.9540.9590.9570.9460.9500.9440.9380.945Coverageprob.(Method3)0.9530.9390.9460.9410.9460.9260.9020.929ScoreCI(ni=25)TotalNumberofNAs5821882116124281NumberofNAsineachendpoint(582,16)(188,100)(19,20)(16,16)(121,121)(278,277)Coverageprob.(Method1)0.9610.9700.9660.9580.9620.9620.9620.961Coverageprob.(Method2)0.9620.9690.9660.9580.9620.9620.9580.960Coverageprob.(Method3)0.9050.9520.9290.9560.9600.9500.9350.950

PAGE 51

ComparisonofthemeanandmedianlengthsofCIsforlargesamplesizeswhen1=1 WaldCI(ni=100)MeanLength0.5500.5850.5680.6690.6300.6760.7750.687MedianLength0.5500.5810.5650.6710.6280.6730.7710.686LRT-basedCI(ni=100)MeanLength(Method1)0.5560.5870.5720.6850.6410.6830.7820.698MeanLength(Method2)0.5560.5870.5720.6850.6410.6830.7820.698MedianLength(Method1)0.5550.5840.5690.6850.6390.6800.7770.695MedianLength(Method2)0.5550.5840.5690.6850.6390.6800.7770.695MedianLength(Method3)0.5550.5840.5690.6850.6390.6800.7770.695ScoreCI(ni=100)MeanLength(Method1)0.5550.5840.5700.6840.6410.6840.7840.698MeanLength(Method2)0.5550.5840.5700.6840.6410.6840.7840.698MedianLength(Method1)0.5530.5810.5670.6830.6370.6800.7790.694MedianLength(Method2)0.5530.5810.5670.6830.6370.6800.7790.694MedianLength(Method3)0.5530.5810.5670.6830.6370.6800.7790.694

PAGE 52

ComparisonofthemeanandmedianlengthsofCIsforsmallsamplesizeswhen1=1 WaldCI(ni=25)MeanLength1.1351.2171.1761.3681.2771.3811.6031.408MedianLength1.1151.1811.1481.3691.2681.3631.5661.392LRT-basedCI(ni=25)MeanLength(Method1)1.1851.2441.2151.4941.3821.4711.7021.512MeanLength(Method2)1.1851.2441.2151.4941.3821.4681.7001.511MedianLength(Method1)1.1511.2041.1771.4611.3411.4301.6461.469MedianLength(Method2)1.1511.2041.1771.4611.3401.4281.6451.469MedianLength(Method3)1.1511.2041.1771.4611.3411.4301.6471.470ScoreCI(ni=25)MeanLength(Method1)1.1631.2381.2001.4581.3571.4691.7231.502MeanLength(Method2)1.1621.2351.1991.4571.3571.4671.7191.500MedianLength(Method1)1.1361.2021.1691.4351.3241.4291.6711.464MedianLength(Method2)1.1361.2001.1681.4351.3231.4281.6671.463MedianLength(Method3)1.1361.2061.1711.4351.3241.4301.6761.466

PAGE 53

ComparisonofthemeanandmedianlengthsofCIsforlargesamplesizeswhen2=1 WaldCI(ni=100)MeanLength0.3590.5480.4530.3600.3870.5110.6600.479MedianLength0.3610.5470.4540.3630.3910.5150.6630.483LRT-basedCI(ni=100)MeanLength(Method1)0.3600.5510.4550.3880.3960.5160.6710.493MeanLength(Method2)0.3600.5510.4550.3880.3960.5160.6700.492MedianLength(Method1)0.3610.5510.4560.3880.3970.5200.6760.496MedianLength(Method2)0.3610.5510.4560.3880.3970.5200.6760.495MedianLength(Method3)0.3610.5510.4560.3880.3970.5210.6770.496ScoreCI(ni=100)MeanLength(Method1)0.3620.5530.4580.3940.4020.5260.6840.502MeanLength(Method2)0.3610.5530.4570.3940.4020.5260.6830.501MedianLength(Method1)0.3590.5510.4550.3930.3990.5260.6850.501MedianLength(Method2)0.3590.5510.4550.3930.3990.5260.6850.501MedianLength(Method3)0.3600.5510.4560.3930.3990.5260.6860.501

PAGE 54

ComparisonofthemeanandmedianlengthsofCIsforsmallsamplesizeswhen2=1 WaldCI(ni=25)MeanLength0.7691.1120.9410.8630.8651.0631.3251.029MedianLength0.7801.1060.9430.8600.8731.0881.3371.039LRT-basedCI(ni=25)MeanLength(Method1)0.7911.1190.9551.0210.9761.1451.4011.136MeanLength(Method2)0.7911.1150.9531.0200.9751.1391.3861.130MedianLength(Method1)0.7821.1240.9530.9980.9581.1431.4041.126MedianLength(Method2)0.7821.1220.9520.9960.9571.1381.3921.121MedianLength(Method3)0.7821.1270.9551.0000.9601.1511.4191.132ScoreCI(ni=25)MeanLength(Method1)0.8201.1390.9791.0731.0271.2111.4901.201MeanLength(Method2)0.8181.1370.9771.0721.0271.2061.4771.196MedianLength(Method1)0.8021.1290.9651.0521.0071.2001.4751.183MedianLength(Method2)0.8011.1270.9641.0521.0061.1961.4641.179MedianLength(Method3)0.8181.1320.9751.0531.0071.2051.4871.188

PAGE 55

Theaverageofcoverageprobabilitiesandtheaverageofabsolutedifferencesbetweenthecoverageprobabilityandthenominalvalue(0.95)forlargeandsmallsamplesizes

PAGE 56

2-10 : Table2-10. CellProbabilitiesUsedtoSimulateDatasetsWhenL=3 1-1-10.3320.3670.3012-110.3320.3670.30131-10.1260.3100.5644110.1260.3100.564 Since2=0,thestereotypemodel,inthiscase,isdenedas logP(Yi=s) 2-11 showsthecoverageprobabilitiesforlargeandsmallsamplesforeachconstraint.When1=1,coverageprobabilitiesforallmethodsareabout0.94forlargesamplesandbetween0.90and0.94forsmallsamples.Theresultsaresimilartothepreviousresults.Ontheotherhand,when2=1,therearetoomanycondenceintervalsthathaveNAforatleastoneoftheendpointsandthecoverageprobabilitiesareverylow,exceptfortheWaldmethod,whichseemstoworkbetterthantheothermethodsinthiscase.Sincetruevalueoftheparameter2is0,whentheidentiabilityconstraint2=1isused,theestimatesforscoreparametersarecloseto0,whichcausestheestimatesandstandarderrorsfor1parameterstobehuge.Therefore,2=1isnotappropriateconstraintinthiscase.Theseproblemsoccurduetothenonlinearstructureofthemodel. 56

PAGE 57

Comparisonofcoverageprobabilitiesforlargeandsmallsamplesizeswhenthetruevaluefor2=0 WaldCI(ni=100)0.9400.9450.9430.9800.9450.962LRT-basedCI(ni=100)TotalNumberofNAs0046809NumberofNAsineachendpoint(0,0)(0,0)(46,46)(808,809)Coverageprob.(Method1)0.9420.9440.9430.2610.8100.536Coverageprob.(Method2)0.9420.9440.9430.2650.8260.545Coverageprob.(Method3)0.9420.9440.9430.2600.7450.503ScoreCI(ni=100)TotalNumberofNAs055347747NumberofNAsineachendpoint(0,0)(5,0)(5347,161)(747,646)Coverageprob.(Method1)0.9450.9440.9450.4790.8710.675Coverageprob.(Method2)0.9450.9440.9450.7570.8800.818Coverageprob.(Method3)0.9450.9440.9450.2230.8060.515WaldCI(ni=25)0.9010.9160.9080.9760.9260.951LRT-basedCI(ni=25)TotalNumberofNAs447251948NumberofNAsineachendpoint(4,4)(47,47)(250,251)(947,948)Coverageprob.(Method1)0.9270.9310.9290.7560.7800.768Coverageprob.(Method2)0.9270.9270.9270.7620.8010.781Coverageprob.(Method3)0.9270.9260.9270.7370.7060.722ScoreCI(ni=25)TotalNumberofNAs11651280640NumberofNAsineachendpoint(1,0)(165,4)(1254,250)(622,588)Coverageprob.(Method1)0.9420.9410.9410.8900.8830.886Coverageprob.(Method2)0.9420.9400.9410.9030.8890.896Coverageprob.(Method3)0.9420.9260.9340.7760.8260.801

PAGE 58

58

PAGE 59

PetersonandHarrell ( 1990 ).Thepartialstereotypemodelfallsbetweenthestereotypemodelandthemultinomiallogisticmodel 59

PAGE 60

60

PAGE 61

61

PAGE 62

62

PAGE 63

63

PAGE 64

64

PAGE 65

65

PAGE 66

3-1 summarizesthenumberofparametersforthementionedmodelswhenq
PAGE 67

ScottandWild ( 1997 )describedanalternativesamplingschemeasfollows.WechooseY=jwithprobabilitynj=nandthensamplexfromf(xjY=j).Repeatingthisntimes,onecanobtainsamplesofrandomsizefromeachcategory.Theydenotedthe 67

PAGE 68

Greenland ( 1994 )mentionedthatunderacorrectlyspeciedstereotypemodel,theestimatesfortheeffectofthepredictorsfromoutcome-stratiedsamplingwillbeconsistent.Wewillalsodenotethesamplingfractionsas1,...,k,foroutcomelevelsY=1,...,L.Hewrotethesampleoddsusingthepseudo-modelasfollows:P(Y=sjx,sampled) 68

PAGE 69

P(Y=s)=ns 1=PLl=1exp(l+'l0x+0lw)=exp(s+'s0x+0sw),wheres=s+log(s=1).Thus,theestimatesfortheeffectofthepredictorsfromoutcome-stratiedsamplingwillbeconsistentforthepartialstereotypemodel. 69

PAGE 70

70

PAGE 71

A.2 .Notethattheequationsfortheelementsthatarecommonwiththestereotypemodelhavethesameform,butwehaveadditionalelementsassociatedwithfsgparameters. 71

PAGE 72

72

PAGE 73

LiuandShao 2003 ).Next,wedoasimulationstudytocheckthedistributionofthelikelihoodratioteststatisticsforcomparisonofvariousmodels. 73

PAGE 74

3.2.3 .Wesimulate100,000datasetsfromthenulldistribution,calculatetheappropriateLRTstatisticandplotthedensityofLRTstatisticandchi-squareddistributionforcomparison.WealsocalculatetheproportionofthetimesthenullmodelischosenusingAICandtheLRTstatisticunderasignicancelevelof0.05and0.1.Weconsiderthreebinaryexplanatoryvariables,x1,x2,andx3,takingvalues-1and1;hencethereare23=8explanatoryvariablecombinations.Wetakethesamplesizenitobe50and500fori=1,...,8toseetheeffectofsamplesize.Weassume 74

PAGE 75

3-2 andTable 3-3 .Weconsiderthepartialstereotypemodelwithq=1,wherew=x1. Table3-2. Cellprobabilitiesusedtosimulatedatasetsforstereotypemodels 10.3140.2730.2630.1110.03920.1980.2280.2900.1860.09830.2300.2440.2860.1640.07640.1280.1790.2790.2420.17250.1540.2000.2860.2210.13960.0740.1280.2420.2840.27370.0930.1480.2590.2700.23080.0390.0820.1900.3000.389 Table3-3. Cellprobabilitiesusedtosimulatedatasetsforpartialstereotypemodels 10.4260.1660.2640.0910.05220.2730.1410.2950.1560.13530.3160.1510.2920.1360.10540.1740.1100.2810.2000.23550.1030.2990.2600.2450.09460.0520.1990.2290.3280.19170.0650.2280.2420.3070.15980.0290.1340.1880.3640.286 1 ,intestingH0:s=0(Norelationship)vs.Ha:s='s(Thestereotypemodelholds),thescoreparametersarenotidentiableunderthehypothesisthat=0.Therefore,thestandardconditionsforthelikelihoodratioteststatisticstohaveapproximatechi-squareddistributionsarenotsatised. 75

PAGE 76

Haberman ( 1981 ).TheRCmodelisthespecialcaseofthestereotypemodel,whentheexplanatoryvariableisnominal.However,inourmainsimulationstudy,wehavethreebinarypredictorsratherthananominalexplanatoryvariable.WecheckiftheresultgivenbyHabermanextendstothiscase.TheWishartdistributionisdeterminedbythenumberofdegreesoffreedomandthenumberofdimensions(variables)fortheWishartmatrix.WeuseqWishartMaxfunctionofRwrittenby Johnstoneetal. ( 2009 )tondthepercentilesofthedistributionofthelargesteigenvalueoftheWishartmatrix.Thesimulationstudyconrmsthatthelikelihoodratioteststatistictotestindependenceforthestereotypemodelcannotbeapproximatedbyachi-squareddistribution.TheplotsofthedensityofLRTstatisticandchi-squareddistributionforsmallandlargesamplesaregiveninFigure 3-1 andpercentilesaregiveninTable 3-4 Bni=500Figure3-1. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthestereotypemodel 76

PAGE 77

3-4 showsthatthepercentilesforthedistributionoftheLRTstatisticissimilartothepercentilesforthedistributionofthelargesteigenvalueofthe33Wishartmatrixwith4degreesoffreedom.Thelowerpercentilesseemtodiffer,especiallyforthe5thpercentile,howeverasimilardifferencewasalsoobservedinthesimulationstudywedidforthecaseofanominalexplanatoryvariable. Table3-4. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthestereotypemodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=6)1.6353.4555.3487.84110.64512.59216.812WishartDistr*0.3584.3277.30610.49113.55215.47519.269LRT(ni=50)3.3505.6107.73410.38413.30115.24219.556LRT(ni=500)3.4555.6657.76210.39513.30915.26919.446 *UsingthedistributionofthelargesteigenvalueofWishartdistribution(ndf=4,pdim=3) 3-5 andFigure 3-2 .WewanttocheckiftheLRTdistributionhasthedistributionofthelargesteigenvalueofWishartmatrix,howevertheargumentsneededbyqWishartMaxarenotclearinthiscase.Thus,thedistributionofthelikelihoodratioteststatistictotestindependenceforthepartialstereotypemodelrequiresfurtherresearch. Table3-5. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthepartialstereotypemodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=9)3.3255.8998.34311.38914.68416.91921.666LRT(ni=50)4.2937.0839.67112.82516.19018.46223.263LRT(ni=500)4.4267.2049.74612.89916.25418.48323.175 77

PAGE 78

Bni=500Figure3-2. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthepartialstereotypemodel 3-6 andFigure 3-3 conrmthisfact. Table3-6. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthemultinomiallogisticmodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=12)5.2268.43811.34014.84518.54921.02626.217LRT(ni=50)5.1328.38011.30414.85418.62221.08426.310LRT(ni=500)5.2878.46711.36714.90818.59921.04426.151 78

PAGE 79

Bni=500Figure3-3. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontotestindependenceforthemultinomiallogisticmodel 3-7 andFigure 3-4 Table3-7. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthepartialstereotypemodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=3)0.3521.2132.3664.1086.2517.81511.345LRT(ni=50)0.3611.2402.4104.1936.3637.93511.543LRT(ni=500)0.3491.2132.3584.1296.2667.79011.364 79

PAGE 80

Bni=500Figure3-4. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthepartialstereotypemodel 3-8 andFigure 3-5 Table3-8. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthemultinomiallogisticmodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=6)1.6353.4555.3487.84110.64512.59216.812LRT(ni=50)1.6303.4425.3317.81410.60312.55416.797LRT(ni=500)1.6183.4425.3317.83510.66112.62116.870 80

PAGE 81

Bni=500Figure3-5. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthestereotypemodelandthemultinomiallogisticmodel 3-9 andFigure 3-6 Table3-9. ComparisonofpercentilesforthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthepartialstereotypemodelandthemultinomiallogisticmodel Percentile5%25%50%75%90%95%99% Chi-squaredDistr(df=3)0.3521.2132.3664.1086.2517.81511.345LRT(ni=50)0.3361.1752.2914.0006.0757.56710.903LRT(ni=500)0.3521.2062.3594.0836.1817.69211.216 81

PAGE 82

Bni=500Figure3-6. ComparisonofthedistributionofLRTdensityandthechi-squareddistributiontocomparetsofthepartialstereotypemodelandthemultinomiallogisticmodel 100,000=0.0007for=0.05,anditisq 100,000=0.0009for=0.1.Theresults,asshowninTable 3-10 andTable 3-11 ,donotdiffermuchforthetwosamplesizes.Thelikelihoodratiotestfailstorejectthenullmodelabout95%ofthetimewhensignicancelevelischosenas=0.05andabout90%ofthetimewhen=0.1,asexpected,forthecasesthatseemtohaveanapproximatechi-squareddistributionfromtheplotsandpercentiles.Theseproportionsarelowerthanexpectedonlyfortestingindependenceforthestereotypemodelandthepartialstereotypemodel,sincetheLRT 82

PAGE 83

ProportionofchoosingnullmodelusingLRT(=0.05) SampleSizeHa:SMHa:PSMHa:MLM *UsingthedistributionofthelargesteigenvalueofWishartdistribution Table3-11. ProportionofchoosingnullmodelusingLRT(=0.1) SampleSizeHa:SMHa:PSMHa:MLM *UsingthedistributionofthelargesteigenvalueofWishartdistribution statisticsforthesehypothesesdonothavealimitingchi-squareddistribution.Whenthedistributionofthelargesteigenvalueofthe33Wishartmatrixwith4degreesoffreedomisused,theproportionofchoosingthenullmodelisveryclosetothenominallevelwhentestingindependenceforthestereotypemodel.Itappearsreasonabletousethelikelihoodratiotesttocomparementionedmodelsexceptforthesetwohypotheses.However,furtherresearchonlargesampledistributionsoftheseteststatisticsareneeded. 3-12 representtheproportionofthetimesAICwassmallerforthenullmodel.AICseemstobechoosingthecorrectmodelatleast85%ofthetime, 83

PAGE 84

Table3-12. ProportionofchoosingnullmodelusingAIC SampleSizeHa:SMHa:PSMHa:MLM 84

PAGE 85

3-13 .Thecellprobabilitiesarethesamefori=1,2,3,4,andsimilarlyfori=5,6,7,8,sincetheirx1valuesarethesame. Table3-13. Cellprobabilitiesusedtosimulatedatasetsforstereotypemodels 1,2,3,40.1920.1170.2870.1920.2125,6,7,80.0950.2850.2580.2580.105 Thelikelihoodratioteststatisticforthesehypothesesdoesnothaveanapproximatechi-squareddistribution,sincethereisalackofidentiability.ThisissupportedbyFigures 3-7 3-8 ,and 3-9 .However,theapproximatedistributionofthisLRTstatisticisatopicforfutureresearch. Bni=500Figure3-7. ComparisonoftheSMandthePSMwhenthereislackofidentiability 85

PAGE 86

Bni=500Figure3-8. ComparisonoftheSMandtheMLMwhenthereislackofidentiability Bni=500Figure3-9. ComparisonofthePSMandtheMLMwhenthereislackofidentiability 86

PAGE 87

( 1990 )analyzeddatafromaclinicaltrialcomparingtwotreatmentsforrespiratoryillness.Patientswererandomlyassigned(withinoneoftwocenters)totwotreatments(active,placebo)insuccessiveblocksof6.Oftheeligiblepatients,54wererandomizedtoactivetreatmentand57toplacebo.Eachpatientwasclassiedintooneofthevecategories(0=terrible,1=poor,2=fair,3=good,4=excellent)accordingtotheirrespiratorystatusatbaselinepriortorandomizationandateachoffoursubsequentvisits.Inouranalysis,weusethemeasureofrespiratorystatusinthelastvisitastheresponsevariableanddrug,center,gender,ageandrespiratorystatusatbaselineastheexplanatoryvariables.Letdbeanindicatorvariablefordrug(1=active,0=placebo),cbeanindicatorvariableforcenter(1=center2,0=center1)andgbeanindicatorvariableforgender(1=females,2=males).Therespiratorystatusatbaselineisdenotedbybandageofthepatientisdenotedbya.Ourpurposeistoseeifthepartialstereotypemodelisappropriateforthisdataset.Wetthestereotypemodel,thepartialstereotypemodelandthemultinomiallogisticmodelwithallpossiblesubsetsofexplanatoryvariables.However,wedidnotconsiderpartialstereotypemodelswithq=p1,sincetheyareequivalenttomultinomiallogisticmodels.Table 3-14 givesthemodelthathasthesmallestAICforeachtypeofmodelwithaspecicnumberofexplanatoryvariables,p.Forthepartialstereotypemodels,weshowtheexplanatoryvariablesthatdonothaveparallelcoefcients,asinthestereotypemodel,byputtingtheminparanthesis.Whenp=1,allmentionedmodelsareequivalentandhaveanAICvalueof767.68.Allthesemodelshavecenterandrespiratorystatusatbaselineasthepredictorswhenp=2.Theadditionalpredictorisdrugwhenp=3.Notethattheexplanatoryvariablesarethesameforeachtypeofmodelexceptwhenp=4.Theadditionalpredictorisageforthepartialstereotypemodelandthemultinomiallogisticmodel, 87

PAGE 88

Stereotype,partialstereotypeandmultinomiallogisticmodelswithsmallestAICvalues 5ThePartialStereotypeModeld,c,g,b,a(d,a)751.64TheMultinomialLogisticModeld,c,g,b,a759.60TheStereotypeModeld,c,g,b,a762.424ThePartialStereotypeModeld,c,a,b(d,a)751.33TheMultinomialLogisticModeld,c,a,b753.46TheStereotypeModeld,c,g,b761.453ThePartialStereotypeModeld,c,b(d)753.33TheMultinomialLogisticModeld,c,b754.49TheStereotypeModeld,c,b760.412ThePartialStereotypeModelc,b(borc)762.56TheMultinomialLogisticModelc,b762.56TheStereotypeModelc,b760.87 whereasitisgenderforthestereotypemodelwithanAICof761.45.Infact,theAICvalueforthestereotypemodelwithdrug,center,ageandrespiratorystatusatbaselineisverysimilar,thatis,761.66.ThemodelwiththesmallestAICisthepartialstereotypemodelwiththesefourpredictors,thatis,logPs(xi) 3-15 Table3-15. MLestimatesforthepartialstereotypemodel,ttedtorespiratoryillnessdata,withtheminimumAIC 88

PAGE 89

89

PAGE 90

90

PAGE 91

91

PAGE 92

Haberman ( 1974 )provedatheoremonnecessaryandsufcientconditionsforthemaximumlikelihoodestimatestoexistforlog-linearmodels. Silvapulle ( 1981 )gavenecessaryandsufcientconditionsfortheexistenceofmaximumlikelihoodestimatesforbinomialresponsemodels. AlbertandAnderson ( 1984 )provedtheoremsontheexistenceofmaximumlikelihoodestimatesforthemultinomiallogisticmodelusingconceptsofdatacongurations:completeseparation,quasi-completeseparation,andoverlap. SantnerandDuffy ( 1986 )modiedthedenitionsandthestatementsinthetheoremofAlbertandAnderson. LesaffreandAlbert ( 1989 )introducedtheconceptof`partialseparation'. Speckmanetal. ( 2009 )generalizedAlbertandAnderson'sresultstomultinomialchoicemodels. Tse ( 1986 )provedatheoremthatgivesnecessaryandsufcientconditionsfortheexistenceofmaximumlikelihoodestimatesinorderedpolytomousmodelsusingcumulativelinks. ChenandShao ( 1999 )and ChenandShao 92

PAGE 93

2001 )investigatedtheseconditionsfortheexistenceofmaximumlikelihoodestimatesandBayesianestimatesforbinaryandorderedpolytomousresponsemodels. TodemandKim ( 2011 )examinedtheexistenceofmaximumlikelihoodestimatesinrandomeffectmodelsforclusteredmultivariatebinarydata.Werstreviewtheliteratureandstatetheoremsontheexistenceofmaximumlikelihoodestimatesfordichotomousreponsemodels,logisticregressionmodelsusingbaseline-categorylogits,andcategoricalregressionmodelsusingcumulativelinks.Wethenextendthedenitionsofseparationandoverlaptothestereotypemodelandtheadjacent-categorieslogitmodel.Finally,wegeneralizeandmodifysomeofthetheoremsontheexistenceofmaximumlikelihoodestimatesandexamineconnectionsbetweenthedifferentapproaches. 1+exp(x0i), 93

PAGE 94

AlbertandAnderson ( 1984 )provedtheoremsontheexistenceofmaximumlikelihoodestimatesforthemultinomiallogisticmodel.Theyshowedthattheexistenceofmaximumlikelihoodestimatesdependonthedatapatterns,whichfallintothreemutuallyexclusiveandexhaustivecategories:completeseparation,quasi-completeseparation,andoverlap.Wenowgothroughtheseinthecontextofabinaryresponse.Thedenitionsofseparationandoverlaparegivenasbelowwhentheresponsevariableisbinary.AlbertandAndersonstatedthatthereiscompleteseparationifthereisavectorthatcorrectlyallocatesallobservationstotheirobservedresponsecategory. 94

PAGE 95

Altmanetal. ( 2004 ),whenthereisonequantitativeexplanatoryvariableandabinaryresponsevariable. Table4-1. Dataexhibitingcompleteseparationwhenthereisasinglequantitativeexplanatoryvariable Thereiscompleteseparationsince0+1xi>0foralli2I1and0+1xi<0foralli2I2.Nextisanexampleofquasi-completeseparationwhereoneobservationwithanxvalueof0isaddedtoeachgroup.Thereisquasi-completeseparationsince0+1xi0foralli2I1and0+1xi0foralli2I2. Table4-2. Dataexhibitingquasi-completeseparationwhenthereisasinglequantitativeexplanatoryvariable Figure 4-1 andFigure 4-2 showhowtheloglikelihoodbehavesasafunctionoftheslopeforcompleteandquasi-completeseparationexamples.Whenthereiscompleteorquasi-completeseparation,themaximumlikelihoodestimateoftheslopedoesnotexistastheloglikelihooddoesnotreachamaximum.Whenthereiscompleteseparation,theloglikelihoodgetscloserto0astheslopeincreases.However,itgetsclosertoanegativenumberthatiscloseto0inthecaseofquasi-completeseparation.Whenthereisasinglebinaryexplanatoryvariableandabinaryresponsevariabletakingvalues0and1,thereiscompleteseparationifdiagonal(maindiagonalorantidiagonal)entriesareall0,asinTable 4-3 95

PAGE 96

Theloglikelihoodasafunctionoftheslopeforthecompleteseparationexample[ReprintedwithpermissionfromWiley.Altman,M.,Gill,J.andMcDonald,M.P.(2004)ConvergenceProblemsinLogisticRegression,inNumericalIssuesinStatisticalComputingfortheSocialScientist.Wiley,NewYork.(Page241,Figure10-1)] Figure4-2. Theloglikelihoodasafunctionoftheslopeforthequasi-completeseparationexample[ReprintedwithpermissionfromWiley.Altman,M.,Gill,J.andMcDonald,M.P.(2004)ConvergenceProblemsinLogisticRegression,inNumericalIssuesinStatisticalComputingfortheSocialScientist.Wiley,NewYork.(Page242,Figure10-2)] 96

PAGE 97

Dataexhibitingcompleteseparationwhenthereisabinaryexplanatoryvariable Thereisquasi-completeseparationifoneofthefourentriesinthetableis0.Ingeneral,foranybinaryindependentvariableinalogisticregression,themaximumlikelihoodestimateoftheregressioncoefcientdoesnotexistwheneverthereisazeroinanycellofa22tableformedbythatbinaryindependentvariableandtheresponsevariable.Notethatwhenthereisasingleemptycell,theoddsratiois0or1,dependingonthelocationoftheemptycell.Therefore,themaximumlikelihoodestimateoftheslope,whichisthelogoddsratio,doesnotexist.However,thereisstillinformationavailableabouttheassociation.Oneoftheendpointsofthelikelihood-ratiocondenceintervalisnite,thatis,ithastheform(,U)or(L,1),dependingonthelocationoftheemptycell.Forexample,whenn11=0butothernij>0ina22table,log=andacondenceintervalhastheform(,U)forsomeniteupperboundU.Whentheresponsevariableisbinaryandthereisamulti-categoryqualitativeexplanatoryvariable,thereiscompleteseparationifalltheobservationswiththesamevaluefortheexplanatoryvariablefallinthesameresponsecategory.Thereisquasi-completeseparationifthereisnotcompleteseparation,butatleastoneoftheentriesinthetableis0.Letxbeaqualitativeexplanatoryvariablethattakesvalues1,2,and3.TherstexamplegiveninTable 4-4 isanexampleofcompleteseparationsinceallobservationswithx=1andx=3havearesponseof0andallobservationswithx=2havearesponseof1.Thesecondandthethirdtablesareexamplesofquasi-completeseparation. 97

PAGE 98

Dataexhibitingcompleteandquasi-completeseparationwhenthereisaqualitativeexplanatoryvariable Nextweconsideranexamplewheretherearetwobinarypredictorsandabinaryresponsevariable.Table 4-6 givesthemarginaltablesforthedatainTable 4-5 Table4-5. DataforwhichMLEdoesnotexistforoneofthepredictorswhentherearetwobinarypredictors Yx1x2Count 0007001501012011101008101011091110 Table4-6. MarginaltablesforthedatagiveninTable 4-5 1917x2=1 150 Themaximumlikelihoodestimateoftheregressioncoefcientforx2doesnotexistsincethereiszeroinoneofthecellsofthe22tableformedbyx2andtheresponsevariable.Table 4-7 andTable 4-8 givethepartialtablesforthedatagiveninTable 4-5 Table4-7. PartialtablesforthedatagiveninTable 4-5 (foreachlevelofx1) 98

PAGE 99

PartialtablesforthedatagiveninTable 4-5 (foreachlevelofx2) Notethatbothentriesinacolumnare0whenx2=1.Inthiscase,theresponsesisofonetype,thatis,y=1.Wehavenoinformationabouttheoddsratiounlikethecasesforwhichthelikelihood-ratiocondenceintervalfortheregressioncoefcienthadoneendpointthatisnite. Haberman ( 1974 )gaveaneasywaytoverifytheexistenceofMLEforthiscaseasfollows:Letthelevelsofthebinaryvariablestakevalues0and1.Let=+1ifx1+x2+yisevenand-1ifx1+x2+yisodd.TheMLEexistsifandonlyifthezerosareatlocationswherethereareonlyeither+1'sor-1's,butnotboth. Table4-9. Haberman'smethodofdetectingexistenceofMLEwhentherearetwobinaryexplanatoryvariablesandabinaryresponsevariable Usingthisidea,wecaneasilyseethattheMLEdoesnotexistforthedatagiveninTable 4-5 sinceoneofthezerosisinalocationwherethereis+1andtheotheroneisinalocationwherethereis-1. Table4-10. AnapplicationofHaberman'smethodwhenMLEdoesnotexist TheMLEexistsforthedatagiveninTable 4-11 sincezerosareonlyinlocationswherethereare-1's.Next,wereviewsomeexamplesofseparationandoverlapgivenby So ( 1995 )inaSAStutorialwhentherearetwoquantitativeexplanatoryvariablesandabinary 99

PAGE 100

AnapplicationofHaberman'smethodwhenMLEexists responsevariable.Figures 4-3 4-4 ,and 4-5 showanexampleofcompleteseparation,quasi-completeseparation,andoverlap,respectively.Notethatthereexistsavectorthatcorrectlyallocatesallobservationstotheirgroupwhenthereiscompleteseparation.Thevector1=(6,2,1)satises 62x1+x2>0foralli2I1and62x1+x2<0foralli2I2.Whenoneofthex2valueswaschangedfrom44to64,wegetanexampleofquasi-completeseparation.Notethatthevector1=(4,2,1)satisesthebelowequations: Silvapulle ( 1981 )gavenecessaryandsufcientconditionsfortheexistenceanduniquenessoftheMLEforthebinaryresponsecaseusinggeometricconceptsinvolvingcones.ThedenitionsofrelatedconceptsweregivenbySpeckmanetal.(2009)asfollows:`AconeCRmisasetofpointssuchthatifx2C,thenx2Cforany0.AconeCRmisaconvexconeifitisalsoclosedunderaddition.Aconicalcombinationofasetofvectorsfa1,...,alg2Rmisx=lXj=1jaj, 100

PAGE 101

AnexampleofcompleteseparationSource:So(1995)[ReprintedwithpermissionfromSAS.SO,Y.(1995).Atutorialonlogisticregression.TechnicalNote450,SASInstituteInc.,Cary,NC.(Page2,Figure1)] Figure4-4. Anexampleofquasi-completeseparation[ReprintedwithpermissionfromSAS.SO,Y.(1995).Atutorialonlogisticregression.TechnicalNote450,SASInstituteInc.,Cary,NC.(Page2,Figure2)] Figure4-5. Anexampleofoverlap[ReprintedwithpermissionfromSAS.SO,Y.(1995).Atutorialonlogisticregression.TechnicalNote450,SASInstituteInc.,Cary,NC.(Page3,Figure3)] 101

PAGE 102

Silvapulle 1981 ):Assumethedesignmatrixisfullrank.Lettheconditionbedenedby:S\T6=;oroneofS,TisRp.(i)TheMLEofexistsandtheminimumsetfgisboundedonlywhenissatised.(ii)Supposethatl()isaproperclosedconvexfunctiononRp.TheMLEexistsandtheminimumsetisboundedifandonlyifissatised.(iii)SupposethatlogFandlog(1F)areconvexandxi1=1foreveryi.TheMLEexistsandtheminimumsetfgisboundedifandonlyifS\T6=;.LetusfurtherassumethatFisstrictlyincreasingateverytsatisfying0
PAGE 103

Jacobsen ( 1989 ).ThesetofstructurevectorsinthebinarycaseisA=f(1)yi+1xi:i=1,...,ng.NotethatthesetAcanalsobewrittenasA=fxifori2I1(yi=1),xifori2I2(yi=0):i=1,...,ng.Thedenitionsofseparationandoverlapusingthesetofstructurevectors,A,aregivenasfollows: 103

PAGE 104

ChenandShao ( 2001 )statedthatSilvapulle'sconditionsontheconvexityoflogForlog(1F)arenotneededandprovedthefollowingtheoremwhichstatesthat(C1)and(C2)aresufcientfortheexistenceofMLEforthebinarycase.Theorem( ChenandShao 2001 ):AssumeFiscontinuous,andthat(C1)thedesignmatrixisfullrank,(C2)thereexistsapositivevectora=(a1,...,an)02Rn,i.e.,eachcomponentai>0,suchthatX0a=0,whereX0isthematrixwithrowsx0iifyi=0,andx0iifyi=1,thentheMLEofexists.Furthermore,if00gT=coni(xi:yi=0)=fx:x=nXi=1,yi=0ixi,i>0g. 104

PAGE 105

Table4-12. DatathatisusedtoillustrateChenandShao'smethod ThedatacanbealsowrittenasinTable 4-13 : Table4-13. DatagiveninTable11 Now,X0=266664111111111100011000010001111110377775. 105

PAGE 106

( 1984 )provedtheoremsontheexistenceofmaximumlikelihoodestimatesforthemultinomiallogisticmodel.Theyshowedthattheexistenceofmaximumlikelihoodestimatesdependonthedatapatterns,whichfallintothreemutuallyexclusiveandexhaustivecategories:completeseparation,quasi-completeseparation,andoverlap. SantnerandDuffy ( 1986 )modiedthestatementsandproofsofAlbertandAnderson'sresults.Beforegivingthedenitionsofthesecategoriesinthegeneralcase,weintroducesomenotation.LetXdenotethen(p+1)matrixwiththex0iasrows,wherexi=(1,xi1,...,xip)0isthecolumnvectorofexplanatoryvariablesforthesubjecti,i=1,...,n.LetYibetheresponsevariableforsubjectithattakesavalueofj(1jL,L>2)withprobabilityP(Yi=jjxi)=exp(0jxi) 106

PAGE 107

4.2 ,itcanbeeasilyseenthatthereiscompleteseparationifthereexistsavector=12Rp+1suchthat01xi>0foralli2I1and01xi<0foralli2I2,whenL=2. 107

PAGE 108

LesaffreandAlbert ( 1989 )introducedtheconceptof`partialseparation'inlogisticdiscrimination.Theystatedthatseparationcanoccurevenwhensomegroupsoverlapsubstantially.Theydenedpartialseparationascompleteseparationofclustersofgroups.Similarly,quasi-partialseparationisdenedasthequasi-completeseparationofclustersofgroups.Theydenedaclustertobesingularifitcontainsobservationsfromonegrouponly.Theremustbeatleastoneclusterthatisnotsingulartotalkaboutpartialseparationsinceotherwisetherewouldbecompleteseparationorquasi-completeseparationofthegroups.Figure 4-6 ,takenfrom LesaffreandAlbert ( 1989 ),summarizesvepossibledatacongurations:a)completeseparationb)quasi-completeseparationc)partialseparationd)quasi-partialseparatione)overlap.InFigure 4-6 (c),(d)and(e),therearesixgroupsHj,j=1,...,6,formingthreeclustersC1(=H1[H2),C2(=H3[H4[H5),andC3(=H6).Figure6(c)isanexampleforpartialseparationsincethreeclustersarecompletelyseparated,whereasFigure6(d)isanexampleforquasi-partialseparationsincetwoclustersarequasi-completelyseparated.Notethattheremightbemanyoverlappingsamonggroupsinaclusterincasesofpartialorquasi-partialseparation.InFigure6(e),althoughC1andC2arecompletelyseparatedfromeachother,theybothintersectwithC3,andhencethereisoverlapandtheMLEexists.Ingeneral,aslongasthereisnogroupthatiscompletelyorquasi-completelyseparatedfromalltheothergroups,thereisoverlap. 108

PAGE 109

Representationofthevepossibledatacongurations[ReprintedwithpermissionfromWiley.LESAFFRE,E.andALBERT,A.(1989).Partialseparationinlogisticdiscrimination.JournalofRoyalStatisticalSociety,SeriesB,51109-116.(Page111,Figure1)] Next,wewillgivesomesimpleexamplesofseparationforthemultinomiallogisticmodelwhenthereisasinglequantitativeexplanatoryvariableandacategoricalresponsevariable.Herearesomeexamplesofcompleteseparation: Table4-14. Examplesofcompleteseparation x=1x=2x=3 Y=1100Y=2010Y=3001 x=1x=2x=3 Y=1001Y=2010Y=3100 x=1x=2x=3 Y=1001Y=2100Y=3010 Someexamplesofquasi-completeseparationaregiveninTable 4-15 .Finally,wegivesomesimpleexamplesofpartialquasi-completeseparationinTable 4-16 .Intheseexamples,althoughthereisoverlapamongsomecategories,thereisatleastonecategorythatdoesnotoverlapwithanyoftheothercategories.Therefore,theMLEdoesnotexist.Intherstexample,thereisoverlapbetweenH1and 109

PAGE 110

Examplesofquasi-completeseparation x=1x=2x=3 Y=1111Y=2100Y=3100 x=1x=2x=3 Y=1111Y=2001Y=3001 Table4-16. Examplesofpartialquasi-completeseparation x=1x=2x=3 Y=1101Y=2100Y=3010 x=1x=2x=3x=4 Y=10100Y=21001Y=31000Y=40010 ( 2009 )gavenecessaryandsufcientconditionsfortheexistenceofmaximumlikelihoodestimatesfortheclassofmultinomialchoicemodels.First,wereviewtheirnotation.SupposeY1,...,Ynareindependentrandomvariables,whereYitakesavalueofj(1jL,L>2)withprobabilityP(Yi=jj)=pj(Xi,),fori=1,...,n.LetXi=(xi1,...,xiL)bethemLdesignmatrixforsubjecti,andbeaparametervectoroflengthm.Choicemodelsandclassicationmodelsarethetwocommontypesofmodels.Usingchoicemodels,subjectimakeschoicejwithprobabilitypj(Xi)=exp(x0ij) 110

PAGE 111

Speckmanetal. ( 2009 )statedthatxijcanbereplacedbyxijxiL,forj=1,...,L1takingxiL=0foridentiability.Usingtheclassicationmodel,subjectiisclassiedintooneofLresponsecategoriesbasedonthevectorofcovariatesxioflengthp.Themultinomiallogisticmodelisawidelyusedclassicationmodel,whereP(Yi=j)=exp(x0ij) 1+PL1l=1exp(x0il),j=1,...,L1,andP(Yi=L)=1 1+PL1l=1exp(x0il).Theclassicationmodelisaspecialcaseofthechoicemodelwherem=p(L1),x0i1=(x0i,00,...,00)x0i2=(00,x0i,...,00)...x0i,L1=(00,00,...,x0i)x0i,L=(00,00,...,00),and=(1,...,L1)0.Speckmanetal.generalizedthedenitionsandresultsgivenbyAlbertandAndersontomultinomialchoicemodels.Thesimplestclassicationruleistoclassifyobservationiintogroupjif (4) AlbertandAndersonstatedthatthereiscompleteseparationifallobservationscanbecorrectlyallocatedtotheirgroupsusing( 4 ).Denitions,givenbyAlbertandAnderson,areextendedtochoicemodelsbySpeckmanetal.asfollows: 111

PAGE 112

112

PAGE 113

4.2 ,aconicalcombinationofasetofvectorsfa1,...,alg2Rmisx=lXj=1jaj,wherej0forj=1,..,l,andtheconicalhullofasetC2RmisthecollectionofallconicalcombinationsofvectorsfromC,i.e.,coni(C)=fiXj=1jaj:aj2C,j0andiisapositiveintegerg.ThenSpeckmanetal.denedafunction,associatedwithasetC,whichhasimportantpropertiesofanormwhenconi(C)=Rmasfollows: (4) Thefunction,givenin( 4 ),hasbeenreferredtoas`quasi-norm'by TodemandKim ( 2011 ).Itsatisesthetriangularinequality,butforthisfunctiontobeanorm,thefollowingthreepropertiesneedtobesatised:C1.jjbjjc>0,b2Rm,C2.jjbjjc=0ifandonlyifb=0,C3.jjbjjc=jjjjbjjcforall2Randb2Rm.C3holdsonlyif0,however,theotherpropertiesofanormaresatisedifthesamplehasoverlap.Speckmanetal.gavethefollowinglemmawhichrelatesoverlappingwiththenotionofconicalhullandthequasi-norm.Lemma( Speckmanetal. 2009 ):LetAbeanitesubsetofRm.Thefollowingstatementsareequivalent:(i)Thesamplehasoverlap.(ii)coni(A)=Rm.(iii)ConditionsC1andC2hold. 113

PAGE 114

4.2 1 )inChapter 1 114

PAGE 115

1 .Otherlinkfunctionsarepossible.Cumulativelinkmodels,thatlinkthecumulativeprobabilitiestothelinearpredictor,can 115

PAGE 116

(4) LetXbethenpdesignmatrixwithrowsx0i,wherexi=(xi1,..,xip)0denotesthep1vectorofcovariatesforsubjecti,=(1,...,p)0isap1vectorofregressioncoefcients,and=(1,...L1)0isthevectorofinterceptssatisfying1<2<...
PAGE 117

Rockafellar 1972 ):`LetC1andC2benonemptysetsinRn.AhyperplaneHissaidtoseparateC1andC2ifC1iscontainedinoneoftheclosedhalf-spacesassociatedwithHandC2liesintheoppositeclosedhalf-space.ItissaidtoseparateC1andC2properlyifC1andC2arenotbothactuallycontainedinHitself.ItissaidtoseparateC1andC2stronglyifthereexistssome>0suchthatC1+Biscontainedinoneoftheopenhalf-spacesassociatedwithHandC2+Biscontainedintheoppositehalf-space,whereBistheunitEuclideanballfxjjxj1g.Forstrictseparation,C1andC2mustsimplybelongtoopposingopenhalf-spaces.'OneofthetheoremsRockafellarprovedisasfollows:Theorem( Rockafellar 1972 ):LetC1andC2benonemptysetsinRn.ThereexistsahyperplaneseparatingC1andC2properlyifandonlyifthereexistsavectorbsuchthat(a)inffhx,bijx2C1gsupfhx,bijx2C2g(b)supfhx,bijx2C1g>inffhx,bijx2C2g.ThereexistsahyperplaneseparatingC1andC2stronglyifandonlyifthereexistsavectorbsuchthat 117

PAGE 118

Tse 1986 ):Considerthemodelgivenin( 4 ).AssumeF(t)isstrictlyincreasing,logf(t)isconvexwithrespecttotand0
PAGE 119

ChenandShao 1999 ):AssumethatZ1jujp+L1dF(u)<1.Ifthefollowingconditionsaresatised:(C1)fi:yi=1gandfi:yi=Lgarenonemptysets;(C2)8l=1,1lp,8al0,br0,withPpl=1al+PL1r=2br>0,minj:yjL1(yjXr=2br+pXl=1xjllal)1(yi1Xr=2br+pXl=1xillal),thentheposteriorisproper,thatis,RRRpL(,jy)dd<1.ChenandShaotooktheinterceptsasincreasingratherthanstrictlyincreasing,allowingfortheresponsecategoriestobeempty,exceptfortherstandthelastcategories.TheconditionsofTheorem1donotseemveryintuitive,butbasically,(C1)isnecessarytohaveenoughinformationtoestimatetheintercepts,1andL1,whereas(C2)isessentialtoestimatetheslopeandalsotohaveaproperposterior.Theorem2( ChenandShao 1999 ):IfF(b)F(a)>0foreveryb>a,then(C1)and(C2)arenecessaryconditionsfortheposteriorpropriety.Combiningthesetwotheorems,(C1)and(C2)arenecessaryandsufcientconditionsfortheposteriorproprietyifFsatisesZ1jujp+L1dF(u)<1andF(b)F(a)>0foreveryb>a.Thesetwoassumptionsaresatisedforthewidelyusedprobit,logitandlog-loglinkmodels.However,condition(C2)maybehardtoverify,andhence,thefollowingpropositionofChenandShaogivestwosufcientconditionsfor(C2). 119

PAGE 120

ChenandShao 1999 ):AssumethattherstandthelastcategoriesarenonemptyandthatthedesignmatrixXisfullcolumnrank.Then(C2)holdsifoneofthefollowingconditionsaresatised:(C3)Thereexistsapositivevectora=(a1,...,an)02Rn,i.e.,eachcomponentai>0,suchthata0X1=0,whereX1=(xi,i:yi=L,xj,j:yj0,suchthata0X2=0,whereX2=(xi,i:yi=1,xj,j:yj>1)withxi=(1,x0i)0.ChenandShaoalsoexaminedtherelationshipbetweentheproprietyandtheexistenceofmaximumlikelihoodestimatesandprovedatheoremthatgivessufcientconditionsfortheexistenceofmaximumlikelihoodestimates.Theorem3( ChenandShao 1999 ):AssumethatFiscontinuous.If(C1)and(C2)aresatised,thenthemaximumlikelihoodestimateof(,)exists.ItcanbeeasilyseenfromTheorem2and3thatwhenFiscontinuousandsatisesF(b)F(a)>0foreveryb>a,thentheposteriorproprietyimpliestheexistenceofmaximumlikelihoodestimates.TheposteriorisnotproperiftheMLEdoesnotexist.Hence,thistheoremcanbeusedtocheckimproprietyoftheposterior.TheLOGISTICprocedureofSASchecksifthereiscompleteorquasi-completeseparationinthedatapatternthatwouldyieldinniteestimatesandgivesawarning.Examplesrelatedtothesetheoremswillbegivenlater.ThetheoremsandpropositionprovedbyChenandShaowillbemodiedinSection 4.4.2 120

PAGE 121

1+exp(j+x0i),withfjgincreasinginj,0=andL=1.Wereviewedthetheoremsontheexistenceofmaximumlikelihoodestimatesforthecumulativelogitmodeloftheproportionaloddsform.Next,wemodifyTse'stheoremandgeneralizeChenandShao'sresults. Table4-17. Anexampleofquasi-completeseparationtoillustrateTse'stheorem x=1x=2x=3 Y=1111Y=2001Y=3001 SASgivesawarningofquasi-completeseparation,andhugestandarderrorsareoutput.Sincep=1inthiscase,wewillcheckifthereexistsl2Rsuchthatmaxx2Ljxl0,maxx2L1xl=3landminx2U1xl=3l.Similarly,maxx2L2xl=3landminx2U2xl=3l.However,3l<3lisnottrue.Ifl<0,maxx2L1xl=landminx2U1xl=3l.Similarly,maxx2L2xl=landminx2U2xl=3l.Again,l<3lisnottruesincel<0inthiscase.Thus,sincetheredoesnotexistl2Rsatisfyingthecondition,theMLEshouldbebounded,butitisnot.ThetheoremgivenbyTserequiredthatthereisnostrict 121

PAGE 122

4-17 ,theMLEdoesnotexistevenifthereisnostrictseparation,butthereisproperseparation.Hence,wereplacethestrictinequalitywith`lessthanorequalto'intheequation:maxx2Ljx0l
PAGE 123

1 ,whoseproofisgiveninAppendix B ,wecaneasilychecktheexistenceofMLE.Now,wegobacktothemotivatingexampleofthissection: Table4-18. Theexampleofquasi-completeseparationusedtoillustrateTse'stheorem x=1x=2x=3 Y=1111Y=2001Y=3001 Inthiscase,sincetherearenodiscordantpairs,therearenodiscordantpairsforanycollapsingsoftheordinalresponsetoabinaryresponse.Hence,theMLEdoesnotexist.Infact,thereisquasi-completeseparation.Now,weconsideratypicalexampleofcompleteseparationwherealltheobservationsfallonadiagonalofthetable.FortheexamplegiveninTable 4-19 example,therearenoconcordantpairs,andtheMLEdoesnotexist. Table4-19. Anexamplewheretherearenoconcordantpairs x=1x=2x=3 Y=1001Y=2010Y=3100 Inthissubsection,weprovedthattheMLEdoesnotexistevenifthereisnostrictseparation,butthereisproperseparation.Hence,wereplacedthestrictinequalitywith`lessthanorequalto'intheequation:maxx2Ljx0l
PAGE 124

ChenandShao 1999 ):AssumethatZ1jujp+L1dF(u)<1.Ifthefollowingconditionsaresatised:(C1)fi:yi=1gandfi:yi=Lgarenonemptysets;(C2)8l=1,1lp,8al0,br0,withPpl=1al+PL1r=2br>0,minj:yjL1(yjXr=2br+pXl=1xjllal)1(yi1Xr=2br+pXl=1xillal),thentheposteriorisproper,thatis,RRRpL(,jy)dd<1.Asmentionedbefore,condition(C2)maybehardtoverify,andhence,thefollowingpropositionofChenandShaogivestwosufcientconditionsfor(C2).Proposition1( ChenandShao 1999 ):AssumethattherstandthelastcategoriesarenonemptyandthatthedesignmatrixXisfullcolumnrank.Then(C2)holdsifoneofthefollowingconditionsaresatised:(C3)Thereexistsapositivevectora=(a1,...,an)02Rn,i.e.,eachcomponentai>0,suchthata0X1=0,whereX1=(xi,i:yi=L,xj,j:yj0,suchthata0X2=0,whereX2=(xi,i:yi=1,xj,j:yj>1)withxi=(1,x0i)0.Itiseasytocheckif(C3)or(C4)issatisedbytheSIMPLEXfunctionofR,andifoneofthemholds,wecansay(C2)holds.However,ifneither(C3)nor(C4)holds,wecannotverify(C2).Forcumulativelinkmodels,ifthereisatleastonecollapsingthatgivesinformationabouttheslope,wecanestimateit.Withthisproposition,Chenand 124

PAGE 125

B forthesakeofcompletenessalthoughsomeofthemhaveonlyminorchangesfromtheoriginalproofs.

PAGE 126

4-20 givesthecountsforeachcombinationofthelevelsoftheexplanatoryvariableandtheresponsevariable. Table4-20. AnexampleforwhichMLEExistsforthecumulativelogitmodel x=1x=2x=3 Y=1001Y=2300Y=3020Y=4100 UsingChenandShao'snotation,X1=0B@111111111212131CAandX2=0B@131111111112121CA.UsingtheSIMPLEXfunctionofR,wecouldnotndavectora=(a1,...,a7)0suchthateachai>0thatsatises(C3)or(C4).Hence,wecannotverifythat(C2)holds.However,usingthegeneralizedversionoftheproposition,wecheckifthereexistsapositivevectorasuchthata0D1=0,a0D2=0ora0D3=0,whereD1=0B@131111111112121CA,D2=0B@111111131112121CA,andD3=0B@111111121213111CA. 126

PAGE 127

Chenetal. ( 2009 ).Lemma( Chenetal. 2009 ):LetXbeannp(p0,i.e.,vi>0fori=1,...,n)suchthatX0v=0, 127

PAGE 128

ChenandShao ( 1999 ).TherstoneisanexampleforwhichtheMLEexistsforthecumulativelogitmodel. Table4-21. TheexampleforwhichtheMLEexistsforthecumulativelogitmodel x=0x=1 Y=111Y=211Y=310 NotethatX1=(0,1,0,1,0)0,andX2=(0,1,0,1,0)0fortheexamplegiveninTable 4-21 .TheMLEexistsinthiscase,andthereexistsapositivevectorasuchthata0X1=0issatised.Next,weconsidertheexampleforwhichtheMLEdoesnotexistforthecumulativelogitmodel.FortheexamplegiveninTable 4-22 ,X1=X2=(0,1,0,0)0.WecanconcludethattheMLEdoesnotexistforthisexamplesincetheredoesnotexistapositivevectorathatsatisesa0X1=0ora0X2=0. 128

PAGE 129

TheexampleforwhichtheMLEdoesnotexistforthecumulativelogitmodel x=0x=1 Y=111Y=210Y=310 ThefollowingcorollaryfollowsdirectlyfromTheorem 5 3 ,wecanconcludethatiftheMLEdoesnotexistwhenasubsetofppredictorsareused,itwillnotexistwhenalloftheppredictorsareusedinthemodel. 129

PAGE 130

SAS 2008 )IfoneusesFREQinsteadofWEIGHTstatement,therewillbeacheckandonegetswarningmessagesifthereiscompleteorquasi-completeseparation.Thischeckisavailableforalllogisticmodelsandforanynumberofresponsecategories.The 130

PAGE 131

Speckmanetal. ( 2009 )andthelemmaby Chenetal. ( 2009 ),wecanwritethefollowinglemma,whichgivesapracticalmethodoftestingifthereisnooverlap. 1 ,iftheMLEexists,thereexistsapositivevectora2Rnsuchthata0X=0,whereXisthematrixwheretheelementsofAaretherowsofXandnisthenumberofelementsinsetA,namely,X0isthematrixwithrowsx0iifyi=0,andx0iifyi=1,asChenandShaostatedintheirtheorem. 131

PAGE 132

DataforExample1 ix1x2yi Hence,A=fx11x12,x21x22,x31x32,x41x42,x52x51,x62x61,x72x71,x82x81g.Therefore,X=26666666666666666666641001001111111011011011103777777777777777777775.TheSIMPLEXfunctiongivesa=(0.18,0.09,0.18,0.09,0.09,0.09,0.09,0.27)0.Notethata=(2,1,2,1,1,1,1,3)0isalsoasolutionandthevectorsaandasatisfya0X[,j]=0forallj,i.e.,a0X[,1]=a0X[,2]=a0X[,3]=0. 132

PAGE 133

Table4-24. DataforExample2 ixiyi Now,wewritexiforeachi.Notethatx1=x2=110000110000,x3=x4=120000120000,andx5=130000130000.Hence,A=fx11x12,x11x13,x23x21,x23x22,x32x31,x32x33,x43x41,x43x42,x51x52,x51x53g.Therefore,X=26666666666666666666666666664111111001100001112120012120000121313130037777777777777777777777777775. 133

PAGE 134

4-25 andnoticingthatthereisnocategorythatiscompletelyorquasi-completelyseparatedfromothers.Notethatalthoughcategories2and3arequasi-completelyseparated,theybothoverlapwithcategory1. Table4-25. CountdataforExample2 Y=1101Y=2010Y=3110 Table4-26. DataforExample3 x=1x=2x=3 Y=1100Y=2001Y=3010 InordertocheckiftheMLEexistsforthecumulativelogitoftheproportionaloddsform,wewriteD1=111312andD2=111312.Inthiscase,whena=(1,1,2)0,wehavea0D2=0.Hence,a0Dj=0foratleastonejasrequired.TheMLEdoesnotexistforthemultinomiallogisticmodelforthisdataset.Inthiscase,Xiswrittenasfollows: 134

PAGE 135

4-26 thatallcategoriesarecompletelyseparatedfromtherestofthecategories,andhencetheMLEdoesnotexist. 135

PAGE 136

1 ,webrieyreviewedlogisticmodelsusedformultinomialresponses.Themultinomiallogisticmodelisthemostexiblemodelforanalyzinganordinalresponse.However,itlacksparsimonyandfailstoexploittheordinalstructureoftheresponsevariable.Thestereotypemodel,whichwasintroducedby Anderson ( 1984 ),canbeusedforbothorderedandunorderedcategoricalresponses.Itisappropriatetousethestereotypemodelwhentheresponsevariableisgeneratedbyanassessorwhomakesthejudgmentoftheleveloftheorderedvariableafterprocessinganindeterminateamountofinformation.Thestereotypemodelisaspecialcaseofthemultinomiallogisticmodelandamoregeneralformoftheadjacent-categorieslogitmodelwithproportionaloddsstructure.Itismoreparsimoniousthanthemultinomiallogisticmodelduetoaproductrepresentationofthelog-odds-ratiosintermsofcategory-specicscoresandacommonparametercorrespondingtoeachpredictor.Itgivesunbiasedandvalidinferenceunderoutcome-stratiedsampling(asincase-controlstudies).Italsoallowsforthestratum-specicnuisanceparameterstobeeliminatedbyusingaconditionallikelihood.Despitethemanyappealingfeatures,thestereotypemodelhasnotbeenwidelyusedbecauseoftheproblemsinestimationduetononlinearityintheparameters.Inprinciple,itcanbeeasilytwithcurrentsoftware,butstartingvaluesfortheparametersandidentiabilityconstraintsneedtobechosencarefully.Wecomparedresultsfromdifferentprograms,andobservedthattheNLMIXEDprocedureofSASandtheGNMandVGAMpackagesofRgivesimilarresults.Therehavebeennostudiescomparingdifferentmethodsforcondenceintervalsinthestereotypemodel.InChapter 2 ,weconsideredthreecondenceintervalmethods: 136

PAGE 137

3 ,weintroducedapartialstereotypemodel,whichisamodelthatfallsbetweenthestereotypemodelandthemultinomiallogisticmodel.Itismoreparsimoniousthanthemultinomiallogisticmodel,butyet,moreexiblethanthestereotypemodel.Weshowedthatthepartialstereotypemodeltsbetterthanalternativemodelstoaclinicaltrialdataset.Therefore,thepartialstereotypemodelisacandidatemodel,butonlywhenthereareatleastthreeexplanatoryvariables.Wedeterminedidentiabilityconstraintsandexaminedsomepropertiesofthepartialstereotypemodel.Thepartialstereotypemodelalsogivesunbiasedandvalidinferenceunderoutcome-stratiedsamplingasthestereotypemodel.Weshowedhowthestandardcondenceintervalmethodsextendtothepartialstereotypemodel.Infuturework,weplantodoasimulationstudytodeterminethemethodthatperformsbetterintermsofachievingerrorratesclosetothenominallevelwithbothsmallandlarge 137

PAGE 138

138

PAGE 139

4 ,wefocusontheexistenceofmaximumlikelihoodestimatesforthemultinomiallogisticmodel,themultinomialchoicemodel,andthecumulativelinkmodelofproportionaloddsform.AlbertandAnderson(1984)showedthattheexistenceofthemaximumlikelihoodestimatesdependsonthedatapatterns,whichfallintothreemutuallyexclusiveandexhaustivecategories:completeseparation,quasi-completeseparationandoverlap.Themaximumlikelihoodestimatesexistonlyifthereisoverlap.Weextendedthedenitionsofseparationandoverlaptothestereotypemodelandtheadjacent-categorieslogitmodel,but,furtherresearchisneededtondpracticalwaysofcheckingtheexistenceofthemaximumlikelihoodestimatesforstereotypeandpartialstereotypemodels.Wesummarizedtheexistingliterature,generalizedsomeresults,andexaminedconnectionsbetweenthedifferentapproaches.Wemodiedthetheoremby Tse ( 1986 )andprovedthatthemaximumlikelihoodestimatesexistforcumulativelinkmodelsifandonlyifthereisnoproperseparation.Wealsoprovedacorollarywhichgivesa 139

PAGE 140

ChenandShao ( 1999 ),relatingtotheproprietyoftheposteriordistributionforBayesianinference.ChenandShaogavetwosufcientconditionsfortheexistenceofmaximumlikelihoodestimates.However,sinceitishardtocheckoneoftheconditions,theyprovedapropositionwhichgivesaneasywayofcheckingthatcondition.Forcumulativelinkmodels,ifthereisatleastonecollapsingthatgivesinformationabouttheslope,wecanestimateit.Withthisproposition,ChenandShaocheckedtherstandthelastcollapsings.However,wecancheckifatleastoneofthecollapsingsisgoodinthissenseratherthanjusttwoofthem.Wegeneralizedthispropositionbyrequiringweakerconditions,whicharemorelikelytohold.Weprovedatheoremthatgivesanecessaryconditionfortheexistenceofmaximumlikelihoodestimatesforcumulativelinkmodels.Wealsoprovedalemmathatgivesapracticalmethodoftestingthepresenceofoverlapusingstructurevectorsandgaveexamplestoillustrateitsapplicationsforthemultinomiallogisticmodel.Theresearchtopicsofconditionsfortheexistenceofthemaximumlikelihoodestimatesforcategoricalregressionmodelsstillhasmanyunsolvedproblems.Weplantoexplorewhenthemaximumlikelihoodestimatesexistforvariouscategoricalresponsemodels,suchastherandomeffectsstereotypemodelsorothercommonlyusedmodelsinthepresenceofmissingdata.Theresearchtopicsofstereotypemodelandpartialstereotypemodelsareworthworkingonsincetheycanbeeasilytwithcurrentsoftwareandhaveappealingfeatures,especiallyforcase-controlstudies. 140

PAGE 141

@m@l 141

PAGE 142

@'jIXi=1LXs=1yisfs+'s0xilogLXl=1exp(l+'l0xi)g!=IXi=1yij0xiIXi=1LXs=1yisexp(j+'j0xi) 142

PAGE 143

@jIXi=1LXs=1yis(s+'s0xilogLXl=1exp(l+'l0xi)!=IXi=1yijIXi=1LXs=1yis!exp(j+'j0xi) 143

PAGE 144

@m@l @mIXi=1yij0xiIXi=1LXs=1yis!ij0xi!=IXi=1yijximIXi=1ni@ij @m@l @mIXi=1fyijniijg!=IXi=1niijxim'jLXt=1it't!.Therefore,E@2l 144

PAGE 145

@'m@l @'mIXi=1fyijniijg!=IXi=1ni@ij A.1 .Letyisbetheobservedcellcountfor(i,s)thcell.Then,thelog-likelihoodisl=logL=IXi=1LXs=1yisfs+'s0xi+0swilogLXl=1exp(l+'l0xi+0swi)g.First,wederive@l=@jr: @jrIXi=1LXs=1yisfs+'s0xi+0swilogLXl=1exp(l+'l0xi+0lwi)g!=IXi=1yijwirIXi=1LXs=1yisexp(j+'j0xi+0jwi)

PAGE 146

@m@l @mIXi=1fyijniijgwir!=IXi=1ni@ij @j@l @jIXi=1fyijniijgwir!=IXi=1ni@ij 146

PAGE 147

@'m@l @'mIXi=1fyijniijgwir!=IXi=1ni@ij @'jIXi=1fyijniijgwir!=IXi=1ni@ij 147

PAGE 148

@mIXi=1fyijniijgwir!=IXi=1ni@ij @mpIXi=1fyijniijgwir!=IXi=1ni@ij @mrIXi=1fyijniijgwir!=IXi=1ni@ij @jpIXi=1fyijniijgwir!=IXi=1ni@ij 148

PAGE 149

149

PAGE 150

1 150

PAGE 151

1 151

PAGE 152

2 152

PAGE 153

153

PAGE 154

154

PAGE 155

3 155

PAGE 156

2 ,wehaveh(u)=Ztr0,r=2,...,L1ZRp(mini:yi>1(uiyi1Xr=2trx0i)+minj:yjs(uj+yjXr=2tr+x0j)).1(mini:yi>1(uiyi1Xr=2trx0i)+minj:yjs(uj+yjXr=2tr+x0j)>0)ddt2...dtL1Ztr1,r=2,...,L1Z(R+)p(mini:yi>1(uiyi1Xr=2trx0i)+minj:yjs(uj+yjXr=2tr+x0j)).1(mini:yi>1(uiyi1Xr=2trx0i)+minj:yjs(uj+yjXr=2tr+x0j)>0)ddt2...dtL1

PAGE 157

157

PAGE 158

158

PAGE 159

8B1+1 8B1)]dds2...dsL1=Zl0,0
PAGE 160

160

PAGE 161

1 161

PAGE 162

2 5 maxx2Ljx0lminx2Ujx0lforatleastonejL1.SincetheMLEexists,theredoesnotexistanonzerol2Rpsuchthat 162

PAGE 163

3 5 .Then,wehavea0Sj=0foratleastonej,1jL1,whereSjisdenedasthematrixwithrowszi(j)s0i,i=1,...,nwithsi=(1,si1,...,siq),whereq
PAGE 164

164

PAGE 165

165

PAGE 166

166

PAGE 167

TezcanOzrazgatBaslantiwasbornin1979inBursa,Turkey.In1996,shewasadmittedtoBogaziciUniversity,Istanbul,fromwheresheearnedherBachelorofScienceinMathematicsin2000.SheworkedinRiskandDataManagementDepartmentinAlternatifBankandProjectManagementDepartmentinOyakBank.ShethenjoinedtheDepartmentofStatisticsattheUniversityofFloridatopursueadoctoratedegree.Duringhergraduateyears,sheservedasanInstructorandTeachingAssistantforseveralundergraduateandgraduatelevelcourses.ShereceivedherPh.D.inStatisticsinAugust2011. 167