This article was downloaded by: [University of California, Los Angeles (UCLA)]On: 16 April 2013, At: 06:10Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Structural Equation Modeling: AMultidisciplinary JournalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/hsem20 Analyzing Mixed-Dyadic Data UsingStructural Equation ModelsJames L. Peugh a , David DiLillo b & Jillian Panuzio ba Cincinnati Children's Hospital Medical Centerb University of Nebraskaâ€“LincolnVersion of record first published: 15 Apr 2013. To cite this article: James L. Peugh , David DiLillo & Jillian Panuzio (2013): Analyzing Mixed-DyadicData Using Structural Equation Models, Structural Equation Modeling: A Multidisciplinary Journal,20:2, 314-337To link to this article: http://dx.doi.org/10.1080/10705511.2013.769395 PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.The publisher does not give any warranty express or implied or make any representationthat the contents will be complete or accurate or up to date. The accuracy of anyinstructions, formulae, and drug doses should be independently verified with primarysources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand, or costs or damages whatsoever or howsoever caused arising directly orindirectly in connection with or arising out of the use of this material.
StructuralEquationModeling ,20:314Â–337,2013 CopyrightÂ©Taylor&FrancisGroup,LLCISSN:1070-5511print/1532-8007onlineDOI:10.1080/10705511.2013.769395 TEACHER'SCORNER AnalyzingMixed-DyadicDataUsingStructural EquationModels JamesL.Peugh, 1 DavidDiLillo, 2 andJillianPanuzio 2 1 CincinnatiChildren'sHospitalMedicalCenter 2 UniversityofNebraskaÂ–Lincoln Mixed-dyadicdata,collectedfromdistinguishable(nonex changeable)orindistinguishable(exchangeable)dyads,requirestatisticalanalysistechniqu esthatmodelthevariationwithindyads andbetweendyadsappropriately.Thepurposeofthisarticl eistoprovideatutorialforperformingstructuralequationmodelinganalysesofcross-sectio nalandlongitudinalmodelsformixed independentvariabledyadicdata,andtoclarifyquestions regardingvariousdyadicdataanalysis specicationsthathavenotbeenaddressedelsewhere.Arti ciallygenerateddatasimilartothe NewlywedProjectandtheSwedishAdoptionTwinStudyonAgin gwereusedtoillustrateanalysis modelsfordistinguishableandindistinguishabledyads,r espectively.Duetotheirwidespreaduse amongappliedresearchers,theAMOSandM plus statisticalanalysissoftwarepackageswereused toanalyzethedyadicdatastructuralequationmodelsillus tratedhere.Theseanalysismodelsare presentedinsufcientdetailtoallowresearcherstoperfo rmtheseanalysesusingtheirpreferred statisticalanalysissoftwarepackage.Keywords :distinguishable,dyadic,exchangeable,indistinguisha ble,mixed Appliedresearchersoftenposeempiricalquestionsdesign edtotesttheoriesregardingindividuals,andsubsequentdataaretypicallyanalyzedassumingt heresponsesfromoneparticipant areindependentofresponsesfromallotherparticipants.A ssuch,appliedresearcherstypically identifythecorrectstatisticalanalytictechniqueneede dtoanswersuchindividualisticresearch questionsbyconsidering(a)thetypeofresearchquestionp osed(e.g.,descriptive,comparative, CorrespondenceshouldbeaddressedtoJamesL.Peugh,Cinci nnatiChildren'sHospital&MedicalCenter,BehavioralMedicine&ClinicalPsychology,UniversityofCincin natiCollegeofMedicine,DepartmentofPediatrics,Cincinnati,OH45229-3026.E-mail:firstname.lastname@example.org
ANALYZINGMIXED-DYADICDATAINSEM 315 orrelationship),(b)thenumberofindependentanddepende ntvariablesunderconsideration, (c)themeasurementscaleofthesevariables,and(d)thesam plingstrategyusedtoobtain thedata.However,researchersalsoinvestigatequestions designedtoclarifyhowindividuals interact,andadyadisdenedasaninteractiverelationshi pbetweenapairofindividuals(e.g., datingormarriedcouples,monozygoticordizygotictwins, oremployerÂ–employee,doctorÂ– patient,orparentÂ–childrelationships).Ifdatawerecoll ectedfromdyadicrelationships,three additionalconsiderationsneedtobeaddressedbeforethec orrectstatisticalanalysistechnique canbeidentiedandapplied:dependence,distinguishabil ity,andthetypeofdyadicvariables tobeanalyzed. Dyadicdependencereferstothefactthatthevariablescore scollectedfromindividuals interactingwithindyadsarenotindependent,butarelikel ytobemorecorrelatedthanscores fromindividualsindifferentdyads(cf.Gonzalez&Grifn, 1999;Kenny,Kashy,&Bolger, 1998).Further,dependenceindyadicdatatakestwoforms.S imilartotraditionalanalysis models(e.g.,ordinaryleastsquares[OLS]regression)tha tspecifysharedpredictorandresponsevariablevariation,intrapersonal(relationships ofvariablescoresfromthesameperson) dyadicdependencereferstothevariationinanindividual' spredictororcovariatescorethat issharedwiththevarianceofhisorherownresponsevariabl escore.However,interpersonal (relationshipsofvariablesfromdifferentpersons)dyadi cdependencereferstothevariationin anindividual'spredictororcovariatescorethatisshared withthevariationoftheotherdyad member'sresponsevariablescore(cf.Kenny,1996).Depend enceisanimportantissuebecause traditionalanalysisapproachesthatassumeindependence ofindividualscores,suchasanalysis ofvarianceandOLSregression,canproducebiasedparamete restimatesandstandarderrorsif appliedincorrectlytodyadicdata.Anumberofanalytictec hniquesareavailablethatestimate thedegreeofdependenceindyadicdata.Perhapsthemostwid elyusedapproachistheintraclass correlation(Gonzalez&Grifn,2002;Kenny&Judd,1996;Mc Graw&Wong,1996),which quantiestheproportionofresponsevariablevariability thatisduetomeandifferencesacross dyads.Dyadicdependenceassessmentisnotreviewedherebe causeresearcherscollectdyadic datatoanswerdyadicresearchquestions,thepresenceofdy adicdependenceistobeexpected, andquantifyingtheamountofdependencepresenttypically doesnotinformtheresearch question.InterestedreaderscanconsultKenny,Kashy,and Cook(2006)foradditionaldetails onassessingdependence.However,asshownintheanalysise xamplesthatfollow,properly specifyingandmodelingboththeintrapersonalandinterpe rsonaldependenceiscrucialto dyadicdatastructuralequationmodeling(SEM)analysis. ThisarticleillustratestheSEManalysisstepsnecessaryt oanalyzemixeddyadicdata (i.e.,datathatvarybothwithinadyadandacrossdyads)sam pledcross-sectionallyand longitudinallyfromeitherdistinguishableorindistingu ishabledyads.Dyadicdistinguishability referstowhetherthetwoindividualscomprisingadyadposs essadistinctivecharacteristicthat candifferentiatetheminamannerthatisrelevanttotheres earchquestionunderinvestigation (e.g.,Kenny&Ledermann,2010).Forexample,individualsm akingupatraditionalmarital dyadcouldbeidentiedbytheirdesignatedgendervalue(e. g.,gender:1 D wives,2 D husbands)andcouldbeconsidereddistinguishable.Bycont rast,same-sexidenticaltwins areanexampleofdyadswhosememberscouldbeconsideredind istinguishablebecausethe designationofÂ“Person1Â”andÂ“Person2Â”withineachdyadwou ldbearbitrary.Although anempiricaltestofdistinguishabilityisavailable(seeG rifn&Gonzalez,1995;Kenny& Cook,1999;Kennyetal.,2006),thattestisnotreviewedher e.Thisarticleproceedsfromthe
316 PEUGH,DiLILLO,PANUZIO assumptionthatitistheresearchquestionunderinvestiga tion,nottheresultsofastatisticaltest, thatdrivesallresearchdesignandstatisticalanalysisde cisionsinvolvingdyaddistinguishability. Tofurtherclarifythedistinguishabilityissue,consider anexampleofidenticalbrother-sister twins.Iftheresearchquestionunderinvestigationwasgen eticinnature,thiswouldsuggestthe twinsbetreatedasindistinguishableinthedataanalysis. However,severalresearchquestions couldbeposed(e.g.,skilllevel)thatwouldsuggestthetwi nsbetreatedasdistinguishablein thedataanalysis.Thekeypointtobemaderegardingtheissu eofdyaddistinguishabilityisthat thequestionisnotwhetheranyvariableexistsonwhichdyad sunderconsiderationcouldbe rendereddistinguishableorindistinguishable,butwheth erthevariablesthatarecrucialtothe theorybeingtestedbytheresearchquestionunderinvestig ationsuggestthedyadsbetreated asdistinguishableorindistinguishableinthedataanalys is. GOALSOFTHISARTICLE Articiallygenerateddataconsistentwiththemeanstruct uresandcovariancematricesof boththeNewlywedProject(DiLilloetal.,2009)andtheSwed ishAdoptionTwinStudy onAging(SATSA;Pedersen,2004)wereusedtoillustratehyp otheticalanalysismodels fordistinguishableandindistinguishabledyads,respect ively.TheNewlywedProjectinvolved datacollectedfromnewlywedcouplesin3consecutiveyears toinvestigatetherelationships betweenvariouspsychologicalphenomenaandtheirsubsequ entimpactonmaritalfunctioning. TheSATSAprojectinvolveddatacollectedfromtwinpairsin 1987,1990,and1993that examinedtherelationshipsbetweenmeasuresofpsychologi calfunctioningandqualityoflife. Asdescribedlater,itisfurtherassumedthattheresearchq uestionsunderinvestigationsuggest thattheNewlywedmaritalcouplesbetreatedasdistinguish ableandtheSATSAtwinsbe treatedasindistinguishableinallofthedyadicdataanaly sisexamples. Thepurposeofthisarticleisto(a)showthatseveraldyadic structuralequationmodels areavailabletomodeltheintrapersonalandinterpersonal dependenceinmixeddyadicdata sampledfromdistinguishableorindistinguishabledyads; (b)providespecicanalysisdetails andexplanationsregardinghowmixeddyadicdatastructura lequationmodelsarespeciedand whycertainmodicationsareneededforindistinguishable dyadstructuralequationmodelsthat havenotbeenaddressedinotherpublications;(c)provideA MOS(Version16)andM plus (Version6.11;MuthÃ©n&MuthÃ©n,1998Â–2010)examplesforalldyad icanalysismodelsdescribed here,aswellassupplementalMicrosoftExcellesdesigned toassistinthecomputationof additionaltindexesfortheindistinguishabledyadanaly sismodels,inanAppendixavailable athttps://bmixythos.cchmc.org/xythoswfs/webui/_xy-4 76611_1-t_AXKarXYG;and(d)provide descriptionsoftheanalysismodelsusedhereinsufcientd etailtoallowresearcherstoapply thesemodelsusingthestatisticalanalysissoftwarepacka geoftheirchoice.Foralldyadicdata analysismodelsshownhere,thedistinguishabledyadanaly sismodelsaredescribedrstso thatthemodelspecicationalterationsneededtoanalyzed atafromindistinguishabledyads withthesameanalysismodelaremoreclearlypresented.Fur ther,Table1presentsanoverview ofthedyadicdataanalysismodelexamplespresented,their respectivelinearmodelequations, theresearchquestionsaddressed,andtheparameterestima teconstraintsinvolvedwitheach. Finally,asdemonstratedinallexampleanalyses,theresea rchquestionunderinvestigationand thedistinguishabilitydecisiontogetherdictatethedyad icstructuralequationmodelneeded
TABLE1 DyadicDataAnalysisModelOrganizationalSchematic Actor-Partner InterdependenceModel (APIM) CommonFate(CF) MediationModel DyadicLatentGrowth CurveModel(DLGCM) Distinguishable Y i D .v i C /b i . Actor / C b i . Partner / C Â© i Y i D .v i C /Âƒ Y Â˜ C Â© i M i D .v i C /Âƒ M Â˜ C Â© i X i D .v i / C Âƒ X ÂŸ C Â© i Â˜ D Â£ 0 ÂŸ C .Â’Â“/ÂŸ C Â— Â˜ Y tid D ÂƒÂ˜ i. INTERCEPT / C Âƒ t Â˜ i. SLOPE / C Â© tid Â˜ i. INTERCEPT / D Â’ i. INTERCEPT / C Â” i .X i / C Â— i. INTERCEPT / Â˜ i. SLOPE / D Â’ i. SLOPE / C Â” i .X i / C Â— i. SLOPE / Equations Indistinguishable Y i D v i C b. Actor & Partner / C Â© i Y i D v C Âƒ Y Â˜ C Â© M i D v C Âƒ M Â˜ C Â© X i D v C Âƒ X ÂŸ C Â© Â˜ D Â£ 0 ÂŸ C .Â’Â“/ÂŸ C Â— Â˜ Y tid D ÂƒÂ˜ . INTERCEPT / C Âƒ t Â˜ . SLOPE / C Â© i Â˜ . INTERCEPT / D Â’ . INTERCEPT / C Â”.X i / C Â— . INTERCEPT / Â˜ . SLOPE / D Â’ . SLOPE / C Â”.X i / C Â— . SLOPE / Unitof Analysis Distinguishable Indistinguishable individualswithindyads dyads dyadsdyads individualswithindyads dyads ParameterEstimateConstraints Distinguishabledyads b 1 . Actor / D b 2 . Actor / I b 1 . Partner / D b 2 . Partner / Indistinguishabledyads b 1 . Actor / D b 2 . Actor / D b 1 . Partner / D b 2 . Partner / v ÂŒv X D v X ;v M D v M ;v Y D v Y Â Â© indistinguishable 2666666664 Â¢ 2 X 0Â¢ 2 X Â¢ X;M 0Â¢ 2 M 0Â¢ X;M 0Â¢ 2 M Â¢ X;Y 0Â¢ M;Y 0Â¢ 2 Y 0Â¢ X;Y 0Â¢ M;Y 0Â¢ 2 Y 3777777775 Â© indistinguishable 2666666664 Â¢ 2 Y 0Â¢ 2 Y 00Â¢ 2 Y Â¢00Â¢ 2 Y 0Â¢00Â¢ 2 Y 00Â¢00Â¢ 2 Y 3777777775 Â˜ indistinguishable ÂŒÂ’ Intercept D Â’ Intercept ;Â’ Slope D Â’ Slope 266664 Â§ Intercept Â§ 1 Â§ Slope 0Â§ 2 Â§ Intercept Â§ 2 0Â§ 1 Â§ Slope 377775 Note. i , 1 ,and 2 subscriptsrefertoindividualswithindyads;symbolslack ingan i subscript,orsymbolswiththesamesubscript(s),indicate parameterestimatesconstrainedtoequalitybetweenindiv idualswithin dyads. .v i C / indicatesresponsevariableinterceptsthatcanbeestimat edindistinguishabledyadanalysestotestforsignicantd ifferencesbetweenindividualswithindyadsasafunctiono fthedistinguishingfactor.317
318 PEUGH,DiLILLO,PANUZIO FIGURE1 DistinguishabledyadactorÂ–partnerinterdependence(API M)analysismodel.** p<:01 . andhowtheintrapersonalandinterpersonaldyadicdepende nceshouldbespeciedwithinthe model. CROSS-SECTIONALDYADICDATAANALYSES Researcherscollectcross-sectionaldatafromdyadstoans werresearchquestionsinvolving interpersonaldynamics,suchaswhetherthepredictorvari ablescoreoftherstdyadmember .X 1 / issignicantlyrelatedtotheirownresponsevariable .Y 1 / score(i.e., X 1 ! Y 1 ;an intrapersonalorÂ“actorÂ”effect),andifthepredictorvari ablescoreoftherstdyadmemberis signicantlyrelatedtotheresponsevariablescoreofthes econddyadmember(i.e., X 1 ! Y 2 ; aninterpersonalorÂ“partnerÂ”effect;Cook&Kenny,2005;Fu rman&Simon,2006;Kenny, 1996).AtraditionalSEManalysismodelusedtoanswerresea rchquestionsinvolvingdyadic intrapersonalandinterpersonalrelationshippatternsis theactorÂ–partnerinterdependencemodel (APIM;cf.Kenny,1996).AnexampleAPIMthatinvestigatest heintrapersonalandinterpersonalrelationshipsbetweenchildhoodpsychologicalm altreatmentandsubsequentmarital satisfactionamongnewlywedcouplesisshowninFigure1.DistinguishableDyadsTheAPIMshowninFigure1estimates(a)predictorvariable( psychologicalmaltreatment)
ANALYZINGMIXED-DYADICDATAINSEM 319 TABLE2 ActorÂ–PartnerInterdependenceModelFitStatistics DistinguishableDyadsIndistinguishableDyads Null Model Analysis Model,Initial Analysis Model,Final Saturated Model Null Model Analysis Model Saturated Model Â¦ 2 387.3813.664.91065.9710.979.99 df 63201036 LogL 4,028.83 3,841.97 3,837.60 3,835.14 3,831.02 3,803.52 3,803.03 means,(b)responsevariable(maritalsatisfaction)inter cepts,(c)predictorvariablevariances, (d)responsevariableresidual .e/ variances,(e)apredictorvariablecovariance( Cx ),(f)a residualscovariance( Cy ),(g)actor( b ACTOR )effects,and(h)partner( b PARTNER )effects.Intrapersonaldyadicdependenceismodeledthroughtheestim ationofactoreffects( b ACTOR ); interpersonaldyadicdependenceismodeledthroughtheest imationofpartnereffects( b PARTNER ) andcovariances( Cx and Cy ).Estimatingeachoftheseparametersseparatelyforthetw odyad membersresultsinasaturatedmodel(inmorecomplexanalys es,theunconstrainedmodel mightnotbesaturated).ThegoalofanAPIManalysisinthedi stinguishabledyadcaseis totestthetofmoreparsimoniousmodelsthatconstrainact orandpartnereffectestimates. Forexample,twoofthemorecommonconstraintpatternsinth edistinguishablecaseinvolve amodelthatconstrainsactoreffects(e.g., b ACTOR ; Wives D b ACTOR ; Husbands )andpartnereffects (e.g., b PARTNER ; Wives D b PARTNER ; Husbands )separatelytoequality(i.e.,where b ACTOR Â¤ b PARTNER ), andamodelconstrainingallfoureffects(e.g., b ACTOR ; Wives D b ACTOR ; Husbands D b PARTNER ; Wives D b PARTNER ; Husbands / toequality.Theseequalityconstraintsallowtestingfors ignicantdifferences inactorandpartnereffectsbetweendistinguishabledyadm embers(Gonzalez&Grifn,2001). Itisimportanttonotethatthestandardnullandsaturateds tructuralequationmodelsarethe appropriatemodelswithwhichtotestthetofanAPIMwithdi stinguishabledyads.However, thedenitionandspecicationoftheappropriatenullands aturatedmodelswilldifferfor indistinguishabledyadanalyses,asshowninthesubsequen tsections. DistinguishableAPIMexample. Asanexample,theAPIMshowninFigure1was estimatedusingtheNewlywedProjectdatatoquantifychild hoodpsychologicalmaltreatment andsubsequentmaritalsatisfactionactorandpartnereffe cts,andtotestforpossibledifferences intheseeffectsbetweennewlywedhusbandsandwives.Assho wnintheleftpanelofTable2, estimatinganinitialanalysismodelthatconstrainedallf ourregressionpathstoequalityresulted inthefollowingchi-squaremodeltindexvalue: Â¦ 23 D 13:66 , p<:01 (notshowninTable1: comparativetindex[CFI] D .97,TuckerÂ–LewisIndex[TLI] D .95,rootmeansquareerrorof approximation[RMSEA] D .09). 1 Modicationindexesshowedthatthehusbandactoreffect 1 Thisarticleusedgeneratedhypotheticaldatatodemonstra tetheproceduresinvolvedinvariousdyadicdata analysesusingSEM,nottoevaluatedyadicdataanalysismod els.ModeltstatisticvaluesarepresentedÂ“asisÂ”;no judgmentsastothequalityofthetofthemodeltothedataar emadeandnotheoreticalconclusionsshouldbedrawn fromthehypotheticalresultspresented.
320 PEUGH,DiLILLO,PANUZIO path( b ACTOR W Husband )shouldbefreed;estimatingthatnalanalysismodelresul tedinthefollowingchi-squaremodeltindexvalue: Â¦ 22 D 4:91 , p>:05 (CFI D .99,TLI D .98,RMSEA D .06).ThenalmodelparameterestimatesareshowninFigure 1.Aschildhoodpsychological maltreatmentforeitherspouseincreased,maritalsatisfa ctionsignicantlydecreasedforboth spouses,butthisdecreasewassignicantlygreaterforthe relationshipbetweenhusbands' childhoodmaltreatmentexperiencesandhusbands'marital satisfaction. IndistinguishableDyadsFittingtheAPIMtodistinguishabledyadicdataisafairlys traightforwardprocess.However, analyzingindistinguishabledyadicdataisamoreinvolved processthatrequiresadditional steps.FromtheSATSAdata,twins'self-reportedfearandli fesatisfactionscoreswillbeused toillustratethestepsinanAPIManalysiswithindistingui shabledyads.Theanalysismodel usedintheexampleisshowninFigure2.However,beforeintr oducingtheanalysissteps,a briefreviewofstructuralequationmodeltisneeded. InSEM,modeltisdeterminedbyestimatingthreemodels:th esubstantiveanalysismodel ofinterest,anullmodel,andasaturatedmodel.Thenull(or baseline)modelestimatesonly FIGURE2 IndistinguishabledyadanalysisactorÂ–partnerinterdepe ndencemodel(APIM).* p<:05 .
ANALYZINGMIXED-DYADICDATAINSEM 321 meansandvariancesforeachanalysisvariable,constrains allpossibleanalysisvariablecovariancestozero,andisconsideredtheworstpossiblemodelto ttoasetofsampledata. 2 Asaturatedmodelisdenedasamodelthatfreelyestimatesallana lysisvariablemeans,variances, andcovariances.Thesaturatedmodelisconsideredthebest ttingmodelpossible,butisnot parsimoniousandisseldomofinteresttoresearchers.Toge therthenull(worstttingmodel possible)andsaturated(bestttingmodelpossible)model sprovideacontinuumwithinwhich toevaluatethetoftheanalysismodel. Theproperchi-squaremodeltstatisticforasubstantives tructuralequationmodelis obtainedbysubtractingthechi-squarestatisticforthesa turatedmodelfromthechi-square statisticfortheanalysismodel.Thischi-squaredifferen cevalueisthentestedbyreferencing ittoachi-squaredistributionatdegreesoffreedomequalt othedifferenceinthenumber ofestimatedparametersbetweentheanalysisandsaturated models.However,thechi-square statisticforatypicalstructuralequationmodelistested directlybyreferencingittoachi-square distributionatdegreesoffreedomequaltothedegreesoffr eedomfortheanalysismodel becauseatypicalsaturatedstructuralequationmodelhasa chi-squarestatisticanddegreesof freedomthatarebothzero.Thischi-squaremodeltstatist icquantiesmisspecication,or thelackoftoftheanalysismodeltothesampledata. Inadditiontotheoreticalmisspecication,indistinguis habledyadstructuralequationmodels containasecondsourceofmodelmist:arbitrarydesignati on.Specically,thedesignation ofPerson1andPerson2withineachindistinguishabledyadw ouldbearbitrary,butnot inconsequential.Asdemonstratedelsewhere(Woody&Sadle r,2005),reversingthisarbitrary Person1/Person2designationforsome,butnotall,ofthedy adsinthesampledatasetcan notablyaltertherelationshipsamonganalysisvariables. Amethodofremovingthisarbitrary mist,leavingonlyaquanticationofanalysismodelmissp ecication,isneededtoaccurately evaluatethetofanAPIMforindistinguishabledyads.Assh ownlater,removingarbitrary designationmistinvolvestheestimationofchi-squarest atisticsanddegreesoffreedomfor nullandanalysisstructuralequationmodelsintheusualfa shion,butitalsoinvolvesthe estimationofaspecialsaturatedmodelwithachi-squarest atisticanddegreesoffreedomthat arebothnonzero.Theterm saturated isusedthroughoutthearticletomaintainaconsistency withthedyadicliteratureeventhoughtheindistinguishab ledyadsaturatedmodelsshownhere arenotsaturatedfromtheusualSEMperspective. Specically,inatypicalstructuralequationmodel,estim atingasaturatedmodelinvolves estimatingallpossibleparametersforasetofdata,whichu sesallavailabledegreesoffreedom andresultsinachi-squaretstatisticvalueofzero.Howev er,theappropriatesaturatedmodel forindistinguishabledyadsinvolvesestimatingallparam etersthatmakelogicalsense,but wouldnotexhaustalldegreesoffreedom.Specically,forA PIMs,separateparameterestimates forthetwodyadmemberswouldnotbeneededforindistinguis habledyadsbecausethere wouldbenotheoreticalreasontoexpectdifferences,andan yobserveddifferenceswould beduetoarbitrarydesignation.Asaresult,anindistingui shabledyadsaturated(orI-SAT; Olsen&Kenny,2006)modelisdenedasamodelthatconstrain sthefollowingsixpairsof 2 Noconsensuscurrentlyexists,eitherintheempiricallite ratureoracrossstatisticalanalysissoftwarepackages,a s towhetherexogenousvariablecovariancesshouldbeestima tedinanullmodel(seeWidaman&Thompson,2003). Forallnullmodelsusedhere(e.g.,Figures4,8,and11),all analysisvariablecovariancesweremanuallyxedtozero inM plus andAMOS.
322 PEUGH,DiLILLO,PANUZIO parameterestimates(i.e.,eachelementofthesamplemeanv ectorandcovariancematrix)to equalitybetweenPerson1andPerson2:(a)predictorvariab lemeans,(b)predictorvariable variances,(c)intrapersonalcovariances,(d)interperso nalcovariances,(e)responsevariable means,and(f)responsevariablevariances,asshowninFigu re3.NoticeinFigure3thatthe modelissaturatedinthetraditionalsense;themodelestim atesallpossibleassociationsamong analysisvariables.However,thedegreesoffreedomvaluef orthismodelisnotzerodueto theequalityconstraintsimposedasaresultofthearbitrar yPerson1/Person2designationof indistinguishabledyadmembers. SeveralpropertiesoftheI-SATmodelarenoteworthy.Ifdya dmemberswereperfectly indistinguishable(i.e.,ifpredictorandresponsevariab lemeans,variances,andcovariances eachwereequivalentbetweenbothdyadmembers),theI-SATm odelchi-squaretstatistic wouldequalzero,themodelwouldhavesixdegreesoffreedom duetotheequalityconstraints, andthemodel-reproducedcovariancematrixwouldmatchthe sampledatacovariancematrix (Carey,2005).Assumingthatthedyadsunderinvestigation arenotperfectlyindistinguishable, FIGURE3 IndistinguishabledyadactorÂ–partnerinterdependencemo del(APIM)saturatedmodel:(a)predictorvariablemeans,(b)predictorvariablevariances,(c)i ntrapersonalcovariances,(d)interpersonalcovariances , (e)responsevariablemeans,and(f)responsevariablevari ancesconstrainedtoequality.Thetwounlabeled covariancesontheleftsideofthemodelarefreelyestimate d(i.e.,notgivenaletterlabel)tomodelinterpersonal dyadicdependence.
ANALYZINGMIXED-DYADICDATAINSEM 323 reversingthePerson1/Person2designationforsome,butno tall,ofthedyadsinthesample datasetwouldchangetheI-SATmodelchi-squaretstatisti cvalue,butthemodelparameter estimateswouldnotchange.ThismeansthattheI-SAT'snonz erochi-squarestatisticvalue quantiesarbitrarydesignationmistonly(Kennyetal.,2 006;Olsen&Kenny,2006). Computingtheproperchi-squaremodeltstatisticforanin distinguishabledyadAPIM requiressubtractingthechi-squaretstatisticforthein distinguishabledyadsaturatedmodel (Figure3,whichcontainsonlyarbitrarydesignationmist )fromthechi-squaretstatisticfor theindistinguishabledyadanalysismodel(Figure2,which containsbothmodelmisspecicationandarbitrarydesignationmist),andtestingthere sultingchi-squaredifferencevalue (whichnowquantiesmodelmisspecicationonly)byrefere ncingachi-squaredistribution withdegreesoffreedomequaltothedifferenceinthenumber ofestimatedparametersbetween theanalysisandsaturatedmodels.(Alternatively,thechi -squaredifferencetestcanalsobeimplementedbysubtractingthelog-likelihoodvaluesofthet womodelsratherthanthechi-square statistics.)Conceptually,theI-SATmodelisthecorrects aturatedmodelforindistinguishable dyadsjustasanunconstrainedAPIMisthecorrectsaturated modelfordistinguishabledyads (Olsen&Kenny,2006).Further,theI-SATmodelservesasthe bestttingindistinguishable modelagainstwhichtotestsubstantiveAPIMsthatcontaina dditionalparameterestimate constraints. AlthoughmostSEManalysissoftwarepackagesprovideanaly sismodelandnullmodel chi-squarestatisticsanddegreesoffreedombydefault,ne itheraretheappropriatestatistics forthepurposesofcomputingadditionalSEMtindexes,suc hastheCFI,TLI,andthe RMSEA,forindistinguishabledyadAPIMs.Computingthecor recttindexesbeginsby estimatingtheappropriateindistinguishabledyadbaseli neornullmodel,whichcanbedone byxingallcovariancesintheindistinguishabledyadsatu ratedmodeltozero.Thisresults inamodelthatestimatesmeansandvariancesonly,asshowni nFigure4.Fortindex computationpurposes,thecorrectanalysismodelchi-squa restatisticanddegreesoffreedom areobtainedbysubtractingtheindistinguishabledyadsat uratedmodel(Figure3)chi-square statisticanddegreesoffreedomfromtheindistinguishabl edyadanalysismodel(Figure2) chi-squarestatisticanddegreesoffreedom.Similarly,th eappropriatechi-squarestatistic anddegreesoffreedomforthenull(orbaseline)modelfort indexcomputationpurposes involvessubtractingtheindistinguishabledyadsaturate dmodel(Figure3)chi-squarestatistic anddegreesoffreedomfromtheindistinguishabledyadnull orbaselinemodel(Figure4) chi-squarestatisticanddegreesoffreedom. IndistinguishableAPIMexample. TheAPIMshowninFigure2wasusedtotestfor differencesintheactorandpartnereffectsrelatingselfreportedfeartolifesatisfactionamong pairsoftwins.TherightpanelofTable2showsthatestimati ngananalysismodelthat constrainedallfourregressionpathstoequalityresulted inthefollowingchi-squaremodel tindexvalue: Â¦ 23 D 10:97 , p<:05 (notshowninTable2:CFI D .74,TLI D .57, RMSEA D .08).However,recallthatthischi-squaretstatisticcon tainsbothmisspecication andarbitrarydesignationmist.Arbitrarydesignationmi stisremovedbysubtractingthe chi-squaretstatisticofthesaturatedmodelfromthechisquaretstatisticoftheanalysis model .10:97 9:99 D 0:98/ .Thischi-squaredifferencevalueisthentestedatachi-sq uare distributionequaltothedifferenceinthenumberofparame tersestimatedbetweenthetwo models .6 3 D 3/ .Afterarbitrarydesignationwasremoved,theanalysismod elshowed
324 PEUGH,DiLILLO,PANUZIO FIGURE4 IndistinguishabledyadactorÂ–partnerinterdependencemo del(APIM)nullmodel:(a)predictor variablemeans,(b)predictorvariablevariances,(e)resp onsevariablemeans,and(f)responsevariablevariances constrainedtoequality. thefollowingchi-squaremodeltindexvalue: Â¦ 23 D :98 , p>:05 (CFI D .96,TLI > 1, RMSEA D .04).Figure2showstheparameterestimatesfromtheanalys ismodel;aseither twin'sfearincreased,lifesatisfactionsignicantlyand equallydecreasedforbothtwins. CommonFateModel:DistinguishableDyadsTheAPIMiswell-suitedtotesttheoreticalrelationshipsa mongvariablesattheindividual level;theactoreffectsquantifyintraindividualinuenc es,andthepartnereffectsquantify theinterindividualinuenceswithindyads.However,cert ainphenomenaofinteresttodyadic researchers,suchasmaritaldiscordorfamilycrises,tend toimpacttherelationshipbetween thedyadmembersandmightbebetterassessedatthedyadleve l.Forexample,ifaresearch questioninvolvedtraditionalmarriedcouplesratingthei rmaritalsatisfaction,theAPIMis neededtoquantifypossibleindividualdifferencesinsati sfactionwithinthedyad.However, iftheresearchquestioninvolvedtheimpactoflifestresso rsoncouples'cohesion,adyadlevelanalysismodelwouldbeneededtoassesstheimpactont herelationships,nottheindividuals.
ANALYZINGMIXED-DYADICDATAINSEM 325 FIGURE5 Distinguishabledyadcommonfatemediationanalysismodel .* p<:05 .** p<:01 . Thecommonfatemodel(cf.Gonzalez&Grifn,1999,2001;Gri fn&Gonzalez,1995; Kenny,1996;Kenny&LaVoie,1985)isastructuralequationm odelthatusesvariablescores collectedfrombothindividualsasindicatorsofadyad-lev elphenomenontotestresearch questionsregardingthedyadicrelationship.Anexampleof acommonfatemediationmodel (e.g.,Ledermann&Macho,2009)isshowninFigure5.Measure mentsofperceivedsocial support,affectionalexpression,andmaritalsatisfactio ntakenfromhusbandsandwivesare usedtotestwhetheraffectionalexpressionmediatesthere lationshipbetweenperceivedsocial supportandmaritalsatisfactionattherelationshipordya dlevel.Themethodsdescribed heretotestforthepresenceofasignicantindirectormedi atedeffect,aswellasthe regressionpathnotation .Â’;Â“;Â£ 0 / showninFigure5,areconsistentwiththemediationstudies ofMacKinnonandcolleagues(seeFritz&MacKinnon,2007;Ma cKinnon,Lockwood,Hoffman,West,&Sheets,2002;MacKinnon,Lockwood,&Williams, 2004)andaredescribed furtherlater. SeveralfeaturesofthecommonfatemodelshowninFigure5ar enoteworthy.Intrapersonal dyadicdependenceismodeledthroughestimatingtheobserv edvariableuniquecovariances (C)foreachdyadmember(i.e.,Husbands:C1,C3,andC5;Wive s:C2,C4,andC6).Although typicallynotestimatedinSEM,theseuniquecovariancesar eneededtomodelspecicintrapersonaldyadicphenomenasuchasacommonmethodvariance,ind ividualperceptualtendencies, oraresponseset,amongothers(Gonzalez&Grifn,1999;Led ermann&Macho,2009; Woody&Sadler,2005).Interpersonaldyadicdependenceism odeledbyusingconrmatory factoranalysis(CFA)methodstocombinetheperceivedsoci alsupport,affectionalexpression, andmaritalsatisfactionscoresofbothhusbandsandwivesi ntothreelatentandshareddyadic variables.AsalsoshowninFigure5,therstfactorloading foreachofthethreeCFAmodels isxedtounity(Bollen,1989;Grifn&Gonzalez,1995).The secondfactorloadingforeach
326 PEUGH,DiLILLO,PANUZIO TABLE3 CommonFateModelFitStatistics DistinguishableDyadsIndistinguishableDyads Null Model Hypothetical Model Saturated Model Null Model Hypothetical Model Saturated Model Â¦ 2 583.282.890660.6113.6913.69 df 153021912 LogL 3,825.44 3,535.241 3,533.79 7,044.93 6,721.47 6,721.47 CFAisfreelyestimated(i.e.,allowedtodiffer)consisten twithdistinguishabledyads(Ledermann&Macho,2009). 3 Forboththedistinguishableandindistinguishablecommon fatemodelanalysisexamples, signicantmediationwaspresentifthebias-correctedboo tstrap95%condenceinterval(e.g., Efron&Tibshirani,1993)forthecombinedindirectormedia tedpath .Â’Â“/ didnotinclude zero(MacKinnon,2008).Ifthecondenceintervaldidnotco ntainzeroandthedirecteffect path .Â£ 0 / wassignicant,partialmediationwaspresent.Ifthecond enceintervaldidnot containzeroanddirecteffectpath .Â£ 0 / wasnonsignicant,fullmediationwaspresent(Fritz& MacKinnon,2007). Distinguishablecommonfateexample. Asanexample,thecommonfatemediation modelshowninFigure5wasestimatedusingtheNewlywedProj ectdatatotestwhether affectionalexpressionmediatedtherelationshipbetween perceivedsocialsupportandmarital satisfactionatthedyadlevel.AsshownintheleftpanelofT able3,estimatingtheanalysis modelresultedinthefollowingchi-squaremodeltindexva lue: Â¦ 23 D 2:89 , p>:05 (not showninTable3:CFI D 1,TLI > 1,RMSEA D 0).AsshowninFigure5,allthreeregression paths .Â’;Â“;Â£ 0 / werestatisticallysignicant,whichsuggestedthepossib ilityofpartialmediation. However,thejointindirectpathestimate .Â’Â“ D 1:51/ showeda95%condenceinterval (basedon5,000bias-correctedbootstrapsamples)thatcon tainedzero[ .16,3.25];affectional expressiondidnotsignicantlymediatetherelationshipb etweenperceivedsocialsupportand maritalsatisfaction.CommonFateModel:IndistinguishableDyadsFigure6showstheanalysismodelusedintheexampleofacomm onfatemediationmodelfor indistinguishabledyads.Thatmodelusestwins'socioecon omicstatus(SES),desireforpersonal growth(Growth),andattitudestowardeducation(ATE)scor estotestwhetheradesirefor personalgrowthmediatestherelationshipbetweenSESanda ttitudestowardeducation.Similar totheAPIMexample,theindistinguishabledyadcommonfate analysismodelalsodiffersfrom thecommonfatemodelfordistinguishabledyadsbytheparam eterestimateconstraintsneeded duetoarbitrarydesignation.Specically,asshowninFigu re6,theindistinguishabledyad 3 Dependingonthestatisticalanalysissoftwarepackageuse d,researchersmightneedtotakeadditionalstepsto avoidestimationerrors.SeetheAdditionalCaveats.docx leintheonlineAppendixmaterials.
ANALYZINGMIXED-DYADICDATAINSEM 327 FIGURE6 Indistinguishabledyadcommonfatemediationanalysismod el:factorloadingsxedtounity; measurementintercepts( I1 D I2 ; I3 D I4 ; I5 D I6 ),uniquevariances( e1 D e2 ; e3 D e4 ; e5 D e6 ),and uniquecovariances( C1 D C2 ; C3 D C4 ; C5 D C6 )eachconstrainedtoequality.** p<:01 . commonfateanalysismodelxesallsixfactorloadingsfort hethreeCFAmodelstounity (Bollen,1989;Grifn&Gonzalez,1995),andconstrainsmea surementintercepts( I1 D I2 ; I3 D I4 ; I5 D I6 ),uniquevariances(i.e., VAR e1 D VAR e2 ; VAR e3 D VAR e4 ; VAR e5 D VAR e6 ), anduniquecovariances(i.e., C1 D C2 ; C3 D C4 , C5 D C6 )toequality(Ledermann& Macho,2009). UnliketheAPIMexample,theI-SATandnull(I-NULL)modelsf oracommonfatemediationmodelanalysiswithindistinguishabledyadsarenotes timatedbecausetheindistinguishable dyadcommonfatemediationmodelandtheappropriateI-SATm odelareequivalentmodels. AsshownintherightpanelofTable3,thechi-squareandloglikelihoodmodeltstatisticsfor boththeindistinguishabledyadcommonfatemediationmode l(Figure6)andtheappropriate saturated(I-SAT)modelforacommonfatemediationmodel(n otshown,butincludedinthe onlineAppendix)areidentical.Further,althoughnotshow nhere,themodel-reproducedmean structuresandcovariancesmatricesforbothmodelsareals oidenticaleventhoughthetwo modelsdifferby .12 9 D 3/3 df . BoththeindistinguishabledyadcommonfatemodelinFigure 6anditsappropriateI-SAT modelconstrainresponsevariablemeansandresponsevaria blevariancestoequalitybetween dyadmembers.Thereasonforthe3 df differencebetweenthesetwoequivalentmodelscanbest beexplainedintermsofdifferencesbetweeninterpersonal covariancesthatareexplicitlyversus implicitlyestimatedandconstrainedtoequality.Specic ally,threepairsofsame-variableinterpersonalcovariances(i.e.,cov[Twin1SES,Twin2SES],cov [Twin1Growth,Twin2Growth],
328 PEUGH,DiLILLO,PANUZIO cov[Twin1ATETwin2ATE])thatwouldbefreelyestimatedint heappropriateI-SATmodel arenotincludedorestimatedinthecommonfatemodel(i.e., cov[ e1;e2 ],cov[ e3;e4 ],and cov[ e5;e6 ],respectively,arenotestimatedinFigure6).Thecommonf atemodeladdresses thisinterpersonaldyadicdependencebyestimatingthrees haredlatentvariablevariancesrather thanthreeinterpersonalcovariances,asdescribedearlie r.The3 df differencebetweenthetwo modelscomesfromtheremainingthreeinterpersonalcovari ances(i.e.,cov[Twin1SES,Twin2 ATE] D cov[Twin1ATE,Twin2SES];cov[Twin1SES,Twin2Growth] D cov[Twin2SES, Twin1Growth;cov[Twin1Growth,Twin2ATE] D cov[Twin2Growth,Twin1ATE])that (a)wouldbeexplicitlyestimatedandconstrainedtoequali ty(using3 df )intheI-SATmodel, (b)arenotexplicitlyincludedorestimatedinFigure6,but (c)areimplicitlyconstrainedto equality(using0 df )intheindistinguishabledyadcommonfatemodel-reproduc edcovariance matrixasadirectresultoftheequalityconstraints( e1 D e2 , e3 D e4 , e5 D e6 and C1 D C2 , C3 D C4 , C5 D C6 )alreadyincludedinthemodel. Asmentionedpreviously,ifdyadswereperfectlyindisting uishable,thechi-squaremodelt statisticwouldequalzero.Thenonzerochi-squarevaluesh owninTable3(13.69)indicates theSATSAtwinsarenotperfectlyindistinguishable.Asals omentionedpreviously,saturated modelsingeneralarenotparsimoniousandareseldomofinte resttoresearchers.Theindistinguishabledyadcommonfatemediationmodelisanexam pleofamodelthat,although essentiallysaturatedintheindistinguishabledyadsense discussedpreviously,canstillbeused totestasubstantiveresearchquestioninvolvingmediatio n.However,similartoamodelthatis saturatedinthetypicalSEMsense,thequestionoftforani ndistinguishabledyadcommon fatemediationmodelisamootpoint;alltindexeswillshow idealvaluesbydenition. Indistinguishablecommonfateexample. Toillustrate,theindistinguishabledyadcommonfatemediationmodelwasusedwiththeSATSAdatatotestw hetheradesireforpersonal growthmediatedtherelationshipbetweenSESandattitudes towardeducation.Asshown inFigure6,bothregressionpath( Â’ & Â“ )coefcientswerestatisticallysignicant,which suggestedthepossibilityofmediation.Thejointindirect pathestimate .Â’Â“ D :12/ showeda 95%condenceinterval(basedon5,000bias-correctedboot strapsamples)thatdidnotcontain zero[.05,.23].Adesireforpersonalgrowthpartiallymedi ated(asshowninFigure6,the Â£ 0 pathcoefcientwasalsostatisticallysignicant)therel ationshipbetweenSESandattitudes towardeducation. LONGITUDINALDYADICDATAANALYSES TheactorÂ–partnerinterdependenceandcommonfateanalysi smodelsshownintheprevious sectionscanmodeldyadicdataintrapersonalandinterpers onaldependenceandcanquantify dyadicrelationshipdynamicsfromdatasampledcross-sect ionally,buttheycannotassess dyadicresponsevariablechangesovertime.However,thesa meintrapersonalandinterpersonal dyadicdependencecanbemodeledlongitudinallytoquantif yseparate,butrelated,changes indyadmembers'responsevariablescoresovertime.Anexam pleofalatentgrowthcurve structuralequationmodel(cf.Meredith&Tisak,1990)that hasbeenmodiedtoaccommodate longitudinaldyadicdatawithcovariates(e.g.,DiLilloet al.,2009;Kashy,Donnellan,Burt,& McGue,2008)isshowninFigure7.RecallthattheNewlywedPr ojectinvolvedpsychological
ANALYZINGMIXED-DYADICDATAINSEM 329 FIGURE7 Distinguishabledyadlongitudinalanalysismodel:(a)int rapersonalintercept-slopecovariances (labeledA&B),(b)inter-personalgrowthtrajectorycovar iances(labeledCÂ–F),(c)interpersonalunique covariances(labeledGÂ–I),(d)uniquevariances(labeled e11;e21;:::e32 ),and(e)covariateeffects( Â” IW , Â” SW , Â” IH , Â” SH )allfreelyestimated. functioningandmaritalsatisfactiondatacollectedover3 consecutiveyears.Asshownin Figure7,theslopeloadings(0,1,2)forbothhusbandsandwi vesdenetheinterceptsasthe expectedmaritalsatisfactionscoreforhusbandsandwives attherstyearofdatacollection, andallowstheslopeestimatestobeinterpretedastheexpec tedlinearchangeinhusbands'and wives'maritalsatisfactionperyear.DistinguishableDyadsForbothdyadmembers,thedyadicgrowthcurvemodelshownin Figure7freelyestimates (a)interceptxedeffects(averageresponsevariablescor eatTime1),(b)slopexedeffects (averagerateofresponsevariablechangeovertime),(c)in terceptrandomeffects(individualresponsevariablevariationatTime1),(d)sloperandom effects(variationinindividual responsevariablechangeovertime),(e)intrapersonalint erceptÂ–slopecovariances(labeledA andB),(f)intrapersonaluniquevariances( e11;e21;:::;e32 ),(g)interpersonalinterceptÂ–slope covariances(labeledCÂ–F),and(h)interpersonaluniqueco variances(labeledGÂ–I).Inaddition toresearchquestionsinvolvingtheformofthemeanrespons evariablegrowthtrajectoryand thepresenceofsignicantvariationininterceptsandslop esforhusbandsandwives,the dyadiclatentgrowthcurvesmodelshowninFigure7canbeuse dtoanswerresearchquestions
330 PEUGH,DiLILLO,PANUZIO FIGURE8 Distinguishabledyadlongitudinalnullmodel:uniquevari ances( e s)freelyestimated,unique covariances,interceptscovariance,covariateregressio ns,andcovariatecovarianceallxedtozero. regardingwhetherdifferencesexistintheaveragegrowtht rajectoriesbetweenhusbandsand wivesinthesampleandtheeffectsofcovariatesonhusbands 'andwives'growthtrajectories. Specically,asshowninFigure7,theeffectofchildhoodps ychologicalmaltreatmenton maritalsatisfactioninterceptsandslopescanbeestimate dseparatelyforhusbands( Â” IH and Â” SH )andwives( Â” IW and Â” SW ). InthepreviousactorÂ–partnerinterdependenceandcommonf atemodelsfordistinguishable dyads,thestandardnullandsaturatedmodelsweretheappro priatemodelsforassessing thetofthosemodelstothedata.However,foralongitudina ldyadicgrowthmodel,the standardsaturatedmodelremainstheappropriatemodel,bu tthecorrectnullmodelisshown inFigure8.Thecorrectnullmodelforalatentgrowthcurvem odelisanintercept-onlymodel thatxesuniquecovariances,covariateregressions,thei nterceptscovariance,andthecovariates covariancetozero.NoticealsoinFigure8thattheuniqueva riances( e11;e21;:::;e32 )that werefreelyestimatedintheanalysismodelarealsofreelye stimatedinthenullmodel(Widaman &Thompson,2003). Distinguishablelongitudinalexample. Theconditionallongitudinaldyadicgrowthcurve modelshowninFigure7wasttotheNewlywedProjectdatatot estforpossibledifferences inmaritalsatisfactionchangesovertimebetweenhusbands andwives. 4 Asshownintheleft 4 Manyresearchersconsidertheestimationofanuncondition almodel(i.e.,omittingthepsychologicalmaltreatment covariatesfromFigure7)priortoaconditionalmodelaprud enttestingstep.Anunconditionalmodelwasttothe Newlyweddata;thatmodelshowedthefollowingtindexes( Â¦ 24 D 4:95 , p>:05 ;CFI&TLI D 1,RMSEA D .02). Theinterceptandslopevarianceestimatesforbothhusband s( Â§ IH D 7:69 , p<:01 ; Â§ SH D 2:35 , p<:01 )and wives( Â§ IW D 11:00 , p<:01 ; Â§ SH D 1:59 , p<:01 )werestatisticallysignicant.
ANALYZINGMIXED-DYADICDATAINSEM 331 TABLE4 LongitudinalDyadicGrowthModelFitStatistics DistinguishableDyadsIndistinguishableDyads Null Model Conditional Model Constrained Model Saturated Model Null Model Conditional Model Saturated Model Â¦ 2 300.7035.7635.770331.1972.4454.91 df 3012140392920 LogL 8,282.86 6,137.98 6,137.98 8,132.51 8,412.74 8,283.37 8,274.60 panelofTable4,estimatingtheconditionalanalysismodel resultedinthefollowingchi-square modeltindexvalue: Â¦ 212 35:76 , p<:05 (notshowninTable4:CFI D .98,TLI D .95, RMSEA D .07).ConsistentwiththepreviousAPIMexample,researche rscouldtestthetof additionallongitudinalmodelsthatconstrainparametere stimatesbetweenhusbandsandwives toequality.Inthisexample,amodelthatconstrainedinter cept(intercept husbands D intercept wives ) andslope(slope husbands D slope wives )xedeffectestimatesseparatelytoequalitywasestimate d. AsalsoshownintheleftpanelofTable4,estimatingthecons trainedanalysismodelresultedin thefollowingchi-squaremodeltindexvalue: Â¦ 214 D 35:77 , p<:05 (CFI D .98,TLI D .96, RMSEA D .06).Anestedmodeltestcomparingtheconditionalmodel .Â¦ 212 D 35:76 )tothe constrainedmodel .Â¦ 214 D 35:77/ showedastatisticallynonsignicantresult( 35:77 35:76 D :01 ; 14 12 D 2 ; ÂÂ¦ 22 D :01 , p>:05 ),indicatingthattheconditionalmodelshouldbe rejectedinfavorofthemoreparsimoniousconstrainedmode l.Additionalresultsshowedno signicantrelationshipbetweenwives'psychologicalmal treatmentandmaritalsatisfaction. However,husbands'psychologicalmaltreatmentwassigni cantlyrelatedtobothintercept ( Â” IH D :24 , p<:01 )andslope( Â” SH D :07 , p<:05 ).Forhusbands,increasedchildhood psychologicalmaltreatmentwasrelatedtosignicantlylo werbaselinemaritalsatisfactionand signicantlydecreasedmaritalsatisfactionovertime.IndistinguishableDyadsFromtheSATSAdata,twins'lifesatisfactionscoresassess edovertimeandtheirself-reported fearscoreswillbeusedtoillustratethelongitudinalgrow thmodelforindistinguishabledyads. TheanalysismodelusedforthisexampleisshowninFigure9. However,similartotheAPIM, alongitudinalgrowthmodelanalysiswithindistinguishab ledyadsalsobeginsbyspecifying theappropriatenullandsaturatedmodels.Identicaltothe logicusedtospecifythesaturated modelsfortheAPIM,theappropriatelongitudinalI-SATmod elisshowninFigure10.The indistinguishabledyadlongitudinalsaturatedmodelcont ainsinterpersonalequalityconstraints onmean(labeledAÂ–D),variance(labeledEÂ–H),andcovarian ceestimates(labeledOÂ–T),as wellasintrapersonalcovarianceequalityconstraints(la beledIÂ–N). 5 Further,similartothe 5 Forbothdistinguishableandindistinguishabledyadiclon gitudinalmodels,theissueofwhetheruniquevariances andcovariancesshouldbefreelyestimated(e.g.,distingu ishable)orconstrainedtoequality(e.g.,indistinguisha ble) betweendyadmembersshouldbetreatedasahypothesistobet estedratherthanviewedasadyadicmethodological requirement(e.g.,Kashyetal.,2008).
332 PEUGH,DiLILLO,PANUZIO FIGURE9 Indistinguishabledyadlongitudinalmodel:interceptand slopemeans(A&B)andvariances(C &D),uniquevariances(E),uniquecovariances(F),intrape rsonalintercept-slopecovariances(G),interpersonal growthtrajectorycovariances(H),covariateinuenceson intercept(I)andslope(J),andcovariatemeansand variances(K&L)eachconstrainedtoequality. longitudinaldyadicmodelfordistinguishabledyads,thea ppropriatenullmodelforadyadic growthcurvemodelforindistinguishabledyadsisalsoanin tercept-onlymodel,asshownin Figure11.NoticeinFigure11that,consistentwithindisti nguishabledyads,thelatentintercept variablemeans(labeledA)andvariances(labeledB),covar iatemeans(labeledC)andvariances (labeledD),anduniquevariances(labeledE)areeachconst rainedtoequality,whereasunique covariances(labeledFinFigure9),covariateregressionp aths,andthecovariancebetweenthe twocovariates,areallxedtozerointhenullmodel. Theindistinguishabledyadlongitudinalsaturatedandnul lmodelsagainprovideacontinuum withinwhichtoevaluatethetofalongitudinalanalysismo del.AsshowninFigure9,the indistinguishabledyadlongitudinalanalysismodeldiffe rsfromthelongitudinalanalysismodel fordistinguishabledyadsbytheparameterestimateconstr aintsneededduetoarbitrarydesignation.Specically,asshowninFigure9,theparameterest imatespreviouslyfreelyestimated fordistinguishabledyadsareconstrainedtoequalitybetw eenindistinguishabledyadmembers: (a)interceptmeans(labeledA),(b)slopemeans(labeledB) ,(c)interceptvariances(labeledC), (d)slopevariances(labeledD),(e)uniquevariances(labe ledE),(f)uniquecovariances(labeled F),(g)intrapersonalinterceptÂ–slopecovariances(label edG),(h)interpersonalinterceptÂ–slope covariances(labeledH),(i)covariateinuencesoninterc epts(labeledI),and(j)covariate inuencesonslopes(labeledJ).Also,recallthattheSATSA datawerecollectedfromtwin
ANALYZINGMIXED-DYADICDATAINSEM 333 FIGURE10 Indistinguishabledyadlongitudinalsaturatedmodel:int erpersonalmeans(labeledAÂ–D)and variances(labeledEÂ–H),intrapersonalcovariances(IÂ–N) ,andinterpersonalcovariances(OÂ–T)eachconstrained toequality.Thefourunlabeledcovariancesontheleftside ofthemodelarefreelyestimated(i.e.,notgivena letterlabel)tomodelinterpersonaldyadicdependence. pairsin1987,1990,and1993.Theslopeloadings(0,3,6)de netheinterceptastheexpected lifesatisfactionscoreforatwinpairin1987,andenableth eslopeestimatetobeinterpreted astheexpectedchangeinlifesatisfactionforevery1yeari ncreaseintime.Similartothe indistinguishabledyadactorÂ–partnerandcommonfateanal ysismodels,arbitrarydesignation isagainasourceofmistforanindistinguishabledyadlong itudinalSEManalysis.However, identicaltotheactorÂ–partnerandcommonfateanalysismod els,arbitrarydesignationmist canagainberemovedandcorrectedmodeltindexes(CFI,TLI ,andRMSEA)computedfor theindistinguishabledyadiclongitudinalanalysismodel (e.g.,seeKashyetal.,2008).
334 PEUGH,DiLILLO,PANUZIO FIGURE11 Indistinguishabledyadlongitudinalnullmodel:Intercep tmeans(A)andvariances(B),covariate means(C)andvariances(D),anduniquevariances(E)eachco nstrainedtoequality.Uniquecovariances(F), interceptscovariance,covariateregressions,andcovari atecovarianceallxedtozero. Longitudinalindistinguishableexample. Asanexampleanalysis,theconditionalanalysismodelshowninFigure9wasttotheSATSAdata. 6 AsshownintherightpanelofTable4, afterremovingarbitrarydesignationmist( 72:44 54:91 D 17:53 ; 29 20 D 9 ),thefollowing chi-squaremodeltindexvaluewasobservedfortheconditi onalmodel: Â¦ 29 D 17:53 , p<:05 (notshowninTable4:CFI D .97,TLI D .93,RMSEA D .05).Resultsfromtheconditional modelshowedthatself-reportedfearwassignicantlyrela tedtotwins'intercepts(labeledIin Figure13; Â” intercept D :96 , p<:01 )butnotslopes;higherself-reportedfearwassignicantl y relatedtolowerinitialstatuslifesatisfaction. ADDITIONALRESOURCES Thegoalsofthisarticleweretoillustratethestepsinvolv edinestimatingpopularcross-sectional (i.e.,APIMandcommonfate)andlongitudinal(i.e.,dyadic longitudinalgrowth)modelsused 6 Anunconditionalmodel(i.e.,omittingtheself-reportedf earcovariatesfromFigure9)wasalsottotheSATSA data.Afterremovingarbitrarydesignationmist( 49:43 42:85 D 6:58 ; 17 12 D 5 ; Â¦ 25 D 6:58 , p>:05 ),the unconditionalmodelshowedthefollowingtindexes:CFI D .99,TLI D .98,RMSEA D .03.Resultsalsoshowed twinshadanexpectedlifesatisfactionthatdifferedsigni cantlyfromzeroin1987(45.96, p<:01 ),butdecreased signicantlyperyear( .30, p<:01 ).Resultsfurthershowedtwins'interceptvariance .Â§ T D 42:76 , p<:01 )and slopevariance .Â§ S D :33 , p<:01 )estimateswerestatisticallysignicant.
ANALYZINGMIXED-DYADICDATAINSEM 335 toanalyzemixed-dyadicdatacollectedfromindistinguish ableordistinguishabledyads,and toclarifywhycertainadditionalstepsandmodicationsar eneededtoanalyzeindistinguishabledyaddatausingstructuralequationmodels.TheAMOSan dM plus statisticalanalysis softwarepackageswereusedinallanalysismodelspresente dinthisarticle;allexamplesare availableinanappendixonlineathttps://bmixythos.cchm c.org/xythoswfs/webui/_xy-476611_ 1-t_AXKarXYG.However,asecondarygoalofthisarticlewas toprovidesufcientdetailto allowresearcherstoanalyzethesemodelsintheanalysisso ftwarepackageoftheirchoice. Inadditiontothemodelsillustratedhere,severalauthors haveproposedadditionalcrosssectionalandlongitudinaldyadicdataanalysismodels.Fo rexample,Newsom(2002)showed howaCFAmodelcouldalsobeusedtotestforsignicantdiffe rencesamongdistinguishable dyadmembers.Also,LaurenceauandBolger(2005)showedhow diarymethodscanbeusedto quantifyrelationshipprocesschangesovertimeinmarital data.Further,althoughmostdyadic dataanalysismodelsassumearesponsevariablemeasuredon acontinuousscale,eachofthese modelscanbeusedtoanalyzecross-sectionalandlongitudi nalresponsevariablesmeasuredon acategoricalscale(e.g.,seeKennyetal.,2006).Inadditi on,manyoftheSEMmodelsshown hereandelsewherecanalsobeequivalentlyestimatedashie rarchicallinearmodels(e.g.,see Atkins,2005;Campbell&Kashy,2002;Gonzalez&Grifn,200 2;Kashy,Campbell,&Harris, 2006;Wendorf,2002;Whisman,Uebelacker,&Weinstock,200 4;Zhang&Willson,2006). Thedyadicmodelsdemonstratedinthisarticlehavealsobee ncombinedandexpandedon inseveralways.KennyandLedermann(2010)showedhowtheAP IMcanbemodiedsothat theindividual-levelactorandpartnereffectscanbeusedt oidentifydyad-levelrelationship patterns(e.g.,actoronly,partneronly,couple,andcontr astpatterns).Severalresearchershave alsoshownhowtheAPIMcanbeexpandedtoaddressdyadicrese archquestionsinvolvingmoderatedmediationandmediatedmoderationpossibilities(e .g.,Bodenmann,Ledermann,&Bradbury,2007;Campbell,Simpson,Kashy,&Fletcher,2001;Led ermann&Bodenmann,2006; Srivastava,McGonigal,Richards,Butler,&Gross,2006).I naddition,Matthews,Conger,and Wickrama(1996)showedhowthecommonfateandactorpartner interdependencemodelscan becombined,andmorethanonemediatingvariableadded,toa nswercomplexdyadicresearch questionsinvolvinghowmediateddynamicsattheindividua llevelcanimpacttherelationship atthedyadlevel.Needlesstosay,appliedresearchersseek ingtoanswerresearchquestions involvingcross-sectionalorlongitudinalmixeddyadicda tacollectedfromdyadsconsideredto bedistinguishableorindistinguishablecanchoosefromam ongseveralSEManalysisoptions. ACKNOWLEDGMENTS WewishtoextendourappreciationandgratitudetoCraigK.E nders,JosephA.Olsen,Pamela Sadler,ErikWoody,the Teacher'sCorner Editor,andseveralanonymousreviewersfortheir insightfulcommentsandhelpfulsuggestionsthatgreatlyi mprovedthequalityofthisarticle. REFERENCES Atkins,D.C.(2005).Usingmultilevelmodelstoanalyzecou pleandfamilytreatmentdata:Basicandadvancedissues. JournalofFamilyPsychology,19, 98Â–110.
336 PEUGH,DiLILLO,PANUZIO Bodenmann,G.,Ledermann,T.,&Bradbury,T.N.(2007).Stre ss,sex,andsatisfactioninmarriage. PersonalRelationships,14, 551Â–569. Bollen,K.A.(1989). Structuralequationswithlatentvariables. NewYork,NY:Wiley. Campbell,L.,&Kashy,D.A.(2002).Estimatingactor,partn er,andinteractioneffectsfordyadicdatausingPROC MIXEDandHLM:Auser-friendlyguide. PersonalRelationships,9, 327Â–342. Campbell,L.,Simpson,J.A.,Kashy,D.A.,&Fletcher,G.J.O .(2001).Idealstandards,theself,andexibilityof idealsincloserelationships. PersonalityandSocialPsychologyBulletin,27, 447Â–462. Carey,G.(2005).Theintraclasscovariancematrix. BehaviorGenetics,35, 667Â–670. Cook,W.L.,&Kenny,D.A.(2005).TheactorÂ–partnerinterde pendencemodel:Amodelofbidirectionaleffectsin developmentalstudies. InternationalJournalofBehavioralDevelopment,29, 101Â–109. DiLillo,D.,Peugh,J.,Walsh,K.,Panuzio,J.,Trask,E.,&E vans,S.(2009).Childmaltreatmenthistoryamong newlywedcouples:Alongitudinalstudyofmaritaloutcomes andmediatingpathways. JournalofConsultingand ClinicalPsychology,77, 680Â–692. Efron,B.,&Tibshirani,R.J.(1993). Anintroductiontothebootstrap. NewYork,NY:Chapman&Hall/CRC. Fritz,M.S.,&MacKinnon,D.P.(2007).Requiredsamplesize todetectthemediatedeffect. PsychologicalScience, 18, 233Â–239. Furman,W.,&Simon,V.A.(2006).Actorandpartnereffectso fadolescents'romanticworkingmodelsandstylesof interactionswithromanticpartners. ChildDevelopment,77, 588Â–604. Gonzalez,R.,&Grifn,D.(1999).Thecorrelationalanalys isofdyad-leveldatainthedistinguishablecase. Personal Relationships,6, 449Â–469. Gonzalez,R.,&Grifn,D.(2001).Testingparametersinstr ucturalequationmodeling:EveryÂ“oneÂ”matters. PsychologicalMethods,6, 258Â–269. Gonzalez,R.,&Grifn,D.(2002).Modelingthepersonality ofdyadsandgroups. JournalofPersonality,70, 901Â–924. Grifn,D.,&Gonzalez,R.(1995).Correlationalanalysiso fdyad-leveldataintheexchangeablecase. Psychological Bulletin,118, 430Â–439. Kashy,D.A.,Campbell,L.,&Harris,D.W.(2006).Advancesi ndataanalyticapproachesforrelationshipsresearch: Thebroadutilityofhierarchicallinearmodeling.InA.Van gelisti&D.Perlman(Eds.), Cambridgehandbookof personalrelationships (pp.73Â–89).NewYork,NY:CambridgeUniversityPress. Kashy,D.A.,Donnellan,M.B.,Burt,S.A.,&McGue,M.(2008) .Growthcurvemodelsforindistinguishable dyadsusingmultilevelmodelingandstructuralequationmo deling:Thecaseofadolescenttwins'conictwiththeir mothers. DevelopmentalPsychology,44, 316Â–329. Kenny,D.A.(1996).Modelsofnon-independenceindyadicre search. JournalofSocialandPersonalRelationships, 13, 279Â–294. Kenny,D.A.,&Cook,W.(1999).Partnereffectsinrelations hipresearch:Conceptualissues,analyticdifculties,an d illustrations. PersonalRelationships,6, 433Â–448. Kenny,D.A.,&Judd,C.M.(1996).Ageneralprocedureforthe estimationofinterdependence. PsychologicalBulletin, 119, 138Â–148. Kenny,D.A.,Kashy,D.A.,&Bolger,N.(1998).Dataanalysis insocialpsychology.InD.T.Gilbert,S.T.Fiske,& G.Lindzey(Eds.), Handbookofsocialpsychology (4thed.,Vol.1,pp.233Â–265).Boston,MA:McGraw-Hill. Kenny,D.A.,Kashy,D.A.,&Cook,W.L.(2006). Dyadicdataanalysis. NewYork,NY:Guilford. Kenny,D.A.,&LaVoie,L.(1985).Separatingindividualand groupeffects. JournalofPersonalityandSocial Psychology,48, 339Â–348. Kenny,D.A.,&Ledermann,T.(2010).Detecting,measuring, andtestingdyadicpatternsintheactorÂ–partnerinterdependencemodel. JournalofFamilyPsychology,24, 359Â–366. Laurenceau,J.P.,&Bolger,N.(2005).Usingdiarymethodst ostudymaritalandfamilyprocesses. JournalofFamily Psychology,19, 86Â–97. Ledermann,T.,&Bodenmann,G.(2006).Moderator-undmedia tor-effektebeidyadischendaten:Zweierweiterungen desakteurpartner-interdependenz-modells[Moderatoran dmediatoreffectsindyadicresearch:Twoextensionsof theactorÂ–partnerinterdependencemodel]. ZeitschriftfÃ¼rSozialpsychologie,37, 27Â–40. Ledermann,T.,&Macho,S.(2009).Mediationindyadicdataa tthelevelofthedyads:Astructuralequationmodeling approach. JournalofFamilyPsychology,23, 661Â–670. MacKinnon,D.P.(2008). Introductiontostatisticalmediationanalysis. NewYork,NY:Erlbaum. MacKinnon,D.P.,Lockwood,C.M.,Hoffman,J.M.,West,S.G. ,&Sheets,V.(2002).Acomparisonofmethodsto testmediationandotherinterveningvariableeffects. PsychologicalMethods,7, 83Â–104.
ANALYZINGMIXED-DYADICDATAINSEM 337 MacKinnon,D.P.,Lockwood,C.M.,&Williams,J.(2004).Con dencelimitsfortheindirecteffect:Distributionof theproductandresamplingmethods. MultivariateBehavioralResearch,39, 99Â–128. Matthews,L.S.,Conger,R.D.,&Wickrama,K.A.S.(1996).Wo rkÂ–familyconictandmaritalquality:Mediating processes. SocialPsychologyQuarterly,59, 62Â–79. McGraw,K.O.,&Wong,S.P.(1996).Forminginferencesabout someintraclasscorrelationcoefcients. Psychological Methods,1, 30Â–46. Meredith,W.,&Tisak,J.(1990).Latentcurveanalysis. Psychometrika,55, 107Â–122. MuthÃ©n,B.O.,&MuthÃ©n,L.K.(1998Â–2010). M plus user'sguide (6thed.).LosAngeles,CA:MuthÃ©n&MuthÃ©n. Newsom,J.T.(2002).Amultilevelstructuralequationmode lfordyadicdata. StructuralEquationModeling,9, 431Â–447. Olsen,J.A.,&Kenny,D.A.(2006).Structuralequationmode lingwithinterchangeabledyads. PsychologicalMethods, 11, 127Â–141. Pedersen,N.L.(2004).SwedishAdoption/TwinStudyonAgin g(SATSA),1984,1987,1990,and1993[Computerle]. ICPSRversion.Stockholm,Sweden:KarolinskaInstitutet[ producer].AnnArbor,MI:Inter-UniversityConsortium forPoliticalandSocialResearch[distributor]. Srivastava,S.,McGonigal,K.M.,Richards,J.M.,Butler,E .A.,&Gross,J.J.(2006).Optimismincloserelationships: Howseeingthingsinapositivelightmakesthemso. JournalofPersonalityandSocialPsychology,91, 143Â–153. Wendorf,C.A.(2002).Comparisonsofstructuralequationm odelingandhierarchicallinearmodelingapproachesto couples'data. StructuralEquationModeling,9, 126Â–140. Whisman,M.A.,Uebelacker,L.A.,&Weinstock,L.M.(2004). Psychopathologyandmaritalsatisfaction:The importanceofevaluatingbothpartners. JournalofConsultingandClinicalPsychology,72, 830Â–838. Widaman,K.F.,&Thompson,J.S.(2003).Onspecifyingthenu llmodelforincrementaltindicesinstructural equationmodeling. PsychologicalMethods,8, 16Â–37. Woody,E.,&Sadler,P.(2005).Structuralequationmodelsf orinterchangeabledyads:Beingthesamemakesadifference. PsychologicalMethods,10, 139Â–158. Zhang,D.,&Willson,V.L.(2006).Comparingempiricalpowe rofmultilevelstructuralequationmodelsand hierarchicallinearmodels:Understandingcross-levelin teractions. StructuralEquationModeling,13, 615Â–630.
TESTING THE EFFECTIVENESS OF THE MUML ESTIMATION METHOD FOR DYADIC DATA IN MULTILEVEL STRUCTURAL EQUATION MODELS By YU SU A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS IN EDUCATION UNIVERSITY OF FLORIDA 2014
Â© 2014 Yu Su
To my parents for all of their love and support
4 ACKNOWLEDGMENTS I would like to express sincere gratitude to my committee chair, Dr. Walter Leite and co chair, Dr. Leite initially fosters my interest in Structural Equation Modeling . He has provided invaluable guidance and supervision throughout all phases of my thesis study, show ed me in person how sedulous a promising scholar can be in academia, without his support, this project would not have been possible. Also, I am indebted to Dr. Mille r, not only for his encouragement and feedback throughout the course of this research, but also for his effectively guidance that introduce d me the beauty of Item Response Theory. I do admire his insight and passion in quantitative research. Very special thanks to Dr. Linda Behar for her in dental grant. She gives me the opportunities to master new research skills and apply what I have learned, and always reminds me the importance of linking methodology to the real w orld research. Most important of all, her unconditional love will never be forgotten. Finally, my sincere thanks g o to my family and my friends .
5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 7 LIST OF FIGURES ................................ ................................ ................................ .......... 8 ABSTRACT ................................ ................................ ................................ ..................... 9 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 11 2 LITERATURE REVIEW ................................ ................................ .......................... 16 Multilevel Structural Equation Modeling ................................ ................................ .. 16 .................. 19 Full Information Maximum Likelih ood Estimation for MSEM ................................ ... 21 Factors Affecting Accuracy of Estimates ................................ ................................ . 23 Intraclass Coefficient and Design Effect ................................ ........................... 23 Sample and Cluster Size Issue ................................ ................................ ........ 24 The Performance of Multi level Modeling with Dyadic Data ................................ .... 28 Current Study to Evaluate the Performance of MSEM with Dyadic Data ................ 30 3 METHOD ................................ ................................ ................................ ................ 33 Data Generation and Analysis Models ................................ ................................ ... 34 Model 1. Equal Between Level Model and Within Level Model ........................ 34 Model 2. Simple Between Level Model and Complex Within Level Model ....... 35 Simulation conditions ................................ ................................ .............................. 35 Sample Size ................................ ................................ ................................ ..... 35 Intra Class Correlation (ICC) ................................ ................................ ............ 36 Dependent Variables and Analysis ................................ ................................ ......... 37 4 RESULTS ................................ ................................ ................................ ............... 45 Non Convergent Problems and Improper Solutions ................................ ............... 45 Parameter Bias ................................ ................................ ................................ ....... 46 Standard Error Bias ................................ ................................ ................................ 48 Confidence Interval Coverage Rates ................................ ................................ ...... 50 Chi Square Statistics ................................ ................................ .............................. 51 5 SUMM ARY AND DISCUSSION ................................ ................................ .............. 61
6 LIST OF REFERENCES ................................ ................................ ............................... 68 BIOGRAPHICAL SKETCH ................................ ................................ ............................ 75
7 LIST OF TABLES Table page 3 1 Population Parameter Values for Data Generation ................................ ............. 40 4 1 Percentage of Non convergent Solutions for Model 1 and Model 2. ................... 52 4 2 Percentage of Non convergent Solutions for Model 1 and Model 2 .................... 52 4 3 Relative Bias in Parameter Estimates for Within Mode ls ................................ .... 53 4 4 Relative Bias in Parameter Estimates for Between Model s ................................ 54 4 5 Relative Bias in Standard Errors for Within Model s ................................ ............ 55 4 6 Relative Bias in Standard Errors for Between Model s ................................ ........ 56 4 7 Coverage of 95% Confidence Interval for Model 1 ................................ ............. 57 4 8 Coverage of 95% Confidence Interval for Model 2 ................................ ............. 57 4 9 Relative bias in chi square test and the percentage of rejected rate .................. 58
8 LIST OF FIGURES Figure page 3 1 Population Model Used for Data Generation in Scenario 1 ................................ 41 3 2 Analysis Model in Scenario 1 ................................ ................................ .............. 42 3 3 Population Model Used for Data Generation in Scenario 2 ................................ 43 3 4 A nalysis Model in Scenario 2 ................................ ................................ .............. 44 4 1 Percentage of Non convergent Solutions for Model 1 ................................ ........ 59 4 2 Percentage of Improper Solutions for Model 1 ................................ ................... 59 4 3 Percentage of Non convergent Solutions for Model 2 ................................ ........ 60 4 4 Percentage of Im proper Solutions for Model 2 ................................ ................... 60
9 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for th e Degree of Master of Arts in Education TESTING THE EFFECTIVENESS OF THE MUML ESTIMATION METHOD FOR DYADIC DATA IN MULTILEVEL STRUCTURAL EQUATION MODELS By Yu Su August 2014 Chair: Walter Leite Cochair: David Miller Major: Resear ch and Evaluation Methodology Dyads, as one special case of hierarchical structured data, are common in social and educational research (e.g., married couples, twins, student partners). When multilevel structural equation modeling (MSEM) become a widespread analytic technique for analy sis of nested or hierarchical structured data. However, practices associated with the employing of MSEM with extremely small group sizes (e.g., dyadic data) are problematic due to the lack of empirical research. At the same time, although pseudobalance app roach (MUML) described by MuthÃ©n (1989, 1990) has been increasingly used to perform MSEM, little is known about how the complexity of study, a Monte Carlo simulation was emp loyed to investigate the performance of MCFA with dyadic data and to examine the accuracy of the MUML estimator, in terms of the parameter estimates, their standard errors and the global chi square model fit. Multilevel sample data were generated in terms of 3 design factors: (a) intraclass correlation, (b) number of dyads and (c) model specification.
10 The results indicated the application of Multilevel CFA with dyadic data is a potential dangerous practice especially when researchers are concerned the varia bility across dyads. It should be noted first that all the design conditions (number of dyads, ICC values and model specification) have clearly effects on the performance of factor models. Besides, these effects are much pronounced in the between group par t. Overall, the within group part of the models presented little to negligible bias with minor difference across all cells of conditions. Although the standard errors for factor loadings exhibited negative bias greater than 10%, these observations were onl y limited to the lower sample size conditions. However, for the between group part of the models, there were serious problems with convergence difficulties, underestimated chi square statistics, parameter bias and attenuated standard errors, particularly a t small sample size and lower ICC. Therefore, to make a valid inference about parameter estimates and their standard errors, at least 200 dyads are required with higher ICC conditions(>0.2). Given our findings, researchers are suggested to be cautious the use of MCFA with dyads, especially when the between group part is the primary focus of investigation.
11 CHAPTER 1 INTRODUCTION Nested (or structured) data are common in social and behavior sciences. Nested data arise when the observed scores collected from individuals interact in some way to produce correlated groups. For example, students are nested within classrooms, patients are nested within hospitals, children are nested within their families and repeated measure s are nested within individuals. Dyads, as one special case of hierarchical structured data, represent data nested within groups of 2 (e.g., married couples, twins, student partners). Statistically, nested dyadic data imply that units interacting within gr oups (dyads) are likely to be more similar than those from different groups (e.g., Gaudreau , Fecteau & Perreault , 2010; Gonzalez & Griffin, 1999; Kenny, 1996). When individuals are nested, the fundamental independence assumption underlying most conventional inferential methods (e.g., ANOVA, OLS regression) are violated, leading to underestimated standard errors, increased Type I error rate and the reduction of p ower in statistical significance tests ( Moerbeek, 2004; Van Landeghem , De Fraine & Van Damme , 2005). To better address these issues, there are currently two approaches for analyzing the effects of clustering on individual performance. The first one is mult ilevel regression ro & Bates, 2000). A second approach is multilevel structural equation modeling which integrates multilevel modeling within a latent variable framework (Heck & Thomas, 2008; Hox,
12 2002; Kaplan, 2008; Mehta & Neale, 2005; MuthÃ©n, 1994). A detailed introducti on to these two approaches will be given next. Multilevel regression models concern the existence of nested data by allowing for residual components at each level in the hierarchy. For example, a two level model which allows for grouping of child outcomes within families would include residuals at the child and family level. Therefore the residual variance is partitioned into a between family component (the variance of the family lev el residuals) and a within family component (the variance of the child level residuals). An advantage of using MRM versus a more traditional approach is the segregation of grouping effects. In the fixed effects model (i.e., Analysis of variance, OLS regres sion), the effects of the cluster level predictors were confounded with the influences of group variables, unless dummy cluster indicators are included in the model or cluster mean centering is performed (Allison, 2009). However, in a multilevel model, eff ects of both types of variables can be estimated. Clusters are treated as random samples to make efficient and valid multilevel regression modeling provides rationale for groupi ng effects and appropriately partitions predictor variance into its within dyad part (e.g., wife) and between dyad part(e.g., couple) (Heck& Thomas, 2008; Raudenbush & Bryk ,2002).Over the past ten years, MRM have been increasingly applied to dyadic area (e.g., Cook, 1994; Barnett , Marshall & Raudenbush , 1993; Kurdek, 2003 ; Raudenbush , Brennan & Barnett , 1995; Atkins, 2005; Newsom, 2002 ; McMahon , Pouget & Tortu , 2006 ; Kenny & Cook,2006). For example, to provide a practical guide, Campbell and Kashy (2002) used two statistical software programs SAS PROC MIXED and HLM performing analysis of
13 dyadic data with continuous outcomes. McMahon and his colleagues (2006) extended nonlinear multilevel modeling of dyadic data with binary outcomes using NLMIXED in SAS. In a simulation study, Newsom & Nishishiba (2002) provided useful information about statistical properties of MRM with dyadic data. The results showed that the inclusion of random slope effects will lead to serious con vergence problems. As many have noted, it is impractical to estimate both the random slope and random intercept effects simultaneously, more specifically, in the presence of dyadic data, MRM models with both random effects result in too many parameters tha t cannot be estimated given the limited number of covariance elements within each level 2 units, thus leading to problems in model identification. As is the case with any statistical procedures, MRM has its intrinsic limitations: (a) MRM does not allow level 2 outcome variables to be predicted with level 1 variables. (b) MRM assumes that all the variables of interest are perfectly reliable (or no mea surement errors are included), yet this assumption is rarely met in practice which can result in biased parameter estimates and attenuated standard errors. (c) MRM does not provide measures of a global model fit. Interestingly, these conventional disadvant ages of MRM can be perfectly offset by the structural equation modeling (SEM) framework (Bauer , 2003; Kline, 2011). More precisely, SEM is a latent variable technique that not only takes measurement error and model fit into account, but also allows more ac curate estimates of the structural relationship between observed/ latent variables (Heck & Thomas, 2008). However, the limitation of SEM lies in the assumption (MuthÃ©n, 1994).
14 Clear ly, since each of these two approaches has unique attributes, it is not surprising that methodologists are interested in fostering their synergy. The result is multilevel structural equation modeling (MSEM), which synthesizes the advantages of MRM and SEM under the general latent analytical framework (e.g., Mehta & Neale, 2005; MuthÃ©n & Asparouhov, 2011 ). MSEM is compared with multilevel regression modeling (MRM) by Zhang and Willson (2006). better capacity for the estima ting unobserved latent variables, whereby the relationships for measurement errors are corrected and error homogeneity is tested. Secondly, MSEM better models a variety of complex theoretical models of interdependence such as the actor partner dyadic model and the mutual influence model (Kenny, 1996; Griffin& Gonzalez, 1995,2012) . Thirdly, several features of SEM such as global model fit test, multi group analysis and nested model comparison are also available in MSEM (Newsom, 2002; Tomarken & Waller, 2005) . Although considerable advances have been made in this respect, MSEM is still a promising area for methodological inquiry. Recent computational advances such as Mplus (MuthÃ©n & MuthÃ©n , 2012), LISREL ( JÃ¶reskog & SÃ¶rbom, 1996) , and GLLAMM (Ra be Hesketh , Skrondal & Pickles , 200 4 ) made MSEM accessible to more general formulations and a few sources suggest it as an approach for dya d analysis (e.g., Newsom, 2002; Peugh , DiLillo & Panuzio , 2013 ). When methodologists consider the utility and generalizability of MSEM, however, little is known about the behavior of estimates, their standard errors and chi square significant test when data is nested within extremely small groups. There is also a paucity of research examining dyad within MSEM framework. Therefore , it is unclear
15 how the complexity of dyadic data affects the performance of estimation methods. The present study examines these issues in a Monte Carlo simulation study of MSEM with dyadic data under varying level of ICC and number of dyads.
16 CHAP TER 2 LITERATURE REVIEW Multilevel Structural Equation Modeling The general concern of multilevel structural equation modeling is testing the hypothesized interrelationships between latent variables and/or their indicators in the presence of hierarchical structure data . Interestingly, some statistical inferences and research questions can only be validly addressed within an MSEM framework, For example, dyadic researchers may have int erests in an important question work performance (individual level latent t rait) depend on marital quality ( d yad level latent trait)? As is introduced by MuthÃ©n & Asparouhov (2009), MSEM has its root in single level SEM which falls primarily into two categories: (a) confirmatory factor analysis (CFA) that focuses on relations among latent variables and measured indicators; and (b) path analysis (PA) that examines the relation s (e.g., direct, indirect effects) among a set of observed items. Likewise, the CFA model and the path model can be synthesized into a full MSEM model with both measurement and structural components. The general formulation of single level CFA is expressed as follows: (2 1) W here refers to an individual unit, is an intercept vector, is the regression coefficients or factor loading matrix, is the unobserved latent variables and is the measurement residual vector (MuthÃ©n ,1994). Thus, the observed variables can be expressed as a function of latent variables, observed covariates and measurement residuals. y i i i
17 When SEM is used to analyze the models for means and covariance/correlation matrix of observed variables, the equation can be expressed as follows: (2 2) (2 3) Where Equation (2) refers to the model implied covariance matrix which i s expressed as a function of factor loading matrix , factor covariance matrix and measurement errors in . And refers to means vectors also expressed as a function of parameters in . In SEM, the parameters are estimated so that the discrepancy between the observed sample covariance matrix and the model implied covariance matrix is minimized (Bollen, 1989). Now, consider for simplicity a two level single factor model where data are gathered on a number of N individuals ( ) nested within G groups (g). Then the MSEM formulation can be illustrated as follows: (2 4 ) where the observed score of individual ( expressed as a function of a measurement intercept vector ( ), a factor loading matrix ( ), underlying factor score ( ) and measurement residual vector ( ) (MuthÃ©n,1994; MuthÃ©n & Satorr a, 1989) which are assumed to be normally distributed with a mean of zero and variance covariance matrix. It is worth noting that in MSEM, the parameter matrix are allowed to vary across groups and the factor means should be specified as a random effect at the between group level. As such, in Equation (4) can be further expressed as follows: (2 5)
18 where refers to the intercepts vector( or the overall expected value for ), refers to a random factor that represents variation across groups and is the individual residual terms that assume to be normally distributed with a mean of zero and variance matrix. It follows then a total factor variance and the residual variation of can be both decomposed into a between group p art and a within group part: In the same manner, a multivariate population covariance matrix can be decomposed into the sum of a between group component and a with group component (MuthÃ©n, 1994; MuthÃ©n & Asparouhov, 2009; Hox, 2002) . That is, (2 8) Finally, the sample covariance matrix is produced (MuthÃ©n, 1994; MuthÃ©n & Asparouhov, 2009; Heck & Thomas, 2008) as follows: (2 9) In MSEM, the hierarchical data are assumed to be estimated by the between group models and within group models simultaneously, however, what is at issue here is that is not an unbiased ML estimator for and the same is true for . Thus it is V ( i g ) B w V ( i g ) B w T B W (2 6) ( 2 7)
19 complicated to test a within groups model for and a between groups model for (MuthÃ©n, 1994; MuthÃ©n & Satorra, 1989). To address this issue, several analytic approaches for two level structure are developed (e.g., MuthÃ©n, 1994; McDonald, 1994; Gold stein, 1995, 2011; Rabe Hesketh et al., 2004; Mehta & Neale, 2005). In the next par t, we will first give the detailed shows easier calculation and faster convergence than ML procedure. Limited Information Maximum Likelihood Approach ( MUML ) In M balanced approach (1989, 1990) the pooled within group covariance matrix is given as the unbiased sample estimate of . The equation shows as follows: (2 10) where N and G refer to the total sample size and number of groups (or sample size at cluster level ) respectively, is the observed score for individual in cluster , and is the group mean of for cluster . Similarly, the sample estimate of between group covariance matrix , denoted as , is calculated as follows: (2 11) where refers to the grand mean of (or the overall sample means) and are defined as before. S P W 1 N G ( y i j i 1 n j j 1 G y j ) ( y i j y j ) '
20 MuthÃ©n indicated (1990) that, in the context of perfectly balanced data in which (e.g., for dyadic data, for all cluster ), (2 12) (2 13) where is the unbiased ML estimator of and is the function of the combination of and the group size weighted , in which the scale factor equals to the common group size . However, in the context of unbalanced data (values of vary across clusters). becomes an estimator for the subset of groups with unique group size d, (2 14) where the scale parameter is distinct for each subset . MuthÃ©n (1990) then further suggested calculating including an ad hoc estimator for the scaling factor, . (2 15) where is almost equal to the average sample size within clusters. By analyzing and simultaneously, we can assess the model fit and parameter estimates. MuthÃ©n (1990) defined the result of this equation as a limited information maximum likelihood solution (MUML), which also refers to a pseudobalanced solution (McDonald, 1994). This approach is currently only available for multilevel models in GLLAMM (Rabe Hesketh et al., 2004) and for SEM in M plus (MuthÃ©n & MuthÃ©n, 201 2 ), S P W W S * B c B W S B d * W c d B
21 generally combined with default robust estimator for parameter estimates and the chi square model test for the heterogeneity correction (See MuthÃ©n & Sat orra, 1995). In the case of balanced data, MUML estimator is claimed to be unbiased (MuthÃ©n, 1989) and produced similar results as full information maximum likelihood (FIML) estimator (Hox & Maas, 2001; McDonald, 1994; MuthÃ©n & Satorra, 1995). Full Inform ation Maximum Likelihood Estimation for MSEM When Hartley and Rao (1967) first provided a description of using maximum likelihood e stimators for multilevel models , Dempster , Laird and Rubin (1977) then developed the expectation maximization (EM) algorith m, based on which, FIML makes it accessible to analyze the hierarchical structural data involving both continuous and discrete outcomes (Raudenbush & Bryk, 2002). For the ML estimator, a discrepancy function is applied to examine the distance between the observed and model implied covariance matrices and vector of means, expressed as follows ( JÃ¶reskog , 1967): (2 16) Where indicates the determinant of a matrix, denotes the trace of a matrix and is the number of observed variables. When , then reflecting the model is correctly matched, if , indicating the model is mis specified. Given the sufficiently large sample size, the correct model specification and multivariate normality distribution, the statistics: (2 17) is distributed as chi square distribution with a degree of freedom, which is
22 (2 18) W here refers to the number of observed indicators and r efers to the number of model parameters. For the two level covariance matrix, the ML fitting function considers individuals nested within groups and the equation under normality can be expressed as follows (MuthÃ©n & Satorra, 1995): (2 19) As a result, the incorporation of multilevel analysis within a general SEM framework allows for both intercepts and slopes to vary across groups (Mehta & Neale, 2005). As mentioned before, evaluation of model fit is one of the unique strengths of the MSEM framework. In particular, the chi square significant test is the most widely used for testing the null hypothesis, which assumes that the observed and reproduced sample means and variance/covariance are perfectly matched given the degree of freedom of the specific model and the value. However, in practice, the elements of the population covariance matrix are unknown but can be estimated by using the sample covariance matrix , so the null hypothesis is then expressed as follows: , (2 20) If the null hypothesis fails to be rejected, we can conclude that the hypothesis model results in the perfect reproduc tion of the population covariance matrix and mean vectors.
23 Factors Affecting Accuracy of Estimates Intraclass Coefficient and Design Effect Hox and Maas (2001) pointed out that the magnitude of intraclass correlation coefficient (ICC) has positive effect s on the accuracy of MUML produced parameter estimates. ICC, which refers to the extent of systematic variance that is due to grouping effects (Raudenbush & Bryk, 2002), can be modeled as follows: (2 21) Where and refer to the between group variance and within group variance, respectively. ICC is the magnitude of the correlation of observations within a cluster. To dyads, ICC quantifies the magnitude of dyadic interdependence. The values of ICC range from 0 to 1, when the value of ICC closes to zero, suggesting that the grouping effects are trivial. Thus, MSEM can be simplified into single level SEM. On the other hand, a higher value of ICC indicates a greater proportion of between group level variance . In other words, individuals within dyads are more homogeneous than those in different dyads, thus leading to greater bias if multilevel modeling techniques are not taken into account (e.g., Gonzalez & Griffin, 2002; Heck & Thomas, 2008). Snijders and Bos ker (1999) indicated that the ICC value should be greater than 0.05 to construct the multilevel analysis. Rather than focus on ICC, some researchers considerd the size of the design effect as the issue in multilevel modeling ( Maas& Hox, 2005; MuthÃ©n & Sato rra , 1995). Shackman (2001) identifies that DEFF is an adjustment that should be used to determine sample size. In cluster sampling, design effect can be expressed as follows (Kish, 1965):
24 DEFF= 1+ (average cluster size 1)*ICC (2 22) As a function of ICC, DEFF measures the magnitude of underestimation in sampling variance, in other words, it measures the extent to which the sample size needs to be adjusted due to clustering effects. Only when t he average cluster size equals to 1 and / or ICC equals to zero, DEFF equals to 1, implying no variance exists in the between group level. Many researchers undertake multilevel modeling when DEFF > 2(see MuthÃ©n & Satorra, 1995, for a report of the Monte C arlo study). If Design Effect 2, analyzing multilevel structured data with conventional SEM does not result in overly misleading results. Note that, MuthÃ©n (2004) further pointed out that the rule of thumb serves as a guideline wh en researchers concern the bias of standard errors and chi square when ignoring the nested structure of the data. However, if multilevel structure itself is of interest, in that case, multilevel modeling can still be done even when there are smaller design effects (DEFF <2), as it clearly is with dyadic data. In general, both analytic work and simulation research demonstrated that the significant test without considering multilevel structure of data will lead to overly misleading ikowski, 1981; Tate & Wongbundhit, 1983; Hox, 2002; Julian, 2001). Sample and Cluster Size I ssue In the multilevel structural equation modeling, the asymptotic maximum likelihood estimator is used most often under the assumption that the large sample siz e goes to infinity, the violation of which will lead to biased parameter estimates and incorrect standard errors. While most social and educational research is based on varying sample sizes at more than one level, especially given that the sample size of t he individual level is by nature larger than that of a group, the investigation of the minimum
25 sample size required for individual level and group level is usually of concern. Eliason (1993) recommends a minimum sample size of 60 for ML estimator, but this would apply to the group level for multilevel modeling, Simulations designed to assess the sample size issue in multilevel structural equation modeling are beginning to appear i n literature (Hox, 1995; Hox & Maas, 2001 ; Hox , Maas & Brinkhuis , 2010; Meuleman & Billiet, 2009; Hox , V an de Schoot & Matthijsse , 2012). Of particular note was Hox and Maas (2001) who investigated the robustness of MUML in estimating a multilevel CFA under varying levels of ICC (0.2 0.33), number of groups (50 100 200), clust er size (10 20 50) and group imbalance. Results showed that the within group part pose no problems across all conditions. However, several problems were found in the between group part when the number of groups is small (50). The problems include inadmiss ible or under estimated parameter estimates, inaccurate standard errors and biased residual variance. Particularly, ICC had the largest effects on parameter estimates and their standard errors, but the effects of cluster size were negligible. Besides, the chi square model test was accurate in the balanced case but shows small positive biases with higher ICC and imbalanced groups. As a result, the authors suggested the minimum number of groups to be 100 for good MUML performance and at least 200 groups are r equired for a valid inference of residual variance. In a later study, Hox and his colleagues (2010) compared the performance of MUML, FIML, and DWLS estimators under similar conditions. Results consistently showed the robustness of the within group part re garding the parameter estimate and their standard errors of factor loadings. The authors also pointed out that ICC has no effects on any of the conditions. But this contradicted to the previous results
26 of Hox and Maas (2001) who indicated that ICC has the greatest effects on convergence problems. This was because the effect of ICC was confounded with the amount of systematic variance in the between group model (Hox et al., 2010). In addition, it was found that the most important factors that influence mode fit test are level 2 sample size and estimation methods, particularly MUML leaded to the most chi square value bias. Interestingly, the bias increased as the average level 1 size increased. Nevertheless, based on analytical study about statistical properti es of MUML, Yuan and Hayashi (2005) discussed the role of level 1 and level 2 sample sizes on the standard errors and test statistics of the MUML procedure. The authors showed a conclusion opposite to Hox and his colleagues (2010), indicating that smaller cluster size leads to a higher coefficient of variation. In addition, only when the group numbers goes to infinity, do MUML produced standard errors and the chi square test lead to valid inference. Recently, Meuleman and Billiet (2009) conducted a simulati on study concerning the required group numbers for accurate multilevel SEM estimation in cross national research. The study varied the average cluster size from 585 2925 with the number of groups varying from 20 100 and differing levels of ICC (0.08 0.25 0 .5). Consistent with those reported by Hox and Maas (2010), although no parameter bias was found in within group part, there were small bias in variance estimates and their standard errors. This bias decreased rapidly with increasing group size. However, i n the between level part, significant estimation problems are present with a small number of groups (20). ICC was found to have no substantial influence on estimation accuracy. Furthermore, the authors suggested a sample of 50 100 counties that is needed f or accurate
27 estimation. Yet, contrary to earlier studies (Snijders & Bosker , 1993; Mok , 1995; Hox & Maas , 2001), no evidence was found for the existence of trade off effects between level 1 and level 2 sample sizes, perhaps this was due to the fact that th e level 1 sample size in the study was relatively large. More recently, Hox and his colleagues (2012) reanalyzes the simulation of Meuleman and Billiet (2009) to investigate the accuracy of the Bayesian estimation method with a very small numbers of countr ies. Results showed that a sample of about 20 countries is sufficient for accurate Bayesian estimation in MSEM. In sum, although there appeared to be substantial problems in making variance inference in the between group part, studies consistently indicat ed accurate parameter estimates in the within group part with a small number of groups. However, those efforts mainly focused on the effects of a small number of groups rather than on marginal group sizes. Even though most findings suggested that the effec ts of group level sample size is more beneficial than that of the level 1sample size (e.g., Hox & Maas, 2001; Snijders & Bosker, 1993), what remains unclear was the extent to which small individual size (e.g., dyads) impact the group level model fit or par ameter estimates in MSEM. Interestingly, simulation studies on the accuracy of multilevel regression estimates also revealed similar results of MSEM. (Busing, 1993; Van der Leeden & Busing, 1994; Clarke & Wheaton, 2007; Maas & Hox , 2004, 2005; Mok, 1995; B ell , Morgan , Kromrey & Ferron , 2013). In general, the regression coefficient of individual level appears to be robust across all conditions, generating accurate estimates even with only 30 groups and small group size (=5) (Maas & Hox , 2004, 2005; Kreft
28 19 96).However, the group level variances and regression coefficients are usually underestimated with small sample size.(Busing , 1993; Van der Leeden & Busing , 1994; Mok , 1995; Maas & Hox , 2004) and this bias is greatest at high er levels of ICC ( Hox & Maas , 2001). of 30 groups with at least 30 units per group. This rule was later develope d by Hox (1998) who suggested increasing the sample size to 50/20(that is, a minimum ratio of 50 groups to 20 units per group) to better test the cross level intersection. Further, Hox provided a 100/10 rule (a minimum ratio of 100 groups to 10 units per g roup) for variance and covariance testing. The Performance of Multi level Modeling with Dyadic D ata The guidelines mentioned above were commonly cited in e mpirical literature (e.g., Hox , 1998; Bell et al., 2013). Researchers continued to work with small cluster size and recommended multilevel regression models as powerful and flexible tools when testing the inter dependence effects on dyadic case (e.g., couples, twins, a class of two students). However, rule of thumb seemed to be followed without real pro of of what the minimum number of groups was required in the presence of extremely small cluster size should be. Besides, no simulation study had been conducted to examine dyads. Concern regarding this gap was first expressed by Newsom and Nishishiba (2002 ) in a small simulation study with 200 replications in each cell of design. They examined the effects of the number of dyads and ICC on parameter bias, their standard errors as well as convergence difficulties with REML estimator under a wide ranging of n umber of groups (50, 100, 200, 500, 1000) and levels of ICC (0.05, 0.1, 0.2, 0.3).
29 Results indicated that fixed effects and their standard errors showed generally low bias across all conditions. However, estimates of random effects leaded to serious non co nvergence problems, upward bias in parameters and their standard errors. Specifically, for intercept variance estimates, with higher ICC values (>0.1), fewer dyads were needed to avoid convergence problems. It was also found that parameter bias decreases a s the number of dyads increase. This increase also varied as a function of ICC. In addition, to obtain unbiased standard error estimates for intercepts, 500, 200, and 200 dyads are required for ICCs of 0.10, 0.20 and 0.30, respectively. However, estimates of slope variance and their standard errors appeared to be positively biased under all conditions and therefore seem to be impractical for dyadic researchers. As such, only random intercept components should be included in the random part of MRM (McMahon e t al., 2006). A recent simulation study by Clark and Wheaton (2007) focused on the effects of level linear models. The study includes sample sizes as few as 2 observations (dyads), with the number of groups varying from 50 100 and different leve ls of ICC (0.1 through 0.30). Consistent with those reported by Newsom and Nishishiba (2002), results showed no evidence of parameter bias in fixed effects across all conditions, but they did find evidence of positive bias in level 2 variance components in the conditions with dyadic data and small number of groups (= 50). However, it was interesting to find that this bias disappeared when the number of groups is larger than 200. Later, in another similar study, Clark (2008) extended the simulation study to examine small cluster size effects across continuous and discrete outcomes with both linear and non linear hierarchical
30 models. He demonstrated that the similar findings hold for both continuous and discrete outcomes, the only exception was the evidence of upward bias in fixed effects for non linear model with unbalanced data and extremely small cluster size (< 2). In general, simulation studies regarding dyads (Newsom & Nishishiba, 2002; Clarke & Wheaton, 2007; Clarke, 2008) consistently showed robust pe rformance of fixed effects under all conditions, but still reported substantial difficulty in making inferences about the variance components. Although these findings had paved the way for more general analytic work about dyads, each of them was still limi ted in ce rtain aspects: (a) Newsom and Nishishiba (2002) conducted simulation study based on simple two level h ierarchical linear models; (b) I cluster size was varied from 2 to 20 while the number of groups was he ld constant, equal to 200 and as such, it was difficult to test the effects of dyads on convergence difficulties, parameter bias and their standard errors with increasing number of groups. Current Study to Evaluate the Performance of MSEM w ith Dyadic Data Eventually, dyadic researchers became interested in not only the performance of simple MRM with extremely small cluster sizes, but also in modeling latent variables and measuring goodness of model fit. MSEM is a competing technique to explore these is sues with a variety of model testing options that are previously unavailable. However, only a few instances of researchers applying the MSEM to dyadic areas were available ( e.g., Newsom, 2002; Dyer , Hanges & Hall , 2005; Geldhof , Preacher & Zyphur, 2014). F or example, Newsom (2002), based on latent growth modeling, provided a strategy of application for second order factor model with small groups using 116 couple data set, results showed that small sample size may lead to empirical under identification probl ems (that is, negative variances occurred for the latent intercept or slope). Dyer
31 confirmatory factor analysis model (MCFA) with leadership data reflecting a unidimension al construct at both the individual and societal level of analysis. It was proved that MCFA leadership constructs do operate at the person and dyad groups (Yammarino, Dans ereau & Kennedy, 2001). In a recent simulation study, Geldhof and his colleagues (201 4) explored the applicability of MCFA for multilevel reliability estimation with varying cluster sizes (2 30), numbers of groups (50 200), and different levels of ICC (0.05 0.75). Results showed that dyadic data leaded to convergence problems when ICC was low (<0.2). Besides, contradictory to the previous findings showing robust performance of parameter estimates in the within group part across all conditions (Newsom& Nishishiba, 2002; Clarke & Wheaton, 2007; Clarke, 2007), results in this study showed that dyadic data had an impact on within group parameter estimates with small sample size and low ICC. Clearly, more information is needed on estimation problems mentioned above. As a result, it is of great importance to further investigate under what conditio ns do non convergence problems, bias in estimated parameter and their standard errors likely to occur. At the same time, when pseudobalance approach described by MuthÃ©n (1989, 1990) has become widely used to perform MSEM, no simulation study has analyzed t he robustness of the MUML estimator for MCFA with dyads. To shed some light on the applicability of MSEM to dyads research, the present study focuses on examining the performance of the MUML estimator for dyadic data by employing multilevel SEM. More spec ifically, the purpose of this paper is to address the following interrelated questions:
32 (1) To what extent the MUML estimator for MCFA models of dyadic data result in convergence difficulty, bias in estimated parameters, standard errors and the chi square test, confidence interval coverage rates of parameter estimates and Type I error rates of the chi square test? (2) To what extent do convergence problems, bias in parameter estimates (factor loadings, factor variance and residual variance) and their standa rd errors, confidence interval coverage rates in MCFA models of dyadic data, bias of chi square statistic and Type I error rate of the chi square test depend on number of dyads (group level sample size) and/or ICC
33 CHAPTER 3 METHOD To provide answers to the above research questions, a Monte Carlo simulation study was employed to examine the performance of MSEM with dyadic data under varying manipulated conditions. All data generation and analyses were completed using M plus versio n 7.1 (MuthÃ©n & MuthÃ©n, 2010 ). Multilevel confirmatory factor analysis (MCFA) model, as a special case of MSEM without structural path linking to latent variables, are widely used in MSEM relate d simulation studies.(e.g., Hox & Maas , 2001, 2010; Yu an & Hayashi, 2005; Me uleman & Billiet, 2009 ; Julian, 2001; Hox et al., 2014). Recently, research work about MCFA with dyadic data was also located (Dyer et al., 2005; Geldhof et al., 2014). In of model structure does not generally bias the results from fitting single level covariance structure model to nested data. The current study is interested in testing if this result is still hold when fitting MCFA to dyads. Therefore, a two level CFA (MCF A) model was employed for data generation across two simulation models that were commonly applied in the empirical study. Model 1: equ al between and within structure (e.g., Hox , 1993 ; Dyer et al., 2005, Yuan &Bentler, 2007 ) Model 2: complex within and simp le be tween structure (e.g., Hox & Maas, 2001, 2010 ; Julian, 2001). Detailed information of each scenario is described later in this study. With cluster size of 2, the following design factors will be varied across two scenarios: number of dyads (level 2 sa mple size= 50, 100, 200, 300). The intraclass
34 correlation coefficient (ICC= 0.15, 0.2, 0.3, 0.33) and model specification (equal or complex between and within structures). These factors lead to an experimental design with a 4 (ICC) 4 (sample size) 2 (model specification) = 32 unique cells. For each cell of the simulation study, 2000 replications were generated. The simulation parameters were selected based on the following criteria: a) Simulation parameters were used in line with previous simulation studies, b) Selection of parameter values ensured that the simulation conditions mimic those found in applied research. A more detail rationale used for the value selection for each of these conditions will be pro vided later. The outcomes of interest in this simulation study were : (i) P ercentage of non convergent solutions and inadmissible solutions. (ii) S tandard errors and 95% coverage rate of the f ixed effects and random effects . (iii) P arameter bias in fixed ef fects and random effects . (iv) T he accuracy of chi square test. Data Generation and Analysis Models Model 1. Equal Between Level Model and Within Level M odel The Multilevel CFA model used to generate data is presented in Figure 3 1. This model is similar to those used in Dyer and his colleagues (2005), the within and between levels were specified to have an equal factor structure with six observed variables y1 y6 that load onto single latent factors. The latent factor and residuals variance were simulate d with normal distributions and diagonal correlation matrix. Consistent with previous simulation work (e.g., Hox & Maas, 2001; Muthen & Satorra, 1995), in the within part, the factor loadings and residual variance of all observed variables were assigned to be 0.8 and 0.36, respectively. In the between part, the parameter values (factor loadings, residual variance) will vary according to the ICC as
35 describes later. The analysis model was equal to population model as shown in Figure 3 2. Model 2. Simple Between Level Model and Complex Within Level M odel In scenario 2, a Multilevel CFA model used to generate data is presented in Figure 3 3. This model structure is similar to those used in Hox and Maas (2001, 2010). The structure of the between level model in Scenario 2 was the same as that of the between level model in Scenario 1 with six observed variables y1 y6 that load onto single one latent factors. While in the within part, the six observed variables y1 y6 that load onto two latent factors. In the wi thin part, the correlations between the common factors were set to be 0.3, the factor loadings and residual variance of all observed variables were assigned to be 0.8 and 0.36, respectively, In the between part, the parameter values (factor loadings, resid ual variance) will vary according to the ICC as describes later. The analysis model that was equal to population model was shown in Figure 3 4. Simulation conditions Sample S ize In general, there are three types of sample size that are commonly analyzed i n MSEM: the total sample size , the number of clusters and the average number of units per cluster . For dyadic data, for all clusters , thus only balanced data were generated to represent the case of dyads. In this study, four level 2 sample size, =50, 100, 200 and 300 (i.e., =100, 200,400 and 600, respectively) were used to represent di fferent number of dyads. The reasons for choosing these levels of sample size were shown as follows: First, in the Hox and Maas (2001) simulation, number of
36 groups that larger than 50 were needed to produce acceptable between model estimates, Second, in l ine with previous literature, the minimum number of 200 groups is usually reported to obtain consistent and efficient estimates when using ML estimation methods (e.g., Boomsma , 1987; Loehlin, 20 13; Clark & Wheaton, 2007). Intra C lass Correlation (ICC) Fo ur levels of ICC were manipulated in the present study: 0.15, 0.20, 0.3 and 0.33. These levels of ICC are selected based on levels of clustering typically found in population based survey data (Gulliford , Ukoumunne & Chinn , 1999). In addition, though ICCs rarely exceed .30 in educational research (LÃ¼dtke et al., 2008) . ICC will sometimes exceed 0.33 in family research (Hox & Maas, 2001). It is of great values to examine the performance of MCFA with dyadic data under more rea listic data conditions, therefore these ranges of ICC were considered. It is important to recall here that in MSEM, the ICC of a variable is a function of between level variance ( ) and within level varian ce ( ) ( MuthÃ©n & Satorra, 1995) as follows: (3 1) Where represents the th diagonal element of a covariance matrix. According to Equation (8), in MCFA, the indicator variance is in turn a combination of three components: loadings between indicators and latent factors, latent factor variances and residual variance of indicators. Thus, ICC can be further expressed as a function of model parameters as follows (MuthÃ©n, 1991, 1994): (3 2)
37 Therefore, the ICC of population model can be controlled by adjuste d portions of factor loadings and residual variance in between and within levels, while holding the factor variance constant, For instance, for a high ICC value of 0.33, the factor loadings and residual variance for the within group part were held constan t to 0.8 and 0.36, respectively. In the between group part, the factor loadings were set to 0.5124 and residual variances were set to 0.23. As the latent factor variances were fixed to 1.0 across two levels for model identification purposes, the latent var iances and their correlation do not contribute to the calculation of the various ICCs. Population parameter values that correspond to different levels of ICC can be found in Table 3 1 . Dependent Variables and Analysis For each generated sample data set, t he two level CFA models were analyzed with the MUML estimator. The models were identified by fixing the latent factor variance to 1 in both the between part and within part. Thus the estimated parameters are the residual variance and factor loadings. With respect to dependent variables, each of the two models mentioned above will be compared based on the following: (i) Percentage of replications with convergence problems or inadmissible solutions (e.g., negative unique variances) across all conditions. More specifically, percentage of convergent solutions is examined by the number of samples that fail to obtain estimates. Improper (inadmissible) solutions, which is when MPLUS is able to obtain estimates, but they are not proper because they contain a negativ e variance or a correlation that is outside of the 1 to 1 range. (ii) 95% coverage rate refers to the proportion of replication for parameter estimate falls within the 95% confidence interval.
38 (iii) The accuracy of parameter bias was calculated by percent age relative bias. Let be the average bias of each combined combination, the percentage relative bias was given by (Bandalos & Leite, 2013): = (3 3) Where refers to the population parameter value. The relative bias was investigated with sets of factor loadings and error va riance as multilevel outcomes. In up to 5% is acceptable. If >0.05, estimation is regarded as not sufficiently accurate. ( iv) Bias in standard errors of parameters (factor loadings and residual variance) was calculated by the following equations (Bandalos & Leite, 2013): = (3 4) <0.1 is considered as a tolerable value (Hoogland &Boomsma, 1998). In the equation, refers to the mean of standard error estimates across iteration s of a combined condition, and refers to the standard deviation of sample estimates, when the number of replications is large, this is considered to be the population standard error. (v) The accuracy of the chi square global test is suggested by calculating the relative bias, comparing the estimated value with expected value. In addition, the percentage of rejected rate at a significance level of 5% is also presented. As the model
39 is correctly fitted in the current study, the expec ted value for the chi square is equals to the degrees of freedom and the rejection rate indicated the Type I error rate. Following the prior simulation studies (e.g., Curran , West & Finch , 1996; Kaplan, 1989), the relative chi square bias less than 5% is c onsidered being acceptable.
40 T able 3 1. Population Parameter Values for Data Generation ICC 0.33 0.5124 0.23 0. 8 0.36 0.3 0 0.4781 0.20 0. 8 0.36 0.2 0 0.3872 0.10 0. 8 0.36 0.15 0.3105 0.08 0. 8 0.36 Note: ICC=Intraclass c orrelation c oefficient; = Within group factor loading; = Between group factor loading; =Between group residual variance; = Within group residual variance.
41 Figure 3 1. Population Model Used for Data G eneration in Scenario 1
42 Figure 3 2 . Analysis M odel in Scenario 1
43 Figure 3 3. Population Model Used for Data G eneration in Scenario 2
44 Figure 3 4 . Analysis M odel in Scenario 2
45 CHAPTER 4 RESULTS Non Convergent Problems and Improper S olutions The percentage of replications with non convergence problems and improper solutions (e.g., negative unique variances) over 2000 replications is presented in Table 4 1 and Table 4 2, respectively. For Model 1, within and between levels were specified to h ave an equal factor structure with six observed variable loaded on a single latent factors. As Figure 4 1 and Figure 4 2 show, the model generally produced poor convergence rates when ICC values were lower than 0.3 and the number of dyads was less than 200 . The model generated the highest percent of a negative definite matrix (60.5%) and non convergent problems (30.5%) in conditions with the smallest number of dyads (50) and the lowest ICC (0.15). However, when the number of dyads increased from 50 to 300, the average proportion of non convergent problems rapidly dropped from 21.7% to 3.9% and the improper solutions dropped from 43.8% to 6.9%. For Model 2, the within levels were specified to have the six observed variables that load onto two latent factors , MUML produced less non convergent or improper solutions versus those of Model 1. Similarly, as Figure 4 3 and Figure 4 4 indicate, Model 2 had the worst non convergent problems (22.7%) and improper solutions (52.8) with ICC= 0.15 and = 50. By increasing the number of dyads from 50 to 300, the corresponding non convergent problems decreased from 13.3% to 2.4% and inadmissible solutions decreased from 45.3% to 4.0%. Overall, there were modest differences evident in the convergence rates for model specifications. The convergence rates for both models increased as a function of the number of dyads and ICC values; with higher ICC values, fewer dyads are needed to avoid non convergence or improper solutions. At the largest sample size (
46 =300) together with the highest ICC (0.33), MUML estimation converged to proper solutions of all models across 100% of replications. Parameter B ias Information about the magnitude of the bias in the parameter estimates can be det ermined by reviewing relative bias. Table 4 3 shows the results for bias estimates for the within group part of Model 1 and Model 2. In the within group part of Model 1, the factor loadings showed an overall positive bias of 0.8% with minor difference acro ss varying levels of ICC. The largest bias was found at the smallest sample size. However, bias rapidly decreased when the number of dyads is increased. For residual variance, MUML consistently produced little negative bias across all the conditions, this bias seemed to be unrelated to ICC values but varied as a function of the increasing sample size. Again, the largest bias ( 2.2%) was established in conditions with the smallest number of groups (50). A similar pattern was found for Model 2, though the es timates of factor loadings were consistently underestimated, which was different from that in Model 1. The overall relative bias was 0.3% for factor loadings and 1.2% for residual variance. Clearly, both the fixed and random parameters indicated little t o negligible bias across all conditions. Once again, the largest bias was found at the lowest sample size, however, MUML rapidly produced accurate parameters with increasing number of dyads. In the between group part of the model, bias in parameter estimat es was quite different. Table 4 4 displays the results for bias estimates for the between group part for both models. The overall bias of factor loadings was 7.8% for Model 1, therefore, a loading of 0.8 was typically estimated as 0.74. Bias in factor loa dings estimates appeared to depend significantly on the sample size. When the number of dyads ranged
47 from 50 100 200 300, the corresponding degree of bias was 11.2%, 10.2%, 6.7% and 4.1%. This bias also varied as a function of increasing ICC, but this effect was more pronounced with lager number of dyads. The model produced less than 5% bias only when the ICC was increased to 0.3 and number of dyads reached to 200. In addition, comparison of the findings for Model 2 with those for Model 1 indicated that the bias in factor loadings was also affected by model specification. The overall bias for factor loadings was 5.0% for Model 2. Similarly, factor loading bias decreased as a function of increasing number of dyads and increasing ICC values. Residual variance estimates showed evidence of severe bias. The between group model consistently indicted negative bias as high as 81.5% when the ICC was set at 0.15 with 50 dyads. Although bias rapidly decreased by increasing both the number of dyads and the ICC values, it still indicated greater than 10% underestimation when ICC was lower than 0.33 and number of groups was less than 200. Parameter estimates of the between group part for Model 2 showed a similar pattern to that for Model 1. Again, when the number of groups was less than 200, parameter estimates of factor loadings and residual variance consistently showed severe bias across all levels of ICC. Bias in residual variance was highly problematic ( 75.4%), this meant that the observed values of error var iance were typically estimated less than half of their true values. Overall, regardless of the underlying model structure, bias estimates of the within group part was typically less than 5% in all conditions, MUML produced accurate estimates when the ICC w as increased to 0.2 and groups reached to 200. However, there appeared to be some indication that the parameter bias for between group parts
48 was affected by model specification, particularly for factor loadings. Within each of the two models, parameter bia s decreased as sample size and ICC increased. Bias in residual variance proved to be highly problematic unless the ICC is greater than 0.3 with a sufficient number of dyads. More specifically, to make a valid inference about variance estimates (bias<5%) fo r the between group part, it appears that 200 dyads are necessary when ICC is larger than 0.2. Standard Error B ias The proportion of relative bias in estimated standard errors for factor loadings and residual variance were presented in Table 4 5. The stan dard errors bias for factor loadings were predominately underestimated across all cells of conditions for both models. In the within part of Model 1, the relative bias of standard errors for factor loadings was 14.1%. When ICC values were less than 0.2, t he standard errors consistently show significant bias beyond 10% with minor difference across all the conditions. However, there was a direct trend toward less bias with increasing number of dyads and increasing ICC. Specifically, when the ICC values is l arger than 0.2, by increasing the number of dyads from50 to 300, the corresponding standard error bias dropped from 15.0% to 6.7%. Interestingly, the effect of sample size and ICC on standard error bias for factor loadings appeared to be difference with model specification. For Model 2, the overall relative bias of standard errors was somewhat smaller ( 6.3%) compared to those from Model 1 and this bias varied as a function of number of dyads and ICC across all the conditions. Although the standard errors exhibit negative bias greater than 10%, these observations were only limited to the smallest sample size combined with lower ICC levels (<0.3).
49 Regardless of underlying model structure, within each of the two models, the standard error bias for residual v ariance was quite smaller, with nearly every bias estimates below 5% across all the cells of design. The degree of bias largely depended on number of dyads but appeared to be little related to ICC. With increasing number of dyads, both models rapidly prod uced accurate standard errors for residual variance. With regard to the between group part of models, the standard errors generally revealed a severe bias. These results can be found in Table 4 6. The standard errors for factor loadings were consistently underestimated. For model 1, the overall relative bias of standard errors for factor loadings was 16.8%. This bias was somewhat reduced at la r ger sample sizes together with higher ICC, however, as many as 300 dyads were not sufficient to keep standard er ror bias under 10% across all ICC levels. In addition, standard errors bias for factor loadings also appeared to be related to model specification. For model 2, the overall relative bias of standard errors for factor loadings was 12.9%, the influence of sample size appeared to be much greater at higher ICC values ( 0.3). To obtain an acceptable standard error bias, = 300, 200 and 200 seemed to be necessary for ICCs of 0.2, 0.3, and 0.33, respectively. The standard error bias for residual variance was much smaller and did not consistently move in a positive or negative direction .For mode 1, although it showed positive bias as high as 71.5% when the number of dyads was 50 and the ICC was 0.15, this bias dra matically decreased and tended to move in a negative direction with increased sample size and higher ICC values. A similar pattern was found for Model 2. Again, severe bias in standard errors (36.8%) was established at the smallest sample size ( =50) together with the lowest ICC (0.15), this bias decreased as a function of
50 sample size and ICC. Therefore, to make a valid inference about variance estimates (bias<10%) for between group part, 200 dyads appear to be necessary when ICC was larger than 0.2. Confidence Interval Coverage R ates To investigate the effects of the combination of biased standard errors and biased parameter estimates, the 95% confidence interval for each parameter was calculated and the proportion of the interva l that contains the population parameter value was then counted. The results for model 1 and model 2 are presented in Table 4 7 and Table 4 8, respectively. In the within group part of Model 1, the mean coverage of 95% confidence interval was 90.9% for fac tor loadings with minor difference across all conditions. For residual variance, the mean coverage rates were 92.9% which appeared to improve as a function of increasing number of dyads. In the between group part, the mean coverage of 95% confidence for fa ctor loadings was too low (85.9%), however, the coverage rates clearly directly increased when number of dyads increased from 50 to 300 and ICC levels increased from 0.15 to 0.33. The mean coverage of residual variance was 93.8% which generally improved wi th the increased number of dyads but appeared to be negatively influenced by ICC values. For Model 2, in the within group part of the model, the mean coverage rate was 93 % for factor loadings and 93.9% for residual variance. In the between part, the mean coverage was 87.7% for factor loadings and 94.1% for residual variance. Overall, the coverage rat e s for model 2 were consistently better than that for model 1. However, the variation trend was almost the same regardless of the underlying model structures.
51 Chi Square S tatistics Because the correct model was fitted in the current study, the expected value for the chi square is equals to the degrees of freedom, which is 18 for model1 and 19 for model 2. At a 5% significance level, the expected proportion of true model rejections out of 2000 replications is 100. The relative bias of the chi square test statistics and Type I 9. For Model 1, MUML chi square test statistics consistently showed downward bias across all the cells of design. This bias decreased as a function of increasing ICC values. When ICC increased from 0.15 to 0.33, the average bias drastically dropped from 9.6% to 1.6%. In addition, there appeared to be some indication that the chi square statistics were also positively affected by sample size. For model 2, the chi square bias again decreased as sample size increased, however, the effect of ICC on the bias in chi square test values varied substantially with model specificati on. When ICC was less than 0.3, the chi square value turned out to be underestimated across all the conditions. However, this bias tended to move into a positive direction and was generally below 5% in a higher ICC condition ( 0.3) . Overall, for both models, by increasing the ICC levels as well as the number of dyads, the size of chi square gradually closed to its expected value which is 18 , and MUML continued to produce Type I error rates that gradually close to the nominal 5%.
52 Ta ble 4 1. Percentage of Non convergent Solutions for Model 1 and Model 2 . Number of dyads ICC .15 ICC .20 ICC .30 ICC .33 n % n % n % n % Model 1 50 611 30.5 468 23.4 359 17.9 33 15.1 100 467 23.4 301 15 178 8.9 129 6.4 200 319 16 .0 156 7.8 40 2 .0 20 1 300 224 11.2 76 3.8 12 0.6 5 0 Model 2 50 455 22.7 297 14.8 178 8.9 137 6.9 100 270 13.5 101 5 .0 26 1.3 16 0.8 200 114 5.7 14 0.7 1 0 0 0 300 47 2.4 1 0 0 0 0 0 Note. n= Numbers of non convergent models. %= Percent of non convergent problems. Table 4 2. Percentage of Non convergent Solutions for Model 1 and Model 2 Number of dyads ICC .15 ICC .20 ICC .30 ICC .33 n % n % n % n % Model 1 50 1210 60.5 1028 51.4 778 38.9 606 30.3 100 868 43.4 660 33 .0 378 18.9 308 15.4 200 542 27.1 356 17.8 142 7.1 79 4 .0 300 264 13.2 196 9.8 92 4.6 0 0 Model 2 50 455 52.8 297 42.8 678 33.9 378 18.9 100 270 34.3 101 25.1 328 16.4 16 8.8 200 114 16.8 14 10.2 102 5.1 0 0 300 47 9.4 1 6.4 0 0 0 0 Note. n= Numbers of improper solutions. %= Percentage of improper solutions .
53 Table 4 3. Relative Bias i n Para meter Estimates for Within Mode ls Number of dyads Bias in factor loadings Bias in residual variance Bias % Bias % Model 1 ICC .15 50 0 0.1 0.007 1.9 100 0.004 0.5 0.003 1 .0 200 0.005 0.6 0.001 0.4 300 0.004 0.5 0.001 0.3 ICC.20 50 0.008 1 .0 0.008 2.1 100 0.01 0 1.2 0.003 0.9 200 0.006 0.8 0.001 0.4 300 0.004 0.6 0.001 0.3 ICC.30 50 0.014 1.7 0.009 2.3 100 0.011 1.4 0.004 1 .0 200 0.004 0.5 0.001 0.4 300 0.001 0.2 0.001 0.3 ICC.33 50 0.015 1.7 0.009 2.4 100 0.011 1.3 0.004 1.1 200 0.003 0.4 0.001 0.4 300 0 0 0.001 0.3 Model 2 ICC .15 50 0.008 1 .0 0.01 0 2.5 100 0.003 0.4 0.005 1.3 200 0.002 0.2 0.002 0.5 300 0.002 0.2 .0.001 0.3 ICC.20 50 0.004 0.5 0.01 0 2.6 100 0.001 0.1 0.005 1.3 200 0.002 0.2 0.002 0.5 300 0.002 0.2 0.001 0.3 ICC.30 50 0.003 0.3 0.01 0 2.9 100 0.001 0.1 0.005 1.4 200 0.002 0.2 0.002 0.5 300 0.002 0.2 0.001 0.3 ICC.33 50 0.003 0.3 0.011 2.9 100 0.001 0.1 0.005 1.4 200 0.001 0.2 0.002 0.5 300 0.002 0.2 0.002 0.3 Note. Bias= ( refers to the average bias of each combined combination, refers to the population parameter value). %= the mean relative bias of parameter estimates .
54 Table 4 4 . Relative Bias i n Parameter Estimates for Between Model s Number of dyads Bias in factor loadings Bias in residual variance Bias % Bias % Model 1 ICC .15 50 0.027 9 .0 0.065 81.5 100 0.037 12.1 0.038 47.5 200 0.035 11.3 0.02 25 .0 300 0.025 8.1 0.014 17.6 ICC.20 50 0.048 12.3 0.033 33.5 100 0.048 10.3 0.063 33.5 200 0.032 8.4 0.017 17 .0 300 0.021 5.4 0.01 10 .0 ICC.30 50 0.057 12.1 0.068 34 .0 100 0.042 8.9 0.03 14.8 200 0.019 4 .0 0.012 6 .0 300 0.009 1.9 0.007 3.3 ICC.33 50 0.044 11.3 0.036 19.2 100 0.037 7.4 0.031 13.4 200 0.015 2.9 0.01 4.5 300 0.005 1 .0 0.006 2.7 Model 2 ICC .15 50 0.019 8.1 0.06 75.4 100 0.017 6.4 0.033 41.5 200 0.01 3.3 0.014 17.3 300 0.007 2.1 0.007 9.4 ICC.20 50 0.023 7.9 0.048 48 .0 100 0.016 5.2 0.024 24.4 200 0.006 2.6 0.007 7.3 300 0.002 0.6 0.004 4.1 ICC.30 50 0.021 6.4 0.047 23.7 100 0.011 4.7 0.018 9.1 200 0.003 1.5 0.006 3.1 300 0.001 0.2 0.004 1.9 ICC.33 50 0.019 6.1 0.044 19.2 100 0.009 4.7 0.015 6.5 200 0.002 0.4 0.006 2.6 300 0 0.1 0.003 1.4 Note. Bias= ( refers to the average bias of each combined combination, refers to the population parameter value). %= the mean relative bias of parameter estimates .
55 Table 4 5. Relative Bias i n Standard Errors for Within Model s Number of dyads Bias in factor loadings Bias in residual variance Bias % Bias % Model 1 ICC .15 50 0.014 12 .0 0.003 3.4 100 0.01 0 11.3 0.001 1.7 200 0.008 13.6 0.001 1.5 300 0.007 13.7 0 0.6 ICC.20 50 0.017 13.9 0.003 3.3 100 0.011 12.9 0.001 1.5 200 0.009 10.9 0.001 1.6 300 0.007 8.9 0 0 ICC.30 50 0.019 15 0.003 3.1 100 0.012 13.3 0.001 1.7 200 0.007 10 .0 0.001 1.3 300 0.004 7.8 0 0.6 ICC.33 50 0.019 15.1 0.003 3 .0 100 0.012 9.2 0.001 1.8 200 0.006 8.6 0.001 1.3 300 0.003 5.6 0 0.6 Model 2 ICC .15 50 0.014 11.4 0.006 4.9 100 0.008 10.5 0.002 2.3 200 0.004 7.4 0.001 1.2 300 0.003 6.7 0.001 1.3 ICC.20 50 0.014 11.3 0.006 4.7 100 0.007 12.5 0.002 2.1 200 0.003 5.4 0.001 1 .0 300 0.002 4 .0 0 1 .0 ICC.30 50 0.013 9.8 0.005 4.1 100 0.006 6.4 0.001 1.5 200 0.002 2.9 0 0.6 300 0.001 2.5 0 0.7 ICC.33 50 0.012 9.2 0.004 3.7 100 0.002 2.4 0.001 0.4 200 0.004 2.4 0.001 0.4 300 0.001 2.2 0 0.7 Note. Bias= ( refers to the mean of standard error estimates across iterations of a combined condition, and refers to the standard deviation of sample estimates,). %= the mean relative bias of standard errors.
56 Table 4 6. Relative Bias i n Standard Errors for Between Model s Number of dyads Bias in factor loadings Bias in residual variance Bias % Bias % Model 1 ICC .15 50 0.018 6 .0 0.244 71.5 100 0.007 3.2 0.139 65.2 200 0.029 16.8 0.02 19.7 300 0.028 19.2 0.003 3 .0 ICC.20 50 0.044 15.7 0.023 18.8 100 0.029 13.1 0.042 21 .0 200 0.03 18.3 0.028 22.1 300 0.029 22.3 0.002 3.1 ICC.30 50 0.025 8.3 0.178 21.4 100 0.05 23.2 0.012 6.2 200 0.029 20.7 0.009 4.2 300 0.015 14.2 0.001 2.4 ICC.33 50 0.031 10.3 0.175 38.7 100 0.036 16.5 0.062 22.8 200 0.026 19.6 0.007 4.1 300 0.01 10.1 0.009 3 .0 Model 2 ICC .15 50 0.035 12.5 0.13 36.8 100 0.029 13.6 0.057 22.7 200 0.026 18.5 0.01 9.4 300 0.021 18.9 0.007 12.6 ICC.20 50 0.042 16.4 0.049 17.3 100 0.033 17.9 0.005 2.8 200 0.016 14.7 0.003 5.5 300 0.008 9.7 0.001 1.4 ICC.30 50 0.047 18.4 0.024 16.5 100 0.022 13.2 0.014 8.4 200 0.007 7.2 0.001 2 .0 300 0.004 5.1 0 0.8 ICC.33 50 0.041 16.5 0.023 12.5 100 0.022 14.2 0.009 9 .0 200 0.006 5.9 0.001 1.5 300 0.003 4.3 0 0.5 Note. Bias= ( refers to the mean of standard error estimates across iterations of a combined condition, and refers to the standard deviation of sample estimates,). %= the mean relative bias of standard errors .
57 Table 4 7. Coverage of 95% Confidence Interval for Model 1 F actor loadings coverage R esidual variance coverage NG NG Model ICC 50 100 200 300 50 100 200 300 Within M odel 0.15 92.0 91.8 90.3 90.2 91.3 93.1 94.0 94.6 0.2 0 91.0 89.4 89.8 90.5 91.2 93.0 93.9 94.5 0.3 0 90.0 89.9 91.5 92.6 91.0 93.0 94.1 94.6 0.33 92.2 90.2 92.2 93.2 93.6 92.9 94.0 94.5 Between 0.15 84.8 84.1 84.2 85.4 94.5 95.0 95.5 95.4 Model 0.2 0 83.8 84.0 85.9 87.6 94.1 88.7 95.3 95.1 0.3 0 83.2 85.4 89.7 92.0 92.9 93.8 94.6 94.9 0.33 87.4 86.8 91.1 93.2 92.9 93.7 94.3 93.8 Note. NG = Number of dyads Table 4 8. Coverage of 95% Confidence Interval for Model 2 F actor loadings coverage R esidual variance coverage NG NG Model ICC 50 100 200 300 50 100 200 300 Within Model 0.15 92 .0 92.8 93.1 93.3 93.3 94 .0 94.5 94.6 0.2 0 91.9 92.8 93.7 93.8 93.5 94.1 94.7 94.8 0.3 0 92.3 93.2 94.3 94.2 93.5 94.1 94.8 94.8 0.33 92.1 93.4 94.4 94.3 91.8 94.1 94.8 94.8 Between Model 0.15 81.9 83.7 86.9 89.1 94.9 95.2 95.2 95.3 0.2 0 83.6 86.7 90.5 92.3 94.3 94.7 94.8 95 .0 0.3 0 86.2 90.5 93.1 94 .0 93.4 93.9 94.4 94.8 0.33 84.4 91.4 93.8 94.2 90.7 93.7 94.3 94.8 Note. NG = Number of dyads
58 Table 4 9 . Relative bias in chi square test and the percentage of rejected rate Note. NG = Number of dyads Bias Chi square test (%) Type I error rates with nominal = 5 (%) ICC NG NG 50 100 200 300 50 100 200 300 Model 1 0.15 9.4 11.0 9.8 8.1 2.5 1.7 2.2 1.8 0.2 7.7 8.5 5.9 4.0 3.1 2.0 2.7 2.7 0.3 4.9 4.2 1.5 0.2 3.3 4.6 3.6 4.3 0.33 3.8 2.5 0.5 0.5 3.4 3.8 4.0 4.6 Model 2 0.15 5.7 7.0 4.0 2.4 4.0 2.5 3.6 3.2 0.2 2.9 3.2 0.5 0.2 4.9 3.5 4.9 3.8 0.3 0.4 0.1 0.9 0.9 5.8 4.9 4.7 4.6 0.33 1.7 0.9 1.2 1.1 6.6 5.0 4.6 4.6
59 Figure 4 1. Percentage of Non convergent Solutions for Model 1 Figure 4 2. Percentage of Improper Solutions for Model 1
60 Figure 4 3. Percentage of Non convergent Solutions for Model 2 Figure 4 4 . Percentage of Improper Solutions for Model 2
61 CHAPTER 5 SUMMARY AND DISCUSSION Multilevel structural equation modeling (MSEM) is a latent variable technique that allows us not only to analyze hierarchical data by specifying within and between models simultaneously, but to also consider the measurement error and model fit at differen t levels. However, practices associated with the employing of MSEM with extremely small group sizes (e.g., dyadic data) are problematic due to the lack of empirical research. In the current study, a Monte Carlo simulation was employed to investigate the pe rformance of MCFA with dyadic data using a MUML estimator. The results of the current study indicated the application of Multilevel CFA with dyadic data is a potential dangerous practice especially when the between group part is the primary focus of invest igation. It should be noted first that the specific model structure considered in this study clearly lead to additional bias of factor models. Model 1 and Model 2 have same between level structure with six indicators loaded on single dimension, however, fo r the within group part, Model 1 and Model 2 both have six observed variables, but in Model 2, the indicators measure two factors, whereas the indicators for model 1 measure a single dimension. Because of this different specification, Model 2 generally lea d to less bias with regard to parameter estimates, their standard errors and chi square test statistics, especially under conditions with low ICC and small sample size. In terms of the within group part, both the factor loadings and residual variance show ed little to negligible bias with minor difference across all cells of conditions. Although the standard errors for factor loadings exhibited negative bias greater than 10%, with minor difference between several cells of design, these observations were
62 onl y limited to the conditions with lower ICC ( 0.2) and smaller number of dyads (less than 200). This finding supported Gonzalez and his colleague s dyadic data do have an impact on within group standard errors estimates in the low ICC condition. With increased number of dyads from 200 to 300, the overall relative bias dropped below 10% with ICC having a larger effect on a large r number of dyads. The standard error bias for residual variance was quite smaller, with nearly every bias estimates below 5%, the degree of bias was directly related to number of dyads but seemed to depend little on the levels of ICC. One general result found is that for the within group part, the varied simulation conditions have no notable impacts on the accuracy of parameter estimates and their standard errors. Previous simulation studies on the accuracy of parameter estimates of individual levels show ed results similar to ours (Hox, 1995; Meuleman & Billiet, 2009; Hox & Maas, 2001, 2010; Hox et al., 2012). Just as MuthÃ©n (1989, 1990) pointed out, in the MUML approach, the pooled within group covariance matrix is given as the maximum likelihood estimato r of the population within group covariance matrix. The early simulation studies suggested a total sample size of 200 for a proper performance o f an ML estimator (Boomsma, 198 7 ). Thus it was not surprising that in the current study, MUML produced unbiased and efficient parameter estimates and their standard errors for the within group part when the number of dyads was larger than 100. In general, these simulation results indicate that MCFA of dyadic data is much more useful when researchers are interested o nly in the within group part of the model; for instance, MCFA is used for data obtaine d from cluster sampling (Hox & Maas, 2001, 2010), thus analyzing pooled within covariance
63 matrix instead of its total covariance matrix is a simple and efficient approac h (MuthÃ©n, 1989). The current study also suggested that the application of MCFA to dyadic data is impractical when researchers are concerned with variability across dyads. For the between group part of the models, the effects of dyad groups (level 2 sample size), ICC values and model specification were more pronounced on the accuracy of the estimates and their standard errors. For model 1,the overall bias of factor loadings was beyond 5%, only when the ICC was increased to 0.3 and number of dyads reached t o 200, MUML produced less than 5% bias that Hoogland and Boomsma (1998) considered still acceptable. For model 2, less bias was established in factor loadings. To obtain acceptable factor loadings estimates, at least 200 dyads are required together with IC C values that larger than 0.2.The residual variances consistently revealed server estimation problems. More specifically, the between group model consistently showed negative bias larger than 70% for both models when ICC was set at 0.15 with 50 dyads, thi s meant that the observed values of error variance were typically estimated less than half of their true values. Although bias decreased with an increased number of dyads together with higher ICC, it still showed greater than 10% underestimation in the con ditions with small sample size and low ICC. The between group standard errors were estimated with considerably less accuracy than parameter estimates. Within each of two models, the standard errors for factor loadings were generally underestimated and move negatively away from the population values. For model 1, although the relative bias of standard errors was somewhat reduced at larger sample size together with higher ICC, as many as 300
64 dyads were not sufficient to keep standard error bias under 10% whi ch Hoogland and Boomsma (1998) indicated as acceptable for standard errors. The effects of sample size appeared to be much greater in standard error bias for model 2, there is a trend toward less bias with increasing number of dyads. Therefore, to obtain a n acceptable standard error bias, = 300, 200 and 200 seemed to be necessary for ICCs of 0.2, 0.3, and 0.33, respectively. The standard errors of variance estimates showed highly bias at the smallest sample size for both models, th is negative bias rapidly decreased with increasing sample size and increasing ICC levels. At least 200 groups or a large ICC are needed here to achieve a relative bias of less than 10%. However, because the parameter estimates were also biased, efficient e stimated standard errors do not necessarily lead to confidence intervals with good coverage. The coverage rates were particularly low (less than 90%) for the between group factor loadings. Thus, to obtain an empirical coverage with a bias of approximatel y 5% around a nominal 95% confidence interval (i.e., 90% 99%), at least 200 groups are required when ICC is reached to 0.3. Difference between designed conditions also appeared in the global chi square statistics. Both the bias in chi square and the probability of rejection rates were related to the model specification, number of dyads and ICC values. Interestingly, Model 1 produced negative chi square bias across all the cells of design. When ICC is less than 0.3, this underestimated bias is generally larger than 5% which decreased Type I error rates for the chi square model fit test, thereby the correctly specified models were rejected less often than expected. However, model 2 produced minimal inflated chi squar e bias at higher ICC level ( 0.3). Overall, for both models, by increasing ICC as
65 well as the number of dyads, the size of chi square gradually closed to its expected value and their Type I error rates gradually closed to nominal 5 %. Additionally, non convergence or improper solutions was another serious problem when employing MCFA with dyadic data. This finding is consistent with previous simulation study (Newsom & Nishishiba, 2002; Clarke & Wheaton, 2007; Clarke, 2008) which indic ated severe convergence problem with the use of multilevel modeling with dyadic data, particularly at small sample size and low ICC. Overall, regardless of the underlying model structure, there were generally poor convergence rates when ICC value was less than 0.2, together with a small number of dyads (less than 200). One rationale for these results can be the simple fact that only limited information was available at the between group level under low ICC and small sample size conditions. It is also possib le that the serious problems which occurred in the between (Kenny , Mannetti , Pierro , Livi & Kashy , 2002). In other words, although ICC is generally defined as the proportion of between group variance to its total variance, it can also be discussed as the correlation between pairs of dyads sampled from a certain cluster (Kenny et al., 2002). In that case, a small level 1 sample size with a low ICC condition is likely to result in negative expec ted covariance between same dyad pairs, making it possible to obtain negative ICC which further leads to model misfit or non convergent problems ( Geldhof et al., 2014). Therefore, further research is needed to investigate if ed results in the estimation and convergence problems identifiable in our simulation study.
66 Given our findings, it is necessary to caution against using multilevel CFA when the number of dyads is smaller than 200, especially if the ICC values turns out to be under 0.2. Although increasing the level 2 sample size and/or improving ICC values are two simplest ways to achieve accurate estimate, both of them are difficult to do in practice. Therefore, some alternative strategies are recommended here. First, usi ng robust chi square statistics which is designed to adjust the chi square model fit and the standard errors of covariance model seem s especially useful (Satorra & Bentler, 1990). However, these approach described by Satorra and Bentler can only alleviate bias in the chi square statistic and the standard errors, therefore bias in the parameter estimates may still exist when MCFA are applied to dyadic data. Another promising solution might be to use Bayesian methods which may allow for several advantages ove r classical methods. First, unlike the asymptotic ML method, the Bayesian approach is efficient in small samples. The estimates are always accurate under the correct probability distribution, and the approach therefore alleviates the problem of negative va riance estimates. Hox and his colleagues (2012) indicated in their simulation study that Bayesian estimation will help in making valid inferences in multilevel SEM when the number of groups is relatively small. As a result, we can expect MCFA to perform be tter by using Bayesian estimates instead of MUML. Some researchers ( van der Leeden et al. , 199 4; Hox & Maas, 2001, 2 010 ) also recommend using boostrap methods for empirical standard errors estimates and future research on this may be promising for testing random effects with dyads. The current study provided useful information on dyadic analysis with multilevel CFA. However, there were still several limitations to this study. Firstly, it was important
67 to note that our study was based on a relatively simple multilevel data structure. In other words, the hierarchical data considered here was completely nested and only two level models were analyzed. Furthermore, two kinds of model structures considered in the simulation process were also relatively simple. That is, for both models, th e between group parts were specified to have same factor structure with six observed variables that load onto single latent factors. The model specification only varied in the within group part with the same number of indicators. However, hierarchical stru ctures were usually much more complex than those in this current study, and additional research was needed to analyze a broader array of multilevel data structures in covariance modeling and to evaluate the effects of small group size for analyzing non nor mal outcomes using generalized multilevel models. In addition, the present study did not consider the issue of model misspecification, in other words, the fixed or random parameters were not varied correctly or incorrectly specified in the within or betwee n the effects of incorrect model specification on parameter estimates, their standard errors and model fit. Furthermore, the Monte Carlo simulation in the current stu dy is conducted by Mplus software package and it is difficult to generalize the effects of using a single program on our results. Thus, more research is required to compare previous results with different software packages. Despite these limitations, the c urrent study clearly underscores the problems of using MCFA with dyadic data, and highlights conditions wherein non convergent problems, bias in estimated parameters, standard errors and the chi square test might be encountered.
68 LIST OF REFEREN CES Allison, P. D. (2009). Fixed effects regression models. Los Angeles: Sage. Atkins, D. C. (2005). Using multilevel models to analyze couple and family treatment data: basic and advanced issues. Journal of Family Psychology , 19 (1), 98. Bandalos, D. L., & Leite, W. L. (2013). Use of monte carlo studies in structural equation modeling research. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed., pp. 625 666). Greenwich, CT: Information Age Publishing. Barcikowski , R. S. (1981). Statistical power with group mean as the unit of analysis. Journal of Educational and Behavioral Statistics , 6 (3), 267 285. Barnett, R. C., Marshall, N. L., Raudenbush, S. W., & Brennan, R. T. (1993). Gender and the relationship between job experiences and psychological distress: a study of dual earner couples. Journal of P ersonality and S ocial psychology , 64 (5), 794. Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics , 28 (2), 135 167. Bell, B. A., Morgan, G. B., Kromrey, J. D., & Ferron, J. M. (2010). The impact of small cluster size on multilevel models: a Monte Carlo examination of two level models with binary and continuous predictors. JSM Proceedings, Su rvey Research Methods Section, 4057 4067. Bollen, K.A. (1989), Structural Equations with Latent Variables , New York: JohnWiley & Sons, Inc. Boomsma, A. (1987). The robustness of maximum likelihood estimation in structural equation models. In P. Cuttance & R. Ecob (Eds.), Structural modeling by example (pp. 160 188). New York: University of Cambridge. Bosker, R., & Snijders, T. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. New York . Busing, F. M. (1993). Distribution characteristics of variance estimates in two level models: a Monte Carlo study . Department of Psychology, University of Leiden. Campbell, L., & Kashy, D. A. (2002). Estimating actor, partner, and interaction effects for dyadic data using PROC MIXED and HLM: A user friendly guide. Personal Relationships , 9 (3), 327 342.
69 Clarke, P., & Wheaton, B. (2007). Addressing data sparseness in contextual pop ulation research using cluster analysis to create synthetic neighborhoods. Sociological methods & research , 35 (3), 311 351. Clarke, P. (2008). When can group level clustering be ignored? Multilevel models versus single level models with sparse data. Journa l of Epidemiology and Community Health , 62 (8), 752 758. Cook, W. L. (1994). A structural equation model of dyadic relationships within the family system. Journal of Consulting and Clinical Psychology , 62 (3), 500. Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological M ethods , 1 (1), 16. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal statistical Society , 39 (1), 1 38. Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The Leadership Quarterly , 16 (1), 149 167. E liason, S. R. (1993). Maximum likelihood estimation: Logic and practice (No. 96). Sage. Gaudreau, P., Fecteau, M. C., & Perreault, S. (2010). Multi level modeling of dyadic data in sport sciences: Conceptual, statistical, and practical issues. Measurement in Physical Education and Exercise Science , 14 (1), 29 50. Geldhof, G. J., Preach er, K. J., & Zyphur, M. J. (201 4 ). Reliability Estimation in a Multilevel Confirmatory Factor Analysis Framework. Goldstein, H. (1995). Multilevel statistical models (2nd ed.). New York: Wiley & Sons. Goldstein, H. (2003). Multilevel statistical models . New York: Halsted. Goldstein, H. (2011). Multilevel statistical models (Vol. 922). John Wiley & Sons. Gonzalez, R., & Griffin, D. (1999). The correlational analysis of dyad level data in the distinguishable case. Personal Relationships , 6 (4), 449 469. Gonzalez, R., & Griffin, D. (2001). Testing parameters in structural equation modeling: every" one" matters. Psychological Methods , 6 (3), 258. Griffin, D., & Gonzalez, R. (1995). Correlational analysis of dyad level data in the exchangeable case. Psychological Bulletin , 118 (3), 430.
70 Gulliford, M. C., Ukoumunne, O. C., & Chinn, S. (1999). Components of variance and intraclass correlations for the design of community based surveys a nd intervention studies: data from the Health Survey for England 1994. American Journal of Epidemiology , 149 (9), 876 883. Hartley, H. O., & Rao, J. N. (1967). Maximum likelihood estimation for the mixed analysis of variance model. Biometrika , 54 (1 2), 93 10 8. Heck, R. H., & Thomas, S. L. (2008). An introduction to multilevel modeling techniques (2nd ed.). New York: Routledge. Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling. An overview and a meta analysis. Sociologi cal Methods & Research , 26 (3), 329 367. Hox, J. J. (1995). Applied multilevel analysis (2nd ed.). Amsterdam, Netherlands: TT Publikaties. Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar & M.Schader (Eds.). Classification, data analysis, and data highways (pp. 147 154). New York: Springer Verlag. Hox, J. J., & Maas, C. J. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural E quation M odeling , 8 (2), 157 174. Hox, J. J. (2002). Multilevel analysis: Techniques and applications . Psychology Press. Hox, J. J., Maas, C. J., & Brinkhuis, M. J. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica , 64 (2), 157 170. Hox, J. J., V an de Schoot, R., & Matthijsse, S. (2012, January). How few countries will do? Comparative survey analysis from a Bayesian perspective. In Survey Research Methods (Vol. 6, No. 2, pp. 87 93). European Survey Research Association. Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling , 8 (3), 325 352. JÃ¶reskog, K. G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika , 32 (4), 443 482. JÃ¶reskog, K. G., & SÃ¶rbom, D. (1996). LISREL 8 user's reference guide . Scientific Software International.
71 Kaplan, D. (1989). A study of the sampling variability and z values of parameter estimates from missp ecified structural equation models. Multivariate Behavioral Research , 24 (1), 41 57. Kaplan, D. (2008). Structural equation modeling: Foundations and extensions (Vol. 10). Sage. Kenny, D. A. (1996). Models of non independence in dyadic research. Journal of Social and Personal Relationships , 13 (2), 279 294. Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of P ersonality and S ocial P sychology , 83 (1), 126. Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis . Guilford Press. Kish, L. 1965. Survey Sampling . New York: John Wiley and Sons. Kline, R. B. (2011). Principles and practice of structural equation modeling . Guilford press. Kreft, I. G. (1996). Are multile vel techniques necessary? An overview, including simulation studies. Unpublished manuscript, California State University, Los Angeles . Kreft, I. G., & De Leeuw, J. (1998). Introducing multilevel modeling . Sage. Kurdek, L. A. (2003). Methodological issues i n growth curve analyses with married couples. Personal Relationships , 10 (2), 235 266. Loehlin, J. C. (2013). Latent variable models: An introduction to factor, path, and structural equation analysis . Psychology Press. Longford, N. T. (1987). A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects. Biometrika , 74 (4), 817 827. LÃ¼dtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & MuthÃ©n, B. (2008). The multilevel latent covariate mode l: a new, more reliable approach to group level effects in contextual studies. Psychological Methods , 13 (3), 203. Maas, C. J., & Hox, J. J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica , 58 (2), 127 137. Maas, C. J., & H ox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences , 1 (3), 86 92.
72 McMahon, J. M., Pouget, E. R., & Tortu, S. (2006). A guide for multilevel modeling of d yadic data with binary outcomes using SAS PROC NLMIXED. Computational S tatistics & D ata A nalysis , 50 (12), 3663 3680. Mehta, P. D., & Neale, M. C. (2005). People are variables too: multilevel structural equations modeling. Psychological methods , 10 (3), 259. Meuleman, B., & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are neede d for accurate multilevel SEM? In Survey Research Methods (Vol. 3, No. 1, pp. 45 58). Mok, M. (1995). Sample size requirements for 2 level designs in education al research. Unpublished manuscript, Multilevel Models Project, Institute of Education, University of London. MuthÃ©n, B. O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika , 54 (4), 557 585. MuthÃ©n, B. O., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. Multilevel A nalysis o f E ducational D ata , 87 99. MuthÃ©n, B. O. (1990). Mean and covariance structure analysis of hierarchical data. UCLA Statistics Series , 62 . MuthÃ©n, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement , 28 (4), 338 354. Muthen, B. O. (1994). Multilevel covariance structure analysis. Sociological M ethods & R esearch , 22 (3), 376 398. MuthÃ©n, B. O. (2004, October 12, 2004 5:04 pm). Intraclass correlations [Web log post]. Message posted to http://www.statmodel.com/discussion/messages/12/18.html MuthÃ©n, B. O. , & Asparouhov, T. (2009). Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework. In Handbook of Advanced Multilevel Analysis . New York, NY: Routledge. MuthÃ©n, L. K., & MuthÃ©n, B. O. (2010). Mplus User's Guide: Stati stical Analysis with Latent Variables: User's Guide . MuthÃ©n & MuthÃ©n. Newsom, J. T. (2002). A multilevel structural equation model for dyadic data. Structural Equation Modeling , 9 (3), 431 447. Newsom, J. T., & Nishishiba, M. (2002). Nonconvergence and sampl e bias in hierarchical linear modeling of dyadic data. Unpublished manuscript, Portland State University .
73 Peugh, J. L., DiLillo, D., & Panuzio, J. (2013). Analyzing Mixed Dyadic Data Using Structural Equation Models. Structural Equation Modeling: A Multidi sciplinary Journal , 20 (2), 314 337. Rabe Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multileve l structural equation modeling. Psychometrika , 69 (2), 167 190. Raudenbush, S. W., Brennan, R. T., & Barnett, R. C. (1995). A multivariate hierarc hical model for studying psychological change within married couples. Journal of Family Psychology , 9 (2), 161. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage. Satorra, A., & Muthen, B. (1995). Complex sample data i n structural equation modeling. Sociological M ethodology , 25 , 267 316. Shackman, G. (2001). Sample size and design effect. Albany Chapter of the American Statistical Association . Snijders, T. A., & Bosker, R. J. (199 3). Standard errors and sample sizes for two level research. Journal of Educational and Behavioral Statistics , 18 (3), 237 259. Tate, R. L., & Wongbundhit, Y. (1983). Random versus nonrandom coefficient models for multilevel analysis. Journal of Educational and Behavioral Statistics , 8 (2), 103 120. Tomarken, A. J., & Waller, N. G. (2005). Structural equation modeling: Strengths, limitations, and misconceptions. Annu. Rev. Clin. Psychol. , 1 , 31 65. Van der Leeden, R., & Busing, F. M. (1994). First iteration versus final IGLS/RIGLS estimators in two level models: A Monte Carlo study with ML3 . Department of Psychology, University of Leiden. Woody, E., & Sadler, P. (2005). Structural equation models for interchangeable dyads: Being the same makes a difference. Psychological Methods, 10, 139 158 Yammarino, F. J., Dansereau, F., & Kennedy, C. J. (2001). A multiple level multidimensional approach to leadership: Viewing leadership through an elephant's eye. Organizational Dynamics . level covariance structure models. P sychometrika , 70 (1), 147 167.
74 Yuan, K. H., & Bentler, P. M. (2007). Multilevel covariance structure analysis by fitting multiple single level models. Sociological Methodology, 37 , 53 82. Zhang, D., & Willson, V. L. (2006). Comparing empirical power of multilevel structural equation models and hierarchical linear models: Understanding cross level interactions. Structural Equation Modeling , 13 (4), 615 6
75 BIOGRAPHICAL SKETCH Yu Su entered Southwestern University of Finance and Economics and majored in electronic commerce in 2008. She received her . Then, she went to graduate school in 2012 at the U niversity of Florida for degree i n the research and evaluation methodology program. She will receive her Master of Arts in Education degree in 2014 and continue pursuing her Ph.D degr ee in educational statistics at the University of Florida .