<%BANNER%>

Psychometric Properties and Rater Effects of the Computer Adaptive Measure of Functional Cognition for Traumatic Brain Injury

Permanent Link: http://ufdc.ufl.edu/UFE0024346/00001

Material Information

Title: Psychometric Properties and Rater Effects of the Computer Adaptive Measure of Functional Cognition for Traumatic Brain Injury
Physical Description: 1 online resource (141 p.)
Language: english
Creator: Wen, Pey-Shan
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: cat, irt, measurement, rehabilitation, tbi
Rehabilitation Science -- Dissertations, Academic -- UF
Genre: Rehabilitation Science thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Individuals with moderate to severe TBI often need extensive rehabilitation. To verify the effectiveness of intervention and design rehabilitation programs that meet individual s needs, precise and efficient outcome measures are crucial. Current assessments for TBI either focus on measuring impairments, such as neuropsychological tests or lack of breadth and depth, such as functional outcome measures. A comprehensive, ecological valid, precise, inexpensive, and efficient assessment for TBI is needed. Using traditional measurement technology to develop a comprehensive, precise and efficient assessment is impractical. To avoid creating gaps in precision, the ideal measure needs to encompass enough items to cover all relevant levels of examinees ability. Accordingly, a large number of items need to be answered, which compromises efficiency. However, contemporary methods, such as item response theory (IRT) and computer adaptive testing (CAT), have the ability to maximize precision and, at the same time, minimize response burden. CAT only directs the items related to respondent s ability level (most informative items) to respondent. Since only the most informative items will be direct to respondent, CAT methods provide efficient measurement without losing precision. The Computer Adaptive Measure of Functional Cognition for Traumatic Brian Injury (CAMFC-TBI) uses contemporary methods (IRT and CAT) to overcome the disadvantage of traditional measurement technology and addresses the drawbacks of current measures. The aim of this proposal is to conduct the fundamental work to move the CAMFC-TBI item pool to an item bank for a CAT application. The psychometric studies showed that the CAMFC-TBI is a sound functional outcome measurement test battery for TBI. Future studies should further confirm the structure of the CAMFC-TBI using larger sample sizes. In addition, CAT version of the test battery should accommodate rater effects.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Pey-Shan Wen.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Velozo, Craig A.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024346:00001

Permanent Link: http://ufdc.ufl.edu/UFE0024346/00001

Material Information

Title: Psychometric Properties and Rater Effects of the Computer Adaptive Measure of Functional Cognition for Traumatic Brain Injury
Physical Description: 1 online resource (141 p.)
Language: english
Creator: Wen, Pey-Shan
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: cat, irt, measurement, rehabilitation, tbi
Rehabilitation Science -- Dissertations, Academic -- UF
Genre: Rehabilitation Science thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Individuals with moderate to severe TBI often need extensive rehabilitation. To verify the effectiveness of intervention and design rehabilitation programs that meet individual s needs, precise and efficient outcome measures are crucial. Current assessments for TBI either focus on measuring impairments, such as neuropsychological tests or lack of breadth and depth, such as functional outcome measures. A comprehensive, ecological valid, precise, inexpensive, and efficient assessment for TBI is needed. Using traditional measurement technology to develop a comprehensive, precise and efficient assessment is impractical. To avoid creating gaps in precision, the ideal measure needs to encompass enough items to cover all relevant levels of examinees ability. Accordingly, a large number of items need to be answered, which compromises efficiency. However, contemporary methods, such as item response theory (IRT) and computer adaptive testing (CAT), have the ability to maximize precision and, at the same time, minimize response burden. CAT only directs the items related to respondent s ability level (most informative items) to respondent. Since only the most informative items will be direct to respondent, CAT methods provide efficient measurement without losing precision. The Computer Adaptive Measure of Functional Cognition for Traumatic Brian Injury (CAMFC-TBI) uses contemporary methods (IRT and CAT) to overcome the disadvantage of traditional measurement technology and addresses the drawbacks of current measures. The aim of this proposal is to conduct the fundamental work to move the CAMFC-TBI item pool to an item bank for a CAT application. The psychometric studies showed that the CAMFC-TBI is a sound functional outcome measurement test battery for TBI. Future studies should further confirm the structure of the CAMFC-TBI using larger sample sizes. In addition, CAT version of the test battery should accommodate rater effects.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Pey-Shan Wen.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Velozo, Craig A.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024346:00001


This item has the following downloads:


Full Text

PAGE 1

1 PSYCHOMETRIC PROPERTIES AND RATER E FFECTS OF THE COMPUTER ADAPTIVE TEST OF FUNCTIONAL COGNITION FO R TRAUMATIC BRAIN INJURY By PEY-SHAN WEN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2009

PAGE 2

2 2009 Pey-Shan Wen

PAGE 3

3 To my family and friends

PAGE 4

4 ACKNOWLEDGMENTS First, I would like to thank m y mentor and committee members. My mentor, Dr. Velozo has been working very hard with me to make this happen. Dr. Rosenbek has been always supported and caring. Dr. Richards and Dr. H eaton provided me a lot of reference for understanding cognition. I thank them all for putting so much effort in helping me accomplish this journey. In addition, my two colleagues, Sande and Dr. Waid-ebbs have supported me 100% in all aspects. Thank you!

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ............................................................................................................... 4LIST OF TABLES ...........................................................................................................................8LIST OF FIGURES .......................................................................................................................10ABSTRACT ...................................................................................................................... .............11 CHAPTER 1 INTRODUCTION .................................................................................................................. 13Overview of Traumatic Brain Injury ...................................................................................... 13Impacts ....................................................................................................................... .....13Common Deficits of Traumatic Brain Injury .................................................................. 13Current Cognitive Assessments for Traumatic Brain Injury .................................................. 14Neuropsychological Tests ...............................................................................................15Functional Outcome Measures ........................................................................................ 16Contemporary Methods for Measurement .............................................................................. 19Classical Test Theory ......................................................................................................19Item Response Theory ..................................................................................................... 20Sample invariance .................................................................................................... 20Test free measurement ............................................................................................. 21Computer Adaptive Testing ............................................................................................ 22Rater Effects ...........................................................................................................................26Research Aims ........................................................................................................................272 FACTOR STRUCTURE OF THE COMPUTER ADAPTIVE MEASURE FOR FUNCTIONAL COGNITION-TRAU MATIC BRAIN INJURY .......................................... 35Introduction .................................................................................................................. ...........35Methods ..................................................................................................................................40Participants .................................................................................................................. ....40Instruments ................................................................................................................... ...41Data Analysis ...................................................................................................................41Results .....................................................................................................................................43Research Question 1 ........................................................................................................43Research Question 2 ........................................................................................................44Research Question 3 ........................................................................................................44Discussion .................................................................................................................... ...........483 PSYCHOMETRIC PROPERTIES OF THE CO MPUTER ADAPTIVE MEASURE OF FUNCTIONAL COGNITI ON FOR TRAUMATIC BRAIN INJURY ................................. 74

PAGE 6

6 Introduction .................................................................................................................. ...........74Methods ..................................................................................................................................77Participants .................................................................................................................. ....77Instruments ................................................................................................................... ...77Data Analysis ...................................................................................................................78Differential item functioning (DIF) analysis ............................................................ 79The item level psychometric properties ................................................................... 79Results .....................................................................................................................................80Differential Item Functioning (DIF) Analysis ................................................................. 80The Item Level Psychometric Properties ........................................................................81Discussion .................................................................................................................... ...........82Item Misfit ................................................................................................................... ....83Item Reliability and Separation ....................................................................................... 83Person Spread ..................................................................................................................84Person Misfit ................................................................................................................. ..84Person Reliability and Separation ...................................................................................85Unidimensionality ........................................................................................................... 854 RATER EFFECTS ON THE COMPU TER ADAPTIVE MEASUR E OF FUNCTIONAL COGNITION-TBI ........................................................................................ 91Introduction .................................................................................................................. ...........91Methods ..................................................................................................................................96Participants .................................................................................................................. ....96Raters ...............................................................................................................................97Instruments ................................................................................................................... ...97Data Analysis ...................................................................................................................98Hypothesis 1 ............................................................................................................. 98Hypothesis 2 ............................................................................................................. 98Hypothesis 3 ............................................................................................................. 99Results ...................................................................................................................................100Hypothesis 1 .................................................................................................................. 101Hypothesis 2 .................................................................................................................. 101Hypothesis 3 .................................................................................................................. 102Discussion .................................................................................................................... .........1035 DISCUSSION .................................................................................................................... ...113APPENDIX A THE CAMFC-TBI ITEMS, DOM AINS, AND SUB-DOMAINS ....................................... 118B ITEMS WITH MORE THAN 50% NOT APPLICABLE RESPONSE ...........................125C MISFITTING ITMES OF THE COMPU TER ADAPTIVE MEASURE OF FUNCTIONAL COGNI TI ON FOR TRAUMATIC BRAIN INJURY ............................... 126D FACETS RULER .................................................................................................................129

PAGE 7

7 LIST OF REFERENCES .............................................................................................................134BIOGRAPHICAL SKETCH .......................................................................................................141

PAGE 8

8 LIST OF TABLES Table page 1-1 Common neuropsychological tests .................................................................................... 28 1-2 Commonly used functional outcome measures for TBI .................................................... 29 1-3 Ten-step guideline to develop an item bank for CAT ........................................................ 32 2-1 The CAMFC-TBI domains, sub-domain s structure and example of items ....................... 55 2-2 Number of participants across recovery stages .................................................................. 55 2-3 Demographic statistics fo r patients and caregivers ............................................................ 56 2-4 Fit indices for the general cognition .................................................................................. 57 2-5 Pearson correlations for patients data ............................................................................... 57 2-6 Pearson correlations for caregivers data ........................................................................... 58 2-7 Confirmatory Factor Analysis Results ............................................................................... 59 2-8 Confirmatory Factor Analysis Results ............................................................................... 59 2-9 Number of factors meeti ng criteria for each domain .........................................................60 2-10 Rotated factor pattern of the at tention doma in for patient data ......................................... 61 2-11 Rotated factor pattern of the at tention doma in for caregiver data .....................................62 2-12 Rotated factor pattern of the me mory for patient data ....................................................... 63 2-13 Rotated factor pattern of the me mory dom ain for caregiver data ...................................... 64 2-14 Factor pattern of the processing speed doma in for patient data ......................................... 65 2-15 Factor pattern of the processing speed doma in for caregiver data .....................................66 2-16 Rotated factor pattern of the execu tive function doma in for patient data .......................... 67 2-17 Rotated factor pattern of the executiv e function doma in for caregiver data ...................... 68 2-18 Factor pattern of the emotional ma nagem ent domain for patient data .............................. 70 2-19 Factor pattern of the emotional ma nagem ent domain for caregiver data .......................... 70 2-20 Rotated factor pattern of the social communication dom ain for patient data .................... 71

PAGE 9

9 2-21 Rotated factor pattern of the social communication dom ain fo r caregiver data ................ 72 2-22 Summary of the number of Cross loaded and not loaded items ........................................ 73 3-1 Number of participants across recovery stages .................................................................. 87 3-2 Demographics statistics fo r patients and caregivers .......................................................... 87 3-3 Demographics statistic fo r healthcare professionals .......................................................... 88 3-4 Adjusted P-values for each domain ................................................................................... 89 3-5 Summary results of Rasch analysis ....................................................................................90 4-1 Number of participants across recovery stages ................................................................ 108 4-2 Demographics statistics fo r patients and caregivers ........................................................ 108 4-3 Demographics statistic fo r healthcare professionals ........................................................ 109 4-4 Rater measure report ...................................................................................................... ..110 4-5 Significant rater effects ................................................................................................. ...111

PAGE 10

10 LIST OF FIGURES Figure page 1-1 Item characteristic curve. As the re spondents estim ated ability increases, the probability of answering the it em correctly is increased. .................................................. 34 4-1 Facets ruler of the attention domain ................................................................................. 112

PAGE 11

11 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy PSYCHOMETRIC PROPERTIES AND RATER E FFECTS OF THE COMPUTER ADAPTIVE MEASURE OF FUNCTIONAL COGNITION FOR TRAUMATIC BRAIN INJURY By Pey-Shan Wen August 2009 Chair: Craig Velozo Major: Rehabilitation Science Individuals with moderate to se vere TBI often need extensive rehabilitation. To verify the effectiveness of intervention and design rehabilitation programs that meet i ndividuals needs, precise and efficient outcome measures are crucia l. Current assessments for TBI either focus on measuring impairments, such as neuropsychological tests or lack of breadth and depth, such as functional outcome measures. A comprehensive, ecological valid, precise, inexpensive, and efficient assessment for TBI is needed. Using traditional measurement technology to develop a comprehensive, precise and efficient assessment is impractical. To avoid cr eating gaps in precision, the ideal measure needs to encompass enough items to cover all relevant levels of examinees ability. Accordingly, a large number of items need to be answered, which compromises efficiency. However, contemporary methods, such as item response theory (IRT) and computer adaptive testing (CAT), have the ability to maximize precision an d, at the same time, minimize response burden. CAT only directs the items related to respondent s ability level (most informative items) to respondent. Since only the most informative items will be direct to respondent, CAT methods provide efficient measurement without losing precision. The Computer Adaptive Measure of Functional Cognition for Traumatic Brian Injury (CAMFC-TB I) uses contemporary methods

PAGE 12

12 (IRT and CAT) to overcome the disadvantage of traditional measurement technology and addresses the drawbacks of current measures. The aim of this proposal is to conduct the fundamental work to move the CAMFC-TBI item pool to an item bank for a CAT application. The psychometric studies showed that th e CAMFC-TBI is a sound functional outcome measurement test battery for TBI. Future studies should further confirm the structure of the CAMFC-TBI using larger sample sizes. In addition, CAT version of th e test battery should accommodate rater effects.

PAGE 13

13 CHAPTER 1 INTRODUCTION Overview of Traumatic Brain Injury Impacts Trauma tic brain injury (TBI) is caused by ex ternal forces, such as a blow, jolt or penetration to the head temporarily or permanently disrupting the brain function (http://www.cdc.gov/ncipc/tbi/TBI.htm). Because of advanced medical care, the survival rate of patients with TBI continues to increase. However, survivors with TBI ofte n experience long-term disability (Moscato, Trevisan, & Willer, 1994). Each year in the United States, 1.4 million individuals sustain TBIs and 235,000 of those require hospitalization (http://www.cdc.gov/ncipc/tbi/TBI.h tm). The estimated direct cost, such as medical spending, and indirect cost, such as loss of productivity, for TBI was $60 b illion in the United States in 2000 (Finkelstein, Corso, & Miller, 2006). The lo ng-term costs of TBI is higher than other causes of brain injury because TBI often occurs in younger popul ations and the TBI survivors usually have a long life expectancy (van Baalen et al., 2003). The lifetime cost for one individual with severe TBI can reach $4 million (National In stitute of Neurological Disorder and Stroke 1989). Common Deficits of Traumatic Brain Injury The extent of the deficits following TBI is broad. Individuals w ith mild TBI might experience slowed information processing, verbal retrieval problems and postconcussional syndrome such as headache, fati gue, anxiety, irritability and sl eep disturbance. In contrast, individuals with severe TBI might have deficits in every aspect of functioning, such as physical deficits, cognitive deficits, emotional, and beha vioral problems (Hellawell, Taylor, & Pentland, 1999; van Baalen et al., 2003). Cognitive deficits are the most concerning problem to patients

PAGE 14

14 with moderate to severe TBI and their caregivers (Brooks, Campsie, Symington, Beattie, & McKinlay, 1986; Hellawell et al., 1999). The most common cognitive deficits following severe TBI are related to attention, memory, executive function, emotion and social participation (Johnston & Hall, 1994; Silver, McAllister, & Yudofsky, 2005). Individual with moderate to severe TBI often need extensive rehabilitation programs (Lezak, 2004). For cognitive rehabilitation programs, the primary concern of patients and their caregivers is the improvement of cognitive act ivities (Stuss, Winocur, & Robertson, 2008). Rehabilitation programs cannot meet an individual s need without precise measures (Johnston & Hall, 1994). Along with the extended rehabilitation services, from acute hospital to community, outcome measures need to go beyond assessing impairments to address activity limitations and participation restriction (C ohen & Marino, 2000). In addition, Johnston, Keith, and Hinderer (1992) stressed the importance of assessing the cognitive impact on everyday life in TBI population (Johnston, Keith, & Hinder er, 1992). Yet, the measurements for the most important outcome of cognitive rehabilita tion, cognitive activities, are poorly developed (Stuss et al., 2008). Current Cognitive Assessments for Traumatic Brain Injury Since the heterogeneity of TBI symp toms a nd prolonged recovery process, assessing TBI outcomes is extremely challenging. A literature search was conducted using PsycInfo and PubMed electronic databases to review common ly used cognitive assessments in TBI. The key words cognitive outcomes after traumatic brain injury were used to search potential studies. In addition, the searches were limited to adults, -2008 and sample sizes larger than 10. A total of 63 studies were retained. One hundr ed and fourteen neuropsychological tests and 44 functional outcomes were used in these studies. Our finding of wide varieties of assessments used in TBI is consistent with Jette and Haleys conclusions that the choi ce of scale is based on

PAGE 15

15 practical considerations and that there is a lack of consensu s on the best measures for rehabilitation outcomes (Jette & Haley, 2005). Current cognitive assessments can be categorized into neuropsychological tests and functional outcome measures. Neuropsychological Tests As evident in our review, there are ma ny neur opsychological tests used in TBI (Table 1-1). The advantages of neuropsychological tests are their strong psychometric properties, objectivity, and capacity measurement. Traditional neurops ychological tests are pe rformance tests with established normative data and sound psychomet ric properties. They are administrated by licensed neuropsychologists following standard testing procedures under well-controlled environments. By using trained examiners, sta ndard procedures and a controlled environment, neuropsychological tests have a dvantages over other types of tests in terms of objectivity. Additionally, neuropsychological tests were developed based on cognitive models to measure impairment of cognition. Neuropsyc hological tests can be used to detect impaired functioning of brain structures and cognitive models can be used to provide causal paths to investigate the potential compromised cogniti on. Moreover, since neuropsychol ogical tests eliminate the environmental barriers/facilitators during the test, they assess the pure ca pacity of individuals under a controlled environment. Through knowing the damaged component and the capacity of individuals, therapists can develop intervention focusing on the missing component and also predict potential problems that the individual may experience when his/her environment changes (Cohen & Marino, 2000). While neuropsychological tests measure patien ts limitations and stre ngths of capacity, patients have been found to improve on daily functional outcomes without change in neuropsychological tests (Cope 1995; Karyl M. Hall & Cope, 1995; Tupper & Cicerone, 1990).

PAGE 16

16 Without accounting for mood or behavior disturban ces or consequences occurring in the real world, neuropsychological tests cannot exactly refl ect the problem of real-life activities (Hart et al., 2003). In addition, with the repetitive instru ctions, frequent breaks, and non-distractible environment, neuropsychological tests automatically provide compensations to the examinees, especially for the individuals with mild TBI (Sbordone, Seyranian, & Ruff, 1998). Thus, individuals with mild TBI ma y perform within normal range on objective neuropsychological tests, but still have c ognitive challenges in re al life (Lezak, 1989). Ther efore, neuropsychological tests are often challenged for their limitation in iden tifying patients difficulties in the real-world. Ecological validity is defined as the relationship between patients performance on neuropsychological tests and his/ her behavior in naturalistic e nvironments (Tupper & Cicerone, 1990). The ecological validity issue of neuropsychologi cal tests is often unde r debate (Chaytor & Schmitter-Edgecombe, 2003; Chaytor, Temk in, Machamer, & Dikmen, 2007; Heaton & Pendleton, 1981; Sbordone & Guilmette, 1999; Sbordone, 2001). While several neuropsychological tests show significant correlations with everyday functioning (Chaytor & Schmitter-Edgecombe, 2003, 2007), many lack an association with everyday function. (Sbordone & Guilmette, 1999; Sbordone, 2001). Another critique of neuropsychological tests are that they are time consuming and expensive. Depending on the testing batteries, ne uropsychological tests can take from one hour up to a full day ( https://www.ucsfhealth.org/childrens/medical_servi ces /neuro/sclerosis/ne uropsychological_testi ng.html). For an hourly fee of $175, a full ne uropsychological evaluation can cost $3,000 (http://www.willowtrees.org/neuro_assessmnt.php). Functional Outcome Measures Since surveys of TBI rehabilitation providers and payers have shown that functional independence was rated as the most important objective of rehabi lita tion (K. M. Hall &

PAGE 17

17 Johnston, 1994), measures for TBI need to be capable of monitoring functional outcomes. The commonly used functional outcome measures for TBI includes the Glasgow Outcome Scale (GOS) (Jennett & Bond, 1975), the Functional Independence Measure (FIM) (Hamilton, Granger, Sherwin, Zielezny, & Ta shman, 1987), the Functional A ssessment Measure (FAM), the Disability Rating Scale (DRS) (Rappaport, Hall, Hopkins, Belleza, & Cope, 1982), and the Community Integration Questi onnaire (CIQ) (Willer, Rosentha l, Kreutzer, & Gordon, 1993) (Table 1-2). These functional outcome measures are short, easy to administrate, and are ecologically valid. The selection of outcome measures is mainly based on practicality (Stuss et al., 2008). Long and taxing outcome measures, even with good reliabilit y and validity, are not feasible in clinical settings. In addition, these measures are often observational instruments most rated by clinicians as they obser ve and evaluate indi viduals with TBI in the healthcare setting, and some rated by caregivers or patients themse lves based on their observations in real life. Therefore, in contrast to neuropsychological tests, functional outcome measur es tend to be short, easy to administer and have at least face ecological validity. On the other hand, the disadvantages of func tional outcome measures are not negligible. Functional outcome used in TBI often are not comprehensive, have poor sensitivity, and are tainted by observer bias. These commonly used functional outcome measures are often global measures. In general, physical function is often the priority focus for re habilitation. Therefore, these functional outcome measures consist of few items addressing cognition. Because cognitive deficits are heterogeneous in individuals with TBI, using these global functional outcome measures do not comprehensively identify the cognitive deficits in TBI. For example, the GOS categorizes abilities into five stages from dead to good recovery using only five statements. No item specifically measures cogniti on. Although the GOS-E extends th e GOS from five stages to

PAGE 18

18 eight stages, only its first ques tion obey simple commands is directly related to cognition. Similarly, only two out of 18 items on the FIM, 3 out of 12 items on the FAM, 3 out of 8 on the DRS and 1 out of 15 on the CIQ are cognitive it ems. The scores on these functional outcome measures are heavily influenced by physical abilities. Thus, regardless of their psychometric properties, these functional outcome measures may be sufficient to assess function in general, but not functional cognition, which is defined as the ability to complete everyday activities that require primarily cognitive abilities (Donovan et al., 2008). Moreover, these functional outcome measures are often designed to asse ss certain recovery stages of TB I or certain severity levels of TBI. Applying these measures across all recovery stages and all severity levels results in poor sensitivity and ceiling effects. For example, b ecause the FIM and the FAM were developed for inpatient rehabilitation. When applying the FIM and FAM to i ndividuals with moderate and severe TBI at one-year post injury, these instrume nts showed substantial ce iling effects; one third of individuals with TBI reached maximal score (K. M. Hall et al., 1996). The DRS also showed ceiling effects at one-year post injury (47%) (K. M. Hall, Hamilton, Gordon, & Zasler, 1993). In addition, two of the CIQ sub-domains showed cei ling effects when assess ing individuals with TBI in the community (K. M. Hall et al., 1996). Furthermore, Sohlberg and Mateer (2001) stressed that these measures are particularly venerable to observer bias (Sohlberg & Mateer, 2001). Self and proxy report on cognitive domains often show discrepancy of the ratings especially on TBI. Biases are caused by impair ed self awareness of individuals with TBI, caregivers burden, and clinicians heuristics (see Chapter 4 for details). In summary, rehabilitation services need comprehensive, e fficient, and psychometrically sound measures to monitor functional outcomes reflecting a patient s real-world cognitive abilities for TBI. Furthermore, the rater bias of these measures need to be addressed.

PAGE 19

19 Contemporary Methods for Measurement While functional outcome measures are need ed for monitoring the most important objective functional independence of individuals with TBI, as we reviewed above, these measures have serious limitations. Functional ou tcome assessments in rehabilitation face the dilemma between precision and efficiency (Jette & Haley, 2005). The ideal measure needs to encompass enough items to cover all relevant levels of the examinees abi lity and eliminate gaps that compromise precision (Jette & Haley, 2005). Th erefore, technically, a large number of items are needed to cover the entire spectrum of daily-life challenges Consequently, lengthy measures result in respondent burden and are not feasib le in busy clinical settings. Traditional measurement technology cannot resolve this dilemma. As a result, precision is often compromised by administrating short-length meas urements. Creating an ideal measure using traditional measurement technology is impractical (Jette & Haley, 2005). To solve these problems, in the past three decades the health-related fields have explored contemporary methods, such as item response theory (IRT), to evaluate existing outcome measures and develop new outcome measures. Classical Test Theory To understand the advantage of applying IRT to measures, first we need to know the basic concepts of class ical test theory (CTT). The CTT assumes a simple linear relationship between observed score and true score (the expected score): X=T+E, wher e X represents observed score; T represents true score; and E re presents error score. The simplic ity of the CTT model results in several limitations. First, all items on a test are tr eated with equivalent it em properties, since item parameters are not included in the CTT model. Accordingly, CTT provides only one reliability estimate and a standard error of measurement even though the precision of measurement should vary across the breath of the measure (i.e., meas ures are often less precise at their extremes)

PAGE 20

20 (Hays, Morales, & Reise, 2000). Moreover, withou t including item parameters in the model, item properties cannot link to the examinees ability. It em properties, such as item difficulty, item discrimination, and reliability need to be comput ed outside the CTT model based on data from a sample. Therefore, these item properties are sample dependent, that is, a measurement has varied reliability, item difficulty and item discriminatio n when applied to different samples (Embretson & Reise, 2000). More detail discussions are in the section sample invariance. Item Response Theory Item response theory (IRT) is the m athematic model that presents how examinees with different levels of abilities respond to an item. Two basic concepts of IR T are latent traits and item characteristic curve (ICC). The major charac teristic distinguishing IRT from CTT is the assumption of latent traits, which account for th e variance of response pa tterns. In fact, most applications assume that only one latent tr ait accounts for the response patterns (Crocker & Algina, 1986). The ICC, with latent trait on the x-axis and the probability of a correct response to the item on the y-axis, connects respondents pr obability of a correct response to his/her estimated ability (latent ability) on the item (Figure 1-1). The ICC demonstrates how the probability of a correct response depends on the latent trait. The most distinguished advantages of a pplying IRT in measurement are (1) sample invariance; and (2) test free measurement (C rocker & Algina, 1986; Fan, 1998; Velozo, Wang, Lehman, & Wang, 2008). Sample invariance In CTT, item difficulty is defined as the succ ess rate of a particular pool of exam inees on an item. Item discrimination is calculated by Pearson correlation coefficient between the score of the item and the score on the to tal test. Thus, these indices are strongly affected by the selected sample. For example, two groups answer item A. Group one with higher abilities will have

PAGE 21

21 higher success rate of answering item A while group two with lower abilities will have lower success rate of answering item A. Therefore, th e item difficulty computed from group one will be relatively lower than that computed from group two. Item discrimination, the correlation between the scores of the item and the scores on the total test, will be different between these two groups as well. Therefore, these two sample s will produce different item difficulties and different item discriminations for item A. Th e varied item properties across samples reflect sample dependency of CTT. In contrast, item diff iculty of IRT is the value of latent ability ( ) corresponding to the point where the examinee with this latent ability has 50% chance of answering the item correctly. Item discrimination in IRT is the slope of the tangent line of the ICC at the point where examinee with latent ability has 50% chance to answer the item correctly (see line in Figure 1-1). Computing item parameters (i.e., item difficulty and item discrimination) is based on the ICC and latent trait, which ar e both invariant no matter what sample was used. Therefore, item parameters are sample invariant in IRT. Test free measurement In CTT, when the examinees tak e two different tests, they will receive higher scores on the easier test but lower scores on the harder test. In CTT, therefore, person statistics change depended on what tests are taken. Test dependence is the other crucial limitation or CTT. In IRT, on the other hand, persons ability is scaled on the invariant latent trait, no matter which set of the items is used to assess their abilities. Theref ore, IRT is test free measurement (Crocker & Algina, 1986). For example, 1, 2, 3, and 4 represent minimal latent trait scores to correctly respond to item 1, item 2, item 3 and item 4, respectively. Based on the response patterns obtained for these 4 items, the respective ICCs show that 1 is easier than 2, and 2 is easier than 3, and 3 is easier than 4. If examinee As latent ability falls between 1and 2, and examinee Bs latent ability falls between 3 and 4, we can conclude that examinee B has higher

PAGE 22

22 ability than examinee A even though they did not answer the same questions (Crocker & Algina, 1986). Moreover, since the item difficulty and pers on ability are defined on the same scale, the items that are way too difficult or too easy relati ve to respondents ability can be skipped. The items are optimally selected to provide the most information of examinees ability. Therefore, using IRT as a foundation to tailo r items based on individuals ability can reduce number of items that examinees need to answer without sacrificing the precision of the measure. The precision and efficiency dilemma of traditional outcome measures can be solved by using IRT. Several models of IRT have been proposed, from one parameter logistic model (1-PL), two-parameter logistic model (2-PL), three-parameter logistic model (3-PL) to more complicated multi-demensional IRT models. Detailed discussion of these models is beyond the scope of this dissertation. However, the basic concepts of 1-PL, 2-PL and 3PL should be noted: 1-PL only includes an item difficulty parameter; 2-PL c onsists of both an item difficulty and an item discrimination parameters; 3-PL incorporates a guessing parameter besides item difficulty and item discrimination parameters. As the model ge tting more complicated, more parameters need to be estimated. Consequently, larger sample sizes are required to stabilize the estimations. Rasch model, a 1-PL model is the most parsimon ious IRT model. Because of its simplicity and ease of interpretation, th e Rasch model is preferred in the rehabilitation field (Jette & Haley, 2005). Even with small sample sizes (n=25), the Rasch model still maintains its robust estimation (de Gruijter, 1986). Therefore, the Rasc h rating scale model is the most appropriate model for applying this primary analysis. Chap ter 3 provides more information on the Rasch model. Computer Adaptive Testing Computer adaptive testing (CAT) is one of th e most im portant applications of IRT (Cook, O'Malley, & Roddey, 2005). CAT met hodologies provide a computer algorithm to select the

PAGE 23

23 items tailored to the un ique ability level of the respondent (Wainer, Dorans, & Eignor, 2000). First, IRT methods estimate item difficulties of a pool of items and scale them into a linear continuum where items are ordered based on their difficulty hierarchy. Then a CAT algorithm directs the first item to the respondent according to the starting rule. Based on the respondents answer, the computer selects the next item that provides the most information to estimate the respondents ability. One method of identifying the most informative item is to choose the item with a difficulty level close to respondents abi lity level. In other words, the most informative items can be identified as those that the individual has a 50% of chance of endorsing (e.g., answering yes/no or being correct/incorrect). Since only the most informative items will be directed to the respondent, thos e extreme difficult/easy items rela tive to respondents estimated abilities will be filtered out. By directing only th e most informative items to an individual, the CAT methods attain efficiency without losing precision. In health related outcome studies, simu lation CAT studies demonstrated empirical evidence that CAT was accurate, precise and effi cient (Haley, Coster, Andres, Kosinski, & Ni, 2004; Haley, Ni, Hambleton, Slavin, & Jette, 200 6; Ware, Gandek, Sinclair, & Bjorner, 2005). In terms of accuracy, studies reported that the simula ted CAT scores and the full scale scores were highly correlated (Cook et al., 2007 ; Haley, Coster, Andres, Kosins ki et al., 2004; Haley, Ni et al., 2006; Ware et al., 2005). Prospective CAT studi es (using real CAT scor es) reported that the correlation of the CAT score and the fixed length que stionnaire scores were slightly lower than those found in simulated CAT studies; however, th e correlations were still good. (Haley, Siebens et al., 2006; Haley et al., 2008; Wa re et al., 2005). Moreover, st udy have compared the precision of CAT to random item selection. A 10-item CAT was more precise than a 10-item random selection test. The 10-item random selection met hod had over 1.5 times the standard error of the

PAGE 24

24 CAT (Haley, Ni et al., 2006). Ha ley et al. (2006) showed that CAT reduced administrating time to almost 60% and required only two third of it ems; In a more recent study, Haley and colleagues (2008) reported that CAT reduced half of the ad ministrating time and half number of items. In addition, because IRT models are used to scale items into a latent tra it, the CAT application exhibits test free measurement characteristics. That is, even though different sets of items are directed to respondents, their estimated abil ities are still comparable. In summary, CAT applications of IRT create precise, efficient and comparable m easures (Revicki & Cella, 1997; Ware, Bjorner, & Kosinski, 2000). The intention of developing the Computer Adaptive Measure of Functional Cognition for Traumatic Brian Injury (CAMFC-TBI) is to resolv e the limitations of current measures used in TBI by using contemporary measurement methods: IRT and CAT. The goal of the CAMFC-TBI development is to build an assessment that is ecological valid, precise, efficient, broad in scope, and adequate for assessi ng all recovery stages of TBI. The items of the CMAFC-TBI were generated in corporating three areas: neuropsychological domains, real world functioni ng, and TBI recovery stages. Th e domains of the CAMFC-TBI represent the most frequently c ited neuropsychological domains of TB I that are relevant to daily cognitive functioning and applicable to recovery stages. In addition, the activity and participation components of the International Classification of Functioning Disa bility and Health model were used as a guideline to design th e items reflecting real world func tioning Based on the literature review our research team constructed four do mains for functional cogn ition attention, memory, processing speed and executive function. We genera ted new items or modified the existing items from other assessments based on this 4-domain structure. Items were generated based on the trajectory of recovery for each domain. Experts were then invited to an advisory panel meeting to

PAGE 25

25 evaluate the prototype of the it em pool. Based on the suggestions from advisory panel, we expanded our structure from four to six domains attention, memory, processing speed and executive function, emotional management and so cial communication. Then a series of focus groups were conducted to evaluate the comple tion, comprehension and clarity of items. The research team then revised items based on the suggestions from focus groups. Finally, an item pool with 269 items across si x cognitive domains (attention, memory, processing speed, executive functioning, social co mmunication, and emotional management) across four recovery stages (acute, inpatient, outpa tient, and 1-year post injury ) was produced. A total 228 items across six cognitive domains and three recovery stages (inpatient, outpatient, and 1-year post injury) were used in field testing. It should be noted that the acute items were not included in field testing. Figure 1-2 presents the item development flow chart for the CAMFC-TBI. Cella, Gershon, Lai, and Choi (2007) proposed a 10-step guideline to develop an item bank without existing datasets for CAT. Table 1-3 shows the detail descriptions of each step and the related process of the CAMFC-TB I development. All steps and th e status of those steps are presented, but only the shaded steps are the focus of the present series of studies; steps 6, 7, and 8. Step 6 data analysis was to examine unidimensionality, item fit and item difficulties. As mentioned above, the domain structure of th e CAMFC-TBI was derived from the six domains that have been identified by neuropsychological literature to be most relevant to TBI. Unidimensionality of each domain was addresse d before moving toward CAT (Haley et al., 2006). Step 7, item equivalence, was to investig ate the item parameter equivalence across subgroups. In order to create unbias ed measurements, the definition of the domain, and the meaning of each item should be the consistent across all respondents regardless of their sociodemographic

PAGE 26

26 or clinical characteristics (Hahn, Cella, Bode Gershon, & Lai, 2006). In the other words, items should function the same way across different su b-groups (Bjorner, Chang, Thissen, & Reeve, 2007). In our study, item calibrations should stay the same for outpa tient and 1-year post injured participants. Step 8 operational item bank was to review the analyses of steps 6 and 7, and find solutions prior to CAT programming. For exampl e, if differential item functioning (DIF) of items was found between outpatient and 1-year post injured participants, these items could be removed from the item bank or the CAT could be designed to have differe nt item calibrations depending on whether an outpati ent or 1-year post patient was administered the CAT). Rater Effects Another issue of functional outcome measures is their subjectivity when rated by different raters. Rater effects have been reported in a number of observati onal measures (Cusick, Gerhart, & Mellick, 2000; Goldstein & McCue, 1995; Hart et al., 2003; Hendryx, 1989; Leathem, Murphy, & Flett, 1998; Malec, 2004; Sander et al., 1997; Tepper, Beatty, & DeJong, 1996). Rater effect is defined as systematic varian ce caused by a rater who a ssigns ratings not the performance of the ratee (Scu llen, Mount, & Goff, 2000). The a dvantage of self-report is to incorporate the affected persons perspective into measures. However, a concern of using selfreport format with individuals with TBI is their impaired self-awareness (ISA) after injury (Hart et al., 2003). Because of ISA, individuals with TBI were found to have tendency to rate themselves more able than their real ability, es pecially in cognitive and emotional domains. In contrast, the advantage of proxy-re port is to obtain poten tially more reliable information and to increase sample size by obtaining data for in capable or unwilling respondents (Magaziner, Simonsick, Kashner, & Hebel, 1988). However, studies also reported ca regiver burden could potentially bias their ratings. Mo reover, the ratings of clinicians could also be influenced by heuristics, payment systems and observation opportunity. Although this systematic error (rater

PAGE 27

27 effect) does not result in inconsistent measur ement (decreased reliability), it may cause inaccurate scores and further affect the useful ness of measurement (Crocker & Algina, 1986). Therefore, to establish an accurate cognitive meas ure, we have to evaluate rater effects on the CAMFC-TBI. Research Aims In order to move the item pool used in paper and pencil format to an item bank applied in CAT, the fundam ental work needs to be establis hed: domain structure, item properties (e.g. item fit, item calibration, and item equivalence), and ot her issues affecting ite m calibrations (e.g. rater effect). Selecting IRT models (unidimensi onal versus multidimensional), managing misfitting items (delete versus revise), solving item non-equivalence (del ete DIF items versus provide separate item calibrations) and evaluating rater effects are cruc ial steps to make CAT more accurate, precise and efficient. The purpose of this dissertation is to establish the item bank psychometrics before moving to developing a com puter adaptive test vers ion of the instrument battery. Four general aims are proposed: (1) to investigate the factor structure of the newly developed CAMFC-TBI item pool; (2) to examine the item parameter equivalence across subgroups (participants from outpatient rehabilitation setting an d participants who are one-year post rehabilitation); (3) to examine the item leve l psychometric properties of the CAMFC-TBI; and (4) to examine the rater e ffects of the CAMFC-TBI.

PAGE 28

28 Table 1-1. Common neuropsychological tests 1 Domains Name of Tests Number of studies (by domain) Number of studies across domains Attention Stroop 4 7 Digit Span 4 9 TMT 5 16 Symbol Digit Modalities 3 8 Digit Symbol Coding 2 7 Memory Digit Span 5 9 RAVLT 7 7 CVLT 5 5 WMS-LM 7 7 ROCFT 5 5 Processing Speed Symbol Digit Modalities 5 8 Digit Symbol Coding 5 7 TMT 4 16 Stroop 3 7 Executive Function TMT 7 16 COWAT 8 8 WCST 7 7 TMT: Trail Making Test; RAVLT: Rey Audito ry Verbal Learning Test; CVLT: California Verbal Learning Test; WMS-LM: Wechsler Me mory Scale-Logical Memory; ROCFT: Rey Osterrieth Complex Figure Test; COWAT: Cont rolled Oral Word Association Test; WCST: Wisconsin Card Sort Test 1 The common neuropsychological tests were selected if they were used in more than five studies in our literature review. Some tests were used to assess different domains across studies. For example th e Stroop test was used to assess the attention domain in four studies and was used to assess the processing speed domain in another three studies.

PAGE 29

29 Table 1-2. Commonly used functional outcome measures for TBI Assessments Items Psychometric properties Glasgow Outcome Scale (GOS) (Jennett & Bond, 1975) -0 item measure cognition Five stages 1. Dead 2. Vegetative State: Unable to interact with environment; unresponsive 3. Severe Disability: Able to follow commands; unable to live independently 4. Moderate Disability: Able to live independently; unable to return to work or school 5. Good Recovery: Able to return to work or school -One item scale -Use to categorize the severity of TBI Glasgow Outcome ScaleExtended (GOS-E) Eight Yes/No questions 1. Obey simple commands 2. Need assistance for ADL 3. Shop without assistance 4. Travel without assistance 5. Work 6. Resume social and leisure activities 7. Family or friend disruption 8. Other current problem Eight final categories 1. Dead 2. Vegetative state 3. Lower severe disability 4. Upper severe disability 5. Lower moderate disability 6. Upper moderate disagility 7. Lower good recovery 8. Upper good recovery

PAGE 30

30 Table 1-2. Continued Assessments Items Psychometric properties Functional Independence Measure (FIM) ) (Hamilton et al., 1987) -18 items -7-level ordinal scale -2 items measure cognition: problem solving and memory Self care (Feeding, Grooming, Bathing, Dressing upper body, Dressing lower body, Toileting) Sphincter control (Bladder management, Bowel management) Mobility (Bed/chair/wheelchair, Toilet, Tub) Locomotion (Walking/wheelchair, Stairs) Communication Comprehension audio/visual Expression verbal/non-verbal Psychosocial adjustment (Social interaction) Cognitive function ( Problem solving, Memory ) The FIM has appropriate validity and interrater agreement (Hamilton, 1991). Precision during rehabilitation has been observed to be high (Granger, 1990) Acceptable content validity and construct validity ( 0.77 for MS and 0.65 for stroke) (Heinemann, 1994) Interrater agreementTotal score ICC=0.97 Subscore ICC=0.93-0.96 Kappa=0.71 (Hamilton, 1991) DrawbacksCeiling effects at one year post injury were found in the moderate and severe TBI population (Hall, 1996) Dearth cognitive behavior, communication and community related items for TBI (Hall 1993) Functional Assessment Measure (FAM) -12 items -Designed to better capture the specificity of disability of TBI patients in post discharge phase -3 items measure cognition: orientation, attention, and safety judgment Swallowing Car transfer Community access Reading Writing Speech intelligibility Emotional status Adjustment to limitation Employability Orientation Attention span Safety judgment FAM contributed significantly to overall proportion of disability. Since the FAM consists of items related to community functioning such as car transfers, employability, community mobility and adjustment to limitations, the FAM may be greater contributor to valid measure of disability during the follow up period (Marosszeky, 1992) Validity Significantly correlated with injury severity (Hall, 1993) The FAM were predictive of return to work and community integration (Gurka, 1999) Inter-rater reliabilityKappa=0.85 (Hall, 1992) Total average ICC for the FIM=0.85; for the FAM =0.83) Improve sensitivity for postactue rehabilitation functional assessment (Hall, 1993) Drawbacksthe FAM was not able to reduce the ceiling effect in those TBI survivors living in the community (K. M. Hall et al., 1996).

PAGE 31

31 Table 1-2. Continued Assessments Items Psychometric properties Disability Rating Scale (DRS) (Rappaport et al., 1982) -8 items -Summation of the scores range from 0 (no disability) to 30 (death) -3 items measure cognition 1. Eye opening (0-3) 2. Communication ability (0-4) 3. Motor response (0-5) 4. Feeding (cognitive ability only) (0-3) 5. Toileting (cognitive ability only) (0-3) 6. Grooming (cognitive ability only) (0-3) 7. Level of Functioning (physical, mental, emotional or social function) (0-5) 8. Employability (0-3) Concurrent Validity The DRS correlated 0.8 with the GOS (Jennett, 1975) and 0.85 with the GOSE (Smith, 1979). Interrater reliability (severe TBI)Pearson correlations=0.97-0.98 (Rappaport, 1982) Spearman rho=0.98 (Gouvier, 1987) Test-retest reiablitySpearman rho=0.95 (Gouvier, 1987) Sensitivity to change71% of TBI showed improvement on DRS vs 33% on the GOS between inpatient rehab admission and discharge DrawbacksGeneral not specific Poor sensitivity to extreme functional levels deficits, not suitable to mild or very severe TBI (Hall, 1993) Community Integration Questionnaire (CIQ) -15 items -Summation of the scores range from 0 (less integration) to 29 -1 item measure cognition Three sub-domains: Home integration 1. Household shopping 2. Meal preparation 3. House keep 4. Child care 5. Social planning. Social integration 6. Personal finances 7. Shopping frequency 8. Leisure activities 9. Visiting friends 10. Do leisure activities alone 11. Best friend Productivity 12. Travel 13. Work 14. School 15. Volunteer ICC for patient and significant other o.43 for home integration, 0.65 for social integration, and 0.81 for productivity (Tepper, 1996) Internal consistency Larger than 0.8 Validity Subscales and total score correlate with the CHART (Heinemann, 1995) DrawbacksHalf of participants with TBI reached a score equal or greater than the mean of control sample in Home integration and Social integration sub-domains

PAGE 32

32 Table 1-3. Ten-step guideline to develop an item bank for CAT Cella 10-Step Guideline Content Process of CAMFC TBI development Step 1. Domain determination Select th e domains related to what is intended to measure by literature review and inputs from professionals and patients Research team reviewed literature, and constructed the domain structure-attention, memory, processing speed, and executive function domains Step 2. Source availability Check whether there are relevant existing datasets. Research team review existing relevant assessments Step 3. Item generation Write new items or acquire items from existing questionnaire Research team generated new items or modified the items from existing assessment by domain across recovery stages (acute, inpatient rehabilitation, outpatient rehabilitation, and one year post injury) Step 4. Content validation Obtain feedbacks from experts or patients regarding the cont ent relevance and representativeness. Experts were invited to an advisory panel meeting to provide further suggestions. Item pool was expanded from 4 domains to 6 domains. Two extra domains were added based on experts suggestions: emotional management and social communication domains Focus groups including i ndividuals with TBI, caregivers, and healthca re professionals were conducted to modify items in terms of comprehension, completion and clarity Research team further evaluated items based on the suggestions of focus groups Step 5. Field testing Collect data for further anal ysis Paper and pencil format of CAMFC-TBI with 228 items were tested on individuals with TBI, and their caregivers from outpatient and one year post rehabilitation stages, and healthcare professionals from outpatient rehabilitation stages

PAGE 33

33 Table 1-3. Continued Cella 10-Step Guideline Content Process of CAMFC TBI development Step 6. Data analysis Examine the unidimensionality, item fit and item difficulties Factor analysis-Chapter 2 Rasch analysis-Chapter 3 Step 7. Item equivalence Investigate the item parameter equivalence across sub-groups Differential Item Functioning Chapter 3 Step 8. Operational item bank Review the analyses of the step six and seven, and find solutions Decisions on domains and items per domain have been finalized for CAT Step 9. CAT implementation Program computer for item selection, starting and stopping rules CAT initial programming completed; presently ongoing alpha testing. Step 10. CAT implementation Utilize item bank to create short-form assessments for clinical use Planned for future studies

PAGE 34

34 Probability of a correct response Latent trait ( ) Figure 1-1. Item characteristic curve. As the respondents estimated ability increases, the probability of answering the item correctly is increased. Adapted from Crocker and Algina, 1986 Item discrimination

PAGE 35

35 CHAPTER 2 FACTOR STRUCTURE OF THE COMPUTER ADAPTIVE MEAS URE FOR FUNCTIONAL COGNITION-TRAUMATIC BRAIN INJURY Introduction Most of functional outcome measures for re habilitation focus on assessing the physical domain. According to the review in Chapte r 1, functional outcome measures, even those commonly used in TBI, only contain few items measuring cognition. With few cognitive items, these measures cannot comprehensively identify the heterogeneous cognitive deficits in TBI. Cognitive problems in attention, memory, executive function, emotion and social participation are commonly seen in individuals with moderate to severe TBI (Johnston & Hall, 1994; Silver et al., 2005). However, the commonly used functiona l outcome measures, such as the Functional Independent Measure (FIM), only address part of these cognitive problems. For example, the FIM addresses only two cognitive problems (p roblem solving and memory) in the entire assessment. It omits other deficits that individua ls with TBI may experience such as attention, emotion, and communication. On the other hand, neuropsychological tests are more extensive in terms of their domain coverage. In addition, they are objective and have strong psycho metric properties. However, neuropsychological tests are impairment measur es which focus on body function and structure. Impairment outcomes cannot provide a complete picture of a patients function in the real world. Studies demonstrated that patients who were within a normal range on neuropsychological tests still experienced challenges in real life (Cope, 1995; Kary l M. Hall & Cope, 1995; Tupper & Cicerone, 1990). Other disadvantages of neuropsychological tests ar e that they are burdensome, both for the administrator and patient. The CAMFC-TBI was developed to overcom e the existing problems of cognitive measures: incomprehensive domain coverage, ec ological limitations a nd inefficiency. To

PAGE 36

36 comprehensively cover the extensive cognitive problems of TBI, we constructed the CAMFCTBI using neuropsychologically-derived domains We identified the domains for the CAMFCTBI using three criteria: the most frequently cite d as deficit domains for TBI, relevant to daily cognitive functioning, and applicable to the typical long-term recovery st ages following injury. Six domains were identified for the CAMFC-TBI: attention, memory, processing speed, executive function, emotional management and social communication. In addition, to thoroughly represent each domain, theoretical models were use to create sub-domain structures of the CAMFC-TBI for further item generation. The sub-domain structure of the CAMFC-TBI is presented in Table2-1. The following is a brief overview of the neuropsychol ogical theories underlying the domains of the CAMFC-TBI. Attention was defined as A control process th at enables the individu al to select, from a number of alternatives, the tasks he will perfor m, or the stimulus he will process, and the cognitive strategy he will adopt to ca rry out these operations. (Moscovitch, 1979) The theoretical model for our attenti on domain was modified from Mirs kys attention model. Four of five components of Mirskeys attention model were included in our attention sub-domains: switching, dividing, focus/execute, and sustai n (Mirsky, Anthony, Duncan, Ahearn, & Kellam, 1991). Encode, one of the five components of Mirskeys attention model overlapping with memory process was addressed in our memory domain. Memory was defined as the ability to store in formation for varying durations and utilize it for adaptive purposes (Lezak, 2004). The theoreti cal model for our memory domain was an integrated model which contains three components: working memory, long-term declarative memory and long-term non-declarative memory. Working memory provides temporary storage and manipulates information and is important for holding the necessary information to complete

PAGE 37

37 a task. Long-term declarative memories include memory of general facts (semantic memory) and memory of particular events related to personal e xperience (episodic memory). Long-term nondeclarative memory involves memory of l earned skills, such as riding a bike. Executive function was defined as A collection of processes that are responsible for guiding, directing, and managing cognitive, emotiona l, and behavioral functions, particularly during active novel problem solvi ng.(Roth, Isquith, & Gioia, 1996). The theoretical model for our executive function domain was modified fr om Stuss and Bensons executive function model (Stuss & Benson, 1986). Five of 9 components of Stuss and Bensons model and a problem solving component were included in our execu tive function sub-domains : initiate, inhibit, plan/organize, monitor, shift, and problem solving. Social communication was defined as the ability to utilize proper language in both social and cultural contexts to achieve communica tive interactions purpos e (Blonder, 2000). The theoretical model for our social communication domain was modified from Pragmatic Protocol (PP) (Prutting & Kirchner, 1987). Three com ponents of PP were included in our social communication sub-domains: topic management conversational ability and non-verbal communication. We did not include a sub-domain structure for pr ocessing speed and emotional management domains. It should be noted that even though sub-domain struct ures were used as a guideline for item generation, the items were generated primary based on the concept of a difficulty hierarchy for each domain. That is, we generated items based on the trajectory of each recovery phase from acute to i npatient rehabilitation, to outpatient rehabilitation, to one-year post injury; representing items from easy to most difficult. While the structure of the CAMFC-TBI wa s based on neuropsychological domains and sub-domains, the measurement goal of the CAMFC-TBI is different from the neuropsychological

PAGE 38

38 tests. In contrast to neuropsychological test s, which focus on assessing impairment of body function/structure, the CAMFC-TBI was inte nded to measure activity and participation component of the Intern ational Classification of Functioning, Disability and Health: functional cognition. Functional cognition is defined as the ability to complete everyday activities that require primarily cognitive abilities (Donovan et al., 2008). Similar to functional cognition, applied cognition emerged as an underlying factor of the Activity Measure of Postacute Care and was defined as discrete functional activities whose performance depends most critically on application of cognitive skills, with limited m ovement requirements (Coster, Haley, Ludlow, Andres, & Ni, 2004). Carrying out f unctional activities su ccessfully involves the cooperation and integration of several cognitive components with environment factors; therefore, impairment measures cannot completely address functional cognition. While the CAMFC-TBI has been designed on the basis of neuropsycho logical constructs, the functiona l nature of the instrument, may result in a different factor structure. In contrast to the sub-domain structure of functional cognition proposed above, there is some empirical support for viewing functional cogn ition as a single construct. For example, the factor analysis of the FIM, one of the mo st pervasive outcome measurements used in rehabilitation field, showed that the FIM ite ms mainly loaded on two factors: motor and cognition (Heinemann, Linacre, Wright, Hamilton, & Granger, 1993). The items that assess memory, problem solving, communicative co mprehension, communicative expression, and social interaction loaded on one factor. Similarly, the principal component analysis of the FIMFAM showed two components accounted for 83.6% of total variance: physical and cognitive functioning (Hawley, Taylor, He llawell, & Pentland, 1999). Fourteen items of the FIMFAM,

PAGE 39

39 including attention, memory, problem solvi ng, emotion, and communicative comprehension loaded mainly on a single component: cognitive functioning. More recently, Haley and colleagues (2004) developed a pool of items, known as the Activity Measure for Postacute Care (AM-PAC), representing activity component of the ICF model. Factor analysis revealed three factors accounted for the variations of this activity measure: Applied Cognition, Personal Care & Instrumental Activities, and Physical & Movement Activities (Haley, Coster, Andres, Ludl ow et al., 2004). While not as detailed as our proposed functional cognitive measure, Coster and colleagues defined 59 items of applied cognition, e.g., Keep track of appointments; Read a newspaper (Coster et al., 2004). Rasch analysis was conducted to assess the psychometri cs of the Applied Cognition Scale. While item and person reliability and separati on were satisfactory, 13 of 59 items did not fit the Rasch model and 25% of participants reached the maximum score. Although the Applied Cognition Scale was concluded to be unidimensional, the conclu sion was based on Rasch fit statistics without further examination using fact or analytic approaches. In summary, based on neuropsychological models, cognition is a multi-dimensional construct. It can be represen ted as components/sub-domains, i.e., sustained attention, working memory, etc.. It can be represented as domai ns, such as attention, memory, and executive function; these domains overlap in certain degree and interact w ith each other. However, the major variance of cognitive activ ities could possibly be accounte d for by single factor-functional cognition. Based on the above review, there are several critical questions a bout the structure of functional cognition: First, can functional cognition be addressed as a single, general construct when assessing individuals with TBI? Second, is a domain-based structure sufficient to explain

PAGE 40

40 the variance of each domain or is a sub-domai n structure derived from neuropsychological models better in explaining the variance of each domain? Third, what is the exploratory factor structure of each functional cognitio n domain in individuals with TBI if the factor structures we proposed are not confirmed? Specifically, we hypothesized that first the CAMFC-TBI can be defined by a general functional cognition construct. Second, underlying this general f unctional cognition construct, there will be six inter-correlated domains: attention, memory, processing speed, executive function, emotional management and social co mmunication (domain-based structure). Within each domain, there will be the small sub-set of items highly correlated based on the neuropsychological model (subdomain structure). However, we hypothesized that the domainbased structure is sufficient to explain the variance of each domain. In other words, we hypothesize that sub-domain struct ure will not be significantly better than domain-based structure. Methods Participants This study was conducted using the existing data from the study Developing a Com puter Adaptive TBI Cognitive Measure, a NIH f unded grant #5R21HD045869-03. This study was approved by the Institutional Review Board of th e University of Florida and the Research and Development Committee at the North Flor ida/South Georgia VA Medical Center. Ninety individuals with TBI and 89 caregivers were recruited from three sites: Shands Hospital in Gainesville, Florida, Brook Health Systems in Jacksonvi lle, Florida and the Shepherd Center, Atlanta, Georgia. The incl usion criteria for patients (1) were receiving rehabilitation for TBI or self-reported to have a TBI; (2) patien ts were at one of the two phases of recovery (outpatient or one-year post injury); (3) patients are between age 18 and 85; (4) patients did not

PAGE 41

41 self-report as having a previous diagnosis of schizophrenia or psychotic disorder or mental retardation; and (5) patients spoke English as their first language. Caregivers, defined as a friend or family member who observes the daily func tioning of the individual with TBI, were approached for consent after th e individual with TBI was recrui ted. Table 2-2 represents the number of participants collected across rec overy stages in the existing database. The demographics of the participants were summarized in Table 2-3. Instruments The CAMFC-TBI consists of 228 items across six dom ains: attention (52 items), memory (35 items), processing speed (33 items), executive function (64 items), emotional management (14 items), and social communication (30 items). Each item of the CAMFC is rated with a 4point rating scale: never, sometimes, often, and al ways. If the participan ts have not performed the activity for the past week, they are in structed to answer not applicable. Data Analysis To examine whether the CAMFC-TBI measures a general concept o f functional cognition in individuals with TBI, a confir matory factor analysis (CFA) was conducted. First, we conducted Rasch analysis on each domain and obtained the person measures for each individual on the six domains. We treated the person measures of the six domains as six item scores. That is, instead of addressing each of the 228 item ratings as individual values, the Winsteps program was used to obtain the person measures for each domain: attention, memory, processing speed, executive function, emotiona l management and social communication. Then, CFA was used to confirm that these six item s/domains contained an underlying factor using Mplus program. We used several fit indices to examine the model fit: P value of the Chi square index is larger than 0.05, the co mparative fit index (CFI) is larg er than 0.95, the Tucker-Lewis index (TLI) is larger than 0.95, the root mean square error of approximation (RMSEA) is less

PAGE 42

42 than 0.06, and the standard root mean square residual (SRMR) is less than 0.08 (Hu & Bentler, 1999). To examine whether domain-based structure is sufficient to explain the variance of each domain or if a sub-domain structure derived from neuropsychological models is significantly better in explaining the variance of each domain, we tested two models for each domain using CFA. For each domain, we conducted CFA to confirm that all items within a domain had an underlying factor. Second, we examined whether the sub-domain model per domain fit the data. The sub-domain mode l was constructed based on the hypothesized neuropsychological sub-domains used to guide item development. For example, the items of the attention domain were genera ted following four sub-domains: focus attention, sustained attention, divided attention, and shifted atten tion. Therefore, a fou r-factor structure was examined for the attention domain of the CAMFC-TBI (Table 2-1). The sub-domain structure of each domain was constructed based on Appendix A. Again, as with the CFA for examining a single cognitive factor, fit indices from the Mplus program were used to determine whether the model has adequate fit. If both models fit, an additional procedure, model comparison, is required to examine whether the sub-domain model fit significantly better than the single-factor model. If the sub-domain model does not show signi ficantly better fit, that is, the Pvalue of Chi-square statistic of the model comparison is not smaller than 0.05, for parsimonious reason we favored single-factor model. If neither the domain-based nor sub-do main model can be confirmed with CFA, to further investigate the potential factor st ructure, we conduct EFA for each domain. Prior to conducting EFA, we used two steps to manage the missing data. First, items with more than 50% Not Applicable response on both patient data and caregiver data were deleted (Appendix B).

PAGE 43

43 Second, for the rest of the items, we replaced th e Not Applicable responses and missing data with value 1, which indicates low ability. The assumption underlying this conversion is that a respondent indicated Not Applicable if they were not able to accomplish the activity. EFA was conducted using SAS program version 9.1. We used unweighted least squares method for estimators. After factor extraction, we conducte d an oblique rotation (obvarimax) following the initial EFA to facilitate the interp retability of factor structure. A factor load ing greater than 0.3 is considered having relationship (Portney & Watkins, 2000b).The Kaiser-Guttman rule, percentage of total variance, scr ee test, and interpretability of th e factors were used to identify the number of factors in each hypothesized dom ain of the CAMFC-TBI (Brown, 2006; Portney & Watkins, 2000b). Based on the Kaiser-Guttman ru le, factors with eigenva lues greater than 1 are retained. Each remained factor is required to account for more than 5% of total variance. In addition, the scree test was used to determine the number of factors by inspecting the point where the slope changes substantially in the f actor versus eigenvalue graph. Finally the number of the retaining factors based on the Kaiser-Guttman rule, percenta ge of variance, and scree test had to be interpretable. Interpretability is the mo st important guideline to decide the final factor structure. Results Research Question 1 The prelimi nary CFA did not completely confirm that the six domains of the CAMFC-TBI contained an underlying factor. Only one fit index, SRMR reached the criteria of model fit when we conducted CFA using patient data (0.069) (Table 2-4). However, the caregiver data showed better model fit. The CFI, TLI and SRMR reached th e criteria of model fit, but not the chi square and the RMSEA (Table 2-4). Although Chi squa re is commonly reported in CFA research, because of its stringent standards, researchers rely more on other fit indices. The Pearson

PAGE 44

44 correlations of the six domains for patient and caregiver data are pres ented in Table 2-5 and Table 2-6 respectively. Research Question 2 The preliminary CFA could not pos itively define the covariance matrix for several domains, and failed to confirm single-factor stru cture and multiple factor structure for the other domains of the CAMFC-TBI. Table 2-7 and Table 28 presents the statistical criteria for each domain across the patient and ca regiver raters. The attention, memory, and executive function domains for both patients and caregivers ratings could not positively define the covariance matrix of latent variables, so the CFA would not successfully compute. In addition, CFA would not successfully compute on the domain-based structure of the processing speed domain for caregivers ratings. Furthermore, CFA did not confirm the doma in-based structure across six domains or the sub-domain structure of social communication domain for both patients and caregivers ratings. Since no model was confirmed, we did not conduct the model comparison procedure. Research Question 3 We conducted EFA to further explor e the fact or structure of each domain on both patient and caregiver raters. Table 2-9 summarizes the nu mber of factors defined by three criteria: 1) eigenvalues greater than 1, 2) f actors accounting for greater than 5% of the variance, and scree plots where the slope changes substantially in the factor versus eigenvalue graph. For the attention domain although nine factors had eigenvalu es greater than 1 for patient and caregiver data, only five factors accounted fo r more than 5% of total variance for patient data and only four factors account ed for more than 5% of total variance for caregiver data. The scree test indicated two factors s hould be retained for patient data but four factors should be retained for caregiver data. Base d on above results, we extracted two, three and four factors to

PAGE 45

45 further investigate the interpretability of the fact or loadings. The rotated three factor pattern was found to be the most interpretable. These thr ee factors accounted for 51% and 56% of total variance for patient data and caregiver data respectively. For patient data, 11 of 50 items cross loaded on more than one factor (see shaded items on Table 2-10) and five items did not load on a ny factor. The items assessing activities requiring focus attention/working memory loaded on factor 1 (9 items). The items assessing ability against distractions loaded on factor 2 (16 items). Th e items assessing activities requiring sustained attention loaded on factor 3 (9 items). For caregiver data, 13 of 50 items cross loaded on more than one factor (see shaded items on Table 2-11) and two items did not load on a ny factor. The items asse ssing activities requiring focus attention/working memory loaded on factor 1 (13 items). The items assessing ability against distractions loaded on factor 2 (12 items). Th e items assessing activities requiring focus attention loaded on factor 3 (10 items). For the memory domain for patient data, five factors ha d eigenvalues greater than 1 and each of these five factor accounted for more than 5% of total variance. In contrast, for caregiver data, three factors had eigenvalues greater than 1 and each of these three factor accounted for more than 5 % of total variance. The scree tests for patient data and caregiver data both indicated three factors should be retained. Based on above criteria we extracted three and four factors to further investigate the interpretability of the fact or loadings. The rotated three factor pattern was found to be the most interpretable. These thr ee factors accounted for 69% and 72 % of total variance for patient data and caregiver data respectively. When three factors retained, patient and careg iver data had 5 of 33 items cross loaded on more than one factor, (see shaded items on Tabl e 2-12 and Table 2-13). In addition, one item did

PAGE 46

46 not load on any factor for patient data and two items did not load on any factor for caregiver data. For patient and caregiver data, items requiring declarative memory had tendency to load on factor 1; items requiring non-declarative memory had tendency to load on factor 2; item requiring working memory had tendency to load on factor 3. For the processing speed domain for patient data, five factors had eigenvalues greater than 1 and each of these five factor accounted for more than 5% of total variance. For caregiver data, four factors had eigenvalues greater than 1 but six factors accounted for more than 5 % of total variance. The scree tests for patient data and caregiver data both indicated three factors should be retained. Based on above criteria, we extracted three and four factors to further investigate the interpretability of the factor loadings. However, both factor patterns are not interpretable and only few items loaded on the th ird or fourth factors. The only interpretable model is to retain one-factor structure. This processing speed factor accounted for 47% and 52% of total variance for patient data and caregiver data, respectively. For patient data, 4 of 28 items did not load on the retain ed factor 3 (Table 2-14 ). For caregiver data, 2/ 28 items did not load on the retained factor (Table 2-15). For the executive function domain for patient data, 15 factor s had eigenvalues greater than 1 but only 3 factors accounted for more than 5% of total variance. In contrary, for caregiver data, 14 factors had eigenvalues gr eater than 1 but only 2 factors accounted for more than 5% of total variance. The scree test fo r patient data indicated 3 factor s should be retained while the scree test for caregiver data s howed 5 factors should be retain ed. Based on above criteria, we extracted 2, 3 and 5 factors to further investigate the interpretability of the factor loadings. The rotated two factor pattern was found to be the mo st interpretable. These two factors accounted for 36% and 45 % of total variance for patient da ta and caregiver data respectively. When two

PAGE 47

47 factors retained, 3 of 62 and 7 of 62 items cross loaded on more than one factor for patient data and caregiver data respectively (shaded items on Table 2-16 and Table 2-17). In addition, 7 and 6 items did not load on any factor for patient and caregiver data respectively. Factor 1 contained items assessing planning and organizing, and factor 2 contained items measuring regulatory control For the emotional management domain one factor had eigenvalue greater than 1 but four factors accounted for more than 5% of total variance for patient data for patient data. For the caregiver data two factors had eige nvalue greater than 1 and four factors accounted for more than 5% of total variance. Th e scree tests for both patient and ca regiver data indi cated two factors should be retained. We extracted one and two factors to further investigate the interpretability of factor loadings. Two-factor stru cture was not interpretable. Th erefore we concluded only one factor represented the emoti onal management domain. This f actor accounted for 75% and 69% of total variance for patient data and caregiver da ta respectively. All items loaded on the retained factor for patient data (Table 218). Two items did not load on this factor for caregiver data (Table 2-19). For the social communication domain for patient data five factors had eigenvalues greater than 1 and all of these factors accounted for more than 5% of total variance. For the caregiver data, four factors had eigenvalues greater than 1 and all of them accounted for more than 5% of total variance. The sc ree tests for both patient data and caregiver data indicated two factors should be retain ed. Based on above criteria, we extrac ted two, three and four factors to further investigate the interpretability of the f actor loadings. The rotate d two-factor pattern was found to be the most interpretable. These tw o factors accounted for 65% and 68 % of total variance for patient data and careg iver data respectively. When tw o factors retained, there were

PAGE 48

48 no cross loaded items for the patient data, but 4 of 30 items cross loaded on more than one factor for the caregiver data (shaded items on Table 2-21 ). Both patient and caregiver data had one item did not load on any factor. Fact or 1 contained items assessing inhibition and factor 2 contained items measuring communication skills (Table 2-20 and Table 2-21). Summary of cross loaded items and not loaded items across six domains were listed in Table 2-22. Discussion A general functional cognition construct of the CAMFC-TBI was confirmed by the caregivers ratings but was not c onfirme d by the patients ratings. Fo r the domain level, we were unable to confirm whether the sub-domain model per domain fit significantly better than 1-factor model per domain. In fact, ne ither the 1-factor model nor sub-domain model derived from neuropsychological models was confirmed using CF A. However, when we further explored the potential factor structures by using EFA, we found multi-factor/sub-domain structures on 4 of 6 domains and 1-factor structur e on 2 remaining domains of the CAMFC-TBI. The attention domain contained 3 inter-correlated sub-domai ns: focus/working memory, against distraction, and sustained attention for patients ratings; focus/working memory, against distraction and focus attention for caregivers ratings. The me mory domain contained 3 inter-correlated subdomains: working memory, long-term declarative memory, and long-term non-declarative memory. The executive function domain cont ained 2 inter-correlated sub-domains: plan/organizing and regu latory control. The social communication domain contained 2 intercorrelated sub-domains: inhibiti on and communication skills. Our findings partially support the first hypothe sis that the entire CAMFC-TBI can be defined by a general functional c ognition construct. CFA confirme d that the entire CAMFC-TBI contained an underlying factor (f unctional cognition) for the caregivers ratings, but CFA did not confirm that one-factor model fit the CAMFC-TBI for the patients ratings. The Pearson

PAGE 49

49 correlations of six domains revealed that the emotional management domain had the lowest correlations with other domains on both patients and caregivers ra tings. In fact, the correlations between emotional management and attention, and between emotional management and memory were only 0.32 and 0.40 respectively for patients ratings. Although thes e correlations were significant, they are relatively low compared to other pairs of correlation for patients ratings (0.62-0.83). On the contrary, the co rrelations between emotional management and attention, and between emotional management and memory we re 0.59 and 0.49, respectively for caregivers ratings. These correlations indicated that the emotional management domain is more closely related to other domains on caregivers ratings than on patients ratings. Perhaps emotional management represents a correlated but distinct domain from general functional cognition for TBI. To further verify this assumption that em otional management is a distinct domain from general functional cognition, CFA will need to co nfirm that the other five domains can be represented by one-factor, general functional cognition construct. We could not successfully compute CFA to support the second hypothesis that domainbased structure per domain is sufficient to expl ain the variance of each model. Neither domainbased structure nor sub-domain structure was confirmed. It is likely that CFA did not compute successfully due to the low ratio of data to number of items. In general, the ratios of the data to the number of items across six domains were small in our study with 1.7, 2.5, 2.7, 1.4, 6.4 and 3.0 respectively. Costello and Osborme (2005) inve stigated the effects of sample size on EFA (Costello & Osborne, 2005). They repor ted that almost one-third of their analyses with the very small sample size (defined as the ratio of data to the number of items equal to 2:1) failed to produce a solution because of Heywood cases (impossible factor loadings) or failure to converge. Since CFA and EFA are based on similar statistical models, the effects of small sample size are

PAGE 50

50 likely to occur for CFA as well. In addition, these preliminary results of CFA need to be interpreted with caution because the low ratio of the data to the number of items per domain can result in incorrect soluti ons (Costello & Osborne, 2005). Since our data either fail to be computed or did not fit the models in CFA, we further used EFA to explore the insight of f actor structures of the CAMFC-TBI. Four of six domains of the CAMFC-TBI showed potential sub-domain stru ctures. On attention domain, EFA showed a 3factor structure on both patients ratings and caregivers ratings However, the three underlying factors of these two raters were slightly different. The three underlying factors of attention domain for patients ratings were labeled as focus attention/working memory, against distractions, and sustained attention while the three underlying factors of attention domain for caregivers ratings were labeled as focus attention/working memory, against distractions, and focus attention. The major difference of the factor stru ctures of these tw o raters was that sustained attention appeared to be one of the dominant underlying factors for the patients ratings while focus attention appear to be one of th e dominant underlying factors for the caregivers ratings. In general, the factor structure of the attention domain of the CAMFC-TBI from EFA was similar to the structure of the theoretical model. All components from the theoretical model were defined on the attention domain of the CAMFC-TBI by EFA, except for the switch component. Few items loading on the switch co mponent may be the reason that switch did not appear to be a dominant factor on the CAMFC-TBI. Our factor structure of the atte ntion domain is similar to the factor structure of the Moss Attention Rating Scale (MARS), a be havioral rating scale intended to assess attention deficits in real world situation. The MARS identified w ith three inter-correlated sub-domains: restlessness/distractibility, init iation, and sustained/consistent attention (Hart et al., 2006)

PAGE 51

51 although it was originally developed to test an ov erall concept of atten tion (Whyte, Hart, Bode, & Malec, 2003). Two underlying factors (restlessn ess/distractibility, and sustained/consistent attention) of the MARS were al so identified in our assessment However, the other component initiation did not emerge on our assessment. Si nce the theoretical model (modified Mirskeys attention model) that we used to guild item ge neration did not include initiation component on attention domain, missing an i nitiation component on our atte ntion domain was not surprising. For the memory domain, EFA showed a 3-fact or structure on both pa tients ratings and caregivers ratings. The three underlying factors of memory were labeled as working memory, long-term declarative memory and long-term non-declarative memory In general, the factor structure of memory domain of the CAMFC-TB I from EFA matched the structure of the theoretical model. In comp arison, the revised Everyday Me mory Questionnaire (EMQ), originally developed for assess ing everyday memory for head in jury, contained 13 items with three underlying factors: retrieval memory, atte ntional tracking and un-id entifiable factor (Royle & Lincoln, 2008). Their items requiri ng retrieval memory were similar to our items that were identified as long-term declarative memory while items requiring attentional tracking were comparable to the ones we identified as working memory. The revised EMQ did not contain a factor paralleled to our l ong-term non-declarative memory. For the executive function domain, EFA showed a 2-factor structure on both patients ratings and caregivers ratings. While we gene rated our items based on 6 components modified by Stuss and Bensons model, the dominant unde rlying factors from EFA did not match the structure of the theoretical model. Rath er, EFA identified two broader factors: plan/organizing/problem solving and regulatory control These two factors paralleled the two indexes of the Behavior Rating Inventory of Executive FunctionAdult version (BRIEF-A) (Roth

PAGE 52

52 et al., 1996). The BRIEF-A is a standardized self and proxy-report measure for executive function in everyday environment. The summary sc ore of the BRIEF-A, consists of two indexes: Behavior Regulation Index (BRI) and Metacognition Index (MI). The regulatory control factor is comparable to the BRI of the BRIEF-A while the plan/organizing/problem solving factor is comparable to the MI. For the social communication domain, EFA show ed a 2-factor structure on both patients ratings and caregivers ratings. The two underlyi ng factors of the social communication domain for both patients ratings and careg ivers ratings we re labeled as inhibition and communication skills. While we generated our items based on thr ee components (conversational ability, topic management, and non-verbal communication) deri ved from the Pragmatic Protocol (PP), the dominant underlying factors from EFA did not completely match the structure of the theoretical model. However, inhibition is comparable to the non-verbal communication component of the PP and communication skills are a combination of topic management and conversational ability components Our preliminary analyses indicated that func tional cognition is a broad dimension with six distinctive but correlated domains. Multiple underlying factors for each domain were obtained from EFA. These sub-domains could be used to monitor the progress of specific aspects of the domains while the domain scores could provide a broader picture re flecting the attention, memory, processing speed, executive function, em otional management or social communication. Selections of an overall cognitive measure or domain/sub-domain measures should be based on the purpose of the evaluation. For example, for screening whether patients have functional cognitive deficits, an overall cognitive meas ure may be most appropriate. For determining whether patients have functional attention deficits, the domain score that contains all aspects of

PAGE 53

53 attention should be applied. For mo nitoring or investigating what sp ecific aspects of attention for which the patient has trouble, the sub-domain scor e should be used. It should be noted that the factor structure of each domain on the CAMFC-TBI may provide a way to track each aspects of that domain, but they cannot replace the role of neuropsychological tests. The impairments of structures or pathways may not be reflected by se lf/proxy reports of the in terference of cognitive deficits on functional activities. In addition, it is possible that the dominant underlying factors are different when the assessment is applied to differe nt phases of recovery stages of TBI or diverse populations. For example, varied factor structures of the Everyday Memory Questionnaire were revealed when applied in different samples acr oss different studies (Eflikides et al., 202; Richardson & Chan, 1995; Royle & Lincoln, 2008). Therefore, th e factor structure of our preliminary analysis may not be adequate for other stages of TBI, such as acute and inpatient rehabilitation stages or other diagnosis. Finall y, because of our small sample size, the factor loadings are likely not stable (Costello & Osborne, 2005). Thus, the items may have been misclassified into the wrong f actors during the computation. T hus, it is very important to replicate this study to ensure that the factor structure we found is stable for our target population. The preliminary results provide a general idea of how functional cognitions could be constructed. Future studies should include larger sample sizes to examine whether our factor structures found in EFA can be confirmed. Examining factor structure provides evidence for the dimensionality of an assessment and this information can furthe r suggest what IRT mode ls should be conducted. Unidimensionality is an important assump tion of unidimensional IRT models. If the unidimensionality assumption does not hold, the it em difficulty estimates and person ability estimates may not be accurate. Therefore, prior to conducting Rasch analysis, our factor structure

PAGE 54

54 findings provide the foundation to further inve stigate the measurement qualities of the CAMFCTBI.

PAGE 55

55 Table 2-1. The CAMFC-TBI domains, sub-dom ains structure and example of items Domains Sub-domains Examples Attention Switching Returns to an activity without a reminder after a short interruption Dividing Has a conversation in a noisy environment (therapy room) Focus/execute Selects meal items from a menu Sustained Participates in a structured activity for 30 minutes without rest break Memory Working Goes to a room to get something and forgets what to get Long-term declarative Recalls information given at a previous therapy or doctor appointment Long-term non-declarative Recalls to turn off the stove or oven Executive Function Initiate Fills free time with activities without been told Inhibit Talks at the wrong time Plan/organize Plan a new activity Monitor Make careless errors during a new activity Shift Gets stuck on a topic Problem solving Tries a different approach to a problem when the first one does not work Social Communication Topic management Jumps to a topic unrelated to the conversation Conversational ability Keeps up with a conversation without asking people to repeat Non-verbal communication Faces the person when speaking Table 2-2. Number of particip ants across recovery stages Outpatient rehabilitation One year post injury Total 47 patient ratings 46 caregiver ratings 43 patient ratings 43 caregiver ratings 90 patient ratings 89 caregiver ratings

PAGE 56

56 Table 2-3. Demographic statisti cs for patients and caregivers Patient (N=90) Caregiver (N=89) Age Mean SD Range 38.2 15.8 years 18-84 49.9 14.6 years 21-88 Gender, n (%) Male Female 63 (70%) 27 (30%) 19 (78.7%) 70 (21.3%) Ethnicity, n (%) White African American Hispanic Other Missing 72 (80%) 10 (11.1%) 4 (4.4%) 3 (3.3%) 1 (1.1%) 75 (84.3%) 10 (11.2%) 3 (3.4%) 1 (1.1%) Education n (%) 8th grade 9th grade 10th grade 11th grade GED 12th grade Some college, no degree Some college, earned degree (AA/AS) Completed 4-year college, earned degree (BS/BA) Graduate/Professional degree (MA/MS/PhD) 2 (2.2%) 0 (0%) 7 (7.8%) 6 (6.7%) 5 (5.6%) 19 (21.1%) 23 (25.6%) 14 (15.6%) 7 (7.8%) 7 (7.8%) 1 (1.1%) 1 (1.1%) 2 (2.2%) 3 (3.4%) 4 (4.5%) 13 (14.6%) 27 (30.3%) 20 (22.5%) 11 (12.4%) 7 (7.9%) Income n (%) Under $5,000 $5,000-$10,000 $11,000-$15,000 $16,000-$20,000 $21,000-$35,000 $35,000-$50,000 Over $50,000 9 (10%) 3 (3.3%) 2 (2.2%) 8 (8.9%) 11 (12.2%) 24 (26.7%) 32 (35.6%) 2 (2.2%) 3 (3.4%) 7 (7.9%) 5 (5.6%) 18 (20.2%) 16 (18%) 36 (40.4%) Work n (%) Yes No Missing 17 (18.9%) 72 (80%) 1 (1.1%) Drive n (%) Yes No Missing 27 (30%) 59 (65.6%) 4 (4.4%)

PAGE 57

57 Table 2-4. Fit indices for the general cognition Fit indices Patient Caregiver Chi Square 73.37 21.77 df 9 9 P-Value (>.05) 0.0000 0.0096 CFI (>0.95) 0.848 0.974 TLI (>0.95) 0.747 0.954 RMSEA (<0.06) 0.282 0.126 SRMR (<0.08) 0.069 0.033 Table 2-5. Pearson correlations for patients data Attention Memory Processing Speed Executive Function Emotional Management Social communication Attention 1.00 Memory 0.76 <.0001 1.00 Processing Speed 0.80 <.0001 0.73 <.0001 1.000 Executive Function 0.63 <.0001 0.72 <.0001 0.67 <.0001 1.00 Emotional Management 0.32 0.0021 0.40 0.0001 0.50 <.0001 0.69 <.0001 1.00 Social Communication 0.62 <.0001 0.68 <.0001 0.69 <.0001 0.83 <.0001 0.65 <.0001 1.00

PAGE 58

58 Table 2-6. Pearson correlations for caregivers data Attention Memory Processing Speed Executive Function Emotional Management Social communication Attention 1.00 Memory 0.80 <.0001 1.00 Processing Speed 0.80 <.0001 0.81 <.0001 1.00 Executive Function 0.84 <.0001 0.84 <.0001 0.82 <.0001 1.00 Emotional Management 0.59 <.0001 0.49 <.0001 0.53 <.0001 0.63 <.0001 1.00 Social Communication 0.72 <.0001 0.72 <.0001 0.76 <.0001 0.80 <.0001 0.67 <.0001 1.00

PAGE 59

59 Table 2-7. Confirmatory Factor Analysis Results Attention (52 items) Memory (35 items) Processing Speed (33 items) Indices Patient Caregiver Patient Caregiver Patient Caregiver (criterion) 1-Factor 4-Factor* 1-Factor 4-Factor* 1-F actor 3-Factor* 1-Factor 3-Factor* 1-Factor 1-Factor* Chi Square 102.886 115.172 115.77 155.793 122.643 112.669 88.843 84.082 df 51 51 50 50 46 41 42 38 P-Value (>0.05) <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 CFI (>0.95) 0.913 0.893 0.927 0.883 0.898 0.936 0.934 0.951 TLI (>0.95) 0.93 0.914 0.959 0.934 0.916 0.969 0.945 0.969 RMSEA (<0.06) 0.106 0.118 0.122 0.154 0.137 0.14 0.111 0.117 WRMR (<0.1) 1.08 1.148 1.119 1.31 1.186 1.114 1.056 1.058 *represents that covariance matrix could not be positively defined and the analysis did not successfully complete Table 2-8. Confirmatory Factor Analysis Results Executive Function (64 items) Emotion (14 items) Social Communication (30 items) Indices Patient Caregiver Patient Caregiver Patient Caregiver (criterion) 1-Factor 6-Factor* 1-Factor 6-Factor* 1-F actor 1-Factor 1-Factor 3Factor 1-Factor 3-Factor Chi Square 161.764 154.061 123.721 114.624 614.35 640.398 608.31 220.09 157.116 156.11 df 59 59 51 51 17 17 31 38 37 37 P-Value (>0.05) <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 <.0000 CFI (>0.95) 0.857 0.868 0.913 0.924 0.913 0.944 0.685 0.685 0.882 0.881 TLI (>0.95) 0.876 0.886 0.94 0.948 0.943 0.966 0.743 0.743 0.93 0.93 RMSEA (<0.06) 0.139 0.134 0.127 0.118 0.149 0.118 0.231 0.231 0.19 0.191 WRMR (<0.1) 1.292 1.257 1.143 1.095 1.004 0.858 1.771 1.764 1.401 1.411 represents that covariance matrix could not be positively defined and the analysis did not successfully complete

PAGE 60

60 Table 2-9. Number of factors meeting criteria for each domain Attention Memory PS EF EM SC Pt Cg Pt Cg Pt Cg Pt Cg Pt Cg Pt Cg Eigenvalue>1 9 9 535415141 2 5 4 > 5% variance 5 4 5356324 4 5 4 Scree Plot 2 4 3333332 2 2 2

PAGE 61

61 Table 2-10. Rotated factor pattern of th e attention domain for patient data Item Factor 1 Factor 2 Factor 3 i7 Writes down a short phone message 0.82 0.00 -0.21 i19 Locates a phone number/address in the phone book 0.70 -0.03 -0.02 i18 Writes down messages from an answering machine 0.69 0.09 -0.02 i39 Writes down message while talk on the phone 0.67 -0.03 0.00 i25 Sorts important mail from junk mail 0.55 -0.21 0.21 i5 Copies daily schedule correctly 0.52 0.14 0.13 i36 Notices when a warning light appears on the dashboard 0.49 0.07 0.08 i47 Use map or follow written direction 0.48 -0.27 0.31 i26 Locates items in the refrigerator 0.45 0.08 0.25 i23 Locates info in a section/article in newspaper 0.43 -0.13 0.40 i41 Locates items in a store using a shopping list 0.42 -0.09 0.31 i20 Locates particular item/brand in grocery store 0.36 0.29 0.24 i45 Answers the phone when it rings 0.36 0.30 0.03 i24 Selects outfits from a dresser or closet 0.29 0.07 0.26 i48 Leaves out steps of a task (R) 0.29 0.28 -0.01 i17 Talks without being distracted by a TV on 0.05 0.71 0.00 i15 Has a conversation in a noisy environment 0.05 0.68 0.14 i11 Stays focused on 5-10 min in a noisy environment 0.12 0.64 0.09 i31 Participates activity for 1 hr with short break -0.11 0.53 0.38 i16 Watches TV without being distracted by other talk 0.12 0.48 0.18 i6 Turns toward a ringing phone 0.28 0.48 -0.09 i9 Participates in activity for 30 min without break 0.12 0.45 0.08 i14 Has a conversation with a small group 0.19 0.42 0.29 i12 Completes 2-3 min conversation using the phone 0.30 0.40 0.14 i13 Completes a meal with distractions 0.15 0.39 0.22 i8 Participates in activity for 30 min with break 0.32 0.35 -0.09 i44 Looks toward person after being touched lightly 0.09 0.33 0.07 i49 Stops in the middle of a task when distracted (R) -0.10 0.33 0.08 i1 Correctly answer questions about himself/herself 0.01 0.33 0.16 i52 Stop chewing food when distracted (R) 0.13 0.32 -0.25 i51 Makes mistakes as the length of the task increases (R) 0.07 0.31 0.15 i3 Greets person when that person enters the room 0.00 0.31 0.14 i43 Able to work on multiple things at the same time 0.18 0.31 0.23 i46 Participates in a structured activity for 5-10 min 0.18 0.30 0.25 i50 Pays attention to the wrong conversation/activity (R) 0.00 0.28 -0.04 i4 Selects meal items from a menu 0.25 0.26 0.16 i2 Goes directly from to a specific location 0.19 0.20 0.13 i35 Continues to work on an extended project 0.16 -0.24 0.63 i34 Picks out important info from a lecture/instruction 0.02 0.24 0.57 i29 Listens for 15-30 minutes quietly and with focus 0.03 -0.06 0.57 i42 Finishes one task before starting another -0.03 -0.09 0.53 i32 Returns to an activity after a short interruption -0.29 0.41 0.50 i30 Participates in a 10-20 min conversation 0.04 0.26 0.44 i27 Sits through an hour long TV program 0.10 0.11 0.44 i28 Reads 30 minutes without taking a break 0.35 -0.20 0.43 i33 Maintains speed accuracy when do a task in distract -0.03 0.37 0.40 i40 Goes back and forth btw read instructions and do a task 0.27 0.22 0.35 i22 Selects meal items from a complex menu 0.16 0.18 0.34 i21 Locates size of clothing on a department store 0.30 0.19 0.33 i10 Completes self-care without getting distracted 0.17 0.27 0.32

PAGE 62

62 Table 2-11. Rotated factor pattern of th e attention domain for caregiver data Item Factor 1 Factor 2 Factor 3 i23 Locates info in a section/article in newspaper 0.78 -0.07 0.07 i39 Writes down message while talk on the phone 0.76 0.06 -0.08 i7 Writes down a short phone message 0.75 -0.03 0.03 i18 Writes down messages from an answering machine 0.74 -0.05 0.04 i19 Locates a phone number/address in the phone book 0.72 0.01 0.13 i25 Sorts important mail from junk mail 0.66 -0.15 0.22 i20 Locates particular item/brand in grocery store 0.60 0.11 0.18 i41 Locates items in a store using a shopping list 0.54 0.02 0.26 i40 Goes back and forth btw read instructions and do a task 0.54 0.29 -0.03 i47 Use map or follow written direction 0.46 0.00 0.25 i43 Able to work on multiple things at the same time 0.44 0.40 0.07 i21 Locates size of clothing on a department store 0.44 0.07 0.24 i5 Copies daily schedule correctly 0.40 -0.07 0.42 i35 Continues to work on an extended project 0.40 0.34 0.11 i36 Notices when a warning light appears on the dashboard 0.35 0.08 -0.05 i24 Selects outfits from a dresser or closet 0.31 0.00 0.23 i33 Maintains speed accuracy when do a task in distract 0.27 0.62 0.15 i17 Talks without being distracted by a TV on -0.07 0.57 0.24 i34 Picks out important info from a lecture/instruction 0.49 0.56 -0.18 i27 Sits through an hour long TV program 0.13 0.55 0.03 i16 Watches TV without being distracted by other talk -0.06 0.54 0.13 i50 Pays attention to the wrong conversation/activity (R) 0.03 0.54 0.12 i31 Participates activity for 1 hr with short break 0.23 0.54 0.23 i49 Stops in the middle of a task when distracted (R) -0.10 0.54 0.11 i11 Stays focused on 5-10 min in a noisy environment 0.13 0.53 0.31 i30 Participates in a 10-20 min conversation 0.17 0.48 0.26 i28 Reads 30 minutes without taking a break 0.45 0.47 -0.14 i13 Completes a meal with distractions 0.08 0.46 -0.09 i15 Has a conversation in a noisy environment -0.05 0.45 0.44 i29 Listens for 15-30 minutes quietly and with focus 0.21 0.44 -0.11 i46 Participates in a structured activity for 5-10 min -0.09 0.42 0.31 i52 Stop chewing food when distracted (R) -0.07 0.42 0.04 i32 Returns to an activity after a short interruption 0.35 0.41 0.18 i51 Makes mistakes as the length of the task increases (R) 0.28 0.36 0.02 i8 Participates in activity for 30 min with break 0.02 0.19 0.16 i12 Completes 2-3 min conversation using the phone 0.07 -0.04 0.70 i45 Answers the phone when it rings 0.22 -0.18 0.61 i26 Locates items in the refrigerator 0.41 -0.30 0.60 i6 Turns toward a ringing phone -0.10 0.09 0.58 i2 Goes directly from to a specific location 0.09 0.04 0.57 i10 Completes self-care without getting distracted 0.18 -0.12 0.56 i14 Has a conversation with a small group -0.20 0.40 0.48 i22 Selects meal items from a complex menu 0.18 0.32 0.42 i48 Leaves out steps of a task (R) 0.11 0.09 0.42 i9 Participates in activity for 30 min without break -0.14 0.38 0.41 i1 Correctly answer questions about himself/herself -0.05 0.14 0.40 i4 Selects meal items from a menu 0.14 0.21 0.39 i3 Greets person when that person enters the room -0.11 0.24 0.34 i44 Looks toward person after being touched lightly 0.06 0.04 0.34 i42 Finishes one task before starting another 0.12 0.23 0.28

PAGE 63

63 Table 2-12. Rotated factor pattern of the memory for patient data Items F1 F2 F3 i53 Recalls a meal later in the day 0.67 -0.27 0.07 i63 Recalls the steps in do ing a simple activity 0.65 0.10 0.00 i56 Recalls what he/she did before the injury 0.62 -0.08 0.17 i62 Recalls where to find thing when not put in its place 0.60 0.14 -0.05 i60 Recalls a visit by a familiar person 0.58 0.17 0.19 i80 Recalls events from last birthdays/vacation 0.55 -0.15 0.12 i54 Knows the current month 0.49 0.35 0.02 i75 Recalls birthdays, holidays or anniversaries 0.47 0.22 0.12 i59 Recalls more than one appointment in a single day 0.47 0.20 0.26 i55 Recalls a visit from a familiar person 0.42 -0.05 0.31 i76 Recalls frequently used phone numbers 0.38 0.29 0.07 i74 Recalls info given at a prev ious therapy appointment 0.35 0.09 0.25 i70 Recalls familiar route without assistance 0.34 0.33 0.11 i67 Recalls to lock the door wh en leaving the house -0.08 0.65 -0.02 i82 Recalls to pay bills -0.06 0.59 0.06 i73 Recalls to use a calendar to track appointments 0.02 0.59 0.12 i64 Recalls to move laundry fr om washer to dryer -0.20 0.57 0.25 i69 Recalls to give someone a telephone message 0.03 0.55 0.06 i81 Recalls to do weekly chores 0.06 0.55 -0.01 i66 Recalls to turn off the stove or oven 0.05 0.52 0.13 i71 Recalls a newly learned r oute without assistance 0.18 0.49 0.16 i72 Recalls where the car is in the mall parking lot -0.01 0.48 0.06 i83 Recalls upcoming deadlines, as signments, or meetings 0.27 0.44 -0.06 i65 Recalls to put food away in the fridge when finished 0.26 0.38 0.12 i79 Recalls to go to doctor appointments 0.37 0.38 0.22 i78 Recalls the story line from one reading to the next 0.17 0.29 0.08 i85 Begins to do something and forgets what was to be done (R) -0.16 -0.02 0.89 i84 Goes to a room to get thing but forgets what to get (R) 0.02 -0.08 0.73 i86 Loses train of thought in a conversation (R) 0.11 -0.09 0.72 i87 Repeats a story that has already been told (R) -0.22 0.03 0.65 i58 Recalls a simple routine 0.42 -0.01 0.49 i57 Recalls basic instructions 0.35 -0.04 0.45 i77 Recalls to get an item at the store not written down 0.23 0.26 0.36 i61 Recalls to take medicine at the right time and amount 0.06 0.13 0.34

PAGE 64

64 Table 2-13. Rotated factor pattern of th e memory domain for caregiver data Items F1 F2 F3 i64 Recalls to move laundry from washer to dryer 0.90 -0.20 -0.06 i66 Recalls to turn off the stove or oven 0.80 -0.09 -0.01 i81 Recalls to do weekly chores 0.74 -0.04 -0.01 i65 Recalls to put food away in the fridge when finished 0.72 0.03 0.03 i67 Recalls to lock the door when leaving the house 0.58 0.08 0.05 i73 Recalls to use a calendar to track appointments 0.54 0.21 0.06 i63 Recalls the steps in do ing a simple activity 0.52 0.17 0.22 i69 Recalls to give someone a telephone message 0.51 0.16 0.15 i83 Recalls upcoming deadlines, assignments, or meetings 0.43 0.23 0.27 i78 Recalls the story line from one reading to the next 0.26 0.14 0.19 i61 Recalls to take medicine at the right time and amount 0.21 0.21 0.19 i58 Recalls a simple routine -0.04 0.64 0.17 i57 Recalls basic instructions -0.12 0.59 -0.10 i60 Recalls a visit by a familiar person 0.22 0.53 0.20 i56 Recalls what he/she did before the injury -0.04 0.53 -0.01 i74 Recalls info given at a prev ious therapy appointment 0.22 0.52 0.18 i53 Recalls a meal later in the day 0.20 0.51 0.20 i72 Recalls where the car is in the mall parking lot 0.26 0.47 0.12 i70 Recalls familiar route without assistance 0.26 0.45 0.24 i54 Knows the current month 0.30 0.44 0.06 i79 Recalls to go to doctor appointments 0.38 0.43 0.23 i77 Recalls to get an item at the store not written down 0.17 0.42 0.18 i62 Recalls where to find thing when not put in its place 0.21 0.42 0.21 i59 Recalls more than one appointment in a single day 0.19 0.39 0.34 i55 Recalls a visit from a familiar person 0.33 0.39 0.16 i71 Recalls a newly learned r oute without assistance 0.26 0.38 0.27 i75 Recalls birthdays, holidays or anniversaries 0.21 0.36 0.31 i82 Recalls to pay bills 0.17 0.35 0.06 i80 Recalls events from la st birthdays/vacation 0.16 0.31 0.29 i84 Goes to a room to get thing but forgets what to get (R) -0.15 -0.22 0.90 i85 Begins to do something and forgets what was to be done (R) -0.11 -0.19 0.88 i86 Loses train of thought in a conversation (R) -0.10 -0.06 0.69 i76 Recalls frequently used phone numbers 0.07 0.26 0.31 i87 Repeats a story that has already been told (R) 0.10 0.10 0.30

PAGE 65

65 Table 2-14. Factor pattern of the proc essing speed domain for patient data Items F1 i110 Pays for a fast-food order within 30 seconds 0.74676 i96 Makes a simple breakfast within 5-10 min if physically able 0.66939 i111 Reads a restaurant menu and makes a selection within 5 min 0.65435 i114 Shops for a few items in a reasonable amount of time 0.59993 i92 Copies schedule in a timely manner 0.59460 i109 Places order in a drive-through without holding up line 0.57640 i91 Writes name in a timely manner 0.56711 i100 Unloads the washing ma chine within 10 minutes 0.56482 i101 Puts away clean dishes within 15 minutes 0.54949 i94 Gets dressed within 15 min 0.54530 i113 Keeps up the pace required of school or work setting 0.54076 i107 Sorts daily mail within 5 min 0.53750 i93 Begins to answer open-ended questions within 2 sec 0.53694 i98 Follows simple directions w ithout asking people to repeat 0.53450 i105 Follows an automated phone menu 0.53410 i106 Keeps up with the story of a 30-min TV without asking 0.52213 i95 Completes tasks or chores by a set deadline 0.48645 i102 Takes a message without asking the caller to repeat 0.48190 i112 Reads a one page letter within 5 min 0.47750 i97 Keeps up with a conversation wi thout asking people to repeat0.46100 i90 Completes menu selection in a timely manner 0.45188 i117 Takes a long time to get dressed (R) 0.38633 i88 Answers the phone within a least 3 rings 0.38390 i119 Makes mistakes when trying to keep up (R) 0.31830 i89 Says "come in" in response to knock on the door 0.28090 i116 Takes a long time to finish eating a meal (R) 0.24269 i118 Needs repeated requests to respond (R) 0.21208 i115 Open-ended questions need to be asked more than once (R) 0.11416

PAGE 66

66 Table 2-15. Factor pattern of the processing speed domain for caregiver data Items F1 i114 Shops for a few items in a reasonable amount of time 0.70297 i101 Puts away clean dishes within 15 minutes 0.68166 i105 Follows an automated phone menu 0.67572 i96 Makes a simple breakfast within 5-10 min if physically able 0.66311 i110 Pays for a fast-food order within 30 seconds 0.66310 i107 Sorts daily mail within 5 min 0.62847 i94 Gets dressed within 15 min 0.62748 i91 Writes name in a timely manner 0.59545 i98 Follows simple directions w ithout asking people to repeat 0.59358 i102 Takes a message without asking the caller to repeat 0.59090 i92 Copies schedule in a timely manner 0.57075 i90 Completes menu selection in a timely manner 0.57066 i100 Unloads the washing ma chine within 10 minutes 0.56436 i93 Begins to answer open-ended questions within 2 sec 0.56040 i95 Completes tasks or chores by a set deadline 0.55307 i112 Reads a one page letter within 5 min 0.52211 i109 Places order in a drive-through without holding up line 0.52156 i89 Says "come in" in response to knock on the door 0.48619 i88 Answers the phone within a least 3 rings 0.47553 i111 Reads a restaurant menu and makes a selection within 5 min 0.47315 i97 Keeps up with a conversation wi thout asking people to repeat0.46710 i113 Keeps up the pace required of school or work setting 0.45338 i117 Takes a long time to get dressed (R) 0.44032 i119 Makes mistakes when trying to keep up (R) 0.43436 i115 Open-ended questions need to be asked more than once (R) 0.34111 i116 Takes a long time to finish eating a meal (R) 0.30155 i106 Keeps up with the story of a 30-min TV without asking 0.29148 i118 Needs repeated requests to respond (R) 0.26368

PAGE 67

67 Table 2-16. Rotated factor pattern of the executive function domain for patient data Items F1 F2 i152 Comes up with an alternate when the first not work 0.71 -0.10 i159 Organizes a short written document 0.71 -0.11 i141 Gathers materials needed to put together a list 0.69 0.02 i143 Organizes an activity several days in advance 0.66 -0.05 i164 Picks up hints that they should end a conversation 0.60 0.01 i124 Plans a new activity 0.60 0.29 i146 Tells someone/takes action when something wrong 0.59 0.02 i155 Tries a different approach when the first does not work 0.59 0.10 i142 Identifies items needed to put together a list 0.59 0.14 i160 Estimates the time needed to meet a deadline 0.58 0.25 i162 Makes changes to a schedule if needed 0.57 0.06 i150 Follows safety rules 0.57 0.15 i154 Suggests or attempts a solution to a problem 0.56 0.04 i161 Adjusts schedule to m eet a deadline 0.55 -0.07 i125 Plans ahead in order to get to an appointment on time 0.53 -0.11 i144 Organizes a written list 0.51 -0.09 i126 Fills free time with activities without being told 0.49 0.33 i127 Starts an activity without being told 0.48 0.16 i122 Complete a complex task that has several steps 0.48 -0.03 i139 Chooses clothes based on the weather 0.48 0.01 i123 Plans a common daily activity 0.48 0.28 i156 Adds a new topic to a conversation 0.47 0.10 i158 Do the things needed to prepare for a bigger project 0.46 0.02 i165 Dresses to match social situation 0.45 -0.06 i137 Starts a task early enough to get it done 0.45 0.20 i135 Gets started on homework/ c hores without being told 0.44 0.11 i149 Makes attempts to solve problems before asking for help 0.44 0.29 i130 Recognizes and corrects mistakes 0.44 0.11 i169 Able to make a quick, simple direction 0.43 0.31 i145 Keeps personal area organized 0.41 0.10 i148 Stops an activity to do something needs to get done 0.41 0.24 i166 Asks questions to get more info about injury 0.40 0.04 i121 Complete a simple task that has several steps 0.36 0.28 i167 Initiates a discussion about future needs 0.32 0.18 i147 Seeks help when needed 0.31 0.20 i131 Readily changes behaviors when an error is pointed out 0.29 -0.18 i157 Listens to another perspective without arguing 0.26 0.16 i153 Stops talking when a discussion becomes heated 0.17 0.05 i180 Bothers other people while they are working (R) -0.14 0.61 i174 Does not know what to do next, so stops(R) 0.07 0.55 i172 Gets stuck in a topic (R) 0.03 0.53 i177 Is overly trusting (R) -0.02 0.52 i178 Buys unnecessary items that look appealing (R) -0.03 0.51 i151 Tries to do an activity befo re having the ability (R) -0.11 0.51 i179 Interrupts while someone is talking on the phone (R) 0.02 0.49 i132 Talks at the wrong time (R) -0.22 0.49 i183 Does not start activities on own (R) 0.00 0.49

PAGE 68

68 Table 2-16. Continued Items F1 F2 i138 Comes up with ideas for thing s to do during free time 0.38 0.48 i140 Demonstrates an understanding of own abilities 0.20 0.48 i128 Makes careless errors during a new activity (R) -0.06 0.48 i171 Makes careless errors in daily tasks (R) -0.04 0.48 i184 Gives up if first attempt is not successful (R) 0.17 0.47 i129 Not recognize limitations when attempting a task (R) 0.11 0.46 i175 Allows others to solve problems could have done selves (R) -0.08 0.46 i176 Jumps to a solution when attempting to solve a problem (R) -0.26 0.46 i170 Does not readily switch from one activity to another (R) -0.25 0.42 i182 Has problems managing money (R) 0.27 0.34 i181 Makes errors when solving a problem with steps (R) 0.26 0.32 i136 Catches own mistakes while working on a task 0.14 0.24 i173 Does not stop or apologizes when bothers others (R) -0.08 0.23 i134 Stays seated until a task is done 0.06 0.20 i133 Not ask embarrassing questions/make hurtful comments 0.15 0.18 Table 2-17. Rotated factor pattern of the ex ecutive function domain for caregiver data Items F1 F2 i162 Makes changes to a schedule if needed 0.88 -0.16 i160 Estimates the time needed to meet a deadline 0.81 0.01 i161 Adjusts schedule to meet a deadline 0.79 0.09 i158 Do the things needed to prepare for a bigger project 0.78 -0.13 i152 Comes up with an alternate when the first not work 0.77 -0.04 i144 Organizes a written list 0.72 -0.03 i124 Plans a new activity 0.70 0.20 i181 Makes errors when solving a problem with steps (R) 0.67 -0.04 i141 Gathers materials needed to put together a list 0.67 0.13 i143 Organizes an activity se veral days in advance 0.64 0.15 i122 Complete a complex task that has several steps 0.62 -0.07 i135 Gets started on homework/ chores without being told 0.60 0.19 i159 Organizes a short written document 0.58 0.15 i142 Identifies items needed to put together a list 0.58 0.30 i155 Tries a different approach when the first does not work 0.58 0.08 i130 Recognizes and corrects mistakes 0.52 0.32 i125 Plans ahead in order to get to an appointment on time 0.52 0.32 i165 Dresses to match social situation 0.51 0.21 i127 Starts an activity without being told 0.51 0.36 i164 Picks up hints that they should end a conversation 0.47 0.04 i175 Allows others to solve problem s could have done selves (R) 0.45 -0.05 i182 Has problems managing money (R) 0.41 0.26 i139 Chooses clothes based on the weather 0.40 0.39 i167 Initiates a discussion about future needs 0.40 0.08 i121 Complete a simple task that has several steps 0.38 0.17 i145 Keeps personal area organized 0.37 0.34 i131 Readily changes behaviors when an error is pointed out 0.36 0.14 i128 Makes careless errors during a new activity (R) 0.35 0.21

PAGE 69

69 Table 2-17. Continued Items F1 F2 i166 Asks questions to get more info about injury 0.28 0.07 i156 Adds a new topic to a conversation 0.21 0.13 i133 Not ask embarrassing questions/make hurtful comments 0.14 0.06 i153 Stops talking when a discussion becomes heated 0.09 0.07 i140 Demonstrates an understanding of own abilities 0.12 0.62 i150 Follows safety rules 0.16 0.61 i129 Not recognize limitations when attempting a task (R) -0.10 0.56 i138 Comes up with ideas for thing s to do during free time 0.34 0.55 i147 Seeks help when needed -0.22 0.54 i177 Is overly trusting (R) 0.08 0.54 i151 Tries to do an activity before having the ability (R) -0.10 0.52 i137 Starts a task early enough to get it done 0.31 0.51 i183 Does not start ac tivities on own (R) 0.22 0.50 i134 Stays seated until a task is done 0.03 0.48 i148 Stops an activity to do so mething needs to get done 0.29 0.47 i126 Fills free time with activ ities without being told 0.38 0.46 i149 Makes attempts to solve problems before asking for help 0.42 0.44 i146 Tells someone/takes action when something wrong 0.32 0.44 i154 Suggests or attempts a solution to a problem 0.37 0.44 i184 Gives up if first attempt is not successful (R) 0.39 0.41 i174 Does not know what to do next, so stops(R) 0.11 0.40 i123 Plans a common daily activity 0.39 0.40 i136 Catches own mistakes while working on a task 0.30 0.39 i171 Makes careless errors in daily tasks (R) 0.13 0.39 i170 Does not readily switch from one activity to another (R) 0.02 0.39 i157 Listens to another perspe ctive without arguing -0.03 0.38 i132 Talks at the wrong time (R) -0.02 0.37 i169 Able to make a quick, simple direction 0.08 0.34 i178 Buys unnecessary items that look appealing (R) 0.20 0.32 i180 Bothers other people whil e they are working (R) 0.27 0.31 i172 Gets stuck in a topic (R) -0.01 0.31 i176 Jumps to a solution when attempting to solve a problem (R) -0.06 0.31 i173 Does not stop or apologizes when bothers others (R) -0.05 0.24 i179 Interrupts while someone is talking on the phone (R) 0.11 0.23

PAGE 70

70 Table 2-18. Factor pattern of the emotiona l management domain for patient data Items F1 i195 Overreacts to challenges (R) 0.77 i189 Accepts help without losing temper 0.72 i196 Gets frustrated/upset when having to wait (R) 0.71 i186 Gets upset with a change of routine (R) 0.68 i197 Overreacts to frustrating situations (R) 0.59 i188 Accepts constructive critici sm without losing temper 0.57 i194 Has angry or tearful outbursts for no reason (R) 0.56 i191 Listens to another pers pective without arguing 0.54 i198 Frustration increases to get physical (R) 0.54 i185 Blames others for problems or mistakes (R) 0.51 i190 Stops an activity and star ts a new one without upset 0.49 i193 Becomes tearful easily when upset (R) 0.43 i187 Does not react when peop le are visibly upset (R) 0.40 i192 Calms down after an argument 0.33 Table 2-19. Factor pattern of the emotiona l management domain for caregiver data Items F1 i197 Overreacts to frustrating situations (R) 0.72 i195 Overreacts to challenges (R) 0.72 i188 Accepts constructive critici sm without losing temper 0.71 i189 Accepts help without losing temper 0.70 i185 Blames others for problems or mistakes (R) 0.65 i191 Listens to another pers pective without arguing 0.64 i186 Gets upset with a change of routine (R) 0.55 i194 Has angry or tearful outbursts for no reason (R) 0.46 i196 Gets frustrated/upset when having to wait (R) 0.46 i190 Stops an activity and star ts a new one without upset 0.43 i198 Frustration increases to get physical (R) 0.38 i187 Does not react when peop le are visibly upset (R) 0.34 i192 Calms down after an argument 0.24 i193 Becomes tearful easily when upset (R) 0.24

PAGE 71

71 Table 2-20. Rotated factor pattern of the social communication domain for patient data Items F1 F2 i223 Blurts out something off topic (R) 0.82 -0.09 i227 Interrupts other people conversation (R) 0.77 -0.03 i218 Gets stuck on a topic (R) 0.70 -0.04 i221 Jumps to a topic unrelated to the conversation (R) 0.69 -0.08 i220 Asks embarrassing questions/makes hurtful comments (R) 0.67 0.00 i219 Talks at the wrong time (R) 0.67 0.06 i226 Interrupts while someone is talking on the phone (R) 0.65 -0.01 i225 Repeats a story that has already been told (R) 0.64 0.06 i228 Provides too much info when telling something (R) 0.63 -0.07 i222 Walks away from conversation before it is finished (R) 0.62 0.06 i217 Gets too close when talking to someone (R) 0.57 -0.02 i224 Loses train of thought in a conversation (R) 0.47 0.12 i215 Sounds rude or demanding when making a request (R) 0.46 0.10 i216 Facial expression does not match the conversation (R) 0.36 0.22 i208 Acknowledges another pers on point of view -0.07 0.74 i207 Shows interests in what ot her people are saying -0.13 0.73 i210 Participates in a 1020 min conversation -0.01 0.71 i211 Picks out important info fr om a lecture/instruction 0.05 0.63 i202 Able to talk with more than one person at a time 0.24 0.62 i203 Begins to answer questions within an appropriate time 0.24 0.61 i212 Adds a new topic to a convers ation at the right time -0.06 0.61 i209 Keeps up with a convers ation without repeat 0.00 0.60 i199 Gets a person attention b4 starting a conversation 0.04 0.60 i201 Greets person when someone enters the room 0.03 0.57 i213 Starts a conversation 0.02 0.54 i204 Provides enough info when telling something 0.17 0.52 i200 Allows others to take a turn in a conversation 0.25 0.48 i206 Appropriate eye contact wh en having a conversation 0.02 0.47 i205 Faces the person when speaking -0.18 0.46 i214 Misunderstands what the speaker intends (R) 0.13 0.15

PAGE 72

72 Table 2-21. Rotated factor pattern of the social communication domain for caregiver data Items F1 F2 i218 Gets stuck on a topic (R) 0.87 -0.21 i219 Talks at the wrong time (R) 0.78 0.07 i225 Repeats a story that has already been told (R) 0.64 -0.05 i227 Interrupts other people conversation (R) 0.64 0.10 i216 Facial expression does not match the conversation (R) 0.56 0.19 i220 Asks embarrassing questions/makes hurtful comments (R) 0.55 0.19 i221 Jumps to a topic unrelated to the conversation (R) 0.55 0.20 i228 Provides too much info when telling something (R) 0.55 -0.07 i223 Blurts out someth ing off topic (R) 0.53 0.24 i226 Interrupts while someone is talking on the phone (R) 0.52 0.16 i212 Adds a new topic to a conve rsation at the right time 0.48 0.42 i224 Loses train of thought in a conversation (R) 0.47 0.23 i214 Misunderstands what the speaker intends (R) 0.46 0.16 i215 Sounds rude or demanding when making a request (R) 0.40 0.33 i222 Walks away from conversation before it is finished (R) 0.36 0.32 i217 Gets too close when talking to someone (R) 0.36 0.16 i199 Gets a person attention b4 starting a conversation 0.28 0.10 i203 Begins to answer questions within an appropriate time -0.21 0.76 i209 Keeps up with a convers ation without repeat 0.09 0.69 i207 Shows interests in what other people are saying 0.16 0.67 i213 Starts a conversation -0.11 0.66 i208 Acknowledges another pe rson point of view 0.23 0.64 i202 Able to talk with more than one person at a time 0.09 0.62 i201 Greets person when someone enters the room 0.06 0.57 i210 Participates in a 10-20 min conversation 0.26 0.57 i205 Faces the person when speaking 0.06 0.56 i204 Provides enough info when telling something 0.11 0.53 i200 Allows others to take a turn in a conversation 0.34 0.44 i211 Picks out important info fr om a lecture/instruction 0.20 0.41 i206 Appropriate eye contact wh en having a conversation 0.07 0.40

PAGE 73

73 Table 2-22. Summary of the number of Cross loaded and not loaded items Domains Raters # of cross loaded items/ # of total items # of not loaded items/ # of total items Attention Patient 11/50 5/50 Caregiver 13/50 2/50 Memory Patient 5/33 1/33 Caregiver 5/33 2/33 Processing speed Patient NA 4/28 caregiver NA 2/28 Executive function Patient 3/62 7/62 Caregiver 7/62 6/62 Emotional management Patient NA 0/14 Caregiver NA 2/14 Social communication Patient 0/30 1/30 caregiver 4/30 1/30 NA represents not applicable: only single factor was extrac ted for the processing speed and emotional management domains; theref ore, there is no cross loaded issue

PAGE 74

74 CHAPTER 3 PSYCHOMETRIC PROPERTIES OF THE CO MPUTER ADAPTIVE MEASURE OF FUNCTIONAL COGNITION FOR TRAUMATIC BRAIN INJURY Introduction The Computer Adaptiv e Measure of Functi onal Cognition for Traumatic Brian Injury (CAMFC-TBI) was recently developed to overcome the limitations of existing cognitive measures for everyday activities. Functional cogn ition is defined as the ability to complete everyday activities that primarily require cogn itive abilities (Donovan et al., 2008). This measure was designed to address the comm on cognitive problems that indi viduals with traumatic brain injury experience in their everyday life. Con ceptually, the CAMFC-TBI was intended to address six cognitive domains that are most frequently a ffected by TBI: attention, memory, processing speed, executive function, emotional management and social communication (Johnston & Hall, 1994; Silver et al., 2005). To empirically validate the factor structur e of the CAMFC-TBI, confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) were conducted. While the domain or subdomain structure could not be s upported with CFA, EFA provided at least par tially support for domain based structure (i.e., attention, memor y, processing speed, etc. represented individual construct). Based on the EFA, the first factor, whic h is the factor that ac counted for the largest variance, accounted for 26% to 75% of total variances across si x domains for patient data. The first factor accounted for 38% to 69% of tota l variance across six domains for caregiver data. Moreover, EFA extracted only one interpretable factor for Emotional management and process speed domains indicating a single factor structur e of these domains. For the remaining domains of the CAMFC-TBI (attention, memory, executive function and social communication domains), 26% to 48% of the total variance were reflected in the dominan t factor for patient data and 38% to 58% of the total variance were reflected in the dominant factor for caregiver data.

PAGE 75

75 The above factor analysis finding of the CAMFC-TBI suggested that this measure actually consists of a battery of six scales. Ther efore, the next logical question is, what are the measurement qualities of each of the separate domain scales? A contemporary method of identifying the measurement qualities of new scales is Item Response Theory (IRT), in particular the Rasch analysis, 1-parameter l ogistic rating scale model. The a dvantages of IRT are that it can be used to achieve sample invariance and test free measurements (Crocker & Algina, 1986; Fan, 1998; Velozo et al., 2008). That is, item prope rties, such as item difficulty and item discrimination, do not vary when applied to differe nt samples. Moreover, the estimated abilities of examinees can be compared across individuals without taking the same set of items. Because of these features, IRT methodologies are the necessary precu rsor to contemporary test administrations such as com puter adaptive testing (CAT). IRT approaches have been applied to seve ral functional cognitive measures. Whyte and colleagues applied Rasch anal ysis to the Moss Attention Rating Scale (MARS), a 45 item clinician-rated assessment of attention deficits in everyday activities for TBI (Whyte et al., 2003). Even though Rasch analysis identified 3 of 45 items as misfitting, the MARS showed high person separation and reliability (5.69 and 0.97 respectively). Its internal consistency was also high (Cronbach of 0.95). Hart and colleagues further in vestigated the fact or structure and measurement qualities of the MARS using EFA, CFA and Rasch analys is (Hart et al., 2006). They identified three factors (Restlessness/Dist ractibility, Initiation, and Sustained/Consistent Attention) and created a 22-item version of the MARS. Th is 22-item MARS exhibited fair measurement qualities. Person separation and reliability for three factors, Restlessness/Distractibility, Initiation, and Su stained/Consistent Atte ntion, were 1.31 and 0.63; 1.23 and 0.6; and 0.89 and 0.44, respectively.

PAGE 76

76 Rasch analysis was also applied to the Dy sexecutive Syndrome (DEX), part of the Behavioral Assessment of the Dysexecutive syndrome (BAD), the most commonly used questionnaire for assessing executive functi on problems in everyday life (Wilson, Alderman, Burgess, Emslie, & Evans, 1996). The DEX is a 20-item observational scale rated by the patient and caregiver. Chan and Bode investigated diffe rential item functioning (DIF) between patient and caregiver ratings (Chan & Bode, 2008). Th e authors presented only limited results, identifying 3 of 20 items as misfitting and 5 of 20 items showing DIF (Chan & Bode, 2008). Finally, the Applied Cognition Sc ale, an instrument similar to the CAMFC-TBI, was also evaluated using Rasch analysis. This scale, part of the Activity Measure for Post-Acute Care, was designed to use self-report format to monitor applied cognition of Postacute patients (Coster et al., 2004; Haley, Coster Andres, Ludlow et al., 2004). Similar to functional cognition, applied cognition was defined as discrete functi onal activities whose performance depends most critically on application of cognitive skills, with limited movement requirements (Coster et al., 2004). The Applied Cognition Scale is not a diagnosis specific measure and has been used to assess patients from inpatient rehabilitation services, transitional care un its, ambulatory services and home care (Haley, Coster, Andres, Ludlow et al., 2004). While cei ling effect (25% of patients reached the maximum score) and item mi sfittings (13 of 59) were identified, person separation and reliability (1.80 a nd 0.77) and item separation and reliability (2.58 and 0.87) were satisfactory. The factor analysis findings of Chapter 2 suggested that the CAMFC-TBI consists of a battery of six scales. In this study, we used Rasch analysis to inve stigate the item-level psychometric properties of the six scales of the CAMFC-TBI: attention, memory, processing speed, executive function, emotional management and social communication. The aim is to

PAGE 77

77 evaluate measurement qualities of each of the domain scales. The objectives of this study are to investigate ceiling/floor effects, item fit, person reliability and person separation of each domain scale of the CAMFC-TBI for individuals with TBI. Methods Participants This study was conducted using the existing data from the study Developing a Com puter Adaptive TBI Cognitive Measure, a NIH f unded grant #5R21HD045869-03. This study was approved by the Institutional Review Board of th e University of Florida and the Research and Development Committee at the North Flor ida/South Georgia VA Medical Center. Ninety individuals with TBI, 89 caregivers and 47 healthcare professionals associated with these individuals with TBI were recruited from three sites: Shands Hospital, Brook Health Systems and the Shepherd Center, Atlanta, Georgi a. The inclusion criteria for patients were (1) patients who had self-identified as having a traumatic brain injury; (2) patient s were at one of the two phases of recovery (outpatient or one-year post injury); (3) patients were between age 18 and 85; (4) patients did not report having previous di agnosis of schizophrenia or psychotic disorder or mental retardation; and (5) patients spoke E nglish as their first language. Caregivers, defined as a friend or family member who observes the daily functioning of the individual with TBI, were approached for consent after the indi vidual with TBI was recruited. Healthcare professionals were recruited afte r the individual with TBI agr eed to participate. Table 3-1 represents the number of participants collected across recovery stages in the existing database. The demographics of the participants are summarized in Table 3-2 and Table 3-3. Instruments The CAMFC-TBI consists of 228 items across six dom ains: attention (52 items), memory (35 items), processing speed (33 items), executive function (64 items), emotional management

PAGE 78

78 (14 items), and social communication (30 items). Each item of the CAMFC is rated with a 4point rating scale: never, sometimes, often, and al ways. If the participan ts have not performed the activity for the past week, they are in structed to answer not applicable. Data Analysis A rating scale Rasch analysis was conducted using Wi nsteps software program (Linacre, 2004a). Rasch analysis is a 1-parameter logistic model (1-PL). Unlike 2-PL models which allow each item to have different discrimination parameters, 1-PL models assume all items have a constant item discrimination parameter. The Rasc h model, the most parsimonious IRT model, is preferred in the rehabilitation field because of its simplicity and ease of interpretation (Jette & Haley, 2005). Since the Rasch model assumes that the item discrimination parameter is constant across all items and does not consider the guessin g parameter, the Rasch model does not require larger sample sizes like the 2 or 3 parameter IR T models (Jette & Haley, 2005). Even with small sample sizes (n=25), the Rasch model still main tains its robust estimation (de Gruijter, 1986). In addition, Wang and Chen demonstrated that in Rasch model standard deviati ons were very stable while sample size increased from 100 to 2000 (Wang & Chen, 2005). Rasch model is based on the concept that when the persons ability matches the item difficulty, he/she has 50% probability to endorse th e item. In contrast, when persons ability is a higher than item difficulty, the probability of endorsing the item is a higher than 50% and when persons ability is a lower than item difficulty, th e probability of endorsing the item is a lower than 50%. The Rasch rating scale formula is presented below. Log[Pnik /(1-Pnik)]=Bn-Di-Fk Where Pnik=probability of person n who passes item i; 1-Pnik =probability of person n who fails item i; Bn =person ability; and

PAGE 79

79 Di =item difficulty for item i Fk =item difficulty of the k threshold Differential item functi oning (DIF) analysis DIF analysis indica tes whether item calibrations are significantly different across subgroups. We first conducted uniform DIF analysis to examine the item calibration equivalence across our two sub-groups (the participants from outpatient rehabi litation settings and participants who were one-year post injury). If no DIF across these subgroups were confirmed, we could combine these sub-groups for later Ra sch analysis. The t statistics provided by Winsteps program and an alpha level of 0.05 with Bonferroni adjustment for multiple comparisons were used to determine significant DIF. The adjusted P-values for significant DIF were listed in Table 3-4. We defi ned no DIF as less than 5% of items per domain with DIF. The item level psychometric properties We conducted a rating scale Rasch analysis us ing Wi nsteps program to investigate item statistics and person statistics of the CMAFC-TBI. Item stat istics including item fit, item reliability, and item separation we re evaluated. In addition, person statistics including person fit, person spread, ceiling/floor effects, person reliability and person separation were evaluated. Item fit is determined by the fit st atistics of each item provided by the Winsteps program. The Winsteps program provides two types of fit statis tics: information-weighted mean square (infit MnSq) and outlier-sensitive mean square fit statis tic (outfit MnSq). By their statistical nature, infit statistics are more responsiv e to the variance of those well-targeted observations while outfit statistics are more sensitive to the variance of outliers or extreme observations. For the focus of this paper, only infit statistics were used to determine item fit. An infit MnSq value equal to 1 indicates the observed response patterns perfectly match the model predic ted pattern. Infit MnSq statistics ranging from 0.6 to 1.4 are considered adequate fit for survey data (Bond & Fox, 2001).

PAGE 80

80 Item reliability represents how well the estimate of the item measures can be replicated when another sample with comparable abili ty are given the same set of items. Item separation estimates how well the items are separated by the measured variable. Similar to item fit, Person fit determined by the fit statistics of persons was also provided by Winsteps program. Person misfit indicates th e person has erratic responses that may be caused by guessing or misunderstanding the meanings of items. Person spread is defined as the difference between maximum person m easure and minimum person measure. Ceiling effect is defined as more than 5% of participants re ach the maximal score, which indicates these participants real abilities are too high to be assessed by this measurement tool while floor effect is defined as more than 5% of participants are rated the minimal score, which indicates that these participants real abilities are too low to be assessed by this m easurement tool. In terms of a reliability index Person reliability in Rasch analysis represents how well the estimate of the persons ability can be replicated when other se ts of items measuring the same construct are given to the same sample of persons. This person reliability as analogous to Cronbachs alpha with values between 0 and 1. Person separation estimates how well the measure separates persons. Person separation is formulated in term s of standard error un its. The value of person separation is not restricted betw een 0 and 1. The statistically di stinct ability strata can be obtained by applying the person se paration in the formula (4Gp+1)/3, where Gp represents the person separation index(Wright & Masters, 1982). An assessment needs at least two strata to significantly distinguish pe ople minimally into a high or low ability group. Results Differential Item Functioning (DIF) Analysis No item showed significant DIF between the outpatient subgroup and one year post injury subgroup. Therefore, these two groups were combined for all subsequent analyses.

PAGE 81

81 The Item Level Psychometric Properties Item Statistics. Table 3-5 presents item misfit, reliab ility and separation. Regardless of rater groups, the average of the Executive function and Emotional management domains showed the highest percentage of item misfitting with 12% and 14%, respectiv ely while the attention and memory domain showed the lowest percentage of item misfitting with 7.33% and 7.67%, respectively. In terms of raters patients had a tendency to show lowest percentage of item misfitting except for the Processing speed and Executive function domains, where healthcare professionals showed the lowest percentage of item misfitting. In contrast, caregivers had a tendency to show the highest percentage of item misfitting, especially on the emotional management and social communication domains. All the misfitting item are identified in Appendix C. Item reliabilities were high across all domains and all raters, ranging from 0.85 to 0.95, being relatively lower for the Processing speed domain with 0.86, 0.87, and 0.88 for patient, caregiver, and healthcare professional ratings, respectiv ely. Within domains, item reliabilities were similar across raters. Item separations were high across all domains ranging from 2.41 to 4.42, being relatively lower for th e processing speed domain with an average of 2.59. In terms of raters, patients had a tendency to show lower item separation especially on the attention and executive function domains, while healthcare prof essionals had a tendency to show higher item separation except for the Emotional management domain. Person Statistics. Table 3-5 also presents person misfit, reliability and separation. Regardless of rater groups, th e average of percentages of person misfitting were similar across all domains ranging from 8% to 12%, except for the Executive function domain, where the average percentage of person misfitting was slightly higher with 18.33%. Relative to raters, healthcare professionals showed a lower percentage of person misfitting than patients and caregivers. on

PAGE 82

82 Emotional management domain (4% versus 9% and 11% respectively). In general, the means of persons abilities were higher than the means of item difficulties across all domains and all raters (see Person Mean and SD in Table 3-5 ). This indi cated that persons of this sample are slightly more able than the mean of item difficulty. However, no ceiling effects were found except for the Emotional Management domain rated by patients and healthcare professionals. No floor effects were found across any of the domains or raters. The Emotional management domain showed relative lower range of person spread with 2.69, 5.99, and 7.56 for patient, caregiver and healthcare professional ratings re spectively. Moreover, the ratings of healthcare professionals showed the greatest person spread than the other two rater groups across all six domains from 7.56 to 12.17. Person reliabilities were high across all domains a nd all raters, ranging from 0.79 to 0.96, being relatively lower for the Emotional Management domain as well (0.79-0.81). Similarly, Cronbachs alpha was high across all domains an d all raters, ranging from 0.92 to 1. Likewise, person separations were high across all domains and all raters, ranging from 2.89 to 6.89, being relatively lower for the Emotional Manageme nt domain (2.89-3.04). Within domains, person separations were similar across raters. Discussion The results of the preliminary Rasch analyses f or evaluating the item-level psychometrics properties of the six domains of CAMFC-TBI we re positive and promising. No DIF between the outpatient rehabilitation subgroup and one-year post injury su bgroup indicated that the item estimates were stable across these two stages of TBI recovery. A lthough some item misfitting were found, none of the domains contained more than 14% average item misfitting. Both item reliability and item separation were high across all domains and all raters. In terms of the person statistics, the percentages of person misfitting were low across all domains and raters except for

PAGE 83

83 the Executive function domain which showed sl ightly higher average percentage of person misfitting. The means of persons abi lities were higher than the mean s of item difficulties across all domains; however, no ceiling effects were found, except for the ratin gs of patients and healthcare professionals on the Emotional management domain. The CAMFC-TBI showed broad person spread across domains especially the ra tings of the healthcar e professionals. Finally both the person reliabilities and person separations were high across all domains and raters. Item Misfit The CAMFC-TBI showed a relative ly low percentage of item misfitting (3%-21%). In general, these percentages were lower than t hose found by Coster and co lleagues for their the Applied Cognition Scale (22%)(Coster et al., 2004). Moreover, the percentage of item misfitting for the attention domain of the CAMFC-TBI (4 %-10%) were lower than that of the MARS (15%) (Whyte et al., 2003). Th e Emotional management domain showed slightly higher percentage of item misfitting (7%, 21%, and 14% for patients, caregivers and healthcare professionals respectively), especially for the ratings of caregivers. This indicated that approximately one fifth of items in the Emo tional management domain may assess different constructs other than emotion when taking the caregivers perspective into account. Item Reliability and Separation The item reliabilities (0.85-0.95) and sepa rations (2.41-4.42) of the CAM FC-TBI were generally higher than those of the Applied Cognitive Scale (0.87 and 2.58). The high item reliabilities suggest that item measures should stay the same when applying the CAMFC-TBI to another comparable sample. In addition, the good item separation indicated that items have good spread after accounting for the measurement error.

PAGE 84

84 Person Spread In general, the CAMFC-TBI showed broad pe rso n spread especially for the ratings by healthcare professionals. The person spreads of the CAMFC-TBI were higher than the person spreads found in the Applied C ognition Scale (5.53) except for th e patient rated the emotional management and executive function domains and caregiver rated Executive function domain (2.69, 4.35 and 4.23 respectively). The estimated ab ilities from the ratings by healthcare professionals were distributed ove r a wider range than the ratings of patients and caregivers. This indicated that healthcare professionals may be more capable of distinguishing differences in functional cognition than other raters. We intended to develop a func tional cognitive measure to co ver the whole spectrum of TBI recovery stages. While our preliminary ex amination of the CAMFC-TBI only included individuals with TBI in outpati ent and one-year post rehabilitati on stages, our findings suggested the CAMFC-TBI may be also appropriate to appl y to individuals with TBI in the inpatient rehabilitation stage. The CAMFC-TBI did not show floor effects for any of the domains, suggesting that the battery may be appropriate fo r individuals with more severe TBI symptoms, such as those receiving inpatient rehabilitation. Similarly, except for patient and caregiver ratings for the emotional management domain, the CAMF C-TBI did not show ceiling effects, suggesting that the instrument may be appropriate for individuals with mild TBI. However, these inferences need to be supported by empirical studies. Person Misfit The percentages of person misfitting on the CAMFC-TBI, in general, were low. Except for the Executive function domain, the percentages of person misfitting on the CAMFC-TBI (4%15%) were similar with that of the applied cogn ition measure developed by Coster and colleague (8%). The executive function domain showed sligh tly higher percentages of misfitting persons

PAGE 85

85 (18%, 18% and 19% for patient, ca regiver and healthcare professiona l, respectively) than other domains. Person misfitting reflects persons responses that did not match the models expectation. In general it can be caused by guessi ng or misunderstanding the meaning of items. Since executive function is a more complex syst em, it could be more challenging to rate the activities assessing this construct. While we suspected that self-report from individuals with TBI might show higher person misfitting rate since thei r cognitive deficits might cause inconsistent or erratic responses, our findi ngs did not support our postulati on. No patterns showed that patients self-report resulted in higher erratic responses. Person Reliability and Separation Person reliability and person separation were high for the CAMFC-TBI. Compared with the Applied Cognitive S cale (0.77 and 1.8), the CAMFC-TBI showed better person reliability and separation (0.79-0.96 and 2.89-6.89). Moreover the CAMFC-TBI was able to distinguish ability levels into more categories (3 to 7 categories depending on domains) than the Applied Cognitive Scale (2 categories). These findings suggested that the CAMFC-TBI may be more sensitive to monitoring change th an the Applied Cognitive Scale. Unidimensionality A limitation of this study is that Rasch an alysis was conducted under the assumption that each domain represents a single dimension. In our previous study (Chapter 2) we failed to confirm unidimensionality of the domains with confirmatory factor analysis. While we tried to confirm the unidimensionality of each domain prior to conducting Rasch analysis, it should be noted that the concept of unidimensionality is often debatable. Cook and colleague (2003) stressed that the statistical model assumptions are always violated to a certain degree (Cook, Monahan, & McHorney, 2003). Box and Draper (1987) stated that all models are wrong; the most empirical question is to what degree of violation is allowable (Box & Draper, 1987). Since

PAGE 86

86 a measure is never truly unidimensional (Reckase, 1985), we may apply more practical criterion: essential unidimensionality, that is ignoran ce of idiosyncratic, me thodological or trivial dimensions (W. Stout, 1987; W. F. Stout, 1990). Under this definition of unidimensionality, Rasch analysis of the CAMFC-TBI may be justif ied. In future research, we should investigate the measurement qualities of the sub-domains that were found in our previous research (Chapter 2). These future studies will uncover the measurement qualities of other factor structures of the CAMFC.

PAGE 87

87 Table 3-1. Number of particip ants across recovery stages Outpatient Rehabilitation One Year Post Injury Total 47 patient ratings 46 caregiver ratings 47 healthcare profe ssional ratings 43 patient ratings 43 caregiver ratings 90 patient ratings 89 caregiver ratings 47 healthcare professional ratings Table 3-2. Demographics statistics for patients and caregivers Demographics Patient (N=90) Caregiver (N=89) Age Mean SD (year) Range 38.2 15.8 18-84 49.9 14.6 21-88 Gender, n (%) Male Female 63 (70%) 27 (30%) 70 (21.3%) 19 (78.7%) Ethnicity, n (%) White African American Hispanic Other Missing 72 (80%) 10 (11.1%) 4 (4.4%) 3 (3.3%) 1 (1.1%) 75 (84.3%) 10 (11.2%) 3 (3.4%) 1 (1.1%) Education, n (%) 8th grade 9th grade 10th grade 11th grade GED 12th grade Some college, no degree Some college, earned degree (AA/AS) Completed 4-year college, earned degree (BS/BA) Graduate/Professional degree (MA/MS/PhD) 2 (2.2%) 0 (0%) 7 (7.8%) 6 (6.7%) 5 (5.6%) 19 (21.1%) 23 (25.6%) 14 (15.6%) 7 (7.8%) 7 (7.8%) 1 (1.1%) 1 (1.1%) 2 (2.2%) 3 (3.4%) 4 (4.5%) 13 (14.6%) 27 (30.3%) 20 (22.5%) 11 (12.4%) 7 (7.9%) Income, n (%) Under $5,000 $5,000-$10,000 $11,000-$15,000 $16,000-$20,000 $21,000-$35,000 $35,000-$50,000 Over $50,000 9 (10%) 3 (3.3%) 2 (2.2%) 8 (8.9%) 11 (12.2%) 24 (26.7%) 32 (35.6%) 2 (2.2%) 3 (3.4%) 7 (7.9%) 5 (5.6%) 18 (20.2%) 16 (18%) 36 (40.4%) Work, n (%) Yes No Missing 17 (18.9%) 72 (80%) 1 (1.1%) Drive, n (%) Yes No Missing 27 (30%) 59 (65.6%) 4 (4.4%)

PAGE 88

88 Table 3-3. Demographics statisti c for healthcare professionals Healthcare Professional (N=47) Age Mean SD Range 40.0 10.9 years 25-55 years Years of experience Mean SD Range 13.7 9.5 years 1-31 years Gender, n (%) Male Female 3 (6%) 44 (94%) Ethnicity, n (%) White African American Missing 45 (95.8%) 1 (2.1%) 1 (2.1%) Employer n (%) Brooks Rehabilitation Shepherd Center Shands Hospital 33 (70.2%) 13 (27.1%) 1(2.1%) Occupation n (%) Care manager Cognitive Rehabilitation Therapist Occupational Therapy Physical Therapy Speech Language Pathologist Resident 4 (8.3%) 25 (54.2%) 4 (8.3%) 6 (12.5%) 7 (14.6%) 1 (2.1%)

PAGE 89

89 Table 3-4. Adjusted P-values for each domain Domains Attention Memory Processing Speed Executive Function Emotional Management Social Communication Number of item 52 353364 14 30 Adjusted P-value .000962 .001429.001515.000781.003571 .001667

PAGE 90

90Table 3-5. Summary results of Rasch analysis Attention 52 items Memory 35 items Processing Speed 33 items Executive Function 64 items Emotional Management 14 items Social Communication 30 items Rater PT CG HP PT CG HP PT CG HP PT CG HP PT CG HP PT CG HP Item Misfittting 2 (4%) 4 (8%) 5 (10%) 1 (3%) 3 (9%) 4 (11%) 4 (12%) 3 (9%) 3 (9%) 8 (13%) 9 (14%) 6 (9%) 1 (7%) 3 (21%) 2 (14%) 1 (3%) 5 (17%) 3 (10%) Average of item misfit 7.33% 7.67% 10% 12% 14% 10% Item Reliability 0.9 0.94 0.95 0.93 0.93 0.93 0.86 0.87 0.88 0.89 0.92 0.91 0.92 0.92 0.85 0.92 0.91 0.94 Item Separation 2.99 4.07 4.42 3.51 3.56 3.57 2.58 2.81 2.48 2.83 3.31 3.11 3.38 3.46 2.41 3.34 3.1 4.06 Person Misfitting 9 (10%) 13 (15%) 5 (11%) 9 (10%) 9 (10%) 5 (11%) 8 (9%) 7 (8%) 6 (13%) 16 (18%) 16 (18%) 9 (19%) 8 (9%) 10 (11%) 2 (4%) 11 (12%) 12 (13%) 5 (11%) Average of person misfit 12% 10.33% 10% 18.33% 8% 8.67% Person Spread range -0.46 5.91 -1.5 4.61 -3.78 8.39 -0.83 4.75 -2.41 5.25 -4.26 4.66 -0.87 5.32 -1.96 5.55 -3.58 4.5 -0.23 4.12 -1.02 3.21 -3.81 6.34 -0.85 1.84 -1.56 4.43 -0.89 6.67 -0.37 5.17 -1.39 6.65 -1.61 8.11 Person Spread 6.37 6.11 12.17 5.58 7.66 8.92 6.19 7.51 8.08 4.35 4.23 10.15 2.69 5.99 7.56 5.54 8.04 9.72 Person Reliability 0.92 0.9 0.96 0.9 0.94 0.94 0.87 0.89 0.86 0.92 0.95 0.96 0.8 0.81 0.79 0.88 0.93 0.92 Person Separation 4.87 5.21 6.89 4.39 5.76 5.81 3.77 4.08 3.64 4.92 5.96 6.6 2.97 3.04 2.89 4.03 5.03 4.73 Cronbachs Alpha 0.98 0.98 1 0.97 0.99 1 0.98 0.98 1 0.96 0.99 1 0.93 0.92 1 0.93 0.97 0.97 Person Mean SD (logits) 1.14 0.92 1.32 1.34 1.35 2.11 1.53 1.22 1.29 1.59 1.26 2.12 1.01 1.04 0.64 0.96 0.66 1.69 1.55 1.19 1.48 1.3 2.45 1.52 1.55 1.19 1.48 1.3 2.45 1.52 1.75 1.11 1.53 1.39 2.09 1.35 Ceiling/Floor N/N N/N N/N N/N N/N N/N N/N N/N N/N N/N N/N N/N Y/N N/N Y/N N/N N/N N/N PT=patient CG=caregiver HP=healthcare professional Y=yes N=no SD=standard deviation Person spread is defined as the difference between maximum person measure and minimum person measure

PAGE 91

91 CHAPTER 4 RATER EFFECTS ON THE CO MPUTER ADAPTIVE ME ASURE OF FUNCTIONAL COGNITION-TBI Introduction Reliability and Validity are two ess ential psychometric propertie s to evaluate the soundness of instruments (Portney & Watkins, 2000a ). The first prerequisite, reliability, is to evaluate the degree of consiste ncy of a measurement tool (Portney & Watkins, 2000a). For example, how consistent are the results from more than one rater rating the same examinee (inter-rater reliability), how stable are the re sults from a rater rating the examinee repeatedly (intra-rater reliability), etc.. Rater reliability assessing the ex tent of inconsistency due to observers/raters is often investigated in psychom etric studies. While measurement error consist of systematic error and random error, reliability only accounts for the degree of random errors. Besides random error, the ratings of different ob servers/raters on the same subjects may also be the result systematic error known as rater effects. The terms rater effects, rater bias, and r ater error are not cl early defined and often used interchangeably. In this paper, we used the term rater effect to mean systematic variance caused by different raters assigning ratings not th e performance of ratees (Scullen et al., 2000). Although not as common as reliab ility studies, a considerable lit erature has investigated rater effects, more specifically on rater leniency/sev erity (Cusick et al., 2000; Goldstein & McCue, 1995; Hart et al., 2003; Hendryx, 1989; Leathem et al., 1998; Malec, 2004; Sander et al., 1997; Tepper et al., 1996). Rater severity /leniency is defined as the tende ncy that a rater assigns ratings consistently lower/higher than ot her raters assign (Myford & Wolf e, 2004). The results of rater effects are varied depending on assessments, constructs, and raters. However, there is a general agreement that items assessing physical ability have higher agreement among raters than do items assessing cognitive ability or emoti onal problems (Hart et al., 2003; Hendryx, 1989;

PAGE 92

92 McKinlay, 1984; M. Sherer et al ., 1998; Tepper et al., 1996). Since rater effects are more evident with cognitive domains, investigat ing rater effects of cognitive meas ures is especially crucial. Cognitive deficits are the most concerning problem for patients with moderate to severe traumatic brain injury (TBI) and their caregiver s (Brooks et al., 1986; Hellawell et al., 1999). This population often requires extensive rehabilitation programs (Lezak, 2004). Cognitive activities are the most important outcomes for ev aluating cognitive rehabilitation (Stuss et al., 2008) The most convenient method to assess patien ts cognitive activities in daily life is to ask patients directly (Heaton & Pendl eton, 1981). This self-report format has been adopted in many outcome measures applied in TBI, such as the Community Integration Questionnaire (CIQ)(Willer et al., 1993), Mayo-Portland Adaptabi lity Inventory (MPAI) (Malec, 1994), Patient Competent Rating Scale (PCRS) (G. P. Prigatano et al., 1986), Behavior Rating Inventory of Executive Function (BRIEF) (Roth et al., 1996 ), and Dysexecutive Questionnaire (DEX) (Wilson et al., 1996). Besides its advantages of convenience and economy, self-report questionnaires incorporate the affected persons perspective into the assessment of everyday difficulties (Leathem et al., 1998). While self-re port questionnaires are common for cognitive outcome measures, using se lf-report with a TBI popul ation raises a concern. TBI, especially with frontal lobes damage, is often associated with impaired self-awareness (ISA) (Stuss & Benson, 1986). As a result of ISA, individuals with TBI te nd to overestimate their ab ilities particularly on cognitive and emotional function (McKinlay, 19 84). Their self-report information might be compromised simply because of cognitive de ficits (G. Prigatano & Schacter, 1991). Other reporting sources or proxies might be mo re reliable reporters of cognitive deficits. Caregivers (significant others) and healthcare professionals are common proxies for reporting outcomes of individuals with TBI Caregivers have the advant age of having many opportunities

PAGE 93

93 to observe patients functional performance on daily basis and providing information for those patients who are incapable of answering ques tionnaires (Magaziner et al., 1988). Several assessments have included caregivers to report pa tients cognitive deficits, such as the CIQ (Willer et al., 1993), MPAI (Malec, 1994), PCRS (G P. Prigatano et al., 1986), BRIEF (Roth et al., 1996), and DEX (Wilson et al., 1996). Howe ver, caregiver reports have potential disadvantages. Heaton and Pendleton (1981) stated that the ratings of family members may be biased by their relations hip, feelings and emotion for the pa tients (Heaton & Pendleton, 1981). In addition, because of feeling responsible for th e patients safety, car egivers may be more conservative in reporting the patients abilitie s in certain activities (Malec, Machulda, & Moessner, 1997). Moreover, affected by the burden, caregiver may tend to remember the patients undesirable behaviors, which may lead to negative ra tings (Malec, 2004). Another important source of ratings are those of healthcare professionals. Healthcare professionals have the advantage of being particularly objective, si nce they are trained to assess and treat patients. A number of assessments are rated by healthcare professionals, such as the Functional Independence Measure (Hamilton et al., 1987), Moss Attention Rating Scale (Whyte et al., 2003), Minimum Data Set (Morris et al., 1990), and Gl asgow Outcome Scale (Jennett & Bond, 1975). Many of these assessments target the acu te and inpatient stages since those are the stages when healthcare professionals have the most contact with patients. While healthcare professional ratings are extensivel y used, these ratings have cons iderable limitations. Healthcare professionals are particularly subject to biases due to their professional training and employment positions. For example, Wolfson, Doctor and Burn s (2000) demonstrated that the clinicians ratings on patients function were affected by heuristics, that is their judgments are biased by external information, such as the inherent di fficulty of the task (Wolfson, Doctor, & Burns,

PAGE 94

94 2000). For example, clinicians may assume that TBI cognitive deficits are more severe than patients view their cognitive deficits. Furthermore, when outcomes are used to decide payment, bias can result in imprecision in functional outcome measurements (Wolfson et al., 2000). As patients need to show sufficient deficits to enroll in the treatment, healthcare professionals might consciously or unconsciously set th eir rating criterion higher than ca regivers or patients. Finally, a particular concern is that he althcare professionals may have too few opportunities to observe patients daily function and may not appropriately assess which basic impairments translate into challenges in the real-world (Malec, 2004). Health care professionals have very limited time with patients especially for those patients receiving ou tpatient treatment. Without directly observing patients everyday activities, hea lthcare professionals can only estimate a patients functional cognition based on the clinical measures, which do not account for varied environmental factors. All of above factors have poten tial to cause the systematic differences in the ratings. The above studies suggest the importance of comparing self, caregiver, and healthcare professional reports for TBI. A number of investigators ha ve reported rater differences (concordance) on questionnaires applied to the TBI population (Cusick et al., 2000; Goldstein & McCue, 1995; Hart et al., 2003; Hendryx, 1989; Leathem et al., 1998; McKinlay, 1984; Sander et al., 1997; Tepper et al., 1996). Tepper and colleagues (1996) inve stigated the rater agreement of 148 individuals with TBI on the CIQ. They rev ealed that patients and th eir significant others had poor agreement on the ratings of the home inte gration construct of th e CIQ (Tepper et al., 1996) Sander and colleagues (1997) studied rater effects on the CIQ using 122 individual with one-year post TBI and their significant others (Sander et al., 1997). The authors found that individuals with TBI rated themselves significan tly higher than did th eir significant others on home integration construct of the CIQ. Leathem and colleagues (1998) inve stigated rater effects

PAGE 95

95 on 53 individuals with TBI. They found that indi viduals with severe TBI underestimated their difficulties on cognitive and emotional items of th e PCRS compared with their significant others. It should be noted that the author s also revealed that the ratings of individuals with mild or moderate TBI were consistent with the ratings of their significant others on cognitive and emotional items (Leathem et al., 1998). Cusick and college (2000) compar ed the ratings of 204 individuals with TBI to the ra tings of their significant othe rs on the CHART, CIQ and FIM (Cusick et al., 2000). They also found that in gene ral, patients tended to rate themselves less impaired on the items related to cognition. Ho wever, when they divided patients based on severity levels of TBI, they f ound the individuals with severe TB I rated themselves more severe than did their caregivers, whic h conflicted with the findings of Leathem and colleagues, 1998). The varied results of rater effects were also found in Malecs study (Malec, 2004). He examined rater effects among three raters (individuals with acquired brain in their ou tpatient rehabilitation stage, their significant others, and clinicians) on the MPAI. By comparing 134 sets of ratings across three domains of the MPAI Malec found that the individua ls with TBI tended to rate themselves less impaired than did the signifi cant others and clinic ians on two domains: Adjustment Index and Participation Index. On the contrary, patients rated themselves more impaired than did clinicians on Ability Index of the MPAI. Studies investigating rater eff ects have showed different rate rs (patients, caregivers, and healthcare professionals) did systematically aff ect the results of outcome measures in TBI. Although the findings of severity/l eniency of raters are somewhat inconsistent across different studies, in general, studies showed that patients were more lenient or less severe in their ratings compared to proxies. The majority of these studies focused on evaluating rater severity/leniency by using correlation indices, such as Pearson correlation coefficient, Kappa coefficient and

PAGE 96

96 intraclass correlation. These methods can be used to examine rater concordance/agreement, but do not provide information on the response patterns of different raters. None of the above studies investigated whether the response patterns of raters were erratic. While including raters experiencing TBI, we considered their cognitive deficits may not only influence the leniency of ratings but may also result in erratic response patterns, for example responding no difficulty to attend to an hour long lecture but having difficulty to at tend to 10-minute activities. Correlation indices are not suffici ent to address this type of erratic response pattern issue. Therefore, we used the Many-Facet Rasch Mode l (MFRM) to investigate the potential erratic responses of raters and rater sever ity. In addition, we operationally identified the better rater as those who showed the highest discrimination es timates in this study. We hypothesized that (1) the ratings of individuals with TBI will not fit the Rasch model; (2a) there will be significant rater severities; (2b) healthcare professionals will be the most se vere raters and the individuals with TBI will be the most lenient raters; and (3) the healthcare professionals will be the better raters. Methods Participants This study was conducted using the existing data from the study Developing a Com puter Adaptive TBI Cognitive Measure, a NIH f unded grant #5R21HD045869-03. This study was approved by the Institutional Review Board of th e University of Florida and the Research and Development Committee at the North Flor ida/South Georgia VA Medical Center. Ninety individuals with TBI, 89 caregivers and 47 healthcare professionals associated with these individuals with TBI were recruited from th ree sites: Shands Hospital, Gainesville, Florida, Brook Health Systems, Jacksonville, Florida, and the Shepherd Center, Atlanta, Georgia. The inclusion criteria for patients we re (1) patients reported having had a traumatic brain injury; (2)

PAGE 97

97 patients were at one of the two phases of rec overy (outpatient or one -year post injury); (3) patients were between age 18 and 85; (4) patients did not report having a previous diagnosis of schizophrenia or psychotic disorder or mental reta rdation; and (5) patients spoke English as their first language. Caregivers, defined as a frie nd or family member who observes the daily functioning of the individual with TBI, were approached for cons ent after the individual with TBI was recruited. Healthcare professionals were recruited after the individual with TBI agreed to participate. Table 4-1 represents the number of participants collected across recovery stages in the existing database. The demographic of the pa rticipants are summarized in Table 4-2 and Table 4-3. Raters Participants rece iving outpatient rehabilitati on were rated by three ra ters: the patient (selfreport), the caregiver, and the healthcare professiona l. Participants at one-year post injury stage were rated by two raters: patient (self-report) and careg iver. Since the previous study (Chapter 3) showed no differential item functioning between the outpatient subgroup and the one-year post subgroup, we combined these two subgroups for our an alyses. That is, we used three raters in this study: 90 patient ratings, 89 caregiver rati ngs and 47 healthcare prof essional ratings. It should be noted that the healthcare professional ratings were only available for the outpatient subgroup. Instruments The CAMFC-TBI consists of 228 items across six dom ains: attention (52 items), memory (35 items), processing speed (33 items), executive function (64 items), emotional management (14 items), and social communication (30 items). Each item of the CAMFC is rated with a 4point rating scale: never, sometimes, often, and al ways. If the participan ts have not performed the activity for the past week, they are in structed to answer not applicable.

PAGE 98

98 Data Analysis Facets softwa re version 3.57 (Linacre, 2004b) which performs the Many-Facet Rasch Model (MFRM), was used to analyze rater e ffects (Linacre, 2005). The MFRM extends the rating scale Rasch model by adding one more component/facet (Cj) into the formula: Log[Pnik /(Pni(k-1))]=Bn-Dgi-Fgk-Cj Where Pnik =probability of observing category k for person n who answers item i; Pni(k-1) =probability of observing category k-1; Bn =person ability; Dgi =item difficulty for item i in group g; Fgk =the difficulty of being observed in categor y k relative to category k-1 for an item in group g; and Cj= severity of judge j, who gives rating k to person n on item i Hypothesis 1 Infit MnSqs and Outfit MnSqs of raters were used to eval uate whether there was erratic raters. Infit statistics is more responsive to th e variance of those well-ta rgeted observations while outfit statistics is sensitive to the variance of outliers or extreme observa tions. Ideal fit is when the observed response patterns exactly match the model predicted pattern (MnSq=1). Infit MnSq and Outfit MnSq ranging from 0.6 to 1.4 were cons idered adequate fit for survey data (Bond & Fox, 2001). Hypothesis 2 The rater effects were exam ined for each domain. First, the fixed chi-square was used to examine whether at least one rater consistently ex ercised the ratings differently from other raters. If the fixed chi-square test wa s significant, which means at l east one rater rated significantly severe/lenient, a t-test was then used to identify the signifi cantly different rater pairs (e.g., self versus caregiver, self versus healthcare professional, caregiver versus healthcare professional).

PAGE 99

99 An alpha level of 0.05 was used to determine a significant rater effect. Bonferroni adjustment was applied to adjust the P-value. For a two-side d t-test with three comparisons, the absolute zvalues have to be no less than 2.39 to reach statistical significance. Measure represents the average ratings of the rater in logits. The greater the measure, the more severe the rater is. Fair-M average represents the average ratings of the rater after partialing out the deviation of the ratees in that raters sample from the overall ratee mean. For example, rater A rates ratee 1, 2, 3, 4 and 5 and rater B rates ratee 4, 5, 6, 7, and 8. When the measure (average of ratee mean) of rater A is higher than B, we cannot be certain what causes the rater A to have higher measure than rater B. The reasons could be ei ther rater A tends to assign lower ratings to ratees (severe rater) or ratee 1, 2, and 3 actually ha ve lower ability than ratee 6, 7, 8. Fair-M averages adjust the ratee differences, so using Fa ir-M averages we can compare rater severities as all raters rate the same ratees. Therefore, Fair-M average provides more accurate information than measure when raters do not rate the same ratees. By combining information of the t-test and Fair-M average, we then identified significant severe/lenient raters. It should be noted that the lower the Fair-M -Average value indicates the severer rater. Hypothesis 3 Estimate discrimination is the item discr imination that is generated using the twoparameter logistic model and the generalized partial credit model, but with the restriction that the discrimination estimate cannot affect the other parameters during the computation (Linacre, 2005). Item discriminations are the slopes of the item characteristic curves (ICC), and are related to the correlations between each item response and the total scores. For the items with high discrimination, the probability of endorsing the item increases ra dically with the increment of trait (true) ability. This item can more sensitively reflect the differences of respondents abilities. On the other hand, for the items with low discrimi nation, the probability of endorsing the item is

PAGE 100

100 almost the same across all trait (true) abiliti es. Such items do not add any information for estimating the examinees ability levels. Since three raters rated the same items, if all raters interpreted and util ize the items the same way, the estimate discriminations (ED) should be the same across all raters. However, if the estima te discrimination is low on one rater but not another, that may indicate this rater misunderstands or is not attending to the answers he/she is providing. For example, if a rater rates hi gh and low-ability individuals the same the discrimination of this rater is low, and he/se does not add information to person measures. The expected value of estimate discrimination is 1 for the Rasch model. Estimate discrimination higher than 1 indicates higher item discrimination than the model expected. On the contrary, estimate discrimination lower than 1 indica tes lower item discrimination than the model expected (Linacre, 2001). Moreover, if items fail high-ability respondents but pass low-ability respondents, the items will show negative discriminations. Results Figure 4-1 de picts three facets (r aters, ratees, items) on the li near interval scale for the attention construct. The furthest left column pres ents the unit of measure in logits. The mean of item difficulties was anchored at zero. The second column displays the severity of raters. More severe raters display on the top while more lenience raters display on the bottom. Based on Figure 4-1, healthcare professionals were the most severe raters (-0.72.03), and caregivers (1.02.02) and patients (-1.02.02) were at similar levels of rater severity (Table 4-4). The third column shows the ability di stribution of the examinees, with lowest ability examinees at the bottom of the scale and highest ability examinees at the top of the scale. No examinees are at the ceiling or floor. The fourth column displays item difficulties. Easier items are at the bottom of the scale and ore difficult items are at the top of the scale. Therefore, item 1 Answer question self was rated the easiest item while item 28 Read 30 minutes with no break was considered

PAGE 101

101 the most difficult item. In addition, the items lo cated on the same horizontal level represent the same item difficulty. Thus, item 28 Read 30 minute s with no break had the same average item difficulty with item 43 Multiple things. Furtherm ore, items within two standard absolute error values were also considered the same difficu lty. The fifth column showed the likelihood of applying the rating scale in relati on with the ratees abilities. For example, if ratees estimated abilities are between 1.5 and 3 l ogit, they are most likely to receive a rating of 4 on this measurement. Figures of Facet s ruler for other domains are presented in Appendix D. Hypothesis 1 All of three rate rs ac ross six domains fit the Rasch model based on Infit and out fit MnSqs The infit MnSqs ranged from 0.73 to 1.18 and out fit MnSqs ranged from 0.74 to 1.23 (Table 44). Hypothesis 2 The fixed chi-squares showed that all six domai ns had at least one rater w ho significantly and consistently exercised the rati ngs differently than others. The values of the fixed chi-square ranged from 21.6 to 596 across the domains with 2 de grees of freedom (df). All P-values of the fixed chi-squares were le ss than 0.001(Table 4-5). The t-tests and the Fair-M Averages s howed the healthcare professionals rated significantly more severely than the patients rated themselves on the memory, executive function and social communication domains (Table 4-4 and Table 4-5). The t-tests of paired comparisons identified the memory, executive function, a nd social communicati on domains as having significant rater effects between th e ratings of healthcare professionals and patients with z-values 2.44, 2.44 and 2.41 respectively (Table 4-5). In addition to the t-te sts, the Fair-M-Averages of the healthcare professionals were lower (i.e., more severe) than patient on these three domains,

PAGE 102

102 2.86 versus 3.29 for the memory domain, 2.61 vers us 3.16 for the executive function domain, and 3.15 versus 3.35 for the social communication domain. While not statistically significan t, the t-tests and the Fair-M Averages showed trends that on the attention and processing speed domains, th e ratings of the healthcare professionals were more severe than the ratings of patients, and the ratings of the healthcare prof essionals were also more severe than the ratings of the caregivers. On the attentio n, and processing speed domains, the z values for pair comparisons between the he althcare professionals and the patients were 2.14 and 2.05 respectively, and the betw een healthcare professionals a nd the caregivers were 2.14 and 2.1 respectively, close to critical value 2.39. Additionally, on the attention and processing speed domains, the healthcare professionals had the lowest Fair-M-Averages (Table 4-4). On the emotional management domain, the t-tests and th e Fair-M Averages showed the trend that the ratings of the caregivers were mo re severe than the ratings of the patients. The Fair-M-Averages of the caregivers were lower th an the patients (3.25 versus 3.38) with Z value 2.25, close to critical value 2.39. Hypothesis 3 Estimate discrim inations for healthcare professi onals, caregivers and patients were within acceptable range (0.91-1.33, 1.04-1.11, and 0.82-1.0 respec tively) (Table 4-5). Except for the emotional management domain, patients se lf-ratings consistently had the lowest estimate discriminations among the three raters for each domain (0.82-0.92) (Table 4-5). On the emotional management domain, the estimate di scriminations showed a different order: healthcare professionals had the lowest estimate discrimination (0.91) followed by the patients (1.0). Caregivers had the highest estimate discrimina tion (1.04) (Table 4-5).

PAGE 103

103 Discussion This study demonstrated the extended applica tion of the Rasch m odel by incorporating rater facet into the Rasch model. In genera l, for the CAMFC-TBI our findings suggested a number of significant rater effect s across raters. Overall, ratings by the patients, caregivers and healthcare professionals all fit th e Rasch measurement model. While all fit the model, there were different severities across raters. He althcare professionals were significantly more severe in their ratings than patients across 3 of 6 of the domains and showed trends of being more severe on 2 of the remaining 3 domains. Furthermore, caregiver raters showed trends of being more severe than patients on the emotional management domain. Finally, estimate disc riminations were used to identify the best rater. Overall, raters showed equally-good di scrimination, suggesting no rater is better or worse than another. Our findings did not support the first hypothesis that ratings of individuals with TBI would not fit Rasch model. Unfortunately, similar studi es have not investigat ed rater fit for TBI patients. There may be several explanations why our hypothesis was not supported. While our hypothesis was based on the expect ation that individuals with c ognitive deficits may not be good raters, possibly self-ratings of functional cognition are not challeng ing for individuals with TBI. The assessment was designed to evaluate cogniti on by asking these individuals to report on challenges they experience in real life. In spite of their cognitive defic its, individuals with TBI may find it easy to report on the cognitive challenges they face on a daily basis. As Hart and colleague pointed out some subtle deficits may only be detected by the individuals themselves, not by outsiders (Hart et al., 2006). Our findings partially support the second hypothe sis (a) that there will be significant group-level of rater severities. We found significant rater effect s on all domains of the CAMFCTBI based on fixed chi-square statistics. Howeve r, the t-tests for paired comparisons did not

PAGE 104

104 identify significant differences across six domai ns. Rather, t-tests showed significant differences on only the ratings between the healthcare prof essionals and the pati ents on the memory, executive function and social communication domains. Even t hough not reaching the significant criterion, the paired comparisons indicated trends of differen ces on the ratings between the healthcare professionals and patients, and be tween the healthcare professionals and the caregivers on the attention and processing sp eed domains. Interestingly, on the emotional management domain, the ratings between the healthcare professionals and the patients did not differ. Instead, pair comparisons showed a tr end of difference between the ratings of the caregivers and the patients. Our findings partially supported the second hypothesis (b) that the healthcare professional is the most severe rater and the individual with TBI is the most lenient rater. Healthcare professionals were either significan tly more severe raters or had a tendency of being more severe raters across 5 of 6 domains of the CAMFC-TBI. This may be due to heuristics bias of healthcare professionals (Wolfson et al., 2000). Because of the assumption that cognitive deficits are prevalent sequelae for individuals with TBI, healthcare professi onals might intuitively rate patients with TBI more dysfunctional on cogniti ve domain. Moreover, Sherer et al. (2003) propose that clinicians experien ces allowed them to foresee the future impacts of TBI and resulted in more severe ratings (Mark Sherer, Hart, & Nick, 2003) The ratings of the patients and caregivers do not significantly differ for 5 of the 6 domains of the CAMFC-TBI. The way the items of the CA MFC-TBI were written may contribute to the concordance between the ratings of the caregiv ers and patients. During the item development stage, the items of the CAMFC-TBI were evaluated by focus groups including individuals with TBI, caregivers, and healthcare pr ofessionals to ensure the item s were well written, clear, and

PAGE 105

105 easy to understand. Such items with more speci fic descriptions (e.g., how often do you forget appointments or other things you need to do?) have been shown by Sherer and colleagues to have higher agreement than general items (e.g., how well is your memo ry for recent events now as compared to before your injury?) when rated by individuals with TBI and their car egivers (M. Sherer et al., 1998). In addition, impaired self awareness (ISA) di d not result in signifi cant lenient ratings by the patients in our study as indicated by the congruency between the patient and caregiver ratings. In a longitudinal study, Godfrey and coll eagues (1993) found ISA in closed head injury improved over time. The individuals with closed head injury showed good insight into their impairments in 1 year, 2 year and 3 year fo llow-up (Godfrey, Partridge, Knight, & Bishara, 1993). Since at least half of our participants were at the one-year post injury stages, the impact of ISA may not be as pronounced. Furthermore, the effects of caregiver burden were perhaps reflected in their ratings of emotional management; having a tendency of being more severe than patient ratings. Since caregivers are more likely than others to e xperience patients emotional outbursts, their memories may be enhanced by the embarrassmen ts of patients inappropriate emotional reactions. Hendryx (1989) reported similar findings that patients and their families had good agreement on some of the cognitive problems but not on emotional problems (Hendryx, 1989). Mckinkay and Brooks (1984) also reported that emotional problems had the worst rater agreement between the ratings of patients a nd caregivers compared to cognitive constructs (McKinlay, 1984). Our findings partially supported the third hypothesis that the h ealthcare profe ssional is the better rater. In general, the three raters showed similar estimate discriminations. Healthcare

PAGE 106

106 professionals showed a slight te ndency of having better discrimination across 5 of the 6 domains. Patients showed slightly lower estimate discrimi nations across 5 of the 6 domains. However, all of these estimate discriminations were close to th e expected value of 1. We expected healthcare professionals to have high estimate discrimination because of their professional training on assessing patients. Surprisingly, even without professional trai ning, the estimate discrimination of the caregivers is almost equivalent to that of the healthcare professionals. Moreover, the estimate discriminations of patient were also in good range. The clarity of items of the CAMFCTBI may contribute to the high esti mate discriminations across raters. Several limitations of this study should caution interpretations of our findings. First, our sample did not include individuals in acute or inpatients stages of TBI; therefore, our findings should not be generalized to individuals at these early stages of TBI. Second, since we did not have sufficient information regarding the severity level of individuals with TBI, our results are not comparable with those findings that targeted a specific severity level. For example, Leathem and colleague revealed that the level of TBI severity had differe nt impacts on rater effects: the severe TBI underestimated their de ficits while the mild to moderate TBI did not (Leathem et al., 1998). Future studies should include severity vari ables to further investigate rater effects based on different severity of TBI. In summary, rater effects represent the differe nt perceptions and frames of reference from different raters. Incorporating ra ter effects into measures provides more accurate results and more meaningful comparisons. Investigating rater effects among rate rs, we may be able to see a more complete picture of the patients function. We may find the problems that one rater report them but the other raters are not aware of especially on the em otional area, such as handling arguments, and controlling crying (Leathem et al., 1998). Rater effects should not be treated as

PAGE 107

107 psychometric problems of an instrument. Instea d, we should investigat e rater effects with caution, and attemp to understand the phenomena. With further understanding of rater effects, the adjustments can be made to make measures comparable across raters.

PAGE 108

108 Table 4-1. Number of particip ants across recovery stages Outpatient Rehabilitation One Year Post Injury Total 47 patient ratings 46 caregiver ratings 47 healthcare profe ssional ratings 43 patient ratings 43 caregiver ratings 90 patient ratings 89 caregiver ratings 47 healthcare professional ratings Table 4-2. Demographics statistics for patients and caregivers Demographics Patient (N=90) Caregiver (N=89) Age Mean SD (year) Range 38.2 15.8 18-84 49.9 14.6 21-88 Gender, n (%) Male Female 63 (70%) 27 (30%) 70 (21.3%) 19 (78.7%) Ethnicity, n (%) White African American Hispanic Other Missing 72 (80%) 10 (11.1%) 4 (4.4%) 3 (3.3%) 1 (1.1%) 75 (84.3%) 10 (11.2%) 3 (3.4%) 1 (1.1%) Education, n (%) 8th grade 9th grade 10th grade 11th grade GED 12th grade Some college, no degree Some college, earned degree (AA/AS) Completed 4-year college, earned degree (BS/BA) Graduate/Professional degree (MA/MS/PhD) 2 (2.2%) 0 (0%) 7 (7.8%) 6 (6.7%) 5 (5.6%) 19 (21.1%) 23 (25.6%) 14 (15.6%) 7 (7.8%) 7 (7.8%) 1 (1.1%) 1 (1.1%) 2 (2.2%) 3 (3.4%) 4 (4.5%) 13 (14.6%) 27 (30.3%) 20 (22.5%) 11 (12.4%) 7 (7.9%) Income, n (%) Under $5,000 $5,000-$10,000 $11,000-$15,000 $16,000-$20,000 $21,000-$35,000 $35,000-$50,000 Over $50,000 9 (10%) 3 (3.3%) 2 (2.2%) 8 (8.9%) 11 (12.2%) 24 (26.7%) 32 (35.6%) 2 (2.2%) 3 (3.4%) 7 (7.9%) 5 (5.6%) 18 (20.2%) 16 (18%) 36 (40.4%) Work, n (%) Yes No Missing 17 (18.9%) 72 (80%) 1 (1.1%) Drive, n (%) Yes No Missing 27 (30%) 59 (65.6%) 4 (4.4%)

PAGE 109

109 Table 4-3. Demographics statisti c for healthcare professionals Healthcare Professional (N=47) Age Mean SD Range 40.0 10.9 years 25-55 years Years of experience Mean SD Range 13.7 9.5 years 1-31 years Gender, n (%) Male Female 3 (6%) 44 (94%) Ethnicity, n (%) White African American Missing 45 (95.8%) 1 (2.1%) 1 (2.1%) Employer n (%) Brooks Rehabilitation Shepherd Center Shands Hospital 33 (70.2%) 13 (27.1%) 1(2.1%) Occupation n (%) Care manager Cognitive Rehabilitation Therapist Occupational Therapy Physical Therapy Speech Language Pathologist Resident 4 (8.3%) 25 (54.2%) 4 (8.3%) 6 (12.5%) 7 (14.6%) 1 (2.1%)

PAGE 110

110Table 4-4. Rater measure report Domains Raters Obs Average Fair-M Average Measure Model S.E. Infit MnSq Infit ZStd Outfit MnSq Outfit ZStd Estim. Discrim Attention Healthcare professional 3.0 3.03 -0.72 0.03 0.8 -6.6 0.77 -6.6 1.2 Patient 3.1 3.24 -1.02 0.02 1.14 6.3 1.23 7.9 0.82 Caregiver 3.2 3.24 -1.02 0.02 0.96 -1.7 0.96 -1.3 1.11 Memory Healthcare professional 2.9 2.86 -0.63 0.04 0.81 -5.1 0.82 -4.4 1.11 Patient 3.2 3.29 -1.29 0.03 1.18 6.4 1.16 4.5 0.87 Caregiver 3.0 3.12 -1.02 0.03 0.93 -2.5 0.90 -3.2 1.10 Processing Speed Healthcare professional 3.0 2.88 -0.46 0.04 0.88 -2.8 0.88 -2.2 1.02 Patient 3.1 3.19 -0.87 0.03 1.10 3.4 1.11 3.0 0.92 Caregiver 3.1 3.20 -0.88 0.03 0.96 -1.2 1.00 0 1.08 Executive Function Healthcare professional 2.7 2.61 -1.97 0.03 0.73 -9.0 0.74 -9.0 1.33 Patient 3.1 3.16 -2.75 0.02 1.14 7.3 1.20 9.0 0.84 Caregiver 2.9 2.92 -2.39 0.02 0.99 -0.3 1.00 -0.1 1.05 Emotional management Healthcare professional 3.3 3.37 -1.45 0.07 1.08 1.2 1.07 0.7 0.91 Patient 3.3 3.38 -1.49 0.04 1,01 0.2 1.01 0.1 1.00 Caregiver 3.1 3.25 -1.22 0.04 0.98 -0.5 1.02 0.4 1.04 Social Communication Healthcare professional 3.2 3.15 -1.14 0.04 0.92 -2.3 0.91 -2.1 1.08 Patient 3.3 3.35 -1.55 0.03 1.10 3.6 1.12 3.6 0.88 Caregiver 3.2 3.23 -1.29 0.03 0.96 -1.6 0.95 3.6 1.08

PAGE 111

111Table 4-5. Significant rater effects Fixed chi-square test Pair comparisons Domain Fixed chi-square value Degree of freedom P-values Raters Differences (logits) Z-values Attention 73.8 2 <0.001* H-P 0.3 2.14 H-C 0.3 2.14 C-P 0 0 Memory 188.0 2 <0.001* H-P* 0.66 2.44* H-C 0.39 1.44 C-P 0.27 1 Processing Speed 78.2 2 <0.001* H-P 0.41 2.05 H-C 0.42 2.1 C-P 0.01 0.05 Executive Function 596.5 2 <0.001* H-P* 0.78 2.44* H-C 0.42 1.31 C-P 0.36 1.13 Emotional Management 21.6 2 <0.001* H-P 0.04 0.33 H-C 0.23 1.92 C-P 0.27 2.25 Social Communication 75.8 2 <0.001* H-P* 0.41 2.41* H-C 0.15 0.88 C-P 0.26 1.53 represents statistically significant Shaded represents close to statistically significant

PAGE 112

112-------------------------------------------------------------------------------------------------------------------------------+ 3 + + + + (4) + | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | + 2 + + + + + | | | | | | | | | | | | | | | ** | | | | | | | | --| | | | | 28.Read 30min no break 43.Multiple things | | | | | | | | + 1 + + *** + 39.Write message while phone + + | | | *** | 33.Speed/accur distract | | | | | | 35.Continue extended project 40.Go btw instruction task 42.Finish before start | 3 | | | | ******** | 18.Write voice message 34.Info from lecture | | | | | ***** | 16.TV not distract other talk 27.Sit 1hour TV 30.Parti 10-20min convers | | : : : : 32.Return to activity 49.Stop when distracted(R) : : | | | ****** | 11.Focus 5-10min noisy 17.Talk not distracted TV 23.Locate article newspaper | | : : : : 29.Listen 15-30min focus 31.Parti 1hr break 47.Follow written direction : : : : : : 51.Mistake length increase(R) : : | | | ***** | 19.Locate number phone book 25.Sort junk mail 38.Drive answer cell | --| : : : : 41.Locate item shop list 7.Write short message : : 0 ********** 15.Convers noisy env 5.Copy daily schedule | | | ****** | 20.Locate item grocery 21.Locate size of clothing 50.Attention wrong convers(R) | | : : : : 8.Parti 30min break : : | | | ***** | 22.Select complex menu 37.Drive while talk 45.Answer phone ring | | : : : : 9.Parti 30min no break : : | | | ****** | 14.Convers small group 36.Notice warning light 48.Leave out steps(R) | | | | | ***** | 12.phone 2-3min 26.Locate item fridge 4.Select menu | 2 | | | H | ****** | 13.Meal with distractions 2.Go directly location 24.Select dresser/closet | | : : : : 46.Parti activity 5-10min : : | | | ******* | 10.Self-care not distract 3.Greet person 44.Look toward touched | | : : : : 52.Stop chewing distracted(R) 6.Turn toward phone : : + -1 + C P + **** + + + | | | | 1.Answer question self | | | | | | | | | | | | | | | | | ** | | --| | | | | | | | | | | | | + -2 + + + + (1) + -------------------------------------------------------------------------------------------------------------------------------|Measr|-Raters| = 1 |-Items |Scale| -------------------------------------------------------------------------------------------------------------------------------Figure 4-1. Facets ruler of the attention domain

PAGE 113

113 CHAPTER 5 DISCUSSION Traumatic brain injury ( TBI) often results in long-term disability (M oscato et al., 1994). Individuals with moderate to severe TBI often require extens ive rehabilitation programs (Lezak, 2004). The most important challenge for patients and their caregiver is their cognitive ability for everyday life. Functional cognition is defined as the ability to complete everyday activities that require primarily on cognitive abilities (Donovan et al., 2008). To identify the cognitive problems of everyday life and monitor the trea tment effect, outcome measures for TBI are necessary. While neuropsychological tests are objective and comprehensive, they are not designed for assessing functional cognition. On the other hand, functional outcome measures aimed at assessing function are often not compre hensive, have poor sensitivity, and are tainted by observer bias. The Computer Adaptive Measur e of Functional Cognition for Traumatic Brain Injury (CAMFC-TBI) was designed to assess f unctional cognition and to overcome many of the limitations of current functional outcome measur es: comprehensiveness, sensitivity, and rater bias. The purpose of this dissertation was to establish the fundamental work so CAMFC-TBI can move forward to computer ad aptive testing version of the instrument battery. Four general aims of this dissertation were proposed to: 1) investigate the factor structure of the newly developed CAMFC-TBI item pool; 2) examine the item parameter equivalence across subgroups (participants from outpatient rehabilitation setting an d participants who are one-year post rehabilitation); 3) examine the item level psyc hometric properties of the CAMFC-TBI; and 4) examine the rater effect s of the CAMFC-TBI. To address the first aim, investigation of factor structure of the CAMFC-TBI, both confirmatory factor analysis (CFA) and explorator y factor analysis (EFA) were applied (Chapter

PAGE 114

114 2). Three specific research questions were tested : (1) whether single underlying factor general functional cognition represents the entire CAMFC-TBI; (2) was a domain-based structure sufficient to explain the variance of each dom ain over a neuropsychologically-based sub-domain structure; and (3) what is the underlying factor structure of each functi onal cognition domain for individuals with TBI if the f actor structures we proposed ba sed on neuropsychological model cannot be confirmed. The result of CFA showed that a general functi onal cognition construct was underlying the CAMFC-TBI for the caregivers ratings but not for the patients ratings. Patients ratings suggested that single underlying factor may be sufficient to represent the CAMFC-TBI if the emotional management doma in was excluded. For the second question, due to the low ratio of item and data, CFA could not be computed successfully. Neither domainbased model nor sub-domain structure was confirmed. For the third question, to further explore the potential factor structures by using EFA, we found mixed results. The finding that the dominant factor of each domain accounted for majority of the variance could possible be interpreted as meet the essential unidimensionality criteria. But factor loadings also supported a sub-domain structures on 4 of 6 domains and do main-based structure on 2 remaining domains of the CAMFC-TBI. The sub-domain structures for six scales are as following: attention domain contained 3 inter-corre lated sub-domains: focus/working memory, against distraction, and sustained attention for patients ratings; focus/working memor y, against distra ction and focus attention for caregivers ratings Memory domain contained 3 inter-correlated sub-domains: working memory, long-term declarative and long-term non-declarative. Executive function domain contained 2 inter-correlated sub-domains: plan/organizing and regulatory control. Social communication domain contains 2 inter-correlat ed sub-domains: inhibition and communication skills. It should be kept in mi nd that because of the small sample size, the structure we obtained

PAGE 115

115 from EFA should not be seen as a definite fact or solution. Rather, thes e findings should provide insight of the factor structure for future CFA. To address the second and third aims, examin ation of item equivalence across sub-groups (participants from outpatient re habilitation setting and partic ipants who are one-year post rehabilitation) and measurement quality of the CAMFC-TBI, Rasch analysis was conducted (Chapter 3). The results suggest ed item estimates were stable cross outpatient rehabilitation subgroup and one-year post injury subgroup. Ther efore, the CAMFC-TBI can be applied on these two groups without adjustment. For example, same set of item difficulty estimates can be applied for both outpatient and one-year pos t injury groups. Moreover, the item-level measurement qualities of six-domain scales were positive and promising. Item and person statistics were either compar able or better than those found for existing functional cognitive measures. Although items misfitting (<14%) and person misfitting (<18%) across six domains were found, none of these misfitting were substantial. Ceiling effects were absent, except for the ratings of patient and healthcare professional on the Emotional management domain. To address the fourth aim, potential rater e ffects of the CAMFC-TBI, Facet analysis was conducted (Chapter 4). We hypothesi zed that (1) the ratings of i ndividuals with TBI would not fit Rasch model; (2a) there would be significant group-level of rate r severities; (2b) healthcare professional would be the most severe rater and the individual with TBI w ill be the most lenient rater; and (3) the healthca re professional would be the better rater. Facet analysis showed that all of the three raters (patient, caregiver and healthcare professi onal) fit Rasch model. That is, none of the rater-groups answered the questions erraticall y. In terms of rater severity, the ratings of healthcare professional were significantly mo re severe than the ratings of patient on the memory, executive function and social communi cation domains. Additionally, healthcare

PAGE 116

116 professional showed the tendency of being more severe rater than patient on the attention and processing speed domains. The severity of caregive rs ratings was generally similar to patients ratings, except for the emotional management domain. On the emotional management domain, caregiver showed the tendency of being more severe rater than patient and healthcare professional. Several limitations of these stud ies should be noticed. The first concern is the power of the analyses. Since this is a preliminary study, the sample size is relatively small, especially considering the size of our item pool. This affects the power of all the analyses. Second, we did not have enough information to cat egorize our patients severity level of TBI. Even though we intended to recruit only moderate to severe TBI, we could not guarantee this criterion. In addition, since mild TBI and moderate to severe TBI have very different symptoms, lack of this information limits the generalizati on of our findings to mild TBI. This series of studies addressed important fundamental research in validating the item bank for the CAMFC-TBI. The first study (Chapter 2) provided insight of factor into the structure of functional cognition. The second study (Chapter 3) provided the measurement quality of the six measures of the CAMFC-TBI: attention, memory, processi ng speed, executive function, emotional management, and social communicati on. The third study (Chapter 4) addressed the rater effects of the CAMFC-TB I. The factor structure suppo rts the potential domain-based structure for constructing computer adaptive te st (CAT). In addition, the emerged sub-domain structure from EFA should be confirmed in the future studies. The sound psychometrics of six batteries of the CAMFC-TBI suggests that CAMFC-TBI is a promising functional cognitive measure. Using this item bank to create CAT, an efficient, comprehensive, and sensitive measure is warranted. Investigation of rater effects clarified the

PAGE 117

117 potential systematic bias from different raters. When a measure is designed for different raters (patient, caregiver and healthcare professional), rater effects are unavoidable. These findings suggest that in their present fo rm, independent measures should be generated for each rater (e.g., having separate computer adaptive tests for each rater). However, comparable measures might be achieved by adjusting scores on the basis of rate r severity. Future studies should investigate whether such adjustments for rater bias are warranted.

PAGE 118

118 APPENDIX A THE CAMFC-TBI ITEMS, DOMAINS, AND SUB-DOMAINS Attention Items Sub-domains 1.Correctly answer questions about himself/herself (for example, What is your name? How old are you? Where are you? What year is it?). Focused 2.Goes directly from his/her room to a specific location (for example, dining room, therapy room) without wandering. Focused 3.Greets a familiar person when that person enters the room. Focused 4.Selects meal items from a menu. Focused 5.Copies daily schedule correctly. Focused 6.Turns toward a ringing phone. Focused 7.Writes down a short phone message correctly. Focused 8.Participates in a structured activity for 30 minutes with rest break (for example, a therapy session). Sustained 9.Participates in a structure activity fo r 30 minutes without rest break (for example, a therapy session). Sustained 10.Completes a self-care activity (for ex ample, brushes teeth, gets dressed) without getting distracted. Sustained 11.Stays focused on a 5 to 10 minutes activ ity in a noisy environment. Sustained 12.Completes 2 to 3 minute conversa tion using the phone. Sustained 13.Completes a meal with distractions (for example, conversations, TV). Divided 14.Has a conversation with a small gr oup (family or few friends). Shift 15.Has a conversation in a noisy envi ronment (therapy room). Divided 16.Watches TV without being distr acted by people talking. Divided 17.Talks with someone without bei ng distracted by a TV on in the background. Divided 18.Correctly writes down message from an answering machine. Divided 19.Locates a phone number or addres s in the telephone book. Focused 20.Locates particular item or brand in the grocery store. Focused 21.Locates particular size of clothing on a de partment store rack or shelf. Focused 22.Selects meal items from a complex menu (for example, restaurant menu). Focused 23.Locates needed information in a section or article in the newspaper. Focused 24.Selects outfit from a dresser (chest of drawers) or closet. Focused 25.Sorts important mail from junk mail. Focused 26.Locates items in the refrigerator. Focused 27.Sits through an hour-long TV program without getting distracted. Sustained 28.Reads 30 minutes without taking a break. Sustained 29.Listens for 15-30 minutes quietly and with focus (for example during a religious service or class lecture). Sustained 30.Participates in a 10-20 minute conver sation, staying on topic. Sustained 31.Participates in a structured activity for one hour with only a short rest break. Sustained 32.Returns to an activity without a remi nder after a short in terruption. Shift

PAGE 119

119 33.Maintains speed and accuracy when doing a task in a distracting environment (for example, people walk ing in and out the room or people talking). Focused 34.Picks out important information from a lecture/instruction. Focused 35.Continues to work on an extended proj ect (for example, one that takes several days). Sustained 36.Notices when a warning light appears on the dashboard (for example, seat belt, door ajar, emerge ncy break, engine service). Focused 37.Maintains safe driving while talking to a person in the car. Focused 38.Maintains safe driving while answering a cell phone. Focused 39.Writes down a phone message while talking on the phone at the same time. Divided 40.Goes back and forth between reading instructions and doing a task (for example, reading a recipe while cook ing, looking at a manual to repair a car, looking at instructions to put together a new purchase). Shift 41.Locates items in a store usi ng a shopping list. Focused 42.Finishes one task before starting another. Sustained 43.Able to work on multiple things at the same time (for example, preparing a second dish while something is already cooking on the stove, keeping an eye on children while doing other things). Divided 44.Looks toward person after being touched lightly. Focused 45.Answers the phone when it rings. Focused 46.Participates in a structure activity for 5 to 10 minutes (for example, short therapy session or simple grooming activity). Sustained 47.Able to use map or follow written direction to get to an unfamiliar location. Focused 48.Leaves out steps of a task (for example, does not remove shaving cream completely from face). Encode (not included) 49.Stops in the middle of a task when distracted (for example, by someone talking). Focused 50.Pays attention to the wrong convers ation or activity (for example, listening to nearby conversation rather than the person they are talking to). Focused 51.Makes more mistakes as the length of the task increases. Sustained 52.Stop chewing food when distracted. Divided Memory Items Sub-domains 53.Recalls a meal later in the day (for example, remembers what they had for breakfast when asked in the afternoon). LT-Declarative 54.Knows the current month. LT-Declarative 55.Recalls a visit from a familiar person (friends, family) earlier in the day. LT-Declarative 56.Recalls what he/she did before the inju ry (job/school/homemak ing). LT-Declarative 57.Recalls basic instructions (for exam ple, using equipment in their room, using call button to call nu rse, turning on TV). LTNonDeclarative 58.Recalls a simple routine (for example, doing an exercise, using memory LT-

PAGE 120

120 book). NonDeclarative 59.Recalls more than one appointment (for example, multiple health care appointments or social activ ities) in a single day. LT-Declarative 60.Recalls a visit by a familiar person (friends, family, therapist) from the previous day. LT-Declarative 61.Recalls to take medicine at the righ t time and right amount. LT-Declarative 62.Recalls where to find something when it is not put in its usual place (for example, looking for keys). LTNonDeclarative 63.Recalls the steps in doing a simple activity (for example, gathering needed materials and items for maki ng a sandwich or cooking breakfast, washing a car, loading and starting a dishwasher). LTNonDeclarative 64.Recalls to move laundry from washer to dryer. LTNonDeclarative 65.Recalls to put food away in the refrigerator when finished. LTNonDeclarative 66.Recalls to turn off the stove or oven. LTNonDeclarative 67.Recalls to lock the door when leaving the house. LTNonDeclarative 68.When driving, remembers to take the key when getting out of the car. LTNonDeclarative (not included) 69.Recalls to give someone a telephone message. LTNonDeclarative 70.Recalls familiar route without assist ance (for example going from home to a local store). LTNonDeclarative 71.Recalls a newly learned r oute without assistance. LTNonDeclarative 72.Recalls where the car is parked in the mall/g rocery store parking lot. LT-Declarative 73.Recalls to use a calendar to keep tr ack of appointments from week to week. LT-Declarative 74.Recalls information given at a previous therapy or doctor appointment. LT-Declarative 75.Recalls birthdays, holidays or anniversaries. LT-Declarative 76.Recalls frequently used phone numbers. LT-Declarative 77.Recalls to get an item at the store th at was not written down. LT-Declarative 78.Recalls the story line in a book from one reading to the next. Working 79.Recalls to go to doctors a ppointments. LT-Declarative 80.Recalls events from last birt hday/vacation. LT-Declarative 81.Recalls to do weekly chores. LT-Declarative 82.Recalls to pay bills (for example, rent, el ectric, phone or credit card). LT-Declarative 83.Recalls upcoming deadlines, assignme nts, or meetings. LT-Declarative 84.Goes to a room to get something, but forgets what to get. Working 85.Begins to do something and forgets what was to be done. Working 86.Loses train of thought in a conversation. Working 87.Repeats a story that has al ready been told. Working

PAGE 121

121 Executive Function Items Sub-domains 121.Complete a simple task that has se veral steps (for example, chooses clothes and gets dressed). Plan/Organize 122.Complete a complex task that has several steps (for example, cooking a complete dinner, doing a house repair or building something). Plan/Organize 123.Plans a common daily activity (for example, gathers items needed for dressing or grooming). Plan/Organize 124.Plans a new activity (for example, gathers items needed for cooking or craft project). Plan/Organize 125.Plans ahead in order to get to an appointment on time. Plan/Organize 126.Fills free time with activities without being told. Initiate 127.Starts an activity without being told (for example, starts getting dressed after getting up in the morning). Initiate 128.Make careless errors duri ng a new activity (for example, doing things out of order or leaving a st ep out of an activity). Monitor 129.Does not recognize limitations when a ttempting a task (for example, tries to walk when unable). Monitor 130.Recognizes and corrects mistakes. Monitor 131.Readily changes behaviors when an error is pointed out. Shift 132.Talks at the wrong time (interrupts conversation, talks when he/she should be listening). Inhibit 133.Does not ask embarrassing questions, or make hurtful/inappropriate comments. Inhibit 134.Stays seated until a task is done. Inhibit 135.Gets started on homework/chores without being told. Initiate 136.Catches own mistakes while working on a task. Monitor 137.Starts a task early enough to get it done (for exampl e, starts to get ready for school/ work/appointment in order to arrive on time). Plan/Organize 138.Comes up with ideas for things to do during free time. Initiate 139.Chooses clothes based on the weather. Plan/Organize 140.Demonstrates an understanding of ow n abilities (for example, does not ask to drive or return to work /school if they are not able). Monitor 141.Gathers materials needed for an activity (for example, necessary materials for work or school). Plan/Organize 142.Identifies items needed to put together a list (for example, grocery items for the week, shopping list, materi als for project or repairs). Plan/Organize 143.Organizes an activity several days in advance (for example, planning a trip, visiting friends, pla nning holiday activities). Plan/Organize 144.Organizes a written list (for exampl e, errands list organized by store types, grocery list by sections of store). Plan/Organize 145.Keeps personal area organized (for ex ample, putting things away in the bedroom, bathroom, kitchen, laundry room). Plan/Organize 146.Tells someone or takes action when something goes wrong (for example, water on the floor, shoes un tied, pot boiling over). Problem Solving

PAGE 122

122 147.Seeks help when needed. Problem Solving 148.Stops an activity to do something else that needs to get done (for example, stops watching TV to get dressed). Shift 149.Makes reasonable attempts to solve problem s before asking for help. Problem Solving 150.Follows safety rules (for example, locks wheelchair brakes when stopped, not opening doors to strangers, looks both ways before crossing street). Problem Solving 151.Tries to do an activity before having the ability to do it (such as standing or walking unassisted, cooking, dr iving, returning to school/work). Monitor 152.Comes up with an alternate solution when the first solution does not work (for example, when a drain cleaner does not work, calls a plumber). Problem Solving 153.Stops talking when a discu ssion becomes heated. Inhibit 154.Suggests or attempts a solution to a problem. Problem Solving 155.Tries a different approach to a problem when the first one does not work. Problem Solving 156.Adds a new topic to a conversation. Inhibit 157.Listens to anothers perspec tive without argu ing. Inhibit 158.Do the things needed to prepare for a bigger project (for example, moving furniture before painting a room). Plan/Organize 159.Organizes a short written document (fo r example, letter, memo, email or school paper) Plan/Organize 160.Estimates the time needed to do a series of tasks to meet a deadline. Plan/Organize 161.Adjusts schedule to meet a deadline. Plan/Organize 162.Makes changes to a schedule if need ed (for example, postpones doing errands to be on time for work. Shift 163.Fills gas tank before it runs out. Monitor (not included) 164.Picks up hints from others that th ey should end a conversation (for example, other person looks at watch). Monitor 165.Dresses to match social situation (for example, dresses casually to go out with friends and dresses more formally for special events). Plan/Organize 166.Asks questions to get more inform ation about injury. Plan/Organize 167.Initiates a discussion about future ne eds (for example, returning home after hospitalization, asking about financial issues, or resuming work or school). Problem Solving 168.Plans a short trip using public tr ansportation (for example, bus or subway). Problem Solving (not included) 169.Able to make a quick, simple decision (for example, where to go to dinner). Problem Solving 170.Does not readily switch from one activ ity to another (for example, will not stop watching TV to begin dressing). Shift 171.Makes careless errors in daily tasks (for example, misses a button, forgets to put toothpaste on toothbrush). Monitor 172.Gets stuck on a topic (keeps talk ing about the same thing). Shift 173.Does not stop or apologizes when behavior bothers others. Monitor 174.Does not know what to do next, so stops in the middle of a task. Plan/Organize

PAGE 123

123 175.Allows others to solve problems for them when they could have done it themselves. Problem Solving 176.Jumps to a solution when attempting to solve a problem. Problem Solving 177.Is overly trusting (does not recognize wh en being taken advantage of). Monitor 178.Buys unnecessary items that look app ealing (impulse buying). Inhibit 179.Interrupts while someone is talking on the phone. Inhibit 180.Bothers other people while they are working. Inhibit 181.Makes errors when solving a problem th at has several steps (for example, cooking/following a recipe, shopping car maintenance, programming electronics). Problem Solving 182.Has problems managing money (for ex ample, tries to make purchase without enough money, overdrawing checking account, running-up credit cards). Problem Solving 183.Does not start activities on ow n (for example, must be told what to do). Initiate 184.Gives up if first attempt to solve a pr oblem is not successful. Plan/Organize Social Communication Items Sub-domains 199.Gets a persons attention before st arting a conversati on (for example, waits for eye contact before talking). Conversational Ability 200.Allows others to take a turn in a co nversation (for example, gives another person a chance to talk). Conversational Ability 201.Greets person when someone enters the room. Conversational Ability 202.Able to talk with more than one person at a time. Conversational Ability 203.Begins to answer open-ended questions within an appropriate amount of time (for example, responds to "What did you do today?" within a few seconds). Topic Management 204.Provides enough information when telling someone about something. Conversational Ability 205.Faces the person when speaking. Nonverbal Communication 206.Appropriate eye contact when having a conversation (for example, looks away occasionally during a conversation). Nonverbal Communication 207.Shows interests in what other people are saying (for example, keep eye contact, comments or nods). Topic Management 208.Acknowledges another person's point of view (for example, by nodding or commenting). Conversational Ability 209.Keeps up with a conversation wit hout asking people to repeat. Conversational Ability 210.Participates in a 10-20 minute c onversation, staying on topic. Topic Management 211.Picks out important information from a lecture/instruction. Conversational Ability 212.Adds a new topic to a conversat ion at the right time. Topic

PAGE 124

124 Management 213.Starts a conversation. Conversational Ability 214.Misunderstands what the speaker intends (for example, does not recognize when someone makes a joke or uses sarcasm). Conversational Ability 215.Sounds rude or demanding when making a request. Nonverbal Communication 216.Facial expression does not match the conversation (for example, blank expression during an emotional conver sation or smiles too much during a serious conversation). Nonverbal Communication 217.Gets too close when talking to someone. Nonverbal Communication 218.Gets stuck on a topic (keeps talking about the same thing). Topic Management 219.Talks at the wrong time (interrupts conversation, talks when he/she should be listening). Topic Management 220.Asks embarrassing questions, or make s hurtful/inappropriate comments. Conversational Ability 221.Jumps to a topic unrelated to the conversation. Topic Management 222.Walks away from conversation before it is finished. Topic Management 223.Blurts out something off t opic during a conversation. Topic Management 224.Loses train of thought in a conversation Topic Management 225.Repeats a story that ha s already been told Topic Management 226.Interrupts while someone is talking on the phone Conversational Ability 227.Interrupts other peoples conversation Conversational Ability 228.Provides too much information wh en telling someone something Conversational Ability

PAGE 125

125 APPENDIX B ITEMS WITH MORE THAN 50% NOT APPLICABLE RESPONSE Item num ber Item I37 Maintains safe driving while talking on to a person in the car I38 Maintains safe driving while answering a cell phone I68 When driving, remembers to take the key when getting out of the car I78 Recalls the story line in a book from one reading to the next I99 Washes a car within 30 minutes I103 Takes a phone message off the answering machine without having to replay the message more than one time I104 Gets money from ATM within 5 minutes I108 Writes a check in a grocery st ore without holding up the line I120 Reacts slowly in driving situations I163 Fills gas tank before it runs out I168 Plan a short trip usi ng public transportation

PAGE 126

126 APPENDEX C MISFITTING ITMES OF THE COMPUTER ADAPTIVE MEASURE OF FUNCT IONAL COGNITION FOR TRAUMAT IC BRAIN INJURY Attention Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTD Infit MNSQ ZSTD (3)Greets person when enter the room 2 3.1 (8) Participates in a structured activity 30 min with break 1.83 4.6 1.79 3.9 2.73 5.2 (9)Participates in activity 30 min without break 1.95 3.7 (24)Selects outfit from a dresser 1.59 2.6 (25)Sorts important mail from junk mail 1.58 2.8 (31)Participates in activity 1 hour with short break 1.47 2.1 (44)Looks toward person after being touched 1.47 2 (52)Stops chewing food when distracted 1.79 4 2.65 2.2 Memory Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTD Infit MNSQ ZSTD (56)Recalls what he did before injury 1.6 3.3 (70)Recalls familiar route without assistant 1.78 2.9 (76)Recalls frequently used phone number 1.61 3.4 (80)Recalls event from last Birthday 1.72 2.5 (82)Recalls to pay bill 1.5 2.3 (86)Loses trains of thought in a conversation 1.56 2.5 (87)Repeats a story that has been told 1.62 3.5 1.82 3.3

PAGE 127

127 Processing Speed Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTD Infit MNSQ ZSTD (91)Writes name in timely manner 2.24 4 (89)Says "come in" in response to knock 1.77 4.3 1.47 2.6 (99)Washes a car within 30 mins 1.63 2.6 (100)Unloads the washing machine within 10 1.42 2.2 (115)Opened-ended questions more than 1 1.51 2.9 (116)Takes a long time to finish eating 1.87 4.2 2.36 3.7 (117)Takes a long time to get dress 1.54 2.5 (118)Needs repeated requests to respond 1.58 2 Executive Function Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTDInfit MNSQ ZSTD (121)Completes a simple task 1.62 2.6 (122)Completes a complex task 1.5 2.6 (131)Readily changes behavior when error 1.48 3 (132)Talks at the wrong time 1.61 2.7 (133)Not ask embarrass question 1.82 5 2.21 6.3 3.89 8.3 (134)Stays seated until done 2.05 4.2 (144)Organizes a written list 1.49 2.9 (151)Tries to do activity before can do it 1.48 3.1 (153)Stops talking when discussion heated 1.82 4.5 (163)Fills gas tank before runs out minimum estimated measure (166)Asks questions to get info about injury 1.54 3.3 (168)Plans a short trip pubic transportation 1.86 3 (170)Not readily switch activity 1.6 3.8 1.42 2.6 (172)Gets stuck on a topic 1.45 2 (173)Not stop when behavior bothers others 1.75 4.3 1.74 4.2 (176)Jumps to a solution 1.49 3.3 1.55 3.2 (182)Has problems manage money 1.71 3.2 (183)Not start activity on own 1.47 2.8

PAGE 128

128 Emotional Management Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTD Infit MNSQ ZSTD (187)Not react when people are upset 1.51 2.9 2.34 3.5 (192)Calms down after an argument 1.83 4.4 (193)Becomes tearful easily when upset 1.71 3.9 2.26 5.8 2.05 3 Social Communication Misfit Item Patient Caregiver Healthcare Professional Infit MNSQ ZSTDInfit MNSQ ZSTD Infit MNSQ ZSTD (199)Gets attention bf start conversation 1.52 3 (201)Greets person when someone enter 1.67 2.8 (206)Eye contact when a conversation 1.54 3.3 1.45 2.6 (213)Starts a conversation 1.51 2.9 1.96 3.9 (217)Gets too close when talk 1.71 2.9 (228)Provides too much info when tell 1.75 4.2 1.64 2.7

PAGE 129

129APPENDIX D FACETS RULER Facets ruler of the m emory domain ----------------------------------------------------------------------------------------------------------------------------+ 3 + + + + (4) + | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | + 2 + + + + + | | | ** | | | | | | | | --| | | | | | | | | | | | | | | | **** | | | | | | **** | | | + 1 + + **** + 62.Find thing not its place 71.New route no assist 77.Get item not written + + : : : : 78.story line : : | | | **** | | 3 | | | | ***** | 80.Events last birthdays | | | | | ***** | 59.Recall app 74.Info previous app 83.Upcoming deadlines | | | | | | 72.Car parking lot 75.Birthday/holiday 76.Frequent phone numbers | | : : : : 87.Repeat story told(R) : : | | | ****** | 81.Do weekly chore | | 0 ***** 53.Recall meal 69.Give phone message 73.Use calendar app --* : : : : 79.Doctor's app 82.Pay bills : : | | | ** | 61.Take medicine 84.Forget what to get(R) 86.Lose thought convers(R) | | | | | ********** | 60.Recall visit previous day 64.Move laundry washer dryer | | | | | ***** | 58.Recall routine 63.Recall the steps simple 65.Put food fridge | | : : : : 70.Familiar route no assist 85.Forget what to be done(R) : : | | | ****** | | | | | H | **** | 55.Recall visit 56.Recall before injury 66.Turn off stove | 2 | : : : : 67.Lock door leave : : | | | **** | 57.Recall basic instructions | | + -1 + C + **** + + + | | | **** | 68.Take Key when get out | | | | P | *** | 54.Know month | | | | | | | | | | | ** | | | | | | | | --| + -2 + + + + + | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | + -3 + + + + (1) + ----------------------------------------------------------------------------------------------------------------------------|Measr|-Raters| = 1 |-Items |Scale| ----------------------------------------------------------------------------------------------------------------------------

PAGE 130

130Facets ruler of the pr ocessing speed domain -------------------------------------------------------------------------------------------------------------------------------+ 2 + + + + (4) + | | | *** | | | | | | | | | | | | | | | | | | | | | | | | | | --| | | | | | | | | | | | | | | | | | | + 1 + + ** + + + | | | | | | | | | **** | 103.Message machine no replay | | | | | ****** | 113.Keep up school/work 99.Wash car in 30min | 3 | | | | *** | 102.Message no caller repeat | | | | | ******* | 95.Complete by deadline | | | | | *** | 114.Shop timely 88.Answer 3 ring | | | | | *** | 108.Write check no hold up 112.Read 1page in 5min 119.Mistake keep up(R) | | : : : : 97.Convers no repeat 98.Simple directions no repeat : : | | | **** | 100.Unload washer 10min 101.Put clean dish 15min 89.Respond to knock | | : : : : 92.Copy schedule timely : : 0 ***** 105.Follow auto phone menu 107.Sort mail in 5min 110.Pay fast-food in 30sec --* : : : : 93.Answer Open-ended 2sec 94.Dress 15min : : | | | ***** | 104.ATM in 5min 109.Drive-through no hold up 111.Read menu decide in 5min | | : : : : 96.Make breakfast 5-10min : : | | | ******* | | | | | | ****** | 106.TV 30min no ask 115.Open-ended ask >1(R) 90.Select menu timely | | | | H | ** | 120.React slow drive(R) | | | | | ** | 116.Long time meal(R) 91.Write name timely | 2 | | | | **** | | | | | | ****** | 117.Long time get dress(R) | | | | C P | *** | | | + -1 + + ** + + + | | | ** | | | | | | ** | | | | | | | | | | | | ** | 118.Repeat to respond(R) | --| | | | | | | | | | | | | | | | | | | | | | | | | + -2 + + + + (1) + -------------------------------------------------------------------------------------------------------------------------------|Measr|-Raters| = 1 |-Items |Scale| -------------------------------------------------------------------------------------------------------------------------------

PAGE 131

131Facets ruler of the execu tive function domain ----------------------------------------------------------------------------------------------------------------------------+ 2 + + + 168.Short trip public trans + (4) + | | | | | --| | | | | | | | | | | | | | | | | | | | | | *. | | | | | | | | 2 | + 1 + + *. + 153.Stop talking heated + + | | | *. | | | | | | *. | 136.Catch own mistake 144.Org written list | | | | | *. | 122.Complete complex 133.No embarrass question 143.Org activity advance | | : : : : 156.Add topic convers 160.Est time needed : : | | | *** | 124.Plan new activity 130.Correct mistake 131.Change when error | | : : : : 138.Idea for free time 164.Pick Hint end convers : : | | | **** | 126.Free time activity 135.Get start not told 152.Alter first not work | | : : : : 155.Differ approach 157.Listen no argue 158.Do thing for bigger : : : : : : 159.Org short docu 161.Adj meet deadline : : | | | **. | 125.Plan to app on time 154.Solution to prob 162.Change if needed | --| : : : : 167.Initiate discuss future 169.Quick direction 176.Jump when solve(R) : : 0 *. 137.Start early get done 145.Keep organized 148.Stop to do thing need : : : : 166.Question injury 177.Over trust(R) : : | | | *****. | 128.Error new act(R) 129.Not recognize limit(R) 140.Understand own ability | | : : : : 141.Gather material 142.Identifies item 149.Solve b4 help : : : : : : 178.Buy unnecessary(R) 181.Error solve prob steps(R) : : | | | ****. | 127.Start activity not told 134.Stay seat til done 147.Seek help when need | | : : : : 151.Do b4 able(R) 170.Not switch act(R) 172.Stuck in topic(R) : : : : : : 173.Not apologize bother(R) 175.Other solve not self(R) 183.Not start own(R) : : | | | **** | 121.Complete simple 123.Plan daily activity 132.Talk wrong time(R) | | : : : : 146.Tell/act when wrong 179.Interrupt when phone(R) 182.Problem money(R) : : | | | **** | 139.Clothes weather 150.Follow safety rule 165.Dress match | | : : : : 174.Not know what next(R) : : | | | *. | 184.Gives up first attempt(R) | | | | | ** | 180.Bothers while working(R) | | + -1 + + *. + 171.Error daily task(R) + + | | | *. | | | | | | | | | | | | | 163.Fills gas | | | | | | | | | | | | | | | | | | | | + -2 + H + + + + | | | | | | | | | | | | | | C | | | 1 | | | | | | | | | P | | | | | | | | | | + -3 + + + + (0) + ----------------------------------------------------------------------------------------------------------------------------|Measr|-Raters| = 2 |-Items |Scale| -----------------------------------------------------------------------------------------------------------------------------

PAGE 132

132Facets Ruler of Emotional Management ------------------------------------------------------------------------------------------------------+ 3 + + ** + + (4) + | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ** | | | + 2 + + + + + | | | | | | | | | ** | | --| | | | *** | | | | | | | | | | | | | | | | | | | | | + 1 + + ** + + + | | | *** | 188.Criticism no temper | | | | | ***** | 191.Listen no argue | 3 | | | | *** | | | | | | ******* | 189.Accept help no temper | | | | | | 186.Upset change routine(R) 192.Calm down argument | | : : : : 196.Frustrated/upset wait(R) : : | | | ******** | 197.Overreact frustrating(R) | | 0 ***** 190.Stop start new not upset 193.Tearful easily(R) --* : : : : 195.Overreact challenge(R) : : | | | ***** | | | | | | ******* | 185.Blame other(R) 187.Not react when other upset(R) | | | | | ****** | | | | | | ***** | | | | | | **** | 194.Angry/tearful no reason(R) | 2 | | | | ** | | | + -1 + + ***** + + + | | | *** | | | | | C | **** | | | | | H P | *** | 198.Frustration physical(R) | | | | | | | | | | | | | --| | | | | | | + -2 + + + + (1) + ------------------------------------------------------------------------------------------------------|Measr|-Raters| = 1 |-Items |Scale| ------------------------------------------------------------------------------------------------------

PAGE 133

133Facets ruler of the soci al communication domain --------------------------------------------------------------------------------------------------------+ 3 + + + + (4) + | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | + 2 + + + + + | | | ** | | --| | | | | | | | | | | | | | | | ** | | | | | | **** | | | | | | | | | + 1 + + *** + 212.Adds topic convers right + + | | | ** | 199.Get attention b4 convers 211.Info from lecture | 3 | | | | ****** | | | | | | *** | 204.Enough info when telling | | | | | ***** | 209.Keep up convers no repeat 210.Partici 10-20min convers | | : : : : 213.Start convers : : | | | ****** | 225.Repeat story told(R) 228.Too much info when tell(R) | | | | | ****** | 202.Talk more than one 203.Answerin appropriate time | | : : : : 206.Eye contact when convers 208.Acknowledge other's view : : : : : : 214.Misunderstand(R) 218.Stuck on a topic(R) : : : : : : 224.Lose thought convers(R) : : 0 ******** 200.Take a turn convers 207.Interest other saying --* | | | ** | 219.Talk wrong time (R) 221.Jumps topic unrelat convers(R) | | : : : : 226.Interrupt other phone(R) : : | | | ** | 216.Facial not match convers(R) 227.Interrupt convers(R) | | | | | ******** | 201.Greet person 205.Face person when speak | | : : : : 215.Rude/demanding request(R) : : | | | ****** | 223.Blurt out off topic(R) | | | | | ******* | 220.Embarrass question(R) | | | | | ******* | 222.Walks away convers(R) | 2 | + -1 + + + + + | | H | **** | | | | | C | *** | | | | | | | | | | | P | | | | | | | ** | 217.Too close when talk(R) | | | | | | | --| + -2 + + + + (1) + --------------------------------------------------------------------------------------------------------|Measr|-Raters| = 1 |-Items |Scale| ---------------------------------------------------------------------------------------------------------

PAGE 134

134 LIST OF REFERENCES Bjorner JB, Chang CH, Thissen D, et al.: Developing tailored instrume nts: item banking and computerized adaptive assessment. Qual Life Res 16 Suppl 1:95-108, 2007. Blonder LX: Language use, in Aphasia and languag e: Theory to practice. Edited by Nadeau SE, Gonzalez-Rothi L, Crosson B. New York, Guilford Press, 2000, pp. 284-295. Bond TG, Fox CM: Applying the Rasch Model. New Jersey, 2001. Box G, Draper N: Empirical Model Building and Response Surfaces. New York: John Wiley and Sons, 1987. Brooks N, Campsie L, Symington C, et al.: The five year outcome of severe blunt head injury: a relative's view. J Neurol Neur osurg Psychiatry 49:764-770, 1986. Brown TA: Confirmatory Factor Analysis for Applied Research. New York: The Guilford Press, 2006. Chan RC, Bode RK: Analysis of patient and proxy ratings on the Dysexecutive Questionnaire: an application of Rasch analysis. J Ne urol Neurosurg Psychiatry 79:86-88, 2008. Chaytor N, Schmitter-Edgecombe M: The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol Rev 13:181-197, 2003. Chaytor N, Schmitter-Edgecombe M: Fractio nation of the dysexecutive syndrome in a heterogeneous neurological sample: compar ing the Dysexecutive Questionnaire and the Brock Adaptive Functioning Questi onnaire. Brain Inj 21:615-621, 2007. Chaytor N, Temkin N, Machamer J, et al.: The ecological validity of neuropsychological assessment and the role of depressive sympto ms in moderate to severe traumatic brain injury. J Int Neuropsychol Soc 13:377-385, 2007. Cohen ME, Marino RJ: The tools of disability outcomes research functional status measures. Archives of Physical Medicine and Rehabilitation 81:S21-S29, 2000. Cook KF, Monahan PO, McHorney CA: Delicate ba lance between theory and practice: health status assessment and item respons e theory. Med Care 41:571-574, 2003. Cook KF, O'Malley KJ, Roddey TS: Dynamic asse ssment of health outcomes: time to let the CAT out of the bag? Hea lth Serv Res 40:1694-1711, 2005. Cook KF, Teal CR, Bjorner JB, et al.: IRT health outcomes data analysis project: an overview and summary. Qual Life Res 16 Suppl 1:121-132, 2007. Cope DN: The effectiveness of traumatic brain in jury rehabilitation: a review. Brain Inj 9:649670, 1995.

PAGE 135

135 Costello AB, Osborne JW: Best pr actices in exploratory factor analysis: four recommendations for getting the most from your analysis. Prac tical Assessment, Res earch & Evaluation 10, 2005. Coster WJ, Haley SM, Ludlow LH, et al.: Develo pment of an applied co gnition scale to measure rehabilitation outcomes. Arch Phys Med Rehabil 85:2030-2035, 2004. Crocker L, Algina J: Introduction to Classical & Modern Te st Theory. Orlando: Harcourt Brace Jovanovich College Publishers, 1986. Cusick CP, Gerhart KA, Mellick DC: Participant-Proxy Reliability in Traumatic Brain Injury Outcome Research. Journal of H ead Trauma Rehabilitation 15, 2000. de Gruijter DNM: Small n does not always justify Rasch model. Applied Psychology Measurement 10:187-194, 1986. Donovan NJ, Kendall DL, Heaton SC, et al.: Con ceptualizing functional cognition in stroke. Neurorehabil Neural Repair 22:122-135, 2008. Eflikides A, Yuiltsi E, Kangellidou T, et al.: Wechsler Memory Scale, Rivermead Behavioral Memory Test, and Everyday Memory Questionn aire in Healthy Adults and Alzheimer Patients. 18:63-77, 202. Embretson SE, Reise SP: Item Response Theo ry for Psychologist. New Jersey: Lawrence Erlbaum Associates, 2000. Fan X: Item response theory and classical te st theory: an empirical comparison of their item/person statistics. Educational an d Psychological Measurement 58:357-372, 1998. Finkelstein E, Corso P, Miller T: The Inciden ce and Economic Burden of Injuries in the United States New York: Oxford University Press, 2006. Godfrey HP, Partridge FM, Knight RG, et al.: Course of insight disorder and emotional dysfunction following closed head injury: a controlled crosssectional follow-up study. J Clin Exp Neuropsychol 15:503-515, 1993. Goldstein G, McCue M: Differences between pa tient and informant functional outcome ratings in head-injured individuals. International Journal of Rehabilitation and Health 1:25-35, 1995. Hahn EA, Cella D, Bode RK, et al.: Item banks a nd their potential applicati ons to health status assessment in diverse populations. Med Care 44:S189-197, 2006. Haley SM, Coster WJ, Andres PL, et al.: Scor e comparability of short forms and computerized adaptive testing: Simulation study with the activity measure for pos t-acute care. Arch Phys Med Rehabil 85:661-666, 2004.

PAGE 136

136 Haley SM, Coster WJ, Andres PL, et al.: Activ ity outcome measurement for postacute care. Med Care 42:I49-61, 2004. Haley SM, Ni P, Hambleton RK, et al.: Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank. J Clin Epidemiol 59:1174-1182, 2006. Haley SM, Siebens H, Coster WJ, et al.: Co mputerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes. Arch Phys Med Rehabil 87:1033-1042, 2006. Haley SM, Gandek B, Siebens H, et al.: Comput erized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Arch Phys Med Rehabil 89:275-283, 2008. Hall KM, Hamilton BB, Gordon WA, et al.: Characteristics a nd comparisons of functional assessment indices: Disability Rating Scal e, Functional Independence Measure, and Functional Assessment Measure. J Head Trauma Rehabil 8:60-74, 1993. Hall KM, Johnston MV: Outcomes evaluation in TB I Rehabilitation. Part II: measurement tools for a nationwide data system. Arch Phys Med Rehabil 75:SC10-18; discussion SC 27-18, 1994. Hall KM, Cope DN: The benefit of rehabilitation in traumatic brain injury: A literature review. Journal of Head Trauma Rehabilitation 10:1-13, 1995. Hall KM, Mann NC, High WM, Jr., et al.: Functio nal measures after traumatic brain injury: ceiling effects of FIM, FIM+ FAM, DRS, and CIQ. Journal of Head Trauma Rehabilitation, 1996. Hamilton BB, Granger CV, Sherwin FS, et al.: Uniform natinal data system for medical rehabilitation, in Rehabilita tion Outcomes: Analysis and Measurement. Edited by Fuhrer M. Baltimore, Brookes, 1987. Hart T, Whyte J, Polansky M, et al.: C oncordance of patient and family report of neurobehavioral symptoms at 1 year after traumatic brain injury. Archives of Physical Medicine and Rehabilitation 84:204-213, 2003. Hart T, Whyte J, Millis S, et al.: Dimensions of Disordered Attention in Traumatic Brain Injury: Further Validation of the Moss Attention Rati ng Scale. Archives of Physical Medicine and Rehabilitation 87:647-655, 2006. Hawley CA, Taylor R, Hellawe ll DJ, et al.: Use of the functional assessment measure (FIM+FAM) in head injury rehabilitation: a psychometric analysis J Neurol Neurosurg Psychiatry 67:749-754, 1999. Hays RD, Morales LS, Reise SP: Item response theory and health outcomes measurement in the 21st century. Med Care 38:II28-42, 2000.

PAGE 137

137 Heaton RK, Pendleton MG: Use of Neuropsychologica l tests to predict adult patients' everyday functioning. J Consult Clin Psychol 49:807-821, 1981. Heinemann AW, Linacre JM, Wright BD, et al.: Relationships between impairment and physical disability as measured by the functional i ndependence measure. Arch Phys Med Rehabil 74:566-573, 1993. Hellawell DJ, Taylor RT, Pentland B: Cognitiv e and psychosocial outcome following moderate or severe traumatic brain injury. Brain Inj 13:489-504, 1999. Hendryx PM: Psychosocial changes perceived by closed-head-injured adults and their families. Arch Phys Med Rehabil 70:526-530, 1989. Hu L, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structure Equation Modeling 6:1-55, 1999. Jennett B, Bond M: Assessment of outcome af ter severe brain damage. Lancet 1:480-484, 1975. Jette AM, Haley SM: Contemporary measuremen t techniques for rehabilitation outcomes assessment. J Rehabil Med 37:339-345, 2005. Johnston MV, Keith RA, Hinderer SR: Measuremen t standards for interdisciplinary medical rehabilitation. Arch Phys Med Rehabil 73:S3-23, 1992. Johnston MV, Hall KM: Outcomes evaluation in TB I Rehabilitation. Part I: overview and system principles. Arch Phys Med Reha bil 75:SC1-9; discussion SC27-28, 1994. Leathem JM, Murphy LJ, Flett RA: Selfand info rmant-ratings on the patient competency rating scale in patients with trau matic brain injury. J Clin Exp Neuropsychol 20:694-705, 1998. Lezak MD: Assessment of the behavioral conseq uences of head trauma. New York: Alan R. Liss, 1989. Lezak MD: Neuropsychological Assessment, 2004. Linacre JM: A User's Gyide to FACETS. Chicago: Winsteps, 2001. Linacre JM: WINSTEPS: Rasc h computer program. Chicago: MESA Press, 2004a. Linacre JM: Facets Rasch measurement co mputer program, in. Edited by. Chicago, Winsteps.com, 2004b. Linacre JM: A User's Guid e to FACETS. Chicago, 2005. Magaziner J, Simonsick EM, Kashner TM, et al.: Patient-proxy re sponse comparability on measures of patient health and functi onal status. J Clin Epidemiol 41:1065-1074, 1988. Malec JF: Relationship of the Mayo-Portland Adap tability Inventory to functional outcome and cognitive performance measures. journal of Head Trauma Rehabilitation 9:1-15, 1994.

PAGE 138

138 Malec JF, Machulda MM, Moessner AM: Differing problem perceptions of st aff, survivors, and significant others after brai n injury. Journal of Head Trauma Rehabilitation 12:1-13, 1997. Malec JF: Comparability of Mayo-Portland Adaptabi lity Inventory ratings by staff, significant others and people with acquired br ain injury. Brain Inj 18:563-575, 2004. McKinlay I: Annual meeting of the American Academy for Cerebral Palsy and Developmental Medicine, 1983. Abstracts. Dev Med Child Neurol 26:236-262, 1984. Mirsky AF, Anthony BJ, Duncan CC, et al.: An alysis of the elements of attention: a neuropsychological approach. Neuropsychol Rev 2:109-145, 1991. Morris JN, Hawes C, Fries BE, et al.: Designing the national resident assessment instrument for nursing homes. Gerontologist 30:293-307, 1990. Moscato BS, Trevisan M, Willer BS: The prevalence of traumatic brain injury and co-occurring disabilities in a national household survey of adults. J Neuropsychiatry Clin Neurosci 6:134-142, 1994. Moscovitch M: Information processi ng and the cerebral hemispheres in Handbook of behavioral neurobiology. Edited by, 1979. Myford CM, Wolfe EW: Detecting and measur ing rater effects using many-facet Rasch measurement: Part II. J Appl Meas 5:189-227, 2004. Portney LG, Watkins MP: Reliability, in Fundati ons of Clinical Research Applications to Practice. Edited by. New Jersey, Pr entice Hall Health, 2000a, pp. 61-77. Portney LG, Watkins MP: Multivar iate Analysis, in Foundations of Clinical Research. Edited by. New Jersey, Prentice-Hall, Inc., 2000b, pp. 587-620. Prigatano G, Schacter D: Awareness of deficit after brain injury: clinical and theoretical issues. New York: Oxford University Press, 1991. Prigatano GP, Fordyce DJ, Zeiner HK, et al.: Neuropsychological rehabilitation after brain injury. Baltimore: Johns Hopki ns University Press, 1986. Prutting CA, Kirchner DM: A clinical appraisal of the pragmatic aspects of language. J Speech Hear Disord 52:105-119, 1987. Rappaport M, Hall KM, Hopkins K, et al.: Disabili ty rating scale for seve re head trauma: coma to community. Arch Phys Med Rehabil 63:118-123, 1982. Reckase MD: The difficulty of test items th at measure more than one ability. Applied Psychological Measurement 9:401-412, 1985.

PAGE 139

139 Revicki DA, Cella DF: Health status assessmen t for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res 6:595-600, 1997. Richardson JT, Chan RC: The c onstituent structure of subjec tive memory questionnaires: evidence from multiple sclerosis. Memory 3:187-200, 1995. Roth RM, Isquith PK, Gioia GA: Behavior rating inventory of executive function-adulat version. Lutz: Psychological Assessment Recourses, Inc., 1996. Royle J, Lincoln NB: The Everyday Memory Questionnaire-revised: deve lopment of a 13-item scale. Disabil Rehabil 30:114-121, 2008. Sander AM, Seel RT, Kreutzer JS, et al.: Agreem ent between persons with traumatic brain injury and their relatives regarding psychosocial outcome using the community integration questionnaire,. Archives of Physical Medicine and Rehabilitation 78:353-357, 1997. Sbordone RJ, Seyranian GD, Ruff RM: Are the subjective complain ts of traumatically brain injured patients reliabl e? Brain Inj 12:505-515, 1998. Sbordone RJ, Guilmette TJ: Prediction of everyday and vocational functioning from neuropsychological test data frensic neuropsychology, 1999. Sbordone RJ: Limitations of neuropsychological te sting to predict the cognitive and behavioral functioning of persons with br ain injury in real-world settings. NeuroRehabilitation NeuroRehabilitation J1 NeuroRehabilitation 16:199, 2001. Scullen SE, Mount MK, Goff M: Understanding the latent structur e of job performance ratings. J Appl Psychol 85:956-970, 2000. Sherer M, Boake C, Levin E, et al.: Characteris tics of impaired awareness after traumatic brain injury. J Int Neuropsychol Soc 4:380-387, 1998. Sherer M, Hart T, Nick TG: Measurement of impa ired self-awareness after traumatic brain injury : a comparison of the patient competency ra ting scale at the awareness questionnaire. Brain Injury 17:25, 2003. Silver JM, McAllister TW, Yudo fsky SC: Textbook of Traumatic Brain Injury. Washington, DC: American Psychiatric Pub., 2005. Sohlberg MM, Mateer CA: Neurological Disorders Associated with Cognitive Impairments, in Cognitive Rehabilitation. Edited by. New York, The Guilford Press, 2001, pp. 25-58. Stout W: A nonparametric approach for assessing latent trait uni dimensionality. Psychometrika 52:589-617, 1987. Stout WF: A new item response theory mode ling approach with applications to unidimensionality assessment and abili ty estimation. Psychometrika 55:293-325, 1990.

PAGE 140

140 Stuss DT, Benson DF: The Frontal Lobes. New York: Raven Press, 1986. Stuss DT, Winocur G, Robertson IH (eds.): Cognitive Neurorehabilitation, Second Edition: Evidence and Application. Cambridge University Press, 2008. Tepper S, Beatty P, DeJong G: Outcomes in trau matic brain injury: self-r eport versus report of significant others. Brain Inj 10:575-581, 1996. Tupper DE, Cicerone KD: Introduction to the neuropsychology of everyday life, in The Neuropsychology of Everyday Life: Assessm ent and Basic Competencies. Edited by Tupper DE, Cicerone KD. Massachusetts, Kluwer Academic Publishers, 1990, pp. 3-18. van Baalen B, Odding E, Maas AI, et al.: Traumatic brain injury: classification of initial severity and determination of functional out come. Disabil Rehabil 25:9-18, 2003. Velozo CA, Wang Y, Lehman L, et al.: Uti lizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running. Disabil Rehabil 30:458467, 2008. Wainer H, Dorans NJ, Eignor D: Computerized Adaptive Testing: A primer. Mahwah, NJ: Lawrence Erlbaum Associates, 2000. Wang W-C, Chen C-T: Item Parameter Recovery, St andard Error Estimates, and Fit Statistics of the WINSTEPS Program for the Family of Ra sch Models. Educational and Psychological Measurement 65:376-404, 2005. Ware JE, Jr., Bjorner JB, Kosinski M: Prac tical implications of item response theory and computerized adaptive testing: a brief su mmary of ongoing studies of widely used headache impact scales. Med Care 38:II73-82, 2000. Ware JE, Jr., Gandek B, Sinclair SJ, et al.: Item response theory and computerized adaptive testing: implications for outcomes meas urement in rehabilitation. Rehabilitation Psychology 50:71-78, 2005. Whyte J, Hart T, Bode RK, et al.: The Moss A ttention Rating Scale for traumatic brain injury: initial psychometric assessment. Arch Phys Med Rehabil 84:268-276, 2003. Willer B, Rosenthal M, Kreutzer JS, et al.: Assessment of community integration following rehabilitation for traumatic brain injury. J ournal of Head Trauma Rehabilitation 8:75-87, 1993. Wilson BA, Alderman N, Burgess PW, et al.: BAD-Behavior Assessment of the Dysexecutive Syndome. Bury St. Edmunds: Thames Valley Test Company, 1996. Wolfson AM, Doctor JN, Burns SP: Clinician j udgments of functional outcomes: how bias and perceived accuracy affect rating. Arch Phys Med Rehabil 81:1567-1574, 2000. Wright BD, Masters GN: Rating Scale Analysis. Chicago: MESA Press, 1982.

PAGE 141

141 BIOGRAPHICAL SKETCH Pey-Shan Wen was born in Taiwan. She rece ived B.S. degree in National Cheng-Kung University in Tainan, Taiwan in 1996 and worked as a licens ed occupational therapist in Taiwan for six years after she graduated. In 2003, Pey-Shan enrolled in the advanced masters program in the Department of Occupational Therapy of the University of Flor ida. While she still worked on her masters degree, in 2004, she enrolled in th e rehabilitation scienc e doctoral program of the University of Florida and started her research training. She was a research assistant for her mentor, Dr. Velozo, and worked as a project coordinator for a NI H grant Develop a computer adaptive TBI measure. Meanwhile, she also involved in teac hing a few classes. In 2007 December, she was award her masters degree and continued wo rking on her Ph.D. In 2009, she was a teaching assistant for two classes: Pat hophysiology and Research method. In the same year, she received her Ph.D. from the University of Florida.