<%BANNER%>

ADHD Assessment

Permanent Link: http://ufdc.ufl.edu/UFE0041447/00001

Material Information

Title: ADHD Assessment Incremental Validity of Objective Measures of Child Behavior
Physical Description: 1 online resource (73 p.)
Language: english
Creator: Stewart, Lindsay
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: adhd, assessment
Clinical and Health Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Attention-Deficit/Hyperactivity Disorder (ADHD) is one of the most common psychological disorders of childhood. As such, determining empirically supported assessment procedures for diagnosing ADHD is an important research goal. Current guidelines highlight the necessity of including multiple informants and settings in the assessment process. While multiple methods (e.g. paper and pencil questionnaires, computerized measures of attention) are also commonly used by psychologists as they conduct ADHD evaluations, there is little research concerning the incremental validity of various methods. In order to examine this issue, the current study evaluated the degree to which clinic-based measures of child behavior (in this case, clinician ratings of child behavior during testing and child performance on a computerized measure of attention) predicted functional impairment above and beyond parent and teacher ratings of ADHD. The study also considered the impact of oppositional and defiant symptoms, which make up the most commonly occurring comorbid condition with ADHD. Participants included a sample of 27 children who met diagnostic criteria for ADHD. Correlational analyses revealed that parent and teacher ratings of ADHD and oppositional symptoms were strongly related to parent and teacher ratings of impairment, respectively. Clinic-based measures of ADHD behavior were mildly to moderately associated with impairment indices. Bivariate correlations also revealed that clinician ratings were significantly associated with age, parent ratings of hyperactivity/impulsivity, and oppositional/defiant symptoms. Computerized attention (CPT) scores were significantly associated with age and moderately associated with teacher ratings of hyperactivity/impulsivity at a level that approached significance. According to hierarchical regression analyses, neither clinician ratings of behavior during testing nor the CPT composite added a significant increment to the prediction of impairment above and beyond ADHD ratings. Interestingly, Oppositional Defiant Disorder (ODD) symptoms explained a high degree of variance in impairment, in some cases greater than ADHD symptoms. The current findings support the notion that clinicians should rely heavily on symptom checklists as cost and time-efficient means of diagnosing ADHD. The outcome of this and other studies question the ecological validity of measures of clinic-based behavior. Taken together, the results call for a strong reliance on assessment measures and methods that tap the child?s everyday attention, impulsivity and activity level.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Lindsay Stewart.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Johnson, James H.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-04-30

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041447:00001

Permanent Link: http://ufdc.ufl.edu/UFE0041447/00001

Material Information

Title: ADHD Assessment Incremental Validity of Objective Measures of Child Behavior
Physical Description: 1 online resource (73 p.)
Language: english
Creator: Stewart, Lindsay
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: adhd, assessment
Clinical and Health Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Attention-Deficit/Hyperactivity Disorder (ADHD) is one of the most common psychological disorders of childhood. As such, determining empirically supported assessment procedures for diagnosing ADHD is an important research goal. Current guidelines highlight the necessity of including multiple informants and settings in the assessment process. While multiple methods (e.g. paper and pencil questionnaires, computerized measures of attention) are also commonly used by psychologists as they conduct ADHD evaluations, there is little research concerning the incremental validity of various methods. In order to examine this issue, the current study evaluated the degree to which clinic-based measures of child behavior (in this case, clinician ratings of child behavior during testing and child performance on a computerized measure of attention) predicted functional impairment above and beyond parent and teacher ratings of ADHD. The study also considered the impact of oppositional and defiant symptoms, which make up the most commonly occurring comorbid condition with ADHD. Participants included a sample of 27 children who met diagnostic criteria for ADHD. Correlational analyses revealed that parent and teacher ratings of ADHD and oppositional symptoms were strongly related to parent and teacher ratings of impairment, respectively. Clinic-based measures of ADHD behavior were mildly to moderately associated with impairment indices. Bivariate correlations also revealed that clinician ratings were significantly associated with age, parent ratings of hyperactivity/impulsivity, and oppositional/defiant symptoms. Computerized attention (CPT) scores were significantly associated with age and moderately associated with teacher ratings of hyperactivity/impulsivity at a level that approached significance. According to hierarchical regression analyses, neither clinician ratings of behavior during testing nor the CPT composite added a significant increment to the prediction of impairment above and beyond ADHD ratings. Interestingly, Oppositional Defiant Disorder (ODD) symptoms explained a high degree of variance in impairment, in some cases greater than ADHD symptoms. The current findings support the notion that clinicians should rely heavily on symptom checklists as cost and time-efficient means of diagnosing ADHD. The outcome of this and other studies question the ecological validity of measures of clinic-based behavior. Taken together, the results call for a strong reliance on assessment measures and methods that tap the child?s everyday attention, impulsivity and activity level.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Lindsay Stewart.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Johnson, James H.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-04-30

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041447:00001


This item has the following downloads:


Full Text

PAGE 1

ADHD ASSESSMENT: INCREMENTAL VALIDITY OF OBJECTIVE MEASURES OF CHILD BEHAVIOR By LINDSAY MCALISTER STEWART A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORID A IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2010 1

PAGE 2

2010 Lindsay McAlister Stewart 2

PAGE 3

To my teachers including the families wh o come seeking guidance from the mental health profession 3

PAGE 4

ACKNOWLEDGMENTS I would like to express my sincere appr eciation and gratitude to my chair, Dr. James Johnson, without whom this project w ould not be possible. He has generously provided his knowledge, encouragement, and critical eye to my work throughout these past years, and from him I have observed wh at it means to be a dedicated scientist, practitioner, and mentor. I would also like to acknowledge the additional members of my doctoral committee: Drs. Stephen Boggs Shelley Heaton, and Fonda Eyler. I extend special thanks to Dr. Shelley Heaton fo r allowing me the position of working in her neuropsychology lab prior to my enteri ng graduate school at the University of Florida. I attribute her ment orship during this time to my developing interests in child psychology and the research area of ADHD. Funding for this study was generously provi ded by mini-grants from the Center for Pediatric Psychology and Family Studies. I would also like to thank the staff and psychologists at the Behavioral Institute of Atlanta who supported data collection efforts for this project. Specifical ly, Drs. Stephen Garber, Frank Ba tkins, and Mary Helen Hunt assisted with the screening process. D ebbie Buchman and Joanne Linder also kindly gave of their time. Finally, I acknowledge the instrumental roles of my family and husband, whose loving support has extended over distance and time to allow me to work towards my academic goals. 4

PAGE 5

TABLE OF CONTENTS page ACKNOWLEDG MENTS .................................................................................................. 4LIST OF TABLES ............................................................................................................ 7ABSTRACT ..................................................................................................................... 8CHAPTER 1 INTRODUC TION .................................................................................................... 10Overview of Attention-Defici t/Hyperactivity Disorder ............................................... 10Assessment of ADHD ............................................................................................. 11Incremental Validity in Asse ssment ........................................................................ 17Current St udy .......................................................................................................... 222 METHOD S .............................................................................................................. 26Participants ............................................................................................................. 26Procedur e ............................................................................................................... 29Measures ................................................................................................................ 30Statistical Analyses ................................................................................................. 343 RESULT S ............................................................................................................... 39Preliminary Analyses .............................................................................................. 39Primary Aims .......................................................................................................... 41Aim 1: To Determine the Incremental Validity of Ratings of ADHD Symptoms and Child Behavior During Te sting in Predicting Functional Impairment .................................................................................................... 41Predicting parent-rated functional impairment ............................................ 41Predicting teacher-rated f unctional impai rment .......................................... 42Aim 2: To Determine the Incremental Validity of Ratings of ADHD Symptoms and Child Performance on the Continuous Performance Test in Predicting Functi onal Impai rment .............................................................. 42Predicting parent-rated functional impairment ............................................ 43Predicting teacher-rated f unctional impai rment .......................................... 43Follow-up Aim: To Determine the Predict ive Value of Oppos itional Defiant Disorder Symptoms Relative to Other Variables in Predicting Functional Impairment .................................................................................................... 44Predicting parent-rated functional impairment ............................................ 44Predicting teacher-rated f unctional impai rment .......................................... 44 5

PAGE 6

4 DISCUSSI ON ......................................................................................................... 49Preliminary Analyses .............................................................................................. 52Parent/Teacher Ratings of A DHD Symptoms and Impairment ............................... 52Parent/Teacher Ratings of ODD Symptoms and Impairment ................................. 54Clinicians Ratings During Test ing .......................................................................... 55Childrens Performance on the C ontinuous Perform ance Test ............................... 58Limitations and Futu re Directi ons ........................................................................... 60Summary ................................................................................................................ 63LIST OF RE FERENCES ............................................................................................... 66BIOGRAPHICAL SKETCH ............................................................................................ 73 6

PAGE 7

LIST OF TABLES Table page 2-1 Demographic characteristi cs of child par ticipants ............................................... 372-2 Demographic characteri stics of parent respondents ........................................... 372-3 Descriptive statistics for the administered measures .......................................... 383-1 Correlations among vari ables of in terest ............................................................ 463-2 Hierarchical regression analyses predicting parent-rated functional impairment .......................................................................................................... 473-3 Hierarchical regression analyses predicting teacher-rated functional impairment .......................................................................................................... 473-4 Follow-up hierarchical r egression analysis predicting pa rent-rated functional impairment .......................................................................................................... 483-5 Follow-up hierarchical regression analysis predicting teacher-rated functional im pairment ......................................................................................... 48 7

PAGE 8

Abstract of Dissertation Pr esented to the Graduate School of the University of Florida in Partial Fulf illment of the Requirements for t he Degree of Doctor of Philosophy ADHD ASSESSMENT: INCREMENTAL VALIDITY OF OBJECTIVE MEASURES OF CHILD BEHAVIOR By Lindsay McAlister Stewart May 2010 Chair: James H. Johnson Major: Psychology Attention-Deficit/Hyperactivity Disor der (ADHD) is one of the most common psychological disorders of childhood. As such, determining empirically supported assessment procedures for diagnosing ADHD is an important research goal. Current guidelines highlight the necessi ty of including multiple informants and settings in the assessment process. While multiple methods (e.g. paper and pencil questionnaires, computerized measures of att ention) are also commonly used by psychologists as they conduct ADHD evaluations, there is little research concerning the incremental validity of various methods. In order to examine this issue, the current study evaluated the degree to which clinic-based measures of child behavior (in th is case, clinician ratings of child behavior during testing and child performance on a com puterized measure of attention) predicted functional impairment above and beyond parent and teacher ratings of ADHD. The study also considered the impact of oppositional and defiant symptoms, which make up the most commonly occurring como rbid condition with ADHD. Participants included a sample of 27 child ren who met diagnostic criteria for ADHD. Correlational analyses revealed that parent and teacher ratings of ADHD and 8

PAGE 9

9 oppositional symptoms were st rongly related to parent and teacher ratings of impairment, respectively. Clinic-based measures of ADHD behavior were mildly to moderately associated with im pairment indices. Bivariat e correlations also revealed that clinician ratings were significantly associated with age, parent ratings of hyperactivity/impulsivity, and oppositional/de fiant symptoms. Co mputerized attention (CPT) scores were significantly associat ed with age and moderately associated with teacher ratings of hyperactivity/impulsivi ty at a level that approached significance. According to hierarchical regression analyse s, neither clinician ratings of behavior during testing nor the CPT comp osite added a significant incr ement to the prediction of impairment above and beyond ADHD ratings. Interestingly, Oppositional Defiant Disorder (ODD) symptoms expl ained a high degree of variance in impairment, in some cases greater than ADHD symptoms. The current findings support the notion that clinicians should rely heavily on symptom checklists as cost and time-effi cient means of diagnosing ADHD. The outcome of this and other studies question the ec ological validity of measures of clinicbased behavior. Taken toget her, the results call for a strong reliance on assessment measures and methods that tap the childs everyday attention, impulsivity and activity level.

PAGE 10

CHAPTER 1 INTRODUCTION Overview of Attention-Deficit/Hyperactivity Disorder Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder of childhood characterized by developmentally inappropriate levels of inattention, hyperactivity, and impulsivity. With an estimated prevalence in the general school-age population of approximately 8% (Center for Disease C ontrol, 2005) and 4-12% in pediatric primary care settings (American Academy of Pediatrics [AAP], 2000), ADHD is one of the most common childhood disorder s. Indeed, ADHD accounts for a large number of referrals to physician and child me ntal health profession als (Barkley, 1998). Research suggests that a disproportionate num ber of males as compared to females are diagnosed with the disorder, with an estimated ratio of appr oximately 9:1 in clinical samples and a ratio of 4:1 in epide miological samples (Cantwell, 1996). According to the Diagnostic and Statistica l Manual of Mental Disorders, Fourth Edition (DSM-IV; APA, 1994), the diagnostic criteria for ADHD incorporates two symptom groupings based on inattention and hy peractivity/impulsivity. Depending upon the number of symptoms pres ent within each grouping, one of three ADHD subtypes of ADHD are considered when making a diagn osis. These include the Predominantly Inattentive, Predominantly Hyperactive-Im pulsive, and the Combined subtypes. The DSM-IV criteria also explicitly incorpor ate requirements regarding symptom onset (before 7 years of age), symptom duration (at least 6 months), developmental deviance, and cross-situational pervasiveness of impai rment from ADHD symptoms. The DSM-IV makes specific mention that there must be evidence of clin ically significant impairment in social, academic, or occupational function ing. Indeed, level of impairment is a 10

PAGE 11

diagnostic consideration that carries very re levant implications wit h regard to treatment, as children with ADHD often have difficulty in the classroom or on t he job, display lower adaptive functioning and poorer self-care, have more social problems, and are at higher risk for unintentional injury (Barkley, 1998; Ba rkley, Fischer, Smallish, & Fletcher, 2006). Assessment of ADHD Assessment of ADHD is a process that in volves the consideration of information from multiple informants (e.g. parents and t eachers), multiple settings (e.g. home and school), and oftentimes, multiple methods (interviews, observations, rating scales). It is important to note that specific assessment practices vary depending on what the goals for assessment are. In res earch settings, for example, a pr imary objective is to obtain a diagnosis. In clinical settings, on the other hand, assessment is broader and may involve making a diagnosis, treatment planni ng, and evaluating a tr eatment over time (Pelham, Fabiano, & Massetti 2005). Further, as Anas topoulos and Shelton (2001) note, other potential influenc es on how assessment is carried out may include the clinicians level of experience, his or her beliefs about the cause of psychopathology, the degree of methodological rigor employed, and the fact that there is quite a large array of instruments to select from. Given the complexity inherent in the assessment process, both the American Academy of Pediatrics (2001) and the American Academy of Child and Adolescent Psychiatry (2007) have published general guidel ines for evaluating children suspected of having ADHD. These guidelines featur e key components, such as screening for ADHD when core symptoms are present, employing DSM-IV criteria as the primary basis from which a diagnosis of ADHD is given, gathering information about symptoms from parents and teachers, assessing for the presence of coexisting conditions, and 11

PAGE 12

using DSM-IV based rating scales rather than global questionnaires (AAP, 2001). Interviews with the parent and teacher th rough using structured and semi-structured interviews, verbal narratives, and/or or open-ended questions are suggested by the American Academy of Pediatrics as an effe ctive means to determine age of onset and frequency and duration of symptoms across setti ngs. While structured interviews such as the National Institute of Mental Health s Diagnostic Interview Schedule for Children (DISC-IV) have strong empirical support (S haffer, Fisher, Lucas, Dulcan, & SchwabStone, 2000), there are lim itations in that the format can be quite lengthy. Open-ended questions, on the other hand, while not empirica lly supported in and of themselves, are valuable in clinical settings as they prov ide some of the information essential for diagnosis, such as age of onset and frequen cy of symptomatologyand are effectively supplemented by structured self-report rating scales. Parent and teacher rating scales are a common and efficient means to get information about the presence and severity of the childs symptoms and, importantly, to determine the degree to which the childs symp toms are significantly different from same-aged peers (Pelham, Fabiano, & Massetti 2005). Such scales tend to be easy to use, efficient, and do not require special trai ning on the part of the informant (Hinshaw & Nigg, 1999). A noteworth y feature of rating scales is that they target appraisals from adults who observe the children in their daily environment (such as the home or school), where disruptive behaviors are likely to occur (Barkley, 1998). Barkley points out, however, that these measures require t he informants opinions and perspectives on child behavior and may reflect biased responding. The author emphasizes that professionals should take multiple informant s into account while at the same time 12

PAGE 13

considering the developmental history in orde r to grasp the key features of a childs behavior. Among the numerous available DSM-IV-based narrow band scales, several stand out as possessing strong empirica l support. The Conners Rating Scales (Conners, 1997, 2008), for example, have been proven useful in a wide variety of clinical and research settings. Available in parent, teacher, and adolescent self-report forms, they are relatively easy to admin ister and have strong psychometric properties, all of which make them appealing. T he scales have been used to establish the convergent validity of other ADHD symptom ratings scales as well as measures of externalizing behavior disorders (Collett, Ohan, & Myers, 2003). Furt her, in treatment outcome studies, the parent and teacher versions of the scale have shown sensitivity to methylphenidate treatment effects in children with ADHD (Gadow, Sverd, Sprafkin, Nolan, & Grossman, 1999). In addition to parent and teacher ratings of the childs ADHD symptoms, the DSMIV also specifies that there must be evi dence of impairment in academic, social, or occupational domains of functioning. I ndeed, children with ADHD have been shown to be impaired in a variety of areas, including adaptive functioning (difficulties with selfhelp and independent behaviors), academic f unctioning, social functioning, and home/family functioning (Biederman et al., 1999; Shelton et al., 1998; Lahey et al., 1998; Pelham et al., 1998). Examples of the range of impairments include not adjusting to changes in routine or showing appropriate concerns for safety, being less likely to comply with adults, breaking classroom rules, being more likely to repeat a grade level, demonstrating less cooperation among peers, and having problems interacting with siblings and family members. Despite the emphasis plac ed on the impairment criterion 13

PAGE 14

by the DSM-IV diagnostic guidelines for ADHD and the research that testifies to the presence of significant functional problems in these children, the DSM-IV does not specify how impairment should be gauged. As Bird et al. (2005) highlight, research instruments offer the potentia l to strengthen the clinical process by helping obtain reliable diagnostic conclusions. There are several general measures of impairment that possess empirical support, including the Vinelan d Adaptive Behavior Scale (Sparrow, Balla, & Cicchetti, 1984), the Childrens Glob al Assessment Scale (Shaffer et al., 1983), the Columbia Impairment Rating Scale (Bir d et al., 1993), and the Brief Impairment Scale (Bird et al., 2005). Some of the limit ations of the existing global measures (the reader is referred to a review by Winters, Collett, and Myers [2005] for a more extensive review of existing measures for use with children and adolescents) are that most are unidimensional and provide only a single rating for overall impairmen t, many are rather lengthy and thus are not ideal for use in clinical settings, and none link impairment to difficulties that occur specifically as a resu lt of ADHD behavior. In response to the need for a multidimensional measur e to quantify impairment in ch ildren and particularly in children with ADHD, Fabiano et al. (2006) developed the Impairment Rating Scale (IRS), whose parent and teacher versions prov ide information on the childs functioning across various domains, including relationship with peers, relationship with parents, relationship with teacher, academic progress, and self-esteem. Mu ltiple areas of impairment are important to consider, especia lly in treatment planning, as these are often the reason for referral and constitute po ssible treatment target s (Angold, Costello, Farmer, Burns, & Erkanli., 1999) Of note, the IR S also taps the raters assessment of the childs need for treatment and special servic es. Preliminary findings from a recent 14

PAGE 15

study, which used samples of children with ADHD and two school-based samples, indicated that both parent and teacher vers ions of the IRS demonstrated good temporal stability, correlated with ot her impairment ratings, and successfully discriminated between children with and without ADHD (Fab iano et al., 2006). The authors also indicated that although the correlations between ADHD symptoms and the IRS ratings of impairment tended to be large, there was substantial unshared variance between these measures. A thorough evaluation of ADHD should also include an assessment for comorbid disorders; for as many as two-thirds of clinic-referred children with ADHD also have a diagnosis of another condition that affe cts their functioning (Cantwell, 1996). Oppositional Defiant Disorder (ODD) and Conduc t Disorder (CD) ar e the most common comorbid conditions associated with ADHD, with estimates ranging between 54 and 84% in some samples. Estimates also in dicate that up to 25-35% of children with ADHD have learning disabilities (Pliszka, Carl son, & Swanson, 1999). Prevalence rates of comorbid internalizing problems, while not uncommon, tend to be lower (MTA Cooperative Group, 1999). To assess for co morbidity, broadband measures such as the Behavioral Assessment System for Ch ildren (BASC; Reynolds & Kamphaus, 2002) or the Child Behavior Checklist (CBCL, Ac henbach, 1991) are commonly used as they tap into multiple dimensions of child functioning, possess strong psychometric properties, and are relatively efficient to administer. Laboratory measures of vigilance and su stained attention are sometimes used during comprehensive clinical ADHD evaluati ons, particularly because they offer the appeal of a controlled assessment that is grounded in a childs actual behavior (Barkley, 15

PAGE 16

1998; Heaton et al., 2001) and t hey are not subject to biases that sometimes plague parent-report measures. The te st most commonly used in research and clinical settings to evaluate sustained attention in children suspected of having AD HD is the Continuous Performance Test (CPT), originally develo ped by Rosvold, Mirsky, Sarason, Bransome, and Beck (1956). CPTs now constitute a broad ca tegory of tests, but they all share a basic design that involves a rapid presentation of changing stimuli and directions requiring test-takers to be vi gilant of an infrequently occurring stimulus (Riccio, Reynolds, & Lowe, 2001). Two types of errors, omissions and commissions, are thought to reflect problems relating to sust ained attention and impulsivity. Omissions occur when a subject misses a target stimul us, while errors of commission occur when a subject responds in the absence of the target stimulus. While the American Academy of Pediatrics (2001) warns that laborator y-based measures should not be used making diagnoses of ADHD due to questionable ecological validity and the lack of data to suggest favorable levels of sensitivity and spec ificity, such measures can be useful in understanding elements of cognition in ADHD (Pelham, Fabiano, & Massetti, 2005). It is noteworthy that research has consist ently shown that children with ADHD perform more poorly than normal control groups on continuous performance tasks (Epstein et al., 2003). While scores may not necessarily imply a diagnosis of ADHD as problems of attention can result from various childho od problems, some measures do seem to reflect the presence of attention problems. With the definition of ADHD being highly grounded in the behaviors displayed by a child in his or her daily life (APA, 1994) it is no surprise that behavioral observations have a long history of use in evaluating for ADHD. Many observational measures, such 16

PAGE 17

as the Classroom Observations of Conduct and Attention Deficit Disorders (COCADD; Atkins, Pelham, & Licht, 1988) and the Cla ssroom Behavior Code (Abikoff, GittelmanKlein, & Klein, 1977), invo lve an independent observer w ho evaluates the childs behavior in a clinic or naturalistic (e.g. classroom) setting. Behavior codes are based on elements such as time spent on-task and out-of-seat behavior. The observational codes from the measures listed here exhibit a cceptable reliability and validity, effectively discriminate between children with ADHD and comparison children, and have demonstrated sensitivity to the effects of treatment (Pelham, Fabiano, & Massetti, 2005). Another observational measure, t he Hillside Behavior Ra ting Scale (HBRS; Gittelman & Klein, 1985), utilizes a seven-item scale where observers rate a childs behavior during a testing situation. The HBRS items assess domains that directly relate to DSM-IV ADHD symptoms as well as disrup tive behavior more generally. Instead of using a Likert-scale format, the rater is asked to c hoose an operationally-defined descriptor that corresponds best to the child s behavior during the observation period. Preliminary evidence provides support for the internal and interrater consistency of the HBRS as well as its convergent and conc urrent validity (Willcutt, Hartung, Lahey, Loney, & Pelham, 1999). Incremental Validity in Assessment As illustrated by the many components involved in a thorough psychological evaluation of ADHD that are des cribed above, assessment of t he disorder is not simply a matter of considering specific behaviors us ing a single measure at a given time point (Johnston & Murray, 2003). The authors point out that assessment of children and adolescents should be multimodal (including, fo r example, interviews, rating scales, and behavior observations), multi-informant (most commonly incorporating information from 17

PAGE 18

parents and teachers), and involves gathering data in different settings. As such, the evaluation process requires ong oing decision-making as well as a synthesis of many pieces of information on the part of the clin ician. With the va riety of measures and methods available for assessing ADHD, the ques tion of how to select and combine data derived from these measures is particularly relevant. Johnston and Murray state that the entire assessment process, including the particular instruments and procedures for how to combine the incoming information, should be empirically supported. Indeed, the American Psychological Associations (APA ) move to establish a Psychological Assessment Work Group in 1996 testifies to the emphasis placed on using scientifically validated assessment instrum ents. In fact, many p sychological tests appear to demonstrate sound psychometric properties, including validity that, in some cases, are comparable to that of medical tests (Meyer et al., 2001; J ohnston & Murray). It is also widely agreed upon that using a battery of te sts is preferred over more simplified approaches. It should be noted, however, that ther e is relatively little empirical data in either the adult or child literature to suggest how to best and most efficiently combine multiple assessment components for purposes of diagnosis, assessing impairment, and/or treatment planning (Hunsley & Meyer, 2003; Johnston & Murray). Questions of incremental validity relate to determining the most efficient and effective means of assessing ADHD. Johns ton and Murray (2003) define incremental validity as answering the following question: Does the procedure or method add to the assessment process in a way that improves the outcome? (p. 496). As mentioned previously, psychometric properties of psychological tests are necessary but not sufficient in determining incremental validity. In their comprehens ive review covering 18

PAGE 19

the incremental validity in the psychological assessment of children and adolescents, Johnston and Murray define various types of incremental validity. First, the incremental validity of multiple informants conveys whet her or not different informants contribute unique, clinically-useful information to t he assessment processes. Smith, Pelham, Gnagy, Molina, and Evans (2000) found that in a sample of adolescents with ADHD, adolescent self-report of ADHD symptoms did not contribute incremental validity over and above parent reports in pr edicting the observed frequency of negative behavior. With respect to combining parent and teac her reports, the results seem to depend on the goal of the assessment. Power et al. ( 1998) and Power, Costigan, Leff, Eiraldi, and Landau (2001) found that in a popul ation of school-age childr en, teacher reports were more useful than parent reports for ruling out a diagnosis of ADHD and for discriminating between subtypes of ADHD. On the other hand, a combination of both parent and teacher reports tends to be more useful for c onfirming a diagnosis of ADHD (as documenting the presence of core symptoms across situations is a prerequisite for diagnosis). A second type of incremental validity, wh ich involves measures of multiple constructs, relates to whether or not consi dering more than one domain of functioning (or construct) adds value to the assessm ent (Johnston & Murray, 2003). The authors note that most studies involving incr emental validity and the assessment of child/adolescent problems are of this type. For example, Hinshaw et al. (2000) reported that assessing more than one aspect of child functioning improves the ability to predict response to treatment. In a report on the findings from t he MTA study, Hinshaw et al. noted that children with ADHD and anxiety responded equally well to medications and 19

PAGE 20

psychosocial treatments, whereas childr en with ADHD and ODD responded best to medication treatment. Studies of incremental validity of multiple constructs have also looked at the contribution of including family and parenting characteristics in the assessment of child behavior problems, with re sults supporting this practice (DeVito & Hopkins, 2001; Ford et al., 1999). A third type of incremental va lidity, and the one most pert inent to the current study, is that of multiple methods. The incremental validity of multiple methods considers the contribution and utility of adding additional methods to an existing strategy of assessment. Studies utilizing multiple methods have most commonly combined the informants responses on questionnaires with objective measures of the childs behavior. Here, objective measures may in clude naturalistic observations, laboratory analogues, and performance on laboratory ta sks (Johnston & Murray, 2003). Even though these types of measures offer the theoretical appeal of being grounded in child behavior and the potential to add unique info rmation beyond traditional parent and teacher report measures, little research has addressed the incremental validity of these methods (Johnston & Murray). The literature t hat does exist reflects mixed results. Lobitz and Johnson (1975) indicated that parent reports of their children demonstrated high levels of sensitivity and specificity: 90% of clinic-referred and 90% of nonreferred children were correctly classified based on t hese reports. Of note, however, neither laboratory based nor home observations of the childs behavior added significantly to these prediction rates. Similarly, Sleator and Ullmann (1981) found that observations of whether or not the child displayed obvious hyperactive behaviors during his or her visit to the physicians office did not correspond well with parent and teacher reports of 20

PAGE 21

ADHD. Further, these observations did not si gnificantly add to the prediction of future problems. Sleator and Ullmann c oncluded that their findings lend evidence to the notion that the patients behavior in t he clinic setting is not always indicative of his or her usual behavior. In comparison to these studies, wh ich suggest negligible incremental validity of clinic-based measures of child behavio r, Glutting, Robins, and de Lancey (1997) found that examiners standardized ratings of inattentiv e, hyperactive, and impulsive behaviors during intellectual testing provided useful information about the childs ADHD features. Willcut, Hartung, Lahey, Loney, and Pelham (1999) discovered that ratings of inattentive, hyperactive, and disrupt ive behavior made by examiners during psychological testing of preschool children did provide a significant increment in the prediction of impairment abov e and beyond parent and teacher behavior ratings and other controlled variables, such as age, gender, IQ, and the presence of other psychopathology in addition to ADHD. In light of this significant increment, the findings suggest that clinician-based ratings of child behavior may, in fact, be useful as an additional indicator of ADHD. This study considered behavior ratings of preschool children only; thus, the authors note that futu re studies of clinic behavior in older children would provide a needed ex tension of their work. Factors that likely influenced the findings of the above studies include the nature of the specific behaviors observed, the st andardization of the observational measures, the type of other informati on that was available during assessment (Johnston & Murray, 2003) as well as the degree of shared varian ce between the predictor and outcome variables. Despite these potential confounds, the question of the incremental validity of multiple methods is an import ant one worth pursuing. Co mpared to other methods of 21

PAGE 22

evaluating child behavior, measures such as naturalistic or clinic-based observation systems and continuous performance tests tend to be relatively more labor and costintensive. In light of the greater co st and time commitments inherent in these techniques, questions exist as to whether they offer unique information by adding incrementally to the assessment process. The current study will provide a framework for evaluating the empirical support for ut ilizing multiple methods (i.e. symptom-based ratings scales and clinic-based behavioral m easures) in the assessment of ADHD. Current Study The purpose of this study is to examine the incremental validit y of measures of child behavior in the context of a comprehen sive clinical evaluation of ADHD. As indicated, there are few publis hed studies that address the i ssue of incremental validity as it relates to the assessment of child and adolescent psychological disorders in general. The several studies that deal with the incremental validity of multiple methods in ADHD assessment have yielde d conflicting results; thus the question persists as to whether or not includ ing additional measures of child behavior adds unique information above and beyond parent and teac her reports of ADHD sym ptoms. In their comprehensive review of issues related to incremental validity in the psychological assessment of children and adolescents, Johnston and Murray (2 003) state that determining incremental validity is facilitat ed when the criterion is objective, widely agreed upon, and not c onfounded with the information used in prediction (p. 499). Although it would be of great interest to determine whether or not clinic-based measures of child behavior add value (above and bey ond parent and teacher ratings of ADHD symptoms) in predicting a diagnosi s of ADHD, it is difficult to arrive at a valid diagnosis without considering these measures (i.e observational measur es, DSM-IV symptom22

PAGE 23

based rating scales). As such, when diagnosis is the criterion variable, it is likely confounded with the information used in predic tion. Willcut, Hartung, Lahey, Loney, and Pelham (1999) lessened the impact of this confound by instead using functional impairment as the criterion variable. As de scribed in an earlier section, children with ADHD have been shown to be impaired in a vari ety of areas, and it is this impairment that often brings affected childre n to the attention of service providers. It has also been suggested that indices of impairment are more predictive of outcome than are measures of ADHD symptom severity (Pelham, Fabiano, & Massetti, 2005). In a large scale study of school-aged youths, Angold, Costello, Farme r, Burns, and Erkanli (1999) found that individuals with symptomatic impairment were much more concentrated among service users compared to individuals with a diagnosis but no impairment. Impairment is also highly relevant in that the domains of functioning (e.g. adaptive, academic, social, home/family functioning) make for viabl e treatment targets (Pelham, Fabiano, & Massetti, 2005). The Willcut, Hartung, Lahey Loney, and Pelham (1999) study, which utilized impairment as the crit erion variable, operationalized this construct with various teacher, parent, and child measures, including global indices of impairment, measures of social competence and social functioning, and measures of academic achievement and IQ. It should be noted that since the time of this study, parent and teacher-report measures have been newly developed to a ssess ADHD-specific impairment across relevant domains of functi oning. The Impairment Rating Scale (IRS; Fabiano et al., 2006) is of particular interest, due to its multidimensional nat ure, preliminary evidence of acceptable psychometric properties, and the availability of both parent and teacher forms. 23

PAGE 24

The current study will therefore, using impairment as the criterion variable, seek to evaluate the incremental validity of measures of ch ild behavior above and beyond parent and teacher ratings of ADHD. The study will also attempt to consider the impact of oppositional and defiant symptoms, which make up the most commonly occurring comorbid condition with ADHD. Explori ng oppositional symptoms is of interest, especially since there is hi storically a very strong relationship between ODD and symptoms of hyperactivity and impulsivity (A merican Academy of Child and Adolescent Psychiatry, 2007). Based on previous rese arch on the link between impairment and ODD symptoms (Fabiano et al., 2006) it is also likely that symptoms of oppositional and defiant behaviors will be very strongly relat ed to impairment. The relationship between child demographic variables (such as age and IQ) and their relationship to impairment will also be considered. The specific ai ms of the current st udy are as follows: Aim 1: To extend the findings of the Willcutt, Hartung, Lahey, Loney, and Pelham (1999) study by using a predominantly schoo l-age (as opposed to preschool) sample of children to determine whether examiner ratings of child behavior during psychological testing provide a significant increment in th e prediction of functional impairment beyond parent and teacher ratings of symptoms afte r controlling for the demographic variables that are found to correlate significantly with impairment. It is hypothesized that ratings of behavior in the clinic will be associated with measures of impairm ent after controlling for significant demographic characteristi cs and will provide unique information beyond parent and teacher ratings of ADHD symptoms. Aim 2: To determine if child performance on a Continuous Performance Test (CPT), a laboratory measure of attention/impulsivity, provides a significant increment in 24

PAGE 25

25 the prediction of functional impairment beyond parent and teac her ratings of symptoms after controlling for significant demographic c haracteristics. It is hypothesized that childrens performance on the CPT will be signi ficantly associated with measures of impairment after controlling for significant demographic variables and will provide new information above and beyond parent and teacher ratings of ADHD symptoms. Aim 3: To determine the contribution of t he examiner ratings of child ADHD behavior and child performance on the CPT in predicting functional impairment beyond parent and teacher ratings of symptoms after taking into account child oppositional/defiant characte ristics and significant demographic variables. It is hypothesized that both the clinic-based ex aminer ratings and the performance index from the CPT will provi de unique information as components of a comprehensive clinical evaluation. The relative contributi on of these indices vers us oppositional/defiant symptoms is an aim that is exploratory in nature.

PAGE 26

CHAPTER 2 METHODS Participants Participants were drawn from cases seen for psychological evaluation at the Behavioral Institute of Atlanta (BIA) in Atlanta, GA. BIA is a group practice that provides assessment and treatment services for indi viduals and families displaying a range of problems (e.g. ADHD, Oppositional Defiant Disorder, anxiety and mood disorders). Participants for the study were required to have a primary diagnosis of ADHD, based on whether or not the child met Diagnostic and St atistical Manual-Fourth Edition (DSM-IV; American Psychiatric Association, 1994) criteria for the disorder. Children, between the ages of 6 and 16, with both ADHD Combi ned Type and ADHD Inattentive Type were recruited, as these are the most frequent ly occurring ADHD cases seen in clinical settings. The diagnoses were based on data derived from parent interviews, information from the childs clinical hi story, information from parent and teacher behavior rating scales, behavioral observations and the laboratory m easures described in the current study. In order to appraise whether or not the child met requirements for impairment as specified by the DSM-IV, c linicians based this assessment on their impressions from the interview as well as from measures routinely completed during the course of the evaluation, including but not limited to measures of IQ, academic achievement, and scores from broadband measur es of psychological functioning. Participants were also considered for inclusi on in the study if they met DSM-IV criteria for other disruptive disorders, including Op positional Defiant Disor der (ODD) or Conduct Disorder (CD). It was appropr iate to include children with these types of comorbid conditions, as ODD and CD are commonly found in children with ADHD (American 26

PAGE 27

Academy of Child and Adolescent Psychiatr y, 2007). Therefor e, including these children was thought to add to the representativeness of the sample. Participants were excluded from the study if they met DSM-IV criteria for Mental Retardation [i.e. they obtai ned a Full Scale IQ score of 70 or less on the Wechsler Intelligence Scale for Children-Fourth Edit ion (WISC-IV; Wechsler, 2003)]. Participants were also excluded if they received a formal diagnosis of Learning Disability or an Axis I diagnosis other than the ones listed (ODD, CD) to ensure that the results of this study were specific to a sample of children with ADHD/other disruptiv e disorder. These specific criteria were also used becaus e the excluded conditions can be expected to result in school impairment and potentially confound the relationship between measures of ADHD symptoms and the major dependent variable of the study, impairment. The final sample consisted of the first 27 children with a diagnosis of ADHD who met the specified inclusion and exclusion criteria. Of this sample, 19 of these children met diagnostic criteria for ADHD Combin ed Type, and 8 of these children met diagnostic criteria for ADHD I nattentive Type. A majority of this sample (77.8%) was not on any kind of medication to treat his/her ADHD symptoms. Consistent with ADHD prevalence rates, a majority of the children were males: tw enty (74.1%) individuals were males, and 7 (25.9%) individuals were females. The age of the children ranged from 6 to 16 ( M = 10.30, SD = 2.83), and their reported Full Scale IQ from the WISC-IV ranged from 73 to 130 ( M = 104.52, SD = 13.68). The IQ score of 73 at the low end of the range was thought to be an underest imate of the childs true functioning (due to severe attention deficits during testi ng) rather than a result of true Borderline intellectual abilities. For those about whom ethnicity information was provided, the ethnicity 27

PAGE 28

breakdown of the sample included 19 ( 70.4%) Caucasians, 3 (11.1%) African Americans, and 2 (7.4%) Hispanic/ Latin Americans. There were three children (11.1%) for whom ethnicity information was not availa ble. Tables 2-1 and 2-2 provide detailed demographic information about the child sa mple and the parent respondents. According to the obtained mean scores across administered measures (Table 23), most child participants were characteriz ed as meeting the clinical cutoff score of TScore 65 for parent and teacher ratings of DSM-IV symptoms of inattention and hyperactivity/impulsivity. The mean parent and teacher ratings of DSM-IV ODD were elevated but not in the significant range. In addition, mean clinician ratings on the ADHD-related items of the Hillside Behavior Ratings Scale (HBRS) were strikingly similar to previously published data fo r HBRS scores in an ADHD sample (Willcut, Hartung, Lahey, Loney, & Pelham, 1999). Simi larly, mean parent and teacher ratings on each version of the Impairment Ratings Sc ale (IRS), when rounded up to the nearest whole number, met the optimal cu t-point of 3. This cut-o ff point was found by Fabiano et al. (2006) to exhibit the optimal predict ive power across parent and teacher versions of the IRS such that a child who received a score of 3 or greater consistently met DSMIV criteria for ADHD. The mean score on the auditory/visual computerized measure of attention (IVA CPT) was very close to the beginning of the Average range (M = 89.08; scores between 90 and 110 are considered Ave rage), which suggests that the current sample, as a whole, scored fairly well on th is measure. As one would expect given the inclusionary criteria for this study, the comb ination of mean scores across a majority of these measures was indicative of a sa mple with clinically significant ADHD characteristics. 28

PAGE 29

Procedure Institutional Review Board (I RB) approval was obtained prior to initiating the study. Parents whose children met inclusion criteria for the study were approached about their willingness to participate as well as their willingness to allow their children to participate in the study. Parents were given a detailed description of the study and were asked to sign an informed consent form. Most of the collected data consisted of measures that were routinely completed during the course of the evaluation. The evaluations themselves occurred either in the morning or afternoon over a two day time period, and the ratings scales were provi ded to parents and teachers by t he first day that the child was seen. The routinely collected measures included parent and te acher versions of the third edition of the Conners Rating Scal es (Conners 3; Conners, 2008) and parent and teacher versions of the Im pairment Rating Scale (IRS; F abiano et al., 2006). Of note, for the evaluations that were carried out during the summer or early fall of the school year, the childs previous years teacher was requested to fill out the teacherreport ratings scales. Childr en also completed a computer ized measure of attention, the Integrated Visual and Auditory Perfo rmance Test (IVA CPT; Sandford & Turner, 1995). Informed consent requested permission for the clinician to make ratings of each participants attention and impulsivity during test ing, as well as to videotape a portion of testing. Ratings were recorded using t he Hillside Behavior Rating Scale (HBRS; Gittelman & Klein, 1985). The clinician co mpleted this measure shortly after the administration of the IQ measure, the Wechsler Intelligenc e Scale for Children-Fourth Edition (WISC-IV, Wechsler, 2003). The WISC-IV was always t he first administered test within the test battery. The particular task and setting were chosen as it was assumed that the administration of the IQ measure allo wed the rater enough time to be able to 29

PAGE 30

observe the behaviors of interest. In addition, the tasks within the intelligence measure include mildly stressful problem-solving tasks, which may approximate academic endeavors experienced by the child on a regular basis. Inter-rater reliability of the clinician ratings during testing was established on approximately 30% of the total sample. T aped recordings of children completing the WISC-IV were provided to a laboratory rese arch assistant. This assistant had received prior training in making observations and a ccurately filling out the Hillside Behavior Rating Scale. An inter-rater reliability ana lysis using the Kappa statistic was performed to determine consistency between the clinician and research assistants ratings. The inter-rater reliability was found to be significant (Kappa = 0.73, p < 0.001), which is considered an acceptable level of agreement (Landis & Koch, 1977). Measures Conners 3rd Edition (Conners 3; Conners, 2008): The long version of both parent and teacher forms were used. The parent and teacher forms (possessing 108 and 113 items, respectively) assess primarily external izing behaviors, including those related to ADHD and oppositional/defiant symptomatology. The directions requests parents (or teachers, given the particular form) to c onsider the childs behavior during the past month, and responses are given on a Likert sca le, ranging from Not at all true (Never, Seldom), Just a little true (O ccasionally), Pretty Much Tr ue (Often, Quite a bit) to Very much true (Very often, Very frequently). Strengths of the Conners rating scales include their large normative base, evidenc e of strong psychometric properties, and its supported factor structure (Conners, 2008). T-scores from the parent and teacher DSM-IV Hyperactive-Impulsive and the DSM-IV Inattentive subscales were of primary interest, as elevated scores on these sub scales reflect above average correspondence 30

PAGE 31

with DSM-IV diagnostic criteria for major su btypes of ADHD. Composite ODD scores, computed by summing parent and teacher DSM-IV Oppositio nal Defiant subscale scores, were also considered in follow-up analyses. Impairment Rating Scale (IRS, Fabiano et al., 2006): The authors of the scale note that the IRS is a rationally derived instrument whose items reflect areas of functioning that are considered to be impaired in children with ADHD. The parent version covers seven domains of functi oning, including relationship with peers, relationship with siblings, relationship with parents, academic progress, self-esteem, influence on family functioning, and overall impairment. The teacher version possesses six domains of functioning, which are relationship with peers, relationship with teacher, academic progress, self-esteem, influence on classroom functioning, and overall impairment. Raters are instructed to determi ne the severity of the childs impairment/ need for treatment based on a continuum by placing an X on a line. For scoring purposes, the line is divided into seven equal segments. The position of the X represents a score between 0 (n o problem/definitely does not need treatment or special services) and 6 (extreme problem/definitely needs treatment or special services). Average scores, used in the current study, were calculated by completing the appropriate permutation across domain scores. Both parent and teacher versions of the IRS have been shown to exhibit good temporal st ability, strong correl ations with other impairment ratings and behavior measur es, and evidence of convergent and discriminant validity. Furt her, the IRS has been shown to contribute unique variance above and beyond ratings of ADHD symptoms w hen predicting future impairment as assessed by global impairment measures (Fabiano et al., 2006). 31

PAGE 32

Integrated Visual and Auditory Performanc e Test (IVA, Sandford & Turner, 1995) is a measure of visual and auditory attent ion. This Continuous Performance Task paradigm requires the participant to click a computer mouse button whenever he or she sees or hears the number and to refrai n from pressing the mouse when he or she sees or hears a As par t of the administration protocol, each participant receives a standard set of instructions and completes a practi ce test prior to the real test. The IVA CPT presents auditory/visual stimuli and common/rare targets in a counterbalanced fashion that reportedly controls for fati gue and learning (Sandford & Turner, 1995). The IVA CPTs scoring yields visual and att ention quotients that have a mean of 100 and a standard deviation of 15. The Combined Attent ion Quotient, a measure that provides a general indicator of attention, was used in the current study. The IVA CPT has been found to be useful in differentiating ch ildren with ADHD from normative samples (Sandford & Turner). Hillside Behavior Rating Scale (HBRS, Gittelman & Klein, 1985): This scale contains seven items on which observers rate a childs behavior during testing. Within each item, the rater is asked to choose a descriptor (the number of descriptor choices ranges from five to seven) that best corresponds with the behavior that the child displays during testing. The three HBRS item s that directly relate to symptoms of DSMIV ADHD include motor activity, distractibility, and impulse control. The four areas that assess disruptive behavior in more general terms include frustration tolerance, cooperation, interest in tasks, and attenti on seeking. A total composite score is obtained by summing all seven HBRS items. The HBRS has been shown to possess satisfactory psychometric properties, with reliability values r anging from .68-.76, 32

PAGE 33

evidence of an ability to discriminate children with and without ADHD, and sensitivity to the effects of stimulant m edication (Willcutt, Hartung, Lahey, Loney, & Pelham, 1999). The above study utilized a composite ADHD score, computed by summing the three HBRS items that directly correspond to DS M-IV ADHD items, in t heir analyses of the HBRS. In an investigation of whether t he utilization of a subset of HRBS items influenced the results, the author s found that the results were virtually identical in the two scenarios in which either the ADHD com posite or HBRS total scores were used. This was likely due to the high correlation (r = .96) between the ADHD composite and total score. In light of this finding and to provide for a greater degree of variation, the total score was used in analyses addressing all current primary aims. As noted earlier, in order to provide reliability data, a research assistant was tr ained by the investigator to complete the HBRS based on child behavior portrayed in specific video segments obtained during testing. The video segments were selected from real-life clinic examples, illustrating varying degrees of i nattentive, hyperactive, and disruptive behaviors. As there is presently no gold st andard by which to judge the reliability of HBRS ratings, trainees ratings were co mpared to those provided by the study investigator. Sufficient agreement (exact correspondence on at leas t 5 of the 7 items) was required on at least two measures before in vivo ratings were made. As previously indicated, the research assistant subsequent ly completed HBRS ratings on a larger subset of 8 children. Comparing these rati ngs to clinicians ratings provided further support for the inter-rater reliability of the HBRS (Kappa agreement = .730). Demographic Form : The parent-completed demogr aphic form consisted of 15 questions asking about the child and family. Child-focused demographic information 33

PAGE 34

included age, gender, grade leve l, classroom placement, ethnicity, psychological diagnoses, medical illness diagnoses, tr eatment services received (including psychological), and current medication regimen. Family-focused demographic information included age, gender, ethnicity, mari tal status, occupation, educational level and nature of the caregivers relationship to the child. Statistical Analyses The current study used impai rment as the criterion va riable to evaluate the incremental validity of measures of child behavior (clin ician ratings of child behavior during testing and child performance on the CPT) above and beyond parent and teacher ratings of ADHD. It was expected th at the clinic-based measures would provide at least some new information to that prov ided by ADHD symptom ratings scales. The study also considered the relationshi p between oppositional/ defiant symptoms and impairment, as ODD symptoms make up th e most commonly occurring comorbid condition with ADHD. Prior to addressing the s pecific aims of incremental validity, initial correlational analyses were conducted to as sess the relationship between hypothesized predictor variables (e.g. parent and teacher ratings of ADHD and ODD symptoms, HBRS ratings, and CPT scores) and parent and teacher ratings of impairment. Subsequently, hierarchical regression models were used to evaluate the incremental validity of the clinic-based meas ures of child behavior. Analysis 1: First, consideration was given to whether age, gender, IQ, and/or medication status were independe ntly related to child impairment, as indexed by ratings from the parent and teacher forms of the IRS. Those demographic and child characteristics found to be significant were included in the first step of a hierarchical multiple regression analysis to ensure that findings reflected a specific relationship 34

PAGE 35

between HBRS ratings and scores of functional impairment. Of note, the impairment scores from parent and teacher versions of the IRS were considered in separate regression analyses as distinct dependent variables for all analyses. This was done in order to consider parents and teachers as uni que informants, particularly given the very low level of correspondence between parent and teacher ratings of impairment ( r = .07) in this sample. Measures of ADHD sympt omatology, indexed by DSM-IV Inattentive and DSM-IV Hyperactive-Impulsive scores from parent and teac her forms of the Conners 3, were each entered as separate indices in the second step of the regression analyses to determine the association betwe en these measures and impairment. Finally, HBRS total composite scores were entered into the third step to determine if these behavioral ratings during testing added si gnificantly to the explained variance in IRS impairment scores. Analysis 2: A second set of hierarchic al multiple regression analyses were conducted to determine whether child per formance on the IVA CP T added incrementally in predicting functional impai rment. Again, the demographi c and child characteristics that were significantly associ ated with impairment we re entered into the first step of the equation to control for these variables. Pa rent and teacher reports of ADHD symptoms were entered in the second step of the regressions. Lastly, the Combined Attention Quotient of the IVA CPT was entered into t he model to establish whether this index provided a significant increase in the explained variance of the IRS impairment scores. Analysis 3: To explore the unique contribution of Oppositi onal-Defiant symptoms in predicting impairment, a fi nal set of regression analyse s was performed. The first step of the equations included relevant demographic variables and the ODD composite. 35

PAGE 36

The second step included parent and teacher reports of ADHD sym ptoms. The IVA CPT Quotient and HBRS scores were entered in a final step only if these were found to be significant predictors of im pairment in earlier analyses. 36

PAGE 37

Table 2-1. Demographic characte ristics of child participants N (%) M SD Child Age ADHD (Total) 27 10.30 2.83 ADHD(Subtypes) Combined 19 9.37 2.54 Inat tentive 8 12.50 2.27 Child Gender Boys 20 Girls 7 Child IQ 104.52 13.68 Child Ethnicity Caucasian (70.4%) Afri can Am. (11.1%) Hispanic (7.4%) Unidentified (11.1%) Special Services No 20 Yes 6 Unidentified 1 Medication Status Yes 5 No 21 Unidentified 1 Table 2-2. Demographic charac teristics of parent respondents N (%) Parent gender Male 4 Female 23 Parent marital status Married (81.5%) Divorced (3.7%) Remarried (14.8%) Parent education Some college/AA (7.4%) College (48.1%) Masters or above (33.3%) Unidentified (11.1%) 37

PAGE 38

38 Table 2-3. Descriptive statisti cs for the administered measures M SD Parent IRS Average 2.52 1.29 Teacher IRS Average 2.76 1.45 Conners Parent Ratings Inattention 71.11 11.28 Hyperactivity/Impulsivity 65.59 14.77 Oppositional/Defiant 57.48 14.18 Conners Teacher Ratings Inattention 68.33 12.32 Hyperactivity/Impulsivity 66.30 17.23 Oppositional /Defiant 57.89 15.95 HBRS ADHD Ratings 6.44 2.69 IVA CPT Composite 89.08 21.34 Note: Higher scores on the IRS, Conners, and HBRS suggest more severe/clinical problems. Lower quotient scores on the IVA CP T suggest greater attention difficulties.

PAGE 39

CHAPTER 3 RESULTS Preliminary Analyses The statistical package SPSS for Windows (SPSS Inc., 2008) was used for all statistical analyses. The data was first anal yzed to determine the presence of outliers as well as possible deviations from norma lity. The standardized residuals for each variable of interest were computed in order to detect outliers. As none of the standardized residuals were more than three dev iations from the mean, outliers did not emerge as a concern (Weinber g & Abramowitz, 2008). All cases were kept for consideration in subsequent analyses. In addition, the di stributions for each of the variables met the regression assumption for normality (e.g. the skewness and kurtosis values were all within an acceptable range). Table 3-1 shows a correlation matrix, which depicts the associations among variables of interest, including relevant demographic variables, symptom ratings, clinician ratings of child behavior duri ng testing, and child performance on a computerized CPT. With respect to demogr aphic variables, neither age nor Full Scale IQ was found be significantly related to par ent ratings of impai rment. In addition, separate t-tests were run to examine the re lationship between the categorical variables of gender and medication status to ratings of impairment. Neither of these factors related significantly to parent-rated or teac her-rated impairment. It can be noted that age did emerge as a significant predictor of teacher-rated impairment ( r = -.57, p <.01). The direction of the relationship was such that teachers tended to rate younger children as more impaired than older children. Age was thus entered into subsequent regression analyses (as a demographic variab le to be controlled for) when teacher 39

PAGE 40

impairment was the dependent variable. Given t heir lack of significant associations with impairment, no other demographic variables were included in subsequent analyses. Other notable correlations with parent ratings of impairment included parent ratings of inattention (r = .72, p < .01), parent ratings of hyperactivity ( r = .50, p < .01), and ratings of oppositional/defiant behavior ( r = .65, p <.01). More severe inattentive, hyperactive, and oppositional symptomatology was associated with higher parental reports of impairment. Clinician ratings of ADHD behavior during testing ( r = .20) and child performance on the CPT ( r = .36) were mildly to m oderately associated with parent ratings of impairment. It should be noted, however, that instead of emerging as a negative association as one would expect, t hose children with higher levels of impairment within the home actually scored better on the computerized test of attention. Teacher ratings of inattention ( r = .45, p < .05), teacher ratings of hyperactivity ( r = .63, p < .01), and ratings of oppos itional/defiant behavior (r = .53, p < .01) were significantly associated with teacher ratings of impairment. The direction of the relationship was such that increased levels of inattentive, hyperactive, and oppositional symptomatology were associat ed with greater teacher reports of impairment. Childrens ADHD behavior during testing ( r = .26) and perform ance on the CPT ( r = -.25) were only mildly to moderately correlated with teacher ratings of impairment, in the expected directions. Despite the non-significant mild to moderate le vel of the obtained correlations, HBRS and CPT ratings were retained in subsequent hierarchical regression analyses (described below), give n the theoretical importance of these variables to the study. 40

PAGE 41

The bivariate correlations also revealed that the clinician ratings of ADHD behavior during testing were significantly associated with age ( r = -.57, p < .05), parent ratings of hyperactivity/impulsivity ( r = .51, p < .05), as well as with oppositional/defiant symptoms ( r = .50, p <.01). CPT scores were signi ficantly associated with age ( r = .49, p <.05) and moderately associated with teacher ratings of hyperactivity/impulsivity at a level that can be viewed as approaching significance in light of the moderate correlation ( r = -.35, p = .08). Primary Aims Aim 1: To Determine the Incremental Vali dity of Ratings of ADHD Symptoms and Child Behavior During Testing in Predicting Functional Impairment For the subsequent hierarchical regr ession analyses, relevant demographic variables, as described above, were entered in to the first block of the equation. The predictor variables, either parent or teacher sco res from the DSM-IV Inattention and Hyperactivity/Impulsivity subscales, were then entered into the next block. Finally, clinician ratings during testing (HBRS scores) were entered into the final block of the analyses. Predicting parent-rated functional impairment None of the demographic variables were found to be significantly related to parentrated functional impairment. Therefore, only two blocks were used in this model. The overall regression model predi cting parent-rated functional impairment was significant ( R2 = .68, F [3,23] = 16.31, p < .001). As can be seen in Table 3-2, the first block showed significant direct effects fo r parent ratings of inattention ( = .66, t = 5.54, p < .001) and parent ratings of hyperactivity/impulsivity ( = .39, t = 3.30, p < .01). Block two of the equation showed a non-significant direct effect for HBRS ratings ( = -.12, t = 41

PAGE 42

-.86, p = .40). In summary, parent ratings of inattention most strongly predicted average parent ratings of functional impairment. Par ent ratings of hyperactivity and impulsivity predicted parent-rated impairment to a m oderate degree. Clinician ratings during testing were a non-significant contributor of unique variance to the regression model. Predicting teacher-rated functional impairment The overall regression model predicting teacher-rated functi onal impairment was significant ( R2 = .52, F [4,22] = 6.03, p < .01). Table 3-3 show s a significant negative effect for age ( = -.56, t = -3.41, p < .01) in the first block of the equation. Block two of the equation showed a non-significant effect for teacher ratings of inattention ( = .19, t = 1.20, p = .24) and a significant effe ct for teacher ratings of hyperactivity/impulsivity ( = .37, t = 2.17, p < .05). The final block show ed that HBRS ratings did not significantly add to the regression equation ( = -.03, t = -.15, p = .88) in predicting teacher-rated functional impairment. Overall, when controlling for age, teacher ratings of hyperactivity and impulsivity emerged as the strongest predictor of teacher-reported impairment. Aim 2: To Determine the Incremental Vali dity of Ratings of ADHD Symptoms and Child Performance on the Continuous Performance Test in Predicting Functional Impairment For the subsequent hierarchical regr ession analyses, relevant demographic variables were entered into the first block of the equation. The predictor variables, either parent or teacher scores from the DSM-IV Inattention and Hyperactivity/Impulsivity subscales, were ent ered into the second block. The total score from the IVA CPT, representi ng a composite index of visual and auditory attention, was then entered into the final block of the analyses. 42

PAGE 43

Predicting parent-rated functional impairment As previously noted, none of the demographic variables were significantly related to parent-rated functional impai rment; therefore, two blocks were used in this model. The overall regression model predicti ng parent-rated functional impairment was significant ( R2 = .68, F [3,23] = 16.29, p < .001). As shown in t able 3-2, the first block showed a significant direct effect for parent ratings of inattention ( = .63, t = 5.00, p < .001) and parent ratings of hyperactivity/impulsivity ( = .40, t = 3.23, p < .01). Block two of the equation showed a nonsignificant direct effect fo r the IVA CPT total score ( = .11, t = .85, p = .40). Again, parent ratings of inattention and hyperactivity/impulsivity were the best predict ors of parent-rated functional impairment. Childrens performance on the CPT was a non-si gnificant contributor of unique variance to the regression model. Predicting teacher-rated functional impairment The overall regression model predicting teacher rated functional impairment was significant ( R2 = .53, F [4,22] = 6.28, p < .01). Table 3-3 show s a significant negative effect for age ( = -.56, t = -3.41, p < .01) in the first block of the equation. Block two of the equation showed a non-significant effect for teacher ratings of inattention ( = .19, t = 1.20, p = .24) and a significant effe ct for teacher ratings of hyperactivity/impulsivity ( = .37, t = 2.17, p < .05). The final block showed that the IVA total score did not significantly add to the regression equation ( = .12, t = .70, p = .49) in predicting teacher-rated functional impairment. Similar to the other teacher m odel, teacher ratings of hyperactivity and impulsivity emerged as the strongest predictor of impairment, while the measure of attention from the CPT was non-significant. 43

PAGE 44

Follow-up Aim: To Determine the Predic tive Value of Oppositional Defiant Disorder Symptoms Relative to Othe r Variables in Predicting Functional Impairment In light of the high correspondence bet ween ODD symptoms and impairment ( r = .65 for parent-rated impairment and r = .53 for teacher-r ated impairment), ODD symptoms were entered into the first blo ck of the equation to determine whether the presence of this variable would suppress the effects of other prev iously significant variables. Parent or teacher sco res from the DSM-IV Inattention and Hyperactivity/Impulsivity subscales were ent ered into the second bl ock of the equations. Given the lack of significant contributions of the HBRS ratings and the IVA CPT scores, these variables were not included in testing subsequent regression models. Predicting parent-rated functional impairment The overall model predicting parent-ra ted impairment wa s significant ( R2 = .75, F [3,23] = 23.35, p < .001). As shown in Table 3-4, block one showed a significant direct effect for ODD symptomatology ( = .65, t = 4.31, p < .001). Block two showed a significant direct effect for parent ratings of inattention ( = .58, t = 5.39, p < .001). Unlike the previous parent models, the effect for parent ratings of hyperactivity/impulsivity was no longer significant ( = .17, t = 1.30, p = .21). The model suggests that ODD symptomatology and inatt ention were the strongest predictors of parent-rated functional impairment, suppressing the effects of parent reports of hyperactivity/impulsivity. Predicting teacher-rated f unctional impairment The overall model predicting teache r-rated impairment wa s significant (R2 = .57, F [4,22] = 7.25, p < .01). As shown in Table 3-5, block one showed an almost significant direct effect for ODD symptomatology ( = .34, t = 1.92, p = .07) as well as a 44

PAGE 45

significant negative effect for age ( = -.42, t = -2.40, p < .05) Block two showed a nonsignificant direct effect for t eacher ratings of inattention ( = .16, t = .99, p = 33) and a significant effect for teacher repor ts of hyperactivity/impulsivity ( = .35, t = 2.09, p < .05). In summary, childrens younger ages, the presence of ODD symptoms, and higher teacher ratings of hyperactivity/impulsivi ty contributed the most to teacher-rated functional impairment. 45

PAGE 46

Table 3-1. Correlations among variables of interest 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 1. Age --.17 .20 -.33 -.16 -.37 -.44* -.57* .49* .09 -.57** 2. IQ --.04 .01 -.02 .17 -.02 -.25 .27 -.12 .00 Parent Ratings 3. Inattention -.16 .13 -.15 .29 .14 .32 .72** -.16 4. H/I --.02 .38* .62** .51* .11 .50** .26 Teacher Ratings 5. Inattention -.44* .25 -.07 -.06 .25 .45* 6. H/I -.30 .17 -.35 .01 .63** 7. ODD composite -.50** .09 .65** .53** 8. HBRS --.27 .20 .26 9. IVA CPT -.36 -.25 10. Parent IRS -.07 11. Teacher IRS -* p < .05, ** p <.01 46

PAGE 47

Table 3-2. Hierarchical regression analyses predicting parent -rated functional impairment DV: Parent IRS B t R 2 Total R 2 Model 1 Block 1 .67 Inattention .66 .08 5.54** Hyperactivity/Im pulsivity .39 .03 3.30** Block 2 .01 .68 HBRS Ratings -.12 -.03 -.86 Model 2 Block 1 .67 Inattention .63 .07 5.00** Hyperactivity/Im pulsivity .40 .03 3.23** Block 2 .01 .68 IVA CPT Composite .11 .01 .85 **p <.01 Table 3-3. Hierarchical regression analyses predicting teacher-rated functional impairment DV: Teacher IRS B t R 2 Total R 2 Model 1 Block 1 .32 Age -.56 -.28 -3.41** Block 2 .20 .52 Inattention .19 .02 1.20 Hyperactivity/Im pulsivity .37 .03 2.17* Block 3 .00 .52 HBRS Ratings -.03 -.01 -.15 Model 2 Block 1 .32 Age -.56 -.28 -3.41** Block 2 .20 .52 Inattention .19 .02 1.20 Hyperactivity/Im pulsivity .37 .03 2.17* Block 3 .01 .53 IVA CPT Composite .12 .01 .70 p < .05, ** p <.01 47

PAGE 48

48 Table 3-4. Follow-up hi erarchical regression analysis predict ing parent-rated functional impairment DV: Parent IRS B t R 2 Total R 2 Block 1 .43 ODD Compos ite .65 .04 4.31** Block 2 .32 .75 Inattention .58 .07 5.39** Hyperactivity /Impulsivity .17 .02 1.30 **p <.01 Table 3-5. Follow-up hierar chical regression analysis predi cting teacher-rated functional impairment DV: Teacher IRS B t R 2 Total R 2 Block 1 .41 Age -.42 -.21 -2.40* ODD Composite .34 .02 1.92 Block 2 .16 .57 Inattention .16 .02 .99 Hyperactivity/Im pulsivity .35 .03 2.09* p <.05

PAGE 49

CHAPTER 4 DISCUSSION Determining empirically supported assessment procedures for diagnosing ADHD in children is an important goal for applied re search. Current guidelines highlight the necessity of including multiple informants (parents and teachers) and settings (home and school) in the assessment process. Multiple methods (e.g. paper and pencil questionnaires, computerized measures of attention) are also commonly used by psychologists as they conduct ADHD evalua tions. While the different approaches tend to vary in cost and the amount of time taken for administration, there is little research concerning the utility of adding additional methods to an existing assessment strategy. As the variety and scope of assessment procedur es expand, this concept of incremental validity has been cited as increasingly impor tant to explore (Hunsley & Meyer, 2003; Johnston & Murray, 2003). Yet, as H unsley and Meyer note, there has been little systematic effort to explore the incremental validity of psychological assessment procedures. The few studies that have looked at the in cremental validity of multiple methods in ADHD assessment have yielded conflicting re sults. For example, Lobitz and Johnson (1975) found that neither c linic based nor home observa tions of child behavior added significantly to the optimal classification rates offered by parent reports. Gathje, Lewandowski and Gordon (2008) discovered that child commission errors on a computerized measure of attent ion explained a very small am ount of unique variance in a Global Impairment Index, composed of impairment scores across a variety of measures. In contrast, other studies have supported the utility of clinic-based methods of assessing ADHD behavior during testing as offering useful information about the 49

PAGE 50

childs ADHD features (Glu tting, Robins, & de Lancey, 1997) One such study found that ratings of inattentive, hyperactive and disruptive behaviors made by examiners during psychological testing of preschool child ren did provide a significant increment in the prediction of impairment above and beyo nd parent and teacher behavior ratings (Willcut, Hartung, Lahey, Loney, & Pelham, 1999). The mixed findings regarding the incremental validity and unique ness of clinic-based measures make the current investigation relevant, especially in light of the time and financial resources required to employ these types of instruments. The current study sought to evaluate t he incremental validity of multiple assessment methods, which included symptom-based parent and teacher ADHD ratings, behavioral observations of the child during testing, and child performance on a combined auditory/visual computerized measure of attention. While it would be of interest to determine whether the above measur es add incremental va lue in predicting a diagnosis of ADHD, it is difficult to arrive at a valid diagnosis without considering these very symptom-based, observational, and perf ormance-based instrum ents. Impairment was selected as the criterion variable not only to avoid this confound but also because it is functional impairment that often brings chil dren to the attention of service providers. Current research also suggests that impai rment is highly predictive of outcome and warrants an important focus in ADHD asse ssment. A growing body of literature highlights the necessity of including fo rmal assessments of impairment in ADHD evaluations (Fabiano et al., 2006; Pelham, Fabiano, & Massetti, 2005). The use of the newly developed Impairment Ratings Scale (Fabiano et al.), which has shown good preliminary reliability and valid ity as a separate standardized measure of impairment, is 50

PAGE 51

an improvement over previous studies, which have looked at this construct using a variety of post-hoc indices that are not AD HD specific. The current study, therefore, using impairment as the criterion variable, was designed to evaluate the incremental validity of measures of child behavior (clinician ratings of child behavior during testing and child performance on the CPT) above and beyond parent and teacher ratings of ADHD. It was expected that the clinic-based measures would provide some new information above and beyond the ADHD symptom rating scales. As an additional exploratory aim, the study also considered the impact of oppositional and defiant symptoms, which are commonly obs erved in children with ADHD The study included a sample of 27 child ren who met diagnostic criteria for ADHD and who did not meet crit eria for a learning, mood, or other Axis I disorder apart from a disruptive behavior disorder. Mean scores across parent and teacher rating scales suggested that most children in the sample me t the clinical cut-off criteria for DSM-IV Inattention and DSM-IV Hyperactivity/Impulsivi ty. Mean DSM-IV Oppositional Defiant scores were elevated, but not in the clinic ally significant range. Further, mean scores from the parent and teacher Impairment Ratings Scale and from the Hillside Behavior Rating Scale suggested clinical levels of d ysfunction. It should be noted that the inclusion of an ADHD sample, as opposed to a combined (ADHD plus a nonclinical) group, was appropriate for addressing the current studys aims, as it is within this type of clinical sample that measures such as the CPT are most commonly used. Enough variability was present in the scores such that the data met requi red assumptions for normality. 51

PAGE 52

Preliminary Analyses First, bivariate correlations were run to determine the relationship among variables of interest, including relev ant demographic variables, symptom ratings, clinician ratings of child behavior during testing, and child pe rformance on a computer ized CPT. Parent ratings of ADHD and ODD symptoms were most strongly related to parent-rated impairment. Age, teacher ratings of A DHD, and ODD symptoms were significantly correlated with teacher-rated impairment. Clinic-based measures of ADHD-related behavior, which included clinician ratings of behavior during testing and child performance on the CPT, were only mild ly to moderately associated with both impairment indices. The bivariate correlations also revealed that the clinician ratings were significantly associated with age and parent ratings of hyperacti vity/impulsivity as well as with oppositional/defiant symptoms. CPT scores were significantly associated with age and moderately associated with teacher ratings of hyperactivity/impulsivity at a level that approached significance. These significant associations indicate some degree of initial overlap across methods. The recurring appearance of age as being significantly associated with a number of vari ables suggests that younger children show the most overt symptoms of i nattention and hyperactivity/impulsivity, while older children may be more subtly affected by their ADHD at least in the one-on-one clinic setting. Parent/Teacher Ratings of ADHD Symptoms and Impairment In order to consider specific aims of the study pertainin g to the incremental validity of ADHD assessment procedures, hierarchical regressions we re conducted. Results of regression analyses showed that parent reports of inattentive and hyperactive/impulsive behaviors accounted for a significant percent of the variance in parent-rated impairment. Parent ratings of inattention were relatively more predictive of impairment than parent 52

PAGE 53

ratings of hyperactivity and impulsivity although both indices of ADHD symptoms demonstrated large effect sizes. With teac her ratings of impairment as the criterion variable, age and teacher ratings of hyperacti vity and impulsivity ex plained a significant portion of the variance. Teacher ratings of i nattention did not contri bute to the model at a level that was statistically significant. Taken together, these findings are consistent with studies that have f ound ADHD symptomatology and im pairment to be moderately associated, with most bivariate correlati ons ranging form .20 to .50 (Gathje, Lewandowski, & Gordon, 2008; Fabiano et al. 2006). Findings also supported the assumption that different aspects of t he childs presentation and behavior are most salient depending on whether the rater is a parent or classroom teacher. Teachers likely notice age-specific deviations because they have a broader ex perience base from which to judge behaviors (Murray et al., 2007) Also, teachers ma y be more apt to notice and report hyperactive-impulsive behaviors, which are more concrete, readily observable, and may disrupt classroom activities, while parents have a range of settings from which to make ratings on both hyperac tive and inattentive behaviors. The degree to which teachers observe inattentive behavior may be influenced by the degree to which specific structured aspects of classroom activity are enforced. That the two main factors of ADHD were differentially related to impairment is not surprising, since the individual symptoms that comprise the factors have varying positive and negative predictive powers that are rater specific (Power, Costigan, Leff, Eiraldi, & Landau, 2001). One implication for ADHD asse ssment, particularly with the impending development of the DSM-V, is how the clinic ian should weight and combine specific ADHD symptom ratings from parents and teachers. 53

PAGE 54

Parent/Teacher Ratings of ODD Symptoms and Impairment Ratings of oppositional/defi ant symptomatology were hi ghly predictive of parent and teacher-rated impairment ( = .65 for the parent model, and = .34 for the teacher model). In most cases, t he unique predictive value of O DD symptoms exceeded that of ratings of hyperactivity and inattention. Conceptually, it makes sense that these symptoms would be highly asso ciated with the problem/ need for treatment response anchor within the Impairment Rating Scale. ODD symptoms such as temper tantrums, argumentativeness with adults, and rule-break ing, are likely the most readily recognizable by raters, closely followed by hyperactive/impulsive symptoms. The high level of correspondence between ratings of ODD symptoms and symptoms of hyperactivity/impulsivity bot h in the current sample ( r = .62) and in a much large sample ( r = .67) from which the Conners-3 was no rmed (Conners, 2008) help explain why these two dimensions of child behavior showed si milar patterns of significance in the regression models. In fact, in the parent model, ODD sympt oms and parent-rated symptoms of hyperactivity and impulsivity were so highly inte rrelated that the effect of the latter became insignificant when the ODD variable was entered first into the model. One could conclude that it is these mo re overt oppositional and hyperactive behaviors that tend to bring families into a psychologis ts office for a compr ehensive evaluation. Other studies also attest to the idea that deviant and impulsive behaviors are highly associated with a perceived need for treatment services (Rowe, Maughan, Costello, & Angold, 2005). Angold and Coste llo (1996) found that children with as few as two or three ODD symptoms involvi ng psychosocial impairment exhibited high rates of psychiatric service utilization and psychiatric morbidity. Thus, children with both ADHD and ODD symptoms are most likely to be identified as having significant difficulties 54

PAGE 55

functioning at home, with peers, and in the classroom. It is arguable that these children require the most immediate, comprehensive, and long-lasting treatment plans. Clinicians Ratings During Testing Contrary to expectations, clinician ratings on the HBRS during testing did not provide a significant incr ement in predicting impairment above and beyond parent and teacher ratings of ADHD symptoms. Indeed, preliminary correlational analyses did not provide support for an initial strong relations hip between these clinician ratings and parent/teacher ratings of impairment. The n egative findings are consistent with the existing literature that has questioned the external validity of clinic-based based observations (Lobitz & Johnson, 1975; Sleator & Ullmann, 1981; Edwards et al., 2005). While there is intuitive appeal for the idea that test behaviors reflect enduring characteristics of child behavior, this a ssumption has been called into question. One study, which conducted a meta-analysis investigating the ecological validity of test observations, reported very conservative or negligible predictive values (average r = .18) (Glutting, Youngstrom, Oakland, & Watkins, 1986). With regards to ADHD, findings pertaining to the general diagnostic utility of clinic-based observations are mixed. A 2005 study (Edwards et al.) found that clinician ratings of child behavior during standardized IQ and achievement testing were related to parent but not to teacher ratings of ADHD behaviors. The relations hip between clinician and parent ratings was much stronger for hyperactive and impulsi ve behaviors than it was for inattentive behaviors. Interestingly, this was borne out in the current study, in which the HBRS ratings were significantly associated with par ent reports of hyperactivity/impulsivity ( r = .51, p < .01) but not parent ratings of inattention. Also similar to the above 2005 study, no significant associations were found between clinician ratings during testing and 55

PAGE 56

teacher ratings of ADHD behaviors. The lack of correspondence between clinic-based observations and teacher ratings provides an important cautionary point, particularly because some surveys have found that physici ans often rely on their own observations of child behavior during the office visi t to make diagnostic decisions about ADHD (Copeland et al., 1987). Clinic-based ratings are certainly not a substitute for gathering cross-situational information about a child s behavior from both parents and teachers. One relevant question regarding clinician ratings of ADHD behavior during testing is the following: if these ratings are not signifi cantly related to overall parent or teacherrated impairment and are limited in their generalizability, what is it that they predict? The study by Willcutt, Hartung, Lahey, Loney, and Pelham (1999) used a variety of very specific indices of impairment (from a vari ety of measures) in their regression models and found that HBRS scores were predictive of teacher and peer reports of social discord, above and beyond ratings of ADHD sym ptomatology. While the design of the current study was to consider average impai rment as opposed to specific indices, the researcher did go back and enter each item of the parent and teacher impairment scale to consider the relationship of clinician ratings to individual domains of child functioning. Most HBRS incremental validity findings re mained negligible; however, HBRS ratings were found to provide a more substantial increment in the degree of variance explained when predicting teacher ratings of specific aspec ts of social impairment (item: How the childs problem affect his or her relationship with other children) [ semipartial r = .12, with semipartial correlations close to .15 at th is point in a hierarchical regression being considered clinically meaningful (Hunsley & Me yer, 2003)]. This result, combined with the 1999 study findings, tentatively suggests that when a child shows overt ADHD 56

PAGE 57

symptomatology in the office setting, he or she is likely to demonstrate social impairment among their peers and teachers as well. Another line of research has focused on the intrasession validity of test observations; that is, the strength of associ ation between test observations of behavior and other measures of formal test scores that are given around the same time. A summary of research in this area suggests that observations of child behavior during testing possess moderate and meaningful levels of intrasession validity (Glutting, Youngstrom, Oakland, & Watkins, 1996). Research has also suggested that standardized ratings of childrens inattentiv e and avoidant behaviors during testing can affect the magnitude of thei r IQ scores (Glutting, Robins & de Lancey, 1997). In the current sample, an exploration of correspondence of behavior ratings with other concurrent measures revealed moderate re lationships. For example, HBRS ratings correlated moderately with childrens working memory index scores from the IQ test ( r = -.40) and modestly with childrens performance on the IVA CPT ( r = -.27). Due to sample size constraints, the above correlations did not quite reach significance. Still, these associations were somewhat higher t han the cross-situationa l associations with average parent and teacher-rated impairment, teacher ratings of inattention and hyperactivity/impulsivity, and parent ratings of inattention. The primary value, then, of behavior observations during testing seems to be the clinicians ability to gauge behavioral impact upon other measures completed by the child during the evaluation. Standardization and the provisio n of norms for behavior rati ngs scales such as the HBRS are worth pursuing. Further development of such measures would help clinicians 57

PAGE 58

quantify their argument for why a childs be havior may have negatively impacted his or her testing scores. Childrens Performance on the Continuous Performance Test Also contrary to expectations, childrens performance on the IVA CPT did not provide a significant incr ement in predicting impai rment above and beyond parent and teacher ratings of ADHD symptoms. Init ial correlational analyses did not provide support for a significant link between CP T scores and parent or teacher-report measures of impairment. CPT scores were however, significantly associated with age and moderately associated with teacher ratings of hyperactivity/impulsivity at a level that approached significance. The appeal of continuous performance tests is their grounding in childrens actual behavior, whic h can be directly measured and quantified. Parameters from continuous performance te sts have been found to be correlated with ADHD symptoms generally (i.e. across sympt om domains), and most all CPT manuals attest to the ability of t he test to differentiate betw een ADHD and normal control groups (Epstein, Erkanli, Conners, Klaric, Coste llo, et al., 2003). Info rmation regarding the sensitivity, specificity, and in cremental validity of CPT measures is sparse, however. The data that is available suggest that none can be used with confidence for diagnostic decision-making, either alone or in conj unction with other assessment procedures (Rapport, Chung, Shore, Denney, & Isaacs, 2000). The authors of the latter study point out that when considering results from a laboratory measure of at tention, one does not know if deviant scores are due to inattention or from other cognitive deficits such as visual-perceptual difficulties and/or working memory problems. Similar to clinicians judgments of childrens ADHD behavior during testing, t he ecological validity of CPT measures has been called into question. St udies suggest little overlap between CPT 58

PAGE 59

outcome measures and parent/teacher ADHD ratings scales (Corkum & Siegel, 1993; Alloway et al., 2009). These studies also po int out that many children who perform well on the CPT do, in fact, meet diagnostic criter ia for ADHD. To reinforce this point, approximately 30% of the current sample (in which all children received a diagnosis of ADHD) achieved overall Attention Quoti ents that were 100 or better (with 100 considered exactly Average). Interestingly, when compared to all other variables, age showed the highest association with CPT performance in this sample ( r = .49) such that older children exhibited better combined auditory/visual attent ion scores. It appears that children acquire self-modulation and behavioral control as they age, which may make the norm-referencing of CPTs for adolescent s and adults less precise. Future studies should help clarif y this issue. Given that CPTs have not consistently been found to relate to parent and teacher ratings of ADHD symptoms or to overall parent and teacher reports of impairment, there needs to be a serious examination into the costs versus the benefits of utilizing such measures during routine AD HD evaluations. Depending upon the model, the cost of purchasing a computerized test of attention can rage from 500 to over 1,000 dollars. Clinicians must consider other cost-efficient alternatives that show signs of promising ecological validity and concordance with children s day-to-day behaviors and functioning. In a study that looked at ex ecutive functions in adolescents with ADHD (Toplak, Bucciarelli, Jain, & Tannock, 2009) researchers found that only a small amount of unique variance in predicting AD HD status was attributable to performance measures, while a bulk of the variance was attributable to parent and teacher ratings on the Behavior Rating Inventory of Executive F unctioning (BRIEF; Gioi a, Isquith, Guy, & 59

PAGE 60

Kenworthy, 2000). A growing body of evi dence suggests that ratings scales that tap into executive processes (such as planning and organizational skills) are clinically relevant. Certainly ratings scales are less ex pensive and more efficient to administer in the context of ADHD evaluations. As an extension of the current study, one might consider the incremental validity of a m easure like the BRIEF in predicting average levels of psychosocial impairment. Limitations and Future Directions The current study is not without its limitations. It woul d be desirable to carry out the investigation with a larger sample size. Despite this limitation, significant correlational findings did emerge between a number of collected meas ures. There were several moderate correlations that might have been significant with a larger sample. Significant main effects for parent and teacher ratings of ADHD and ODD were also found in the incremental validity analyses utiliz ing hierarchical regressions. The small effect sizes obtained for the clinic-based meas ures of child behavior (the HBRS and IVA CPT) suggest that even with many more pa rticipants, significant findings for these particular methods of capturing ADHD behavior would not be likely. The small sample size speaks to the logistical challenges of re stricting the inclusion criteria to children who met diagnostic criteria for ADHD or AD HD plus ODD/CD. This methodological decision was made in order to ensure that the primary infl uence of overall impairment was ADHD/other disruptive diso rder and not a learni ng, mood, or anxiety disorder. A fair number of individually screened children fa iled to meet inclusion criteria because of the presence of comorbid learning disabilit ies. Indeed, Pliszka, Carlson, and Swanson (1999) estimated that up to 25-35% of children with ADHD also have learning disabilities. Most children referred for eval uations are likely identified because of the 60

PAGE 61

greater level of impairment t hat behavioral plus significant academic underachievement difficulties present. Despite the recruitment challenges, a la rger sample of children who meet the specified inclusionar y/exclusionary criteria would be helpful to better sort out the influence of variables such as medi cation status, ADHD type, and gender on impairment levels. The small sample made it impossible to examine impairment levels by group. Beyond the methodological cons ideration of sample size, it would be desirable to have a higher degree of control over factors such as the time frame for recruiting participants (e.g. limit participant s to those seen during the middle of the school year so that all parent and teacher ratings as well as performance-based measures can be completed concurrently). Another important limitation of this study relates to the idea of what Hunsley and Meyer (2003) describe as the criterion probl em in incremental validity research. Generally, the criterion must be reliable. Along this vein, the authors note that aggregate variables make superior and more reli able criterion variabl es over individual indicators. For this reason, an average impairment index was selected as the dependent variable as opposed to individual domains of psychosocial impairment. Another important point the authors make is that relationships between predictor and criterion variables may be artificially elevat ed if they are from the same source. The same could be said if they are obtained by the same method (e.g. paper and pencil questionnaires). In the current study, both ADHD symptom measures and impairment measures were collected from parents and t eachers. It is plausible that method variance may have contributed to some degree to the significant findings in the study. While this may have been the case, the obtai ned findings are precisely what one would 61

PAGE 62

expect to find; namely that increased le vels ADHD symptomatol ogy are related to increased levels of impairment. It is also possible that the potent ial confound of method variance made it more difficult for the clin ic-based measures of child behavior to demonstrate unique incremental validity. A future direction, then, would be to methodologically define either symptoms or impairment in a distinct manner. For example, an alternative approach of captur ing ADHD symptoms might be for observers to quantify specific symptoms, such as the frequency of classroom interruptions and incidences of forgotten homew ork, both at home and at school over a period of time. The use of an observational system such as the COCADD (Atkins, Pelham, & Licht, 1988) would be one such data collection technique about which it would be helpful to acquire incremental validity information in the future. A discussion of the criterion variable, impai rment, raises issues regarding the role of this construct in diagnosing ADHD. Pelham, Fabiano, and Massetti (2005) discussed limitations of the most co mmon methods of diagnosis (i.e. symptom checklists) by saying that the symptom is not informati ve for treatment without knowledge of the impaired functioning that it reflects and it s context (p. 468). The authors argue that a focus on impairment, rather t han symptoms, may help improv e childrens functioning in the long run. The current study and other studies have found modest to moderate correlations between ADHD symptoms and psychosocial impairment. The modest associations suggest a great deal of unexplained variance in the construct of impairment. Researchers should continue to look into other potential factors that influence impairment. Another practical future direction is to identify the role that impairment ought to play in diagnostic dec ision-making in general. Evidence suggests 62

PAGE 63

that the rates of ADHD diagnosis c hange substantially depending on the degree to which clinicians do or do not use formal meas ures of impairment (G athje, Lewandowski, & Gordon, 2008). Without specific guidelines about how to incorporate the construct of impairment, diagnostic decision-making is likely to continue to vary by clinician. Summary In summary, the current findings revealed significant associations between parent and teacher reports of ina ttentive, hyperactive, and oppos itional/defiant symptoms and impairment. Further, the general findings from this study suggest that the clinic-based measures used in this investigation did not add incrementally to information gained form parents and teachers in predictin g ADHD-specific impairment. The small sample size and methodological consideratio ns temper the generalizability of the findings; however, many of the specific effect sizes are consis tent with those from other studies with larger sample sizes. One plausible explanation of why the clinic-based measures did not strongly relate to impairment is that children with ADHD tend to behave more appropriately in the context of one-onone testing situations due to demand characteristics. The activities and ta sks involved in comprehensive psychological evaluations are usually novel for the child, and the pace of presentation is fairly quick with an examiner who may differentially re inforce appropriate behavior. Also, when an examiner is present to redirect and engage a child, he or she may be less likely to wander from the task or to act out. Data suggest that when children do show inattentive, hyperactive, and impulsive behavio rs in a more controlled setting such as the clinic (a lower base rate occurrence a ccording to the literat ure), these symptoms should be heeded as true signs of difficulty. Clinicians observations of off-task 63

PAGE 64

behaviors during individual testing are predict ive of social impairment across other situations as well, such as at school. The current findings offer support for the not ion that clinicians should rely heavily on symptom checklists as the most cost an d time-efficient means of diagnosing ADHD. In their article on evidence-based assessment of ADHD, Pelham, Fabiano, and Massetti (2005), arrived at the above conclusion and conjectured that day-long, clinic-based ADHD evaluations may not provide increment al validity for treatm ent planning purposes above and beyond parent and teacher rating scales. The authors note that in addition to collecting information from parent and teachers about ADHD symptoms, a clinical interview or intake questionnaire is necessary to obtain the age of onset as well as to form a basis from which to rule in or rule out other disorders. They argue that beyond these core requirements, information about the incremental va lidity of available combinations of ADHD assessment approaches is sorely needed to help answer questions about cost-effectiveness and value. The results of the present study and other studies question the ec ological validity, and indeed t he incremental validity, of measures of clinic-based behavior such as the ones employed here. Taken together, the results call for a strong reliance on asse ssment measures and methods that tap the childs everyday attention, impulsivity and acti vity level. This translates into symptom checklists, real-world observation of behav ior, and measures of impairment (Pelham, Fabiano, & Massetti). As indicated, experts assert that psychologists should be focusing most on measuring and quantifying impairment, gi ven its relevance to selecting appropriate targets for behavior change in tr eatment and its high correspondence with long-term outcome. Continued exploration of what best accounts for and influences 64

PAGE 65

functional impairment across home and school se ttings is an important research goal. For example, the current study provides su pport for targeting comorbid oppositional symptoms, as well as ADHD symptoms, given their stro ng, unique link to impairment. Future research that sheds light on those as pects of child behavior that are most highly associated with impairment, as well as the context in which these behaviors occur, will be valuable to both the study and practice of ADHD assessment. 65

PAGE 66

LIST OF REFERENCES Abikoff, H., Gittelman-Klein, R., & Klein, D.F. (1977). Validati on of a classroom observation code for hyperactive children. Journal of Consulting and Clinical Psychology, 45, 772. Achenbach, T.M. (1991). Integrative guide for the 1991 CBCL/4-18, YSR, and TRF profiles. Burlington: University of Vermont, Department of Psychiatry. Alloway, T.P., Gathercole, S.E ., Holmes, J., Place, M., Ellio tt, J.G., & Hilton, K. (2009). The diagnostic utility of behavioral check lists in identifying children with ADHD and children with working memory deficits. Child Psychiatry and Human Development, 40, 353. American Academy of Child and Adolescent Psych iatry (2007). Practi ce parameters for the assessment and treatment of children and adults with AttentionDeficit/Hyperactivity Disorder. Journal of Child and Adolescent Psychiatry 46, 894. American Academy of Pediatrics. (2001). Clinical practice guideline: Diagnosis and evaluation of the child with attenti on-deficit/ hyperactivity disorder. Pediatrics, 105, 1158. American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Anastopoulos, A.D., & Shelton, T.L. (2001). Assessing attention-deficit/hyperactivity disorder. Dordrecht, Netherlands: Kluwer Academic Publishers. Angold, A., & Costello, E.J. (1996). The relative diagnostic utility of child and parent reports of oppositional defiant behaviors. International Journal of Methods in Psychiatric Research, 6, 253. Angold, A., Costello, E.J., Farmer, E.M.Z., Bu rns, B.J., & Erkanli, A. (1999). Impaired but undiagnosed. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 129. Atkins, M.S., Pelham, W.E., & Licht, M. (1998). The development and validat ion of objective measures for the assessment of conduct and attention deficit disorders. In R.J. Prinz (Ed.). Advances in behavioral assessment of children and families (Vol. 4, pp. 3) Greenwich, CT: JAI. Barkley, R.A. (1998). Attention Deficit Hyperactivity Disorder: A handbook for diagnosis and treatment (2nd ed.). New York, NY: The Guilford Press. 66

PAGE 67

Barkley, R.A., Fischer, M., Smallish, L., & Fletcher, K. (2006). Young adult outcomes of hyperactive children: Adaptive functioning in major life activities. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 192. Biederman, J., Faraone, S.V. Mick, E., Williamson, S., W ilens, T.E., Spencer, T.J., et al. (1999). Clinical correlates of ADHD in females: findings from a large group of girls ascertained from pediatric and psychiatric referral sources. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 966. Bird, H.R., Canino, G.J., Davi es, M., Ramrez, R., Chvez, L., Duarte, C., & Shen, S. (2005). The Brief Impairment Rating Scal e (BIS): A multidimensional scale of functional impairment for children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 699. Bird, H.R., Shaffer, D., Fisher P., Gould, M.S., Staghezza, B. Chen, J.Y., et al. (1993). The Columbia Impairment Scales (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research, 3, 167. Cantwell, D.P. (1996). Attention Deficit Disorder: A revi ew of the past 10 years. Journal of the American Academy of Child and Adolescent Psychiatry 35, 978 987. Centers for Disease Control and Prevention (2005). Prevalence of diagnosis and medication treatment for attention deficit/h yperactivity disorder United States 2003. Morbidity and Mortality Weekly Report: MMWR, 54, 842. Collett, B.R., Ohan, J.L., & My ers, K.M. (2003). Ten-year re view of rating scales: V. Scales assessing attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 1015. Conners, C.K. (1997). Conners Rating Scales Revised Technical Manual. North Tonawanda, N.Y.: MultiHealth Systems, Inc. Conners, C.K. (2008). Conners 3rd Edition Technical Manual. Toronto, ON: MultiHealth Systems, Inc. Copeland, L., Wolraich, M., Lindgren, S., Milich, R., et al. (1987). Pedi atricians reported practices in the assessment and treatment of attention deficit disorders. J ournal of Developmental and B ehavioral Pediatrics, 8, 191. Corkum, P.V., & Siegel, L.S. (1993). Is the continuous performance task a valuable research tool for use with children with a ttention-deficit/hyperactivity disorder? Journal of Child Psychology and Psychiatry, 34, 1217. 67

PAGE 68

DeVito, C., & Hopkins, J. (2001). Attachment, parenting, and marital dissatisfaction as predictors of disruptive behavior in preschoolers. Development and Psychopathology, 13, 215. Edwards, M.C., Schulz, E.G., Ch elonis, J., Gardner, E., Philyaw A., & Young, J. (2005). Estimates of the validity and utility of unstructured clinical observations of children in the assessment of ADHD. Clinical Pediatrics, 44, 49. Epstein, J.N., Erkanli, A., C onners, C.K., Klaric, J., Costello J.E., & Angold, A. (2003). Relations between continuous performance test performance measures and ADHD behaviors, Journal of Abnormal Child Psychology, 31, 543. Fabiano, G.A., Pelham, W.E. Waschbusch, D.A., Gnagy, E.M ., Lahey, B.B., Chronis, A.M., et al. (2006). A practi cal measure of impairment: Psychometric properties of the Impairment Rating Scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369. Ford, J.D., Racusin, R., Davis, W.B., Ellis, C.G., Thomas, J., Rogers, K., et al. (1999). Trauma exposure among childr en with oppositional defiant disorder and attention deficit-hyperactivity disorder. Journal of Consulting and Clinical Psychology, 67, 786. Gadow, K.D., Sverd, J., Spra fkin, J., Nolan, E.E., & Gr ossman, S. (1999). Long-term methylphenidate therapy in children with co morbid attention-deficit hyperactivity disorder and chronic multiple tic disorder. Archives of General Psychiatry, 56, 330. Gathje, R.A., Lewandowski, L.J., & Gordon, M. (2008). The role of impairment in the diagnosis of ADHD. Journal of Attention Disorders, 11, 529. Gioia, G.A., Isquith, P.K., Guy, S., & Kenworthy, L. (2000). BRIEF: Behavior Rating Inventory of Executive F unction professional manual. Lutz, FL: Psychological Assessment Resources, Inc. Gittelman, R.G., & Klein, D. ( 1985). Hillside Behavior Rating Sale. Psychopharmacology Bulletin, 21, 898. Glutting, J.J., Robins, P.M., & de Lancey, E. (1997). Discriminant validity of test observations for children with attention deficit/hyperactivity. Journal of School Psychology, 35, 391. Glutting, J.J., Youngstrom, E. A., Oakland, T., & Watkins, M.W., (1996). Situational specificity and generality of test behavio rs for samples of normal and referred children. School Psychology Review, 25, 94. 68

PAGE 69

Heaton, S.C., Reader, S.K., Preston, A.S., Fennel l, E.B., Puyana, O.E., Gill, N.G., et al. (2001). The Test of Everyday Attenti on for Children (TEA-Ch): patterns of performance in children with ADHD and clinical controls. Child Neuropsychology, 7, 251. Hinshaw, S.P., & Nigg, J.T. (1999). Behavio r rating scales in the assessment of disruptive behavior problems in childhood. In D. Shaffer & P. Christopher, et al. (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp. 91 126). New York, NY: Guilford Press. Hinshaw, S.P., Owens, E.B., Wells K.C., Kraemer, H.C., Abiko ff, H.B., Arnold, E.L., et al. (2000). Family processes and tr eatment outcome in the MTA: Negative/Ineffective parenting practices in relation to multimodal treatment. Journal of Abnormal Child Psychology, 28, 555. Hunsley, J., & Meyer, G.J. (2003). The increment al validity of psychological testing and assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15, 446. Johnston, C., & Murray, C. (2003). Incrementa l validity in the psyc hological assessment of children and adolescents. Psychological Assessment, 15, 496. Lahey, B.B., Pelham, W.E., Stein, M.A., Loney, J., Trapani, C., Nugent, K., et al. (1998). Validity of DSM-IV attention-deficit/hyper activity disorder for younger children. Journal of the American Academy of Child and Adolescent Psychiatry, 37, 695 702. Landis, J. R., Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159. Lobitz, G.K., & Johnson, S.M. (1975). Normal ve rsus deviant children: A multimethod comparison. Journal of Abnormal Child Psychology, 3, 353. Meyer, G.J., Finn, S.E., Eyde, L. D., Kay, G.G., Moreland, K.L. Dies, R.R., et al. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128. MTA Cooperative Group. ( 1999). 14-month randomized clinical trial of treatment strategies for attention deficit hyperactivity disorder. Archives of General Psychiatry, 56, 1073. Murray, D.W., Kollins, S.H., Hardy, K.K., Ab ikoff, H.B., Swanson, J.M., Cunningham, C., et al. (2007). Parent versus teacher rati ngs of attention-deficit/hyperactivity disorder symptoms in the Preschoolers with Attention-Deficit/Hyperactivity Disorder Treatment Study (PATS). Journal of Child and Adolescent Psychopharmacology, 17, 605. 69

PAGE 70

Pelham, W.E., Fabiano, G.A., & Massetti, G.M. (2005). Evidenc e-based assessment of attention deficit hyperactivity diso rder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 449. Pelham, W.E., Lang, A.R., Atkes on, B., Murphy, D.A., Gnagy, E. M., Greiner, A.R., et al. (1998). Effects of deviant child behavior on alcohol consumption. American Journal of Addictions, 7, 103. Pliszka, S.R., Carlson, C.L., & Swanson, J.M. (1999). ADHD with comorbid disorders: Clinical assessment and management. New York, NY: Guilford. Power, T.J., Andrews, T.J., Eira ldi, R.B., Doherty, B.J., Ikeda, M.J., DuPaul, G.J., et al. (1998). Evaluating attention defic it hyperactivity disorder using multiple informants: The incremental utility of combin ing teacher with parent reports. Psychological Assessment, 10, 250. Power, T.J., Costigan, T.E ., Leff, S.S., Eiraldi, R.B., & Landau, S. (2001). Assessing ADHD across settings: Contributions of behavioral assessment to categorical decision making. Journal of Clinical Child Psychology, 30, 3992. Rapport, M.D., Chung, K.M., Shore, D., Denney D.B., & Isaacs, P. (2000). Upgrading the science and technology of assessment and diagnosis: Laboratory and clinicbased assessment of children with ADHD. Journal of Clinical Child Psychology, 29, 555. Reynolds, C.R., & Kamphaus, R.W. (2002). The clinicians guide to the Behavior Assessment Scale for Children. New York, NY: Guilford. Riccio, C.A., Reynolds, C.R., & Lowe, P. (2001). Neurobiology of attention and executive control. In Clinical applications of continuous performance tests Measuring attention and impulsive responding in children and adults (pp. 1). New York, NY: Wiley. Rosvold, H., Mirsky, A ., Sarason, I., Bransome, E., & Beck, L. (1956). A continuous performance test of brain damage. Journal of Consulting Psychology, 20, 343 350. Rowe, R., Maughan, B., Costello, E.J., & An gold, A. (2005). Defining oppositional defiant disorder. Journal of Child Psychology and Psychiatry, 46, 1309 Sandford, J.A., & Turner, A. (1995). Manual for the Integrated Visual and Auditory Continuous Performance Test. Richmond, VA: Braintrain. 70

PAGE 71

Shaffer, D., Fisher, P., Lucas, C.P., Dulcan, M.K., & Schwab-Stone, M.E. (2000). NIMH Diagnostic Interview Schedule for Child ren Version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28. Shaffer, D., Gould, M.S., Bras ic, J., et al. (1983). A Children s Global Assessment Scale (CGAS). Archives of General Psychiatry, 40, 1228. Shelton, T.L., Barkley, R.A ., Crosswait, C., Moorehouse, M., Fletcher, K., Barrett, S., et al. (1998). Psychiatric and psychological morbidity as a function of adaptive disability in preschool children with aggressive and hyperactive-impulsiveinattentive behavior. Journal of Abnormal Child Psychology, 26, 475. Sleator, E.K. & Ullmann, R.K. (1981). Can the physician di agnose hyperactivity in the office? Pediatrics, 67, 13. Smith, B.H., Pelham, W.E., Gn agy, E., Molina, B., & Evans, S. (2000). The reliability, validity, and unique contributions of self-r eport by adolescents receiving treatment for attention-deficit/hyperactivity disorder. Journal of Consulting and Clinical Psychology, 68, 489. Sparrow, S.S., Balla, D.A ., & Cicchetti, D.V. (1984). Vineland Adaptive Behavior Scales-Interview Edition, Survey Form Circle Pines, MN: American Guidance Services. SPSS Inc. (2008). SPSS 17.0 for Windows, Rel 17.0.0. Chicago: SPSS Inc. Toplak, M.E., Bucciarelli, S. M., Jain, U., & Tannock, R. (2009). Executive Functions: Performance-based measures and the Behav ior Rating Inventory of Executive Function (BRIEF) in adolescents with attention deficit/hyperactivity disorder (ADHD). Child Neuropsychology, 15, 53. Wechsler, D. (2003). Wechsler Intelligence Scale for Children Fourth Edition: Manual. San Antonio: The Psychol ogical Corporation. Weinberg, S.L., & Abramowitz, S. (2008). Statistics using SPSS: An integrative approach (2nd ed.). London, UK: Cambridge University Press. Willcutt, E.G., Hartung, C.M., Lahey, B.B., Loney J., & Pelham, W.E. (1999). Utility of behavior ratings by examiners during a ssessments of preschool children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 27, 463. 71

PAGE 72

72 Winters, N.C., Collett, B.R., & Myers, K.M. (2005). Ten-year review of rating scales VII: Scales assessing functional impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 309.

PAGE 73

BIOGRAPHICAL SKETCH Lindsay McAlister Stewart was born in 1981 in Jacksonville, Florida. She received a Bachelor of Arts, with dist inction, from the University of Virginia, where she double majored in psychology and Spanish. Lindsay earned a Master of Science in clinical psychology in 2005 from the University of Florida and went on to complete her predoctoral internship at the Kennedy Krieger Institute/Johns Hopkins School of Medicine during the 2007-2008 year. She and her husband curr ently reside in Atlanta, Georgia. 73