Citation
Detecting significant change in the neurosychological performance of college football players

Material Information

Title:
Detecting significant change in the neurosychological performance of college football players
Creator:
Phalin, Benjamin R
Publication Date:
Language:
English
Physical Description:
iii, 142 leaves : ill. ; 29 cm.

Subjects

Subjects / Keywords:
Cognitive models ( jstor )
Concussions ( jstor )
Confidence interval ( jstor )
Craniocerebral trauma ( jstor )
Mathematical variables ( jstor )
Modeling ( jstor )
Physical trauma ( jstor )
Population control ( jstor )
Regression analysis ( jstor )
Statistical models ( jstor )
Dissertations, Academic -- Psychology-Clinical Psychology -- UF ( lcsh )
Psychology-Clinical Psychology thesis, Ph. D ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 2004.
Bibliography:
Includes bibliographical references.
General Note:
Printout.
General Note:
Vita.
Statement of Responsibility:
by Benjamin R. Phalin.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
3103770 ( ALEPH )
775499329 ( OCLC )

Downloads

This item has the following downloads:

detectingsignifi00phal ( .pdf )

AA00004699_00001.pdf

detectingsignifi00phal_0043.txt

detectingsignifi00phal_0062.txt

detectingsignifi00phal_0149.txt

detectingsignifi00phal_0038.txt

detectingsignifi00phal_0112.txt

detectingsignifi00phal_0055.txt

detectingsignifi00phal_0069.txt

detectingsignifi00phal_0040.txt

detectingsignifi00phal_0147.txt

detectingsignifi00phal_0056.txt

detectingsignifi00phal_0133.txt

detectingsignifi00phal_0093.txt

detectingsignifi00phal_0001.txt

detectingsignifi00phal_0046.txt

detectingsignifi00phal_0119.txt

detectingsignifi00phal_0111.txt

detectingsignifi00phal_0116.txt

detectingsignifi00phal_0052.txt

detectingsignifi00phal_0132.txt

detectingsignifi00phal_0050.txt

detectingsignifi00phal_0071.txt

detectingsignifi00phal_0037.txt

detectingsignifi00phal_0121.txt

detectingsignifi00phal_0070.txt

detectingsignifi00phal_0067.txt

detectingsignifi00phal_0012.txt

detectingsignifi00phal_0021.txt

detectingsignifi00phal_0092.txt

detectingsignifi00phal_0013.txt

detectingsignifi00phal_0115.txt

detectingsignifi00phal_0063.txt

detectingsignifi00phal_0102.txt

detectingsignifi00phal_0097.txt

detectingsignifi00phal_pdf.txt

detectingsignifi00phal_0101.txt

detectingsignifi00phal_0066.txt

detectingsignifi00phal_0059.txt

detectingsignifi00phal_0118.txt

detectingsignifi00phal_0006.txt

detectingsignifi00phal_0008.txt

detectingsignifi00phal_0077.txt

detectingsignifi00phal_0142.txt

detectingsignifi00phal_0026.txt

detectingsignifi00phal_0042.txt

detectingsignifi00phal_0123.txt

detectingsignifi00phal_0107.txt

detectingsignifi00phal_0048.txt

EIW1DEPWB_SO5ET4_xml.txt

detectingsignifi00phal_0065.txt

detectingsignifi00phal_0031.txt

detectingsignifi00phal_0068.txt

detectingsignifi00phal_0082.txt

detectingsignifi00phal_0045.txt

detectingsignifi00phal_0015.txt

detectingsignifi00phal_0016.txt

detectingsignifi00phal_0126.txt

detectingsignifi00phal_0014.txt

detectingsignifi00phal_0104.txt

detectingsignifi00phal_0018.txt

detectingsignifi00phal_0060.txt

detectingsignifi00phal_0051.txt

detectingsignifi00phal_0007.txt

detectingsignifi00phal_0094.txt

detectingsignifi00phal_0044.txt

detectingsignifi00phal_0109.txt

detectingsignifi00phal_0000.txt

detectingsignifi00phal_0145.txt

detectingsignifi00phal_0098.txt

detectingsignifi00phal_0053.txt

detectingsignifi00phal_0049.txt

detectingsignifi00phal_0124.txt

detectingsignifi00phal_0078.txt

detectingsignifi00phal_0072.txt

detectingsignifi00phal_0034.txt

detectingsignifi00phal_0002.txt

detectingsignifi00phal_0075.txt

detectingsignifi00phal_0125.txt

detectingsignifi00phal_0088.txt

detectingsignifi00phal_0090.txt

detectingsignifi00phal_0061.txt

detectingsignifi00phal_0117.txt

detectingsignifi00phal_0081.txt

detectingsignifi00phal_0030.txt

detectingsignifi00phal_0005.txt

detectingsignifi00phal_0151.txt

detectingsignifi00phal_0041.txt

detectingsignifi00phal_0022.txt

detectingsignifi00phal_0146.txt

detectingsignifi00phal_0024.txt

detectingsignifi00phal_0027.txt

detectingsignifi00phal_0017.txt

detectingsignifi00phal_0087.txt

detectingsignifi00phal_0025.txt

detectingsignifi00phal_0138.txt

detectingsignifi00phal_0010.txt

detectingsignifi00phal_0105.txt

detectingsignifi00phal_0140.txt

detectingsignifi00phal_0079.txt

detectingsignifi00phal_0122.txt

detectingsignifi00phal_0019.txt

detectingsignifi00phal_0127.txt

detectingsignifi00phal_0028.txt

detectingsignifi00phal_0139.txt

detectingsignifi00phal_0131.txt

detectingsignifi00phal_0032.txt

detectingsignifi00phal_0039.txt

detectingsignifi00phal_0096.txt

detectingsignifi00phal_0137.txt

detectingsignifi00phal_0080.txt

detectingsignifi00phal_0148.txt

detectingsignifi00phal_0074.txt

detectingsignifi00phal_0054.txt

detectingsignifi00phal_0113.txt

detectingsignifi00phal_0106.txt

detectingsignifi00phal_0089.txt

detectingsignifi00phal_0108.txt

detectingsignifi00phal_0110.txt

detectingsignifi00phal_0004.txt

detectingsignifi00phal_0144.txt

detectingsignifi00phal_0033.txt

detectingsignifi00phal_0103.txt

detectingsignifi00phal_0036.txt

detectingsignifi00phal_0029.txt

detectingsignifi00phal_0135.txt

detectingsignifi00phal_0130.txt

detectingsignifi00phal_0099.txt

detectingsignifi00phal_0134.txt

detectingsignifi00phal_0114.txt

detectingsignifi00phal_0095.txt

AA00004699_00001_pdf.txt

detectingsignifi00phal_0143.txt

detectingsignifi00phal_0035.txt

detectingsignifi00phal_0086.txt

detectingsignifi00phal_0100.txt

detectingsignifi00phal_0120.txt

detectingsignifi00phal_0009.txt

detectingsignifi00phal_0011.txt

detectingsignifi00phal_0083.txt

detectingsignifi00phal_0023.txt

detectingsignifi00phal_0085.txt

detectingsignifi00phal_0058.txt

detectingsignifi00phal_0057.txt

detectingsignifi00phal_0091.txt

detectingsignifi00phal_0020.txt

detectingsignifi00phal_0150.txt

detectingsignifi00phal_0003.txt

detectingsignifi00phal_0084.txt

detectingsignifi00phal_0128.txt

detectingsignifi00phal_0073.txt

detectingsignifi00phal_0064.txt

detectingsignifi00phal_0141.txt

detectingsignifi00phal_0047.txt

detectingsignifi00phal_0076.txt

detectingsignifi00phal_0129.txt

detectingsignifi00phal_0136.txt


Full Text











DETECTING SIGNIFICANT CHANGE IN THE NEUROPSYCHOLOGICAL
PERFORMANCE OF COLLEGE FOOTBALL PLAYERS













By

BENJAMIN R. PHALIN


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2004






























Copyright 2004

by

Benjamin R. Phalin















ACKNOWLEDGEMENTS

A project of this scope involves the collaboration of a great number of people.

First and foremost, I would like to acknowledge my parents, whose emphasis on the

value of education and unswerving support throughout my own educational endeavors

has allowed me to continue to strive to reach for my potential. Secondly, I would like to

thank my wife for her encouragement throughout the process of the preparation of this

paper. Last, but certainly not least, I would like to thank Dr. Eileen Fennell and Dr.

Duane Dede for their guidance, and for providing me with outstanding examples of true

scientist-practitioners.















TABLE OF CONTENTS


Page

ACKNOWLEDGEMENTS............................................................. iii

ABSTRACT................................................................................... vi

CHAPTER

ONE INTRODUCTION................................................................. 1

Historical Context...................................................... 1
Minor Head Injury: Definition............................................ 4
Minor Head Injury: Pathology............................................ 10
Neuropsychological Effects of Minor Head Injury..................... 16
Sports-related Minor Head Injury...................................... 20
Reliable Change ....................................................... .. 28
Reliable Change Index............................................ 29
Reliable Change Index with Correction for Practice.......... 30
Linear Regression................................................ 31
Multiple Regression Model....................................... 31
Specific Aims and Hypotheses.......................................... 33

TWO METHODS.......................... ......... ...................................... 35

Subjects ................................. ...................................... 35
Protocol.............................. .... .................................. 36
Baseline............................................................. 35
In-Season Concussion Evaluation............................... 39
Data Analysis............................................................... 40
Basic Models...................................................... 41
Reliable Change Indices..................................... ..... 41
Regression Models............................................ .... 42
Sensitivity and Specificity................................... ..... 42

THREE RESULTS...................................................................... 44

Demographic Data for Baseline Population............................. 44









Demographic Characteristics for Concussed and Control............. 47
Participants
Baseline Performance..................................................... 49
Overview of Repeated Testing............................................51
Detecting Significant Decline Using Basic Statistical Methods......51
Model 1 .............................. ................ .......... 56
Model 2............................................................ 63
Reliable Change Index Models................................ ...... 71
M odel 3........................................................... 71
Model 4................................ ............................. 79
Model 5 .......................................................... 87
Regression Models......................................... ...............95
M odel 6................................... ......................... 95
Model 7.......................................................... 103

FOUR DISCUSSION ..................................................................... 112
Basic Models....................................................... .......... 116
RCI Models......................................................... ....... 117
Regression Models...........................................................121
Limitations and Implications for Future Research..................... 129
Contributions of the Current Study...................................... 131

REFERENCES............................................................................ 135

BIOGRAPHICAL SKETCH................................................................ 142


























v















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

DETECTING SIGNIFICANT CHANGE IN THE NEUROPSYCHOLOGICAL
PERFORMANCE OF COLLEGE FOOTBALL PLAYERS

By

Benjamin R. Phalin

August 2004


Chair: Eileen B. Fennell
Cochair: Duane E. Dede
Major Department: Clinical and Health Psychology

The use of neuropsychological test instruments has been advocated by researchers

in the detection of cognitive decline following sports-related concussion as being a

sensitive means of producing objective results with which to aid in making return-to-play

decisions. However, no study to date has provided a systematic examination of serial

neuropsychological testing over several days following concussion. In the current study,

a brief battery ofneuropsychological tests was administered to 443 college football

players at four Division I universities. Of the 443 initial participants, 26 concussed

players were identified by team trainers and subsequently underwent retesting using the

same battery of tests within 24 hours, 3 days, 5 days, and 7 days of concussion.

Concussed players were assigned controls matched on position, history of diagnosed








learning disability, and history of previous concussion who were also re-administered the

battery at the same test intervals.

Seven different statistical models conceptually organized into three groups were

examined to determine their ability to distinguish between the concussed and

nonconcussed participants. Basic models included a comparison of retest performance to

normative data and comparison of retest performances to baseline performance. RCI

models included a basic RCI approach, RCI with practice correction, and indices created

based on the magnitude of change of the control population. Regression models included

simple linear regression and multiple regression models, though multiple regression

analysis was ended due to instability of prediction equations. Optimal

sensitivity/specificity combinations occurred when classification was made based on one

significant variable finding. Good sensitivity was observed for all models at the 24-hour

retest period, although specificity was a limiting factor for the most basic models. Linear

regression models and an analysis of magnitude of change over re-test intervals were the

only models to distinguish between groups at Day 3 and Day 7. Results suggested the

importance of systematic examination of cognitive decline following sports-related

concussion, especially given currently-used AAN return-to-play guidelines, as well as

need for appropriate comparison data and statistical analyses to aid in the determination

of cognitive recovery following concussive injury. Further data collection is necessary,

especially at later sessions, to accurately use multiple regression models.















CHAPTER ONE
INTRODUCTION

Historical Context

The use of neuropsychological measures has, over the course of the continuing

evolution of the field, been the focus of considerable debate in terms of utility and

correlates of specific test performance. This evolution has been described in terms of

distinct eras (McSweeny et al.,1993) or areas of focus in the use of neuropsychological

test performance. The first has been described by Rourke (quoted in Mcsweeny et al.,

1993), and involved the interpretation of test performance and the use of cognitive testing

mainly to imbue some veracity to the hypothesis that performance on certain specifically

designed test measures indirectly suggested dysfunction in certain localized brain

anatomy. This endeavor, of course, has been the mainstay of the various academic

subspecialties concerned with the study of neuroanatomy and the associated cognitive

sciences and theories. The very roots of this goal can be traced back as far as the time of

Gall, phrenology, and the notion that various protrusions on the scalp of an individual

revealed certain personality traits and tendencies (Cowey, 2001). While the theory of

phrenology itself has long since been discredited, it is the notion of localization that has

stood the test of time and has led to a greater understanding and logical organization of

the study of the structure and function of the human brain. During the infancy of the field

of neuropsychological theory in general, the primary opportunity to further corroborate

structure and function was found to be the study of the performance of individuals with








localized lesions. From the performance of these individuals, inferences concerning the

function of the absent or damaged areas were formulated. One of the most cited cases is

that of patient H.M., who received resection of the anterior portions of both of his

temporal lobes as part of a radical surgical correction for intractable epilepsy. Brenda

Milner and her colleagues detailed the cognitive effects of this resection and established

the utility of neuropsychological tests in the measurement of deficient cognitive abilities

(Milner, 1958). As McSweeny et al. (1993) noted, neuropsychological tests have been

utilized recently in order to aid in diagnosis and to provide clinically significant

contributions to the diagnostic process. A vast literature has been amassed, for instance,

in the application of neuropsychological techniques in the examination and treatment of

epileptic patients subsequent to patient H.M. (Bennett, 1987; Hermann, 1985; Hermann

and Wyler, 1988). Cognitive tests have been used to aid in the process of localizing the

focus of epileptic activity by suggesting relatively discrete anatomical areas of

dysfunction which are thought to contribute to poor performance on certain measures.

Further, cognitive tests have been used in conjunction with reversible lesion methods

(e.g., WADA) to localize brain regions responsible for language and memory in

individuals undergoing surgical resection methods in order to avoid the accidental

destruction of these areas and their subsequent debilitating effects (Loring et al., 1992).

Determination of the language- dominant hemisphere via neuropsychological methods is

an important step in identifying those individuals who might best be served with this

surgical treatment.

A further use ofneuropsychological techniques becomes salient when one seeks

to quantify the effect of a surgical resection or any other neurological insult. By








comparing an individual's performance before a potentially neurologically altering event

to his/her performance afterward, one is able to objectively describe the resulting

cognitive effects. Another commonly used methodology, in general, involves comparing

the performance of one individual to that of a group of other individuals, or normative

population. In this sense, the term "normative" implies that the mean score of a group of

individuals is the standard to which other individuals can be compared. This is perhaps

the most typical method used in neuropsychology to gauge the performance of the

individual in question, and has often been used without question as to the validity of such

practice. Within the past several years, neuropsychological test performance has been

used somewhat less frequently to localize neurological insult due to the continuing

growth and advance of imaging technology (Ruff, 2003). While the use of imaging

devices such as positron emission topography (PET), magnetic resonance imaging (MRI),

and functional magnetic resonance imaging (fMRI) has continued to hone the ability of

the investigator to view the brain itself in a non-obtrusive way, and have continued to

lend themselves effectively to the diagnostic process, neuropsychological test results

have played a somewhat decreased role in this capacity. However, the focus of

neuropsychological examination and analysis of test performance have clearly proven

their merit in other ways. For example, the measurement of cognitive abilities and

inherent strengths and weaknesses within the individual offers the neuropsycholgist the

unique opportunity to contribute to the practical knowledge about the cognitive

functioning of a patient, and allows him or her to recommend strategies designed to aid

the individual in returning to previous levels. In the event that a baseline measure of an

individual's performance is available, a comparison with post-event findings allows for








an objective review of resulting cognitive deficits and a gauge of changing cognitive

status. In this sense, a measure of individual cognitive change over time has become a

major focus of neuropsycholgical measures. With this increasing focus on longitudinal

follow-up and repeated testing, several issues regarding serial test administration and the

sensitivity of established neuropsychological measures have arisen. Recently, the use of

repeated assessment over time has been applied in various circumstances in order to

measure the progressive nature of cognitive change. This has been the case, as

McSweeny et al. (1993) pointed out, in the assessment of the dementias, which involve

the insidious loss of cognitive abilities due to the progressive alteration of brain structure

or chemistry. Measures of neuropsychological change over time have also been used to

assess treatment affects and interventions.

Recently, the use of neuropsycholgical testing has been applied to the study of

one of the most common causes of cognitive change and continuing sequelae: head

injury. While the results of moderate to severe head injury are often obvious, both to

current imaging techniques and measures of cognitive status, and the recovery from such

is often prolonged and some effects permanent, the effects of more subtle, or mild head

injuries are not so easy to unilaterally identify. Therefore, the use of neuropsychological

measures in this regard has become more valuable.

Minor Head Injury: Definition

While several researchers have endeavored to elucidate the consequent effects of

minor head injury, the lack of consistency in defining the phenomenon has made it

difficult, if not impossible, to reconcile the findings of different investigations. The use

of the term "minor head injury" generally suggests the occurrence of mild to moderate








traumatic brain injury (Kibby and Long, 1996). However there has been little agreement

on the terminology utilized in describing various levels of severity within this broad

category. A review of the literature reveals a basic framework underlying the working

model of minor head injury. However, from this basic outline, many slight variations

have been created and used in experimental design. In general, a minor head injury is

considered to be one in which a force is applied to the head, producing posttraumatic

memory difficulties or other fast-resolving neurological symptoms" (Kibby and Long,

1996).

In examining the origins of this very basic foundation for defining minor head

injury, Kibby and Long (1996) refer to research findings suggesting that the severity of

head injury is directly related to the force that is applied to it. They indicate that this

finding is supported by the work of Graham et al. (1995) which suggested that the degree

of axonal injury is naturally also affected by the amount and direction of force applied.

In other words, the amount of force applied to the head is directly correlated with the

amount of damage expected to be observed in the axons of the brain and the amount of

metabolic disruption (McIntosh et al., 1996; Meaney et al., 1995). In a natural

progression, one might assume that this also applies to any observed cognitive

dysfunction, which may be directly related to the extent of axonal injury incurred. Kibby

and Long suggest that this is shown through the work of Langfitt and Gennarelli (1982)

and Levin et al., (1987). In their respective investigations, these authors concluded that a

minor head injury resulting in a loss of consciousness of less than 20 minutes was not

associated with any lingering deficits or detectable neuroanatomical damage. Therefore,

the extent of the head injury was assumed to be minor.








Aside from the adoption of some common findings to the definition of minor

head injury, one of the most commonly used indicators of head injury severity is the

Glasgow Coma Scale (GCS) (Jennett and Bond, 1975). The GCS is a measure

commonly employed at the scene of a head injury-inducing accident by immediate care

providers or in an emergency medicine situation. In performing this assessment

technique, the degree of coma is measured by evaluating motor, verbal, and eye-opening

responses to a set of standard stimuli. The resulting score is based on a scale from 3 to

15, the lowest score representing no response to verbal commands or somatic stimulation,

and the highest suggesting the intact ability to follow verbal commands, orientation, and

obvious awareness of sensory input. In the current literature, it is generally suggested

that a minor head injury is one that results in a GCS of 13-15, scores that are often

correlated with a positive long-term outcome. However, the practice of using the GCS by

itself in the assessment of injury severity is questionable since the work of researchers

such as Cantu (1992) and Strauss and Savitsky (1934) has suggested that symptoms

subsequent to injury without the observation of loss of consciousness are also common.

Another commonly used indicator of head injury severity is posttraumatic

amnesia, which has been defined as the period of time following the applied trauma to the

brain in which new memories are unable to be stored. This time period is associated with

the inability to recall events, sequence time, or learn new information (Ahmed et al.,

2000; McAllister, 1992; McFarland et al., 2001). A correlation has been noted between

the length of PTA and the duration of LOC, though McAllister indicates that this

relationship is not always linear, and warns against using either measure as the sole

predictor of head injury severity and potential outcome.








While some commonalities exist between research projects in their inclusion

criteria for cases of minor head injury, a significant amount of slight variation around the

central theme has remained. The reasons for this occurrence may be many, from

theoretical differences to the utilization of samples of convenience or the use of

nonprospective research designs. Researchers have historically not applied a common

definition to their inclusion criteria when studying minor head injury. For instance, in

the often-cited study by Rimel et al. in 1981, patients were included as cases of minor

head injury if they had LOC of less than 20 minutes. This group added another specifier

to the basic definition. In order to control for cognitive changes observed that could

possibly have been related to other event-related trauma, Rimel et al. excluded patients

with a hospital stay of greater than three days. It was their supposition that a longer

hospital stay was indicative of more severe injuries that might introduce confounds to

results obtained that otherwise would have been attributed to the head injury alone.

Another definition was provided by the Centers for Disease Control in 1985, and

while this definition is not specific to minor head injury, a sense of the vague boundaries

that might be inferred from it is still recognizable. The CDC states that a traumatic brain

injury is the "occurrence of injury to the head arising from blunt or penetrating trauma, or

from acceleration-deceleration forces, that is associated with any of these symptoms or

signs attributable to the injury: decreased level of consciousness, amnesia, other

neurologic or neuropsychologic abnormalities"(Thurman et al., 1998).

Subsequent researchers have included patients who met some, but not all, of these

criteria; others have included patients with more severe injuries and confounding

neurological histories (Dikmen et al., 1986; Leininger et al., 1990; Levin et al., 1987).








What is clear in this rather cloudy picture is that the only consistency is inconsistency.

Table 1 indicates the variability in defining minor head injury among some of the most

cited authors in this field.

Table 1. Varying inclusion criteria for minor head injury
Author LOC PTA GCS

Barth et al. (1983) <20 minutes <60 minutes >12
Dikmen et al. (1986) <1 hour >1 hour >11
Parker (1996) <5 minutes
Alexander (1992) <15 minutes <2 hours
King (1996) <24
LOC=loss of consciousness, PTA=posttraumatic amnesia, GCS=Glasgow Coma Scale
In an attempt to assemble these various criteria, the basic definition of minor head

injury, as outlined by Kibby and Long (1996), is as follows. A minor head injury is

incurred by any blow to the head resulting in a loss of consciousness of less than 20

minutes, an initial GCS score of greater than or equal to 13, and posttraumatic amnesia of

less than 24 hours. This definition, while serving as a solid basis for the nosology of

minor head injury and indicating the upper limits of what is thought to constitute minor

head injury, fails to define the boundaries at the minimum degree of that which can be

considered in this category.

Recognizing the variability in defining minor head injury from one study to the

next, a definition of mild traumatic brain injury (MTBI) was formulated by the Mild

Traumatic Brain Injury Committee of the Head Injury Interdisciplinary Special Interest

Group of the American Congress of Rehabilitation Medicine (1993). According to the

consensus reached by this organization, MTBI is defined by, at least one of the following:








1. any period of loss of consciousness, 2. any loss of memory for events
immediately before of after the accident, 3. any alteration of mental state at the
time of the accident, 4. focal neurological deficits, which may or may not be
transient, but when the severity of the accident does not exceed the following: a.
loss of consciousness of 30 minutes or less; b. after 30-minutes, an initial GCS
score of 13-15; and c. posttraumatic amnesia not greater than 24 hours. (Busch
and Alpem, 1998).

This attempt at providing a uniform definition, while again clearly delineating an

upper limit for symptoms indicative of minor head injury, fails to account for the minimal

symptomatology necessary to qualify as such. The minimum limits of this definition

include a wide gradation of injuries, for which the definition itself does not account or

enumerate. Within this category is the type of injury known as concussion, which

according to the Mild Traumatic Brain Injury Subcommittee, also falls under the general

rubric of minor head injury.

Concussion has come to be defined as a "clinical syndrome characterized by

immediate and transient posttraumatic impairment of neural function" (Maddocks and

Saling, 1996). In general, those suffering from concussion are thought to recover without

residual impairment. The categorization of concussion severity was the topic of a recent

set of guidelines authored by the Quality Standards Subcommittee of the American

Academy of Neurology (AAN) in 1997 (cited in Hinton-Bayre et al., 1999). In this

classification scheme, the severity of concussion is based upon a graded scale ranging

from 1, the least severe, to 3, the most severe. Grade 1 concussions are those in which

the individual does not lose consciousness as the result of the insulting force, but

experiences transient confusion (inattention, poor concentration, or inability to process

information or sequence tasks) that resolves in 15 minutes or less. Grade 2 concussions

are those in which, like grade 1 concussions, the individual does not lose consciousness,








but experiences transient confusion that does not remit within a 15 minute time period.

The most severe concussion, grade 3, according the AAN guidelines, includes a period of

loss of consciousness, which may last seconds to minutes, as well as transient

neurological disruption (Kelly and Rosenberg, 1998). These transient neurological

symptoms have also been the subject of many research investigations. Patients suffering

from minor head injuries, including concussion, often complain of a cluster of symptoms,

which was recognized by Strauss and Savitsky (1934). In their study, these investigators

referred to this complex of symptoms as "post-concussion syndrome" (Watson et al.,

1995). This cluster includes those symptoms mentioned above plus many others, which

may persist for as long as weeks or months after injury. The current study will focus on

the neuropsychological effects observed in concussions incurred in a college football

population.

Minor Head Injury: Pathology

Researchers have long sought to identify the core features and underlying

mechanism for the cognitive sequelae that are evidenced following mild traumatic brain

injury. Key to this search is the fact that little evidence has surfaced regarding the

biological changes that are thought to occur in these instances (Montgomery et al., 1991).

Lishman (1988) reviews some of the early research involving animal models of

concussion study in the search for the pathophysiological roots of concussion

symptomatology. Initially, he states, cerebral ischaemia as a result of a blow to the head

was thought to be responsible for the pattern of symptoms observed. This was

hypothesized to arise from the rise in intracranial pressure and obstruction of intracranial

circulation. While this hypothesis theoretically accounted for the brief loss of








consciousness observed in many cases of minor head injury, laboratory findings were not

supportive, and other theories were subsequently formulated. Lishman (1988) points to

the contribution of Denny-Brown and Russell in 1941 concerning the theory that shear

strain resultant from rotational forces are in some part responsible for transient cognitive

deficits in mild head injury. The studies of these researchers on the effect of

acceleration-deceleration effects upon the brain itself promulgated further studies of the

effect of these forces. Denny-Brown's studies with animals suggested that what he

described as "uncomplicated" mild head injury, or concussion, was achieved with his

subjects only when the head was free to move about when the actual insult occurred. His

conclusions, therefore, implicated the acceleration and deceleration forces that were

applied to the skull during the course of the injury.

It was at this point that researchers began to appreciate the fluidity of the

environment within the cranium. That is, investigators moved toward experimental

designs that would allow them to directly observe and measure those forces acting within

the skull itself. Lishman (1988) describes the studies of Pudenz and Shelden in 1946 on

monkeys in which the skullcaps were removed and replaced with transparent material.

By so doing, these researchers were able to observe the movement of the brain at the

moment of impact. They indicated in their report that they observed a "swirling" motion

both at the point of impact and in other areas of the brain not directly impacted by the

blow. It was this swirling motion that was thought to produce many of the symptoms

associated with concussion.

Since the early work of these researchers, many others have confirmed and

elaborated upon these hypotheses. The brain can be thought of as a semi-gelatinous mass








that is suspended in a fluid within a confined space. It is held in place both by its limited

housing within the skull and through its connection via the brainstem and spinal cord.

Because it is "floating" in a homeostatic amount of cerebrospinal fluid (CSF), and

"tethered, it moves out of phase relative to the skull in response to changes in speed or

direction"(McAllister, 1992). The anatomically disrupting effects seen in mild head

injury are thought to be directly correlated with the force of the impact imparted to the

head. Further, relatively few instances of concussion have been noted in cases of static

injury. That is, when the individual is not free to move his head, the results of minor

head injury have not been found to be as substantial (Kingsley, 1996; McAllister, 1992).

Therefore, it would appear that the risk of incurring complications as a result of applied

forces is greatly increased when the head does not have a substance through which it may

conduct a portion of the force vectors acting upon it.

In the case of minor head injury, the most common insults are classified as

diffuse. In this sense, a diffuse injury is one in which a definitive location of focal impact

cannot be identified. Because minor head injury rarely results in fatality, it is often

impossible to detect the subtle neuroanatomical changes inherent to the injury.

Montgomery et al. (1991) suggested that one of the first researchers to identify

anatomical anomalies subsequent to minor head injury was Oppenheimer in 1968. In his

studies, Oppenheimer had the opportunity to examine the brains of five individuals who,

having suffered from mild traumatic brain injuries, died of complications from other

injuries also incurred during the causative incident. Microscopic examination of brain

tissue revealed diffuse axonall disruptions" throughout the white matter. In particular, he

indicated that these injuries were more common in midbrain tegmentum and the corpus








collosum (Montgomery et al., 1991). Subsequent research over the years has indicated

that this diffuse damage to white matter tracts is the probable cause of cognitive

disruption in minor head injury. The motion of the brain within the skull is now thought

to be responsible for the "twisting and stretching" of neuronal axons and the subsequent

damage that occurs. Because this damage can be caused by inertial forces, direct impact

with the head is not necessary to induce detrimental effects (Busch and Alpern, 1998).

Furthermore, Busch and Alpem (1998) point out that damage to axons by shearing forces

causes tearing and the disruption of other cell milieu including blood vessels, possibly

altering blood transport to the directly affected neurons and those immediately

surrounding. In reviewing the histopathology of minor head injury, Busch also refers to

the work of Strich (1956) and Povlishock et al. (1983) in which damage to neurons by

shear or stretching was observed extensively in the white matter of the hemispheres and

in the brainstem. In addition, these researchers discovered that retraction balls could be

viewed in the instances where axonal fibers had been disconnected.

McAllister (1992) suggests that a predilection for these types of anatomical

disruptions occurs in the corpus collosum, superior cerebellar peduncle, basal ganglia,

and periventricular white matter. In each of these cases, the long white matter tracts

appear to be more susceptible to the damaging effects of axonal twisting than are other

areas. In addition, because the brain is situated in a hard casing, the opportunity for

injury by impact with the skull wall is also present, particularly in those regions

vulnerable to bearing the brunt of the type of motion that is often seen in cases of mild

head injury.








Gennarelli's research in the area of traumatic brain injury has provided evidence

to support the notion of what he has termed "coup/contrecoup" distinction. In general, an

injury resultant from a coup force is observed locally, at the site of the impact of the head

with the offending object (Gennarelli and Graham, 1998). He further asserted that the

area and severity of damage are directly related to the size and force of the impacting

object. Therefore, he continued, but warned against generalization, coup injuries are

often associated with the acceleration of the head. Contrecoup injuries, on the other

hand, are often seen as arising from the sudden deceleration of the head. These insults

are "away from the site of injury" (Gennarelli and Graham, 1998). While they are often

observed in a linear relation with the site of impact, and from the coup-related injury site,

because of the nature of the internal environment of the cranium, this is not always the

case. The location of the brain within the skull, as mentioned above, predisposes certain

areas to damage. These areas include the temporal lobe poles, the frontal lobe poles, and

the parasagittal region (Parker, 1996), all of which are thought commonly affected to a

degree in many cases of mild head injury.

In the early 1980's, Gennarelli and his colleagues suggested the term "diffuse

axonal injury" (DAI) in the description of the anatomical effects observed in many

instances of traumatic brain injury. As described above, focal damage produced by a

non-missile, closed-head injury is rather rare. More often, diffuse axonal damage is

prevalent. In elaborating on the definition of DAI, researchers formulated three

categories of DAI (Gennarelli and Graham, 1998). In a DAI of grade I, damage to axons

is notable within the white matter of the hemispheres and occasionally in the brain stem

and corpus collosum. This is most likely the extent of the damage seen in most cases of








minor head injury. A grade II DAI also evidences some "focal abnormalities in the

corpus collosum, often associated with small hemorrhages, called tissue tear

hemorrhages" (Gennarelli and Graham, 1998). According to this gradation system, grade

HI DAI's also include extensive damage to the rostral brainstem. In a 1968 paper,

Oppenheimer and his colleagues reported their finding that "clusters ofmicroglia" were

found in the white matter of patients who, having suffered minor head injury, had died of

unrelated causes. Gennarelli suggests that this finding has been supported recently by

other investigators and has been elaborated upon by others who have discovered the

presence of what he terms "microscopic DAI"(Gennarelli and Graham, 1998).

However, Gennarelli suggested that a possibly more important discovery was

made by Blumbergs et al. (1994) when these investigators reported having performed

immunostaining with an amyloid precursor protein (APP) antibody. In their results, these

researchers reported observing multiple areas of diffuse axonal injury when using this

staining technique in five patients who had received minor head injury. Experimentation

with other antibody staining techniques have met with less success in identifying those

regions that have suffered diffuse axonal injury. Staining for the detection of P-APP

remains the most sensitive to elucidating this phenomenon. Gennarelli warns, though,

that this should not be considered a marker for head injury since heightened production

and collection of P-Apolipoprotein is also seen in the "dystrophic axons" of some elderly

patients and in some cases of known transient ischemic attack and infarction. He

suggests that an accumulation of this precursor protein should suggest an area in which

metabolic change has taken place, which certainly could be the case in minor head injury,

though it may also be true in other instances as well.








Recent work by Roberts et al. (1994) has also suggested a link between the

accumulation of P-APP and the increased risk for the development of Alzheimer's

Disease (AD). The work of these researchers supports the hypothesis that the expression

of P-APP is indeed part of "an acute phase response" to the type of neuronal damage that

is seen in minor head injury. They further suggest that the overaccumulation of the

precursor protein may lead to a build-up of P-APP and subsequently lead to the

expression of certain AD symptoms, or an increase in an individual's predilection to

develop AD later in life if they are already genetically susceptible. In addition to these

important biochemical findings, Busch and Alpem (1998) sites the work ofPovlishock et

al. (1983) in which it was found that minor head injury can lead to a disruption in

electrolyte transfer and capillary permeability.

While the neuroanatomical effects of minor head injury are not easily identified

by common medical measures such as imaging devices and biochemical markers, recent

research in the area of diffuse microscopic damage and the detection of the accumulation

of amyloid precursor protein and its subsequent correlation with outcome have suggested

to investigators that there are biological consequences of even minor head injury. It is

becoming more commonly accepted that these changes do play a role in many of the

neuropsychological and other cognitive changes that are often observed following even

the mildest forms of head injury.

Neuropsychological Effects of Minor Head Injury

While the psychosocial ramifications of minor head injury and the effect of

persisting post concussion syndrome symptoms have received a large amount of attention

in the literature, the issue of cognitive impairment suffered as a result of minor head








injury has been one that has produced varying and debated results. This is due in large

part to the methodology used in different studies. However, there are some common

aspects to the findings from various research groups that suggest that minor head injury

does contribute to detectable cognitive sequelae.

The studies ofRimel et al. (1981) and Barth et al. (1983) are often cited in the

literature when referring to the finding that minor head injury produces a measurable

neuropsychological change (Dikmen et al., 1986). The findings by these two research

groups, using the same population, suggested that there are noticeable cognitive as well

as psychosocial deficits in those having suffered minor head injury. Dikmen argues that

these findings may not be accurate, however, since these groups used what she describes

as an inappropriate normative comparison group. While the groups were impaired

compared to this normative group, she states, it is difficult to parse out the effects of other

factors that are not known, including premorbid neurological status. In her own study on

the effects of minor head injury on neuropsychological functioning, Dikmen indicates

that the findings were much less definitive when testing her subjects at 1 and 12 months

post injury.

Dikmen performed a wide range of neuropsychological tests on 20 consecutive

hospitalized patients who had experienced a period of coma not over 1 hour or, in the

absence of coma with PTA of at least 1 hour, a GCS score of greater than or equal to 12

on admission, with no evidence of cortical or brainstem contusion, no history of prior

head injury or neurological insult, and between the ages of 15 to 60. She utilized a control

group of friends and relatives of those suffering from minor head injury rather than the

published normative data for the individually administered tests. Although her results








were not as conclusive as those of Rimel and Barth, she reported that there was a trend

for her minor head injury population to perform more poorly on tests of concentration

and short-term memory at 1 month after injury. This was supportive of the findings of a

previous and oft-cited study by Gronwall and Wrightson (1974) who showed that those

suffering minor head injury and suffering from post concussion symptoms tended to

demonstrate impairment in tests of attention and processing speed. She tempers her

finding, though, by stating that her results may not be indicative of significant differences

between the groups if one takes into account the premorbid differences in intellectual

ability between them (Dikmen et al., 1986). She did not have any measurements of

baseline functioning with which to compare her findings.

In another study of a large group of head injured individuals participating in three

separate experiments, Dikmen again sought to clarify the neuropsychological

ramifications of traumatic brain injury (Dikmen et al., 1995). The effort included 436

adult head-injured patients, 243 of whom had a GCS of 13-15 on admission. In this

study, she indicated that the extent of neuropsychological impairment observed was

directly related to the duration of coma associated with the injury. In the group that she

labeled as 'mild head injury' (those with coma of duration less than 1 hour and GCS of

13-15 on admission), she found that there was no statistically significant difference from

a trauma control group in neuropsychological performance. She suggests that the results

are similar to her previous study of neuropsychological outcome in minor head injury

with respect to the finding that measures of attention and short-term memory show a

trend toward persisting difference in those with minor head injury. In her discussion of

results, she stressed the fact that measures of attention and memory tend to show decline








with increasing severity of head injury. Again, she had no baseline comparative

measures.

Dikmen's studies of the neuropsychological functioning of those suffering minor

head injury have taken place at relatively long periods after the injurious incident (1

month and 1 year), and have shown limited persisting differences between the injured

group and the control population. Likewise, a meta-analytic review by Binder et al.

(1997) using 8 studies comprised of 11 samples of individuals with a history of mild head

trauma at least 3 months after receiving their injury resulted in an overall nonsignificant

effect size of .07 when comparing a composite neuropsychological test score to control

groups. However, when they used the d statistic, they found a significant finding of. 12,

p < .03 for measures of attention. It was their conclusion then, much like Dikmen's, that

the neuropsychological effects, if any, of minor head injury are generally resolved from 1

to 3 months post-injury.

In a study by Montgomery et al. (1991) of 26 consecutive admissions to an

accident and emergency room for minor head injury (defined by this group as an injury

requiring an overnight inpatient hospital stay as well as a PTA period of less than 12

hours), investigators utilized a Four-Choice Reaction-Time measurement in order to

detect possible cognitive sequelae of the injury. In addition, a measurement of brainstem

evoked potential and subjective measure of post concussive complaints were performed

on the same day as the injury, six weeks later, and at six months post-injury. Fenton

reported that head-injured patients exhibited a significantly prolonged reaction time

measurement on the day of the injury when compared to a control group matched for age,

sex, and social status. Results indicated "serial improvement" at the six-week and six-








month follow-up. Interestingly, over half of the patient's involved in this study still

complained of post-concussion syndrome symptoms at the six-week and six-month

follow-up assessments. In addition, more than half of these patients exhibited what

Fenton called "significant delays" in brainstem conduction during the first testing session.

Other studies examining the cognitive sequelae of minor head injury have shown

that more significant differences are detectable in the acute and early recovery phases of

head injury and are often associated with the degree of persisting post concussional

symptoms. This is particularly true in investigations examining the effects of minor head

injuries incurred in contact sport related incidents, which usually involve lesser amounts

of applied forces (acceleration and deceleration) and generally result in milder concussive

effects (Maddocks and Saling, 1996).

Sports-related Minor Head Injury

The interest in sports related head injuries in the head injury literature has a long

history, though relatively little methodologically sound research has been conducted in

the area until recently. A 1991 report by the National Health Interview Survey estimated

that approximately 300,000 people in the United States suffered a head injury as a direct

result of participating in a sporting activity. Of these, 34% (approximately 103,000) were

categorized as "mild" (Kelly and Rosenberg, 1998; Thurman et al., 1998). In addition,

the survey contended that many of those individuals suffering head injuries never sought

medical treatment, and therefore many estimates of national head injury prevalence may

be significantly below the actual number. Until the last 10 to 15 years, however, most

investigation into the cognitive effects of sports-related head injury had been isolated or








anecdotal, and some investigations continue to employ methods that bring the validity of

findings into question.

In an article published in the Journal of the American Medical Association in

1928, Martland described the symptoms that he observed in boxers after receiving

repeated blows to the head. He termed the condition "punch drunk" and described the

symptoms as including a slight unsteadiness in gait or uncertainty in equilibrium, slight

mental confusion, and distinct slowing of muscular action. While he noted that most of

these cases remained relatively minor, some resulted in more severe neurological

complication including hesitancy in speech, tremors of the hands, and nodding

movements of the head (Martland, 1928). In addition, he postulated that the effects of

repeated blow to the head might cause lasting damage. While he indicated that he had to

rely in some cases on observation garnered from "laymen", he suggested that he could

identify many individuals known to suffer from persisting symptoms thought to be

related to repeated blows to the head. He was unable to empirically evaluate many of

them, he stated, as they were housed at the time in asylums across the country.

Further corroboration of this syndrome was provided by Millspaugh (1937) in his

review of the history of naval boxers. Millspaugh stated conclusively that there are

indeed cognitive sequelae to the repeated blows received in a boxing match, although the

cognitive deficits he described were of an objective nature. He also described a complex

of symptoms acutely accompanying repeated blow to the head, which included "fullness

in the head, cephalgia, vertigo, irritability, insomnia, increased fatiguability, and memory

defects" (Millspaugh, 1937). In addition, he described what appeared to him to be a

period of amnesia in both those suffering loss of consciousness during the fight, and








those without LOC. He took this to indicate that there was some physical damage to the

brain itself which may manifest for long periods of time. In his concluding remarks,

Millspaugh states: "repeated and frequent concussions, occasionally very severe, often

undoubtedly associated with intracranial capillary hemorrhages are to say the least not

conducive to stabilized mental equilibrium."

In a study by Tysvaer et al. (1989), 37 former professional soccer players were

examined in order to determine the existence of persisting neuropsychological deficits

caused by heading a soccer ball. The players ranged in age from 35 to 64, and had no

reported history of alcohol abuse. Investigators interviewed the players and used the

Wechsler Adult Intelligence Scale (WAIS), the Trail Making Test (parts A and B), a

modified version of the Halstead-Reitan aphasia screening test, tests of sensory-

perceptual functioning, motor tests, and the Benton visual retention test, Form C, in order

to determine neuropsychological status. Their scores on these tests were compared to a

control group consisting of 20 hospitalized patients who manifested sings of a variety of

disorders. These researchers found that the former soccer players exhibited a

significantly larger mean split between Performance IQ (PIQ) and Verbal IQ (VIQ) on

the WAIS, and significantly worse scores on both Trail Making Tests A and B. They

concluded that their results suggested that brain injury incurred as a result of soccer

injuries can result in permanent brain injury as exhibited by performance on

neuropsychological tests measuring perception, learning, manual dexterity, speed, and

attention. While these results may be suggestive of persisting cognitive deficits as a

result of sports-related head injury, the methodology is fraught with problems. Most

importantly, the authors do not report the neurologic history of any of the players tested.








Therefore, it is impossible to distinguish the extent of head injuries any one individual

may have suffered. In addition, they do not appear to use the time since the supposed

injury as a covariate in examining test performance, nor do they provide a history of other

factors that may play a role in affecting cognitive functioning. Further, researchers

reported no knowledge of cognitive functioning before these individuals played soccer.

Finally, the researchers report that the control group used was one consisting of

hospitalized patients with "varying" injuries. Without knowledge of the history of the

group, it would appear that it is an inappropriate normative comparison.

The recent examination of minor head injury incurred in sporting events has

several advantages over other scenarios in which the state of the individual at the time of

the accident and observation of their immediate behavior is not possible (Collins et al.,

1999). Within the realm of organized sporting events involving physical contact between

players, the opportunity exists to assess injury very soon after the incident because of the

immediate availability of medical staff. In addition, there is often a means of reviewing

the incident via television recording. This affords the opportunity to assess the types of

forces applied to the head of the injured player. Further, some investigators now collect

baseline measures ofneuropsychological functioning before a player can compete in the

contact sport. This allows for comparison of findings subsequent to the injurious incident

and more accurate analysis of subsequent cognitive changes following head injury.

A prospective study by Barth et al. (1989) of the cognitive effects of minor head

injury in a college football population suggested that neuropsychological testing is a

sensitive device in the detection of subtle cognitive changes following concussion. This

group collected baseline measures for 2,350 college football players at a total of 10








universities. Subsequently, they retested 182 players with documented cases of

concussion within 24 hours of the injury, 5 and 10 days later. The battery used in this

investigation consisted of Trail Making Tests A and B, the Symbol Digit Test, and the

PASAT. As a comparison group, Barth used a population of 59 players sustaining "mild

orthopedic injuries and 48 male college students" (Barth et al., 1989). These individuals

were tested with the same protocol using the same time period as the concussed players.

Barth concluded that while the control group performed significantly better on the

"within 24 hour" PASAT and Symbol Digit test, constituting what he termed "normal

testing behavior," (exhibiting practice effects) the concussed group failed to show

improvement on these two measures within 24 hours of receiving a concussion.

However, he noted that these individuals tended to show the same improvement as

controls on the day 5 tests of these measures with a leveling at day 10, which suggested

to Barth that these individuals had returned to baseline functioning at that point. The

investigators failed to show the same pattern of results on the Trail Making Tests, which

Barth suggested may be due to the strong effects of practice inherent to its successive

administration. As a result of this study, Barth concluded that a single incident of mild

concussion in a college football player results in detectable neuropsychological changes

within 24 hours of the incident, and that a recovery to baseline functioning occurs quickly

within the 5 to 10 days following injury.

In a study of 130 Australian Rules Football players by Maddocks and Saling

(1996), baseline measures of cognitive function were collected using the Paced Auditory

Serial Addition Test (PASAT), the Digit Symbol Substitution Test (DSST), and Four-

Choice Reaction time, which involves measures of decision and movement times. Of the








130 with baseline cognitive measures, Maddocks reported on 10 players who

subsequently received concussion (5 were described as incurring no LOC, 5 had LOC of

less than 1 minute and PTA of less than 30 minutes) and were re-tested at 5 days post-

injury. He indicated that while the neurological features of concussion, including

headache, nausea and dizziness, had remitted after 5 days, persisting neuropsychological

deficits were noted on the DSST and Four Choice Reaction time (decision time). He

noted that significant differences were not found on the PASAT, a test that previous

authors had found to be particularly sensitive to the effects of minor head injury due to

the attentional component inherent to performing the task. Maddocks suggested that

since his population of concussed individuals presented with milder forms of minor head

injury than most minor head injury populations examined, the sensitivity of the test may

be less effective. Of importance, Maddocks utilized a control group consisting of age-

matched umpires who were tested on two separate occasions with the same test measures.

Although this may allow for comparison with the head injured population taking into

account the effect of practice on the various measures, Maddocks fails to elaborate on

other factors that may have made this an inappropriate comparison group (e.g., history of

past head injury, inter-test duration, educational history, etc.).

In another study by Hinton-Bayre et al. (1997), 54 players from a professional

rugby team were tested at baseline using the Symbol Digit Modalities Test (SDMT), the

Digit Symbol Substitution Test (DSST), and the Speed of Comprehension Test. Ten

players subsequently received concussions and were tested within 24-48 hours of the

incident. The authors reported that those measures of speed of information processing

were sensitive to the cognitive effects of the concussion, though an untimed test of word








recognition was not. They further stated that tests of speed of information processing

were more sensitive than either the SDMT or the DSST based on comparison with

baseline measures.

In a 1998 study by McCrea et al., researchers administered the Standardized

Assessment of Concussion (SAC) to 568 high school and college football players before

the 1995 and 1996 seasons. The SAC consists of a brief measure of orientation, short-

term delayed verbal memory (5 words recalled immediately and after a 5 minute delay), a

brief measure of attention and shot-term memory (reverse Digit Span), and concentration

(recitation of the months of the year in reverse order), as well as several exertional

exercise in order to create conditions of intracranial pressure during which post

concussive symptoms are most likely to be seen. The sum of the scores is added to

produce a composite total with a maximum score of 30. Of the 568 individuals tested at

baseline, 33 subsequently experienced concussion and were tested immediately. 28 of

the 33 were also tested 2 days after the injury occurred. McCrea found that the

concussed players performed significantly below both the control group and their own

baseline composite measures on the SAC. At 48 hours, he indicated the concussed

players had returned to their baseline level. He concluded that this brief battery of

modified neuropsychological measures is sensitive to the cognitive deficits obtained from

minor head injury incurred in football, and suggests that his observation that all

concussed players were not different from baseline at 48 hours is consistent with other

findings. Of note, however, was the lack of attention to the effects of practice, and the

possibility that they may have contributed to improvements seen at the follow-up testing,

thus potentially masking any persistent cognitive effects of concussion. Further, the lack








of matched controls excludes the researchers from making comment on the potential

effects of age and education level on results obtained, although they contend that these

variables have "minimal" effect on performance and that there were no significant

differences between the scores of high school and college players.

A more recent test-retest study by Barr and McCrea (2001) described the results

of the administration of the SAC to concussed players immediately following the injury

compared to their baseline performance. A control group of non-concussed players was

also tested at the same time intervals. These researcher found that using both a reliable

change index approach and a linear regression approach were successful at identifying

detriments in cognitive performance immediately following concussion. Receiver

operating characteristic curves were used to determine the statistical significance of the

models in accurately classifying a player as concussed or not concussed.

A study by Collins et al. (1999) tested 393 athletes from 4 university football

teams using a brief neuropsychological battery that included the Hopkins Verbal

Learning Test (HVLT, verbal learning, and delayed memory), Trail Making Tests A and

B, Digit Span test, Symbol Digit Modalities Test, Grooved Pegboard Test, and Controlled

Oral Word Association. In addition the athletes completed the Concussion Symptom

Scale, a self-report, Likert scale of post concussion symptoms. Subsequently, the group

retested 16 individuals who suffered concussion within 24 hours of the incident, and

within 3, 5, and 7 days afterwards. Collins concluded, after performing a discriminant

analysis on the resulting data, that the use of this brief neuropsychological battery

resulted in an overall correct classification rate of 89.5% for the concussed and control

groups at the 24-hour testing period. In particular, he noted that concussed individuals








were significantly worse on the Hopkins Total score and Hopkins Delay score than the

controls, and that moderate differences between the groups persisted until the 5-day

testing. While not included in the analysis of concussed players, Collins, et al. analyzed

the baseline scores of all 393 athletes and discovered an interaction between history of

diagnosed learning disability and history of previous concussion, and significantly lower

baseline performance on two measures (Trail Making Test B and Digit Symbol

Modalities Test). This finding suggests that these demographic variables may be

important to examine and take into account when evaluating the neuropsychological

performance of an athlete after concussion.

Reliable Change

The tests of cognitive change and recovery of function described above generally

have utilized one of two methods to indicate recovery after concussion and fitness to

return to play. One assesses a player's return to baseline functioning (Barth et al., 1989;

McCrea et al., 1998), and the other a comparison to a control group involving the

determination of differences between serial testing results (Collins et al.,1999; Maddocks

and Saling, 1996). Hinton-Bayre et al. (1999) suggested that the former does not

properly account for the effects of practice, while the latter provides limited information

without the availability of "suitable" normative data. The results of neuropsychological

testing are influenced by many factors in addition to the cognitive effects of minor head

injury. Especially in repeated, or serial testing, as is performed in the evaluation of

recovery of a concussed athlete in order to determine return-to-play status, the results of

testing may be influenced by such things as the reliability of the test being used, practice

effects, and the attitude of the test taker (Temkin et al., 1999). Even when there is no true








change in cognitive status, these variables may suggest that a change has taken place. In

this sense, the ability to detect true change in an individual case can be affected by many

factors. Temkin et al. (1999) examined four statistical models in order to determine

which was most sensitive to detecting "real change" in the individual case. They used 7

test measures and retest scores from 384 "neurologically stable" individuals in the

comparison of four statistical models in order to address this question. These models

were the basic Reliable Change Index (RCI), the RCI with correction for practice effects,

a simple regression model, and a multiple regression model. Each of these models was

used in order to generate a predicted retest score which was then compared to the actual

retest score. The difference between the two, or residual, was then compared to a

confidence interval that was calculated to contain 90% of those scores expected given no

real change. This method was based on Za=Z.95=l.645. This value suggests that 5% of

scores were thought to be significantly higher than the normal population, while 5% were

assumed to be significantly lower. In other words, any residual score lying outside this

interval was assumed to reflect real change.

Reliable Change Index

The Reliable Change Index was originally developed to assess the effectiveness

of clinical interventions (Hinton-Bayre et al., 1999). It was used by Jacobson and Truax

(1991) with the idea that clinically significant change was related to a return to normal

functioning. The model is based on the assumption that a change in test scores from one

time to the next is significant if the size of that change is large compared to the standard

error of the difference of the test. Jacobson and Truax (1991) indicate that the error

variance of a test is calculated through the use of test-retest reliability and the variation of








scores around the mean. According to Hinton-Bayre, the formula for obtaining an

individual's RCI score is:

RCI= (x2 1)
Sdff

In this instance, xl is the individual's first score, x2 is the retest score and sdiff is the

standard error of the difference, which is calculated as follows:

Sdf = 2(s 2)

The standard error of the measurement, SE is obtained through the use of the standard

deviation of the control group, S. In addition, the reliability of the measure, r12, is used in

obtaining this statistic:

SE = S1- r2 (Hinton-Bayre et al., 1999)

Again, in the use of this statistic, an RCI z value exceeding 1.96 is considered to be a

significant change.

In the analysis of Temkin et al. (1999), a change was considered to be reliable,

using the RCI, if the absolute value of the post-test minus the pre-test exceeded the

standard error of the test-retest difference of the normative sample multiplied by the z-

score cutoff described above. This created a prediction interval that should contain 90%

of the scores in a normal population. Any change score lying outside of this interval in

either direction was considered "real" change.

Reliable Change Index with Correction for Practice

The second model used by Temkin et al. (1999) was a slight variation on the RCI

model described above. This model has been used by others in the evaluation of

cognitive outcome following various surgical procedures (Chelune et al., 1993; Hermann








et al., 1996; Kneebone et al., 1998; Sawrie et al, 1996). In this model, practice effects for

individual tests are determined by calculating the mean difference between the test and

retest scores of the normative population. This mean difference is then added to the

baseline score of the individual in question in order to calculate a predicted follow-up

score. A reliable change is then determined by the amount of difference between the

actual retest score and the predicted. A confidence interval is determined just as it was

for the basic RCI. That is, the confidence interval is determined by the standard error of

the difference of the normative sample multiplied by the z-score that determines the

cutoff point (generally Z.95).

Linear Regression

The third model utilized by Temkin et al. (1999) was a basic linear regression. A

formula that can be used for predicting any score in the population under investigation is

calculated by regressing the retest score of the normative sample on their initial scores.

Originally suggested by McSweeny et al. (1993), Temkin suggests that this model

controls for both practice effects and regression toward the mean. Reliable change is

determined again by examining the difference between the observed retest score and the

predicted score. A prediction interval is calculated based on the residuals estimated by

the equation multiplied by a z-cutoff score (again, generally Z.95).

Multiple Regression Model

The final model investigated by Temkin et al. (1999) incorporated the use of

stepwise linear regression in order to determine a predicted retest score. Multiple

measures were used in the stepwise regression which the investigators thought might be

important in determining follow-up scores. These included baseline test score, test-retest








interval, demographic variables, and a measure of overall neuropsychological

competence. In order to account for nonlinear relationships between test and retest score,

these investigators also included the square and cube of baseline scores in their multiple

regression models for each measure. Temkin explains that variables were added to the

stepwise linear regression if they had the "highest partial correlation among the variables

not already in the equation, and if the significance level associated with the variable was

under .05." Variables were removed from the equation if, after adding another variable,

the significance was above .10.

The results of Temkin et al. (1999) suggest that the prediction intervals associated

with the four models became narrower, and therefore more sensitive to the detection of

change, with the growing complexity of the methods used. In this instance, the term

"complexity" does not necessarily refer to the difficulty of mathematical operations, but

to the amount of error variance accounted for (e.g., test-retest reliability, practice effects,

regression to the mean, etc.). That is, the RCI was less sensitive than the RCI with

practice effects, which was less sensitive than the linear regression, which in turn was

less sensitive than the multiple regression model. However, they indicate that the RCI

produced high error rates due to the non-normality of the population residuals. This was

in large part due to the practice effects seen in the individual measures, which caused the

population data to become somewhat skewed. When practice effects were taken into

account, the residuals better estimated a normal curve, and predicted scores were more

closely correlated with observed scores. They report that overall prediction accuracy was

similar for the last three models, though, with the addition of demographic variables in

the multiple regression model, significantly better prediction was possible.








Specific Aims and Hypotheses

Given the magnitude of the occurrence of head injury in the United States, and the

subsequent cognitive as well as psychosocial consequences inherent to it, the importance

of detecting neuropsychological deficits is clear. The literature suggests that minor head

injury in sports, a particularly common occurrence in a college football population, varies

significantly from person to person, and that subtle deficits may be indicative of the

lingering effects of concussion. The assessment of recovery from concussion is

particularly important when one considers the potential effects of repeated concussion

and its long-term consequences (Cantu and Voy, 1995; McCrory and Berkovic, 1998).

As Collins et al. (1999) have indicated, the possibility of a correlation between multiple

concussion and reduced baseline neuropsychological functioning make the assessment of

concussion a particularly salient issue in the student athlete population.

Studies on the effects of minor head injury to this point have produced varying,

often contradictory results. This is in no small part due to the variability of the

methodology utilized. In many cases, investigators have used inappropriate normative

samples or comparison groups, while in others, factors contributing to change (or lack

thereof) in serial testing have not been taken into account. More sensitive and sound

statistical measures are needed in the assessment of head injury and the period of

recovery after head injury.

The present study is designed to further the research on the sensitivity of

neuropsychological measures over serial testing sessions in assessing the functioning of

individuals suffering minor head injury in a college football population. The aims of this

investigation are (1) to analyze the effectiveness of various statistical models in detecting








ongoing cognitive sequelae in concussed college football players via serial

neuropsychological testing, (2) to determine appropriate thresholds for test performance

resulting in acceptable sensitivity and specificity classification values, (3) to determine

those functions which appear most indicative of enduring cognitive complaints over the

7-day period following concussion, and (4) to calculate representative models in order to

aid in prediction of cognitive recovery following concussion. It is hypothesized that each

progressively "complex" model will exhibit increasing sensitivity in terms of

demonstrating lower test performance relative to baseline at the retest session. It is also

hypothesized, based on the findings of Temkin et al. (1999), that only the most complex

models (i.e., regression models) will allow for accurate classification of concussed

individuals beyond the 24-hour test session. As was the case in the Temking et al. (1999)

study, the term "complex" is used here to denote the level to which potential error

variance is taken into account. Therefore, in relation to the current study, it is

hypothesized that those models which account for more potential error variance will be

more successful at detecting real post-concussive cognitive decline at the more distal

testing sessions.















CHAPTER TWO
METHODS

Subjects

This study included the subjects that were originally reported by Collins et al.

(1999) as well as others from a continuing data collection project at the University of

Florida. Participants included male football players from four universities: the University

of Florida, Michigan State University, the University of Pittsburgh, and the University of

Utah. All participants in the study were volunteers who agreed to participate in the study

without compensation and without financial responsibility. All participants on the

football teams of these respective schools were asked to participate.

In the initial meeting with the student athlete, baseline data was collected

regarding age, football position, handedness, college entrance exam scores (SAT and/or

ACT), high school grade point average, highest level of education achieved by mother

and father, occupation of mother and father, history of diagnosed learning disability

(LD), history of diagnose attention deficit-hyperactivity disorder (ADHD), history of

neurological problems (e.g., epilepsy, migraine headaches, etc.), history of significant

medical illness (e.g., asthma, diabetes, heart problems, etc.), history of psychological

illness (e.g., depression, anxiety, etc.), history of alcohol and substance abuse, and history

of recent medication use. In addition, each participant was asked to provide a history of

previous concussion (including date, description of circumstances surrounding the

incident, incidence of loss of consciousness, incidence of post concussive symptoms,








length of symptom presentation, and the results of imaging tests if any). All concussions

were subsequently classified according the grading system outlined by the American

Academy of Neurology (1997).

Protocol

Baseline

Approval by the Institutional Review Boards for each respective school was

obtained. Each participant was provided the opportunity to review the nature, purpose,

and projected use of the data with either a Ph.D. psychologist or a doctoral level student.

All data were collected by Ph.D.-level psychologists, doctoral students, or in some

instances, team physicians and athletic trainers who had been trained in administering

this brief battery. Each had attended a training session in test administration in order to

ensure standardized administration techniques. Tests were scored in a standardized

fashion, and the results from other institutions were sent to this investigator for entry into

a centralized database.

Each participant was administered a brief neuropsychological battery including

the following tests: Hopkins Verbal Learning Test (HVLT), Trail Making Tests A and B,

the Digit Span subtest of the Wechsler Adult Intelligence Scale-Revised, Symbol Digit

Modalities Test (DSMT), Grooved Pegboard, and the Controlled Oral Word Association

test (COWA). This battery is one that is currently used in a professional football study

on the effects of concussion and was recommended by Lovell and Collins (1998) for

study of concussion in a college football population.

The Hopkins Verbal Learning Test (Brandt, 1991) consists of a presentation of 12

words belonging to three semantic categories. Three learning trials are administered,








after which a delay period is interjected for approximately 15-20 minutes. The subject is

then asked to recall as many words as they can remember. After this delayed free recall,

a 24-word recognition list is presented which contains all of the target words in addition

to 6 semantically related words and 6 unrelated words. As the words are read to the

subject, they are simply asked to identify those words that were presented previously.

This test allows for a measure of short-term verbal memory, attention, concentration, and

learning. In addition, a measure of delayed verbal memory is calculated as well as a

score for recognition memory. In addition, within the recognition trial, scores for related

false positive responses and unrelated false positives are calculated. A discrimination

score is also calculated to determine the subject's ability to differentiate between

previously presented words and foils. The HVLT has six alternate forms (Lezak, 1995).

For the purposes of this study, Hopkins Total score and Hopkins Delay score were

examined.

Trail Making Test A (Reitan and Wolfson, 1985) consists of 25 numbered circles.

The subject is asked to connect the numbered circles in the correct order (from 1 to 25) as

quickly as they can, without making any mistakes or lifting their pencil from the paper.

Score on this measure is recorded in seconds to completion. Errors during the

completion of the task are immediately pointed out to the subject and they are directed to

continue from the point of their last correct mark. No score is kept for the number of

errors committed; it is felt that this is reflected in an increased time of completion. Trail

Making Test A is thought to be a measure of visual scanning and sequencing ability as

well as motor speed. Trail Making Test B, (Reitan and Wolfson, 1985) while similar in

appearance, encompasses circles with both letters and numbers. The subject is asked to








start with the number 1 and connect the circles in alternating increasing sequence (i.e., 1-

A-2-B-3-C, etc.). The score for this measure too is recorded in seconds to completion.

This measure is thought to reflect a person's ability to effectively shift mental set and

utilize cognitive flexibility in addition to visual scanning, sequencing, and motor speed.

These functions are often associated with frontal-executive functioning (Spreen and

Strauss, 1998).

The Digit Span subtest of the WAIS-III (Wechsler, 1997) consists of the

presentation of increasingly long sequences of numbers. In the Forward condition, the

subject is simply asked to listen to the numbers and repeat them in the same order in

which they were given. In the reverse condition, they are asked to recite them

backwards. Each part is discontinued when the subject fails to correctly recall two trials

of numbers of the same length. This measure is thought to reflect attention and

concentration. For the purposes of this study, the Digit Span total score (forward plus

back) was examined.

The Symbol Digit Modalities Test (SDMT) (Smith, 1982) is similar to the Digit

Symbol subtest of the WAIS-R in that is requires the subject to provide quick

substitutions, but it in this measure, the subject is asked to write the correct number under

each symbol based on a key. Subjects are given 90 second to complete as many items as

they can. This test was chosen based on its ability to measure visual scanning and visuo-

motor speed. Immediately following its administration, the subject is asked to recall as

many of these associations as they can, which provides a measure of incidental learning.

The variable examined as part of this study was total number of symbols correctly

produced in the 90 second test administration.








The Grooved Pegboard test (Matthews and Klove, 1964) consists of a small board

containing a 5x5 matrix of slotted holes arranged in various angles. The subject is given

25 pegs, each the same shape, and asked to put them in successive slots starting with the

top row as quickly as they can without dropping. They are allowed only to hold one peg

at a time with one hand. Time to completely fill the holes are measured for each hand

independently (Lezak, 1995). This test was chosen for inclusion in the battery due to its

sensitivity to motor speed and coordination. Dominant hand and non-dominant hand

completion times were examined in the current study.

The Controlled Oral Word Association Test (COWA) (Benton and Hamsher,

1989) consists of three trials. In each, the subject is presented with a letter and asked to

produce as many words as they can until the examiner asks them to stop. They are told

that they must not use proper nouns or the same words with different endings. The

subject is given 60 seconds to produce as many words as they can. This measure was

chosen for its ability to reflect abilities of word fluency and retrieval. The current study

utilized total words produced over all three trials as the variable under examination.

In addition to these neuropsychological tests, the participant were administered

the Concussion Symptom Scale in order to examine baseline levels of symptoms

commonly associated with post concussion syndrome (e.g., headache, memory

difficulties, concentration difficulties, sensitivity to light or noise, etc.). This is a

subjective questionnaire utilizing a Likert-type scale.

In-season concussion evaluation

Those athletes who sustained a concussion during the football season, either in

practice or in a game, were evaluated with this same battery (utilizing alternate forms








where available) within 24 hours of the incident, and at 3, 5, and 7 days post-concussion.

In-season concussion was initially evaluated by trainers or team physicians and was then

referred for neuropsychological testing as part of this study. Grade of concussion was

evaluated based on the AAN guidelines. In addition, within 24 hours of concussion,

circumstances contributing to the concussion were obtained through interview with

trainers, and videotape of the incident (if available) was reviewed in some cases.

Football players within each respective team served as controls. They were

matched as closely as possible to the concussed player in terms of the following

variables: race, position, history of learning disability, and history of head injury.

Athletes were encouraged to continue with testing protocol over all test intervals.

However, some difficulty was encountered with attrition, especially in the control

population. Ultimately, data from participating individuals was excluded listwise. That

is, a player's data was utilized over each retest interval only if they generated data for

each corresponding test session. This resulted in a decreasing study population over each

successive test interval. At the 24-hour test session, total number of concussed players

who had data at both baseline and 24 hours was 21. The number of controls at the same

interval was 13. For the Day 3 test session, the total number of concussed players was 19

with 11 controls. At Day 5 the number of concussed players was 18 with 10 controls. At

Day 7, there were 19 concussed and 6 control participants.

Data Analysis

Data analysis included the application of 7 statistical models of significance

detection. These were conceptually divided into three broad categories based on their

statistical methodology.








Basic Models

The first subgroup consisted of two basic, clinically-oriented approaches (Model

1 and Model 2). For Model 1, each of the concussed and control participants' retest

scores for each measure was compared to their respective normative mean scores and

standard deviations, which were derived from the larger baseline group of football

players (n=409), ultimately resulting in a standardized z-score. Significant findings were

classified using a standard clinical approach (Reitan and Wolfson, 1985), where 1

standard deviation below the mean was interpreted as denoting mild impairment. In

Model 2, each concussed and control participants' retest scores for each variable were

compared to their own baseline performance again using a criterion of 1 standard

deviation (based on the normative population) decrease in score to denote a finding

suggesting significant cognitive decline following concussion.

Reliable Change Indices

The second conceptual group of statistical models included three variants of the

Reliable Change Index (Model 3, Model 4, and Model 5). Model 3 involved the

application of a basic RCI analysis as described by Jacobson and Truax (1991) and used

in studies by Temkin et al. (1991) and Barr and McCrae (2001) to detect change in serial

neuropsychological testing. Model 4 utilized the application of the same RCI procedures

with correction for practice. For both Model 3 and Model 4, findings for each variable

were deemed significant if the retest score fell outside the computed change index in the

negative direction. For Model 5, each concussed participants' change on the individual

variables from baseline to retest interval was directly compared to the average change

score for the control population over the same time period. A significant finding was








denoted by a change score which fell below a 95% confidence interval based on the

standard deviation of change scores for the control group on each variable and test

interval.

Regression Models

The third conceptual subgroup of statistical models included two regression

approaches (Model 6 and Model 7). Consistent with the research ofTemkin et al. (1999)

and Barr and McCrea (2001), in Model 6, simple linear regression equations were

calculated, based on control population data, in order to predict retest scores for each

variable over the four test intervals. The observed scores were then compared to the

predicted scores and a standard z-score was calculated using the standard error of the

residuals for the model based on the control participants. Observed retest scores falling

beyond a significance value of 1.96 in the negative direction were classified as

significantly below expectation. In Model 7, a multiple regression equation approach

using history of diagnosed learning disability and history of previous concussion as

additional predictive indicators was analyzed.

Sensitivity and Specificity

Sensitivity values were calculated for each variable over the 4 test sessions for

each model. For individual variables, sensitivity was determined by that variable's

probability of accurately classifying a concussed player as being significantly below

expectation based on the applied statistical model.

Each model was then examined over all test session to determine how many

variables from the battery of neuropsychological tests as a whole were found to have

been significantly below expectation for both concussed and control participants. A








classification cut-point (the optimal number of significant variables below expectation)

was determined based on the highest degree of sensitivity/specificity combination at each

time interval, where sensitivity was determined by the number of concussed participants

who were accurately classified as concussed at each cut-point, and specificity was

determined by the number of control participants who were accurately classified as "not

concussed" at those same cut-points.

Classification accuracy was then assessed using Receiver Operating Characteristic

curves, which provided a statistical indication of each model's ability to accurately

distinguish between the concussed and nonconcussed participants relative to chance

(Hanley, 1989).

















CHAPTER THREE
RESULTS

Demographic Data for Baseline Population

During the current study, data was collected using the above-described pre-season

neuropsychological test battery, and demographic variables were collected from a total of

443 football players at four universities. The total number of pre-season participants

from each location is presented in Table 2, and is presented graphically in Figure 1.

Table 2. Numbers and percentage of total pre-season participants' locations

N Percent Cumulative Percent

University of Florida 155 35.2 35.2
Michigan State University 119 26.9 62.1
University of Utah 85 19.2 81.3
University of Pittsburgh 83 18.7 100.0

Location


Univ. of Pittsburgh
18 7o*


Univ of Florda
35 2%


Uni, ol Utan
19 2.


S Mich. St. Univ.
26.9%


Figure 1. Percent of baseline participants from each contributing school.













Table 3 and Figure 2 indicate the racial composition of the total number of baseline

participants. The majority of these were African American (47.9%), closely followed by


Caucasians (47.2%). There were significantly fewer Hispanic, Asian, and Samoan

participants.


Table 3. Racial composition of baseline population

N Percent Cumulative Percent


African American 212 47.9 47.9
Caucasian 209 47.2 95.0
Hispanic 6 1.4 96.4
Asian 2 0.5 96.8
Haitian 1 0.2 97.1
Samoan 13 2.9 100.0




RACE
Samoan
2.9%
Haitian
.2%
Asian
.5%
Hispanic
1.4%

7 African American
Caucaswar. 47.9%
472"




Figure 2. Racial composition of baseline population across sites.


Table 4 indicates the frequencies of ages of the baseline population. The mean age of

baseline population participants was 19.85 years with a standard deviation of 1.75 years.









Table 4. Distribution of ages across the baseline population
Age (years) N Percent Cumulative Percent

17 6 1.4 1.4
18 113 22.5 26.9
19 101 22.8 49.7
20 77 17.4 67.0
21 72 16.3 83.3
22 39 8.8 92.1
23 22 5.0 97.1
24 7 1.6 98.6
25 2 0.5 99.1
26 2 0.5 99.5
27 2 0.5 100.0


AGE
30



20







a. 0
17 18 19 20 21 22 23 24 25 26 27
Figure 3. Distribution of ages for baseline population.

Table 5 indicates the frequencies of year in school for all participants in the baseline

population.

Table 5. Distribution of class status for the baseline population
N Percent Cumulative Percent

Freshman 171 38.6 38.6
Sophomore 92 20.8 59.4
Junior 95 21.4 80.8
Senior 67 15.1 95.9
Fifth Year Senior 18 4.1 100.0








Demographic Characteristics for Concussion and Control Participants

As indicated above, in-season concussion data utilizing re-administration of the

baseline neuropsychological test battery was conducted for those individuals who were

reported by team trainers to have received a concussion either during practice or during a

game. Initial re-testing took place within 24-hours of the identified concussive event, at

three days, five days, and seven days post-concussion. During the time of the

investigation, trainer-identified concussive events resulted in a total of 26 concussed

players, including three players who incurred two concussions during the time of the

study. Table 6 indicates the demographic characteristics of the concussed players.

Table 6. Demographics characteristics of the concussed players
N Percent

Age
18 7 26.9
19 11 42.3
20 6 23.1
21 2 7.7

Race
African American 14 53.8
Caucasian 12 46.2
History ofLD
Yes 6 23.1
No 20 76.9

When a concussed player was identified by a trainer, a matched control was

assigned to each for purposes of neuropsychological test result comparison over time. As

outlined above, control subjects were matched as closely as possible to concussed

individuals in terms of race, history of diagnosed learning disability, history of

concussion, and position. However, control subjects' continued participation in the time-

sensitive research design was less than optimal. Ultimately, when control participants








failed to continue in the study, the time requirements did not allow for replacement with

another control participant. Subsequently, the total number of control participants was

less than the total number of concussed subjects, resulting in a total of 14 individuals.

Table 7 indicates the demographic characteristics of the control population.

Table 7. Demographic characteristics of the control population

N Percent

Age
18 6 42.9
19 4 28.6
20 1 7.1
21 3 21.4

Race
African American 7 50.0
Caucasian 7 50.0

History ofLD
Yes 2 14.3
No 12 85.7

Examination of the racial composition of the baseline population and the concussed

subjects indicates that both groups were nearly half African American and half

Caucasian.

A one-way ANOVA was performed to determine whether there were any

statistically significant differences between the baseline population (excluding those

players who eventually suffered concussion and their matched controls), the concussed

subjects, and their controls in terms of age, race, and year in school. Using status as the

between-subjects factor (baseline, concussed, control), no significant differences were

found between the groups in terms of racial composition [F(2,440) = 1.12, p=0.33], nor

were there any significant differences between the groups with regard to year in school








[F(2,440) = 2.97, p=0.052] although there was a trend for the concussed players to be of

lower school year. It was not surprising, therefore, that a significant effect for age was

observed [F(2,440) = 3.87, p=0.022]. However, paired t-tests using a Bonferroni

corrected p-value of 0.017 (.05/3) failed to demonstrate a significant difference between

the groups in terms of age. Again, a trend was observed for the concussed players to be

slightly younger than the baseline population as a whole, though this was not statistically

significant. For the purposes of this study, which examined baseline to retest session

performance (with 24 hours, day 3, day 5, and day 7), participants were excluded

listwise. That is, participants must have had a score at both test sessions for inclusion

into examination of performance for a particular test interval. This resulted in fewer

included participants for each successive time interval. Ultimately 21 concussed and 13

controls were included for the 24 hour analysis, 19 concussed and 11 controls for the Day

3 analysis, 18 concussed and 10 controls for Day 5, and 19 concussed and 6 controls for

Day 7.

Baseline Performance

As noted above, ten variables were examined from the battery of

neuropsychological tests administered during the current study. Those variables were

Hopkins total score, Trails A total time in seconds, Trails B total time in seconds,

Grooved Pegboard Test dominant and nondominant hand time in seconds, digit span

forward, digit span backward, Digit Symbol time in seconds, COWA total score, and

Hopkins Delay score. Table 8 shows the means and standard deviations for the total

Baseline population as well as the concussed participants and control subjects at baseline.








Table 8. Baseline Performance of total population, concussed, and controls
Total Baseline Controls Concussed
n=409 n=13 n=21

M (SD) M (SD) M (SD)
Measure

Hopkins Total 24.96 (4.02) 25.77 (3.44) 24.19 (4.13)
Hopkins Delay 8.35 (2.09) 8.85 (1.77) 7.71 (1.98)
Trails A 20.99 (5.96) 21.97 (6.17) 22.18 (5.53)
Trails B 52.84 (17.39) 45.95 (13.13) 57.83 (24.83)
Digits Forward 9.05 (2.18) 8.31 (2.14) 9.00 (1.89)
Digits Back 7.20 (2.48) 7.08 (2.53) 6.67 (2.29)
Digit Symbol 58.75 (9.07) 64.92 (8.03) 55.81 (8.98)
Pegs Dominant 66.56 (10.50) 65.48 (8.41) 70.73 (13.79)
Pegs Nondominant 73.34 (11.81) 72.72 (11.78) 79.62 (16.96)
COWA Total 39.08 (9.23) 35.77 (6.70) 41.32 (10.21)

A series of one-way ANOVA's was run in order to detect any significant

differences between the groups with regard to their baseline performance on any of these

ten measures. Table 9 provides a summary of those findings.

There was a significant finding with regard to baseline performance between the

three groups on the Digit Symbol test [F (2,440) = 4.14, p=0.02]. Subsequently, paired t-

tests were performed using a Bonferroni corrected significance alpha value of 0.017

(.05/3). Results indicated that the control subjects scored, on average, significantly

better than did the concussed participants (t = 2.99, p=0.013) at baseline. Further, there

was also a trend indicating that the control participants' scores were significantly better

than the control group at baseline, although this did not reach significance after applying

correction for multiple t-tests. However, because the difference between the concussed

group and control group scores on the Digit Symbol test at baseline was found to be

significant, this variable was omitted from further analysis.








Table 9. One-way ANOVA's using status as between-subjects factor
Measure df F Significance

Hopkins Total (2,440) 0.65 0.52
Hopkins Delay (2,440) 1.35 0.26
Trails A (2,440) 0.56 0.57
Trails B (2,440) 1.82 0.16
Digits Forward (2.440) 0.73 0.48
Digits Backward (2,440) 0.48 0.62
Digit Symbol (2,440) 4.14 0.02
Grooved Pegboard Dom (2,440) 1.63 0.19
Grooved Pegboard Nondom (2,440) 2.73 0.07
COWA Total (2,440) 1.40 0.25

Overview of Repeated Testing

Repeated testing utilizing the above-described neuropsychological battery

generally yielded results consistent with historical reviews of test-retest performance for

those having suffered minor head injury. That is, the concussed participants in the

current study, on average, exhibited a decline (or a stasis) in performance across test

variables within 24 hours of concussion when compared to their baseline performance.

Examination of subsequent performances on days 3, 5, and 7 days post concussive event

generally suggested a gradual improvement over these days. This is particularly evident

when comparing the graphical representation of these performances compared to the

control population's average performance over this same time interval. Figures 4 through

12 represent the average performance of individuals in both the concussed and control

groups over the testing period.

Detecting Significant Decline Using Basic Statistical Methods

Generally speaking, neuropsychological testing involves the examination of an

individual after a known or suspected neurological insult has occurred.











Hopkins Total Score


= in-season concussion

22 o_ control
Baseline 24 Hours Day 3 Day 5 Day 7

Testing Session

Figure 4. Hopkins Verbal Learning Test -Retest Performance for Both Groups


Trail Making Test A


--
S18-N in-season concussion

I 16 o control
Baseline 24 hours Day 3 Day 5 Day 7

Testing Session
Figure 5. Trail Making Test A Test-Retest Performance for Both Groups











Trail Making Test B


(D in-season concussion

S 30 o control
Baseline 24 Hours Day 3 Day 5 Day 7

Test Session

Figure 6. Trail Making Test B Test-Retest Performance for Both Groups


Digit Span Forward





,/\
/ '
/ "
/
/\



/
P/


8-
E in-season concussion

Z 7 ____ control
Baseline 24 Hours Day 3 Day 5 Day 7

Test Session

Figure 7. Digit Span Forward Test-Retest Performance for Both Groups










Digit Span Backward
9.5

0 9.0.







0
S8.5 n-season concussion
Z 8.0 control



Baseline 24 Hours Day 3 Day 5 Day 7
4-
Q 7.5
7.0

E 6.5 in-season concussion
Z 6.0.______._______,_________________ 0 control
Baseline 24 Hours Day 3 Day 5 Day 7

Test Session
Figure 8. Digit Span Backward Test-Retest Performance for Both Groups


Grooved Pegboard Dominant Hand


E 58 in-season concussion
0 ---
o 56 o control
Baseline 24 Hours Day 3 Day 5 Day 7

Test Session
Figure 9. Grooved Pegboard (Dominant Hand) Test-Retest Performance for Both Groups











Grooved Pegboard Nondominant Hand
90.
8-
80
C







o 70
0




E in-season concussion
0 80


0- --

0 60 1o control
Baseline 24 Hours Day 3 Day 5 Day 7

Test Session

Figure 10. Grooved Pegboard (Nondom. Hand) Test-Retest Performance for Both
Groups





o Controlled Oral Word Association

0
0.

48 \
/ \
S46 \
0

44-

42-

/ \
E \


6 38! "-----in-season concussion
0 o
1- 36 _control
Baseline 24 Hours Day 3 Day 5 Day 7


Test Session

Figure 11. COWA Test-Retest Performance for Both Groups










Hopkins Delay Score
11


10 -,

-0
o






7 in-season concussion
E
z 6 control
Baseline 24 Hours Day 3 Day 5 Day 7
Figure 12. Hopkins Delay Performance for Both Groups

In most instances, the examiner is without the benefit of prior knowledge regarding the

individual's baseline cognitive abilities. Therefore, certain assumptions are made with

regard to that person's premorbid cognitive abilities, and a comparison is made between

their status-post neuropsychological performance and some normative group that is

deemed appropriate. In many circumstances, an arbitrary cut-point is determined for a

neuropsychological measure, a performance under which (or significantly different from)

is considered to indicate abnormality. In this study, the group of more than 400 football

players who participated in the baseline study provided an appropriate comparison group.

Model 1

The most basic model (referred to as Model 1 for the purposes of this study) for

detecting if a concussed player's performance suggested ongoing concussive sequelae

was a comparison at each post-concussive test interval to the normative baseline

population mean score for that measure. Using a liberal criterion, a difference of greater

than 1.0 SD below that expected (normative mean) score would suggest a performance








below expectation, perhaps indicating ongoing concussive effects. Using that approach,

Table 10 indicates the cut-points for the 9 neuropsychological measures that were

studied. These were derived by determining a score for each particular measure which

was 1 SD worse than the mean.

Using these criteria, the number of concussed and control participants who were

classified as significantly below the cut-point was determined for each test session

(within 24 hours of concussion, 3 days, 5 days, and 7 days post-concussion). Table 11

indicates the sensitivity of each measure for each test session.

Table 10. Cut Points Using 1 SD Difference from Normative Population Mean (Model 1)

Measure Normative Mean Score Standard Deviation Cut-Point

Hopkins Total 24.96 4.02 20.94
Trails A (sec) 20.99 5.96 26.95
Trails B 52.84 17.39 70.23
Digit Span Forward 9.05 2.18 6.87
Digit Span Backward 7.20 2.47 4.73
Grooved Pegboard (D) 66.56 10.50 77.06
Grooved Pegboard (N) 73.34 11.81 88.15
COWAT 39.08 9.23 29.85
Hopkins Delay 8.35 2.09 6.26

As Table 11 indicates, within 24 hours of concussion, the most sensitive measure

appears to be total words recalled after the delay period for the Hopkins Verbal Learning

Test. However, none of the measures alone was particularly sensitive. Examining the

battery as a whole using Model 1, the number of concussed and control participants who

scored below the cut-points for each measure were totaled. Tables 12-15 show the

cumulative number for each group who were significantly different than the normative

mean group for all 9 variables examined.









Table 11. Sensitivity for Neuropsychological Measures for Model 1
Measure 24 Hours Day 3 Day 5 Day 7

Hopkins Total 0.29 0.11 0.17 0.05
Trails A 0.29 0.05 0.00 0.11
Trails B 0.19 0.11 0.06 0.00
Digit Span Forward 0.19 0.16 0.00 0.05
Digit Span Back 0.14 0.05 0.05 0.00
Grooved Pegboard D 0.14 0.16 0.06 0.05
Grooved Pegboard N 0.38 0.16 0.11 0.21
COWAT 0.19 0.05 0.11 0.00
Hopkins Delay 0.43 0.42 0.11 0.16

Table 12 demonstrates that, within 24 hours of concussion, using the criterion of

falling below the cut-points (derived using Model 1) for 2 or more variables achieved the

greatest (Se + Sp) value of 1.37. That is, 11 of the 21 concussed individuals fell below

the cut-point for 2 or more variables while only 2 of the 13 control participants did so at

that test session. However, as has been noted in the literature, in the case of sports

concussion it is likely better to err on the side of caution, and while using a criteria of

only one variable below significance value resulted in a greater number of false positive

errors (Sp=0.38), it also resulted in a far better true positive rate (Se = 0.91).

Table 12. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables Within 24 Hours with Model 1

No. of Sig.Measures Concussed Control Se Sp (Se + Sp)
0 21 13 1.0 0.0 (1.00)
1 19 8 0.91 0.38 (1.29)
2 11 2 0.52 0.85 (1.37)
3 8 0 0.38 1.00 (1.38)
4 3 0 0.14 1.00 (1.14)
5 2 0 0.10 1.00 (1.10)
6 2 0 0.10 1.00 (1.10)
7 1 0 0.05 1.00 (1.05)
8 1 0 0.05 1.00 (1.05)
9 1 0 0.05 1.00 (1.05)
Se=Sensitivity, Sp=Specificity








Again, Table 13 demonstrates that the highest combined sensitivity and

specificity rates were achieved at Day 3 using a criteria of 3 or more variables falling

below their respective cut-points (Se + Sp = 1.12). However, sensitivity is extremely

negatively affected by the more conservative inclusion criteria. This more conservative

criterion, however, also resulted in a significantly higher specificity rate, demonstrating

the need for a pre-determined tolerance for higher false-positive rates when attempting to

detect ongoing effects of concussion.

Table 13. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 1
No. of Measures Concussed Control Se Sp (Se + Sp)

0 19 11 1.00 0.0 (1.00)
1 10 5 0.53 0.55 (1.08)
2 6 3 0.32 0.73 (1.05)
3 4 1 0.21 0.91 (1.12)
4 2 0 0.11 1.00 (1.11)
5 1 0 0.05 1.00 (1.05)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

At day 7 (Table 15), the highest combined sensitivity and specificity score was achieved

using a criterion of one variable below cut-point (1.25).

Certainly, the sensitivity and specificity values using a comparison of

performance at these time intervals to a normative population suggests that the model

itself has some value within 24 hours of concussion. Although the model is not an

optimal one, it was able to detect a degree of performance aberrance after concussion

compared to the appropriate standard scores, though specificity was conversely poor.








Receiver operating characteristic curves were calculated for this model using Analyse-It

software for Excel at each of the time interval and are shown in Figures 13-16.

Table 14. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 1
No. of Measures Concussed Control Se Sp (Se + Sp)

0 18 10 1.00 0.0 (1.00)
1 8 5 0.44 0.50 (0.94)
2 3 1 0.17 0.90 (1.07)
3 0 1 0.00 0.90 (0.90)
4 0 1 0.00 0.90 (0.90)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 15. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 1
No. of Measures Concussed Control Se Sp (Se + Sp)

0 19 6 1.00 0.0 (1.00)
1 8 1 0.42 0.83 (1.25)
2 4 0 0.21 1.00 (1.21)
3 0 0 0.00 1.00 (1.00)
4 0 0 0.00 1.00 (1.00)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Figure 13 shows the receiver operating characteristic curve for the classification

rate for Model 1 at 24 hours. The diagonal line represents chance level, and by

definition, the area underneath it is equal to 0.50. The area underneath the curve for

Model 1 at 24 hours was 0.773 (95% Confidence Interval = .62-.93; SE = 0.08) and was

found to be statistically different than chance (p=0.0003). Using generally accepted









accuracy classification, this model at 24 hours after concussion would be characterized

only as fair.


ROC Curve for Model 1 at 24 Hours


0.9
S0.8
0.7
0 No discrimination

0.4
S0.6 / -0-Model 1
f0.5 -

0.3
d& 0.2
0.1

0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)


Figure 13. ROC Curve for Model 1 at 24 Hours

At Day 3, detection of significant scores on the test variable compared to the baseline

normative population was not better than chance using Model 1 (Figure 14). The area

under the curve (AUC) was calculated as 0.56 (95% Confidence Interval = .35-.77; SE =

0.11) and was not statistically different than the reference diagonal (p=0.30). A similar

pattern was observed at Day 5 as can be seen in Figure 15. Using Model 1, the detection

rate was not statistically difference than chance. The AUC was found to be 0.517 (95%

Confidence Interval = 0.29-0.74; SE = 0.12).

The calculated AUC for Model 1 at day 7 was 0.65 (95% Confidence Interval =

0.41-0.88; SE = 0.12), and again, was not statistically better than chance (p=0.11) at

differentiating between concussed participants and controls (Figure 16).











ROC Curve for Model 1 at Day 3


No discrimination
--o- mdell


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)


Figure 14. Receiver Operating Characteristic Curve for Model 1 at Day 3



ROC Curve for Model 1 at Day 5

1

0.9
0.8

S0.7
-No discrimination
S0.6 --0- model

S0.5

S0.4

S0.3-

I 0.2

0.1 <
/-

0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)


Figure 15. Receiver Operating Characteristic Curve for Model 1 at Day 5










ROC Curve for Model 1 at Day 7



0.9
0.8
I 0.7
No discrimination
o 0.6
6 --model 1
2 0.5
0.4
0.3-
0.2-
0.1 -

0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)


Figure 16. Receiver Operating Characteristic Curve for Model 1 at Day 7

Model 2

In the circumstance that one has baseline test measures for an individual,

comparison of their performance following neurological insult to that baseline

performance may be more appropriate in determining change in cognitive status during

subsequent testing sessions. In the second basic model examined in this study (here

referred to as Model 2), performance of concussed individuals at each testing session

following concussion was compared to their baseline performance. This was achieved by

determining significance based again on a liberal criterion of 1 SD (derived from the

normative population as a whole, n=409) below their baseline performance. In this

manner, each subject was afforded an individualized comparator as illustrated for one

concussed participant in Table 16.








Please note that those variables for which time is the recorded score, a cut-point

was determined by the addition of 1 SD to the baseline performance, and ordinal

variables were rounded to the nearest whole number. Therefore, as can be seen in Table

13, for this participant's time on Trails A to be classified as significantly different at 24

hours than baseline, his score would have to increase by nearly 6 seconds, from 33" to

nearly 39" total time.

Table 16. Example of Cut-Point Determination for Study Participants
Measure Baseline Performance 1 SD(from normative pop.) Cut-Point

Hopkins Total 20 4.02 16
Trails A 33" 5.96" 38.96"
Trails B 68" 17.39" 85.39"
Digits For. 12 2.18 10
Digits Back. 6 2.48 4
Pegs Dominant 73" 10.50" 83.50"
Pegs Nondominant 100" 11.81" 111.81"
COWAT 52 9.23 43
Hopkins Delay 8 2.09 6

The same procedure was used for each concussed and control participant, and their retest

results at each testing session were compared to their appropriate cut-point score in order

to determine if there was a significant change from their baseline performance. The

sensitivities for individual variables were calculated for each testing session and are

presented in Table 17.

As was noted in Model 1, the sensitivities for the individual variables was

relatively poor at detecting significant change. The most sensitive individual variables at

24 hours post-concussion were Hopkins Total score and Grooved Pegboard nondominant

hand completion time. As would be expected, using Model 2, the individual variables

were most effective in detecting significant change relative to baseline performance

within 24 hours of concussion, with a trend toward decreasing sensitivity at each








successive testing session. The one exception was the Hopkins Delay score, which

remained relatively stable over time.

Table 17. Sensitivity for Neuropsychological Measures for Model 2
Measure 24 Hours Day 3 Day 5 Day 7

Hopkins Total 0.43 0.11 0.17 0.11
Trails A 0.29 0.16 0.11 0.11
Trails B 0.29 0.05 0.06 0.05
Digit Span Forward 0.14 0.11 0.06 0.05
Digit Span Back 0.19 0.11 0.06 0.05
Grooved Pegboard D 0.19 0.05 0.00 0.05
Grooved Pegboard N 0.43 0.16 0.06 0.16
COWAT 0.14 0.11 0.00 0.00
Hopkins Delay 0.33 0.26 0.17 0.26

The battery as a whole was again examined to determine the number of concussed

and control participants who scored below their individual cut-points across variables

using Model 2. Sensitivity and specificity characteristics of the Model for each test

session are presented in Tables 18-21.

As Table 18 demonstrates, the greatest degree of classification accuracy occurred

at 24 hours for Model 2 when a criterion of a significant reduction in score on 1 or more

variables was used. Specificity greatly increased when the criteria was two or more

significant variables, though this also resulted in the largest single drop in sensitivity.

Using the same criterion on Days 3 through 7, a predictable drop in sensitivity was

observed over time, while specificity remained relatively constant. Notably, this model

saw its lowest sensitivity using the one-variable criterion at the Day 5 test session (Table

20), even relative to the Day 7 test results. This pattern was consistent in most of the

models examined as part of this study.








Table 18. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables Within 24 Hours with Model 2

No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 21 13 1.0 0.0 (1.00)
1 17 5 0.81 0.62 (1.43)
2 10 1 0.48 0.92 (1.40)
3 8 0 0.38 1.00 (1.38)
4 4 0 0.19 1.00 (1.19)
5 4 0 0.19 1.00 (1.19)
6 3 0 0.14 1.00 (1.14)
7 3 0 0.14 1.00 (1.14)
8 1 0 0.05 1.00 (1.05)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 19. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 2
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 11 1.0 0.0 (1.00)
1 12 5 0.63 0.55 (1.18)
2 4 2 0.21 0.82 (1.03)
3 2 0 0.11 1.00 (1.11)
4 1 0 0.05 1.00 (1.05)
5 1 0 0.05 1.00 (1.05)
6 1 0 0.05 1.00 (1.05)
7 1 0 0.05 1.00 (1.05)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se-Sensitivity, Sp=Specificity

As with the use of this model in the previous testing sessions, the greatest degree

of sensitivity was achieved when a criteria of one significant variable was used.

However, this also resulted in falsely classifying greater than half of the control

participants as concussed. Receiver operating characteristic curves for Model 2 are

presented in Figures 17 through 20.









Table 20. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 2
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 18 10 1.0 0.0 (1.00)
1 6 3 0.33 0.70 (1.03)
2 4 1 0.22 0.90 (1.12)
3 2 1 0.11 0.90 (1.01)
4 0 1 0.00 0.90 (0.90)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 21. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 2
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 6 1.0 0.0 (1.00)
1 10 4 0.53 0.33 (0.86)
2 5 0 0.26 1.00 (1.26)
3 1 0 0.05 1.00 (1.05)
4 0 0 0.00 1.00 (1.00)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Data for the 24-hour test session resulted in a ROC curve suggesting that this

model had "fair" classification accuracy. AUC was calculated to be 0.788 (SE =

0.0773; 95% Confidence Interval = 0.636-0.939), and was significantly greater than

change with a significance value ofp<0.0001. At the Day 3 test session (Figure 18),

Model 2 was found not to be statistically better than chance at classifying the concussed


and control participants (AUC = 0.566, SE = 0.1135, p<0.28).













ROC Curve for Model 2 at 24 Hours


1
0.9
' 0.8
= 0.7
.0.6
. 0.5
S0.4
0.3
I -2
S0.2
0.1


No discrimination
-0-Model 2


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)



Figure 17. Receiver Operating Characteristic Curve for Model 2 at 24 Hour Test Session




ROC Curve for Model2 at Day 3


1

0.9

0.8

:_ 0.7
S No discrimination
S0.6 --Model 2
______


Figure 18. Receiver Operating Characteristic Curve for Model 2 at Day 3 Test Session


0 0.2 0.4 0.6 0.8 1
1 -Specificity (false positives)











ROC Curve for Model 2 at Day 5


No discrimination
---- Model 2


0 0.2 0.4 0.6 0.8
1 Specificity (false positives)


Figure 19. Receiver Operating Characteristic Curve for Model 2 at Day 5 Test Session


ROC Curve for Model 2 at Day 7

1
0.9
S0.8
0.7
No discrimination
& 0.6- -o- tModel 2
2 0.5
0.4-
S0.3
r 0.2
0.1 -
0
0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 20. Receiver Operating Characteristic Curve for Model 2 at Day 7 Test Session

Similarly, Model 2 was found not to be statistically different than chance at the

Day 5 and Day 7 test sessions, as shown in Figures 19 and 20. The AUC for Model 2 at

Day 5 was found to be 0.522 (SE = 0.1154, 95% Confidence Interval = 0.296-0.748,









p<0.42), while the AUC for Model 2 at Day 7 was 0.518 (SE = 0.1211, 95% Confidence

Interval = 0.280-0.755, p<0.44).

As the ROC analysis suggested, in the repeated testing of the concussed football

players, comparing their re-test performance to either an appropriate normative sample

(Model 1) or to their own baseline performance using a liberal criteria of 1.0 SD reduced

performance (Model 2) yielded rather poor sensitivity for the individual variables

included in this test battery. Further, examining the variables collectively, both models

were only fair in accurately discriminating between concussed and control participants

even within 24 hours of concussion, a time period in which the cognitive sequelae of

concussion are generally most evident.


Model 1 vs. Model 2 ROC at 24 Test Session (diff = 0.022,
p<0.82)

1
0.9
0.8 D0
5 I Nodiscrimination
S0.7
S------ Model 2
o 0.6
0
CX 0 .. -Model 1
0.5
0.4
0.3
| 0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)


Figure 21. Comparison of Model 1 and Model 2 at the 24-Hour Test Session


Neither was better than chance at discriminating between the two groups at the Day 3,

Day 5, or Day 7 test sessions. A comparison of Model 1 and Model 2 for the 24 Test








Session suggested that Model 1, with an AUC of 0.788 was slightly more sensitive than

Model 2, which had an AUC of 0.766. However, there was not a statistically significant

difference between the two (p=0.82) as illustrated in Figure 21.

Reliable Change Index Models

Model 3

Three variants of the Reliable Change Index (RCI) were subsequently applied to

the data set. The first, a basic RCI model (referred to as Model 3 for the purposes of this

study), essentially examined whether the test-retest performance difference for each

individual was large relative to the error variance of the variable under examination,

providing indication of a change from baseline performance. The following formula was

applied for each individual's performance on each variable:


RCI (x2 x)
Sdiff

In this equation, x, is the individual's score at the Baseline testing session, x, is the

individual's score at the second test session (in this case, within 24 hours of concussion,

day 3, day 5, and day 7), and sd the standard error of the difference, was calculated

using sE, the standard error of the measure with the equation:


S diff= 2E 2

sE was derived from the correlation of the test-retest scores of the control group for each

variable (r,2) and the standard deviation (s) of the control group at baseline using the

equation:


SE = S1 -r2









An index, outside of which a re-test score was considered significant, was calculated by

multiplying the sdy by a cutoff level of 1.96, a widely accepted statistical significance

level. Tables 22 through 25 indicate the data used to calculate these measures as well as

the RCI ranges used as comparitors for each variable at each testing session.

Table 22. Data Used in the Calculation of Reliable Change Indices at 24 Hours
Measure r2 sd SE sd RCI(+/-)

Hopkins Total 0.841 3.52 1.40 1.99 3.89
Trails A 0.204 5.59 4.99 7.05 13.82
Trails B 0.623 11.21 6.88 9.74 19.08
Digits Forward 0.942 2.27 0.55 0.77 1.51
Digits Backward 0.548 2.41 1.62 2.29 4.49
Pegs Dominant 0.720 7.87 4.17 5.89 11.55
Pegs Nondominant 0.737 12.02 6.16 8.72 17.09
COWAT 0.282 6.97 5.91 8.35 16.37
Hopkins Delay 0.682 1.73 0.98 1.38 2.70
a r12 based on the correlation for control participants between baseline and 24 hour performance, listwise n=13
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference


Table 23. Data Used in the Calculation of Reliable Change Indices at Day 3
Measure r2 Sd SE d d RCI(+/-)


Hopkins Total 0.679 3.52 1.99 2.82 5.53
Trails A 0.760 5.59 2.74 3.87 7.59
Trails B 0.215 11.21 9.93 14.05 27.53
Digits Forward 0.672 2.27 1.30 1.84 3.60
Digits Backward 0.659 2.41 1.41 1.99 3.90
Pegs Dominant 0.704 7.87 4.28 6.06 11.87
Pegs Nondominant 0.617 12.02 7.44 10.52 20.62
COWAT 0.429 6.97 5.27 7.45 14.60
Hopkins Delay 0.419 1.73 1.32 1.87 3.66
a r12 based on the correlation for control participants between baseline and Day 3 performance, listwise n=13
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference









Table 24. Data Used in the Calculation of Reliable Change Indices at Day 5
Measure r2Sdb SE Sdid RCI(+/-)

Hopkins Total 0.628 3.52 2.15 3.04 5.95
Trails A 0.392 5.59 4.36 6.16 12.08
Trails B 0.427 11.21 8.49 12.00 23.52
Digits Forward 0.418 2.27 1.73 2.45 4.80
Digits Backward 0.228 2.41 2.12 2.99 5.87
Pegs Dominant 0.297 7.87 6.60 9.34 18.30
Pegs Nondominant 0.735 12.02 6.19 8.75 17.15
COWAT 0.604 6.97 4.39 6.20 12.16
Hopkins Delay 0.386 1.73 1.36 1.92 3.76
a r12 based on the correlation for control participants between baseline and Day 5 performance, listwise n=10
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference


Table 25. Data Used in the Calculation of Reliable Change Indices at Day 7
Measure r2 Sd bSE sd RCI(+/-)


Hopkins Total 0.492 3.52 2.51 3.55 6.96
Trails A 0.301 5.59 4.67 6.61 12.95
Trails B 0.365 11.21 8.93 12.63 24.76
Digits Forward 0.600 2.27 1.44 2.03 3.98
Digits Backward 0.497 2.41 1.71 2.42 4.74
Pegs Dominant 0.674 7.87 4.50 6.36 12.46
Pegs Nondominant 0.883 12.02 4.11 5.81 11.40
COWAT 0.025 6.97 6.88 9.73 19.08
Hopkins Delay 0.884 1.73 0.59 0.83 1.63
a r12 based on the correlation for control participants between baseline and Day 7 performance
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference

Please note that a change was considered to indicate a significant difference between a

performance at a particular test session and the appropriate baseline performance only if

the difference exceeded the RCI cutoff point associated with a significant decrease in

performance at the second test session. That is, it was considered significant if the second

test performance on a particular measure was significantly below the baseline


performance based on the RCI.








Table 26 demonstrates the use of this method on one concussed participant for

two test variables. As is illustrated, for the variable 'Hopkins Total words recalled,' a

reduction of greater than 4 recalled words was calculated to be a significant change

relative to the error variance of the test as indicated by the sd This participant's

reduction of 8 words at 24 hours relative to his baseline performance resulted in a z-score

of-4.03, which, using a cut-point ofz=1.96, was found to be a highly significant change

in the negative direction. That is, his performance on Hopkins Total was significantly

below his performance on the same measure at baseline. For the variable Trails A, which

is measured in seconds, his increase of 51" in total completion time from baseline to the

24 hour test session ultimately resulted in a Trails A time at 24 hours of 84", greatly

exceeding the maximum value in the calculated RCI of 46.8". This resulted in a z-score

of 7.23.

Table 26. Example of RCI Significance Determination for One Concussed Participant at
the 24 Hour Test Session

Measure Observed Change RCI Interval z-score
From Baseline
To 24 Hours


Hopkins Total -8 words 20 +/- 3.89 = (16-24) -4.03
Trails A +51.0" 33" +/- 13.82 = (19.2-46.8) 7.23



Table 27 presents the sensitivities for the individual measures at each test session using

Model 3, the basic RCI approach. Again, the data indicated that the sensitivities for the

individual variables were relatively poor, even within 24 hours, which was also seen in

Models 1 and 2. The sensitivities for the individual variables using Model 3 also








generally represent relatively lower values than in Model 2, which is not surprising since

the criteria used for Models 1 and 2 were more liberal (1.0 SD).

Table 27. Sensitivity for Neuropsychological Measures for Model 3
Measure 24 Hours Day 3 Day 5 Day 7

Hopkins Total 0.43 0.11 0.11 0.11
Trails A 0.14 0.11 0.00 0.00
Trails B 0.19 0.00 0.06 0.05
Digit Span Forward 0.29 0.05 0.00 0.00
Digit Span Back 0.14 0.00 0.06 0.05
Grooved Pegboard D 0.19 0.11 0.00 0.00
Grooved Pegboard N 0.29 0.05 0.00 0.16
COWAT 0.05 0.00 0.00 0.00
Hopkins Delay 0.38 0.16 0.11 0.37

The battery as a whole was again examined to determine the number of concussed and

control participants who scored below their individual cut-points across multiple

variables using Model 3. Sensitivity and specificity characteristics of Model 3 for each

test session are presented in Tables 28-31.

Table 28. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours using Model 3
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 21 13 1.0 0.0 (1.00)
1 16 1 0.76 0.92 (1.68)
2 10 0 0.48 1.00 (1.48)
3 7 0 0.33 1.00 (1.33)
4 4 0 0.19 1.00 (1.19)
5 3 0 0.14 1.00 (1.14)
6 2 0 0.10 1.00 (1.10)
7 1 0 0.05 1.00 (1.05)
8 1 0 0.05 1.00 (1.05)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

As with the first two models, Model 3 was most effective in detecting a concussed

participant when a criterion of one variable below the respective RCI cutoff was used.

Notably, the RCI model resulted in a much greater specificity value when using this








criterion than either Model 1 or Model 2, suggesting that the basic RCI model may be

more effective in separating the groups. This is not altogether surprising since the cutoff

criteria are based on the performance of the control population. At Day 3 (Table 29),

Model 2 saw a significant decline in sensitivity, even when using the minimal criteria of

one variable falling outside the RCI.

Table 29. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 3
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)


1.0
0.21
0.16
0.05
0.05
0.05
0.05
0.00
0.00
0.00


5 1
6 1
7 C
8 C
9 C
Se=Sensitivity, Sp=Specificity


0.0
0.82
0.82
1.00
1.00
1.00
1.00
1.00
1.00
1.00


(1.00)
(1.03)
(0.98)
(1.05)
(1.05)
(1.05)
(1.05)
(1.00)
(1.00)
(1.00)


Table 30. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 3
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)


1.0
0.22
0.06
0.05
0.00
0.00
0.00
0.00
0.00
0.00


0.0
0.90
0.90
0.90
0.90
1.00
1.00
1.00
1.00
1.00


(1.00)
(1.12)
(0.96)
(0.96)
(0.90)
(1.00)
(1.00)
(1.00)
(1.00)
(1.00)


As was seen in the previous two models, detection of concussed players saw its

most extensive drop at the Day 5 test session (Table 30), but saw resurgence in sensitivity


at the Day 7 test session (Table 31).








Table 31. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 3
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 6 1.0 0.0 (1.00)
1 9 3 0.47 0.50 (0.97)
2 5 0 0.26 1.00 (1.26)
3 0 0 0.00 1.00 (1.00)
4 0 0 0.00 1.00 (1.00)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Receiver operating characteristic curves for Model 3 at each test session are

shown in Figures 22 through 25. As Figure 22 indicates, Model 3 was relatively good at

discriminating between concussed and control participants at the 24 hour test session.

Area under the curve was found to be 0.861 (SE = 0.0641, 95% Confidence Interval =

0.735-0.986), which was significantly better than chance level (p<0.0001).

While the use of Model3 was generally good at distinguishing between concussed

and control participants within 24 hours of concussion, Figures 23-25 indicate that the

model performed no better than chance at Day 3, Day 5, and Day 7. The AUC for Model

3 at Day 3 was 0.514 (SE = 0.111, 95% Confidence Interval = 0.298-0.731, p<0.45);

AUC for Model 3 at Day 5 was 0.556 (SE = 0.111, 95% Confidence Interval = 0.338-

0.773, p<0.31); and the AUC for Model 3 at Day 7 was 0.553 (SE = 0.123, 95%

Confidence Interval = 0.312-0.793, p<0.33).











ROC Curve for Model 3 at 24 Hours


No discrimination
--- Model 3


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 22. Receiver Operating Characteristic Curve for Model 3 at 24 Hour Test Session



ROC Curve for Model 3 at Day 3

1

0.9

S0.8
0.7
0 No discrimination
o 0.6
S-0- Model 3
2 0.5
0.4

S0.3
0.2
0.1


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 23. Receiver Operating Characteristic Curve for Model 3 at Day 3 Test Session










ROC Curve for Model 3 at Day 5


No discrimination
---- tModel 3


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 24. Receiver Operating Characteristic Curve for Model 3 at Day 5 Test Session


ROC Curve for Model 3 at Day 7


0 0.2 0.4 0.6 0.8
1 Specificity (false positives)

Figure 25. Receiver Operating Characteristic

Model 4


No discrimination
-o- Model 3









1


Curve for Model 3 at Day 7 Test Session


The second RCI variant analyzed as part of the current study (referred to as Model

4) utilized the same data indicators as were used in the basic RCI model (Model 3), but








also included a correction for observed practice effects in the control population for

individual variables at each testing session. Consistent with the literature, practice

effects were calculated by finding the mean change in score over a test-retest period for

the comparison population. For the current study, this involved calculating the mean

change for the control population for each variable from baseline to 24 hour test session,

baseline to Day 3 test session, baseline to Day 5 test session, and baseline to Day 7 test

session. Again, individual subjects were omitted listwise. Table 32 indicates the mean

change scores for the control population.

Table 32. Mean Change Scores for Individual Variables for Control Participants
Measure 24 Hour(n=13) Day 3(n=1 1) Day5(n=10) Day7(n=6)

Hopkins Total 0.38 -0.91 -0.40 0.50
Trails A -3.69" -5.03" -4.87" -8.48"
Trails B 2.34"* -1.01" -8.40" -11.03"
Digits Forward 1.38 1.64 2.00 1.50
Digits Backward 0.77 1.73 2.50 0.67
Pegs Dominant -2.97" -4.05" -4.20" -8.07"
Pegs Nondominant -0.61" -3.35" 0.46"* -6.55"
COWAT 3.08 5.45 5.80 1.67
Hopkins Delay 0.00* -1.27* 0.60 -2.33*
*indicates no practice effect detected

In order to correct for practice effect, the mean change score for a variable was

subtracted from each participants score on that variable at the retest session. The

resulting number was then entered into the RCI equation as outlined above, and

calculated as before with the basic RCI method in Model 3. Therefore, the equation for

the RCI became:

(x2 -d12)-x1
Sdiff








where x, was the participant's score on a variable at baseline, x, was their score at the

retest session, and d12 was the average change score for the control group on the variable

over that test-retest interval. The sda was calculated as in the basic RCI model using the

correlations and standard deviations for the control group as described above. Please

note that for those variables that were found, on average, not to have practice effects for

the control group at the retest session, no changes were made in the study participants'

retest score in the RCI model. This occurred five times, most frequently for the Hopkins

Verbal Learning Test Total and Delay scores, which was not unexpected given that these

tests used alternate forms over test sessions. Again, for the purposes of this study, scores

were classified as significant only if they exceeded their RCI cutoff in the negative or

declined performance level.

Table 33 presents the sensitivities for the individual measures at each test session

using Model 4, the RCI approach with correction for practice effects. Once again, it is

apparent that the individual variables, despite correction for practice effect, have

relatively low sensitivities. The practice correction does appear to have resulted in a

slight improvement in sensitivity for each re-test session, suggesting the detection of one

or two more significantly decreased scores for each. It is notable that both Hopkins

Delay and Grooved Pegboard nondominant hand performance have relatively larger

sensitivities when compared to the other variables at day 7, suggesting some lingering

short-term memory and motor speed decline in the concussed population. Tables 34

through 37 indicate the sensitivity and specificity of the battery of variables as a whole

for Model 4 at each test session.








Table 34 indicates a relatively high sensitivity/specificity combination was

achieved for the RCI model with correction for practice at the 24 Hour test session using

a criterion of 1 significant variable. Sixteen of the 21 concussed participants at this test

session fell below the cutoff for 1 or more of the 9 variables under examination while

only 1 control participants fell below the cutoff for 1 or more variables. By the Day 3

test session (Table 35), the RCI model with practice correction continued to have fairly

high classification success using a criterion of 1 significant variable, achieving a Se + Sp

value of 1.31.

Table 33. Sensitivity for Neuropsychological Measures for Model 4
Measure 24 Hours Day 3 Day 5 Day 7

Hopkins Total 0.43 0.11 0.11 0.11
Trails A 0.19 0.21 0.00 0.16
Trails B 0.19 0.00 0.06 0.05
Digit Span Forward 0.33 0.16 0.01 0.05
Digit Span Back 0.14 0.11 0.01 0.05
Grooved Pegboard D 0.24 0.11 0.00 0.11
Grooved Pegboard N 0.29 0.11 0.00 0.32
COWAT 0.10 0.11 0.11 0.00
Hopkins Delay 0.38 0.16 0.11 0.37

Table 34. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours with Model 4
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 21 13 1.0 0.0 (1.00)
1 16 1 0.76 0.92 (1.68)
2 11 0 0.52 1.00 (1.52)
3 7 0 0.33 1.00 (1.33)
4 5 0 0.24 1.00 (1.24)
5 4 0 0.19 1.00 (1.19)
6 2 0 0.10 1.00 (1.10)
7 2 0 0.10 1.00 (1.10)
8 1 0 0.05 1.00 (1.05)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity








At the Day 5 test session (Table 36), the application of Model 4 resulted in an

accuracy of 33% when using a criterion of significance on one variable while producing 2

false positive identifications. Although slightly better than the corresponding accuracy

using Model 3, the ability to distinguish between the groups was relatively poor.

However, unlike the previous models, Model 4 achieved some success in distinguishing

between concussed and control participants by Day 7 as can be seen in Table 37. Using a

criterion of significance on one variable, Model 4 achieved a sensitivity level of 0.68

although it also produced a high number of false positive identifications.

Table 35. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 4
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 11 1.0 0.0 (1.00)
1 11 3 0.58 0.73 (1.31)
2 4 2 0.21 0.82 (1.03)
3 1 0 0.05 1.00 (1.05)
4 1 0 0.05 1.00 (1.05)
5 1 0 0.05 1.00 (1.05)
6 1 0 0.05 1.00 (1.05)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 36. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 4
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 18 10 1.0 0.0 (1.00)
1 6 2 0.33 0.80 (1.13)
2 2 1 0.11 0.90 (1.01)
3 1 1 0.05 0.90 (0.95)
4 0 1 0.00 0.90 (0.90)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)








Using a criterion of two significant variables, it achieved its highest sensitivity/specificity

combination, but saw a substantial decline in sensitivity. While cursory examination of

the sensitivity/specificity values suggested some clinical utility in identifying lingering

cognitive complaints at this most distal test session, further analysis of statistical

significance using ROC curves indicated otherwise.

Table 37. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 4
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 6 1.0 0.0 (1.00)
1 13 3 0.68 0.50 (1.18)
2 7 1 0.37 0.83 (1.20)
3 2 0 0.11 1.00 (1.11)
4 1 0 0.05 1.00 (1.05)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Receiver operating characteristic curves for Model 4 at each testing session are presented

in Figures 26 through 29. As with the basic RCI model, the RCI with correction for

practice resulted in a model with a relatively good ability to distinguish between

concussed and control participants within 24 hours of concussion. The AUC for this

model was 0.863 (SE = 0.0636, 95% Confidence Interval = 0.738-0.987) and was

significantly better than chance with p<0.0001 (Fig. 26). As seen in Figure 27, Model 4

resulted in slightly better classification rate at the Day 3 test session, although the model

was again not statistically difference than chance level with an AUC of 0.634 (SE =

0.1082, 95% Confidence Interval = 0.422-0.846, p=0.1079). At Day 5, Model 4 was

clearly not better than chance at distinguishing between concussed and control






85


participants (Fig. 28). AUC was calculated as 0.556, p=0.3151 (SE = 0.1154, 95%

Confidence Interval = 0.329-0.782).


ROC Curve for Model 4 at 24 Hours


No discrimination
---- Model 4


0 0.2 0.4 0.6 0.8
1 Specificity (false positives)


Figure 26. Receiver Operating Characteristic Curve for Model 4 at 24 Hour Test Session

ROC Curve for Model 4 at Day 3


No discrimination
0-Model 4


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 27. Receiver Operating Characteristic Curve for Model 4 at Day 3 Test Session











ROC Curve for Model 4 at Day 5


No discrimination
-o- Model 4


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 28. Receiver Operating Characteristic Curve for Model 4 at Day 5 Test Session


ROC Curve for Model 4 at Day 7

1
0.9
S0.8
a
S0.7
S No discrimination
0 0.6 4
O 0.6 --o Model 4
2 0.5
0.4-
S0.3
0 0.2-
0.1


0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)

Figure 29. Receiver Operating Characteristic Curve for Model 4 at Day 7 Test Session

As seen in Figure 29, the application of Model 4 at Day 7 appears to have been somewhat

more successful than the preceding models at distinguishing between concussed and

control participants. This was suggested by the sensitivity/specificity values in Table 37.









The AUC for this model at Day 7 was 0.636 (SE = 0.1245, 95% Confidence Interval =

0.392-0.880); however, this was again not statistically different than the reference

diagonal (p=0.1374).

A comparison between the basic RCI model and the RCI with correction for

practice applied to the data at the 24 Hour test session is shown in Figure 30. Although

the AUC for Model 4 is slightly larger than for Model 3 at the 24 Hour test session (0.863

vs. 0.861), the difference between the two was not statistically significant (p=l).


Comparison of RCI and RCI with Correction at 24 Hours



0.9

S0.8 ..... No discrimination
S0.7-
"-o RCI with Correction for
0.6 Pactice
i 0.5 ---D--RCa
0.4
0.3
0.2
0.1

0 0.2 0.4 0.6 0.8 1
1 Specificity (false positives)
Figure 30. Comparison of ROC Curves for the RCI and RCI with Correction for
Practice

Model 5
The third variant of the RCI model (referred to as Model 5) approached the task

of determining whether there was a real difference between baseline and retest

performance by abandoning some assumptions used in the previous two RCI models.

Those assumptions included that 1) the control participants' true scores for a particular








variable at test and retest sessions were equal, and 2) that the error variances for a

particular test variable at test and retest were also equal. Using the equation

(Y, Xi) -(Y -X)
[Sx2 -Sy2 2S xS yr

where (Y X), or D, is the mean change score for a particular variable for the control

population, and Sx2 S2 2S Syry or SD ,is the standard deviations of the change

scores for the control population, the participants' change scores from baseline to each

successive test session were examined to determine if they were large relative to the

distribution of change scores for the control group for each of those intervals.

Table 38 shows the average change scores for each variable at each testing

session with the standard deviation of the change scores for the control population. Z-

scores were calculated using the above equation. This process is illustrated in Table 39

for one concussed participant at the 24 Hour test session. As with the previous models, a

change score was considered significant if it exceeded 1.96 in the negative direction.

Therefore, as seen in Table 39, a z-score indicating a significant change relative to the

variance of the change scores for the control population for Hopkins Total score is

indicated by a z-score less than -1.96. By contrast, a greater-than-average change score

in the negative direction for Trails A (and all other variables where time is the measured

variable) is indicated by a z-score greater than +1.96.

Table 40 presents the sensitivities for the individual measures at each test session

using Model 5. As the data illustrates, the sensitivities for individual variables using

Model 5 at the 24 Hour test session are once again relatively higher than any of the other

days as a whole. Singly, the sensitivity values are not very high. As with most batteries








of neuropsychological test batteries, however, their combined use resulted in greater

sensitivity for this model over the repeated test sessions.

Table 38. Average Change Scores and Standard Deviations of Change Scores for the
Control Population
Test Session
24 Hour Day 3 Day 5 Day 7
Measure D SD D SD D SD D SD


Hopkins Total 0.38 2.53 -0.91 3.36 -0.40 3.95 -4.33 10.27
Trails A -3.69 6.03 -5.03 3.73 -4.87 5.55 -8.48 4.19
Trails B 2.34 10.55 -1.01 26.02 -8.40 13.96 -11.03 10.44
Digits Forward 1.38 1.12 1.64 1.63 2.00 1.94 1.50 1.87
Digits Backward 0.08 2.22 1.73 2.00 2.50 3.14 0.67 2.07
Pegs Dominant -2.97 6.08 -4.05 5.80 -4.20 13.25 -8.07 7.77
Pegs Nondominant -0.61 8.27 -3.35 10.65 0.46 12.61 -6.55 5.99
COWAT 3.08 8.92 5.45 8.36 5.80 7.93 1.67 11.45
Hopkins Delay 0.00 1.41 -1.27 2.53 0.60 1.84 -2.33 3.56
D =average change, SD =standard deviation of change score

Table 39. Example of Z-score Derivation using Model 5 for one Concussed Participant at
24 Hour Test Session
Measure Baseline 24 hour Difference D S, z-score

Hopkins Total 20 12 -8 0.38 2.53 -3.31
Trails A 33.0 84.0 51 -3.69 6.03 9.06
D =average change, SD =standard deviation of change score

Table 40. Sensitivity for Neuropsychological Measures for Model 5
Measure 24 Hours Day 3 Day 5 Day 7

Hopkins Total 0.43 0.05 0.00 0.00
Trails A 0.29 0.16 0.11 0.32
Trails B 0.19 0.00 0.06 0.05
Digit Span Forward 0.33 0.16 0.22 0.05
Digit Span Back 0.14 0.11 0.06 0.16
Grooved Pegboard D 0.24 0.11 0.00 0.11
Grooved Pegboard N 0.29 0.11 0.00 0.32
COWAT 0.05 0.11 0.00 0.00
Hopkins Delay 0.38 0.00 0.11 0.00








As can be seen, the sensitivity values at day 3 and day 5 are consistently poor.

Interestingly, however, the sensitivities for the individual variables at day 7 are somewhat

higher using Model 5 than the previous 4, suggesting that this may be a more appropriate

model to use at more distal testing sessions. Tables 41 through 44 indicated the

sensitivity and specificity of the battery of variables as a whole for Model 5 at each test

session.

Table 41. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours with Model 5
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 21 13 1.0 0.0 (1.00)
1 17 1 0.81 0.92 (1.73)
2 12 0 0.57 1.00 (1.57)
3 7 0 0.33 1.00 (1.33)
4 5 0 0.24 1.00 (1.24)
5 4 0 0.19 1.00 (1.19)
6 2 0 0.10 1.00 (1.10)
7 1 0 0.05 1.00 (1.05)
8 1 0 0.05 1.00 (1.05)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 41 indicates that Model 5 was very successful at distinguishing between concussed

and control participants at the 24 Hour test session. Using a criterion of one variable

falling below cutoff, the combined sensitivity/specificity was 1.73, with both good

sensitivity (0.81) and specificity (0.92).

Day 3 sensitivity/specificity data using Model 5 (Table 42) again indicated a

pattern of significantly decreased sensitivity and a less substantial drop in specificity with

the one variable criterion. Using two variables as a criterion results in an unacceptable

drop in sensitivity. A continued reduction in sensitivity at day 5, as was seen with all

previous models, was again evident for Model 5 (Table 43). However, at the Day 7 test








session, Model 5 resulted in fair sensitivity and good specificity using the one-variable

criterion (Table 44).

Table 42. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 5
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 11 1.0 0.0 (1.00)
1 10 2 0.53 0.82 (1.35)
2 2 0 0.11 1.00 (1.11)
3 1 0 0.05 1.00 (1.05)
4 1 0 0.05 1.00 (1.05)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

Table 43. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 5
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 18 10 1.0 0.0 (1.00)
1 7 2 0.39 0.80 (1.19)
2 1 0 0.06 1.00 (1.06)
3 0 0 0.00 1.00 (1.00)
4 0 0 0.00 1.00 (1.00)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity

The statistical significance of Model 5 at each testing session was again determined

through ROC curve analysis using non-parametric measurement of area under the curve.


Results of this analysis are depicted in Figures 31 through 34.









Table 44. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 5
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)

0 19 6 1.0 0.0 (1.00)
1 12 0 0.63 1.00 (1.63)
2 6 0 0.32 1.00 (1.32)
3 1 0 0.05 1.00 (1.05)
4 0 0 0.00 1.00 (1.00)
5 0 0 0.00 1.00 (1.00)
6 0 0 0.00 1.00 (1.00)
7 0 0 0.00 1.00 (1.00)
8 0 0 0.00 1.00 (1.00)
9 0 0 0.00 1.00 (1.00)
Se=Sensitivity, Sp=Specificity


ROC Curve for Model 5 at 24 Hours



0.9
S0.8
0.7
S- No discrimination
L 0.6 -Model 5
20.5
0.4-
0.3
(c 0.2
0.1

0 0.2 0.4 0.6 0.8 1
1 -Specificity (false positives)

Figure 31. Receiver Operating Characteristic Curve for Model 5 at 24 Hour Test Session

Figure 31 depicts the ROC curve for Model 5 at 24 hours. The AUC of 0.888 (SE =

0.0577, 95% Confidence Interval = 0.775-1.00) was larger than any of the preceding

models and was highly significant (p<0.0001), indicating that this model was good at

distinguishing between the change scores of the concussed and control participants.

Further, Figure 32 indicates that Model 5 was also able to distinguish between the









concussed and control populations at the Day 3 test session at a statistically significant

level, an achievement which no previous model was able to attain. The AUC for the

model at this test session was 0.682 (SE=0.0985, 95% Confidence Interval = 0.489-

0.875), which was statistically different than chance level (p=0.033).


ROC Curve for Model 5 at Day 3

1
0.9
S0.8
0.7
No discrimination
S0.6 -- odel 5
2 0.5.
S0.4
0.3
& 0.2-
0.1

0 0.2 0.4 0.6 0.8 1
I Specificity (false positives)

Figure 32. Receiver Operating Characteristic Curve for Model 5 at Day 3 Test Session

By day 5, however, Model 5 was not statistically different than chance at

distinguishing between the two groups (Fig. 33). The AUC was calculated to be 0.597

(SE = 0.1107, 95% Confidence Interval = 0.380-0.814), with a p-value of 0.19.

Interestingly, as was suggested by the sensitivity/specificity values for Model 5 at day 7

in Table 44, Figure 34 indicates that Model 5 was indeed successful at distinguishing

between concussed and control participants at the day 7 test session. The ROC plot

produced an AUC of 0.816 (SE = 0.0837, 95% Confidence Interval = 0.652-0.980),

which was statistically different than chance level (p<0.0001).




Full Text

PAGE 1

'(7(&7,1* 6,*1,),&$17 &+$1*( ,1 7+( 1(85236<&+2/2*,&$/ 3(5)250$1&( 2) &2//(*( )227%$// 3/$<(56 %\ %(1-$0,1 5 3+$/,1 $ ',66(57$7,21 35(6(17(' 72 7+( *5$'8$7( 6&+22/ 2) 7+( 81,9(56,7< 2) )/25,'$ ,1 3$57,$/ )8/),//0(17 2) 7+( 5(48,5(0(176 )25 7+( '(*5(( 2) '2&725 2) 3+,/2623+< 81,9(56,7< 2) )/25,'$

PAGE 2

&RS\ULJKW E\ %HQMDPLQ 5 3KDOLQ

PAGE 3

$&.12:/('*(0(176 $ SURMHFW RI WKLV VFRSH LQYROYHV WKH FROODERUDWLRQ RI D JUHDW QXPEHU RI SHRSOH )LUVW DQG IRUHPRVW ZRXOG OLNH WR DFNQRZOHGJH P\ SDUHQWV ZKRVH HPSKDVLV RQ WKH YDOXH RI HGXFDWLRQ DQG XQVZHUYLQJ VXSSRUW WKURXJKRXW P\ RZQ HGXFDWLRQDO HQGHDYRUV KDV DOORZHG PH WR FRQWLQXH WR VWULYH WR UHDFK IRU P\ SRWHQWLDO 6HFRQGO\ ZRXOG OLNH WR WKDQN P\ ZLIH IRU KHU HQFRXUDJHPHQW WKURXJKRXW WKH SURFHVV RI WKH SUHSDUDWLRQ RI WKLV SDSHU /DVW EXW FHUWDLQO\ QRW OHDVW ZRXOG OLNH WR WKDQN 'U (LOHHQ )HQQHOO DQG 'U 'XDQH 'HGH IRU WKHLU JXLGDQFH DQG IRU SURYLGLQJ PH ZLWK RXWVWDQGLQJ H[DPSOHV RI WUXH VFLHQWLVWSUDFWLWLRQHUV

PAGE 4

7$%/( 2) &217(176 3DJH $&.12:/('*(0(176 LLL $%675$&7 YL &+$37(5 21( ,1752'8&7,21 +LVWRULFDO &RQWH[W 0LQRU +HDG ,QMXU\ 'HILQLWLRQ 0LQRU +HDG ,QMXU\ 3DWKRORJ\ 1HXURSV\FKRORJLFDO (IIHFWV RI 0LQRU +HDG ,QMXU\ 6SRUWVUHODWHG 0LQRU +HDG ,QMXU\ 5HOLDEOH &KDQJH 5HOLDEOH &KDQJH ,QGH[ 5HOLDEOH &KDQJH ,QGH[ ZLWK &RUUHFWLRQ IRU 3UDFWLFH /LQHDU 5HJUHVVLRQ 0XOWLSOH 5HJUHVVLRQ 0RGHO 6SHFLILF $LPV DQG +\SRWKHVHV 7:2 0(7+2'6 6XEMHFWV 3URWRFRO %DVHOLQH ,Q6HDVRQ &RQFXVVLRQ (YDOXDWLRQ 'DWD $QDO\VLV %DVLF 0RGHOV 5HOLDEOH &KDQJH ,QGLFHV 5HJUHVVLRQ 0RGHOV 6HQVLWLYLW\ DQG 6SHFLILFLW\ 7+5(( 5(68/76 'HPRJUDSKLF 'DWD IRU %DVHOLQH 3RSXODWLRQ ,9

PAGE 5

'HPRJUDSKLF &KDUDFWHULVWLFV IRU &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV %DVHOLQH 3HUIRUPDQFH 2YHUYLHZ RI 5HSHDWHG 7HVWLQJ 'HWHFWLQJ 6LJQLILFDQW 'HFOLQH 8VLQJ %DVLF 6WDWLVWLFDO 0HWKRGV 0RGHO 0RGHO 5HOLDEOH &KDQJH ,QGH[ 0RGHOV 0RGHO 0RGHO 0RGHO 5HJUHVVLRQ 0RGHOV 0RGHO 0RGHO )285 ',6&866,21 %DVLF 0RGHOV 5&, 0RGHOV 5HJUHVVLRQ 0RGHOV /LPLWDWLRQV DQG ,PSOLFDWLRQV IRU )XWXUH 5HVHDUFK &RQWULEXWLRQV RI WKH &XUUHQW 6WXG\ 5()(5(1&(6 %,2*5$3+,&$/ 6.(7&+ Y

PAGE 6

$EVWUDFW RI 'LVVHUWDWLRQ 3UHVHQWHG WR WKH *UDGXDWH 6FKRRO RI WKH 8QLYHUVLW\ RI )ORULGD LQ 3DUWLDO )XOILOOPHQW RI WKH 5HTXLUHPHQWV IRU WKH 'HJUHH RI 'RFWRU RI 3KLORVRSK\ '(7(&7,1* 6,*1,),&$17 &+$1*( ,1 7+( 1(85236<&+2/2*,&$/ 3(5)250$1&( 2) &2//(*( )227%$// 3/$<(56 %\ %HQMDPLQ 5 3KDOLQ $XJXVW &KDLU (LOHHQ % )HQQHOO &RFKDLU 'XDQH ( 'HGH 0DMRU 'HSDUWPHQW &OLQLFDO DQG +HDOWK 3V\FKRORJ\ 7KH XVH RI QHXURSV\FKRORJLFDO WHVW LQVWUXPHQWV KDV EHHQ DGYRFDWHG E\ UHVHDUFKHUV LQ WKH GHWHFWLRQ RI FRJQLWLYH GHFOLQH IROORZLQJ VSRUWVUHODWHG FRQFXVVLRQ DV EHLQJ D VHQVLWLYH PHDQV RI SURGXFLQJ REMHFWLYH UHVXOWV ZLWK ZKLFK WR DLG LQ PDNLQJ UHWXPWRSOD\ GHFLVLRQV +RZHYHU QR VWXG\ WR GDWH KDV SURYLGHG D V\VWHPDWLF H[DPLQDWLRQ RI VHULDO QHXURSV\FKRORJLFDO WHVWLQJ RYHU VHYHUDO GD\V IROORZLQJ FRQFXVVLRQ ,Q WKH FXUUHQW VWXG\ D EULHI EDWWHU\ RI QHXURSV\FKRORJLFDO WHVWV ZDV DGPLQLVWHUHG WR FROOHJH IRRWEDOO SOD\HUV DW IRXU 'LYLVLRQ XQLYHUVLWLHV 2I WKH LQLWLDO SDUWLFLSDQWV FRQFXVVHG SOD\HUV ZHUH LGHQWLILHG E\ WHDP WUDLQHUV DQG VXEVHTXHQWO\ XQGHUZHQW UHWHVWLQJ XVLQJ WKH VDPH EDWWHU\ RI WHVWV ZLWKLQ KRXUV GD\V GD\V DQG GD\V RI FRQFXVVLRQ &RQFXVVHG SOD\HUV ZHUH DVVLJQHG FRQWUROV PDWFKHG RQ SRVLWLRQ KLVWRU\ RI GLDJQRVHG YL

PAGE 7

OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ ZKR ZHUH DOVR UHDGPLQLVWHUHG WKH EDWWHU\ DW WKH VDPH WHVW LQWHUYDOV 6HYHQ GLIIHUHQW VWDWLVWLFDO PRGHOV FRQFHSWXDOO\ RUJDQL]HG LQWR WKUHH JURXSV ZHUH H[DPLQHG WR GHWHUPLQH WKHLU DELOLW\ WR GLVWLQJXLVK EHWZHHQ WKH FRQFXVVHG DQG QRQFRQFXVVHG SDUWLFLSDQWV %DVLF PRGHOV LQFOXGHG D FRPSDULVRQ RI UHWHVW SHUIRUPDQFH WR QRUPDWLYH GDWD DQG FRPSDULVRQ RI UHWHVW SHUIRUPDQFHV WR EDVHOLQH SHUIRUPDQFH 5&, PRGHOV LQFOXGHG D EDVLF 5&, DSSURDFK 5&, ZLWK SUDFWLFH FRUUHFWLRQ DQG LQGLFHV FUHDWHG EDVHG RQ WKH PDJQLWXGH RI FKDQJH RI WKH FRQWURO SRSXODWLRQ 5HJUHVVLRQ PRGHOV LQFOXGHG VLPSOH OLQHDU UHJUHVVLRQ DQG PXOWLSOH UHJUHVVLRQ PRGHOV WKRXJK PXOWLSOH UHJUHVVLRQ DQDO\VLV ZDV HQGHG GXH WR LQVWDELOLW\ RI SUHGLFWLRQ HTXDWLRQV 2SWLPDO VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQV RFFXUUHG ZKHQ FODVVLILFDWLRQ ZDV PDGH EDVHG RQ RQH VLJQLILFDQW YDULDEOH ILQGLQJ *RRG VHQVLWLYLW\ ZDV REVHUYHG IRU DOO PRGHOV DW WKH KRXU UHWHVW SHULRG DOWKRXJK VSHFLILFLW\ ZDV D OLPLWLQJ IDFWRU IRU WKH PRVW EDVLF PRGHOV /LQHDU UHJUHVVLRQ PRGHOV DQG DQ DQDO\VLV RI PDJQLWXGH RI FKDQJH RYHU UHWHVW LQWHUYDOV ZHUH WKH RQO\ PRGHOV WR GLVWLQJXLVK EHWZHHQ JURXSV DW 'D\ DQG 'D\ 5HVXOWV VXJJHVWHG WKH LPSRUWDQFH RI V\VWHPDWLF H[DPLQDWLRQ RI FRJQLWLYH GHFOLQH IROORZLQJ VSRUWVUHODWHG FRQFXVVLRQ HVSHFLDOO\ JLYHQ FXUUHQWO\XVHG $$1 UHWXPWRSOD\ JXLGHOLQHV DV ZHOO DV QHHG IRU DSSURSULDWH FRPSDULVRQ GDWD DQG VWDWLVWLFDO DQDO\VHV WR DLG LQ WKH GHWHUPLQDWLRQ RI FRJQLWLYH UHFRYHU\ IROORZLQJ FRQFXVVLYH LQMXU\ )XUWKHU GDWD FROOHFWLRQ LV QHFHVVDU\ HVSHFLDOO\ DW ODWHU VHVVLRQV WR DFFXUDWHO\ XVH PXOWLSOH UHJUHVVLRQ PRGHOV 9OO

PAGE 8

&+$37(5 21( ,1752'8&7,21 +LVWRULFDO &RQWH[W 7KH XVH RI QHXURSV\FKRORJLFDO PHDVXUHV KDV RYHU WKH FRXUVH RI WKH FRQWLQXLQJ HYROXWLRQ RI WKH ILHOG EHHQ WKH IRFXV RI FRQVLGHUDEOH GHEDWH LQ WHUPV RI XWLOLW\ DQG FRUUHODWHV RI VSHFLILF WHVW SHUIRUPDQFH 7KLV HYROXWLRQ KDV EHHQ GHVFULEHG LQ WHUPV RI GLVWLQFW HUDV 0F6ZHHQ\ HW DOf RU DUHDV RI IRFXV LQ WKH XVH RI QHXURSV\FKRORJLFDO WHVW SHUIRUPDQFH 7KH ILUVW KDV EHHQ GHVFULEHG E\ 5RXUNH TXRWHG LQ 0FVZHHQ\ HW DO f DQG LQYROYHG WKH LQWHUSUHWDWLRQ RI WHVW SHUIRUPDQFH DQG WKH XVH RI FRJQLWLYH WHVWLQJ PDLQO\ WR LPEXH VRPH YHUDFLW\ WR WKH K\SRWKHVLV WKDW SHUIRUPDQFH RQ FHUWDLQ VSHFLILFDOO\ GHVLJQHG WHVW PHDVXUHV LQGLUHFWO\ VXJJHVWHG G\VIXQFWLRQ LQ FHUWDLQ ORFDOL]HG EUDLQ DQDWRP\ 7KLV HQGHDYRU RI FRXUVH KDV EHHQ WKH PDLQVWD\ RI WKH YDULRXV DFDGHPLF VXEVSHFLDOWLHV FRQFHUQHG ZLWK WKH VWXG\ RI QHXURDQDWRP\ DQG WKH DVVRFLDWHG FRJQLWLYH VFLHQFHV DQG WKHRULHV 7KH YHU\ URRWV RI WKLV JRDO FDQ EH WUDFHG EDFN DV IDU DV WKH WLPH RI *DOO SKUHQRORJ\ DQG WKH QRWLRQ WKDW YDULRXV SURWUXVLRQV RQ WKH VFDOS RI DQ LQGLYLGXDO UHYHDOHG FHUWDLQ SHUVRQDOLW\ WUDLWV DQG WHQGHQFLHV &RZH\ f :KLOH WKH WKHRU\ RI SKUHQRORJ\ LWVHOI KDV ORQJ VLQFH EHHQ GLVFUHGLWHG LW LV WKH QRWLRQ RI ORFDOL]DWLRQ WKDW KDV VWRRG WKH WHVW RI WLPH DQG KDV OHG WR D JUHDWHU XQGHUVWDQGLQJ DQG ORJLFDO RUJDQL]DWLRQ RI WKH VWXG\ RI WKH VWUXFWXUH DQG IXQFWLRQ RI WKH KXPDQ EUDLQ 'XULQJ WKH LQIDQF\ RI WKH ILHOG RI QHXURSV\FKRORJLFDO WKHRU\ LQ JHQHUDO WKH SULPDU\ RSSRUWXQLW\ WR IXUWKHU FRUURERUDWH VWUXFWXUH DQG IXQFWLRQ ZDV IRXQG WR EH WKH VWXG\ RI WKH SHUIRUPDQFH RI LQGLYLGXDOV ZLWK

PAGE 9

ORFDOL]HG OHVLRQV )URP WKH SHUIRUPDQFH RI WKHVH LQGLYLGXDOV LQIHUHQFHV FRQFHUQLQJ WKH IXQFWLRQ RI WKH DEVHQW RU GDPDJHG DUHDV ZHUH IRQQXODWHG 2QH RI WKH PRVW FLWHG FDVHV LV WKDW RI SDWLHQW +0 ZKR UHFHLYHG UHVHFWLRQ RI WKH DQWHULRU SRUWLRQV RI ERWK RI KLV WHPSRUDO OREHV DV SDUW RI D UDGLFDO VXUJLFDO FRUUHFWLRQ IRU LQWUDFWDEOH HSLOHSV\ %UHQGD 0LOQHU DQG KHU FROOHDJXHV GHWDLOHG WKH FRJQLWLYH HIIHFWV RI WKLV UHVHFWLRQ DQG HVWDEOLVKHG WKH XWLOLW\ RI QHXURSV\FKRORJLFDO WHVWV LQ WKH PHDVXUHPHQW RI GHILFLHQW FRJQLWLYH DELOLWLHV 0LOQHU f $V 0F6ZHHQ\ HW DO f QRWHG QHXURSV\FKRORJLFDO WHVWV KDYH EHHQ XWLOL]HG UHFHQWO\ LQ RUGHU WR DLG LQ GLDJQRVLV DQG WR SURYLGH FOLQLFDOO\ VLJQLILFDQW FRQWULEXWLRQV WR WKH GLDJQRVWLF SURFHVV $ YDVW OLWHUDWXUH KDV EHHQ DPDVVHG IRU LQVWDQFH LQ WKH DSSOLFDWLRQ RI QHXURSV\FKRORJLFDO WHFKQLTXHV LQ WKH H[DPLQDWLRQ DQG WUHDWPHQW RI HSLOHSWLF SDWLHQWV VXEVHTXHQW WR SDWLHQW +0 %HQQHWW +HUPDQQ +HUPDQQ DQG :\OHU f &RJQLWLYH WHVWV KDYH EHHQ XVHG WR DLG LQ WKH SURFHVV RI ORFDOL]LQJ WKH IRFXV RI HSLOHSWLF DFWLYLW\ E\ VXJJHVWLQJ UHODWLYHO\ GLVFUHWH DQDWRPLFDO DUHDV RI G\VIXQFWLRQ ZKLFK DUH WKRXJKW WR FRQWULEXWH WR SRRU SHUIRUPDQFH RQ FHUWDLQ PHDVXUHV )XUWKHU FRJQLWLYH WHVWV KDYH EHHQ XVHG LQ FRQMXQFWLRQ ZLWK UHYHUVLEOH OHVLRQ PHWKRGV HJ :$'$f WR ORFDOL]H EUDLQ UHJLRQV UHVSRQVLEOH IRU ODQJXDJH DQG PHPRU\ LQ LQGLYLGXDOV XQGHUJRLQJ VXUJLFDO UHVHFWLRQ PHWKRGV LQ RUGHU WR DYRLG WKH DFFLGHQWDO GHVWUXFWLRQ RI WKHVH DUHDV DQG WKHLU VXEVHTXHQW GHELOLWDWLQJ HIIHFWV /RULQJ HW DO f 'HWHUPLQDWLRQ RI WKH ODQJXDJH GRPLQDQW KHPLVSKHUH YLD QHXURSV\FKRORJLFDO PHWKRGV LV DQ LPSRUWDQW VWHS LQ LGHQWLI\LQJ WKRVH LQGLYLGXDOV ZKR PLJKW EHVW EH VHUYHG ZLWK WKLV VXUJLFDO WUHDWPHQW $ IXUWKHU XVH RI QHXURSV\FKRORJLFDO WHFKQLTXHV EHFRPHV VDOLHQW ZKHQ RQH VHHNV WR TXDQWLI\ WKH HIIHFW RI D VXUJLFDO UHVHFWLRQ RU DQ\ RWKHU QHXURORJLFDO LQVXOW %\

PAGE 10

FRPSDULQJ DQ LQGLYLGXDOfV SHUIRUPDQFH EHIRUH D SRWHQWLDOO\ QHXURORJLFDOO\ DOWHULQJ HYHQW WR KLVKHU SHUIRUPDQFH DIWHUZDUG RQH LV DEOH WR REMHFWLYHO\ GHVFULEH WKH UHVXOWLQJ FRJQLWLYH HIIHFWV $QRWKHU FRPPRQO\ XVHG PHWKRGRORJ\ LQ JHQHUDO LQYROYHV FRPSDULQJ WKH SHUIRUPDQFH RI RQH LQGLYLGXDO WR WKDW RI D JURXS RI RWKHU LQGLYLGXDOV RU QRUPDWLYH SRSXODWLRQ ,Q WKLV VHQVH WKH WHUP fQRUPDWLYHf LPSOLHV WKDW WKH PHDQ VFRUH RI D JURXS RI LQGLYLGXDOV LV WKH VWDQGDUG WR ZKLFK RWKHU LQGLYLGXDOV FDQ EH FRPSDUHG 7KLV LV SHUKDSV WKH PRVW W\SLFDO PHWKRG XVHG LQ QHXURSV\FKRORJ\ WR JDXJH WKH SHUIRUPDQFH RI WKH LQGLYLGXDO LQ TXHVWLRQ DQG KDV RIWHQ EHHQ XVHG ZLWKRXW TXHVWLRQ DV WR WKH YDOLGLW\ RI VXFK SUDFWLFH :LWKLQ WKH SDVW VHYHUDO \HDUV QHXURSV\FKRORJLFDO WHVW SHUIRUPDQFH KDV EHHQ XVHG VRPHZKDW OHVV IUHTXHQWO\ WR ORFDOL]H QHXURORJLFDO LQVXOW GXH WR WKH FRQWLQXLQJ JURZWK DQG DGYDQFH RI LPDJLQJ WHFKQRORJ\ 5XII f :KLOH WKH XVH RI LPDJLQJ GHYLFHV VXFK DV SRVLWURQ HPLVVLRQ WRSRJUDSK\ 3(7f PDJQHWLF UHVRQDQFH LPDJLQJ 05,f DQG IXQFWLRQDO PDJQHWLF UHVRQDQFH LPDJLQJ I05,f KDV FRQWLQXHG WR KRQH WKH DELOLW\ RI WKH LQYHVWLJDWRU WR YLHZ WKH EUDLQ LWVHOI LQ D QRQREWUXVLYH ZD\ DQG KDYH FRQWLQXHG WR OHQG WKHPVHOYHV HIIHFWLYHO\ WR WKH GLDJQRVWLF SURFHVV QHXURSV\FKRORJLFDO WHVW UHVXOWV KDYH SOD\HG D VRPHZKDW GHFUHDVHG UROH LQ WKLV FDSDFLW\ +RZHYHU WKH IRFXV RI QHXURSV\FKRORJLFDO H[DPLQDWLRQ DQG DQDO\VLV RI WHVW SHUIRUPDQFH KDYH FOHDUO\ SURYHQ WKHLU PHULW LQ RWKHU ZD\V )RU H[DPSOH WKH PHDVXUHPHQW RI FRJQLWLYH DELOLWLHV DQG LQKHUHQW VWUHQJWKV DQG ZHDNQHVVHV ZLWKLQ WKH LQGLYLGXDO RIIHUV WKH QHXURSV\FKROJLVW WKH XQLTXH RSSRUWXQLW\ WR FRQWULEXWH WR WKH SUDFWLFDO NQRZOHGJH DERXW WKH FRJQLWLYH IXQFWLRQLQJ RI D SDWLHQW DQG DOORZV KLP RU KHU WR UHFRPPHQG VWUDWHJLHV GHVLJQHG WR DLG WKH LQGLYLGXDO LQ UHWXUQLQJ WR SUHYLRXV OHYHOV ,Q WKH HYHQW WKDW D EDVHOLQH PHDVXUH RI DQ LQGLYLGXDOfV SHUIRUPDQFH LV DYDLODEOH D FRPSDULVRQ ZLWK SRVWHYHQW ILQGLQJV DOORZV IRU

PAGE 11

DQ REMHFWLYH UHYLHZ RI UHVXOWLQJ FRJQLWLYH GHILFLWV DQG D JDXJH RI FKDQJLQJ FRJQLWLYH VWDWXV ,Q WKLV VHQVH D PHDVXUH RI LQGLYLGXDO FRJQLWLYH FKDQJH RYHU WLPH KDV EHFRPH D PDMRU IRFXV RI QHXURSV\FKROJLFDO PHDVXUHV :LWK WKLV LQFUHDVLQJ IRFXV RQ ORQJLWXGLQDO IROORZXS DQG UHSHDWHG WHVWLQJ VHYHUDO LVVXHV UHJDUGLQJ VHULDO WHVW DGPLQLVWUDWLRQ DQG WKH VHQVLWLYLW\ RI HVWDEOLVKHG QHXURSV\FKRORJLFDO PHDVXUHV KDYH DULVHQ 5HFHQWO\ WKH XVH RI UHSHDWHG DVVHVVPHQW RYHU WLPH KDV EHHQ DSSOLHG LQ YDULRXV FLUFXPVWDQFHV LQ RUGHU WR PHDVXUH WKH SURJUHVVLYH QDWXUH RI FRJQLWLYH FKDQJH 7KLV KDV EHHQ WKH FDVH DV 0F6ZHHQ\ HW DO f SRLQWHG RXW LQ WKH DVVHVVPHQW RI WKH GHPHQWLDV ZKLFK LQYROYH WKH LQVLGLRXV ORVV RI FRJQLWLYH DELOLWLHV GXH WR WKH SURJUHVVLYH DOWHUDWLRQ RI EUDLQ VWUXFWXUH RU FKHPLVWU\ 0HDVXUHV RI QHXURSV\FKRORJLFDO FKDQJH RYHU WLPH KDYH DOVR EHHQ XVHG WR DVVHVV WUHDWPHQW DIIHFWV DQG LQWHUYHQWLRQV 5HFHQWO\ WKH XVH RI QHXURSV\FKROJLFDO WHVWLQJ KDV EHHQ DSSOLHG WR WKH VWXG\ RI RQH RI WKH PRVW FRPPRQ FDXVHV RI FRJQLWLYH FKDQJH DQG FRQWLQXLQJ VHTXHODH KHDG LQMXU\ :KLOH WKH UHVXOWV RI PRGHUDWH WR VHYHUH KHDG LQMXU\ DUH RIWHQ REYLRXV ERWK WR FXUUHQW LPDJLQJ WHFKQLTXHV DQG PHDVXUHV RI FRJQLWLYH VWDWXV DQG WKH UHFRYHU\ IURP VXFK LV RIWHQ SURORQJHG DQG VRPH HIIHFWV SHUPDQHQW WKH HIIHFWV RI PRUH VXEWOH RU PLOG KHDG LQMXULHV DUH QRW VR HDV\ WR XQLODWHUDOO\ LGHQWLI\ 7KHUHIRUH WKH XVH RI QHXURSV\FKRORJLFDO PHDVXUHV LQ WKLV UHJDUG KDV EHFRPH PRUH YDOXDEOH 0LQRU +HDG ,QMXU\ 'HILQLWLRQ :KLOH VHYHUDO UHVHDUFKHUV KDYH HQGHDYRUHG WR HOXFLGDWH WKH FRQVHTXHQW HIIHFWV RI PLQRU KHDG LQMXU\ WKH ODFN RI FRQVLVWHQF\ LQ GHILQLQJ WKH SKHQRPHQRQ KDV PDGH LW GLIILFXOW LI QRW LPSRVVLEOH WR UHFRQFLOH WKH ILQGLQJV RI GLIIHUHQW LQYHVWLJDWLRQV 7KH XVH RI WKH WHUP fPLQRU KHDG LQMXU\f JHQHUDOO\ VXJJHVWV WKH RFFXUUHQFH RI PLOG WR PRGHUDWH

PAGE 12

WUDXPDWLF EUDLQ LQMXU\ .LEE\ DQG /RQJ f +RZHYHU WKHUH KDV EHHQ OLWWOH DJUHHPHQW RQ WKH WHUPLQRORJ\ XWLOL]HG LQ GHVFULELQJ YDULRXV OHYHOV RI VHYHULW\ ZLWKLQ WKLV EURDG FDWHJRU\ $ UHYLHZ RI WKH OLWHUDWXUH UHYHDOV D EDVLF IUDPHZRUN XQGHUO\LQJ WKH ZRUNLQJ PRGHO RI PLQRU KHDG LQMXU\ +RZHYHU IURP WKLV EDVLF RXWOLQH PDQ\ VOLJKW YDULDWLRQV KDYH EHHQ FUHDWHG DQG XVHG LQ H[SHULPHQWDO GHVLJQ ,Q JHQHUDO D PLQRU KHDG LQMXU\ LV FRQVLGHUHG WR EH RQH LQ ZKLFK D IRUFH LV DSSOLHG WR WKH KHDG SURGXFLQJ SRVWWUDXPDWLF PHPRU\ GLIILFXOWLHV RU RWKHU IDVWUHVROYLQJ QHXURORJLFDO V\PSWRPVf .LEE\ DQG /RQJ f ,Q H[DPLQLQJ WKH RULJLQV RI WKLV YHU\ EDVLF IRXQGDWLRQ IRU GHILQLQJ PLQRU KHDG LQMXU\ .LEE\ DQG /RQJ f UHIHU WR UHVHDUFK ILQGLQJV VXJJHVWLQJ WKDW WKH VHYHULW\ RI KHDG LQMXU\ LV GLUHFWO\ UHODWHG WR WKH IRUFH WKDW LV DSSOLHG WR LW 7KH\ LQGLFDWH WKDW WKLV ILQGLQJ LV VXSSRUWHG E\ WKH ZRUN RI *UDKDP HW DO f ZKLFK VXJJHVWHG WKDW WKH GHJUHH RI D[RQDO LQMXU\ LV QDWXUDOO\ DOVR DIIHFWHG E\ WKH DPRXQW DQG GLUHFWLRQ RI IRUFH DSSOLHG ,Q RWKHU ZRUGV WKH DPRXQW RI IRUFH DSSOLHG WR WKH KHDG LV GLUHFWO\ FRUUHODWHG ZLWK WKH DPRXQW RI GDPDJH H[SHFWHG WR EH REVHUYHG LQ WKH D[RQV RI WKH EUDLQ DQG WKH DPRXQW RI PHWDEROLF GLVUXSWLRQ 0F,QWRVK HW DO 0HDQH\ HW DO f ,Q D QDWXUDO SURJUHVVLRQ RQH PLJKW DVVXPH WKDW WKLV DOVR DSSOLHV WR DQ\ REVHUYHG FRJQLWLYH G\VIXQFWLRQ ZKLFK PD\ EH GLUHFWO\ UHODWHG WR WKH H[WHQW RI D[RQDO LQMXU\ LQFXUUHG .LEE\ DQG /RQJ VXJJHVW WKDW WKLV LV VKRZQ WKURXJK WKH ZRUN RI /DQJILWW DQG *HQQDUHOOL f DQG /HYLQ HW DO f ,Q WKHLU UHVSHFWLYH LQYHVWLJDWLRQV WKHVH DXWKRUV FRQFOXGHG WKDW D PLQRU KHDG LQMXU\ UHVXOWLQJ LQ D ORVV RI FRQVFLRXVQHVV RI OHVV WKDQ PLQXWHV ZDV QRW DVVRFLDWHG ZLWK DQ\ OLQJHULQJ GHILFLWV RU GHWHFWDEOH QHXURDQDWRPLFDO GDPDJH 7KHUHIRUH WKH H[WHQW RI WKH KHDG LQMXU\ ZDV DVVXPHG WR EH PLQRU

PAGE 13

$VLGH IURP WKH DGRSWLRQ RI VRPH FRPPRQ ILQGLQJV WR WKH GHILQLWLRQ RI PLQRU KHDG LQMXU\ RQH RI WKH PRVW FRPPRQO\ XVHG LQGLFDWRUV RI KHDG LQMXU\ VHYHULW\ LV WKH *ODVJRZ &RPD 6FDOH *&6f -HQQHWW DQG %RQG f 7KH *&6 LV D PHDVXUH FRPPRQO\ HPSOR\HG DW WKH VFHQH RI D KHDG LQMXU\LQGXFLQJ DFFLGHQW E\ LPPHGLDWH FDUH SURYLGHUV RU LQ DQ HPHUJHQF\ PHGLFLQH VLWXDWLRQ ,Q SHUIRUPLQJ WKLV DVVHVVPHQW WHFKQLTXH WKH GHJUHH RI FRPD LV PHDVXUHG E\ HYDOXDWLQJ PRWRU YHUEDO DQG H\HRSHQLQJ UHVSRQVHV WR D VHW RI VWDQGDUG VWLPXOL 7KH UHVXOWLQJ VFRUH LV EDVHG RQ D VFDOH IURP WR WKH ORZHVW VFRUH UHSUHVHQWLQJ QR UHVSRQVH WR YHUEDO FRPPDQGV RU VRPDWLF VWLPXODWLRQ DQG WKH KLJKHVW VXJJHVWLQJ WKH LQWDFW DELOLW\ WR IROORZ YHUEDO FRPPDQGV RULHQWDWLRQ DQG REYLRXV DZDUHQHVV RI VHQVRU\ LQSXW ,Q WKH FXUUHQW OLWHUDWXUH LW LV JHQHUDOO\ VXJJHVWHG WKDW D PLQRU KHDG LQMXU\ LV RQH WKDW UHVXOWV LQ D *&6 RI VFRUHV WKDW DUH RIWHQ FRUUHODWHG ZLWK D SRVLWLYH ORQJWHUP RXWFRPH +RZHYHU WKH SUDFWLFH RI XVLQJ WKH *&6 E\ LWVHOI LQ WKH DVVHVVPHQW RI LQMXU\ VHYHULW\ LV TXHVWLRQDEOH VLQFH WKH ZRUN RI UHVHDUFKHUV VXFK DV &DQWX f DQG 6WUDXVV DQG 6DYLWVN\ f KDV VXJJHVWHG WKDW V\PSWRPV VXEVHTXHQW WR LQMXU\ ZLWKRXW WKH REVHUYDWLRQ RI ORVV RI FRQVFLRXVQHVV DUH DOVR FRPPRQ $QRWKHU FRPPRQO\ XVHG LQGLFDWRU RI KHDG LQMXU\ VHYHULW\ LV SRVWWUDXPDWLF DPQHVLD ZKLFK KDV EHHQ GHILQHG DV WKH SHULRG RI WLPH IROORZLQJ WKH DSSOLHG WUDXPD WR WKH EUDLQ LQ ZKLFK QHZ PHPRULHV DUH XQDEOH WR EH VWRUHG 7KLV WLPH SHULRG LV DVVRFLDWHG ZLWK WKH LQDELOLW\ WR UHFDOO HYHQWV VHTXHQFH WLPH RU OHDUQ QHZ LQIRUPDWLRQ $KPHG HW DO 0F$OOLVWHU 0F)DUODQG HW DO f $ FRUUHODWLRQ KDV EHHQ QRWHG EHWZHHQ WKH OHQJWK RI 37$ DQG WKH GXUDWLRQ RI /2& WKRXJK 0F$OOLVWHU LQGLFDWHV WKDW WKLV UHODWLRQVKLS LV QRW DOZD\V OLQHDU DQG ZDUQV DJDLQVW XVLQJ HLWKHU PHDVXUH DV WKH VROH SUHGLFWRU RI KHDG LQMXU\ VHYHULW\ DQG SRWHQWLDO RXWFRPH

PAGE 14

:KLOH VRPH FRPPRQDOLWLHV H[LVW EHWZHHQ UHVHDUFK SURMHFWV LQ WKHLU LQFOXVLRQ FULWHULD IRU FDVHV RI PLQRU KHDG LQMXU\ D VLJQLILFDQW DPRXQW RI VOLJKW YDULDWLRQ DURXQG WKH FHQWUDO WKHPH KDV UHPDLQHG 7KH UHDVRQV IRU WKLV RFFXUUHQFH PD\ EH PDQ\ IURP WKHRUHWLFDO GLIIHUHQFHV WR WKH XWLOL]DWLRQ RI VDPSOHV RI FRQYHQLHQFH RU WKH XVH RI QRQSURVSHFWLYH UHVHDUFK GHVLJQV 5HVHDUFKHUV KDYH KLVWRULFDOO\ QRW DSSOLHG D FRPPRQ GHILQLWLRQ WR WKHLU LQFOXVLRQ FULWHULD ZKHQ VWXG\LQJ PLQRU KHDG LQMXU\ )RU LQVWDQFH LQ WKH RIWHQFLWHG VWXG\ E\ 5LPHO HW DO LQ SDWLHQWV ZHUH LQFOXGHG DV FDVHV RI PLQRU KHDG LQMXU\ LI WKH\ KDG /2& RI OHVV WKDQ PLQXWHV 7KLV JURXS DGGHG DQRWKHU VSHFLILHU WR WKH EDVLF GHILQLWLRQ ,Q RUGHU WR FRQWURO IRU FRJQLWLYH FKDQJHV REVHUYHG WKDW FRXOG SRVVLEO\ KDYH EHHQ UHODWHG WR RWKHU HYHQWUHODWHG WUDXPD 5LPHO HW DO H[FOXGHG SDWLHQWV ZLWK D KRVSLWDO VWD\ RI JUHDWHU WKDQ WKUHH GD\V ,W ZDV WKHLU VXSSRVLWLRQ WKDW D ORQJHU KRVSLWDO VWD\ ZDV LQGLFDWLYH RI PRUH VHYHUH LQMXULHV WKDW PLJKW LQWURGXFH FRQIRXQGV WR UHVXOWV REWDLQHG WKDW RWKHUZLVH ZRXOG KDYH EHHQ DWWULEXWHG WR WKH KHDG LQMXU\ DORQH $QRWKHU GHILQLWLRQ ZDV SURYLGHG E\ WKH &HQWHUV IRU 'LVHDVH &RQWURO LQ DQG ZKLOH WKLV GHILQLWLRQ LV QRW VSHFLILF WR PLQRU KHDG LQMXU\ D VHQVH RI WKH YDJXH ERXQGDULHV WKDW PLJKW EH LQIHUUHG IURP LW LV VWLOO UHFRJQL]DEOH 7KH &'& VWDWHV WKDW D WUDXPDWLF EUDLQ LQMXU\ LV WKH fRFFXUUHQFH RI LQMXU\ WR WKH KHDG DULVLQJ IURP EOXQW RU SHQHWUDWLQJ WUDXPD RU IURP DFFHOHUDWLRQGHFHOHUDWLRQ IRUFHV WKDW LV DVVRFLDWHG ZLWK DQ\ RI WKHVH V\PSWRPV RU VLJQV DWWULEXWDEOH WR WKH LQMXU\ GHFUHDVHG OHYHO RI FRQVFLRXVQHVV DPQHVLD RWKHU QHXURORJLF RU QHXURSV\FKRORJLF DEQRUPDOLWLHVf7KXUPDQ HW DO f 6XEVHTXHQW UHVHDUFKHUV KDYH LQFOXGHG SDWLHQWV ZKR PHW VRPH EXW QRW DOO RI WKHVH FULWHULD RWKHUV KDYH LQFOXGHG SDWLHQWV ZLWK PRUH VHYHUH LQMXULHV DQG FRQIRXQGLQJ QHXURORJLFDO KLVWRULHV 'LNPHQ HW DO /HLQLQJHU HW DO /HYLQ HW DO f

PAGE 15

:KDW LV FOHDU LQ WKLV UDWKHU FORXG\ SLFWXUH LV WKDW WKH RQO\ FRQVLVWHQF\ LV LQFRQVLVWHQF\ 7DEOH LQGLFDWHV WKH YDULDELOLW\ LQ GHILQLQJ PLQRU KHDG LQMXU\ DPRQJ VRPH RI WKH PRVW FLWHG DXWKRUV LQ WKLV ILHOG 7DEOH 9DU\LQJ LQFOXVLRQ FULWHULD IRU PLQRU KHDG LQMXU\ $XWKRU /2& 37$ *&6 %DUWK HW DO f 'LNPHQ HW DO f 3DUNHU f $OH[DQGHUf .LQJ f PLQXWHV KRXU PLQXWHV PLQXWHV PLQXWHV KRXU KRXUV ! /2& ORVV RI FRQVFLRXVQHVV 37$ SRVWWUDXPDWLF DPQHVLD *&6 *ODVJRZ &RPD 6FDOH ,Q DQ DWWHPSW WR DVVHPEOH WKHVH YDULRXV FULWHULD WKH EDVLF GHILQLWLRQ RI PLQRU KHDG LQMXU\ DV RXWOLQHG E\ .LEE\ DQG /RQJ f LV DV IROORZV $ PLQRU KHDG LQMXU\ LV LQFXUUHG E\ DQ\ EORZ WR WKH KHDG UHVXOWLQJ LQ D ORVV RI FRQVFLRXVQHVV RI OHVV WKDQ PLQXWHV DQ LQLWLDO *&6 VFRUH RI JUHDWHU WKDQ RU HTXDO WR DQG SRVWWUDXPDWLF DPQHVLD RI OHVV WKDQ KRXUV 7KLV GHILQLWLRQ ZKLOH VHUYLQJ DV D VROLG EDVLV IRU WKH QRVRORJ\ RI PLQRU KHDG LQMXU\ DQG LQGLFDWLQJ WKH XSSHU OLPLWV RI ZKDW LV WKRXJKW WR FRQVWLWXWH PLQRU KHDG LQMXU\ IDLOV WR GHILQH WKH ERXQGDULHV DW WKH PLQLPXP GHJUHH RI WKDW ZKLFK FDQ EH FRQVLGHUHG LQ WKLV FDWHJRU\ 5HFRJQL]LQJ WKH YDULDELOLW\ LQ GHILQLQJ PLQRU KHDG LQMXU\ IURP RQH VWXG\ WR WKH QH[W D GHILQLWLRQ RI PLOG WUDXPDWLF EUDLQ LQMXU\ 07%,f ZDV IRUPXODWHG E\ WKH 0LOG 7UDXPDWLF %UDLQ ,QMXU\ &RPPLWWHH RI WKH +HDG ,QMXU\ ,QWHUGLVFLSOLQDU\ 6SHFLDO ,QWHUHVW *URXS RI WKH $PHULFDQ &RQJUHVV RI 5HKDELOLWDWLRQ 0HGLFLQH f $FFRUGLQJ WR WKH FRQVHQVXV UHDFKHG E\ WKLV RUJDQL]DWLRQ 07%, LV GHILQHG E\ DW OHDVW RQH RI WKH IROORZLQJ

PAGE 16

DQ\ SHULRG RI ORVV RI FRQVFLRXVQHVV DQ\ ORVV RI PHPRU\ IRU HYHQWV LPPHGLDWHO\ EHIRUH RI DIWHU WKH DFFLGHQW DQ\ DOWHUDWLRQ RI PHQWDO VWDWH DW WKH WLPH RI WKH DFFLGHQW IRFDO QHXURORJLFDO GHILFLWV ZKLFK PD\ RU PD\ QRW EH WUDQVLHQW EXW ZKHQ WKH VHYHULW\ RI WKH DFFLGHQW GRHV QRW H[FHHG WKH IROORZLQJ D ORVV RI FRQVFLRXVQHVV RI PLQXWHV RU OHVV E DIWHU PLQXWHV DQ LQLWLDO *&6 VFRUH RI DQG F SRVWWUDXPDWLF DPQHVLD QRW JUHDWHU WKDQ KRXUV %XVFK DQG $OSHP f 7KLV DWWHPSW DW SURYLGLQJ D XQLIRUP GHILQLWLRQ ZKLOH DJDLQ FOHDUO\ GHOLQHDWLQJ DQ XSSHU OLPLW IRU V\PSWRPV LQGLFDWLYH RI PLQRU KHDG LQMXU\ IDLOV WR DFFRXQW IRU WKH PLQLPDO V\PSWRPDWRORJ\ QHFHVVDU\ WR TXDOLI\ DV VXFK 7KH PLQLPXP OLPLWV RI WKLV GHILQLWLRQ LQFOXGH D ZLGH JUDGDWLRQ RI LQMXULHV IRU ZKLFK WKH GHILQLWLRQ LWVHOI GRHV QRW DFFRXQW RU HQXPHUDWH :LWKLQ WKLV FDWHJRU\ LV WKH W\SH RI LQMXU\ NQRZQ DV FRQFXVVLRQ ZKLFK DFFRUGLQJ WR WKH 0LOG 7UDXPDWLF %UDLQ ,QMXU\ 6XEFRPPLWWHH DOVR IDOOV XQGHU WKH JHQHUDO UXEULF RI PLQRU KHDG LQMXU\ &RQFXVVLRQ KDV FRPH WR EH GHILQHG DV D fFOLQLFDO V\QGURPH FKDUDFWHUL]HG E\ LPPHGLDWH DQG WUDQVLHQW SRVWWUDXPDWLF LPSDLUPHQW RI QHXUDO IXQFWLRQf 0DGGRFNV DQG 6DOLQJ f ,Q JHQHUDO WKRVH VXIIHULQJ IURP FRQFXVVLRQ DUH WKRXJKW WR UHFRYHU ZLWKRXW UHVLGXDO LPSDLUPHQW 7KH FDWHJRUL]DWLRQ RI FRQFXVVLRQ VHYHULW\ ZDV WKH WRSLF RI D UHFHQW VHW RI JXLGHOLQHV DXWKRUHG E\ WKH 4XDOLW\ 6WDQGDUGV 6XEFRPPLWWHH RI WKH $PHULFDQ $FDGHP\ RI 1HXURORJ\ $$1f LQ FLWHG LQ +LQWRQ%D\UH HW DO f ,Q WKLV FODVVLILFDWLRQ VFKHPH WKH VHYHULW\ RI FRQFXVVLRQ LV EDVHG XSRQ D JUDGHG VFDOH UDQJLQJ IURP WKH OHDVW VHYHUH WR WKH PRVW VHYHUH *UDGH FRQFXVVLRQV DUH WKRVH LQ ZKLFK WKH LQGLYLGXDO GRHV QRW ORVH FRQVFLRXVQHVV DV WKH UHVXOW RI WKH LQVXOWLQJ IRUFH EXW H[SHULHQFHV WUDQVLHQW FRQIXVLRQ LQDWWHQWLRQ SRRU FRQFHQWUDWLRQ RU LQDELOLW\ WR SURFHVV LQIRUPDWLRQ RU VHTXHQFH WDVNVf WKDW UHVROYHV LQ PLQXWHV RU OHVV *UDGH FRQFXVVLRQV DUH WKRVH LQ ZKLFK OLNH JUDGH FRQFXVVLRQV WKH LQGLYLGXDO GRHV QRW ORVH FRQVFLRXVQHVV

PAGE 17

EXW H[SHULHQFHV WUDQVLHQW FRQIXVLRQ WKDW GRHV QRW UHPLW ZLWKLQ D PLQXWH WLPH SHULRG 7KH PRVW VHYHUH FRQFXVVLRQ JUDGH DFFRUGLQJ WKH $$1 JXLGHOLQHV LQFOXGHV D SHULRG RI ORVV RI FRQVFLRXVQHVV ZKLFK PD\ ODVW VHFRQGV WR PLQXWHV DV ZHOO DV WUDQVLHQW QHXURORJLFDO GLVUXSWLRQ .HOO\ DQG 5RVHQEHUJ f 7KHVH WUDQVLHQW QHXURORJLFDO V\PSWRPV KDYH DOVR EHHQ WKH VXEMHFW RI PDQ\ UHVHDUFK LQYHVWLJDWLRQV 3DWLHQWV VXIIHULQJ IURP PLQRU KHDG LQMXULHV LQFOXGLQJ FRQFXVVLRQ RIWHQ FRPSODLQ RI D FOXVWHU RI V\PSWRPV ZKLFK ZDV UHFRJQL]HG E\ 6WUDXVV DQG 6DYLWVN\ f ,Q WKHLU VWXG\ WKHVH LQYHVWLJDWRUV UHIHUUHG WR WKLV FRPSOH[ RI V\PSWRPV DV fSRVWFRQFXVVLRQ V\QGURPHf :DWVRQ HW DO f 7KLV FOXVWHU LQFOXGHV WKRVH V\PSWRPV PHQWLRQHG DERYH SOXV PDQ\ RWKHUV ZKLFK PD\ SHUVLVW IRU DV ORQJ DV ZHHNV RU PRQWKV DIWHU LQMXU\ 7KH FXUUHQW VWXG\ ZLOO IRFXV RQ WKH QHXURSV\FKRORJLFDO HIIHFWV REVHUYHG LQ FRQFXVVLRQV LQFXUUHG LQ D FROOHJH IRRWEDOO SRSXODWLRQ 0LQRU +HDG ,QMXU\ 3DWKRORJ\ 5HVHDUFKHUV KDYH ORQJ VRXJKW WR LGHQWLI\ WKH FRUH IHDWXUHV DQG XQGHUO\LQJ PHFKDQLVP IRU WKH FRJQLWLYH VHTXHODH WKDW DUH HYLGHQFHG IROORZLQJ PLOG WUDXPDWLF EUDLQ LQMXU\ .H\ WR WKLV VHDUFK LV WKH IDFW WKDW OLWWOH HYLGHQFH KDV VXUIDFHG UHJDUGLQJ WKH ELRORJLFDO FKDQJHV WKDW DUH WKRXJKW WR RFFXU LQ WKHVH LQVWDQFHV 0RQWJRPHU\ HW DO f /LVKPDQ f UHYLHZV VRPH RI WKH HDUO\ UHVHDUFK LQYROYLQJ DQLPDO PRGHOV RI FRQFXVVLRQ VWXG\ LQ WKH VHDUFK IRU WKH SDWKRSK\VLRORJLFDO URRWV RI FRQFXVVLRQ V\PSWRPDWRORJ\ ,QLWLDOO\ KH VWDWHV FHUHEUDO LVFKDHPLD DV D UHVXOW RI D EORZ WR WKH KHDG ZDV WKRXJKW WR EH UHVSRQVLEOH IRU WKH SDWWHUQ RI V\PSWRPV REVHUYHG 7KLV ZDV K\SRWKHVL]HG WR DULVH IURP WKH ULVH LQ LQWUDFUDQLDO SUHVVXUH DQG REVWUXFWLRQ RI LQWUDFUDQLDO FLUFXODWLRQ :KLOH WKLV K\SRWKHVLV WKHRUHWLFDOO\ DFFRXQWHG IRU WKH EULHI ORVV RI

PAGE 18

FRQVFLRXVQHVV REVHUYHG LQ PDQ\ FDVHV RI PLQRU KHDG LQMXU\ ODERUDWRU\ ILQGLQJV ZHUH QRW VXSSRUWLYH DQG RWKHU WKHRULHV ZHUH VXEVHTXHQWO\ IRUPXODWHG /LVKPDQ f SRLQWV WR WKH FRQWULEXWLRQ RI 'HQQ\%URZQ DQG 5XVVHOO LQ FRQFHUQLQJ WKH WKHRU\ WKDW VKHDU VWUDLQ UHVXOWDQW IURP URWDWLRQDO IRUFHV DUH LQ VRPH SDUW UHVSRQVLEOH IRU WUDQVLHQW FRJQLWLYH GHILFLWV LQ PLOG KHDG LQMXU\ 7KH VWXGLHV RI WKHVH UHVHDUFKHUV RQ WKH HIIHFW RI DFFHOHUDWLRQGHFHOHUDWLRQ HIIHFWV XSRQ WKH EUDLQ LWVHOI SURPXOJDWHG IXUWKHU VWXGLHV RI WKH HIIHFW RI WKHVH IRUFHV 'HQQ\%URZQfV VWXGLHV ZLWK DQLPDOV VXJJHVWHG WKDW ZKDW KH GHVFULEHG DV fXQFRPSOLFDWHGf PLOG KHDG LQMXU\ RU FRQFXVVLRQ ZDV DFKLHYHG ZLWK KLV VXEMHFWV RQO\ ZKHQ WKH KHDG ZDV IUHH WR PRYH DERXW ZKHQ WKH DFWXDO LQVXOW RFFXUUHG +LV FRQFOXVLRQV WKHUHIRUH LPSOLFDWHG WKH DFFHOHUDWLRQ DQG GHFHOHUDWLRQ IRUFHV WKDW ZHUH DSSOLHG WR WKH VNXOO GXULQJ WKH FRXUVH RI WKH LQMXU\ ,W ZDV DW WKLV SRLQW WKDW UHVHDUFKHUV EHJDQ WR DSSUHFLDWH WKH IOXLGLW\ RI WKH HQYLURQPHQW ZLWKLQ WKH FUDQLXP 7KDW LV LQYHVWLJDWRUV PRYHG WRZDUG H[SHULPHQWDO GHVLJQV WKDW ZRXOG DOORZ WKHP WR GLUHFWO\ REVHUYH DQG PHDVXUH WKRVH IRUFHV DFWLQJ ZLWKLQ WKH VNXOO LWVHOI /LVKPDQ f GHVFULEHV WKH VWXGLHV RI 3XGHQ] DQG 6KHOGHQ LQ RQ PRQNH\V LQ ZKLFK WKH VNXOOFDSV ZHUH UHPRYHG DQG UHSODFHG ZLWK WUDQVSDUHQW PDWHULDO %\ VR GRLQJ WKHVH UHVHDUFKHUV ZHUH DEOH WR REVHUYH WKH PRYHPHQW RI WKH EUDLQ DW WKH PRPHQW RI LPSDFW 7KH\ LQGLFDWHG LQ WKHLU UHSRUW WKDW WKH\ REVHUYHG D fVZLUOLQJf PRWLRQ ERWK DW WKH SRLQW RI LPSDFW DQG LQ RWKHU DUHDV RI WKH EUDLQ QRW GLUHFWO\ LPSDFWHG E\ WKH EORZ ,W ZDV WKLV VZLUOLQJ PRWLRQ WKDW ZDV WKRXJKW WR SURGXFH PDQ\ RI WKH V\PSWRPV DVVRFLDWHG ZLWK FRQFXVVLRQ 6LQFH WKH HDUO\ ZRUN RI WKHVH UHVHDUFKHUV PDQ\ RWKHUV KDYH FRQILUPHG DQG HODERUDWHG XSRQ WKHVH K\SRWKHVHV 7KH EUDLQ FDQ EH WKRXJKW RI DV D VHPLJHODWLQRXV PDVV

PAGE 19

WKDW LV VXVSHQGHG LQ D IOXLG ZLWKLQ D FRQILQHG VSDFH ,W LV KHOG LQ SODFH ERWK E\ LWV OLPLWHG KRXVLQJ ZLWKLQ WKH VNXOO DQG WKURXJK LWV FRQQHFWLRQ YLD WKH EUDLQVWHP DQG VSLQDO FRUG %HFDXVH LW LV fIORDWLQJf LQ D KRPHRVWDWLF DPRXQW RI FHUHEURVSLQDO IOXLG &6)f DQG fWHWKHUHG LW PRYHV RXW RI SKDVH UHODWLYH WR WKH VNXOO LQ UHVSRQVH WR FKDQJHV LQ VSHHG RU GLUHFWLRQf0F$OOLVWHU f 7KH DQDWRPLFDOO\ GLVUXSWLQJ HIIHFWV VHHQ LQ PLOG KHDG LQMXU\ DUH WKRXJKW WR EH GLUHFWO\ FRUUHODWHG ZLWK WKH IRUFH RI WKH LPSDFW LPSDUWHG WR WKH KHDG )XUWKHU UHODWLYHO\ IHZ LQVWDQFHV RI FRQFXVVLRQ KDYH EHHQ QRWHG LQ FDVHV RI VWDWLF LQMXU\ 7KDW LV ZKHQ WKH LQGLYLGXDO LV QRW IUHH WR PRYH KLV KHDG WKH UHVXOWV RI PLQRU KHDG LQMXU\ KDYH QRW EHHQ IRXQG WR EH DV VXEVWDQWLDO .LQJVOH\ 0F$OOLVWHU f 7KHUHIRUH LW ZRXOG DSSHDU WKDW WKH ULVN RI LQFXUULQJ FRPSOLFDWLRQV DV D UHVXOW RI DSSOLHG IRUFHV LV JUHDWO\ LQFUHDVHG ZKHQ WKH KHDG GRHV QRW KDYH D VXEVWDQFH WKURXJK ZKLFK LW PD\ FRQGXFW D SRUWLRQ RI WKH IRUFH YHFWRUV DFWLQJ XSRQ LW ,Q WKH FDVH RI PLQRU KHDG LQMXU\ WKH PRVW FRPPRQ LQVXOWV DUH FODVVLILHG DV GLIIXVH ,Q WKLV VHQVH D GLIIXVH LQMXU\ LV RQH LQ ZKLFK D GHILQLWLYH ORFDWLRQ RI IRFDO LPSDFW FDQQRW EH LGHQWLILHG %HFDXVH PLQRU KHDG LQMXU\ UDUHO\ UHVXOWV LQ IDWDOLW\ LW LV RIWHQ LPSRVVLEOH WR GHWHFW WKH VXEWOH QHXURDQDWRPLFDO FKDQJHV LQKHUHQW WR WKH LQMXU\ 0RQWJRPHU\ HW DO f VXJJHVWHG WKDW RQH RI WKH ILUVW UHVHDUFKHUV WR LGHQWLI\ DQDWRPLFDO DQRPDOLHV VXEVHTXHQW WR PLQRU KHDG LQMXU\ ZDV 2SSHQKHLPHU LQ ,Q KLV VWXGLHV 2SSHQKHLPHU KDG WKH RSSRUWXQLW\ WR H[DPLQH WKH EUDLQV RI ILYH LQGLYLGXDOV ZKR KDYLQJ VXIIHUHG IURP PLOG WUDXPDWLF EUDLQ LQMXULHV GLHG RI FRPSOLFDWLRQV IURP RWKHU LQMXULHV DOVR LQFXUUHG GXULQJ WKH FDXVDWLYH LQFLGHQW 0LFURVFRSLF H[DPLQDWLRQ RI EUDLQ WLVVXH UHYHDOHG GLIIXVH fD[RQDO GLVUXSWLRQVf WKURXJKRXW WKH ZKLWH PDWWHU ,Q SDUWLFXODU KH LQGLFDWHG WKDW WKHVH LQMXULHV ZHUH PRUH FRPPRQ LQ PLGEUDLQ WHJPHQWXP DQG WKH FRUSXV

PAGE 20

FROORVXP 0RQWJRPHU\ HW DO f 6XEVHTXHQW UHVHDUFK RYHU WKH \HDUV KDV LQGLFDWHG WKDW WKLV GLIIXVH GDPDJH WR ZKLWH PDWWHU WUDFWV LV WKH SUREDEOH FDXVH RI FRJQLWLYH GLVUXSWLRQ LQ PLQRU KHDG LQMXU\ 7KH PRWLRQ RI WKH EUDLQ ZLWKLQ WKH VNXOO LV QRZ WKRXJKW WR EH UHVSRQVLEOH IRU WKH fWZLVWLQJ DQG VWUHWFKLQJf RI QHXURQDO D[RQV DQG WKH VXEVHTXHQW GDPDJH WKDW RFFXUV %HFDXVH WKLV GDPDJH FDQ EH FDXVHG E\ LQHUWLDO IRUFHV GLUHFW LPSDFW ZLWK WKH KHDG LV QRW QHFHVVDU\ WR LQGXFH GHWULPHQWDO HIIHFWV %XVFK DQG $OSHP f )XUWKHUPRUH %XVFK DQG $OSHP f SRLQW RXW WKDW GDPDJH WR D[RQV E\ VKHDULQJ IRUFHV FDXVHV WHDULQJ DQG WKH GLVUXSWLRQ RI RWKHU FHOO PLOLHX LQFOXGLQJ EORRG YHVVHOV SRVVLEO\ DOWHULQJ EORRG WUDQVSRUW WR WKH GLUHFWO\ DIIHFWHG QHXURQV DQG WKRVH LPPHGLDWHO\ VXUURXQGLQJ ,Q UHYLHZLQJ WKH KLVWRSDWKRORJ\ RI PLQRU KHDG LQMXU\ %XVFK DOVR UHIHUV WR WKH ZRUN RI 6WULFK f DQG 3RYOLVKRFN HW DO f LQ ZKLFK GDPDJH WR QHXURQV E\ VKHDU RU VWUHWFKLQJ ZDV REVHUYHG H[WHQVLYHO\ LQ WKH ZKLWH PDWWHU RI WKH KHPLVSKHUHV DQG LQ WKH EUDLQVWHP ,Q DGGLWLRQ WKHVH UHVHDUFKHUV GLVFRYHUHG WKDW UHWUDFWLRQ EDOOV FRXOG EH YLHZHG LQ WKH LQVWDQFHV ZKHUH D[RQDO ILEHUV KDG EHHQ GLVFRQQHFWHG 0F$OOLVWHU f VXJJHVWV WKDW D SUHGLOHFWLRQ IRU WKHVH W\SHV RI DQDWRPLFDO GLVUXSWLRQV RFFXUV LQ WKH FRUSXV FROORVXP VXSHULRU FHUHEHOODU SHGXQFOH EDVDO JDQJOLD DQG SHULYHQWULFXODU ZKLWH PDWWHU ,Q HDFK RI WKHVH FDVHV WKH ORQJ ZKLWH PDWWHU WUDFWV DSSHDU WR EH PRUH VXVFHSWLEOH WR WKH GDPDJLQJ HIIHFWV RI D[RQDO WZLVWLQJ WKDQ DUH RWKHU DUHDV ,Q DGGLWLRQ EHFDXVH WKH EUDLQ LV VLWXDWHG LQ D KDUG FDVLQJ WKH RSSRUWXQLW\ IRU LQMXU\ E\ LPSDFW ZLWK WKH VNXOO ZDOO LV DOVR SUHVHQW SDUWLFXODUO\ LQ WKRVH UHJLRQV YXOQHUDEOH WR EHDULQJ WKH EUXQW RI WKH W\SH RI PRWLRQ WKDW LV RIWHQ VHHQ LQ FDVHV RI PLOG KHDG LQMXU\

PAGE 21

*HQQDUHOOLfV UHVHDUFK LQ WKH DUHD RI WUDXPDWLF EUDLQ LQMXU\ KDV SURYLGHG HYLGHQFH WR VXSSRUW WKH QRWLRQ RI ZKDW KH KDV WHUPHG fFRXSFRQWUHFRXSf GLVWLQFWLRQ ,Q JHQHUDO DQ LQMXU\ UHVXOWDQW IURP D FRXS IRUFH LV REVHUYHG ORFDOO\ DW WKH VLWH RI WKH LPSDFW RI WKH KHDG ZLWK WKH RIIHQGLQJ REMHFW *HQQDUHOOL DQG *UDKDP f +H IXUWKHU DVVHUWHG WKDW WKH DUHD DQG VHYHULW\ RI GDPDJH DUH GLUHFWO\ UHODWHG WR WKH VL]H DQG IRUFH RI WKH LPSDFWLQJ REMHFW 7KHUHIRUH KH FRQWLQXHG EXW ZDUQHG DJDLQVW JHQHUDOL]DWLRQ FRXS LQMXULHV DUH RIWHQ DVVRFLDWHG ZLWK WKH DFFHOHUDWLRQ RI WKH KHDG &RQWUHFRXS LQMXULHV RQ WKH RWKHU KDQG DUH RIWHQ VHHQ DV DULVLQJ IURP WKH VXGGHQ GHFHOHUDWLRQ RI WKH KHDG 7KHVH LQVXOWV DUH fDZD\ IURP WKH VLWH RI LQMXU\f *HQQDUHOOL DQG *UDKDP f :KLOH WKH\ DUH RIWHQ REVHUYHG LQ D OLQHDU UHODWLRQ ZLWK WKH VLWH RI LPSDFW DQG IURP WKH FRXSUHODWHG LQMXU\ VLWH EHFDXVH RI WKH QDWXUH RI WKH LQWHUQDO HQYLURQPHQW RI WKH FUDQLXP WKLV LV QRW DOZD\V WKH FDVH 7KH ORFDWLRQ RI WKH EUDLQ ZLWKLQ WKH VNXOO DV PHQWLRQHG DERYH SUHGLVSRVHV FHUWDLQ DUHDV WR GDPDJH 7KHVH DUHDV LQFOXGH WKH WHPSRUDO OREH SROHV WKH IURQWDO OREH SROHV DQG WKH SDUDVDJLWWDO UHJLRQ 3DUNHU f DOO RI ZKLFK DUH WKRXJKW FRPPRQO\ DIIHFWHG WR D GHJUHH LQ PDQ\ FDVHV RI PLOG KHDG LQMXU\ ,Q WKH HDUO\ fV *HQQDUHOOL DQG KLV FROOHDJXHV VXJJHVWHG WKH WHUP fGLIIXVH D[RQDO LQMXU\f '$,f LQ WKH GHVFULSWLRQ RI WKH DQDWRPLFDO HIIHFWV REVHUYHG LQ PDQ\ LQVWDQFHV RI WUDXPDWLF EUDLQ LQMXU\ $V GHVFULEHG DERYH IRFDO GDPDJH SURGXFHG E\ D QRQPLVVLOH FORVHGKHDG LQMXU\ LV UDWKHU UDUH 0RUH RIWHQ GLIIXVH D[RQDO GDPDJH LV SUHYDOHQW ,Q HODERUDWLQJ RQ WKH GHILQLWLRQ RI '$, UHVHDUFKHUV IRUPXODWHG WKUHH FDWHJRULHV RI '$, *HQQDUHOOL DQG *UDKDP f ,Q D '$, RI JUDGH GDPDJH WR D[RQV LV QRWDEOH ZLWKLQ WKH ZKLWH PDWWHU RI WKH KHPLVSKHUHV DQG RFFDVLRQDOO\ LQ WKH EUDLQ VWHP DQG FRUSXV FROORVXP 7KLV LV PRVW OLNHO\ WKH H[WHQW RI WKH GDPDJH VHHQ LQ PRVW FDVHV RI

PAGE 22

PLQRU KHDG LQMXU\ $ JUDGH ,, '$, DOVR HYLGHQFHV VRPH fIRFDO DEQRUPDOLWLHV LQ WKH FRUSXV FROORVXP RIWHQ DVVRFLDWHG ZLWK VPDOO KHPRUUKDJHV FDOOHG WLVVXH WHDU KHPRUUKDJHVf *HQQDUHOOL DQG *UDKDP f $FFRUGLQJ WR WKLV JUDGDWLRQ V\VWHP JUDGH +, '$,fV DOVR LQFOXGH H[WHQVLYH GDPDJH WR WKH URVWUDO EUDLQVWHP ,Q D SDSHU 2SSHQKHLPHU DQG KLV FROOHDJXHV UHSRUWHG WKHLU ILQGLQJ WKDW fFOXVWHUV RI PLFURJOLDf ZHUH IRXQG LQ WKH ZKLWH PDWWHU RI SDWLHQWV ZKR KDYLQJ VXIIHUHG PLQRU KHDG LQMXU\ KDG GLHG RI XQUHODWHG FDXVHV *HQQDUHOOL VXJJHVWV WKDW WKLV ILQGLQJ KDV EHHQ VXSSRUWHG UHFHQWO\ E\ RWKHU LQYHVWLJDWRUV DQG KDV EHHQ HODERUDWHG XSRQ E\ RWKHUV ZKR KDYH GLVFRYHUHG WKH SUHVHQFH RI ZKDW KH WHUPV fPLFURVFRSLF '$,f*HQQDUHOOL DQG *UDKDP f +RZHYHU *HQQDUHOOL VXJJHVWHG WKDW D SRVVLEO\ PRUH LPSRUWDQW GLVFRYHU\ ZDV PDGH E\ %OXPEHUJV HW DO f ZKHQ WKHVH LQYHVWLJDWRUV UHSRUWHG KDYLQJ SHUIRUPHG LPPXQRVWDLQLQJ ZLWK DQ DP\ORLG SUHFXUVRU SURWHLQ $33f DQWLERG\ ,Q WKHLU UHVXOWV WKHVH UHVHDUFKHUV UHSRUWHG REVHUYLQJ PXOWLSOH DUHDV RI GLIIXVH D[RQDO LQMXU\ ZKHQ XVLQJ WKLV VWDLQLQJ WHFKQLTXH LQ ILYH SDWLHQWV ZKR KDG UHFHLYHG PLQRU KHDG LQMXU\ ([SHULPHQWDWLRQ ZLWK RWKHU DQWLERG\ VWDLQLQJ WHFKQLTXHV KDYH PHW ZLWK OHVV VXFFHVV LQ LGHQWLI\LQJ WKRVH UHJLRQV WKDW KDYH VXIIHUHG GLIIXVH D[RQDO LQMXU\ 6WDLQLQJ IRU WKH GHWHFWLRQ RI $33 UHPDLQV WKH PRVW VHQVLWLYH WR HOXFLGDWLQJ WKLV SKHQRPHQRQ *HQQDUHOOL ZDUQV WKRXJK WKDW WKLV VKRXOG QRW EH FRQVLGHUHG D PDUNHU IRU KHDG LQMXU\ VLQFH KHLJKWHQHG SURGXFWLRQ DQG FROOHFWLRQ RI $SROLSRSURWHLQ LV DOVR VHHQ LQ WKH fG\VWURSKLF D[RQVf RI VRPH HOGHUO\ SDWLHQWV DQG LQ VRPH FDVHV RI NQRZQ WUDQVLHQW LVFKHPLF DWWDFN DQG LQIDUFWLRQ +H VXJJHVWV WKDW DQ DFFXPXODWLRQ RI WKLV SUHFXUVRU SURWHLQ VKRXOG VXJJHVW DQ DUHD LQ ZKLFK PHWDEROLF FKDQJH KDV WDNHQ SODFH ZKLFK FHUWDLQO\ FRXOG EH WKH FDVH LQ PLQRU KHDG LQMXU\ WKRXJK LW PD\ DOVR EH WUXH LQ RWKHU LQVWDQFHV DV ZHOO

PAGE 23

5HFHQW ZRUN E\ 5REHUWV HW DO f KDV DOVR VXJJHVWHG D OLQN EHWZHHQ WKH DFFXPXODWLRQ RI $33 DQG WKH LQFUHDVHG ULVN IRU WKH GHYHORSPHQW RI $O]KHLPHUfV 'LVHDVH $'f 7KH ZRUN RI WKHVH UHVHDUFKHUV VXSSRUWV WKH K\SRWKHVLV WKDW WKH H[SUHVVLRQ RI $33 LV LQGHHG SDUW RI fDQ DFXWH SKDVH UHVSRQVHf WR WKH W\SH RI QHXURQDO GDPDJH WKDW LV VHHQ LQ PLQRU KHDG LQMXU\ 7KH\ IXUWKHU VXJJHVW WKDW WKH RYHUDFFXPXODWLRQ RI WKH SUHFXUVRU SURWHLQ PD\ OHDG WR D EXLOGXS RI $33 DQG VXEVHTXHQWO\ OHDG WR WKH H[SUHVVLRQ RI FHUWDLQ $' V\PSWRPV RU DQ LQFUHDVH LQ DQ LQGLYLGXDOfV SUHGLOHFWLRQ WR GHYHORS $' ODWHU LQ OLIH LI WKH\ DUH DOUHDG\ JHQHWLFDOO\ VXVFHSWLEOH ,Q DGGLWLRQ WR WKHVH LPSRUWDQW ELRFKHPLFDO ILQGLQJV %XVFK DQG $OSHP f VLWHV WKH ZRUN RI 3RYOLVKRFN HW DO f LQ ZKLFK LW ZDV IRXQG WKDW PLQRU KHDG LQMXU\ FDQ OHDG WR D GLVUXSWLRQ LQ HOHFWURO\WH WUDQVIHU DQG FDSLOODU\ SHUPHDELOLW\ :KLOH WKH QHXURDQDWRPLFDO HIIHFWV RI PLQRU KHDG LQMXU\ DUH QRW HDVLO\ LGHQWLILHG E\ FRPPRQ PHGLFDO PHDVXUHV VXFK DV LPDJLQJ GHYLFHV DQG ELRFKHPLFDO PDUNHUV UHFHQW UHVHDUFK LQ WKH DUHD RI GLIIXVH PLFURVFRSLF GDPDJH DQG WKH GHWHFWLRQ RI WKH DFFXPXODWLRQ RI DP\ORLG SUHFXUVRU SURWHLQ DQG LWV VXEVHTXHQW FRUUHODWLRQ ZLWK RXWFRPH KDYH VXJJHVWHG WR LQYHVWLJDWRUV WKDW WKHUH DUH ELRORJLFDO FRQVHTXHQFHV RI HYHQ PLQRU KHDG LQMXU\ ,W LV EHFRPLQJ PRUH FRPPRQO\ DFFHSWHG WKDW WKHVH FKDQJHV GR SOD\ D UROH LQ PDQ\ RI WKH QHXURSV\FKRORJLFDO DQG RWKHU FRJQLWLYH FKDQJHV WKDW DUH RIWHQ REVHUYHG IROORZLQJ HYHQ WKH PLOGHVW IRUPV RI KHDG LQMXU\ 1HXURSV\FKRORJLFDO (IIHFWV RI 0LQRU +HDG ,QMXU\ :KLOH WKH SV\FKRVRFLDO UDPLILFDWLRQV RI PLQRU KHDG LQMXU\ DQG WKH HIIHFW RI SHUVLVWLQJ SRVW FRQFXVVLRQ V\QGURPH V\PSWRPV KDYH UHFHLYHG D ODUJH DPRXQW RI DWWHQWLRQ LQ WKH OLWHUDWXUH WKH LVVXH RI FRJQLWLYH LPSDLUPHQW VXIIHUHG DV D UHVXOW RI PLQRU KHDG

PAGE 24

LQMXU\ KDV EHHQ RQH WKDW KDV SURGXFHG YDU\LQJ DQG GHEDWHG UHVXOWV 7KLV LV GXH LQ ODUJH SDUW WR WKH PHWKRGRORJ\ XVHG LQ GLIIHUHQW VWXGLHV +RZHYHU WKHUH DUH VRPH FRPPRQ DVSHFWV WR WKH ILQGLQJV IURP YDULRXV UHVHDUFK JURXSV WKDW VXJJHVW WKDW PLQRU KHDG LQMXU\ GRHV FRQWULEXWH WR GHWHFWDEOH FRJQLWLYH VHTXHODH 7KH VWXGLHV RI 5LPHO HW DO f DQG %DUWK HW DO f DUH RIWHQ FLWHG LQ WKH OLWHUDWXUH ZKHQ UHIHUULQJ WR WKH ILQGLQJ WKDW PLQRU KHDG LQMXU\ SURGXFHV D PHDVXUDEOH QHXURSV\FKRORJLFDO FKDQJH 'LNPHQ HW DO f 7KH ILQGLQJV E\ WKHVH WZR UHVHDUFK JURXSV XVLQJ WKH VDPH SRSXODWLRQ VXJJHVWHG WKDW WKHUH DUH QRWLFHDEOH FRJQLWLYH DV ZHOO DV SV\FKRVRFLDO GHILFLWV LQ WKRVH KDYLQJ VXIIHUHG PLQRU KHDG LQMXU\ 'LNPHQ DUJXHV WKDW WKHVH ILQGLQJV PD\ QRW EH DFFXUDWH KRZHYHU VLQFH WKHVH JURXSV XVHG ZKDW VKH GHVFULEHV DV DQ LQDSSURSULDWH QRUPDWLYH FRPSDULVRQ JURXS :KLOH WKH JURXSV ZHUH LPSDLUHG FRPSDUHG WR WKLV QRUPDWLYH JURXS VKH VWDWHV LW LV GLIILFXOW WR SDUVH RXW WKH HIIHFWV RI RWKHU IDFWRUV WKDW DUH QRW NQRZQ LQFOXGLQJ SUHPRUELG QHXURORJLFDO VWDWXV ,Q KHU RZQ VWXG\ RQ WKH HIIHFWV RI PLQRU KHDG LQMXU\ RQ QHXURSV\FKRORJLFDO IXQFWLRQLQJ 'LNPHQ LQGLFDWHV WKDW WKH ILQGLQJV ZHUH PXFK OHVV GHILQLWLYH ZKHQ WHVWLQJ KHU VXEMHFWV DW DQG PRQWKV SRVW LQMXU\ 'LNPHQ SHUIRUPHG D ZLGH UDQJH RI QHXURSV\FKRORJLFDO WHVWV RQ FRQVHFXWLYH KRVSLWDOL]HG SDWLHQWV ZKR KDG H[SHULHQFHG D SHULRG RI FRPD QRW RYHU KRXU RU LQ WKH DEVHQFH RI FRPD ZLWK 37$ RI DW OHDVW KRXU D *&6 VFRUH RI JUHDWHU WKDQ RU HTXDO WR RQ DGPLVVLRQ ZLWK QR HYLGHQFH RI FRUWLFDO RU EUDLQVWHP FRQWXVLRQ QR KLVWRU\ RI SULRU KHDG LQMXU\ RU QHXURORJLFDO LQVXOW DQG EHWZHHQ WKH DJHV RI WR 6KH XWLOL]HG D FRQWURO JURXS RI IULHQGV DQG UHODWLYHV RI WKRVH VXIIHULQJ IURP PLQRU KHDG LQMXU\ UDWKHU WKDQ WKH SXEOLVKHG QRUPDWLYH GDWD IRU WKH LQGLYLGXDOO\ DGPLQLVWHUHG WHVWV $OWKRXJK KHU UHVXOWV

PAGE 25

ZHUH QRW DV FRQFOXVLYH DV WKRVH RI 5LPHO DQG %DUWK VKH UHSRUWHG WKDW WKHUH ZDV D WUHQG IRU KHU PLQRU KHDG LQMXU\ SRSXODWLRQ WR SHUIRUP PRUH SRRUO\ RQ WHVWV RI FRQFHQWUDWLRQ DQG VKRUWWHUP PHPRU\ DW PRQWK DIWHU LQMXU\ 7KLV ZDV VXSSRUWLYH RI WKH ILQGLQJV RI D SUHYLRXV DQG RIWFLWHG VWXG\ E\ *URQZDOO DQG :ULJKWVRQ f ZKR VKRZHG WKDW WKRVH VXIIHULQJ PLQRU KHDG LQMXU\ DQG VXIIHULQJ IURP SRVW FRQFXVVLRQ V\PSWRPV WHQGHG WR GHPRQVWUDWH LPSDLUPHQW LQ WHVWV RI DWWHQWLRQ DQG SURFHVVLQJ VSHHG 6KH WHPSHUV KHU ILQGLQJ WKRXJK E\ VWDWLQJ WKDW KHU UHVXOWV PD\ QRW EH LQGLFDWLYH RI VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH JURXSV LI RQH WDNHV LQWR DFFRXQW WKH SUHPRUELG GLIIHUHQFHV LQ LQWHOOHFWXDO DELOLW\ EHWZHHQ WKHP 'LNPHQ HW DO f 6KH GLG QRW KDYH DQ\ PHDVXUHPHQWV RI EDVHOLQH IXQFWLRQLQJ ZLWK ZKLFK WR FRPSDUH KHU ILQGLQJV ,Q DQRWKHU VWXG\ RI D ODUJH JURXS RI KHDG LQMXUHG LQGLYLGXDOV SDUWLFLSDWLQJ LQ WKUHH VHSDUDWH H[SHULPHQWV 'LNPHQ DJDLQ VRXJKW WR FODULI\ WKH QHXURSV\FKRORJLFDO UDPLILFDWLRQV RI WUDXPDWLF EUDLQ LQMXU\ 'LNPHQ HW DO f 7KH HIIRUW LQFOXGHG DGXOW KHDGLQMXUHG SDWLHQWV RI ZKRP KDG D *&6 RI RQ DGPLVVLRQ ,Q WKLV VWXG\ VKH LQGLFDWHG WKDW WKH H[WHQW RI QHXURSV\FKRORJLFDO LPSDLUPHQW REVHUYHG ZDV GLUHFWO\ UHODWHG WR WKH GXUDWLRQ RI FRPD DVVRFLDWHG ZLWK WKH LQMXU\ ,Q WKH JURXS WKDW VKH ODEHOHG DV fPLOG KHDG LQMXU\f WKRVH ZLWK FRPD RI GXUDWLRQ OHVV WKDQ KRXU DQG *&6 RI RQ DGPLVVLRQf VKH IRXQG WKDW WKHUH ZDV QR VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFH IURP D WUDXPD FRQWURO JURXS LQ QHXURSV\FKRORJLFDO SHUIRUPDQFH 6KH VXJJHVWV WKDW WKH UHVXOWV DUH VLPLODU WR KHU SUHYLRXV VWXG\ RI QHXURSV\FKRORJLFDO RXWFRPH LQ PLQRU KHDG LQMXU\ ZLWK UHVSHFW WR WKH ILQGLQJ WKDW PHDVXUHV RI DWWHQWLRQ DQG VKRUWWHUP PHPRU\ VKRZ D WUHQG WRZDUG SHUVLVWLQJ GLIIHUHQFH LQ WKRVH ZLWK PLQRU KHDG LQMXU\ ,Q KHU GLVFXVVLRQ RI UHVXOWV VKH VWUHVVHG WKH IDFW WKDW PHDVXUHV RI DWWHQWLRQ DQG PHPRU\ WHQG WR VKRZ GHFOLQH

PAGE 26

ZLWK LQFUHDVLQJ VHYHULW\ RI KHDG LQMXU\ $JDLQ VKH KDG QR EDVHOLQH FRPSDUDWLYH PHDVXUHV 'LNPHQfV VWXGLHV RI WKH QHXURSV\FKRORJLFDO IXQFWLRQLQJ RI WKRVH VXIIHULQJ PLQRU KHDG LQMXU\ KDYH WDNHQ SODFH DW UHODWLYHO\ ORQJ SHULRGV DIWHU WKH LQMXULRXV LQFLGHQW PRQWK DQG \HDUf DQG KDYH VKRZQ OLPLWHG SHUVLVWLQJ GLIIHUHQFHV EHWZHHQ WKH LQMXUHG JURXS DQG WKH FRQWURO SRSXODWLRQ /LNHZLVH D PHWDDQDO\WLF UHYLHZ E\ %LQGHU HW DO f XVLQJ VWXGLHV FRPSULVHG RI VDPSOHV RI LQGLYLGXDOV ZLWK D KLVWRU\ RI PLOG KHDG WUDXPD DW OHDVW PRQWKV DIWHU UHFHLYLQJ WKHLU LQMXU\ UHVXOWHG LQ DQ RYHUDOO QRQVLJQLILFDQW HIIHFW VL]H RI ZKHQ FRPSDULQJ D FRPSRVLWH QHXURSV\FKRORJLFDO WHVW VFRUH WR FRQWURO JURXSV +RZHYHU ZKHQ WKH\ XVHG WKH G VWDWLVWLF WKH\ IRXQG D VLJQLILFDQW ILQGLQJ RI S IRU PHDVXUHV RI DWWHQWLRQ ,W ZDV WKHLU FRQFOXVLRQ WKHQ PXFK OLNH 'LNPHQfV WKDW WKH QHXURSV\FKRORJLFDO HIIHFWV LI DQ\ RI PLQRU KHDG LQMXU\ DUH JHQHUDOO\ UHVROYHG IURP WR PRQWKV SRVWLQMXU\ ,Q D VWXG\ E\ 0RQWJRPHU\ HW DO f RI FRQVHFXWLYH DGPLVVLRQV WR DQ DFFLGHQW DQG HPHUJHQF\ URRP IRU PLQRU KHDG LQMXU\ GHILQHG E\ WKLV JURXS DV DQ LQMXU\ UHTXLULQJ DQ RYHUQLJKW LQSDWLHQW KRVSLWDO VWD\ DV ZHOO DV D 37$ SHULRG RI OHVV WKDQ KRXUVf LQYHVWLJDWRUV XWLOL]HG D )RXU&KRLFH 5HDFWLRQ7LPH PHDVXUHPHQW LQ RUGHU WR GHWHFW SRVVLEOH FRJQLWLYH VHTXHODH RI WKH LQMXU\ ,Q DGGLWLRQ D PHDVXUHPHQW RI EUDLQVWHP HYRNHG SRWHQWLDO DQG VXEMHFWLYH PHDVXUH RI SRVW FRQFXVVLYH FRPSODLQWV ZHUH SHUIRUPHG RQ WKH VDPH GD\ DV WKH LQMXU\ VL[ ZHHNV ODWHU DQG DW VL[ PRQWKV SRVWLQMXU\ )HQWRQ UHSRUWHG WKDW KHDGLQMXUHG SDWLHQWV H[KLELWHG D VLJQLILFDQWO\ SURORQJHG UHDFWLRQ WLPH PHDVXUHPHQW RQ WKH GD\ RI WKH LQMXU\ ZKHQ FRPSDUHG WR D FRQWURO JURXS PDWFKHG IRU DJH VH[ DQG VRFLDO VWDWXV 5HVXOWV LQGLFDWHG fVHULDO LPSURYHPHQWf DW WKH VL[ZHHN DQG VL[

PAGE 27

PRQWK IROORZXS ,QWHUHVWLQJO\ RYHU KDOI RI WKH SDWLHQWfV LQYROYHG LQ WKLV VWXG\ VWLOO FRPSODLQHG RI SRVWFRQFXVVLRQ V\QGURPH V\PSWRPV DW WKH VL[ZHHN DQG VL[PRQWK IROORZXS DVVHVVPHQWV ,Q DGGLWLRQ PRUH WKDQ KDOI RI WKHVH SDWLHQWV H[KLELWHG ZKDW )HQWRQ FDOOHG fVLJQLILFDQW GHOD\Vf LQ EUDLQVWHP FRQGXFWLRQ GXULQJ WKH ILUVW WHVWLQJ VHVVLRQ 2WKHU VWXGLHV H[DPLQLQJ WKH FRJQLWLYH VHTXHODH RI PLQRU KHDG LQMXU\ KDYH VKRZQ WKDW PRUH VLJQLILFDQW GLIIHUHQFHV DUH GHWHFWDEOH LQ WKH DFXWH DQG HDUO\ UHFRYHU\ SKDVHV RI KHDG LQMXU\ DQG DUH RIWHQ DVVRFLDWHG ZLWK WKH GHJUHH RI SHUVLVWLQJ SRVW FRQFXVVLRQDO V\PSWRPV 7KLV LV SDUWLFXODUO\ WUXH LQ LQYHVWLJDWLRQV H[DPLQLQJ WKH HIIHFWV RI PLQRU KHDG LQMXULHV LQFXUUHG LQ FRQWDFW VSRUW UHODWHG LQFLGHQWV ZKLFK XVXDOO\ LQYROYH OHVVHU DPRXQWV RI DSSOLHG IRUFHV DFFHOHUDWLRQ DQG GHFHOHUDWLRQf DQG JHQHUDOO\ UHVXOW LQ PLOGHU FRQFXVVLYH HIIHFWV 0DGGRFNV DQG 6DOLQJ f 6SRUWVUHODWHG 0LQRU +HDG ,QMXU\ 7KH LQWHUHVW LQ VSRUWV UHODWHG KHDG LQMXULHV LQ WKH KHDG LQMXU\ OLWHUDWXUH KDV D ORQJ KLVWRU\ WKRXJK UHODWLYHO\ OLWWOH PHWKRGRORJLFDOO\ VRXQG UHVHDUFK KDV EHHQ FRQGXFWHG LQ WKH DUHD XQWLO UHFHQWO\ $ UHSRUW E\ WKH 1DWLRQDO +HDOWK ,QWHUYLHZ 6XUYH\ HVWLPDWHG WKDW DSSUR[LPDWHO\ SHRSOH LQ WKH 8QLWHG 6WDWHV VXIIHUHG D KHDG LQMXU\ DV D GLUHFW UHVXOW RI SDUWLFLSDWLQJ LQ D VSRUWLQJ DFWLYLW\ 2I WKHVH b DSSUR[LPDWHO\ f ZHUH FDWHJRUL]HG DV fPLOGf .HOO\ DQG 5RVHQEHUJ 7KXUPDQ HW DO f ,Q DGGLWLRQ WKH VXUYH\ FRQWHQGHG WKDW PDQ\ RI WKRVH LQGLYLGXDOV VXIIHULQJ KHDG LQMXULHV QHYHU VRXJKW PHGLFDO WUHDWPHQW DQG WKHUHIRUH PDQ\ HVWLPDWHV RI QDWLRQDO KHDG LQMXU\ SUHYDOHQFH PD\ EH VLJQLILFDQWO\ EHORZ WKH DFWXDO QXPEHU 8QWLO WKH ODVW WR \HDUV KRZHYHU PRVW LQYHVWLJDWLRQ LQWR WKH FRJQLWLYH HIIHFWV RI VSRUWVUHODWHG KHDG LQMXU\ KDG EHHQ LVRODWHG RU

PAGE 28

DQHFGRWDO DQG VRPH LQYHVWLJDWLRQV FRQWLQXH WR HPSOR\ PHWKRGV WKDW EULQJ WKH YDOLGLW\ RI ILQGLQJV LQWR TXHVWLRQ ,Q DQ DUWLFOH SXEOLVKHG LQ WKH -RXUQDO RI WKH $PHULFDQ 0HGLFDO $VVRFLDWLRQ LQ 0DUWODQG GHVFULEHG WKH V\PSWRPV WKDW KH REVHUYHG LQ ER[HUV DIWHU UHFHLYLQJ UHSHDWHG EORZV WR WKH KHDG +H WHUPHG WKH FRQGLWLRQ fSXQFK GUXQNf DQG GHVFULEHG WKH V\PSWRPV DV LQFOXGLQJ D VOLJKW XQVWHDGLQHVV LQ JDLW RU XQFHUWDLQW\ LQ HTXLOLEULXP VOLJKW PHQWDO FRQIXVLRQ DQG GLVWLQFW VORZLQJ RI PXVFXODU DFWLRQ :KLOH KH QRWHG WKDW PRVW RI WKHVH FDVHV UHPDLQHG UHODWLYHO\ PLQRU VRPH UHVXOWHG LQ PRUH VHYHUH QHXURORJLFDO FRPSOLFDWLRQ LQFOXGLQJ KHVLWDQF\ LQ VSHHFK WUHPRUV RI WKH KDQGV DQG QRGGLQJ PRYHPHQWV RI WKH KHDG 0DUWODQG f ,Q DGGLWLRQ KH SRVWXODWHG WKDW WKH HIIHFWV RI UHSHDWHG EORZ WR WKH KHDG PLJKW FDXVH ODVWLQJ GDPDJH :KLOH KH LQGLFDWHG WKDW KH KDG WR UHO\ LQ VRPH FDVHV RQ REVHUYDWLRQ JDUQHUHG IURP fOD\PHQf KH VXJJHVWHG WKDW KH FRXOG LGHQWLI\ PDQ\ LQGLYLGXDOV NQRZQ WR VXIIHU IURP SHUVLVWLQJ V\PSWRPV WKRXJKW WR EH UHODWHG WR UHSHDWHG EORZV WR WKH KHDG +H ZDV XQDEOH WR HPSLULFDOO\ HYDOXDWH PDQ\ RI WKHP KH VWDWHG DV WKH\ ZHUH KRXVHG DW WKH WLPH LQ DV\OXPV DFURVV WKH FRXQWU\ )XUWKHU FRUURERUDWLRQ RI WKLV V\QGURPH ZDV SURYLGHG E\ 0LOOVSDXJK f LQ KLV UHYLHZ RI WKH KLVWRU\ RI QDYDO ER[HUV 0LOOVSDXJK VWDWHG FRQFOXVLYHO\ WKDW WKHUH DUH LQGHHG FRJQLWLYH VHTXHODH WR WKH UHSHDWHG EORZV UHFHLYHG LQ D ER[LQJ PDWFK DOWKRXJK WKH FRJQLWLYH GHILFLWV KH GHVFULEHG ZHUH RI DQ REMHFWLYH QDWXUH +H DOVR GHVFULEHG D FRPSOH[ RI V\PSWRPV DFXWHO\ DFFRPSDQ\LQJ UHSHDWHG EORZ WR WKH KHDG ZKLFK LQFOXGHG fIXOOQHVV LQ WKH KHDG FHSKDOJLD YHUWLJR LUULWDELOLW\ LQVRPQLD LQFUHDVHG IDWLJXDELOLW\ DQG PHPRU\ GHIHFWVf 0LOOVSDXJK f ,Q DGGLWLRQ KH GHVFULEHG ZKDW DSSHDUHG WR KLP WR EH D SHULRG RI DPQHVLD LQ ERWK WKRVH VXIIHULQJ ORVV RI FRQVFLRXVQHVV GXULQJ WKH ILJKW DQG

PAGE 29

WKRVH ZLWKRXW /2& +H WRRN WKLV WR LQGLFDWH WKDW WKHUH ZDV VRPH SK\VLFDO GDPDJH WR WKH EUDLQ LWVHOI ZKLFK PD\ PDQLIHVW IRU ORQJ SHULRGV RI WLPH ,Q KLV FRQFOXGLQJ UHPDUNV 0LOOVSDXJK VWDWHV fUHSHDWHG DQG IUHTXHQW FRQFXVVLRQV RFFDVLRQDOO\ YHU\ VHYHUH RIWHQ XQGRXEWHGO\ DVVRFLDWHG ZLWK LQWUDFUDQLDO FDSLOODU\ KHPRUUKDJHV DUH WR VD\ WKH OHDVW QRW FRQGXFLYH WR VWDELOL]HG PHQWDO HTXLOLEULXPf ,Q D VWXG\ E\ 7\VYDHU HW DO f IRUPHU SURIHVVLRQDO VRFFHU SOD\HUV ZHUH H[DPLQHG LQ RUGHU WR GHWHUPLQH WKH H[LVWHQFH RI SHUVLVWLQJ QHXURSV\FKRORJLFDO GHILFLWV FDXVHG E\ KHDGLQJ D VRFFHU EDOO 7KH SOD\HUV UDQJHG LQ DJH IURP WR DQG KDG QR UHSRUWHG KLVWRU\ RI DOFRKRO DEXVH ,QYHVWLJDWRUV LQWHUYLHZHG WKH SOD\HUV DQG XVHG WKH :HFKVOHU $GXOW ,QWHOOLJHQFH 6FDOH :$,6f WKH 7UDLO 0DNLQJ 7HVW SDUWV $ DQG %f D PRGLILHG YHUVLRQ RI WKH +DOVWHDG5HLWDQ DSKDVLD VFUHHQLQJ WHVW WHVWV RI VHQVRU\ SHUFHSWXDO IXQFWLRQLQJ PRWRU WHVWV DQG WKH %HQWRQ YLVXDO UHWHQWLRQ WHVW )RUP & LQ RUGHU WR GHWHUPLQH QHXURSV\FKRORJLFDO VWDWXV 7KHLU VFRUHV RQ WKHVH WHVWV ZHUH FRPSDUHG WR D FRQWURO JURXS FRQVLVWLQJ RI KRVSLWDOL]HG SDWLHQWV ZKR PDQLIHVWHG VLQJV RI D YDULHW\ RI GLVRUGHUV 7KHVH UHVHDUFKHUV IRXQG WKDW WKH IRUPHU VRFFHU SOD\HUV H[KLELWHG D VLJQLILFDQWO\ ODUJHU PHDQ VSOLW EHWZHHQ 3HUIRUPDQFH ,4 3,4f DQG 9HUEDO ,4 9,4f RQ WKH :$,6 DQG VLJQLILFDQWO\ ZRUVH VFRUHV RQ ERWK 7UDLO 0DNLQJ 7HVWV $ DQG % 7KH\ FRQFOXGHG WKDW WKHLU UHVXOWV VXJJHVWHG WKDW EUDLQ LQMXU\ LQFXUUHG DV D UHVXOW RI VRFFHU LQMXULHV FDQ UHVXOW LQ SHUPDQHQW EUDLQ LQMXU\ DV H[KLELWHG E\ SHUIRUPDQFH RQ QHXURSV\FKRORJLFDO WHVWV PHDVXULQJ SHUFHSWLRQ OHDUQLQJ PDQXDO GH[WHULW\ VSHHG DQG DWWHQWLRQ :KLOH WKHVH UHVXOWV PD\ EH VXJJHVWLYH RI SHUVLVWLQJ FRJQLWLYH GHILFLWV DV D UHVXOW RI VSRUWVUHODWHG KHDG LQMXU\ WKH PHWKRGRORJ\ LV IUDXJKW ZLWK SUREOHPV 0RVW LPSRUWDQWO\ WKH DXWKRUV GR QRW UHSRUW WKH QHXURORJLF KLVWRU\ RI DQ\ RI WKH SOD\HUV WHVWHG

PAGE 30

7KHUHIRUH LW LV LPSRVVLEOH WR GLVWLQJXLVK WKH H[WHQW RI KHDG LQMXULHV DQ\ RQH LQGLYLGXDO PD\ KDYH VXIIHUHG ,Q DGGLWLRQ WKH\ GR QRW DSSHDU WR XVH WKH WLPH VLQFH WKH VXSSRVHG LQMXU\ DV D FRYDULDWH LQ H[DPLQLQJ WHVW SHUIRUPDQFH QRU GR WKH\ SURYLGH D KLVWRU\ RI RWKHU IDFWRUV WKDW PD\ SOD\ D UROH LQ DIIHFWLQJ FRJQLWLYH IXQFWLRQLQJ )XUWKHU UHVHDUFKHUV UHSRUWHG QR NQRZOHGJH RI FRJQLWLYH IXQFWLRQLQJ EHIRUH WKHVH LQGLYLGXDOV SOD\HG VRFFHU )LQDOO\ WKH UHVHDUFKHUV UHSRUW WKDW WKH FRQWURO JURXS XVHG ZDV RQH FRQVLVWLQJ RI KRVSLWDOL]HG SDWLHQWV ZLWK fYDU\LQJf LQMXULHV :LWKRXW NQRZOHGJH RI WKH KLVWRU\ RI WKH JURXS LW ZRXOG DSSHDU WKDW LW LV DQ LQDSSURSULDWH QRUPDWLYH FRPSDULVRQ 7KH UHFHQW H[DPLQDWLRQ RI PLQRU KHDG LQMXU\ LQFXUUHG LQ VSRUWLQJ HYHQWV KDV VHYHUDO DGYDQWDJHV RYHU RWKHU VFHQDULRV LQ ZKLFK WKH VWDWH RI WKH LQGLYLGXDO DW WKH WLPH RI WKH DFFLGHQW DQG REVHUYDWLRQ RI WKHLU LPPHGLDWH EHKDYLRU LV QRW SRVVLEOH &ROOLQV HW DO f :LWKLQ WKH UHDOP RI RUJDQL]HG VSRUWLQJ HYHQWV LQYROYLQJ SK\VLFDO FRQWDFW EHWZHHQ SOD\HUV WKH RSSRUWXQLW\ H[LVWV WR DVVHVV LQMXU\ YHU\ VRRQ DIWHU WKH LQFLGHQW EHFDXVH RI WKH LPPHGLDWH DYDLODELOLW\ RI PHGLFDO VWDII ,Q DGGLWLRQ WKHUH LV RIWHQ D PHDQV RI UHYLHZLQJ WKH LQFLGHQW YLD WHOHYLVLRQ UHFRUGLQJ 7KLV DIIRUGV WKH RSSRUWXQLW\ WR DVVHVV WKH W\SHV RI IRUFHV DSSOLHG WR WKH KHDG RI WKH LQMXUHG SOD\HU )XUWKHU VRPH LQYHVWLJDWRUV QRZ FROOHFW EDVHOLQH PHDVXUHV RI QHXURSV\FKRORJLFDO IXQFWLRQLQJ EHIRUH D SOD\HU FDQ FRPSHWH LQ WKH FRQWDFW VSRUW 7KLV DOORZV IRU FRPSDULVRQ RI ILQGLQJV VXEVHTXHQW WR WKH LQMXULRXV LQFLGHQW DQG PRUH DFFXUDWH DQDO\VLV RI VXEVHTXHQW FRJQLWLYH FKDQJHV IROORZLQJ KHDG LQMXU\ $ SURVSHFWLYH VWXG\ E\ %DUWK HW DO f RI WKH FRJQLWLYH HIIHFWV RI PLQRU KHDG LQMXU\ LQ D FROOHJH IRRWEDOO SRSXODWLRQ VXJJHVWHG WKDW QHXURSV\FKRORJLFDO WHVWLQJ LV D VHQVLWLYH GHYLFH LQ WKH GHWHFWLRQ RI VXEWOH FRJQLWLYH FKDQJHV IROORZLQJ FRQFXVVLRQ 7KLV JURXS FROOHFWHG EDVHOLQH PHDVXUHV IRU FROOHJH IRRWEDOO SOD\HUV DW D WRWDO RI

PAGE 31

XQLYHUVLWLHV 6XEVHTXHQWO\ WKH\ UHWHVWHG SOD\HUV ZLWK GRFXPHQWHG FDVHV RI FRQFXVVLRQ ZLWKLQ KRXUV RI WKH LQMXU\ DQG GD\V ODWHU 7KH EDWWHU\ XVHG LQ WKLV LQYHVWLJDWLRQ FRQVLVWHG RI 7UDLO 0DNLQJ 7HVWV $ DQG % WKH 6\PERO 'LJLW 7HVW DQG WKH 3$6$7 $V D FRPSDULVRQ JURXS %DUWK XVHG D SRSXODWLRQ RI SOD\HUV VXVWDLQLQJ fPLOG RUWKRSHGLF LQMXULHV DQG PDOH FROOHJH VWXGHQWVf %DUWK HW DO f 7KHVH LQGLYLGXDOV ZHUH WHVWHG ZLWK WKH VDPH SURWRFRO XVLQJ WKH VDPH WLPH SHULRG DV WKH FRQFXVVHG SOD\HUV %DUWK FRQFOXGHG WKDW ZKLOH WKH FRQWURO JURXS SHUIRUPHG VLJQLILFDQWO\ EHWWHU RQ WKH fZLWKLQ KRXUf 3$6$7 DQG 6\PERO 'LJLW WHVW FRQVWLWXWLQJ ZKDW KH WHUPHG fQRUPDO WHVWLQJ EHKDYLRUf H[KLELWLQJ SUDFWLFH HIIHFWVf WKH FRQFXVVHG JURXS IDLOHG WR VKRZ LPSURYHPHQW RQ WKHVH WZR PHDVXUHV ZLWKLQ KRXUV RI UHFHLYLQJ D FRQFXVVLRQ +RZHYHU KH QRWHG WKDW WKHVH LQGLYLGXDOV WHQGHG WR VKRZ WKH VDPH LPSURYHPHQW DV FRQWUROV RQ WKH GD\ WHVWV RI WKHVH PHDVXUHV ZLWK D OHYHOLQJ DW GD\ ZKLFK VXJJHVWHG WR %DUWK WKDW WKHVH LQGLYLGXDOV KDG UHWXUQHG WR EDVHOLQH IXQFWLRQLQJ DW WKDW SRLQW 7KH LQYHVWLJDWRUV IDLOHG WR VKRZ WKH VDPH SDWWHUQ RI UHVXOWV RQ WKH 7UDLO 0DNLQJ 7HVWV ZKLFK %DUWK VXJJHVWHG PD\ EH GXH WR WKH VWURQJ HIIHFWV RI SUDFWLFH LQKHUHQW WR LWV VXFFHVVLYH DGPLQLVWUDWLRQ $V D UHVXOW RI WKLV VWXG\ %DUWK FRQFOXGHG WKDW D VLQJOH LQFLGHQW RI PLOG FRQFXVVLRQ LQ D FROOHJH IRRWEDOO SOD\HU UHVXOWV LQ GHWHFWDEOH QHXURSV\FKRORJLFDO FKDQJHV ZLWKLQ KRXUV RI WKH LQFLGHQW DQG WKDW D UHFRYHU\ WR EDVHOLQH IXQFWLRQLQJ RFFXUV TXLFNO\ ZLWKLQ WKH WR GD\V IROORZLQJ LQMXU\ ,Q D VWXG\ RI $XVWUDOLDQ 5XOHV )RRWEDOO SOD\HUV E\ 0DGGRFNV DQG 6DOLQJ f EDVHOLQH PHDVXUHV RI FRJQLWLYH IXQFWLRQ ZHUH FROOHFWHG XVLQJ WKH 3DFHG $XGLWRU\ 6HULDO $GGLWLRQ 7HVW 3$6$7f WKH 'LJLW 6\PERO 6XEVWLWXWLRQ 7HVW '667f DQG )RXU &KRLFH 5HDFWLRQ WLPH ZKLFK LQYROYHV PHDVXUHV RI GHFLVLRQ DQG PRYHPHQW WLPHV 2I WKH

PAGE 32

ZLWK EDVHOLQH FRJQLWLYH PHDVXUHV 0DGGRFNV UHSRUWHG RQ SOD\HUV ZKR VXEVHTXHQWO\ UHFHLYHG FRQFXVVLRQ ZHUH GHVFULEHG DV LQFXUULQJ QR /2& KDG /2& RI OHVV WKDQ PLQXWH DQG 37$ RI OHVV WKDQ PLQXWHVf DQG ZHUH UHWHVWHG DW GD\V SRVWn LQMXU\ +H LQGLFDWHG WKDW ZKLOH WKH QHXURORJLFDO IHDWXUHV RI FRQFXVVLRQ LQFOXGLQJ KHDGDFKH QDXVHD DQG GL]]LQHVV KDG UHPLWWHG DIWHU GD\V SHUVLVWLQJ QHXURSV\FKRORJLFDO GHILFLWV ZHUH QRWHG RQ WKH '667 DQG )RXU &KRLFH 5HDFWLRQ WLPH GHFLVLRQ WLPHf +H QRWHG WKDW VLJQLILFDQW GLIIHUHQFHV ZHUH QRW IRXQG RQ WKH 3$6$7 D WHVW WKDW SUHYLRXV DXWKRUV KDG IRXQG WR EH SDUWLFXODUO\ VHQVLWLYH WR WKH HIIHFWV RI PLQRU KHDG LQMXU\ GXH WR WKH DWWHQWLRQDO FRPSRQHQW LQKHUHQW WR SHUIRUPLQJ WKH WDVN 0DGGRFNV VXJJHVWHG WKDW VLQFH KLV SRSXODWLRQ RI FRQFXVVHG LQGLYLGXDOV SUHVHQWHG ZLWK PLOGHU IRUPV RI PLQRU KHDG LQMXU\ WKDQ PRVW PLQRU KHDG LQMXU\ SRSXODWLRQV H[DPLQHG WKH VHQVLWLYLW\ RI WKH WHVW PD\ EH OHVV HIIHFWLYH 2I LPSRUWDQFH 0DGGRFNV XWLOL]HG D FRQWURO JURXS FRQVLVWLQJ RI DJH PDWFKHG XPSLUHV ZKR ZHUH WHVWHG RQ WZR VHSDUDWH RFFDVLRQV ZLWK WKH VDPH WHVW PHDVXUHV $OWKRXJK WKLV PD\ DOORZ IRU FRPSDULVRQ ZLWK WKH KHDG LQMXUHG SRSXODWLRQ WDNLQJ LQWR DFFRXQW WKH HIIHFW RI SUDFWLFH RQ WKH YDULRXV PHDVXUHV 0DGGRFNV IDLOV WR HODERUDWH RQ RWKHU IDFWRUV WKDW PD\ KDYH PDGH WKLV DQ LQDSSURSULDWH FRPSDULVRQ JURXS HJ KLVWRU\ RI SDVW KHDG LQMXU\ LQWHUWHVW GXUDWLRQ HGXFDWLRQDO KLVWRU\ HWFf ,Q DQRWKHU VWXG\ E\ +LQWRQ%D\UH HW DO f SOD\HUV IURP D SURIHVVLRQDO UXJE\ WHDP ZHUH WHVWHG DW EDVHOLQH XVLQJ WKH 6\PERO 'LJLW 0RGDOLWLHV 7HVW 6'07f WKH 'LJLW 6\PERO 6XEVWLWXWLRQ 7HVW '667f DQG WKH 6SHHG RI &RPSUHKHQVLRQ 7HVW 7HQ SOD\HUV VXEVHTXHQWO\ UHFHLYHG FRQFXVVLRQV DQG ZHUH WHVWHG ZLWKLQ KRXUV RI WKH LQFLGHQW 7KH DXWKRUV UHSRUWHG WKDW WKRVH PHDVXUHV RI VSHHG RI LQIRUPDWLRQ SURFHVVLQJ ZHUH VHQVLWLYH WR WKH FRJQLWLYH HIIHFWV RI WKH FRQFXVVLRQ WKRXJK DQ XQWLPHG WHVW RI ZRUG

PAGE 33

UHFRJQLWLRQ ZDV QRW 7KH\ IXUWKHU VWDWHG WKDW WHVWV RI VSHHG RI LQIRUPDWLRQ SURFHVVLQJ ZHUH PRUH VHQVLWLYH WKDQ HLWKHU WKH 6'07 RU WKH '667 EDVHG RQ FRPSDULVRQ ZLWK EDVHOLQH PHDVXUHV ,Q D VWXG\ E\ 0F&UHD HW DO UHVHDUFKHUV DGPLQLVWHUHG WKH 6WDQGDUGL]HG $VVHVVPHQW RI &RQFXVVLRQ 6$&f WR KLJK VFKRRO DQG FROOHJH IRRWEDOO SOD\HUV EHIRUH WKH DQG VHDVRQV 7KH 6$& FRQVLVWV RI D EULHI PHDVXUH RI RULHQWDWLRQ VKRUWn WHUP GHOD\HG YHUEDO PHPRU\ ZRUGV UHFDOOHG LPPHGLDWHO\ DQG DIWHU D PLQXWH GHOD\f D EULHI PHDVXUH RI DWWHQWLRQ DQG VKRWWHUP PHPRU\ UHYHUVH 'LJLW 6SDQf DQG FRQFHQWUDWLRQ UHFLWDWLRQ RI WKH PRQWKV RI WKH \HDU LQ UHYHUVH RUGHUf DV ZHOO DV VHYHUDO H[HUWLRQDO H[HUFLVH LQ RUGHU WR FUHDWH FRQGLWLRQV RI LQWUDFUDQLDO SUHVVXUH GXULQJ ZKLFK SRVW FRQFXVVLYH V\PSWRPV DUH PRVW OLNHO\ WR EH VHHQ 7KH VXP RI WKH VFRUHV LV DGGHG WR SURGXFH D FRPSRVLWH WRWDO ZLWK D PD[LPXP VFRUH RI 2I WKH LQGLYLGXDOV WHVWHG DW EDVHOLQH VXEVHTXHQWO\ H[SHULHQFHG FRQFXVVLRQ DQG ZHUH WHVWHG LPPHGLDWHO\ RI WKH ZHUH DOVR WHVWHG GD\V DIWHU WKH LQMXU\ RFFXUUHG 0F&UHD IRXQG WKDW WKH FRQFXVVHG SOD\HUV SHUIRUPHG VLJQLILFDQWO\ EHORZ ERWK WKH FRQWURO JURXS DQG WKHLU RZQ EDVHOLQH FRPSRVLWH PHDVXUHV RQ WKH 6$& $W KRXUV KH LQGLFDWHG WKH FRQFXVVHG SOD\HUV KDG UHWXUQHG WR WKHLU EDVHOLQH OHYHO +H FRQFOXGHG WKDW WKLV EULHI EDWWHU\ RI PRGLILHG QHXURSV\FKRORJLFDO PHDVXUHV LV VHQVLWLYH WR WKH FRJQLWLYH GHILFLWV REWDLQHG IURP PLQRU KHDG LQMXU\ LQFXUUHG LQ IRRWEDOO DQG VXJJHVWV WKDW KLV REVHUYDWLRQ WKDW DOO FRQFXVVHG SOD\HUV ZHUH QRW GLIIHUHQW IURP EDVHOLQH DW KRXUV LV FRQVLVWHQW ZLWK RWKHU ILQGLQJV 2I QRWH KRZHYHU ZDV WKH ODFN RI DWWHQWLRQ WR WKH HIIHFWV RI SUDFWLFH DQG WKH SRVVLELOLW\ WKDW WKH\ PD\ KDYH FRQWULEXWHG WR LPSURYHPHQWV VHHQ DW WKH IROORZXS WHVWLQJ WKXV SRWHQWLDOO\ PDVNLQJ DQ\ SHUVLVWHQW FRJQLWLYH HIIHFWV RI FRQFXVVLRQ )XUWKHU WKH ODFN

PAGE 34

RI PDWFKHG FRQWUROV H[FOXGHV WKH UHVHDUFKHUV IURP PDNLQJ FRPPHQW RQ WKH SRWHQWLDO HIIHFWV RI DJH DQG HGXFDWLRQ OHYHO RQ UHVXOWV REWDLQHG DOWKRXJK WKH\ FRQWHQG WKDW WKHVH YDULDEOHV KDYH fPLQLPDOf HIIHFW RQ SHUIRUPDQFH DQG WKDW WKHUH ZHUH QR VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH VFRUHV RI KLJK VFKRRO DQG FROOHJH SOD\HUV $ PRUH UHFHQW WHVWUHWHVW VWXG\ E\ %DUU DQG 0F&UHD f GHVFULEHG WKH UHVXOWV RI WKH DGPLQLVWUDWLRQ RI WKH 6$& WR FRQFXVVHG SOD\HUV LPPHGLDWHO\ IROORZLQJ WKH LQMXU\ FRPSDUHG WR WKHLU EDVHOLQH SHUIRUPDQFH $ FRQWURO JURXS RI QRQFRQFXVVHG SOD\HUV ZDV DOVR WHVWHG DW WKH VDPH WLPH LQWHUYDOV 7KHVH UHVHDUFKHU IRXQG WKDW XVLQJ ERWK D UHOLDEOH FKDQJH LQGH[ DSSURDFK DQG D OLQHDU UHJUHVVLRQ DSSURDFK ZHUH VXFFHVVIXO DW LGHQWLI\LQJ GHWULPHQWV LQ FRJQLWLYH SHUIRUPDQFH LPPHGLDWHO\ IROORZLQJ FRQFXVVLRQ 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYHV ZHUH XVHG WR GHWHUPLQH WKH VWDWLVWLFDO VLJQLILFDQFH RI WKH PRGHOV LQ DFFXUDWHO\ FODVVLI\LQJ D SOD\HU DV FRQFXVVHG RU QRW FRQFXVVHG $ VWXG\ E\ &ROOLQV HW DO f WHVWHG DWKOHWHV IURP XQLYHUVLW\ IRRWEDOO WHDPV XVLQJ D EULHI QHXURSV\FKRORJLFDO EDWWHU\ WKDW LQFOXGHG WKH +RSNLQV 9HUEDO /HDUQLQJ 7HVW +9/7 YHUEDO OHDUQLQJ DQG GHOD\HG PHPRU\f 7UDLO 0DNLQJ 7HVWV $ DQG % 'LJLW 6SDQ WHVW 6\PERO 'LJLW 0RGDOLWLHV 7HVW *URRYHG 3HJERDUG 7HVW DQG &RQWUROOHG 2UDO :RUG $VVRFLDWLRQ ,Q DGGLWLRQ WKH DWKOHWHV FRPSOHWHG WKH &RQFXVVLRQ 6\PSWRP 6FDOH D VHOIUHSRUW /LNHUW VFDOH RI SRVW FRQFXVVLRQ V\PSWRPV 6XEVHTXHQWO\ WKH JURXS UHWHVWHG LQGLYLGXDOV ZKR VXIIHUHG FRQFXVVLRQ ZLWKLQ KRXUV RI WKH LQFLGHQW DQG ZLWKLQ DQG GD\V DIWHUZDUGV &ROOLQV FRQFOXGHG DIWHU SHUIRUPLQJ D GLVFULPLQDQW DQDO\VLV RQ WKH UHVXOWLQJ GDWD WKDW WKH XVH RI WKLV EULHI QHXURSV\FKRORJLFDO EDWWHU\ UHVXOWHG LQ DQ RYHUDOO FRUUHFW FODVVLILFDWLRQ UDWH RI b IRU WKH FRQFXVVHG DQG FRQWURO JURXSV DW WKH KRXU WHVWLQJ SHULRG ,Q SDUWLFXODU KH QRWHG WKDW FRQFXVVHG LQGLYLGXDOV

PAGE 35

ZHUH VLJQLILFDQWO\ ZRUVH RQ WKH +RSNLQV 7RWDO VFRUH DQG +RSNLQV 'HOD\ VFRUH WKDQ WKH FRQWUROV DQG WKDW PRGHUDWH GLIIHUHQFHV EHWZHHQ WKH JURXSV SHUVLVWHG XQWLO WKH GD\ WHVWLQJ :KLOH QRW LQFOXGHG LQ WKH DQDO\VLV RI FRQFXVVHG SOD\HUV &ROOLQV HW DO DQDO\]HG WKH EDVHOLQH VFRUHV RI DOO DWKOHWHV DQG GLVFRYHUHG DQ LQWHUDFWLRQ EHWZHHQ KLVWRU\ RI GLDJQRVHG OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ DQG VLJQLILFDQWO\ ORZHU EDVHOLQH SHUIRUPDQFH RQ WZR PHDVXUHV 7UDLO 0DNLQJ 7HVW % DQG 'LJLW 6\PERO 0RGDOLWLHV 7HVWf 7KLV ILQGLQJ VXJJHVWV WKDW WKHVH GHPRJUDSKLF YDULDEOHV PD\ EH LPSRUWDQW WR H[DPLQH DQG WDNH LQWR DFFRXQW ZKHQ HYDOXDWLQJ WKH QHXURSV\FKRORJLFDO SHUIRUPDQFH RI DQ DWKOHWH DIWHU FRQFXVVLRQ 5HOLDEOH &KDQJH 7KH WHVWV RI FRJQLWLYH FKDQJH DQG UHFRYHU\ RI IXQFWLRQ GHVFULEHG DERYH JHQHUDOO\ KDYH XWLOL]HG RQH RI WZR PHWKRGV WR LQGLFDWH UHFRYHU\ DIWHU FRQFXVVLRQ DQG ILWQHVV WR UHWXUQ WR SOD\ 2QH DVVHVVHV D SOD\HUfV UHWXUQ WR EDVHOLQH IXQFWLRQLQJ %DUWK HW DO 0F&UHD HW DO f DQG WKH RWKHU D FRPSDULVRQ WR D FRQWURO JURXS LQYROYLQJ WKH GHWHUPLQDWLRQ RI GLIIHUHQFHV EHWZHHQ VHULDO WHVWLQJ UHVXOWV &ROOLQV HW DO 0DGGRFNV DQG 6DOLQJ f +LQWRQ%D\UH HW DO f VXJJHVWHG WKDW WKH IRUPHU GRHV QRW SURSHUO\ DFFRXQW IRU WKH HIIHFWV RI SUDFWLFH ZKLOH WKH ODWWHU SURYLGHV OLPLWHG LQIRUPDWLRQ ZLWKRXW WKH DYDLODELOLW\ RI fVXLWDEOHf QRUPDWLYH GDWD 7KH UHVXOWV RI QHXURSV\FKRORJLFDO WHVWLQJ DUH LQIOXHQFHG E\ PDQ\ IDFWRUV LQ DGGLWLRQ WR WKH FRJQLWLYH HIIHFWV RI PLQRU KHDG LQMXU\ (VSHFLDOO\ LQ UHSHDWHG RU VHULDO WHVWLQJ DV LV SHUIRUPHG LQ WKH HYDOXDWLRQ RI UHFRYHU\ RI D FRQFXVVHG DWKOHWH LQ RUGHU WR GHWHUPLQH UHWXPWRSOD\ VWDWXV WKH UHVXOWV RI WHVWLQJ PD\ EH LQIOXHQFHG E\ VXFK WKLQJV DV WKH UHOLDELOLW\ RI WKH WHVW EHLQJ XVHG SUDFWLFH HIIHFWV DQG WKH DWWLWXGH RI WKH WHVW WDNHU 7HPNLQ HW DO f (YHQ ZKHQ WKHUH LV QR WUXH

PAGE 36

FKDQJH LQ FRJQLWLYH VWDWXV WKHVH YDULDEOHV PD\ VXJJHVW WKDW D FKDQJH KDV WDNHQ SODFH ,Q WKLV VHQVH WKH DELOLW\ WR GHWHFW WUXH FKDQJH LQ DQ LQGLYLGXDO FDVH FDQ EH DIIHFWHG E\ PDQ\ IDFWRUV 7HPNLQ HW DO f H[DPLQHG IRXU VWDWLVWLFDO PRGHOV LQ RUGHU WR GHWHUPLQH ZKLFK ZDV PRVW VHQVLWLYH WR GHWHFWLQJ fUHDO FKDQJHf LQ WKH LQGLYLGXDO FDVH 7KH\ XVHG WHVW PHDVXUHV DQG UHWHVW VFRUHV IURP fQHXURORJLFDOO\ VWDEOHf LQGLYLGXDOV LQ WKH FRPSDULVRQ RI IRXU VWDWLVWLFDO PRGHOV LQ RUGHU WR DGGUHVV WKLV TXHVWLRQ 7KHVH PRGHOV ZHUH WKH EDVLF 5HOLDEOH &KDQJH ,QGH[ 5&,f WKH 5&, ZLWK FRUUHFWLRQ IRU SUDFWLFH HIIHFWV D VLPSOH UHJUHVVLRQ PRGHO DQG D PXOWLSOH UHJUHVVLRQ PRGHO (DFK RI WKHVH PRGHOV ZDV XVHG LQ RUGHU WR JHQHUDWH D SUHGLFWHG UHWHVW VFRUH ZKLFK ZDV WKHQ FRPSDUHG WR WKH DFWXDO UHWHVW VFRUH 7KH GLIIHUHQFH EHWZHHQ WKH WZR RU UHVLGXDO ZDV WKHQ FRPSDUHG WR D FRQILGHQFH LQWHUYDO WKDW ZDV FDOFXODWHG WR FRQWDLQ b RI WKRVH VFRUHV H[SHFWHG JLYHQ QR UHDO FKDQJH 7KLV PHWKRG ZDV EDVHG RQ =D = 7KLV YDOXH VXJJHVWV WKDW b RI VFRUHV ZHUH WKRXJKW WR EH VLJQLILFDQWO\ KLJKHU WKDQ WKH QRUPDO SRSXODWLRQ ZKLOH b ZHUH DVVXPHG WR EH VLJQLILFDQWO\ ORZHU ,Q RWKHU ZRUGV DQ\ UHVLGXDO VFRUH O\LQJ RXWVLGH WKLV LQWHUYDO ZDV DVVXPHG WR UHIOHFW UHDO FKDQJH 5HOLDEOH &KDQJH ,QGH[ 7KH 5HOLDEOH &KDQJH ,QGH[ ZDV RULJLQDOO\ GHYHORSHG WR DVVHVV WKH HIIHFWLYHQHVV RI FOLQLFDO LQWHUYHQWLRQV +LQWRQ%D\UH HW DO f ,W ZDV XVHG E\ -DFREVRQ DQG 7UXD[ f ZLWK WKH LGHD WKDW FOLQLFDOO\ VLJQLILFDQW FKDQJH ZDV UHODWHG WR D UHWXUQ WR QRUPDO IXQFWLRQLQJ 7KH PRGHO LV EDVHG RQ WKH DVVXPSWLRQ WKDW D FKDQJH LQ WHVW VFRUHV IURP RQH WLPH WR WKH QH[W LV VLJQLILFDQW LI WKH VL]H RI WKDW FKDQJH LV ODUJH FRPSDUHG WR WKH VWDQGDUG HUURU RI WKH GLIIHUHQFH RI WKH WHVW -DFREVRQ DQG 7UXD[ f LQGLFDWH WKDW WKH HUURU YDULDQFH RI D WHVW LV FDOFXODWHG WKURXJK WKH XVH RI WHVWUHWHVW UHOLDELOLW\ DQG WKH YDULDWLRQ RI

PAGE 37

VFRUHV DURXQG WKH PHDQ $FFRUGLQJ WR +LQWRQ%D\UH WKH IRUPXOD IRU REWDLQLQJ DQ LQGLYLGXDOfV 5&, VFRUH LV 5&O-A 6GLII ,Q WKLV LQVWDQFH [L LV WKH LQGLYLGXDOfV ILUVW VFRUH LV WKH UHWHVW VFRUH DQG 6GLIU LV WKH VWDQGDUG HUURU RI WKH GLIIHUHQFH ZKLFK LV FDOFXODWHG DV IROORZV 9 O6ef 7KH VWDQGDUG HUURU RI WKH PHDVXUHPHQW 6H LV REWDLQHG WKURXJK WKH XVH RI WKH VWDQGDUG GHYLDWLRQ RI WKH FRQWURO JURXS 6 ,Q DGGLWLRQ WKH UHOLDELOLW\ RI WKH PHDVXUH 7Q LV XVHG LQ REWDLQLQJ WKLV VWDWLVWLF 6e 6MO UO +LQWRQ%D\UH HW DO f $JDLQ LQ WKH XVH RI WKLV VWDWLVWLF DQ 5&, ] YDOXH H[FHHGLQJ LV FRQVLGHUHG WR EH D VLJQLILFDQW FKDQJH ,Q WKH DQDO\VLV RI 7HPNLQ HW DO f D FKDQJH ZDV FRQVLGHUHG WR EH UHOLDEOH XVLQJ WKH 5&, LI WKH DEVROXWH YDOXH RI WKH SRVWWHVW PLQXV WKH SUHWHVW H[FHHGHG WKH VWDQGDUG HUURU RI WKH WHVWUHWHVW GLIIHUHQFH RI WKH QRUPDWLYH VDPSOH PXOWLSOLHG E\ WKH ] VFRUH FXWRII GHVFULEHG DERYH 7KLV FUHDWHG D SUHGLFWLRQ LQWHUYDO WKDW VKRXOG FRQWDLQ b RI WKH VFRUHV LQ D QRUPDO SRSXODWLRQ $Q\ FKDQJH VFRUH O\LQJ RXWVLGH RI WKLV LQWHUYDO LQ HLWKHU GLUHFWLRQ ZDV FRQVLGHUHG fUHDOf FKDQJH 5HOLDEOH &KDQJH ,QGH[ ZLWK &RUUHFWLRQ IRU 3UDFWLFH 7KH VHFRQG PRGHO XVHG E\ 7HPNLQ HW DO f ZDV D VOLJKW YDULDWLRQ RQ WKH 5&, PRGHO GHVFULEHG DERYH 7KLV PRGHO KDV EHHQ XVHG E\ RWKHUV LQ WKH HYDOXDWLRQ RI FRJQLWLYH RXWFRPH IROORZLQJ YDULRXV VXUJLFDO SURFHGXUHV &KHOXQH HW DO +HUPDQQ

PAGE 38

HW DO .QHHERQH HW DO 6DZULH HW DO f ,Q WKLV PRGHO SUDFWLFH HIIHFWV IRU LQGLYLGXDO WHVWV DUH GHWHUPLQHG E\ FDOFXODWLQJ WKH PHDQ GLIIHUHQFH EHWZHHQ WKH WHVW DQG UHWHVW VFRUHV RI WKH QRUPDWLYH SRSXODWLRQ 7KLV PHDQ GLIIHUHQFH LV WKHQ DGGHG WR WKH EDVHOLQH VFRUH RI WKH LQGLYLGXDO LQ TXHVWLRQ LQ RUGHU WR FDOFXODWH D SUHGLFWHG IROORZXS VFRUH $ UHOLDEOH FKDQJH LV WKHQ GHWHUPLQHG E\ WKH DPRXQW RI GLIIHUHQFH EHWZHHQ WKH DFWXDO UHWHVW VFRUH DQG WKH SUHGLFWHG $ FRQILGHQFH LQWHUYDO LV GHWHUPLQHG MXVW DV LW ZDV IRU WKH EDVLF 5&, 7KDW LV WKH FRQILGHQFH LQWHUYDO LV GHWHUPLQHG E\ WKH VWDQGDUG HUURU RI WKH GLIIHUHQFH RI WKH QRUPDWLYH VDPSOH PXOWLSOLHG E\ WKH ]VFRUH WKDW GHWHUPLQHV WKH FXWRII SRLQW JHQHUDOO\ =f /LQHDU 5HJUHVVLRQ 7KH WKLUG PRGHO XWLOL]HG E\ 7HPNLQ HW DO f ZDV D EDVLF OLQHDU UHJUHVVLRQ $ IRUPXOD WKDW FDQ EH XVHG IRU SUHGLFWLQJ DQ\ VFRUH LQ WKH SRSXODWLRQ XQGHU LQYHVWLJDWLRQ LV FDOFXODWHG E\ UHJUHVVLQJ WKH UHWHVW VFRUH RI WKH QRUPDWLYH VDPSOH RQ WKHLU LQLWLDO VFRUHV 2ULJLQDOO\ VXJJHVWHG E\ 0F6ZHHQ\ HW DO f 7HPNLQ VXJJHVWV WKDW WKLV PRGHO FRQWUROV IRU ERWK SUDFWLFH HIIHFWV DQG UHJUHVVLRQ WRZDUG WKH PHDQ 5HOLDEOH FKDQJH LV GHWHUPLQHG DJDLQ E\ H[DPLQLQJ WKH GLIIHUHQFH EHWZHHQ WKH REVHUYHG UHWHVW VFRUH DQG WKH SUHGLFWHG VFRUH $ SUHGLFWLRQ LQWHUYDO LV FDOFXODWHG EDVHG RQ WKH UHVLGXDOV HVWLPDWHG E\ WKH HTXDWLRQ PXOWLSOLHG E\ D ]FXWRII VFRUH DJDLQ JHQHUDOO\ =f 0XOWLSOH 5HJUHVVLRQ 0RGHO 7KH ILQDO PRGHO LQYHVWLJDWHG E\ 7HPNLQ HW DO f LQFRUSRUDWHG WKH XVH RI VWHSZLVH OLQHDU UHJUHVVLRQ LQ RUGHU WR GHWHUPLQH D SUHGLFWHG UHWHVW VFRUH 0XOWLSOH PHDVXUHV ZHUH XVHG LQ WKH VWHSZLVH UHJUHVVLRQ ZKLFK WKH LQYHVWLJDWRUV WKRXJKW PLJKW EH LPSRUWDQW LQ GHWHUPLQLQJ IROORZXS VFRUHV 7KHVH LQFOXGHG EDVHOLQH WHVW VFRUH WHVWUHWHVW

PAGE 39

LQWHUYDO GHPRJUDSKLF YDULDEOHV DQG D PHDVXUH RI RYHUDOO QHXURSV\FKRORJLFDO FRPSHWHQFH ,Q RUGHU WR DFFRXQW IRU QRQOLQHDU UHODWLRQVKLSV EHWZHHQ WHVW DQG UHWHVW VFRUH WKHVH LQYHVWLJDWRUV DOVR LQFOXGHG WKH VTXDUH DQG FXEH RI EDVHOLQH VFRUHV LQ WKHLU PXOWLSOH UHJUHVVLRQ PRGHOV IRU HDFK PHDVXUH 7HPNLQ H[SODLQV WKDW YDULDEOHV ZHUH DGGHG WR WKH VWHSZLVH OLQHDU UHJUHVVLRQ LI WKH\ KDG WKH fKLJKHVW SDUWLDO FRUUHODWLRQ DPRQJ WKH YDULDEOHV QRW DOUHDG\ LQ WKH HTXDWLRQ DQG LI WKH VLJQLILFDQFH OHYHO DVVRFLDWHG ZLWK WKH YDULDEOH ZDV XQGHU f 9DULDEOHV ZHUH UHPRYHG IURP WKH HTXDWLRQ LI DIWHU DGGLQJ DQRWKHU YDULDEOH WKH VLJQLILFDQFH ZDV DERYH 7KH UHVXOWV RI 7HPNLQ HW DO f VXJJHVW WKDW WKH SUHGLFWLRQ LQWHUYDOV DVVRFLDWHG ZLWK WKH IRXU PRGHOV EHFDPH QDUURZHU DQG WKHUHIRUH PRUH VHQVLWLYH WR WKH GHWHFWLRQ RI FKDQJH ZLWK WKH JURZLQJ FRPSOH[LW\ RI WKH PHWKRGV XVHG ,Q WKLV LQVWDQFH WKH WHUP fFRPSOH[LW\f GRHV QRW QHFHVVDULO\ UHIHU WR WKH GLIILFXOW\ RI PDWKHPDWLFDO RSHUDWLRQV EXW WR WKH DPRXQW RI HUURU YDULDQFH DFFRXQWHG IRU HJ WHVWUHWHVW UHOLDELOLW\ SUDFWLFH HIIHFWV UHJUHVVLRQ WR WKH PHDQ HWFf 7KDW LV WKH 5&, ZDV OHVV VHQVLWLYH WKDQ WKH 5&, ZLWK SUDFWLFH HIIHFWV ZKLFK ZDV OHVV VHQVLWLYH WKDQ WKH OLQHDU UHJUHVVLRQ ZKLFK LQ WXUQ ZDV OHVV VHQVLWLYH WKDQ WKH PXOWLSOH UHJUHVVLRQ PRGHO +RZHYHU WKH\ LQGLFDWH WKDW WKH 5&, SURGXFHG KLJK HUURU UDWHV GXH WR WKH QRQQRUPDOLW\ RI WKH SRSXODWLRQ UHVLGXDOV 7KLV ZDV LQ ODUJH SDUW GXH WR WKH SUDFWLFH HIIHFWV VHHQ LQ WKH LQGLYLGXDO PHDVXUHV ZKLFK FDXVHG WKH SRSXODWLRQ GDWD WR EHFRPH VRPHZKDW VNHZHG :KHQ SUDFWLFH HIIHFWV ZHUH WDNHQ LQWR DFFRXQW WKH UHVLGXDOV EHWWHU HVWLPDWHG D QRUPDO FXUYH DQG SUHGLFWHG VFRUHV ZHUH PRUH FORVHO\ FRUUHODWHG ZLWK REVHUYHG VFRUHV 7KH\ UHSRUW WKDW RYHUDOO SUHGLFWLRQ DFFXUDF\ ZDV VLPLODU IRU WKH ODVW WKUHH PRGHOV WKRXJK ZLWK WKH DGGLWLRQ RI GHPRJUDSKLF YDULDEOHV LQ WKH PXOWLSOH UHJUHVVLRQ PRGHO VLJQLILFDQWO\ EHWWHU SUHGLFWLRQ ZDV SRVVLEOH

PAGE 40

6SHFLILF $LPV DQG +\SRWKHVHV *LYHQ WKH PDJQLWXGH RI WKH RFFXUUHQFH RI KHDG LQMXU\ LQ WKH 8QLWHG 6WDWHV DQG WKH VXEVHTXHQW FRJQLWLYH DV ZHOO DV SV\FKRVRFLDO FRQVHTXHQFHV LQKHUHQW WR LW WKH LPSRUWDQFH RI GHWHFWLQJ QHXURSV\FKRORJLFDO GHILFLWV LV FOHDU 7KH OLWHUDWXUH VXJJHVWV WKDW PLQRU KHDG LQMXU\ LQ VSRUWV D SDUWLFXODUO\ FRPPRQ RFFXUUHQFH LQ D FROOHJH IRRWEDOO SRSXODWLRQ YDULHV VLJQLILFDQWO\ IURP SHUVRQ WR SHUVRQ DQG WKDW VXEWOH GHILFLWV PD\ EH LQGLFDWLYH RI WKH OLQJHULQJ HIIHFWV RI FRQFXVVLRQ 7KH DVVHVVPHQW RI UHFRYHU\ IURP FRQFXVVLRQ LV SDUWLFXODUO\ LPSRUWDQW ZKHQ RQH FRQVLGHUV WKH SRWHQWLDO HIIHFWV RI UHSHDWHG FRQFXVVLRQ DQG LWV ORQJWHUP FRQVHTXHQFHV &DQWX DQG 9R\ 0F&URU\ DQG %HUNRYLF f $V &ROOLQV HW DO f KDYH LQGLFDWHG WKH SRVVLELOLW\ RI D FRUUHODWLRQ EHWZHHQ PXOWLSOH FRQFXVVLRQ DQG UHGXFHG EDVHOLQH QHXURSV\FKRORJLFDO IXQFWLRQLQJ PDNH WKH DVVHVVPHQW RI FRQFXVVLRQ D SDUWLFXODUO\ VDOLHQW LVVXH LQ WKH VWXGHQW DWKOHWH SRSXODWLRQ 6WXGLHV RQ WKH HIIHFWV RI PLQRU KHDG LQMXU\ WR WKLV SRLQW KDYH SURGXFHG YDU\LQJ RIWHQ FRQWUDGLFWRU\ UHVXOWV 7KLV LV LQ QR VPDOO SDUW GXH WR WKH YDULDELOLW\ RI WKH PHWKRGRORJ\ XWLOL]HG ,Q PDQ\ FDVHV LQYHVWLJDWRUV KDYH XVHG LQDSSURSULDWH QRUPDWLYH VDPSOHV RU FRPSDULVRQ JURXSV ZKLOH LQ RWKHUV IDFWRUV FRQWULEXWLQJ WR FKDQJH RU ODFN WKHUHRIf LQ VHULDO WHVWLQJ KDYH QRW EHHQ WDNHQ LQWR DFFRXQW 0RUH VHQVLWLYH DQG VRXQG VWDWLVWLFDO PHDVXUHV DUH QHHGHG LQ WKH DVVHVVPHQW RI KHDG LQMXU\ DQG WKH SHULRG RI UHFRYHU\ DIWHU KHDG LQMXU\ 7KH SUHVHQW VWXG\ LV GHVLJQHG WR IXUWKHU WKH UHVHDUFK RQ WKH VHQVLWLYLW\ RI QHXURSV\FKRORJLFDO PHDVXUHV RYHU VHULDO WHVWLQJ VHVVLRQV LQ DVVHVVLQJ WKH IXQFWLRQLQJ RI LQGLYLGXDOV VXIIHULQJ PLQRU KHDG LQMXU\ LQ D FROOHJH IRRWEDOO SRSXODWLRQ 7KH DLPV RI WKLV LQYHVWLJDWLRQ DUH f WR DQDO\]H WKH HIIHFWLYHQHVV RI YDULRXV VWDWLVWLFDO PRGHOV LQ GHWHFWLQJ

PAGE 41

RQJRLQJ FRJQLWLYH VHTXHODH LQ FRQFXVVHG FROOHJH IRRWEDOO SOD\HUV YLD VHULDO QHXURSV\FKRORJLFDO WHVWLQJ f WR GHWHUPLQH DSSURSULDWH WKUHVKROGV IRU WHVW SHUIRUPDQFH UHVXOWLQJ LQ DFFHSWDEOH VHQVLWLYLW\ DQG VSHFLILFLW\ FODVVLILFDWLRQ YDOXHV f WR GHWHUPLQH WKRVH IXQFWLRQV ZKLFK DSSHDU PRVW LQGLFDWLYH RI HQGXULQJ FRJQLWLYH FRPSODLQWV RYHU WKH GD\ SHULRG IROORZLQJ FRQFXVVLRQ DQG f WR FDOFXODWH UHSUHVHQWDWLYH PRGHOV LQ RUGHU WR DLG LQ SUHGLFWLRQ RI FRJQLWLYH UHFRYHU\ IROORZLQJ FRQFXVVLRQ ,W LV K\SRWKHVL]HG WKDW HDFK SURJUHVVLYHO\ fFRPSOH[f PRGHO ZLOO H[KLELW LQFUHDVLQJ VHQVLWLYLW\ LQ WHUPV RI GHPRQVWUDWLQJ ORZHU WHVW SHUIRUPDQFH UHODWLYH WR EDVHOLQH DW WKH UHWHVW VHVVLRQ ,W LV DOVR K\SRWKHVL]HG EDVHG RQ WKH ILQGLQJV RI 7HPNLQ HW DO f WKDW RQO\ WKH PRVW FRPSOH[ PRGHOV LH UHJUHVVLRQ PRGHOVf ZLOO DOORZ IRU DFFXUDWH FODVVLILFDWLRQ RI FRQFXVVHG LQGLYLGXDOV EH\RQG WKH KRXU WHVW VHVVLRQ $V ZDV WKH FDVH LQ WKH 7HPNLQJ HW DO f VWXG\ WKH WHUP fFRPSOH[f LV XVHG KHUH WR GHQRWH WKH OHYHO WR ZKLFK SRWHQWLDO HUURU YDULDQFH LV WDNHQ LQWR DFFRXQW 7KHUHIRUH LQ UHODWLRQ WR WKH FXUUHQW VWXG\ LW LV K\SRWKHVL]HG WKDW WKRVH PRGHOV ZKLFK DFFRXQW IRU PRUH SRWHQWLDO HUURU YDULDQFH ZLOO EH PRUH VXFFHVVIXO DW GHWHFWLQJ UHDO SRVWFRQFXVVLYH FRJQLWLYH GHFOLQH DW WKH PRUH GLVWDO WHVWLQJ VHVVLRQV

PAGE 42

&+$37(5 7:2 0(7+2'6 6XEMHFWV 7KLV VWXG\ LQFOXGHG WKH VXEMHFWV WKDW ZHUH RULJLQDOO\ UHSRUWHG E\ &ROOLQV HW DO f DV ZHOO DV RWKHUV IURP D FRQWLQXLQJ GDWD FROOHFWLRQ SURMHFW DW WKH 8QLYHUVLW\ RI )ORULGD 3DUWLFLSDQWV LQFOXGHG PDOH IRRWEDOO SOD\HUV IURP IRXU XQLYHUVLWLHV WKH 8QLYHUVLW\ RI )ORULGD 0LFKLJDQ 6WDWH 8QLYHUVLW\ WKH 8QLYHUVLW\ RI 3LWWVEXUJK DQG WKH 8QLYHUVLW\ RI 8WDK $OO SDUWLFLSDQWV LQ WKH VWXG\ ZHUH YROXQWHHUV ZKR DJUHHG WR SDUWLFLSDWH LQ WKH VWXG\ ZLWKRXW FRPSHQVDWLRQ DQG ZLWKRXW ILQDQFLDO UHVSRQVLELOLW\ $OO SDUWLFLSDQWV RQ WKH IRRWEDOO WHDPV RI WKHVH UHVSHFWLYH VFKRROV ZHUH DVNHG WR SDUWLFLSDWH ,Q WKH LQLWLDO PHHWLQJ ZLWK WKH VWXGHQW DWKOHWH EDVHOLQH GDWD ZDV FROOHFWHG UHJDUGLQJ DJH IRRWEDOO SRVLWLRQ KDQGHGQHVV FROOHJH HQWUDQFH H[DP VFRUHV 6$7 DQGRU $&7f KLJK VFKRRO JUDGH SRLQW DYHUDJH KLJKHVW OHYHO RI HGXFDWLRQ DFKLHYHG E\ PRWKHU DQG IDWKHU RFFXSDWLRQ RI PRWKHU DQG IDWKHU KLVWRU\ RI GLDJQRVHG OHDUQLQJ GLVDELOLW\ /'f KLVWRU\ RI GLDJQRVH DWWHQWLRQ GHILFLWK\SHUDFWLYLW\ GLVRUGHU $'+'f KLVWRU\ RI QHXURORJLFDO SUREOHPV HJ HSLOHSV\ PLJUDLQH KHDGDFKHV HWFf KLVWRU\ RI VLJQLILFDQW PHGLFDO LOOQHVV HJ DVWKPD GLDEHWHV KHDUW SUREOHPV HWFf KLVWRU\ RI SV\FKRORJLFDO LOOQHVV HJ GHSUHVVLRQ DQ[LHW\ HWFf KLVWRU\ RI DOFRKRO DQG VXEVWDQFH DEXVH DQG KLVWRU\ RI UHFHQW PHGLFDWLRQ XVH ,Q DGGLWLRQ HDFK SDUWLFLSDQW ZDV DVNHG WR SURYLGH D KLVWRU\ RI SUHYLRXV FRQFXVVLRQ LQFOXGLQJ GDWH GHVFULSWLRQ RI FLUFXPVWDQFHV VXUURXQGLQJ WKH LQFLGHQW LQFLGHQFH RI ORVV RI FRQVFLRXVQHVV LQFLGHQFH RI SRVW FRQFXVVLYH V\PSWRPV

PAGE 43

OHQJWK RI V\PSWRP SUHVHQWDWLRQ DQG WKH UHVXOWV RI LPDJLQJ WHVWV LI DQ\f $OO FRQFXVVLRQV ZHUH VXEVHTXHQWO\ FODVVLILHG DFFRUGLQJ WKH JUDGLQJ V\VWHP RXWOLQHG E\ WKH $PHULFDQ $FDGHP\ RI 1HXURORJ\ f 3URWRFRO %DVHOLQH $SSURYDO E\ WKH ,QVWLWXWLRQDO 5HYLHZ %RDUGV IRU HDFK UHVSHFWLYH VFKRRO ZDV REWDLQHG (DFK SDUWLFLSDQW ZDV SURYLGHG WKH RSSRUWXQLW\ WR UHYLHZ WKH QDWXUH SXUSRVH DQG SURMHFWHG XVH RI WKH GDWD ZLWK HLWKHU D 3K' SV\FKRORJLVW RU D GRFWRUDO OHYHO VWXGHQW $OO GDWD ZHUH FROOHFWHG E\ 3K'OHYHO SV\FKRORJLVWV GRFWRUDO VWXGHQWV RU LQ VRPH LQVWDQFHV WHDP SK\VLFLDQV DQG DWKOHWLF WUDLQHUV ZKR KDG EHHQ WUDLQHG LQ DGPLQLVWHULQJ WKLV EULHI EDWWHU\ (DFK KDG DWWHQGHG D WUDLQLQJ VHVVLRQ LQ WHVW DGPLQLVWUDWLRQ LQ RUGHU WR HQVXUH VWDQGDUGL]HG DGPLQLVWUDWLRQ WHFKQLTXHV 7HVWV ZHUH VFRUHG LQ D VWDQGDUGL]HG IDVKLRQ DQG WKH UHVXOWV IURP RWKHU LQVWLWXWLRQV ZHUH VHQW WR WKLV LQYHVWLJDWRU IRU HQWU\ LQWR D FHQWUDOL]HG GDWDEDVH (DFK SDUWLFLSDQW ZDV DGPLQLVWHUHG D EULHI QHXURSV\FKRORJLFDO EDWWHU\ LQFOXGLQJ WKH IROORZLQJ WHVWV +RSNLQV 9HUEDO /HDUQLQJ 7HVW +9/7f 7UDLO 0DNLQJ 7HVWV $ DQG % WKH 'LJLW 6SDQ VXEWHVW RI WKH :HFKVOHU $GXOW ,QWHOOLJHQFH 6FDOH5HYLVHG 6\PERO 'LJLW 0RGDOLWLHV 7HVW '607f *URRYHG 3HJERDUG DQG WKH &RQWUROOHG 2UDO :RUG $VVRFLDWLRQ WHVW &2:$f 7KLV EDWWHU\ LV RQH WKDW LV FXUUHQWO\ XVHG LQ D SURIHVVLRQDO IRRWEDOO VWXG\ RQ WKH HIIHFWV RI FRQFXVVLRQ DQG ZDV UHFRPPHQGHG E\ /RYHOO DQG &ROOLQV f IRU VWXG\ RI FRQFXVVLRQ LQ D FROOHJH IRRWEDOO SRSXODWLRQ 7KH +RSNLQV 9HUEDO /HDUQLQJ 7HVW %UDQGW f FRQVLVWV RI D SUHVHQWDWLRQ RI ZRUGV EHORQJLQJ WR WKUHH VHPDQWLF FDWHJRULHV 7KUHH OHDUQLQJ WULDOV DUH DGPLQLVWHUHG

PAGE 44

DIWHU ZKLFK D GHOD\ SHULRG LV LQWHUMHFWHG IRU DSSUR[LPDWHO\ PLQXWHV 7KH VXEMHFW LV WKHQ DVNHG WR UHFDOO DV PDQ\ ZRUGV DV WKH\ FDQ UHPHPEHU $IWHU WKLV GHOD\HG IUHH UHFDOO D ZRUG UHFRJQLWLRQ OLVW LV SUHVHQWHG ZKLFK FRQWDLQV DOO RI WKH WDUJHW ZRUGV LQ DGGLWLRQ WR VHPDQWLFDOO\ UHODWHG ZRUGV DQG XQUHODWHG ZRUGV $V WKH ZRUGV DUH UHDG WR WKH VXEMHFW WKH\ DUH VLPSO\ DVNHG WR LGHQWLI\ WKRVH ZRUGV WKDW ZHUH SUHVHQWHG SUHYLRXVO\ 7KLV WHVW DOORZV IRU D PHDVXUH RI VKRUWWHUP YHUEDO PHPRU\ DWWHQWLRQ FRQFHQWUDWLRQ DQG OHDUQLQJ ,Q DGGLWLRQ D PHDVXUH RI GHOD\HG YHUEDO PHPRU\ LV FDOFXODWHG DV ZHOO DV D VFRUH IRU UHFRJQLWLRQ PHPRU\ ,Q DGGLWLRQ ZLWKLQ WKH UHFRJQLWLRQ WULDO VFRUHV IRU UHODWHG IDOVH SRVLWLYH UHVSRQVHV DQG XQUHODWHG IDOVH SRVLWLYHV DUH FDOFXODWHG $ GLVFULPLQDWLRQ VFRUH LV DOVR FDOFXODWHG WR GHWHUPLQH WKH VXEMHFWfV DELOLW\ WR GLIIHUHQWLDWH EHWZHHQ SUHYLRXVO\ SUHVHQWHG ZRUGV DQG IRLOV 7KH +9/7 KDV VL[ DOWHUQDWH IRUPV /H]DN f )RU WKH SXUSRVHV RI WKLV VWXG\ +RSNLQV 7RWDO VFRUH DQG +RSNLQV 'HOD\ VFRUH ZHUH H[DPLQHG 7UDLO 0DNLQJ 7HVW $ 5HLWDQ DQG :ROIVRQ f FRQVLVWV RI QXPEHUHG FLUFOHV 7KH VXEMHFW LV DVNHG WR FRQQHFW WKH QXPEHUHG FLUFOHV LQ WKH FRUUHFW RUGHU IURP WR f DV TXLFNO\ DV WKH\ FDQ ZLWKRXW PDNLQJ DQ\ PLVWDNHV RU OLIWLQJ WKHLU SHQFLO IURP WKH SDSHU 6FRUH RQ WKLV PHDVXUH LV UHFRUGHG LQ VHFRQGV WR FRPSOHWLRQ (UURUV GXULQJ WKH FRPSOHWLRQ RI WKH WDVN DUH LPPHGLDWHO\ SRLQWHG RXW WR WKH VXEMHFW DQG WKH\ DUH GLUHFWHG WR FRQWLQXH IURP WKH SRLQW RI WKHLU ODVW FRUUHFW PDUN 1R VFRUH LV NHSW IRU WKH QXPEHU RI HUURUV FRPPLWWHG LW LV IHOW WKDW WKLV LV UHIOHFWHG LQ DQ LQFUHDVHG WLPH RI FRPSOHWLRQ 7UDLO 0DNLQJ 7HVW $ LV WKRXJKW WR EH D PHDVXUH RI YLVXDO VFDQQLQJ DQG VHTXHQFLQJ DELOLW\ DV ZHOO DV PRWRU VSHHG 7UDLO 0DNLQJ 7HVW % 5HLWDQ DQG :ROIVRQ f ZKLOH VLPLODU LQ DSSHDUDQFH HQFRPSDVVHV FLUFOHV ZLWK ERWK OHWWHUV DQG QXPEHUV 7KH VXEMHFW LV DVNHG WR

PAGE 45

VWDUW ZLWK WKH QXPEHU DQG FRQQHFW WKH FLUFOHV LQ DOWHUQDWLQJ LQFUHDVLQJ VHTXHQFH LH $%& HWFf 7KH VFRUH IRU WKLV PHDVXUH WRR LV UHFRUGHG LQ VHFRQGV WR FRPSOHWLRQ 7KLV PHDVXUH LV WKRXJKW WR UHIOHFW D SHUVRQfV DELOLW\ WR HIIHFWLYHO\ VKLIW PHQWDO VHW DQG XWLOL]H FRJQLWLYH IOH[LELOLW\ LQ DGGLWLRQ WR YLVXDO VFDQQLQJ VHTXHQFLQJ DQG PRWRU VSHHG 7KHVH IXQFWLRQV DUH RIWHQ DVVRFLDWHG ZLWK IURQWDOH[HFXWLYH IXQFWLRQLQJ 6SUHHQ DQG 6WUDXVV f 7KH 'LJLW 6SDQ VXEWHVW RI WKH :$,6,,, :HFKVOHU f FRQVLVWV RI WKH SUHVHQWDWLRQ RI LQFUHDVLQJO\ ORQJ VHTXHQFHV RI QXPEHUV ,Q WKH )RUZDUG FRQGLWLRQ WKH VXEMHFW LV VLPSO\ DVNHG WR OLVWHQ WR WKH QXPEHUV DQG UHSHDW WKHP LQ WKH VDPH RUGHU LQ ZKLFK WKH\ ZHUH JLYHQ ,Q WKH UHYHUVH FRQGLWLRQ WKH\ DUH DVNHG WR UHFLWH WKHP EDFNZDUGV (DFK SDUW LV GLVFRQWLQXHG ZKHQ WKH VXEMHFW IDLOV WR FRUUHFWO\ UHFDOO WZR WULDOV RI QXPEHUV RI WKH VDPH OHQJWK 7KLV PHDVXUH LV WKRXJKW WR UHIOHFW DWWHQWLRQ DQG FRQFHQWUDWLRQ )RU WKH SXUSRVHV RI WKLV VWXG\ WKH 'LJLW 6SDQ WRWDO VFRUH IRUZDUG SOXV EDFNf ZDV H[DPLQHG 7KH 6\PERO 'LJLW 0RGDOLWLHV 7HVW 6'07f 6PLWK f LV VLPLODU WR WKH 'LJLW 6\PERO VXEWHVW RI WKH :$,65 LQ WKDW LV UHTXLUHV WKH VXEMHFW WR SURYLGH TXLFN VXEVWLWXWLRQV EXW LW LQ WKLV PHDVXUH WKH VXEMHFW LV DVNHG WR ZULWH WKH FRUUHFW QXPEHU XQGHU HDFK V\PERO EDVHG RQ D NH\ 6XEMHFWV DUH JLYHQ VHFRQG WR FRPSOHWH DV PDQ\ LWHPV DV WKH\ FDQ 7KLV WHVW ZDV FKRVHQ EDVHG RQ LWV DELOLW\ WR PHDVXUH YLVXDO VFDQQLQJ DQG YLVXR PRWRU VSHHG ,PPHGLDWHO\ IROORZLQJ LWV DGPLQLVWUDWLRQ WKH VXEMHFW LV DVNHG WR UHFDOO DV PDQ\ RI WKHVH DVVRFLDWLRQV DV WKH\ FDQ ZKLFK SURYLGHV D PHDVXUH RI LQFLGHQWDO OHDUQLQJ 7KH YDULDEOH H[DPLQHG DV SDUW RI WKLV VWXG\ ZDV WRWDO QXPEHU RI V\PEROV FRUUHFWO\ SURGXFHG LQ WKH VHFRQG WHVW DGPLQLVWUDWLRQ

PAGE 46

7KH *URRYHG 3HJERDUG WHVW 0DWWKHZV DQG .ORYH f FRQVLVWV RI D VPDOO ERDUG FRQWDLQLQJ D [ PDWUL[ RI VORWWHG KROHV DUUDQJHG LQ YDULRXV DQJOHV 7KH VXEMHFW LV JLYHQ SHJV HDFK WKH VDPH VKDSH DQG DVNHG WR SXW WKHP LQ VXFFHVVLYH VORWV VWDUWLQJ ZLWK WKH WRS URZ DV TXLFNO\ DV WKH\ FDQ ZLWKRXW GURSSLQJ 7KH\ DUH DOORZHG RQO\ WR KROG RQH SHJ DW D WLPH ZLWK RQH KDQG 7LPH WR FRPSOHWHO\ ILOO WKH KROHV DUH PHDVXUHG IRU HDFK KDQG LQGHSHQGHQWO\ /H]DN f 7KLV WHVW ZDV FKRVHQ IRU LQFOXVLRQ LQ WKH EDWWHU\ GXH WR LWV VHQVLWLYLW\ WR PRWRU VSHHG DQG FRRUGLQDWLRQ 'RPLQDQW KDQG DQG QRQGRPLQDQW KDQG FRPSOHWLRQ WLPHV ZHUH H[DPLQHG LQ WKH FXUUHQW VWXG\ 7KH &RQWUROOHG 2UDO :RUG $VVRFLDWLRQ 7HVW &2:$f %HQWRQ DQG +DPVKHU f FRQVLVWV RI WKUHH WULDOV ,Q HDFK WKH VXEMHFW LV SUHVHQWHG ZLWK D OHWWHU DQG DVNHG WR SURGXFH DV PDQ\ ZRUGV DV WKH\ FDQ XQWLO WKH H[DPLQHU DVNV WKHP WR VWRS 7KH\ DUH WROG WKDW WKH\ PXVW QRW XVH SURSHU QRXQV RU WKH VDPH ZRUGV ZLWK GLIIHUHQW HQGLQJV 7KH VXEMHFW LV JLYHQ VHFRQGV WR SURGXFH DV PDQ\ ZRUGV DV WKH\ FDQ 7KLV PHDVXUH ZDV FKRVHQ IRU LWV DELOLW\ WR UHIOHFW DELOLWLHV RI ZRUG IOXHQF\ DQG UHWULHYDO 7KH FXUUHQW VWXG\ XWLOL]HG WRWDO ZRUGV SURGXFHG RYHU DOO WKUHH WULDOV DV WKH YDULDEOH XQGHU H[DPLQDWLRQ ,Q DGGLWLRQ WR WKHVH QHXURSV\FKRORJLFDO WHVWV WKH SDUWLFLSDQW ZHUH DGPLQLVWHUHG WKH &RQFXVVLRQ 6\PSWRP 6FDOH LQ RUGHU WR H[DPLQH EDVHOLQH OHYHOV RI V\PSWRPV FRPPRQO\ DVVRFLDWHG ZLWK SRVW FRQFXVVLRQ V\QGURPH HJ KHDGDFKH PHPRU\ GLIILFXOWLHV FRQFHQWUDWLRQ GLIILFXOWLHV VHQVLWLYLW\ WR OLJKW RU QRLVH HWFf 7KLV LV D VXEMHFWLYH TXHVWLRQQDLUH XWLOL]LQJ D /LNHUWW\SH VFDOH ,QVHDVRQ FRQFXVVLRQ HYDOXDWLRQ 7KRVH DWKOHWHV ZKR VXVWDLQHG D FRQFXVVLRQ GXULQJ WKH IRRWEDOO VHDVRQ HLWKHU LQ SUDFWLFH RU LQ D JDPH ZHUH HYDOXDWHG ZLWK WKLV VDPH EDWWHU\ XWLOL]LQJ DOWHUQDWH IRUPV

PAGE 47

ZKHUH DYDLODEOHf ZLWKLQ KRXUV RI WKH LQFLGHQW DQG DW DQG GD\V SRVWFRQFXVVLRQ ,QVHDVRQ FRQFXVVLRQ ZDV LQLWLDOO\ HYDOXDWHG E\ WUDLQHUV RU WHDP SK\VLFLDQV DQG ZDV WKHQ UHIHUUHG IRU QHXURSV\FKRORJLFDO WHVWLQJ DV SDUW RI WKLV VWXG\ *UDGH RI FRQFXVVLRQ ZDV HYDOXDWHG EDVHG RQ WKH $$1 JXLGHOLQHV ,Q DGGLWLRQ ZLWKLQ KRXUV RI FRQFXVVLRQ FLUFXPVWDQFHV FRQWULEXWLQJ WR WKH FRQFXVVLRQ ZHUH REWDLQHG WKURXJK LQWHUYLHZ ZLWK WUDLQHUV DQG YLGHRWDSH RI WKH LQFLGHQW LI DYDLODEOHf ZDV UHYLHZHG LQ VRPH FDVHV )RRWEDOO SOD\HUV ZLWKLQ HDFK UHVSHFWLYH WHDP VHUYHG DV FRQWUROV 7KH\ ZHUH PDWFKHG DV FORVHO\ DV SRVVLEOH WR WKH FRQFXVVHG SOD\HU LQ WHUPV RI WKH IROORZLQJ YDULDEOHV UDFH SRVLWLRQ KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI KHDG LQMXU\ $WKOHWHV ZHUH HQFRXUDJHG WR FRQWLQXH ZLWK WHVWLQJ SURWRFRO RYHU DOO WHVW LQWHUYDOV +RZHYHU VRPH GLIILFXOW\ ZDV HQFRXQWHUHG ZLWK DWWULWLRQ HVSHFLDOO\ LQ WKH FRQWURO SRSXODWLRQ 8OWLPDWHO\ GDWD IURP SDUWLFLSDWLQJ LQGLYLGXDOV ZDV H[FOXGHG OLVWZLVH 7KDW LV D SOD\HUfV GDWD ZDV XWLOL]HG RYHU HDFK UHWHVW LQWHUYDO RQO\ LI WKH\ JHQHUDWHG GDWD IRU HDFK FRUUHVSRQGLQJ WHVW VHVVLRQ 7KLV UHVXOWHG LQ D GHFUHDVLQJ VWXG\ SRSXODWLRQ RYHU HDFK VXFFHVVLYH WHVW LQWHUYDO $W WKH KRXU WHVW VHVVLRQ WRWDO QXPEHU RI FRQFXVVHG SOD\HUV ZKR KDG GDWD DW ERWK EDVHOLQH DQG KRXUV ZDV 7KH QXPEHU RI FRQWUROV DW WKH VDPH LQWHUYDO ZDV )RU WKH 'D\ WHVW VHVVLRQ WKH WRWDO QXPEHU RI FRQFXVVHG SOD\HUV ZDV ZLWK FRQWUROV $W 'D\ WKH QXPEHU RI FRQFXVVHG SOD\HUV ZDV ZLWK FRQWUROV $W 'D\ WKHUH ZHUH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV 'DWD $QDO\VLV 'DWD DQDO\VLV LQFOXGHG WKH DSSOLFDWLRQ RI VWDWLVWLFDO PRGHOV RI VLJQLILFDQFH GHWHFWLRQ 7KHVH ZHUH FRQFHSWXDOO\ GLYLGHG LQWR WKUHH EURDG FDWHJRULHV EDVHG RQ WKHLU VWDWLVWLFDO PHWKRGRORJ\

PAGE 48

%DVLF 0RGHOV 7KH ILUVW VXEJURXS FRQVLVWHG RI WZR EDVLF FOLQLFDOO\RULHQWHG DSSURDFKHV 0RGHO DQG 0RGHO f )RU 0RGHO HDFK RI WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWVf UHWHVW VFRUHV IRU HDFK PHDVXUH ZDV FRPSDUHG WR WKHLU UHVSHFWLYH QRUPDWLYH PHDQ VFRUHV DQG VWDQGDUG GHYLDWLRQV ZKLFK ZHUH GHULYHG IURP WKH ODUJHU EDVHOLQH JURXS RI IRRWEDOO SOD\HUV Q f XOWLPDWHO\ UHVXOWLQJ LQ D VWDQGDUGL]HG ]VFRUH 6LJQLILFDQW ILQGLQJV ZHUH FODVVLILHG XVLQJ D VWDQGDUG FOLQLFDO DSSURDFK 5HLWDQ DQG :ROIVRQ f ZKHUH VWDQGDUG GHYLDWLRQ EHORZ WKH PHDQ ZDV LQWHUSUHWHG DV GHQRWLQJ PLOG LPSDLUPHQW ,Q 0RGHO HDFK FRQFXVVHG DQG FRQWURO SDUWLFLSDQWVf UHWHVW VFRUHV IRU HDFK YDULDEOH ZHUH FRPSDUHG WR WKHLU RZQ EDVHOLQH SHUIRUPDQFH DJDLQ XVLQJ D FULWHULRQ RI VWDQGDUG GHYLDWLRQ EDVHG RQ WKH QRUPDWLYH SRSXODWLRQf GHFUHDVH LQ VFRUH WR GHQRWH D ILQGLQJ VXJJHVWLQJ VLJQLILFDQW FRJQLWLYH GHFOLQH IROORZLQJ FRQFXVVLRQ 5HOLDEOH &KDQJH ,QGLFHV 7KH VHFRQG FRQFHSWXDO JURXS RI VWDWLVWLFDO PRGHOV LQFOXGHG WKUHH YDULDQWV RI WKH 5HOLDEOH &KDQJH ,QGH[ 0RGHO 0RGHO DQG 0RGHO f 0RGHO LQYROYHG WKH DSSOLFDWLRQ RI D EDVLF 5&, DQDO\VLV DV GHVFULEHG E\ -DFREVRQ DQG 7UXD[ f DQG XVHG LQ VWXGLHV E\ 7HPNLQ HW DO f DQG %DUU DQG 0F&UDH f WR GHWHFW FKDQJH LQ VHULDO QHXURSV\FKRORJLFDO WHVWLQJ 0RGHO XWLOL]HG WKH DSSOLFDWLRQ RI WKH VDPH 5&, SURFHGXUHV ZLWK FRUUHFWLRQ IRU SUDFWLFH )RU ERWK 0RGHO DQG 0RGHO ILQGLQJV IRU HDFK YDULDEOH ZHUH GHHPHG VLJQLILFDQW LI WKH UHWHVW VFRUH IHOO RXWVLGH WKH FRPSXWHG FKDQJH LQGH[ LQ WKH QHJDWLYH GLUHFWLRQ )RU 0RGHO HDFK FRQFXVVHG SDUWLFLSDQWVf FKDQJH RQ WKH LQGLYLGXDO YDULDEOHV IURP EDVHOLQH WR UHWHVW LQWHUYDO ZDV GLUHFWO\ FRPSDUHG WR WKH DYHUDJH FKDQJH VFRUH IRU WKH FRQWURO SRSXODWLRQ RYHU WKH VDPH WLPH SHULRG $ VLJQLILFDQW ILQGLQJ ZDV

PAGE 49

GHQRWHG E\ D FKDQJH VFRUH ZKLFK IHOO EHORZ D b FRQILGHQFH LQWHUYDO EDVHG RQ WKH VWDQGDUG GHYLDWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO JURXS RQ HDFK YDULDEOH DQG WHVW LQWHUYDO 5HJUHVVLRQ 0RGHOV 7KH WKLUG FRQFHSWXDO VXEJURXS RI VWDWLVWLFDO PRGHOV LQFOXGHG WZR UHJUHVVLRQ DSSURDFKHV 0RGHO DQG 0RGHO f &RQVLVWHQW ZLWK WKH UHVHDUFK RI 7HPNLQ HW DO f DQG %DUU DQG 0F&UHD f LQ 0RGHO VLPSOH OLQHDU UHJUHVVLRQ HTXDWLRQV ZHUH FDOFXODWHG EDVHG RQ FRQWURO SRSXODWLRQ GDWD LQ RUGHU WR SUHGLFW UHWHVW VFRUHV IRU HDFK YDULDEOH RYHU WKH IRXU WHVW LQWHUYDOV 7KH REVHUYHG VFRUHV ZHUH WKHQ FRPSDUHG WR WKH SUHGLFWHG VFRUHV DQG D VWDQGDUG ]VFRUH ZDV FDOFXODWHG XVLQJ WKH VWDQGDUG HUURU RI WKH UHVLGXDOV IRU WKH PRGHO EDVHG RQ WKH FRQWURO SDUWLFLSDQWV 2EVHUYHG UHWHVW VFRUHV IDOOLQJ EH\RQG D VLJQLILFDQFH YDOXH RI LQ WKH QHJDWLYH GLUHFWLRQ ZHUH FODVVLILHG DV VLJQLILFDQWO\ EHORZ H[SHFWDWLRQ ,Q 0RGHO D PXOWLSOH UHJUHVVLRQ HTXDWLRQ DSSURDFK XVLQJ KLVWRU\ RI GLDJQRVHG OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ DV DGGLWLRQDO SUHGLFWLYH LQGLFDWRUV ZDV DQDO\]HG 6HQVLWLYLW\ DQG 6SHFLILFLW\ 6HQVLWLYLW\ YDOXHV ZHUH FDOFXODWHG IRU HDFK YDULDEOH RYHU WKH WHVW VHVVLRQV IRU HDFK PRGHO )RU LQGLYLGXDO YDULDEOHV VHQVLWLYLW\ ZDV GHWHUPLQHG E\ WKDW YDULDEOHfV SUREDELOLW\ RI DFFXUDWHO\ FODVVLI\LQJ D FRQFXVVHG SOD\HU DV EHLQJ VLJQLILFDQWO\ EHORZ H[SHFWDWLRQ EDVHG RQ WKH DSSOLHG VWDWLVWLFDO PRGHO (DFK PRGHO ZDV WKHQ H[DPLQHG RYHU DOO WHVW VHVVLRQ WR GHWHUPLQH KRZ PDQ\ YDULDEOHV IURP WKH EDWWHU\ RI QHXURSV\FKRORJLFDO WHVWV DV D ZKROH ZHUH IRXQG WR KDYH EHHQ VLJQLILFDQWO\ EHORZ H[SHFWDWLRQ IRU ERWK FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV $

PAGE 50

FODVVLILFDWLRQ FXWSRLQW WKH RSWLPDO QXPEHU RI VLJQLILFDQW YDULDEOHV EHORZ H[SHFWDWLRQf ZDV GHWHUPLQHG EDVHG RQ WKH KLJKHVW GHJUHH RI VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQ DW HDFK WLPH LQWHUYDO ZKHUH VHQVLWLYLW\ ZDV GHWHUPLQHG E\ WKH QXPEHU RI FRQFXVVHG SDUWLFLSDQWV ZKR ZHUH DFFXUDWHO\ FODVVLILHG DV FRQFXVVHG DW HDFK FXWSRLQW DQG VSHFLILFLW\ ZDV GHWHUPLQHG E\ WKH QXPEHU RI FRQWURO SDUWLFLSDQWV ZKR ZHUH DFFXUDWHO\ FODVVLILHG DV fQRW FRQFXVVHGf DW WKRVH VDPH FXWSRLQWV &ODVVLILFDWLRQ DFFXUDF\ ZDV WKHQ DVVHVVHG XVLQJ 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF FXUYHV ZKLFK SURYLGHG D VWDWLVWLFDO LQGLFDWLRQ RI HDFK PRGHOfV DELOLW\ WR DFFXUDWHO\ GLVWLQJXLVK EHWZHHQ WKH FRQFXVVHG DQG QRQFRQFXVVHG SDUWLFLSDQWV UHODWLYH WR FKDQFH +DQOH\ f

PAGE 51

&+$37(5 7+5(( 5(68/76 'HPRJUDSKLF 'DWD IRU %DVHOLQH 3RSXODWLRQ 'XULQJ WKH FXUUHQW VWXG\ GDWD ZDV FROOHFWHG XVLQJ WKH DERYHGHVFULEHG SUHVHDVRQ QHXURSV\FKRORJLFDO WHVW EDWWHU\ DQG GHPRJUDSKLF YDULDEOHV ZHUH FROOHFWHG IURP D WRWDO RI IRRWEDOO SOD\HUV DW IRXU XQLYHUVLWLHV 7KH WRWDO QXPEHU RI SUHVHDVRQ SDUWLFLSDQWV IURP HDFK ORFDWLRQ LV SUHVHQWHG LQ 7DEOH DQG LV SUHVHQWHG JUDSKLFDOO\ LQ )LJXUH 7DEOH 1XPEHUV DQG SHUFHQWDJH RI WRWDO SUHVHDVRQ SDUWLFLSDQWVf ORFDWLRQV 1 3HUFHQW &XPXODWLYH 3HUFHQW 8QLYHUVLW\ RI )ORULGD 0LFKLJDQ 6WDWH 8QLYHUVLW\ 8QLYHUVLW\ RI 8WDK 8QLYHUVLW\ RI 3LWWVEXUJK /RFDWLRQ )LJXUH 3HUFHQW RI EDVHOLQH SDUWLFLSDQWV IURP HDFK FRQWULEXWLQJ VFKRRO

PAGE 52

7DEOH DQG )LJXUH LQGLFDWH WKH UDFLDO FRPSRVLWLRQ RI WKH WRWDO QXPEHU RI EDVHOLQH SDUWLFLSDQWV 7KH PDMRULW\ RI WKHVH ZHUH $IULFDQ $PHULFDQ bf FORVHO\ IROORZHG E\ &DXFDVLDQV bf 7KHUH ZHUH VLJQLILFDQWO\ IHZHU +LVSDQLF $VLDQ DQG 6DPRDQ SDUWLFLSDQWV 7DEOH 5DFLDO FRPSRVLWLRQ RI EDVHOLQH SRSXODWLRQ 1 3HUFHQW &XPXODWLYH 3HUFHQW $IULFDQ $PHULFDQ &DXFDVLDQ +LVSDQLF $VLDQ +DLWLDQ 6DPRDQ 5$&( 6DPRDQ b $IULFDQ $PHULFDQ b )LJXUH 5DFLDO FRPSRVLWLRQ RI EDVHOLQH SRSXODWLRQ DFURVV VLWHV 7DEOH LQGLFDWHV WKH IUHTXHQFLHV RI DJHV RI WKH EDVHOLQH SRSXODWLRQ 7KH PHDQ DJH RI EDVHOLQH SRSXODWLRQ SDUWLFLSDQWV ZDV \HDUV ZLWK D VWDQGDUG GHYLDWLRQ RI \HDUV

PAGE 53

7DEOH 'LVWULEXWLRQ RI DJHV DFURVV WKH EDVHOLQH SRSXODWLRQ $JH \HDUVf 1 3HUFHQW &XPXODWLYH 3HUFHQW $*( )LJXUH 'LVWULEXWLRQ RI DJHV IRU EDVHOLQH SRSXODWLRQ 7DEOH LQGLFDWHV WKH IUHTXHQFLHV RI \HDU LQ VFKRRO IRU DOO SDUWLFLSDQWV LQ WKH EDVHOLQH SRSXODWLRQ 7DEOH 'LVWULEXWLRQ RI FODVV VWDWXV IRU WKH EDVHOLQH SRSXODWLRQ 1 3HUFHQW &XPXODWLYH 3HUFHQW )UHVKPDQ 6RSKRPRUH -XQLRU 6HQLRU )LIWK
PAGE 54

'HPRJUDSKLF &KDUDFWHULVWLFV IRU &RQFXVVLRQ DQG &RQWURO 3DUWLFLSDQWV $V LQGLFDWHG DERYH LQVHDVRQ FRQFXVVLRQ GDWD XWLOL]LQJ UHDGPLQLVWUDWLRQ RI WKH EDVHOLQH QHXURSV\FKRORJLFDO WHVW EDWWHU\ ZDV FRQGXFWHG IRU WKRVH LQGLYLGXDOV ZKR ZHUH UHSRUWHG E\ WHDP WUDLQHUV WR KDYH UHFHLYHG D FRQFXVVLRQ HLWKHU GXULQJ SUDFWLFH RU GXULQJ D JDPH ,QLWLDO UHWHVWLQJ WRRN SODFH ZLWKLQ KRXUV RI WKH LGHQWLILHG FRQFXVVLYH HYHQW DW WKUHH GD\V ILYH GD\V DQG VHYHQ GD\V SRVWFRQFXVVLRQ 'XULQJ WKH WLPH RI WKH LQYHVWLJDWLRQ WUDLQHULGHQWLILHG FRQFXVVLYH HYHQWV UHVXOWHG LQ D WRWDO RI FRQFXVVHG SOD\HUV LQFOXGLQJ WKUHH SOD\HUV ZKR LQFXUUHG WZR FRQFXVVLRQV GXULQJ WKH WLPH RI WKH VWXG\ 7DEOH LQGLFDWHV WKH GHPRJUDSKLF FKDUDFWHULVWLFV RI WKH FRQFXVVHG SOD\HUV 7DEOH 'HPRJUDSKLFV FKDUDFWHULVWLFV RI WKH FRQFXVVHG SOD\HUV 1 3HUFHQW $JH 5DFH $IULFDQ $PHULFDQ &DXFDVLDQ +LVWRU\ RI /'
PAGE 55

IDLOHG WR FRQWLQXH LQ WKH VWXG\ WKH WLPH UHTXLUHPHQWV GLG QRW DOORZ IRU UHSODFHPHQW ZLWK DQRWKHU FRQWURO SDUWLFLSDQW 6XEVHTXHQWO\ WKH WRWDO QXPEHU RI FRQWURO SDUWLFLSDQWV ZDV OHVV WKDQ WKH WRWDO QXPEHU RI FRQFXVVHG VXEMHFWV UHVXOWLQJ LQ D WRWDO RI LQGLYLGXDOV 7DEOH LQGLFDWHV WKH GHPRJUDSKLF FKDUDFWHULVWLFV RI WKH FRQWURO SRSXODWLRQ 7DEOH 'HPRJUDSKLF FKDUDFWHULVWLFV RI WKH FRQWURO SRSXODWLRQ 1 3HUFHQW $JH 5DFH $IULFDQ $PHULFDQ &DXFDVLDQ +LVWRU\ RI /' )f S @ QRU ZHUH WKHUH DQ\ VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH JURXSV ZLWK UHJDUG WR \HDU LQ VFKRRO

PAGE 56

>)f S @ DOWKRXJK WKHUH ZDV D WUHQG IRU WKH FRQFXVVHG SOD\HUV WR EH RI ORZHU VFKRRO \HDU ,W ZDV QRW VXUSULVLQJ WKHUHIRUH WKDW D VLJQLILFDQW HIIHFW IRU DJH ZDV REVHUYHG >)f S @ +RZHYHU SDLUHG WWHVWV XVLQJ D %RQIHUURQL FRUUHFWHG SYDOXH RI f IDLOHG WR GHPRQVWUDWH D VLJQLILFDQW GLIIHUHQFH EHWZHHQ WKH JURXSV LQ WHUPV RI DJH $JDLQ D WUHQG ZDV REVHUYHG IRU WKH FRQFXVVHG SOD\HUV WR EH VOLJKWO\ \RXQJHU WKDQ WKH EDVHOLQH SRSXODWLRQ DV D ZKROH WKRXJK WKLV ZDV QRW VWDWLVWLFDOO\ VLJQLILFDQW )RU WKH SXUSRVHV RI WKLV VWXG\ ZKLFK H[DPLQHG EDVHOLQH WR UHWHVW VHVVLRQ SHUIRUPDQFH ZLWK KRXUV GD\ GD\ DQG GD\ f SDUWLFLSDQWV ZHUH H[FOXGHG OLVWZLVH 7KDW LV SDUWLFLSDQWV PXVW KDYH KDG D VFRUH DW ERWK WHVW VHVVLRQV IRU LQFOXVLRQ LQWR H[DPLQDWLRQ RI SHUIRUPDQFH IRU D SDUWLFXODU WHVW LQWHUYDO 7KLV UHVXOWHG LQ IHZHU LQFOXGHG SDUWLFLSDQWV IRU HDFK VXFFHVVLYH WLPH LQWHUYDO 8OWLPDWHO\ FRQFXVVHG DQG FRQWUROV ZHUH LQFOXGHG IRU WKH KRXU DQDO\VLV FRQFXVVHG DQG FRQWUROV IRU WKH 'D\ DQDO\VLV FRQFXVVHG DQG FRQWUROV IRU 'D\ DQG FRQFXVVHG DQG FRQWUROV IRU 'D\ %DVHOLQH 3HUIRUPDQFH $V QRWHG DERYH WHQ YDULDEOHV ZHUH H[DPLQHG IURP WKH EDWWHU\ RI QHXURSV\FKRORJLFDO WHVWV DGPLQLVWHUHG GXULQJ WKH FXUUHQW VWXG\ 7KRVH YDULDEOHV ZHUH +RSNLQV WRWDO VFRUH 7UDLOV $ WRWDO WLPH LQ VHFRQGV 7UDLOV % WRWDO WLPH LQ VHFRQGV *URRYHG 3HJERDUG 7HVW GRPLQDQW DQG QRQGRPLQDQW KDQG WLPH LQ VHFRQGV GLJLW VSDQ IRUZDUG GLJLW VSDQ EDFNZDUG 'LJLW 6\PERO WLPH LQ VHFRQGV &2:$ WRWDO VFRUH DQG +RSNLQV 'HOD\ VFRUH 7DEOH VKRZV WKH PHDQV DQG VWDQGDUG GHYLDWLRQV IRU WKH WRWDO %DVHOLQH SRSXODWLRQ DV ZHOO DV WKH FRQFXVVHG SDUWLFLSDQWV DQG FRQWURO VXEMHFWV DW EDVHOLQH

PAGE 57

7DEOH %DVHOLQH 3HUIRUPDQFH RI WRWDO SRSXODWLRQ FRQFXVVHG DQG FRQWUROV 0HDVXUH 7RWDO %DVHOLQH Q &RQWUROV Q &RQFXVVHG Q 0 6'f 0 6'f 0 6'f +RSNLQV 7RWDO f f f +RSNLQV 'HOD\ f f f 7UDLOV $ f f f 7UDLOV % f f f 'LJLWV )RUZDUG f f f 'LJLWV %DFN f f f 'LJLW 6\PERO f f f 3HJV 'RPLQDQW f f f 3HJV 1RQGRPLQDQW f f f &2:$ 7RWDO f f f $ VHULHV RI RQHZD\ $129$fV ZDV UXQ LQ RUGHU WR GHWHFW DQ\ VLJQLILFDQW GLIIHUHQFHV EHWZHHQ WKH JURXSV ZLWK UHJDUG WR WKHLU EDVHOLQH SHUIRUPDQFH RQ DQ\ RI WKHVH WHQ PHDVXUHV 7DEOH SURYLGHV D VXPPDU\ RI WKRVH ILQGLQJV 7KHUH ZDV D VLJQLILFDQW ILQGLQJ ZLWK UHJDUG WR EDVHOLQH SHUIRUPDQFH EHWZHHQ WKH WKUHH JURXSV RQ WKH 'LJLW 6\PERO WHVW >) f S @ 6XEVHTXHQWO\ SDLUHG W WHVWV ZHUH SHUIRUPHG XVLQJ D %RQIHUURQL FRUUHFWHG VLJQLILFDQFH DOSKD YDOXH RI f 5HVXOWV LQGLFDWHG WKDW WKH FRQWURO VXEMHFWV VFRUHG RQ DYHUDJH VLJQLILFDQWO\ EHWWHU WKDQ GLG WKH FRQFXVVHG SDUWLFLSDQWV W S f DW EDVHOLQH )XUWKHU WKHUH ZDV DOVR D WUHQG LQGLFDWLQJ WKDW WKH FRQWURO SDUWLFLSDQWVf VFRUHV ZHUH VLJQLILFDQWO\ EHWWHU WKDQ WKH FRQWURO JURXS DW EDVHOLQH DOWKRXJK WKLV GLG QRW UHDFK VLJQLILFDQFH DIWHU DSSO\LQJ FRUUHFWLRQ IRU PXOWLSOH WWHVWV +RZHYHU EHFDXVH WKH GLIIHUHQFH EHWZHHQ WKH FRQFXVVHG JURXS DQG FRQWURO JURXS VFRUHV RQ WKH 'LJLW 6\PERO WHVW DW EDVHOLQH ZDV IRXQG WR EH VLJQLILFDQW WKLV YDULDEOH ZDV RPLWWHG IURP IXUWKHU DQDO\VLV

PAGE 58

7DEOH 2QHZD\ $129$fV XVLQJ VWDWXV DV EHWZHHQVXEMHFWV IDFWRU 0HDVXUH GI ) 6LJQLILFDQFH +RSNLQV 7RWDO f +RSNLQV 'HOD\ f 7UDLOV $ f 7UDLOV % f 'LJLWV )RUZDUG f 'LJLWV %DFNZDUG f 'LJLW 6\PERO f *URRYHG 3HJERDUG 'RP f *URRYHG 3HJERDUG 1RQGRP f &2:$ 7RWDO f 2YHUYLHZ RI 5HSHDWHG 7HVWLQJ 5HSHDWHG WHVWLQJ XWLOL]LQJ WKH DERYHGHVFULEHG QHXURSV\FKRORJLFDO EDWWHU\ JHQHUDOO\ \LHOGHG UHVXOWV FRQVLVWHQW ZLWK KLVWRULFDO UHYLHZV RI WHVWUHWHVW SHUIRUPDQFH IRU WKRVH KDYLQJ VXIIHUHG PLQRU KHDG LQMXU\ 7KDW LV WKH FRQFXVVHG SDUWLFLSDQWV LQ WKH FXUUHQW VWXG\ RQ DYHUDJH H[KLELWHG D GHFOLQH RU D VWDVLVf LQ SHUIRUPDQFH DFURVV WHVW YDULDEOHV ZLWKLQ KRXUV RI FRQFXVVLRQ ZKHQ FRPSDUHG WR WKHLU EDVHOLQH SHUIRUPDQFH ([DPLQDWLRQ RI VXEVHTXHQW SHUIRUPDQFHV RQ GD\V DQG GD\V SRVW FRQFXVVLYH HYHQW JHQHUDOO\ VXJJHVWHG D JUDGXDO LPSURYHPHQW RYHU WKHVH GD\V 7KLV LV SDUWLFXODUO\ HYLGHQW ZKHQ FRPSDULQJ WKH JUDSKLFDO UHSUHVHQWDWLRQ RI WKHVH SHUIRUPDQFHV FRPSDUHG WR WKH FRQWURO SRSXODWLRQfV DYHUDJH SHUIRUPDQFH RYHU WKLV VDPH WLPH LQWHUYDO )LJXUHV WKURXJK UHSUHVHQW WKH DYHUDJH SHUIRUPDQFH RI LQGLYLGXDOV LQ ERWK WKH FRQFXVVHG DQG FRQWURO JURXSV RYHU WKH WHVWLQJ SHULRG 'HWHFWLQJ 6LJQLILFDQW 'HFOLQH 8VLQJ %DVLF 6WDWLVWLFDO 0HWKRGV *HQHUDOO\ VSHDNLQJ QHXURSV\FKRORJLFDO WHVWLQJ LQYROYHV WKH H[DPLQDWLRQ RI DQ LQGLYLGXDO DIWHU D NQRZQ RU VXVSHFWHG QHXURORJLFDO LQVXOW KDV RFFXUUHG

PAGE 59

+RSNLQV 7RWDO 6FRUH 7HVWLQJ 6HVVLRQ )LJXUH +RSNLQV 9HUEDO /HDUQLQJ 7HVW 5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV 7UDLO 0DNLQJ 7HVW $ 7HVWLQJ 6HVVLRQ )LJXUH 7UDLO 0DNLQJ 7HVW $ 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV

PAGE 60

7UDLO 0DNLQJ 7HVW % 7HVW 6HVVLRQ )LJXUH 7UDLO 0DNLQJ 7HVW % 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV 'LJLW 6SDQ )RUZDUG 7HVW 6HVVLRQ )LJXUH 'LJLW 6SDQ )RUZDUG 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV

PAGE 61

'LJLW 6SDQ %DFNZDUG 7HVW 6HVVLRQ )LJXUH 'LJLW 6SDQ %DFNZDUG 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV *URRYHG 3HJERDUG 'RPLQDQW +DQG 7HVW 6HVVLRQ )LJXUH *URRYHG 3HJERDUG 'RPLQDQW +DQGf 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV

PAGE 62

*URRYHG 3HJERDUG 1RQGRPLQDQW +DQG 7HVW 6HVVLRQ )LJXUH *URRYHG 3HJERDUG 1RQGRP +DQGf 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV Y 7HVW 6HVVLRQ )LJXUH &2:$ 7HVW5HWHVW 3HUIRUPDQFH IRU %RWK *URXSV

PAGE 63

+RSNLQV 'HOD\ 6FRUH LQVHDVRQ FRQFXVVLRQ FRQWURO )LJXUH +RSNLQV 'HOD\ 3HUIRUPDQFH IRU %RWK *URXSV ,Q PRVW LQVWDQFHV WKH H[DPLQHU LV ZLWKRXW WKH EHQHILW RI SULRU NQRZOHGJH UHJDUGLQJ WKH LQGLYLGXDOfV EDVHOLQH FRJQLWLYH DELOLWLHV 7KHUHIRUH FHUWDLQ DVVXPSWLRQV DUH PDGH ZLWK UHJDUG WR WKDW SHUVRQfV SUHPRUELG FRJQLWLYH DELOLWLHV DQG D FRPSDULVRQ LV PDGH EHWZHHQ WKHLU VWDWXVSRVW QHXURSV\FKRORJLFDO SHUIRUPDQFH DQG VRPH QRUPDWLYH JURXS WKDW LV GHHPHG DSSURSULDWH ,Q PDQ\ FLUFXPVWDQFHV DQ DUELWUDU\ FXWSRLQW LV GHWHUPLQHG IRU D QHXURSV\FKRORJLFDO PHDVXUH D SHUIRUPDQFH XQGHU ZKLFK RU VLJQLILFDQWO\ GLIIHUHQW IURPf LV FRQVLGHUHG WR LQGLFDWH DEQRUPDOLW\ ,Q WKLV VWXG\ WKH JURXS RI PRUH WKDQ IRRWEDOO SOD\HUV ZKR SDUWLFLSDWHG LQ WKH EDVHOLQH VWXG\ SURYLGHG DQ DSSURSULDWH FRPSDULVRQ JURXS 0RGHO 7KH PRVW EDVLF PRGHO UHIHUUHG WR DV 0RGHO IRU WKH SXUSRVHV RI WKLV VWXG\f IRU GHWHFWLQJ LI D FRQFXVVHG SOD\HUfV SHUIRUPDQFH VXJJHVWHG RQJRLQJ FRQFXVVLYH VHTXHODH ZDV D FRPSDULVRQ DW HDFK SRVWFRQFXVVLYH WHVW LQWHUYDO WR WKH QRUPDWLYH EDVHOLQH SRSXODWLRQ PHDQ VFRUH IRU WKDW PHDVXUH 8VLQJ D OLEHUDO FULWHULRQ D GLIIHUHQFH RI JUHDWHU WKDQ 6' EHORZ WKDW H[SHFWHG QRUPDWLYH PHDQf VFRUH ZRXOG VXJJHVW D SHUIRUPDQFH

PAGE 64

EHORZ H[SHFWDWLRQ SHUKDSV LQGLFDWLQJ RQJRLQJ FRQFXVVLYH HIIHFWV 8VLQJ WKDW DSSURDFK 7DEOH LQGLFDWHV WKH FXWSRLQWV IRU WKH QHXURSV\FKRORJLFDO PHDVXUHV WKDW ZHUH VWXGLHG 7KHVH ZHUH GHULYHG E\ GHWHUPLQLQJ D VFRUH IRU HDFK SDUWLFXODU PHDVXUH ZKLFK ZDV 6' ZRUVH WKDQ WKH PHDQ 8VLQJ WKHVH FULWHULD WKH QXPEHU RI FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZKR ZHUH FODVVLILHG DV VLJQLILFDQWO\ EHORZ WKH FXWSRLQW ZDV GHWHUPLQHG IRU HDFK WHVW VHVVLRQ ZLWKLQ KRXUV RI FRQFXVVLRQ GD\V GD\V DQG GD\V SRVWFRQFXVVLRQf 7DEOH LQGLFDWHV WKH VHQVLWLYLW\ RI HDFK PHDVXUH IRU HDFK WHVW VHVVLRQ 7DEOH &XW 3RLQWV 8VLQJ 6' 'LIIHUHQFH IURP 1RUPDWLYH 3RSXODWLRQ 0HDQ 0RGHO f 0HDVXUH 1RUPDWLYH 0HDQ 6FRUH 6WDQGDUG 'HYLDWLRQ &XW3RLQW +RSNLQV 7RWDO 7UDLOV $ VHFf 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFNZDUG *URRYHG 3HJERDUG 'f *URRYHG 3HJERDUG 1f &2:$7 +RSNLQV 'HOD\ $V 7DEOH LQGLFDWHV ZLWKLQ KRXUV RI FRQFXVVLRQ WKH PRVW VHQVLWLYH PHDVXUH DSSHDUV WR EH WRWDO ZRUGV UHFDOOHG DIWHU WKH GHOD\ SHULRG IRU WKH +RSNLQV 9HUEDO /HDUQLQJ 7HVW +RZHYHU QRQH RI WKH PHDVXUHV DORQH ZDV SDUWLFXODUO\ VHQVLWLYH ([DPLQLQJ WKH EDWWHU\ DV D ZKROH XVLQJ 0RGHO WKH QXPEHU RI FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZKR VFRUHG EHORZ WKH FXWSRLQWV IRU HDFK PHDVXUH ZHUH WRWDOHG 7DEOHV VKRZ WKH FXPXODWLYH QXPEHU IRU HDFK JURXS ZKR ZHUH VLJQLILFDQWO\ GLIIHUHQW WKDQ WKH QRUPDWLYH PHDQ JURXS IRU DOO YDULDEOHV H[DPLQHG

PAGE 65

7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\ 7DEOH GHPRQVWUDWHV WKDW ZLWKLQ KRXUV RI FRQFXVVLRQ XVLQJ WKH FULWHULRQ RI IDOOLQJ EHORZ WKH FXWSRLQWV GHULYHG XVLQJ 0RGHO f IRU RU PRUH YDULDEOHV DFKLHYHG WKH JUHDWHVW 6H 6Sf YDOXH RI 7KDW LV RI WKH FRQFXVVHG LQGLYLGXDOV IHOO EHORZ WKH FXWSRLQW IRU RU PRUH YDULDEOHV ZKLOH RQO\ RI WKH FRQWURO SDUWLFLSDQWV GLG VR DW WKDW WHVW VHVVLRQ +RZHYHU DV KDV EHHQ QRWHG LQ WKH OLWHUDWXUH LQ WKH FDVH RI VSRUWV FRQFXVVLRQ LW LV OLNHO\ EHWWHU WR HUU RQ WKH VLGH RI FDXWLRQ DQG ZKLOH XVLQJ D FULWHULD RI RQO\ RQH YDULDEOH EHORZ VLJQLILFDQFH YDOXH UHVXOWHG LQ D JUHDWHU QXPEHU RI IDOVH SRVLWLYH HUURUV 6S f LW DOVR UHVXOWHG LQ D IDU EHWWHU WUXH SRVLWLYH UDWH 6H f 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV :LWKLQ +RXUV ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f ‘A6HQVLWLYLW\ 6" 6SHFLILFLW\

PAGE 66

$JDLQ 7DEOH GHPRQVWUDWHV WKDW WKH KLJKHVW FRPELQHG VHQVLWLYLW\ DQG VSHFLILFLW\ UDWHV ZHUH DFKLHYHG DW 'D\ XVLQJ D FULWHULD RI RU PRUH YDULDEOHV IDOOLQJ EHORZ WKHLU UHVSHFWLYH FXWSRLQWV 6H 6S f +RZHYHU VHQVLWLYLW\ LV H[WUHPHO\ QHJDWLYHO\ DIIHFWHG E\ WKH PRUH FRQVHUYDWLYH LQFOXVLRQ FULWHULD 7KLV PRUH FRQVHUYDWLYH FULWHULRQ KRZHYHU DOVR UHVXOWHG LQ D VLJQLILFDQWO\ KLJKHU VSHFLILFLW\ UDWH GHPRQVWUDWLQJ WKH QHHG IRU D SUHGHWHUPLQHG WROHUDQFH IRU KLJKHU IDOVHSRVLWLYH UDWHV ZKHQ DWWHPSWLQJ WR GHWHFW RQJRLQJ HIIHFWV RI FRQFXVVLRQ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6! 6SHFLILFLW\ $W GD\ 7DEOH f WKH KLJKHVW FRPELQHG VHQVLWLYLW\ DQG VSHFLILFLW\ VFRUH ZDV DFKLHYHG XVLQJ D FULWHULRQ RI RQH YDULDEOH EHORZ FXWSRLQW f &HUWDLQO\ WKH VHQVLWLYLW\ DQG VSHFLILFLW\ YDOXHV XVLQJ D FRPSDULVRQ RI SHUIRUPDQFH DW WKHVH WLPH LQWHUYDOV WR D QRUPDWLYH SRSXODWLRQ VXJJHVWV WKDW WKH PRGHO LWVHOI KDV VRPH YDOXH ZLWKLQ KRXUV RI FRQFXVVLRQ $OWKRXJK WKH PRGHO LV QRW DQ RSWLPDO RQH LW ZDV DEOH WR GHWHFW D GHJUHH RI SHUIRUPDQFH DEHUUDQFH DIWHU FRQFXVVLRQ FRPSDUHG WR WKH DSSURSULDWH VWDQGDUG VFRUHV WKRXJK VSHFLILFLW\ ZDV FRQYHUVHO\ SRRU

PAGE 67

5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYHV ZHUH FDOFXODWHG IRU WKLV PRGHO XVLQJ $QDO\VH,W VRIWZDUH IRU ([FHO DW HDFK RI WKH WLPH LQWHUYDO DQG DUH VKRZQ LQ )LJXUHV 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6" 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6" 6SHFLILFLW\ )LJXUH VKRZV WKH UHFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYH IRU WKH FODVVLILFDWLRQ UDWH IRU 0RGHO DW KRXUV 7KH GLDJRQDO OLQH UHSUHVHQWV FKDQFH OHYHO DQG E\ GHILQLWLRQ WKH DUHD XQGHUQHDWK LW LV HTXDO WR 7KH DUHD XQGHUQHDWK WKH FXUYH IRU 0RGHO DW KRXUV ZDV b &RQILGHQFH ,QWHUYDO 6( f DQG ZDV IRXQG WR EH VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH S f 8VLQJ JHQHUDOO\ DFFHSWHG

PAGE 68

DFFXUDF\ FODVVLILFDWLRQ WKLV PRGHO DW KRXUV DIWHU FRQFXVVLRQ ZRXOG EH FKDUDFWHUL]HG RQO\ DV IDLU 52& &XUYH IRU 0RGHO DW +RXUV 4! R D D! ‘V B! Z 6SHFLILFLW\ IDOVH SRVLWLYHVf )LJXUH 52& &XUYH IRU 0RGHO DW +RXUV $W 'D\ GHWHFWLRQ RI VLJQLILFDQW VFRUHV RQ WKH WHVW YDULDEOH FRPSDUHG WR WKH EDVHOLQH QRUPDWLYH SRSXODWLRQ ZDV QRW EHWWHU WKDQ FKDQFH XVLQJ 0RGHO )LJXUH f 7KH DUHD XQGHU WKH FXUYH $8&f ZDV FDOFXODWHG DV b &RQILGHQFH ,QWHUYDO 6( f DQG ZDV QRW VWDWLVWLFDOO\ GLIIHUHQW WKDQ WKH UHIHUHQFH GLDJRQDO S f $ VLPLODU SDWWHUQ ZDV REVHUYHG DW 'D\ DV FDQ EH VHHQ LQ )LJXUH 8VLQJ 0RGHO WKH GHWHFWLRQ UDWH ZDV QRW VWDWLVWLFDOO\ GLIIHUHQFH WKDQ FKDQFH 7KH $8& ZDV IRXQG WR EH b &RQILGHQFH ,QWHUYDO 6( f 7KH FDOFXODWHG $8& IRU 0RGHO DW GD\ ZDV b &RQILGHQFH ,QWHUYDO 6( f DQG DJDLQ ZDV QRW VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH S f DW GLIIHUHQWLDWLQJ EHWZHHQ FRQFXVVHG SDUWLFLSDQWV DQG FRQWUROV )LJXUH f

PAGE 69

6HQVLWLYLW\ WUXH SRVLWLYHVf H 6HQVLWLYLW\ WUXH SRVLWLYHVf 52& &XUYH IRU 0RGHO DW 'D\ UH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\

PAGE 70

52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 0RGHO ,Q WKH FLUFXPVWDQFH WKDW RQH KDV EDVHOLQH WHVW PHDVXUHV IRU DQ LQGLYLGXDO FRPSDULVRQ RI WKHLU SHUIRUPDQFH IROORZLQJ QHXURORJLFDO LQVXOW WR WKDW EDVHOLQH SHUIRUPDQFH PD\ EH PRUH DSSURSULDWH LQ GHWHUPLQLQJ FKDQJH LQ FRJQLWLYH VWDWXV GXULQJ VXEVHTXHQW WHVWLQJ VHVVLRQV ,Q WKH VHFRQG EDVLF PRGHO H[DPLQHG LQ WKLV VWXG\ KHUH UHIHUUHG WR DV 0RGHO f SHUIRUPDQFH RI FRQFXVVHG LQGLYLGXDOV DW HDFK WHVWLQJ VHVVLRQ IROORZLQJ FRQFXVVLRQ ZDV FRPSDUHG WR WKHLU EDVHOLQH SHUIRUPDQFH 7KLV ZDV DFKLHYHG E\ GHWHUPLQLQJ VLJQLILFDQFH EDVHG DJDLQ RQ D OLEHUDO FULWHULRQ RI 6' GHULYHG IURP WKH QRUPDWLYH SRSXODWLRQ DV D ZKROH Q f EHORZ WKHLU EDVHOLQH SHUIRUPDQFH ,Q WKLV PDQQHU HDFK VXEMHFW ZDV DIIRUGHG DQ LQGLYLGXDOL]HG FRPSDUDWRU DV LOOXVWUDWHG IRU RQH FRQFXVVHG SDUWLFLSDQW LQ 7DEOH

PAGE 71

3OHDVH QRWH WKDW WKRVH YDULDEOHV IRU ZKLFK WLPH LV WKH UHFRUGHG VFRUH D FXWSRLQW ZDV GHWHUPLQHG E\ WKH DGGLWLRQ RI 6' WR WKH EDVHOLQH SHUIRUPDQFH DQG RUGLQDO YDULDEOHV ZHUH URXQGHG WR WKH QHDUHVW ZKROH QXPEHU 7KHUHIRUH DV FDQ EH VHHQ LQ 7DEOH IRU WKLV SDUWLFLSDQWfV WLPH RQ 7UDLOV $ WR EH FODVVLILHG DV VLJQLILFDQWO\ GLIIHUHQW DW KRXUV WKDQ EDVHOLQH KLV VFRUH ZRXOG KDYH WR LQFUHDVH E\ QHDUO\ VHFRQGV IURP f WR QHDUO\ f WRWDO WLPH 7DEOH ([DPSOH RI &XW3RLQW 'HWHUPLQDWLRQ IRU 6WXG\ 3DUWLFLSDQWV 0HDVXUH %DVHOLQH 3HUIRUPDQFH 6'LIRP QRUPDWLYH SRSf &XW3RLQW +RSNLQV 7RWDO 7UDLOV $ f f f 7UDLOV % f f f 'LJLWV )RU 'LJLWV %DFN 3HJV 'RPLQDQW f f f 3HJV 1RQGRPLQDQW f f &2:$7 +RSNLQV 'HOD\ 7KH VDPH SURFHGXUH ZDV XVHG IRU HDFK FRQFXVVHG DQG FRQWURO SDUWLFLSDQW DQG WKHLU UHWHVW UHVXOWV DW HDFK WHVWLQJ VHVVLRQ ZHUH FRPSDUHG WR WKHLU DSSURSULDWH FXWSRLQW VFRUH LQ RUGHU WR GHWHUPLQH LI WKHUH ZDV D VLJQLILFDQW FKDQJH IURP WKHLU EDVHOLQH SHUIRUPDQFH 7KH VHQVLWLYLWLHV IRU LQGLYLGXDO YDULDEOHV ZHUH FDOFXODWHG IRU HDFK WHVWLQJ VHVVLRQ DQG DUH SUHVHQWHG LQ 7DEOH $V ZDV QRWHG LQ 0RGHO WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDULDEOHV ZDV UHODWLYHO\ SRRU DW GHWHFWLQJ VLJQLILFDQW FKDQJH 7KH PRVW VHQVLWLYH LQGLYLGXDO YDULDEOHV DW KRXUV SRVWFRQFXVVLRQ ZHUH +RSNLQV 7RWDO VFRUH DQG *URRYHG 3HJERDUG QRQGRPLQDQW KDQG FRPSOHWLRQ WLPH $V ZRXOG EH H[SHFWHG XVLQJ 0RGHO WKH LQGLYLGXDO YDULDEOHV ZHUH PRVW HIIHFWLYH LQ GHWHFWLQJ VLJQLILFDQW FKDQJH UHODWLYH WR EDVHOLQH SHUIRUPDQFH ZLWKLQ KRXUV RI FRQFXVVLRQ ZLWK D WUHQG WRZDUG GHFUHDVLQJ VHQVLWLYLW\ DW HDFK

PAGE 72

VXFFHVVLYH WHVWLQJ VHVVLRQ 7KH RQH H[FHSWLRQ ZDV WKH +RSNLQV 'HOD\ VFRUH ZKLFK UHPDLQHG UHODWLYHO\ VWDEOH RYHU WLPH 7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG > 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\ 7KH EDWWHU\ DV D ZKROH ZDV DJDLQ H[DPLQHG WR GHWHUPLQH WKH QXPEHU RI FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZKR VFRUHG EHORZ WKHLU LQGLYLGXDO FXWSRLQWV DFURVV YDULDEOHV XVLQJ 0RGHO 6HQVLWLYLW\ DQG VSHFLILFLW\ FKDUDFWHULVWLFV RI WKH 0RGHO IRU HDFK WHVW VHVVLRQ DUH SUHVHQWHG LQ 7DEOHV $V 7DEOH GHPRQVWUDWHV WKH JUHDWHVW GHJUHH RI FODVVLILFDWLRQ DFFXUDF\ RFFXUUHG DW KRXUV IRU 0RGHO ZKHQ D FULWHULRQ RI D VLJQLILFDQW UHGXFWLRQ LQ VFRUH RQ RU PRUH YDULDEOHV ZDV XVHG 6SHFLILFLW\ JUHDWO\ LQFUHDVHG ZKHQ WKH FULWHULD ZDV WZR RU PRUH VLJQLILFDQW YDULDEOHV WKRXJK WKLV DOVR UHVXOWHG LQ WKH ODUJHVW VLQJOH GURS LQ VHQVLWLYLW\ 8VLQJ WKH VDPH FULWHULRQ RQ 'D\V WKURXJK D SUHGLFWDEOH GURS LQ VHQVLWLYLW\ ZDV REVHUYHG RYHU WLPH ZKLOH VSHFLILFLW\ UHPDLQHG UHODWLYHO\ FRQVWDQW 1RWDEO\ WKLV PRGHO VDZ LWV ORZHVW VHQVLWLYLW\ XVLQJ WKH RQHYDULDEOH FULWHULRQ DW WKH 'D\ WHVW VHVVLRQ 7DEOH f HYHQ UHODWLYH WR WKH 'D\ WHVW UHVXOWV 7KLV SDWWHUQ ZDV FRQVLVWHQW LQ PRVW RI WKH PRGHOV H[DPLQHG DV SDUW RI WKLV VWXG\

PAGE 73

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV :LWKLQ +RXUV ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ S 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ AA6SHFLILFLW\ $V ZLWK WKH XVH RI WKLV PRGHO LQ WKH SUHYLRXV WHVWLQJ VHVVLRQV WKH JUHDWHVW GHJUHH RI VHQVLWLYLW\ ZDV DFKLHYHG ZKHQ D FULWHULD RI RQH VLJQLILFDQW YDULDEOH ZDV XVHG +RZHYHU WKLV DOVR UHVXOWHG LQ IDOVHO\ FODVVLI\LQJ JUHDWHU WKDQ KDOI RI WKH FRQWURO SDUWLFLSDQWV DV FRQFXVVHG 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYHV IRU 0RGHO DUH SUHVHQWHG LQ )LJXUHV WKURXJK

PAGE 74

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S L6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ S 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6S 6SHFLILFLW\ 'DWD IRU WKH KRXU WHVW VHVVLRQ UHVXOWHG LQ D 52& FXUYH VXJJHVWLQJ WKDW WKLV PRGHO KDG fIDLUf FODVVLILFDWLRQ DFFXUDF\ $8& ZDV FDOFXODWHG WR EH 6( b &RQILGHQFH ,QWHUYDO f DQG ZDV VLJQLILFDQWO\ JUHDWHU WKDQ FKDQJH ZLWK D VLJQLILFDQFH YDOXH RI S $W WKH 'D\ WHVW VHVVLRQ )LJXUH f 0RGHO ZDV IRXQG QRW WR EH VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH DW FODVVLI\LQJ WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV $8& 6( Sf

PAGE 75

52& &XUYH IRU 0RGHO DW +RXUV )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW +RXU 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 76

52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ 1R GLVFULPLQDWLRQ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 6LPLODUO\ 0RGHO ZDV IRXQG QRW WR EH VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH DW WKH 'D\ DQG 'D\ WHVW VHVVLRQV DV VKRZQ LQ )LJXUHV DQG 7KH $8& IRU 0RGHO DW 'D\ ZDV IRXQG WR EH 6( b &RQILGHQFH ,QWHUYDO

PAGE 77

Sf ZKLOH WKH $8& IRU 0RGHO DW 'D\ ZDV 6( b &RQILGHQFH ,QWHUYDO Sf $V WKH 52& DQDO\VLV VXJJHVWHG LQ WKH UHSHDWHG WHVWLQJ RI WKH FRQFXVVHG IRRWEDOO SOD\HUV FRPSDULQJ WKHLU UHWHVW SHUIRUPDQFH WR HLWKHU DQ DSSURSULDWH QRUPDWLYH VDPSOH 0RGHO f RU WR WKHLU RZQ EDVHOLQH SHUIRUPDQFH XVLQJ D OLEHUDO FULWHULD RI 6' UHGXFHG SHUIRUPDQFH 0RGHO f \LHOGHG UDWKHU SRRU VHQVLWLYLW\ IRU WKH LQGLYLGXDO YDULDEOHV LQFOXGHG LQ WKLV WHVW EDWWHU\ )XUWKHU H[DPLQLQJ WKH YDULDEOHV FROOHFWLYHO\ ERWK PRGHOV ZHUH RQO\ IDLU LQ DFFXUDWHO\ GLVFULPLQDWLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV HYHQ ZLWKLQ KRXUV RI FRQFXVVLRQ D WLPH SHULRG LQ ZKLFK WKH FRJQLWLYH VHTXHODH RI FRQFXVVLRQ DUH JHQHUDOO\ PRVW HYLGHQW 0RGHO YV 0RGHO 52& DW 7HVW 6HVVLRQ GLII Sf )LJXUH &RPSDULVRQ RI 0RGHO DQG 0RGHO DW WKH +RXU 7HVW 6HVVLRQ 1HLWKHU ZDV EHWWHU WKDQ FKDQFH DW GLVFULPLQDWLQJ EHWZHHQ WKH WZR JURXSV DW WKH 'D\ 'D\ RU 'D\ WHVW VHVVLRQV $ FRPSDULVRQ RI 0RGHO DQG 0RGHO IRU WKH 7HVW

PAGE 78

6HVVLRQ VXJJHVWHG WKDW 0RGHO ZLWK DQ $8& RI ZDV VOLJKWO\ PRUH VHQVLWLYH WKDQ 0RGHO ZKLFK KDG DQ $8& RI +RZHYHU WKHUH ZDV QRW D VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFH EHWZHHQ WKH WZR S f DV LOOXVWUDWHG LQ )LJXUH 5HOLDEOH &KDQJH ,QGH[ 0RGHOV 0RGHO 7KUHH YDULDQWV RI WKH 5HOLDEOH &KDQJH ,QGH[ 5&,f ZHUH VXEVHTXHQWO\ DSSOLHG WR WKH GDWD VHW 7KH ILUVW D EDVLF 5&, PRGHO UHIHUUHG WR DV 0RGHO IRU WKH SXUSRVHV RI WKLV VWXG\f HVVHQWLDOO\ H[DPLQHG ZKHWKHU WKH WHVWUHWHVW SHUIRUPDQFH GLIIHUHQFH IRU HDFK LQGLYLGXDO ZDV ODUJH UHODWLYH WR WKH HUURU YDULDQFH RI WKH YDULDEOH XQGHU H[DPLQDWLRQ SURYLGLQJ LQGLFDWLRQ RI D FKDQJH IURP EDVHOLQH SHUIRUPDQFH 7KH IROORZLQJ IRUPXOD ZDV DSSOLHG IRU HDFK LQGLYLGXDOfV SHUIRUPDQFH RQ HDFK YDULDEOH ,Q WKLV HTXDWLRQ [ LV WKH LQGLYLGXDOfV VFRUH DW WKH %DVHOLQH WHVWLQJ VHVVLRQ [ LV WKH LQGLYLGXDOfV VFRUH DW WKH VHFRQG WHVW VHVVLRQ LQ WKLV FDVH ZLWKLQ KRXUV RI FRQFXVVLRQ GD\ GD\ DQG GD\ f DQG VGLII WKH VWDQGDUG HUURU RI WKH GLIIHUHQFH ZDV FDOFXODWHG XVLQJ V( WKH VWDQGDUG HUURU RI WKH PHDVXUH ZLWK WKH HTXDWLRQ V( ZDV GHULYHG IURP WKH FRUUHODWLRQ RI WKH WHVWUHWHVW VFRUHV RI WKH FRQWURO JURXS IRU HDFK YDULDEOH UXf DQG WKH VWDQGDUG GHYLDWLRQ Vf RI WKH FRQWURO JURXS DW EDVHOLQH XVLQJ WKH HTXDWLRQ

PAGE 79

$Q LQGH[ RXWVLGH RI ZKLFK D UHWHVW VFRUH ZDV FRQVLGHUHG VLJQLILFDQW ZDV FDOFXODWHG E\ PXOWLSO\LQJ WKH VGLII E\ D FXWRII OHYHO RI D ZLGHO\ DFFHSWHG VWDWLVWLFDO VLJQLILFDQFH OHYHO 7DEOHV WKURXJK LQGLFDWH WKH GDWD XVHG WR FDOFXODWH WKHVH PHDVXUHV DV ZHOO DV WKH 5&, UDQJHV XVHG DV FRPSDULWRUV IRU HDFK YDULDEOH DW HDFK WHVWLQJ VHVVLRQ 7DEOH 'DWD 8VHG LQ WKH &DOFXODWLRQ RI 5HOLDEOH &KDQJH ,QGLFHV DW +RXUV 0HDVXUH UQ & E F 6( G 6GLII 5&,f +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ D UO EDVHG RQ WKH FRUUHODWLRQ IRU FRQWURO SDUWLFLSDQWV EHWZHHQ EDVHOLQH DQG KRXU SHUIRUPDQFH OLVWZLVH Q E VG LV WKH VWDQGDUG GHYLDWLRQ IRU WKH FRQWURO JURXS DW EDVHOLQH F VWDQGDUG HUURU RI WKH PHDVXUH G VWDQGDUG HUURU RI WKH GLIIHUHQFH 7DEOH 'DWD 8VHG LQ WKH &DOFXODWLRQ RI 5HOLDEOH &KDQJH ,QGLFHV DW 'D\ 0HDVXUH UQ VG & 6( G 6GLII 5&,f +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ D UO EDVHG RQ WKH FRUUHODWLRQ IRU FRQWURO SDUWLFLSDQWV EHWZHHQ EDVHOLQH DQG 'D\ SHUIRUPDQFH OLVWZLVH Q E VG LV WKH VWDQGDUG GHYLDWLRQ IRU WKH FRQWURO JURXS DW EDVHOLQH F VWDQGDUG HUURU RI WKH PHDVXUH G VWDQGDUG HUURU RI WKH GLIIHUHQFH

PAGE 80

7DEOH 'DWD 8VHG LQ WKH &DOFXODWLRQ RI 5HOLDEOH &KDQJH ,QGLFHV DW 'D\ 0HDVXUH UD E 6G & 6( G 6GLII 5&,f +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ D U EDVHG RQ WKH FRUUHODWLRQ IRU FRQWURO SDUWLFLSDQWV EHWZHHQ EDVHOLQH DQG 'D\ SHUIRUPDQFH OLVWZLVH Q E VG LV WKH VWDQGDUG GHYLDWLRQ IRU WKH FRQWURO JURXS DW EDVHOLQH F VWDQGDUG HUURU RI WKH PHDVXUH G VWDQGDUG HUURU RI WKH GLIIHUHQFH 7DEOH 'DWD 8VHG LQ WKH &DOFXODWLRQ RI 5HOLDEOH &KDQJH ,QGLFHV DW 'D\ 0HDVXUH 9 E rG F 6( G 6GLII 5&,f +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ D UO EDVHG RQ WKH FRUUHODWLRQ IRU FRQWURO SDUWLFLSDQWV EHWZHHQ EDVHOLQH DQG 'D\ SHUIRUPDQFH E VG LV WKH VWDQGDUG GHYLDWLRQ IRU WKH FRQWURO JURXS DW EDVHOLQH F VWDQGDUG HUURU RI WKH PHDVXUH G VWDQGDUG HUURU RI WKH GLIIHUHQFH 3OHDVH QRWH WKDW D FKDQJH ZDV FRQVLGHUHG WR LQGLFDWH D VLJQLILFDQW GLIIHUHQFH EHWZHHQ D SHUIRUPDQFH DW D SDUWLFXODU WHVW VHVVLRQ DQG WKH DSSURSULDWH EDVHOLQH SHUIRUPDQFH RQO\ LI WKH GLIIHUHQFH H[FHHGHG WKH 5&, FXWRII SRLQW DVVRFLDWHG ZLWK D VLJQLILFDQW GHFUHDVH LQ SHUIRUPDQFH DW WKH VHFRQG WHVW VHVVLRQ 7KDW LV LW ZDV FRQVLGHUHG VLJQLILFDQW LI WKH VHFRQG WHVW SHUIRUPDQFH RQ D SDUWLFXODU PHDVXUH ZDV VLJQLILFDQWO\ EHORZ WKH EDVHOLQH SHUIRUPDQFH EDVHG RQ WKH 5&,

PAGE 81

7DEOH GHPRQVWUDWHV WKH XVH RI WKLV PHWKRG RQ RQH FRQFXVVHG SDUWLFLSDQW IRU WZR WHVW YDULDEOHV $V LV LOOXVWUDWHG IRU WKH YDULDEOH f+RSNLQV 7RWDO ZRUGV UHFDOOHGf D UHGXFWLRQ RI JUHDWHU WKDQ UHFDOOHG ZRUGV ZDV FDOFXODWHG WR EH D VLJQLILFDQW FKDQJH UHODWLYH WR WKH HUURU YDULDQFH RI WKH WHVW DV LQGLFDWHG E\ WKH VGLII 7KLV SDUWLFLSDQWfV UHGXFWLRQ RI ZRUGV DW KRXUV UHODWLYH WR KLV EDVHOLQH SHUIRUPDQFH UHVXOWHG LQ D ]VFRUH RI ZKLFK XVLQJ D FXWSRLQW RI ] ZDV IRXQG WR EH D KLJKO\ VLJQLILFDQW FKDQJH LQ WKH QHJDWLYH GLUHFWLRQ 7KDW LV KLV SHUIRUPDQFH RQ +RSNLQV 7RWDO ZDV VLJQLILFDQWO\ EHORZ KLV SHUIRUPDQFH RQ WKH VDPH PHDVXUH DW EDVHOLQH )RU WKH YDULDEOH 7UDLOV $ ZKLFK LV PHDVXUHG LQ VHFRQGV KLV LQFUHDVH RI f LQ WRWDO FRPSOHWLRQ WLPH IURP EDVHOLQH WR WKH KRXU WHVW VHVVLRQ XOWLPDWHO\ UHVXOWHG LQ D 7UDLOV $ WLPH DW KRXUV RI f JUHDWO\ H[FHHGLQJ WKH PD[LPXP YDOXH LQ WKH FDOFXODWHG 5&, RI f 7KLV UHVXOWHG LQ D ]VFRUH RI 7DEOH ([DPSOH RI 5&, 6LJQLILFDQFH 'HWHUPLQDWLRQ IRU 2QH &RQFXVVHG 3DUWLFLSDQW DW WKH +RXU 7HVW 6HVVLRQ 0HDVXUH 2EVHUYHG &KDQJH )URP %DVHOLQH 5&, ,QWHUYDO ]VFRUH 7R +RXUV +RSNLQV 7RWDO ZRUGV f 7UDLOV $ f f f 7DEOH SUHVHQWV WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO PHDVXUHV DW HDFK WHVW VHVVLRQ XVLQJ 0RGHO WKH EDVLF 5&, DSSURDFK $JDLQ WKH GDWD LQGLFDWHG WKDW WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDULDEOHV ZHUH UHODWLYHO\ SRRU HYHQ ZLWKLQ KRXUV ZKLFK ZDV DOVR VHHQ LQ 0RGHOV DQG 7KH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDULDEOHV XVLQJ 0RGHO DOVR

PAGE 82

JHQHUDOO\ UHSUHVHQW UHODWLYHO\ ORZHU YDOXHV WKDQ LQ 0RGHO ZKLFK LV QRW VXUSULVLQJ VLQFH WKH FULWHULD XVHG IRU 0RGHOV DQG ZHUH PRUH OLEHUDO 6'f 7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\ 7KH EDWWHU\ DV D ZKROH ZDV DJDLQ H[DPLQHG WR GHWHUPLQH WKH QXPEHU RI FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZKR VFRUHG EHORZ WKHLU LQGLYLGXDO FXWSRLQWV DFURVV PXOWLSOH YDULDEOHV XVLQJ 0RGHO 6HQVLWLYLW\ DQG VSHFLILFLW\ FKDUDFWHULVWLFV RI 0RGHO IRU HDFK WHVW VHVVLRQ DUH SUHVHQWHG LQ 7DEOHV 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW +RXUV XVLQJ 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f H 6HQVLWLYLW\ S 6SHFLILFLW\ $V ZLWK WKH ILUVW WZR PRGHOV 0RGHO ZDV PRVW HIIHFWLYH LQ GHWHFWLQJ D FRQFXVVHG SDUWLFLSDQW ZKHQ D FULWHULRQ RI RQH YDULDEOH EHORZ WKH UHVSHFWLYH 5&, FXWRII ZDV XVHG 1RWDEO\ WKH 5&, PRGHO UHVXOWHG LQ D PXFK JUHDWHU VSHFLILFLW\ YDOXH ZKHQ XVLQJ WKLV

PAGE 83

FULWHULRQ WKDQ HLWKHU 0RGHO RU 0RGHO VXJJHVWLQJ WKDW WKH EDVLF 5&, PRGHO PD\ EH PRUH HIIHFWLYH LQ VHSDUDWLQJ WKH JURXSV 7KLV LV QRW DOWRJHWKHU VXUSULVLQJ VLQFH WKH FXWRII FULWHULD DUH EDVHG RQ WKH SHUIRUPDQFH RI WKH FRQWURO SRSXODWLRQ $W 'D\ 7DEOH f 0RGHO VDZ D VLJQLILFDQW GHFOLQH LQ VHQVLWLYLW\ HYHQ ZKHQ XVLQJ WKH PLQLPDO FULWHULD RI RQH YDULDEOH IDOOLQJ RXWVLGH WKH 5&, 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f $V ZDV VHHQ LQ WKH SUHYLRXV WZR PRGHOV GHWHFWLRQ RI FRQFXVVHG SOD\HUV VDZ LWV PRVW H[WHQVLYH GURS DW WKH 'D\ WHVW VHVVLRQ 7DEOH f EXW VDZ UHVXUJHQFH LQ VHQVLWLYLW\ DW WKH 'D\ WHVW VHVVLRQ 7DEOH f

PAGE 84

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6" 6SHFLILFLW\ 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYHV IRU 0RGHO DW HDFK WHVW VHVVLRQ DUH VKRZQ LQ )LJXUHV WKURXJK $V )LJXUH LQGLFDWHV 0RGHO ZDV UHODWLYHO\ JRRG DW GLVFULPLQDWLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH KRXU WHVW VHVVLRQ $UHD XQGHU WKH FXUYH ZDV IRXQG WR EH 6( b &RQILGHQFH ,QWHUYDO f ZKLFK ZDV VLJQLILFDQWO\ EHWWHU WKDQ FKDQFH OHYHO S2f :KLOH WKH XVH RI 0RGHO ZDV JHQHUDOO\ JRRG DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZLWKLQ KRXUV RI FRQFXVVLRQ )LJXUHV LQGLFDWH WKDW WKH PRGHO SHUIRUPHG QR EHWWHU WKDQ FKDQFH DW 'D\ 'D\ DQG 'D\ 7KH $8& IRU 0RGHO DW 'D\ ZDV 6( b &RQILGHQFH ,QWHUYDO Sf $8& IRU 0RGHO DW 'D\ ZDV 6( b &RQILGHQFH ,QWHUYDO Sf DQG WKH $8& IRU 0RGHO DW 'D\ ZDV 6( b &RQILGHQFH ,QWHUYDO Sf

PAGE 85

52& &XUYH IRU 0RGHO DW +RXUV )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW +RXU 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 86

52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ 1R GLVFULPLQDWLRQ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 0RGHO 7KH VHFRQG 5&, YDULDQW DQDO\]HG DV SDUW RI WKH FXUUHQW VWXG\ UHIHUUHG WR DV 0RGHO f XWLOL]HG WKH VDPH GDWD LQGLFDWRUV DV ZHUH XVHG LQ WKH EDVLF 5&, PRGHO 0RGHO f EXW

PAGE 87

DOVR LQFOXGHG D FRUUHFWLRQ IRU REVHUYHG SUDFWLFH HIIHFWV LQ WKH FRQWURO SRSXODWLRQ IRU LQGLYLGXDO YDULDEOHV DW HDFK WHVWLQJ VHVVLRQ &RQVLVWHQW ZLWK WKH OLWHUDWXUH SUDFWLFH HIIHFWV ZHUH FDOFXODWHG E\ ILQGLQJ WKH PHDQ FKDQJH LQ VFRUH RYHU D WHVWUHWHVW SHULRG IRU WKH FRPSDULVRQ SRSXODWLRQ )RU WKH FXUUHQW VWXG\ WKLV LQYROYHG FDOFXODWLQJ WKH PHDQ FKDQJH IRU WKH FRQWURO SRSXODWLRQ IRU HDFK YDULDEOH IURP EDVHOLQH WR KRXU WHVW VHVVLRQ EDVHOLQH WR 'D\ WHVW VHVVLRQ EDVHOLQH WR 'D\ WHVW VHVVLRQ DQG EDVHOLQH WR 'D\ WHVW VHVVLRQ $JDLQ LQGLYLGXDO VXEMHFWV ZHUH RPLWWHG OLVWZLVH 7DEOH LQGLFDWHV WKH PHDQ FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ 7DEOH 0HDQ &KDQJH 6FRUHV IRU ,QGLYLGXDO 9DULDEOHV IRU &RQWURO 3DUWLFLSDQWV 0HDVXUH +RXUQ f 'D\ Q OOf 'D\Q f 'D\Q f +RSNLQV 7RWDO r r 7UDLOV $ f f f f 7UDLOV % fr f f f 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW f f f f 3HJV 1RQGRPLQDQW f f fr f &2:$7 +RSNLQV 'HOD\ r r r fLQGLFDWHV QR SUDFWLFH HIIHFW GHWHFWHG ,Q RUGHU WR FRUUHFW IRU SUDFWLFH HIIHFW WKH PHDQ FKDQJH VFRUH IRU D YDULDEOH ZDV VXEWUDFWHG IURP HDFK SDUWLFLSDQWV VFRUH RQ WKDW YDULDEOH DW WKH UHWHVW VHVVLRQ 7KH UHVXOWLQJ QXPEHU ZDV WKHQ HQWHUHG LQWR WKH 5&, HTXDWLRQ DV RXWOLQHG DERYH DQG FDOFXODWHG DV EHIRUH ZLWK WKH EDVLF 5&, PHWKRG LQ 0RGHO 7KHUHIRUH WKH HTXDWLRQ IRU [ GQf[[ VGLII WKH 5&, EHFDPH

PAGE 88

ZKHUH [ ZDV WKH SDUWLFLSDQWfV VFRUH RQ D YDULDEOH DW EDVHOLQH [ ZDV WKHLU VFRUH DW WKH UHWHVW VHVVLRQ DQG G Q ZDV WKH DYHUDJH FKDQJH VFRUH IRU WKH FRQWURO JURXS RQ WKH YDULDEOH RYHU WKDW WHVWUHWHVW LQWHUYDO 7KH VGLII ZDV FDOFXODWHG DV LQ WKH EDVLF 5&, PRGHO XVLQJ WKH FRUUHODWLRQV DQG VWDQGDUG GHYLDWLRQV IRU WKH FRQWURO JURXS DV GHVFULEHG DERYH 3OHDVH QRWH WKDW IRU WKRVH YDULDEOHV WKDW ZHUH IRXQG RQ DYHUDJH QRW WR KDYH SUDFWLFH HIIHFWV IRU WKH FRQWURO JURXS DW WKH UHWHVW VHVVLRQ QR FKDQJHV ZHUH PDGH LQ WKH VWXG\ SDUWLFLSDQWVf UHWHVW VFRUH LQ WKH 5&, PRGHO 7KLV RFFXUUHG ILYH WLPHV PRVW IUHTXHQWO\ IRU WKH +RSNLQV 9HUEDO /HDUQLQJ 7HVW 7RWDO DQG 'HOD\ VFRUHV ZKLFK ZDV QRW XQH[SHFWHG JLYHQ WKDW WKHVH WHVWV XVHG DOWHUQDWH IRUPV RYHU WHVW VHVVLRQV $JDLQ IRU WKH SXUSRVHV RI WKLV VWXG\ VFRUHV ZHUH FODVVLILHG DV VLJQLILFDQW RQO\ LI WKH\ H[FHHGHG WKHLU 5&, FXWRII LQ WKH QHJDWLYH RU GHFOLQHG SHUIRUPDQFH OHYHO 7DEOH SUHVHQWV WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO PHDVXUHV DW HDFK WHVW VHVVLRQ XVLQJ 0RGHO WKH 5&, DSSURDFK ZLWK FRUUHFWLRQ IRU SUDFWLFH HIIHFWV 2QFH DJDLQ LW LV DSSDUHQW WKDW WKH LQGLYLGXDO YDULDEOHV GHVSLWH FRUUHFWLRQ IRU SUDFWLFH HIIHFW KDYH UHODWLYHO\ ORZ VHQVLWLYLWLHV 7KH SUDFWLFH FRUUHFWLRQ GRHV DSSHDU WR KDYH UHVXOWHG LQ D VOLJKW LPSURYHPHQW LQ VHQVLWLYLW\ IRU HDFK UHWHVW VHVVLRQ VXJJHVWLQJ WKH GHWHFWLRQ RI RQH RU WZR PRUH VLJQLILFDQWO\ GHFUHDVHG VFRUHV IRU HDFK ,W LV QRWDEOH WKDW ERWK +RSNLQV 'HOD\ DQG *URRYHG 3HJERDUG QRQGRPLQDQW KDQG SHUIRUPDQFH KDYH UHODWLYHO\ ODUJHU VHQVLWLYLWLHV ZKHQ FRPSDUHG WR WKH RWKHU YDULDEOHV DW GD\ VXJJHVWLQJ VRPH OLQJHULQJ VKRUWWHUP PHPRU\ DQG PRWRU VSHHG GHFOLQH LQ WKH FRQFXVVHG SRSXODWLRQ 7DEOHV WKURXJK LQGLFDWH WKH VHQVLWLYLW\ DQG VSHFLILFLW\ RI WKH EDWWHU\ RI YDULDEOHV DV D ZKROH IRU 0RGHO DW HDFK WHVW VHVVLRQ

PAGE 89

7DEOH LQGLFDWHV D UHODWLYHO\ KLJK VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQ ZDV DFKLHYHG IRU WKH 5&, PRGHO ZLWK FRUUHFWLRQ IRU SUDFWLFH DW WKH +RXU WHVW VHVVLRQ XVLQJ D FULWHULRQ RI VLJQLILFDQW YDULDEOH 6L[WHHQ RI WKH FRQFXVVHG SDUWLFLSDQWV DW WKLV WHVW VHVVLRQ IHOO EHORZ WKH FXWRII IRU RU PRUH RI WKH YDULDEOHV XQGHU H[DPLQDWLRQ ZKLOH RQO\ FRQWURO SDUWLFLSDQWV IHOO EHORZ WKH FXWRII IRU RU PRUH YDULDEOHV %\ WKH 'D\ WHVW VHVVLRQ 7DEOH f WKH 5&, PRGHO ZLWK SUDFWLFH FRUUHFWLRQ FRQWLQXHG WR KDYH IDLUO\ KLJK FODVVLILFDWLRQ VXFFHVV XVLQJ D FULWHULRQ RI VLJQLILFDQW YDULDEOH DFKLHYLQJ D 6H 6S YDOXH RI 7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW +RXUV ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f H 6HQVLWLYLW\ 6! 6SHFLILFLW\

PAGE 90

$W WKH 'D\ WHVW VHVVLRQ 7DEOH f WKH DSSOLFDWLRQ RI 0RGHO UHVXOWHG LQ DQ DFFXUDF\ RI b ZKHQ XVLQJ D FULWHULRQ RI VLJQLILFDQFH RQ RQH YDULDEOH ZKLOH SURGXFLQJ IDOVH SRVLWLYH LGHQWLILFDWLRQV $OWKRXJK VOLJKWO\ EHWWHU WKDQ WKH FRUUHVSRQGLQJ DFFXUDF\ XVLQJ 0RGHO WKH DELOLW\ WR GLVWLQJXLVK EHWZHHQ WKH JURXSV ZDV UHODWLYHO\ SRRU +RZHYHU XQOLNH WKH SUHYLRXV PRGHOV 0RGHO DFKLHYHG VRPH VXFFHVV LQ GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV E\ 'D\ DV FDQ EH VHHQ LQ 7DEOH 8VLQJ D FULWHULRQ RI VLJQLILFDQFH RQ RQH YDULDEOH 0RGHO DFKLHYHG D VHQVLWLYLW\ OHYHO RI DOWKRXJK LW DOVR SURGXFHG D KLJK QXPEHU RI IDOVH SRVLWLYH LGHQWLILFDWLRQV 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6" 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f

PAGE 91

8VLQJ D FULWHULRQ RI WZR VLJQLILFDQW YDULDEOHV LW DFKLHYHG LWV KLJKHVW VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQ EXW VDZ D VXEVWDQWLDO GHFOLQH LQ VHQVLWLYLW\ :KLOH FXUVRU\ H[DPLQDWLRQ RI WKH VHQVLWLYLW\VSHFLILFLW\ YDOXHV VXJJHVWHG VRPH FOLQLFDO XWLOLW\ LQ LGHQWLI\LQJ OLQJHULQJ FRJQLWLYH FRPSODLQWV DW WKLV PRVW GLVWDO WHVW VHVVLRQ IXUWKHU DQDO\VLV RI VWDWLVWLFDO VLJQLILFDQFH XVLQJ 52& FXUYHV LQGLFDWHG RWKHUZLVH 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ S 6SHFLILFLW\ 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF FXUYHV IRU 0RGHO DW HDFK WHVWLQJ VHVVLRQ DUH SUHVHQWHG LQ )LJXUHV WKURXJK $V ZLWK WKH EDVLF 5&, PRGHO WKH 5&, ZLWK FRUUHFWLRQ IRU SUDFWLFH UHVXOWHG LQ D PRGHO ZLWK D UHODWLYHO\ JRRG DELOLW\ WR GLVWLQJXLVK EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV ZLWKLQ KRXUV RI FRQFXVVLRQ 7KH $8& IRU WKLV PRGHO ZDV 6( b &RQILGHQFH ,QWHUYDO f DQG ZDV VLJQLILFDQWO\ EHWWHU WKDQ FKDQFH ZLWK S )LJ f $V VHHQ LQ )LJXUH 0RGHO UHVXOWHG LQ VOLJKWO\ EHWWHU FODVVLILFDWLRQ UDWH DW WKH 'D\ WHVW VHVVLRQ DOWKRXJK WKH PRGHO ZDV DJDLQ QRW VWDWLVWLFDOO\ GLIIHUHQFH WKDQ FKDQFH OHYHO ZLWK DQ $8& RI 6( b &RQILGHQFH ,QWHUYDO S f $W 'D\ 0RGHO ZDV FOHDUO\ QRW EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO

PAGE 92

SDUWLFLSDQWV )LJ f $8& ZDV FDOFXODWHG DV S 6( b &RQILGHQFH ,QWHUYDO f 52& &XUYH IRU 0RGHO DW +RXUV )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW +RXU 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 93

52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ $V VHHQ LQ )LJXUH WKH DSSOLFDWLRQ RI 0RGHO DW 'D\ DSSHDUV WR KDYH EHHQ VRPHZKDW PRUH VXFFHVVIXO WKDQ WKH SUHFHGLQJ PRGHOV DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV 7KLV ZDV VXJJHVWHG E\ WKH VHQVLWLYLW\VSHFLILFLW\ YDOXHV LQ 7DEOH

PAGE 94

7KH $8& IRU WKLV PRGHO DW 'D\ ZDV 6( b &RQILGHQFH ,QWHUYDO f KRZHYHU WKLV ZDV DJDLQ QRW VWDWLVWLFDOO\ GLIIHUHQW WKDQ WKH UHIHUHQFH GLDJRQDO S f $ FRPSDULVRQ EHWZHHQ WKH EDVLF 5&, PRGHO DQG WKH 5&, ZLWK FRUUHFWLRQ IRU SUDFWLFH DSSOLHG WR WKH GDWD DW WKH +RXU WHVW VHVVLRQ LV VKRZQ LQ )LJXUH $OWKRXJK WKH $8& IRU 0RGHO LV VOLJKWO\ ODUJHU WKDQ IRU 0RGHO DW WKH +RXU WHVW VHVVLRQ YV f WKH GLIIHUHQFH EHWZHHQ WKH WZR ZDV QRW VWDWLVWLFDOO\ VLJQLILFDQW S Of &RPSDULVRQ RI 5&, DQG 5&, ZLWK &RUUHFWLRQ DW +RXUV )LJXUH &RPSDULVRQ RI 52& &XUYHV IRU WKH 5&, DQG 5&, ZLWK &RUUHFWLRQ IRU 3UDFWLFH 0RGHO 7KH WKLUG YDULDQW RI WKH 5&, PRGHO UHIHUUHG WR DV 0RGHO f DSSURDFKHG WKH WDVN RI GHWHUPLQLQJ ZKHWKHU WKHUH ZDV D UHDO GLIIHUHQFH EHWZHHQ EDVHOLQH DQG UHWHVW SHUIRUPDQFH E\ DEDQGRQLQJ VRPH DVVXPSWLRQV XVHG LQ WKH SUHYLRXV WZR 5&, PRGHOV 7KRVH DVVXPSWLRQV LQFOXGHG WKDW f WKH FRQWURO SDUWLFLSDQWVf WUXH VFRUHV IRU D SDUWLFXODU

PAGE 95

YDULDEOH DW WHVW DQG UHWHVW VHVVLRQV ZHUH HTXDO DQG f WKDW WKH HUURU YDULDQFHV IRU D SDUWLFXODU WHVW YDULDEOH DW WHVW DQG UHWHVW ZHUH DOVR HTXDO 8VLQJ WKH HTXDWLRQ .[ fU[f V-n669Z ZKHUH \ ;f RU LV WKH PHDQ FKDQJH VFRUH IRU D SDUWLFXODU YDULDEOH IRU WKH FRQWURO 9 f f f 6[ 6\ 6[6\U[\ RU 6' LV WKH VWDQGDUG GHYLDWLRQV RI WKH FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ WKH SDUWLFLSDQWVf FKDQJH VFRUHV IURP EDVHOLQH WR HDFK VXFFHVVLYH WHVW VHVVLRQ ZHUH H[DPLQHG WR GHWHUPLQH LI WKH\ ZHUH ODUJH UHODWLYH WR WKH GLVWULEXWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO JURXS IRU HDFK RI WKRVH LQWHUYDOV 7DEOH VKRZV WKH DYHUDJH FKDQJH VFRUHV IRU HDFK YDULDEOH DW HDFK WHVWLQJ VHVVLRQ ZLWK WKH VWDQGDUG GHYLDWLRQ RI WKH FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ = VFRUHV ZHUH FDOFXODWHG XVLQJ WKH DERYH HTXDWLRQ 7KLV SURFHVV LV LOOXVWUDWHG LQ 7DEOH IRU RQH FRQFXVVHG SDUWLFLSDQW DW WKH +RXU WHVW VHVVLRQ $V ZLWK WKH SUHYLRXV PRGHOV D FKDQJH VFRUH ZDV FRQVLGHUHG VLJQLILFDQW LI LW H[FHHGHG LQ WKH QHJDWLYH GLUHFWLRQ 7KHUHIRUH DV VHHQ LQ 7DEOH D ]VFRUH LQGLFDWLQJ D VLJQLILFDQW FKDQJH UHODWLYH WR WKH YDULDQFH RI WKH FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ IRU +RSNLQV 7RWDO VFRUH LV LQGLFDWHG E\ D ]VFRUH OHVV WKDQ %\ FRQWUDVW D JUHDWHUWKDQDYHUDJH FKDQJH VFRUH LQ WKH QHJDWLYH GLUHFWLRQ IRU 7UDLOV $ DQG DOO RWKHU YDULDEOHV ZKHUH WLPH LV WKH PHDVXUHG YDULDEOHf LV LQGLFDWHG E\ D ]VFRUH JUHDWHU WKDQ 7DEOH SUHVHQWV WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO PHDVXUHV DW HDFK WHVW VHVVLRQ XVLQJ 0RGHO $V WKH GDWD LOOXVWUDWHV WKH VHQVLWLYLWLHV IRU LQGLYLGXDO YDULDEOHV XVLQJ 0RGHO DW WKH +RXU WHVW VHVVLRQ DUH RQFH DJDLQ UHODWLYHO\ KLJKHU WKDQ DQ\ RI WKH RWKHU GD\V DV D ZKROH 6LQJO\ WKH VHQVLWLYLW\ YDOXHV DUH QRW YHU\ KLJK $V ZLWK PRVW EDWWHULHV

PAGE 96

RI QHXURSV\FKRORJLFDO WHVW EDWWHULHV KRZHYHU WKHLU FRPELQHG XVH UHVXOWHG LQ JUHDWHU VHQVLWLYLW\ IRU WKLV PRGHO RYHU WKH UHSHDWHG WHVW VHVVLRQV 7DEOH $YHUDJH &KDQJH 6FRUHV DQG 6WDQGDUG 'HYLDWLRQV RI &KDQJH 6FRUHV IRU WKH &RQWURO 3RSXODWLRQ 0HDVXUH +RXU 6' 7HVW 'D\ 6HVVLRQ 6Q 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ DYHUDJH FKDQJH 6' VWDQGDUG GHYLDWLRQ RI FKDQJH VFRUH 7DEOH ([DPSOH RI =VFRUH 'HULYDWLRQ XVLQJ 0RGHO IRU RQH &RQFXVVHG 3DUWLFLSDQW DW +RXU 7HVW 6HVVLRQ 0HDVXUH %DVHOLQH KRXU 'LIIHUHQFH ]VFRUH +RSNLQV 7RWDO 7UDLOV $ DYHUDJH FKDQJH 6' VWDQGDUG GHYLDWLRQ RI FKDQJH VFRUH 7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\

PAGE 97

$V FDQ EH VHHQ WKH VHQVLWLYLW\ YDOXHV DW GD\ DQG GD\ DUH FRQVLVWHQWO\ SRRU ,QWHUHVWLQJO\ KRZHYHU WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDULDEOHV DW GD\ DUH VRPHZKDW KLJKHU XVLQJ 0RGHO WKDQ WKH SUHYLRXV VXJJHVWLQJ WKDW WKLV PD\ EH D PRUH DSSURSULDWH PRGHO WR XVH DW PRUH GLVWDO WHVWLQJ VHVVLRQV 7DEOHV WKURXJK LQGLFDWHG WKH VHQVLWLYLW\ DQG VSHFLILFLW\ RI WKH EDWWHU\ RI YDULDEOHV DV D ZKROH IRU 0RGHO DW HDFK WHVW VHVVLRQ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW +RXUV ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6" 6SHFLILFLW\ 7DEOH LQGLFDWHV WKDW 0RGHO ZDV YHU\ VXFFHVVIXO DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH +RXU WHVW VHVVLRQ 8VLQJ D FULWHULRQ RI RQH YDULDEOH IDOOLQJ EHORZ FXWRII WKH FRPELQHG VHQVLWLYLW\VSHFLILFLW\ ZDV ZLWK ERWK JRRG VHQVLWLYLW\ f DQG VSHFLILFLW\ f 'D\ VHQVLWLYLW\VSHFLILFLW\ GDWD XVLQJ 0RGHO 7DEOH f DJDLQ LQGLFDWHG D SDWWHUQ RI VLJQLILFDQWO\ GHFUHDVHG VHQVLWLYLW\ DQG D OHVV VXEVWDQWLDO GURS LQ VSHFLILFLW\ ZLWK WKH RQH YDULDEOH FULWHULRQ 8VLQJ WZR YDULDEOHV DV D FULWHULRQ UHVXOWV LQ DQ XQDFFHSWDEOH GURS LQ VHQVLWLYLW\ $ FRQWLQXHG UHGXFWLRQ LQ VHQVLWLYLW\ DW GD\ DV ZDV VHHQ ZLWK DOO SUHYLRXV PRGHOV ZDV DJDLQ HYLGHQW IRU 0RGHO 7DEOH f +RZHYHU DW WKH 'D\ WHVW

PAGE 98

VHVVLRQ 0RGHO UHVXOWHG LQ IDLU VHQVLWLYLW\ DQG JRRG VSHFLILFLW\ XVLQJ WKH RQHYDULDEOH FULWHULRQ 7DEOH f 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ S 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6S 6SHFLILFLW\ 7KH VWDWLVWLFDO VLJQLILFDQFH RI 0RGHO DW HDFK WHVWLQJ VHVVLRQ ZDV DJDLQ GHWHUPLQHG WKURXJK 52& FXUYH DQDO\VLV XVLQJ QRQSDUDPHWULF PHDVXUHPHQW RI DUHD XQGHU WKH FXUYH 5HVXOWV RI WKLV DQDO\VLV DUH GHSLFWHG LQ )LJXUHV WKURXJK

PAGE 99

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ L6" 6SHFLIFLW\ 52& &XUYH IRU 0RGHO DW +RXUV )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW +RXU 7HVW 6HVVLRQ )LJXUH GHSLFWV WKH 52& FXUYH IRU 0RGHO DW KRXUV 7KH $8& RI 6( b &RQILGHQFH ,QWHUYDO f ZDV ODUJHU WKDQ DQ\ RI WKH SUHFHGLQJ PRGHOV DQG ZDV KLJKO\ VLJQLILFDQW S2f LQGLFDWLQJ WKDW WKLV PRGHO ZDV JRRG DW GLVWLQJXLVKLQJ EHWZHHQ WKH FKDQJH VFRUHV RI WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV )XUWKHU )LJXUH LQGLFDWHV WKDW 0RGHO ZDV DOVR DEOH WR GLVWLQJXLVK EHWZHHQ WKH

PAGE 100

FRQFXVVHG DQG FRQWURO SRSXODWLRQV DW WKH 'D\ WHVW VHVVLRQ DW D VWDWLVWLFDOO\ VLJQLILFDQW OHYHO DQ DFKLHYHPHQW ZKLFK QR SUHYLRXV PRGHO ZDV DEOH WR DWWDLQ 7KH $8& IRU WKH PRGHO DW WKLV WHVW VHVVLRQ ZDV 6(A2 b &RQILGHQFH ,QWHUYDO f ZKLFK ZDV VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH OHYHO S f 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ %\ GD\ KRZHYHU 0RGHO ZDV QRW VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ WKH WZR JURXSV )LJ f 7KH $8& ZDV FDOFXODWHG WR EH 6( b &RQILGHQFH ,QWHUYDO f ZLWK D SYDOXH RI ,QWHUHVWLQJO\ DV ZDV VXJJHVWHG E\ WKH VHQVLWLYLW\VSHFLILFLW\ YDOXHV IRU 0RGHO DW GD\ LQ 7DEOH )LJXUH LQGLFDWHV WKDW 0RGHO ZDV LQGHHG VXFFHVVIXO DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH GD\ WHVW VHVVLRQ 7KH 52& SORW SURGXFHG DQ $8& RI 6( b &RQILGHQFH ,QWHUYDO f ZKLFK ZDV VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH OHYHO S2f

PAGE 101

52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ 1R GLVFULPLQDWLRQ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 102

5HJUHVVLRQ 0RGHOV 0RGHO 7ZR PRGHOV EDVHG RQ UHJUHVVLRQ DQDO\VLV ZHUH XVHG WR H[DPLQH WKHLU DELOLW\ WR GHWHFW VLJQLILFDQW FKDQJH IURP EDVHOLQH SHUIRUPDQFH RYHU VXEVHTXHQW UHWHVW VHVVLRQV 7KH ILUVW ZDV D VLPSOH OLQHDU UHJUHVVLRQ DSSURDFK XVLQJ EDVHOLQH SHUIRUPDQFH WR SUHGLFW SHUIRUPDQFH IRU HDFK VXFFHVVLYH WHVW VHVVLRQ UHIHUUHG WR DV 0RGHO f )RU WKLV PRGHO UHJUHVVLRQ HTXDWLRQV ZHUH FDOFXODWHG IRU HDFK RI WKH YDULDEOHV IRU HDFK WHVW VHVVLRQ IRU WKH FRQWURO SRSXODWLRQ LQ RUGHU WR GHULYH WKH QHFHVVDU\ GDWD IRU HQWU\ LQWR OLQHDU UHJUHVVLRQ HTXDWLRQV LQ WKH IRUP < D E; ZKHUH < LV WKH FRQWURO JURXSfV SHUIRUPDQFH RQ D YDULDEOH DW D UHWHVW VHVVLRQ D LV D FDOFXODWHG FRQVWDQW YDOXH DQG E LV WKH %HWD ZHLJKW IRU ; WKH FRQWUROVf SHUIRUPDQFH RQ WKDW YDULDEOH DW EDVHOLQH 5HJUHVVLRQ HTXDWLRQV ZHUH FDOFXODWHG XVLQJ 6366 VRIWZDUH IRU HDFK YDULDEOH 7KH YDULDEOHV GHULYHG IURP WKRVH UHJUHVVLRQ DQDO\VHV DUH SUHVHQWHG LQ WDEOH IRU HDFK WHVW VHVVLRQ DORQJ ZLWK WKH 5 YDOXH DVVRFLDWHG ZLWK WKH PRGHO IRU WKH FRQWURO SRSXODWLRQ 7DEOH 5HJUHVVLRQ (TXDWLRQ 'DWD IRU (DFK 9DULDEOH DQG 7HVW 6HVVLRQ 0HDVXUH FRQVWDQW Df EHWD Ef 5 +RXU +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\

PAGE 103

7DEOH &RQWLQXHG 0HDVXUH FRQVWDQW Df EHWD Ef 5 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV GHOD\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLWV )RUZDUG 'LJLWV %DFNZDUG 3HJV 'RPLQDQW 3HJV 1RQGRPLQDQW &2:$7 +RSNLQV 'HOD\ 8VLQJ WKHVH SDUDPHWHUV SUHGLFWHG UHWHVW SHUIRUPDQFHV IRU HDFK YDULDEOH ZHUH FDOFXODWHG IRU ERWK WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW HDFK WHVWLQJ VHVVLRQ 7KH SUHGLFWHG VFRUH \ IRU HDFK YDULDEOH ZDV FRPSDUHG WR WKH FRUUHVSRQGLQJ REVHUYHG VFRUH \ WR GHWHUPLQH LI WKH GLIIHUHQFH ZDV ODUJH UHODWLYH WR WKH GLVWULEXWLRQ RI UHVLGXDOV IRU WKH

PAGE 104

FRQWURO SDUWLFLSDQWV 6'U EDVHG RQ WKH UHJUHVVLRQ HTXDWLRQ ZKLFK ZDV FDOFXODWHG ZLWK WKH HTXDWLRQ $ ]VFRUH ZDV GHULYHG IRU HDFK YDULDEOH WR GHWHUPLQH LI WKH REVHUYHG VFRUH ZDV VLJQLILFDQWO\ GLIIHUHQW WKDQ WKH SUHGLFWHG VFRUH ZKLFK ZDV EDVHG RQ WKH SHUIRUPDQFH RI WKH FRQWURO SRSXODWLRQ 7KH HTXDWLRQ IRU GHULYLQJ WKHVH ]VFRUH ZDV \\ ZKHUH \ ZDV WKH REVHUYHG VFRUH IRU DQ LQGLYLGXDO YDULDEOH DQG \ ZDV WKH SUHGLFWHG VFRUH $V LQ WKH SUHYLRXV PRGHOV D ]VFRUH H[FHHGLQJ WKH FXWRII SRLQW RI ZDV WDNHQ WR LQGLFDWH D FKDQJH IURP EDVHOLQH WR UHWHVW VHVVLRQ ZKLFK ZDV VLJQLILFDQWO\ GLIIHUHQW WKDQ H[SHFWDWLRQ DQG VLQFH RQO\ GHYLDWLRQ LQ WKH QHJDWLYH GLUHFWLRQ ZDV WDNHQ DV VLJQLILFDQW D SRVLWLYH ILQGLQJ ZRXOG LQGLFDWH WKDW WKH UHWHVW SHUIRUPDQFH ZDV LQGLFDWLYH RI GHFOLQHG SHUIRUPDQFH UHODWLYH WR WKRVH RI WKH QRQFRQFXVVHG SRSXODWLRQ DW UHWHVW VHVVLRQ 7KH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDULDEOHV XVLQJ 0RGHO DUH SUHVHQWHG LQ 7DEOH $V 7DEOH LQGLFDWHV WKH VHQVLWLYLWLHV IRU WKH LQGLYLGXDO YDOXHV XVLQJ 0RGHO ZHUH JHQHUDOO\ FRPSDUDEOH WR WKRVH ZLWK 0RGHO 7KHUH ZDV D VOLJKW LQFUHDVH LQ VHQVLWLYLWLHV XVLQJ 0RGHO DW 'D\ UHODWLYH WR WKH VDPH WHVW VHVVLRQ XVLQJ 0RGHO +RZHYHU DW WKH 'D\ DQG 'D\ WHVW VHVVLRQV WKH VHQVLWLYLWLHV XVLQJ 0RGHO DUH YHU\ ORZ ZKLFK LV OLNHO\ UHIOHFWLYH RI WKH SRRU SUHGLFWLYH SRZHU RI EDVHOLQH SHUIRUPDQFH IRU SHUIRUPDQFH DW 'D\ DQG 'D\ RQ LQGLYLGXDO YDULDEOHV 7KLV LV SUREDEO\ D IXQFWLRQ RI

PAGE 105

WKH ORZ QXPEHU RI OLVWZLVH FRQWURO SDUWLFLSDQWV DW WKH ODWHU WHVWLQJ VHVVLRQV RQ ZKRVH GDWD WKH UHJUHVVLRQ HTXDWLRQV ZHUH EDVHG 7DEOH 6HQVLWLYLW\ IRU 1HXURSV\FKRORJLFDO 0HDVXUHV IRU 0RGHO 0HDVXUH +RXUV 'D\ 'D\ 'D\ +RSNLQV 7RWDO 7UDLOV $ 7UDLOV % 'LJLW 6SDQ )RUZDUG 'LJLW 6SDQ %DFN *URRYHG 3HJERDUG *URRYHG 3HJERDUG 1 &2:$7 +RSNLQV 'HOD\ 7DEOHV WKURXJK LQGLFDWHG WKH VHQVLWLYLW\ DQG VSHFLILFLW\ RI WKH EDWWHU\ RI YDULDEOHV DV D ZKROH IRU 0RGHO DW HDFK WHVW VHVVLRQ $V ZLWK WKH SUHYLRXV PRGHOV WKH KLJKHVW GHJUHH RI FODVVLILFDWLRQ DFFXUDF\ RFFXUUHG DW WKH +RXU WHVW VHVVLRQ 7DEOH f 6HQVLWLYLW\ ZDV REYLRXVO\ KLJKHVW DJDLQ ZKHQ XVLQJ VLJQLILFDQFH RQ RQH YDULDEOH DV WKH FULWHULRQ IRU FODVVLI\LQJ FRQFXVVHG SDWLHQWV f 7KLV FULWHULRQ DOVR UHVXOWHG LQ D IDLUO\ JRRG VSHFLILFLW\ UDWH f $ VLPLODU SDWWHUQ ZDV REVHUYHG XVLQJ 0RGHO DW WKH GD\ WHVW VHVVLRQ 7DEOH f /LNH WKH SUHYLRXV PRGHOV DW WKH 'D\ WHVW VHVVLRQ WKH PRGHO ZDV OHVV VXFFHVVIXO DW GLVWLQJXLVKLQJ EHWZHHQ WKH JURXSV GXH LQ ODUJH SDUW WR D VXEVWDQWLDO GURS LQ VHQVLWLYLW\ 7DEOH f $V ZDV WKH FDVH ZLWK 0RGHO WKH XVH RI UHJUHVVLRQ SUHGLFWLRQV LQ DWWHPSW WR GLVWLQJXLVK EHWZHHQ WKH JURXSV UHVXOWHG LQ D UHODWLYHO\ ODUJH LQFUHDVH LQ VSHFLILFLW\ UHODWLYH WR WKH RWKHU PRGHOV DW WKH 'D\ WHVW VHVVLRQ 7DEOH f $OWKRXJK VHQVLWLYLW\ ZDV UHODWLYHO\ ORZ HYHQ XVLQJ WKH OHDVW UHVWULFWLYH FULWHULRQ RI RQH YDULDEOH H[FHHGLQJ VLJQLILFDQFH OHYHO WKH PRGHO ZDV UHODWLYHO\ JRRG DW GLVWLQJXLVKLQJ EHWZHHQ WKH WZR JURXSV

PAGE 106

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW +RXUV ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ S 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f 6H 6HQVLWLYLW\ 6S 6SHFLILFLW\ 7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f

PAGE 107

7DEOH 1XPEHU RI &RQFXVVHG DQG &RQWURO 3DUWLFLSDQWV )DOOLQJ %HORZ &XW3RLQW IRU 0XOWLSOH 9DULDEOHV DW 'D\ ZLWK 0RGHO 1R RI 6LJ0HDVXUHV &RQFXVVHG &RQWURO 6H 6S 6H 6Sf f f f f f f f f f f L6H 6HQVLWLYLW\ AA6SHFLILFLW\ )LJXUHV WKURXJK SUHVHQW WKH 52& FXUYHV IRU 0RGHO DW HDFK WHVW VHVVLRQ 5HVXOWV VXJJHVWHG WKDW WKLV PRGHO OLNH WKH SUHYLRXVO\ H[DPLQHG PRGHOV SURYLGHG JRRG GLVWLQFWLRQ EHWZHHQ FRQFXVVHG DQG QRQFRQFXVVHG SDUWLFLSDQWV DW WKH +RXU WHVW VHVVLRQ )LJXUH f $8& ZDV 6( b &RQILGHQFH ,QWHUYDO f ZKLFK ZDV VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH Sf 0RGHO ZDV DOVR VLJQLILFDQWO\ EHWWHU WKDQ FKDQFH DW GLVFULPLQDWLQJ EHWZHHQ WKH JURXSV DW WKH 'D\ WHVW VHVVLRQ )LJXUH f ZLWK DQ $8& RI 6( b &RQILGHQFH ,QWHUYDO f ZLWK S $OWKRXJK VRPHZKDW EHWWHU WKDQ 0RGHOV DW GLVFULPLQDWLQJ EHWZHHQ JURXSV DW 'D\ 0RGHO ZDV QRW VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH DW WKLV WHVW VHVVLRQ )LJXUH f $8& ZDV 6( b &RQILGHQFH ,QWHUYDO f ZLWK D S YDOXH RI

PAGE 108

52& &XUYH IRU 0RGHO DW +RXUV 1R GLVFULPLQDWLRQ f§Rf§ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW +RXU 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 109

:KLOH VRPHZKDW OHVV VXFFHVVIXO WKDQ 0RGHO 0RGHO ZDV UHODWLYHO\ VXFFHVVIXO DW GLVWLQJXLVKLQJ EHWZHHQ WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW 'D\ DV ZHOO $8& ZDV 6( f ZLWK D SYDOXH RI )LJ f 52& &XUYH IRU 0RGHO DW 'D\ 1R GLVFULPLQDWLRQ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ 52& &XUYH IRU 0RGHO DW 'D\ 1R GLVFULPLQDWLRQ f§f§ 0RGHO )LJXUH 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF &XUYH IRU 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 110

0RGHO 7KH ILQDO PRGHO UHIHUUHG WR DV 0RGHO f H[DPLQHG DV SDUW RI WKH FXUUHQW VWXG\ LQYROYHG WKH DSSOLFDWLRQ RI PXOWLSOH UHJUHVVLRQ PRGHOV WR WKH GDWD LQ RUGHU WR GHWHUPLQH WKH HIIHFWLYHQHVV RI WZR GHPRJUDSKLF YDULDEOHV DW PRUH DFFXUDWHO\ SURYLGLQJ SUHGLFWHG UHWHVW VFRUHV DW HDFK RI WKH WHVWLQJ VHVVLRQV ,W ZDV K\SRWKHVL]HG WKDW KLVWRU\ RI GLDJQRVHG OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ ZRXOG FRQWULEXWH WR WKH DFFXUDF\ RI WKH SUHGLFWLRQ HTXDWLRQV WKXV ORZHULQJ WKH VWDQGDUG GHYLDWLRQ RI WKH DYHUDJH UHVLGXDOV IRU WKH FRQWURO JURXS DQG SURYLGLQJ QDUURZHU FRQILGHQFH LQWHUYDOV RXWVLGH RI ZKLFK D FRQFXVVHG SDUWLFLSDQWfV REVHUYHG VFRUH ZRXOG EH FRQVLGHUHG VLJQLILFDQWO\ GLIIHUHQW WKDQ D SUHGLFWHG VFRUH UHODWLYH WR EDVHOLQH SHUIRUPDQFH VXJJHVWLQJ FRQWLQXLQJ HIIHFWV RI FRQFXVVLRQ ,Q RUGHU WR H[DPLQH WKHVH YDULDEOHVf FRQWULEXWLRQ WR SUHGLFWLRQ VFRUHV WKH\ ZHUH HQWHUHG LQ DGGLWLRQ WR WKH EDVHOLQH VFRUH RQ WKH LQGLYLGXDO PHDVXUHf LQWR D PXOWLSOH UHJUHVVLRQ DQDO\VLV XVLQJ 6366 VWDWLVWLFDO VRIWZDUH IRU HDFK YDULDEOH DW HDFK WHVWLQJ VHVVLRQ IRU WKH FRQWURO SRSXODWLRQ 9DULDEOHV ZHUH HQWHUHG LQ D VWHSZLVH IDVKLRQ DQG ZHUH LQFOXGHG LQWR WKH HTXDWLRQ LI WKHLU ) SUREDELOLW\ ZDV OHVV WKDQ EXW ZHUH UHPRYHG LI WKH ) SUREDELOLW\ H[FHHGHG 7DEOHV WKURXJK LQGLFDWH WKH UHVXOWLQJ PRGHOV IRU HDFK YDULDEOH IRU WKH WHVW VHVVLRQV $V 7DEOH LQGLFDWHV QHLWKHU KLVWRU\ RI OHDUQLQJ GLVDELOLW\ QRU KLVWRU\ RI SUHYLRXV FRQFXVVLRQ HQWHUHG VLJQLILFDQWO\ LQWR DQ\ RI WKH SUHGLFWLRQ HTXDWLRQV DW WKH +RXU WHVW VHVVLRQ 8OWLPDWHO\ WKLV UHVXOWHG IRU PRVW PHDVXUHV LQ D VLPSOH UHJUHVVLRQ SUHGLFWLRQ HTXDWLRQ XVLQJ EDVHOLQH VFRUH DV WKH VROH SUHGLFWRU IRU WKDW SDUWLFXODU YDULDEOH ZKLFK ZDV HTXLYDOHQW WR 0RGHO

PAGE 111

7DEOH (TXDWLRQV 5HVXOWLQJ IURP 0XOWLSOH 5HJUHVVLRQ $QDO\VLV DW +RXUV 0HDVXUH 0RGHO 5 ) VLJ 6( +RSNLQV 7RWDO \ fEDVHOLQHf 7UDLOV $ QR VLJQLILFDQW SUHGLFWRUV 7UDLOV % \ fEDVHOLQHf 'LJLWV )RUZDUG \ fEDVHOLQHf 'LJLWV %DFNZDUG QR VLJQLILFDQW SUHGLFWRUV 3HJV 'RPLQDQW \ fEDVHOLQHf 3HJV 1RQGRPLQDQW \ fEDVHOLQHf &2:$7 QR VLJQLILFDQW SUHGLFWRUV +RSNLQV 'HOD\ \ fEDVHOLQHf 6( VWDQGDUG HUURU 7DEOH DOVR GHPRQVWUDWHV WKDW QRQH RI WKH SUHGLFWRU YDULDEOHV HQWHUHG VLJQLILFDQWO\ LQWR WKH UHJUHVVLRQV IRU WKUHH RI WKH QLQH YDULDEOHV XQGHU LQYHVWLJDWLRQ 7UDLOV $ 'LJLWV %DFNZDUG DQG &2:$7 WRWDO VFRUHf $W GD\ 7DEOH f WKHUH ZDV DOVR D WUHQG IRU WKH EDVHOLQH VFRUHV RQ LQGLYLGXDO PHDVXUHV WR EH WKH RQO\ VLJQLILFDQW SUHGLFWRU YDULDEOH HQWHUHG LQWR WKH UHJUHVVLRQ HTXDWLRQ +RZHYHU QRWDEOH H[FHSWLRQV LQFOXGHG 'D\ 7UDLOV % VFRUH IRU ZKLFK KLVWRU\ RI OHDUQLQJ GLVDELOLW\ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU HQWHULQJ WKH HTXDWLRQ ZLWK DQ 5 YDOXH RI &2:$7 WRWDO VFRUH DW 'D\ IRU ZKLFK KLVWRU\ RI FRQFXVVLRQ ZDV WKH VROH VLJQLILFDQW SUHGLFWRU ZLWK DQ 5 YDOXH RI DQG *URRYHG 3HJERDUG 1RQGRPLQDQW +DQG VFRUH IRU ZKLFK ERWK EDVHOLQH SHUIRUPDQFH DQG KLVWRU\ RI OHDUQLQJ GLVDELOLW\ HQWHUHG VLJQLILFDQWO\ LQWR WKH PRGHO UHVXOWLQJ LQ DQ RYHUDOO 5 YDOXH RI 1R SUHGLFWRU YDOXHV ZHUH VLJQLILFDQW IRU WKH 'D\ +RSNLQV 'HOD\ VFRUH $W WKH 'D\ WHVW VHVVLRQ 7DEOH f KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG EDVHOLQH SHUIRUPDQFH ERWK HQWHUHG VLJQLILFDQWO\ LQWR WKH UHJUHVVLRQ HTXDWLRQV IRU +RSNLQV 7RWDO VFRUH 5 f DQG 7UDLOV % 5 f +LVWRU\ RI OHDUQLQJ GLVDELOLW\ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU *URRYHG 3HJERDUG 'RPLQDQW +DQG 5 f DQG *URRYHG

PAGE 112

3HJERDUG 1RQGRPLQDQW +DQG 5 f 1R VWDWLVWLFDOO\ VLJQLILFDQW SUHGLFWRUV ZHUH IRXQG IRU WKH UHPDLQLQJ YDULDEOHV DW WKDW WHVW VHVVLRQ 7DEOH (TXDWLRQV 5HVXOWLQJ IURP 0XOWLSOH 5HJUHVVLRQ $QDO\VLV DW 'D\ 0HDVXUH 0RGHO 5 ) VLJ 6( +RSNLQV 7RWDO \ fEDVHOLQHf 7UDLOV $ \ fEDVHOLQHf 7UDLOV % \ fK[ RI /'f 'LJLWV )RUZDUG \ fEDVHOLQHf 'LJLWV %DFNZDUG \ fEDVHOLQHf 3HJV 'RPLQDQW \ fEDVHOLQHf 3HJV 1RQGRP \ fK[ /'f fEDVHOLQHf &2:$7 \ fK[ RI FRQHf +RSNLQV 'HOD\ QR VLJQLILFDQW SUHGLFWRUV 6( VWDQGDUG HUURU 7DEOH (TXDWLRQV 5HVXOWLQJ IURP 0XOWLSOH 5HJUHVVLRQ $QDO\VLV 'D\ 0HDVXUH 0RGHO 5 ) VLJ 6( +RSNLQV 7RWDO \ fK[ /'f fEDVHOLQHf 7UDLOV $ QR VLJQLILFDQW SUHGLFWRUV 7UDLOV % \ fK[ /'f fEDVHOLQHf 'LJLWV )RUZDUG QR VLJQLILFDQW SUHGFLWRUV 'LJLWV %DFNZDUG QR VLJQLILFDQW SUHGLFWRUV 3HJV 'RPLQDQW \ fK[ /'f 3HJV 1RQGRP \ fK[ /'f &2:$7 QR VLJQLILFDQW SUHGLFWRUV +RSNLQV 'HOD\ QR VLJQLILFDQW SUHGLFWRUV 6( VWDQGDUG HUURU 7DEOH VKRZV WKH UHVXOWV RI WKH PXOWLSOH UHJUHVVLRQ DQDO\VLV IRU HDFK YDULDEOH DW WKH 'D\ WHVW VHVVLRQ $W WKDW GD\ QR VLJQLILFDQW SUHGLFWRUV ZHUH IRXQG IRU VL[ RI WKH QLQH YDULDEOHV +LVWRU\ RI FRQFXVVLRQ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU &2:$ WRWDO VFRUH 5 f DQG RQO\ EDVHOLQH SHUIRUPDQFH IRU *URRYHG 3HJERDUG 1RQGRPLQDQW KDQG DQG +RSNLQV 'HOD\ VFRUH HQWHUHG VLJQLILFDQWO\ LQWR WKRVH UHJUHVVLRQ HTXDWLRQV

PAGE 113

*LYHQ WKH UHODWLYHO\ ORZ QXPEHU RI OLVWZLVH FRQWURO SDUWLFLSDQWV DW HDFK WHVW LQWHUYDO LW LV QRW VXUSULVLQJ WKDW WKH WZR DGGLWLRQDO GHPRJUDSKLF YDULDEOHV GLG QRW FRQWULEXWH PRUH VLJQLILFDQWO\ WR WKH UHJUHVVLRQ HTXDWLRQV +RZHYHU LW LV QRWDEOH WKDW ERWK KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ FRQWULEXWHG VLJQLILFDQWO\ WR VRPH YDULDEOHV DW WKH 'D\ 'D\ DQG 'D\ WHVW VHVVLRQV GHVSLWH WKH ORZ OLVWZLVH Q ,W LV HYHQ PRUH QRWDEOH WKDW ERWK ZHUH EHWWHU SUHGLFWRUV LQ VRPH YDULDEOHV WKDQ ZDV EDVHOLQH SHUIRUPDQFH 7DEOH (TXDWLRQV 5HVXOWLQJ IURP 0XOWLSOH 5HJUHVVLRQ $QDO\VLV DW 'D\ 0HDVXUH 0RGHO 5 ) VLJ 6( +RSNLQV 7RWDO QR VLJQLILFDQW SUHGLFWRUV 7UDLOV $ QR VLJQLILFDQW SUHGLFWRUV 7UDLOV % QR VLJQLILFDQW SUHGLFWRUV 'LJLWV )RUZDUG QR VLJQLILFDQW SUHGLFWRUV 'LJLWV %DFNZDUG QR VLJQLILFDQW SUHGLFWRUV 3HJV 'RPLQDQW QR VLJQLILFDQW SUHGLFWRUV 3HJV 1RQGRPLQDQW \ fEDVHOLQHf &2:$7 \ fK[ FRQFXVVLRQf +RSNLQV 'HOD\ \ fEDVHOLQHf 6( VWDQGDUG HUURU $V LQ WKH DQDO\VLV RI WKH EDVLF UHJUHVVLRQ PRGHO 0RGHO f WKH DERYH HTXDWLRQV ZHUH DSSOLHG WR WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWVf VFRUH IRU HDFK PHDVXUH RYHU WKH LQWHUYDO IURP EDVHOLQH WR HDFK WHVWLQJ VHVVLRQ 3UHGLFWHG VFRUHV ZHUH VXEWUDFWHG IURP DFWXDO REVHUYHG VFRUHV DQG GLYLGHG E\ WKH VWDQGDUG HUURU RI WKH UHVLGXDOV IRU WKH FRQWURO SRSXODWLRQ WR REWDLQ D ]VFRUH $ ]VFRUH H[FHHGLQJ LQ WKH QHJDWLYH GLUHFWLRQ ZDV FRQVLGHUHG D VLJQLILFDQW ILQGLQJ DV EHIRUH 7DEOHV WKURXJK SURYLGH D FRPSDULVRQ RI LQGLYLGXDO PHDVXUH VHQVLWLYLWLHV IRU WKRVH PRGHOV LQ ZKLFK KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI FRQFXVVLRQ ZHUH VLJQLILFDQW SUHGLFWRUV WR WKH VHQVLWLYLWLHV RI WKH RWKHU PRGHOV DW WKH FRUUHVSRQGLQJ WHVW VHVVLRQV

PAGE 114

7DEOH &RPSDULVRQ WR 3UHYLRXV 0RGHOV RI 6HQVLWLYLWLHV IRU 5HJUHVVLRQ (TXDWLRQV LQ ZKLFK +LVWRU\ RI /HDUQLQJ 'LVDELOLW\ /'f ZDV WKH 6ROH 6LJQLILFDQW 3UHGLFWRU /' 0RGHO 'D\ 7UDLOV % 'D\ 3HJV 'RP 'D\ 3HJV 1GRP 7DEOH &RPSDULVRQ WR 3UHYLRXV 0RGHOV RI 6HQVLWLYLWLHV IRU 5HJUHVVLRQ (TXDWLRQV LQ ZKLFK +LVWRU\ RI 3UHYLRXV &RQFXVVLRQ &RQHf ZDV WKH 6ROH 6LJQLILFDQW 3UHGLFWRU 0RGHO &RQH 'D\ &2:$7 'D\ &2:$7 7DEOH &RPSDULVRQ WR 3UHYLRXV 0RGHOV RI 6HQVLWLYLWLHV IRU 5HJUHVVLRQ (TXDWLRQV LQ ZKLFK +LVWRU\ RI /' DQG %DVHOLQH 3HUIRUPDQFH ZHUH 6LJQLILFDQW 3UHGLFWRUV /'%DVHOLQH 0RGHO 'D\ +RSNLQV 7 'D\ 7UDLOV % $V FDQ EH VHHQ LQ WDEOH ZKHQ KLVWRU\ RI GLDJQRVHG OHDUQLQJ GLVDELOLW\ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU LW GLG QRW SURYLGH D VXEVWDQWLDO LQFUHDVH LQ VHQVLWLYLW\ UHODWLYH WR WKH SUHYLRXVO\ GHVFULEHG PRGHOV 1RU GLG KLVWRU\ RI OHDUQLQJ GLVDELOLW\ SURYLGH DQ LQFUHDVH LQ VHQVLWLYLW\ UHODWLYH WR WKH RWKHU PRGHOV ZKHQ LW ZDV HQWHUHG LQWR UHJUHVVLRQ HTXDWLRQV ZLWK EDVHOLQH SHUIRUPDQFH 7DEOH f 7DEOH VXJJHVWV WKDW KLVWRU\ RI SUHYLRXV FRQFXVVLRQ ZDV UHODWLYHO\ VXFFHVVIXO DW LQFUHDVLQJ WKH VHQVLWLYLW\ RI &2:$ WRWDO VFRUH DW WKH 'D\ WHVW VHVVLRQ DQG GLG SURYLGH D YHU\ VOLJKW LQFUHDVH LQ VHQVLWLYLW\ RI &2:$ WRWDO VFRUH DW 'D\ $Q RYHUDOO FRPSDULVRQ RI WKH VHQVLWLYLWLHV IRXQG E\ WKH

PAGE 115

DSSOLFDWLRQ RI DOO PRGHOV IRU HDFK YDULDEOH RYHU WKH WHVW VHVVLRQV LV SUHVHQWHG LQ 7DEOH 7DEOH 2YHUDOO &RPSDULVRQ RI 6HQVLWLYLWLHV 7HVW 6HVVLRQ 0RGHO +RS 7RW 7U $ 7U % 'LJ ) 'LJ % 3HJ 3HJ 1' &2:$ +RS %DVH +RXU %DVH'D\ %DVH'D\ %DVH'D\ $V ZDV QRWHG DERYH IRU HDFK PRGHO VHQVLWLYLWLHV ZHUH JHQHUDOO\ KLJKHVW DW WKH +RXU WHVW VHVVLRQ DFURVV YDULDEOHV ([DPLQDWLRQ RI 7DEOH VXJJHVWV WKDW WKH ODWHU PRGHOV LQFOXGLQJ 5&, ZLWK FRUUHFWLRQ IRU SUDFWLFH 0RGHO f 5&, H[DPLQDWLRQ RI GLIIHUHQFH VFRUHV 0RGHO f DQG WKH VLPSOH UHJUHVVLRQ PRGHO 0RGHO f WHQGHG WR EH VOLJKWO\ PRUH VHQVLWLYH DW +RXUV DFURVV YDULDEOHV $W 'D\ 'D\ DQG 'D\ WHVWLQJ VHVVLRQV D GLIIHUHQFH LQ VHQVLWLYLWLHV EHWZHHQ PRGHOV ZDV PRUH VXEWOH ZLWK D WHQGHQF\ WRZDUG ODWHU

PAGE 116

PRGHOV EHLQJ VOLJKWO\ PRUH VHQVLWLYH WKDQ WKH HDUOLHU OHVV FRPSOH[ PRGHOV 7KLV LV PRUH HDVLO\ REVHUYHG LQ D FRPSDULVRQ EHWZHHQ $8& PHDVXUHV 7KLV LV SUHVHQWHG LQ 7DEOH 7DEOH &RPSDULVRQ RI $8& 0HDVXUHV IRU $OO 0RGHOV $FURVV 7HVWLQJ 6HVVLRQV 0RGHO %DVH +RXUV %DVH'D\ %DVH'D\ %DVH'D\ r r r r r r r r r r r GHQRWHV VWDWLVWLFDOO\ GLIIHUHQW WKDQ FKDQFH VHH DERYH IRU LQGLYLGXDO SYDOXHV $V 7DEOH VXJJHVWV DOO PRGHOV ZHUH VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH KRXU WHVW VHVVLRQ ,W ZDV QRWHG DERYH WKDW 0RGHO DQG 0RGHO ZHUH QRW GLIIHUHQW WKDQ HDFK RWKHU LQ WKLV UHJDUG /LNHZLVH 0RGHO DQG 0RGHO ZHUH QRW VWDWLVWLFDOO\ GLIIHUHQW LQ GLVWLQJXLVKLQJ EHWZHHQ WKH WZR JURXSV 0RGHO 5&, GLIIHUHQFH VFRUHVf KDG WKH KLJKHVW $8& YDOXH DW WKH +RXU WHVW VHVVLRQ DQG DSSHDUHG WR EH WKH PRVW VHQVLWLYH RYHUDOO 6WDWLVWLFDO FRPSDULVRQ RI WKH $8& PHDVXUHV IRU 0RGHOV DQG LQGLFDWHG WKDW WKH\ ZHUH QRW VWDWLVWLFDOO\ VLJQLILFDQW S!f ZKLFK LV VHHQ JUDSKLFDOO\ LQ )LJXUH $W WKH 'D\ WHVW VHVVLRQ RQO\ 0RGHO DQG 0RGHO ZHUH VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ WKH FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV :LWK DQ $8& YDOXH RI 0RGHO VLPSOH OLQHDU UHJUHVVLRQf DSSHDUHG PRUH DFFXUDWH LQ GLVWLQJXLVKLQJ EHWZHHQ WKH WZR JURXSV WKDQ 0RGHO 5&, IRU GLIIHUHQFH VFRUHVf ZKLFK KDG DQ $8& YDOXH RI &RPSDULVRQ RI WKH WZR PRGHOV KRZHYHU LQGLFDWHG WKDW WKH\ ZHUH QRW VWDWLVWLFDOO\ GLIIHUHQW S!f ZKLFK LV H[KLELWHG LQ )LJXUH

PAGE 117

&RPSDULVRQ RI 6LJQLILFDQW 'HWHFWLRQ 0RGHOV DW +RXUV )LJXUH &RPSDULVRQ RI 0RGHOV DQG DW WKH +RXU 7HVW 6HVVLRQ &RPSDULVRQ RI 0RGHOV DQG DW 'D\ )LJXUH &RPSDULVRQ RI 52& &XUYHV IRU 0RGHO DQG 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 118

,OO $V 7DEOH VXJJHVWV QRQH RI WKH PRGHOV ZDV VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ WKH WZR JURXSV DW 'D\ DOWKRXJK 0RGHO KDG WKH KLJKHVW $8& YDOXH f $W WKH 'D\ WHVW VHVVLRQ 0RGHOV DQG ZHUH DJDLQ WKH RQO\ VWDWLVWLFDOO\ DFFXUDWH PHWKRGV IRU GLVWLQJXLVKLQJ EHWZHHQ WKH JURXSV 0RGHO ZLWK DQ $8& YDOXH RI ZDV WKH PRVW DFFXUDWH ZKLOH 0RGHO ZDV VLJQLILFDQW ZLWK DQ $8& YDOXH RI $ VWDWLVWLFDO FRPSDULVRQ RI WKH PRGHOV LQGLFDWHG KRZHYHU WKDW WKH WZR ZHUH QRW VLJQLILFDQWO\ GLIIHUHQW S!f DV VKRZQ LQ )LJXUH &RPSDULVRQ RI 0RGHO DQG DW 'D\ 1R GLVFULPLQDWLRQ f§Rf§0RG HO 0RGHO )LJXUH &RPSDULVRQ RI 52& &XUYHV IRU 0RGHO DQG 0RGHO DW 'D\ 7HVW 6HVVLRQ

PAGE 119

&+$37(5 )285 ',6&866,21 7KH GHWHFWLRQ RI OLQJHULQJ FRJQLWLYH GHFOLQH LQ WKRVH KDYLQJ VXIIHUHG PLOG KHDG LQMXU\ KDV D QXPEHU RI LPSOLFDWLRQV $OWKRXJK LW KDV EHHQ VKRZQ LQ QXPHURXV VWXGLHV DQG WKURXJK REVHUYDWLRQDO GDWD WKDW WKH FRJQLWLYH HIIHFWV RI FRQFXVVLRQ WHQG WR GLVVLSDWH UHODWLYHO\ TXLFNO\ DQG XVXDOO\ UHVROYH ZLWKLQ WR GD\V DIWHU WKH LQFLGHQW WKH PDQLIHVWDWLRQ RI FRQWLQXLQJ FRJQLWLYH VHTXHODH PD\ EH DQ LQGLFDWRU RI D PRUH VHULRXV XQGHUO\LQJ SDWKRORJLFDO FRQGLWLRQ 7KH LPSRUWDQFH RI GHWHFWLQJ OLQJHULQJ FRJQLWLYH HIIHFWV LV PDJQLILHG ZKHQ RQH FRQVLGHUV WKDW WKRVH KDYLQJ VXIIHUHG IURP PLOG KHDG LQMXU\ DUH PRUH DW ULVN IRU WKH IXWXUH RFFXUUHQFH RI KHDG LQMXU\ 5REHUWV HW DO f HVSHFLDOO\ ZLWKLQ WKH WLPH IUDPH LQ ZKLFK WKH\ DUH UHFRYHULQJ IURP WKH LQLWLDO LQFLGHQW 7KH ELRORJLFDO XQGHUSLQQLQJV RI WKLV DUH VRPHZKDW XQFOHDU WKRXJK PD\ EH UHODWHG WR FKDQJHV LQ WKH ELRFKHPLFDO HQYLURQPHQW LQGXFHG E\ WKH LQMXU\ LWVHOI 7KH H[SHULHQFH RI FRQFXVVLRQ FDQ OHDG WR ORQJHUWHUP HIIHFWV %DUWK HW DK 5LPHO HW DK 5REHUWV HW DK f DQG WKH H[SHULHQFH RI D PXOWLSOH FRQFXVVLRQV ZLWKLQ D VKRUW WLPH SHULRG KDV OHG WR GHDWK LQ VRPH FLUFXPVWDQFHV 7KH DERYH LV HVSHFLDOO\ LPSRUWDQW ZKHQ FRQVLGHULQJ WKH HIIHFW RI FRQFXVVLRQ LQ VSRUWV %\ QDWXUH IRRWEDOO SURYLGHV WKH HQYLURQPHQW IRU D UHODWLYHO\ KLJK LQFLGHQFH RI FRQFXVVLRQ DPRQJ LWV SDUWLFLSDQWV $ GHFLVLRQ UHJDUGLQJ ILWQHVV IRU D SOD\HU WR UHWXUQ WR DFWLRQ IROORZLQJ FRQFXVVLRQ LV RIWHQ PDGH E\ WHDP WUDLQHUV RU SK\VLFLDQV ZKR DUH DIIRUGHG OLWWOH REMHFWLYH LQIRUPDWLRQ XSRQ ZKLFK WR PDNH WKLV GHFLVLRQ RWKHU WKDQ JURVV

PAGE 120

PDUNHUV RI RULHQWDWLRQ DQG D SOD\HUfV DVVXUDQFH WKDW WKH FREZHEV KDYH FOHDUHG VR WR VSHDN 7KH DSSOLFDWLRQ RI QHXURSV\FKRORJLFDO WHVWV WR D UHWXPWRSOD\ GHFLVLRQ SURYLGHV IDU PDU REMHFWLYH FULWHULD DQG IDU EHWWHU VHQVLWLYH PHDVXUH RI FRJQLWLYH IXQFWLRQLQJ 8QWLO UHFHQWO\ IHZ UHVHDUFKHUV KDG LPSOHPHQWHG FRQWUROOHG VWXGLHV RI WKH XVHIXOQHVV RI QHXURSV\FKRORJLFDO PHDVXUHV LQ GHWHFWLQJ FRJQLWLYH VHTXHODH RI FRQFXVVLRQ DPRQJ DWKOHWHV LQ WKH DFXWH VWDJHV ,Q D VWXG\ SXEOLVKHG LQ E\ :LOOLDP %DUU DQG 0LFKDHO 0F&UHD WKH UHVHDUFKHUV DVVHVVHG WKH XWLOLW\ RI WKH 6LGHOLQH $VVHVVPHQW RI &RQFXVVLRQ 6$&f LQ GHWHFWLQJ FRJQLWLYH GHFOLQH LQ WKH LPPHGLDWH VWDJHV IROORZLQJ FRQFXVVLRQ DQG IRXQG WKLV WR EH D XVHIXO WRRO LQ TXDQWLI\LQJ GHWULPHQW LQ SHUIRUPDQFH +RZHYHU IHZ VWXGLHV WR GDWH KDYH XVHG D VWDQGDUGL]HG EDWWHU\ RI QHXURSV\FKRORJLFDO PHDVXUHV WR DVVHVV WKH SDWWHUQ RI FRJQLWLYH GHFOLQH DQG UHWXUQ WR EDVHOLQH IXQFWLRQLQJ RYHU D PRUH SURORQJHG SHULRG %DUWK HW DO f 7KH FXUUHQW VWXG\ XWLOL]HG GDWD FROOHFWHG IURP FROOHJH IRRWEDOO SOD\HUV DW IRXU PDMRU 'LYLVLRQ XQLYHUVLWLHV UHVXOWLQJ LQ D UHSUHVHQWDWLYH VDPSOH RI FROOHJH IRRWEDOO SOD\HUV DW WKLV OHYHO %DVHOLQH SHUIRUPDQFH RQ D EULHI QHXURSV\FKRORJLFDO WHVW EDWWHU\ ZKLFK LQFOXGHG WKH +RSNLQV 9HUEDO /HDUQLQJ 7HVW 7UDLO 0DNLQJ 7HVW $ DQG % 'LJLW 6SDQ WHVW *URRYHG 3HJERDUG DQG &RQWUROOHG 2UDO :RUG $VVRFLDWLRQ ZHUH UHFRUGHG 6XEVHTXHQWO\ SOD\HUV ZKR VXIIHUHG FRQFXVVLRQ HLWKHU LQ SUDFWLFH RU LQ D JDPH VLWXDWLRQ ZHUH LGHQWLILHG E\ WHDP WUDLQHUV DQG UHDGPLQLVWHUHG WKH QHXURSV\FKRORJLFDO EDWWHU\ DW HDFK RI WHVWLQJ VHVVLRQV ZLWKLQ KRXUV RI FRQFXVVLRQ GD\V GD\V DQG GD\V SRVWFRQFXVVLRQ 7KHVH SOD\HUV ZHUH DVVLJQHG FRQWURO SDUWLFLSDQWV ZKR ZHUH PDWFKHG DV FORVHO\ DV SRVVLEOH ZLWK UHJDUG WR SRVLWLRQ KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ

PAGE 121

6HYHQ VWDWLVWLFDO PRGHOV ZHUH H[DPLQHG 7KHVH ZHUH FRQFHSWXDOO\ GLYLGHG LQWR WKUHH JURXSV FRQVLVWLQJ RI EDVLF FOLQLFDO FRPSDULVRQV UHOLDEOH FKDQJH LQGH[ PRGHOV DQG UHJUHVVLRQ DQDO\VLV PRGHOV 7KHVH JURXSV LQFOXGHG IRXU PRGHOV LQ SDUWLFXODU ZKLFK KDYH EHHQ H[DPLQHG E\ UHVHDUFKHUV LQ WKH GHWHFWLRQ RI fWUXHf FKDQJH LQ QHXURSV\FKRORJLFDO SHUIRUPDQFH RYHU UHSHDWHG WHVWLQJ LQWHUYDOV %DUU DQG 0F&UHD 7HPNLQ HW DO f 7KHVH ZHUH WKH 5HOLDEOH &KDQJH ,QGH[ 5&,f D YDULDQW RI WKH 5&, ZLWK FRUUHFWLRQ IRU SUDFWLFH D OLQHDU UHJUHVVLRQSUHGLFWHG VFRUH PRGHO DQG D PXOWLSOH UHJUHVVLRQSUHGLFWHG VFRUH PRGHO XWLOL]LQJ GHPRJUDSKLF YDULDEOHV WR HQKDQFH SUHGLFWLRQ SRZHU 7KH FXUUHQW VWXG\ DWWHPSWHG WR SURYLGH IXUWKHU HYLGHQFH WKDW WKHVH VWDWLVWLFDO PRGHOV FDQ EH XVHG LQ WKH DQDO\VLV RI QHXURSV\FKRORJLFDO WHVW SHUIRUPDQFH WR GHWHFW OLQJHULQJ FRJQLWLYH VHTXHODH RI FRQFXVVLRQ WKURXJK VHULDO WHVWLQJ DQG PD\ DOVR EH YDOXDEOH LQ KHOSLQJ WR GHWHUPLQH DQ DWKOHWHfV ILWQHVV IRU UHWXUQ WR SOD\ $Q RYHUDOO DQDO\VLV RI WKH WHVWUHWHVW SHUIRUPDQFH RI FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV LQGLFDWHG WKDW WKRVH KDYLQJ VXIIHUHG FRQFXVVLRQ H[KLELWHG D SDWWHUQ RI SHUIRUPDQFH RYHU WLPH WKDW ZDV VLPLODU WR WKDW REVHUYHG E\ RWKHUV %DUWK HW DO +LQWRQ%D\UH HW DO 0DGGRFNV DQG 6DOLQJ f 7KDW LV SHUIRUPDQFH ZLWKLQ WKH YHU\ HDUO\ VWDJHV DIWHU FRQFXVVLRQ ZLWKLQ KRXUVf JHQHUDOO\ VDZ D GHFOLQH LQ SHUIRUPDQFH OHYHO RU IDLOHG WR H[KLELW H[SHFWHG SUDFWLFH HIIHFWV %\ FRQWUDVW LW ZDV IDLUO\ REYLRXV HYHQ ZLWKRXW WKH DSSOLFDWLRQ RI VWDWLVWLFDO PHWKRGV WKDW WKH PDWFKHG FRQWUROV DV D ZKROH WHQGHG WR H[KLELW LQFUHDVHG SHUIRUPDQFH RYHU WKH QLQH YDULDEOHV H[DPLQHG DV SDUW RI WKH QHXURSV\FKRORJLFDO EDWWHU\ XWLOL]HG ([DPLQDWLRQ RI JUDSKLFDO UHSUHVHQWDWLRQ RI UHFRYHU\ WUHQGV RYHU WKH UHPDLQLQJ WKUHH UHWHVW VHVVLRQV LQGLFDWHG DV WKH OLWHUDWXUH VXJJHVWV JUDGXDO LPSURYHPHQW RYHU WKH ZHHN IROORZLQJ FRQFXVVLRQ

PAGE 122

,Q H[DPLQLQJ WKH FULWHULD XVHG WR FODVVLI\ SOD\HUV DV FRQWLQXLQJ WR H[KLELW FRJQLWLYH HIIHFWV RI FRQFXVVLRQ UHVHDUFKHUV KDYH FDXWLRQHG WKDW HUULQJ RQ WKH VLGH RI FDXWLRQ LV WKH EHVW FRXUVH RI DFWLRQ EHFDXVH RI WKH LPSRUWDQW LPSOLFDWLRQV WKDW IDLOLQJ WR GHWHFW OLQJHULQJ HIIHFWV PD\ KDYH 7KHUHIRUH VRPH KDYH HOHFWHG WR IDYRU WKH XVH RI UDWKHU OLEHUDO FULWHULD LQ FODVVLILFDWLRQ %DUU DQG 0F&UHD f 7KH UDPLILFDWLRQ RI WKLV RI FRXUVH LV WKH DFFHSWDQFH RI ORZHU VSHFLILFLW\ UDWHV 2SWLPDOO\ D XVHIXO VWDWLVWLFDO PRGHO ZRXOG VHHN WR PD[LPL]H ERWK VHQVLWLYLW\ DQG VSHFLILFLW\ 2SWLPDO FXWSRLQWV IRU WKHVH PRGHOV FDQ EH LQGLFDWHG E\ FRPSDULVRQ RI WKH VXP RI VHQVLWLYLW\ DQG VSHFLILFLW\ DW HDFK FXW SRLQW )RU WKH FXUUHQW VWXG\ WKLV PD[LPL]HG VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQ ZDV JHQHUDOO\ DFKLHYHG XVLQJ D FXWSRLQW RI RQH RU PRUH YDULDEOHV IDOOLQJ VLJQLILFDQWO\ EHORZ H[SHFWDWLRQ EDVHG RQ WKH SDUWLFXODU PRGHO H[DPLQHG ,Q VRPH LQVWDQFHV XVLQJ D FULWHULRQ RI WZR RU PRUH YDULDEOHV IDOOLQJ EHORZ H[SHFWDWLRQ DFKLHYHG WKH PD[LPXP VHQVLWLYLW\VSHFLILFLW\ FRPELQDWLRQ EXW WKLV FDPH DW WKH JUHDWHVW H[SHQVH IRU VHQVLWLYLW\ (UULQJ RQ WKH VLGH RI FDXWLRQ WKHUHIRUH WKH FXWSRLQWV ZLWK WKH PD[LPXP VHQVLWLYLW\ DSSHDU WR EH WKH PRVW VDOLHQW LQ WKLV FDVH RQH RU PRUH YDULDEOHV IDOOLQJ EHORZ H[SHFWDWLRQ EDVHG RQ WKH SDUWLFXODU VWDWLVWLFDO PRGHO $QRWKHU SRWHQWLDO DSSURDFK PD\ EH WR H[DPLQH WKH SRVLWLYH DQG QHJDWLYH SUHGLFWLYH SRZHUV IRU WKH LQGLYLGXDO PHDVXUHV +RZHYHU VLQFH WKH DLP RI WKH WHVWV LV WR GHWHFW UDWKHU WKDQ SUHGLFW GHFUHPHQW LQ SHUIRUPDQFH VHQVLWLYLW\ DQG VSHFLILFLW\ YDOXHV DUH PRUH DSSURSULDWH LQ WKLV LQVWDQFH 7KLV VWXG\ XWLOL]HG D FRPSDULVRQ RI PRGHOV XVLQJ 5HFHLYHU 2SHUDWLQJ &KDUDFWHULVWLF FXUYHV D PHWKRG ZKLFK KDV EHHQ XVHG RIWHQ LQ WKH PHGLFDO OLWHUDWXUH WR GHWHUPLQH WKH XVHIXOQHVV RI SDUWLFXODU ELRORJLFDO PDUNHUV RU WHVWV WR DFFXUDWHO\ FODVVLI\ WKH SUHVHQFH RI SDWKRORJ\ &KD HW DO 'H
PAGE 123

f 7KLV PHWKRG KDV DOVR JDLQHG DQ LQFUHDVLQJ XVH DFURVV YDULHG VSHFLDOWLHV LQ WKH SV\FKRORJLFDO OLWHUDWXUH %DUU DQG 0F&UHD *ODVVPLUH HW DO +HDWKFRWH 5RVV HW DO f $ VLPLODU SURFHGXUH ZDV XVHG E\ %DUU HW DO WR H[DPLQH WKH FODVVLILFDWLRQ DFFXUDF\ RI DQ 5&, DQG D UHJUHVVLRQ PRGHO LQ HDUO\VWDJH ZLWKLQ KRXUVf FRQFXVVLRQ WHVWLQJ ZLWK WKH 6$& $Q 52& DQDO\VLV LQYROYHV WKH QRQSDUDPHWULF PHDVXUH RI WKH DUHD XQGHU WKH FXUYH VHQVLWLYLW\ SORWWHG DJDLQVW VSHFLILFW\f ZKHUH DQ DUHD RI GHQRWHV FKDQFH OHYHO 7KHUHIRUH D FDOFXODWHG DUHD FDQ EH FRPSDUHG WR FKDQFH OHYHO WR GHWHUPLQH WKH FODVVLILFDWLRQ DFFXUDF\fV VWDWLVWLFDO VLJQLILFDQFH /LNHZLVH WKH $8& RI WZR RU PRUH FXUYHV FDQ EH FRPSDUHG WR GHWHUPLQH LI RQH PRGHO LV PRUH DFFXUDWH WKDQ DQRWKHU %DVLF 0RGHOV ,Q WKH PRVW EDVLF PRGHOV 0RGHO DQG 0RGHO f H[DPLQHG DV SDUW RI WKLV VWXG\ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWVf VFRUHV DW HDFK WHVWLQJ VHVVLRQ ZHUH FRPSDUHG WR WKH QRUPDWLYH SRSXODWLRQ PHDQ VFRUH RI WKH OHVV FRQFXVVHG SOD\HUV DQG FRQWUROf LQGLYLGXDOV IURP WKH IRXU VFKRROV DQG WR WKHLU RZQ EDVHOLQH SHUIRUPDQFH UHVSHFWLYHO\ LQ RUGHU WR GHWHFW ODUJH GLIIHUHQFHV UHODWLYH WR WKH VWDQGDUG GHYLDWLRQ RI WKH EDVHOLQH SRSXODWLRQ IRU HDFK PHDVXUH 6XFK FRPSDULVRQV UHVXOW LQ VWDQGDUG ]VFRUH DQG DUH QRW XQOLNH WKH DQDO\VLV RI QHXURSV\FKRORJLFDO WHVW GDWD XVHG LQ WKH FOLQLFDO VHWWLQJ 6LJQLILFDQFH OHYHOV ZHUH GHWHUPLQHG IRU WKH LQGLYLGXDO YDULDEOHV EDVHG RQ D OLEHUDO FULWHULD RI 6' EDVHG RQ WKH QRUPDWLYH VDPSOHf EHORZ WKH QRUPDWLYH PHDQ RU EHORZ WKHLU EDVHOLQH SHUIRUPDQFH ZKLFK LV RIWHQ XVHG LQ FRPPRQ QHXURSV\FKRORJLFDO WHVWV WR GHQRWH PLOG LPSDLUPHQW 5HLWDQ DQG :ROIVRQ f 7KH DSSOLFDWLRQ RI WKLV PHWKRG UHVXOWHG LQ FRPSDUDEO\ KLJK GHJUHHV RI VHQVLWLYLW\ DW WKH KRXU WHVW VHVVLRQ IRU ERWK PRGHOV DQG UHVSHFWLYHO\f 7KH XVH RI HYHQ RQH YDULDEOH IDOOLQJ EHORZ WKH GHVLJQDWHG

PAGE 124

FODVVLILFDWLRQ FULWHULD LQ 0RGHO KRZHYHU UHVXOWHG LQ D YHU\ ORZ VSHFLILFLW\ UDWH DW WKDW WHVW VHVVLRQ f 0RUH WKDQ KDOI RI WKH FRQWURO SDUWLFLSDQWV ZHUH IDOVHO\ FODVVLILHG DV FRQFXVVHG 8VLQJ WKH VDPH FULWHULD RQH YDULDEOH IDOOLQJ EHORZ H[SHFWDWLRQf IRU 0RGHO RQ WKH RWKHU KDQG UHVXOWHG LQ D UHODWLYHO\ ORZHU IDOVHSRVLWLYH UDWH VSHFLILFLW\ f VXJJHVWLQJ VOLJKWO\ EHWWHU FODVVLILFDWLRQ DFFXUDF\ ZLWKLQ KRXUV 7KLV GLIIHUHQFH ZDV QRW VWDWLVWLFDOO\ VLJQLILFDQW EXW WKH LQFUHDVH LQ VSHFLILFLW\ SRLQWV WR WKH VWDWLVWLFDO EHQHILW RI KDYLQJ D EDVHOLQH FRPSDUDWRU ([DPLQDWLRQ RI $8& GDWD LQGLFDWHG WKDW QHLWKHU 0RGHO QRU 0RGHO ZHUH VWDWLVWLFDOO\ EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH 'D\ 'D\ RU 'D\ WHVW VHVVLRQV :KLOH WKH\ GLG FRQWLQXH WR KDYH VRPH VHQVLWLYLW\ ZKHQ XVLQJ WKH RQHYDULDEOH FODVVLILFDWLRQ FULWHULRQ WKH FRPELQHG VHQVLWLYLW\VSHFLILFLW\ UDWHV ZHUH VXFK WKDW QHLWKHU PRGHO FRXOG EH FKDUDFWHUL]HG DV VXFFHVVIXO DW GLVFULPLQDWLQJ EHWZHHQ FRQFXVVHG DQG QRQFRQFXVVHG SDUWLFLSDQWV DW DQ\ RI WKHVH GD\V 1HLWKHU WKHUHIRUH DSSHDUV WR KDYH DQ\ FOLQLFDO YDOXH LQ WKH GHWHUPLQDWLRQ RI UHWXPWRSOD\ VWDWXV DIWHU WKH KRXU SHULRG 5&, 0RGHOV )RU WKH UHPDLQGHU RI WKH PRGHOV H[DPLQHG VLJQLILFDQFH IRU LQGLYLGXDO YDULDEOHV ZDV GHWHUPLQHG E\ GDWD JDWKHUHG IURP WKH FRQWURO SDUWLFLSDQWV )RU WKH EDVLF 5&, PRGHO DQG WKH 5&, PRGHO ZLWK SUDFWLFH IRU FRUUHFWLRQ WKH VWDWLVWLFDO PHWKRG HVVHQWLDOO\ DVNHG WKH TXHVWLRQ ZDV WKH LQGLYLGXDOfV VFRUH DW D UHWHVW VHVVLRQ RXWVLGH WKH QRUPDO UDQJH RI FKDQJH VFRUHV EDVHG RQ WKH PHDVXUHfV UHOLDELOLW\" 7KDW VWDQGDUG HUURU LQ WXUQ ZDV GHULYHG IURP WKH FRUUHODWLRQ RI WKH FRQWURO SDUWLFLSDQWVf VFRUHV RQ WKH LQGLYLGXDO YDULDEOH DW HDFK WHVWUHWHVW LQWHUYDO 7KH VWDQGDUG GHYLDWLRQ RQ WKH RWKHU KDQG ZDV REWDLQHG RQO\

PAGE 125

IURP WKH EDVHOLQH WHVW VHVVLRQ $ b FRQILGHQFH LQWHUYDO ZDV XVHG LQ HDFK FDVH ZLWK D ] YDOXH RI $OWKRXJK VHHQ DV VRPHZKDW WRR FRQVHUYDWLYH E\ VRPH UHVHDUFKHUV %DUU DQG 0F&UHD f WKLV OHYHO RI VLJQLILFDQFH ZDV FKRVHQ LQ RUGHU WR SURYLGH JUHDWHU EDODQFH EHWZHHQ VHQVLWLYLW\ DQG VSHFLILFLW\ HVSHFLDOO\ VLQFH PRUH OLEHUDO FULWHULD ZHUH XVHG LQ ODWHU VWHSV RI WKH FODVVLILFDWLRQ SURFHVV ,W KDV EHHQ VXJJHVWHG E\ VRPH UHVHDUFKHUV WKDW QDUURZHU FRQILGHQFH LQWHUYDOV FUHDWHG E\ WKLV PHWKRG PD\ PDNH WUXH FKDQJH LQ WHVW SHUIRUPDQFH PRUH UHDGLO\ DSSDUHQW 7HPNLQ HW DO f 7KH XVH RI FRUUHFWLRQ IRU SUDFWLFH HIIHFWV LQ WKH 5&, PRGHO DV 7HPNLQ QRWHG GRHV QRW FKDQJH WKH ZLGWK RI WKH FRQILGHQFH LQWHUYDO EXW GRHV SURYLGH D PHWKRG IRU UHYHDOLQJ D GHJUHH RI FKDQJH WKDW PD\ KDYH PDVNHG LQ WKLV FDVH D WUXH GHFUHDVH LQ SHUIRUPDQFH FRPSDUHG WR EDVHOLQH 8VH RI WKH EDVLF 5&, PRGHO UHVXOWHG DJDLQ LQ IDLUO\ JRRG FODVVLILFDWLRQ DFFXUDF\ ZLWKLQ KRXUV RI FRQFXVVLRQ XVLQJ WKH RQHRUPRUH VLJQLILFDQW YDULDEOH FXWSRLQW VHQVLWLYLW\ f DV ZHOO DV D QRWDEOH LQFUHDVH LQ VSHFLILFLW\ DW WKDW WHVW VHVVLRQ f RYHU HLWKHU RI WKH ILUVW WZR PRGHOV 7KH FDOFXODWHG $8& YDOXH DW WKH KRXU WHVW VHVVLRQ RI VXJJHVWHG D IDLU DELOLW\ WR GLVWLQJXLVK EHWZHHQ FRQFXVVHG DQG QRQFRQFXVVHG LQGLYLGXDOV +RZHYHU LW ZDV QR EHWWHU WKDQ FKDQFH DW GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV RYHU WKH VXEVHTXHQW WHVW VHVVLRQV &RUUHFWLRQ IRU SUDFWLFH LQ WKH 5&, PRGHO 0RGHO f SURYLGHG QR LPSURYHPHQW LQ VHQVLWLYLW\ f RU VSHFLILFLW\ f DW WKH KRXU WHVW VHVVLRQ XVLQJ WKH RQHRUPRUH YDULDEOH GHWHFWLRQ FULWHULD :KLOH FRUUHFWLQJ IRU SUDFWLFH GLG SURYLGH PLQLPDO LPSURYHPHQW LQ FODVVLILFDWLRQ DFFXUDF\ RYHU WKH 'D\ 'D\ DQG 'D\ WHVW VHVVLRQV WKH PRGHO ZDV QRW VLJQLILFDQWO\ GLIIHUHQW WKDQ FKDQFH DW WKRVH LQWHUYDOV LQ GLVWLQJXLVKLQJ

PAGE 126

EHWZHHQ WKH JURXSV EDVHG RQ $8& PHDVXUHV 5HVXOWV ZHUH FRPSDUDEOH WR WKRVH REWDLQHG E\ 7HPNLQ ZKR LQGLFDWHG WKDW LW ZDV WKH 5&, WKDW ZDV WKH OHDVW HIIHFWLYH RI WKH PRGHOV VWXGLHG E\ WKRVH UHVHDUFKHUV ([SORULQJ WKH DSSDUHQWO\ ORZ XWLOLW\ RI WKHVH PRGHOV DW PRUH GLVWDO WHVW VHVVLRQV IURP FRQFXVVLRQ VRPH XQGHUO\LQJ DVVXPSWLRQV LQKHUHQW WR LWV DSSOLFDWLRQ PD\ EH LQIOXHQWLDO )LUVWO\ WKH PRGHO DVVXPHV WKDW D VXEMHFWfV VFRUH DW WLPH RQH DQG WLPH WZR DUH HTXDO 6HFRQGO\ LW DVVXPHV WKDW WKH VWDQGDUG GHYLDWLRQV DW WKRVH WZR VHVVLRQV DUH DOVR HTXDO :KLOH WKH PRGHO GRHV XWLOL]H WKH FRUUHODWLRQ EHWZHHQ D FRQWURO VXEMHFWfV WLPH RQH DQG WLPH WZR SHUIRUPDQFH WKH UHVXOWLQJ VWDQGDUG HUURU RI WKH GLIIHUHQFH WHQGHG WR EH ODUJH UHODWLYH WR WKH REVHUYHG FKDQJH LQ VFRUHV RYHU WKH WHVW LQWHUYDOV FUHDWLQJ UHODWLYHO\ ZLGHU FRQILGHQFH LQWHUYDOV HVSHFLDOO\ DW WKH ODWHU WHVWLQJ VHVVLRQV DQG QHFHVVDULO\ PDNLQJ GHWHFWLRQ RI WUXH FKDQJH PRUH GLIILFXOW ,Q D PRGHO QRW SUHYLRXVO\ H[DPLQHG LQ WKH OLWHUDWXUH LQ WKH FRQWH[W RI VHULDO QHXURSV\FKRORJLFDO SHUIRUPDQFH 0RGHO f WKH FKDQJH RI FRQFXVVHG SOD\HUV VFRUHV ZDV GLUHFWO\ FRPSDUHG WR WKH DYHUDJH FKDQJH VFRUHV RI WKH FRQWURO SDUWLFLSDQWV RYHU WKH VDPH WHVW LQWHUYDOV 7KH TXHVWLRQ DVNHG LQ WKLV PRGHO HVVHQWLDOO\ ZDV LV WKH FKDQJH REVHUYHG IRU D FRQFXVVHG SOD\HU ODUJH UHODWLYH WR WKH GLVWULEXWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ" /LNH WKH 5&, PRGHOV H[DPLQHG EHIRUH LW WKLV PRGHO FUHDWHG DQ LQGH[ RXWVLGH RI ZKLFK D FKDQJH LQ VFRUH IURP EDVHOLQH ZDV FRQVLGHUHG VLJQLILFDQW 8QOLNH WKH 5&, PRGHOV KRZHYHU LW GLG QRW DVVXPH WKDW WLPH RQH DQG WZR WHVW VFRUHV ZHUH HTXDO RU WKDW WKH VWDQGDUG GHYLDWLRQV RI WKRVH PHDVXUHV ZHUH HTXDO /LNH WKH SUHYLRXV PRGHOV XWLOL]DWLRQ RI 0RGHO XVLQJ D RQHRUPRUH YDULDEOH FXW SRLQW UHVXOWHG LQ D UHODWLYHO\ JRRG VHQVLWLYLW\ f DW KRXUV ,W DOVR DFKLHYHG D KLJK

PAGE 127

GHJUHH RI VSHFLILFLW\ GHVSLWH WKH OLEHUDO FXWSRLQW f 8QOLNH WKH SUHYLRXV PRGHOV KRZHYHU 0RGHO ZDV DOVR DEOH WR GLVWLQJXLVK EHWZHHQ FRQFXVVHG DQG FRQWURO SDUWLFLSDQWV DW WKH 'D\ WHVW VHVVLRQ DV ZHOO DV DW WKH 'D\ WHVW VHVVLRQ EXW QRW DW WKH 'D\ WHVW VHVVLRQ $OWKRXJK WKHUH ZDV DQ REYLRXV GURS LQ VHQVLWLYLW\ RYHU WLPH FRPSDUHG WR WKH KRXU WHVW VHVVLRQ ZKLFK PD\ KDYH EHHQ DFFRXQWHG IRU E\ WUXH UHFRYHU\ RI IXQFWLRQ LQ VRPH LQGLYLGXDOV WKH GLVFUHSDQF\ EHWZHHQ WKH VHQVLWLYLW\ DQG VSHFLILFLW\ YDOXHV DW WKRVH GD\V VXJJHVWHG WKDW WKLV PRGHO LV XVHIXO LQ WKH GHWHFWLRQ RI OLQJHULQJ FRJQLWLYH G\VIXQFWLRQ DW WHVW VHVVLRQV PRUH GLVWDQW IURP WKH FRQFXVVLYH HYHQW ([DPLQDWLRQ RI WKH VHQVLWLYLW\ RI LQGLYLGXDO YDULDEOHV IRU WKLV PRGHO VXJJHVWHG D GLVWLQFWLRQ EHWZHHQ WKRVH IXQFWLRQV WKDW ZHUH PRVW HDVLO\ GHWHFWHG LQ WKH LPPHGLDWH SHULRG KRXUVf IROORZLQJ FRQFXVVLRQV DQG WKRVH WKDW FRQWLQXHG WR PDQLIHVW GHWHFWDEOH HQGXULQJ FKDQJH DW WKH ODWHU WHVW VHVVLRQV 'D\V DQG f 6SHFLILFDOO\ DW WKH KRXU WHVW VHVVLRQ +RSNLQV 7RWDO VFRUH f +RSNLQV 'HOD\ VFRUH f DQG 'LJLW 6SDQ )RUZDUG f ZHUH WKH PRVW VHQVLWLYH LQGLYLGXDO YDULDEOHV 7KLV VXJJHVWHG WKDW VKRUWn WHUP YHUEDO PHPRU\ ZDV DIIHFWHG LQ WKH KLJKHVW QXPEHU RI SDUWLFLSDQWV LPPHGLDWHO\ IROORZLQJ FRQFXVVLRQ 7KLV LV QRW WR VD\ WKDW D FKDQJH LQ RWKHU IXQFWLRQV ZDV QRW GHWHFWDEOH E\ WKH PRGHO 7UDLOV $ WLPH DV ZHOO DV *URRYHG 3HJERDUG GRPLQDQW DQG QRQGRPLQDQWf FRPSOHWLRQ WLPHV DOVR SURYLGHG D VPDOOHU GHJUHH RI VHQVLWLYLW\ DW WKH KRXU WHVW VHVVLRQ :KLOH ZRUNLQJ PHPRU\ DV LQGLFDWHG E\ 'LJLW 6SDQ )RUZDUG FRQWLQXHG WR SURYLGH D GHJUHH RI VHQVLWLYLW\ DW WKH 'D\ DQG 'D\ WHVW VHVVLRQV LW ZDV PRUH RIWHQ WKRVH YDULDEOHV PHDVXULQJ SURFHVVLQJ VSHHG DQG VSHHGHG PRWRU GH[WHULW\ WKDW SURYLGHG WKH JUHDWHVW GHJUHH RI VHQVLWLYLW\ IRU WKLV PRGHO HVSHFLDOO\ DW WKH 'D\ WHVW VHVVLRQ 7UDLOV $ VHQVLWLYLW\ *URRYHG 3HJERDUG QRQGRPLQDQW VHQVLWLYLW\ f

PAGE 128

7KHVH ILQGLQJV ZHUH FRQVLVWHQW ZLWK WKRVH UHSRUWHG E\ +LQWRQ%D\UH HW DO f 0DGGRFNV f DQG %DUWK f DOO RI ZKRP IRXQG WKDW SURFHVVLQJ VSHHG UHDFWLRQ WLPH DQG DWWHQWLRQ FRQWLQXHG WR VKRZ VRPH GHFUHPHQW LQ WKH GD\V IROORZLQJ FRQFXVVLRQ GHVSLWH LQ VRPH FDVHV WKH UHVROXWLRQ RI VXEMHFWLYH QHXURORJLFDO IHDWXUHV VXFK DV FRQIXVLRQ QDXVHD DQG KHDGDFKH $8& PHDVXUHV IRU 0RGHO LQGLFDWHG IDLUO\ UREXVW GHWHFWLRQ DELOLW\ DW WKH KRXU WHVW VHVVLRQ f D VLJQLILFDQWO\EHWWHUWKDQFKDQFH DELOLW\ WR GLVWLQJXLVK EHWZHHQ WKH JURXSV DW 'D\ f DQG DOVR VLJQLILFDQW FODVVLILFDWLRQ DELOLW\ DW 'D\ f 2I DOO PRGHOV WHVWHG DV SDUW RI WKLV VWXG\ 0RGHO \LHOGHG WKH EHVW FODVVLILFDWLRQ UDWH DW KRXUV DQG DW 'D\ 5HJUHVVLRQ 0RGHOV 6LPLODU FODVVLILFDWLRQ UDWHV ZHUH DFKLHYHG XVLQJ D VLPSOH OLQHDU UHJUHVVLRQ PRGHO 0RGHO f ZKHUH EDVHOLQH SHUIRUPDQFH ZDV XVHG WR SUHGLFW UHWHVW SHUIRUPDQFH DW WKH YDULRXV WHVW LQWHUYDOV DQG WKH GLIIHUHQFH EHWZHHQ REVHUYHG DQ SUHGLFWHG VFRUHV ZDV DQDO\]HG WR GHWHUPLQH LI WKH GLIIHUHQFH ZDV ODUJH UHODWLYH WR WKH UHVLGXDOV RI WKH FRQWURO SRSXODWLRQ 7KLV PHWKRG UHVXOWHG LQ JRRG VHQVLWLYLW\ fVSHFLILFLW\ f UDWHV DW WKH KRXU WHVW VHVVLRQ XVLQJ WKH RQHYDULDEOH FXWSRLQW PXFK OLNH WKRVH REVHUYHG IRU 0RGHO $OVR VLPLODU WR 0RGHO WKH XVH RI WKH UHJUHVVLRQ HTXDWLRQ PRGHO H[KLELWHG JRRG FODVVLILFDWLRQ UDWHV DW WKH 'D\ DQG 'D\ WHVW VHVVLRQV ZKLFK ZDV LQGLFDWHG E\ WKH $8& PHDVXUHV IRU WKRVH WLPH LQWHUYDOV DQG UHVSHFWLYHO\f $JDLQ LQ H[DPLQLQJ ZK\ WKLV PRGHO ZDV VXSHULRU WR WKH ILUVW LQ DFFXUDWHO\ FODVVLI\LQJ FRQFXVVHG DQG QRQFRQFXVVHG LQGLYLGXDOV DW WKRVH GLVWDO VHVVLRQV LW ZDV QRWHG WKDW ZKLOH WKH VHQVLWLYLW\ DW WKH RQHYDULDEOH FXW SRLQW DW HDFK WHVW LQWHUYDO GLG VHH D SUHGLFWDEOH GHFUHDVH WKH

PAGE 129

FRUUHVSRQGLQJ VSHFLILFLW\ UDWHV UHPDLQHG UHODWLYHO\ KLJK DOORZLQJ IRU UHODWLYHO\ DFFXUDWH GLVWLQFWLRQ EHWZHHQ WKH JURXSV XVLQJ HYHQ D OLEHUDO LGHQWLILFDWLRQ FULWHULRQ 7KH ILQDO PRGHO H[DPLQHG DV SDUW RI WKH FXUUHQW VWXG\ LQYROYHG WKH XVH RI D PXOWLSOH UHJUHVVLRQ DQDO\VLV 3UHYLRXV UHVHDUFKHUV KDYH LQGLFDWHG WKDW FHUWDLQ GHPRJUDSKLF YDULDEOHV LQFOXGLQJ DJH DQG WHVWUHWHVW LQWHUYDO FRQWULEXWHG VLJQLILFDQWO\ WR WKH SUHGLFWLRQ RI UHWHVW VFRUHV DOORZLQJ IRU VPDOOHU VWDQGDUG GHYLDWLRQV DQG E\ GHILQLWLRQ VPDOOHU FKDQJH LQGLFHV 7HPNLQ HW DO f 7KLV DOORZHG IRU PRUH VHQVLWLYLW\ WR FKDQJH D SURSHUW\ ZKLFK LV YDOXDEOH LQ GHWHFWLQJ WKH OLQJHULQJ HIIHFWV RI FRQFXVVLRQ )RU WKH FXUUHQW VWXG\ WKH XVH RI DJH ZDV QRW D YLDEOH GHPRJUDSKLF SUHGLFWRU VLQFH WKH UDQJH RI SDUWLFLSDQWVf DJHV ZDV OLPLWHG +RZHYHU LW ZDV K\SRWKHVL]HG EDVHG RQ SUHYLRXV ILQGLQJV UHJDUGLQJ WKH FXUUHQW GDWDEDVH &ROOLQV HW DK f WKDW KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI SUHYLRXV FRQFXVVLRQ PD\ KDYH VRPH LPSDFW RQ WKH LQFLGHQFH RI FRQFXVVLRQ DQG SRVVLEO\ UHWHVW SHUIRUPDQFH 7KHUHIRUH WKH\ ZHUH LQFOXGHG DORQJ ZLWK EDVHOLQH SHUIRUPDQFH LQ VWHSZLVH PXOWLSOH UHJUHVVLRQ DQDO\VHV RI HDFK YDULDEOH RYHU HDFK WHVW VHVVLRQ +RZHYHU WKH ORZ OLVWZLVH Q SUHFOXGHG D UHOLDEOH DQDO\VLV RI WKLV PRGHO LQ WKH FXUUHQW VWXG\ $W WKH KRXU WHVW VHVVLRQ WKH WRWDO OLVWZLVH SRSXODWLRQ RI FRQWURO VXEMHFWV ZDV D QXPEHU VXEVWDQWLDOO\ EHORZ WKDW QHFHVVDU\ WR DFKLHYH UHOLDEOH UHVXOWV LQ WHUPV RI DFFXUDWH UHJUHVVLRQ PRGHO JHQHUDWLRQ 7KHUH ZHUH IHZHU OLVWZLVH FRQWURO SDUWLFLSDQWV DW HDFK VXFFHVVLYH WHVW VHVVLRQ PDNLQJ PXOWLSOH UHJUHVVLRQ DQDO\VLV HYHQ OHVV UHOLDEOH DW WKRVH LQWHUYDOV ,W ZDV WKHUHIRUH QRW SUDFWLFDO WR FRPSDUH WKH GHWHFWLRQ UDWH RI WKLV PRGHO WR WKH RWKHUV VWXGLHG 1HYHUWKHOHVV WKLV DQDO\VLV GLG UHYHDO VRPH LQWHUHVWLQJ VLJQLILFDQW ILQGLQJV GHVSLWH WKH ORZ SRZHU

PAGE 130

$W WKH KRXU UHWHVW VHVVLRQ WKH RQO\ VLJQLILFDQW SUHGLFWRU RI SHUIRUPDQFH IRU DQ\ YDULDEOH ZDV EDVHOLQH SHUIRUPDQFH ZLWK QR VLJQLILFDQW SUHGLFWRUV GHWHFWHG IRU WKUHH RI WKH QLQH YDULDEOHV $W WKH 'D\ WHVW VHVVLRQ D KLVWRU\ RI OHDUQLQJ GLVDELOLW\ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU 7UDLOV % ZKLOH KLVWRU\ RI SUHYLRXV FRQFXVVLRQ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU WRWDO &2:$ VFRUH +LVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG EDVHOLQH SHUIRUPDQFH ERWK FRQWULEXWHG VLJQLILFDQWO\ WR SUHGLFWLRQ RI *URRYHG 3HJERDUG 1RQGRPLQDQW KDQG FRPSOHWLRQ WLPH $W 'D\ D KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG EDVHOLQH SHUIRUPDQFH HQWHUHG VLJQLILFDQWO\ LQWR WKH UHJUHVVLRQ HTXDWLRQV IRU +RSNLQV 7RWDO VFRUH DQG 7UDLOV % +LVWRU\ RI OHDUQLQJ GLVDELOLW\ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU *URRYHG 3HJERDUG 'RPLQDQW DQG 1RQGRPLQDQW WLPHV DW WKDW WHVW VHVVLRQ )LQDOO\ IRU WKH 'D\ WHVW VHVVLRQ KLVWRU\ RI FRQFXVVLRQ ZDV WKH RQO\ VLJQLILFDQW SUHGLFWRU IRU &2:$ WRWDO VFRUH $OWKRXJK WKHVH SUHGLFWRU YDULDEOHV ZHUH IRXQG WR EH VWDWLVWLFDOO\ VLJQLILFDQW LQ WKH UHJUHVVLRQ PRGHOV EDVHG RQ WKH FRQWURO SRSXODWLRQ UHWHVW VFRUHV WKH\ GLG QRW JHQHUDOO\ SURYLGH VXEVWDQWLDOO\ LPSURYHG VHQVLWLYLW\ ZKHQ DSSOLHG WR WKH FRQFXVVHG SDUWLFLSDQWVf GDWD UHODWLYH WR WKH RWKHU PRGHOV VWXGLHG 7KH RQO\ H[FHSWLRQ WR WKLV ZDV WKH XVH RI KLVWRU\ RI FRQFXVVLRQ LQ WKH UHJUHVVLRQ PRGHO IRU &2:$ WRWDO VFRUH DW WKH 'D\ WHVW VHVVLRQ ,Q WKLV LQVWDQFH WKH VHQVLWLYLW\ YDOXH RI DSSHDUHG WR EH VXEVWDQWLDOO\ EHWWHU WKDQ WKH VHQVLWLYLW\ YDOXHV XVLQJ WKH RWKHU PRGHOV IRU WKH VDPH YDULDEOH ZKLFK UDQJHG IURP WR 2YHUDOO WKH XVH RI WKHVH SUHGLFWRU YDULDEOHV LQ WKH FXUUHQW VWXG\ FDQQRW EH FRQVLGHUHG UHOLDEOH GXH WR WKH ORZ QXPEHU RI LQGLYLGXDOV ZKR ZHUH SRVLWLYH IRU HDFK

PAGE 131

,Q VXPPDU\ DOO PRGHOV H[DPLQHG DV SDUW RI WKH FXUUHQW VWXG\ ZHUH JHQHUDOO\ JRRG DW GHWHFWLQJ VLJQLILFDQW FKDQJH ZLWKLQ WKH KRXU SHULRG IROORZLQJ FRQFXVVLRQ XVLQJ WKH RQHYDULDEOH VLJQLILFDQFH FULWHULRQ ZLWK HDFK VXFFHVVLYH PRGHO SURYLGLQJ VOLJKWO\ EHWWHU FODVVLILFDWLRQ UDWHV DV H[SHFWHG 2YHUDOO WKH FODVVLILFDWLRQ UDWHV IRU HDFK ZHUH FRPSDUDEOH WR WKRVH IRXQG E\ %DUU DQG 0F&UHD f LQ H[DPLQLQJ WKH 6$& ZLWK DQ 5&, DQG D OLQHDU UHJUHVVLRQ PRGHO ,Q WKH DEVHQFH RI EDVHOLQH PHDVXUHV RQ DQ LQGLYLGXDO IRRWEDOO SOD\HU 0RGHO LQGLFDWHG WKDW FRPSDULVRQ RI SHUIRUPDQFH DFURVV WKH YDULDEOHV VWXGLHG FDQ EH FRPSDUHG WR DQ DSSURSULDWH QRUPDWLYH SRSXODWLRQ DQG GDWD FROOHFWHG DV SDUW RI WKLV VWXG\ FDQ EH XVHG IRU VXFK SXUSRVHV $ VLJQLILFDQW GLIIHUHQFH IURP EDVHOLQH RQ MXVW RQH YDULDEOH WKH FKRVHQ FODVVLILFDWLRQ FXW SRLQWf UHVXOWHG LQ DQ DFFHSWDEOH OHYHO RI VHQVLWLYLW\ +RZHYHU WKH FRUUHVSRQGLQJ VSHFLILFLW\ UDWH ZDV TXLWH ORZ KLJKOLJKWLQJ WKH QHHG WR GHILQH D OHYHO RI VSHFLILFLW\ WKDW LV DFFHSWDEOH WR WKH H[DPLQLQJ KHDOWK SURIHVVLRQDO 7KLV PD\ XOWLPDWHO\ OHDG WR ZLWKKROGLQJ WKH SOD\HU IURP UHWXUQLQJ WR DFWLRQ ZKHQ WKLV LV LQ IDFW QRW QHFHVVDU\ $V ZDV PHQWLRQHG DERYH WKRXJK HUULQJ RQ WKH VLGH RI FDXWLRQ PD\ SUHYHQW FDWDVWURSKLF HIIHFWV :KHQ RQH KDV WKH EHQHILW RI EDVHOLQH WHVW PHDVXUHV D FRPSDULVRQ RI UHWHVW SHUIRUPDQFH WR WKDW OHYHO LV REYLRXVO\ PRUH DSSURSULDWH $V 0RGHO LQGLFDWHG VHQVLWLYLW\ DW WKDW WHVW LQWHUYDO UHWDLQHG FRPSDUDEOH YDOXH WR 0RGHO EXW H[KLELWHG VXEVWDQWLDOO\ LQFUHDVHG VSHFLILFLW\ 1HLWKHU RI WKHVH PHWKRGV KRZHYHU DSSHDUHG WR KDYH DQ\ VLJQLILFDQW XVH LQ GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG QRQFRQFXVVHG SOD\HUV DW VXEVHTXHQW WHVW VHVVLRQV 2QH UHDVRQ IRU WKLV PD\ EH WKDW WKHUH ZHUH QR GLIIHUHQFHV EHWZHHQ WKH FRQFXVVHG DQG QRQFRQFXVVHG SOD\HUV DW WKRVH WLPHV 7KLV DVVXPSWLRQ PD\ OHDG RQH WR EHOLHYH WKDW DQ\ FRJQLWLYH GHFOLQH VXIIHUHG DV D UHVXOW RI WKH FRQFXVVLRQ LWVHOI KDG UHVROYHG

PAGE 132

7KH RWKHU PRGHOV WHVWHG HDFK SURYLGHG DFFHSWDEOH VHQVLWLYLW\ OHYHOV DW WKH KRXU WHVW VHVVLRQ ZLWK VXFFHVVLYHO\ EHWWHU FODVVLILFDWLRQ DFFXUDF\ DV SUHGLFWHG ZKLFK ZDV LQGLFDWHG E\ WKH $8& PHDVXUHV ,Q H[DPLQLQJ WKH LPSURYHPHQW LQ FODVVLILFDWLRQ DFFXUDF\ LW ZDV DSSDUHQW PRVW REYLRXVO\ DW WKH KRXU WHVW VHVVLRQVf WKDW WKLV ZDV LQ SDUW GXH WR DQ LQFUHDVH LQ VHQVLWLYLW\ EXW ZDV ODUJHO\ D IXQFWLRQ RI DQ LQFUHDVH LQ VSHFLILFLW\ ZKLFK FUHDWHG D JUHDWHU GLVWLQFWLRQ EHWZHHQ WKH JURXSV 7KLV QDWXUDOO\ IROORZHG IURP WKH XVH RI WKH PDWFKHG FRQWURO SRSXODWLRQ WHVWHG RYHU WKH VDPH WLPH LQWHUYDOV ZKR ZHUH E\ GHILQLWLRQ DVVXPHG QRW WR EH VXIIHULQJ IURP FRJQLWLYH LPSDLUPHQW GXH WR FRQFXVVLYH LQMXU\ 7KLV LQFUHDVH LQ VHSDUDWLRQ EHWZHHQ WKH JURXSV PD\ LQGLFDWH WKDW WKH PRGHOV ZKLFK ZHUH EDVHG DFFRUGLQJ WR WKHLU UHVSHFWLYH PHWKRGRORJLHV RQ GDWD JDWKHUHG IURP FRQWURO VXEMHFWV DUH PRUH WROHUDQW RI OLEHUDO FRQILGHQFH LQWHUYDOV 7KDW LV WKH XVH RI D b FRQILGHQFH LQWHUYDO ]YDOXH RI PD\ SURYLGH PRUH GHWHFWLRQ SRZHU IRU WKH PRGHOV ZLWKRXW VXEVWDQWLDOO\ LPSDFWLQJ WKH VSHFLILFLW\ DV ZDV WKH FDVH LQ WKH VLPSOH FRPSDULVRQ WR D QRUPDWLYH SRSXODWLRQ 0RGHO f RU FRPSDULVRQ WR EDVHOLQH SHUIRUPDQFH UHODWLYH WR WKH VWDQGDUG GHYLDWLRQ RI WKH QRUPDWLYH SRSXODWLRQ 0RGHO f 3UHYLRXV VWXGLHV KDYH XVHG WKLV FRQILGHQFH LQWHUYDO YDOXH %DUU DQG 0F&UHD 7HPNLQ HW DO f DQG VXJJHVWHG WKDW WKH FRVW WR VSHFLILFLW\ UHSUHVHQWHG DQ DFFHSWDEOH ORVV $V ZDV K\SRWKHVL]HG QHLWKHU WKH EDVLF FRPSDULVRQ PRGHOV 0RGHOV DQG f WKH 5&, PRGHO 0RGHO f QRU WKH 5&, PRGHO ZLWK SUDFWLFH FRUUHFWLRQ 0RGHO f ZHUH DFFXUDWH DW GHWHFWLQJ HQGXULQJ FRJQLWLYH VHTXHODH DQG DFFXUDWHO\ GLVWLQJXLVKLQJ EHWZHHQ FRQFXVVHG DQG QRQFRQFXVVHG SOD\HUV DW WKH 'D\ 'D\ DQG 'D\ WHVW VHVVLRQV DQG WKHUHIRUH DSSHDU QRW WR SURYLGH UHOLDEOH PHDQV IRU SURYLGLQJ UHWXPWRSOD\ FULWHULD IRU

PAGE 133

FRQFXVVHG IRRWEDOO SOD\HUV DW WKRVH WLPHV 7KH RQO\ PRGHOV WR SURYLGH VWDWLVWLFDOO\ VLJQLILFDQW FODVVLILFDWLRQ UDWHV DW 'D\ ZHUH WKH VLPSOH OLQHDU UHJUHVVLRQ PRGHO 0RGHO f DQG WKH DQDO\VLV RI FKDQJH VFRUHV UHODWLYH WR WKH GLVWULEXWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ 0RGHO f ,Q DGGLWLRQ WKHVH WZR PRGHOV ZHUH WKH RQO\ VWDWLVWLFDO PHWKRGV WR VXFFHVVIXOO\ GLVWLQJXLVK EHWZHHQ WKH WZR JURXSV DW WKH 'D\ WHVW LQWHUYDO DOWKRXJK WKLV UHVXOW VKRXOG EH YLHZHG ZLWK VRPH FDXWLRQ GXH WR WKH ORZ QXPEHU RI OLVWZLVH SDUWLFLSDQWV DW WKDW WHVW LQWHUYDO ,W LV QRWDEOH WKDW GHVSLWH WKH UHODWLYHO\ ORZ SRZHU WKH PRGHOV GLG SURYLGH VWDWLVWLFDOO\ VLJQLILFDQW FODVVLILFDWLRQ DFFXUDF\ DV LQGLFDWHG E\ WKHLU UHVSHFWLYH $8& YDOXHV $V ZDV WKH FDVH IRU ERWK 7HPNLQ HW DO f DQG %DUU DQG 0F&UHD f WKH XVH RI D OLQHDU UHJUHVVLRQ PRGHO ZDV PRUH VHQVLWLYH WKDQ HLWKHU 5&, PRGHO 7KLV PRGHO ZKLFK LQKHUHQWO\ SURYLGHV FRUUHFWLRQ IRU ERWK SUDFWLFH HIIHFWV DQG UHJUHVVLRQ WR WKH PHDQ UHVXOWHG LQ QDUURZHU FRQILGHQFH LQWHUYDOV GXH WR WKH PRUH DFFXUDWH PHDQV RI SUHGLFWLQJ UHWHVW VFRUHV ZKLFK ZDV UHIOHFWHG LQ WKH VWDQGDUG HUURU RI WKH UHVLGXDOV WKH FRPSDUDWRUf IRU WKH FRQWURO SRSXODWLRQ 2YHUDOO LW SURYLGHG WKH EHVW FODVVLILFDWLRQ DFFXUDF\ DW WKH 'D\ WHVW VHVVLRQ DV LQGLFDWHG E\ WKH $8& PHDVXUH 7KH PRVW DFFXUDWH FODVVLILFDWLRQ PRGHO RYHUDOO WKRXJK ZDV 0RGHO LQ ZKLFK WKH FKDQJH LQ D FRQFXVVHG SOD\HUfV SHUIRUPDQFH RQ HDFK YDULDEOH ZDV FRPSDUHG WR WKH DYHUDJH FRQWURO FKDQJH RQ WKH YDULDEOH UHODWLYH WR WKH GLVWULEXWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ 7KLV PRGHO WRR HVVHQWLDOO\ IDFWRUHG LQ WKH HIIHFWV RI SUDFWLFH RYHU WKH UHSHDWHG WHVWLQJ LQWHUYDOV DFFRXQWHG IRU WKH YDULDELOLW\ RI WKHVH SUDFWLFH HIIHFWV LQ WKH FRQWURO SRSXODWLRQ DQG DOORZHG IRU WKH GLUHFW UDWKHU WKDQ LQGLUHFW FRPSDULVRQ RI D FRQFXVVHG SOD\HUfV FKDQJH LQ VFRUH IURP EDVHOLQH WR WKH FRQWURO SRSXODWLRQ 8VLQJ WKLV PRGHO UHODWLYHO\ JRRG

PAGE 134

DFFXUDF\ FODVVLILFDWLRQ UDWHV ZHUH REVHUYHG IRU WKH KRXU 'D\ DQG 'D\ WHVW VHVVLRQV $ VWDWLVWLFDO FRPSDULVRQ RI 0RGHO DQG 0RGHO DW WKRVH WHVW LQWHUYDOV UHYHDOHG WKDW WKH\ ZHUH QRW VLJQLILFDQWO\ GLIIHUHQW LQ WHUPV RI FODVVLILFDWLRQ DFFXUDF\ +RZHYHU $8& YDOXHV VXJJHVWHG D VOLJKWO\ JUHDWHU DFFXUDF\ DW KRXUV DQG HVSHFLDOO\ DW 'D\ PDNLQJ LW PRUH YDOXDEOH LQ GHWHFWLQJ SRVVLEOH HQGXULQJ FRJQLWLYH GHFOLQH ZKLFK DV QRWHG DERYH DSSHDUHG PRVW OLNHO\ WR PDQLIHVW LQ VORZHU SURFHVVLQJ VSHHG WLPH 7KH UHVXOWV REWDLQHG LQ WKH FXUUHQW VWXG\ DUH SDUWLFXODUO\ VDOLHQW WR WKH GHFLVLRQ UHJDUGLQJ D SOD\HUVf ILWQHVV IRU UHWXUQ WR SOD\ 2YHU WKH SDVW WZR GHFDGHV DV PRUH DWWHQWLRQ KDV EHHQ SDLG WR WKH SRWHQWLDOO\ GHYDVWDWLQJ VKRUWWHUP DV ZHOO DV ORQJWHUP HIIHFWV RI FRQFXVVLRQ D QXPEHU RI SURSRVLWLRQV DQG JXLGHOLQHV KDYH HPHUJHG FRQFHUQLQJ WKH FULWHULD WKDW VKRXOG EH PHW EHIRUH D SOD\HU LV GHHPHG UHDG\ WR UHWXUQ WR SOD\ $V HDUO\ DV &DQWX KDG SURYLGHG VRPH JXLGHOLQHV IRU WKH FODVVLILFDWLRQ RI FRQFXVVLRQ VHYHULW\ ZKLFK OHG WR VRPH UHFRPPHQGDWLRQV IRU UHWXPWRSOD\ JXLGHOLQHV 7KHVH ZHUH ODWHU UHYLVHG LQ ,Q WKH &RORUDGR 0HGLFDO 6RFLHW\ UHOHDVHG DQRWKHU V\VWHP IRU WKH JUDGLQJ RI FRQFXVVLRQ VHYHULW\ DQG ZLWK LW IXUWKHU UHFRPPHQGDWLRQV IRU UHWXPWR SOD\ GHFLVLRQPDNLQJ 7KH &DQWX UHFRPPHQGDWLRQV DSSHDUHG WR EH WKH PRVW FRQVHUYDWLYH ZLWK D FDOO IRU UHPRYDO IURP SOD\ IRU DW OHDVW RQH V\PSWRPIUHH ZHHN DIWHU VXIIHULQJ MXVW RQH *UDGH FRQFXVVLRQ 7KH &RORUDGR JXLGHOLQHV RQ WKH RWKHU KDQG UHFRPPHQGHG MXVW D PLQXWH SHULRG GXULQJ ZKLFK VXEMHFWLYH VLJQV RI FRQFXVVLRQ KDG UHVROYHG EHIRUH UHWXPWRSOD\ 7KH FULWHULD IRU UHWXPWRSOD\ ZHUH PRUH UHVWULFWLYH IRU UHSHDWHG *UDGH FRQFXVVLRQV UDQJLQJ IURP D RQHZHHN V\PSWRPIUHH KLDWXV WR SUHVFULSWLRQ IRU UHPRYDO IURP SOD\ IRU WKH VHDVRQ LI V\PSWRPV HQGXUHG )RU *UDGH DQG *UDGH FRQFXVVLRQV &DQWXfV UHFRPPHQGDWLRQV ZHUH DJDLQ WKH PRVW FRQVHUYDWLYH

PAGE 135

UDQJLQJ IURP D PLQLPXP RI RQH V\PSWRPIUHH ZHHN IRU WKH ILUVW *UDGH FRQFXVVLRQ WR D PLQLPXP RI RQH V\PSWRPIUHH PRQWK IRU UHSHDWHG *UDGH FRQFXVVLRQ RU WKH ILUVW *UDGH FRQFXVVLRQ .HOO\ DQG 5RVHQEHUJ f 7KH &RORUDGR JXLGHOLQHV ZHUH VOLJKWO\ OHVV UHVWULFWLYH EXW VLPLODU WR WKRVH RI &DQWX ,Q WKH &RORUDGR JXLGHOLQHV ZHUH UHDGGUHVVHG E\ WKH $PHULFDQ $FDGHP\ RI 1HXURORJ\ $$1f ZKR WKHQ UHOHDVHG WKHLU RZQ YHUVLRQ RI UHWXPWRSOD\ JXLGHOLQHV 7KHVH ZHUH VXEVHTXHQWO\ HQGRUVHG E\ D QXPEHU RI RUJDQL]DWLRQV DVVRFLDWHG ZLWK PHGLFLQH LQ VSRUWV )XHUVW f EXW DGYRFDWHG RQO\ DV fSUDFWLFH RSWLRQVf UDWKHU WKDQ XQLYHUVDO JXLGHOLQHV 7KHVH SUDFWLFH RSWLRQV QRZ DSSHDU WR EH WKH PRVW ZLGHO\ XWLOL]HG SDUDPHWHUV IRU UHWXPWRSOD\ GHFLVLRQ PDNLQJ 7KH\ DUH KRZHYHU WKH OHDVW UHVWULFWLYH RI WKH PRVW FRPPRQO\ FLWHG JXLGHOLQHV )RU H[DPSOH WKH $$1 JXLGHOLQHV VXJJHVW WKDW D SOD\HU EH ZLWKKHOG IURP SUDFWLFH RU JDPHSOD\ IRU RQO\ V\PSWRPIUHH PLQXWHV LQ WKH HYHQW RI WKH ILUVW *UDGH FRQFXVVLRQ ,Q WKH FDVH RI PXOWLSOH *UDGH FRQFXVVLRQV RU RQH *UDGH FRQFXVVLRQ OHQJWK RI V\PSWRPIUHH KLDWXV ZDV UHFRPPHQGHG WR EH RQH ZHHN )XUWKHU RQO\ D RQHZHHN UHPRYDO IURP SOD\ ZDV UHFRPPHQGHG IRU D ILUVW *UDGH FRQFXVVLRQ JLYHQ WKH DSSDUHQW UHVROXWLRQ RI QHXURORJLF V\PSWRPV 5HVXOWV RI WKH FXUUHQW VWXG\ VXJJHVW WKDW QHXURSV\FKRORJLFDO WHVWV FDQ SURYLGH REMHFWLYH HYLGHQFH RI HQGXULQJ FRJQLWLYH VHTXHODH RI FRQFXVVLRQ HYHQ DW WKH RQHZHHN SHULRG $V QRWHG DERYH IRU WKHVH SDUWLFLSDQWV WKH PRVW VHQVLWLYH PHDVXUHV XVLQJ 0RGHOV DQG DW WKDW WHVW LQWHUYDO DSSHDU WR EH WKRVH DVVRFLDWHG ZLWK SURFHVVLQJ VSHHG DQG VSHHGHG SV\FKRPRWRU IXQFWLRQV *LYHQ WKLV LW LV UHFRPPHQGHG WKDW D QHXURSV\FKRORJLFDO EDWWHU\ LQFOXGLQJ PHDVXUHV RI WKHVH IXQFWLRQV EH XWLOL]HG LQ FRQMXQFWLRQ ZLWK WKRVH JXLGHOLQHV LQ HVWDEOLVKLQJ WKH UHVROXWLRQ RI FRQFXVVLRQLQGXFHG

PAGE 136

FRJQLWLYH GHFOLQH 2EYLRXVO\ WKH IXUWKHU FROOHFWLRQ RI FRQWURO QRUPDWLYHf GDWD ZLOO EH QHFHVVDU\ WR PDLQWDLQ DQ DSSURSULDWH FRPSDULVRQ IRU WKLV SXUSRVH /LPLWDWLRQV DQG ,PSOLFDWLRQV IRU )XWXUH 5HVHDUFK 7KH ODUJHVW OLPLWDWLRQ LQ WKH FXUUHQW VWXG\ ZDV REYLRXVO\ WKH ORZ QXPEHU RI SDUWLFLSDQWV RYHU WLPH GHVSLWH WKH FROOHFWLRQ RI GDWD RYHU D WZR\HDU SHULRG $WWULWLRQ HVSHFLDOO\ DPRQJ WKH FRQWURO SRSXODWLRQ OLNHO\ UHVXOWHG LQ OHVV DFFXUDWH SUHGLFWLRQ DQG FRPSDULVRQV DFURVV 0RGHOV 7KLV ZDV PRVW REYLRXV LQ WKH DQDO\VLV RI SRWHQWLDO PXOWLSOH UHJUHVVLRQ PRGHOV ZKLFK ZHUH UHQGHUHG XQUHOLDEOH DQG XOWLPDWHO\ QRW D YLDEOH SUHGLFWLRQ RSWLRQ LQ WKH FXUUHQW VWXG\ 'HVSLWH WKLV WKHUH DSSHDUHG WR EH VRPH UHODWLRQVKLSV EHWZHHQ KLVWRU\ RI OHDUQLQJ GLVDELOLW\ DQG KLVWRU\ RI FRQFXVVLRQ LQ VHULDO WHVWLQJ XVLQJ WKH YDULDEOHV H[DPLQHG KHUH ,Q DGGLWLRQ WKH ORZ OLVWZLVH QXPEHU RI FRQWURO SDUWLFLSDQWV DOVR FUHDWHG WKH SRWHQWLDO IRU VNHZLQJ RI WKH GDWD E\ DQ\ SDUWLFXODU RXWOLHU 7KLV LQ IDFW PD\ KDYH FRQWULEXWHG WR WKH LQDELOLW\ RI DQ\ RI WKH PRGHOV WR VXFFHVVIXOO\ GLVWLQJXLVK EHWZHHQ WKH FRQFXVVHG DQG FRQWURO LQGLYLGXDOV DW WKH 'D\ WHVW VHVVLRQ &HUWDLQO\ DV QRWHG DERYH GDWD IRU WKH 'D\ WHVW VHVVLRQ VKRXOG EH YLHZHG ZLWK VRPH FDXWLRQ EHFDXVH RI WKH ORZ OLVWZLVH Q WKRXJK VLJQLILFDQW ILQGLQJV ZHUH REVHUYHG GHVSLWH WKH ORZ SRZHU ,W PXVW EH QRWHG WKDW WKH QXPEHU RI SDUWLFLSDQWV LQ WKH FXUUHQW VWXG\ ZDV JHQHUDOO\ VLPLODU WR WKRVH REWDLQHG LQ SUHYLRXV VWXGLHV H[DPLQLQJ WKH XVH RI QHXURSV\FKRORJLFDO WHVW PHDVXUHV LQ D FRQFXVVLRQ SRSXODWLRQ %DUWK HW DO +LQWRQ %D\UH HW DK 0DGGRFNV DQG 6DOLQJ f 'XH WR WKH WLPHLQWHQVLYH UHTXLUHPHQWV RI D SURVSHFWLYH UHVHDUFK GHVLJQ ZLWK VLPLODU SRSXODWLRQV WKH FROOHFWLRQ RI ODUJHU GDWD VHWV ZLOO UHTXLUH H[WHQVLYH FROODERUDWLRQ EHWZHHQ UHVHDUFKHUV DFURVV PXOWLSOH VLWHV

PAGE 137

%\ LWV YHU\ QDWXUH RQH PD\ H[SHFW WKHUH WR EH D KLJKHU LQFLGHQFH RI FRQFXVVLRQ LQ WKH JDPH RI IRRWEDOO HVSHFLDOO\ RI WKH *UDGH W\SH *LYHQ WKH HIIHFW WKDW UHSRUWLQJ WR D FRDFK RU WUDLQHU WKDW RQH KDV VXIIHUHG D FRQFXVVLRQ PD\ KDYH LH UHVWULFWLRQ LQ SOD\LQJ WLPHf LW ZRXOG QRW EH VXUSULVLQJ IRU SOD\HUV QRW WR DFNQRZOHGJH KDYLQJ UHFHLYHG D KLW WKDW FDXVHG VRPH PRPHQWDU\ FRQIXVLRQ RU GLVRULHQWDWLRQ 7KH LGHQWLILFDWLRQ RI FRQFXVVHG SOD\HUV ZDV WKHUHIRUH UHOLDQW RQ REVHUYDWLRQ E\ WHDP WUDLQHUV RU FRDFKHV ZKLFK FDQ RQO\ EH DVVXPHG WR EH WKH PRUH REYLRXV FDVHV DQG WKHUHIRUH D UHGXFHG VDPSOH RI WKH HQWLUH SRSXODWLRQ RI FRQFXVVHG LQGLYLGXDOV )XWXUH UHVHDUFK VKRXOG LQFOXGH WKH FROOHFWLRQ RI D ODUJHU SRSXODWLRQ RI UHWHVW FRQWUROV :KLOH XVLQJ D \RNHG FRQWURO DSSURDFK GLG HQDEOH WKH FROOHFWLRQ RI FRQWURO SDUWLFLSDQWV ZKR ZHUH FDUHIXOO\ PDWFKHG RQ KLVWRU\ RI /' DQG KLVWRU\ RI FRQFXVVLRQ WKH DWWULWLRQ UDWH DPRQJ WKRVH LQGLYLGXDOV VXJJHVWV WKDW VHOHFWLQJ D UHSUHVHQWDWLYH VDPSOH RI SOD\HUV IURP D WHDP DQG DGPLQLVWHULQJ WKH VHULHV RI QHXURSV\FKRORJLFDO WHVWV RYHU WKH EDVHOLQH DQG UHWHVW VHVVLRQV ZRXOG DOVR SURYLGH WKH GDWD QHHGHG IRU WKH XWLOL]DWLRQ RI WKH PRGHOV H[DPLQHG 7KLV PD\ SURYLGH IRU D ODUJHU FRPSDULVRQ SRSXODWLRQ 7KLV PD\ DIIHFW WKH HYDOXDWLRQ RI DQ\ RI WKHVH LQGLYLGXDOV VKRXOG WKH\ UHFHLYH D FRQFXVVLRQ LQ WKH IXWXUH KRZHYHU ,Q RUGHU WR IXUWKHU DVVHVV WKH FRQWULEXWLRQ RI /' DQG KLVWRU\ RI FRQFXVVLRQ WR UHWHVW SHUIRUPDQFH HPSKDVLV VKRXOG EH SODFHG RQ UHFUXLWLQJ WKRVH LQGLYLGXDOV SRVLWLYH IRU WKHVH YDULDEOHV IRU XVH DV FRQWUROV LQ VHULDO WHVWLQJ ZLWK WKLV EDWWHU\ RI WHVWV *LYHQ WKH UHODWLYHO\ KLJKHU GHJUHH RI VSHFLILFLW\ DFKLHYHG ZLWK WKH PRUH HIIHFWLYH PRGHOV DQDO\VLV RI WKHLU VHQVLWLYLWLHV VKRXOG EH SHUIRUPHG XWLOL]LQJ D PRUH OLEHUDO FRQILGHQFH LQWHUYDO FULWHULD :KLOH D ]VFRUH RI ZDV XVHG LQ WKH FXUUHQW VWXG\ LQ RUGHU WR SURYLGH D EHWWHU EDODQFH LQ VHQVLWLYLW\ DQG VSHFLILFLW\ DQG WR EH PRUH FRQVLVWHQW

PAGE 138

ZLWK VWDWLVWLFDO OLWHUDWXUH D ]VFRUH RI UHSUHVHQWLQJ D b FRQILGHQFH LQWHUYDO PD\ SURYLGH D KLJKHU GHJUHH RI VHQVLWLYLW\ ZKLOH QRW GUDPDWLFDOO\ LPSDFWLQJ WKH VSHFLILFLW\ UDWH )XWXUH UHVHDUFK PD\ DOVR IRFXV RQ WKH LQWHJUDWLRQ RI GDWD FRQFHUQLQJ VXEMHFWLYH FRPSODLQWV RI SRVWFRQFXVVLYH V\PSWRPV $ EULHI UHYLHZ RI VHOIUHSRUWHG FRQFXVVLYH FRPSODLQWV LQ WKH FXUUHQW SRSXODWLRQ VXJJHVWHG WKDW VRPH VXEMHFWLYH FRPSODLQWV LQFOXGLQJ VOHHS GLIILFXOWLHV DQG IHHOLQJV RI GL]]LQHVV DPRQJ RWKHUV &ROOLQV HW DO f PD\ DOVR HQGXUH SDVW WKH WLPH GXULQJ ZKLFK PRVW FRQFXVVLYH FRPSODLQWV DUH H[SHFWHG WR KDYH UHVROYHG DQG PD\ FRQWULEXWH WR GHFOLQH LQ FRJQLWLYH SHUIRUPDQFH ,Q DGGLWLRQ RWKHU FRJQLWLYH YDULDEOHV ZHUH LQFOXGHG LQ WKH FXUUHQW EDWWHU\ DIWHU GDWD ZDV FROOHFWHG IRU WKLV VWXG\ 7KRVH PHDVXUHV LQFOXGHG WKH %ULHI 9LVXDO 0HPRU\ 7HVW DQG D VPHOO LGHQWLILFDWLRQ WHVW ([DPLQDWLRQ RI WKH FRQWULEXWLRQ RI WKHVH YDULDEOHV WR WKH LGHQWLILFDWLRQ RI SRVW FRQFXVVLYH FRJQLWLYH GHFOLQH DQG UHWXUQ WR EDVHOLQH PD\ SURYH XVHIXO LQ WKH IXWXUH &RQWULEXWLRQV RI WKH &XUUHQW 6WXG\ 2QH RI WKH DQFLOODU\ EHQHILWV RI WKLV VWXG\ LQFOXGHG LWV FRQWULEXWLRQ WR WKH HYDOXDWLRQ E\ WHDP WUDLQHUV DQG SK\VLFLDQV RI FRQFXVVLRQ LQ D FROOHJH IRRWEDOO SRSXODWLRQ DW VSHFLILF LQWHUYDOV DIWHU LQMXU\ ZKLFK PD\ EH KHOSIXO LQ GHWHUPLQLQJ UHWXPWRSOD\ VWDWXV $V WKH VWXG\ GHPRQVWUDWHG WKH XVH RI ERWK D OLQHDU UHJUHVVLRQ SUHGLFWLRQ PRGHO 0RGHO f DQG DQ DQDO\VLV RI FKDQJH IURP EDVHOLQH UHODWLYH WR WKH GLVWULEXWLRQ RI FKDQJH VFRUHV IRU WKH FRQWURO SRSXODWLRQ 0RGHO f SURYLGHG UHODWLYHO\ JRRG FODVVLILFDWLRQ DFFXUDF\ ZLWKLQ KRXUV DW 'D\ DQG DW 'D\ $V QRWHG DERYH WKHUH ZDV QRW D VWDWLVWLFDOO\ VLJQLILFDQW GLIIHUHQFH EHWZHHQ WKH PRGHOVf FODVVLILFDWLRQ DFFXUDF\ DOWKRXJK 0RGHO DSSHDUV WR KDYH SURYLGHG VRPHZKDW EHWWHU FODVVLILFDWLRQ DFFXUDF\ DW 'D\ ,W LV

PAGE 139

FOHDU KRZHYHU WKDW ORQJHUWHUP FRJQLWLYH HIIHFWV PD\ EH SUHVHQW IROORZLQJ FRQFXVVLRQ ZKLFK DUH QRW UHDGLO\ DSSDUHQW ZLWKRXW WKH DSSURSULDWH GDWD ZLWK ZKLFK WR FRPSDUH UHWHVW UHVXOWV 0RGHO OHQGV LWVHOI WR WKH SURYLVLRQ RI UHDGLO\ REVHUYDEOH VLJQLILFDQFH GHWHUPLQDWLRQ IRU HDFK YDULDEOH HVSHFLDOO\ IRU WKRVH QRW IDPLOLDU ZLWK WKH VWDWLVWLFV LQYROYHG LQ WKH UHJUHVVLRQ SUHGLFWLRQ $ ZRUNVKHHW GHVLJQHG WR SURYLGH DQDO\VLV RI UHWHVW SHUIRUPDQFH LQ LGHQWLILHG FRQFXVVHG SOD\HUV ZDV GHVLJQHG IRU XVH E\ WHDP WUDLQHUV DQG SK\VLFLDQV ZKR KDYH EHHQ WUDLQHG LQ WKH DGPLQLVWUDWLRQ RI WKLV EULHI QHXURSV\FKRORJLFDO EDWWHU\ LQ RUGHU WR SURYLGH GDWD IRU UHWXPWRSOD\ GHFLVLRQPDNLQJ 6LJQLILFDQFH GHWHUPLQDWLRQ SRLQWV IRU HDFK YDULDEOH ZDV EDVHG RQ WKH FRQWURO SRSXODWLRQ IURP WKH FXUUHQW VWXG\ XVLQJ 0RGHO VWDWLVWLFDO PHWKRGRORJ\ %HWWHU FODVVLILFDWLRQ DFFXUDF\ LV OLNHO\ ZLWK WKLV PRGHO ZLWK WKH DGGLWLRQ RI IXUWKHU FRQWURO SRSXODWLRQ GDWD FROOHFWLRQ 7KLV ZRUNVKHHW LV VKRZQ LQ ILJXUH

PAGE 140

3OD\HU 1DPH 0HDVXUH %DVHOLQH 3HUIRUPDQFH +RXU 3HUIRUPDQFH 'LIIHUHQFH &XWRII 6LJQLILFDQW +RSNLQV 7RWDO GHFUHDVH RI WRWDO ZRUGV UHFDOOHG 7UDLOV $ LQFUHDVH RI VHFRQGV 7UDLOV % LQFUHDVH RI VHFRQGV 'LJLWV )RUZDUG GHFUHDVH RI GLJLW UHFDOOHG 'LJLWV %DFN GHFUHDVH RI GLJLWV UHFDOOHG 3HJV 'RPLQDQW LQFUHDVH RI VHFRQGV 3HJV 1RQGRP !LQFUHDVH RI VHFRQGV &2:$ 7RWDO !B GHFUHDVH RI ZRUGV SURGXFHG +RSNLQV 'HOD\ GHFUHDVH RI ZRUGV UHFDOOHG )LQGLQJV DUH VLJQLILFDQW LI RQH RU PRUH YDULDEOHV DUH IRXQG WR EH VLJQLILFDQW 0HDVXUH %DVHOLQH 3HUIRUPDQFH 'D\ 3HUIRUPDQFH 'LIIHUHQFH &XWRII 6LJQLILFDQW +RSNLQV 7RWDO GHFUHDVH RI WRWDO ZRUGV UHFDOOHG 7UDLOV $ LQFUHDVH RI VHFRQGV 7UDLOV % !LQFUHDVH RI VHFRQGV 'LJLWV )RUZDUG GHFUHDVH RI GLJLW UHFDOOHG 'LJLWV %DFN GHFUHDVH RI GLJLWV UHFDOOHG 3HJV 'RPLQDQW LQFUHDVH RI VHFRQGV 3HJV 1RQGRP LQFUHDVH RI VHFRQGV &2:$ 7RWDO !B GHFUHDVH RI ZRUGV SURGXFHG +RSNLQV 'HOD\ GHFUHDVH RI ZRUGV UHFDOOHG

PAGE 141

0HDVXUH %DVHOLQH 3HUIRUPDQFH 'D\ 3HUIRUPDQFH 'LIIHUHQFH &XWRII 6LJQLILFDQW +RSNLQV 7RWDO GHFUHDVH RI WRWDO ZRUGV UHFDOOHG 7UDLOV $ FKDQJH RI VHFRQGV 7UDLOV % LQFUHDVH RI VHFRQGV 'LJLWV )RUZDUG GHFUHDVH RI GLJLW UHFDOOHG 'LJLWV %DFN GHFUHDVH RI GLJLWV UHFDOOHG 3HJV 'RPLQDQW LQFUHDVH RI VHFRQGV 3HJV 1RQGRP LQFUHDVH RI VHFRQGV &2:$ 7RWDO !B GHFUHDVH RI ZRUGV SURGXFHG +RSNLQV 'HOD\ GHFUHDVH RI ZRUGV UHFDOOHG )LQGLQJV DUH VLJQLILFDQW LI RQH RU PRUH YDULDEOHV DUH IRXQG WR EH VLJQLILFDQW )LJXUH ([DPSOH RI :RUNVKHHW IRU WKH 'HWHFWLRQ RI 2QJRLQJ (IIHFWV RI &RQFXVVLRQ LQ &ROOHJH )RRWEDOO 3OD\HUV

PAGE 142

5()(5(1&(6 $KPHG 6 %LHUOH\ 5 DQG 6KHLNK -, f 3RVWWUDXPDWLF DPQHVLD DIWHU FORVHG KHDG LQMXU\ $ UHYLHZ RI WKH OLWHUDWXUH DQG VRPH VXJJHVWLRQV IRU IXWXUH UHVHDUFK %UDLQ ,QMXU\ $OH[DQGHU 03 f 1HXURSV\FKLDWULF FRUUHODWHV RI SHUVLVWHQW SRVWFRQFXVVLYH V\QGURPH -RXUQDO RI +HDG 7UDXPD 5HKDELOLWDWLRQ %DUU :% DQG 0F&UHD 0 f 6HQVLWLYLW\ DQG VSHFLILFLW\ RI VWDQGDUGL]HG QHXURFRJQLWLYH WHVWLQJ LPPHGLDWHO\ IROORZLQJ VSRUWV FRQFXVVLRQ -RXUQDO RI WKH ,QWHUQDWLRQDO 1HXURSV\FKRORJLFDO 6RFLHW\ %DUWK -7 $OYHV :0 5\DQ 79 f 0LOG KHDG LQMXU\ LQ VSRUWV ,Q /HYLQ +6 (LVHQEHUJ +0 DQG %HQWRQ $/ HGVf 0LOG KHDG LQMXU\ 2[IRUG 8QLYHUVLW\ 3UHVV 1HZ
PAGE 143

%UDQGW f 7KH +RSNLQV 9HUEDO /HDUQLQJ 7HVW 'HYHORSPHQW RI D QHZ YHUEDO PHPRU\ WHVW ZLWK VL[ HTXLYDOHQW IRUPV 7KH &OLQLFDO 1HXURSV\FKRORJLVW %XVFK &5 DQG $OSHP +3 f 'HSUHVVLRQ DIWHU PLOG WUDXPDWLF EUDLQ LQMXU\ 1HXURSV\FKRORJ\ 5HYLHZ &DQWX 5& f *XLGHOLQHV IRU UHWXUQ WR FRQWDFW VSRUWV DIWHU D FHUHEUDO FRQFXVVLRQ 7KH 3K\VLFLDQ DQG 6SRUWVPHGLFLQH &DQWX 5& f &HUHEUDO FRQFXVVLRQ LQ VSRUWV 6SRUWV 0HGLFLQH &DQWX 5& DQG 9R\ 5 f 6HFRQG LPSDFW V\QGURPH D ULVN LQ DQ\ VSRUW 7KH 3K\VLFLDQ DQG 6SRUWVPHGLFLQH &KHOXQH *1DXJOH 5, DQG /HXGHUV + f ,QGLYLGXDO FKDQJH DIWHU HSLOHSV\ VXUJHU\ SUDFWLFH HIIHFWV DQG EDVHUDWH LQIRUPDWLRQ 1HXURSV\FKRORJ\ &KD '+ &KDQJ 0< &KHQ %< DQG &KDQJ ++ f 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF DQDO\VLV RI DQ HU\WKUREODVW PRUSKRORJLF VFRULQJ V\VWHP WR LPSURYH LGHQWLILFDWLRQ RI IHWDO FHOOV LQ PDWHUQDO EORRG 3UHQDWDO 'LDJQRVLV &ROOLQV 0: *ULQGHO 6+ /RYHOO 05 'HGH '( 0RVHU '3KDOLQ %5 1RJOH 6 :DVLN 0 &RUGU\ 'DXJKHUW\ 0. 6HDUV 6) 1LFROHWWH ,QGHOLFDWR 3 DQG 0F.HDJ '% f 5HODWLRQVKLS EHWZHHQ FRQFXVVLRQ DQG QHXURSV\FKRORJLFDO SHUIRUPDQFH LQ FROOHJH IRRWEDOO SOD\HUV -$0$ &RZH\ $ f )XQFWLRQDO ORFDOL]DWLRQ LQ WKH EUDLQ )URP DQFLHQW WR PRGHP 3V\FKRORJLVW 'H
PAGE 144

)HQWRQ 0F&OHOODQG 5 0RQWJRPHU\ $ 0DF)O\QQ DQG 5XWKHUIRUG : f 7KH SRVWFRQFXVVLRQDO V\QGURPH VRFLDO DQWHFHGHQWV DQG SV\FKRORJLFDO VHTXHODH %ULWLVK -RXUQDO RI 3V\FKLDWU\ )HXUVW 0/ f 1HXURORJLVWV RIIHU FRQFXVVLRQ RSWLRQV 7KH 3K\VLFLDQ DQG 6SRUWVPHGLFLQH RQOLQH 85/ KWWSZZZSK\VVSRUWVPHGFRPLVVXHVDSUQEBQHXURKWP *HQQDUHOOL 7$ DQG *UDKDP ', f 1HXURSDWKRORJ\ RI WKH KHDG LQMXULHV 6HPLQDUV LQ &OLQLFDO 1HXURSV\FKLDWU\ *ODVVPLUH '0 .LQQH\ ', DQG *UHHQH 5/ f 6HQVLWLYLW\ DQG VSHFLILFLW\ RI 003, QHXURORJLF FRUUHFWLRQ IDFWRUV 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF DQDO\VLV $VVHVVPHQW *UDKDP ', $GDPV -+ 1LFROO -$ 0D[ZHOO :/ *HQQDUHOOL 7$ f 7KH QDWXUH GLVWULEXWLRQ DQG FDXVHV RI WUDXPDWLF EUDLQ LQMXU\ %UDLQ 3DWKRORJ\ *URQZDOO '0 DQG :ULJKWVRQ 3 f 'HOD\HG UHFRYHU\ RI LQWHOOHFWXDO IXQFWLRQLQJ DIWHU PLQRU KHDG LQMXU\ /DQFHW LL +DQOH\ -$ f 5HFHLYHU RSHUDWLQJ FKDUDFWHULVWLF 52&f PHWKRGRORJ\ WKH VWDWH RI WKH DUW &ULWLFDO 5HYLHZ LQ 'LDJQRVWLF ,PDJLQJ +DUULVRQ &/ DQG 'LMNHUV 0 f 7UDXPDWLF EUDLQ LQMXU\ UHJLVWULHV LQ WKH 8QLWHG 6WDWHV $Q RYHUYLHZ %UDLQ ,QMXU\ +HDWKFRWH $ f ,WHP UHFRJQLWLRQ PHPRU\ DQG WKH UHFHLYHU RSHUDWLQJ FKDUDFWHULVWLF -RXUQDO RI ([SHULPHQWDO 3V\FKRORJ\ /HDUQLQJ 0HPRU\ t &RJQLWLRQ +HUPDQQ %3 f 1HXURSV\FKRORJLFDO IXQFWLRQ DQG SV\FKRSDWKRORJ\ LQ HSLOHSV\ 3V\FKRORJLFDO 5HSRUWV +HUPDQQ %3 DQG :\OHU $5 f 1HXURSV\FKRORJLFDO RXWFRPH RI DQWHULRU WHPSRUDO OREHFWRP\ -RXUQDO RI (SLOHSV\ +HUPDQQ %3 6HLGHQEHUJ 0 6FKRHQIHOG 3HWHUVRQ /HYHURQL & DQG :\OHU $5 f (PSLULFDO WHFKQLTXHV IRU GHWHUPLQLQJ WKH UHOLDELOLW\ PDJQLWXGH DQG SDWWHUQ RI QHXURSV\FKRORJLFDO FKDQJH DIWHU HSLOHSV\ VXUJHU\ (SLOHSVLD +LQWRQ%D\UH $' *HIIHQ *0 DQG 0F)DUODQG f 0LOG KHDG LQMXU\ DQG VSHHG RI LQIRUPDWLRQ SURFHVVLQJ $ SURVSHFWLYH VWXG\ RI SURIHVVLRQDO UXJE\

PAGE 145

OHDJXH SOD\HUV -RXUQDO RI &OLQLFDO DQG ([SHULPHQWDO 1HXURSV\FKRORJ\ +LQWRQ%D\UH $' *HIIHQ *0 DQG *HIIHQ /% f &RQFXVVLRQ LQ FRQWDFW VSRUWV 5HOLDEOH FKDQJH LQGLFHV RI LPSDLUPHQW DQG UHFRYHU\ -RXUQDO RI &OLQLFDO DQG ([SHULPHQWDO 1HXURSV\FKRORJ\ -DFREVRQ 16 DQG 7UXD[ 3 f &OLQLFDO VLJQLILFDQFH D VWDWLVWLFDO DSSURDFK WR GHILQLQJ PHDQLQJIXO FKDQJH LQ SV\FKRWKHUDS\ UHVHDUFK -RXUQDO RI &RQVXOWLQJ DQG &OLQLFDO 3V\FKRORJ\ -HQQHWW % DQG %RQG 0 f $VVHVVPHQW RI RXWFRPH DIWHU VHYHUH EUDLQ GDPDJH $ SUDFWLFDO VFDOH 7KH /DQFHW L .HOO\ -3 DQG 5RVHQEHUJ -+ f 3UDFWLFH SDUDPHWHU WKH PDQDJHPHQW RI FRQFXVVLRQ LQ VSRUWV 1HXURORJ\ .HOO\ -3 DQG 5RVHQEHUJ -+ f 7KH GHYHORSPHQW RI JXLGHOLQHV IRU WKH PDQDJHPHQW RI FRQFXVVLRQ LQ VSRUWV -RXUQDO RI +HDG 7UDXPD 5HKDELOLWDWLRQ .LEE\ 0< DQG /RQJ &f 0LQRU KHDG LQMXU\ DWWHPSWV DW FODULI\LQJ WKH FRQIXVLRQ %UDLQ ,QMXU\ .LQJ 16 f (PRWLRQDO QHXURSV\FKRORJLFDO DQG RUJDQLF IDFWRUV WKHLU XVH LQ WKH SUHGLFWLRQ RI SRVWFRQFXVVLRQ V\PSWRPV DIWHU PLOG DQG PRGHUDWH KHDG LQMXULHV -RXUQDO RI 1HXURORJ\ 1HXURVXUJHU\ DQG 3V\FKLDWU\ .LQJVOH\ 5( f &RQFLVH WH[W RI QHXURVFLHQFH %DOWLPRUH :LOOLDPV t :LONLQV .QHHERQH $& $QGUHZ 0%DNHU 5$ DQG .QLJKW -/ f 1HXURSV\FKRORJLF FKDQJHV DIWHU FRURQDU\ DUWHU\ E\SDVV JUDIW LQ XVH RI UHOLDEOH FKDQJH LQGLFHV $QQDOV RI 7KRUDFLF 6XUJHU\ .UDHPHU +& 0RUJDQ *$ /HHFK 1/ *HOQHU -$ 9DVNH -+DUPRQ 5f 0HDVXUHV RI FOLQLFDO VLJQLILFDQFH -RXUQDO RI WKH $PHULFDQ $FDGHP\ RI &KLOG DQG $GROHVFHQW 3V\FKLDWU\ /DQJILWW 7: DQG *HQQDUHOOL 7$ f &DQ WKH RXWFRPH IURP KHDG LQMXU\ EH LPSURYHG" -RXUQDO RI 1HXURVXUJHU\ /HLQLQJHU %( *UDPOLQJ 6( DQG )DUUHOO $' f 1HXURSV\FKRORJLFDO GHILFLWV LQ V\PSWRPDWLF PLQRU KHDG LQMXU\ SDWLHQWV DIWHU FRQFXVVLRQ DQG PLOG FRQFXVVLRQ -RXUQDO RI 1HXURORJ\ 1HXURVXUJHU\ DQG 3V\FKLDWU\

PAGE 146

/HYLQ +6 0DWWLV 6 DQG 5XII 50 f 1HXUREHKDYLRUDO RXWFRPH IROORZLQJ PLQRU KHDG LQMXU\ D WKUHHFHQWHU VWXG\ -RXUQDO RI 1HXURVXUJHU\ /H]DN 0' f 1HXURSV\FKRORJLFDO DVVHVVPHQW WKLUG HGLWLRQ 1HZ
PAGE 147

0HDQH\ ') 6PLWK '+ 6KUHLEHU ', %DLQ $& 0LOOHU 57 5RVV '7 DQG *HQQDUHOOL 7$ f %LRPHFKDQLFDO DQDO\VLV RI H[SHULPHQWDO GLIIXVH D[RQDO LQMXU\ -RXUQDO RI 1HXURWUDXPD 0LOG 7UDXPDWLF %UDLQ ,QMXU\ &RPPLWWHH RI WKH +HDG ,QMXU\ ,QWHUGLVFLSOLQDU\6SHFLDO ,QWHUHVW *URXS RI WKH $PHULFDQ &RQJUHVV RI 5HKDELOLWDWLRQ 0HGLFLQFH f 'HILQLWLRQ -RXUQDO RI +HDG 7UDXPD 5HKDELOLWDWLRQ 0LOOVSDXJK -$ f 'HPHQWLD SXJLOLVWLFD 86 1DYDO 0HGLFDO %XOOHWLQ 0LOQHU % f 3V\FKRORJLFDO GHIHFWV SURGXFHG E\ WHPSRUDO OREH H[FLVLRQ 5HVHDUFK 3XEOLFDWLRQV RI WKH $VVRFLDWLRQ IRU 5HVHDUFK LQ 1HUYRXV t 0HQWDO 'LVHDVH 0RQWJRPHU\ ($ )HQWRQ *: 0F&OHOODQG 50DF)O\QQ DQG 5XWKHUIRUG :+ f 7KH SV\FKRELRORJ\ RI PLQRU KHDG LQMXU\ 3V\FKRORJLFDO 0HGLFLQH 2SSHQKHLPHU '5 f 0LFURVFRSLF OHVLRQV LQ WKH EUDLQ IROORZLQJ KHDG LQMXU\ -RXUQDO RI 1HXURVXUJHU\ DQG 3V\FKLDWU\ 3DUNHU 56 f 7KH VSHFWUXP RI HPRWLRQDO GLVWUHVV DQG SHUVRQDOLW\ FKDQJHV DIWHU PLOG KHDG LQMXU\ LQFXUUHG LQ D PRWRU YHKLFOH DFFLGHQW %UDLQ ,QMXU\ 3RYOLVKRFN -7 %HFNHU '3 &KHQJ &/ DQG 9DXJKDQ *: f $[RQDO FKDQJH LQ PLQRU KHDG LQMXU\ -RXUQDO RI 1HXURSDWKRORJ\ DQG ([SHULPHQWDO 1HXURORJ\ 5HLWDQ 5 DQG :ROIVRQ f 7KH +DOVWHDG5HLWDQ QHXURSV\FKRORJLFDO WHVW EDWWHU\ WKRU\ DQG FOLQLFDO LQWHUSUHWDWLRQ 7XVFRQ $= 1HXURSV\FKRORJ\ 3UHVV 5LPHO 5 *LRUGDQL 0 %DUWK %ROO 7 -DQH f 'LVDELOLW\ FDXVHG E\ PLQRU KHDG LQMXU\ 1HXURVXUJHU\ 5REHUWV *: *HQWOHPHQ 60 /\QFK $ 0XUUD\ / /DQGRQ 0 DQG *UDKDP ', f $P\ORLG SURWHLQ GHSRVLWLRQ LQ WKH EUDLQ DIWHU VHYHUH KHDG LQMXU\ LPSOLFDWLRQV IRU WKH SDWKRJHQHVLV RI $O]KHLPHUfV 'LVHDVH -RXUQDO RI 1HXURORJ\ 1HXURVXUJHU\ DQG 3V\FKLDWU\ 5RVV 65 0LOOLV 65 .UXNRZVNL 5$ f 'HWHFWLQJ LQFRPSOHWH HIIRUW RQ WKH 003, $Q H[DPLQDWLRQ RI WKH )DNH%DG 6FDOH LQ PLOG KHDG LQMXU\ -RXUQDO RI &OLQLFDO DQG ([SHULPHQWDO 1HXURSV\FKRORJ\ 5XII 50 f $ IULHQGO\ FULWLTXH RI QHXURSV\FKRORJ\ IDFLQJ WKH FKDOOHQJHV RI RXU IXWXUH $UFKLYHV RI &OLQLFDO 1HXURSV\FKRORJ\

PAGE 148

6DZULH 60 &KHOXQH *1DXJOH 5, DQG /XGHUV +2 f (PSLULFDO PHWKRGV IRU DVVHVVLQJ QHXURSV\FKRORJLFDO FKDQJH IROORZLQJ HSLOHSV\ VXUJHU\ -RXUQDO RI WKH ,QWHUQDWLRQDO 1HXURSV\FKRORJLFDO 6RFLHW\ 6PLWK $ f 6\PERO 'LJLW 0RGDOLWLHV 7HVW 6'07f PDQXDO UHYLVHGf /RV $QJHOHV :HVWHUQ 3V\FKRORJLFDO 6HUYLFHV 6SUHHQ 2 DQG 6WUDXVV ( f $ FRPSHQGLXP RI QHXURSV\FKRORJLFDO WHVWV VHFRQG HGLWLRQ 1HZ
PAGE 149

%,2*5$3+,&$/ 6.(7&+ %HQMDPLQ 5 3KDOLQ ZDV ERP DQG UDLVHG LQ )ORULGD +H DWWHQGHG :LQWHU 3DUN +LJK 6FKRRO LQ 2UODQGR ZKHUH KH JUDGXDWHG DV D )ORULGD $FDGHPLF 6FKRODU LQ %HQMDPLQ ZHQW RQ WR DWWHQG WKH 8QLYHUVLW\ RI )ORULGD LQ DFKLHYLQJ KLV %DFKHORU RI 6FLHQFH GHJUHH LQ SV\FKRORJ\ LQ DQG ZDV DQ LQGXFWHH LQWR WKH 3KL %HWD .DSSD DQG *ROGHQ .H\ +RQRU 6RFLHWLHV DW WKDW LQVWLWXWLRQ +H ZDV DGPLWWHG WR WKH GRFWRUDO SURJUDP LQ FOLQLFDO DQG KHDOWK SV\FKRORJ\ DW WKH 8QLYHUVLW\ RI )ORULGD LQ DQG DIWHU FRPSOHWLRQ RI WKH FRXUVH DQG SUDFWLFD UHTXLUHPHQWV RI WKDW SURJUDP FRPSOHWHG KLV LQWHUQVKLS LQ QHXURSV\FKRORJ\ DW WKH -DPHV $ +DLOH\ 9$ 0HGLFDO &HQWHU LQ 7DPSD )ORULGD +H UHFHLYHG KLV 'RFWRU RI 3KLORVRSK\ LQ $XJXVW DQG FXUUHQWO\ ZRUNV DV D SRVWGRFWRUDO UHVLGHQW LQ SULYDWH SUDFWLFH LQ *DLQHVYLOOH )ORULGD ZKHUH KH LV VXSHUYLVHG E\ (UQHVW %RUGLQL 3K'

PAGE 150

, FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ -M8/W8 Mef! 4&W9$M// (LOHHQ % )HQQHOO &KDLU 3URIHVVRU RI &OLQLFDO DQG +HDOWK 3V\FKRORJ\ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 'XDQH ( 'HGH &RFKDLU &OLQLFDO $VVRFLDWH 3URIHVVRU RI &OLQLFDO DQG +HDOWK 3V\FKRORJ\ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ +HDOWK 3V\FKRORJ\ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ -DUIHV $OJLQD 3URIHVVRU RI (GXFDWL 3V\FKRORJ\

PAGE 151

7KLV GLVVHUWDWLRQ ZDV VXEPLWWHG WR WKH *UDGXDWH )DFXOW\ RI WKH &ROOHJH RI 3XEOLF +HDOWK DQG +HDOWK 3URIHVVLRQV DQG WR WKH *UDGXDWH 6FKRRO DQG ZDV DFFHSWHG DV SDUWLDO IXOILOOPHQW RI WKH UHTXLUHPHQWV IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ $XJXVW 'HDQ &ROOHJH RI 3XEOLF +HDOWK DQG +HDOWK 3URIHVVLRQV \Lf§IFM $ 'HDQ *UDGXDMH 6FKRRO

PAGE 152

81,9(56,7< 2) )/25,'$


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID EIW1DEPWB_SO5ET4 INGEST_TIME 2011-11-02T15:17:09Z PACKAGE AA00004699_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES


DETECTING SIGNIFICANT CHANGE IN THE NEUROPSYCHOLOGICAL
PERFORMANCE OF COLLEGE FOOTBALL PLAYERS
By
BENJAMIN R. PHALIN
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2004

Copyright 2004
by
Benjamin R. Phalin

ACKNOWLEDGEMENTS
A project of this scope involves the collaboration of a great number of people.
First and foremost, I would like to acknowledge my parents, whose emphasis on the
value of education and unswerving support throughout my own educational endeavors
has allowed me to continue to strive to reach for my potential. Secondly, I would like to
thank my wife for her encouragement throughout the process of the preparation of this
paper. Last, but certainly not least, I would like to thank Dr. Eileen Fennell and Dr.
Duane Dede for their guidance, and for providing me with outstanding examples of true
scientist-practitioners.

TABLE OF CONTENTS
Page
ACKNOWLEDGEMENTS iii
ABSTRACT vi
CHAPTER
ONE INTRODUCTION 1
Historical Context 1
Minor Head Injury: Definition 4
Minor Head Injury: Pathology 10
Neuropsychological Effects of Minor Head Injury 16
Sports-related Minor Head Injury 20
Reliable Change 28
Reliable Change Index 29
Reliable Change Index with Correction for Practice 30
Linear Regression 31
Multiple Regression Model 31
Specific Aims and Hypotheses 33
TWO METHODS 35
Subjects 35
Protocol 36
Baseline 35
In-Season Concussion Evaluation 39
Data Analysis 40
Basic Models 41
Reliable Change Indices 41
Regression Models 42
Sensitivity and Specificity 42
THREE RESULTS 44
Demographic Data for Baseline Population 44
IV

Demographic Characteristics for Concussed and Control 47
Participants
Baseline Performance 49
Overview of Repeated Testing 51
Detecting Significant Decline Using Basic Statistical Methods 51
Model 1 56
Model 2 63
Reliable Change Index Models 71
Model 3 71
Model 4 79
Model 5 87
Regression Models 95
Modeló 95
Model 7 103
FOUR DISCUSSION 112
Basic Models 116
RCI Models 117
Regression Models 121
Limitations and Implications for Future Research 129
Contributions of the Current Study 131
REFERENCES 135
BIOGRAPHICAL SKETCH 142
v

Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
DETECTING SIGNIFICANT CHANGE IN THE NEUROPSYCHOLOGICAL
PERFORMANCE OF COLLEGE FOOTBALL PLAYERS
By
Benjamin R. Phalin
August 2004
Chair: Eileen B. Fennell
Cochair: Duane E. Dede
Major Department: Clinical and Health Psychology
The use of neuropsychological test instruments has been advocated by researchers
in the detection of cognitive decline following sports-related concussion as being a
sensitive means of producing objective results with which to aid in making retum-to-play
decisions. However, no study to date has provided a systematic examination of serial
neuropsychological testing over several days following concussion. In the current study,
a brief battery of neuropsychological tests was administered to 443 college football
players at four Division I universities. Of the 443 initial participants, 26 concussed
players were identified by team trainers and subsequently underwent retesting using the
same battery of tests within 24 hours, 3 days, 5 days, and 7 days of concussion.
Concussed players were assigned controls matched on position, history of diagnosed
vi

learning disability, and history of previous concussion who were also re-administered the
battery at the same test intervals.
Seven different statistical models conceptually organized into three groups were
examined to determine their ability to distinguish between the concussed and
nonconcussed participants. Basic models included a comparison of retest performance to
normative data and comparison of retest performances to baseline performance. RCI
models included a basic RCI approach, RCI with practice correction, and indices created
based on the magnitude of change of the control population. Regression models included
simple linear regression and multiple regression models, though multiple regression
analysis was ended due to instability of prediction equations. Optimal
sensitivity/specificity combinations occurred when classification was made based on one
significant variable finding. Good sensitivity was observed for all models at the 24-hour
retest period, although specificity was a limiting factor for the most basic models. Linear
regression models and an analysis of magnitude of change over re-test intervals were the
only models to distinguish between groups at Day 3 and Day 7. Results suggested the
importance of systematic examination of cognitive decline following sports-related
concussion, especially given currently-used AAN retum-to-play guidelines, as well as
need for appropriate comparison data and statistical analyses to aid in the determination
of cognitive recovery following concussive injury. Further data collection is necessary,
especially at later sessions, to accurately use multiple regression models.
Vll

CHAPTER ONE
INTRODUCTION
Historical Context
The use of neuropsychological measures has, over the course of the continuing
evolution of the field, been the focus of considerable debate in terms of utility and
correlates of specific test performance. This evolution has been described in terms of
distinct eras (McSweeny et al.,1993) or areas of focus in the use of neuropsychological
test performance. The first has been described by Rourke (quoted in Mcsweeny et al.,
1993), and involved the interpretation of test performance and the use of cognitive testing
mainly to imbue some veracity to the hypothesis that performance on certain specifically
designed test measures indirectly suggested dysfunction in certain localized brain
anatomy. This endeavor, of course, has been the mainstay of the various academic
subspecialties concerned with the study of neuroanatomy and the associated cognitive
sciences and theories. The very roots of this goal can be traced back as far as the time of
Gall, phrenology, and the notion that various protrusions on the scalp of an individual
revealed certain personality traits and tendencies (Cowey, 2001). While the theory of
phrenology itself has long since been discredited, it is the notion of localization that has
stood the test of time and has led to a greater understanding and logical organization of
the study of the structure and function of the human brain. During the infancy of the field
of neuropsychological theory in general, the primary opportunity to further corroborate
structure and function was found to be the study of the performance of individuals with
1

2
localized lesions. From the performance of these individuals, inferences concerning the
function of the absent or damaged areas were fonnulated. One of the most cited cases is
that of patient H.M., who received resection of the anterior portions of both of his
temporal lobes as part of a radical surgical correction for intractable epilepsy. Brenda
Milner and her colleagues detailed the cognitive effects of this resection and established
the utility of neuropsychological tests in the measurement of deficient cognitive abilities
(Milner, 1958). As McSweeny et al. (1993) noted, neuropsychological tests have been
utilized recently in order to aid in diagnosis and to provide clinically significant
contributions to the diagnostic process. A vast literature has been amassed, for instance,
in the application of neuropsychological techniques in the examination and treatment of
epileptic patients subsequent to patient H.M. (Bennett, 1987; Hermann, 1985; Hermann
and Wyler, 1988). Cognitive tests have been used to aid in the process of localizing the
focus of epileptic activity by suggesting relatively discrete anatomical areas of
dysfunction which are thought to contribute to poor performance on certain measures.
Further, cognitive tests have been used in conjunction with reversible lesion methods
(e.g., WADA) to localize brain regions responsible for language and memory in
individuals undergoing surgical resection methods in order to avoid the accidental
destruction of these areas and their subsequent debilitating effects (Loring et al., 1992).
Determination of the language- dominant hemisphere via neuropsychological methods is
an important step in identifying those individuals who might best be served with this
surgical treatment.
A further use of neuropsychological techniques becomes salient when one seeks
to quantify the effect of a surgical resection or any other neurological insult. By

3
comparing an individual’s performance before a potentially neurologically altering event
to his/her performance afterward, one is able to objectively describe the resulting
cognitive effects. Another commonly used methodology, in general, involves comparing
the performance of one individual to that of a group of other individuals, or normative
population. In this sense, the term “normative” implies that the mean score of a group of
individuals is the standard to which other individuals can be compared. This is perhaps
the most typical method used in neuropsychology to gauge the performance of the
individual in question, and has often been used without question as to the validity of such
practice. Within the past several years, neuropsychological test performance has been
used somewhat less frequently to localize neurological insult due to the continuing
growth and advance of imaging technology (Ruff, 2003). While the use of imaging
devices such as positron emission topography (PET), magnetic resonance imaging (MRI),
and functional magnetic resonance imaging (fMRI) has continued to hone the ability of
the investigator to view the brain itself in a non-obtrusive way, and have continued to
lend themselves effectively to the diagnostic process, neuropsychological test results
have played a somewhat decreased role in this capacity. However, the focus of
neuropsychological examination and analysis of test performance have clearly proven
their merit in other ways. For example, the measurement of cognitive abilities and
inherent strengths and weaknesses within the individual offers the neuropsycholgist the
unique opportunity to contribute to the practical knowledge about the cognitive
functioning of a patient, and allows him or her to recommend strategies designed to aid
the individual in returning to previous levels. In the event that a baseline measure of an
individual’s performance is available, a comparison with post-event findings allows for

4
an objective review of resulting cognitive deficits and a gauge of changing cognitive
status. In this sense, a measure of individual cognitive change over time has become a
major focus of neuropsycholgical measures. With this increasing focus on longitudinal
follow-up and repeated testing, several issues regarding serial test administration and the
sensitivity of established neuropsychological measures have arisen. Recently, the use of
repeated assessment over time has been applied in various circumstances in order to
measure the progressive nature of cognitive change. This has been the case, as
McSweeny et al. (1993) pointed out, in the assessment of the dementias, which involve
the insidious loss of cognitive abilities due to the progressive alteration of brain structure
or chemistry. Measures of neuropsychological change over time have also been used to
assess treatment affects and interventions.
Recently, the use of neuropsycholgical testing has been applied to the study of
one of the most common causes of cognitive change and continuing sequelae: head
injury. While the results of moderate to severe head injury are often obvious, both to
current imaging techniques and measures of cognitive status, and the recovery from such
is often prolonged and some effects permanent, the effects of more subtle, or mild head
injuries are not so easy to unilaterally identify. Therefore, the use of neuropsychological
measures in this regard has become more valuable.
Minor Head Injury: Definition
While several researchers have endeavored to elucidate the consequent effects of
minor head injury, the lack of consistency in defining the phenomenon has made it
difficult, if not impossible, to reconcile the findings of different investigations. The use
of the term “minor head injury” generally suggests the occurrence of mild to moderate

5
traumatic brain injury (Kibby and Long, 1996). However there has been little agreement
on the terminology utilized in describing various levels of severity within this broad
category. A review of the literature reveals a basic framework underlying the working
model of minor head injury. However, from this basic outline, many slight variations
have been created and used in experimental design. In general, a minor head injury is
considered to be one in which a force is applied to the head, producing posttraumatic
memory difficulties or other fast-resolving neurological symptoms” (Kibby and Long,
1996).
In examining the origins of this very basic foundation for defining minor head
injury, Kibby and Long (1996) refer to research findings suggesting that the severity of
head injury is directly related to the force that is applied to it. They indicate that this
finding is supported by the work of Graham et al. (1995) which suggested that the degree
of axonal injury is naturally also affected by the amount and direction of force applied.
In other words, the amount of force applied to the head is directly correlated with the
amount of damage expected to be observed in the axons of the brain and the amount of
metabolic disruption (McIntosh et al., 1996; Meaney et al., 1995). In a natural
progression, one might assume that this also applies to any observed cognitive
dysfunction, which may be directly related to the extent of axonal injury incurred. Kibby
and Long suggest that this is shown through the work of Langfitt and Gennarelli (1982)
and Levin et al., (1987). In their respective investigations, these authors concluded that a
minor head injury resulting in a loss of consciousness of less than 20 minutes was not
associated with any lingering deficits or detectable neuroanatomical damage. Therefore,
the extent of the head injury was assumed to be minor.

6
Aside from the adoption of some common findings to the definition of minor
head injury, one of the most commonly used indicators of head injury severity is the
Glasgow Coma Scale (GCS) (Jennett and Bond, 1975). The GCS is a measure
commonly employed at the scene of a head injury-inducing accident by immediate care
providers or in an emergency medicine situation. In performing this assessment
technique, the degree of coma is measured by evaluating motor, verbal, and eye-opening
responses to a set of standard stimuli. The resulting score is based on a scale from 3 to
15, the lowest score representing no response to verbal commands or somatic stimulation,
and the highest suggesting the intact ability to follow verbal commands, orientation, and
obvious awareness of sensory input. In the current literature, it is generally suggested
that a minor head injury is one that results in a GCS of 13-15, scores that are often
correlated with a positive long-term outcome. However, the practice of using the GCS by
itself in the assessment of injury severity is questionable since the work of researchers
such as Cantu (1992) and Strauss and Savitsky (1934) has suggested that symptoms
subsequent to injury without the observation of loss of consciousness are also common.
Another commonly used indicator of head injury severity is posttraumatic
amnesia, which has been defined as the period of time following the applied trauma to the
brain in which new memories are unable to be stored. This time period is associated with
the inability to recall events, sequence time, or learn new information (Ahmed et al.,
2000; McAllister, 1992; McFarland et al., 2001). A correlation has been noted between
the length of PTA and the duration of LOC, though McAllister indicates that this
relationship is not always linear, and warns against using either measure as the sole
predictor of head injury severity and potential outcome.

7
While some commonalities exist between research projects in their inclusion
criteria for cases of minor head injury, a significant amount of slight variation around the
central theme has remained. The reasons for this occurrence may be many, from
theoretical differences to the utilization of samples of convenience or the use of
nonprospective research designs. Researchers have historically not applied a common
definition to their inclusion criteria when studying minor head injury. For instance, in
the often-cited study by Rimel et al. in 1981, patients were included as cases of minor
head injury if they had LOC of less than 20 minutes. This group added another specifier
to the basic definition. In order to control for cognitive changes observed that could
possibly have been related to other event-related trauma, Rimel et al. excluded patients
with a hospital stay of greater than three days. It was their supposition that a longer
hospital stay was indicative of more severe injuries that might introduce confounds to
results obtained that otherwise would have been attributed to the head injury alone.
Another definition was provided by the Centers for Disease Control in 1985, and
while this definition is not specific to minor head injury, a sense of the vague boundaries
that might be inferred from it is still recognizable. The CDC states that a traumatic brain
injury is the “occurrence of injury to the head arising from blunt or penetrating trauma, or
from acceleration-deceleration forces, that is associated with any of these symptoms or
signs attributable to the injury: decreased level of consciousness, amnesia, other
neurologic or neuropsychologic abnormalities”(Thurman et al., 1998).
Subsequent researchers have included patients who met some, but not all, of these
criteria; others have included patients with more severe injuries and confounding
neurological histories (Dikmen et al., 1986; Leininger et al., 1990; Levin et al., 1987).

8
What is clear in this rather cloudy picture is that the only consistency is inconsistency.
Table 1 indicates the variability in defining minor head injury among some of the most
cited authors in this field.
Table 1. Varying inclusion criteria for minor head injury
Author LOC PTA GCS
Barth et al. (1983)
Dikmen et al. (1986)
Parker (1996)
Alexander(1992)
King (1996)
<20 minutes
<1 hour
<5 minutes
<15 minutes
<60 minutes
>1 hour
<2 hours
<24
>12
>11
LOC=loss of consciousness, PTA=posttraumatic amnesia, GCS=Glasgow Coma Scale
In an attempt to assemble these various criteria, the basic definition of minor head
injury, as outlined by Kibby and Long (1996), is as follows. A minor head injury is
incurred by any blow to the head resulting in a loss of consciousness of less than 20
minutes, an initial GCS score of greater than or equal to 13, and posttraumatic amnesia of
less than 24 hours. This definition, while serving as a solid basis for the nosology of
minor head injury and indicating the upper limits of what is thought to constitute minor
head injury, fails to define the boundaries at the minimum degree of that which can be
considered in this category.
Recognizing the variability in defining minor head injury from one study to the
next, a definition of mild traumatic brain injury (MTBI) was formulated by the Mild
Traumatic Brain Injury Committee of the Head Injury Interdisciplinary Special Interest
Group of the American Congress of Rehabilitation Medicine (1993). According to the
consensus reached by this organization, MTBI is defined by, at least one of the following:

9
1. any period of loss of consciousness, 2. any loss of memory for events
immediately before of after the accident, 3. any alteration of mental state at the
time of the accident, 4. focal neurological deficits, which may or may not be
transient, but when the severity of the accident does not exceed the following: a.
loss of consciousness of 30 minutes or less; b. after 30-minutes, an initial GCS
score of 13-15; and c. posttraumatic amnesia not greater than 24 hours. (Busch
and Alpem, 1998).
This attempt at providing a uniform definition, while again clearly delineating an
upper limit for symptoms indicative of minor head injury, fails to account for the minimal
symptomatology necessary to qualify as such. The minimum limits of this definition
include a wide gradation of injuries, for which the definition itself does not account or
enumerate. Within this category is the type of injury known as concussion, which
according to the Mild Traumatic Brain Injury Subcommittee, also falls under the general
rubric of minor head injury.
Concussion has come to be defined as a “clinical syndrome characterized by
immediate and transient posttraumatic impairment of neural function” (Maddocks and
Saling, 1996). In general, those suffering from concussion are thought to recover without
residual impairment. The categorization of concussion severity was the topic of a recent
set of guidelines authored by the Quality Standards Subcommittee of the American
Academy of Neurology (AAN) in 1997 (cited in Hinton-Bayre et al., 1999). In this
classification scheme, the severity of concussion is based upon a graded scale ranging
from 1, the least severe, to 3, the most severe. Grade 1 concussions are those in which
the individual does not lose consciousness as the result of the insulting force, but
experiences transient confusion (inattention, poor concentration, or inability to process
information or sequence tasks) that resolves in 15 minutes or less. Grade 2 concussions
are those in which, like grade 1 concussions, the individual does not lose consciousness,

10
but experiences transient confusion that does not remit within a 15 minute time period.
The most severe concussion, grade 3, according the AAN guidelines, includes a period of
loss of consciousness, which may last seconds to minutes, as well as transient
neurological disruption (Kelly and Rosenberg, 1998). These transient neurological
symptoms have also been the subject of many research investigations. Patients suffering
from minor head injuries, including concussion, often complain of a cluster of symptoms,
which was recognized by Strauss and Savitsky (1934). In their study, these investigators
referred to this complex of symptoms as “post-concussion syndrome” (Watson et al.,
1995). This cluster includes those symptoms mentioned above plus many others, which
may persist for as long as weeks or months after injury. The current study will focus on
the neuropsychological effects observed in concussions incurred in a college football
population.
Minor Head Injury: Pathology
Researchers have long sought to identify the core features and underlying
mechanism for the cognitive sequelae that are evidenced following mild traumatic brain
injury. Key to this search is the fact that little evidence has surfaced regarding the
biological changes that are thought to occur in these instances (Montgomery et al., 1991).
Lishman (1988) reviews some of the early research involving animal models of
concussion study in the search for the pathophysiological roots of concussion
symptomatology. Initially, he states, cerebral ischaemia as a result of a blow to the head
was thought to be responsible for the pattern of symptoms observed. This was
hypothesized to arise from the rise in intracranial pressure and obstruction of intracranial
circulation. While this hypothesis theoretically accounted for the brief loss of

11
consciousness observed in many cases of minor head injury, laboratory findings were not
supportive, and other theories were subsequently formulated. Lishman (1988) points to
the contribution of Denny-Brown and Russell in 1941 concerning the theory that shear
strain resultant from rotational forces are in some part responsible for transient cognitive
deficits in mild head injury. The studies of these researchers on the effect of
acceleration-deceleration effects upon the brain itself promulgated further studies of the
effect of these forces. Denny-Brown’s studies with animals suggested that what he
described as “uncomplicated” mild head injury, or concussion, was achieved with his
subjects only when the head was free to move about when the actual insult occurred. His
conclusions, therefore, implicated the acceleration and deceleration forces that were
applied to the skull during the course of the injury.
It was at this point that researchers began to appreciate the fluidity of the
environment within the cranium. That is, investigators moved toward experimental
designs that would allow them to directly observe and measure those forces acting within
the skull itself. Lishman (1988) describes the studies of Pudenz and Shelden in 1946 on
monkeys in which the skullcaps were removed and replaced with transparent material.
By so doing, these researchers were able to observe the movement of the brain at the
moment of impact. They indicated in their report that they observed a “swirling” motion
both at the point of impact and in other areas of the brain not directly impacted by the
blow. It was this swirling motion that was thought to produce many of the symptoms
associated with concussion.
Since the early work of these researchers, many others have confirmed and
elaborated upon these hypotheses. The brain can be thought of as a semi-gelatinous mass

12
that is suspended in a fluid within a confined space. It is held in place both by its limited
housing within the skull and through its connection via the brainstem and spinal cord.
Because it is “floating” in a homeostatic amount of cerebrospinal fluid (CSF), and
“tethered, it moves out of phase relative to the skull in response to changes in speed or
direction”(McAllister, 1992). The anatomically disrupting effects seen in mild head
injury are thought to be directly correlated with the force of the impact imparted to the
head. Further, relatively few instances of concussion have been noted in cases of static
injury. That is, when the individual is not free to move his head, the results of minor
head injury have not been found to be as substantial (Kingsley, 1996; McAllister, 1992).
Therefore, it would appear that the risk of incurring complications as a result of applied
forces is greatly increased when the head does not have a substance through which it may
conduct a portion of the force vectors acting upon it.
In the case of minor head injury, the most common insults are classified as
diffuse. In this sense, a diffuse injury is one in which a definitive location of focal impact
cannot be identified. Because minor head injury rarely results in fatality, it is often
impossible to detect the subtle neuroanatomical changes inherent to the injury.
Montgomery et al. (1991) suggested that one of the first researchers to identify
anatomical anomalies subsequent to minor head injury was Oppenheimer in 1968. In his
studies, Oppenheimer had the opportunity to examine the brains of five individuals who,
having suffered from mild traumatic brain injuries, died of complications from other
injuries also incurred during the causative incident. Microscopic examination of brain
tissue revealed diffuse “axonal disruptions” throughout the white matter. In particular, he
indicated that these injuries were more common in midbrain tegmentum and the corpus

13
collosum (Montgomery et al., 1991). Subsequent research over the years has indicated
that this diffuse damage to white matter tracts is the probable cause of cognitive
disruption in minor head injury. The motion of the brain within the skull is now thought
to be responsible for the “twisting and stretching” of neuronal axons and the subsequent
damage that occurs. Because this damage can be caused by inertial forces, direct impact
with the head is not necessary to induce detrimental effects (Busch and Alpem, 1998).
Furthermore, Busch and Alpem (1998) point out that damage to axons by shearing forces
causes tearing and the disruption of other cell milieu including blood vessels, possibly
altering blood transport to the directly affected neurons and those immediately
surrounding. In reviewing the histopathology of minor head injury, Busch also refers to
the work of Strich (1956) and Povlishock et al. (1983) in which damage to neurons by
shear or stretching was observed extensively in the white matter of the hemispheres and
in the brainstem. In addition, these researchers discovered that retraction balls could be
viewed in the instances where axonal fibers had been disconnected.
McAllister (1992) suggests that a predilection for these types of anatomical
disruptions occurs in the corpus collosum, superior cerebellar peduncle, basal ganglia,
and periventricular white matter. In each of these cases, the long white matter tracts
appear to be more susceptible to the damaging effects of axonal twisting than are other
areas. In addition, because the brain is situated in a hard casing, the opportunity for
injury by impact with the skull wall is also present, particularly in those regions
vulnerable to bearing the brunt of the type of motion that is often seen in cases of mild
head injury.

14
Gennarelli’s research in the area of traumatic brain injury has provided evidence
to support the notion of what he has termed “coup/contrecoup” distinction. In general, an
injury resultant from a coup force is observed locally, at the site of the impact of the head
with the offending object (Gennarelli and Graham, 1998). He further asserted that the
area and severity of damage are directly related to the size and force of the impacting
object. Therefore, he continued, but warned against generalization, coup injuries are
often associated with the acceleration of the head. Contrecoup injuries, on the other
hand, are often seen as arising from the sudden deceleration of the head. These insults
are “away from the site of injury” (Gennarelli and Graham, 1998). While they are often
observed in a linear relation with the site of impact, and from the coup-related injury site,
because of the nature of the internal environment of the cranium, this is not always the
case. The location of the brain within the skull, as mentioned above, predisposes certain
areas to damage. These areas include the temporal lobe poles, the frontal lobe poles, and
the parasagittal region (Parker, 1996), all of which are thought commonly affected to a
degree in many cases of mild head injury.
In the early 1980’s, Gennarelli and his colleagues suggested the term “diffuse
axonal injury” (DAI) in the description of the anatomical effects observed in many
instances of traumatic brain injury. As described above, focal damage produced by a
non-missile, closed-head injury is rather rare. More often, diffuse axonal damage is
prevalent. In elaborating on the definition of DAI, researchers formulated three
categories of DAI (Gennarelli and Graham, 1998). In a DAI of grade I, damage to axons
is notable within the white matter of the hemispheres and occasionally in the brain stem
and corpus collosum. This is most likely the extent of the damage seen in most cases of

15
minor head injury. A grade II DAI also evidences some “focal abnormalities in the
corpus collosum, often associated with small hemorrhages, called tissue tear
hemorrhages” (Gennarelli and Graham, 1998). According to this gradation system, grade
HI DAI’s also include extensive damage to the rostral brainstem. In a 1968 paper,
Oppenheimer and his colleagues reported their finding that “clusters of microglia” were
found in the white matter of patients who, having suffered minor head injury, had died of
unrelated causes. Gennarelli suggests that this finding has been supported recently by
other investigators and has been elaborated upon by others who have discovered the
presence of what he terms “microscopic DAI”(Gennarelli and Graham, 1998).
However, Gennarelli suggested that a possibly more important discovery was
made by Blumbergs et al. (1994) when these investigators reported having performed
immunostaining with an amyloid precursor protein (APP) antibody. In their results, these
researchers reported observing multiple areas of diffuse axonal injury when using this
staining technique in five patients who had received minor head injury. Experimentation
with other antibody staining techniques have met with less success in identifying those
regions that have suffered diffuse axonal injury. Staining for the detection of [3-APP
remains the most sensitive to elucidating this phenomenon. Gennarelli warns, though,
that this should not be considered a marker for head injury since heightened production
and collection of (3-Apolipoprotein is also seen in the “dystrophic axons” of some elderly
patients and in some cases of known transient ischemic attack and infarction. He
suggests that an accumulation of this precursor protein should suggest an area in which
metabolic change has taken place, which certainly could be the case in minor head injury,
though it may also be true in other instances as well.

16
Recent work by Roberts et al. (1994) has also suggested a link between the
accumulation of (3-APP and the increased risk for the development of Alzheimer’s
Disease (AD). The work of these researchers supports the hypothesis that the expression
of (3-APP is indeed part of “an acute phase response” to the type of neuronal damage that
is seen in minor head injury. They further suggest that the overaccumulation of the
precursor protein may lead to a build-up of (3-APP and subsequently lead to the
expression of certain AD symptoms, or an increase in an individual’s predilection to
develop AD later in life if they are already genetically susceptible. In addition to these
important biochemical findings, Busch and Alpem (1998) sites the work of Povlishock et
al. (1983) in which it was found that minor head injury can lead to a disruption in
electrolyte transfer and capillary permeability.
While the neuroanatomical effects of minor head injury are not easily identified
by common medical measures such as imaging devices and biochemical markers, recent
research in the area of diffuse microscopic damage and the detection of the accumulation
of amyloid precursor protein and its subsequent correlation with outcome have suggested
to investigators that there are biological consequences of even minor head injury. It is
becoming more commonly accepted that these changes do play a role in many of the
neuropsychological and other cognitive changes that are often observed following even
the mildest forms of head injury.
Neuropsychological Effects of Minor Head Injury
While the psychosocial ramifications of minor head injury and the effect of
persisting post concussion syndrome symptoms have received a large amount of attention
in the literature, the issue of cognitive impairment suffered as a result of minor head

17
injury has been one that has produced varying and debated results. This is due in large
part to the methodology used in different studies. However, there are some common
aspects to the findings from various research groups that suggest that minor head injury
does contribute to detectable cognitive sequelae.
The studies of Rimel et al. (1981) and Barth et al. (1983) are often cited in the
literature when referring to the finding that minor head injury produces a measurable
neuropsychological change (Dikmen et al., 1986). The findings by these two research
groups, using the same population, suggested that there are noticeable cognitive as well
as psychosocial deficits in those having suffered minor head injury. Dikmen argues that
these findings may not be accurate, however, since these groups used what she describes
as an inappropriate normative comparison group. While the groups were impaired
compared to this normative group, she states, it is difficult to parse out the effects of other
factors that are not known, including premorbid neurological status. In her own study on
the effects of minor head injury on neuropsychological functioning, Dikmen indicates
that the findings were much less definitive when testing her subjects at 1 and 12 months
post injury.
Dikmen performed a wide range of neuropsychological tests on 20 consecutive
hospitalized patients who had experienced a period of coma not over 1 hour or, in the
absence of coma with PTA of at least 1 hour, a GCS score of greater than or equal to 12
on admission, with no evidence of cortical or brainstem contusion, no history of prior
head injury or neurological insult, and between the ages of 15 to 60. She utilized a control
group of friends and relatives of those suffering from minor head injury rather than the
published normative data for the individually administered tests. Although her results

18
were not as conclusive as those of Rimel and Barth, she reported that there was a trend
for her minor head injury population to perform more poorly on tests of concentration
and short-term memory at 1 month after injury. This was supportive of the findings of a
previous and oft-cited study by Gronwall and Wrightson (1974) who showed that those
suffering minor head injury and suffering from post concussion symptoms tended to
demonstrate impairment in tests of attention and processing speed. She tempers her
finding, though, by stating that her results may not be indicative of significant differences
between the groups if one takes into account the premorbid differences in intellectual
ability between them (Dikmen et al., 1986). She did not have any measurements of
baseline functioning with which to compare her findings.
In another study of a large group of head injured individuals participating in three
separate experiments, Dikmen again sought to clarify the neuropsychological
ramifications of traumatic brain injury (Dikmen et al., 1995). The effort included 436
adult head-injured patients, 243 of whom had a GCS of 13-15 on admission. In this
study, she indicated that the extent of neuropsychological impairment observed was
directly related to the duration of coma associated with the injury. In the group that she
labeled as ‘mild head injury’ (those with coma of duration less than 1 hour and GCS of
13-15 on admission), she found that there was no statistically significant difference from
a trauma control group in neuropsychological performance. She suggests that the results
are similar to her previous study of neuropsychological outcome in minor head injury
with respect to the finding that measures of attention and short-term memory show a
trend toward persisting difference in those with minor head injury. In her discussion of
results, she stressed the fact that measures of attention and memory tend to show decline

19
with increasing severity of head injury. Again, she had no baseline comparative
measures.
Dikmen’s studies of the neuropsychological functioning of those suffering minor
head injury have taken place at relatively long periods after the injurious incident (1
month and 1 year), and have shown limited persisting differences between the injured
group and the control population. Likewise, a meta-analytic review by Binder et al.
(1997) using 8 studies comprised of 11 samples of individuals with a history of mild head
trauma at least 3 months after receiving their injury resulted in an overall nonsignificant
effect size of .07 when comparing a composite neuropsychological test score to control
groups. However, when they used the d statistic, they found a significant finding of .12,
p < .03 for measures of attention. It was their conclusion then, much like Dikmen’s, that
the neuropsychological effects, if any, of minor head injury are generally resolved from 1
to 3 months post-injury.
In a study by Montgomery et al. (1991) of 26 consecutive admissions to an
accident and emergency room for minor head injury (defined by this group as an injury
requiring an overnight inpatient hospital stay as well as a PTA period of less than 12
hours), investigators utilized a Four-Choice Reaction-Time measurement in order to
detect possible cognitive sequelae of the injury. In addition, a measurement of brainstem
evoked potential and subjective measure of post concussive complaints were performed
on the same day as the injury, six weeks later, and at six months post-injury. Fenton
reported that head-injured patients exhibited a significantly prolonged reaction time
measurement on the day of the injury when compared to a control group matched for age,
sex, and social status. Results indicated “serial improvement” at the six-week and six-

20
month follow-up. Interestingly, over half of the patient’s involved in this study still
complained of post-concussion syndrome symptoms at the six-week and six-month
follow-up assessments. In addition, more than half of these patients exhibited what
Fenton called “significant delays” in brainstem conduction during the first testing session.
Other studies examining the cognitive sequelae of minor head injury have shown
that more significant differences are detectable in the acute and early recovery phases of
head injury and are often associated with the degree of persisting post concussional
symptoms. This is particularly true in investigations examining the effects of minor head
injuries incurred in contact sport related incidents, which usually involve lesser amounts
of applied forces (acceleration and deceleration) and generally result in milder concussive
effects (Maddocks and Saling, 1996).
Sports-related Minor Head Injury
The interest in sports related head injuries in the head injury literature has a long
history, though relatively little methodologically sound research has been conducted in
the area until recently. A 1991 report by the National Health Interview Survey estimated
that approximately 300,000 people in the United States suffered a head injury as a direct
result of participating in a sporting activity. Of these, 34% (approximately 103,000) were
categorized as “mild” (Kelly and Rosenberg, 1998; Thurman et al., 1998). In addition,
the survey contended that many of those individuals suffering head injuries never sought
medical treatment, and therefore many estimates of national head injury prevalence may
be significantly below the actual number. Until the last 10 to 15 years, however, most
investigation into the cognitive effects of sports-related head injury had been isolated or

21
anecdotal, and some investigations continue to employ methods that bring the validity of
findings into question.
In an article published in the Journal of the American Medical Association in
1928, Martland described the symptoms that he observed in boxers after receiving
repeated blows to the head. He termed the condition “punch drunk” and described the
symptoms as including a slight unsteadiness in gait or uncertainty in equilibrium, slight
mental confusion, and distinct slowing of muscular action. While he noted that most of
these cases remained relatively minor, some resulted in more severe neurological
complication including hesitancy in speech, tremors of the hands, and nodding
movements of the head (Martland, 1928). In addition, he postulated that the effects of
repeated blow to the head might cause lasting damage. While he indicated that he had to
rely in some cases on observation garnered from “laymen”, he suggested that he could
identify many individuals known to suffer from persisting symptoms thought to be
related to repeated blows to the head. He was unable to empirically evaluate many of
them, he stated, as they were housed at the time in asylums across the country.
Further corroboration of this syndrome was provided by Millspaugh (1937) in his
review of the history of naval boxers. Millspaugh stated conclusively that there are
indeed cognitive sequelae to the repeated blows received in a boxing match, although the
cognitive deficits he described were of an objective nature. He also described a complex
of symptoms acutely accompanying repeated blow to the head, which included “fullness
in the head, cephalgia, vertigo, irritability, insomnia, increased fatiguability, and memory
defects” (Millspaugh, 1937). In addition, he described what appeared to him to be a
period of amnesia in both those suffering loss of consciousness during the fight, and

22
those without LOC. He took this to indicate that there was some physical damage to the
brain itself which may manifest for long periods of time. In his concluding remarks,
Millspaugh states: “repeated and frequent concussions, occasionally very severe, often
undoubtedly associated with intracranial capillary hemorrhages are to say the least not
conducive to stabilized mental equilibrium.”
In a study by Tysvaer et al. (1989), 37 former professional soccer players were
examined in order to determine the existence of persisting neuropsychological deficits
caused by heading a soccer ball. The players ranged in age from 35 to 64, and had no
reported history of alcohol abuse. Investigators interviewed the players and used the
Wechsler Adult Intelligence Scale (WAIS), the Trail Making Test (parts A and B), a
modified version of the Halstead-Reitan aphasia screening test, tests of sensory-
perceptual functioning, motor tests, and the Benton visual retention test, Form C, in order
to determine neuropsychological status. Their scores on these tests were compared to a
control group consisting of 20 hospitalized patients who manifested sings of a variety of
disorders. These researchers found that the former soccer players exhibited a
significantly larger mean split between Performance IQ (PIQ) and Verbal IQ (VIQ) on
the WAIS, and significantly worse scores on both Trail Making Tests A and B. They
concluded that their results suggested that brain injury incurred as a result of soccer
injuries can result in permanent brain injury as exhibited by performance on
neuropsychological tests measuring perception, learning, manual dexterity, speed, and
attention. While these results may be suggestive of persisting cognitive deficits as a
result of sports-related head injury, the methodology is fraught with problems. Most
importantly, the authors do not report the neurologic history of any of the players tested.

23
Therefore, it is impossible to distinguish the extent of head injuries any one individual
may have suffered. In addition, they do not appear to use the time since the supposed
injury as a covariate in examining test performance, nor do they provide a history of other
factors that may play a role in affecting cognitive functioning. Further, researchers
reported no knowledge of cognitive functioning before these individuals played soccer.
Finally, the researchers report that the control group used was one consisting of
hospitalized patients with “varying” injuries. Without knowledge of the history of the
group, it would appear that it is an inappropriate normative comparison.
The recent examination of minor head injury incurred in sporting events has
several advantages over other scenarios in which the state of the individual at the time of
the accident and observation of their immediate behavior is not possible (Collins et al.,
1999). Within the realm of organized sporting events involving physical contact between
players, the opportunity exists to assess injury very soon after the incident because of the
immediate availability of medical staff. In addition, there is often a means of reviewing
the incident via television recording. This affords the opportunity to assess the types of
forces applied to the head of the injured player. Further, some investigators now collect
baseline measures of neuropsychological functioning before a player can compete in the
contact sport. This allows for comparison of findings subsequent to the injurious incident
and more accurate analysis of subsequent cognitive changes following head injury.
A prospective study by Barth et al. (1989) of the cognitive effects of minor head
injury in a college football population suggested that neuropsychological testing is a
sensitive device in the detection of subtle cognitive changes following concussion. This
group collected baseline measures for 2,350 college football players at a total of 10

24
universities. Subsequently, they retested 182 players with documented cases of
concussion within 24 hours of the injury, 5 and 10 days later. The battery used in this
investigation consisted of Trail Making Tests A and B, the Symbol Digit Test, and the
PASAT. As a comparison group, Barth used a population of 59 players sustaining “mild
orthopedic injuries and 48 male college students” (Barth et al., 1989). These individuals
were tested with the same protocol using the same time period as the concussed players.
Barth concluded that while the control group performed significantly better on the
“within 24 hour” PASAT and Symbol Digit test, constituting what he termed “normal
testing behavior,” (exhibiting practice effects) the concussed group failed to show
improvement on these two measures within 24 hours of receiving a concussion.
However, he noted that these individuals tended to show the same improvement as
controls on the day 5 tests of these measures with a leveling at day 10, which suggested
to Barth that these individuals had returned to baseline functioning at that point. The
investigators failed to show the same pattern of results on the Trail Making Tests, which
Barth suggested may be due to the strong effects of practice inherent to its successive
administration. As a result of this study, Barth concluded that a single incident of mild
concussion in a college football player results in detectable neuropsychological changes
within 24 hours of the incident, and that a recovery to baseline functioning occurs quickly
within the 5 to 10 days following injury.
In a study of 130 Australian Rules Football players by Maddocks and Saling
(1996), baseline measures of cognitive function were collected using the Paced Auditory
Serial Addition Test (PASAT), the Digit Symbol Substitution Test (DSST), and Four-
Choice Reaction time, which involves measures of decision and movement times. Of the

25
130 with baseline cognitive measures, Maddocks reported on 10 players who
subsequently received concussion (5 were described as incurring no LOC, 5 had LOC of
less than 1 minute and PTA of less than 30 minutes) and were re-tested at 5 days post¬
injury. He indicated that while the neurological features of concussion, including
headache, nausea and dizziness, had remitted after 5 days, persisting neuropsychological
deficits were noted on the DSST and Four Choice Reaction time (decision time). He
noted that significant differences were not found on the PASAT, a test that previous
authors had found to be particularly sensitive to the effects of minor head injury due to
the attentional component inherent to performing the task. Maddocks suggested that
since his population of concussed individuals presented with milder forms of minor head
injury than most minor head injury populations examined, the sensitivity of the test may
be less effective. Of importance, Maddocks utilized a control group consisting of age-
matched umpires who were tested on two separate occasions with the same test measures.
Although this may allow for comparison with the head injured population taking into
account the effect of practice on the various measures, Maddocks fails to elaborate on
other factors that may have made this an inappropriate comparison group (e.g., history of
past head injury, inter-test duration, educational history, etc.).
In another study by Hinton-Bayre et al. (1997), 54 players from a professional
rugby team were tested at baseline using the Symbol Digit Modalities Test (SDMT), the
Digit Symbol Substitution Test (DSST), and the Speed of Comprehension Test. Ten
players subsequently received concussions and were tested within 24-48 hours of the
incident. The authors reported that those measures of speed of information processing
were sensitive to the cognitive effects of the concussion, though an untimed test of word

26
recognition was not. They further stated that tests of speed of information processing
were more sensitive than either the SDMT or the DSST based on comparison with
baseline measures.
In a 1998 study by McCrea et al., researchers administered the Standardized
Assessment of Concussion (SAC) to 568 high school and college football players before
the 1995 and 1996 seasons. The SAC consists of a brief measure of orientation, short¬
term delayed verbal memory (5 words recalled immediately and after a 5 minute delay), a
brief measure of attention and shot-term memory (reverse Digit Span), and concentration
(recitation of the months of the year in reverse order), as well as several exertional
exercise in order to create conditions of intracranial pressure during which post
concussive symptoms are most likely to be seen. The sum of the scores is added to
produce a composite total with a maximum score of 30. Of the 568 individuals tested at
baseline, 33 subsequently experienced concussion and were tested immediately. 28 of
the 33 were also tested 2 days after the injury occurred. McCrea found that the
concussed players performed significantly below both the control group and their own
baseline composite measures on the SAC. At 48 hours, he indicated the concussed
players had returned to their baseline level. He concluded that this brief battery of
modified neuropsychological measures is sensitive to the cognitive deficits obtained from
minor head injury incurred in football, and suggests that his observation that all
concussed players were not different from baseline at 48 hours is consistent with other
findings. Of note, however, was the lack of attention to the effects of practice, and the
possibility that they may have contributed to improvements seen at the follow-up testing,
thus potentially masking any persistent cognitive effects of concussion. Further, the lack

27
of matched controls excludes the researchers from making comment on the potential
effects of age and education level on results obtained, although they contend that these
variables have “minimal” effect on performance and that there were no significant
differences between the scores of high school and college players.
A more recent test-retest study by Barr and McCrea (2001) described the results
of the administration of the SAC to concussed players immediately following the injury
compared to their baseline performance. A control group of non-concussed players was
also tested at the same time intervals. These researcher found that using both a reliable
change index approach and a linear regression approach were successful at identifying
detriments in cognitive performance immediately following concussion. Receiver
operating characteristic curves were used to determine the statistical significance of the
models in accurately classifying a player as concussed or not concussed.
A study by Collins et al. (1999) tested 393 athletes from 4 university football
teams using a brief neuropsychological battery that included the Hopkins Verbal
Learning Test (HVLT, verbal learning, and delayed memory), Trail Making Tests A and
B, Digit Span test, Symbol Digit Modalities Test, Grooved Pegboard Test, and Controlled
Oral Word Association. In addition the athletes completed the Concussion Symptom
Scale, a self-report, Likert scale of post concussion symptoms. Subsequently, the group
retested 16 individuals who suffered concussion within 24 hours of the incident, and
within 3, 5, and 7 days afterwards. Collins concluded, after performing a discriminant
analysis on the resulting data, that the use of this brief neuropsychological battery
resulted in an overall correct classification rate of 89.5% for the concussed and control
groups at the 24-hour testing period. In particular, he noted that concussed individuals

28
were significantly worse on the Hopkins Total score and Hopkins Delay score than the
controls, and that moderate differences between the groups persisted until the 5-day
testing. While not included in the analysis of concussed players, Collins, et al. analyzed
the baseline scores of all 393 athletes and discovered an interaction between history of
diagnosed learning disability and history of previous concussion, and significantly lower
baseline performance on two measures (Trail Making Test B and Digit Symbol
Modalities Test). This finding suggests that these demographic variables may be
important to examine and take into account when evaluating the neuropsychological
performance of an athlete after concussion.
Reliable Change
The tests of cognitive change and recovery of function described above generally
have utilized one of two methods to indicate recovery after concussion and fitness to
return to play. One assesses a player’s return to baseline functioning (Barth et al., 1989;
McCrea et al., 1998), and the other a comparison to a control group involving the
determination of differences between serial testing results (Collins et al.,1999; Maddocks
and Saling, 1996). Hinton-Bayre et al. (1999) suggested that the former does not
properly account for the effects of practice, while the latter provides limited information
without the availability of “suitable” normative data. The results of neuropsychological
testing are influenced by many factors in addition to the cognitive effects of minor head
injury. Especially in repeated, or serial testing, as is performed in the evaluation of
recovery of a concussed athlete in order to determine retum-to-play status, the results of
testing may be influenced by such things as the reliability of the test being used, practice
effects, and the attitude of the test taker (Temkin et al., 1999). Even when there is no true

29
change in cognitive status, these variables may suggest that a change has taken place. In
this sense, the ability to detect true change in an individual case can be affected by many
factors. Temkin et al. (1999) examined four statistical models in order to determine
which was most sensitive to detecting “real change” in the individual case. They used 7
test measures and retest scores from 384 “neurologically stable” individuals in the
comparison of four statistical models in order to address this question. These models
were the basic Reliable Change Index (RCI), the RCI with correction for practice effects,
a simple regression model, and a multiple regression model. Each of these models was
used in order to generate a predicted retest score which was then compared to the actual
retest score. The difference between the two, or residual, was then compared to a
confidence interval that was calculated to contain 90% of those scores expected given no
real change. This method was based on Za=Z.95=1.645. This value suggests that 5% of
scores were thought to be significantly higher than the normal population, while 5% were
assumed to be significantly lower. In other words, any residual score lying outside this
interval was assumed to reflect real change.
Reliable Change Index
The Reliable Change Index was originally developed to assess the effectiveness
of clinical interventions (Hinton-Bayre et al., 1999). It was used by Jacobson and Truax
(1991) with the idea that clinically significant change was related to a return to normal
functioning. The model is based on the assumption that a change in test scores from one
time to the next is significant if the size of that change is large compared to the standard
error of the difference of the test. Jacobson and Truax (1991) indicate that the error
variance of a test is calculated through the use of test-retest reliability and the variation of

30
scores around the mean. According to Hinton-Bayre, the formula for obtaining an
individual’s RCI score is:
RClJ-^1
Sdiff
In this instance, xi is the individual’s first score, is the retest score and Sdifr is the
standard error of the difference, which is calculated as follows:
The standard error of the measurement, Se , is obtained through the use of the standard
deviation of the control group, S. In addition, the reliability of the measure, r\2, is used in
obtaining this statistic:
S£ = St]1 - rl2 (Hinton-Bayre et al., 1999)
Again, in the use of this statistic, an RCI z value exceeding 1.96 is considered to be a
significant change.
In the analysis of Temkin et al. (1999), a change was considered to be reliable,
using the RCI, if the absolute value of the post-test minus the pre-test exceeded the
standard error of the test-retest difference of the normative sample multiplied by the z-
score cutoff described above. This created a prediction interval that should contain 90%
of the scores in a normal population. Any change score lying outside of this interval in
either direction was considered “real” change.
Reliable Change Index with Correction for Practice
The second model used by Temkin et al. (1999) was a slight variation on the RCI
model described above. This model has been used by others in the evaluation of
cognitive outcome following various surgical procedures (Chelune et al., 1993; Hermann

31
et al., 1996; Kneebone et al., 1998; Sawrie et al, 1996). In this model, practice effects for
individual tests are determined by calculating the mean difference between the test and
retest scores of the normative population. This mean difference is then added to the
baseline score of the individual in question in order to calculate a predicted follow-up
score. A reliable change is then determined by the amount of difference between the
actual retest score and the predicted. A confidence interval is determined just as it was
for the basic RCI. That is, the confidence interval is determined by the standard error of
the difference of the normative sample multiplied by the z-score that determines the
cutoff point (generally Z.95).
Linear Regression
The third model utilized by Temkin et al. (1999) was a basic linear regression. A
formula that can be used for predicting any score in the population under investigation is
calculated by regressing the retest score of the normative sample on their initial scores.
Originally suggested by McSweeny et al. (1993), Temkin suggests that this model
controls for both practice effects and regression toward the mean. Reliable change is
determined again by examining the difference between the observed retest score and the
predicted score. A prediction interval is calculated based on the residuals estimated by
the equation multiplied by a z-cutoff score (again, generally Z.95).
Multiple Regression Model
The final model investigated by Temkin et al. (1999) incorporated the use of
stepwise linear regression in order to determine a predicted retest score. Multiple
measures were used in the stepwise regression which the investigators thought might be
important in determining follow-up scores. These included baseline test score, test-retest

32
interval, demographic variables, and a measure of overall neuropsychological
competence. In order to account for nonlinear relationships between test and retest score,
these investigators also included the square and cube of baseline scores in their multiple
regression models for each measure. Temkin explains that variables were added to the
stepwise linear regression if they had the “highest partial correlation among the variables
not already in the equation, and if the significance level associated with the variable was
under .05.” Variables were removed from the equation if, after adding another variable,
the significance was above .10.
The results of Temkin et al. (1999) suggest that the prediction intervals associated
with the four models became narrower, and therefore more sensitive to the detection of
change, with the growing complexity of the methods used. In this instance, the term
“complexity” does not necessarily refer to the difficulty of mathematical operations, but
to the amount of error variance accounted for (e.g., test-retest reliability, practice effects,
regression to the mean, etc.). That is, the RCI was less sensitive than the RCI with
practice effects, which was less sensitive than the linear regression, which in turn was
less sensitive than the multiple regression model. However, they indicate that the RCI
produced high error rates due to the non-normality of the population residuals. This was
in large part due to the practice effects seen in the individual measures, which caused the
population data to become somewhat skewed. When practice effects were taken into
account, the residuals better estimated a normal curve, and predicted scores were more
closely correlated with observed scores. They report that overall prediction accuracy was
similar for the last three models, though, with the addition of demographic variables in
the multiple regression model, significantly better prediction was possible.

33
Specific Aims and Hypotheses
Given the magnitude of the occurrence of head injury in the United States, and the
subsequent cognitive as well as psychosocial consequences inherent to it, the importance
of detecting neuropsychological deficits is clear. The literature suggests that minor head
injury in sports, a particularly common occurrence in a college football population, varies
significantly from person to person, and that subtle deficits may be indicative of the
lingering effects of concussion. The assessment of recovery from concussion is
particularly important when one considers the potential effects of repeated concussion
and its long-term consequences (Cantu and Voy, 1995; McCrory and Berkovic, 1998).
As Collins et al. (1999) have indicated, the possibility of a correlation between multiple
concussion and reduced baseline neuropsychological functioning make the assessment of
concussion a particularly salient issue in the student athlete population.
Studies on the effects of minor head injury to this point have produced varying,
often contradictory results. This is in no small part due to the variability of the
methodology utilized. In many cases, investigators have used inappropriate normative
samples or comparison groups, while in others, factors contributing to change (or lack
thereof) in serial testing have not been taken into account. More sensitive and sound
statistical measures are needed in the assessment of head injury and the period of
recovery after head injury.
The present study is designed to further the research on the sensitivity of
neuropsychological measures over serial testing sessions in assessing the functioning of
individuals suffering minor head injury in a college football population. The aims of this
investigation are (1) to analyze the effectiveness of various statistical models in detecting

34
ongoing cognitive sequelae in concussed college football players via serial
neuropsychological testing, (2) to determine appropriate thresholds for test performance
resulting in acceptable sensitivity and specificity classification values, (3) to determine
those functions which appear most indicative of enduring cognitive complaints over the
7-day period following concussion, and (4) to calculate representative models in order to
aid in prediction of cognitive recovery following concussion. It is hypothesized that each
progressively “complex” model will exhibit increasing sensitivity in terms of
demonstrating lower test performance relative to baseline at the retest session. It is also
hypothesized, based on the findings of Temkin et al. (1999), that only the most complex
models (i.e., regression models) will allow for accurate classification of concussed
individuals beyond the 24-hour test session. As was the case in the Temking et al. (1999)
study, the term “complex” is used here to denote the level to which potential error
variance is taken into account. Therefore, in relation to the current study, it is
hypothesized that those models which account for more potential error variance will be
more successful at detecting real post-concussive cognitive decline at the more distal
testing sessions.

CHAPTER TWO
METHODS
Subjects
This study included the subjects that were originally reported by Collins et al.
(1999) as well as others from a continuing data collection project at the University of
Florida. Participants included male football players from four universities: the University
of Florida, Michigan State University, the University of Pittsburgh, and the University of
Utah. All participants in the study were volunteers who agreed to participate in the study
without compensation and without financial responsibility. All participants on the
football teams of these respective schools were asked to participate.
In the initial meeting with the student athlete, baseline data was collected
regarding age, football position, handedness, college entrance exam scores (SAT and/or
ACT), high school grade point average, highest level of education achieved by mother
and father, occupation of mother and father, history of diagnosed learning disability
(LD), history of diagnose attention deficit-hyperactivity disorder (ADHD), history of
neurological problems (e.g., epilepsy, migraine headaches, etc.), history of significant
medical illness (e.g., asthma, diabetes, heart problems, etc.), history of psychological
illness (e.g., depression, anxiety, etc.), history of alcohol and substance abuse, and history
of recent medication use. In addition, each participant was asked to provide a history of
previous concussion (including date, description of circumstances surrounding the
incident, incidence of loss of consciousness, incidence of post concussive symptoms,
35

36
length of symptom presentation, and the results of imaging tests if any). All concussions
were subsequently classified according the grading system outlined by the American
Academy of Neurology (1997).
Protocol
Baseline
Approval by the Institutional Review Boards for each respective school was
obtained. Each participant was provided the opportunity to review the nature, purpose,
and projected use of the data with either a Ph.D. psychologist or a doctoral level student.
All data were collected by Ph.D.-level psychologists, doctoral students, or in some
instances, team physicians and athletic trainers who had been trained in administering
this brief battery. Each had attended a training session in test administration in order to
ensure standardized administration techniques. Tests were scored in a standardized
fashion, and the results from other institutions were sent to this investigator for entry into
a centralized database.
Each participant was administered a brief neuropsychological battery including
the following tests: Hopkins Verbal Learning Test (HVLT), Trail Making Tests A and B,
the Digit Span subtest of the Wechsler Adult Intelligence Scale-Revised, Symbol Digit
Modalities Test (DSMT), Grooved Pegboard, and the Controlled Oral Word Association
test (COWA). This battery is one that is currently used in a professional football study
on the effects of concussion and was recommended by Lovell and Collins (1998) for
study of concussion in a college football population.
The Hopkins Verbal Learning Test (Brandt, 1991) consists of a presentation of 12
words belonging to three semantic categories. Three learning trials are administered,

37
after which a delay period is interjected for approximately 15-20 minutes. The subject is
then asked to recall as many words as they can remember. After this delayed free recall,
a 24-word recognition list is presented which contains all of the target words in addition
to 6 semantically related words and 6 unrelated words. As the words are read to the
subject, they are simply asked to identify those words that were presented previously.
This test allows for a measure of short-term verbal memory, attention, concentration, and
learning. In addition, a measure of delayed verbal memory is calculated as well as a
score for recognition memory. In addition, within the recognition trial, scores for related
false positive responses and unrelated false positives are calculated. A discrimination
score is also calculated to determine the subject’s ability to differentiate between
previously presented words and foils. The HVLT has six alternate forms (Lezak, 1995).
For the purposes of this study, Hopkins Total score and Hopkins Delay score were
examined.
Trail Making Test A (Reitan and Wolfson, 1985) consists of 25 numbered circles.
The subject is asked to connect the numbered circles in the correct order (from 1 to 25) as
quickly as they can, without making any mistakes or lifting their pencil from the paper.
Score on this measure is recorded in seconds to completion. Errors during the
completion of the task are immediately pointed out to the subject and they are directed to
continue from the point of their last correct mark. No score is kept for the number of
errors committed; it is felt that this is reflected in an increased time of completion. Trail
Making Test A is thought to be a measure of visual scanning and sequencing ability as
well as motor speed. Trail Making Test B. (Reitan and Wolfson, 1985) while similar in
appearance, encompasses circles with both letters and numbers. The subject is asked to

38
start with the number 1 and connect the circles in alternating increasing sequence (i.e., 1-
A-2-B-3-C, etc.). The score for this measure too is recorded in seconds to completion.
This measure is thought to reflect a person’s ability to effectively shift mental set and
utilize cognitive flexibility in addition to visual scanning, sequencing, and motor speed.
These functions are often associated with frontal-executive functioning (Spreen and
Strauss, 1998).
The Digit Span subtest of the WAIS-III (Wechsler, 1997) consists of the
presentation of increasingly long sequences of numbers. In the Forward condition, the
subject is simply asked to listen to the numbers and repeat them in the same order in
which they were given. In the reverse condition, they are asked to recite them
backwards. Each part is discontinued when the subject fails to correctly recall two trials
of numbers of the same length. This measure is thought to reflect attention and
concentration. For the purposes of this study, the Digit Span total score (forward plus
back) was examined.
The Symbol Digit Modalities Test (SDMT) (Smith, 1982) is similar to the Digit
Symbol subtest of the WAIS-R in that is requires the subject to provide quick
substitutions, but it in this measure, the subject is asked to write the correct number under
each symbol based on a key. Subjects are given 90 second to complete as many items as
they can. This test was chosen based on its ability to measure visual scanning and visuo-
motor speed. Immediately following its administration, the subject is asked to recall as
many of these associations as they can, which provides a measure of incidental learning.
The variable examined as part of this study was total number of symbols correctly
produced in the 90 second test administration.

39
The Grooved Pegboard test (Matthews and Klove, 1964) consists of a small board
containing a 5x5 matrix of slotted holes arranged in various angles. The subject is given
25 pegs, each the same shape, and asked to put them in successive slots starting with the
top row as quickly as they can without dropping. They are allowed only to hold one peg
at a time with one hand. Time to completely fill the holes are measured for each hand
independently (Lezak, 1995). This test was chosen for inclusion in the battery due to its
sensitivity to motor speed and coordination. Dominant hand and non-dominant hand
completion times were examined in the current study.
The Controlled Oral Word Association Test (COWA) (Benton and Hamsher,
1989) consists of three trials. In each, the subject is presented with a letter and asked to
produce as many words as they can until the examiner asks them to stop. They are told
that they must not use proper nouns or the same words with different endings. The
subject is given 60 seconds to produce as many words as they can. This measure was
chosen for its ability to reflect abilities of word fluency and retrieval. The current study
utilized total words produced over all three trials as the variable under examination.
In addition to these neuropsychological tests, the participant were administered
the Concussion Symptom Scale in order to examine baseline levels of symptoms
commonly associated with post concussion syndrome (e.g., headache, memory
difficulties, concentration difficulties, sensitivity to light or noise, etc.). This is a
subjective questionnaire utilizing a Likert-type scale.
In-season concussion evaluation
Those athletes who sustained a concussion during the football season, either in
practice or in a game, were evaluated with this same battery (utilizing alternate forms

40
where available) within 24 hours of the incident, and at 3, 5, and 7 days post-concussion.
In-season concussion was initially evaluated by trainers or team physicians and was then
referred for neuropsychological testing as part of this study. Grade of concussion was
evaluated based on the AAN guidelines. In addition, within 24 hours of concussion,
circumstances contributing to the concussion were obtained through interview with
trainers, and videotape of the incident (if available) was reviewed in some cases.
Football players within each respective team served as controls. They were
matched as closely as possible to the concussed player in terms of the following
variables: race, position, history of learning disability, and history of head injury.
Athletes were encouraged to continue with testing protocol over all test intervals.
However, some difficulty was encountered with attrition, especially in the control
population. Ultimately, data from participating individuals was excluded listwise. That
is, a player’s data was utilized over each retest interval only if they generated data for
each corresponding test session. This resulted in a decreasing study population over each
successive test interval. At the 24-hour test session, total number of concussed players
who had data at both baseline and 24 hours was 21. The number of controls at the same
interval was 13. For the Day 3 test session, the total number of concussed players was 19
with 11 controls. At Day 5 the number of concussed players was 18 with 10 controls. At
Day 7, there were 19 concussed and 6 control participants.
Data Analysis
Data analysis included the application of 7 statistical models of significance
detection. These were conceptually divided into three broad categories based on their
statistical methodology.

41
Basic Models
The first subgroup consisted of two basic, clinically-oriented approaches (Model
1 and Model 2). For Model 1, each of the concussed and control participants’ retest
scores for each measure was compared to their respective normative mean scores and
standard deviations, which were derived from the larger baseline group of football
players (n=409), ultimately resulting in a standardized z-score. Significant findings were
classified using a standard clinical approach (Reitan and Wolfson, 1985), where 1
standard deviation below the mean was interpreted as denoting mild impairment. In
Model 2, each concussed and control participants’ retest scores for each variable were
compared to their own baseline performance again using a criterion of 1 standard
deviation (based on the normative population) decrease in score to denote a finding
suggesting significant cognitive decline following concussion.
Reliable Change Indices
The second conceptual group of statistical models included three variants of the
Reliable Change Index (Model 3, Model 4, and Model 5). Model 3 involved the
application of a basic RCI analysis as described by Jacobson and Truax (1991) and used
in studies by Temkin et al. (1991) and Barr and McCrae (2001) to detect change in serial
neuropsychological testing. Model 4 utilized the application of the same RCI procedures
with correction for practice. For both Model 3 and Model 4, findings for each variable
were deemed significant if the retest score fell outside the computed change index in the
negative direction. For Model 5, each concussed participants’ change on the individual
variables from baseline to retest interval was directly compared to the average change
score for the control population over the same time period. A significant finding was

42
denoted by a change score which fell below a 95% confidence interval based on the
standard deviation of change scores for the control group on each variable and test
interval.
Regression Models
The third conceptual subgroup of statistical models included two regression
approaches (Model 6 and Model 7). Consistent with the research of Temkin et al. (1999)
and Barr and McCrea (2001), in Model 6, simple linear regression equations were
calculated, based on control population data, in order to predict retest scores for each
variable over the four test intervals. The observed scores were then compared to the
predicted scores and a standard z-score was calculated using the standard error of the
residuals for the model based on the control participants. Observed retest scores falling
beyond a significance value of 1.96 in the negative direction were classified as
significantly below expectation. In Model 7, a multiple regression equation approach
using history of diagnosed learning disability and history of previous concussion as
additional predictive indicators was analyzed.
Sensitivity and Specificity
Sensitivity values were calculated for each variable over the 4 test sessions for
each model. For individual variables, sensitivity was determined by that variable’s
probability of accurately classifying a concussed player as being significantly below
expectation based on the applied statistical model.
Each model was then examined over all test session to determine how many
variables from the battery of neuropsychological tests as a whole were found to have
been significantly below expectation for both concussed and control participants. A

43
classification cut-point (the optimal number of significant variables below expectation)
was determined based on the highest degree of sensitivity/specificity combination at each
time interval, where sensitivity was determined by the number of concussed participants
who were accurately classified as concussed at each cut-point, and specificity was
determined by the number of control participants who were accurately classified as “not
concussed” at those same cut-points.
Classification accuracy was then assessed using Receiver Operating Characteristic
curves, which provided a statistical indication of each model’s ability to accurately
distinguish between the concussed and nonconcussed participants relative to chance
(Hanley, 1989).

CHAPTER THREE
RESULTS
Demographic Data for Baseline Population
During the current study, data was collected using the above-described pre-season
neuropsychological test battery, and demographic variables were collected from a total of
443 football players at four universities. The total number of pre-season participants
from each location is presented in Table 2, and is presented graphically in Figure 1.
Table 2, Numbers and percentage of total pre-season participants’ locations
N Percent Cumulative Percent
University of Florida
155
35.2
35.2
Michigan State University
119
26.9
62.1
University of Utah
85
19.2
81.3
University of Pittsburgh
83
18.7
100.0
Location
Figure 1. Percent of baseline participants from each contributing school.
44

45
Table 3 and Figure 2 indicate the racial composition of the total number of baseline
participants. The majority of these were African American (47.9%), closely followed by
Caucasians (47.2%). There were significantly fewer Hispanic, Asian, and Samoan
participants.
Table 3. Racial composition of baseline population
N
Percent
Cumulative Percent
African American
212
47.9
47.9
Caucasian
209
47.2
95.0
Hispanic
6
1.4
96.4
Asian
2
0.5
96.8
Haitian
1
0.2
97.1
Samoan
13
2.9
100.0
RACE
Samoan
2.9%
African American
47.9%
Figure 2. Racial composition of baseline population across sites.
Table 4 indicates the frequencies of ages of the baseline population. The mean age of
baseline population participants was 19.85 years with a standard deviation of 1.75 years.

46
Table 4. Distribution of ages across the baseline population
Age (years)
N
Percent
Cumulative Percent
17
6
1.4
1.4
18
113
22.5
26.9
19
101
22.8
49.7
20
77
17.4
67.0
21
72
16.3
83.3
22
39
8.8
92.1
23
22
5.0
97.1
24
7
1.6
98.6
25
2
0.5
99.1
26
2
0.5
99.5
27
2
0.5
100.0
AGE
17 18 19 20 21 22 23 24 25 26 27
Figure 3. Distribution of ages for baseline population.
Table 5 indicates the frequencies of year in school for all participants in the baseline
population.
Table 5. Distribution of class status for the baseline population
N
Percent
Cumulative Percent
Freshman
171
38.6
38.6
Sophomore
92
20.8
59.4
Junior
95
21.4
80.8
Senior
67
15.1
95.9
Fifth Year Senior
18
4.1
100.0

47
Demographic Characteristics for Concussion and Control Participants
As indicated above, in-season concussion data utilizing re-administration of the
baseline neuropsychological test battery was conducted for those individuals who were
reported by team trainers to have received a concussion either during practice or during a
game. Initial re-testing took place within 24-hours of the identified concussive event, at
three days, five days, and seven days post-concussion. During the time of the
investigation, trainer-identified concussive events resulted in a total of 26 concussed
players, including three players who incurred two concussions during the time of the
study. Table 6 indicates the demographic characteristics of the concussed players.
Table 6. Demographics characteristics of the concussed players
N
Percent
Age
18
7
26.9
19
11
42.3
20
6
23.1
21
2
7.7
Race
African American
14
53.8
Caucasian
12
46.2
History of LD
Yes
6
23.1
No
20
76.9
When a concussed player was identified by a trainer, a matched control was
assigned to each for purposes of neuropsychological test result comparison over time. As
outlined above, control subjects were matched as closely as possible to concussed
individuals in terms of race, history of diagnosed learning disability, history of
concussion, and position. However, control subjects’ continued participation in the time-
sensitive research design was less than optimal. Ultimately, when control participants

48
failed to continue in the study, the time requirements did not allow for replacement with
another control participant. Subsequently, the total number of control participants was
less than the total number of concussed subjects, resulting in a total of 14 individuals.
Table 7 indicates the demographic characteristics of the control population.
Table 7. Demographic characteristics of the control population
N
Percent
Age
18
6
42.9
19
4
28.6
20
1
7.1
21
3
21.4
Race
African American
7
50.0
Caucasian
7
50.0
History of LD
Yes
2
14.3
No
12
85.7
Examination of the racial composition of the baseline population and the concussed
subjects indicates that both groups were nearly half African American and half
Caucasian.
A one-way ANOVA was performed to determine whether there were any
statistically significant differences between the baseline population (excluding those
players who eventually suffered concussion and their matched controls), the concussed
subjects, and their controls in terms of age, race, and year in school. Using status as the
between-subjects factor (baseline, concussed, control), no significant differences were
found between the groups in terms of racial composition [F(2,440) = 1.12, p=0.33], nor
were there any significant differences between the groups with regard to year in school

49
[F(2,440) = 2.97, p=0.052] although there was a trend for the concussed players to be of
lower school year. It was not surprising, therefore, that a significant effect for age was
observed [F(2,440) = 3.87, p=0.022]. However, paired t-tests using a Bonferroni
corrected p-value of 0.017 (.05/3) failed to demonstrate a significant difference between
the groups in terms of age. Again, a trend was observed for the concussed players to be
slightly younger than the baseline population as a whole, though this was not statistically
significant. For the purposes of this study, which examined baseline to retest session
performance (with 24 hours, day 3, day 5, and day 7), participants were excluded
listwise. That is, participants must have had a score at both test sessions for inclusion
into examination of performance for a particular test interval. This resulted in fewer
included participants for each successive time interval. Ultimately 21 concussed and 13
controls were included for the 24 hour analysis, 19 concussed and 11 controls for the Day
3 analysis, 18 concussed and 10 controls for Day 5, and 19 concussed and 6 controls for
Day 7.
Baseline Performance
As noted above, ten variables were examined from the battery of
neuropsychological tests administered during the current study. Those variables were
Hopkins total score, Trails A total time in seconds, Trails B total time in seconds,
Grooved Pegboard Test dominant and nondominant hand time in seconds, digit span
forward, digit span backward, Digit Symbol time in seconds, COWA total score, and
Hopkins Delay score. Table 8 shows the means and standard deviations for the total
Baseline population as well as the concussed participants and control subjects at baseline.

50
Table 8. Baseline Performance of total population, concussed, and controls
Measure
Total Baseline
n=409
Controls
n=13
Concussed
n=21
M
(SD)
M
(SD)
M
(SD)
Hopkins Total
24.96
(4.02)
25.77
(3.44)
24.19
(4.13)
Hopkins Delay
8.35
(2.09)
8.85
(1.77)
7.71
(1.98)
Trails A
20.99
(5.96)
21.97
(6.17)
22.18
(5.53)
Trails B
52.84
(17.39)
45.95
(13.13)
57.83
(24.83)
Digits Forward
9.05
(2.18)
8.31
(2.14)
9.00
(1.89)
Digits Back
7.20
(2.48)
7.08
(2.53)
6.67
(2.29)
Digit Symbol
58.75
(9.07)
64.92
(8.03)
55.81
(8.98)
Pegs Dominant
66.56
(10.50)
65.48
(8.41)
70.73
(13.79)
Pegs Nondominant 73.34
(11.81)
72.72
(11.78)
79.62
(16.96)
COWA Total
39.08
(9.23)
35.77
(6.70)
41.32
(10.21)
A series of one-way ANOVA’s was run in order to detect any significant
differences between the groups with regard to their baseline performance on any of these
ten measures. Table 9 provides a summary of those findings.
There was a significant finding with regard to baseline performance between the
three groups on the Digit Symbol test [F (2,440) = 4.14, p=0.02]. Subsequently, paired t-
tests were performed using a Bonferroni corrected significance alpha value of 0.017
(.05/3). Results indicated that the control subjects scored, on average, significantly
better than did the concussed participants (t = 2.99, p=0.013) at baseline. Further, there
was also a trend indicating that the control participants’ scores were significantly better
than the control group at baseline, although this did not reach significance after applying
correction for multiple t-tests. However, because the difference between the concussed
group and control group scores on the Digit Symbol test at baseline was found to be
significant, this variable was omitted from further analysis.

51
Table 9. One-way ANOVA’s using status as between-subjects factor
Measure
df
F
Significance
Hopkins Total
(2,440)
0.65
0.52
Hopkins Delay
(2,440)
1.35
0.26
Trails A
(2,440)
0.56
0.57
Trails B
(2,440)
1.82
0.16
Digits Forward
(2.440)
0.73
0.48
Digits Backward
(2,440)
0.48
0.62
Digit Symbol
(2,440)
4.14
0.02
Grooved Pegboard Dom
(2,440)
1.63
0.19
Grooved Pegboard Nondom
(2,440)
2.73
0.07
COWA Total
(2,440)
1.40
0.25
Overview of Repeated Testing
Repeated testing utilizing the above-described neuropsychological battery
generally yielded results consistent with historical reviews of test-retest performance for
those having suffered minor head injury. That is, the concussed participants in the
current study, on average, exhibited a decline (or a stasis) in performance across test
variables within 24 hours of concussion when compared to their baseline performance.
Examination of subsequent performances on days 3,5, and 7 days post concussive event
generally suggested a gradual improvement over these days. This is particularly evident
when comparing the graphical representation of these performances compared to the
control population’s average performance over this same time interval. Figures 4 through
12 represent the average performance of individuals in both the concussed and control
groups over the testing period.
Detecting Significant Decline Using Basic Statistical Methods
Generally speaking, neuropsychological testing involves the examination of an
individual after a known or suspected neurological insult has occurred.

52
® Hopkins Total Score
Testing Session
Figure 4. Hopkins Verbal Learning Test -Retest Performance for Both Groups
Trail Making Test A
Testing Session
Figure 5. Trail Making Test A Test-Retest Performance for Both Groups

53
Trail Making Test B
Test Session
Figure 6. Trail Making Test B Test-Retest Performance for Both Groups
Digit Span Forward
Test Session
Figure 7. Digit Span Forward Test-Retest Performance for Both Groups

54
Digit Span Backward
Test Session
Figure 8. Digit Span Backward Test-Retest Performance for Both Groups
Grooved Pegboard Dominant Hand
Test Session
Figure 9. Grooved Pegboard (Dominant Hand) Test-Retest Performance for Both Groups

55
Grooved Pegboard Nondominant Hand
Test Session
Figure 10. Grooved Pegboard (Nondom. Hand) Test-Retest Performance for Both
Groups
v
Test Session
Figure 11. COWA Test-Retest Performance for Both Groups

56
Hopkins Delay Score
in-season concussion
control
Figure 12. Hopkins Delay Performance for Both Groups
In most instances, the examiner is without the benefit of prior knowledge regarding the
individual’s baseline cognitive abilities. Therefore, certain assumptions are made with
regard to that person’s premorbid cognitive abilities, and a comparison is made between
their status-post neuropsychological performance and some normative group that is
deemed appropriate. In many circumstances, an arbitrary cut-point is determined for a
neuropsychological measure, a performance under which (or significantly different from)
is considered to indicate abnormality. In this study, the group of more than 400 football
players who participated in the baseline study provided an appropriate comparison group.
Model 1
The most basic model (referred to as Model 1 for the purposes of this study) for
detecting if a concussed player’s performance suggested ongoing concussive sequelae
was a comparison at each post-concussive test interval to the normative baseline
population mean score for that measure. Using a liberal criterion, a difference of greater
than 1.0 SD below that expected (normative mean) score would suggest a performance

57
below expectation, perhaps indicating ongoing concussive effects. Using that approach,
Table 10 indicates the cut-points for the 9 neuropsychological measures that were
studied. These were derived by determining a score for each particular measure which
was 1 SD worse than the mean.
Using these criteria, the number of concussed and control participants who were
classified as significantly below the cut-point was determined for each test session
(within 24 hours of concussion, 3 days, 5 days, and 7 days post-concussion). Table 11
indicates the sensitivity of each measure for each test session.
Table 10. Cut Points Using 1 SD Difference from Normative Population Mean (Model 1)
Measure
Normative Mean Score Standard Deviation Cut-Point
Hopkins Total
24.96
4.02
20.94
Trails A (sec)
20.99
5.96
26.95
Trails B
52.84
17.39
70.23
Digit Span Forward
9.05
2.18
6.87
Digit Span Backward
7.20
2.47
4.73
Grooved Pegboard (D)
66.56
10.50
77.06
Grooved Pegboard (N)
73.34
11.81
88.15
COWAT
39.08
9.23
29.85
Hopkins Delay
8.35
2.09
6.26
As Table 11 indicates, within 24 hours of concussion, the most sensitive measure
appears to be total words recalled after the delay period for the Hopkins Verbal Learning
Test. However, none of the measures alone was particularly sensitive. Examining the
battery as a whole using Model 1, the number of concussed and control participants who
scored below the cut-points for each measure were totaled. Tables 12-15 show the
cumulative number for each group who were significantly different than the normative
mean group for all 9 variables examined.

58
Table 11. Sensitivity for Neuropsychological Measures for Model 1
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.29
0.11
0.17
0.05
Trails A
0.29
0.05
0.00
0.11
Trails B
0.19
0.11
0.06
0.00
Digit Span Forward
0.19
0.16
0.00
0.05
Digit Span Back
0.14
0.05
0.05
0.00
Grooved Pegboard D
0.14
0.16
0.06
0.05
Grooved Pegboard N
0.38
0.16
0.11
0.21
COWAT
0.19
0.05
0.11
0.00
Hopkins Delay
0.43
0.42
0.11
0.16
Table 12 demonstrates that, within 24 hours of concussion, using the criterion of
falling below the cut-points (derived using Model 1) for 2 or more variables achieved the
greatest (Se + Sp) value of 1.37. That is, 11 of the 21 concussed individuals fell below
the cut-point for 2 or more variables while only 2 of the 13 control participants did so at
that test session. However, as has been noted in the literature, in the case of sports
concussion it is likely better to err on the side of caution, and while using a criteria of
only one variable below significance value resulted in a greater number of false positive
errors (Sp=0.38), it also resulted in a far better true positive rate (Se = 0.91).
Table 12. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables Within 24 Hours with Model 1
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
19
8
0.91
0.38
(1.29)
2
11
2
0.52
0.85
(1.37)
3
8
0
0.38
1.00
(1.38)
4
3
0
0.14
1.00
(1.14)
5
2
0
0.10
1.00
(1.10)
6
2
0
0.10
1.00
(1.10)
7
1
0
0.05
1.00
(1.05)
8
1
0
0.05
1.00
(1.05)
9
1
0
0.05
1.00
(1.05)
¿■^Sensitivity, S/?=Specificity

59
Again, Table 13 demonstrates that the highest combined sensitivity and
specificity rates were achieved at Day 3 using a criteria of 3 or more variables falling
below their respective cut-points (Se + Sp = 1.12). However, sensitivity is extremely
negatively affected by the more conservative inclusion criteria. This more conservative
criterion, however, also resulted in a significantly higher specificity rate, demonstrating
the need for a pre-determined tolerance for higher false-positive rates when attempting to
detect ongoing effects of concussion.
Table 13. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 1
No. of Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.00
0.0
(1.00)
1
10
5
0.53
0.55
(1.08)
2
6
3
0.32
0.73
(1.05)
3
4
1
0.21
0.91
(1.12)
4
2
0
0.11
1.00
(1.11)
5
1
0
0.05
1.00
(1.05)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/>=Specificity
At day 7 (Table 15), the highest combined sensitivity and specificity score was achieved
using a criterion of one variable below cut-point (1.25).
Certainly, the sensitivity and specificity values using a comparison of
performance at these time intervals to a normative population suggests that the model
itself has some value within 24 hours of concussion. Although the model is not an
optimal one, it was able to detect a degree of performance aberrance after concussion
compared to the appropriate standard scores, though specificity was conversely poor.

60
Receiver operating characteristic curves were calculated for this model using Analyse-It
software for Excel at each of the time interval and are shown in Figures 13-16.
Table 14. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 1
No. of Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
18
10
1.00
0.0
(1.00)
1
8
5
0.44
0.50
(0.94)
2
3
1
0.17
0.90
(1.07)
3
0
1
0.00
0.90
(0.90)
4
0
1
0.00
0.90
(0.90)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/?=Specificity
Table 15. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 1
No. of Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.00
0.0
(1.00)
1
8
1
0.42
0.83
(1.25)
2
4
0
0.21
1.00
(1.21)
3
0
0
0.00
1.00
(1.00)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/?=Specificity
Figure 13 shows the receiver operating characteristic curve for the classification
rate for Model 1 at 24 hours. The diagonal line represents chance level, and by
definition, the area underneath it is equal to 0.50. The area underneath the curve for
Model 1 at 24 hours was 0.773 (95% Confidence Interval = .62-.93; SE = 0.08) and was
found to be statistically different than chance (p=0.0003). Using generally accepted

61
accuracy classification, this model at 24 hours after concussion would be characterized
only as fair.
Figure 13. ROC Curve for Model 1 at 24 Hours
At Day 3, detection of significant scores on the test variable compared to the baseline
normative population was not better than chance using Model 1 (Figure 14). The area
under the curve (AUC) was calculated as 0.56 (95% Confidence Interval = .35-.77; SE =
0.11) and was not statistically different than the reference diagonal (p=0.30). A similar
pattern was observed at Day 5 as can be seen in Figure 15. Using Model 1, the detection
rate was not statistically difference than chance. The AUC was found to be 0.517 (95%
Confidence Interval = 0.29-0.74; SE = 0.12).
The calculated AUC for Model 1 at day 7 was 0.65 (95% Confidence Interval =
0.41-0.88; SE = 0.12), and again, was not statistically better than chance (p=0.11) at
differentiating between concussed participants and controls (Figure 16).

Sensitivity (true positives) E Sensitivity (true positives)
62
ROC Curve for Model 1 at Day 3
re 14. Receiver Operating Characteristic Curve for Model 1 at Day 3
ROC Curve for Model 1 at Day 5
Figure 15. Receiver Operating Characteristic Curve for Model 1 at Day 5

63
ROC Curve for Model 1 at Day 7
Figure 16. Receiver Operating Characteristic Curve for Model 1 at Day 7
Model 2
In the circumstance that one has baseline test measures for an individual,
comparison of their performance following neurological insult to that baseline
performance may be more appropriate in determining change in cognitive status during
subsequent testing sessions. In the second basic model examined in this study (here
referred to as Model 2), performance of concussed individuals at each testing session
following concussion was compared to their baseline performance. This was achieved by
determining significance based again on a liberal criterion of 1 SD (derived from the
normative population as a whole, n=409) below their baseline performance. In this
manner, each subject was afforded an individualized comparator as illustrated for one
concussed participant in Table 16.

64
Please note that those variables for which time is the recorded score, a cut-point
was determined by the addition of 1 SD to the baseline performance, and ordinal
variables were rounded to the nearest whole number. Therefore, as can be seen in Table
13, for this participant’s time on Trails A to be classified as significantly different at 24
hours than baseline, his score would have to increase by nearly 6 seconds, from 33” to
nearly 39” total time.
Table 16. Example of Cut-Point Determination for Study Participants
Measure Baseline Performance 1 SD(from normative pop.) Cut-Point
Hopkins Total
20
4.02
16
Trails A
33”
5.96”
38.96”
Trails B
68”
17.39”
85.39”
Digits For.
12
2.18
10
Digits Back.
6
2.48
4
Pegs Dominant
73”
10.50”
83.50”
Pegs Nondominant
100”
11.81”
111.81
COWAT
52
9.23
43
Hopkins Delay
8
2.09
6
The same procedure was used for each concussed and control participant, and their retest
results at each testing session were compared to their appropriate cut-point score in order
to determine if there was a significant change from their baseline performance. The
sensitivities for individual variables were calculated for each testing session and are
presented in Table 17.
As was noted in Model 1, the sensitivities for the individual variables was
relatively poor at detecting significant change. The most sensitive individual variables at
24 hours post-concussion were Hopkins Total score and Grooved Pegboard nondominant
hand completion time. As would be expected, using Model 2, the individual variables
were most effective in detecting significant change relative to baseline performance
within 24 hours of concussion, with a trend toward decreasing sensitivity at each

65
successive testing session. The one exception was the Hopkins Delay score, which
remained relatively stable over time.
Table 17. Sensitivity for Neuropsychological Measures for Model 2
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.43
0.11
0.17
0.11
Trails A
0.29
0.16
0.11
0.11
Trails B
0.29
0.05
0.06
0.05
Digit Span Forward
[ 0.14
0.11
0.06
0.05
Digit Span Back
0.19
0.11
0.06
0.05
Grooved Pegboard D 0.19
0.05
0.00
0.05
Grooved Pegboard N 0.43
0.16
0.06
0.16
COWAT
0.14
0.11
0.00
0.00
Hopkins Delay
0.33
0.26
0.17
0.26
The battery as a whole was again examined to determine the number of concussed
and control participants who scored below their individual cut-points across variables
using Model 2. Sensitivity and specificity characteristics of the Model for each test
session are presented in Tables 18-21.
As Table 18 demonstrates, the greatest degree of classification accuracy occurred
at 24 hours for Model 2 when a criterion of a significant reduction in score on 1 or more
variables was used. Specificity greatly increased when the criteria was two or more
significant variables, though this also resulted in the largest single drop in sensitivity.
Using the same criterion on Days 3 through 7, a predictable drop in sensitivity was
observed over time, while specificity remained relatively constant. Notably, this model
saw its lowest sensitivity using the one-variable criterion at the Day 5 test session (Table
20), even relative to the Day 7 test results. This pattern was consistent in most of the
models examined as part of this study.

66
Table 18. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables Within 24 Hours with Model 2
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
17
5
0.81
0.62
(1.43)
2
10
1
0.48
0.92
(1.40)
3
8
0
0.38
1.00
(1.38)
4
4
0
0.19
1.00
(1.19)
5
4
0
0.19
1.00
(1.19)
6
3
0
0.14
1.00
(1.14)
7
3
0
0.14
1.00
(1.14)
8
1
0
0.05
1.00
(1.05)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5p=Specificity
Table 19. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 2
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.0
0.0
(1.00)
1
12
5
0.63
0.55
(1.18)
2
4
2
0.21
0.82
(1.03)
3
2
0
0.11
1.00
(1.11)
4
1
0
0.05
1.00
(1.05)
5
1
0
0.05
1.00
(1.05)
6
1
0
0.05
1.00
(1.05)
7
1
0
0.05
1.00
(1.05)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5/>=Specificity
As with the use of this model in the previous testing sessions, the greatest degree
of sensitivity was achieved when a criteria of one significant variable was used.
However, this also resulted in falsely classifying greater than half of the control
participants as concussed. Receiver operating characteristic curves for Model 2 are
presented in Figures 17 through 20.

67
Table 20. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 2
No. of Sig.Measures
Concussed
Control
Se
Sp
(iSe + Sp)
0
18
10
1.0
0.0
(1.00)
1
6
3
0.33
0.70
(1.03)
2
4
1
0.22
0.90
(1.12)
3
2
1
0.11
0.90
(1.01)
4
0
1
0.00
0.90
(0.90)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5/?=Specificity
Table 21. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 2
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.0
0.0
(1.00)
1
10
4
0.53
0.33
(0.86)
2
5
0
0.26
1.00
(1.26)
3
1
0
0.05
1.00
(1.05)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 6p=Specificity
Data for the 24-hour test session resulted in a ROC curve suggesting that this
model had “fair” classification accuracy. AUC was calculated to be 0.788 (SE =
0.0773; 95% Confidence Interval = 0.636-0.939), and was significantly greater than
change with a significance value of p<0.0001. At the Day 3 test session (Figure 18),
Model 2 was found not to be statistically better than chance at classifying the concussed
and control participants (AUC = 0.566, SE = 0.1135, p<0.28).

68
ROC Curve for Model 2 at 24 Hours
Figure 17. Receiver Operating Characteristic Curve for Model 2 at 24 Hour Test Session
ROC Curve for Model2 at Day 3
Figure 18. Receiver Operating Characteristic Curve for Model 2 at Day 3 Test Session

69
ROC Curve for Model 2 at Day 5
Figure 19. Receiver Operating Characteristic Curve for Model 2 at Day 5 Test Session
ROC Curve for Model 2 at Day 7
No discrimination
Model 2
Figure 20. Receiver Operating Characteristic Curve for Model 2 at Day 7 Test Session
Similarly, Model 2 was found not to be statistically different than chance at the
Day 5 and Day 7 test sessions, as shown in Figures 19 and 20. The AUC for Model 2 at
Day 5 was found to be 0.522 (SE = 0.1154, 95% Confidence Interval = 0.296-0.748,

70
p<0.42), while the AUC for Model 2 at Day 7 was 0.518 (SE = 0.1211, 95% Confidence
Interval = 0.280-0.755, p<0.44).
As the ROC analysis suggested, in the repeated testing of the concussed football
players, comparing their re-test performance to either an appropriate normative sample
(Model 1) or to their own baseline performance using a liberal criteria of 1.0 SD reduced
performance (Model 2) yielded rather poor sensitivity for the individual variables
included in this test battery. Further, examining the variables collectively, both models
were only fair in accurately discriminating between concussed and control participants
even within 24 hours of concussion, a time period in which the cognitive sequelae of
concussion are generally most evident.
Model 1 vs. Model 2 ROC at 24 Test Session (diff = 0.022,
p<0.82)
Figure 21. Comparison of Model 1 and Model 2 at the 24-Hour Test Session
Neither was better than chance at discriminating between the two groups at the Day 3,
Day 5, or Day 7 test sessions. A comparison of Model 1 and Model 2 for the 24 Test

71
Session suggested that Model 1, with an AUC of 0.788 was slightly more sensitive than
Model 2, which had an AUC of 0.766. However, there was not a statistically significant
difference between the two (p=0.82) as illustrated in Figure 21.
Reliable Change Index Models
Model 3
Three variants of the Reliable Change Index (RCI) were subsequently applied to
the data set. The first, a basic RCI model (referred to as Model 3 for the purposes of this
study), essentially examined whether the test-retest performance difference for each
individual was large relative to the error variance of the variable under examination,
providing indication of a change from baseline performance. The following formula was
applied for each individual’s performance on each variable:
In this equation, x, is the individual’s score at the Baseline testing session, x2 is the
individual’s score at the second test session (in this case, within 24 hours of concussion,
day 3, day 5, and day 7), and sdiff , the standard error of the difference, was calculated
using sE, the standard error of the measure with the equation:
sE was derived from the correlation of the test-retest scores of the control group for each
variable (ru) and the standard deviation (s) of the control group at baseline using the
equation:

72
An index, outside of which a re-test score was considered significant, was calculated by
multiplying the sdiff by a cutoff level of 1.96, a widely accepted statistical significance
level. Tables 22 through 25 indicate the data used to calculate these measures as well as
the RCI ranges used as comparitors for each variable at each testing session.
Table 22. Data Used in the Calculation of Reliable Change Indices at 24 Hours
Measure
rn
c b
d
c
SE
d
Sdiff
RCI(+/-)
Hopkins Total
0.841
3.52
1.40
1.99
3.89
Trails A
0.204
5.59
4.99
7.05
13.82
Trails B
0.623
11.21
6.88
9.74
19.08
Digits Forward
0.942
2.27
0.55
0.77
1.51
Digits Backward
0.548
2.41
1.62
2.29
4.49
Pegs Dominant
0.720
7.87
4.17
5.89
11.55
Pegs Nondominant
0.737
12.02
6.16
8.72
17.09
COWAT
0.282
6.97
5.91
8.35
16.37
Hopkins Delay
0.682
1.73
0.98
1.38
2.70
a rl2 based on the correlation for control participants between baseline and 24 hour performance, listwise n=13
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference
Table 23. Data Used in the Calculation of Reliable Change Indices at Day 3
Measure
rn
sd
C
SE
d
Sdiff
RCI(+/-)
Hopkins Total
0.679
3.52
1.99
2.82
5.53
Trails A
0.760
5.59
2.74
3.87
7.59
Trails B
0.215
11.21
9.93
14.05
27.53
Digits Forward
0.672
2.27
1.30
1.84
3.60
Digits Backward
0.659
2.41
1.41
1.99
3.90
Pegs Dominant
0.704
7.87
4.28
6.06
11.87
Pegs Nondominant
0.617
12.02
7.44
10.52
20.62
COWAT
0.429
6.97
5.27
7.45
14.60
Hopkins Delay
0.419
1.73
1.32
1.87
3.66
a rl2 based on the correlation for control participants between baseline and Day 3 performance, listwise n=13
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference

73
Table 24. Data Used in the Calculation of Reliable Change Indices at Day 5
Measure
rn
b
Sd
C
SE
d
Sdiff
RCI(+/-)
Hopkins Total
0.628
3.52
2.15
3.04
5.95
Trails A
0.392
5.59
4.36
6.16
12.08
Trails B
0.427
11.21
8.49
12.00
23.52
Digits Forward
0.418
2.27
1.73
2.45
4.80
Digits Backward
0.228
2.41
2.12
2.99
5.87
Pegs Dominant
0.297
7.87
6.60
9.34
18.30
Pegs Nondominant
0.735
12.02
6.19
8.75
17.15
COWAT
0.604
6.97
4.39
6.20
12.16
Hopkins Delay
0.386
1.73
1.36
1.92
3.76
a r 12 based on the correlation for control participants between baseline and Day 5 performance, listwise n=10
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference
Table 25. Data Used in the Calculation of Reliable Change Indices at Day 7
Measure
V
b
*d
c
SE
d
Sdiff
RCI(+/-)
Hopkins Total
0.492
3.52
2.51
3.55
6.96
Trails A
0.301
5.59
4.67
6.61
12.95
Trails B
0.365
11.21
8.93
12.63
24.76
Digits Forward
0.600
2.27
1.44
2.03
3.98
Digits Backward
0.497
2.41
1.71
2.42
4.74
Pegs Dominant
0.674
7.87
4.50
6.36
12.46
Pegs Nondominant
0.883
12.02
4.11
5.81
11.40
COWAT
0.025
6.97
6.88
9.73
19.08
Hopkins Delay
0.884
1.73
0.59
0.83
1.63
a rl2 based on the correlation for control participants between baseline and Day 7 performance
b sd is the standard deviation for the control group at baseline
c standard error of the measure
d standard error of the difference
Please note that a change was considered to indicate a significant difference between a
performance at a particular test session and the appropriate baseline performance only if
the difference exceeded the RCI cutoff point associated with a significant decrease in
performance at the second test session. That is, it was considered significant if the second
test performance on a particular measure was significantly below the baseline
performance based on the RCI.

74
Table 26 demonstrates the use of this method on one concussed participant for
two test variables. As is illustrated, for the variable ‘Hopkins Total words recalled,’ a
reduction of greater than 4 recalled words was calculated to be a significant change
relative to the error variance of the test as indicated by the sdiff . This participant’s
reduction of 8 words at 24 hours relative to his baseline performance resulted in a z-score
of-4.03, which, using a cut-point of z=1.96, was found to be a highly significant change
in the negative direction. That is, his performance on Hopkins Total was significantly
below his performance on the same measure at baseline. For the variable Trails A, which
is measured in seconds, his increase of 51” in total completion time from baseline to the
24 hour test session ultimately resulted in a Trails A time at 24 hours of 84”, greatly
exceeding the maximum value in the calculated RCI of 46.8”. This resulted in a z-score
of 7.23.
Table 26. Example of RCI Significance Determination for One Concussed Participant at
the 24 Hour Test Session
Measure
Observed Change
From Baseline
RCI Interval
z-score
To 24 Hours
Hopkins Total
-8 words
20+/-3.89 = (16-24)
-4.03
Trails A
+51.0”
33”+/- 13.82 = (19.2-46.8)
7.23
Table 27 presents the sensitivities for the individual measures at each test session using
Model 3, the basic RCI approach. Again, the data indicated that the sensitivities for the
individual variables were relatively poor, even within 24 hours, which was also seen in
Models 1 and 2. The sensitivities for the individual variables using Model 3 also

75
generally represent relatively lower values than in Model 2, which is not surprising since
the criteria used for Models 1 and 2 were more liberal (1.0 SD).
Table 27. Sensitivity for Neuropsychological Measures for Model 3
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.43
0.11
0.11
0.11
Trails A
0.14
0.11
0.00
0.00
Trails B
0.19
0.00
0.06
0.05
Digit Span Forward
0.29
0.05
0.00
0.00
Digit Span Back
0.14
0.00
0.06
0.05
Grooved Pegboard D
0.19
0.11
0.00
0.00
Grooved Pegboard N
0.29
0.05
0.00
0.16
COWAT
0.05
0.00
0.00
0.00
Hopkins Delay
0.38
0.16
0.11
0.37
The battery as a whole was again examined to determine the number of concussed and
control participants who scored below their individual cut-points across multiple
variables using Model 3. Sensitivity and specificity characteristics of Model 3 for each
test session are presented in Tables 28-31.
Table 28. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours using Model 3
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
16
1
0.76
0.92
(1.68)
2
10
0
0.48
1.00
(1.48)
3
7
0
0.33
1.00
(1.33)
4
4
0
0.19
1.00
(1.19)
5
3
0
0.14
1.00
(1.14)
6
2
0
0.10
1.00
(1.10)
7
1
0
0.05
1.00
(1.05)
8
1
0
0.05
1.00
(1.05)
9
0
0
0.00
1.00
(1.00)
5e=Sensitivity, 5p=Specificity
As with the first two models, Model 3 was most effective in detecting a concussed
participant when a criterion of one variable below the respective RCI cutoff was used.
Notably, the RCI model resulted in a much greater specificity value when using this

76
criterion than either Model 1 or Model 2, suggesting that the basic RCI model may be
more effective in separating the groups. This is not altogether surprising since the cutoff
criteria are based on the performance of the control population. At Day 3 (Table 29),
Model 2 saw a significant decline in sensitivity, even when using the minimal criteria of
one variable falling outside the RCI.
Table 29. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 3
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.0
0.0
(1.00)
1
4
2
0.21
0.82
(1.03)
2
3
2
0.16
0.82
(0.98)
3
1
0
0.05
1.00
(1.05)
4
1
0
0.05
1.00
(1.05)
5
1
0
0.05
1.00
(1.05)
6
1
0
0.05
1.00
(1.05)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5/>=Specificity
Table 30. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 3
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
18
10
1.0
0.0
(1.00)
1
4
1
0.22
0.90
(1.12)
2
1
1
0.06
0.90
(0.96)
3
1
1
0.05
0.90
(0.96)
4
0
1
0.00
0.90
(0.90)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
As was seen in the previous two models, detection of concussed players saw its
most extensive drop at the Day 5 test session (Table 30), but saw resurgence in sensitivity
at the Day 7 test session (Table 31).

77
Table 31. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 3
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.0
0.0
(1.00)
1
9
3
0.47
0.50
(0.97)
2
5
0
0.26
1.00
(1.26)
3
0
0
0.00
1.00
(1.00)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/?=Specificity
Receiver operating characteristic curves for Model 3 at each test session are
shown in Figures 22 through 25. As Figure 22 indicates, Model 3 was relatively good at
discriminating between concussed and control participants at the 24 hour test session.
Area under the curve was found to be 0.861 (SE = 0.0641, 95% Confidence Interval =
0.735-0.986), which was significantly better than chance level (pO.0001).
While the use of ModeB was generally good at distinguishing between concussed
and control participants within 24 hours of concussion, Figures 23-25 indicate that the
model performed no better than chance at Day 3, Day 5, and Day 7. The AUC for Model
3 at Day 3 was 0.514 (SE = 0.111, 95% Confidence Interval = 0.298-0.731, p<0.45);
AUC for Model 3 at Day 5 was 0.556 (SE = 0.111, 95% Confidence Interval = 0.338-
0.773, p<0.31); and the AUC for Model 3 at Day 7 was 0.553 (SE = 0.123, 95%
Confidence Interval = 0.312-0.793, p<0.33).

78
ROC Curve for Model 3 at 24 Hours
Figure 22. Receiver Operating Characteristic Curve for Model 3 at 24 Hour Test Session
ROC Curve for Model 3 at Day 3
Figure 23. Receiver Operating Characteristic Curve for Model 3 at Day 3 Test Session

79
ROC Curve for Model 3 at Day 5
Figure 24. Receiver Operating Characteristic Curve for Model 3 at Day 5 Test Session
ROC Curve for Model 3 at Day 7
No discrimination
Model 3
Figure 25. Receiver Operating Characteristic Curve for Model 3 at Day 7 Test Session
Model 4
The second RCI variant analyzed as part of the current study (referred to as Model
4) utilized the same data indicators as were used in the basic RCI model (Model 3), but

80
also included a correction for observed practice effects in the control population for
individual variables at each testing session. Consistent with the literature, practice
effects were calculated by finding the mean change in score over a test-retest period for
the comparison population. For the current study, this involved calculating the mean
change for the control population for each variable from baseline to 24 hour test session,
baseline to Day 3 test session, baseline to Day 5 test session, and baseline to Day 7 test
session. Again, individual subjects were omitted listwise. Table 32 indicates the mean
change scores for the control population.
Table 32. Mean Change Scores for Individual Variables for Control Participants
Measure
24 Hour(n=
=13) Day 3(n=ll)
Day5(n=10)
Day7(n=6)
Hopkins Total
0.38
-0.91*
-0.40*
0.50
Trails A
-3.69”
-5.03”
-4.87”
-8.48”
Trails B
2.34”*
-1.01”
-8.40”
-11.03”
Digits Forward
1.38
1.64
2.00
1.50
Digits Backward
0.77
1.73
2.50
0.67
Pegs Dominant
-2.97”
-4.05”
-4.20”
-8.07”
Pegs Nondominant
-0.61”
-3.35”
0.46”*
-6.55”
COWAT
3.08
5.45
5.80
1.67
Hopkins Delay
0.00*
-1.27*
0.60
-2.33*
“"indicates no practice effect detected
In order to correct for practice effect, the mean change score for a variable was
subtracted from each participants score on that variable at the retest session. The
resulting number was then entered into the RCI equation as outlined above, and
calculated as before with the basic RCI method in Model 3. Therefore, the equation for
(x2 -dn)-xx
sdiff
the RCI became:

81
where x, was the participant’s score on a variable at baseline, x2 was their score at the
retest session, and d n was the average change score for the control group on the variable
over that test-retest interval. The sdiff was calculated as in the basic RCI model using the
correlations and standard deviations for the control group as described above. Please
note that for those variables that were found, on average, not to have practice effects for
the control group at the retest session, no changes were made in the study participants’
retest score in the RCI model. This occurred five times, most frequently for the Hopkins
Verbal Learning Test Total and Delay scores, which was not unexpected given that these
tests used alternate forms over test sessions. Again, for the purposes of this study, scores
were classified as significant only if they exceeded their RCI cutoff in the negative or
declined performance level.
Table 33 presents the sensitivities for the individual measures at each test session
using Model 4, the RCI approach with correction for practice effects. Once again, it is
apparent that the individual variables, despite correction for practice effect, have
relatively low sensitivities. The practice correction does appear to have resulted in a
slight improvement in sensitivity for each re-test session, suggesting the detection of one
or two more significantly decreased scores for each. It is notable that both Hopkins
Delay and Grooved Pegboard nondominant hand performance have relatively larger
sensitivities when compared to the other variables at day 7, suggesting some lingering
short-term memory and motor speed decline in the concussed population. Tables 34
through 37 indicate the sensitivity and specificity of the battery of variables as a whole
for Model 4 at each test session.

82
Table 34 indicates a relatively high sensitivity/specificity combination was
achieved for the RCI model with correction for practice at the 24 Hour test session using
a criterion of 1 significant variable. Sixteen of the 21 concussed participants at this test
session fell below the cutoff for 1 or more of the 9 variables under examination while
only 1 control participants fell below the cutoff for 1 or more variables. By the Day 3
test session (Table 35), the RCI model with practice correction continued to have fairly
high classification success using a criterion of 1 significant variable, achieving a Se + Sp
value of 1.31.
Table 33. Sensitivity for Neuropsychological Measures for Model 4
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.43
0.11
0.11
0.11
Trails A
0.19
0.21
0.00
0.16
Trails B
0.19
0.00
0.06
0.05
Digit Span Forward
0.33
0.16
0.01
0.05
Digit Span Back
0.14
0.11
0.01
0.05
Grooved Pegboard D
0.24
0.11
0.00
0.11
Grooved Pegboard N
0.29
0.11
0.00
0.32
COWAT
0.10
0.11
0.11
0.00
Hopkins Delay
0.38
0.16
0.11
0.37
Table 34. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours with Model 4
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
16
1
0.76
0.92
(1.68)
2
11
0
0.52
1.00
(1.52)
3
7
0
0.33
1.00
(1.33)
4
5
0
0.24
1.00
(1.24)
5
4
0
0.19
1.00
(1.19)
6
2
0
0.10
1.00
(1.10)
7
2
0
0.10
1.00
(1.10)
8
1
0
0.05
1.00
(1.05)
9
0
0
0.00
1.00
(1.00)
5e=Sensitivity, Sp=Specificity

83
At the Day 5 test session (Table 36), the application of Model 4 resulted in an
accuracy of 33% when using a criterion of significance on one variable while producing 2
false positive identifications. Although slightly better than the corresponding accuracy
using Model 3, the ability to distinguish between the groups was relatively poor.
However, unlike the previous models, Model 4 achieved some success in distinguishing
between concussed and control participants by Day 7 as can be seen in Table 37. Using a
criterion of significance on one variable, Model 4 achieved a sensitivity level of 0.68
although it also produced a high number of false positive identifications.
Table 35. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 4
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.0
0.0
(1.00)
1
11
3
0.58
0.73
(1.31)
2
4
2
0.21
0.82
(1.03)
3
1
0
0.05
1.00
(1.05)
4
1
0
0.05
1.00
(1.05)
5
1
0
0.05
1.00
(1.05)
6
1
0
0.05
1.00
(1.05)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/?=Specificity
Table 36. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 4
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
18
10
1.0
0.0
(1.00)
1
6
2
0.33
0.80
(1.13)
2
2
1
0.11
0.90
(1.01)
3
1
1
0.05
0.90
(0.95)
4
0
1
0.00
0.90
(0.90)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)

84
Using a criterion of two significant variables, it achieved its highest sensitivity/specificity
combination, but saw a substantial decline in sensitivity. While cursory examination of
the sensitivity/specificity values suggested some clinical utility in identifying lingering
cognitive complaints at this most distal test session, further analysis of statistical
significance using ROC curves indicated otherwise.
Table 37. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 4
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.0
0.0
(1.00)
1
13
3
0.68
0.50
(1.18)
2
7
1
0.37
0.83
(1.20)
3
2
0
0.11
1.00
(1.11)
4
1
0
0.05
1.00
(1.05)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5/>=Specificity
Receiver operating characteristic curves for Model 4 at each testing session are presented
in Figures 26 through 29. As with the basic RCI model, the RCI with correction for
practice resulted in a model with a relatively good ability to distinguish between
concussed and control participants within 24 hours of concussion. The AUC for this
model was 0.863 (SE = 0.0636, 95% Confidence Interval = 0.738-0.987) and was
significantly better than chance with p<0.0001 (Fig. 26). As seen in Figure 27, Model 4
resulted in slightly better classification rate at the Day 3 test session, although the model
was again not statistically difference than chance level with an AUC of 0.634 (SE =
0.1082, 95% Confidence Interval = 0.422-0.846, p=0.1079). At Day 5, Model 4 was
clearly not better than chance at distinguishing between concussed and control

85
participants (Fig. 28). AUC was calculated as 0.556, p=0.3151 (SE = 0.1154, 95%
Confidence Interval = 0.329-0.782).
ROC Curve for Model 4 at 24 Hours
Figure 26. Receiver Operating Characteristic Curve for Model 4 at 24 Hour Test Session
ROC Curve for Model 4 at Day 3
Figure 27. Receiver Operating Characteristic Curve for Model 4 at Day 3 Test Session

86
ROC Curve for Model 4 at Day 5
Figure 28. Receiver Operating Characteristic Curve for Model 4 at Day 5 Test Session
ROC Curve for Model 4 at Day 7
Figure 29. Receiver Operating Characteristic Curve for Model 4 at Day 7 Test Session
As seen in Figure 29, the application of Model 4 at Day 7 appears to have been somewhat
more successful than the preceding models at distinguishing between concussed and
control participants. This was suggested by the sensitivity/specificity values in Table 37.

87
The AUC for this model at Day 7 was 0.636 (SE = 0.1245, 95% Confidence Interval =
0.392-0.880); however, this was again not statistically different than the reference
diagonal (p=0.1374).
A comparison between the basic RCI model and the RCI with correction for
practice applied to the data at the 24 Hour test session is shown in Figure 30. Although
the AUC for Model 4 is slightly larger than for Model 3 at the 24 Hour test session (0.863
vs. 0.861), the difference between the two was not statistically significant (p=l).
Comparison of RCI and RCI with Correction at 24 Hours
Figure 30. Comparison of ROC Curves for the RCI and RCI with Correction for
Practice
Model 5
The third variant of the RCI model (referred to as Model 5) approached the task
of determining whether there was a real difference between baseline and retest
performance by abandoning some assumptions used in the previous two RCI models.
Those assumptions included that 1) the control participants’ true scores for a particular

88
variable at test and retest sessions were equal, and 2) that the error variances for a
particular test variable at test and retest were also equal. Using the equation
(.K-x, )-(r-x)
1/s,2-S/-2S,S,r,>
where (y - X), or D, is the mean change score for a particular variable for the control
V2 2 • • •
Sx - Sy - 2 SxSyrxy , or SD ,is the standard deviations of the change
scores for the control population, the participants’ change scores from baseline to each
successive test session were examined to determine if they were large relative to the
distribution of change scores for the control group for each of those intervals.
Table 38 shows the average change scores for each variable at each testing
session with the standard deviation of the change scores for the control population. Z-
scores were calculated using the above equation. This process is illustrated in Table 39
for one concussed participant at the 24 Hour test session. As with the previous models, a
change score was considered significant if it exceeded 1.96 in the negative direction.
Therefore, as seen in Table 39, a z-score indicating a significant change relative to the
variance of the change scores for the control population for Hopkins Total score is
indicated by a z-score less than -1.96. By contrast, a greater-than-average change score
in the negative direction for Trails A (and all other variables where time is the measured
variable) is indicated by a z-score greater than +1.96.
Table 40 presents the sensitivities for the individual measures at each test session
using Model 5. As the data illustrates, the sensitivities for individual variables using
Model 5 at the 24 Hour test session are once again relatively higher than any of the other
days as a whole. Singly, the sensitivity values are not very high. As with most batteries

89
of neuropsychological test batteries, however, their combined use resulted in greater
sensitivity for this model over the repeated test sessions.
Table 38. Average Change Scores and Standard Deviations of Change Scores for the
Control Population
Measure
24 Hour
D SD
Test
Day 3
D
Session
Sn
Day 5
D
Day 7
D
Sn
Hopkins Total
0.38
2.53
-0.91
3.36
-0.40
3.95 -4.33
10.27
Trails A
-3.69
6.03
-5.03
3.73
-4.87
5.55 -8.48
4.19
Trails B
2.34
10.55
-1.01
26.02
-8.40
13.96 -11.03
10.44
Digits Forward
1.38
1.12
1.64
1.63
2.00
1.94 1.50
1.87
Digits Backward
0.08
2.22
1.73
2.00
2.50
3.14 0.67
2.07
Pegs Dominant
-2.97
6.08
-4.05
5.80
-4.20
13.25 -8.07
7.77
Pegs Nondominant
-0.61
8.27
-3.35
10.65
0.46
12.61 -6.55
5.99
COWAT
3.08
8.92
5.45
8.36
5.80
7.93 1.67
11.45
Hopkins Delay
0.00
1.41
-1.27
2.53
0.60
1.84 -2.33
3.56
D =average change, SD =standard deviation of change score
Table 39. Example of Z-score Derivation using Model 5 for one Concussed Participant at
24 Hour Test Session
Measure
Baseline
24 hour
Difference
D
z-score
Hopkins Total
20
12
-8
0.38
2.53
-3.31
Trails A
33.0
84.0
51
-3.69
6.03
9.06
D =average change, SD =
=standard deviation of change score
Table 40. Sensitivity for Neuropsychological Measures for Model 5
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.43
0.05
0.00
0.00
Trails A
0.29
0.16
0.11
0.32
Trails B
0.19
0.00
0.06
0.05
Digit Span Forward
0.33
0.16
0.22
0.05
Digit Span Back
0.14
0.11
0.06
0.16
Grooved Pegboard D
0.24
0.11
0.00
0.11
Grooved Pegboard N
0.29
0.11
0.00
0.32
COWAT
0.05
0.11
0.00
0.00
Hopkins Delay
0.38
0.00
0.11
0.00

90
As can be seen, the sensitivity values at day 3 and day 5 are consistently poor.
Interestingly, however, the sensitivities for the individual variables at day 7 are somewhat
higher using Model 5 than the previous 4, suggesting that this may be a more appropriate
model to use at more distal testing sessions. Tables 41 through 44 indicated the
sensitivity and specificity of the battery of variables as a whole for Model 5 at each test
session.
Table 41. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours with Model 5
No. of Sig.Measures Concussed Control Se Sp (Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
17
1
0.81
0.92
(1.73)
2
12
0
0.57
1.00
(1.57)
3
7
0
0.33
1.00
(1.33)
4
5
0
0.24
1.00
(1.24)
5
4
0
0.19
1.00
(1.19)
6
2
0
0.10
1.00
(1.10)
7
1
0
0.05
1.00
(1.05)
8
1
0
0.05
1.00
(1.05)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, S/?=Specificity
Table 41 indicates that Model 5 was very successful at distinguishing between concussed
and control participants at the 24 Hour test session. Using a criterion of one variable
falling below cutoff, the combined sensitivity/specificity was 1.73, with both good
sensitivity (0.81) and specificity (0.92).
Day 3 sensitivity/specificity data using Model 5 (Table 42) again indicated a
pattern of significantly decreased sensitivity and a less substantial drop in specificity with
the one variable criterion. Using two variables as a criterion results in an unacceptable
drop in sensitivity. A continued reduction in sensitivity at day 5, as was seen with all
previous models, was again evident for Model 5 (Table 43). However, at the Day 7 test

91
session, Model 5 resulted in fair sensitivity and good specificity using the one-variable
criterion (Table 44).
Table 42. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 5
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.0
0.0
(1.00)
1
10
2
0.53
0.82
(1.35)
2
2
0
0.11
1.00
(1.11)
3
1
0
0.05
1.00
(1.05)
4
1
0
0.05
1.00
(1.05)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, 5p=Specificity
Table 43. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 5
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
18
10
1.0
0.0
(1.00)
1
7
2
0.39
0.80
(1.19)
2
1
0
0.06
1.00
(1.06)
3
0
0
0.00
1.00
(1.00)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, Sp=Specificity
The statistical significance of Model 5 at each testing session was again determined
through ROC curve analysis using non-parametric measurement of area under the curve.
Results of this analysis are depicted in Figures 31 through 34.

92
Table 44. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 5
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.0
0.0
(1.00)
1
12
0
0.63
1.00
(1.63)
2
6
0
0.32
1.00
(1.32)
3
1
0
0.05
1.00
(1.05)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, iS/?=Specifícity
ROC Curve for Model 5 at 24 Hours
Figure 31. Receiver Operating Characteristic Curve for Model 5 at 24 Hour Test Session
Figure 31 depicts the ROC curve for Model 5 at 24 hours. The AUC of 0.888 (SE =
0.0577, 95% Confidence Interval = 0.775-1.00) was larger than any of the preceding
models and was highly significant (pO.0001), indicating that this model was good at
distinguishing between the change scores of the concussed and control participants.
Further, Figure 32 indicates that Model 5 was also able to distinguish between the

93
concussed and control populations at the Day 3 test session at a statistically significant
level, an achievement which no previous model was able to attain. The AUC for the
model at this test session was 0.682 (SE=0.0985, 95% Confidence Interval = 0.489-
0.875), which was statistically different than chance level (p=0.033).
ROC Curve for Model 5 at Day 3
Figure 32. Receiver Operating Characteristic Curve for Model 5 at Day 3 Test Session
By day 5, however, Model 5 was not statistically different than chance at
distinguishing between the two groups (Fig. 33). The AUC was calculated to be 0.597
(SE = 0.1107, 95% Confidence Interval = 0.380-0.814), with a p-value of 0.19.
Interestingly, as was suggested by the sensitivity/specificity values for Model 5 at day 7
in Table 44, Figure 34 indicates that Model 5 was indeed successful at distinguishing
between concussed and control participants at the day 7 test session. The ROC plot
produced an AUC of 0.816 (SE = 0.0837, 95% Confidence Interval = 0.652-0.980),
which was statistically different than chance level (pO.0001).

94
ROC Curve for Model 5 at Day 5
Figure 33. Receiver Operating Characteristic Curve for Model 5 at Day 5 Test Session
ROC Curve for Model 5 at Day 7
No discrimination
Model 5
Figure 34. Receiver Operating Characteristic Curve for Model 5 at Day 7 Test Session

95
Regression Models
Model 6
Two models based on regression analysis were used to examine their ability to
detect significant change from baseline performance over subsequent retest sessions. The
first was a simple linear regression approach using baseline performance to predict
performance for each successive test session (referred to as Model 6). For this model,
regression equations were calculated for each of the 9 variables for each test session for
the control population in order to derive the necessary data for entry into linear regression
equations in the form:
Y = a + bX
where Y is the control group’s performance on a variable at a retest session, a is a
calculated constant value, and b is the Beta weight for X, the controls’ performance on
that variable at baseline.
Regression equations were calculated using SPSS software for each variable. The
variables derived from those regression analyses are presented in table 45 for each test
session along with the R2 value associated with the model for the control population.
Table 45. Regression Equation Data for Each Variable and Test Session
Measure constant (a) beta (b) R2
24 Hour
Hopkins Total
-1.86
1.09
0.71
Trails A
15.98
0.12
0.04
Trails B
18.14
0.67
0.39
Digits Forward
-0.38
1.23
0.89
Digits Backward
3.66
0.46
0.30
Pegs Dominant
15.69
0.71
0.52
Pegs Nondominant
22.13
0.68
0.54
COWAT
28.00
0.31
0.08
Hopkins Delay
2.82
0.68
0.47

96
Table 45. Continued
Measure
constant (a)
beta (b)
R2
Day 3
Hopkins Total
2.12
0.88
0.46
Trails A
8.26
0.43
0.58
Trails B
25.33
0.46
0.05
Digits Forward
3.91
0.70
0.39
Digits Backward
4.79
0.53
0.43
Pegs Dominant
26.31
0.54
0.50
Pegs Nondominant
39.47
0.40
0.38
COWAT
24.11
0.48
0.18
Hopkins delay
1.18
1.18
0.73
Day 5
Hopkins Total
2.95
0.87
0.40
Trails A
10.70
0.35
0.15
Trails B
13.47
0.56
0.18
Digits Forward
7.77
0.31
0.18
Digits Backward
7.75
0.25
0.05
Pegs Dominant
27.64
0.52
0.09
Pegs Nondominant
0.45
1.00
0.54
COWAT
14.55
0.76
0.37
Hopkins Delay
5.59
0.46
0.15
Day 7
Hopkins Total
-17.76
1.51
0.24
Trails A
12.48
0.17
0.09
Trails B
25.30
0.24
0.13
Digits Forward
4.65
0.65
0.36
Digits Backward
5.49
0.41
0.25
Pegs Dominant
9.70
0.73
0.45
Pegs Nondominant
8.78
0.80
0.78
COWAT
6.83
9.17
0.01
Hopkins Delay
-19.54
2.88
0.78
Using these parameters, predicted re-test performances for each variable were calculated
for both the concussed and control participants at each testing session. The predicted
score, y, for each variable was compared to the corresponding observed score, y, to
determine if the difference was large relative to the distribution of residuals for the

97
control participants, SDr, based on the regression equation, which was calculated with the
equation:
A z-score was derived for each variable to determine if the observed score was
significantly different than the predicted score which was based on the performance of
the control population. The equation for deriving these z-score was:
y-y
where y was the observed score for an individual variable and y was the predicted score.
As in the previous models, a z-score exceeding the cutoff point of 1.96 was taken to
indicate a change from baseline to retest session which was significantly different than
expectation, and since only deviation in the negative direction was taken as significant, a
positive finding would indicate that the retest performance was indicative of declined
performance relative to those of the non-concussed population at retest session. The
sensitivities for the individual variables using Model 6 are presented in Table 46.
As Table 46 indicates, the sensitivities for the individual values using Model 6
were generally comparable to those with Model 5. There was a slight increase in
sensitivities using Model 6 at Day 3 relative to the same test session using Model 5.
However, at the Day 5 and Day 7 test sessions, the sensitivities using Model 6 are very
low, which is likely reflective of the poor predictive power of baseline performance for
performance at Day 5 and Day 7 on individual variables. This is probably a function of

98
the low number of listwise control participants at the later testing sessions on whose data
the regression equations were based.
Table 46. Sensitivity for Neuropsychological Measures for Model 6
Measure
24 Hours
Day 3
Day 5
Day 7
Hopkins Total
0.43
0.05
0.00
0.00
Trails A
0.38
0.26
0.00
0.16
Trails B
0.19
0.00
0.00
0.05
Digit Span Forward
0.38
0.21
0.17
0.11
Digit Span Back
0.10
0.05
0.00
0.00
Grooved Pegboard D
0.24
0.16
0.00
0.05
Grooved Pegboard N
0.33
0.21
0.00
0.21
COWAT
0.05
0.00
0.06
0.00
Hopkins Delay
0.33
0.00
0.17
0.05
Tables 47 through 50 indicated the sensitivity and specificity of the battery of
variables as a whole for Model 6 at each test session. As with the previous models, the
highest degree of classification accuracy occurred at the 24 Hour test session (Table 47).
Sensitivity was obviously highest again when using significance on one variable as the
criterion for classifying concussed patients (0.86). This criterion also resulted in a fairly
good specificity rate (0.85). A similar pattern was observed using Model 6 at the day 3
test session (Table 48). Like the previous models, at the Day 5 test session, the model
was less successful at distinguishing between the groups, due in large part to a substantial
drop in sensitivity (Table 49).
As was the case with Model 5, the use of regression predictions in attempt to
distinguish between the groups resulted in a relatively large increase in specificity
relative to the other models at the Day 7 test session (Table 50). Although sensitivity was
relatively low, even using the least restrictive criterion of one variable exceeding
significance level, the model was relatively good at distinguishing between the two
groups.

99
Table 47. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at 24 Hours with Model 6
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
21
13
1.0
0.0
(1.00)
1
18
2
0.86
0.85
(1.71)
2
11
1
0.52
0.92
(1.44)
3
8
0
0.38
1.00
(1.38)
4
4
0
0.19
1.00
(1.19)
5
4
0
0.19
1.00
(1.19)
6
2
0
0.10
1.00
(1.10)
7
1
0
0.05
1.00
(1.05)
8
1
0
0.05
1.00
(1.05)
9
1
0
0.05
1.00
(1.05)
Se=Sensitivity, 5p=Specificity
Table 48. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 3 with Model 6
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
11
1.0
0.0
(1.00)
1
11
2
0.58
0.82
(1.40)
2
4
1
0.21
0.91
(1.12)
3
2
0
0.11
1.00
(1.11)
4
1
0
0.05
1.00
(1.05)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
Se=Sensitivity, Sp=Specificity
Table 49. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 5 with Model 6
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
18
10
1.0
0.0
(1.00)
1
6
1
0.33
0.90
(1.23)
2
1
1
0.06
0.90
(0.96)
3
0
0
0.00
1.00
(1.00)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)

100
Table 50. Number of Concussed and Control Participants Falling Below Cut-Point for
Multiple Variables at Day 7 with Model 6
No. of Sig.Measures
Concussed
Control
Se
Sp
(Se + Sp)
0
19
6
1.0
0.0
(1.00)
1
7
0
0.37
1.00
(1.37)
2
3
0
0.16
1.00
(1.16)
3
2
0
0.11
1.00
(1.11)
4
0
0
0.00
1.00
(1.00)
5
0
0
0.00
1.00
(1.00)
6
0
0
0.00
1.00
(1.00)
7
0
0
0.00
1.00
(1.00)
8
0
0
0.00
1.00
(1.00)
9
0
0
0.00
1.00
(1.00)
iSe=Sensitivity, ^/^Specificity
Figures 35 through 38 present the ROC curves for Model 6 at each test session.
Results suggested that this model, like the previously examined models, provided good
distinction between concussed and non-concussed participants at the 24 Hour test session
(Figure 35). AUC was 0.874 (SE = 0.0627, 95% Confidence Interval = 0.751-0.996),
which was statistically better than chance (p<0.0001). Model 6 was also significantly
better than chance at discriminating between the groups at the Day 3 test session (Figure
36) with an AUC of 0.696 (SE = 0.0992, 95% Confidence Interval = 0.502-0.891) with
p=0.024. Although somewhat better than Models 1-4 at discriminating between groups
at Day 5, Model 6 was not statistically different than chance at this test session (Figure
37). AUC was 0.603 (SE = 0.1120, 95% Confidence Interval = 0.383-0.822) with a p-
value of 0.18.

101
ROC Curve for Model 6 at 24 Hours
No discrimination
—o— Model 6
Figure 35. Receiver Operating Characteristic Curve for Model 6 at 24 Hour Test Session
ROC Curve for Model 6 at Day 3
Figure 36. Receiver Operating Characteristic Curve for Model 6 at Day 3 Test Session

102
While somewhat less successful than Model 5, Model 6 was relatively successful at
distinguishing between the concussed and control participants at Day 7 as well. AUC
was 0.684 (SE = 0.469-0.899) with a p-value of 0.05 (Fig. 38).
ROC Curve for Model 6 at Day 5
No discrimination
Model 6
Figure 37. Receiver Operating Characteristic Curve for Model 6 at Day 5 Test Session
ROC Curve for Model 6 at Day 7
No discrimination
—0— Model 6
Figure 38. Receiver Operating Characteristic Curve for Model 6 at Day 7 Test Session

103
Model 7
The final model (referred to as Model 7) examined as part of the current study
involved the application of multiple regression models to the data in order to determine
the effectiveness of two demographic variables at more accurately providing predicted
retest scores at each of the testing sessions. It was hypothesized that history of diagnosed
learning disability and history of previous concussion would contribute to the accuracy of
the prediction equations, thus lowering the standard deviation of the average residuals for
the control group and providing narrower confidence intervals outside of which a
concussed participant’s observed score would be considered significantly different than a
predicted score relative to baseline performance, suggesting continuing effects of
concussion.
In order to examine these variables’ contribution to prediction scores, they were
entered (in addition to the baseline score on the individual measure) into a multiple
regression analysis using SPSS statistical software for each variable at each testing
session for the control population. Variables were entered in a stepwise fashion, and
were included into the equation if their F probability was less than 0.05, but were
removed if the F probability exceeded 0.10. Tables 51 through 54 indicate the resulting
models for each variable for the 4 test sessions.
As Table 51 indicates, neither history of learning disability nor history of previous
concussion entered significantly into any of the prediction equations at the 24 Hour test
session. Ultimately, this resulted for most measures in a simple regression prediction
equation using baseline score as the sole predictor for that particular variable, which was
equivalent to Model 6.

104
Table 51. Equations Resulting from Multiple Regression
Analysis at 24 Hours
Measure
Model
R2
F
sig.
SE
Hopkins Total
y =-1.861 + (1.09)(baseline)
0.71
26.65
<0.001
2.63
Trails A
no significant predictors
Trails B
y =18.14 + (0.67)(baseline)
0.39
6.98
0.023
10.27
Digits Forward
y =-3.8 + (1.227)(baseline)
0.89
87.01
<0.001
1.04
Digits Backward
no significant predictors
Pegs Dominant
y =15.69 + (0.71)(baseline)
0.52
11.86
0.005
5.86
Pegs Nondominant
y =22.13 + (0.68)(baseline)
0.54
13.04
0.004
7.69
COWAT
no significant predictors
Hopkins Delay
y =2.81 + (0.68)(baseline)
0.47
9.55
0.010
1.35
SE=standard error
Table 51 also demonstrates that none of the predictor variables entered significantly into
the regressions for three of the nine variables under investigation (Trails A, Digits
Backward, and COWAT total score).
At day 3 (Table 52), there was also a trend for the baseline scores on individual
measures to be the only significant predictor variable entered into the regression
equation. However, notable exceptions included Day 3 Trails B score, for which history
of learning disability was the only significant predictor, entering the equation with an R2
value of 0.53; COWAT total score at Day 3, for which history of concussion was the sole
significant predictor with an R2 value of 0.45; and Grooved Pegboard Nondominant Hand
score, for which both baseline performance and history of learning disability entered
significantly into the model, resulting in an overall R2 value of 0.66. No predictor values
were significant for the Day 3 Hopkins Delay score.
At the Day 5 test session (Table 53), history of learning disability and baseline
performance both entered significantly into the regression equations for Hopkins Total
score (R2=0.70) and Trails B (R2=0.84). History of learning disability was the only
significant predictor for Grooved Pegboard Dominant Hand (R2=0.69) and Grooved

105
Pegboard Nondominant Hand (R2=0.73). No statistically significant predictors were
found for the remaining variables at that test session.
Table 52. Equations Resulting from Multiple Regression Analysis at Day 3
Measure
Model
R2
F
sig.
SE
Hopkins Total
y =2.12 + (0.88)(baseline)
0.46
7.72
0.021
3.51
Trails A
y =8.26 + (0.43)(baseline)
0.58
12.31
0.007
2.15
Trails B
y =132.64 + (-46.64)(hx of LD)
0.53
10.33
0.011
18.56
Digits Forward
y =3.91 + (0.70)(baseline)
0.45
7.41
0.024
1.60
Digits Backward
y =4.79 + (0.53)(baseline)
0.43
6.91
0.027
1.66
Pegs Dominant
y =26.31 + (0.54)(baseline)
0.50
8.85
0.016
4.64
Pegs Nondom.
y=66.31 + (-11.64)(hx LD) + (0.32)(baseline)
0.66
7.75
0.013
5.66
COWAT
y =61.46 + (-11.79)(hx of cone)
0.45
7.39
0.024
6.41
Hopkins Delay
no significant predictors
SE=standard error
Table 53. Equations Resulting from Multiple Regression
Analysis Day 5
Measure
Model
R2
F
sig.
SE
Hopkins Total
y =-10.88 + (8.84)(hx LD) + (0.76)(baseline)
0.70
8.01
0.016
3.15
Trails A
no significant predictors
Trails B
y =84.58 + (-37.01)(hx LD) + (0.55)(baseline)
0.84
17.69
0.002
6.67
Digits Forward
no significant predcitors
Digits Backward
no significant predictors
Pegs Dominant
y =129.08 + (-35.08)(hx LD)
0.69
17.90
0.003
7.86
Pegs Nondom.
y =168.12 +(-50.12)(hx LD)
0.73
21.27
0.002
10.31
COWAT
no significant predictors
Hopkins Delay
no significant predictors
SE=standard error
Table 54 shows the results of the multiple regression analysis for each variable at the Day
7 test session. At that day, no significant predictors were found for six of the nine
variables. History of concussion was the only significant predictor for COWA total score
2
(R =0.79) and only baseline performance for Grooved Pegboard Nondominant hand and
Hopkins Delay score entered significantly into those regression equations.

106
Given the relatively low number of listwise control participants at each test
interval, it is not surprising that the two additional demographic variables did not
contribute more significantly to the regression equations. However, it is notable that both
history of learning disability and history of previous concussion contributed significantly
to some variables at the Day 3, Day 5, and Day 7 test sessions despite the low listwise n.
It is even more notable that both were better predictors in some variables than was
baseline performance.
Table 54. Equations Resulting from Multiple Regression
Analysis at Day 7
Measure
Model
R2
F
sig.
SE
Hopkins Total
no significant predictors
Trails A
no significant predictors
Trails B
no significant predictors
Digits Forward
no significant predictors
Digits Backward
no significant predictors
Pegs Dominant
no significant predictors
Pegs Nondominant
y =8.78 + (0.80)(baseline)
0.78
14.18
0.020
6.03
COWAT
y =63.33 + (-15.0)(hx concussion)
0.79
15.11
0.018
4.73
Hopkins Delay
y =-19.54 + (2.88)(baseline)
0.78
14.25
0.020
2.51
SE=standard error
As in the analysis of the basic regression model (Model 6), the above equations
were applied to the concussed and control participants’ score for each measure over the
interval from baseline to each testing session. Predicted scores were subtracted from
actual observed scores and divided by the standard error of the residuals for the control
population to obtain a z-score. A z-score exceeding 1.96 in the negative direction was
considered a significant finding as before.
Tables 55 through 57 provide a comparison of individual measure sensitivities for
those models in which history of learning disability and history of concussion were
significant predictors to the sensitivities of the other models at the corresponding test
sessions.

107
Table 55. Comparison to Previous Models of Sensitivities for Regression Equations in
which History of Learning Disability (LD) was the Sole Significant Predictor
LD
1
2
Model
3 4
5
6
Day 3 Trails B
0.05
0.11
0.05
0.00
0.00
0.00
0.00
Day 5 Pegs Dom
0.11
0.06
0.00
0.00
0.00
0.00
0.00
Day 5 Pegs Ndom
0.11
0.11
0.06
0.00
0.00
0.00
0.00
Table 56. Comparison to Previous Models of Sensitivities for Regression Equations in
which History of Previous Concussion (Cone) was the Sole Significant Predictor
Model
Cone
1
2 3
4
5
6
Day 3 COWAT 0.26
0.05
0.11 0.00
0.11
0.11
0.00
Day 7 COWAT 0.06
0.00
0.00 0.00
0.00
0.00
0.00
Table 57. Comparison to Previous Models of Sensitivities for Regression Equations in
which History of LD and Baseline Performance were Significant Predictors
LD+Baseline
Model
1 2
3
4
5
6
Day 5 Hopkins T
0.00
0.17
0.17
0.11
0.11
0.00
0.00
Day 5 Trails B
0.06
0.06
0.06
0.06
0.06
0.06
0.00
As can be seen in table 55, when history of diagnosed learning disability was the
only significant predictor, it did not provide a substantial increase in sensitivity relative to
the previously described models. Nor did history of learning disability provide an
increase in sensitivity relative to the other models when it was entered into regression
equations with baseline performance (Table 57). Table 56 suggests that history of
previous concussion was relatively successful at increasing the sensitivity of COWA total
score at the Day 3 test session, and did provide a very slight increase in sensitivity of
COWA total score at Day 7. An overall comparison of the sensitivities found by the

108
application of all models for each variable over the 4 test sessions is presented in Table
58.
Table 58. Overall Comparison of Sensitivities
Test Session
Model
Hop Tot. Tr. A
Tr. B
Dig F
Dig B
Peg D
Peg ND
COWA
Hop D
Base-24 Hour
1
0.29
0.29
0.19
0.19
0.14
0.14
0.38
0.19
0.43
2
0.43
0.29
0.29
0.14
0.19
0.19
0.43
0.14
0.33
3
0.43
0.14
0.19
0.29
0.14
0.19
0.29
0.05
0.38
4
0.43
0.19
0.19
0.33
0.14
0.24
0.29
0.10
0.38
5
0.43
0.29
0.19
0.33
0.14
0.24
0.29
0.05
0.38
6
0.43
0.38
0.19
0.38
0.10
0.24
0.33
0.05
0.33
Base-Day 3
1
0.11
0.05
0.11
0.16
0.05
0.16
0.16
0.05
0.42
2
0.11
0.16
0.05
0.11
0.11
0.05
0.16
0.11
0.26
3
0.11
0.11
0.00
0.05
0.00
0.11
0.05
0.00
0.16
4
0.11
0.21
0.00
0.16
0.11
0.11
0.11
0.11
0.16
5
0.05
0.16
0.00
0.16
0.11
0.11
0.11
0.11
0.00
6
0.05
0.26
0.00
0.21
0.05
0.16
0.21
0.00
0.00
7
0.05
0.26
Base-Day 5
1
0.17
0.00
0.06
0.00
0.05
0.06
0.11
0.11
0.11
2
0.17
0.11
0.06
0.06
0.06
0.00
0.06
0.00
0.17
3
0.11
0.00
0.06
0.00
0.06
0.00
0.00
0.00
0.11
4
0.11
0.00
0.06
0.01
0.01
0.00
0.00
0.11
0.11
5
0.00
0.11
0.06
0.22
0.06
0.00
0.00
0.00
0.11
6
0.00
0.00
0.00
0.17
0.00
0.00
0.00
0.06
0.17
7
0.00
0.06
0.11
0.11
Base-Day 7
1
0.05
0.11
0.00
0.05
0.00
0.05
0.21
0.00
0.16
2
0.11
0.11
0.05
0.05
0.05
0.05
0.16
0.00
0.26
3
0.11
0.00
0.05
0.00
0.05
0.00
0.16
0.00
0.37
4
0.11
0.16
0.05
0.05
0.05
0.11
0.32
0.00
0.37
5
0.00
0.32
0.05
0.05
0.16
0.11
0.32
0.00
0.00
6
0.00
0.16
0.05
0.11
0.00
0.05
0.21
0.00
0.05
7
0.06
As was noted above for each model, sensitivities were generally highest at the 24 Hour
test session across variables. Examination of Table 58 suggests that the later models,
including RCI with correction for practice (Model 4), RCI examination of difference
scores (Model 5), and the simple regression model (Model 6) tended to be slightly more
sensitive at 24 Hours across variables. At Day 3, Day 5, and Day 7 testing sessions, a
difference in sensitivities between models was more subtle, with a tendency toward later

109
models being slightly more sensitive than the earlier, less complex models. This is more
easily observed in a comparison between AUC measures. This is presented in Table 59.
Table 59. Comparison of AUC Measures for All Models Across Testing Sessions
Model Base-24 Hours
Base-Day 3
Base-Day 5
Base-Day 7
1
0.773*
0.560
0.517
0.650
2
0.788*
0.566
0.522
0.518
3
0.861*
0.514
0.556
0.553
4
0.863*
0.634
0.556
0.636
5
0.888*
0.682*
0.597
0.816*
6
0.874*
0.696*
0.603
0.684*
* denotes statistically different than chance, see above for individual p-values
As Table 59 suggests, all models were statistically better than chance at
distinguishing between the concussed and control participants at the 24 hour test session.
It was noted above that Model 1 and Model 2 were not different than each other in this
regard. Likewise, Model 3 and Model 4 were not statistically different in distinguishing
between the two groups. Model 5 (RCI difference scores) had the highest AUC value at
the 24 Hour test session and appeared to be the most sensitive overall. Statistical
comparison of the AUC measures for Models 4, 5, and 6 indicated that they were not
statistically significant (p>0.50), which is seen graphically in Figure 39. At the Day 3
test session, only Model 5 and Model 6 were statistically better than chance at
distinguishing between the concussed and control participants. With an AUC value of
0.696, Model 6 (simple linear regression) appeared more accurate in distinguishing
between the two groups than Model 5 (RCI for difference scores), which had an AUC
value of 0.682. Comparison of the two models, however, indicated that they were not
statistically different (p>0.50), which is exhibited in Figure 40.

110
Comparison of Significant Detection Models at 24 Hours
Figure 39. Comparison of Models 3, 4, and 5 at the 24 Hour Test Session
Comparison of Models 5 and 6 at Day 3
Figure 40. Comparison of ROC Curves for Model 5 and Model 6 at Day 3 Test Session

Ill
As Table 59 suggests, none of the models was statistically better than chance at
distinguishing between the two groups at Day 5, although Model 6 had the highest AUC
value (0.603). At the Day 7 test session, Models 5 and 6 were again the only statistically
accurate methods for distinguishing between the groups. Model 5, with an AUC value of
0.816 was the most accurate, while Model 6 was significant with an AUC value of 0.684.
A statistical comparison of the models indicated, however, that the two were not
significantly different (p>.05), as shown in Figure 41.
Comparison of Model 5 and 6 at Day 7
No discrimination
—o—Mod el 5
Model 6
Figure 41. Comparison of ROC Curves for Model 5 and Model 6 at Day 7 Test Session

CHAPTER FOUR
DISCUSSION
The detection of lingering cognitive decline in those having suffered mild head
injury has a number of implications. Although it has been shown in numerous studies
and through observational data that the cognitive effects of concussion tend to dissipate
relatively quickly, and usually resolve within 5 to 7 days after the incident, the
manifestation of continuing cognitive sequelae may be an indicator of a more serious
underlying pathological condition. The importance of detecting lingering cognitive
effects is magnified when one considers that those having suffered from mild head injury
are more at risk for the future occurrence of head injury (Roberts et al., 1994), especially
within the time frame in which they are recovering from the initial incident. The
biological underpinnings of this are somewhat unclear, though may be related to changes
in the biochemical environment induced by the injury itself. The experience of
concussion can lead to longer-term effects (Barth et ah, 1983; Rimel et ah, 1981; Roberts
et ah, 1994), and the experience of a multiple concussions within a short time period has
led to death in some circumstances.
The above is especially important when considering the effect of concussion in
sports. By nature, football provides the environment for a relatively high incidence of
concussion among its participants. A decision regarding fitness for a player to return to
action following concussion is often made by team trainers or physicians who are
afforded little objective information upon which to make this decision other than gross
112

113
markers of orientation and a player’s assurance that the cobwebs have cleared, so to
speak. The application of neuropsychological tests to a retum-to-play decision provides
far mar objective criteria and far better sensitive measure of cognitive functioning.
Until recently, few researchers had implemented controlled studies of the
usefulness of neuropsychological measures in detecting cognitive sequelae of concussion
among athletes in the acute stages. In a study published in 2001 by William Barr and
Michael McCrea, the researchers assessed the utility of the Sideline Assessment of
Concussion (SAC) in detecting cognitive decline in the immediate stages following
concussion, and found this to be a useful tool in quantifying detriment in performance.
However, few studies to date have used a standardized battery of neuropsychological
measures to assess the pattern of cognitive decline and return to baseline functioning over
a more prolonged period (Barth et al., 1983).
The current study utilized data collected from 443 college football players at four
major Division I universities, resulting in a representative sample of college football
players at this level. Baseline performance on a brief neuropsychological test battery,
which included the Hopkins Verbal Learning Test, Trail Making Test A and B, Digit
Span test, Grooved Pegboard, and Controlled Oral Word Association were recorded.
Subsequently, 26 players who suffered concussion either in practice or in a game
situation were identified by team trainers and re-administered the neuropsychological
battery at each of 4 testing sessions: within 24 hours of concussion, 3 days, 5 days, and 7
days post-concussion. These players were assigned control participants who were
matched as closely as possible with regard to position, history of learning disability, and
history of previous concussion.

114
Seven statistical models were examined. These were conceptually divided into
three groups consisting of basic clinical comparisons, reliable change index models, and
regression analysis models. These groups included four models in particular which have
been examined by researchers in the detection of “true” change in neuropsychological
performance over repeated testing intervals (Barr and McCrea, 2001; Temkin et al.,
1999). These were: the Reliable Change Index (RCI), a variant of the RCI with
correction for practice, a linear regression/predicted score model, and a multiple
regression/predicted score model utilizing demographic variables to enhance prediction
power. The current study attempted to provide further evidence that these statistical
models can be used in the analysis of neuropsychological test performance to detect
lingering cognitive sequelae of concussion through serial testing, and may also be
valuable in helping to determine an athlete’s fitness for return to play.
An overall analysis of the test-retest performance of concussed and control
participants indicated that those having suffered concussion exhibited a pattern of
performance over time that was similar to that observed by others (Barth et al., 1989;
Hinton-Bayre et al., 1997; Maddocks and Saling, 1996;). That is, performance within the
very early stages after concussion (within 24 hours) generally saw a decline in
performance level or failed to exhibit expected practice effects. By contrast, it was fairly
obvious, even without the application of statistical methods, that the matched controls, as
a whole, tended to exhibit increased performance over the nine variables examined as
part of the neuropsychological battery utilized. Examination of graphical representation
of recovery trends over the remaining three retest sessions indicated, as the literature
suggests, gradual improvement over the week following concussion.

115
In examining the criteria used to classify players as continuing to exhibit
cognitive effects of concussion, researchers have cautioned that erring on the side of
caution is the best course of action because of the important implications that failing to
detect lingering effects may have. Therefore, some have elected to favor the use of rather
liberal criteria in classification (Barr and McCrea, 2001). The ramification of this, of
course, is the acceptance of lower specificity rates. Optimally, a useful statistical model
would seek to maximize both sensitivity and specificity. Optimal cut-points for these
models can be indicated by comparison of the sum of sensitivity and specificity at each
cut point. For the current study, this maximized sensitivity/specificity combination was
generally achieved using a cut-point of one or more variables falling significantly below
expectation based on the particular model examined. In some instances, using a criterion
of two or more variables falling below expectation achieved the maximum
sensitivity/specificity combination, but this came at the greatest expense for sensitivity.
Erring on the side of caution, therefore, the cut-points with the maximum sensitivity
appear to be the most salient; in this case, one or more variables falling below expectation
based on the particular statistical model. Another potential approach may be to examine
the positive and negative predictive powers for the individual measures. However, since
the aim of the tests is to detect rather than predict decrement in performance, sensitivity
and specificity values are more appropriate in this instance.
This study utilized a comparison of models using Receiver Operating
Characteristic curves, a method which has been used often in the medical literature to
determine the usefulness of particular biological markers or tests to accurately classify
the presence of pathology (Cha et al., 2004; DeYebenes et al., 2003; Kraemer et al.,

116
2003). This method has also gained an increasing use across varied specialties in the
psychological literature (Barr and McCrea, 2001; Glassmire et al., 2003; Heathcote,
2003; Ross et al., 2004). A similar procedure was used by Barr, et al. to examine the
classification accuracy of an RCI and a regression model in early-stage (within 24 hours)
concussion testing with the SAC. An ROC analysis involves the nonparametric measure
of the area under the curve (sensitivity plotted against 1-specificty), where an area of 0.5
denotes chance level. Therefore, a calculated area can be compared to chance level to
determine the classification accuracy’s statistical significance. Likewise, the AUC of two
or more curves can be compared to determine if one model is more accurate than another.
Basic Models
In the most basic models (Model 1 and Model 2) examined as part of this study,
concussed and control participants’ scores at each testing session were compared to the
normative population mean score of the 409 (443 less 21 concussed players and 13
control) individuals from the four schools and to their own baseline performance
respectively in order to detect large differences relative to the standard deviation of the
baseline population for each measure. Such comparisons result in standard z-score and
are not unlike the analysis of neuropsychological test data used in the clinical setting.
Significance levels were determined for the individual variables based on a liberal criteria
of 1 SD (based on the normative sample) below the normative mean or below their
baseline performance, which is often used in common neuropsychological tests to denote
mild impairment (Reitan and Wolfson, 1985). The application of this method resulted in
comparably high degrees of sensitivity at the 24-hour test session for both models (0.91
and 0.81 respectively). The use of even one variable falling below the designated

117
classification criteria in Model 1, however, resulted in a very low specificity rate at that
test session (0.38). More than half of the control participants were falsely classified as
concussed. Using the same criteria (one variable falling below expectation) for Model 2,
on the other hand, resulted in a relatively lower false-positive rate (specificity = 0.62),
suggesting slightly better classification accuracy within 24-hours. This difference was
not statistically significant, but the increase in specificity points to the statistical benefit
of having a baseline comparator.
Examination of AUC data indicated that neither Model 1 nor Model 2 were
statistically better than chance at distinguishing between concussed and control
participants at the Day 3, Day 5, or Day 7 test sessions. While they did continue to have
some sensitivity when using the one-variable classification criterion, the combined
sensitivity/specificity rates were such that neither model could be characterized as
successful at discriminating between concussed and non-concussed participants at any of
these days. Neither, therefore, appears to have any clinical value in the determination of
retum-to-play status after the 24-hour period.
RCI Models
For the remainder of the models examined, significance for individual variables
was determined by data gathered from the control participants. For the basic RCI model
and the RCI model with practice for correction, the statistical method essentially asked
the question: was the individual’s score at a retest session outside the normal range of
change scores based on the measure’s reliability? That standard error, in turn, was
derived from the correlation of the control participants’ scores on the individual variable
at each test-retest interval. The standard deviation, on the other hand, was obtained only

118
from the baseline test session. A 95% confidence interval was used in each case with a z-
value of 1.96. Although seen as somewhat too conservative by some researchers (Barr
and McCrea, 2001), this level of significance was chosen in order to provide greater
balance between sensitivity and specificity, especially since more liberal criteria were
used in later steps of the classification process. It has been suggested by some
researchers that narrower confidence intervals created by this method may make true
change in test performance more readily apparent (Temkin et al., 1999). The use of
correction for practice effects in the RCI model, as Temkin noted, does not change the
width of the confidence interval, but does provide a method for revealing a degree of
change that may have masked, in this case, a true decrease in performance compared to
baseline.
Use of the basic RCI model resulted again in fairly good classification accuracy
within 24 hours of concussion using the one-or-more significant variable cut-point
(sensitivity = 0.76) as well as a notable increase in specificity at that test session (0.92)
over either of the first two models. The calculated AUC value at the 24-hour test session
of 0.861 suggested a fair ability to distinguish between concussed and nonconcussed
individuals. However, it was no better than chance at distinguishing between concussed
and control participants over the subsequent test sessions.
Correction for practice in the RCI model (Model 4) provided no improvement in
sensitivity (0.76) or specificity (0.92) at the 24-hour test session using the one-or-more
variable detection criteria. While correcting for practice did provide minimal
improvement in classification accuracy over the Day 3, Day 5, and Day 7 test sessions,
the model was not significantly different than chance at those intervals in distinguishing

119
between the groups based on AUC measures. Results were comparable to those obtained
by Temkin who indicated that it was the RCI that was the least effective of the models
studied by those researchers.
Exploring the apparently low utility of these models at more distal test sessions
from concussion, some underlying assumptions inherent to its application may be
influential. Firstly, the model assumes that a subject’s score at time one and time two are
equal. Secondly, it assumes that the standard deviations at those two sessions are also
equal. While the model does utilize the correlation between a control subject’s time one
and time two performance, the resulting standard error of the difference tended to be
large relative to the observed change in scores over the test intervals, creating relatively
wider confidence intervals, especially at the later testing sessions, and necessarily making
detection of true change more difficult.
In a model not previously examined in the literature in the context of serial
neuropsychological performance (Model 5), the change of concussed players scores was
directly compared to the average change scores of the control participants over the same
test intervals. The question asked in this model essentially was: is the change observed
for a concussed player large relative to the distribution of change scores for the control
population? Like the RCI models examined before it, this model created an index outside
of which a change in score from baseline was considered significant. Unlike the RCI
models, however, it did not assume that time one and two test scores were equal or that
the standard deviations of those measures were equal.
Like the previous models, utilization of Model 5 using a one-or-more variable cut
point resulted in a relatively good sensitivity (0.81) at 24 hours. It also achieved a high

120
degree of specificity, despite the liberal cut-point (0.92). Unlike the previous models,
however, Model 5 was also able to distinguish between concussed and control
participants at the Day 3 test session as well as at the Day 7 test session, but not at the
Day 5 test session. Although there was an obvious drop in sensitivity over time
compared to the 24-hour test session, which may have been accounted for by true
recovery of function in some individuals, the discrepancy between the sensitivity and
specificity values at those days suggested that this model is useful in the detection of
lingering cognitive dysfunction at test sessions more distant from the concussive event.
Examination of the sensitivity of individual variables for this model suggested a
distinction between those functions that were most easily detected in the immediate
period (24 hours) following concussions and those that continued to manifest detectable
enduring change at the later test sessions (Days 3, 5, and 7). Specifically, at the 24-hour
test session, Hopkins Total score (0.43), Hopkins Delay score (0.38), and Digit Span
Forward (0.33) were the most sensitive individual variables. This suggested that short¬
term verbal memory was affected in the highest number of participants immediately
following concussion. This is not to say that a change in other functions was not
detectable by the model. Trails A time as well as Grooved Pegboard (dominant and
nondominant) completion times also provided a smaller degree of sensitivity at the 24-
hour test session. While working memory, as indicated by Digit Span Forward,
continued to provide a degree of sensitivity at the Day 3 and Day 5 test sessions, it was
more often those variables measuring processing speed and speeded motor dexterity that
provided the greatest degree of sensitivity for this model, especially at the Day 7 test
session (Trails A sensitivity = 0.32; Grooved Pegboard nondominant sensitivity = 0.32).

121
These findings were consistent with those reported by Hinton-Bayre et al. (1997),
Maddocks (1996), and Barth (1989), all of whom found that processing speed, reaction
time, and attention continued to show some decrement in the 2-5 days following
concussion, despite, in some cases, the resolution of subjective neurological features such
as confusion, nausea, and headache.
AUC measures for Model 5 indicated fairly robust detection ability at the 24-hour
test session (0.888), a significantly-better-than-chance ability to distinguish between the
groups at Day 3 (0.682), and also significant classification ability at Day 7 (0.816). Of all
models tested as part of this study, Model 5 yielded the best classification rate at 24 hours
and at Day 7.
Regression Models
Similar classification rates were achieved using a simple linear regression model
(Model 6), where baseline performance was used to predict retest performance at the
various test intervals, and the difference between observed an predicted scores was
analyzed to determine if the difference was large relative to the residuals of the control
population. This method resulted in good sensitivity (0.86)/specificity (0.85) rates at the
24-hour test session using the one-variable cut-point, much like those observed for Model
5. Also similar to Model 5, the use of the regression equation model exhibited good
classification rates at the Day 3 and Day 7 test sessions, which was indicated by the AUC
measures for those time intervals (0.696 and 0.684 respectively). Again, in examining
why this model was superior to the first 4 in accurately classifying concussed and
nonconcussed individuals at those distal sessions, it was noted that while the sensitivity at
the one-variable cut point at each test interval did see a predictable decrease, the

122
corresponding specificity rates remained relatively high, allowing for relatively accurate
distinction between the groups using even a liberal identification criterion.
The final model examined as part of the current study involved the use of a
multiple regression analysis. Previous researchers have indicated that certain
demographic variables, including age and test-retest interval contributed significantly to
the prediction of retest scores, allowing for smaller standard deviations, and by definition,
smaller change indices (Temkin et al., 1999). This allowed for more sensitivity to
change, a property which is valuable in detecting the lingering effects of concussion. For
the current study, the use of age was not a viable demographic predictor since the range
of participants’ ages was limited.
However, it was hypothesized based on previous findings regarding the current
database (Collins et al., 1999), that history of learning disability and history of previous
concussion may have some impact on the incidence of concussion and possibly retest
performance. Therefore, they were included, along with baseline performance, in
stepwise multiple regression analyses of each variable over each test session. However,
the low listwise n precluded a reliable analysis of this model in the current study. At the
24-hour test session, the total listwise population of control subjects was 13, a number
substantially below that necessary to achieve reliable results in terms of accurate
regression model generation. There were fewer listwise control participants at each
successive test session, making multiple regression analysis even less reliable at those
intervals. It was therefore not practical to compare the detection rate of this model to the
others studied. Nevertheless, this analysis did reveal some interesting significant findings
despite the low power.

123
At the 24-hour retest session, the only significant predictor of performance for
any variable was baseline performance, with no significant predictors detected for three
of the nine variables. At the Day 3 test session, a history of learning disability was the
only significant predictor for Trails B, while history of previous concussion was the only
significant predictor for total COWA score. History of learning disability and baseline
performance both contributed significantly to prediction of Grooved Pegboard
Nondominant hand completion time. At Day 5, a history of learning disability and
baseline performance entered significantly into the regression equations for Hopkins
Total score and Trails B. History of learning disability was the only significant predictor
for Grooved Pegboard Dominant and Nondominant times at that test session. Finally, for
the Day 7 test session, history of concussion was the only significant predictor for
COWA total score.
Although these predictor variables were found to be statistically significant in the
regression models based on the control population retest scores, they did not generally
provide substantially improved sensitivity when applied to the concussed participants’
data relative to the other models studied. The only exception to this was the use of
history of concussion in the regression model for COWA total score at the Day 3 test
session. In this instance, the sensitivity value of 0.26 appeared to be substantially better
than the sensitivity values using the other 6 models for the same variable, which ranged
from 0.00 to 0.11. Overall, the use of these predictor variables in the current study
cannot be considered reliable due to the low number of individuals who were positive for
each.

124
In summary, all models examined as part of the current study were generally good
at detecting significant change within the 24 hour period following concussion using the
one-variable significance criterion, with each successive model providing slightly better
classification rates as expected. Overall, the classification rates for each were
comparable to those found by Barr and McCrea (2001) in examining the SAC with an
RCI and a linear regression model. In the absence of baseline measures on an individual
football player, Model 1 indicated that comparison of performance across the variables
studied can be compared to an appropriate normative population, and data collected as
part of this study can be used for such purposes. A significant difference from baseline
on just one variable (the chosen classification cut point) resulted in an acceptable level of
sensitivity. However, the corresponding specificity rate was quite low, highlighting the
need to define a level of specificity that is acceptable to the examining health
professional. This may ultimately lead to withholding the player from returning to action
when this is in fact not necessary. As was mentioned above though, erring on the side of
caution may prevent catastrophic effects. When one has the benefit of baseline test
measures, a comparison of retest performance to that level is obviously more appropriate.
As Model 2 indicated, sensitivity at that test interval retained comparable value to Model
1, but exhibited substantially increased specificity. Neither of these methods, however,
appeared to have any significant use in distinguishing between concussed and
nonconcussed players at subsequent test sessions. One reason for this may be that there
were no differences between the concussed and nonconcussed players at those times.
This assumption may lead one to believe that any cognitive decline suffered as a result of
the concussion itself had resolved.

125
The other models tested each provided acceptable sensitivity levels at the 24-
hour test session, with successively better classification accuracy as predicted, which was
indicated by the AUC measures. In examining the improvement in classification
accuracy, it was apparent (most obviously at the 24-hour test sessions) that this was in
part due to an increase in sensitivity, but was largely a function of an increase in
specificity, which created a greater distinction between the groups. This naturally
followed from the use of the matched control population tested over the same time
intervals who were, by definition, assumed not to be suffering from cognitive impairment
due to concussive injury. This increase in separation between the groups may indicate
that the models which were based, according to their respective methodologies, on data
gathered from control subjects are more tolerant of liberal confidence intervals. That is,
the use of a 90% confidence interval z-value of 1.645 may provide more detection power
for the models without substantially impacting the specificity as was the case in the
simple comparison to a normative population (Model 1) or comparison to baseline
performance relative to the standard deviation of the normative population (Model 2).
Previous studies have used this confidence interval value (Barr and McCrea, 2001;
Temkin et al., 1999) and suggested that the cost to specificity represented an acceptable
loss.
As was hypothesized, neither the basic comparison models (Models 1 and 2), the
RCI model (Model 3), nor the RCI model with practice correction (Model 4) were
accurate at detecting enduring cognitive sequelae and accurately distinguishing between
concussed and nonconcussed players at the Day 3, Day 5, and Day 7 test sessions, and
therefore appear not to provide reliable means for providing retum-to-play criteria for

126
concussed football players at those times. The only models to provide statistically
significant classification rates at Day 3 were the simple linear regression model (Model
6) and the analysis of change scores relative to the distribution of change scores for the
control population (Model 5). In addition, these two models were the only statistical
methods to successfully distinguish between the two groups at the Day 7 test interval,
although this result should be viewed with some caution due to the low number of
listwise participants at that test interval. It is notable that, despite the relatively low
power, the models did provide statistically significant classification accuracy as indicated
by their respective AUC values.
As was the case for both Temkin et al. (1999) and Barr and McCrea (2001), the
use of a linear regression model was more sensitive than either RCI model. This model,
which inherently provides correction for both practice effects and regression to the mean,
resulted in narrower confidence intervals due to the more accurate means of predicting
retest scores, which was reflected in the standard error of the residuals (the comparator)
for the control population. Overall, it provided the best classification accuracy, at the
Day 3 test session as indicated by the AUC measure. The most accurate classification
model overall, though, was Model 5, in which the change in a concussed player’s
performance on each variable was compared to the average control change on the
variable relative to the distribution of change scores for the control population. This
model, too, essentially factored in the effects of practice over the repeated testing
intervals, accounted for the variability of these practice effects in the control population,
and allowed for the direct rather than indirect comparison of a concussed player’s change
in score from baseline to the control population. Using this model, relatively good

127
accuracy classification rates were observed for the 24-hour, Day 3, and Day 7 test
sessions. A statistical comparison of Model 5 and Model 6 at those test intervals revealed
that they were not significantly different in terms of classification accuracy. However,
AUC values suggested a slightly greater accuracy at 24 hours and especially at Day 7,
making it more valuable in detecting possible enduring cognitive decline, which, as noted
above, appeared most likely to manifest in slower processing speed time.
The results obtained in the current study are particularly salient to the decision
regarding a players’ fitness for return to play. Over the past two decades, as more
attention has been paid to the potentially devastating short-term as well as long-term
effects of concussion, a number of propositions and guidelines have emerged concerning
the criteria that should be met before a player is deemed ready to return to play. As early
as 1986, Cantu had provided some guidelines for the classification of concussion
severity, which led to some recommendations for retum-to-play guidelines. These were
later revised in 2001. In 1991, the Colorado Medical Society released another system for
the grading of concussion severity, and with it, further recommendations for retum-to-
play decision-making. The Cantu recommendations appeared to be the most
conservative, with a call for removal from play for at least one symptom-free week after
suffering just one Grade 1 concussion. The Colorado guidelines on the other hand,
recommended just a 20-minute period, during which subjective signs of concussion had
resolved, before retum-to-play. The criteria for retum-to-play were more restrictive for
repeated Grade 1 concussions ranging from a one-week symptom-free hiatus, to
prescription for removal from play for the season if symptoms endured. For Grade 2 and
Grade 3 concussions, Cantu’s recommendations were again the most conservative,

128
ranging from a minimum of one symptom-free week for the first Grade 2 concussion to a
minimum of one symptom-free month for repeated Grade 2 concussion or the first Grade
3 concussion (Kelly and Rosenberg, 1997). The Colorado guidelines were slightly less
restrictive, but similar to those of Cantu.
In 1997, the Colorado guidelines were re-addressed by the American Academy of
Neurology (AAN), who then released their own version of retum-to-play guidelines.
These were subsequently endorsed by a number of organizations associated with
medicine in sports (Fuerst, 1997), but advocated only as “practice options” rather than
universal guidelines. These practice options now appear to be the most widely utilized
parameters for retum-to-play decision making. They are, however, the least restrictive of
the most commonly cited guidelines. For example, the AAN guidelines suggest that a
player be withheld from practice or gameplay for only 15 symptom-free minutes in the
event of the first Grade 1 concussion. In the case of multiple Grade 1 concussions or one
Grade 2 concussion, length of symptom-free hiatus was recommended to be one week.
Further, only a one-week removal from play was recommended for a first Grade 3
concussion given the apparent resolution of neurologic symptoms.
Results of the current study suggest that neuropsychological tests can provide
objective evidence of enduring cognitive sequelae of concussion, even at the one-week
period. As noted above, for these participants the most sensitive measures using Models
5 and 6 at that test interval appear to be those associated with processing speed and
speeded psycho-motor functions. Given this, it is recommended that a
neuropsychological battery including measures of these functions be utilized in
conjunction with those guidelines in establishing the resolution of concussion-induced

129
cognitive decline. Obviously, the further collection of control (normative) data will be
necessary to maintain an appropriate comparison for this purpose.
Limitations and Implications for Future Research
The largest limitation in the current study was obviously the low number of
participants over time, despite the collection of data over a two-year period. Attrition,
especially among the control population likely resulted in less accurate prediction and
comparisons across Models 3-7. This was most obvious in the analysis of potential
multiple regression models, which were rendered unreliable and ultimately not a viable
prediction option in the current study. Despite this, there appeared to be some
relationships between history of learning disability and history of concussion in serial
testing using the variables examined here. In addition, the low listwise number of control
participants also created the potential for skewing of the data by any particular outlier.
This, in fact, may have contributed to the inability of any of the models to successfully
distinguish between the concussed and control individuals at the Day 5 test session.
Certainly, as noted above, data for the Day 7 test session should be viewed with some
caution because of the low listwise n , though significant findings were observed despite
the low power. It must be noted that the number of participants in the current study was
generally similar to those obtained in previous studies examining the use of
neuropsychological test measures in a concussion population (Barth et al., 1989; Hinton-
Bayre et ah, 1997; Maddocks and Saling, 1996). Due to the time-intensive requirements
of a prospective research design with similar populations, the collection of larger data
sets will require extensive collaboration between researchers across multiple sites.

130
By its very nature, one may expect there to be a higher incidence of concussion in
the game of football, especially of the Grade 1 type. Given the effect that reporting to a
coach or trainer that one has suffered a concussion may have (i.e., restriction in playing
time), it would not be surprising for players not to acknowledge having received a hit that
caused some momentary confusion or disorientation. The identification of concussed
players was therefore reliant on observation by team trainers or coaches, which can only
be assumed to be the more obvious cases, and therefore a reduced sample of the entire
population of concussed individuals.
Future research should include the collection of a larger population of retest
controls. While using a yoked control approach did enable the collection of control
participants who were carefully matched on history of LD and history of concussion, the
attrition rate among those individuals suggests that selecting a representative sample of
players from a team and administering the series of neuropsychological tests over the
baseline and 4 retest sessions would also provide the data needed for the utilization of the
models examined. This may provide for a larger comparison population. This may affect
the evaluation of any of these individuals should they receive a concussion in the future,
however. In order to further assess the contribution of LD and history of concussion to
retest performance, emphasis should be placed on recruiting those individuals positive for
these variables for use as controls in serial testing with this battery of tests.
Given the relatively higher degree of specificity achieved with the more effective
models, analysis of their sensitivities should be performed utilizing a more liberal
confidence interval criteria. While a z-score of 1.96 was used in the current study in
order to provide a better balance in sensitivity and specificity and to be more consistent

131
with statistical literature, a z-score of 1.645, representing a 90% confidence interval may
provide a higher degree of sensitivity while not dramatically impacting the specificity
rate.
Future research may also focus on the integration of data concerning subjective
complaints of post-concussive symptoms. A brief review of self-reported concussive
complaints in the current population suggested that some subjective complaints,
including sleep difficulties and feelings of dizziness among others (Collins et al., 1999),
may also endure past the time during which most concussive complaints are expected to
have resolved, and may contribute to decline in cognitive performance. In addition, other
cognitive variables were included in the current battery after data was collected for this
study. Those measures included the Brief Visual Memory Test and a smell identification
test. Examination of the contribution of these variables to the identification of post-
concussive cognitive decline and return to baseline may prove useful in the future.
Contributions of the Current Study
One of the ancillary benefits of this study included its contribution to the
evaluation by team trainers and physicians of concussion in a college football population
at specific intervals after injury, which may be helpful in determining retum-to-play
status. As the study demonstrated, the use of both a linear regression prediction model
(Model 6) and an analysis of change from baseline relative to the distribution of change
scores for the control population (Model 5) provided relatively good classification
accuracy within 24 hours, at Day 3, and at Day 7. As noted above, there was not a
statistically significant difference between the models’ classification accuracy, although
Model 5 appears to have provided somewhat better classification accuracy at Day 7. It is

132
clear, however, that longer-term cognitive effects may be present following concussion,
which are not readily apparent without the appropriate data with which to compare retest
results.
Model 5 lends itself to the provision of readily observable significance
determination for each variable, especially for those not familiar with the statistics
involved in the regression prediction. A worksheet designed to provide analysis of retest
performance in identified concussed players was designed for use by team trainers and
physicians who have been trained in the administration of this brief neuropsychological
battery in order to provide data for retum-to-play decision-making. Significance
determination points for each variable was based on the control population from the
current study using Model 5 statistical methodology. Better classification accuracy is
likely with this model with the addition of further control population data collection.
This worksheet is shown in figure 16.

133
Player Name
Measure
Baseline
Performance
24 Hour
Performance
Difference
Cutoff
Significant
Hopkins Total
> decrease of 5
total words
recalled
Trails A
> increase of
+8.14 seconds
Trails B
> increase of
23.02 seconds
Digits Forward
> decrease of 1
digit recalled
Digits Back
> decrease of 4
digits recalled
Pegs Dominant
> increase of
8.95 seconds
Pegs Nondom.
>increase of
15.59 seconds
COWA Total
>_ decrease of
14 words
produced
Hopkins Delay
> decrease of 3
words recalled
Findings are significant if one or more variables are found to be significant
Measure
Baseline
Performance
Day 3
Performance
Difference
Cutoff
Significant
Hopkins Total
> decrease of 7
total words
recalled
Trails A
> increase of
+2.29 seconds
Trails B
>increase of
50.00 seconds
Digits Forward
> decrease of 2
digit recalled
Digits Back
> decrease of 2
digits recalled
Pegs Dominant
> increase of
7.32 seconds
Pegs Nondom.
> increase of
17.51 seconds
COWA Total
>_ decrease of
11 words
produced
Hopkins Delay
> decrease of 6
words recalled

134
Measure
Baseline
Performance
Day 7
Performance
Difference
Cutoff
Significant
Hopkins Total
> decrease of
24 total words
recalled
Trails A
> change of
-0.26 seconds
Trails B
> increase of
9.42 seconds
Digits Forward
> decrease of 2
digit recalled
Digits Back
> decrease of 3
digits recalled
Pegs Dominant
> increase of
7.15 seconds
Pegs Nondom.
> increase of
5.20 seconds
COWA Total
>_ decrease of
21 words
produced
Hopkins Delay
> decrease of 9
words recalled
Findings are significant if one or more variables are found to be significant
Figure 16. Example of Worksheet for the Detection of Ongoing Effects of Concussion in
College Football Players

REFERENCES
Ahmed, S., Bierley, R., and Sheikh, J.I. (2000). Post-traumatic amnesia after closed head
injury: A review of the literature and some suggestions for future research. Brain
Injury, 14: 765-780.
Alexander, M.P. (1992). Neuropsychiatric correlates of persistent postconcussive
syndrome. Journal of Head Trauma Rehabilitation, 1: 50-69.
Barr, W.B. and McCrea, M. (2001). Sensitivity and specificity of standardized
neurocognitive testing immediately following sports concussion. Journal of the
International Neuropsychological Society, 7: 693-701.
Barth, J.T., Alves, W.M., Ryan, T.V. (1989). Mild head injury in sports. In Levin, H.S.,
Eisenberg, H.M., and Benton, A.L. (eds.), Mild head injury, Oxford University
Press, New York, pp. 257-275.
Barth, J.T., Macciocchi, S.N., Giordani, B., Rimel, R., Jane, R.A., and Boll, J.T. (1983).
Neuropsychological sequelae of minor head injury. Neurosurgery, 13: 529-533.
Bennett, T.L. (1987). Neuropsychological aspects of complex partial seizures: diagnostic
and treatment issues. Clinical Neuropsychology, 9: 37-45.
Benton, A.L. and Hamsher, K.deS. (1989). Multilingual Aphasia Examination. Iowa
City, Iowa: AJA Associates.
Binder, L.M, Rohling, M.L., and Larabee, G.J. (1997). A review of mild head trauma.
Part I: meta-analytic review of neuropsychological studies. Journal of Clinical
and Experimental Neuropsychology, 19: 421-431.
Blumbergs, P.C., Scott, G., Manavis, J., Wainwright, H., Simpson, D.A., and McLean,
A.J. (1994). Staining of amyloid precursor protein to study axonal damage in mild
head injury. Lancet, 15;344: 1055-1056.
Busch, C. R. and Alpem, H.P. (1998). Depression after mild traumatic brain injury: a
review of current research. Neuropsychology Review, 8: 95-108.
135

136
Brandt, J. (1991). The Hopkins Verbal Learning Test: Development of a new verbal
memory test with six equivalent forms. The Clinical Neuropsychologist, 5: 125-
142.
Busch, C.R. and Alpem, H.P. (1998). Depression after mild traumatic brain injury.
Neuropsychology Review. 8: 95-108.
Cantu, R.C. (1986). Guidelines for return to contact sports after a cerebral concussion.
The Physician and Sportsmedicine, 14: 75-83.
Cantu, R.C. (1992). Cerebral concussion in sports. Sports Medicine, 14: 64-74.
Cantu, R.C. and Voy, R. (1995). Second impact syndrome: a risk in any sport. The
Physician and Sportsmedicine, 23: 27-36.
Chelune, G.J., Naugle, R.I., and Leuders, H. (1993). Individual change after epilepsy
surgery: practice effects and base-rate information. Neuropsychology, 7: 41-52.
Cha, D.H., Chang, M.Y., Chen, B.Y., and Chang, H.H. (2004). Receiver operating
characteristic analysis of an erythroblast morphologic scoring system to improve
identification of fetal cells in maternal blood. Prenatal Diagnosis, 24: 117-120.
Collins, M.W., Grindel, S.H., Lovell, M.R., Dede, D.E., Moser, D.J., Phalin, B.R., Nogle,
S., Wasik, M., Cordry, D., Daugherty, M.K., Sears, S.F., Nicolette, G., Indelicato,
P., and McKeag, D.B. (1999). Relationship between concussion and
neuropsychological performance in college football players. JAMA, 282: 964-970.
Cowey, A. (2001). Functional localization in the brain. From ancient to modem.
Psychologist, 14: 250-254.
DeYebenes, M, Ortero, A, Zunzunegui, M, Rodríguez-Laso, A., Sanchez-Sanchez, F, Del
Ser, T. (2003). Validation of a short cognitive tool for the screening of dementia
in elderly people with low education. International Journal of Geriatric
Psychiatry, 18: 925-936.
Dikmen, S., McLean, A., and Temkin, N. (1986). Neuropsychological and psychosocial
consequences of minor head injury. Journal of Neurology, Neurosurgery, and
Psychiatry, 49: 1227-1232.
Dikmen, S.S., Machamer, J.E., Winn, H.R., and Temkin, N.R. (1995).
Neuropsychological outcome at 1-year post head injury. Neuropsychology, 9: 80-
90.
Dikmen, S.S., Ross, B.L., Machamer, J.E, and Temkin, N.R. (1995). One year
psychosocial outcome in head injury. Journal of the International
Neuropsychological Society, 1:67-77.

137
Fenton, G., McClelland, R., Montgomery, A., MacFlynn, G, and Rutherford, W. (1993).
The postconcussional syndrome: social antecedents and psychological sequelae.
British Journal of Psychiatry, 162: 493-497.
Feurst, M.L. (1997). Neurologists offer concussion options. The Physician and
Sportsmedicine online, 25 URL:
http://www.physsportsmed.com/issues/1997/04apr/nb_neuro.htm, 02-06-2004.
Gennarelli, T.A and Graham, D.I. (1998). Neuropathology of the head injuries. Seminars
in Clinical Neuropsychiatry, 3: 160-175.
Glassmire, D.M., Kinney, D.I., and Greene R.L. (2003). Sensitivity and specificity of
MMPI-2 neurologic correction factors: Receiver operating characteristic analysis.
Assessment, 10: 299-309.
Graham, D.I., Adams, J.H., Nicoll, J.A., Maxwell, W.L., Gennarelli, T.A. (1995). The
nature, distribution and causes of traumatic brain injury. Brain Pathology, 5: 397-
406.
Gronwall, D.M. and Wrightson, P. (1974). Delayed recovery of intellectual functioning
after minor head injury. Lancet ii, 605-609.
Hanley, J.A. (1989). Receiver operating characteristic (ROC) methodology: the state of
the art. Critical Review in Diagnostic Imaging, 29: 307-335.
Harrison, C.L. and Dijkers, M. (1992). Traumatic brain injury registries in the United
States: An overview. Brain Injury, 6: 203-212.
Heathcote, A. (2003). Item recognition memory and the receiver operating characteristic.
Journal of Experimental Psychology: Learning, Memory, & Cognition, 29: 1210-
1230.
Hermann, B.P. (1985). Neuropsychological function and psychopathology in epilepsy.
Psychological Reports, 57: 275-278.
Hermann, B.P. and Wyler, A.R. (1988). Neuropsychological outcome of anterior
temporal lobectomy. Journal of Epilepsy, 1: 35-45.
Hermann, B.P., Seidenberg, M., Schoenfeld, J., Peterson, J., Leveroni, C, and Wyler,
A.R. (1996). Empirical techniques for determining the reliability, magnitude, and
pattern of neuropsychological change after epilepsy surgery. Epilepsia, 37: 942-
950.
Hinton-Bayre, A.D., Geffen, G.M., and McFarland, K. (1997). Mild head injury and
speed of information processing: A prospective study of professional rugby

138
league players. Journal of Clinical and Experimental Neuropsychology, 19: 275-
289.
Hinton-Bayre, A.D., Geffen, G.M., and Geffen, L.B. (1999). Concussion in contact
sports: Reliable change indices of impairment and recovery. Journal of Clinical
and Experimental Neuropsychology, 21: 70-86.
Jacobson, N.S. and Truax, P. (1991). Clinical significance: a statistical approach to
defining meaningful change in psychotherapy research. Journal of Consulting and
Clinical Psychology, 59: 12-19.
Jennett, B. and Bond, M. (1975). Assessment of outcome after severe brain damage. A
practical scale. The Lancet, i: 480-484.
Kelly, J.P. and Rosenberg, J.H. (1997). Practice parameter: the management of
concussion in sports. Neurology, 48: 581-585.
Kelly, J.P. and Rosenberg, J.H. (1998). The development of guidelines for the
management of concussion in sports. Journal of Head Trauma Rehabilitation, 13:
53-65.
Kibby, M.Y. and Long, C.J. (1996). Minor head injury: attempts at clarifying the
confusion. Brain Injury, 10: 159-186.
King, N.S. (1996). Emotional, neuropsychological, and organic factors: their use in the
prediction of postconcussion symptoms after mild and moderate head injuries.
Journal of Neurology, Neurosurgery, and Psychiatry, 61: 75-81.
Kingsley, R.E. (1996). Concise text of neuroscience. Baltimore: Williams & Wilkins.
Kneebone, A.C., Andrew, M.J., Baker, R.A., and Knight, J.L. (1998). Neuropsychologic
changes after coronary artery bypass graft in: use of reliable change indices.
Annals of Thoracic Surgery, 65: 1320-1325.
Kraemer, H.C, Morgan, G.A, Leech, N.L., Gelner, J.A., Vaske J.J, Harmon, R.J. (2003).
Measures of clinical significance. Journal of the American Academy of Child and
Adolescent Psychiatry, 42: 1524-1529.
Langfitt, T.W. and Gennarelli, T.A. (1982). Can the outcome from head injury be
improved? Journal of Neurosurgery, 56: 19-25.
Leininger, B.E., Gramling, S.E., and Farrell, A.D. (1990). Neuropsychological deficits in
symptomatic minor head injury patients after concussion and mild concussion.
Journal of Neurology, Neurosurgery, and Psychiatry, 53: 293-296.

139
Levin, H.S., Mattis, S., and Ruff, R.M. (1987). Neurobehavioral outcome following
minor head injury: a three-center study. Journal of Neurosurgery, 66: 234-243.
Lezak, M.D. (1995). Neuropsychological assessment: third edition. New York, NY:
Oxford University Press.
Lishman, W.A. (1988). Organic psychiatry: the psychological consequences of cerebral
disorder-third edition. Malden, MA: Blackwell Science, Inc.
Loring, D.W., Meador, K.J., and Lee, G.P. (1992). Criteria and validity issues in WADA
assessment. In: Bennett, T.L. (Ed); The neuropsychology of epilepsy. New York,
NY: Plenum Press, pp. 233-245.
Lovell, M.R. and Collins, M.W. (1998). Neuropsychological assessment of the college
football player. Journal of Head Trauma Rehabilitation, 13: 9-26.
Maddocks, D. and Saling, M. (1996). Neuropsychological deficits following concussion.
Brain Injury, 10: 99-103.
Matthews, C.G. and Klove, H. (1964). Instruction manual for the Adult Neuropsychology
Test Battery. Madison, WI: University of Wisconsin Medical School.
Martland, H.S. (1928). Punch drunk. JAMA, 91: 1103-1107.
McAllister, Thomas W. (1992). Neuropsychiatric sequelae of head injuries. Psychiatric
Clinics of North America, 15: 395-413.
McCrea, M., Kelly, J.P., Randolph, C., Kluge, J., Bartolic, E., Finn, G., and Baxter, B.
(1998). Standardized assessment of concussion (SAC): On-site mental status
evaluation of the athlete. Journal of Head Trauma Rehabilitation, 13: 27-35.
McCrory, P.R. and Berkovic, S.F. (1998). Second impact syndrome. Neurology, 50: 677-
683.
McFarland, K., Jackson, L., and Geffen, G. (2001). Post-traumatic amnesia: Consistency-
of-recovery and duration-to-recovery following traumatic brain impairment. The
Clinical Neuropsychologist, 15: 59-68.
McIntosh, T.K., Smith, D.H., Meaney, D.F., Kotapka, M.J., Gennarelli, T.A., and
Graham, D.I. (1996). Neuropathological sequelae of traumatic brain injury:
relationship to neurochemical and biomechanical mechanisms. Laboratory
Investigation, 74: 315-342.
McSweeny, J.A., Naugle, R.I., Chelune, G.J., Luders, H. (1993). “T Scores for Change”:
An illustration of a regression approach to depicting change in clinical
neuropsychology. The Clinical Neuropsychologist, 7: 300-312.

140
Meaney, D.F., Smith, D.H., Shreiber, D.I., Bain A.C., Miller R.T., Ross, D.T., and
Gennarelli, T.A. (1995). Biomechanical analysis of experimental diffuse axonal
injury. Journal of Neurotrauma, 12: 689-694.
Mild Traumatic Brain Injury Committee of the Head Injury InterdisciplinarySpecial
Interest Group of the American Congress of Rehabilitation Medicince. (1993).
Definition. Journal of Head Trauma Rehabilitation, 8: 86-87.
Millspaugh, J.A. (1937). Dementia pugilistica. U.S. Naval Medical Bulletin, 35: 297-303.
Milner, B. (1958). Psychological defects produced by temporal lobe excision. Research
Publications of the Association for Research in Nervous & Mental Disease, 36:
244-257.
Montgomery, E.A., Fenton, G.W., McClelland, R.J., MacFlynn, G., and Rutherford,
W.H. (1991). The psychobiology of minor head injury. Psychological Medicine,
21:375-384.
Oppenheimer, D.R. (1968). Microscopic lesions in the brain following head injury.
Journal of Neurosurgery and Psychiatry, 31: 299-306.
Parker, R.S. (1996). The spectrum of emotional distress and personality changes after
mild head injury incurred in a motor vehicle accident. Brain Injury, 10: 287-302.
Povlishock, J.T., Becker, D.P., Cheng, C.L., and Vaughan, G.W. (1983). Axonal change
in minor head injury. Journal of Neuropathology and Experimental Neurology,
42: 225-242.
Reitan, R. and Wolfson, D. (1985). The Halstead-Reitan neuropsychological test battery:
thory and clinical interpretation. Tuscon AZ: Neuropsychology Press.
Rimel, R., Giordani, M., Barth, J., Boll, T., Jane, J. (1981). Disability caused by minor
head injury. Neurosurgery, 9: 221-228.
Roberts, G.W., Gentlemen, S.M., Lynch, A., Murray, L., Landon, M., and Graham, D.I.
(1994). (3 Amyloid protein deposition in the brain after severe head injury:
implications for the pathogenesis of Alzheimer’s Disease. Journal of Neurology,
Neurosurgery, and Psychiatry, 57: 419-425.
Ross, S.R., Millis, S.R., Krukowski, R.A. (2004). Detecting incomplete effort on the
MMPI-2: An examination of the Fake-Bad Scale in mild head injury. Journal of
Clinical and Experimental Neuropsychology, 26: 115-124.
Ruff, R.M. (2003). A friendly critique of neuropsychology: facing the challenges of our
future. Archives of Clinical Neuropsychology, 18: 847-864.

141
Sawrie, S.M., Chelune, G.J., Naugle, R.I., and Luders, H.O. (1996). Empirical methods
for assessing neuropsychological change following epilepsy surgery. Journal of
the International Neuropsychological Society, 2: 556-564.
Smith, A. (1982). Symbol Digit Modalities Test (SDMT). manual (revised). Los Angeles:
Western Psychological Services.
Spreen, O. and Strauss, E. (1998). A compendium of neuropsychological tests: second
edition. New York, NY: Oxford University Press.
Strauss, I. and Savitsky, N. (1934). Head injury: neurologic and psychiatric aspects.
Archives of Neurology and Psychiatry, 31: 893-954.
Strich, S.J. (1956). Diffuse degeneration of the cerebral white matter in severe dementia
following head injury. Journal of Neurochemistry, 19: 163-185.
Temkin, N.R., Heaton, R.K., Grant, I., and Dikmen, S.S. (1999). Detecting significant
change in neuropsychological test performance: A comparison of four models.
Journal of the International Neuropsychological Society, 5: 357-369.
Thurman, D.J., Branche, C.M., and Sniezek, J.E. (1998). The epidemiology of sports-
related traumatic brain injuries in the united states: recent developments. Journal
of Head Trauma Rehabilitation, 13: 1-8.
Tysvaer, A.T., Storli, O.V., and Bachen, N.I. (1989). Soccer injuries to the brain: A
neurologic and electroencephalographic study of former players. Acta
Neurological Scandinavica, 80: 151-156.
Watson, M.R., Fenton, G.W., McClelland, R.J., Lumsden, J., Headley, M., and
Rutherford, W.H. (1995). The post-concussional state: neuropsychological
aspects. British Journal of Psychiatry, 167: 514-521.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale-Third Edition, administration
and scoring manual. San Antonio, TX: The Psychological Corporation.

BIOGRAPHICAL SKETCH
Benjamin R. Phalin was bom and raised in Florida. He attended Winter Park
High School in Orlando, where he graduated as a Florida Academic Scholar in 1989.
Benjamin went on to attend the University of Florida in 1989, achieving his Bachelor of
Science degree in psychology in 1994, and was an inductee into the Phi Beta Kappa and
Golden Key Honor Societies at that institution. He was admitted to the doctoral program
in clinical and health psychology at the University of Florida in 1995, and, after
completion of the course and practica requirements of that program, completed his
internship in neuropsychology at the James A. Hailey V.A. Medical Center in Tampa,
Florida. He received his Doctor of Philosophy in August, 2004, and currently works as a
post-doctoral resident in private practice in Gainesville, Florida, where he is supervised
by Ernest J. Bordini, Ph.D.
142

I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
¿JjULtU (j£)> QCtVAjLÍL
Eileen B. Fennell, Chair
Professor of Clinical and
Health Psychology
I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
Duane E. Dede, Cochair
Clinical Associate Professor
of Clinical and Health
Psychology
I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
Health Psychology
I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
Jadíes J. Algina
Professor of Educati
Psychology

This dissertation was submitted to the Graduate Faculty of the College of Public
Health and Health Professions and to the Graduate School and was accepted as partial
fulfillment of the requirements for the degree of Doctor of Philosophy.
August 2004
Dean, College of Public
Health and Health
Professions
yi—fcj
A
Dean, Graduaje School

UNIVERSITY OF FLORIDA
1262 07332 077 1