<%BANNER%>

Quantifying the Risk of Financial Events Using Kernel Methods and Information Retrieval


PAGE 1

QUANTIFYING THE RISK OF FINANCIA L EVENTS USING KERNEL METHODS AND INFORMATION RETRIEVAL By MARK CECCHINI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2005

PAGE 2

Copyright 2005 by Mark Cecchini

PAGE 3

This document is dedicated to Tara, Julian and Campbell, who were my inspiration in pursuing and finishing a PhD.

PAGE 4

iv ACKNOWLEDGMENTS I would like to thank Tara and the rest of the Cecchi nis for putting up with me throughout this process. I would also lik e to thank our families for their support throughout this endeavor. Without my comm ittee there would be no dissertation. So, I would like to acknowledge my Advisor Gary Koehler, who came up with the initial research idea and has seen this research through from the beginning, Haldun Aytug, who has been working on this project for three years, Praveen Pathak for his information retrieval expertise and Gary McGill fo r helping me to understand the accounting relevance of the work. Finally, I’d like to thank Karl Hackenbrack for his guidance in the early stages of this work.

PAGE 5

v TABLE OF CONTENTS page ACKNOWLEDGMENTS.................................................................................................iv LIST OF TABLES...........................................................................................................viii LIST OF FIGURES.............................................................................................................x LIST OF OBJECTS...........................................................................................................xi ABSTRACT......................................................................................................................x ii CHAPTER 1 INTRODUCTION AND MOTIVATION....................................................................1 2 FINANCIAL EVENTS................................................................................................5 2.1 Fraud Detection.....................................................................................................5 2.2 Bankruptcy Detection.............................................................................................8 2.3 Restatement Detection..........................................................................................12 3 INFORMATION RETRIEVAL METHODOLOGIES..............................................16 3.1 Overview...............................................................................................................16 3.2 Vector Space Model.............................................................................................18 3.3 WordNet...............................................................................................................21 3.4 Ontology Creation................................................................................................23 4 MACHINE LEARNING METHODOLOGIES.........................................................27 4.1 Statistical Learning Theory...................................................................................28 4.2 Support Vector Machines.....................................................................................29 4.3 Kernel Methods....................................................................................................33 4.3.1 General Kernel Methods.............................................................................34 4.3.2 Domain Specific Kernels...........................................................................40 5 THE FINANCIAL KERNEL.....................................................................................43 6 THE ACCOUNTING ONTOLOGY AND CONVERSION OF DOCUMENTS TO TEXT VECTORS.................................................................................................54

PAGE 6

vi 6.1 The Accounting Ontology....................................................................................54 6.1.1 Step 1: Determine Concepts and Novel Terms that are specific to the accounting domain...........................................................................................55 6.1.2 Step 2: Merge Novel Terms with Concepts..............................................61 6.1.3 Step 3: Add multi-word domain concepts to WordNet.............................64 6.2 Converting Text to a Vector via the Accounting Ontology..................................65 7 COMBINING QUANTITATIVE AND TEXT DATA.............................................69 8 RESEARCH QUESTIONS, METHODOLOGY AND DATA.................................72 8.1 Hypotheses............................................................................................................72 8.2 Research Model....................................................................................................74 8.3 Datasets.................................................................................................................76 8.3.1 Fraud Data..................................................................................................76 8.3.2 Bankruptcy Data.........................................................................................77 8.3.3 Restatement Data........................................................................................79 8.4 The Ontology........................................................................................................80 8.5 Data Gathering and Preprocessing........................................................................82 8.5.1 Preprocessing-Quantitative Data................................................................84 8.5.2 Preprocessing-Text Data............................................................................84 9 RESULTS...................................................................................................................88 9.1 Fraud Results........................................................................................................89 9.2 Discussion of Fraud Results.................................................................................92 9.3 Bankruptcy Results...............................................................................................94 9.4 Discussion of Bankruptcy Results........................................................................97 9.5 Restatement Results..............................................................................................98 9.6 Discussion of Restatement Results.....................................................................102 9.7 Support of Hypotheses........................................................................................103 10 SUMMARY, CONCLUSION AN D FUTURE RESEARCH..................................104 10.1 Summary...........................................................................................................104 10.2 Conclusion........................................................................................................105 10.3 Future Research................................................................................................106 APPENDIX A ONTOLOGIES AND STOPLIST............................................................................109 A.1 Ontologies..........................................................................................................109 A.1.1 GAAP, 300 Dimensions, 100 concepts, 100 2-grams, 100 3-grams.......109 A.1.2 GAAP, 60 Dimensions, 40 concepts, 10 2-grams, 10 3-grams...............115 A.1.3 GAAP, 10 Dimensions, 10 concepts.......................................................117 A.1.4 10K, Bankruptcy, 100 Dimensions..........................................................117 A.1.5 10K, Bankruptcy, 50 Dimensions, 50 Concepts......................................119

PAGE 7

vii A.1.6 10K, Bankruptcy, 25 Dimensions, 25 concepts.......................................120 A.1.7 10K, Fraud, 150 Dimensions, 50 concepts, 50 2-grams, 50 3-grams......121 A.1.8 10K, Fraud, 50 Dimensions, 50 concepts................................................124 A.1.9 10K, Fraud, 25 Dimensions, 25 concepts................................................125 A.2 Stoplist...............................................................................................................127 B QUANTITATIVE AND TEXT DATA....................................................................128 B.1 Quantitative Data...............................................................................................129 B.2 Text Data............................................................................................................162 LIST OF REFERENCES.................................................................................................163 BIOGRAPHICAL SKETCH...........................................................................................172

PAGE 8

viii LIST OF TABLES Table page 1 – Financial Kernel Validation.........................................................................................51 2 – Fraud Detection Results using Financial Kernel.........................................................89 3 – Fraud Detection Results using Text Kernel, 300 Dim GAAP Ont..............................90 4 – Fraud Detection Results using Comb. Kernel, 300 Dim GAAP Ont...........................90 5 – Fraud Detection Results using Text Kernel, 60 Dim GAAP Ont................................90 6 – Fraud Detection Results using Comb. Kernel, 60 Dim GAAP Ont.............................90 7 – Fraud Detection Results using Text Kernel, 10 Dim GAAP Ont................................90 8 – Fraud Detection Results using Comb. Kernel, 10 Dim GAAP Ont.............................91 9 – Fraud Detection Results usi ng Text Kernel, 150 Dim 10K Ont..................................91 10 – Fraud Detection Results usi ng Comb. Kernel, 150 Dim 10K Ont.............................91 11 – Fraud Detection Results usi ng Text Kernel, 50 Dim 10K Ont..................................91 12 – Fraud Detection Results usi ng Comb. Kernel, 50 Dim 10K Ont...............................91 13 – Fraud Detection Results usi ng Text Kernel, 25 Dim 10K Ont..................................92 14 – Fraud Detection Results usi ng Comb. Kernel, 25 Dim 10K Ont...............................92 15 – Bankruptcy Prediction Results using Financial Kernel.............................................94 16 – Bankruptcy Prediction Results us ing Text Kernel, 300 Dim GAAP Ont..................94 17 – Bankruptcy Prediction Results usi ng Comb. Kernel, 300 Dim GAAP Ont..............94 18 – Bankruptcy Prediction Results usi ng Text Kernel, 60 Dim GAAP Ont....................95 19 – Bankruptcy Prediction Results using Combination Kernel, 60 Dim GAAP Ont......95 20 – Bankruptcy Prediction Results usi ng Text Kernel, 10 Dim GAAP Ont....................95

PAGE 9

ix 21 – Bankruptcy Prediction Results using Combination Kernel, 10 Dim GAAP Ont......95 22 – Bankruptcy Prediction Results us ing Text Kernel, 100 Dim 10K Ont......................95 23 – Bankruptcy Prediction Results using Combination Kernel, 100 Dim 10K Ont........96 24 – Bankruptcy Prediction Results using Text Kernel, 50 Dim 10K Ont........................96 25 – Bankruptcy Prediction Results using Combination Kernel, 50 Dim 10K Ont..........96 26 – Bankruptcy Prediction Results using Text Kernel, 25 Dim 10K Ont........................96 27 – Bankruptcy Prediction Results using Te xt Kernel combined with Financial Attributes, 25 Dim 10K Ont.....................................................................................97 28 – Restatement (1,379 cases) Prediction Results using Financial Kernel......................99 29 – Restatement Prediction Resu lts using Financial Kernel............................................99 30 – Restatement Prediction Results us ing Text Kernel, 300 Dim GAAP Ont.................99 31 – Restatement Prediction Results usi ng Comb. Kernel, 300 Dim GAAP Ont.............99 32 – Restatement Prediction Results us ing Text Kernel, 60 Dim GAAP Ont.................100 33 – Restatement Prediction Results usi ng Combination Kernel, 60 Dim GAAP Ont...100 34 – Restatement Prediction Results us ing Text Kernel, 10 Dim GAAP Ont.................100 35 – Restatement Prediction Results usi ng Combination Kernel, 10 Dim GAAP Ont..100 36 – Restatement Prediction Results us ing Text Kernel, 150 Dim 10K Ont...................100 37 – Restatement Prediction Results usi ng Combination Kernel, 150 Dim 10K Ont.....101 38 – Restatement Prediction Results us ing Text Kernel, 50 Dim 10K Ont.....................101 39 – Restatement Prediction Results usi ng Combination Kernel, 50 Dim 10K Ont.......101 40 – Restatement Prediction Results us ing Text Kernel, 25 Dim 10K Ont.....................101 41 – Restatement Prediction Results using Text Kernel combined with Financial Attributes, 25 Dim 10K Ont...................................................................................101

PAGE 10

x LIST OF FIGURES Figure page 1 – Ontology Hierarchy.....................................................................................................23 2 – Basic Graph Kernel......................................................................................................38 3 – Graph Kernel............................................................................................................... .40 4 – The Financial Kernel 1.................................................................................................48 6 – Updated Financial Kernel............................................................................................53 7 – Accounting Ontology Creation Process.......................................................................56 8 – WordNet Noun hierarchy with Domain Concepts.......................................................61 9 – WordNet Noun hierarchy with Domain Concepts enriched with Novel Terms..........63 10 – WordNet Noun Hierarchy with Domain Concepts, Novel Terms and Multi-Word Concepts...................................................................................................................65 11 – Text Kernel............................................................................................................... .69 12 – Combined Kernel.......................................................................................................70 13 – The Discovery Process...............................................................................................75 14 – Fraud Features............................................................................................................ 77 15 – Bankruptcy Features..................................................................................................78 16 – Fraud Dataset...........................................................................................................13 0 17 – Bankruptcy Dataset..................................................................................................134 18 Restatement Dataset.................................................................................................139

PAGE 11

xi LIST OF OBJECTS Object page 1. Text Data................................................................................................................... ...162

PAGE 12

xii Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy QUANTIFYING THE RISK OF FINANCIA L EVENTS USING KERNEL METHODS AND INFORMATION RETRIEVAL By Mark Cecchini August, 2005 Chair: Gary Koehler Major Department: Decision and Information Sciences A financial event is any happening which dram atically changes the value of a firm. Examples of financial events are manageme nt fraud, bankruptcy, exceptional earnings announcements, restatements, and changes in corporate structure. This dissertation creates a method for timely detection of fina ncial events using machine learning methods to create a discriminant function. As th ere are a myriad of possible causes for any financial event, the method created must be pow erful. In order to increase the power of current methods of detection text related to the company is analyzed together with quantitative information on the company. Th e text variables are chosen based on an automatically created accounting ontology. The quantitative variables are mapped to a higher dimension which takes into account ratios and year-over-year changes. The mapping is achieved via a kernel. Support vector machines use the kernel to perform the learning task. The methodology is tested em pirically on three datasets: management fraud, bankruptcy, and financial restatements. The results show that the methodology is competitive with the leading management fr aud detection methods. The bankruptcy and restatement results show promise.

PAGE 13

1 CHAPTER 1 INTRODUCTION AND MOTIVATION SAS 99, Consideration of Fraud in a Financial Statement Audit establishes external auditors’ responsibility to plan and perform audits to provide a reasonable assurance that the audited financial statements are free of material fraud. Recent events highlight that failing to detect fraudulent financial reporting not only exposes the audit firm to adverse legal consequences (e.g., th e demise of Arthur Andersen LLP), but exposes the audit profession to increased publ ic and governmental scrutiny that can lead to fundamental changes in the structure of the public accounting industry, accounting firm conduct, and government oversight of the accounting profession (consider, for example, the Sarbanes-Oxley Act of 2002 89 and subsequent actions of the SEC 92 and NYSE 71). Research that help s auditors better as sess the risk of material misstatement during the planning phase of an audit will re duce instances of fraudulent reporting. Such research is of interest to academics, sta ndard setters, regulators, and audit firms. Current research in accounting has exam ined methods to assess the risk of fraudulent financial reporting. The methodologi es are varied and usually combine some behavioral and quantitative factors. For example, Loebbecke, Eining and Willingham 55 compiled an extensive list of company char acteristics associated with fraudulent reporting (called "red flags"). This list contains financial ra tios and behavioral characteristics of company ma nagement. Other methods scrutin ize accounting entries that are not easily verified by outsi de sources; these entries are called discretionary accruals.

PAGE 14

2 Board composition and executive compensation are also used to model the type of environment that is ripe for fraud. This dissertation proposes a methodology that can estimate the likelihood of fraudulent financial reporting. The resulting decision-aid ha s the potential to complement the unaided auditor risk asse ssments envisioned in SAS 99. Our approach combines novel aspects of the fraud assessment research in accounting with computational methods and theory used in Information Retrieval (IR) and machine learning/datamining. Machine learning uses computational tech niques to automate the discovery of patterns that may be difficul t to find by normal analytic te chniques. Machine learning methodologies have been used in order to determine financial statement validity or, somewhat related, the likelihood of bankruptcy and credit worthiness. There are many models commonly used in machine learning wi th neural networks 66, linear discriminant functions 34, logit functions 3, and decision trees 80 being popular choices. Attempts have been made to recognize patterns in fraudulent companies using neural networks, linear discriminant functions, logit functions, and decision trees. These studies utilized quantitative data from financial statements and surveys from auditors. Unlike these earlier studies, recent advances in machine l earning theory consider generalization ability and domain knowledge while the learning task is undertaken. Existing work on fraud detection has left out a key component of and about the company, text documents. In most public documents, the preponderance of information is textual but most automated methods fo r detecting fraud are based on quantitative information alone. So, either an expert has to distill the text to numbers, which is a monumental task, or the text-based informati on is largely ignored. We hypothesize that

PAGE 15

3 there is information hidden “between the line s” that is overlooked. Our approach can incorporate textual materials like management discussion and analysis, news articles, and so on. An area of research called Information Retrieval (IR) can help us to make use of the text. IR is often employed in library sc ience and, more recently, in powerful Internet search engines (such as Google 39). IR is used for varied purposes, including question answering, document sorting, knowledge engi neering, query expansion, and inferencing. We use IR methodologies to cull the financial text down to numbers, which can be used in conjunction with numerical attributes obtained from the financial statements to automatically predict the likelihood of fraud. What distinguishes the propos ed approach from prior attempts to understand and aid fraud-risk assessments are advances in machine learning theory, both through a theory that addresses generalization errors and methods incorpor ating domain knowledge while the learning task is undertaken, and in IR that enable computer programs to analyze textual materials. The methodologies we create can be genera lized to other accounti ng issues, such as the early detection of bankrupt cy, detection of earnings ma nagement, early detection of increased market value, and general industry stability. Each of these issues has the potential to impact a company’ s value significantly shortly after a related first press release or news item is made public. As a resu lt of the speedy impact, these issues can be called financial events. In this dissertation, we look at the early de tection of bankruptcy, together with the detecti on of management fraud.

PAGE 16

4 The goal of this dissertation is discusse d below. In the following chapter we review financial events detection literatu re and summarize key concepts and results. Chapter 3 summarizes relevant machine le arning literature. Chapter 4 summarizes relevant Information Retrieval literature. In Chapter 5 we develop a machine learning methodology that handles quantitative financial data. In Chapter 6 we develop the IR methodologies that enable us to utilize text fo r financial event detection. In Chapter 7 we explain how we put the text data together wi th the quantitative data. We also extend the methodology we create in Chapter 5 to include text. These methods are used to study some actual data on which we ask a number of questions. The research model and hypotheses are developed in Chapter 8 and test ed. Chapter 9 explains the results along with a conclusion and an explanation of future work.

PAGE 17

5 CHAPTER 2 FINANCIAL EVENTS As explained in the Introduction, a financia l event is any event that significantly alters the value of a company. One can think of such an event as one that raises or lowers the value of the company. A partial list of possible events which lower the value of the firm are as follows: civil or criminal litigation, bankruptcy, management fraud, defalcations, restatements, ear nings management and poor press. We focus on three such events in particular, management fraud, bankr uptcy and restatements. In Section 2.1 we look at the fraud detection literature from accounting and machine learning. In section 2.2 we look at bankruptcy detection literature from those perspectives as well. In section 2.3 we look at the Restatements literature. 2.1 Fraud Detection A key result in audit research was given by Loebbecke, Eining and Willingham 55. They partitioned a large set of indicators into three main components: conditions, motivation, and attitude. They find in 86% of the fraud cases at least one factor from each component was present, indicating it is ex tremely rare for fraud to exist without all three components existing simultaneously. Hackenbrack 41 finds th e relative influence of such components on auditor fraud-risk asse ssments varies systematically with auditor experiences. This research influenced standa rd setting and much of the fraud assessment research that has followed. Bell and Carcello 9 developed a logistic regression model to estimate the likelihood of fraudulent financial reporting. The significant risk factors considered were as follows:

PAGE 18

6 weak internal control envir onment, rapid company growth, inadequate or inconsistent relative profitability, management places undue emphasis on meeting earnings projections, management lied to the auditors or was overly evasive, the ownership status (public vs. private) of the entity, and an interaction term between a weak control environment and an aggressive management attitude toward financial reporting. The logistic regression model was tested on a sample of 77 fraud engagements and 305 nonfraud engagements. The model scored better than auditing professionals in the detection of fraud. The model performed equally as we ll as audit professionals for the non-fraud portion. The authors suggest that the use of th is model might be used to satisfy the SAS 82 requirements for assessing the risk of material misstatement due to fraud. Hansen, McDonald, Messier, and Bell 42 develop a generalized qualitativeresponse model to analyze management fraud. They use the same dataset of 77 fraud and 305 nonfraud cases as collected by Loebbecke, Eining, and Willingham. They first tested the model with symmetric costs between type I a nd type II errors. Over 20 trials they got an 89.3% predictive accuracy. They adjusted the model to allow for asymmetric costs and the accuracy dropped to 85.5%; however, the type II erro rs decreased markedly. The consideration of type I and II errors is important in fraud detection research. Minimizing the type II error is minimizing the chance that the model will miss an actual fraud company. When type II error is minimized, t ype I error will increa se. In the case of fraud detection, type I error is much le ss important than type II error. In fraud detection, discretionary accruals are a cause for concer n as discretionary accruals have been known to be used to help “smooth” fluctuations in periodic income. Accounts that are used in disc retionary accruals, such as Bad Debts Expense, Inventory

PAGE 19

7 and Accounts Receivable, are susceptible to “engineering” on the part of management. By considering year-over-year changes in ra tios, which include th ese accounts, a clearer picture of the company emerges. McNichols and Wilson 56 look at the provi sion for bad debts and consider how it should be reported in the absence of earnings management. Earnings management is a term that describes a spectrum of “cheating” that at a minimum is aggressive and not in strict compliance with GAAP, and at maxi mum is management fraud. The research found that firms use the provision for bad debt s as an income smoothing method; in other words, it is raised in times of high earnings and lowered in times of low earnings. Ragothaman, Carpenter, and Buttars 82 de veloped an expert system to help auditors in the planning stage of an audit. The system is de signed to detect error potential in order to determine if additional substantiv e testing is necessary for the auditors. The expert system rules were deve loped using financial statemen t data. The expert system methodology was rule induction. The system deci des whether the firm is an "error" firm or a "non-error" firm. If the firm is an "e rror" firm then the auditor should consider additional substantive testing. A training sa mple of 55 firms (22 error firms and 33 nonerror firms) was used. A holdout sample of 37 firms was used. The training sample was able to group 86.4% of errors correctly and 100% of non-error firms correctly. The holdout sample classified 83.3% of error firms correctly and 92% of non-error firms. This study was limited by the available data. The accounting literature on fraud detection is covered at great length in Davia 25 and Rezaee 84. Beneish 10 developed a probit model and c onsidered several qua ntitative financial variables for fraud detection. Five of the 8 variables involved yea r-over-year changes.

PAGE 20

8 The study considered differing levels of relative error cost. At the 40:1 error cost ratio (Type I:Type II) 76% of the manipulators were correctly identified. Also, descriptive statistics showed that the Days Receivable Index and the Sales Growth Index were most effective in separating the manipul ators from the non-manipulators. 2.2 Bankruptcy Detection Bankruptcy detection is a we ll-studied area. Many met hodologies have been used to solve this problem, including discriminant analysis, neural networks, fuzzy networks, ID3 (a decision tree classification algorithm), logistic regression a nd genetic algorithms. In this section we describe some ma jor contributions to the literature. In 1966 Beaver 8 showed the efficacy of fi nancial ratios for detecting bankruptcy. The study was performed on a dataset of 79 ba nkrupt and 79 nonbankrupt firms. Beaver computed the mean values of fourteen financ ial ratios for all companies in the study for a five year period prior to bankr uptcy. Many of the ratios prov ed to be valuable to the detection, because the mean values of th e bankrupt companies were significantly different than the mean value for the nonbankrupt companies. Altman’s paper in 1968 4 was a semina l work in bankruptcy detection. He developed a discriminant analysis model usi ng financial ratios. Using a paired-sample approach, Altman compared twenty-two rati os for efficacy in bankruptcy prediction. Five ratios stood out as they were able to accurately predict bankruptcy one year preceding the event. The model predicted bankruptcy correctly 95% of the time and nonbankruptcy correctly 80% of the time. The resulting function, dubbed the Altman ZScore, has been the benchmark for bankruptcy detection work ever since. The specific ratios of the Altman Z-Score are as follows: Working Capital/Total Assets (WC/TA)

PAGE 21

9 Retained Earnings/Total Assets (RE/TA) Earnings Before Interest and Ta xes/Retained Earnings (EBIT/RE) Book Value of Equity/Total Liabilities (BVE/TL). The function has had several in carnations and its weights differ based on industry. For the manufacturing industry it is 6.56(/)3.26(/)6.72(/)1.05(/) WCTARETAEBITREBVETLZscore The weights on each ratio indicate the rati o’s relative importance for classification of healthy and unhealthy companies. A score wh ich is less than some threshold means the company is likely in financial distress while a score greater than or equal to this threshold means the company is likely safe from bankruptcy, at least for the short term. There is a gray area around the threshold that can be construed as an area of concern. The predictive accuracy of this discriminant analysis function is still competitive today for sorting out healthy companie s from unhealthy ones. Altman et al. noted that the discrimina nt analysis technique had limitations, one being its inability to handle time series 5. Bankruptcy is the sum product of many events. A company which goes bankr upt is likely to have been in a deteriorating state for more than one period. Year-over-year changes can cap ture this deterioration better than single year measures. Ohlson 72 was the first to utilize a logi stic regression approach to bankruptcy prediction. He identified four factors as statistically significant in affecting the probability of failure within one year. The factors are as follows: the size of the company, a measure of financial structure, a measure of performance, and a measure of current liquidity. Another finding of the resear ch was that the predictive powers of linear

PAGE 22

10 transforms of a vector of ratios appear to be robust for estimating the probability of bankruptcy. Abdel-khalik and El-Sheshai 1 designed an experiment testing human judgment. Decision makers (loan officers) were allowed to choose the in formation cues they use to make their judgments. The information cues were used to determine whether a loan would end in default. In comparison to mech anical models (discriminant analysis), loan officers performed worse. The finding was that the choice of information cues is more responsible for the lack of correct predic tion than the processing of the cues. Frydman, Altman and Kao 36 developed a recursive partiti oning algorithm (RPA) for bankruptcy classification. The RPA is a Ba yesian procedure, with classification rules derived in order to minimize the expected cost of misclassification. In most cases, the RPA outperformed Altman’s previous results via discriminant analysis. Messier and Hansen 62 use inductive infere nce to analyze examples of bankrupt companies and loan defaults to infer a set of general rules in the form of if-then-else statements. The set of output statements is called a production sy stem. The bankrupt study used only the following ratios: current ra tio, earnings to total tangible assets and retained earnings to total tangible asset. The production system was 100% accurate on a very small holdout set (12 bankrupt and 4 nonbankrupt). The study also used the production system to detect potential loan defaults. The method was 100% accurate on the training sample and 87.5% accurate on a validation sample. In both studies, the production system used fewer ratios and was mo re accurate than the discriminant models it was compared against.

PAGE 23

11 It should be noted that Tsai and Koehler 104 tested the robustne ss of the results of several papers using inductive learning, includ ing the results of Messi er and Hansen 62. The authors determined the accuracy of the i nduced concepts when tested on the same or similar domains. In the case of Messier and Hansen, their findings included a probability of error on the learned concept of the bankrupt cy sample. The probability that the error of the learned concept exceeds 20% is 30.96%. This is due, in part, to the small sample size. The study throws up a caution flag, wa rning readers that the true accuracy of concepts learned by induction may not be re vealed in studies of small sample size. Tam and Kiang 101 use a back propagati on neural network to predict bank defaults. They compare their results with k neares t neighbor, discriminant analysis 34, logistic regression 79 and ID3 80. When c onsidering the year prior to bankruptcy, a multilayer neural network got the best results. When considering two years prior to bankruptcy, logistic regre ssion performed the best. Charalambous, Charitou, and Kaourou 17 comp are the performance of three neural network methods, namely Learning Vector Quan tization, Radial Basis Function, and the Feedforward network. They test their results on 139 matched pa irs of bankrupt and nonbankrupt U.S. firms. Their results indi cate that Learning Vect or Quantization gave superior results over feedforward networks and logit analysis. Piramuthu, Raghavan, and Shaw 77 develop a method of feature construction. The method finds the features that are most pertinent to th e classification problem and discards the ones that are not. The “constructe d” features are fed in to a back propagation neural network. The method was tested on Tam and Kiang’s 101 bankruptcy data. The

PAGE 24

12 network showed a significant improvement in both classification results and computation time. 2.3 Restatement Detection The literature covering restatements is found in conjunction with earnings management literature as well as fraud liter ature. Each company for which the SEC discovers fraud is forced to restate. A ll restatements, however, are not fraudulent. Restatements can be made for various reas ons, including stock splits, errors, accounting irregularities, and fraud. Restatements may be voluntary or in voluntary. For the purposes of this research, restatements ar e defined as in General Accounting Office report GAO-03-138 37. These restatements ma y be voluntary or i nvoluntary and only arise as a result of accounting irregularities. An accounting irregular ity is fraudulent if committed with intention and nonfraudulent if committed by mistake. Restatements can be seen as a superset of fraud. The restatem ent literature specifically related to detection is limited as compared to fraud and bankruptcy The literature review ed in this section gives an overview of the research problems related to restatements. Dechow et al. 26 evaluate the performa nce of competing models of earnings management detection. The models tested are the Jones model, the Modified Jones model, the DeAngelo model and the Industr y model. These models are based on the amount of discretionary accruals made by a comp any in a particular year. Discretionary accruals are not readily observable based on publ icly available reports. The models infer the amount of discretionary accru als based on other inputs and total accruals. The results show that all methods are accura te for detecting earnings management for extreme cases. However, all methods gave poor results when faced with discretionary accruals which were a small percentage of total assets (1% 5%). Earnings management is more likely

PAGE 25

13 to occur at the 1% 5% levels so the practical value of th e methods used is brought into question. Koch and Wall develop an economic m odel of earnings management, which elucidates the situations in which earnings ma nagement are most likel y to occur, based on executive compensation packages. The authors determine how accruals can be used to manage reported earnings. In the paper the authors explain several earnings management tactics. A partial list follows; (1) The “Live for Today” strategy Mana gers minimize accrued expenses in order to maximize profit. (2) The “Occasional Big Bath” strategy ma nagers attempt to meet earnings targets whenever possible. If it looks impossible to meet targets then they attempt to accrue a high amount of e xpenses in that period to allow for meeting earnings targets the next. (3) Miscellaneous Cookie Jar Reserves stra tegy – This is defined as the usage of unrealistic assumptions in the process of estimating accruals. These methods can be readily detected after-the -fact. It is much more difficult to detect these tactics as they are happening. Abbot et al. 1 study the impact of the audit committee on the likelihood of restatement. The authors find that an independent and active audit committee significantly reduces the likelihood of restatemen t. An audit committee which contains at least one member with financia l expertise further reduces th e likelihood of restatement. This empirical study gives weight to the arguments for having an audit committee.

PAGE 26

14 Feroz et al. 33 study the eff ect of Accounting and Aud iting Enforcement Releases on company valuation. A reporting violati on leads to a 13% decline over a two day period, on average. The study also finds th at the companies which are in violation substantially under perform the ma rket in the years prior to th e release, indicating that the incentive to cheat is at leas t in part due economic pressures on the executives of the company. Hribar and Jenkins 43 study the effect of restatements on a firm’s cost of equity capital. The authors find that restatements l ead to a decrease in expected future earnings and increases in the firm’s cost of equity cap ital. The increases were found to be between 7% and 12%. Over the long-term the rates re main higher than before the restatement by at least 6%. Another finding of the work are that firms with greater leverage are associated with larger increases in capital. Kinney et al. 50 approach the problem from the auditor’s perspective. They study the correlation between restatements and th e amount of non-audit services performed by the auditor. This topic became especially interesting when the Sarbanes-Oxley Act of 2002 specifically forbade auditors to provide ce rtain non-audit services to their clients. The study found no significant positive correlatio n between financial information systems services and restatements. There was a significant positive correlation between unspecified non-audit service f ees and restatements. This study supports the notion that auditor independence can be compromised by non-audit consulting engagements with audit clients. Peasnell et al. 75 focus on the factors associated with low earnings quality by looking at a sample of 47 firms which have b een identified as having defective financial

PAGE 27

15 statements. A positive correlation between defective financial statements and losses or significant earnings decreases was found. Re stating firms were less likely to increase dividends, provide optimistic forecasts, and more likely to be involved in corporate restructuring. Restating firms were also less likely to employ a Big 4 auditor and often carried higher debt as a percen tage of total assets as compared to nonrestating firms. The study also found that firms which employed activ e audit committees were less likely to have defective financial statements. In this Chapter the literature on Financia l Events was reviewed. Bankruptcy, fraud and restatement research was reviewed. The next two chapters explain the methodologies used for this research project. Chapter 3 reviews Information Retrieval Methodologies and Chapter 4 review s Machine Learning methodologies.

PAGE 28

16 CHAPTER 3 INFORMATION RETRIEVAL METHODOLOGIES This chapter presents a general overview of re search in the area of IR. As this is an enormous area of research, the main focus is on contributions to the field as they relate to this dissertation. Specifically, we focus on methods of ontology creation and WordNet. The sections are as follows: Section 4.1 provi des a brief overview of general IR research. Section 4.2 explains the Vector Space Model. Section 4.3 explains the basic concepts of lexical databases with specific details a bout WordNet. Section 4.4 explains the fundamentals of ontology creation. 3.1 Overview “An Information Retrieval system does not inform (i.e., change the knowledge of) the user on the subject of his inquiry. It merely informs on the existence (or nonexistence) and whereabouts of documents relati ng to his request” 105. This field of study has exploded with the reality of massive amounts of text in an online environment – the Internet. The need to correctly choose the documents that are relevant to a keyword search has become important to industry (in the form of search engines), decision scientists, and computer scientists. There is much more to the field of IR than merely document retrieval. Some of these are as follows. Question answering systems take natural language questions as input, allowing the user to avoid learning tedious query structures. In response, the system outputs a number of short responses, designed to answer the specific question of the user. The goal of question-answering is to give a more precise response to the user. Whereas normal

PAGE 29

17 document retrieval outputs a list of docu ments, question-answering outputs small passages from documents 74. Query expansion is a research area which has grown tremendously as a result of the internet. Query expansion is most commonly used by search engines as a means to improve the accuracy of results of user queries A user types a few words as a query, and the system expands that query by adding words which will presumably give better results. There are many methods of query expansion. Automatic query expansion uses machine learning techniques to choose the best expanded query 64. Inferencing systems are a gene ralization of query expansion. They can be used at all levels of the IR process. They attemp t to “infer” the meaning of a query and add further detail. The inference is usually base d on semantic relatedness of the words in the query. Semantic relatedness can be determined by parsing a particular corpus as in the case of latent semantic analysis 52, whic h uses statistical techniques to find cooccurrences between words in a corpus, or it can be determined by using a lexical reference system, such as WordNet. Literature-based discovery uses IR tec hniques to discover hidden truths from a particular domain. The basic idea is: parse a set of documents A related to a particular subject and find a list of subjects that A re fers to. Parse a second set of documents B related to the subjects A refers to in order to find the subjects B refers to. The subjects B refers to are called C. If some subjects in C are unexplored in relation to A, then they may be worth looking at. The seminal work in this area is by Swanson 99. Using Medline (a medical documen t repository) he was able to find previously unknown connections between Raynaud’s disease and fish oil. Those connections were tested

PAGE 30

18 empirically by medical researchers. The result s showed that fish oil can actually reduce the symptoms of Raynaud’s disease. 3.2 Vector Space Model A primary goal of IR research is to rela te relevant documents to user queries. Using IR methods, one seeks to separate re levant textual documents from non-relevant ones. A powerful method in IR research is called the vector space model 18, 54, 88. This approach begins by truncating all words in th e document into word stems. Word stems are the base form of words, without suffixes. Stemming is important because a computer cannot see that “stymies” and “stymied” are basi cally the same thing. If we stem the two words, they both become “stymie.” This allo ws the computer to see the two as one word, thus adding to the words importance (via word count) in the document. Then it transforms the document into a vector by c ounting the frequency of each word in the document. Various ways of normalizing thes e vectors are available. A key observation is that these vectors are now quantitative representations of the textual parts of documents. Here is a more formal explanation of the vector space model 48. Each document in the vector space model is represented by a vector of keywords as follows: 1,2,,(,,...)jjjnjdwww where n is the number of keywords and ijw is the weight per keyword i in document j This characterization allows us to view the whole documen t collection as a matrix of weights and is called the term-by-document matr ix. The columns of this matrix are the documents and the terms are the rows. A doc ument is translated into a point in an n

PAGE 31

19 dimensional vector space. For this method to be useful, the vectors must be normalized. The dot product between normalized vectors gi ves the cosine of the angle between the two vectors. When the vectors representing tw o documents are identical, they will have a cosine of 1; when they are orthogonal, they will receive a cosine of 0. The similarity measure between documents j and k is as follows: ,, 1 22 ,, 11(,)n ijik i jk nn ijik iiww simdd ww Finding a w that most accurately depicts the importance of the keywords in the collection is very important to document cla ssification. Sparck Jones 97 made a seminal breakthrough on this problem with the TF-ID F function. The function stands for Term Frequency, Inverse Document Frequency. The basic TF-IDF func tion is as follows: n N tf wij ijlog ijtf is the frequency of term jt in document id N is the number of documents in the collection and n is th e number of documents where the term jt occurs at least once. The logic is as follows: for ijtf or term frequency, a word that occurs more often in a document is more likely to be important to the classification of that document. A measure of inverse docu ment frequency, idf, is defined by ()logijN idf n The logic is that a word that occurs in all doc uments is not helpful in the classification of the document (hence the inverse) and therefore gets a 0 value. A word that appears in only one document is likely to be helpful in classifying that document and gets a value of 1.

PAGE 32

20 Many researchers have attempted to impr ove upon the basic vector space model. Improvements take the form of making the doc ument vector more accurately depict the document itself. Part-of-speech tagging is one such improvement. A part-of-speech tagger reads a document in natu ral language and tags every wo rd with a part-of-speech, such as noun, verb, adjective and adverb. Th e tags are created using sentence structure. All part-of-speech taggers are heuristics w ith no guaranteed accuracy. However, the recent taggers have become so accurate that they only make a few mistakes on entire corpuses 11. Another improvement is word sense disambiguation (WSD). WSD is the attempt to understand the actual definition of a word, in the context of a sentence. Often words that are spelled identically have several mean ings. In the basic vector space model, the document vector would take all instances of the word “crane” and add them up. What if one sentence read, “The crane is part of the animal kingdom” and another sentence read, “The crane was the only thing that could move the 2 ton truck to safety”? Crane in the first sense is referring to a bird whereas crane in the s econd sentence is referring to a mechanical device. A word sense disambigua ted vector would have two versions of the word crane if both showed up in the corpus. This avoids some confusion that might arise were we comparing the similarity between two documents, one which was about the bird called a crane, and the other which was a bout the piece of equipment. How is WSD accomplished? One method is to look at a previously hand-tagged corpus. One such corpus is called SemCor 22. It is a co rpus of documents, which are all tagged with particular word meanings. Researchers use SemCor as a tool to learn WSD. For example, take all sets of word pairs from a corpus and compare with SemCor, looking for

PAGE 33

21 pairs that appear together of ten enough to be considered statistically significant. The phrase “crane lifts beams” may show up in the corpus. It is possibl e to determine if the noun “crane” and the verb “lifts” are found together often enough in SemCor to be considered significant. If this co-occurrence pa ir is considered significant, then “crane” will be given the particular sense number for which it was tagged in SemCor. 3.3 WordNet A lexical reference system is one which allo ws a user to type a word in and get in return that word’s relationships with othe r words. “WordNet is an online lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory. English nouns, verbs, adjectives and adverbs are organized into synonym sets, each representing one underlying le xical concept 22.” The current version of WordNet has 114,648 nouns, 11,306 verbs, 21, 436 adjectives and 4669 adverbs in its system today 22. WordNet is hand-crafted by linguists. The basic relation in WordNet is called synonymy. Sets of synonyms (called s ynsets) form its basic building blocks. For example, the word “history” is in the same block as the words past, past times, yesteryear, and yore. Due to synonymy, WordNet would be mu ch closer to a thesaurus than a dictionary. Nouns are organized into a separate lexical hier archy as are verbs, adjectives and adverbs. There are two main types of relations in WordNet, lexical relations and semantic relations. Lexical relations are between words and sema ntic relations are between concepts. A concept is another word for a syns et. A relationship between concepts can be hierarchical, as is th e case of hyponyms and hypernyms. The hyponym/hypernym is a relation on nouns. Nouns are separated fr om other parts of speech because their relationships are considered di fferent than the relationships between verbs and adjectives.

PAGE 34

22 A hyponym/hypernym relationship is an “is a” relation. WordNet can be represented as a tree. Starting at the top or root node, the concept is very general (as in the case of “entity” or “psychological feature”). As you go down the tree, you encounter more finegrained concepts For example, a robin is a subordinate of the noun bird and bird is a superordinate of robin. The subordinates ar e called hyponyms (is a kind of bird) and the superordinates are called hyper nyms (robin is a kind of). M odifiers, which are adverbs and adjectives are connected similarly as are verbs. Hyponomy is only one of many relations in WordNet. Below is a list of other WordNet relations with examples 94: Relation Example Applicable POS Has-Member Faculty – Professor Noun Member-of Copilot – Crew Noun Has-part Table – Leg Noun Part-of Course – Meal Noun Antonym Leader – Follower Increase-Decrease Noun Verb Troponym Walk – Stroll Verb Entails Snore – Sleep Verb Traditional vector space m odel retrieval techniques focus on the amount of times a word stem appears in a document without cons idering the context of the word. Consider the following two sentences, "What are you ea ting?" "What's eating you?" The words “what,” “are” and “you” would most likely be st op words. (A stop word is any word that

PAGE 35

23 is thought to have little im pact on the classification of any document. Common stop words are “the”, “and”, “but”, “what”, “are” a nd “you”. The list of stop words is usually determined by taking statistics on the document se t. If a word appears too often it is said to carry little weight. This word becomes a stop word. Stop words do not appear in the document vector.) The two sentences above w ould have identical meaning in the vector space model. The meaning of the two sent ences are however, completely different. Using concepts and contexts it is possible to create a lexical re ference system that interprets data specific to a particular area of interest. 3.4 Ontology Creation Figure 1 67 shows that there are three types of ontologies. There are top ontologies, upper domain ontologies and specific domain ontologies. Top ontologies are populated with general, ab stract concepts. Upper domain ontologies are more specialized, but still very general. Specific domai n ontologies are populated with concepts that are specific to a particular s ubject. Top ontologies for the English language are relatively complete. Upper domain ontol ogies and specific domain ontologies are still under construction 67. Figure 1 – Ontology Hierarchy WordNet is a top ontology. Many domain engineers attempt to make domain specific ontologies using the b ackbone of top ontologies. Ofte n a problem arises in that

PAGE 36

24 there is a gap between the top ontology and the specific domain ontology. In this case, an upper domain ontology is necessary. An upper domain ontology connects the top ontology to the specific domain ontology. The upper domain ontology forms the root nodes for the Domain Specific Ontologies. Domain specific ontologies ar e usually created for a spec ific purpose and these are very difficult to obtain. Navigli and Velardi explain “A domain ontology seeks to reduce or eliminate conceptual and terminological confusion among the members of a user community who need to share various kinds of electronic documents and information 68.” Domain ontology creation is a new and active research ar ea in IR. Here are some papers which highlight the current state of the research. Khan and Luo 49 construct ontologies using domain corpora and clustering algorithms. The hierarchy is created using a self-organizing tree. WordNet is used to find domain concepts. The concept hyponyms are added to the tree, under the concept. This is a novel usage of WordNet and a completely automated method of ontology construction. The method is tested on the Reuters 21578 text document corpus. Navigli and Velardi 68 give a step-by-st ep method explaining the process of obtaining ontology. Candidate terminology is extracted from a domain corpus and filtered by contrastive corpora. The contras tive corpora are used to ignore candidate terms which are in actuality part of the general domain. The word senses of domain terminology are discovered via SemCor and WordNet. New domain specific relationships are determined based on rule based machine learning techniques. These relationships are used to determine multi-wor d terms which are domain specific. Finally,

PAGE 37

25 the domain ontology is trimmed and pruned. This methodology was used to create a tourism domain ontology. Vossen 108 describes a methodology of ex tending WordNet to the technical domain. The domain corpus is parsed into head er and modifier structures. A header is a noun or verb and a modifier is an adjective or adverb respectively. A header may have more than one modifier, as in the ex ample “inkjet printer technology”. Here “technology” is the head and “inkjet” and “p rinter” are modifiers. Salient multiword terms are hierarchically orga nized creating a domain concept forest. A domain concept forest is a set of concepts related to a speci fic domain together with relationships between the concepts. The root node of each of the domain concepts is attached to a WordNet concept. In the above example “technology” would be the root node. The result is a domain concept forest attached to WordNet. Buitelaar and Sacaleanu 15 create a method of ranking synsets by domain relevance. The relevance of a synset is determined by its importance to the domain corpus. The importance is determined by the amount of times the c oncept appears in the corpus. A contrastive corpora is used to filter out concepts that are general, as in Navigli and Velardi 68. A unique cont ribution of this research is the usage of hyponyms to determine domain relevance. A hyponym is lower on the tree, therefore it is a specialization of the concep t. The authors look at how often a hyponym to a concept appears in the document as part of the relevance measure. The result is an ordered list of domain terms. The authors tested th e methodology on the medical domain by parsing medical journal abstracts.

PAGE 38

26 Buitelaar and Sacaleanu 14 extend their wo rk by adding words to domain concepts based on lexico-syntactic patterns. The domai n corpus is parsed to look at the syntax patterns of seven word combinations. Each pattern is separately considered for relevance. For all salient patterns, mutual in formation scores are given to co-occurrences within the pattern. Novel terms from the domain which are not in WordNet are added to WordNet concepts if it is determined that they are statistically significant. This methodology is tested on the medical domain. In this Chapter Information Retrieval Me thodologies were explai ned. The specific areas reviewed were the Vect or Space Model, WordNet and Ontology creation. These areas were chosen because of their relevance to the contributions of this work. In the Chapter 4 Machine Learning Methodologies are reviewed.

PAGE 39

27 CHAPTER 4 MACHINE LEARNING METHODOLOGIES Most machine learning/datamining methods 66 start with a training set of data from past cases illustrating positive and negative examples of the concept to be learned. This is called supervised learning. For example, if we are trying to lear n how to discriminate between companies likely to default on loans in the coming year from those unlikely to default, we would collect past cases of defaulting and non-defaulting companies as done in studies such as 1 62. Su ch a training set consists of l observations and a classification for each. That is, there are l pairs of the form (,)iiizy u where inX u represent the n input attributes (the independent variab les) with X called the instance space of all possible companies, {1,1}iy the classification (+1 means a positive example and -1 a negative example of the concept) for 1,..., i and the sample S is 11((,),...(,))() yyXY uu 20. Unless otherwise stat ed, a vector is denoted by a bold, lowercase letter. The superscript on th e vector is reserved for the observation number. An unbolded, subscripted, lowercase letter refers to the components of the vector. The subscript repres ents the index of the compone nt. In Chapter 5 we add a second subscript to denote the year (or period). Typical approaches, such as neural networks, logit, etc. start with a training set and try to fit the data as best as possible using the concept structure chosen (i.e., a neural ne twork, a logit function, etc. respectively). This invariably leads to over-fitting. To am eliorate this, the training set is often broken into two (or more) sets where part of the cases are used to fit a function and part to test

PAGE 40

28 it’s ability to predict on a data set not used for fitting. These approaches do help with over-fitting but are largely ad hoc. 4.1 Statistical Learning Theory Statistical learning theory 106 formally develops the goal of learning a function from examples as that of minimizing a risk functional RLz,gz,dFz over where L is a loss function, and gz, is a set of target functions parametrically defined by (the family of functions we are investigating). In this approach it is assumed that observations, z, are drawn randomly and independently according to an unknown probability distribution Fz. Since Fz is unknown, an induction principle must be invoked. One co mmon induction principle is to minimize the number of misclassifications. Minimizing the number of misclassifications is directly equivalent to minimizing the empirical risk with the loss function as a simple indicator function. Other loss functions give different risk functions. For example, the classical method for linear discriminant functions, de veloped by Fisher 34, is equivalent to minimizing the probability of misclassification. As is well known, empirical risk minimization often results in over-fitting. That is, for small sample sizes, a small empirical risk does not guarantee a small overall risk. This has been observed in many studies. For example, Eisenbeis 28 critiques studies based on such over-fitting. Statistical learning theory approaches this problem by using a structural risk minimization principle 106. For an indicator loss function, it has been shown 106 that for any with a probability at least 1 the bound

PAGE 41

29 emp struct emp bound struct4R Rh,, RR11R 2Rh,, l l holds where the structural risk structR depends on the sample size, l, the confidence level, and the capacity, h, of th e target function. The structR expression is as follows 20: structh(4ln(2/h)4)ln(/4) Rh,, l l l The capacity, h, measures the expressiveness of the target cla ss of functions. In particular, for binary classifi cation, h is the maximal number of points (k) that can be separated into two classes in all possible k2 ways using functions in the target class of functions. This measure is called the VC-dim ension. For linear discriminant functions, without additional assumptio ns, the VC-dimension is hn1 107, 20. The empirical risk is measured by a loss function on the set of examples as follows 91: empii i11 RLx,gx,ll Since we cannot directly minimize R the structural risk minimization principle instead tries to minimize boundR It is almost always the case that the smaller the VCdimension, the lower this bound. 4.2 Support Vector Machines Support Vector Machines (SVM) are growi ng in popularity rapidly in part because both theoreticians and applied sc ientists find them useful. SVMs incorporate ideas from many fields of study including applied math ematics, operations research, machine learning, and more. Based on Statistical Learning Theory, early research suggests that

PAGE 42

30 SVMs have had good success with supervised le arning. They have compared well with other learning algorithms such as Neural Networks, k-Means, a nd Decision Trees 20. Joachmis 46 used SVMs to categorize news stor ies. Pontil and Verri 78 used SVMs for object recognition (independent of aspect). Cortes and Vapnik 23 tested SVMs on hand written zip code identification, getting accuracy just shy of human error. Brown et al. 12 applied SVMs to the problem of classifying unseen genes with success. Support vector machines determine a hyperp lane in the feature space that best separates positives from negative examples. Features are mappings of original attributes (we discuss this shortly). Th e margin of an example (,)iiyu with respect to a hyperplane (,)bw is (,)iiiybwu where w is a weight vector is and b is a bias term. The margin about the hyperplane is the minimum of the margin distribution with respect to a training sample S. The VC-dimension is bounded by 2 2R h1minn, where R is the radius of a ball large enough to contain the input at tribute space. If a margin is large enough, the VC-dimension may be much smaller than n + 1. SVMs learn by maximizing the margin which, in turn, minimizes the VC-dimension and, usually, the bound of the risk functional. This distinguishes them from other popular methods such as neural networks which use heuristic methods to help fi nd parameters that best genera lize. In addition, and unlike most methods, SVM learning is theoretically guaranteed to find the best such linear concept, if the data are separable. Neural networks, decision trees, et c. do not carry this guarantee leading to a plethora of heuristic approaches to find acceptable results. For

PAGE 43

31 example, most decision tree induction use pruning algorith ms that try to create the smallest tree that produces an acceptable tr aining error in the hope s that smaller trees generalize better (this is th e so called Occam’s razor or minimum description length principle) 80. Unfortunately, there is no guarantee that the tree produced minimizes generalization error. SVMs also scale-up to ve ry large data sets and have been applied to problems involving text data, pictures, etc. The SVM is formulated as a quadratic op timization problem with linear inequality constraints. Below is the primal form ulation assuming the data is separable. min,ww st (,)1iiyb wu 1,... i ,ww is minimized in the objective function in order to maximize thus potentially minimizing the bound on the VC-dimension which was expressed above. This can be explained as follows. We replace the functiona l margin with the geometric margin. The geometric margin will equal the functional marg in if the weight vector is a unit vector. Thus we normalize the linear function 11 (,)iiyb wu ww and 1 w because the inequality will be tight at a suppo rt vector. In order to maximize we merely minimize w. This problem has a dual formulation. The dual solution is useful as w is no longer explicitly computed and the expl icit usage of the data points is collapsed into a matrix of inner products, allowing for hi gher, possibly infinite dimensional feature spaces. These

PAGE 44

32 feature spaces are implicitly calculated by a ke rnel which we explain in great detail below. The dual formulation is: 1,11 max(), 2iijijij iijWyyuu st 10ii iy 0i 1,..., i where are the dual variables. w is no longer in the formul ation and all data appears inside the dot product, which is ke y to using kernels in the SVM. A kernel is an implicit mapping of an input attribute space X onto a potentially higher dimensional feature space F. The kernel improves the computational power of the learning machine by implicitly allowing comb inations and functions of the original input variables. For example, if only price and earnings are inputs, a PE ratio would not be explicitly considered by a linear learni ng mechanism. A kernel, properly chosen, would allow many different re lationships between variab les to be simultaneously examined, presumably including price divided by earnings. The PE measure is termed a “feature” of the input variab les. There are many powerful, generic kernels 20, 38 but kernels can also be made to suit a specific ap plication area as we do later in this study. Some application areas are sensitive to periodic changes, making correct pattern recognition more likely with th e usage of time series analys is. Ruping 87 shows how to extend a number of kernels to handle time series data. Jin, Lu, and Shi 45 show that the right subset of attributes for a particular domain is important to time series classification for knowledge discovery applic ations. Their methodology trimmed the attributes to

PAGE 45

33 include only data per tinent to the domain’s time series. Preliminary research suggests that kernels which are constructed with the he lp of application specific information tend to have better results 20. 4.3 Kernel Methods A kernel is a central component of the SV M. Shawe-Taylor and Cristianini call it the information bottleneck of the SVM 95. This is because all data input into a SVM goes through the kernel function and ends up in the kernel matrix. The kernel matrix is a matrix with entries (),()ij ijK uu, where is a mapping mR X :, and ,ij X uu Often the dimension of the feature space is much larger than the attributes space, and may even be infinite (ref. the Gau ssian kernel in Section 4.3.1). Key to the value of kernel methods is th e ability to implicitly captu re this feature space via a mapping The dual formulation expressed in Se ction 4.2 can be generalized to allow the usage of kernels as follows: 1,11 max()(,) 2iijijij iijWyyKuu The kernel function is an inner product be tween feature vectors and is denoted as (,)(),()K uvuv where {,} X uv The feature vectors may not have to be explicitly calculated if the kernel function can create a mapping implicitly. In Section 4.3.1 we show how a kernel can increase th e dimension of the attribute space, thus allowing for more unique featur es, without significantly incr easing computational cost. An alternative to using a kernel is to explic itly create all features deemed necessary for classification as direct input to the SVM as attributes. However, this is both time consuming and computationally costly. Cr eating a kernel unleashes the potentially

PAGE 46

34 nonlinear power of the learning machine, allowing it to find patterns on the attributes that were previously unknown. In Section 4.3.1 we explain the properties of general kernels. In Section 4.3.2 we extend our explanati on of kernels by considering domain specific kernels. These kernels are designed with the structure of a particular domain in mind. 4.3.1 General Kernel Methods As explained above, a kernel is evalua ted within an inne r product between mappings of examples iu where examples are vectors of attributes from the instance space X There are many known kernels and th e list is growing 214687. Two specific kernels can be used to illustrate the nature and expressive power of these functions. The polynomial kernel is: ˆ (,)((,))dKKRuvuv where (,)Kuv is the normal inner product uv, d is a positive integer and R is fixed. Consider a set of examples 11((,),...(,))()SyyXYuueach with four attributes, 1234(,,,)i iiiiuuuu u and 1234(,,,)i iiiivvvv v with d=1 and R = 0. 11223344(,) Kuvuvuvuv uv and with R = 0 and d=2, 2 11223344ˆ (,)()Kuvuvuvuv uv. While (,) K uv has four features, namely 1234(,,,) uuuu ˆ (,) K uv has 10 features, namely all monomials of degree 2, or 2222 1234121314232434(,,,,2,2,2,2,2,2) uuuuuuuuuuuuuuuu Consider a d of arbitrary dimension with n attributes, the number of features is d d n 1 The computational complexity becomes unreasonable as n and d grow.

PAGE 47

35 Due to the implicit mapping in the polynomial kernel (between examples via the inner product), the monomials of degr ee d can be features of an SVM without their explicit creation. An even more powerful kernel is the Gaussian, which is defined as: 2 2(,)exp(/(2)) K uvuv where is the 2-norm 91 and is a positive parameter. An exponential function can be approx imated by polynomials with positive coefficients, making the Gaussian kernel a limit of the sum of polynomial kernels 95. The features of the Gaussian can be best i llustrated by considering the Taylor expansion of the exponential function 0! 1 ) exp(i ix i x 95. The features are all possible monomials with no restriction on the degree. This f eature space has infinitely many dimensions. Now that it is obvious that kernels are a powerful tool, we will look at their properties. To be useful in SVM work, a kernel function must have the following minimum characteristics (Crist ianini and Shawe-Taylor 20): (1) the function must be symmetric (ie (,)(,) KK uvvu) (2) the function must be positive semidefinite, and 3. the function must obey the Cauchy-Schwarz inequality #1 is easy to check. #2 is a little more complicated and it is usually determined by studying a related square-symmetric matrix, A and its eigen decomposition. Let (,) Kuv be a symmetric function on X (,) Kuv is a kernel function if and only if the matrix ,1((,))ijijAK uu is positive semi-definite (has non-negative eigenvalues) 20. #3 is satisfied as long as the function obeys the Cauchy-Schwarz inequality. The Cauchy-

PAGE 48

36 Schwarz inequality as applied to kernels is defined by Cristianini and Shawe-Taylor 20 as: 22 2(,)(),()()() Kuvuvuv (),() uu = (),()(),()uuvv(,)(,) KK uuvv A kernel function often alters the dimensionality of the data, mapping it into feature space. The inner product between all feature vectors is carried out using a kernel matrix. A matrix formed by such inner products is cal led a Gram matrix. The Gram matrix has some useful properties, for exam ple it is positive semidefinite. Since all of the entries in the Gram matrix are in the form of an inne r product, we must be concerned with their proper existence. An inner product space is a vector space endow ed with an inner product. The inner product is actually the metric used to determine the distance between two points. The inner product space is enough structure to properly define each element of the Gram matrix when considering the fini te dimensional case. However, if we want to take advantage of an infinite dimensiona l feature space (as in the Gaussian case) we need the inner product to define a complete metric (defined below). If the inner product defines a complete metric, then it is a Hilbert space 59. A complete metric is one in which every Cauchy sequence is convergent. Consider all countable sequences of real numbers. The Hilbert space is a su bset of all countable sequences ,...} ,..., {2 1 ix x x x such that 1 2 2 2 i ix x The inner product of sequences can be defined as 1,ii i x y xy This infinite space is also called 2L 59. An important characteristic of

PAGE 49

37 an Hilbert space is that it is isomorphic to n R in the finite case and 2L in the infinite case. A compelling property of kernel methods is the ability to form new kernels from existing kernels. For example, one could take a polynomial and a gaussian kernel and add them up to get the features from each. Kernels are also multiplicative. Cristianini and Shawe-Taylor 20 show that the following f unctions of kernels ar e in fact kernels: 12(,)(,)(,) KKK uvuvuv 1(,)(,) KK uvuv 12(,)(,)(,) KKK uvuvuv 3(,)((),()) KK uvuv where 1K ,2K and 3K are kernels, mR X : and 0 Until this point, we have looked at two ke rnels, the Polynomial and the Gaussian. These kernels are very powerful, but offer litt le opportunity for crafting kernels that are specific to a domain. The graph kernel is a general kernel which can be made domain specific, as long as certain rules are followe d. A powerful characteristic of the graph kernel is its intuitive nature. The graphic representation allows us to better understand how a kernel works. Before formulating th is kernel, a simple example is useful. Consider a graph (,) GAE with nodes a and edges e. See the Figure 2 below:

PAGE 50

38 1e 2e 4e s t 5e 3e1a2a3a4a5a Figure 2 – Basic Graph Kernel All ie in this graph are base kernels (for example, a polynomial kernel on a component of the attribute space) To differentiate base kern els from general kernels, the base kernels are denoted as (,)iiKuv Any path from s to t is a feature. This feature is arrived at via the product of all edges in the path between s and t In general, all paths from s to t create features. This allows the re searcher to create his own kernel, by choosing the structure of the graph. Here is a more formal explanation of the graph kernel. It is based on a directed graph G with a source vertex s of in-degree 0 and a sink vertex t of out degree 0. A directed graph is one where the flow on each edge is in a single direction. Each edge is labeled with a base kernel. It is assumed that this is a simple graph, meaning that there are no directed loops. In general, loops are allowed but that makes proving that the resulting mapping is, indeed, a kernel extr emely complicated. Takimoto and Warmuth 100 proved that a directed, acycl ical graph with base kernel s on the edges is indeed a kernel.

PAGE 51

39 Shawe-Taylor and Cristianini 95 describe the kernel as follows: Let s tP be the set of directed paths from s to t for a path 01(...)d p aaa The product of the kernels associated with the edges of p can be seen as follows: 1() 1(,)(,)iid Paa iKK uvuv The graph kernel is the aggregation of all (,)PK uv and can be seen as follows: 1() 1(,)(,)(,)ii ststd GPaa pPpP iKKK uvuvuv Here is another example for clarification. Look at the Figure 3 below. It is a slightly more complex version of the one above. The nodes are labeled for explanatory purposes and the edges are labe led with the base kernel (,),iiiiKuvuv. If 1s and 2t, then there would be a single feature, 1u If 1s and 3t there would also be a single feature 12uu but the feature would be the produ ct of the two base kernels on the path 123() p aaa Three paths converge at node 5, specifically 11235() p aaaa and 2125() p aaa and 31245() p aaaa Node 5 can be seen as a kernel which sums the products of the base kernels on each path. If Node 5 were t the output would be the sum of all paths into node 5, 123 p pp or 12514136uuuuuuuu In general, at each node a (except s), all paths from s to a are summed. The contribution of a path to the kernel is based on the product of its edges.

PAGE 52

40 11uv 33uv 44uv 55uv s t 66uv 77uv 22uv5a4a3a2a1a6a Figure 3 – Graph Kernel 4.3.2 Domain Specific Kernels A kernel should have two properties for a particular applicati on. First, it should capture a similarity measure appropriate to th e domain. The features that offer the most information content for a particular domain n eed to be represented by the kernel. Second, its evaluation should require significantly less computation than would be needed by using the explicit feature mapping 95. The firs t point is key to the contribution of this dissertation. General kernels are building bl ocks but the goal of a kernel method is to determine patterns correctly a nd tuning a kernel to a specifi c domain best does this. Much empirical research has been done wher e a dataset is tested using several kernels and results are given as to which kernel perfor ms better. This is ad hoc. It seems likely that a kernel which is tuned to a domain will better capture the features necessary to correctly classify instances in that domain. Ultimately we will combine kernels that deal with quantitative financial information and textual information. Belo w is a brief summary of some text-based kernels.

PAGE 53

41 Joachims 46 uses the polynomial and gaussian kernel for text categorization. He compares several parameters for each kernel (d for polynomial and for Gaussian). He showed that the parameter that elicited the lowest estimated VC dimension was the one with the best performance on the empiri cal tests. Thus, he has tailored general kernels to the text domain. This is an early and simplistic example of domain specificity. Cristianini et al. 21 develop a Latent Sema ntic Kernel designed to sort documents into categories by keywords, which are automatic ally derived from the text corpus. The kernel implicitly maps keywords into a “s emantic” space, which allows documents which share no keywords to be related. This is accomplished by analyzing co-occurrence patterns. A co-occurrence patt ern is where two terms which are often found in the same document are considered related. The co-occurrence information is extracted using a singular value decomposition of the term by docum ent matrix. This paper illustrates the usage of domain knowledge in the development of a kernel. Another kernel adaptable to text problems is the string subsequence kernel 20. A string is a finite set of characters from a set T In the case of a subsequence kernel, T is the alphabet. The goal of this kernel is to define the similarity between two documents by calculating the number of subsequences th ese documents have in common. The subsequences do not have to be contiguous. However, there is a penalty incorporated into the function based on the distance between words of a subsequence. Early researchers in kernel methods have given us several general forms with which to work. Recent applications of ke rnel methods to domains include protein folding, handwriting recognition, face recognition, image retrieval, and text retrieval. Finding the right kernel for a particular problem has proven to be an ad hoc, yet

PAGE 54

42 extremely important step. The real power of kernels is harnessing those general forms to create kernels that are specific to these domai ns. This work has just begun. A domain may be defined by more than one type of data, thus complicating matters. In the case of the accounting domain, both quant itative and text attributes contain information on a firm. In order to utilize the text data, we must first understand how to narrow down our potential attributes, by looki ng at text specific to the domain of accounting. The next Chapter utilizes the methodologies reviewed in this Chapter to create a Financial Kernel. A review of Chapters 2 as well as this Chapter should give the reader an understanding of the reasons for the part icular design of the Financial Kernel in Chapter 5.

PAGE 55

43 CHAPTER 5 THE FINANCIAL KERNEL Defining a domain specific kernel for fi nance entails looking to the finance and accounting literature to see what attributes and fe atures are often utilized for classification. It also requires us to cons ider the kernels availa ble and which ones would fit our work the best. As this work focuse s specifically on financial events the main publications reviewed were in the realm of management frau d and bankruptcy, as seen in Chapter 2. Without fail, most financial anal yses look to ratios of items on the financial statements. Models for earnings quality in accounting utilize ratios, such as the study by Francis, LaFond, Olsson and Schipper 35. Loebbecke, Eining and Willingham 55 use financial ratios as part of their management fraud model as well. All of the studies detailed in Section 2.2 on Bankruptcy Detection use financial ratios. McNichols and Wilson 56 used year-over-y ear changes in key account values to help determine earnings management. Fran cis, LaFond, Olsson and Schipper 35 utilized year-over-year changes extensively in th eir study on earnings quality. Beneish 10 utilized year-over-year changes to help determine management fraud. The majority of the bankruptcy prediction methods which we re reviewed in S ection 2.2 show the accuracy of their methodologies for the year of bankruptcy, the year prior to bankruptcy, and sometimes further back. As years pr ior to bankruptcy increase, the predictive accuracy of the models decreases. In general, the picture is not clear. However, a trend may be emerging. This trend can be captured by year-over-year changes in key ratios. As explained in Section 2.2 Altm an 5 notes that a limitation on his discriminant analysis

PAGE 56

44 function for bankruptcy detection was its lack of year-over-yea r changes. Year-over-year changes in ratios are capt ured by this function: 21 21 2 2 ii jj i juu uu u u where ,1... ijn are the attribute numbers and the second subscript is the year (or period). We created two kernels to handle ratios and year-over-year changes. The first kernel utilizes the polynomial kernel st ructure on a mapping of the data to produce inverses. Recall, the general polynomial kernel is ˆ (,)((,))dKKR uvuv where R is a constant and d is the degree of the polynom ial. We apply the polynomial kernel to a mapping of the input attributes () uu, where 12(,,...,)nuuu u and 12 12111 ,,...,,,,...,n nuuu uuu u Setting R to zero and 2 d, (,)(),() K uvuv gives all possible ratios of individual attributes j iu u In addition, we get the following attributes: 2 iu and j iu u 1. This can be seen in a simple example. Consider 123(,,) uuu u and 123(,,) vvv v for all X uv. 123 123111 (),,,,, uuu uuu u and 123 123111 (),,,,, vvv vvv v.

PAGE 57

45 The function result is: 2(,)((),()) K uvuv 3 2 3 2 3 1 3 1 2 3 2 3 2 2 2 2 2 1 2 1 2 1 2 12 2 2 v v u u v v u u v u v u v u v v u u 2 1 2 1 1 1 2 2 2 2 3 3 1 1 3 3 3 3 2 2 3 3 1 12 2 2 2 2 2 v v u u v u v u v u v u v u v u v u v u v u v u 2 3 2 3 2 2 2 2 2 1 2 1 3 2 3 21 1 1 2 v u v u v u v v u u which gives the following feature vector: 2 3 2 2 2 1 3 2 2 1 1 2 2 3 1 3 3 2 3 1 2 1 3 2 3 1 2 3 2 2 2 1 2 11 1 1 2 2 2 2 2 2 2 2 2 2 , , 2 u u u u u u u u u u u u u u u u u u u u u u u u u u u u We validated this kernel on simulated data. We used the Altman Z-Score with weights for the manufacturing industry (ref Ch. 2). We created attributes for each variable in the Z-score. The attributes we re TA, EBIT, RE, B.V.E., TL, WC, as defined in Section 2.2. The attribut e values were created using a normal distribution with means and variances appropriate to the domain. Wh en we created the variables we preserved the structure of the balance sheet (i.e. TA = TL + B.V.E. + RE). Each example was input into the Altman Z-Score function to obtain its score. The examples with scores were sorted by score. The top 50% of scores we re labeled with a +1 and the bottom 50% of scores were labeled -1. We onl y input the attributes and labels into the SVM. We were able to separate perfectly on the Altman Z sc ore, but had problems rediscovering weights from the actual function. We determined this is due to the fact that many extra features are created by this kernel and are highly co rrelated with each other. This correlation is due in part to the structure of the Altman Z score. Total Assets and Retained Earnings are two of the six attributes used in creating the ratios of the Altman Z-score. Both of

PAGE 58

46 these attributes are used in two diffe rent ratios. Our kernel creates (2)dn features, some of which did not add a significant amount of information to the learning algorithm (i.e. 2 iu and j iu u 1). To add a time series representation for this kernel, we would have to represent the following relationship: 2 1 2 1 2 2 1 1 2 21i j j i j i j i j iu u u u u u u u u u The left hand side of this function is the year-over-year changes as explained above. The right side is a representation that can be constructed by our kernel by dropping the constant. The attri bute vector would double in size as the second year would be concatenat ed onto the end. In order to get year-overyear changes in this format 2 1 2 1 i j j iu u u u we need 3 d. The number of features in the yearover-year case would be at least 3(2(2))n. Even a modest number of attributes causes a huge explosion in features. The 4n case would generate 4,096 features. The second kernel we created was built as a response to the problems we had with the first, namely dimensionality explosi on and unnecessary featur es. We design this kernel with the goal of getting all the importa nt features, including al l possible intra-year ratios and year-over-year ratios. However, we want to avoid the problem of unwanted features. For this we chose the graph kernel. As discussed already, th e graph kernel is extremely flexible, which makes it a natural choice when trying to construct specific features. We exploit the research of Takimoto and Warmuth 100 to build this kernel. We

PAGE 59

47 call this kernel the Financial Kernel, and denote it as (,)FKuv (,)FKuv is a directed graph (,)GAE with base kernels on all edges e and (,)Kuv on a. The Financial Kernel has as input n attributes per year for 2 year s. The attributes vector is 111122(,...,,,...,)nnuuuu u where the first index is the attribute number and the second index is the time period. See Figures 4 and 5 for an illustration of financial domain kernel. Figure 4 illustrates one of 1 n graphs that make up the Financial Kernel. Each of the 1 n graphs has a source node is and a sink node it. The graphs decrease in size with n. The reason is that each graph car ries information for attributes i through n. Each path from source to sink is a feature. The number of features are equal to the number of paths. All 1 n graphs from Figure 4 are br ought together by the graph in Figure 5. The paths from s to t make up all of the features in (,)FKuv The kernels on e are base kernels. As defined in Chapter 4, a base kernel is a kernel function on a vector component. We can have as many different kernels as there are edges. For the creation of a financial kernel, we limited the base kernels to two forms, one is the standard inner product kernel of i i i iv u v u K, ) ( the second is i i i iv u v u K1 ) ( ~

PAGE 60

48 Figure 4 – The Financial Kernel 1 1 1 1 1 1 1 1 11 11 1 1 1 1 1 1 1 1 1 1 1 is it 1 ) 1 ( 1 ) 1 (1 i iv u 1 ) 2 ( 1 ) 2 (1 i iv u 1 11n nv u 2 21i iv u 2 ) 2 ( 2 ) 2 ( i iv u 2 ) 1 ( 2 ) 1 ( i iv u 1 1 i iv u 2 2 n nv u

PAGE 61

Figure 5 – The Financial Kernel 2 According to Takimoto and Warmuth 100, in order to prove that (,)FKuv is a kernel, we need only have a directed gra ph without cycles and show that each edge e is a valid kernel. (For details of their proof, see pg. 33 of Takimoto and Warmuth 100.) Examination of Figures 4 and 5 clearly show that the graph is directed and free of cycles. We need to show that both ) (i iv u K and (,)iiKuv are kernels. ) (i iv u K is simply the standard inner product kernel. ) ( ~ i iv u K can be shown to be a kernel as follows: (1) n i u u fi i... 1 ) (1 (2) n i v v fi i... 1 ) (1 (3) i i i i i iv u v u v f u f1 ) ( ) (1 1 (4) By Cristianini and Shaw e-Taylor [2000] (pg. 42) 20 ) ( ) ( ) ( ~ i i i iv f u f v u K The features of the Financial Kernel are: 212 1 1212(),,, ,1...,jij i jijiuuu u ijnij uuuu u Here is a small example. In this example 11211222(,,,) uuuu u and 11211222(,,,) vvvv v

PAGE 62

50 1111222211112222 2121121221211212(,)Fuvuvuvuv K uvuvuvuv uv which gives the following features: 11221122 21122112,,uuuu uuuu In general, for year 1 we get all ratios in the form of j iu u In year two we get all ratios in the form of i ju u which is the inverse of year 1. We structure the ratios in this form in order to get yearover-year changes of the form 2 1 2 1 i j j iu u u u The feature space we have constructed so far with intra-year ratios has the structure 1 1 j iu u and 2 2 i ju u It is evident that with this kernel we get the feature or its inverse. In other words, if the true feature is 1 1 i ju u this mapping only gives the inverse. By constructing the features in this manner, we reduce the dime nsionality necessary to get year-over-year changes, but we lose a potentially important se t of features in the pr ocess. For the yearover-year changes all we need to do is ge t the product of the intra-year ratios 1 1 j iu u and 2 2 i ju u The computational complexity of the Financial Kernel is (1) 3 2 nn for n attributes and 2 periods. This is easy to see as each pair of attributes ij are represented three times, 212 1 1212,,jij i jijiuuu u uuuu and the number of attribute pairs are (1) 2 nn

PAGE 63

51 We validate the financial kernel on simulated data to test the kernel’s ability on inputs of a known function. We take the Altman Z-Score and modify it slightly, to add a time series component. The function we create is: 1 1 1 1) / ( 05 1 ) / ( 72 6 ) / ( 26 3 ) / ) (( 56 6TL BE RE EBIT TA RE TA CL CA 2 2 2 2) / ( 05 1 ) / ( 72 6 ) / ( 26 3 ) / ) (( 56 6TL BE RE EBIT TA RE TA CL CA 2 1 2 1 2 1) / ( ) / ( 3 ) / ( ) / ( 3 )) /( ( ) / ) (( 2 EBIT RE RE EBIT RE TA TA RE CL CA TA TA CL CA score BE TL TL BE 2 1) / ( ) / ( The first and second rows of this function are year 1 and year 2 individual Altman Z Scores. The third row is year-over-year ch anges in the ratios of the Altman Z-Score. The weights on the year-over-year change s were chosen arbitrarily. Our dataset contains 2,000 randomly genera ted examples labeled with the modified Altman Z-score function. We divide the ex amples up by sorting the data on the score. The threshold value for our modified Altman function is chosen as a midpoint between the score of sorted item 1,000 and 1,001. Thus a ll of the top-half are labeled as +1 and all of the bottom half as -1. We run experi ments using the financial kernel, a polynomial kernel of degree 2, a Gaussian kernel, and a linear kernel. The results are as follows: Table 1 – Financial Kernel Validation SVTest on Train10 fold cross validation Linear87785%84% Polynomial (deg 2)199875%55% Gaussian105686%86% Financial Kernel70792%91% The results show that the Financial Kernel achieves superior results when using 10fold cross validation. The first column is the number of support vectors. A bound on

PAGE 64

52 generalization error is # SV The generalization error of the Financial Kernel is the lowest of the listed ke rnels. The result is not quite as expected though. One would expect the Financial Kernel to achieve perfect separation. The reason for the error is an assumption we made when developing the ke rnel. The assumption was that we could represent both of th e following ratios i j j iu u u u, by only one of i ju u and j iu u. In order to get perfect separation, we hypothesize that we need to have both i ju u and j iu u as features. This has been easily achieved by adding a mirror image of Figure 4, with the components being inverses of the com ponents of Figure 4. Figure 6 shows the updated Financial Kernel. This Chapter detailed the development of the Financial Kernel, one of the two main methodological contributions of this research In Chapter 6 the development of the Accounting Ontology is explained.

PAGE 65

53 Figure 6 – Updated Financial Kernel

PAGE 66

54 CHAPTER 6 THE ACCOUNTING ONTOLOGY AND CONV ERSION OF DOCUMENTS TO TEXT VECTORS We describe a methodology for creating an accounting ontology in Section 6.1. Section 6.2 describes how the ontology is us ed in conjunction w ith the vector space model to turn accounting documents into text vectors. 6.1 The Accounting Ontology The accounting ontology is built using an accounting corpus to represent the accounting domain and general corpora to re present the general domain. The accounting corpus is the US Master GAA P Guide 16. We chose this because it explai ns generally accepted accounting principles in a fairly non-technical manner. It uses all the terminology, but in more regular language th an a legal publication. We get our general corpora from the Text Resear ch Collection, which is syndicated by TREC 103. This collection includes material from the Associ ate Press, The Wall Street Journal, the Congressional Record, and various newspapers The Text Research Collection has been used in many natural language processing appl ications and is ofte n used to test IR methodologies. A domain specific ontology is created by a se ries of major steps, each with its own series of minor steps. Figure 7 shows how th e ontology is created. There are two classes of corpora, the domain corpora and the genera l corpora. Both are part-of-speech tagged and fed into the function that determine wh ich concepts are germane to the accounting domain, as described in Step 1 below. A se t of concepts and othe r domain specific terms

PAGE 67

55 called Novel Concepts are put through a proce ss that uses the syntactic structure of the accounting corpora, as described in Step 2 below. The result of this step is a WordNet enriched with novel terms from the accoun ting domain. The final step in ontology creation is to add new multi-word concepts to WordNet based on an algorithm that uses the syntactic structure of domain concepts, as described in St ep 3 below. The details of each step are explained in the remainder of the section. 6.1.1 Step 1: Determine Concepts a nd Novel Terms that are specific to the accounting domain We start with a part-of-speech (POS) tagger used to tag the natu ral language text. This puts additional structure on the indivi dual words. The POS tagger used is a derivative of the Brill tagger, called MontyT agger 110. The tagger is run on both the accounting corpus and the general corpora. Th e POS tagged data is culled down to the following form: Word1#POS#WordCount Word2#POS#WordCount WordN#POS#WordCount Word1#POS#WordCount Word2#POS#WordCount

PAGE 68

56 Accounting Domain General Domain POS Tagger POS Tagger Modified TF-IDF LexicoSyntactic Patterns k-Nearest Neighbor Domain Concepts and Novel Terms (Step 1) Domain Concepts enriched with Novel Terms (Step 2) Header Modifier Algorithm Add Multiword Domain Concepts (Step 3) Figure 7 – Accounting Ontology Creation Process

PAGE 69

57 … WordN#POS#WordCount
Word1#POS#WordCount Word2#POS#WordCount WordN#POS#WordCount Word1#POS#WordCount Word2#POS#WordCount WordN#POS#WordCount where Word#POS#WordCount is as follows: Word – Stemmed Word POS – Part of Speech of Word WordCount – Number of times a word appears in a document. The word counts are run through a function in order to detect words that have the highest amount of information for that particular domain. For example, when considering the accounting domain versus a general domain, the word “defeasance” will have a higher score for the accounting domain because it is specific to accounting, while the word “balance” will have a lower score as it can be found equally in the accounting

PAGE 70

58 and the general domain. The function used is a modification of the basic TF-IDF function which includes WordNet Concepts. Recall the basic TF-IDF function: n N tf wij ijlog ) (, where ijw is the weight of term jt in document id, ijtf is the frequency of term jt in document id, N is the number of documents in the collection and n is the number of documents where the term jt occurs at least once 44 Th e inverse document frequency n N idfijlog ) (. We modify the TF-IDF as follows: (1) t d tdf N tf d t rlv log ) log( ) | (, (2) c td t rlv d c rlv) | ( ) | ( (3) c tc T d t rlv d c rlv ) | ( ) | ( (4) i c t i cc T d t rlv d c rlvi) | ( ) | ( max Function (1) mimics the basic TF-IDF function, only the d stands for domain instead of document in our research. rlv is the domain relevance of a term t on domain d N is the number of domains. In function (2), we introduce c for concept, where } ,..., {2 1nt t t c. This introduces the notion of a syns et. By considering the relevance of terms and their synonyms, we get a clearer understanding of the domain. Function (2) sums up the relevance rlv for all terms t in the synset c. This is a concept relevance score. Function (3) sharpens this, considering hyponyms. The c is the concept,

PAGE 71

59 unioned with all of its direct hyponym sets. Recall, the hyponym of a concept i is a concept j which is a specific instance of concept i. For ex ample, a bank account is an account. Function (3) sums up the relevance rlv for all terms t in c and all of c’s direct hyponyms. Looking at the dir ect hyponyms gives us one more measure of a concept’s relevance. Function 3 a dds an additional term c T where T is the total number of terms it within a concept c which are found in the domain corpus. The c is the cardinality of the concept. This set of func tions was developed by Sacaleanu and Buitelaar 15. We add a measure of word sense disamb iguation in Function 4 by comparing the domain frequency of various senses of a term t. In other words, consider concept 1c with terms ) , (3 2 1t t t and concept 2c with terms ) (5 4 3t t t. Notice that 3t is in both 1c and 2c. We determine which concept 3t actually belongs to by comparing the Function 3 scores of the two concepts We choose the concept (1c or 2c) which achieves the maximum value in Function 3. Here is an illustrative example. Th e noun “stocks” in WordNet has 17 different senses (definitions). Listed below are 4 of the 17 senses. 1. stock -(the capital raised by a corpor ation through the issue of shares entitling holders to an ownership intere st (equity); "he owns a controlling share of the company's stock") => capital, working capital -(a ssets available for use in the production of further assets) 2. broth, stock-(liquid in which meat and vegetables are simmered; used as a basis for e.g. soups or sauces; "she made gravy with a base of beef stock")

PAGE 72

60 => soup-(liquid food especially of meat or fish or vegetable stock often containing pieces of solid food) 3. stock, inventory-(the merchandise that a shop has on hand; "they carried a vast inventory of hardware") => merchandise, wares, product-(commodities offered for sale; "good business depends on having good mercha ndise"; "that store offers a variety of products") 4. livestock, stock, farm animal-(not used technically; any animals kept for use or profit) => placental, placental mammal, eutherian, eutherian mammal-(mammals having a placenta; all mammals ex cept monotremes and marsupials) Senses 1 and 3 are much more likely to come up in an accounting context than senses 2 and 4. In order to test which sens e is the most likely sense in the context of a document or corpus, we compare relevance scores, which include for Sense 1 “stock” and its hyponyms “capital” and “working capital”, with Sense 3 “stock”, “inventory”, and its hyponyms “merchandise”, “wears”, and “product” The Sense with the highest relevance becomes the candidate to be a domain specific concept. All word sense disambiguated concepts ar e sorted based on score, and the highest scoring concepts become domain specific concepts. Novel terms are those terms that have high scores but do not fit into a WordNe t category. These terms are very important as they give us an opportunity to en rich WordNet with domain knowledge. WordNet can be viewed as a hierarchical tree where the nodes are concepts and the edges are relationships. Figure 8 shows a simplified WordNet tree after Step 1. In this tree accounting domain concepts are filled in w ith the color gray. We also show that

PAGE 73

61 there is a listing of novel term s which are highly important to the accounting domain but cannot be matched to current WordNet concepts These terms are the subject of Step 2. Domain Concept Domain Concept Domain Concept Novel Term List Term 1 Term 2 … Term n Figure 8 – WordNet Noun hierar chy with Domain Concepts 6.1.2 Step 2: Merge Novel Terms with Concepts In this step, we take the novel terms that were not ma tched to concepts and we attempt to fit them into a domain concep t. We use the methodology of Buitelaar and Sacaleanu 14. This process is done using lexi co-syntatic patterns. Consider the natural text, before preprocessing. In such a text there are certain syntactic patterns that arise, such as [determiner, adjective, noun, verb, noun] A sentence with this structure would be “The large crane eats breakfast.” The “the” is a determiner, large is an adjective (ADJ), crane is a noun (NN), eats is a verb (V) and breakfast is a noun (NN) We consider the syntactic patterns that arise in 7-grams, that is, contiguous 7 word structures. We look for patterns with three words to the left and three words to the right of a central word. This central word will always be either a domain concept (which includes all constituent terms) or a novel term. The basic idea is as follows: l ook for patterns where novel terms

PAGE 74

62 and concepts appear together, often interchange ably. A visual representation of a 7-gram is below: This 7-gram has words in the following se quence: two words on the left that are not important to us (signified by null), an adjective, the term or concept, another unimportant word, and then a verb and a noun. The parts-of-speech we are concerned with are nouns, verbs and adjec tives. All other parts-of-speec h are considered “null”. In order to determine patterns which are populated with words that are related, we use a mutual information score based on co-occurrence. The sore is used to determine the semantic similarity of two-word pairs based on how often pair s of words are found together relative to chance. The mutual information score MI is the following function: N w f t f w t f w c MIc t i c t ii i) ( ) ( ) ( log ) (2 where c is a concept, w is a word in the pattern, and N is the total number of words in the pattern. MI is an approximation of the probabilis tic mutual information function: 2(,) log ()()Pxy PxPx. The details of the derivation of ) (w C MI can be found in 14. In order to determine if a novel term belongs inside a particular concept we have to first decide whether the pattern is reliable. We assume a pattern is reliable if all the terms of a concept are assigned back to the concep t, using an unsupervise d clustering algorithm called k-nearest neighbor. Be low is the data structure fo r the example pattern above: [null, null, ADJ, Term/Concept null, V, NN]

PAGE 75

63 [c,MI, ADJ, V, NN]. As “null” attributes are unimportant, we simply leave them out. For reliability testing we expect the concept c in its representation above to be clustered together will all term instances, represented as [it ,MI,ADJ,V,NN]. If this is not the case, the pattern is considered unreliable. For all reliable patterns, we use the k-nearest neighbor to cluster the concepts as seen above together with the novel terms (NT) in the following representation: [NT, MI, ADJ, V, NN]. If a NT is clustered with a Concept, then we add the NT to the concept, thus enriching WordNet. Figure 9 shows the WordNet tree afte r Step 2. This Figur e updates Figure 7. The domain concepts which were found in Step 1 are shaded gray. Figure 8 illustrates that after Step 3 some of the domain con cepts include novel terms, thus enriching WordNet. } { NovelTerms Concept Domain Concept } { NovelTerms Concept Figure 9 – WordNet Noun hierar chy with Domain Concepts enriched with Novel Terms {} ConceptNovelTerms

PAGE 76

64 6.1.3 Step 3: Add multi-word domain concepts to WordNet At this point, we have domain concepts enriched with novel te rms. We would like to extend WordNet, by adding new nodes. We do this using a slightly altered form of the method described by Vossen 108. Vossen utiliz ed the header-modifier relationship to determine multi-word concepts. For our purpos es, a header is a noun and a modifier is one or more adjectives describing it. For example, “bank account” is a two-word structure with account as the header and ba nk as the modifier. Vossen considers all header-modifier structures, limiting the final se t to the ones above a statistical threshold for a particular domain. We already have our domain concepts from Steps 1 and 2, so we consider only header-modifiers where the header is one of the domain concepts. If an instance of the header-modifier struct ure is considered statistically significant, then it is added as a node below the header in the tree. This means it becomes a hyponym of the domain concept. The potential for mo re than one layer exists. Consider the following phrases, “federal tax expense” and stat e tax expense.” Both of these multiword phrases are actually line items on an income st atement. “Expense” is a term specific to the accounting domain. A “tax expense” is a term that belongs below “tax” in a hierarchy. There can be an additional set of nodes below “tax expense” called “federal tax expense” and “state tax expense.” Th ere can be any number of modifiers for any noun, although it is likely that the number of modifiers will be between one and three. The WordNet tree takes on new nodes undernea th domain concepts. The new nodes are the header modifiers deemed significant to the domain. Figure 10 shows a simplified representation of the tree after Step 3. Th e figure shows (as Figures 8 and 9) the domain concepts as shaded light gray Additional nodes are added below some domain concepts.

PAGE 77

65 This is represented by nodes which are sh aded dark gray. These new nodes are the domain specific multi-word concepts, added in this step. Figure 10 – WordNet Noun Hierarchy with Do main Concepts, Novel Terms and MultiWord Concepts 6.2 Converting Text to a Vect or via the Accounting Ontology Above we developed an accounting ont ology methodology. Now we use this ontology to aid in detection of financial events by using it as domain filter to get rid of unwanted noise. Recall that the output of the Accounting On tology is a set of concepts specific to the accounting domain as well as relationships be tween those concepts. The process of getting a quantitativ e form of a text vector is as follows: We input the company reports in natural language and use a Part-of-Speech tagger 110 as a

PAGE 78

66 preprocessor. The preprocessed document is then parsed down to word counts labeled with a Part-of-speech (e.g. Word#POS#WordC ount). Each word is compared to all domain concepts. If a word fits into one of the domain conc epts, its word count is added to the vector. The power of concepts rests in the fact that all words inside a concept will count the same. For example, the word “l iability” has the following words as synonyms, “indebtedness”, and “financial ob ligation”. All three of these words are part of the same concept. If one document has the word “lia bility”, a word count is placed in the index reserved for the “liability” concept. If anot her document has the word “indebtedness” or “financial obligation”, a word count is placed in the index reserved for the “liability” concept. Below is a class of concepts: 12{,,...,} 1,...,|| 1,...,n ij jccc wc ic jn where jc are concepts, iw are words and | |jc is the size of the concept set. The filtering process leaves us with only concepts jc that are specifically related to the accounting domain. We take this reduction a step further by considering the relationships between the concepts in the Accounting Ontology. We do this by utilizing th e tree structure of WordNet. We need a measure to determine the similarity between nodes (or concepts). There is a vast litera ture on similarity measures, so we choose an off-the-shelf measure that has proven to be among the best. Based on the work of Budan itsky and Hirst 13, we choose the Jiang and Conrath measure, which has been shown to be more accurate on the Miller-Charles 63 set than competing similarity measures. We create a similarity matrix,

PAGE 79

67 comparing each of the concepts, where ijs is the similarity between concepts ic and jc for } ,..., 2 1 { ,n j i. Here is a view of this matrix: nn n n n ns s s s s s s s s 2 1 2 22 21 1 12 11 We input the similarity matrix into an agglomerative clustering algorithm 47. This algorithm clusters the most similar items a nd shrinks the matrix. This algorithm is iterative, in each run concepts which are less si milar are added to existing clusters, so we choose a parameter k where k is the minimum level of similarity with which two concepts can be clustered. The clustered concepts c are called super-concepts sc. sc c where is the size. In turn, the total number of super-concepts sc are less than or equal to the total number of concepts c. There are two goals to creating super-concepts: (1) The super-concepts are designed to clus ter concepts that are similar, therefore financial documents which share accounting super-c oncepts are more likely to be similar. (2) The super-concepts allow us to shrink the size of an undoubtedly large vector. This can help us avoid overfitting on the em pirical data, which is possible due the small datasets available for fraud and bankruptcy Below is a class of super-concepts: 12{,,...,} 1,..., 1,..., s jk kscscsc csc jm ks where km is the number of such concepts in super-concept k.

PAGE 80

68 Chapter 6 explained the methods used to develop the Accounting Ontology. The procedure for converting text to a vector of numbers was also explained. In the next Chapter the method of combining the text da ta with the quantitativ e data is detailed.

PAGE 81

69 CHAPTER 7 COMBINING QUANTITATIVE AND TEXT DATA In this chapter we combine quantitative a nd textual financial data for subsequent analyses. We turn the text into a numeric vector as discussed in Chapter 6, here we concatenate the quantitative form of text to the vector of quantitative financial data. Since we will be applying a kernel to this concatenated vector, we need to expand the financial kernel developed in Chapter 5. We concatenate the text and quantitative at tribute vectors as a single, partitioned vector ) ,..., | ,..., , ,..., (2 2 1 2 2 22 12 1 21 11 m n n n nu u u u u u u u uu. The Financial Kernel is applied to 2 22 12 1 11, , ,...,n nu u u u u and the text kernel is applied to m nu u,...,1 2 where these m – n values are the quantitative representa tion of text. This is a two step process. (1) We create a graph kernel (,)TKuv for the text. (2) We add the text graph to the Financial Kernel graph. (1) Text Graph: The text kern el is a linear kernel (,),TKuvuv. We show (,)TKuv in graph form (Figure 11): Figure 11 – Text Kernel

PAGE 82

70 This is a directed graph w ith no cycles and each edge e is a base kernel. This graph is a kernel by Takimoto a nd Warmuth [2004] 100 (pg. 33). (2) Add (,)TKuv to (,)FKuv to create a combined kernel (,)CKuv. See Figure 12 below: 2FG 1FGkFG 1s 1t 2t s t 2s Ts TG Tt 1 ns 1 nt Figure 12 – Combined Kernel The text graph TG is added to the Financial Kernel. The addition of TG does not alter the fundamental structure of the Financial Kernel graph. The graph is still directed and still contains no cycles. Thus (,)CKuv is a kernel. A simple example illustrates the Combin ed Kernel. There is an input of 2 quantitative attribut es for both years, 22 12 21 11, ,u u u u and 4 text attributes,

PAGE 83

71 8 7 6 5, ,u u u u.The input vectors are 112112225678(,,,|,,,)uuuuuuuu u and 112112225678(,,,|,,,)vvvvvvvv v. 1111222211112222 55667788 2121121221211212(,)Cuvuvuvuv Kuvuvuvuv uvuvuvuvuv with the following features: 11221122 5678 21122112,,,,,,uuuu uuuu uuuu Other kernels could be used in place of the lin ear kernel, giving addi tional features on the text. For this study it is not necessary due to the extensive pr eprocessing steps used during the creation of the text vector. This Chapter explained the method used to combine text and quantitative data. Chapter 7 is the final chapter in the met hodology creation. The following three Chapters delve into the empirical research, testing, re sults and a conclusion. Specifically, Chapter 8 details the research questions the three datasets used fo r testing (management fraud, bankruptcy, and restatements), and the ontolog ies created. Chapter 9 gives the results from the tests on the datasets. Chapter 10 gives a summary, conclu sion and explanation of future research.

PAGE 84

72 CHAPTER 8 RESEARCH QUESTIONS, METHODOLOGY AND DATA This chapter explains the research hy potheses, the data gathering methodology, accounting ontology creation and da ta preprocessing. In Sec tion 8.1 the Hypotheses and test mechanisms are articulated. Section 8. 2 outlines the Research Model. Section 8.3 explains the methods used for gathering data for the Fraud, Bankruptcy and Restatement datasets. Section 8.4 details the ontologies created and Section 8.5 explains data preprocessing. 8.1 Hypotheses The main contributions of this research ar e threefold. (1) We have developed a financial kernel that operates on quantitative fi nancial attributes. (2) We have developed an accounting ontology to aid in using textual data in learni ng tasks. (3) We have combined these two kernels to simultaneously analyze quantitative and text information. These methods will be tested to for their e ffectiveness in early detection of financial events. Our first testable hypothesis is as follows. Hypothesis 1: A support vector machine using the Combined Kernel, which includes the Financial Kernel for quantitative data and the Text Kernel for text data detects financial events with greater accuracy than quantitative methods alone, including the Financial Kernel. A series of tests are run on the financial events data, using the Combination kernel. All available data, both quantitat ive and text is used. We use 10-fold cross validation as

PAGE 85

73 a method for estimating generalization error. We compare the classification accuracy of our method with the other methods as expl ained in Chapter 2, including linear discriminant functions, and logit functions. The concepts in WordNet include semantic relationships between individual words. Developing an ontology specific to the domain of accounting allows us to utilize these relationships when creating the text vector. The basic vector space model does not take these relationships into account. The expecta tion is that the ontol ogy driven text vector will provide a better representation of accountin g-related documents than the basic vector space model. Hypothesis 2: A Support Vector Machine using da ta from a text vector filtered through the accounting ontology wi ll detect financial even ts with greater accuracy than a Support Vector Machine us ing only the vector space model. Two tests are run on the fina ncial events data, using the combination kernel. One test uses a vector created by filtering the text through the accounting ontology. The other is run using a vector of word counts. The re sults of the tests’ 10-fold cross validation are reported and compared. Comparing the classification accuracy of the text and quantitative data allows us to effectively compare the “informa tion content” in the numbers against that of the text. Hypothesis 3: Text filtered through an account ing ontology will detect financial events at least as accurately as co mpared to pure quantitative methods.

PAGE 86

74 Two tests are run on the fina ncial events data, one usi ng only the quantitative data which is fed into the Financial Kernel and the SVM. The other uses only the text data in the form of the concept vector which is preprocessed using the accounting ontology. The concept vector is fed into the Text Kernel a nd the SVM. The results of the tests’ 10-fold cross validation are compared. 8.2 Research Model In this section, the Resear ch Model is explained. Fi gure 13 shows the process we use to study the efficacy of our approach. The em pirical analysis is ca rried out to test our methodology. Starting on the left of the figure, we gather our dataset, which consists of companies that were shown to be fraudulen t and/or bankrupt. We match the fraud and bankrupt companies with nonfraud and nonbankr upt companies based on year, sector, and total assets. Once we have chosen the companies in our dataset, we gather quantitative data from financia l statements and text data from the 10Ks. The financial data is converted into a vector of attribute values. The text data is filtered through the accounting ontology and turned into a numerical vector using the counts of the concepts in the ontology. The text and financial vectors are concatenated and run through the combination kernel. An SVM using the comb ination kernel is used to determine a classifier to distinguish the companies as fraud/nonfraud and bankrupt/nonbankrupt. The financial vector is similarly processed using the financial kernel to get classification results for the quantitative data alone. We compare the quantitative results against the results for the text-only case by feeding the text vector into the text kernel SVM.

PAGE 87

75 Figure 13 – The Discovery Process FraudBankruptFinancial Data SEC AAERsCompustat ResearchFinancial Statement AAA MonographFigures NonfraudNonbankruptText Data Match to fraudMatch to bankruptSEC filings companies on yearcompanies on yearPress releases industry andindustry, and total assetsExogenous press total assetsChat room data Data Sets Financial Kernel Text Kernel Acconting Ontology Vector Space Model Financial Vector Text Vector Combination Kernel Decision Fraudulent Company Nonfraudulent Company Bankrupt Company Nonbankrupt Company SVM

PAGE 88

76 8.3 Datasets The data gathering methods are described in this section for the Fraud, Bankruptcy and Restatement datasets. Text and quantitat ive data are gathered for all companies in the datasets. 8.3.1 Fraud Data Gathering fraud data is a task which requi res considerable time and effort. The main data sources are the SEC Accounting and Auditing Enforcement Releases 93 as well as the Accounting and Auditing Association Monograph by Palmrose 73. The set was limited to fraud which occurred no earlier than 1993. The extr acted financial data consists of financial statement figures for two years. The text data set consists of the text portion of annual reports (10Ks). As the fra ud dataset required both text and quantitative attributes, any company which was missing either the text or quantit ative attributes was deleted from the dataset. The quantitative da taset is shown in Figure 16 of Appendix B. The attribute definitions are as follows: Ticker – Company ticker for stock market Label – fraudulent (-1) nonfraudulent (1) Ind – Industry Number Year – 1st year of data collection Salesyr[1,2] – Sales ARyr[1,2] – Accounts Receivable INVyr[1,2] – Inventory TAyr[1,2] – Total Assets OAyr[1,2] – Other Assets CEyr[1,2] – Capital Expenditures The attributes were chosen based on their re ported occurrence in cases of fraud. A secondary reason for choosing these particular attributes was the likelihood of getting reported data. This is in c ontrast to other hi ghly reported fraud at tributes, such as Advertising Expense, Research and Developmen t Expense and Allowance for Bad Debts.

PAGE 89

77 The dimension of the feature space for the Financial Kernel in this experiment is 90. The features are listed in Figure 14. A “YOY” in front of a ratio means the yearover-year change for that ratio. He re is a listing of the features: Salesyr1/ARyr1 ARyr1/Salesyr1 Salesyr2/ARyr2 ARyr2/Salesyr2 YOYSalesyr1/ARyr1 YOYARyr1/Salesyr1 Salesyr1/INVyr1 INVyr1/Salesyr1 Salesyr2/INVyr2 INVyr2/Salesyr2 YOYSalesyr1/INVyr1 YOYINVyr1/Salesyr1 Salesyr1/TAyr1 TAyr1/Salesyr1 Salesyr2/TAyr2 TAyr2/Salesyr2 YOYSalesyr1/TAyr1 YOYTAyr1/Salesyr1 Salesyr1/OAyr1 OAyr1/Salesyr1 Salesyr2/OAyr2 OAyr2/Salesyr2 YOYSalesyr1/OAyr1 YOYOAyr1/Salesyr1 Salesyr1/CEyr1 CEyr1/Salesyr1 Salesyr2/CEyr2 CEyr2/Salesyr2 YOYSalesyr1/CEyr1 YOYCEyr1/Salesyr1 ARyr1/INVyr1 INVyr1/ARyr1 ARyr2/INVyr2 INVyr2/ARyr2 YOYARyr1/INVyr1 YOYINVyr1/ARyr1 ARyr1/TAyr1 TAyr1/ARyr1 ARyr2/TAyr2 TAyr2/ARyr2 YOYARyr1/TAyr1 YOYTAyr1/ARyr1 ARyr1/OAyr1 OAyr1/ARyr1 ARyr2/OAyr2 OAyr2/ARyr2 YOYARyr1/OAyr1 YOYOAyr1/ARyr1 ARyr1/CEyr1 CEyr1/ARyr1 ARyr2/CEyr2 CEyr2/ARyr2 YOYARyr1/CEyr1 YOYCEyr1/ARyr1 INVyr1/TAyr1 TAyr1/INVyr1 INVyr2/TAyr2 TAyr2/INVyr2 YOYINVyr1/TAyr1 YOYTAyr1/INVyr1 INVyr1/OAyr1 OAyr1/INVyr1 INVyr2/OAyr2 OAyr2/INVyr2 YOYINVyr1/OAyr1 YOYOAyr1/INVyr1 INVyr1/CEyr1 CEyr1/INVyr1 INVyr2/CEyr2 CEyr2/INVyr2 YOYINVyr1/CEyr1 YOYCEyr1/INVyr1 TAyr1/OAyr1 OAyr1/TAyr1 TAyr2/OAyr2 OAyr2/TAyr2 YOYTAyr1/OAyr1 YOYOAyr1/TAyr1 TAyr1/CEyr1 CEyr1/TAyr1 TAyr2/CEyr2 CEyr2/TAyr2 YOYTAyr1/CEyr1 YOYCEyr1/TAyr1 OAyr1/CEyr1 CEyr1/OAyr1 OAyr2/CEyr2 CEyr2/OAyr2 YOYOAyr1/CEyr1 YOYCEyr1/OAyr1 Figure 14 – Fraud Features 8.3.2 Bankruptcy Data The bankrupt companies were chosen using the Compustat Research database 19. All chosen companies are from the Manuf acturing sector (Industry codes 2000 – 3999). The companies chosen were delisted betw een 1993 and 2002. A company is delisted when it does not meet the minimal requirement s of financial stabil ity according to the

PAGE 90

78 market (NYSE, NASDAQ, AMEX). The anal ysis is limited to post-1992 company years due to the fact that the model requires the text of the 10Ks to be in electronic form. Electronic 10Ks were not available until 1993. Figure 17 in Appendix B shows the entire quantitative dataset. The attribute definitions are as follows: Label – bankrupt (-1) nonbankrupt (1) Ticker – company ticker for stock market Ind – Industry Number Year – 1st year of data collection TAyr[1,2] – Total Assets REyr[1,2] – Retained Earnings WCyr[1,2] – Working Capital EBITyr[1,2] – Earnings before Interest and Taxes SEyr[1,2] – Stockholder’s Equity TLyr[1,2] – Total Liabilities These attributes chosen were the co mponents of the Altman Z score for manufacturing4. The dimension of the featur e space for the Financial Kernel in this experiment is 90. The features are listed in Figure 15. A “YOY” in front of a ratio means the year-over-year change for that ratio. Here is a listing of the features: TAyr1/REyr1 REyr1/TAyr1 TAyr2/REyr2 REyr2/TAyr2 YOYTAyr1/REyr1 YOYREyr1/TAyr1 TAyr1/WCyr1 WCyr1/TAyr1 TAyr2/WCyr2 WCyr2/TAyr2 YOYTAyr1/WCyr1 YOYWCyr1/TAyr1 TAyr1/EBITyr1 EBITyr1/TAyr1 TAyr2/EBITyr2 EBITyr2/TAyr2 REyr1/WCyr1 WCyr1/REyr1 REyr2/WCyr2 WCyr2/REyr2 YOYREyr1/WCyr1 YOYWCyr1/REyr1 REyr1/EBITyr1 EBITyr1/REyr1 REyr2/EBITyr2 EBITyr2/REyr2 YOYREyr1/EBITyr1 YOYEBITyr1/REyr1 REyr1/SEyr1 SEyr1/REyr1 REyr2/SEyr2 SEyr2/REyr WCyr1/SEyr1 SEyr1/WCyr1 WCyr2/SEyr2 SEyr2/WCyr2 YOYWCyr1/SEyr1 YOYSEyr1/WCyr1 WCyr1/TLyr1 TLyr1/WCyr1 WCyr2/TLyr2 TLyr2/WCyr2 YOYWCyr1/TLyr1 YOYTLyr1/WCyr1 EBITyr1/SEyr1 SEyr1/EBITyr1 EBITyr2/SEyr2 SEyr2/EBITyr2 Figure 15 – Bankruptcy Features

PAGE 91

79 YOYTAyr1/EBITyr1 YOYEBITyr1/TAyr1 TAyr1/SEyr1 SEyr1/TAyr1 TAyr2/SEyr2 SEyr2/TAyr2 YOYTAyr1/SEyr1 YOYSEyr1/TAyr1 TAyr1/TLyr1 TLyr1/TAyr1 TAyr2/TLyr2 TLyr2/TAyr2 YOYTAyr1/TLyr1 YOYTLyr1/TAyr1 YOYREyr1/SEyr1 YOYSEyr1/REyr1 REyr1/TLyr1 TLyr1/REyr1 REyr2/TLyr2 TLyr2/REyr2 YOYREyr1/TLyr1 YOYTLyr1/REyr1 WCyr1/EBITyr1 EBITyr1/WCyr1 WCyr2/EBITyr2 EBITyr2/WCyr2 YOYWCyr1/EBITyr1 YOYEBITyr1/WCyr1 YOYEBITyr1/SEyr1 YOYSEyr1/EBITyr1 EBITyr1/TLyr1 TLyr1/EBITyr1 EBITyr2/TLyr2 TLyr2/EBITyr2 YOYEBITyr1/TLyr1 YOYTLyr1/EBITyr1 SEyr1/TLyr1 TLyr1/SEyr1 SEyr2/TLyr2 TLyr2/SEyr2 YOYSEyr1/TLyr1 YOYTLyr1/SEyr1 Figure 15 Continued 8.3.3 Restatement Data Restatements as defined in this resear ch are annual reports by publicly traded companies, which have been restated either voluntarily or involuntarily. Restatements are a much more loosely defined dataset than that of bankruptcy or fraud. There is a strong interest as to the underlying causes of restatements, which was a primary motivation for the addition of this dataset. Th e restatements analyzed in this study were a subset of all restatements of publicly traded companies for the years of 1997 – 2002 (details are explained below). The Restatem ent dataset was gathered using report code GAO-03-138 37 by the General Accounting Office. The restatements in this report involve accounting irregularities resulting in ma terial misstatements of financial results. Restatements can be seen as a superset wh ich includes fraud and ear nings management as subsets. When a company is deemed to have committed fraudulent activity or managed earnings, the SEC requires that the company restate its financials. The GAO report includes an appendix which li sts all restatements for th e years between 1997 and 2002. The restatement dataset is th e largest of the da tasets tested (800/1,379), (i.e. the fraud dataset had 122 cases and the bankruptcy dataset had 156 cases). There were 919

PAGE 92

80 restatements for publicly traded companie s between the years of 1997 and 2002 37. The quantitative dataset was 1379 companies, 690 of which were restatements and 689 of which were non restatements. The smaller, 800 case dataset is a subset of the 1,379 case dataset which includes text and quantitative attributes. The size 800 dataset is split evenly between restatements and nonrestat ements. The reduction from 919 to 690 was due completely to the lack of quantitative data available for some of the companies in the GAO report. The drop from 690 to 400 restatements for the combined dataset was due to the inability to get 10K data for some of the GAO companies. This was due in part to the GAOs inclusion of foreign companies and companies traded on Over The Counter markets, both of which are not required to s ubmit the same type of 10K. The quantitative attributes for this dataset are as follows: Ticker – Company ticker for stock market Label – restatement (-1) nonrestatement (1) Ind – Industry Number Year – 1st year of data collection Salesyr[1,2] – Sales ARyr[1,2] – Accounts Receivable INVyr[1,2] – Inventory TAyr[1,2] – Total Assets OAyr[1,2] – Other Assets CEyr[1,2] – Capital Expenditures The entire quantitative dataset is in A ppendix B under the title of Figure 18. The features for the Restatement Dataset are the sa me as the features in the Fraud Dataset, under Figure 14. 8.4 The Ontology The ontology is a three-level ontology com posed of concepts, two-grams and threegrams. The concepts may be one word or two word concepts. The two-grams and threegrams are built on top of the concepts. The si ze of the ontology is determined at three

PAGE 93

81 levels, the concept, two-gram, and three-gram level. A concept can have many children at the two-gram and three-gram levels. A two-gram can have many children at the threegram level. A two-gram is always a direct child of a concept. A three-gram may be a direct child of a concept or a two-gram. Appendix A shows a 300 dimension ontology. This ontology was built using the entire GAAP text [28] as the accounti ng corpus. The 300 dimensions include 100 concepts, 100 two-grams and 100 three-grams. Given the small number of examples in the fraud and bankruptcy datasets, 300 dimensi ons was the largest ontology created. The concepts are determined by the functions desc ribed in Chapter 6. The concepts chosen for this ontology are the ones that had the highest scores as described in Chapter 6. The two-grams and three-grams are chosen ba sed on mutual information scores, using respectively the Dice Coefficient and ll3 [5]. Commonly accepted Mutual Information scores are available for two and three-grams. Higher order n-grams do not have accepted Mutual Information scores, theref ore this analysis is limited to two and three-grams. An ontology of 100 two-grams and 100 three-grams ma kes it feasible to have some concepts with both children and grandch ildren. The deeper the tree the more specific the ontology gets. The effect is a more precise ontology. The prediction accuracy on the test datasets ultimately determine which ontologies are the best for this particular project. The twograms and three-grams are preceded by their part -of-speech (n-noun, a-adjective, v-verb). As seen in Appendix A, there are two numbers after the two a nd three-grams. The first is the mutual information score and the sec ond is the overall ranking of the n-gram’s importance as compared to all n-grams. Th e ranking is used to determine which two and

PAGE 94

82 three-grams are used in the ontology. A two or three-gram is eligible for the ontology if at least one of its component words is a concept in the ontology. Ontology creation is an iterative process. The process must be refined based on the actual results achieved. The 300 dimens ion GAAP ontology Appendix A was used in conjunction with 10Ks of bankrupt and nonba nkrupt companies (see Section 9.x for further details). Due to the small size of the dataset the 300 dimension ontology appears to be overfitting. Two additional GAAP ontologies were created having 60 and 10 dimensions, respectively. These ontol ogies are available in Appendix A. Choosing an accounting text as the basis of the ontology has a major impact on the results. GAAP was chosen because it is a general purpose text that covers all major accounting topics and is written in natural language. A drawback of GAAP is its indirect relationship to the MDNAs. A more direct accounting text would be the MDNAs. A set of ontologies were created using the MDNAs from the bankrupt and nonbankrupt companies as the accounting text. These ont ologies are of the fo llowing dimensions, 150 (including 50 concepts, 50 2-grams, 50 3-gr ams), 50 (including 50 concepts), and 25 (including 25 concepts). All ontol ogies are availabl e in Appendix A. 8.5 Data Gathering and Preprocessing The financial information for bankrupt fi rms was gathered for two consecutive years prior to delisting. In the event that th e financial information was not available for the two years directly prior to delisting, the latest two years of pre-delisted data were taken instead. In the case of fraud the financ ial data was gathered for the first year of fraud and the year prior to fraud, as reported by the SEC. For example, if the first year of fraudulent activity was 2000, then data from 1999 and 2000 is gathered. In the case of restatements, the restatements were gathered for the year of the restatement and the year

PAGE 95

83 prior to the restatement. The fraudulent/b ankrupt/restatement companies were matched with nonfraudulent/nonbankrupt/nonrestatement companies based on industry, year and total assets. A match was accep ted if total assets of a nonfraudulent/nonbankrupt/nonrestatement company were within 10% of the fraudulent/bankrupt company for year one. If no company met this re quirement, then the company with the nearest to tal asset value was chosen. The Compustat Industrial Annual database was used in conjunction with a script created using Perl to download the quantitative financial data for all three datasets. The 10Ks were gathered directly from www.sec.gov There is one 10K per company and the year of the 10K matches (i n most instances) the last year of the financial information. If the 10K was not av ailable for the last year, then the 10K was chosen as follows: (1) The 10K for the year prior to the final year (2) If (1) was not available, the year after the final year (as long as it is not past the delist year in the bankruptcy case, the rest atement year in the restatement case or the fraud year in the fraud case). If (1) and (2) were not available, both the company and its match company were deleted from the analysis. The text analysis was limited to the sect ion entitled “Management’s Discussion and Analysis of Financial Condition and Resu lts of Operations (MDNA).” The MDNA section is a natural choice as it is the por tion of the 10K which allows management to explain the underlying causes of the company’s financial condition. It also is a section where forward-looking statements are allowed.

PAGE 96

84 Using the Financial Kernel, the attributes are mapped to features, as explained in Chapter 5. The total size of the attribut e space is 12 for the fraud, bankruptcy and restatement datasets. The attributes in the fr aud and restatement data sets are described in Section 8.2. The attributes in the bankruptcy dataset sets are descri bed in Section 8.3. The feature space is determined by the function /1 6(/*) 2AY AY where / A Y = the number of attributes per year. 8.5.1 Preprocessing-Quantitative Data There are three issues to consider for quantitative attributes: missing data, “0valued” data, and scaling. Missing data is a common problem with publicly available financial information. The method used to fill in the missing data for this paper is called multiple imputation [81]. This method takes into account not only the statistics of the missing variable over the entire dataset, but also the re lationship between the missing variable and the other variables in the ex ample. The data is put through a multiple imputation routine in the statistics package R 81. Quantitative attributes with a value of 0 is a problem in this analysis because of th e extensive usage of ratios in the Financial Kernel. A ratio of the form 0 x for any x is undefined. In order to avoid this problem, 0 data are given a value of .0001 and the enti re dataset is scaled between 1 and -1. 8.5.2 Preprocessing-Text Data The preprocessing of the text data involved the following steps: (1) Making all text lowercase. This is done to avoid the problem that a computer will see the same words as different if they are different cases. For example, the word

PAGE 97

85 Asset, asset and ASSET would be considered three separate words. Making all letters lowercase avoids this problem. (2) Deleting stopwords. Stopwords are common words that add noise to text analysis. Deleting these stopwords is a met hod of cleaning the text. The stopword list used for preprocessing the ontology is the same stoplist used for preprocessing the MDNAs. The stoplist is available in Appendix A. (3) Part-of-speech tagging and stemming. Part-of-speech tagging assures that matches between the MDNAs and the ontology w ill occur only for words with the same spelling and part-of-speech. Stemming removes the suffixes from the words to facilitate matching of concepts that are the sa me but used in different tenses. (4) Concept-naming. For this step, all synsets from each concept from the ontology are given a single, representa tive word. For example, th e concept liability has three synonyms; liability, indebtedness and financial obligation. The MDNA is searched for all three words and each instance is replaced with a single representative word. This allows for correct matches between the ont ology (which was preprocessed with conceptnaming as well) and the MDNAs. Simple counts of each component of the ontology are placed in vector form for each company MDNA. The size of the ontology is a user-defined parameter. The size of an ontology is limited to the top scoring concep ts, two-grams and three-grams. The user decides how many of each should be in the on tology. The main limitation is that only two and three-grams that have an ontology conc ept as one of their components words can be in the ontology.

PAGE 98

86 Below is an example of one MDNA, (company name Fifth Dimension Inc., year 1996) This is a sample of the raw text. The Corporation spent $122,128 on capital additions during 1996 while recording $124,611 of depreciation expense. A reduc tion in capital spending is projected for 1997 while depreciation reserv es are projected at slight ly lower levels than in 1996. This is the text after Steps (1) and (2). corporation spent 122,128 capital additi ons 1996 recording 124,611 depreciation expense. reduction capital spending projected 1997 depreciation reserves projected slightly lower levels 1996. This is the text after Steps (3) and (4). null/JJ/null corporation/NN/corporati on spent/VBD/spend 122/CD/122 ,/,/, 128/CD/128 capital/NN/capital additions/NNS/addition 1996/CD/1996 recording/NN/recording 124/CD/124 ,/,/, 611/CD/611 depreciation/NN/depreciati on expense/NN/expense ././. reduction/NN/reduction capital/ NN/capital spending/NN/spending projected/VBN/project 1997/CD/ 1997 depreciation/NN/depreciation reserves/NNS/reserve projected/VBN/proje ct slightly/RB/slightly lower/JJR/lower levels/NNS/level 1996/CD/1996 ././. The complete MDNA is available via a link in Appendix B. The text vectors are created by totaling the number of times each ontology component is encountered in the text of a company’s MDNA. The text vectors are

PAGE 99

87 normalized by dividing each vect or component by the total wo rd count of the company’s MDNA text. This normalization procedure assu res that the importance of concepts to a particular document is not diminished due to the difference in sizes between documents. This Chapter gave the resear ch questions along with deta iled explanations of the bankruptcy, fraud and restatement datasets. Da ta preprocessing was explained as well as ontology creation. In the next Chapter test results are give n on the three datasets along with discussions on each.

PAGE 100

88 88 CHAPTER 9 RESULTS This chapter gives the results of the empi rical tests. Each dataset is tested individually and the results ar e listed in table format. Following the results for each dataset is an discussion of th e results. The format of the results is explained below. The experiments are set up so that the hypotheses in Chapter 8 can be either supported or negated. There are three main cat egories of tests. The quantitative data is tested using a SVM with the Financial Kernel. The Text Kernel is tested using various sizes and types of ontologies. The Combination Kernel is te sted using various sizes and types of ontologies as well. Th e results are given in tables 2 40. The table headings are described as follows: “SV” is the number of support vectors. “/SV” is a rough measure of the genera lizability of the “Test on Training” results. Here is the number of examples in the dataset. “C” is a user-defined parameter that de fines the penalty for a mistake in the quadratic optimization problem. Deciding on the right C is more of an art than a science. After raising C to a certain point, the results will level off or decline. Results are given for various values of C. “Test on Training” is the test results of th e examples used to train the SVM. The number shown is the predic tion accuracy of the model. “10-fold Cross Validation” results are the average prediction accuracy of 10 SVM runs where 10% of the examples are left out from training on each run and used for

PAGE 101

89 testing. This method is often used to te st the generalizability of a function on small datasets. A few of the tests were left running wit hout completing for over a week. In these instances it was decided to cancel the runs. The cancelled runs are shown as blanks and are highlighted in gray. 9.1 Fraud Results In this section we report the results of the experiments using the Financial Kernel, the Text Kernel, and the Combination Kernel. Text Kernels of various dimensions were tested. One set of text kernels is based on the GAAP text and the other is based on MDNAs. For the fraud experiments, all MDNAs in both the fraud and bankruptcy datasets are used to create the ontology. Table 2 shows (starting from left) the Test number, the number of Support Vect ors for that Test number, the /SV function to determine generalizability base d on the training set, the erro r penalty (C) used for that Test number, the Test on Training Results and the 10-fold Cross Validation results. Results tables are in this form for the Bankruptcy and Restatement datasets as well. Tables 2 – 14 illustrate the results on the Fraud datasets. Table 2 – Fraud Detection Resu lts using Financial Kernel Test #SVSV/lC Test on Training 10-fold Cross Validation 12318.85%1 98.36%94.26% 22218.03%100 100.00%95.90% 32218.03%1000 100.00%95.90% 42218.03%10000 100.00%95.90%

PAGE 102

90 Table 3 – Fraud Detection Results usi ng Text Kernel, 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 17057.38%1 100.00%46.49% 27057.38%100 100.00%46.49% 37057.38%1000 100.00%46.49% 47057.38%10000 100.00%46.49% / SV Table 4 – Fraud Detection Results us ing Comb. Kernel, 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 14839.34%1 100.00%92.11% 24839.34%100 100.00%92.11% 34839.34%1000 100.00%92.11% 44839.34%10000 100.00%92.11% / SV Table 5 – Fraud Detection Results us ing Text Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 17460.66%1 80.70%54.39% 27662.30%100 85.09%53.51% 37662.30%1000 85.09%53.51% 47662.30%10000 85.09%53.51% / SV Table 6 – Fraud Detection Results us ing Comb. Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 13125.41%1 100.00%92.98% 22822.95%100 100.00%92.98% 32822.95%1000 100.00%92.98% 42822.95%10000 100.00%92.98% / SV Table 7 – Fraud Detection Results us ing Text Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 110485.25%1 62.28%44.74% 210485.25%100 62.28%44.74% 310485.25%1000 62.28%44.74% 410485.25%10000 62.28%44.74% / SV

PAGE 103

91 Table 8 – Fraud Detection Results us ing Comb. Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 13024.59%1 98.25%92.98% 22520.49%100 100.00%91.23% 32520.49%1000 100.00%91.23% 42520.49%10000 100.00%91.23% / SV Table 9 – Fraud Detection Results us ing Text Kernel, 150 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 17158.20%1 94.74%39.47% 26452.46%100 100.00%42.98% 36452.46%1000 100.00%42.98% 46452.46%10000 100.00%42.98% / SV Table 10 – Fraud Detection Results using Comb. Kernel, 150 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 13931.97%1 100.00%92.98% 23931.97%100 100.00%92.98% 33931.97%1000 100.00%92.98% 43931.97%10000 100.00%92.98% / SV Table 11 – Fraud Detection Results us ing Text Kernel, 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 18166.39%1 84.21%50.00% 27763.11%100 78.95% 37359.84%1000 46.77%48.25% 47359.84%10000 50.00% / SV Table 12 – Fraud Detection Results using Comb. Kernel, 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 13528.69%1 99.12%92.98% 23629.51%100 100.00%92.98% 33629.51%1000 100.00%92.98% 43629.51%10000 100.00%92.98% / SV

PAGE 104

92 Table 13 – Fraud Detection Results us ing Text Kernel, 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19477.05%1 68.42%43.86% 29275.41%100 70.18%49.12% 39275.41%1000 70.18%47.36% 49275.41%10000 70.18%47.36% / SV Table 14 – Fraud Detection Results us ing Comb. Kernel, 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 12722.13%1 99.12%92.11% 22318.85%100 100.00%93.86% 32318.85%1000 100.00%93.86% 42318.85%10000 100.00%93.86% / SV 9.2 Discussion of Fraud Results Tables 3, 4, 6, 8, 9, 10 and 11 show under “Testing on Training” that the SVM was able to perfectly separate the Fraudulent fr om the Nonfraudulent companies. The 10-fold Cross Validation results degrade quite a bit though. An explana tion for this is overfittting, as the number of examples is often outnumbered by the number of features. The exception was the Financial Kernel resu lts as shown in Table 2. The 10-fold cross validation results shown in Ta ble 2 are 95.9% accurate. also, the #/SV is 18.03%, which can be interpreted as the risk of incorrect categoriza tion on unseen data. This is the lowest #/SV for the Fraud and Bankr uptcy experiments. The strong results from the Financial Kernel are contrasted against the results from the Text Kernel. No Text Kernel gave 10-fold cross validation results more accurate than 54.39%. Given that the sample is split 50/50 between the positive and the negative classes this result is not very encouraging. The Combination Kernel got strong results, which can be attributed to the Financial Kern el portion. The results are slightly worse with the Combination Kernel than with the Financial Kernel alone which again signals

PAGE 105

93 that the large number of featur es introduced by the Text Kernel causes overfitting. These results do not support Hypothesis 4, which states that the text methods will do at least as good a job predicting financial events as the quantitative methods. The fraud 10K ontology was created using MDNAs from the bankruptcy and the fraud datasets. The possibility exists that the MDNAs from the bankruptcy dataset added noise to the fraud ontology. Another possibility is that the be tter ontology is based on the MDNAs of all publicly traded companies. The small samples used in these experiments may not give enough information to create the true ontology. An interpretation of the results is that mo re information regarding fraud is given in the quantitative financial values than the text or that the ontology is not strong enough for the task. The patterns in th e financial ratios of fraudulent companies are different than those of nonfraudulent companies. The word patterns are not as strong or are not detectable by the ontology created in this work. An experiment was run using the Financial Ke rnel to test for per-class error. The results of the cross-validation show that the Financial Kernel correctly classified fraudulent companies 95.1% of the time and nonfraudulent companies 93. 4% of the time. It is clearly more important to err on the side of detection, and that is what the Financial Kernel did. For comparison purposes, ratios of the attrib utes used in the fraud dataset were fed into a Linear Discriminant Analysis (LDA) function and a Logit function. Ratios of all features of the second-year data were used. The ratios for this experiment are one-way. That is, x y but not y x The LDA, using ratios of the attr ibutes predicted fraud with 65% accuracy. The 10-fold cross validation resu lts for Logit were 54.10%. These results

PAGE 106

94 validate the power of the Financial Kernel together with SVM. The Financial Kernel gives two years of ratios wh ich are two way, that is x y and y x as well as year-over-year changes. In the case of a complex dataset, such as fraud, the additional ratios were necessary for correct classification. 9.3 Bankruptcy Results The results section consists of experime nts using the Financial Kernel, the Text Kernel, and the Combination Kernel. Text Kern els of various dimensions were tested, One set of text kernels is based on the GAAP text and the other is based on MDNAs. For the bankruptcy experiments, the MDNAs in th e bankruptcy dataset are used to create the ontology. Tables 15 -27 illustrate the results. Table 15 – Bankruptcy Prediction Re sults using Financial Kernel Test #SVSV/lC Test on Training 10-fold Cross Validation 111976.28%1 76.92%64.74% 28453.85%100 87.82%64.10% 38151.92%1000 94.23%56.41% 47447.44%10000 94.87%58.97% 57246.15%100000 96.15%62.18% Table 16 – Bankruptcy Predic tion Results using Text Ke rnel, 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19661.54%1 95.51%50.00% 28252.56%100 100.00%52.56% 38252.56%1000 100.00%52.56% 48252.56%10000 100.00%52.56% / SV Table 17 – Bankruptcy Prediction Results using Comb. Kernel 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 18453.85%1 100.00%62.18% 28453.85%100 100.00%62.18% 38453.85%1000 100.00%62.18% 48453.85%10000 100.00%62.18% / SV

PAGE 107

95 Table 18 – Bankruptcy Prediction Results using Text Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 111473.08%1 75.64%50.46% 210164.74%100 79.49%51.92% 310164.74%1000 79.49%52.56% 410265.38%10000 79.49%52.56% / SV Table 19 – Bankruptcy Predic tion Results using Combination Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 110667.95%1 96.79%55.13% 28755.77%100 100.00%58.33% 38755.77%1000 100.00%58.33% 48755.77%10000 100.00%58.33% / SV Table 20 – Bankruptcy Prediction Results using Text Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 114592.95%1 58.33%39.74% 214492.31%500 58.33% 31000 410000 / SV Table 21 – Bankruptcy Predic tion Results using Combination Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 112177.56%1 75.00%58.97% 210265.38%10 78.21%62.18% 38554.49%100 78.21%64.74% 48252.56%1000 83.33%62.82% / SV Table 22 – Bankruptcy Prediction Results using Text Kernel 100 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 18252.56%1 95.51%65.81% 26944.23%5 97.44%67.10% 36139.10%500 99.36%63.87% 46139.10%1000 99.36%63.87% 56139.10%10000 99.36%63.87% / SV

PAGE 108

96 Table 23 – Bankruptcy Prediction Results usi ng Combination Kernel, 100 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19661.54%1 92.31%66.03% 27346.79%0.5 100.00%67.31% 37648.72%100 100.00%60.26% 47648.72%1000 100.00%60.26% 57648.72%10000 100.00%60.26% / SV Table 24 – Bankruptcy Prediction Results using Text Kernel 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 18353.21%1 84.62%66.03% 27648.72%100 87.82%62.82% 37648.72%1000 87.82%62.82% 47648.72%10000 87.82%62.82% / SV Table 25 – Bankruptcy Prediction Results usi ng Combination Kernel, 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19460.26%1 89.74%67.95% 28353.21%50 85.26%71.15% 38051.28%100 85.26%70.51% 47850.00%1000 87.82%68.59% 57749.36%10000 87.82%69.23% / SV Table 26 – Bankruptcy Prediction Results using Text Kernel 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19359.62%1 82.69%71.15% 29258.97%100 82.69%67.95% 39258.97%1000 82.69%67.95% 49258.97%10000 82.69%67.31% / SV

PAGE 109

97 Table 27 – Bankruptcy Prediction Results usi ng Text Kernel combined with Financial Attributes, 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 19460.26%1 89.74%67.95% 27346.79%10 92.31%67.31% 36340.38%100 98.72%70.51% 46239.74%1000 99.36%68.59% 56239.74%10000 100.00%69.23% / SV 9.4 Discussion of Bankruptcy Results Tables 16, 17, 19 and 23 show under “Testi ng on Training” that the SVM was able to perfectly separate the Ba nkrupt from the Nonbankrupt companies. The 10-fold Cross Validation results degrade quite a bit though. An explanation for this is overfittting, as the number of examples is often outnum bered by the number of features. The highest prediction accuracy using 10-fold cross validation was 71.15% and was achieved by the Combination Kernel us ing a 50 dimension 10K ontology (Table 25) and the Text Kernel using the 25 dimension 10K ontology (Table 26). The best results for the Financial Kernel were 64.74% (Table 15). One explanation for these results is that there is more discriminatory informa tion in the 10K ontology than the features for the Financial Kernel. The featur es are a mapping of the attribut es and the attributes were chosen based on the Altman Z Score. Perhap s there are other attributes which, when mapped to feature space, would provide more discriminatory power. An explanation for the Combination Kernel’s inability to improve re sults is that it is likely to be overfitting due to the small training set size compared to the number of total features. The Text Kernel based on 10Ks performed markedly bett er than the Text Kernel based on GAAP. The best results for the GAAP ontology are sh own in Table 16 and are 52.56% which is significantly less than the 71.15% achieved by the 25 dimension 10K ontology. The

PAGE 110

98 results lend support to Hypothesi s 4, which states that the text is as good or better a discriminator than the quantitative data. The Text Kernel with 25 dimension ontol ogy was used to get per-class accuracies of the bankruptcy dataset. The cross-validati on results show that cl assification accuracy for bankruptcy was 67.9% and nonbankruptcy was 72%. It is more important to correctly classify bankrupt companies, this is a weakness of the model. For comparison purposes, two tests were pe rformed using this dataset with the Altman ratios as inputs into an LDA. The LDA was tested using the training set and predicted with 65% accuracy, which is significantly lower than the highest prediction accuracy achieved using these methodologies. The second test was with Logit. The 10fold cross validation results were 66.03%. Thes e results were also lower than the highest predicted accuracy using this methodology. Th e LDA and Logit perfor med slightly better than the Financial Kernel alone. Given the sm all dataset and the large number of features in the Financial Kernel, overfitting is the li kely culprit. Anothe r issue could be the chosen attributes. The Altm an ratios did not perform par ticularly well on the dataset using LDA, which was Altman’s method. Perhaps the optimal set of features for bankruptcy prediction have changed since the Altman publication. 9.5 Restatement Results The Restatement Data is broken into two sets, one which is quantitative data alone, and has 1,379 cases, and the other which is te xt and quantitative da ta and has 800 cases. The set of experiments listed in Table 28 ar e related to the Financial Kernel and the dataset of 1,379 cases.

PAGE 111

99 Table 28 – Restatement (1,379 cases) Predic tion Results using Financial Kernel Test #SVSV/lC Test on Training 10-fold Cross Validation 162545.32%1 93.40%92.89% 213810.01%100 96.76%95.07% 31118.05%1000 97.03%94.49% / SV The following results consist of experiment s using the Financial Kernel, the Text Kernel, and the Combination Kernel on the data set of 800 cases. Text Kernels of various dimensions were tested, one set of text kern els is based on the GAAP text and the other is based on MDNAs. For the restatements expe riments, the MDNAs in the restatements dataset are used to create the ontology. Tables 29 -41 illustrate the results. Table 29 – Restatement Prediction Results using Financial Kernel Test #SVSV/lC Test on Training 10-fold Cross Validation 177496.75%1 52.67%51.02% 274593.13%100 55.33%51.02% 372991.13%1000 57.36%51.52% 469386.63%10000 60.46%53.56% Table 30 – Restatement Prediction Results using Text Kernel, 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 167684.50%1 76.14%55.58% 260175.13%100 76.14%54.70% 359874.75%1000 76.14%55.33% 457772.13%10000 75.35%55.58% Table 31 – Restatement Prediction Results using Comb. Kernel, 300 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 166983.63%1 70.18%56.73% 257672.00%100 75.00%54.44% 355269.00%1000 76.14%54.95% 450863.50%10000 75.25%55.84%

PAGE 112

100 Table 32 – Restatement Prediction Results using Text Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 171889.75%1 68.27%54.31% 269787.13%100 68.27%52.54% 369386.63%1000 68.27%52.92% 469186.38%10000 66.62%52.92% Table 33 – Restatement Prediction Results using Combination Kernel, 60 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 171088.75%1 61.55%54.06% 266282.75%100 66.50%55.20% 364280.25%1000 68.27%54.94% 457672.00%10000 66.62%54.57% Table 34 – Restatement Prediction Results using Text Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 175594.38%1 64.59%52.67% 274693.25%100 64.59%52.79% 374693.25%1000 64.59%52.92% 474693.25%10000 64.59%52.54% Table 35 – Restatement Prediction Results using Combination Kernel, 10 Dim GAAP Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 173992.38%1 56.73%53.55% 270187.63%100 60.28%54.70% 368685.75%1000 62.31%54.82% 464780.88%10000 64.59%54.44% Table 36 – Restatement Prediction Results using Text Kernel 150 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 170688.25%1 61.29%54.19% 267584.38%100 63.20%54.42% 366783.38%1000 63.20%54.19% 463379.13%10000 64.34% / SV

PAGE 113

101 Table 37 – Restatement Prediction Results usin g Combination Kernel, 150 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 169486.75%1 64.72%53.05% 263479.25%100 67.51%52.67% 362277.75%1000 69.29%52.16% 455369.13%10000 68.65%55.58% / SV Table 38 – Restatement Prediction Results using Text Kernel, 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 172590.63%1 58.63%54.06% 270087.50%100 59.90%54.06% 369386.63%1000 60.15%53.93% 468285.25%10000 60.28% / SV Table 39 – Restatement Prediction Results usin g Combination Kernel, 50 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 171188.88%1 60.79%54.44% 266182.63%100 64.47%54.44% 363679.50%1000 67.26%54.70% 456470.50%10000 67.89%55.33% / SV Table 40 – Restatement Prediction Results using Text Kernel, 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 173692.00%1 56.98%52.54% 272090.00%100 58.76%52.79% 372090.00%1000 58.50%52.16% 471689.50%10000 58.25% / SV Table 41 – Restatement Prediction Results usin g Text Kernel combined with Financial Attributes, 25 Dim 10K Ont. Test #SVSV/lC Test on Training 10-fold Cross Validation 172290.25%1 57.99%52.92% 268085.00%100 63.58%55.46% 366483.00%1000 64.21%53.93% 463078.75%10000 63.96%55.70% / SV

PAGE 114

102 9.6 Discussion of Restatement Results The obvious standout results on this dataset are the Financ ial Kernel results on the large restatement dataset. The 95% results shown in Table 28 validate that the Financial Kernel, together with simple attributes, can accurately predict restatements. The results for the Financial Kernel on th e 800 case dataset are shown in Table 29. The best result was 53.56%, much lower than the 95% results fr om the larger dataset. It seems likely that the 579 cases which are in the large da taset but not the small dataset provide the SVM more separation information. The per-cl ass accuracies were obtained for the results of the Financial Kernel. Restatements were correctly classified 95% of the time and nonrestatements were correctly cl assified 95.1% of the time. The distinction in per-class accuracy is very minor. The results on the size 800 dataset are much lower than the results achieved on the larger dataset. The results in Tables 29 – 41 show that the more complex ontologies achieve the best results. The 300 dimension GAAP ontology is the larges t in this study. This ontology combined with the Financial Ke rnel achieved 56.73% accuracy, as seen in Table 31. Although this is the highest accur acy for this dataset the accuracy does not compare to the accuracies achieved in the ma nagement fraud and bankruptcy datasets and is not much better than mere chance. The results in Tables 29 – 41 show that improvement is achieved with increases in the dimension and complexity of the ontologies. This indicates that overfitting is not as extreme as was the case with the other, smaller datasets. It also indicates th at the additional features are important to the discovery of a separating function. In Sec tion 10.3 it is suggested that larger, more complex ontologies should be created as a means to achieve better separation between

PAGE 115

103 restatements and nonrestatements, although th en overfitting becomes a potentially more important issue. 9.7 Support of Hypotheses Hypothesis 1 states that the Combination Kernel classifies more accurately than the Text Kernel and Financia l Kernel. Hypothesis 1 wa s supported for the 800 case restatement dataset, as shown in Tables 29 41. The test results show that prediction results improve as the dimension of the traini ng vector increases. The best results come from the 390 dimension Combination Kernel as shown in Table 31. As previously discussed, the restatement dataset was the onl y set large enough to avoid overfitting the data when using the Combination Kernel. The fraud, bankruptcy and 1,379 restatement datasets did not support the hypothesis. Hypothesis 2 states that Text Kernel preprocessed with the Accounting Ontology cl assifies more accurately than the Text Kernel using basic word counts. Th is Hypothesis is yet to be tested. Hypothesis 3 states that the Text Kernel preproce ssed with the Accounting Ontology classifies as accurately or more accu rately than quantitative methods, including the Financial Kernel. Hypothesis 3 was s upported bankruptcy dataset. The fraud and restatement datasets did not support Hypothesis 3 as the clas sification accuracy for text was much lower than the classification accura cy for the Financial Kernel in both cases. This Chapter gave the results of the tests on the datasets along with an analysis for each. Section 9.7 explained how the results supported or refuted the research hypotheses given in Chapter 8. The next and final Chap ter is Chapter 10. In Chapter 10 a summary of the paper is given along with a conclusion and an expl anation of future research.

PAGE 116

CHAPTER 10 SUMMARY, CONCLUSION AND FUTURE RESEARCH In this chapter a summary of the resear ch is given (Section 10.1), a conclusion (Section 10.2) and an explanation of future research (Section 10.3). 10.1 Summary A methodology was created to combine text and quantitative vari ables in domains where the combination of the two can provide insight into underl ying structure. The quantitative variables were mapped into a hi gher dimensional space vi a a kernel function which was constructed using domain knowledge from finance. This is the first, to our knowledge, domain specific kernel desi gned for financial problems. Accounting ontologies were created as a means of fi nding concepts which were salient in an accounting context. The methodology was created with help from the literature. The ultimate process, however, was quite new. Utilizing text as a discriminator for predicting financial events is unprecedented. The methodology was tested on fraud, bankr uptcy and restatements datasets. The datasets used for empirical testing were chosen based on their complexity. The expectation is that complex financial datasets would benefit from using text together with quantitative attributes. Interest in mechanis ms of detecting the lik elihood of management fraud and restatement is strong. Bankruptcy was chosen because it is well-studied subject with many models to benchmark against.

PAGE 117

10.2 Conclusion The results attained on management fraud detection were very strong. These results were achieved by the Financial Kernel using simple quantitative attributes as inputs. Using the same attributes and so me reasonable assumptions about mappings, the data was tested with LDA and Logit. In bot h cases, the results were much lower (65/66% vs. 95%). These results were achieved usi ng only publicly available data. The best results in past research were obtained by surveying audit partners, who filled out checklists of both quantitative and qualitative company attrib utes. A positive feature of this research is that it can be applied usi ng a computer with intern et access, without the high costs of surveys or personal inte ractions with top management. Bankruptcy is a well-studied subject. Successful models of bankruptcy detection can be found in the literature (s ee Section 2.2 for details). Th e results achieved with this methodology did not match the best A possible reason for this could be that the wrong attributes were chosen for the Financial Kernel However, the text results in bankruptcy were very promising. Achieving 71.15% accuracy using a 25 dimension ontology showed promise for using the text from co mpany reports as attributes in a machine learning context. An intuition here is that there are obvious differences between the text in the MDNAs of healthy companies and those bankrupt companies. The Financial Kernel got surprising resu lts (95% cross-validation accuracy) on the full (1379 case) restatement dataset. The re sults were much lower for the Financial Kernel on the smaller, 800 case dataset. The difference in results mi ght be explained by a much higher discriminatory power in the cases that appear in the large set and not the small set. The text results achieved on the re statements dataset were not much better than chance. The text data as represented in the re search is inadequate. The ontologies are not

PAGE 118

powerful enough to detect differences between the restatements and nonrestatements. The interesting result of the restatemen t tests was the continued improvement in prediction accuracy with highe r dimensional ontologies. 10.3 Future Research The work on this dissertation opened up ma ny possibilities for future research. Improvements can be made to the ontology development process. The relationships between the concepts can be more fully explor ed. Ontologies for mo re granular issues can be hand made. For example, one area of future research which is currently underway is an analysis of the types of risk listed in th e risk statements that are required as part of a proxy statement. Prior research in this area has shown that th e types of risk listed have a positive correlation with actual future risk. Th e risk types and their relationships with other keywords are being hand crafted by a pers on who has expertise in this area. This handcrafted ontology can be test ed against an automatically created ontology, using the methodology from this paper. Improving the classification accuracy of th e text is an important area of future research. Currently, the base text (GAAP and 10Ks) are used to build the ontology. There is no preprocessing which gives highe r weights to the ont ology components which differ the most between datasets. A feature selection mechanism, such as the 1-Norm SVM, can be used to choose features which are valuable to the separation of the two classes in the datasets. A technique for tying the ontologies back to the data might be to use the vector space model on the text vector s. The weights on the vector components would boost the importance of components which aid in separation and give 0 weight to components which do not aid in separation.

PAGE 119

The text and quantitative results reported in Chapter 9 varied widely based on dataset, financial attributes and kernel. In future work, many different base texts will be tested in ontology creation and many financ ial attribute combinations as well. The methodology can be applied to other financial events such as stock price changes after earnings announcements. Abnormal changes, su ch as better-than-expected returns or steep drops can be analyzed. In the fraud c ontext the speech transcripts of the executive team can be analyzed textually. Speech is diffe rent than written text as it allows for more human judgment, which may betray clues of future problems. The current analyses on bankruptcy, fraud and restatements can be extended by thoroughly analyzing the result s for underlying causes. The function created by the support vector machine can be analyzed and th e features with the hi ghest weights can be extracted as potential tr ue features that separate the good fr om the bad. This is especially interesting in this research si nce a feature can be text or fi nancial ratios. The companies can be ranked based on their margin (i.e., th e distance from the separating hyperplane). Those which are close on either side may be in a gray area, while those that are furthest from the plane are likely to be protot ypes for fraud/nonfraud, bankrupt/nonbankrupt, restatement/nonrestatement. As a tool for ma nagers and regulatory agencies, a function which helps determine firms in the gray area fo r fraud and bankruptcy can be created. It is possible to determine thresholds ba sed on the output function for fraud and restatements, like the Altman Z-score does for bankruptcy. Future work related specifically to solidifyi ng the results of the datasets tested are as follows. For the fraud dataset, compara tive testing against other methods was difficult due to the fact that the dataset used in fraud detection was created by this author. Other

PAGE 120

research using this particular dataset is not available. In order to compare the research fully with other quantitative methods, the data set will be tested using the precise methods and attributes presented in previous research Future work should be done to extract the optimal size of an ontology for bankruptcy. Th e base text should al so be explored, as there may be a more suitable text with wh ich to build an ontology. The Financial Kernel’s failed to predict bankruptcy as accurately as the Text Kernel. Additional work should be done to test this dataset using the Financial Ke rnel with combinations of different quantitative attributes. Future work on the restatement dataset is to build more complex ontologies for testing. Other quantitativ e attributes should be tested as well. Obtaining large datasets for fraudulent a nd/or bankrupt firms is problematic. The number of features tends to grow rapidly when using th e Financial Kernel and the Accounting Ontology. As a resu lt, there was a prevalence of overfitting in the datasets tested in the dissertation. In the context of a larger dataset, such as the restatement set, a more complex ontology allowed for improvements in prediction. For this reason, future research should include larger datasets with both text and quantita tive attributes possibly from other areas of interest. For example, a potentially large datase t could be the hourly stock market data and newswire reports for a set of companies in an industry. After factoring out the market bias the goal would be to find th e keywords that determine short-term market moves in a large percentage of cases.

PAGE 121

109 APPENDIX A ONTOLOGIES AND STOPLIST A.1 Ontologies A.1.1 GAAP, 300 Dimensions, 100 co ncepts, 100 2-grams, 100 3-grams receivable n/describe a/receivable,0.1197,448 n/note a/receivable,0.2089,263 n/receivable n/payable,0.3651,132 derivative v/embed n/derivative,0.1538,353 v/embed a/derivative,0.2095,260 a/derivative n/instrument,0.2569,214 option incur hedge n/flow n/hedge,0.1350,403 a/ineffective a/hedge,0.1538,353 goodwill convertible a/convertible a/preferred,0.1111,486 submit v/submit n/tax,0.3687,131 report a/reportable n/section,0.1260,429 folder a/last n/folder,0.6556,40 n/folder v/add,0.6735,34 receive a/receive a/pay,0.3333,152 cease evaluate resolution segment serve measure n/measure n/plan,0.1072,503 a/fair n/measure n/plan,10614.3481,1 v/carry n/measure,0.1115,485 a/present n/measure,0.3084,168 a/fair n/measure,0.5445,64

PAGE 122

110 v/deduction a/fair n/measure,9755.2099,138 a/fair n/measure v/take,9931.4677,5 v/describe a/fair n/measure,9813.3973,27 a/fair n/measure v/find,9753.4331,164 n/service a/fair n/measure,9753.7499,158 a/fair n/measure n/service,9754.0683,154 n/alternative a/fair n/measure,9756.3290,125 a/fair n/measure n/alternative,9763.4930,84 n/collateral a/fair n/measure,9762.1074,90 n/security a/fair n/measure,9755.3780,136 v/finish a/fair n/measure,9757.7996,117 v/give a/fair n/measure,9753.2997,167 a/fair n/measure n/model,9753.3943,165 a/fair n/measure n/change,9753.4471,163 a/fair n/measure n/sale,9753.5225,161 v/write a/fair n/measure,9753.6607,159 a/end a/fair n/measure,9753.7720,157 n/swap a/fair n/measure,9753.9024,155 n/situation a/fair n/measure,9754.2699,153 n/example a/fair n/measure,9754.4194,151 a/fair n/measure n/loan,9754.4308,149 a/fair n/measure v/quote,9754.4576,147 n/earnings a/fair n/measure,9754.5644,145 n/determining a/fair n/measure,9754.6199,143 n/estimating a/fair n/measure,9754.6199,143 a/fair n/measure a/undelivered,9754.6199,143 a/fair n/measure a/fixed,9754.6199,143 n/section a/fair n/measure,9754.7761,141 r/far a/fair n/measure,9755.0124,139 a/fair n/measure n/interest,9755.2144,137 n/shares a/fair n/measure,9755.5109,135 PRP/I a/fair n/measure,9755.7400,133 v/impair a/fair n/measure,9755.8796,131 a/fair n/measure v/underlie,9755.9758,128 a/current a/fair n/measure,9756.2491,126 a/fair n/measure v/be,9756.4682,124 a/fair n/measure n/loss,9756.9329,122 a/fair n/measure r/therefore,9757.0797,120 a/fair n/measure a/uncommitted,9757.7408,118 a/fair n/measure v/estimate,9757.9922,116 a/fair n/measure v/carry,9758.5970,114 n/year a/fair n/measure,9758.9432,112 a/fair n/measure n/indebtedness,9758.9662,110 a/fair n/measure a/common,9759.6877,107 a/fair n/measure a/long,9759.7499,105 a/fair n/measure v/describe,9760.1308,103 a/fair n/measure n/finish,9760.3855,101

PAGE 123

111 n/comparison a/fair n/measure,9760.5235,99 v/carry a/fair n/measure,9760.8054,97 n/case a/fair n/measure,9761.3039,95 v/be a/fair n/measure,9761.5544,93 n/guarantee a/fair n/measure,9762.1038,91 a/fair n/measure n/amount,9762.2297,89 a/fair n/measure n/building,9762.9635,87 a/fair n/measure a/fair,9763.3867,85 n/benefit a/fair n/measure,9763.5560,83 a/fair n/measure n/investment,9763.6185,81 a/fair n/measure n/deaccessed,9765.1990,79 a/fair n/measure n/stock,9765.7866,77 n/land a/fair n/measure,9766.6677,75 a/fair n/measure v/require,9767.0052,73 v/use a/fair n/measure,9767.4356,71 v/make a/fair n/measure,9767.9011,69 n/payment a/fair n/measure,9768.7069,67 n/company a/fair n/measure,9769.9191,65 n/accounting a/fair n/measure,9770.8518,63 n/compare a/fair n/measure,9772.2532,61 a/fair n/measure n/debt,9772.9091,59 a/fair n/measure n/computing,9773.3937,57 n/computing a/fair n/measure,9774.1220,55 n/gain a/fair n/measure,9775.2631,53 n/note a/fair n/measure,9775.8582,51 a/fair n/measure n/accounting,9777.7385,49 n/measure a/fair n/measure,9780.6804,47 v/measure a/fair n/measure,9783.0919,45 a/fair n/measure n/grace,9785.1442,43 n/determination a/fair n/measure,9788.4779,41 n/statement a/fair n/measure,9789.0426,39 a/fair n/measure n/cash,9790.6759,37 v/record a/fair n/measure,9793.4155,35 a/total a/fair n/measure,9803.2856,32 a/fair n/measure n/item,9803.9466,30 a/fair n/measure n/5,9809.7595,28 v/compare a/fair n/measure,9814.0377,26 v/exceed a/fair n/measure,9817.1247,24 v/imply a/fair n/measure,9827.0485,22 n/estimate a/fair n/measure,9830.4516,20 r/less a/fair n/measure,9860.9167,18 a/fair n/measure n/collateral,9867.5056,16 a/relative a/fair n/measure,9871.5648,14 v/determine a/fair n/measure,9879.2428,12 n/percent a/fair n/measure,9884.1552,10 a/fair n/measure n/future,9907.9137,8 n/change a/fair n/measure,9918.8661,6

PAGE 124

112 a/fair n/measure n/asset,9936.6050,4 a/fair n/measure n/hedge,10023.6594,2 a/fair n/measure n/derivative,9755.9489,129 a/fair n/measure a/derivative,9802.5892,33 a/receivable a/fair n/measure,9759.1733,108 withdrawal insurance n/insurance n/contract,0.1436,377 capitalization liability n/litigation a/liability,0.1250,430 finish v/finish n/inventory,0.1126,480 v/finish v/december,0.1806,306 journal n/journal n/entry,0.6516,42 liquidation amortize a/unamortized n/deduction,0.1730,315 warrant security a/favorite n/security,0.1781,309 n/equity n/security,0.1846,302 n/security a/sec,0.3160,161 package remit get curtailment n/closure n/curtailment,0.2094,261 calculation collateral v/support n/collateral,0.1553,351 n/collateral a/dependent,0.2000,278 apportion calculate alternative n/alternative n/guarantee,0.1060,508 n/stock n/alternative,0.1074,502 understanding long-term rental a/rental n/payment,0.1271,424 a/minimum a/rental,0.1288,420 a/contingent n/rental,0.1495,366 obtain closure a/partial n/closure,0.1438,376

PAGE 125

113 equity v/stockholder n/equity,0.1541,352 n/stockholder n/equity,0.1963,283 warranty express v/walter v/express,0.1250,430 n/link n/express,0.6600,39 n/express a/favorite,0.9900,3 postpone run n/run v/art,0.1961,284 v/run n/section,0.2744,197 revenue weighted a/weighted a/average,0.5259,71 engage service v/service n/cost,0.1330,408 n/customer n/service,0.2538,216 n/service a/835,0.2955,177 r/prior v/service,0.4715,92 year-end taxation n/taxation n/recognition,0.1419,380 a/taxation n/profit,0.1809,304 v/taxation a/pretax,0.2000,278 section n/operating n/section,0.1051,511 r/later v/section,0.1127,479 creditor indebtedness n/asset n/indebtedness,0.1380,393 module n/module PRP/s,0.6911,31 n/accountant n/module,0.8319,17 amortization n/subject n/amortization,0.1317,414 intangible a/intangible n/asset,0.1103,489 guidebook rent selection n/site n/selection,0.1081,497 lease a/lease a/tax,0.1053,510 find account

PAGE 126

114 a/sec n/accountant,0.7645,25 v/proceed a/account,0.2857,188 sec a/sec v/sab,0.1404,386 hire incremental a/incremental n/shares,0.1209,442 a/incremental n/borrowing,0.3019,172 describe MD/should v/describe,0.1435,378 gross table settlement v/force n/settlement,0.1818,303 computation lessor n/lessor a/implicit,0.1235,435 n/5b a/lessor,0.1538,353 n/5a a/lessor,0.1538,353 lessee computing n/defer n/computing,0.1593,346 damage n/grace n/damage,0.1081,497 v/test n/damage,0.1722,317 contingent defer v/defer n/counterparty,0.1053,510 n/defer v/show,0.1204,446 n/stone v/defer,0.1290,419 r/indefinitely v/defer,0.1429,379 impairment guarantee v/guarantee a/residual,0.1887,292 grace v/cure v/grace,0.1818,303 allocate payment gaap n/gaap n/guide,0.4800,87 n/guide n/gaap,0.4824,84 n/master n/gaap,0.6326,44 take n/take n/term,0.1128,478 n/take n/agreement,0.1148,467 a/take n/throughput,0.1176,457 n/capital n/take,0.2313,240

PAGE 127

115 v/take n/property,0.2804,191 agreement allocation n/allocation n/process,0.1142,471 discount ending deduction n/deduction n/premium,0.1876,295 growth increase v/increase v/decrease,0.1161,463 n/increase n/decrease,0.1921,289 unearned subsidiary a/majority n/subsidiary,0.1317,414 depreciation a/straight n/depreciation,0.1316,415 v/accumulate n/depreciation,0.1523,357 guide a/equities n/guide,0.1145,469 n/industry n/guide,0.1684,325 n/guide a/equities,0.1938,287 n/guide a/specialized,0.2097,258 long-run debtor software n/software n/product,0.1176,457 n/computer n/software,0.4370,107 transaction A.1.2 GAAP, 60 Dimensions, 40 concepts, 10 2-grams, 10 3-grams receivable n/receivable n/payable,0.3651,132 derivative a/fair n/value a/derivative,9827.7976,33 option report serve withdrawal liability security n/security a/sec,0.3160,161 package calculation apportion calculate

PAGE 128

116 alternative a/fair n/value n/alternative,9788.4971,83 understanding equity run weighted a/weighted a/average,0.5259,71 engage service r/prior v/service,0.4715,92 creditor indebtedness a/fair n/value n/indebtedness,9785.6007,96 module n/accountant n/module,0.8319,17 amortization intangible guidebook rent selection lease account hire describe a/fair n/value v/describe,9784.7731,104 v/describe a/fair n/value,9838.5665,27 computation computing a/fair n/value n/computing,9798.7456,56 n/computing a/fair n/value,9799.5029,54 allocate payment n/payment a/fair n/value,9794.0008,67 gaap n/gaap n/guide,0.4800,87 n/guide n/gaap,0.4824,84 n/master n/gaap,0.6326,44 take a/fair n/value v/take,9956.6650,5 agreement ending a/fair n/value n/ending,9784.7560,105 guide software n/computer n/software,0.4370,107

PAGE 129

117 A.1.3 GAAP, 10 Dimensions, 10 concepts receivable option report liability run account describe payment ending calculate A.1.4 10K, Bankruptcy, 100 Dimensions liquidity receivable n/describe a/receivable,0.5066,88 wear adversely r/adversely v/impact,0.2025,301 r/materially r/adversely,0.3049,187 r/adversely v/affect,0.4801,93 title housing n/student n/housing,0.6667,46 modern write-off variation forward-looking indebtedness n/indebtedness n/year v/end,9205.2412,97 order n/order n/year v/end,9205.5218,94 option convertible report write-down fund selection a/common n/stock n/selection,6910.4783,211 v/selection n/year v/end,9202.6130,115 subordinate a/exchangeable v/subordinate,0.2507,247 v/subordinate n/debenture,0.3138,182 account fluctuation

PAGE 130

118 describe v/end v/december v/describe,6783.5351,340 v/end v/december n/describe,6787.2828,286 v/describe n/year v/end,9206.0437,88 sale n/sale a/common n/stock,6597.9698,445 a/common n/stock n/sale,6601.1560,431 v/end v/december n/sale,6788.2856,272 n/sale n/year v/end,9309.7086,20 n/net v/sale,0.2105,289 a/net n/sale,0.2910,201 n/percentage a/net n/sale,7012.7677,207 impact r/negatively v/impact,0.3333,165 legend v/end v/december n/legend,6784.9553,321 n/legend n/year v/end,9453.2650,13 n/table n/legend,0.4671,100 v/section v/legend,0.5000,89 debenture n/debenture a/common n/stock,6608.1459,401 a/exchangeable n/debenture,0.2780,216 exchangeable a/common n/stock a/exchangeable,6595.3678,474 a/exchangeable a/common n/stock,6883.0620,214 rotate v/rotate n/fleet,0.3077,185 caption package mix clear revolve v/revolve n/credit,0.2811,210 property n/property n/plant,0.2085,294 a/intellectual n/property,0.2727,222 calculate subsidiary alternative funding merchandising advantage n/take n/advantage,0.2051,299 due a/common n/stock a/due,6595.8806,467 v/end v/december a/due,6786.1335,298 income

PAGE 131

119 v/end v/december n/income,6792.3332,248 n/income n/year v/end,9250.5221,32 n/income n/tax,0.3764,142 covenant liquid n/liquid n/year v/end,9199.4622,178 n/liquid n/capital,0.2007,304 a/liquid a/nutritional,0.2105,289 a/investible a/liquid,0.2105,289 software mixture n/mixture n/year v/end,9197.7610,197 innovative a/innovative n/statement,0.3274,170 run v/end v/december v/run,6789.0392,263 v/run n/capital,0.2536,245 choice financing a/common n/stock n/financing,6595.1399,478 modify A.1.5 10K, Bankruptcy, 50 Dimensions, 50 Concepts liquidity receivable wear adversely title housing modern write-off variation forward-looking indebtedness order option convertible report write-down fund selection subordinate account fluctuation describe sale

PAGE 132

120 impact legend debenture exchangeable rotate caption package mix clear revolve property calculate subsidiary alternative funding merchandising advantage due income covenant liquid software mixture innovative run choice financing modify A.1.6 10K, Bankruptcy, 25 Dimensions, 25 concepts liquidity receivable wear adversely title modern forward-looking order convertible roll impact debenture exchangeable rotate package clear

PAGE 133

121 revolve property due income covenant liquid software innovative run modify A.1.7 10K, Fraud, 150 Dimensions, 50 concepts, 50 2-grams, 50 3-grams receivable n/receivable n/year v/end,22793.5777,323 n/account a/receivable,0.5381,89 wear a/perfect n/wear,0.1250,631 proceeding a/legal n/proceeding,0.1923,405 pick option a/black a/option,0.2222,342 furnish v/end v/december v/furnish,22028.3176,418 v/furnish n/year v/end,22808.1907,123 n/furnish n/chain,0.1645,481 convertible shareowner consolidated n/c n/c a/consolidated,16521.5030,725 v/end v/december a/consolidated,22020.1943,535 n/note a/consolidated,0.2302,325 a/consolidated a/financial,0.2911,250 a/consolidated a/financial n/statement,16554.2831,672 transportation n/c n/c n/transportation,16517.5027,799 roll n/roll a/roll,0.1429,561 a/roll n/roll,0.1429,561 a/roll a/flatracks,0.1818,426 n/roll n/lottoworld,0.5000,102 subtitle impact r/adversely v/impact,0.1547,518 r/negatively v/impact,0.3562,197

PAGE 134

122 legend security n/security v/exchange,0.1674,468 package contract v/contract n/year v/end,22792.7000,341 n/contract n/year v/end,22848.2800,48 r/forward v/contract,0.2640,280 land v/end v/december n/land,22026.2369,428 n/land n/year v/end,22818.6300,88 a/land n/radio,0.1250,631 v/unit n/land,0.4816,112 revolve n/c n/c v/revolve,16517.5492,798 v/revolve n/credit,0.2125,360 alternative n/alternative a/common n/stock,14689.9426,901 a/common n/stock n/alternative,15957.3718,809 v/alternative n/year v/end,22795.6095,257 advantage liquid v/end v/december n/liquid,22017.9340,646 n/liquid n/year v/end,22798.4667,204 n/liquid n/capital,0.1739,449 opening run a/common n/stock v/run,14686.9042,919 n/c n/c v/run,16522.7855,713 v/end v/december v/run,22025.2302,438 v/run n/capital,0.1861,418 n/sort a/run,1.0000,1 offset v/end v/december v/offset,22025.6727,436 r/partially v/offset,0.5690,79 choice revenue n/revenue n/services,0.1333,598 n/services n/revenue,0.1333,598 n/licenses n/revenue,0.1667,472 n/revenue n/licenses,0.1667,472 liquidity taxation a/common n/stock n/taxation,14705.0936,861 n/c n/c a/taxation,16571.2286,670 n/c n/c n/taxation,16711.7954,667 v/end v/december a/taxation,22022.2295,471

PAGE 135

123 v/end v/december n/taxation,22055.5955,376 n/year v/end a/taxation,22805.6324,131 n/taxation n/year v/end,22956.1653,31 a/total n/taxation,0.1271,623 a/taxation n/margin,0.4467,135 a/taxation n/profit,0.5455,86 adversely v/end v/december r/adversely,22019.5484,565 r/materially r/adversely,0.2655,278 r/adversely v/affect,0.4768,115 title order n/order n/year v/end,22809.2141,116 n/interest n/order,0.2461,299 shareholder vantage n/take n/vantage,0.2500,292 county n/martin n/county,0.1231,640 selection account n/c n/c n/account,16528.6923,689 v/end v/december v/account,22017.7061,653 v/end v/december n/account,22019.0151,587 n/account n/year v/end,22810.5497,111 a/doubtful n/account,0.1553,515 n/account a/payable,0.1715,454 gross stockholder n/stockholder n/year v/end,22799.4411,185 n/stockholder n/equity,0.1479,543 exchangeable a/common n/stock a/exchangeable,14699.7800,872 a/exchangeable a/common n/stock,15327.4995,811 a/exchangeable n/debenture,0.1669,471 a/exchangeable a/preferred,0.1857,420 a/exchangeable v/subordinate,0.2746,265 associate n/castle n/associate,0.2412,312 caption v/end v/december n/caption,22019.7923,551 n/caption n/year v/end,23758.4579,10 v/section v/caption,0.2500,292 v/caption r/hereby,0.2857,257 n/table n/caption,0.4600,125 payment n/payment v/end v/december,22018.9151,592

PAGE 136

124 v/end v/december n/payment,22021.8242,480 n/payment n/year v/end,22808.3087,122 alteration clear property n/property n/year v/end,22804.5746,139 n/property n/plant,0.1583,501 a/intellectual n/property,0.1977,384 a/property n/insurer,0.2500,292 subsidiary n/subsidiary n/year v/end,22799.9517,180 due n/c n/c a/due,16532.7101,683 v/end v/december n/due,22019.1944,580 v/end v/december a/due,22020.4701,528 v/end v/december r/due,22020.7710,511 n/year v/end r/due,22800.0229,179 r/due a/higher,0.1344,593 r/primarily a/due,0.1999,381 software v/end v/december n/software,22023.5838,459 r/retail v/software,0.1905,409 modify A.1.8 10K, Fraud, 50 Dimensions, 50 concepts receivable wear proceeding pick option furnish convertible shareowner consolidated transportation roll subtitle impact legend security package contract land revolve alternative advantage

PAGE 137

125 liquid opening run offset choice revenue liquidity taxation adversely title order shareholder vantage county selection account gross stockholder exchangeable associate caption payment alteration clear property subsidiary due software modify A.1.9 10K, Fraud, 25 Dimensions, 25 concepts receivable wear proceeding furnish security package contract land advantage run revenue liquidity taxation title order

PAGE 138

126 vantage county gross associate payment alteration clear property software modify

PAGE 139

127 A.2 Stoplist a aboard about above across after against all along alongside although amid amidst among amongst an and another anti any anybody anyone anything around as astride at aught bar barring because before behind below beneath beside besides between beyond both but by circa concerning considering despite down during each either enough everybody everyone except excepting excluding few fewer following for from he her hers herself him himself his hisself i idem if ilk in including inside into it its itself like many me mine minus more most myself naught near neither nobody none nor nothing notwithstanding of off on oneself onto opposite or other otherwise our ourself ourselves outside over own past pending per plus regarding round save self several she since so some somebody someone something somewhat such suchlike sundry than that the thee theirs them themselves there they thine this thou though through throughout thyself till to tother toward towards twain under underneath unless unlike until up upon us various versus via vis-a-vis we what whatall whatever whatsoever when whereas wherewith wherewithal which whichever whichsoever while who whoever whom whomever whomso whomsoever whose whosoever with within without worth ye yet yon yonder you you-all yours yourself yourselves be is are were sfas sfa 20x1 3 500 0 100 400 20x4 6 d 2 21 109 22 8 4 025 920 525 607 150 10 983 145 070 600 570 419 40 911 250 750 40 425 45 000 x9 3 2 800 a b c d e f g h i j k l m n o p q r s t u v w x y z 2003sec mrpa 0000 sop inc 1977

PAGE 140

APPENDIX B QUANTITATIVE AND TEXT DATA

PAGE 141

129 B.1 Quantitative Data

PAGE 142

130 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 1NJR49241993-1454.74626.07244.373738.6620.00165.505797.34712.993-11.80796.377250.163525.114 2PNT.492419931719.486206.68837.77130.243719.4860.001735.1768.46535.79169.17172.012527.783 3BOL28341994-11850.552271.99312.7812457.731140.06584.8072550.066985.21770.87342.705929.31620.766 4DNA2834199411745.124752.642146.267103.21745.12460.9892010.995318.022811.97209.6721602.047408.948 53SSFT73721994-127.90910.1994.97548.33229.5981.23523.912-45.565-3.091-5.7574.1115.802 6LEAF73721994150.79387.85622.7660.00150.7934.40648.916-57.1881.1146.5715.32733.497 73CECN15311995-12.4630.5440.18111.1161.7480.0017.662-2.4262.4741.0657.0330.618 83DOVRA15311995128.124.6150.00124.00828.121.17326.7250.88442.862-0.35119.0967.629 93DNKY23301995-1210.2764.37943.072161.6470.0010.962139.4338.79416.917-2.97955.27884.155 103CYDS233019951117.145540.06335.11729.999117.1452.80347.142-94.08318.9832.32126.95920.183 11PRST35551995-127.6117.8885.86226.6690.5862.38868.8238.10129.38412.37157.44311.38 12TRDT35551995123.13317.332.9291.04923.13314.91138.331-4.16721.05610.45935.9622.369 133CREGE36511995-188.67613.1914.55732.531.7220.1539.169-5.1873.609-3.3853.91735.252 14PKAU36511995130.19152.17115.6617.92830.1911.01724.99715.76213.5683.90516.5448.453 15MWHS59611995-11308.009116.399114.395429.6640.52711.631607.842112.642271.5377.594384.168223.674 163HOSN596119951436.2951018.62523.634101.564436.29531.365446.49969.5623.07355.945206.443240.056 17JDN67981995-131.4872.6280.001295.8680.5420.001371.986-7.08955.156-118.931226.539145.447 18MGI679819951274.65145.3893.3540.001274.6517.158339.66415.68755.1563.319194.435145.229 19IBM73701995-1719402340263238029219594474481132137586695127072137559504 20LU7370199511972221413535432221972221832262619120681186268619940 213ITEX73891995-123.6311.1170.00115.5785.7330.22723.4063.9580.2612.46720.3833.023 22MTY73891995116.60827.6724.4442.42216.6085.36822.191-7.7965.7832.98313.5768.615 23KMB26211996-113149.11660.91348.311845.7680.1883.7112663918.3-217.32512.14125.37140.7 24CHA.22621199619819.9925880.443579.393458.0439819.992485.9339110.5982115.2428.5708.132105900.6 25IG28341996-135.3029.7099.35734.7944.0670.91334.044-8.649-4.4690.8748.32825.716 26EPMN28341996134.2910.8921.1560.00134.291.32528.147-107.21817.779-14.3824.3923.755 27SRM36691996-1994.6295.9157.81630.3639.286.61643.642.4280.8117.2772.9870.7 28PRY.A366919961839.0931111.575208.182203.254839.09312.084971.45441.62244.039138.998487.134484.316 29THO37901996-1602.07849.77463.494175.8845.6954.722170.969113.18179.15931.764119.8251.149 30PMSI379019961197.75370.63818.1880.001197.7531.948225.8267.82110.15960.35592.064133.762 31MTST38251996-150.44217.8356.16343.3131.2091.05641.94-0.18625.6411.95932.0949.846 32DAIO38251996139.31960.42310.278.2639.3193.11557.73618.20233.2263.54534.61423.122 33SMD.138421996-1667.13146.77180.937620.4167.00219.041610.54957.737102.23159.653279.42331.129 34BMET384219961628.356580.347162.135151.523628.35628.677848.739572.497472.733206.707667.418181.321 35CRNS47001996-1161.47182.4511.485399.30114.095143.301327.1457.011-900.65135.54873.713253.432 Figure 16 – Fraud Dataset

PAGE 143

131 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 363CFAC470019961145.88736.427120.3910.001145.8873.536135.14917.76754.2313.15865.74369.406 373ADIN50801996-124.8717.9063.70517.1080.2470.19724.024-1.9154.8023.1083.77520.249 38VENT50801996118.05328.3984.2543.66818.0534.15523.888-7.3624.951-1.4439.36114.527 39MSFT73721996-186716390.001100932534941438752886763548397973610 40CA7372199616084404015140.001608450767062762379235124814225 41GYMM79901996-15.5981.1760.00124.5150.5561.43222.89-7.401-3.977-1.30817.2345.656 423BLRGZ79901996123.80316.0380.4540.2523.8030.03723.589.631.7093.48510.41413.166 43CO87111996-1139.60432.62220.529106.9181.472.123136.2759.14661.7417.27657.96178.314 44FDGT871119961100.39339.50865.3550.001100.3936.929103.1361.19259.7793.85983.36319.773 45PSTI87411996-1608.313193.880.001815.624448.38951.135874.027-177.94993.49753.89501.781372.246 46MGLN8741199611140.1371345.279191.2014.7531140.137306.597895.62-108.025287.662170.919158.25737.37 47CQB1001997-12433.726272.214349.9482401.613215.17376.2482509.133-215.809308.805244.687540.5051715.153 480491B100199712463.8954336.12534.844468.6922463.895223.7422915.053157.38365.777308.306621.8322293.221 493SOCNQ23901997-11073.09228.46304.91058.9282.2660.5443405.517-864.027488.507-308.66260.4373145.08 503WSPT2390199711286.1061657.51192.99340.8181286.10666.9731391.211-588.045178.178327.823-487.4521878.663 51MSC34701997-1320.16356.30760.892418.0740.64319.108395.321104.42440.55959.031148.932246.389 52BMMI347019971319.407312.53835.45270.111319.40715.231399.46585.54394.97137.482133.257266.208 53CENL35771997-128.2633.1462.30917.0780.4171.26518.804-72.7097.4452.8411.6967.108 54LINK35771997117.55519.1535.6845.46117.5550.56619.577-10.02914.1391.42214.6654.912 55PCTL36611997-1466.425108.72944.901355.05127.88921.645352.994-32.388112.32217.716190.242162.752 56ASPT366119971370.343390.64286.89612.306370.34345.808560.659156.025258.177101.752298.157262.502 57LSR38121997-110.1623.752.79911.1450.4410.45412.5161.5178.4341.73611.0451.471 58BTHS38121997110.07617.0482.172.06310.0760.23410.6327.6096.0031.2468.4432.189 59STCO38251997-1102.27915.90122.70764.3820.8465.40448.98314.36519.701-3.15326.48722.496 60CRPB38251997169.45577.1111.1698.48369.4552.85463.6863.31830.51913.95253.47410.212 61IMDC38421997-1106.72814.5823.11758.8420.345.10680.707-53.34-0.98816.438-15.62593.332 62ATSI38421997154.38614.5164.44722.68654.3860.37158.431-15.60854.231.82255.822.611 63INSG73721997-138.8696.7540.18525.4570.410.75321.011-27.4719.712-12.89911.4189.593 64GAEX73721997124.26336.045.5484.99924.2631.99414.114-56.302-13.751-4.116-9.92724.041 65CYLK35771998-142.7611.50310.28994.3189.052.0681.289-63.42642.862-13.33761.97919.31 66KTCC35771998197.085170.0523.10329.57897.0855.968100.94713.63141.7912.03451.90449.043 67XRX35771998-1194497891249830024290356628814270540353815491123533 68SBL357719981977.901205.416196.986838.399183.92889.3341139.29252.14216.7091047.944239.86470.041 69DIGL36691998-124.1917.1525.47627.5581.064.38139.998-40.65712.9137.39922.15317.845 70AMXC36691998131.50969.2739.79610.9931.5090.73337.1269.40714.9810.17222.95814.168 Figure 16 Continued

PAGE 144

132 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 71BSX38411998-12233.576537.786461.9813892.711705.084174.03935726360.00194117241848 72BDX3841199813846.0383116.873726.558536.7913846.038451.324436.9582356.377354.403754.6711742.2812668.27 73BLS48131998-123123462943139410102852124345311098-6008114281481528638 74GTE48131998125473478566843615168756092533650587025083216594940 75OHP63241998-14719.411146.7940.0011637.75119.37940.0451686.888-390.095442.693287.81798.7551243.817 76SIE6324199811045.121037.203145.7280.0011045.1263.7921130.112125.144114.7451.301278.412851.7 773PTUS73721998-131.5324.6710.00113.7230.512.7394.293-103.0791.64-1.0012.3721.921 78GAEX73721998114.11430.6664.1651.98614.1141.91811.166-55.492-6.790.473-7.5318.696 79EAII73721998-1106.97639.5080.001116.7815.26610.51980.564-39.88820.265-24.68355.19125.373 80AMSWA737219981107.358109.17721.0730.001107.35817.826113.04719.41223.2043.2369.70643.341 81TLXN35781999-1365.75175.37681.923636.298.42913.159348.844-67.44134.275-81.61116.966331.878 82HYC357819991276.28261.51552.58957.482276.2822.565369.23759.92375.218-14.743208.147161.09 833SCEPE38411999-12.7170.2730.3962.1060.0360.1091.18-16.389-1.036-5.685-1.3342.514 843ANTR3841199912.012.6990.1670.4292.010.0012.972-29.084-5.103-3.064-3.8416.563 853TSRG49221999-11.0930.1270.0015.2910.2650.1854.3-23.766-4.55-2.373-0.9865.286 86APL.149221999117.7433.5410.3740.00117.7430.00122.0923.9581.8447.6220.1071.985 87RAD59121999-113338.947299.6342472.4379909.8472133.643573.2877913.911-3121.5471955.87795.286-688.4098248.889 88CVS5912199917275.418098.3699.33445.57275.4359.57949.52944.11972.51600.14037.13644.9 89ASFD59611999-139.9314.52724.205177.6083.2876.85356.266-210.00723.861-40.60943.26812.998 90HNV596119991191.419549.85229.28754.816191.41913.904203.019-471.65116.835-40.288-24.452155.843 91BAC60201999-15152636383416363257423043.566335.967642191389431972.51827947556594563 923614B6020199914284823877448673885708.42949464503357627139075516075540985.66 93MSTR73721999-1151.25837.5860.001203.3684.21423.733259.087-297.81642.616-118.931-145.538285.04 94MNS737219991187.22149.23537.9950.001187.2296.146208.6545.06117.35538.55839.982168.672 95EDSN82001999-1132.76221.7470.001106.875.31125.534251.03-115.51861.031-17.039182.51368.517 96KIDS82001999198.631115.47724.6920.00198.6311.00390.5694.07323.22514.20854.64735.922 97SERO28362000-1147.7617.06421.186131.4950.789.56175.33857.71155.15631.633152.47522.863 98ENZN283620001130.25217.0185.4420.947130.2520.427549.676-117.604446.1113.319138.989410.687 99AVP28442000-15681.7594.2610.62811.3499.9193.53192.6899.9428.1911.4-75.13267.7 100EL2844200013043.34366.8550.2546.33043.3143.93218.81122.2882.2712.41352.11506.7 101BHI35332000-15233.81310.4898.56452.7220599.26676.2-127.51484.81076.53327.83348.4 102WFT3533200013461.5791814.261498.663443.5883461.5791246.9674296.36290.846471.736596.0751838.242458.122 103ADELQ48412000-12909.351251.6530.00121499.48476.8012208.00117267.5-1994.314-6008366.0993721.15713397.95 104CHTR48412000123043.5663249.222224.1470.00123043.566207.88824961.824-2091.135-1007.8151765.2422861.79222049.466 105WMI49532000-11249215757518565789131319490909-5973034539214098 Figure 16 Continued

PAGE 145

133 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 106AW49532000114513.6345707.485823.2590.00114513.634423.95514347.093-471.536-235.0491931.307585.77912592.27 1073BIGTQ67982000-1175.64818.0560.0011469.6077.8130.0011034.333-689.481-900.651596.07583.798950.535 108WRI6798200011517.581252.24563.1210.0011517.58117.0252095.747-146.9778.434209.672920.8091174.675 1093CAWC73722000-11.7721.2370.0018.3550.070.1051.262-12.025-1.73-0.898-2.6543.916 1103AUGRE7372200018.1538.3231.2810.2518.1530.1155.801-2.101-2.632-1.4672.1013.7 111IINT73722000-1145.68952.8570.001140.7323.32912.346137.737-60.36742.1933.97360.94676.791 112NETE737220001138.37954.03615.7580.001138.3791.041206.179-20.47692.4853.854176.14130.038 113LGTO73722000-1231.39547.6550.001414.8645.09924.998355.261-69.992167.281-15.679259.95995.302 1140485B737220001399.752487.707106.20183.824399.75220.359315.049-353.663-8.9711.276-219.049514.415 115DJT79902000-11351.37255.40212.3242199.31361.1920.7422219.666-346.224-15.578253.36489.4632130.203 116IGT7990200011623.716898.404296.268146.9891623.716335.9671923.4391247.226596.775314.897296.1131627.326 117WCOM.CM48132001-13517953080.00110391433497886989032688-7918138095540943494 118SBC481320011963224590893760.00196322160249505721351-594181983319961858 119CPTH73702001-1104.17326.6920.001199.95255.22811.189104.006-2174.41219.961-31.421-8.55485.66 1203ANCPA737020011207.818306.34844.7554.937207.8182.325180.083-5.148-7.13526.79491.83488.249 121PNC60202002-16356347771613663771517.581335.967681687702-15.5782714664561523 122BK6020200215756305081775648.42947446336346150.00192397664585.66 Figure 16 Continued

PAGE 146

134 Obs #TickerIndYearLabelTAyr1REyr1WCyr1EBITyr1SEyr1TLyr1TAyr2REyr2WCyr2EBITyr2SEyr2TLyr2 1CRYYQ35711993-148.599-202.72517.718-39.22342.6525.94726.166-240.511-6.702-31.3549.09517.071 2ETRC35711993122.8489.88911.8733.49820.2732.57525.43611.09812.3433.64421.4913.945 3FPNQ36611993-135.873-82.14821.22-26.35619.75516.11819.651-109.2292.603-24.522.46317.188 4VTEK36611994117.9471.048.8062.6094.05713.8927.471.49811.9442.56613.8113.66 53PRCAE20331994-110.831-24.2696.810.717.773.06121.861-2.426-1.093-0.72910.30411.557 6ODWA20331994112.072-0.8692.5161.1848.7193.35335.4810.12817.9182.45428.4996.982 73KNITE22531994-112.5873.2274.5561.2997.6584.92913.1983.2574.841.0937.6035.595 8MRSA22531994140.716.90625.14714.38936.0724.63854.00917.05435.78816.04946.2237.786 9LTTO1027211994-11.242-2.0940.894-1.7320 .0030.2392.722-7.790 .056-5.5450.5971.124 10IXDP2721199414.655-8.110.068-1.0171.6752.9816.366-5.1815.4660.810.4695.897 113MLLEQ27411994-120.307-4.92110.382-4.4815.0865.22118.766-9.3979-3.83210.616.135 12AMEP27411994119.3311.7251.1121.547.67111.6611.24-1.2510.7781.1044.8396.401 133CFNEE27411994-124.229-17.323-23.125-3.005-13.27137.521.411-21.854-16.537-3.347-6.93928.35 14TUTR27411994126.931-5.4116.7880.07715.94110.9933.66-1.7216.3526.96219.50214.158 153PWLUA27501994-17.713-5.0230.165-1.4256.4441.2697.032-5.651-0.012-0.7435.8811.151 16GGIT27501994110.615-0.8621.8362.0872.8037.81224.7380.375.8313.659.9914.748 17BISYQ28701994-112.737-50.9763.079-11.3847.2595.47816.357-128.645-5.689-11.48-2.32418.681 18ALCD28701994111.911-6.4235.8762.37310.3291.18613.769-4.0988.583.25711.9261.458 193521B35721994-1233.915-34.499121.022-3.75889.63144.285180.393-118.78765.957-60.4287.173173.22 203EXBT357219941242.765139.686157.97862.169196.90745.858250.336127.251137.14327.087186.36663.97 213MRESE36211994-12.638-0.9890.337-0.2620.4222.2161.773-1.9070.067-0.321-0.4742.247 223DEWY3621199416.417-1.4562.2740.8680.8765.5415.555-1.3491.7690.6210.9834.572 233CPTX.36721994-150.476-56.92617.1821.209-26.94264.46747.711-61.64118.3915.381-31.75960.191 24BHE36721994148.33320.32630.8910.66740.1318.20257.03726.47437.28511.30946.62410.413 25BYDSQ37141994-16.326-0.806-0.1751.6771.8814.26411.782-0.9282.1132.3745.8565.926 26BOWE3714199418.47803.1273.093.9753.579.2921.7094.2963.5345.6972.662 27CNMWQ38121994-132.83926.921-1.701-5.5086.90225.93754.19613.8878.368-8.48924.37829.818 28DBAS38121994129.06119.26714.1992.44924.6324.42932.20920.54815.9982.85426.4245.785 29YESS39441994-129.657-43.5635.082-17.783-43.16422.88948.87-40.92725.6717.17228.58420.286 30EDIN39441994128.2824.18419.4326.1222.8285.45428.2543.9417.9820.54822.5845.67 313UMMF39601994-183.562-113.10828.086-8.431-34.589117.70158.428-140.0037.73-4.652-61.484119.462 323VITC.39601994150.673-23.71123.8970.5739.34541.32847.951-25.02419.93.0548.03239.919 33ATREQ20301995-110.786-29.448-4.242-5.763-2.34613.1327.462-39.81-6.917-8.365-3.92111.383 34ARMF2030199519.054-1.3775.5171.9268.0371.01711.926-0.937.1311.94210.6221.304 35UNCB20501995-111.535-3.0360.0090.8084.7386.79715.883-6.613-0.944-0.7821.16114.722 Figure 17 – Bankruptcy Dataset

PAGE 147

135 Obs #TickerIndYearLabelTAyr1REyr1WCyr1EBITyr1SEyr1TLyr1TAyr2REyr2WCyr2EBITyr2SEyr2TLyr2 36PIFI.20501995112.3613.6233.4970.4436.4385.9239.3971.6791.205-1.5924.5634.834 37SIRNQ23301995-114.738-13.296.124.30410.7683.9718.678-17.7019.9-3.21414.773.908 38JLTA2330199519.8550.0264.108-2.0965.3374.5189.1730.0533.7380.6925.3643.809 39NCCDQ23401995-175.36931.5731.6727.20538.83236.53747.33418.63819.905-7.18725.921.434 40MSI23401995157.2048.42722.6483.7138.748.50434.613.14919.4060.5473.42231.188 419691B28101995-1305.93250.66229.26958.70651.296249.865312.36550.2726.83228.21350.906257.282 42CCC281019951338.001168.11584.58451.891218.187119.814397.251174.44568.6752.801216.895180.356 433MRCFQ28201995-194.9665.04627.96913.88825.63269.334102.6169.4830.47914.69830.17372.443 443GLMA28201995112.5871.9331.2961.4293.4289.15912.2272.0151.4971.5513.5298.698 453QQQQQ28351995-12.736-8.085-4.404-1.605-1.7094.4455.514-10.4832.791-2.0595.2830.231 46ICCC2835199513.234-5.8821.8490.181.9051.3293.131-5.9481.4050.0771.8771.254 479524B29111995-1472.208-150.7855.965-37.32693.357378.851564.241-167.45-407.018-41.2781.363482.878 48TSO291119951519.15335.78577.52999.932216.514302.639582.587110.29599.475109.217304.065278.522 49LAGR30211995-1159.575-168.72103.999-34.888-40.62792.456100.956-239.06246.467-35.048-110.96996.452 50VANS30211995190.461-23.40141.40413.31472.72817.733105.824-12.8453.39819.84388.28217.542 51PHTA32311995-114.193-3.437-3.2170.6323.23410.95919.282-9.4020.621-1.78415.224.062 52CVTL32311995112.04-12.626-4.058-4.935-2.20414.24121.947-17.1793.573-2.3446.72815.219 53CCSTQ33121995-142.2634.18117.9247.27310.52431.73951.2521.43216.7332.3989.21842.034 543CIIIE33121995140.612-2.048-1.0251.783.5 7937.02943.0010.4023 .1285.6418.73934.258 553NATT34431995-12.07-6.715-4.516-1.39-3.6675.7373.053-0.53-1.7530.013-0.5083.561 563BSTM34431995115.09111.4279.7941.60512.2082.883316.17812.3088.8762.18213.0893.089 57NEXR35711995-11.469-2.2610.582-2.259-2.2613.7319.589-9.77110.425-7.477-9.77129.36 58XYBR3571199511.394-3.534-0.039-2.086-1.0932.4878.015-8.7736.412-5.2816.891.125 59GANDF35761995-179.375-5.61229.3617.42748.58630.78945.159-52.03-6.254-30.223.7141.449 60MYLX35761995180.45817.2163.57621.09465.20115.257116.58640.64197.93128.941104.17212.414 613SCRH.35771995-11.461-50.558-10.404-8.33-10.06311.5240.733-62.505-11.238-9.622-11.01611.749 623MITK3577199512.864-2.0880.6020.2181.3431.5213.762-0.8591.8841.8772.6521.11 63MTTRQ37131995-129.6673.58511.2146.59913.66316.00436.5646.90413.5086.76617.09619.468 64COLL37131995146.881-11.51713.4525.4038.80538.07645.744-6.50514.2059.41213.89131.853 65EXCE37511995-11.883-1.6241.551-1.2481.7290.15410.023-4.1369.039-2.6339.60.392 663RSHX37511995126.932-40.1152.32715.834-39.61559.1945.87512.01923.72225.32631.56114.314 673AWCIQ38421995-178.416-7.02312.3710.427.03451.38254.405-36.93110.972-2.966-2.87457.279 68MNMD38421995151.643-1.66431.8092.98342.3629.28159.5033.00837.2097.18649.6269.877 69VOXQ38451995-14.98-15.5413.822-5.5270.7420.6387.625-21.9816.825-6.4797.4060.219 70SPSIQ3845199514.644-40.9474.231-1.312.9220.2553.551-42.4932.95-1.652.2990.393 Figure 17 Continued

PAGE 148

136 Obs #TickerIndYearLabelTAyr1REyr1WCyr1EBITyr1SEyr1TLyr1TAyr2REyr2WCyr2EBITyr2SEyr2TLyr2 71AMCL38511995-13.842-5.815-0.89-1.6390.073.7721.951-9.199-3.112-1.204-3.0765.027 72SEYE3851199517.261.8251.8592.1911.9465.31410.2932.5081.7822.6422.9297.364 73LAZR39441995-12.023-0.164-0.2840.5920.4131.615.401-2.910.756-1.4483.352.051 743KIDZE3944199513.589-4.7-2.654-1.3980.4133.1711.958-5.763-2.363-0.387-0.6262.579 75TAVI20111996-1302.78665.96956.17827.35777.08225.706253.91315.68139.98721.0127.095226.818 76SFD201119961995.254191.87164.312129.713307.486687.7681083.645245.27259.188167.753361.01722.635 77BEANQ20901996-1109.303-80.71311.397.02765.0344.27399.013-93.48410.1952.55457.7640.953 78WFDS20901996180.73827.56120.1316.97348.7332.00895.48634.6725.11318.86256.41639.07 793SSMKQ23001996-113.673-8.9-4.129-4.49-2.82312.39615.727-8.385-0.5861.5760.88310.653 80INNO2300199619.433-20.64-0.61-1.3522.2757.1589.168-21.955-0.179-1.0513.7915.377 81STAZQ23001996-1188.89521.31973.18311.86103.25385.642160.521-15.61833.957-21.22467.43593.086 82GOSHA230019961196.033137.959104.64133.214138.07757.956174.788113.05882.76241.243113.15761.631 83NDRE23301996-117.806-46.191-2.208-0.368-5.95823.76429.379-0.078-0.5141.5172.90926.47 84NICH23301996118.1799.06512.2091.85811.7246.45515.0799.26111.510.62411.6823.397 85APAR23301996-120.408-69.3881.4071.405-4.45720.29722.722-76.045-4.386-3.072-11.11429.226 86VARS23301996137.79118.25318.0069.76629.8977.89429.24313.77715.2127.31824.7943.397 873ABNKQ27501996-1480.378-21.28135.53359.02946.277434.101503.536-26.95656.1260.88546.715448.495 88CVO275019961470.94627.40622.12581.931121.207349.739586.20148.58792.72296.595146.401439.8 893SFFP28341996-12.774-26.6281.434-7.0911.6961.0780.69-36.157-0.838-7.772-4.7172.939 903SQES2834199612.88-7.8570.64-0.9340.3732.1321.409-10.2980.045-1.798-0.2521.311 91CMTR28351996-18.841-36.3084.656-6.1785.033.8114.285-41.5031.104-7.1242.2082.077 923OXIS2835199617.997-33.099-1.405-5.2474.5023.47212.575-38.4280.958-4.5596.7385.82 933MABAE28351996-116.473-42.26813.697-5.7485.54210.9319.388-49.4156.961-6.2516.6832.705 94AVAN28351996117.224-57.12911.672-9.53615.6191.6059.827-70.2374.629-7.2056.3163.511 953LOCKE34201996-12.094-8.092-0.317-2.7461.1320.9625.522-13.3571.8-5.0313.3992.123 96QEPC34201996116.4344.73812.6953.02913.1162.98143.0266.60514.2124.115.29627.393 97MRSIQ35591996-113.428-26.58110.299-4.3049.873.5587.884-31.7234.625-4.8154.8323.052 98JMAR35591996115.396-26.0745.7441.1729.3696.02717.269-24.2789.6351.63812.4884.781 99SUBM35591996-1125.934-13.00625.618-16.8528.67697.25859.708-60.5599.025-27.209-7.4962.298 100PRIA355919961123.78625.04380.84620.29896.92226.864156.98442.121105.1728.447119.38437.6 101TNNYB35691996-13.935-0.9711.0590.5211.3242.6113.24-2.239-0.296-1.1720.0583.182 1023QPDC3569199611.437-26.822-1.456-0.372-1.9293.3662.023-25.812-0.461.04-0.8152.838 103PNLEQ35721996-140.238-20.05812.71-12.4198.50331.73512.544-50.462-15.928-25.344-15.4728.014 104ADIC35721996136.716.05724.5956.2926.38710.32375.19414.30253.35912.02960.1115.084 105SYQTQ35721996-175.181-120.11-37.351-91.498-30.371105.53482.649-204.3041.436-56.0975.61362.951 Figure 17 Continued

PAGE 149

137 Obs #TickerIndYearLabelTAyr1REyr1WCyr1EBITyr1SEyr1TLyr1TAyr2REyr2WCyr2EBITyr2SEyr2TLyr2 106NTAP35721996168.941-0.62441.91920.68354.02914.912115.73620.34169.63138.20686.26529.471 107ADPVQ35761996-17.212-27.0592.828-4.9994.0453.1674.838-28.2132.361-0.0313.2491.589 108SBEI3576199617.894-5.1962.049-6.4473.2313.91311.269-1.8637.4923.2977.9663.303 1093APSP36611996-10.308-16.1480.1550.0160.1560.1520.324-16.0960.208-0.0220.2080.116 110TLKP3661199611.682-5.165-1.329-2.131-1.7752.60823.083-17.14516.423-10.76218.8724.211 111CODDQ36631996-15.521-30.239-0.4680.7120.4475.0735.309-30.2010.5140.9550.6024.706 112AMCM3663199614.969-34.7651.32-1.482-0.3972.7835.45-33.8282.4171.5430.5482.319 113EAIN36701996-150.971-73.245-9.166-7.0387.08643.88547.862-91.307-16.787-4.274-1.10148.963 114VIDE36701996140.88714.12213.7845.27417.74323.14440.58217.58916.4418.35821.14619.436 1153FIVDE36791996-12.1030.9371.0990.0251.7050.3981.255-1.2240.13-1.987-0.4561.711 1163POWDQ3679199610.427-2.026-1.424-0.324-1.2051.6326.501-7.152-9.595-4.02-5.76912.267 117AI.36901996-173.11224.01938.1251.21731.29641.81659.33312.76726.25-0.68120.04339.29 118MPAA36901996175.5111.08651.810.8740.10835.40298.24517.13675.33313.60868.12730.118 119BDTTZ37141996-1503.802199.05490.171114.88275.08228.722877.153189.121-47.20499.21266.419610.734 120STNT371419961581.57145.05178.56884.02200.562381.009573.53622.58595.71873.372178.096395.44 121IMTI1038411996-1137.159-165.0031.714-7.804-14.511151.67202.786-278.252-55.656-43.517-49.923252.709 122BMP384119961142.46594.21875.67541.393135.9246.541186.449121.7592.36251.987179.6896.76 123URMD38421996-1110.488-71.32598.013-19.19235.95274.53676.593-105.86160.907-29.6281.78574.808 124RSND384219961114.752-38.30925.3778.67452.37157.15689.775-59.76619.88310.12937.01952.756 125PLSIQ38451996-119.321-20.2288.018-3.30216.6322.68947.708-61.6319.017-18.05631.45616.252 1263BCHM38451996118.4293.52311.7828.41115.4852.94422.768.16316.3476.8619.9452.815 127ONTAQ22531997-186.977-86.449-42.595-10.537-8.74295.71951.851-126.822-62.395-13.472-49.115100.966 128HAMP22531997180.58530.94936.30316.59457.7122.875100.84835.27151.28314.16563.40337.445 129TLTXQ22531997-1538.226150.005295.72152.758186.081344.114447.328107.994236.1525.582144.74300.89 130PLUAQ225319971165.98726.81932.9749.38463.668102.319158.543-9.24-44.664-18.76327.609130.934 131BISD23301997-134.817-19.0613.9441.5967.65827.15914.384-37.905-11.185-10.756-11.18525.569 132NICH223301997115.0799.26111.510.62411.6823.39714.0589.2639.384-0.97210.5713.487 1333NMPCQ28441997-1132.75924.60238.60212.3723.479109.28138.75121.63129.89916.93121.194117.557 134DLI284419971149.31472.86753.57634.65854.5394.784177.47479.73862.72830.78859.097118.377 135DVLGQ35401997-1121.444-8.50618.27313.90725.71395.731123.915-14.94713.6344.17219.406104.509 136FNSTQ35401997188.83225.42335.1269.86946.9241.91288.71726.0527.58910.57747.54741.17 137RACE35761997-19.47-24.7794.888-5.9023.7842.6074.009-34.2651.056-8.6340.9521.452 138FSCXQ3576199719.226-17.5113.057-6.4113.2465.985.581-23.061.922-5.1611.0364.545 139AXHM1035771997-1204.044-41.93332.24930.031-18.081222.125171.726-72.36531.09633.294-47.998219.724 1403CLCP357719971209.457-211.59519.154-64.08475.733133.72421.939-35.684-6586.939 Figure 17 Continued

PAGE 150

138 Obs #TickerIndYearLabelTAyr1REyr1WCyr1EBITyr1SEyr1TLyr1TAyr2REyr2WCyr2EBITyr2SEyr2TLyr2 141GECMQ35771997-1250.04918.32366.1932.41545.396204.653229.977-4.71575.40617.42324.617205.36 142DIMD357719971337.554-70.615158.287-49.557180.521157.033306.91-110.10492.848-54.291146.37160.54 1433SHELQ36721997-1139.36713.73622.943-0.12182.91756.435136.306-23.459.2190.3978.71657.549 1443IECE367219971152.0737.36734.62222.04775.46176.60998.66531.3431.7648.80969.56829.097 145AURLQ36741997-16.35-138.543-0.645-10.056-24.14930.49913.638-172.567-3.586-11.217-1.30814.946 1463SODI3674199716.835-1.6721.1680.7020.9665.8695.425-1.1961.340.8261.4423.983 1473ECTHE38411997-13.373-11.4560.958-0.514-0.1363.5093.183-12.5340.286-0.637-1.194.373 1483SPSGE3841199713.328-9.4061.122-0.214-1.1473.8732.997-9.7921.3130.029-0.5162.775 149SNRS38451997-12.949-37.0621.382-6.9980.8492.111.479-54.8836.773-14.0840.211.279 150EVMD3845199713.083-19.6122.121-0.182-3.5011.2784.097-18.9522.5440.891-2.4571.248 151STUA1039441997-1137.824-11.24137.57811.16516.372121.452136.699-32.53929.1798.04-4.891141.59 152GAL.394419971207.783-9.89682.8-20.813162.0345.753196.905-20.679134.39424.415149.79147.114 153NWSW33121998-1383.199-31.948122.79586.51486.7296.499318.409-71.82264.5030.98246.826271.583 154KESNQ331219981405.857-9.2430.55535.62253.077352.78410.918-16.727-13.9224.41346.315364.603 155APMPQ36791998-1299.518-106.06513.399-91.69185.96213.558226.903-342.055-72.298-4.342231.245 156LGL36791998148026.8318.76849.47339.793440.207211.1923.80323.2147.10515.991195.201 Figure 17 Continued

PAGE 151

139 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 1CVGYQ49911995-146.42112.014045.5661.2664.34858.15612.176039.9010.0122.52 2WTI.149911995140.91631.2169.9461.32840.9166.12533.81612.4880.77240.2186.3340.521 3IMTC.251101995-115.0212.2540.60150.5310.0910.70571.86235.2694.301360.503232.9754.568 4CLPI51101995-1199.692001239.15759.0280238.373001238.80565.3690 53PCNIQ73701995-112.9184.3048.45918.7620.0950.4085.45176.7131432.4330.1660.154 6CTG73701995114.11430.6664.1651.98614.1141.91828.8684.1351.7711.1661.9030.237 7EQMD80931995-1831.70474.95960.2592021.460100.4461932.813218.912106.5052896.8710123.776 8RTEL80931995117261172.1355.9208.717261165793164.9232472.4378916.705702.5493.5 93WLGN251119961443.092456.04372.3374.918443.09219.064425.02663.8351.851487.4418.40719.28 10PLSSQ25111996135.87872.393.27713.64135.8785.23316.5379.5524.20917.3080.8830 113ABNKQ275019961145.6942.14300145.6940.3041.4130.2820371.4913.39237.761 12CVO275019961217.291207.865117.50646.467217.29137.218385.758189.996134.599429.18223.84446.827 13UMED28341996-12553.7285.3120.31758.8221.634.32518.2305.1161.61675.1321.243.5 14CARN2834199612619.5333647.03292.638385.7992619.53304184.498386.353397.0482984.3830306.749 15SYC2840199615554.472472.6912889.10816.5665554.472936.8387.2712830.38110.4276099.4027.6311.682 16USAD2840199611501.81962.3220.7340.31501.8421917243.8359.51636.752.778.4 173NSTLQ33121996-145.6056.92510.68629.3861.2082.77721.9891.6625.61216.291.3920.129 18NUE331219961102.62593.33323.6580102.62520.36489.42817.5510111.87923.043.009 19FSCXQ35761996-114.4270.0881.456.910.2690.81616.8510.081.9046.0681.3830.544 20MRVC35761996-115.6443.026035.1281.0173.48924.6362.636021.5220.9631.836 21FCSE35761996-133.54514.582066.85502.00544.36120.3190127.5211.7014.488 22ANCR35761996-163.5118.8970272.7453.51955.126116.13229.063050.22458.91619.39 23SBEI3576199618195.0591760.076771.68208195.059227.9071831.2657239.68408809.898216.72138.109 24ANET357619961145.042154.67631.29553.702145.0423.456262.83350.963107.358320.2052.915.155 25MDEA35771996-1340.81680384.489.911.4430.5199.80475.4107.14.9 26CENL357719961514806.27144.29266.8655142.695826.746122.2644.538575.6952.23438.411 27ASD35851996-110513168476774081466531102617528427906146525 28ASD13585199611208.246188.623.17501208.2462.941217.9611.66101427.8814.0850 29RAM3630199611517.581252.24563.12101517.58117.025320.43938.4502095.74726.9310 30HMII363019961447.998960.377138.347114.127447.99821.2896.572109.52281.401470.76237.50444.583 31PCTL36611996-1204.0422.29928.63795.3972.4881.279214.1482.13634.65287.1642.2321.306 32CMVT36611996193.198119.7226.15527.4893.1983.0395.10521.94824.93288.0123.3240.984 33CAMD36701996-125.18510.82367.24419.351.2531.2131441.587144.119119.5658.5510135.848 34VIDE3670199612642.8322087.112585.21602642.83247.2751786.594444.39801811.59933.537144.155 35SVRI36741996130.7478.2131.9554.18530.747010.6171.7784.38724.0900.564 Figure 18 Restatement Dataset

PAGE 152

140 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 363MPAD36741996130.20457.2957.95813.45730.2040.38660.6746.27619.56338.2051.2142.858 373DGIX39601996-1125.08619.76082.6831.1211.494110.13914.125050.2240.5440.097 383LEVC396019961557.28840.785397.2860.328557.2881.15141.146347.4650.606552.0790.9630 39OMPT48121996-1268.83642.9947.306367.94725.91120.271350.95477.9898.843580.46662.86722.077 40PAGE481219961117.064174.34632.86322.125117.06413.069188.81437.36728.205150.23320.6869.365 41TCAT48411996-125.5998.0471.73432.1460.212.85632.7955.580.70232.2480.2287.135 42GOAL48411996-1121.23232.9643.747290.2218.47121.333186.14365.4656.6466318.61224.994 43AMXI48991996-1254.60341.65235.146235.44516.2016.225278.90736.62934.824246.0697.7097.909 44CCIX.148991996-11330.296386.4819.643964.87989.04452.6381222.58296.61925.697851.3729.1243.915 45PHX.248991996-12293638152703316913367112122859286525312835517461177 46SMTKQ489919961132.866196.4862.9620132.86616.171266.96440.7164.442315.7677.3137.564 47MLTNQ49551996-1102.711610.69920.6621404.1162729.445617.436116.808712.87525.0461660.6491881.00952.605 48HDSN495519961126.141286.12339.76134.052126.1415.056325.41747.04536.195134.9473.04914.548 49BERT58121996-1278.01737.9630.952254.0890.72855.599320.16356.30760.892418.0740.64319.108 503FINE58121996195.276132.23935.14926.495.2760.395109.74427.70128.47390.1873.5843.437 51STFR60351996-127.05312.6915.94140.6770.786.58733.79915.4417.28450.2250.1957.502 52EGFC603519961304.08528.048209.6610304.0854.07436.094232.40464.3094.10.315 53FFBA60351996129.03631.0376.0940.10729.0360.38528.4136.8270.11928.570.4251.504 543PHPC63241996-1178.818116.4470376.4984.7181.594267.296116.8160380.72210.6117.548 55HPLX106324199617.89413.352.0532.7427.8940.61924.972.780.85111.2690.3160.291 56SUHI632419961965.7951189.103145.36422.019965.79534.251441.587180.25228.2141085.34923.26170.23 57UDCI6324199615 49.396606.45106.261234.257 549.39616.031608.073117.965 177.291509.42926.05613.74 58ENVY641119961580.9451014.913155.7960580.9451.5731220.852191.1920685.1462.78311.709 59KAYE6411199611973.424148.4491067.6362.0511973.42418.311251.1232151.54711.8123488.3061.17417.171 60TMBS73721996-1172.42431.57828.8398.4763.07518.846227.26931.46917.258102.8082.82123.69 61MTCI7372199617.4995.0810.2207.4990.4794.7290.22906.3710.4760.183 623AUGRE73721996127.10738.54116.3314.46427.1072.84752.3918.6785.29532.2793.0081.35 63SSAXQ7372199613510.7041554.93417.199333.883510.704517.1211666.10819.088452.1144141.688639.08389.964 64IDNX73731996-14.2030.7230.162.580.0340.2155.4651.2580.1182.5790.0370.133 653FNIX73731996-145.6056.92510.68629.3861.2082.77721.9891.6625.61216.291.3920.129 66SUMC80511996-1488.1183326.8210.8195499.4431995.727112.681435.5212742.28210.0574899.7173514.0950 673UNHC8051 1996130.20457.2957.95813. 45730.2040.38660.6746.276 19.56338.2051.2142.858 68AHCI80821996-140.8488.823056.2792.9382.67846.42112.014045.5661.2664.348 69IHHI808219961142.854229.18629.95713.452142.8541.025236.92328.71312.222139.0343.0942.254 703VLFIQ20301997-1388.29488.515129.049348.84412.23736.024365.75175.37681.923636.298.42913.159 Figure 18 Continued

PAGE 153

141 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 71IHF20301997127.0845.4579.5646.80127.081.8554.63110.8138.56647.87712.7462.454 72RTRO23301997-1273.83468.82796.844275.8047.31938.837143.77520.4646.665256.61135.3167.085 73JLNY23301997-1361510582214298253145630762473482511238244252 743SOCNQ23901997-1118.22846.3460186.9198.2594.871114.64418.310146.92191.0814.134 753WSPT23901997-1368.646161.736118.995591.8375.82819.513626.84194.716229.051005.888.65330.199 76WILK28421997-1260.58153.26111.412163.743023.829246.82746.66617.098140.159010.359 773KYZN28421997146.51114.7223.2110.53446.512.583107.3591.16814.81227.02723.2720.169 78ORXR306019971107.358109.17721.0730107.35817.826105.5120.3760113.04725.7382.178 793FHCO30601997117.32119.3023.2932.35917.3210.81118.7373.5682.35216.190.9490.15 80STMT33501997-171.22719.8513.5960.1030.2843.901259.3147649.068905.9841.17344.7 81WIRE335019971318.409349.34534.39151.485318.40966.27365.26933.51144.975259.84620.87848.188 82ACU3420199719.70721.113.48709.7073.33915.7932.03705.2081.1150.09 833LOCKE342019971714.19582.59426.4750714.19528.654141.17122.230731.88531.0040 84NEXR35711997-1553.496524.57201347.70254.7715.659398.496317.9360915.80933.0032.294 85NWRE3571199716.0937.1191.9340.6766.0931.6286.4481.9740.676.6052.1610.2 86COMS35761997152.95343.5047.8067.94152.95319.5731.2565.3521.94536.89610.0590.132 87CSCO357619971191.191198.19943.34847.181191.1916.68481.93715.63534.677201.6464.7234.364 88MEDP35771997-114.3032.81509.3440.1330.60513.63.27907.7170.0780.038 89CYLK35771997-1265.4044.93519.4471.5340.55926.3916.4612.40216.5680.8230.797 903RGFX35771997-199.66215.7030687.2428.0980107.35913.6670673.4678.3790 913AEXCA357719971608.504335.78419.9670608.50418.594362.92718.8390676.11636.1227.824 923GRDC357919971177.474274.86247.11655.619177.4748.018267.34647.90459.155180.5617.8977.288 93SORT357919971259.57521.04186.68960.561259.5758.93222.24188.12866.507256.52706.709 944360B36691997-11830.778365.463391.581924.2795.928220.0972233.576537.786461.9813892.711705.084174.039 95DETC3669199712363.1421112.711156.734173.612363.14244.1731128.683137.695175.9722121.35745.34662.062 96ISNR367419971141.998125.82630.1050141.99816.102144.38539.8680198.7621.5454.829 97LOGC3674199712627.3681405.305438.2863.762627.36844.3371866.426370.47263.3692429.91430.06677.943 98MODI37141997-1282.33199.49527.534206.99553.1686.166300.52855.31678.311243.80956.7484.93 99BDTTZ37141997147.57945.9088.5469.3147.57922.8722.2841.7262.4712.6955.6592.748 100DAIO38251997-11676.838184.312196.7332446.569.676247.6511808.77187.819223.2062542.44578.939166.422 101TVL.138251997-158518103627543626243466374457108.1999365.6825074.88361686.4734182.1632927.762 1023STRN38291997-110.4781.7940.1616.4990.9560.2619.0893.1190.20111.7754.1960.485 103MDLG3829199711619.3551798.456325.039278.9481619.355294.5721800307.732218.031083.9633360 104CTU38421997-14816.6573968.55578.3491689.56129.25590.4066574.986124.6754.8441394.329108.95390.945 105CRSS384219971336.186206.1794.1542.3336.1865.303264.4194.2913.614351.7373.92432.312 Figure 18 Continued

PAGE 154

142 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 1063KTTEQ452219971409.55728.093317.020409.55725.28132.205342.6870449.337229.9077.244 107OLG452219971317.945510.14667.011112.451317.9456.355438.22475.81689.748310.3937.1715.323 108ITSW45811997-118.2264.5232.27349.9150.3750.8819.95311.43214.63957.228069.369 109HGC.45811997-1340.44959.51542.4988701.01644.684412.025837.4537239.68405896.245185292.284 110PTEK48991997-1372.96357.69263.574352.1291.11113.071372.72851.23276.449339.6791.25416.315 111SMTKQ48991997114.8273.0481.517014.82705.5592.016010.7810.2310.094 112DANKY50401997-1102.14425.3376.89156.5840.1952.17965.3497.8870.05688.0670.0544.134 113IKN50401997130.444.5520.2550.45130.40.65244.0880.2610.1726.1730.6120.118 114MTLM50931997186.6736.0499.59315.83386.673.68451.4848.04621.69160.1650.0034.05 115NR5093199719285899514021190928576761815575859622158274 116MCD58121997-1126.34323.50538.15494.0444.4752.832138.04520.91637.76292.8124.673.354 117YUM581219971261.749310.60267.74147.27261.74912.432207.87740.19725.6169.78611.3723.42 118VTA633119971148.419360.74253.5550148.4190.988371.11553.7140154.1861.1741.75 119HSB.6331199711093.331654.34256.13869.8691093.3311.278836.62390.10164.0271439.5993.79729.821 120RST70111997-148.8761.69812.64118.4640.1820.21598.7250.4047.34644.6041.090.73 121MCS7011199715451.9846440.1711170.401254.6775451.984617.1928458.7771297.867361.9868916.7051096.153414.843 122ISLI73721997-11.4620.20.2094.2560.1121.4920.3570.0280.0564.2320.5040.735 123FDPC73721997-1910.27294.021103.716292.18335.0035.8541084.633110.584120.705334.00940.2218.624 124CYBR.73721997-154.26216.8655.763600.39213.71.494296.18932.6636.4991128.20721.20347.806 1253SOFT7372199711989.321247.4481089.22701989.32111.8081217.0131028.21801899.806102.17657.978 126PEGA7372199711246.659460.98676.62201246.6594.932482.26173.28301335.3477.3080 1273PTUS73721997127.53843.7915.3437.16527.5386.69130.4273.7925.7324.7366.5861.447 1283QDEK7372199711773.529543.15185.0627.6021773.5290940.388151.10211.8122636.8810451.116 129FILE73721997197.04415.87414.90215.43997.042.352504.37920.31123.825118.4063.16113.032 130TEALQ73721997181.67192.86113.24616.3881.6710.04573.9111.78919.766116.0013.353.554 131CORL737219971188.576139.89733.20319.947188.57610.224275.0684.8410.176151.3950.7360.028 132HYBR73731997120.95429.947.1331.38820.9541.43542.42713.693.53141.6416.0820.284 1333DSYS73731997120.54432.6794.9754.15620.5440.38527.3516.2665.11917.9781.1720.701 134QSII737319971119.006377.4127.33344.477119.0065.636318.45131.73841.657135.8397.0150.896 135PSCDQ76001997-1494.35100597.39420.51522.051592.68620.6070766.31152.90334.276 136ISER76001997-128931.95630.44202.440404.317605.41814.936047.8387595101.3111287.346023.54042 137UTLV78121997-146.2845.6788.15857.7361.6431.19735.3388.7184.44240.0894.4050.482 138DCPI781219971163.3337.0893.3731.648163.3333.27811.4512.5761.833122.7173.4012.177 1393JACK79901997-1603.606135.6090357.95422.68372.334496.72293.5220309.787.0539.058 140ELSO799019971759.0241040.418162.177152.674759.02461.1261111.447182.91178.949915.73982.55290.86 Figure 18 Continued

PAGE 155

143 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 141VSR87111997-1157.95750.1820.575140.33.0036.473131.328.761066.6148.7792.103 142QTEC87111997-1661.98373.40952.0361070.248125.77540.9221020.993135.048131.3072348.63943.344143.746 143MRSA225319981308.08270.5439.0210308.08217.75661.4822.850278.25115.8152.199 144SIAYQ22531998131.16439.32512.115.79831.1641.12139.45512.6145.46530.8851.3870.854 145ZQK23201998-1224.5823.541.212270.58310.2542.835246.82718.8390176.3310.1864.935 146TNFI23201998159.392201.74537.986059.3921.639158.69334.769052.732.3990.572 147JH27801998-167.72511.3578.01649.1495.4872.34355.0828.2314.72529.3390.3311.583 1483DAYR27801998-1894.709167.347141.898729.79672.65657.629894.534164.555153.006789.457100.60784.015 149DLI28441998-119.9021.6510.24129.4260.2050.12523.121.8960.14922.4191.6540.192 150STYL28441998116624.87411205.0911200.63874.616624.87408252.523745.13158.58317356.8201258.965 151HIPC28601998118.03819.1371.8071.06118.0382.782.886258.8957.314791.291046.827 152NZYM2860199815.7775.6610.6734.0115.7770.1425.7210.5533.5875.0990.110.012 153SCNYB30211998-1979.5153.2176.91179129.536.31025.7153.5174.41082.1239.234.4 154DECK302119981108009673232901080067994752221011064542291 155HAVA30601998119.90933.6088.8324.97319.9091.28813.87324.199069.5380.5531.573 156SFSK30601998156656625885707566517808402126796584371318496 157RCKY31401998-1171.39939.075.597140.90403.269196.5665.5385.168125.73504.625 158CAND31401998116850.8166168.432899.314016850.81612315.7266853.652976.638017889.0913159.653185.406 159ACRN34201998149.08353.62719.124049.0831.2561.54229.14060.7810.9896.882 160LCUT3420199811900.391119.584243.29721.2831900.3957.5512089.444115.958114.1282429.91458.798319.234 161ADPC35401998-1129.74429.55933.361156.8985.68516.291147.7617.06421.186131.4950.789.56 162DVLGQ35401998-12505.1218.411.5570.31624.328092489.6619.92814.5 163BEHP35671998180.7584.5420.812080.7580.121278.907242.1590752.65358.52618.115 164GNCI35671998176.55773.72510.41217.75176.55734.64434.8725.6958.86957.3610.4210.619 165XIRC35761998-1575.04131.9040427.58699.36566.657603.606135.6090357.95422.68372.334 166DGII357619981856.2471426.288282.423312.13856.24734.2881169.511264.233240.488823.9862.20329.39 167TLXN35781998-1656.527167.59299.438487.70737.63432.116539.852154.03691.799495.20536.24519.355 168HYC35781998131.2599.3440.973031.25925.9097.6810.575017.9830.5490.025 169SORT35791998-1365.75175.37681.923636.298.42913.159479.59465.296184.387208.293108.95322.077 170IDN3579199812178.9413348.986628.052482.6562178.941615.9022897.22571.47356.1391905.14296.667191.054 171TNB36401998-139.70915.174039.875.7872.01831.5324.671013.7230.512.739 172RYC36401998196.59888.69917.2547.1196.5981.0198.09923.94132.57389.3331.769.675 173CUBE3663199814.6929.9412.2071.5284.6920.0399.8882.8981.8285.7440.0370.438 1743PCOM36631998112615.3335060.605332.553103.07812615.3331181.4734794.01707.731126.93818016.4551268.5691508.085 175ALTR36741998-141.80519.4674.599100.2653.9581.48295.79721.1035.798150.16574.8151.59 Figure 18 Continued

PAGE 156

144 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 176WFR36741998-156.0641.1830248.7583.2791.092135.52196.4840874.6427.3251.747 177XLNX36741998-11512135.8437.51124.1112.345.42171.1222.4773.13544.911.5197.2 178ADI3674199813.9073.4590.5331.0373.90703.8460.4371.2286.01102.497 1793SAFY37141998-1667.6557.91587.031285.88943.0511.577743.32364.74185.707308.68649.71314.548 180NER37141998-12964182501222620462531161854011054185230 181ETN371419981317.249168.91331.3221.112317.24989.448164.55338.121.093354.504102.9959.031 182TRW.13714199815546.5565877.8591680.683662.6065546.556153.0649020.9291887.926736.0196025.218199.964114.534 183WNC37151998-11019.174263.016204.168817.1977.69883.5421049.495243.642191.905857.7328.73572.065 184FTHR37151998189.73374.33127.23927.10489.7332.742439.131146.29564.764417.7871.4644.671 185IFRS38251998-181.81227.4563.94938.7350.4870.65153.5313.5584.3927.8080.4720.694 186GEN38251998-15128.4331436.444442.2075323.886159.622193.2385628.6631616.503431.8375748.796149.4212.169 187TLGD38251998-12634710878.4011283955025.9123.526818.91834.7047445.714776.236.9 188STCO38251998149.22339.4239.65312.92649.2231.21930.667.88714.43331.9825.1170.642 189AH38421998-11565.238294.353108.1331249.30995.48553.9451692.792283.326100.9221166.66573.55341.31 190SLS38421998-11163.92810198.57237.52317020.231679.822151625.88312381.731213.73920280.91377860.809 191ASE38441998-178.85112.6848.88741.2480.6362.9687.17219.9617.284221.55926.9311.447 1923SCHK38441998155253921110552541515389089041473436 193CMED38451998-1244.49839.7680392.41712.61755.697115.89822.9170240.13619.5781.091 194DYNT38451998127.49312.5197.30412.39927.4931.5589.5634.64312.53623.5990.9690.925 195NMTX.1384519981336.039425.76133.55169.722336.03913.994424.35430.58682.977344.0611.1326.971 196MGCC3845199813235.565573.223119.68397.0313235.56568.984909.388192.91473.9254559.53871.024460.251 1973IGTI3845199816260.3635492.915636.404126.0526260.363119.9053842.3668.6123.16631.42.468692.4 198AVID38611998-12249.936131.964483.1112343.062183.1110.0621671.256301.887418.9021985.45511.61224.727 199IMAX38611998-15055.42403241620202.1679.81708.2538415561112151491141277 200GMTC39441998-142.24812.4428.13584.4613.8783.62651.4192.09110.40163.5380.135.44 201JUST39441998-1307.98740.11601596.9115.3840419.41842.60901907.56314.6240 202OAR3944199816.69832.2661.1960.3766.6980.07735.3382.5830.6158.9090.0120.433 2033DSIT39441998145130.66827362.1426150.954256.62645130.668030183.3357943.485358.24659847.61405680.97 2043TRGP42101998-1193.20633.51722.843162.4598.8878.224190.18631.93123.743166.8686.50510.019 205DDN42101998148.471120.99929.931.60848.4713.80781.81717.3891.36843.9025.9560.23 206ITSW45811998-159.75417.897035.8771.3682.08885.7717.119036.81.2232.739 2073LYNGD45811998-1102.22719.131.79898.0080.72425.489119.66727.9753.187118.4430.35410.521 208BTY48131998118.2580.8290.399018.2580.6280.0430.065010.1320.0290.032 209BLS481319981953331878122559095333414661999423330095088386900 210MCN49321998-15.460.8020.343.2260.1470.155.8780.8320.3463.0020.1770.207 Figure 18 Continued

PAGE 157

145 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 211UGI493219981319.10580.98415.08811.062319.10537.546102.6939.8110.17155.0831.2587.673 2123PDGE49551998-123.4082.35829.49541.1280.290.41212.8450.00885.7077.7488.77911.189 213BIKO49551998-1959713648131710212626798687757617139151881.009747 214CDSC50451998-14.7290.22906.3710.4760.1838.8470.01608.650.1380.242 215CHSWQ50451998-192.08315.21311.35848.9830.0051.60882.44913.93710.44657.6010.0281.928 216TECD50451998-11957.58149.13584.546615.57460.52722.0441953.392127.76778.905507.65213.87750.683 2173TRNT50451998149.82353.7880.6230.38249.8231.65555.90.4730.44549.7481.5052.117 218GFIHQ50471998-19600.6544.755315.0694152.544822.156314.8045937.896501.445120.272986.857131.03199.493 219PDCO504719981139.58713.6891.320139.58715.64911.2871.948078.77514.130.954 220MI.150651998117.01341.748.6960.42617.0130.50733.3022.4260.6087.6680.7720.748 221CLST5065199811001.10776.513648.0360.6721001.107227.284.988727.04272.4641081.354147.5360 222MCK51221998-11490.701245.538178.1072799.997167.602329.0922089.444275.62243.8963206.605301.189205.172 223CAH51221998124.6356.3467.1890.62524.638.0837.5874.3380.32516.4455.3740.134 224POCC51711998-116.8510.081.9046.0681.3830.54419.2230.1641.944.4890.3340.079 2253EVSI51711998-1567.815165.8680649.49416.42620.136526.867124.670523.40817.35412.926 226BMHC52111998-19172.2051750.8271887.285244.355224.36786.92712814.011629.5661917.0445864.148294.637125.421 227WIKSQ52111998118.71336.8893.8214.90918.7130.04156.75420.2122.10188.8052.0484.93 228FFPM55001998-1172.1937.86614.567109.4534.4480.969104.2223.0417.06676.9831.940.351 229SCHA55001998-144.36120.3190127.5211.7014.48861.76943.4780139.269.5057.52 230RINO590019981156.10253.98965.6450156.1024.94858.54945.6760141.0253.6571.481 231OFLD590019981161.697178.56954.94822.241161.6975.785191.3250.60720.355148.5633.85410.083 232FOOT60201998-154.59811.1297.22170.1422.467085.34813.31710.17480.4622.5751.096 233STL60201998-164.94511.0715.813396.92616.28210.7100.16417.34312.76672.583167.3978.656 234CBSS60201998-1203.2724.8310621.8843.01411.911588.60854.42601469.82116.72949.548 235ZION60201998-1167.5491463.83511.2211982.8311060.576468.381171.1311532.1946.2462087.09414617.138 236NCBM602019981260.144197.09791.8810260.14424.973218.23687.8130309.92630.0225.615 237BLMT6020199817.61416.4491.8531.4247.6141.61912.4381.4791.3316.2011.40.254 238UBMT602019981493.348555.8687.8434.199493.3489.591712.965.49635.251537.3677.397104.052 239CWBC6020199811583.1851808.639390.506383.7241583.18501696.944370.286341.2731448.691095.562 240CAT161591998-114.278.873.36843.1191.5630.64312.4181.4335.22415.420.4953.907 2413FNVG6159199812.7620.8980.2680.4872.7620.09610.6470.8640.3036.9630.0860.321 242NDB621119981104.766339.40758.5460104.7662.49365.07655.9480121.2813.1863.584 243ZCOI6211199817604.54112959.252635.5952972.6617604.541250.88810127.6041458.5531403.0755358.984326.02464.355 2443SFGD6324199817.6348.7162.02307.6340.11519.5724.894058.51900.608 245AMIC632419981556.887786.43476.73320.736556.88735.291864.11615.82219.577704.81650.68411.877 Figure 18 Continued

PAGE 158

146 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 246ALFA63311998136.0619.97911.0960.33836.0614.30730.5579.1260.33532.66913.261.055 247VTA633119981504.641760.553200.283103.751504.64131.704653.098162.603114.773472.9086.30327.908 248FNF63611998-115273.6650.33190.26686.2261.2502.318098.3699.33445.57275.4359.5493.5 249STC63611998126.50227.5212.9835.05926.5020.15221.6712.4555.43815.4360.4580.193 2503HDSGE67941998-110.4184.9534.48347.7080.4340.88814.0371.3436.97722.5640.0220.384 251BLM67941998-176.58420.4352.586133.8142.4364.784137.60533.512.585166.6254.4318.744 2523POMH67941998-12249.936131.964483.1112343.062183.1110.0621671.256301.887418.9021985.45511.61224.727 253CD67941998-18388.3391074.941561.8637074.559234.789270.5958169.6391338.081594.3086675.93232.605233.268 254OLS736319981141.619283.2879.55762.136141.6190.192404.72914.66385.967235.0381.26320.96 255MAN73631998111716.4897249.6892438.1613729.9111716.48966.9416937.7272525.9593263.28411183.6410309.955 256YHOO73701998-15.92861.1350.01181.73349.74810.5196.72476.713092.4976.9093.107 257IMRS73701998-184.8645.3916.32983.250.338.35996.736.9995.03880.6130.0262.183 258BBOX73701998-12103.7377.7386.12580.5976.176.22289.4343.7392.12430.895.380.7 259DTLN73701998-12549.808470.532537.4012652.68655.942133.0831756.083360.671298.8522087.76334.57869.369 2603ATHMQ73701998-16710.7352602.8152202.2956649.043246.44752.4427748.434062.8262704.4448985.455333.14360.38 261LCOS.73701998167.95254.88812.1623.96667.95211.64353.35612.2626.40768.4379.7233.377 262CMGI737019981530.815631.115182.07427.575530.81575.8515793164.9232472.4375748.796149.470.2 263EDGW7370199811279.407569.237129.933116.2661279.40728.348631.801134.176109.2911311.39510.99134.095 264PQE7370199814473.763486.8162193.8384473.763168.19719.0645957.8419.3182675.5715335.56191.1140 265CSRE73721998-121.0515.313021.7920.0480.56414.3032.81509.3440.1330.605 266MTMS73721998-131.20913.8161.13523.2914.0850.5854.726034.37179.94826.9311.146 267ADSK73721998-120.973.101058.0163.8351.23518.6371.792031.0562.7960.331 2683VOXW73721998-192.95519.02627.52881.65814.4420.67798.13219.25220.95677.69715.4551.15 2693ASFT73721998-1194.5461.69339.33481.8672.0935.909167.9591.57630.94165.581.9920.847 270PRGN73721998-1570.035122.40461.942497.234182.4737.569805.274103.7660.443667.73620.21619.022 271CGFW73721998-148.04510.6580780.631298.06316.793336.95570.53209104.2798005.27660.809 272EAII73721998-110064.646132.4011046.3662531.62336.156165.69812494.023189.3011183.6812995.34258.798361.024 2733OBJS73721998-18902150713739536266211899514021190928576201 274PEGA73721998-18460134359611250467103194681843845150289341373 275SEGU73721998-112703.4692385.911256.22922715.198824.7411651.48913126.921907.287116.20722681.424757.5381326.684 276OMKT73721998-11965345140255105729382817154893.02903029922012954 277INFM73721998-16096.29545587.361062624963222785.66577.24156039.179482590222.377144431184 2783AUGRE737219981372.037278.3156.5020372.0375.707247.97742.6470376.9748.8911.337 279AMSWA7372199811709.453234.253001709.45317.576244.7126.9701718.1295.7450 280GAEX7372199814175.538763.294115.55904175.538189.623999.143155.93704880.443324.0750 Figure 18 Continued

PAGE 159

147 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 281INGR73721998146.76863.46222.415046.7689.32872.51521.753068.0329.8383.022 282SSNC737219981109.599130.60937.6070109.5993.62767.06925.595096.7231.4642.854 2833SOFT73721998140.91631.2169.9461.32840.9166.12533.81612.4880.77240.2186.3340.521 284INLQ7372199818102571603513128102516024575630508177564144431206.128 285MNS73721998146.2611.0115.72513.93346.2611.9143.0986.8478.76645.87414.3424.18 2863TRDXQ7372199814999.4814745.748619.53621.0354999.48173.0892529.0391402678.6096025.21854.5660 287WALL737219981247.72361.04831.628144.579247.728.406155.34425.88478.64220.46357.958.373 288FILE73721998116996.1119633.962738.076220.69916996.11108481.86906.301195.45118820.3101206.128 2893FLXI7372199814902153073732027490213517838.8486.52462.65906.791.1696.3 290FFTI73731998-115.0850.3662.1188.0370.1040.19920.7441.522.3728.8070.0740.231 291BI73731998-12.9160.6110.9478.3390.2440.0255.4511.1380.9267.5590.1660.059 292LVLT73731998-1605.03345.45869.497285.7897.63423.964669.13546.97279.161310.7429.43427.23 293UIS73731998-1537.15978.4880442.91640.81330.902543.10759.3110459.41832.73120.861 294DSLGF73731998-123123462943139410102852122522451774514345315646200 295CSPI737319981636.854542.62787.5660636.85413.226612.34677.57770.358640.2110.67614.159 296AZTC7373199810.7410.6430.17200.7410.016267.3464.5360732.0316.7290 297HX73731998142.56281.1912.1775.43542.5620.40853.5299.432.85520.2070.0240.673 298IUSA73741998-11222.434108.132220.5651262.05340.97613.5631699.6141.422235.731446.197332.84330.968 299INOC73741998168.1885.0640.54068.18805.7832.063073.46600.178 300LTBG73741998142.84764.0250.066042.8471.94224501.0672392482522058.9819851206.128 301CSGS7374199811399.21451.8218.7278.51399.2791427.6209.8298.61454.574.348.6 302EQUUS79481998-15.9740.3180.08727.4490.4840.0815.7110.2050.13322.4840.5480.037 303FGRD7948199812956.9065076.909215.7687.3122956.9063.0566052.684317.58797.454346.8424.201311.978 304CCRIQ7990 1998-133.68613.3980.726 45.94723.5111.276 10.6741.66905.8841.757-0.043 305ELSO79901998149.86546.2778.18913.20249.8650.33961.11111.217.33666.2020.3682.273 306CTEN80511998-15.7111.487018.0800.0822.393.087066.1500.201 307NHC8051199814595.5216070.568239.421295.8784595.521218.1945911.122267.0621306.6674579.356201.296142.355 308LABS80711998-13.4030.7581.52515.87600.2792.9280.3031.87410.69900.116 309LABS.80711998-1920.582249.792240.1621218.44886.0343.0671030.663324.848277.6011489.311159.75135.261 310AMS80901998-19.0814.788.1244.3610.1645.99424.1917.1525.47627.5581.064.381 311ASGR809019981261.749310.60267.74147.27261.74912.432207.87740.19725.6169.78611.3723.42 312NRES87111998-11013.1831589.7303539.34484.24929.4871025.0561431.09203048.03432.94917.138 313TTEK8711199811153.421148.136236.84901153.42192.288284.091243.14501189.86485.7010 314IBP20111999-1704.44168.513105.45512.3195.7621.61749.201158.28119.498559.2574.17227.62 3151970B20111999151.47745.9934.0866.97451.4771.93828.6382.6268.03642.1343.2263.287 Figure 18 Continued

PAGE 160

148 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 316SEB2011199911014.3041066.451256.47718.451014.30433.9231671.925338.32838.2071356.04240.72229.486 317HRL2011199911176.256232.76191.016120.9391176.25624.429679.989114.839102.7481277.1655.796188.019 318WEAR23201999-1329.807204.07602398.70312.880332.46197.22602431.1539.2130 319SPOR23201999-14584.97639383.70522.557982.6993751.337261.27392.07364144.64434.993169.93222012954 320PLANQ25221999-1619.143167.9660602.73878.09110.3431030.663275.6201511.4853.3295.135 321MITY252219991178.945145.6595.6880178.9453.141181.32216.3850333.3351.1391.4 322BOTX26701999111.42512.2684.493011.4251.24811.0283.36809.1472.1560.252 323WYNT267019991661.2071804.102295.572246.671661.2078.7182007.102299.108195.791643.6877.63112.542 324BPIE26731999-13.74911528.99970.530.130.045261.217154226.9273445.58916.7050209.874 325EPTG26731999133.08832.8177.6380.77433.0880.29140.93410.1540.78739.6670.4063.287 326SG2750199916669.5725649.56138.0780.5646669.57218.3115555.17471.28370.5618997.14131.435997.843 3273MAGRQ2750199912746.73961.5533.75132746.7148.94366.8550.2546.33043.3143.9180.9 328CIMA28341999-196.94927.3798.78690.6589.7417.719109.67534.80114.945109.8979.6454.893 329NSTK28341999-18.48997.8043.468115.9970.216.4129.463109.6113.122131.0341.49619.777 330DEX28341999-11252.904173.46189.43766.96232.41422.4821328.094191.54698.01813.577130.97915.582 331ZONE283419991345.376464.8277.38712.796345.37613.213393.62988.56713.557336.46611.8327.587 332ALO28341999119121.11829308.227432.044874.96419121.1181893.68226755.233439.871905.50819326.1211631.7251210.845 333IG2834199912060.5999807.363612.521243.1532060.59976.20911645.021623.9611570.5042458.56772.98616.619 334IMCL28361999111.6614.1411.2750.17411.6610.2012.6030.3130.08515.440.1650.143 335GERN283619991144.007507.221.59144.0070.27141.8644.7891.96151.3950.2373.565 336ATISZ283619991173.408215.87450.35959.608173.4082.053246.04757.76864.044185.9214.5557.074 337SERO2836199913799.374676.734200.06298.6743799.3741280.5411007.795164.923131.0044674.249076.746 338BNET287019991511.3396.967.534.3511.340.8419.769.841.3536.836.737.7 3393IGNE287019991170.959421.4965.11256.021170.9599.701587.3859.11868.105231.69610.74853.12 340GSE30811999-113.3923.0582.77219.270.3192.8223.9198.6511.381189.4620.0417.47 3413SWTX308119991469.07741.323289.0073.379469.07777.28640.072339.5332.101458.67610.9890.826 342NWSW33121999-129.178270.1480.922420.1198.1165.05530.228283.4370.476424.591.12.294 343RESC331219991270.759212.46831.3418.461270.7596.775240.4233.9819.58875.9513.7282.086 344CAST33201999-127.5315.67113.28332.4670.670.78823.1394.7959.65926.6220.3790.679 345AHNCQ33201999116 85.5853357.757266.059270.239 1685.585138.3953675.132307.732 281.4041641.94145.267100.125 346CMI.335311999-11645.948261.682265.836711.8132904.595112.6812081.733303.298356.9467134.8471288.884213.351 347GEHL3531199918807.039568.98594708807.0398775.291590.8167943.48508610.4978588.2450 348HIRI35591999-11073.09228.46304.91058.9282.2660.5441836.871361.774519.1893405.5171859.37753.686 349TDSC355919991143.87105.54122.650143.878.31791.43220.6860134.71359.581.739 350NXWXQ35761999-11343.817262.484194.926779.3927.02439.9341359.676271.677234.661835.67421.70435.662 Figure 18 Continued

PAGE 161

149 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 351ZOOXQ3576199911540.2603.4150.201540.2192579.2172.802144702.620.8 3523COPY35771999-100.074069.8940.520.7990.1920.05066.8940.520.263 353DMTI35771999-1132.61921.0622.96989.6122.4052.366107.88810.54914.27546.1680.6751.518 354MTLG35771999-12362.8526.3440.73491.7930.7201.51756.1438239.63273.5764.6155.9 3553SOCR357719991830.94962.98574.7540.492830.9490.36963.014608.4340.176936.33220.6861.172 356MXIP357719991126.35385.53615.52123.392126.353064454.56527681.25101.361686.4731881.0091184 3573MITK3577199913085.91772.4417.2247.73085.9936.81711.9380.7253.42993.5210.796.4 358TLXN35781999-1134.6457.1571.033124.3083.1837.742385.75820.3760.007220.11611.249102.453 359CMIV35781999-1465.87148.688108.178390.53917.10626.124388.29488.515129.049348.84412.23736.024 360HYC35781999-15681.7594.2610.62811.3499.9193.55957.8519.5612.53192.6530.8155.3 361ASPE357819991767.651252.904173.46189.43767.6533.0251328.094191.54698.01695.28334.6215.582 362SATC36211999-173.175622.0591.312859.14362.070.73656.364543.1411.284747.91110.489104.052 363EXX.A36211999119.7850.5893.4776.37119.7850.86201.024018.20.0150 364TNB36401999-11052.628254.17704302.806251.116469.7891806.6498.708418.9023492.061324.075224 365HUB.B364019991131.002316.3711.6850131.0021.522347.52412.4140150.8961.67229.393 366ACRO36631999-12236.2212.655.9996.599.41051468.7142.139.678879.352.4 367SNR3663199911286.1061657.51192.99340.8181286.10666.9731778.99170.086381.0221391.21173.359147.463 368RMTR36741999-1536.721113.729105.488383.6970.9294.46762.508178.86279.16125.516499.916.619 3693HDTC36741999175.2649.9980.531.37275.2641.956140.0321.8512.828113.4666.89712.876 370MFCO367919991429.64226.468383.5260429.6422.04991.1031036.67601160.6055.2793.906 371EMA36791999123.96978.47612.1350.06923.9693.436117.489503.42487.12404.594152.659.038 372WGO37161999-15.5592.016010.7810.2310.094117.489127.64418.107187.5390.1660.758 373COA37161999120.71615.6081.3261.81420.7160.13620.731.4272.77924.890.0991.567 374ATK37601999-1525.20343.95917.378548.71913.26445.804384.44833.40414.81406.4342.52737.052 375ORB376019991349.646146.91846.9970349.64625.636184.92937.9810439.62184.2144.527 376FLIR38121999-127.5865.4663.58574.0143.5754.42854.98315.1415.29305.42151.6369.351 377NOC38121999-1184.92937.9810439.62184.2144.52719.90252.5890413.093.329193.5 378EDO38121999179.22272.92513.7416.40479.2224.06267.38812.4348.38477.3723.0140.383 379RTN381219991490.091190.35554.5218.747490.0912.168203.83576.53731.141538.2378.81438.91 380EW38421999-12.1390.5090.0575.6820.0890.4076.8322.5370.0516.6051.2930.6 381BMET38421999-11311.81510.9375.46263.7917.31095.31488.61475.713.36109.7164.11141.1 382INVN38441999-119.2787.903079.2520.3433.335228.689.0380403.174183.4879.661 383DGTC384419991271.693316.447154.4120.666271.69313.28843.661294.6831.901532.3753.7297.849 384SHFL39901999-134.753200.9771.734438.2830.0890.73227.543155.4321.845315.76735.3161.464 385MIKN39901999-1209.33698.857342.853713.67122.5599.456335.456195.598456.0091063.48740.07414.728 Figure 18 Continued

PAGE 162

150 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 386YRKG399019991174.721184.49752.6310174.7214.103225.377.4180321.0826.82132.619 387ZSCO399019991107.901117.35210.3141.491107.9010.856143.64417.8081.764132.5831.1829.395 3883PERF399019991139.453130.40516.16116.792139.4531.53101.53610.51812.491180.8281.3011.317 389IGCA399019991509896811497350981998468155684531200460 390CEXP421319991186.91118.22846.3460186.9198.259114.64418.310146.92191.0814.134 391IRNE42131999124627.17424185.2434172.0874266.89624627.1743383.75722027.0683459.3563396.21221551.1822979.8261573.288 392PAA42201999-118.5262.5162.69723.2531.4461.87810.689134.412104.4311083.96322.854143.746 393IRM422019991114.31664.01719.90240.056114.3169.15894.50920.35222.942114.55920.261.146 394HPAC45811999-1107.75824.55330.123112.63302.882111.41420.21230.891108.86704.935 395ITSW45811999173.52597.93420.0518.23873.52512.8598.40727.56123.89682.25113.5841.977 396CMLS48321999-1319.96966.52667.244364.9215.60321.602336.95680.59465.805359.2086.64913.535 397CXR48321999155.386163.57315.07215.0755.3861.05174.06321.3788.6662.1492.6270.511 3983POWR499119991326.233108.35510.8960.314326.23342.14139.16211.9550.361370.34538.1374.549 399USEY499119991229.995139.77224.33212.635229.9953.629245.30740.23938.616918.014593.19411.657 400ANIC50631999-11255.304171.931200.3821285.326162.61167.7131583.696243.643218.031312.84827.485116.933 401HMSI506319991525.293292.64215.7563.125525.29311.179255.74412.9642.801520.0111.5033.979 402PBSI51101999-112.0861.6924.73828.9790.5070.50511.3122.2413.72722.9160.4010.908 4033BCTI51101999-198.7527.63637.67165.2277.5761.921117.00813.03241.939181.1314.7377.765 404MDRX51221999-1903.937204.4110781.62592.3236.362867.469199.3030696.60477.17638.91 4059956B512219991723.6511200.945180.929117.058723.65161.9211261.171150.015117.681678.98455.54546.778 406DBRN56211999-113761200820241001473276083963360 407CTR5621199913865.576648.817417.203865.5763689.7731437.81878.159.8638892.8167.1 408GES56511999-12165.453565.604736.0192747.21719.0983823483.831014.221533.8910161.04081.64 409SMRT56511999197.62782.24228.077097.62763.592128.60742.3324.574245.18411.9581.942 410LECH57001999-1563.684773.46201675.68529.8744.182553.496524.57201347.70254.7715.659 411FRAE57001999147.692108.91410.55318.08247.6920.208116.3292.75921.18559.50911.9680.966 412GGUY57311999-1341.92838.13352.828236.968.79915.692332.8828.02448.835256.7278.85520.239 413TWTR57311999-1264.50832.6842.5851543.61658.087115.062462.36813.89416.1322005.12777.06244.743 414ITN5731199913652.6712195.293597.64410.7813652.671286.3542017.918503.42456.5663597.532268.82764.191 415ULTE573119991361.082316.66656.81644.548361.0822.402287.36256.24541.313359.4153.4474.754 416WHENQ57351999-1212.0547.51268.283383.17148.3549.202118.58324.88250.01248.4453.3018.23 417HAST573519991688.903593.04435.61453.194688.90360.947310.75417.248104.649708.49526.9696.636 418AUBN60201999-1219.472120.75869.91492.150.37828.782118.65969.9442.723261.51210.9111.091 419NBAK60201999180.58964.6820.192080.58912.69721.6969.168080.0022.1351.015 420CEBK602019991167.788132.85416.9859.838167.7881.491101.64916.1384.355147.2281.5444.699 Figure 18 Continued

PAGE 163

151 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 421SRCE6020199915996156973448615599612111.2579337750146350322012954 422FFWC60351999-143.9597.8560.95237.7840.0243.30548.9347.2161.07340.2990.2340.568 423FFBZ60351999-11319.331149.7650416.0244.00810.3191335.66140.4510399.2621.4746.538 4243AMMBQ61621999-175.2638.6059.545152.1470.6368.74738.3623.7153.874122.0810.3771.302 425ICII61621999-111408.8483.570.518241.5608.52111.212421.4609.477.319784.4538.31879.3 426GCAP62111999-11369.471147.348213.8771503.17467.81896.9661869.082154.501145.0461617.01553.7963.123 427PLCC621119991349.646146.91846.9970349.64625.636184.92937.9810439.62184.2144.527 428LGAM62821999-1415.26382.698118.214716.722155.48620.998484.79776.09944.552578.047.8524.608 429AMPH62821999178.3765.54655.7720.96778.3760.7396.16758.7910.02280.4482.25510.386 430CI63241999-1214.52531.257121.902215.4758.1165.717228.24827.972105.195192.910.0453.473 431AET6324199911839.2112331.134347.515181.5941839.211299.5341854.461354.026205.1521283.029147.536139.01 432ISNS679419991419.395475.29273.8151.81419.3954.662466.60281.3361.144441.6568.6297.157 4333NVIC679419991646.278725.884182.242132.519646.27847.799817.509194.4139.179684.02844.35343.228 434TCI67981999-114.563.282018.48811.9010.06315.7772.08407.063.9610.005 435CMO67981999-128.0625.54611.4434.7041.7841.78834.8236.024.00532.4333.3614.585 436TCO67981999-13237.248153.032176.2451277.791187.61930.1364438.383220.676239.8461617.71713.61657.656 437HMT67981999119.224.2481.0041.52419.220.0371.7740.3881.0098.880.0420.091 438BRE679819991224.487228.78338.31532.602224.4876.823234.82936.03918.138310.3092.52113.052 439RFS6798199914892.1165400.717903.177783.24892.11683.084459.695675.233660.6014602.20266.296166.891 440LGN701119991189.69660.17213.80512.385189.6960.57284.82512.86318.567209.0550.2637.976 441PDQ701119991872150513834387201462.1109.2425.31040.779.567.8 442AVSV73591999-184.68817.245260.5192.196232.97513.563361.62865.48115.415172.6974.3845.055 443RWY735919991346.188117.4894.0690346.18810.186139.4232.4480336.45815.0563.543 444HWCR73631999-1441.19597.2310473.41125.0325.007230.0441.7190348.81610.4890.753 4453SCBI736319991404.17226.941194.7170.564404.17218.89930.161218.360.077413.091357.244768 446CPLXQ73701999-10.220.35301.3170.4570.0070.1580.02602.9350.0120.095 447BBOX73701999-132.24223.2884.264248.7395.9247.94916.48769.9418.56731.0561.38512.926 448ONHN73701999-189.3845.1270414.66912.746117.972117.4894.0690346.18810.18627.824 4493GEEK737019991450.855135.65338.9380450.85586.292104.17326.6920199.95255.22811.189 450ETAD73701999116.82916.9315.0043.41316.8290.24226.8293.5654.50516.6880.3580.278 4513SRCM737019991458.01532.061158.87324.125458.0158.889501.177152.18719.397835.6434.25310.713 452EDS73701999126.91935.1621.33230.59726.91907.1561.226036.9860.2350.131 453IFXCV737019991105.072116.74613.30644.938105.0721.108106.76122.44354.046116.3841.1732.552 4543TSCN.737019991970.2454107.434166.808274.995970.24534.3993874.672165.527245.477947.92215.47852.605 455LU73701999156504.39851029.048479.3354633.69156504.3981994.21664454.5659156.3654911.44463147.5461881.0092585.845 Figure 18 Continued

PAGE 164

152 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 456EVIS73721999-111.0283.36809.1472.1560.2527.9632.89905.1411.2570.023 457MDSI73721999-13.2190011.04200.401174.06325.1320234.97637.5048.177 458EPIC73721999-119.1535.978015.5460.45813.5472.44299.6340274.9680.10426.855 459TMBS73721999-133.5717.525025.4480.5690.63834.89514.631021.5640.7750.482 460YND73721999-1113.28713.761028.3761.690.739272.92639.738098.7270.8361.91 4613COVR73721999-136.95410.9639.54643.0547.1771.151477.736.03910.004207.818186.3710.083 462IINT73721999-172.7591614.56675.8080.2742.05181.04515.5720.66691.7840.7783.529 463DLVAZ73721999-1275.765.65631.88781.8422.2310.369269.7394.88339.16963.8161.5032.439 4643VETX73721999-1103.25419.15930.89889.2421.1273.29560.08515.98518.01850.2620.4321.697 465SEGU73721999-1102.70121.5950114.9822.269.503106.02311.864095.3862.43.304 466MCTR73721999-1361.62865.48115.415172.6974.3845.055290.85638.4557.31479.9070.7022.629 4673IPLYE73721999-1108.88513.3570217.6681.52976.376132.4881.3182.7796318.8393.5 4683TCSI73721999-1475.55983.23568.777375.76615.15320.038461.13787.40156.123342.42310.72820.531 469SDRC.173721999-1995. 41896.332123.9671851.11638. 8525.2811000.13182.628104. 3191794.28675.13612.78 470MSTR73721999-1988.5169652921.6453.442.6332.46338.4502636.8813.926168.706 4713GLOB73721999-11451.675215.08143.3353220.193299.58637.805952.312123.16441.6323046.783429.6370.602 4723LHSPQ73721999-1338753131302655798105111012972331163382571008511184 473THQI73721999114.8273.0481.517014.82705.5592.016010.7810.2310.094 4743TSSW73721999128.5039.6483.8370.38128.5032.36932.6276.8710.51469.5380.09812.477 4753UNFY7372199911070.978668.242168.6530.8041070.97816.59700.07199.2672.7451247.29764.563275.081 476QNTSQ73721999127.87210.240.5011.37727.8720.44510.8821.6122.12927.370.50210.219 477PTEC737219991159.079255.97423.7197.698159.07931.894262.23624.0997.006162.91225.413.433 478DOCC73721999113.4432.7080.6198.20513.44307.4041.486.68813.44800.086 479MNS73721999173.08863.61811.53928.73773.0882.32265.04912.58521.80168.1261.124.729 480MLOG73721999178.859159.37718.27535.55278.8590182.34433.02534.81688.0670.7422.874 481LGTO737219991123.915116.76724.89545.459123.91515.442266.96420.3763.531274.96816.5639.031 482SSAXQ737219991470.946778.52451.85968.275470.9468.723897.5670.40678.143586.20113.7124.699 483JKHY737319991206.995282.33199.49527.534206.99553.168300.52855.31678.311243.80956.7484.93 484AZTC7373199913091.1628972.161565.525393.383091.162617.4369958.956503.399449.9893142.151586.80963.014 485THLC73891999-127.9388.4412.37730.470.4312.28353.8212.8640.10659.8990.55314.149 486MSGI73891999-1348.18212.2133.603577.922.132128.842458.05618.1943.305553.39713.38555.71 487AAC738919991231.06834.6993.9880231.0680.20114.3073.8850250.8065.87248.828 488ANLT738919991720.9891124.305324.654105.032720.98953.5661032.79312.12338.001695.97461.2417.264 489RENT78221999-19564.4122351.3791503.75110545.73239.466383.2557343.248776.4511412.9979828.51228.546710.62 4903PNEC78221999148.52432.1223.603048.5243.64428.1513.24017.9351.1291.334 Figure 18 Continued

PAGE 165

153 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 491ESI82001999-132.7955.580.70232.2480.2287.135296.18918.3187.1160.5850.5544.699 492POSO8200199919.8536.7391.2140.1999.8533.1323.9630.1440.0072.7080.350.007 493CLCXQ82001999194.71459.73414.6210.51994.7140.77763.29912.1258.10985.0580.4511.102 4943RUSS8200199911693.4332542.30231.74531.5431693.43337.3852333.805325.509519.1892135.971.5148 495CMDCQ83001999-1263.6892022.9791.1672872.94578.09155.972309.3062264.4181.6983182.181450160.809 4963ASLC830019991963.982213.472523.010963.98241.737132.176611.8460960.57754.4410 497AERS87111999-162.898387.3380436.8869.37081.53564.4830614.7157.6570 498EACO87111999116249.228054.5285036.1943290.11816249.2209139.5476718.6543341.79117224.681434.369476.738 499MGLN87411999-148.9729.13413.56116.8614.9114.63257.6467.82410.743103.8122.6472.75 500PMCOQ87411999-16190.833606.212268.7616482.935858.967391.476034.747479.716215.9985588.938406.251112.394 5013CASL8741199913053.505287.8111628.99767.3183053.505502.38042264.41803282.187449100.865 502ADPI87411999163.29866.1294.221895.19263.2982.86986.2516.673189.86673.2151.8962.227 5033EBTI99951999-1622.0818.64120.742104.0077.9836.36551.4191.4271.462387.79064.207 504ITRNA99951999-13896.302252.067429.362960.539122.91350.6564258.425408.052267.7751103.539250.45840.174 505KCS13112000-1224.78969.4932.989208.22516.61615.157301.94881.27649.068248.63921.23616.241 506CHOHQ1311200019.221.0251.2410.4989.2204.8820.8440.1754.4500.169 507PLLL13112000125.50824.7933.0920.91725.5081.56722.3042.3331.21422.5581.1631.142 508EQTY13112000175.96895.21310.85935.20675.9682.83591.77920.452.535103.8892.54232.113 509DF.1202020001752.858116.73439.470752.8588.704117.46137.3030932.9329.5580 510DF20202000184.02647.11514.5340.21784.0260.125853.658275.29438.616762.285119.2737.909 511PENX2040200018.06514.3992.3733.7718.065014.7222.4122.7088.45800.218 512RVFD2040200019647.7167656.3281473.876757.9279647.71607757.4011396.34680.1059995.6020306.74 513LNCE20522000-123.2250.1260.09646.4350.0630.3136.83241.8454.16231.9822.1610.094 514KBL2052200019035.159430.4221383.551407.9619035.15984.06494311449.1471527.55410278.3541164.639213.387 515WRNC23202000-1383031043861513877538222215338139558755848792137782701 516TOM23202000111.64212.2832.2442.72311.6420.8315.9572.8634.49313.8540.8260.104 517RDA27312000111.64519.7253.3846.38911.6450.03927.5354.0378.47112.9110.060.039 518SCHL27312000170.66489.0414.22213.47470.66413.62574.2487.4885.76828.90200.692 519WCS27612000-120.4495.2634.91711.4260.5730.067495.13146.0310676.1162.26155.648 520NEB27612000-1228.67861.3440270.773112.20620.582266.07382.5460473.344320.3839.048 521SR27612000-180.87602.1190.468887.712124.90.507108.427829.9611.4621274.16619.578256.135 522EBF27612000-11170.318496.129390.3011848.1634.17968.03891.395387.436274.641416.92744.37631.743 523ARNX28342000-11607.0790259.127852.76734.00957.9211993.84311.633300.4071132.67727.86963.046 5243NRDC2834 2000164.03956.1162.869064.0390. 73771.3253.085070.9430.70110.113 5253DSCI28342000165.30513.9332.6990.37565.3052.7933.2533.615049.0561.2442.717 Figure 18 Continued

PAGE 166

154 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 526NSTK283420001185.034302.35797.6065.455185.0347.479347.003135.04810.707229.94211.2497.877 527CVM28362000-116.1771.677042.4771.1945.68341.6633.652058.80320.3348.341 528BCRX28362000-12.4980.0782.50865.823.8061.1052.9810.4622.70645.634.0450.701 529CRXA28362000-1106.97639.5080116.7815.26610.51970.73621.65080.56413.5510.948 530VICL28362000-183.33927.8620462.546039.581147.59532.3460189.392017.307 531SUPG28362000-1592.4226.9529.191423.97713.068120.854580.89720.6247.8051163.94714.98879.241 532TKTX28362000-12032.7231.34171455.2124.954.21806.6207.62771802.1156.849.7 533CRGN28362000121.05319.5872.397021.0530.63421.6242.186024.121.0430.088 534ANIK28362000114.3544.7151.553014.3540.0278.3272.86027.0251.4230.647 535NPSP283620001702.099846.8980109.464702.09933.321060.4170112.104668.53433.36945.459 536CSON2836200011714.0112264.313326.937168.221714.011105.1052354.723514.074279.7852525175.218157.937 5373ORGG283620001691.6591977.011208.116226.785691.65915.8281825.169242.159137.549609.4215.3427.442 538HEB2836200013021.761530.55536.4561369.3513021.7610635.0839.631431.8373362.960209.874 539GRKA29112000-115140215947134778020411495523925243747802150 540HWY29112000132.66930.5579.1260.33532.66913.2619.8655.1080.05426.01713.1271.464 541LAF3270200015.3915.6712.4812.1175.390.08417.1512.041.1793.9930.0550.137 542USG327020001185.034302.35797.6065.455185.0347.479347.003135.04810.707229.94211.2497.877 543ATI33122000-151.92615.165052.9188.8421.91652.33113.893050.2768.4074.194 5443NSTLQ33122000149.74855.90.4730.44549.7481.50556.7540.6940.59448.7931.6433.046 545NCS344820001432.679865.2121.46140.434432.6792.769802.315115.15936.022409.1163.94410.466 546BBR344820001188.26325.14933.30958.36188.261.112277.6332.92671.368188.7032.4368.608 547TTC35232000-16.8211.2712.09521.7650.1862.001214.37364.0333.129410.6085.97216.315 548ALG35232000120.5990.8020.0150.99820.5994.9160.2470.0960.34930.6283.8890.134 549CMCO35302000118.8620.7250.458018.8620.65510.6892.928062.5313.6944.647 550MTW3530200012.4865.4611.1280.4342.4860.0745.0611.3180.4755.6540.0430.202 551LUFK35332000-118.1191.123018.9139.1953.39629.3452.161039.28732.8850.887 552HYDL35332000111.4342.3830.7810.55711.4342.47114.0653.233024.2531.561.949 553ATU35402000115.39770.4256.7911.40815.3970.48944.3066.7360.88717.1190.4290.256 5543THMD35402000135.741.79517.2348.91335.70.05258.81519.2688.43852.7080.2711.233 555ASTX35592000-163.35118.832057.7777.6556.92989.71616.786075.8574.25610.386 556TRKN355920001388.549442.22399.0670388.54968.591451.937112.2670376.9664.97112.918 557IVAC355920001214.254206.82269.02324.914214.25418.994259.96183.40722.937285.6310.61114.298 558SMTL355920001736.011426.89385.54376.139736.01110.024466.4489.07782.853732.039.52219.219 559WDC35722000-1741.41234.383144.22419.68313.56679.134640.740.596.105362.46316.98122.866 560RDRTQ35722000-1 1871.636202.09801881.615 1364.06248.1191873.554 141.6401803.7871357.24436.924 Figure 18 Continued

PAGE 167

155 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 561ELX35762000-139.8238.6795.17622.6644.7620.48741.46910.8296.02825.5164.7940.738 562PROX.35762000-134.583107.159.077252.034232.5615.99459.516293.391158.62523.84711.02129.39 563VNWK35772000-1157.48729.3933.117218.5870.96519.511134.3219.9610.799235.69907.817 564ENCD35772000150681.7462031960.52014.950681.72785.61495542.7870.0833142.151431840.227 565MMAN35802000-11.9780.8082.44525.880.0541.0146.320.5559.56312.89101.657 566ILI35802000-1114.3827.3830106.41514.4673.528122.00516.51093.88111.0796.473 567ANEN36632000-115.5526.5950.91613.0760.2790.91113.8732.0210.218.2770.1050.182 568CAMP36632000-1151.55834.06712.657136.41130.8989.502253.360.90141.686676.11610.98955.648 569ADAPQ366320001796.952477.302145.77810.062796.95233.16455.683144.11916.096780.78128.36256.135 570AND366320001124.49197.93632.81812.188124.49123.432206.82269.02324.914214.25418.9943.861 571ILXI36692000-1207.74627.1710128.5540.673.318239.63125.9450130.4221.8413.432 572CKP3669200011271.786523.89957.0924.7471271.78633.207571.67753.3125.071516.681107.83135.848 573UTSI3669200014646.2995355.337840.04974.1964646.299581.5315979.604922.3251008.8645337.661638.963334.748 574ROV36902000-112297.81465.21283.811687.8901.4669.513006.81600.61239.912815.5702.5786.4 575SMP3690200011.3292.8980.3520.8761.32903.3920.5372.0272.66500.004 576HAYZ37142000-1679.738175.794102.034879.23799.69862.448301.71880.862123.219757.41991.65937.052 577TEN37142000-1696.649229.7141.7343437.1186.957482.775793.071226.749461.9813814.474149.89670.2 5783WMCO371420001419.313324.49487.0530419.3136.28371.86238.4550187.222.44.549 579UVSL371420001436.6271250.60453.555207.72436.62710.0511192.54668.56179.971396.2349.79815.258 580TWAV38232000-166.51111.17512.08433.7011.9362.28870.68411.2211.92932.9241.3710.331 5813RVSI38232000-1249.843122.19137.158836.74613.97113.46714.938140.01450.135982.58515.628217.179 582ARXX38252000-127.2556.085.30122.3670.6270.44436.6027.3446.63427.0850.1981.104 583LCRY38252000-1214.6430.695071.2120.1050.826186.19825.387031.5430.1180.941 584COHU38252000-113.6891.320139.58715.6498.00611.2871.948078.77514.130.954 585DATM38252000-154.42814.4030282.2262.71318.19694.63519.1330236.6280.55412.69 586LTXX382520001107.358109.17721.0730107.35817.826105.5120.3760113.04725.7382.178 587CMOS38252000159.38642.8032.1854.29559.3868.53625.2711.4745.77359.1065.3130.873 588BIO38262000-163.8132.2142.3214.3240.0140.43218.630.8780.6997.7550.0110.178 589VARI38262000-15286.041214.084520.7621160.5124.046138.0986248.518214.96667.5141481.71542.5585.28 590CPWY38412000-1887.87162.59203163.54414.299130.1881091.293238.58703402.39636.86992.284 591IJX3841200014.9025.3951.17804.9020.11419.8657.72204.7460.8230.482 592POSS384120001498.481968.76146.7320498.48126.9231072.55448.580535.74130.51425.307 593EMBX384120001352.21034.5614.472245.186352.24.8951206.6244.758277.453389.9895.54320.914 594VFOX38432000-11880.194488.00875.3474392.898168.645482.7752481.068647.128431.8374239.054108.288294.258 5953BLLI384320001156.172480.35577.4050156.1725.464519.70476.9740150.4019.2541.023 Figure 18 Continued

PAGE 168

156 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 596ABCX38612000-123.121.8960.14922.4191.6540.19226.3922.3550.13324.9812.5970.154 597LENS386120001125.63197.56849.0744.043125.636.71268.68868.3459.634182.8952.7035.082 598GMT347002000-11.4881.17506.910.7231.024.6061.2580.249.0990.5340.498 599HOLL47002000111994.1093170.3282359.204011994.1091217.6233569.3014088.229015072.0631834.49652.516 600CTIX47002000116.09512.3621.8960.19616.0951.0893.2270.7220.12934.5680.5150.87 601GMT47002000111167.55210378.931514.963187.35511167.55252.2672988.487546.37186.03510600.2792.468403.142 6023SNHDE48132000-19 15.50819.021110.276226.09 7.64621.086860.54315.106 121.458220.1160.4481.802 6033DTIX48132000117850139856874441785033073381079491165483081236 604EWST49242000-1184.07535.42135.31185.6931.0629.331263.04667.72680.191425.94489.52529.877 605RGCO49242000-14459.695675.233660.6014602.20266.296166.8914183.664669.885595.0714437.385216.743102.305 606CMN50472000-1225.88748.1266.11183.0611.0821.229297.92546.6928.83398.9931.1992.85 607ABIX50472000-1762.80859.78647.48342.5498.97625.899492.108470.53201610.435013.651 608ARW50652000-120.0451.575040.9580.0590.12321.0062.142040.8940.1750.41 609AVT5065200011217.293924.381547.90215.2871217.293124.9591004.012546.32410.8461142.35133.3551.383 610FLMIQ51412000-11113.4088944.318220.110441.236293.115261.21346.74312096.516167.98314050.293279.225573.287 6113089B514120001186.476156.99633.9822.339186.47642.08128.70416.31214.87198.84865.0219.268 612AE51722000-1124.97912.3710.37349.8120.5014.337100.7158.2199.47256.6880.2871.534 613ENRNQ51722000-16 40.493386.22903395.688349.4770 512.886437.84403440.01357.3210 614DG5331200011289.57191.716353.79901289.571096.837427.98901395.31360.89944.743 615AMESQ5331200013609.2071917.898116.433178.5323609.20701922.21166.64198.8513492.0610242.078 616KKD54002000-120.7441.522.3728.8070.0740.23121.5863.3161.9999.6170.0650.564 6170295B54002000-1178.50549.1670168.9016.5516.402145.68952.8570140.7323.32912.346 618KR54112000-11659.3964.13455.6828364.0595624.91349.7831646.98450.81841.8399690.5286467.15240.227 619ABS5411200010.4510.08600.0170.4510.0060.0290.0140.1877.2080.0090.094 620CHRS56212000-1 543.7826.69126.8911188.128 29.08996.986615.2631.837 32.3261447.7134.159144.939 621ANN5621200018.60319.3470.3652.7128.6030.13118.0710.0371.9647.1410.1020.613 622GES565120001499.16320.8381.40499.1635.19123.4750.8560538.7010.13111.382 623URBN565120001706.288652.419153.5780706.288206.895274.982158.1277.453359.957219.005286.91 624GDYS565120001413.683765.89110.142247.8413.68311.857706.41111.168197.147341.69120.58513.267 625FTUSQ5651200013961.07715918.138901.6491895.5253961.07780.91525033.61742.82931.48289205.5319.9 626FRS58122000-172.1967.732.550.55304.757119.9410.3325.39961.26103.276 6273GRLL58122000-110.68519.263070.777.9240.94152.9224.1990133.58121.9460.353 628CHMD59122000-147.93114.1316.665103.4960.2922.77133.292.92210.62634.4870.7955.286 629CURE59122000-1 98.83622.42737.67271.677 8.4351.972117.00813.166 41.939284.9773.2467.803 630JILL59612000-12295930914623586226651814.91249528453473369619.0523032 Figure 18 Continued

PAGE 169

157 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 631LVC5961200014.7725.5081.6104.7720.3124.1661.54604.75200.049 632RIGS60202000-142.5624.445.88811.5370.2780.36259.2161.0310.59214.8170.7850.127 633PBKS60202000165.08346.9266.5886.20665.0835.96312.43869.579456.009104.2281.69314.728 634LEDG60362000-117008581813642768944532191584957241222254584318146 635ASBI6036200014.2265.4160.3690.1594.2260.0525.9280.5030.1064.2580.0340.093 636AMPH62822000-1413.916104.81101143.024619.02267.515244.49839.7680392.41712.61755.697 637ECMN628220001110.97459.70310.50425.591110.9744.51377.1313.8656.67371.0890.7412.219 638PHLY63312000-193.55816.73814.60866.7283.2961.94637.58755.15118.13865.581.1249.6 639PXT63312000170.74291.1080070.7424.17991.9040075.2673.3973.393 640CBG.265312000-151.42416.0591.97727.0933.3970.22640.5339.4571.98316.4550.0310.47 641JLL65312000130680.54446225.8372149.136579.67330680.54411017922.498917.474351.81619634.279985854.376 6423BIGTQ67982000-101.158025.3310.0311.31100.615022.8950.040.672 643WRI67982000-1226.87223.811070.6340.1042.373214.6430.695071.2120.1050.826 644HCP67982000-191.48423.968.25237.462.5338.189175.66641.7948.3672404.594128.10216.211 645CDX679820001295.845280.9763.02239.443295.84520.868237.6843.64529.506260.80455.79522.817 646GREY73112000-1148.67428.6080.938230.921148.1893.587224.36632.8551.819192.167120.6611.535 647LEAP73112000-1439.54792.17285.765549.04823.49314.416456.66680.47989.685569.45729.32315.694 648TNO73112000-1493.10695.6914.406772.83297.73614.513659.21978.37313.7734.82489.8516.524 649BULL73112000-1690.811176.479122.267872.00532.37813.172658.535145.76891.51752.65389.5789.572 650HC73592000-131129.3571798.724174.03512291.706839.9086171.73711557.9981692.488318.89317636.820660.526 651UCO7359200016.42618.0231.1550.3756.4260.15314.5271.6280.2784.7180.1960.224 652KEYN73702000-15.390.76903.5881.6310.2184.3791.02104.3481.450.045 653EDS73702000-17.1541.710.92110.8715.4060.1656.4320.5080.8322.8310.2130.238 654CPTH73702000-16.8211.2712.09521.7650.1862.001107.35911.4320236.628142.911.657 6553CMEE73702000-136.7851.2190.11849.1440.2833.47841.7951.2610.12548.9520.0540.758 656BGNK73702000-119.1155.822077.282.38919.10215.3181.1909.5450.5813.265 657AVCS73702000-1167.84135.14837.61101.1791.1587.175196.21243.64942.361216.3111.79310.495 6583ARIS73702000-1258.17675.2630170.17738.90511.392219.76860.4240134.78710.5035.135 659EDGW73702000-1101.086720.3686.9221337.224104.7660.067122.807893.0291.841505.79659.56990.86 6603HGRD73702000-11068.859272.45275.5042272.96653.617661.3911009.999211.722223.352154.02961.289250.17 661DGIN73702000-16009.847388.08321.7764037.2230119.905980.446321.09122.0473614.133064.207 6623WEBB73702000112522.318534.24423.4012522.32960.919226.84837.1012700.33268.3768 663AVRT73702000129.3824.8711.7034.17429.38026.0791.717.1938.36216.4640.635 6643ZMBA73702000121.55123.275.9025.89421.5511.48422.845.5745.85922.2361.470.813 665LU73702000121551.18222027.0683459.3563396.21221551.1822979.82624501.0674154.8923602.09224522.553395.2331655.85 Figure 18 Continued

PAGE 170

158 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 666MGEN73722000-10.6720.31504.2520.1030.0922.9810.8304.6540.2590.127 667ONXS73722000-118.3463.40.03926.774.5320.57224.4313.763025.7975.2481.644 668GSOF73722000-1130.63912.0653.2695.9935.5173.534275.06829.7126.241290.17811.118.187 6693UNFY73722000-15491.8952.11079.25764.1305.1227.25318.3860.6856.65300.9383.3207.3 6703EGAM73722000-1462631960.52014.950681.72785.6218.1945275.127681.21019.542710.53371.71877.2 6713LTWO7372200019632245908937609632216024431388540095057144436808 672FRTL7372200016.5615.8120.9514.3636.5610.1335.6610.6734.0115.7770.1420.047 673EBIX737220001143.497199.99833.12326.371143.4970.714161.70725.13226.255122.262.0951.203 674REY73732000-1186.44857.77763.299195.0621.4697.47186.35739.66355.844166.99112.6117.279 675BVSN737320001209.8616.613178.9490209.864.51818.06207.0490236.8771357.244768 676FDC73742000-131129.3571798.724174.03512291.706839.9086171.73711557.9981692.488318.89317636.820660.526 677ADP7374200013621.284474.414518.95503621.284404.442472.907692.84903646.261344.730 678SRCP73892000-1447.1556.272152.065247.9333.70447.31458.2037.759130.676213.4840.01230.482 679FMKT73892000-136.97416.4370504.886.9453.01958.0656.6890367.3826.92518.115 680FNIS7389200017.1329.2181.8820.1237.1320.14810.0362.3660.0838.8520.4591.172 681MTY7389200015050.6116581.236193.2581522.2035050.611232.196070.568239.421295.8784595.521218.194219.838 682TMTV78122000-1186.35739.66355.844166.99112.6117.279214.37357.96547.605185.038174.242 683LGF781220001254.731.54221.9960254.740.67726.5744.0450112.84722.0880.717 684ROMN781220001153.304232.30751.1241.46153.3046.71400.1373.5891.487253.5818.44910.329 685WWE781220001103.16382.31818.50624.68103.1635.15877.05914.95427.39897.9991.7275.885 6869136B78302000140.25564.6830.1460.66940.2550.73675.7050.6910.7877.8490.99916.098 687AEN78302000114.11430.6664.1651.98614.1141.91828.8684.1351.7711.1661.9030.237 688SLOT79902000-12475.682345.996265.644858.82425.6565.4612433.803216.002218.927646.0725.9185.558 6895530B79902000-162674435715196182006664442597592725200 690HWD799020001246.275329.97462.84132.258246.2750.66499.816115.95852.535452.2891.94412.566 691OCA80002000-1284.838.078076.8413.1671.5592.84482.628069.284024.998 692TLCV8000200016.56211.7641.99206.5620.08912.8872.447010.4260.240.193 6933MHCA80512000-19.5661.5621.2825.58100.0238.9470.9141.7874.7200.039 694KIND8051200011944.426429.86921.088295.0411944.426257.291170.6574.08497.981715.03530.832361.024 6953CRHEQ8082 2000-148.2683.7040.53358.7821.2720.829 54.3037.3370.63169.2841.1130.003 6963AHOM808220001691.6591977.011208.116226.785691.65915.8281825.169242.159137.549609.4215.3427.442 697VSIH87002000-110.8913.0163.8158.6640.1150.3669.282.7122.7146.9770.1050.32 698METG8700200011995.7273968.55517.039744.1321995.727468.3813264.56512.631532.4071083.73152.28325.245 699CBIZ87212000144.57758.1215.462044.5770.36538.73512.053037.7050.5352.178 7000131B87212000162.98758.43610.63317.4662.9870.95779.14622.54720.20769.3320.542.652 Figure 18 Continued

PAGE 171

159 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 701OXY13112001-123.8417.6489.45830.2460.8554.1624.0984.3136.76425.3450.0561.934 702APC13112001-112731.9749.6062893.14310421.7413751.337776.32713338.947299.6342472.4379909.8472133.643573.287 703VTS13822001-1300.71524.73312.031171.4932.9725.655394.35441.21616.159255.3768.30937.31 7043SEIEQ138220011217.718332.0024.84981.991217.7182.11438.9114.668130.15255.9782.17629.555 705FLR16002001-115.8063.9880.0247.7120.640.08815.55443.478010.3530.4711.055 7063FWLRF160020011485471585209654854200107511199191110632501261 707HAIN20002001113.80900.04013.8090.1400.0404.5090.140.169 708DLM200020011824.3652.781646.3850.632824.362.203112.8341661.381104.9532145.406451.94264.207 709OME20702001-1537.41838.41715.998429.8527.1939.533535.33832.898.402571.76713.44517.309 710DAR20702001126.72148.13613.587026.7210.06979.06126.9131.705124.3050.2332.375 711TGX2834200115.9111.9261.27705.9110.2416.1783.36108.4460.8130.228 712IMGN2834200112381.18814.2721674.72902381.1110.9349770.11897.602718.7152.674.7 713FMXI30862001-140.11919.503030.8670.390.54737.71513.3122.77925.5240.6061.417 7148412B308620011134.21540.0914.3396.825134.21523.0236.2554.4454.37445.97813.445.134 715DEVC32702001142.2350.3099.3299.54142.232.31182.39911.43912.61742.4991.7921.578 716EGBT327020011232.969280.48724.97950.451232.96917.424312.53835.45270.111319.40715.23175.11 717DOV35592001-1258.1261.6260736.60225.331109.909428.51470.7848.628738.98431.709140.899 718GRB35592001-11263.274123.9764.4492842.58682.482260.5061425.277149.23280.943236.88895.171269.371 719GSLI3559200114.4261.2463.74604.4260.0082.0245.7590.1147.91100.004 720NVLS355920011315.49621.321233.2160315.4960.424.898244.5030345.2392.7030.564 721ETS35762001-1671.25238.84887.7621357.94381.33269.8871825.1691.5762542121.357186.37155.3 722JNIC35762001188.69289.75321.354088.6928.87663.97214.723063.4557.3131.578 723EFII3576200118.1020.0470.8624.5398.1020.0271.4720.5951.7696.8952.970.03 724DGII357620011390.901197.18163.5431.494390.90115.062280.28179.78335.22457.56320.112.58 725MMAN35802001-1730.662215.1158.141895.19225.04636.294874.911295.31554.4831092.91259.56955.648 726ILI35802001172.721193.3186.1560.5972.7214.366203.3048.2861.025115.5372.3591.858 727NMSS36612001-110.1473.0650.7584.83800.372238.37326.8011.17484.9391.470.134 728AVNX36612001-12171.3270.1201.22811.1160.1157.12039.1266.2155.22358.155.2133.8 7293CARD36902001-119.5713.4069.06228.7752.1082.02623.0051.9043.75512.5040.0951.126 730IGOC369020011802.784782.023114.7770802.784122.337891.963138.0490936.351212.62749.532 731TLGD38252001-136.8285.3651.3569.5640.5640.51328.486.2331.34610.3530.5540.67 732PHTN382520011366.79190.18211.5170366.79129.42176.9568.1980377.14323.27235.811 733IO38292001-121.1410.5584.439479.160.691196.81787.1271.778121.4582127.577139.186696.3 734OYOG38292001121.9472.680.3430.33821.9471.3791.4290.1460.5219.4370.8721.051 735TRMB382920011139.749287.09419.48331.944139.7495.66259.63419.11826.618120.7576.1721.096 Figure 18 Continued

PAGE 172

160 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 736MSS38292001126508.25329222.36410842.2544652.50526508.253022369.4917695.0592382.11423912.5570794.293 737ARRO38412001-137.84816.27318.22955.7820.2923.89928.2633.1462.30917.0780.4171.265 738HAE3841200112451.81615.2359.856.5852451.831.4242298.166002885.41.423158.4 739SYD38432001-1490006874066181903151623500986794178190873242139 740ALGN38432001112.62211.1633.1492.89712.6220.9058.9073.1263.1189.2260.0340.318 741ESMC38452001134.00747.38.5644.22634.0070.79965.34910.7264.59945.6880.7671.175 742POCI384520011443.986793.09228.511268.214443.98619.888845.81537.566367.366563.25128.12520.132 743AYE49112001-115.0763.2141.9757.9070.110.30621.0265.5383.98913.9210.2890.654 744RRI4911200112333.2941271.27294597.6432333.29486.2861282.173275.294522.6092143.89593.44444.797 745ED49312001114.57817.4463.402014.5780.625190.88441.21616.1596.963149.55219.69 746CMS49312001112.6278.562.3913.40312.6273.7592.8440.0732.1997.843.2030.362 747SVU54112001-1348.486150.520333.5658.37418.91378.15161.4470350.91914.94620.067 7483PUSH54112001-1421.09567.1487.8952511.31974.2332.711605.176102.77212.9082558.23570.08932.644 7493HCAR55002001-1349.73660.70232.835810.90513.77737.769392.74873.26937.349981.96426.23659.528 750MAJR55002001-14616.791005.68590.3832058.80718.24192.8265046.87572.622158.6221440191.054 751TRVS55002001-116949.608673.485873.5444426.256168.548428.2176853.6521458.55310.6877.74800 752FFPM550020011936.2582.210.977.8936.20477.710.873.5890.8049.6 753BBA57122001-1110.1261061.5242.1481514.08862.072.816116.131164.6020.7481556.0263.30120.132 754RSTO5712200111646.491125.9431149.7081.3661646.49141137.821268.5781.7921816.91833.00316.619 755PMORQ59122001-1114.95630.50118.36591.710.2376.48550.4914.4564.15355.390.1331.463 756ECMV59122001-1221.27951.86721.445222.8522.19413.854228.26641.84514.826175.560.8577.741 757PAB6020200111402.809114.509814.4883.8141402.8090139.281128.3816.2322091.0961.2582.177 758IFCJ602020011448.0447022.16346.15235.453448.0440.1784717.242167.55610.004227.0273.8233.591 759FBC60352001-1128.43420.28636.879138.75130.6697.02544.30672.6224.162206.8223.9247.564 760HTHR60352001-121.718151.2630.472317.87489.90.15224.845182.8690380.96420.118.444 761SUFI603520011402.42955.06615.2990402.4290.745141.86227.9701172.466.683286.91 762WFSL60352001123.61935.0587.3327.86223.6191.21942.6757.8346.27823.3320.2710.983 763CIT61722001-199.7916.018079.9440.7792.44996.2253.458028.5773.1111.22 764CIT.61722001-1209.5531.39514.501133.25245.0175.338190.88418.55414.639120.1221.4726.961 765HC73592001-1231.78432.51137.057248.96140.69165.36244.04447638.001905.9843.92432.644 766WSC73592001-11462.28319.808188.211507.629044.5531569.06213.297227.22599.122.66840.514 7673ALRC73632001-15.8383.40881.92314.3710.0650.4183.5820.791179.9717.74800.015 768BBSI73632001128863549487422288628533643953262681292127 769ONES73702001-1730.662215.1158.141895.19225.04636.294874.911295.31554.4831092.91259.56955.648 770IFXCV73702001133.86234.5946.4938.76233.8624.81919.2522.8952.58512.3870.2230.237 Figure 18 Continued

PAGE 173

161 Obs#TickerIndYearLabelSalesyr1ARyr1INVyr1TAyr1OAyr1CEyr1Salesyr2ARyr2INVyr2TAyr2OAyr2CEyr2 771CA73722001-13.2870.5940.6952.8060.0660.2063.6460.5580.5211.5540.0070.092 772ORCL73722001-198.62538.6120226.8150.3644.563138.2938.920160.8911.8326.18 773RDTA7372200119.8361.1460.0560.0759.8360.2150.1120.230.0115.6860.0370.641 774IATV73722001138.2732.9363.9388.84338.273.32433.0435.0868.09235.9943.3141.132 775CSRE737220011203.633228.57348.65515.281203.63320.71187.12775.3765.80569.2844.4317.849 776AVGO73722001173.112103.60316.77837.31373.1121.9787.50113.67930.9259.3331.5670.644 777MDSI737220011905.9841077.52244.88153.629905.98481.4621141.949214.72454.136879.504117.71724.755 778AKLM73722001132143781305271321450832963502543464355109 779BVSN73732001-10.3980.1691.28710.6660.8580.120.5480.0496.48211.1221.2840.413 780SONE73732001-116.27973.6960127.2922.6590.8337.47632.970103.9628.9821.091 781DV820020011238.289383.19755.3238.297238.2892.059306.34844.7554.937207.8182.3254.825 782EDSN820020011213.651198.04235.0133.98213.6519.835192.49329.59610.531189.74411.8963.685 783AFAM83002001-1261.54152.33417.144187.4393.3512.05231.3952.75182.977450.11975.13649.548 784CRN83002001-194.867752.32401025.682270.75920.58297.86824.70201140.1837.8476.059 785CSU83002001152.85263.9811.602052.8521.466126.5518.5540367.3821.3872.297 786RGNT8300200111380.22212.7332.5115.91380.2133.72349385.5124.51447.156.596.9 787MDTL87312001-114.7530068.4125.9090145.6097.763092.511.2852.302 788SONO87312001-1909.13256.1180914.04518.38645.804873.116235.990835.72734.47735.792 Figure 18 Continued

PAGE 174

B.2 Text Data Object 1. Text Data

PAGE 175

163 LIST OF REFERENCES 1. Abbot, L.J, Parker, S., and Peters, G. (2004). Audit Committee Characteristics and Restatements. Auditing, March, Vol 23, pp. 69–77. 2. Abdel-khalik, A. R. and K. M. El-She shai. (1980). Information Choice and Utilization in an Experiment on Default Prediction. Journal of Accounting Research, autumn, Vol. 18, pp. 325-342. 3. Agresti, A. (1990), Categorical Data Analysis, John Wiley & Sons, Inc., New York, NY. 4. Altman, E. (1968) Financial Ratios, Discri minant Analysis and the Prediction of Corporate Bankruptcy,Journal of Finance, Vol. 23, pp. 193-194. 5. Altman, E. Eisenbeis, R.A., and Sinkey, J. (1981). Applications of Classification Techniques in Business, Banking, and Finance, JAI Press, Greenwich, CT. 6. Banerjee, S. and Pedersen, T (2004). The Design, Implementation and Use of the Ngram Statistics Package. Working Paper, University of Minnesota, Duluth. 7. Baeza-Yates, R. and Ribeiro-Neto B. (1999) Modern Information Retrieval, Addison-Wesley Pub Co; Boston, MA. 8. Beaver (1966) Financial Ratios as Predictors of Failure. Journal of Accounting Research, Vol. 4, pp. 71–111. 9. Bell, T.B. and Carcello, J.V. (2000). Res earch Notes, A Decision Aid for Assessing the Likelihood of Fraudul ent Financial Reporting. Auditing: A Journal of Practice and Theory, Vol. 19(1), pp. 169 – 175. 10. Beneish, M. (1999) The Detecti on of Earnings Manipulation. Financial Analysts Journal, Vol. 55, pp. 24-36. 11. Brill, E. (1992) A Simple Rule-Based Part of Speech Tagger. Proceedings of the Third Conference on Applied Natural Language Processing, pp. 152-155, Trento, Italy. 12. Brown, M., Grundy, W., Lin, D., Cristianini N., Sugnet, C., Furey, T., Ares, M. and Haussler, D. (1999). Knowledge-Bas ed Analysis of Microarray Gene Expression Data using S upport Vector Machines. Technical report, University of California in Santa Cruz. (submitted for publication).

PAGE 176

164 13. Budanitsky, A. and Hirst, G. (2001), Semantic Distance in WordNet: An Experimental, Application-Oriented Evaluation of Five Measures, http://ftp.cs.toronto.edu/pub/gh/Budanitsky+Hirst-2001.pdf, June, 2003. 14. Buitelaar, P. and Sacaleanu, B. (2002) Extending Synsets with Medical Terms. Proceedings of the First International WordNet Conference, Mysore, India, January 21 25. 15. Buitelaar, P. and Sacaleanu, B. (2001) Ranking and Selecting Synsets by Domain Relevance. Proceedings of WordNet and other Lexical Resources: Applications, Extensions and Customizations, NAACL 2001 Workshop, Carnegie Mellon University, Pittsburgh. 16. CCH, Generally Accepted Accounting Principles, 2005, http://business.cch.com/primesrc/bin/highwire.dll, March, 2005. 17. Charalambous, C., Charitou, A., and Kaourou, F. (1999) Comparative Analysis of Artificial Neural Network Models: App lication in Bankrupt cy Prediction. IEEE. 18. Choo, C. W. (1995). Information Manageme nt for the Intel ligent Organization: The Art of Scanning the Environment.: Information Today, Inc. Medford, NJ. 19. Standard & Poor’s, Compustat Data, 2005, http://www.compustat.com/, June, 2005. 20. Cristianini N. and Sh awe-Taylor J. (2000) An Introduction to Support Vector Machines and other Kernel-Based Learning Methods. Cambridge University Press, Cambridge, United Kingdom. 21. Cristianini, N., Shawe-Ta ylor J, and Lodhi, H. Latent Semantic Kernels. Kluwer Academic Publishers, Forthcoming. 22. Princeton University, WordNet, 2005, http://www.cogsci.princeton.edu/~wn/index.shtml, June, 2005. 23. Cortes, C. and Vapnik, V. ( 1995). Support Vector Networks. Machine Learning, Vol. 20, pp. 273-297. 24. Michael I. Jordan, Advanced Topics in Learning & Decision Making, 2004, http://www.cs.berkeley.edu /~jordan/courses/281B-spring04/lectures/lec10.pdf, September, 2004. 25. Davia, H.R. (2000). Accountant's Guide to Fraud Detection and Control. 2nd ed., Wiley New York, NY. 26. Dechow, P.M., Sloan, R.G. and Sweene y, A.P. (1995) Detecting Earnings Management. The Accounting Review, Vol. 70(2), pp. 193-225.

PAGE 177

165 27. DeFond, M.L. and Jiambalyo, J. (Jul. 1991). Incidence and Circumstances of Accounting Errors. The Accounting Review, Volume 66(3), pp. 643-655. 28. Eisenbeis, R. (July 1987) Discussion, Supplem ent to Srinivasan, V. and Kim, Y. H. (1987) Credit Granting: A Co mparative Analysis of Cl assification Procedures. J. Fin. Vol. XLII (3), pp. 665-680. 29. Fanning, K., Cogger, K.O., and Srivastava R. (1995). Detection of Management Fraud: A Neural Network Approach. Proceedings, 11th Conference on Artificial Intelligence for Applications, pp. 220-223. 30. Federal Accounting Standards Advisory Board, Generally Accepted Accounting Principles, 2005, http://www.fasab.gov/accepted.html, November, 2004. 31. Fayyad, U.M., Uthurusamy R., Piatets ky-Shapiro G., and Smyth P. (1996). Advances in Knowledge Discovery and Data Mining. MIT Press, Cambridge, MA. 32. Felbaum, Christiane. (1999) WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. 33. Feroz, E.H., Park, K., and Pastena, V. ( 1991). The Financial and Market Effects of the SEC’s Accounting and Aud iting Enforcement Releases, Journal of Accounting Research, Vol. 29, pp. 107-142. 34. Fisher, R. A., (1936). The Use of Multiple Measurements in Taxonomic Problems, Annals of Eugenics, Vol. 7, pp. 179-188. 35. Francis, J., LaFond, R., Olsson, P. and Sc hipper, K. (2005). Th e Market Pricing of Accruals Quality. Journal of Accounting and Economics, Vol. 39(2), pp. 295-327. 36. Frydman, H., Altman, E., and Duen-Li, K. (1985) Introducing Recursive Partitioning for Financial Classification: The Case of Financial Distress. J. Fin Vol. XL(1), pp. 269–291. 37. General Accounting Office, Financial Statement Restatements: Trends, Market Impacts, Regulatory Responses, and Remaining Challenges, 2002, http://www.gao.gov/htext/d03138.html, June, 2004. 38. Genton M.G. (2001). Classes of Kernels for Machine-Learning: A Statistics Perspective. Journal of Machine Learning Research, Vol. 2, pp. 299-312. 39. Google Corporation, Homepage, 2005, http://www.google.com, June, 2005. 40. Gur-Ali, O. and Wallace W.A. (1995). Classifying Delinquent Customers for Credit Collections: an Application of Probabilistic I nductive Learning. International Journal of Human-Computer Studies, Vol. 42, pp. 633-646.

PAGE 178

166 41. Hackenbrack, Karl.(1993) The Effect Of Experience With Different Sized Clients On Auditor Evaluations Of Fraudulen t Financial Reporting Indicators, Auditing: A Journal of Practice and Theory, Vol. 12(1), pp. 99-100. 42. Hansen, J.V., McDonald, J.B., Messier, W. F., and Bell, T.B.(1996) A Generalized Qualitative-response Model and the Analysis of Management Fraud. Management Science, Vol 42(7), pp. 1022-1033. 43. Hribar, P. and Jenkins, N.T.,(2004). The Effect of Accounting Restatements on Earnings Revisions and the Estimated Cost of Capital: Accounting, Disclosure, and the Cost of Capital. Review of Accounting Studies, Vol. 9(2-3), pp. 356–375. 44. Unknown Author, Term Frequency, Inverse Document Frequency, 2003, http://instruct.uwo.ca/gplis/601/week3/tfidf.html, November, 2004. 45. Jin, X., Lu, Y., and Shi, C. (2002) Similarity Measure Based on Partial Information of Time Series. Conference on Knowledge Discovery in Data Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Ming, pp. 544-549. 46. Joachims, T. (1999). Text Categoriz ation with Support Vector Machines. Proceedings of the 1999 Conf erence on AI and Statistics, pp. 137-142. 47. Johnson, S. (1967). Hierarch ical Clustering Schemes, Psychometrika, Vol. 2, pp. 241-254. 48. Jurafsky, D. and Martin J. (2000) Speech and Language Processing, Prentice-Hall, Inc., Upper Saddle River, NJ. 49. Khan, L. and Luo, F. (2002). Automatic Ontology Derivation from Documents. Submitted to IEEE Transactions on Knowledge and Data Engineering, 50. Kinney, W.R., Palmrose, Z.V., and Scholz, S. (2004) Auditor Independence, NonAudit Services, and Restatements: Was the U.S. Government Right? Journal of Accounting Research, Vol. 42(3), pp. 561-588. 51. Koch, T.W. and Wall, L.D. (2000) The Use of Accruals to Manage Reported Earnings: Theory and Evidence, Working Paper 2000-23, Federal Reserve Bank of Atlanta. 52. Landauer, T.K.,Foltz, P.W., and Laham, D. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, Vol. 25, pp. 259-284. 53. Langlois, Shawn, WorldCom's $3.8 Billion Scandal, 2002, CBSMarketWatch.com., June, 2004.

PAGE 179

167 54. Lewis, D. D., Schapire, R. E., Callan, J. P., and Papka, R. (1996). Training Algorithms for Linear Text Classifiers. Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 298-306. 55. Loebbecke, J.K., Eining, M.M., and Willingham J.J. (Fall 1989) Auditors’ Experience with Material Irregularities: Frequency, Nature, and Detectability. Auditing: A Journal of Practice and Theory, Vol. 9(1), pp. 1-28. 56. McNichols, M. and Wilson, P. (1988). Evidence of Earnings Management from the Provision for Bad Debts. Journal of Accounting Research, Vol. 26, pp. 1-31. 57. Mangasarian, O.L., Mathematical Programming in Data Mining. (1997). Data Mining and Knowledge Discovery, Vol. 1(2): pp. 183-201. 58. Weisstein, E., Hilbert Space, 1999, http://mathworld.wolfram.com/HilbertSpace.html, October, 2004. 59. Weisstein, E. Inner Product Space, 1999, http://mathworld.wolfram.com/InnerProductSpace.html, October, 2004. 60. Matsumura, E.M. and Tucker R.R.. (1992) Fraud Detection: A Theoretical Foundation. The Accounting Review, Vol. 67(4), pp. 753-782. 61. McKendall, MA. and Wagner, J.A., III. ( 1997). Motive, Opportunity, Choice, and Corporate Illegality. Organization Science, Vol. 8(6), pp. 624-647. 62. Messier, W.F. and Hansen J.V. (1988) Inducing Rules for Expert System Development: An Example using Default Bankruptcy Data. Management Science, Vol. 34(12), pp. 1403-1416. 63. Miller, G. and Charles, W. (1991) Contex tual Correlates of Semantic Similarity. Language and Cognitive Processes, Vol. 6(1), pp. 1-28. 64. Mitra, M. and Singhal, A. (2004). Improving Automatic Query Expansion. Working Paper, Cornell University. 65. Moler C. and Van Loan C., (2003) Nine teen Dubious Ways to Compute the Exponential of a Matrix, Tw enty-Five Years Later,” SIAM Rev., Vol. 45(1), pp. 3– 49. 66. Natarajan, B. (1991). Machine Learning: A Theoretical Approach. Morgan Kaufmann, Palo Alto, CA. 67. Navigli, R.(2002) Extending, Reducing a nd Trimming General Purpose Ontologies. Proc. of 2nd IEEE International Conference on Systems, Man and Cybernetics, Tunisy, Italy.

PAGE 180

168 68. Navigli, R. and Velardi. P. (2002) Automatic Adaptation of WordNet to Domains, Proc. of 3rd International Conference on Language Resources and Evaluation, Las Palmas, Canary Island, Spain. 69. Navigli, R., and Velardi, P. (2003) On tology Learning and Its Application to Automated Terminology Translation. IEEE Intelligent Systems, January/February, pp. 22-31. 70. Nelson, K. and Kogan, A, Srivastava, R ., Vasarhelyi, M., and Lu, Hai. (2000). Virtual Auditing Agents: The EDGAR Agent Challenge. Decision Support Systems Vol. 28(3), pp 241-255. 71. New York Stock Exchange, Homepage, 2004, http://www.nyse.com/, June, 2004. 72. Ohlson, J. (1980). Financial Ratios and the Probabilistic Prediction of Bankruptcy. Journal of Accounting Research. Vol. 18(1), pp. 109-131. 73. Palmrose, Z.V. (1999) Studies in Accounting Research #33: Empirical Research in Adutior Litigation: Considerations and Data. American Accounting Association, Sarasota, FL. 74. Pasca, M. (2003). Question-Driven Sema ntic Filters for Answer Retrieval. International Journal of Pattern Recognition and Artificial Intelligence. Vol. 17(5), pp. 741-756. 75. Peasnell, K.V., Pope, P.F., and Young, S. (1999) What Factors Drive Low Accounting Quality? An Analysis of Firms Subject to Adverse Rulings by the Financial Reporting Review Panel, Working Paper, Lancaster University. 76. Pincus, K, (1989) The Efficacy of a Red Flags Questionnaire for Assessing the Possibility of Fraud, Accounting, Organizations and Society Vol. 14, pp. 153-63. 77. Piramuthu, S., Raghavan, H. and Shaw. M (1998). Using Feature Construction to Improve the Performance of Neural Networks. Management Science, Vol. 44, pp. 416–430. 78. Pontil, M. and Verri, A. (1998). Object R ecognition with Support Vector Machines. IEEE Trans. on PAMI, 20, Vol. 20(6), pp. 637-646. 79. Press, S.J. and Wilson, S. (1978). C hoosing between Logistic Regression and Discriminant Analysis. J. Amer. Statist. Assoc., Vol. 73, pp. 699-705. 80. Quinlan, J. R. (1996). Decision Tr ees and Instance-Based Classifiers. In CRC Handbook of Computer Science and Engineering. A. B. Tucker, Ed., CRC Press, Boca Raton, FL. 81. R Project Foundation, The R Project for Statistical Computing, 2002, http://www.rproject.org/, June, 2005.

PAGE 181

169 82. Ragothaman, S., Carpenter, J. and Buttars T. (1995). Using Rule Induction for Knowledge Acquisition: An Expert Syst ems Approach to Evaluating Material Errors and Irregularities. Expert Systems with Applications, Vol. 9(4), pp. 483-490. 83. Rodriguez, M., and Gomez-Hidalgo, J. (1997). Using WordNet to Complement Training Information in Text Categorization. Working Paper, Universidad Complutense de Madrid. 84. Rezaee, Z. (2002). Financial Statement Fraud: Prevention and Detection. Chichester: Wiley, New York, NY. 85. Roberts, J. and Thomas, E., Enron's Dirty Laundry, www.Newsweek.com, January, 2005. 86. Rudorfer, G. (1995) Early Bankruptcy Detection Using Ne ural Networks. APL Quote Quad, ACM New York, Vol. 25(4), pp. 171-178. 87. Ruping, S. (2004) SVM Kernels for Time Series Analysis. Working Paper, University of Dortmund. 88. Salton, G. and Buckley, C. (1988). Term Weighting Approaches in Automatic Text Retrieval. Information Processing and Management, Vol. 24(5), pp. 513-523. 89. Sarbanes-Oxley, Financial and Accounting Disclosure Information, 2002, http://www.sarbanes-oxley.com/, August, 2003. 90. Schafer, J. and Olsen, M. (1998) Multip le Imputation for Multivariate MissingData Problems: A Data Analyst’s Perspective. Multivariate Behavioral Research, Vol. 33(4), pp. 545-571. 91. Scholkopf, B. and Smola, A. (2002) Learning with Kernels. MIT Press, Cambridge, MA. 92. Securities and Exchange Co mmission, Homepage, 1995, http://www.sec.gov, July, 2005. 93. Securities and Exchange Commission, Litigation Releases, 1995, http://www.sec.gov/litigation/litreleases.shtml, August, 2003. 94. Sarkar, A., Computational Li nguistics (Course notes), 2004, http://www.sfu.ca/~anoop/courses/CMPT-413-Spring-2004/lexicalsem.pdf, January, 2005. 95. Shawe-Taylor, J., and Cristianini, N. (2004) Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, United Kingdom.

PAGE 182

170 96. Siolas, G. and d’Alche-Buc, F. (2000) Support Vector Machines Based on a Semantic Kernel for Text Categorization. Proceedings of the IEEE-INNS-ENNS International Joint Confer ence on Neural Networks, Vol. 5, pp. 5205 5218. 97. Sparck Jones, K. (1972) A Statistical Interpretation of Term Specificity and its Application in Retrieval. Journal of Documentation, Vol. 28(1), pp. 11-21. 98. Srinivasan, V. and Kim, Y. H. (Jul y 1987) Credit Granting: A Comparative Analysis of Classification Procedures. J.Fin., Vol. XLII(3), pp. 665-681. 99. Swanson, D.R. (1986). Fish-oil, Rayna ud’s Syndrome, and Undiscovered Public Knowledge. Perspectives in Bi ology and Medicine Vol. 30(1), pp. 7-18. 100. Takimoto, E., and Warmuth, M. Path Kernels and Multiplicative Updates. Journal of Machine Learning Research, forthcoming. 101. Tam, K. and Kiang, M. (1992) Managerial Applications of Neural Networks: The Case of Bank Failure Predictions. Management Science, Vol. 38(7), pp. 926-947. 102. Tay, Francis, Shen Lixiang, and Cao, Lijuan (2003). Ordinary Shares, Exotic Methods, World Scientific, Singapore. 103. Information Technology Laboratory’s Retrie val Group, Text Retrieval Conference, 2000, http://trec.nist.gov/data/docs_eng.html, February, 2005. 104. Tsai, L. and Koehler, G. (1993). The Accuracy of Concepts Learned from Induction. Decision Support Systems. Vol. 10, pp. 161-172. 105. van Rijsbergen, C.J. (1979) Information Retrieval. Butterworths, London, United Kingdom, pp. 1. 106. Vapnik, V. (1995) Statistical Learning Theory. Springer Verlag, New York NY. 107. Vapnik V. and Chervonenkis A. (1981). Th e Necessary and Sufficient Conditions for Uniform Convergence of M eans to their Expectations. Theory of Probability and its Applications. Vol. 26(3), pp. 532-553. 108. Vossen, P. (2001) Extending, Trimmi ng and Fusing Wordnet for Technical Documents, Proceedings on NAACL-2001 Workshop on WordNet and Other Lexical Resources Applications, Extensions and Customizations, Pittsburgh, USA. 109. Wallin, J. and Stefan S. (1995). Using Linear Programming to Predict Business Failure: An Empirical Study, Liik etaloudellinen aikakausikirja, http://www.shh.fi/depts/redovis/research/jwss95/jwssma95.htm 110. Liu, Hugo, MontyTagger v 1.2, 2002, http://web.media.mit.edu/~hugo/montytagger/, March, 2005.

PAGE 183

171 111. Zimbelman, M.F. (1997). The Effects of SAS No. 82 on Auditors' Attention to Fraud Risk Factors and A udit Planning Decisions. Journal of Accounting Research, Volume 35, Issue Studies on Experts and the Application of Expertise in Accounting, Auditing, and Tax, pp. 75-97.

PAGE 184

172 BIOGRAPHICAL SKETCH Mark Cecchini received Bachel or of Science degrees in accounting and finance at The Florida State University in 1992. After six years of profe ssional experience he attained an MBA from the Crummer Graduate School at Rollins College in 2000. This experience inspired him to continue his educ ation and get a PhD in a business school. Mark chose decision and information sciences as it looked to be the most challenging and thus most rewarding. He chose the Univer sity of Florida because of its excellent reputation. Mark matriculated in 2001 and gr aduates in 2005. He will subsequently be joining the faculty at the Univ ersity of South Carolina as an assistant professor in the accounting department of the Darla Moore Sc hool of Business. Mark’s wife and inspiration is Tara Cecchini and he has two very cool boys named Julian and Campbell.


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101124_AAAAAP INGEST_TIME 2010-11-24T09:04:25Z PACKAGE UFE0011430_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 16456 DFID F20101124_AAAPNJ ORIGIN DEPOSITOR PATH cecchini_m_Page_149.QC.jpg GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
7857fd287505111b2429a63716528ed4
SHA-1
881bedcc03867ca32d9596c45bfc7491598d743e
7191 F20101124_AAAPMV cecchini_m_Page_137.QC.jpg
363c6a425b466c79f376f02e75ddb0a5
fdaaf61ee7175b296970e738ca2b6d53c501ae1e
158007 F20101124_AAAOJT cecchini_m_Page_161.jp2
a4e76df50ea09105aec798815f0912c0
771d7e7bda0b8fb363323b5cedc38a66aa43ae56
1051986 F20101124_AAAOKH cecchini_m_Page_178.jp2
7a7cee8ed8c803d7fccc0b86ab574317
8cdf88c2ec0404172d2d5471ad9198aee64984e0
4166 F20101124_AAAPNK cecchini_m_Page_149thm.jpg
bbafb26b1509075075a5a5d76d4e603b
73d7428ef4848c3af24b5acb83924c848eea80a2
2370 F20101124_AAAPMW cecchini_m_Page_137thm.jpg
8c5c400b4425a5bfcf99d9021480f2f7
83d2d537d987ecbe3d78076492aeee2634379376
163398 F20101124_AAAOJU cecchini_m_Page_162.jp2
835e1079bcdadf69039d944210d93f6d
decf547f6e34c8c122c09862b9303f9965400623
1051983 F20101124_AAAOKI cecchini_m_Page_180.jp2
569da4e5ef18eee0bfedd30f736bf181
c64edc0480e70169d6412a8a6968fe89d9488d88
3815 F20101124_AAAPOA cecchini_m_Page_168thm.jpg
121ccfa716676cd979d7d90c6a3a01b3
d8bb14eb97d954143836c373aaa149fd9cecc400
2669 F20101124_AAAPNL cecchini_m_Page_150thm.jpg
4cde9f9e293836200b6f4fa42d1ae09c
744c896f1f2b36ff68ebdb97b4926e47a6fba816
3689 F20101124_AAAPMX cecchini_m_Page_138.QC.jpg
e2930525f27db436044ee702ab51993e
f0bfe5343658750a315ca1c4ed359635d1500e44
161771 F20101124_AAAOJV cecchini_m_Page_163.jp2
6d0b6341c9f4a56f8152706525db1ade
9e6e53dae72c221b2c2bc48c80da2ee00c9be3b6
1051976 F20101124_AAAOKJ cecchini_m_Page_181.jp2
82d320c3f80b04e7d48dc584c0cb52dd
52f71e0fbe1ac1c1d0c5ec6293695dd8220511b2
14524 F20101124_AAAPOB cecchini_m_Page_169.QC.jpg
517b145a88f2b12994552401c23034af
918492d242b572fbe21de79069a3f15c57febd80
14624 F20101124_AAAPNM cecchini_m_Page_152.QC.jpg
5d9244839a9508b83f4ab3f8a2443823
25f0ca8a7e7f73b4b0738feec17d6aea36f52fd8
5268 F20101124_AAAPMY cecchini_m_Page_139thm.jpg
38a777459002ddab63422880e44b118f
343e6e1eb093b5e2c3fa2c04939284db6dc49404
155038 F20101124_AAAOJW cecchini_m_Page_166.jp2
567fb1e6c590576017b336c0f252d70a
21b4a5e7f3dbdd4d6d922543e34c5cbd87aeee61
1051977 F20101124_AAAOKK cecchini_m_Page_182.jp2
28aeebdd4ec16aa11a602eb75b8bdaaa
c688b608bf68d83454995b49fe7e6a48a578abf7
3792 F20101124_AAAPOC cecchini_m_Page_169thm.jpg
3ba1a39b496d5e3c178c4c591962bd97
7e5cd2bc7f4ba61c8f4134dee785b810e8b4a1c5
3788 F20101124_AAAPNN cecchini_m_Page_152thm.jpg
9cfc7b24efb07975aaa468ff9765c14d
2b8427d8e3025d56e696a2fadc538145f6289b72
1362 F20101124_AAAPMZ cecchini_m_Page_140thm.jpg
099058354b1039dbc2bb9ddaf996f5b5
10ef30b2ee09fd3b3ca927a34534f9cfde40b7da
160480 F20101124_AAAOJX cecchini_m_Page_168.jp2
37ec5a7ddd719c8f40128c3c3ae5c740
74cfe7dfe077df0d00dc77c25b737de0cc2c6823
1053954 F20101124_AAAOLA cecchini_m_Page_017.tif
42d981b66fe6e193224de837cea1f251
bb268905f8d9d7a8f11ea22574124039eaf50303
19040 F20101124_AAAOKL cecchini_m_Page_183.jp2
c0cf6cee187ffe879642c28cc2149f00
7f72aa21c5429bae0a7907bdc1a11d13f73acef6
14473 F20101124_AAAPOD cecchini_m_Page_170.QC.jpg
5de63e9c0dec8b7d7ff1c12adb3fb207
39a34b2a4635bcaf8dfaed86c1810c690206c0e5
14592 F20101124_AAAPNO cecchini_m_Page_153.QC.jpg
34c03e70566249d8584b146a1c79b406
9bb09d5dd8b0c866041503c4800d67d2487cfe96
156935 F20101124_AAAOJY cecchini_m_Page_169.jp2
e89a86443bf74871792ddc09e2ff5acc
7047dcc1963caec5dd49bf4ced8f1fc049c73d4b
F20101124_AAAOLB cecchini_m_Page_018.tif
da5efb4324775e91613fb7e00a8516df
880b94d424e5dfafd2418878a38e24242c3286ca
53002 F20101124_AAAOKM cecchini_m_Page_184.jp2
ad6eee7fda99f7cf52a438b365ffad92
a6e721e79680da9b6ebf65c39fa7a012eda6522a
3763 F20101124_AAAPOE cecchini_m_Page_170thm.jpg
db52fae2d9ebb6b284424d26ce4e69d7
9a077788d02bbcdb4d395087d83d5a128ab355d3
14510 F20101124_AAAPNP cecchini_m_Page_154.QC.jpg
ed072ec69bd189b32924fccb8b47a8b1
dd99eefeb09111be53f9f805f11ca142a7d4f12b
157214 F20101124_AAAOJZ cecchini_m_Page_170.jp2
971ec2d749571e1e8f77b67ed6a8f93b
4269abeac429ea5333c79dfe6578d4d69df2f470
F20101124_AAAOLC cecchini_m_Page_019.tif
41239f1376c0a284f21d1a4e598ed512
2809365cbd181368b4551be6c847c05897999c4f
F20101124_AAAOKN cecchini_m_Page_001.tif
340ce63e38fbe1e77a8b8c51b0c2a871
b301e4383a61e046d1896cda63321aeadeaf14fd
3742 F20101124_AAAPNQ cecchini_m_Page_154thm.jpg
99abeba646e8d714e57d0e78f182becd
19a36dae97c2bfcc9858012ea69b4fcb626fec68
F20101124_AAAOKO cecchini_m_Page_002.tif
af9d2fa7d0ee603ad4ea43e09015e023
2bbcd71b44335df27aa58f559813e8977fb28a98
14304 F20101124_AAAPOF cecchini_m_Page_171.QC.jpg
018a65ef4960ce75085e8fe4d053059f
69d8619b760d2b536783467989cf04a5f59c4122
3812 F20101124_AAAPNR cecchini_m_Page_155thm.jpg
7a71aeca8037e467fc8f5cb73c502e4f
c276f8bf78fa546e4f02dd9eed430bc75786e071
F20101124_AAAOLD cecchini_m_Page_020.tif
854d8434d00ed55bc63988ef32fdf83f
81bd1dd49053d6c00c80fcd2e9364b1486943e55
F20101124_AAAOKP cecchini_m_Page_004.tif
4d51121edc8c8f8b7ce4f2f61e0ef8c0
943ead8fd3457ed2f2a7a953282a291f331578b9
3729 F20101124_AAAPOG cecchini_m_Page_171thm.jpg
e7a7dc64dcd90e9b98c319244deb64f7
78143f078c513d86f699d96511a0008d1c760b72
14437 F20101124_AAAPNS cecchini_m_Page_159.QC.jpg
b626844ece2b625d39c9d4d893f27dd3
e2df9a79ee259a2f89cbdd2afbe17708f893d19f
F20101124_AAAOLE cecchini_m_Page_021.tif
93730ed02bdf89dec71fde81178888be
186b4705bb8f0245e249c9850091225bcc2481c1
25271604 F20101124_AAAOKQ cecchini_m_Page_005.tif
03547c5f2db38008b50a921e887dad4e
82415a35433af122ef264dbaf113bdea8adb679e
14466 F20101124_AAAPOH cecchini_m_Page_172.QC.jpg
af2ce4c9a33c3ebee7a223146dfe683a
12d52d1c557d66a4033b219b2de5e69e788f93a4
3863 F20101124_AAAPNT cecchini_m_Page_160thm.jpg
df88468db17789766c7c0b8e6722bed0
365ec8159cd793defc3ef15bbe1ebf8adb1a745a
F20101124_AAAOLF cecchini_m_Page_022.tif
3f40b2252bb9e21e947b8330bfadbf6b
ebb44aa45ce1fd32ca9ba9866f836edf1b1d28e5
F20101124_AAAOKR cecchini_m_Page_006.tif
0eaf2797855a12c0a762b300333b1752
06807eb8e55950bb2b0e08d6761d1e59b385cef7
3778 F20101124_AAAPOI cecchini_m_Page_172thm.jpg
f92ebb01d5f2079c27cf01d8bcf93fea
e42811fc624bfc4777604ee39caf67b1c0ad4539
14539 F20101124_AAAPNU cecchini_m_Page_161.QC.jpg
205fc4a28c82c81bca545a07c85b26fb
cf4ac09a9832e14114e2a7d5cbde5f5dcf311ff0
F20101124_AAAOLG cecchini_m_Page_023.tif
24d52a0774ebafac2f1299b455884020
dd36fbacb68fda0e9df162c2936c03158060f936
F20101124_AAAOKS cecchini_m_Page_007.tif
03d16890d827489f25b8be24d0525ae6
70fb7c624042b2f2a8a2c6718762a97c7ca8225c
8923 F20101124_AAAPOJ cecchini_m_Page_173.QC.jpg
60c67602826fd9957e9aa962ad42de3c
d6f9eb7bf855a5e3e6590699962c3e92760cb371
14697 F20101124_AAAPNV cecchini_m_Page_163.QC.jpg
c70c46c347f2ef513d6b6854fa172179
2a54cf6bc3035fd1d980d2ce00d6a635ecb3440b
F20101124_AAAOLH cecchini_m_Page_024.tif
a96a6e03adfd380cce677d40222d1a0a
fb5a04d961c68da7ade8f4e0aed5fde929b2d63d
F20101124_AAAOKT cecchini_m_Page_008.tif
be5e9693ec68b555dc55878b49c89607
496d96505a90affed64fd59f2e31e9205a94771a
2604 F20101124_AAAPOK cecchini_m_Page_173thm.jpg
4f225fbfe3348572d96bf32cd54909e2
c40ad7bc2fae59ee7ba0cf6b7e3e8083f28993e1
3817 F20101124_AAAPNW cecchini_m_Page_163thm.jpg
de2f6ad2746b615637eb1a759fda53e1
0a938800213eb2274806070bb82b7ff765bbb25d
F20101124_AAAOLI cecchini_m_Page_025.tif
df4bec402ecb26d4cc13a7d5f5670621
3fceb2353d60f12b4392912de40ce719f35570bd
F20101124_AAAOKU cecchini_m_Page_010.tif
dae97618ff5595568c0fa8261d52e92e
aa5f23d5713bed6efd18237bf1d5ec9447f43b70
5649 F20101124_AAAPPA cecchini_m_Page_183.QC.jpg
c127c1cdfe4f1273c8830720c1586cc9
f047ac53fa30a1634fa055a4385ae30300b714a1
3296 F20101124_AAAPOL cecchini_m_Page_174.QC.jpg
1e115392523a8042c1c5f19687e6b678
94d624a03d4177214ef03a42f4998a7f301d4953
14415 F20101124_AAAPNX cecchini_m_Page_164.QC.jpg
7b7231793b407033bc962f024796a63b
4cde498c9ee3a9ea67d03683d89ba110b8e1ead7
F20101124_AAAOLJ cecchini_m_Page_026.tif
e0364a2395895b1cb7c21d1485b224f3
0eecee861d72bba72a3d1e804ffe3fdef0039d9c
F20101124_AAAOKV cecchini_m_Page_011.tif
29a195e8f8b74ab2c968b5345e6231a8
9fd3e81197d863d6ec98b5692ad47f173f832ae6
1845 F20101124_AAAPPB cecchini_m_Page_183thm.jpg
0125585ab07470740ddf3c5b5113c832
658d9a2d64c30e37723280ea8400f5fef4869111
21997 F20101124_AAAPOM cecchini_m_Page_175.QC.jpg
d3784d26a2c39eda78914a3d39316d3e
1e0dbf43fc61e5ea198d7f6bde0a0749f8281762
3725 F20101124_AAAPNY cecchini_m_Page_164thm.jpg
d2ff7c6e1d6a66b188c6d22a279d2150
823f963b64940e4ece9bc89f258dbd89d1c8b6ab
F20101124_AAAOLK cecchini_m_Page_027.tif
9f99b2815682a793eaa2c6236eb88fcc
d14705268e0be2d5b8c3f92cae78c44eed44a26b
F20101124_AAAOKW cecchini_m_Page_012.tif
b6be3f462698088c677b515e02fe993d
1bca44f85e1ae607c90bae91af3f1a330fdd1c67
12875 F20101124_AAAPPC cecchini_m_Page_184.QC.jpg
f67135dcdb448ceaf5fb66c831b060f7
c069631ca48b3c701da235e6904b11fc181a2837
5969 F20101124_AAAPON cecchini_m_Page_175thm.jpg
d27fbc87c12ea7777657f3d7bc740829
d4259046949105ec2f45aa7f246febc36777566b
F20101124_AAAPNZ cecchini_m_Page_167thm.jpg
cb722fbd6bacb4e20febdefa809f5458
5d3bf446d941e7b0b7eaab7e5768a9216058f912
F20101124_AAAOMA cecchini_m_Page_043.tif
d6365f5c8af57fe9d3bf33534dc09bcc
68c4447c2646c6509c7f2352d51d0fc3b759b127
F20101124_AAAOLL cecchini_m_Page_028.tif
86daffdf0565293cb3c51817ebb09c06
42ef760503521e8433fea0e6d4af3c81e00e7352
F20101124_AAAOKX cecchini_m_Page_013.tif
dc47e23f3f6c857393d82438a3cc2fe0
f3f26dbb278cd2e73f33a0ed7bce29f5e4065797
3883 F20101124_AAAPPD cecchini_m_Page_184thm.jpg
cbfcf34861aa279697526fd5c8e8ca76
918dcb3e790e7162e844181c9bdc4cf21de6d766
25980 F20101124_AAAPOO cecchini_m_Page_176.QC.jpg
67db98392bc8b329b865c186420b1c3e
4cc781e0fa110821cabe399880cc999ac0d33304
F20101124_AAAOMB cecchini_m_Page_044.tif
9f654fb3c9ebda5b23f57ac9b682aa4b
7267f1e1d9be249a3a973cb4139033820aa32a19
F20101124_AAAOLM cecchini_m_Page_029.tif
64ae43b114598e138e0c6831d64b77cd
e4af4d67b31ee1511e36ecbb8e6d0f4b87e0da19
F20101124_AAAOKY cecchini_m_Page_014.tif
99f80b1a551e55da829102cc8b7fe4f4
ee7e6ce9fade3a3ae9bdcd8c06877d476cc31a35
6793 F20101124_AAAPOP cecchini_m_Page_176thm.jpg
f31f497289ece299cbd51a14c106a91f
82e1661dc4eba7d492bb4a383d33875e579be4aa
F20101124_AAAOMC cecchini_m_Page_045.tif
a2485aeec9102bf158064eca04250a8b
bdb09d3e8a2cf8ac9c67ab51692c890f5958bc1b
F20101124_AAAOLN cecchini_m_Page_030.tif
7b3d77fdfec9813d227e74d367d45ed5
672a1541cca8148f8d8f2f0cf75f374d272eef19
F20101124_AAAOKZ cecchini_m_Page_015.tif
ee3b01aa93842ce6d28d35af97a1db57
c4f4903943675d2fa03340136e87e0edd1d1d81b
6926 F20101124_AAAPOQ cecchini_m_Page_177thm.jpg
4d836db71bdf23747d8c92c5835e8e07
208fe2ffa98a496d7c077c2bdde3fa745cc96c03
F20101124_AAAOMD cecchini_m_Page_047.tif
187e34046b05fdbf4c3d2fe78deed724
ed284cad5da195d3c0ea245cc8c74f3b3456e437
F20101124_AAAOLO cecchini_m_Page_031.tif
bf39bd1e7e6ceae4aabe561ac68e9c93
07235d95ca4df67bce1d44114eae62f088349ab7
25562 F20101124_AAAPOR cecchini_m_Page_178.QC.jpg
426b36ade7349b337d3d94e6315e0968
e7a09ba55da5bdda31de0d5398120bd912cac90a
F20101124_AAAOLP cecchini_m_Page_032.tif
9a51ef7fb908254508ba35a03bf98da0
0461d68bb2ec903e92fc9fdb0dae11b0d4d854ea
7053 F20101124_AAAPOS cecchini_m_Page_178thm.jpg
86d496ef844697d5e06479dfbd7d2673
abc49a7f09f1cae5425c20a6bacabf9cbdca3e52
F20101124_AAAOME cecchini_m_Page_048.tif
1334efe78ac1431e01e5c439f8958c3f
55a6601253ed601e77f09f174de3ff2edd2a6502
F20101124_AAAOLQ cecchini_m_Page_033.tif
843235645dbc9da2dff547c4d6367ab3
d54f2c83d16fd15c4bce688e497f156ace910a30
26738 F20101124_AAAPOT cecchini_m_Page_179.QC.jpg
04e78e727f814ad383cf9943ad2cb48b
d29af0b8701dcf562695ebfdd333612cdb2bc17f
F20101124_AAAOMF cecchini_m_Page_049.tif
74fb970c80724b17fd99e82e347a8d4a
2c292d18869ac0e6cbcfa54b0c90d8d87e844daa
F20101124_AAAOLR cecchini_m_Page_034.tif
c5be8e71cd429c7364bbadcb22ba6a2e
14eb7ff15d781dfd41f474e333efd7a3edb9b0dd
6898 F20101124_AAAPOU cecchini_m_Page_179thm.jpg
bfa0f4aaa6c69a3e44bcf83ec1f07ca1
5729a8cfb744a87192943c2870d1b7709d25ce7d
F20101124_AAAOMG cecchini_m_Page_050.tif
466e061e327f490776010e8536959ad9
cbfea6c2b3640e1cbc1f9e20cd14f20b9d2082a5
F20101124_AAAOLS cecchini_m_Page_035.tif
a49daf6568a09a1e4923dbe7338c81e9
39cad99b353fc1a7c89d6fbf28603e12c8dbdc80
27556 F20101124_AAAPOV cecchini_m_Page_180.QC.jpg
3291f55af0bc7f84aee126c2b4f8810d
83a595c26631262fc0e8f273e3909da623815fb1
F20101124_AAAOMH cecchini_m_Page_052.tif
0f11ae897920b894e557bf94d1367a4c
ec41f5c3c1a026c67a4d6415c6bb9c056fe9253b
F20101124_AAAOLT cecchini_m_Page_036.tif
7215a2df9170fb7516b4b2e6ecf2e06a
df312e721ebde4b400b61ac6463a9158a0e38a53
7390 F20101124_AAAPOW cecchini_m_Page_180thm.jpg
a3afbf314629ae41258f5bacad379e8e
4b2b924ccacf206d110cc61e2efaa90ef9868862
F20101124_AAAOMI cecchini_m_Page_053.tif
622f7b0e5819efe034f771f418f8d67f
5aaaa6b877b36e31833d7bacdde6da30c8b18e42
F20101124_AAAOLU cecchini_m_Page_037.tif
3c77df2978a66587948506651caa6ae9
3c0f82fedc105614b24dfd61cc2794593a6d49b6
24665 F20101124_AAAPOX cecchini_m_Page_181.QC.jpg
8bf76bd65fdba452a6d318b7db12fadc
88657c9a5e11f087cb138253352d1b077eaa9cf6
F20101124_AAAOMJ cecchini_m_Page_054.tif
cd1b727b35419d25f7058b2166c915e2
0ce3ecc2e13baff85952b67002fcb9b3d6b33f98
F20101124_AAAOLV cecchini_m_Page_038.tif
dbbc34bac8bbff3c0f3cd92985a31a64
a7f2bbad2e64258854595bcc0f84076e94399d75
27981 F20101124_AAAPOY cecchini_m_Page_182.QC.jpg
2fdaf65b47e941762835d4ecbeca0469
05abd52bde7ba073dc0a795ca08106241f1e4192
F20101124_AAAOMK cecchini_m_Page_055.tif
bbeddf4833d30f7f712283e508c35dc2
d5f2fb9aa90d1fad588799ce19bc8002b1e775a5
F20101124_AAAOLW cecchini_m_Page_039.tif
033ef2266c081690d0e2419a29911382
bc2ba7fcc5ca643ad55ce4ada50ca4b27b185c4e
7104 F20101124_AAAPOZ cecchini_m_Page_182thm.jpg
096baf0afe9428583e35e013db671ba0
7048f59f11ad29afbbbeb1905753e76f6e98d7b8
F20101124_AAAOML cecchini_m_Page_056.tif
765413904b0b73ae2ecd8c52a7aa7026
7f52cfcd75f39fd226dd104d32793983ff53377c
F20101124_AAAOLX cecchini_m_Page_040.tif
70eac0dd1b4f593e126ccbdacbdb29db
379f15337eb7b211427e4b78632b10a6aab74b27
F20101124_AAAONA cecchini_m_Page_072.tif
89d407d49d1fa4c395d924308e1364e2
241f19f12de3b405a99479c0760988070d20938a
F20101124_AAAOMM cecchini_m_Page_057.tif
ee0a77299fba5ef5dbbc4f7278f76bb9
984bd2c306cec7d9899d90aad3fd7b08683b00ce
F20101124_AAAOLY cecchini_m_Page_041.tif
530111eca4133b727f2bdc3ff6ea1fea
83f7f126e7b3b6d84d98c959736b032da3944709
F20101124_AAAONB cecchini_m_Page_073.tif
df878604b1a7babd881e261f7e00d947
dfd08e3617493f45da5c66ecefcc8e28f907bc48
F20101124_AAAOMN cecchini_m_Page_058.tif
19cacbb340c08688d48ad2cbccc24965
9a81e9fb76a10620ac6c5d31424e121a8d7a9c5e
F20101124_AAAOLZ cecchini_m_Page_042.tif
16df1d2a03b502993cde4c09a8ebe43b
829acb25274feee2cb26a1e1929ff5d41c065258
F20101124_AAAONC cecchini_m_Page_074.tif
d64f6032c368d6437a8c70d684e15602
3e5a5570cdd36eff7bd1228d183bad0535094b9e
F20101124_AAAOMO cecchini_m_Page_059.tif
6f1c390834175ad15c85d5080601d60e
84b1690a5ef5a8775e1edb58b866e649861a347c
F20101124_AAAOND cecchini_m_Page_075.tif
c685d1728e4ebe88e0dc83571dba84b0
665425906f340b8c54fa5a94052afa4aa7d2b8bd
1054428 F20101124_AAAOMP cecchini_m_Page_060.tif
771a2dbf195fd9a518631785c506584a
74b70ab274b3fb455c6254bb0983a860600b2435
F20101124_AAAONE cecchini_m_Page_076.tif
ee575b2d23e7e0d81207d15dc45b864e
67d01be0975e42c0f7025600762096d17c1c831f
F20101124_AAAOMQ cecchini_m_Page_061.tif
f0acb895fc3ed3bedab6965e7252f259
96d4a07fa76cc7fe8a9bdcd912e287f78cc72783
F20101124_AAAOMR cecchini_m_Page_062.tif
accc893bfb3efd91b9980362fa9aba9f
8cb52b993f78b13e04f97f0ba0ec8ecc87416c3b
F20101124_AAAONF cecchini_m_Page_077.tif
c910a6057a5ec124cae485793ed1a727
90fe2493e3c5ab36f6f761257baf66bfeefa28f6
F20101124_AAAOMS cecchini_m_Page_063.tif
6cdb04b3890061fa9a144e2ef46935da
cd619920ee055c67b4163fc0aa9103638a63f49d
F20101124_AAAONG cecchini_m_Page_079.tif
88fb0c7658b15b55b1d3f9029f277b40
dabf3dd8d6ff60f1e26a236ea3e7e15b615e90c1
F20101124_AAAOMT cecchini_m_Page_064.tif
37b9789eaa6586345d4ff13dd75dde10
f8c50757fb9b6497e355d22aa5b156af8850df02
F20101124_AAAONH cecchini_m_Page_081.tif
841cb366e5be8c5a03729af64c24e858
ad377700c28584c951e0ffcf4d7cd7992747a789
25265604 F20101124_AAAOMU cecchini_m_Page_065.tif
de22f9150678b1cfce7df384497ad504
ca7f4f242b8a7e19a981ea1da178d659e7240609
F20101124_AAAONI cecchini_m_Page_082.tif
4325a28442487d3e830096f34390ba1f
9abfc6cceb092f8595ebcd077e4becc4ad326433
F20101124_AAAOMV cecchini_m_Page_067.tif
75629ea3994185ddd6f0e79f08dc39a7
3f9b1c4bdbfd31fcf5649fa570b954eaa1cd0d5c
F20101124_AAAONJ cecchini_m_Page_083.tif
a7d009283e30cda068f6038006b842f1
1a283a8021b46cac79e8978076d399b6760bdbbd
8423998 F20101124_AAAOMW cecchini_m_Page_068.tif
e8f160cdeaced138e8b258c58b56996c
be90060554c59e19266a5bfc030348e52be123e8
F20101124_AAAONK cecchini_m_Page_084.tif
66a68ab3ea90dd599e2925b2e5112405
bf64306a36ce1a45db32ffc5d20ecbd8f0760bf1
F20101124_AAAOMX cecchini_m_Page_069.tif
a316206f696ad4e910c99f0d97648774
7f1a61ab124fbb56cc4856ee23bf25234e621d7e
F20101124_AAAOOA cecchini_m_Page_102.tif
2ff28c931494f3b24318fbb93cbc36eb
30b75b370155b53ee15dea6299e560e9bd8bb60d
F20101124_AAAONL cecchini_m_Page_085.tif
eb0e47f3cc02210da51675d445a53d7d
7f9417db8aeaaa769454ed2027537aa50b508e59
F20101124_AAAOMY cecchini_m_Page_070.tif
68cca9f43a1e58761b989a24e63e43fe
aa31c3dd9e154c22e2f5e5e7cdecc942be2b75a8
F20101124_AAAOOB cecchini_m_Page_104.tif
8d64564ff0f6e92abd73f9b991fc0727
fbf1a9b9f9aeafde9b7f8af5c427207fa4d8798f
F20101124_AAAONM cecchini_m_Page_086.tif
f53118ca09aba8e01aed91d32e2596b8
0dac90ff434525f145faf8eb192ec4914991b5b4
F20101124_AAAOMZ cecchini_m_Page_071.tif
93a0af7e9444dab6d44187e274f7019e
2a66a9bb0d75a0c9e72c2d05d0655c7b055a8529
F20101124_AAAOOC cecchini_m_Page_105.tif
5261c853dea25986e7f2e2f2db505fa0
cb1ab6239ba25cf152c73a060fcaba7a53b4c4f5
8425398 F20101124_AAAONN cecchini_m_Page_087.tif
9f41c83bd451b36e536bbb1c582702bf
50b0a585ac6e3cb983f546b9ee963de53d6e0be6
F20101124_AAAOOD cecchini_m_Page_108.tif
a7270aca514fcd5fe34f0a15134c47fb
1588e7f8e999f98d8d4df1f7f13764b755cba00a
F20101124_AAAONO cecchini_m_Page_088.tif
f87942bdb27365de442fcfcfca824a91
42e35f83766f9b3d68ccb21499571f415927cee3
F20101124_AAAOOE cecchini_m_Page_109.tif
02d8c42f08f23a6fcb661504ec58cfe5
6e4e14159279eaff4e22913a6f88533750683c35
F20101124_AAAONP cecchini_m_Page_089.tif
d34d950eaf10040710a04451e330ca24
eb29549d4fe6bf771f9bc58fd998b0991dfbc042
F20101124_AAAOOF cecchini_m_Page_110.tif
26a821a7924a46519d89beffc6e9fefb
c7b9b9242d874e39e5136eab7afd290346a5a57c
F20101124_AAAONQ cecchini_m_Page_090.tif
5a96143af6f60b782835736de36c9290
e67bdaf75d3b9802da3ea72652833fecfb805b74
F20101124_AAAONR cecchini_m_Page_092.tif
8c1ba368739ca09735342241d175585d
33673ef4f06d2236aebaeec7684e0b60bf2fe8cf
F20101124_AAAOOG cecchini_m_Page_111.tif
ed2824f76a106d6d5618cacb1f39b7f7
f340eb68acafc078d98110a811d42113e2c553bb
F20101124_AAAONS cecchini_m_Page_093.tif
915b149141352359ff42a192ad716bcf
4f9209cf07fc638ac73ddfb9bf06252216a93bf5
F20101124_AAAOOH cecchini_m_Page_112.tif
b8d2926c13fd46aac51ed7d13f11aa89
2d70779dcfec3d4114e6232e408f7bde9c61292e
F20101124_AAAONT cecchini_m_Page_094.tif
d60e1871ed3cb512ba55c54ab73ddd89
872ad3bc62050e28bdc000f6930e8b5d0b086995
F20101124_AAAOOI cecchini_m_Page_113.tif
0fe73d0fd97c5e555bc9702a0d5f8e89
fd63390ca299da3edd34e57500357a5f67d28d44
F20101124_AAAONU cecchini_m_Page_095.tif
87b82b61dd5e9fda3c64468727e16cb2
e422e16c3191df182fa66f9eee6261c3b51bbce6
F20101124_AAAOOJ cecchini_m_Page_114.tif
f1521d933396fe8db6292d99cd43d8a1
6ac72b9f9858adf4a771afd6360b2ee49009fd30
F20101124_AAAONV cecchini_m_Page_096.tif
dd1d2544a73a4ed4f43efbf88037181d
ba120eee24074e79489ec439b017df48e93d044e
F20101124_AAAOOK cecchini_m_Page_115.tif
bf7df39e3189a81e578c9cb8189af0d7
9c1a6f9b3d4cbc79707dc12b5e7b2df99f15514c
F20101124_AAAONW cecchini_m_Page_097.tif
7391394320366dbc4cd8bca172c34786
79a0cab429558eaf25cd111cace2407a6b29fbdb
F20101124_AAAOPA cecchini_m_Page_134.tif
6773c768167c26af882fffb839dc16e1
89a26105d435ea1ff0967aef183d8f65522fe0c3
F20101124_AAAOOL cecchini_m_Page_116.tif
11a7bbc3f1f9ec177734fbeda79c3390
7e6a944424ef5f0f5443b25880dad58866e34d20
F20101124_AAAONX cecchini_m_Page_099.tif
8a4d02068f8ce272b6d06082a143825a
e1ab262aedcd5f9f8936952132205e2b44b8f7c5
F20101124_AAAOPB cecchini_m_Page_135.tif
a6c946e2d84cf61628677339927a0d59
23aa18df2f43d2fcd37a220ca4b22720d7a7bfd4
F20101124_AAAOOM cecchini_m_Page_118.tif
aa25036f35e761d5461830d9fa1e3b0c
27079de8dc56861870b6a83e5bcf4114967df074
F20101124_AAAONY cecchini_m_Page_100.tif
13c881f4a748c60a50bbaac36a14c947
6f65a889d6f35eed9a8c201528d664a92b5842b0
F20101124_AAAOON cecchini_m_Page_119.tif
9aa141247fc792c5189a4106ab37f8e7
a82a1ff66e2361dee0f847676de98b1091377293
F20101124_AAAONZ cecchini_m_Page_101.tif
a0b5bb3e56b6c61710eeb54dbe21018d
f621931ce691fa5ebf23784422cd2220a78fa55e
F20101124_AAAOPC cecchini_m_Page_137.tif
9da30e177b75fd0a7caf08788fd6a4e3
d8bb0a1a432029ae068a30814b69d70b84c0d93c
F20101124_AAAOOO cecchini_m_Page_120.tif
42898f408463fa88f800ff60a5385bec
ecfed724320e169257e4d4108d2ebe52795bbe62
F20101124_AAAOPD cecchini_m_Page_138.tif
c31e900667fbb16e368a47fc57ebfba0
c558ceeb16ab162f8ed3b2227b15af75ba32389e
F20101124_AAAOOP cecchini_m_Page_122.tif
5c95352db69a0f01de655751d840c59a
56d6e93c6f23eebf0ca45324b55edde6212a269d
F20101124_AAAOPE cecchini_m_Page_139.tif
ebdf0b942a301aefa85dd0db7d7fcb77
0a2e39ceabb3cdc3fe0422cf972cc50ac65c0b21
F20101124_AAAOOQ cecchini_m_Page_123.tif
3d97a8d0b070e66a41b239aa260d75f5
0263a038a4a23b4ce24a4213667dbcf1cd069203
F20101124_AAAOPF cecchini_m_Page_140.tif
6c4d3b14201585754bae82bc395a7a94
6380ec24929aa84b13bc9015cc2e389015ae0d02
F20101124_AAAOOR cecchini_m_Page_125.tif
866931cbd1ccb6170a38e3eb22718f5b
a0f15c2db325022dea95cdea00dc5121b62c2b40
F20101124_AAAOPG cecchini_m_Page_141.tif
b30d7c3e04341f57d224b690d9081889
e23a440cfe9c4719185d28ce93612105ebe3b8b8
F20101124_AAAOOS cecchini_m_Page_126.tif
e14a5f318ef575d7e65d787a10dd71d3
dda4b7b1446aa477b18a71ee75787cb04f132fa4
F20101124_AAAOOT cecchini_m_Page_127.tif
5aed0b74e4eb1a3344d1b90990729a09
8fcc754ed04506a5d3867a540aab7925e4c06003
F20101124_AAAOPH cecchini_m_Page_142.tif
9e2d3dab8eca250a75a59c29289028e6
bc7f25e9c366901021e7e2af101f20915573e75d
F20101124_AAAOOU cecchini_m_Page_128.tif
d2ffa242e56d04e9d4abb73d507326f3
69b906dc8b9156b5664ff6122bb89b7d624fb90a
F20101124_AAAOPI cecchini_m_Page_143.tif
e43a48afd698686f5fe5da1110b56828
b8ea20217d8a5e69f8196f0e65303daec8eacc41
F20101124_AAAOOV cecchini_m_Page_129.tif
104c49fed3dcb63d61ea1db329813bad
c4c6e0df06f2dda4bce1c44cc5d6e5e978805961
F20101124_AAAOPJ cecchini_m_Page_144.tif
379bedc1a3fc35894f1581768f74e163
8a2c9427533459896259910ad522fe98816ac501
F20101124_AAAOOW cecchini_m_Page_130.tif
767aa4a11075551700e57c5dcb569579
0ca2c2cd9a6ae673771fe336a95b1314576f2dbf
F20101124_AAAOPK cecchini_m_Page_145.tif
e4b84f2cac5cfa5a4f59102b32845753
ce383e49cbbc1619eb7a5b5be4fd65ce5b1b9c7e
F20101124_AAAOOX cecchini_m_Page_131.tif
87d2fe09c767d9161105830a60b643f5
b0b900c58ad71d70d8caf0004dc4df81f4c6c624
F20101124_AAAOQA cecchini_m_Page_164.tif
6e8b8e768df92db15c52150752c73b61
284d626006a3cc64f3b0cd236eaa7891c3551ce0
F20101124_AAAOPL cecchini_m_Page_146.tif
f1eb144788af7db05c384a8cd9b11854
37559239a2103bda8a6109275b48a7a50ca95e4d
F20101124_AAAOOY cecchini_m_Page_132.tif
4de621fd93f657a245acb5d4c2eccf8b
0e8a9bbf97dc74335d7b0ae4a4c1791453a0e241
F20101124_AAAOQB cecchini_m_Page_167.tif
ce530f2561ad55da037790fdc3c9de2e
9c88c7befb7473019d6bbfa5d2b4866060db2cc4
F20101124_AAAOPM cecchini_m_Page_147.tif
e55bf95e0d47ac4867d794adaf4803fb
58df340fe588f86678c40bad58042079bcf20f3b
F20101124_AAAOOZ cecchini_m_Page_133.tif
f38d3c9b719b1d618b5b5169834b287b
6d2034323618a5f698d8116f703b8dd852cda63d
F20101124_AAAOQC cecchini_m_Page_168.tif
093e031f704368f4e812717c0b6bd24e
7fb7f7a13a18fb081fe3ee994a2925c7fd1bace3
F20101124_AAAOPN cecchini_m_Page_150.tif
4a25c34d7d6aebc7f2ac1e883ce96f6b
68f6d11ddcb92d41351046406fdc77db0d883cb5
F20101124_AAAOQD cecchini_m_Page_169.tif
ca2dd9dca110cd41921f6dbadfa229c1
c076c2f614957d2f6232e87a978ea8a19bb99557
F20101124_AAAOPO cecchini_m_Page_151.tif
3572a961e5b383cbccbf29337e8c07e7
bd7fe2241d5ef12d349a0adc15c847365492b429
F20101124_AAAOQE cecchini_m_Page_171.tif
4c34aa922198ff086616c63f0c03153d
e2917aba19567e1b435c560fb1fcd7bf087fa73e
F20101124_AAAOPP cecchini_m_Page_152.tif
1bf4e63d78db5fb4cc23862b48127a68
0b0374b166786eec07769336c2d9d6da49bfb451
F20101124_AAAOQF cecchini_m_Page_172.tif
3c5a45b3073c907e701a815db21dd0d1
4ad5cc371c3c38eead20de2cb2abaaa6f167ab8d
F20101124_AAAOPQ cecchini_m_Page_154.tif
e4afda36588a1b40e7ef9b5cfbebce7f
8a3c316d0b9cbe304a4101b3b8669bec55f4cc02
F20101124_AAAOQG cecchini_m_Page_173.tif
60444b7de2f8c143998126805a55fb8b
c1679d99d1a1709708a410f1009f8206d4965c9b
F20101124_AAAOPR cecchini_m_Page_155.tif
69ce3afe4d92ae1e70735cc55cd64b01
600d5455b290f8f3b6f23d9418e3456855d38edc
F20101124_AAAOQH cecchini_m_Page_174.tif
8dcb6f40fec371b2a825a4a8aa0bb64e
0c634bd770d66a1e3a7b19a389456433dcaa6dc6
F20101124_AAAOPS cecchini_m_Page_156.tif
77c59863f0859e8f96ff19533a20e1f0
4c0ec98013b8be9a11e9362940ca675feb852b9a
F20101124_AAAOPT cecchini_m_Page_157.tif
bfffbabf59918a4c22241955eda15f95
f1c23ccfe45b0a46cb8518612aac560497fadfd7
F20101124_AAAOQI cecchini_m_Page_175.tif
a8c6c8453e030f32e4b6b275bfbd4736
f5512b6ebd55dac01a6f4e4400abcfc4d49f6cdc
F20101124_AAAOPU cecchini_m_Page_158.tif
21e186b623a12b4a700bbae364cfdc30
9ccc7286091a4e5cbd1a22f6875e6163aab438ad
F20101124_AAAOQJ cecchini_m_Page_176.tif
b727eb9019ae34668a3797cf4345ae48
7d8895e0feffbac87147b6b978f199b85045bccc
F20101124_AAAOPV cecchini_m_Page_159.tif
083d816a9013fad2a601e9d309cf8843
13bfe1d0328ab31d788e49fab53e6042b6bbfd0c
F20101124_AAAOQK cecchini_m_Page_177.tif
3697d97e9d9e3878c9f47da67dee13db
6836c12cb9a8c58d3ca2999695f968a8c816b1ba
F20101124_AAAOPW cecchini_m_Page_160.tif
ff6ec36d397f0cfd3d2f826f45d2c79c
2e674296d5303be017d495209df971a0828c097a
4230 F20101124_AAAORA cecchini_m_Page_011.pro
fdd2d6063c9faf8e79983d08732a7a76
57807485027959f273bad3cb8aff755fdf05b87e
F20101124_AAAOQL cecchini_m_Page_178.tif
c56974b789df737a6f232d7baf99fb14
2dc41e4b95afefeb08dbf6106a84d6dc63064864
F20101124_AAAOPX cecchini_m_Page_161.tif
673f9b348403f5752332c195106f03f6
56809209983a0c2d0bb96022d22b9f14916c4733
42116 F20101124_AAAORB cecchini_m_Page_012.pro
cf96eefffa4643cd5176e67987e7dbf1
eb110dbec9abb06089df5f36aeb6e4828e282410
F20101124_AAAOQM cecchini_m_Page_179.tif
3b3ae540fba497b9ce0d5c3048f26b88
f6fbad6a46755bed6f23943675a67f02c3674d0e
F20101124_AAAOPY cecchini_m_Page_162.tif
64f15c87dadd37765ac135bed8ab5f1a
358755badbfc60f1ab34d7305cbcf4af3262f138
44204 F20101124_AAAORC cecchini_m_Page_013.pro
1b047b23f158ee9df6be7094308a383d
cffcd622985fd0a25ba3e324214babc13161a2e9
F20101124_AAAOQN cecchini_m_Page_180.tif
d137b7fb936e76940790d3e63f1d3376
01f8030b9040405bb1e2b2e8d3ea7368fa06a327
F20101124_AAAOPZ cecchini_m_Page_163.tif
268776d440d78f48ad252462fbdc43a5
497569ff4d7c64f9dbce9d1e72ca9c11ae04b50a
49859 F20101124_AAAORD cecchini_m_Page_014.pro
f826e708ad742f8146e6dee3c4e934f9
ee14170f509ec436e82c3df90b8631cad4852285
F20101124_AAAOQO cecchini_m_Page_181.tif
70775d24be1a13e027942affbd1e01e4
13f695f5735e85a6ef92112ea7821c1bbf627ae0
44655 F20101124_AAAORE cecchini_m_Page_015.pro
27b5d858eb262c02776434797fd357d3
2eae4c604f24953bf0b21c51fe2cd88dade90245
F20101124_AAAOQP cecchini_m_Page_182.tif
bc21fb9f1df1895525877355f7bab728
ef7c2ae5d9b52061f686c5e78c6fd4187ce2253a
23761 F20101124_AAAORF cecchini_m_Page_016.pro
6568b929c756362d9bdc1c06fcc39c3a
5bfc395e33014cecd559e98d14ebc9e4dd6c9779
F20101124_AAAOQQ cecchini_m_Page_183.tif
bd225d35900f1b5a9321f12495fdd35f
7c573a0dcbf8de8d6db441f32d9e8e398380213d
51730 F20101124_AAAORG cecchini_m_Page_018.pro
e313b3c8c8f7bb121d21c1f62bf07f2e
d996d267422822b3732f577aeeec38041487fdab
8227 F20101124_AAAOQR cecchini_m_Page_001.pro
e960aec6c06b11c6811c79ab2b884f34
240fe88ffab59ffa5f0a827f6bee712bdf96bf4f
50156 F20101124_AAAORH cecchini_m_Page_019.pro
ee1ac562e62cf73b736dc86717c16544
2f6528da45d6338634e061f1273521a46f651332
1124 F20101124_AAAOQS cecchini_m_Page_002.pro
38024a820bcc0751441670e90839dcbe
f2eacec4591f0b8f44b1d2373830989b5df2122b
48881 F20101124_AAAORI cecchini_m_Page_020.pro
040f15d655bc5143e48c60219b2a5f4b
b92bc7c7d2bae97efd9b4e1b1192ca653bd1e4ed
3241 F20101124_AAAOQT cecchini_m_Page_003.pro
c01713572d4d2daa86c2104f337b0440
0fae636cfd695d9116a141c0f4e680a63f3217be
18962 F20101124_AAAOQU cecchini_m_Page_004.pro
ffaf73c0786a9bca459830770408afc3
f2ad6c88164681b3891f2ace7cc74689e8707e50
45838 F20101124_AAAORJ cecchini_m_Page_021.pro
70ae60287c5e3b6d4c22b68e921d2e83
d58f1d7a0224b0c0645fd1ffd676f19f64e0766c
69182 F20101124_AAAOQV cecchini_m_Page_005.pro
d7291b2806f0228bed422975251dcfb4
eb7e83781d212a8865b8a19937066a42b1d7a92f
45539 F20101124_AAAORK cecchini_m_Page_022.pro
738e9d744f6ae4880bcb59a165fb4282
52c5d84cf55f936220bfac4341fb6d7f79d5555f
97157 F20101124_AAAOQW cecchini_m_Page_006.pro
4e2d1ee41eda9a72e4ee7629a12527ba
4fc48704dbc9c4e1a1a3da7badffe3231f42041e
48030 F20101124_AAAORL cecchini_m_Page_023.pro
92f2fe9757e8b00e29eb39119aa91c54
21595e4916f09e88ef12ac8c47ac68086e7d37dc
24866 F20101124_AAAOQX cecchini_m_Page_007.pro
455b0e145e4fac62ebf1003d1dcc4c80
dae9598a599bbf67f6d89880a16b877f82d84b33
41777 F20101124_AAAOSA cecchini_m_Page_044.pro
ba31d20523eb07162543ed2140fd1239
f92ae74340164af739672a1ea5a640c2e726d54e
50443 F20101124_AAAORM cecchini_m_Page_024.pro
d77aff53ab31b1b1c9ecbfa33a809c00
2007f4feac447434cf6dc0fb80522d3405abf8c5
53160 F20101124_AAAOQY cecchini_m_Page_008.pro
7d88d8b3bc47f745f3dc5aab5f3e7233
be3eda39dac04414d5b068ef3ef590c05b5b54ce
44613 F20101124_AAAOSB cecchini_m_Page_045.pro
68f95574b4795be5fe6b4a0ad98a1953
3fa5cb2c1c6b8197dc5d6fb6688c9a140000bb19
23137 F20101124_AAAORN cecchini_m_Page_027.pro
250e1094e52d95a6183556f7277849a9
e02ea6af13b4370675a02389d467d760cd6b08cf
55569 F20101124_AAAOQZ cecchini_m_Page_009.pro
e41b09051f64a536000a01f060dee802
81c7142e959b3133063fda4a34dce7a7f48766ac
40136 F20101124_AAAOSC cecchini_m_Page_046.pro
087ae388599f61bd758b80bd04b821c0
1173e3bf03b7060eaa681dd4bef3b0a2b28fef53
42989 F20101124_AAAORO cecchini_m_Page_028.pro
68ec3fb153e3c45bbd3b1a90bb88cc82
e4fc49e7656b0358aad08e0041fb7774832b143a
42016 F20101124_AAAOSD cecchini_m_Page_047.pro
2c66385a03a48472012fed4399fd52de
06fe4860af2d5e074ac4daae27eefd61994af818
43132 F20101124_AAAORP cecchini_m_Page_030.pro
bbf8b74a04619316915e078490dc1cd5
bf863847f12e78cd18a367e2fedd7700c937349e
43837 F20101124_AAAOSE cecchini_m_Page_048.pro
d1b6f032bfa51f947693c098349148bd
bfd96d2c62d0c49d88b72ffe27d77c9ad80860a8
39546 F20101124_AAAORQ cecchini_m_Page_031.pro
fb7e9600f92f2e9ae63946bc1dc6ecb0
8f7661d5f3f0a03909663363d237551a89dd593a
30522 F20101124_AAAOSF cecchini_m_Page_050.pro
bd565c353e23749a862f5e616627a1ab
a16fe0fd32cc518a1a0e5f5515a5d6fb4f6ad8ed
49902 F20101124_AAAORR cecchini_m_Page_032.pro
073de2b63fd89afe3248bd5905034964
9f9f2966516a7a3efbd45a0a294a3acfcd82dffb
31783 F20101124_AAAOSG cecchini_m_Page_052.pro
ca49358d598079fa3663308ddb4a8d32
3f26d5bac8dd3b54e893bfc0235365069f22e551
37546 F20101124_AAAORS cecchini_m_Page_034.pro
4ad4f8d3e77f87335f28cc4f41d462bd
d23367cb1f55917a9b6599840fa867b1f6e6cb61
50633 F20101124_AAAOSH cecchini_m_Page_053.pro
a047589e125a41a97ec794504a1a0fe8
ed8a28c23553d6828a1b7230d703f3f9a54f3596
40025 F20101124_AAAORT cecchini_m_Page_035.pro
b675d533951ed5090633e29f4a4ea038
5de6ecc393fdfe1ba3c0a6a2dc83e74e28e95908
20730 F20101124_AAAOSI cecchini_m_Page_054.pro
4f8e59857b871771f4df345f407353d5
648e2e4e4aced6265256d8f51543e0a0f449c2fb
46365 F20101124_AAAORU cecchini_m_Page_036.pro
60171d8e26f47bec3656d73c90677fab
be3c51289204af337adc7f4cc7c900fffa44f9e1
45981 F20101124_AAAOSJ cecchini_m_Page_055.pro
d7a72b0822b90776fb2c40a8a28e54a6
49024e7aeaec26d99f5db1facc292582d05bdc6d
44694 F20101124_AAAORV cecchini_m_Page_037.pro
6881599ad43894522f75ab72ad8a4103
d4ec487b7b2bd277264d04500ab437c9bf5916f1
45746 F20101124_AAAORW cecchini_m_Page_039.pro
782547e096fb65450a2121c5cc4ceabb
872135eb1a274c8476c93ae3bb7b4cf0533435ed
28631 F20101124_AAAOSK cecchini_m_Page_056.pro
97ea28473ad11ca5acd6b11908de6d13
e3a0fb8aabd55fc879022fcc7418ea66a919914e
39868 F20101124_AAAORX cecchini_m_Page_041.pro
b2e1eedc6f0458d15ba45efac556dab4
ed11229cad4b3356b01889f969c86ca7cd527fcb
39657 F20101124_AAAOTA cecchini_m_Page_074.pro
672c0054b80412814544d4b64c26191b
837e613fec48b9f0d19ac4dccbcce7ef5ae68c78
46722 F20101124_AAAOSL cecchini_m_Page_057.pro
6ce6893052d232631f69a9f9ad19e96c
3b11654c6dc0c8d22d1f9a579f6f8c9ecb14ba8b
45984 F20101124_AAAORY cecchini_m_Page_042.pro
7435e2c93f45fd1c2b77efe675d1790e
e0fe7e43b2fcc79d77cecce3738275d0f5d0d308
26800 F20101124_AAAOTB cecchini_m_Page_075.pro
1203f18493dbc53705b472054cc2491a
395ec65e4779cf99b0cbb91521619b1585f6ff0a
40567 F20101124_AAAOSM cecchini_m_Page_058.pro
f54d5976a6d8679b7a2bc3a471709b3f
6660e45cd3f6ad50d0ce4a1e8384f5274fe08fd4
39348 F20101124_AAAORZ cecchini_m_Page_043.pro
65e9a2094b80276a051c358edfbe6755
02db23a3eec56b0c641fa447cb233c3451f489c2
51515 F20101124_AAAOTC cecchini_m_Page_076.pro
1ba8d0c92461103f923c62894f03e0b7
391cceed91cf4fe4e9cad794f74133957efb1399
34026 F20101124_AAAOSN cecchini_m_Page_059.pro
6cbcf1728ea167619803920711b76ce6
8be1d64d1fd77f5c81c635b9300675ae3daee58e
21652 F20101124_AAAOTD cecchini_m_Page_077.pro
cd98c59e6b5ed228552ff182e73aaed3
e3c19ae77ac1f7cd6264a5f0605af73530ce32a5
3751 F20101124_AAAOSO cecchini_m_Page_060.pro
6f30c7c8d54888591f941c3cea123b3e
7dc8d9e51a4a3ab6e291e877b8710aad2974ca18
44562 F20101124_AAAOTE cecchini_m_Page_078.pro
150e35f6138bdb8bfe463cf81d5cb0ab
5e94ed4ac6529cc5ae233283bff831a36cb2fe8d
35055 F20101124_AAAOSP cecchini_m_Page_062.pro
60b8de5694e9076f79dea98331519965
44193499d12d6d949223ded21e465b8c1b3cd578
35401 F20101124_AAAOTF cecchini_m_Page_079.pro
8a189ce021d995013b38232ae2dac75a
6eecad00ed4deba7666f052bb299862dbb7a7407
44912 F20101124_AAAOSQ cecchini_m_Page_063.pro
821757b790c736c62b5c798478fe3c0d
374090be015fea2dd992f57a9791d068e31aa5c6
6826 F20101124_AAAOTG cecchini_m_Page_080.pro
b11f5e13e9f23f90e08483ee398df839
c1be827469967abeb4c139b91a71de903d8518a4
25015 F20101124_AAAOSR cecchini_m_Page_064.pro
b688631d2a453f006308cabf091d385d
ffaf01fade7cf14a497f2aa793f3de1b96a6ba19
27909 F20101124_AAAOTH cecchini_m_Page_081.pro
03589cfdc4cf39a635688943ba58002d
dca07c88ac9e7b79bc3a4bb1e487545269c354cb
2788 F20101124_AAAOSS cecchini_m_Page_065.pro
aa87168de9fdf53abccd10fd7d2533a9
e78cbfa867cb69da0cacbebaf48af581a1a8bf6d
38374 F20101124_AAAOTI cecchini_m_Page_084.pro
2bbb97701b3a082b58ff12bf1768d9a0
f0c54223cbba5f8366f0c38a45531d861affd275
29100 F20101124_AAAOST cecchini_m_Page_067.pro
1365c905406a3f7ee1208cc275a0c525
db2e142826d108d845272bfc30dc51429564c62f
39907 F20101124_AAAOTJ cecchini_m_Page_085.pro
aa6d7830f0b53c648a8e98f2fe21d3f3
7a50fdb6683b90f2c18e0e929466c4892d6c6380
4360 F20101124_AAAOSU cecchini_m_Page_068.pro
ad418990b4a2efdf57e3ac3641920f4f
b8a9f60bbc8c8ab947d418e77b4cf1d8684b638f
44796 F20101124_AAAOTK cecchini_m_Page_086.pro
8b010a0b8f726049eb219ca141458439
65d7299bfb999b2fa654d63f0f319e4a613bf457
22156 F20101124_AAAOSV cecchini_m_Page_069.pro
5ad08ad059745d4461ec6b52f486c9cf
14840881be20d0fea5ee44786a087e2638706a44
32714 F20101124_AAAOSW cecchini_m_Page_070.pro
55b9a1a602de1180b66f825489062002
53a15f100452ce07b98a39251073a4be775df1fb
1146 F20101124_AAAOTL cecchini_m_Page_087.pro
68f3db63e975ad8dd28e9b925e400b0f
8ac3ed7c64d9ea0068d3bc06bd3eccdd734a9c1f
43061 F20101124_AAAOSX cecchini_m_Page_071.pro
18c7e52a18e2a56a8043d8138b72b358
df44f7677002073291b392808a02344012afc590
44922 F20101124_AAAOUA cecchini_m_Page_102.pro
ce4d85c5aeaee60662539cccd2e6e5f4
7e76ffa20a6560b0f29d1ffdd4b8c8449deefc68
42611 F20101124_AAAOTM cecchini_m_Page_088.pro
f80fe02c5855b40ce2a7f81649c996dc
a6ee01036d21647680a6626aa8fadc09d9cbd7e7
44339 F20101124_AAAOSY cecchini_m_Page_072.pro
cbd756716abe754ad3ec50b8b8c575db
91cd3882758ff91f2677b81c6c26d6a04e3492c6
44616 F20101124_AAAOUB cecchini_m_Page_103.pro
fe4eacf58141a448eec27affcbf48a23
47d74849c1d2d010e77c429ae6376a995bb80b6e
52173 F20101124_AAAOTN cecchini_m_Page_089.pro
cc9d440896b69b5b878914270e4cc7a4
2cf581afa3dbc13b064663c333ed9c4c346252b5
37471 F20101124_AAAOSZ cecchini_m_Page_073.pro
e25f449be080a4a05c8eee60c465baa9
7c9364d3e47d7f2d2f3571142ed30ff6442747d5
50345 F20101124_AAAOUC cecchini_m_Page_104.pro
ba89e67ec67dfc5fb08930e6d4328faa
2a394779031132602efb29f97f3476ecbc70bd01
45123 F20101124_AAAOTO cecchini_m_Page_090.pro
e72867d249e0048b83c7bf6c8d7b4a0b
825658e247f7e43a209934807bf058ed5e22adb4
49329 F20101124_AAAOUD cecchini_m_Page_105.pro
d205326622e9fcb71db16f0211a1bdac
61a187150ff8985fa77b7a7bb0b03602252e5857
51726 F20101124_AAAOTP cecchini_m_Page_091.pro
c4f918a8723dd2face36942525779de1
7978ced6fad75bcf099c0b4f933baae1ab8bbf94
47314 F20101124_AAAOUE cecchini_m_Page_106.pro
bcd455a08911db034048ff2990af0d77
747b8b52cb8a8aa0e8103cd9b4fd3264161f0e3a
44447 F20101124_AAAOTQ cecchini_m_Page_092.pro
58421e7f7e6c439725925a60dc56e88e
ef06d13b1fbeab32adff49383f15811ee270c585
45212 F20101124_AAAOUF cecchini_m_Page_107.pro
f56a29fde0b381a94917fb7cd9102868
2282c5348cf5df02e179e2adaad9bfaefb0bbfee
1883 F20101124_AAAPAA cecchini_m_Page_092.txt
c49e1dddfdf61b4478706da056e2cfa2
c1e85526a51f935c1d2a5e07c256b75f6bf24c45
48968 F20101124_AAAOTR cecchini_m_Page_093.pro
6eb98cc17dbee446a19b05776d0c9294
7ba03a3f3330d72f63f7170a24188d06bd4a1659
38411 F20101124_AAAOUG cecchini_m_Page_108.pro
54747975ed77c955a293e6d69405842f
b4f7be4932701efaffeee70d7d4935743c0e50ea
1927 F20101124_AAAPAB cecchini_m_Page_093.txt
7e7898e3eaced997a86288e2e34ccebd
c10bdd452c0749b3c7da55366904bcc09e815aa3
52360 F20101124_AAAOTS cecchini_m_Page_094.pro
8cbdcfa7086c6c9719193d079d412e5e
8a834a58198876597ebe06c95c474708531d1fd6
50510 F20101124_AAAOUH cecchini_m_Page_109.pro
a4f68c43c7e76255a2e9fa250271832a
323f8bb4fe14df1264f9ee93bb0e36699a7c0757
2083 F20101124_AAAPAC cecchini_m_Page_094.txt
fe7b86826c94e2614844f8e87976bc3a
6061154ef2f7277a3e3b5a87d2e152194b86f899
43014 F20101124_AAAOTT cecchini_m_Page_095.pro
f6e5f47ebd2fc0a4618b0b04b02c1d07
f0047d3fe10fd3a3112726091815c9df0473288d
44387 F20101124_AAAOUI cecchini_m_Page_110.pro
d2514357ee8b0a2d6fa88f8a8dc18947
f0dcc668d14fbadf4289b8cfe2a1353a9940d7cf
1539 F20101124_AAAPAD cecchini_m_Page_098.txt
f3a657211f8fe199b9bc1b672418eede
5dbd0b8d4f19ab82500feeaf5ff57f6c138cbdeb
43700 F20101124_AAAOTU cecchini_m_Page_096.pro
0026b8528b7ac813fdc48f81fd23df58
c2b724c71f7a60b21b06391c66729ab4dd2ae1e0
46091 F20101124_AAAOUJ cecchini_m_Page_111.pro
6a103f7991c6409d8b8670e54076c86f
07e8e80e6009c5d9e90fe82d493decd81065c05d
1687 F20101124_AAAPAE cecchini_m_Page_100.txt
ade1a84c181b3c7aaa4fb76ca385d5c1
31aebe5c94512a8651f87155af42153a8eede14e
43187 F20101124_AAAOTV cecchini_m_Page_097.pro
d8351a8d7b3fef927c23edd4b29e2524
375825c99458ec45d9eae84b36742418fb0e57df
46329 F20101124_AAAOUK cecchini_m_Page_113.pro
4bc1105420544bcfde84db71ad02b112
f05612238a06261e5b33495e6ed961bce8bbdae1
1711 F20101124_AAAPAF cecchini_m_Page_101.txt
c0b2b61e560ddabd8aae2aa2cce2d232
a6f1b9419cbd39c85b43d0ebd1b582589d7b3888
35824 F20101124_AAAOTW cecchini_m_Page_098.pro
bb12cc44fff14c1f36c164d5f62500b1
7f5100aa07f0f88a7532b9f4e58aabb87678e7f0
49910 F20101124_AAAOUL cecchini_m_Page_114.pro
4d5080e7f7c773e18eb343766fc3f2e6
3afb3cd94552990cfbcbdbf2557ba722f6854215
2398 F20101124_AAAPAG cecchini_m_Page_102.txt
8ccd0b57f7682aa5e97163cf4483e7b1
5d106e575b95888fd649578c1875350b054c0345
14582 F20101124_AAAOTX cecchini_m_Page_099.pro
92fb27f7b364f1a11c21e594392be843
5db2d07692c0a81377d58dbb920b2f4ce4c0d32f
12089 F20101124_AAAOVA cecchini_m_Page_132.pro
85ffcff3957d39be6c22f23754a2697f
821c43f97c322ec2689b1376d357a279a1295615
2391 F20101124_AAAPAH cecchini_m_Page_103.txt
a8cdec17da34dea2b6224724089242f6
4657a2a05aede2e4a760dde0ae83dddf93f8f692
39842 F20101124_AAAOTY cecchini_m_Page_100.pro
72cf752f7459488b02881be50caff9e4
30ed08f51a1f3c39d0ff4fbf33905cacef50f91e
26292 F20101124_AAAOVB cecchini_m_Page_133.pro
0a615c9d812a212f43e01e7de7a3d1ec
85e49af5b90cef62b9182a58ffdb27355168ce97
3733 F20101124_AAANRK cecchini_m_Page_121thm.jpg
283d8de4d0ba8a398a66bfdefbca00d6
eafa6de23349cd4ce56df7a933b4072adc7254c1
45814 F20101124_AAAOUM cecchini_m_Page_115.pro
daa61b84a20aebcdd6562cb256a954da
27959b05b2522b9d07b9471be60d498c0ccf005b
2277 F20101124_AAAPAI cecchini_m_Page_104.txt
060e71fb02fa4d341351f015242d11ac
3af7c2edd3f4b1053b841965b013d7e419a4dcaf
38700 F20101124_AAAOTZ cecchini_m_Page_101.pro
b03577e41bd8d67e951f16478f34f0ef
5c176904235d2222a92675874db71f3a1987451a
F20101124_AAANSA cecchini_m_Page_106.tif
94beb31dae8f2187474f872df1cb6034
1936d6682fd9129a4489741b1bc2b711afb4b782
32810 F20101124_AAAOVC cecchini_m_Page_134.pro
822fcfddaf92615059eb2a0fa4bea9f3
283eeaa4bc334f00a88c424347394cee495b438c
3591 F20101124_AAANRL cecchini_m_Page_038thm.jpg
a53a381dfb630fe292152a493a7c7cf2
577e2d55bc97781c69c4b53d817a1798116ddbcb
36852 F20101124_AAAOUN cecchini_m_Page_116.pro
56c855569de4f98a32340d4342736cb6
fd1d9b4c1897439c5885d7e403ab26eb4e051f9c
1962 F20101124_AAAPAJ cecchini_m_Page_105.txt
a75566100a77c4b917d902780cc29f62
2276aa84bcac192be16864288afe40d9992e4e59
71428 F20101124_AAANSB cecchini_m_Page_024.jpg
9039208f04a4b3bf4bb362e62d5b8c2e
db291f7b41dc5a39038819e2b606069af830b358
35623 F20101124_AAAOVD cecchini_m_Page_135.pro
2e73e08a6d100b0c5d8c7eac5cedac18
5e906913ab263c9c9bf01ed95668e5d14d3fe829
5650 F20101124_AAANRM cecchini_m_Page_167.txt
0596e04e8f9bb26ffcd5bf21a13ce92d
8e89d6aaadf227de41c4cf4d35edcdc7331dd0cc
52294 F20101124_AAAOUO cecchini_m_Page_117.pro
8bab36519b122ae3673d3464b269ef9a
04453ed1602334cc35f0979f90d3022d58ce85fe
2353 F20101124_AAAPAK cecchini_m_Page_106.txt
06f732ae882b39b67df282022c626d5d
9b1fe5da05597d7292ea295c8e76db9cb3f5a62e
884 F20101124_AAANSC cecchini_m_Page_131.txt
821749e04be6c7d8cac81900cc26f67b
bb1398c66ffb227dc45ba8fb8a873832b502f03b
23263 F20101124_AAAOVE cecchini_m_Page_136.pro
bfdafc911a076de061b08e4cf119b878
b1ae125bc3153a570becc19b75764071db89a879
86884 F20101124_AAANRN cecchini_m_Page_135.jp2
36d120a7205d7755246c7366aa7755a5
c6b48c5a3b2f02529c8c878f1c15d72b66d379ae
47450 F20101124_AAAOUP cecchini_m_Page_118.pro
4c7062eef96e28584c521267c699b2a0
b5f6c578e2942b6a4ce1159b1e59802e5ae122ff
2412 F20101124_AAAPAL cecchini_m_Page_107.txt
080f432571b18b3c04891d8378a59f5f
29a85bb08fe0a30bb41b8785e04d6281ea8b3363
74534 F20101124_AAANSD cecchini_m_Page_175.jpg
d4b763de17e91214179e0c869e6dcf9d
12add6693881821233c24fb57c02110f669d276c
11675 F20101124_AAAOVF cecchini_m_Page_137.pro
0b8ef34e2f13f74e4b0c21cf37c4c941
f7c5d11dc729427f4a5a8fb54ace2545ce5fc5ab
31122 F20101124_AAANRO cecchini_m_Page_049.pro
1ebf6256195adf74efbae988adc0263c
d3c5cc86a8f95138e11325c33434cee0ad345904
51835 F20101124_AAAOUQ cecchini_m_Page_119.pro
36e6258451c0151940779e96fc681d3b
de3d6b36890db97eba38490fc05a26198be30fc1
1917 F20101124_AAAPBA cecchini_m_Page_123.txt
f810b797beaadd5ae08cffc7d1f82119
4e93299c909ad92e3e385dd64e1ba115d4e66dab
2085 F20101124_AAAPAM cecchini_m_Page_108.txt
00b9aa986e01f7e89b297907af3a900a
828c91791be4933d60b2143e8a4fb270ea89c290
13243 F20101124_AAANSE cecchini_m_Page_133.QC.jpg
9597e681908f6e8381eadba523ca307f
99c30c945143233529c4d6275b2d8fc42a43cc7b
45315 F20101124_AAAOVG cecchini_m_Page_139.pro
8a6a29aa784ad288deb07d2c504fedf5
5b4f15407c2287a65efc34b3f51ffa52385577cb
4404 F20101124_AAANRP cecchini_m_Page_135thm.jpg
7b71eae96d94fd17c59ca01c55f760b5
289b7b58de498dee1c466fcccadc8e4d64800168
42887 F20101124_AAAOUR cecchini_m_Page_120.pro
c952839273e8dfdc0a00747d57b2e308
c431c345875aa3159c11c29c2e9d181ac2de9e26
1229 F20101124_AAAPBB cecchini_m_Page_124.txt
c0540a6f855461143eb5cb7a60a9c077
c72a565397345a791c7e89ea5c00702c74b91ea4
2153 F20101124_AAAPAN cecchini_m_Page_109.txt
8f623f3e3ccdc6cda69311b8daffbe72
940325773e4a32f9c62660ced4e74c81aefb6f05
F20101124_AAANSF cecchini_m_Page_080.tif
9affaa516f8e028caf040a13eab6c72c
75c7a97d223ac337d28ede09da1665dc71684cd0
1265 F20101124_AAAOVH cecchini_m_Page_140.pro
2b9b0a9c9bc6a7378cf8de7a1193c4ca
40c60b2cb2b23d745eba4f7d243aedc442bf0317
159068 F20101124_AAANRQ cecchini_m_Page_157.jp2
f5eaa0b8ad25e90c18774e5e5bd74899
2c15e59965b4d093bb581142769ffd34f1e7b0cc
51162 F20101124_AAAOUS cecchini_m_Page_122.pro
258072b380b46830e4ee93e10070baa9
a5c23945e9431bcaa0e9230e024539bc0666cbea
1029 F20101124_AAAPBC cecchini_m_Page_127.txt
f8a7b1315d30c7a308941cf952c0374b
c32424428bb2e1259b8f11f477d49be293de63c6
1798 F20101124_AAAPAO cecchini_m_Page_110.txt
b98857ba97f9cfa9d25783a287229d8a
cb221111636c7a0e5a59eed27d3d4c6650457693
2032 F20101124_AAANSG cecchini_m_Page_033.txt
51f70102076961df166dd7a16b921132
d62c86671663adac4276a5bac13f892480f97e78
944 F20101124_AAAOVI cecchini_m_Page_141.pro
d8ef4eed00e9de2e9d409cebe4be22be
ed213bed2fbc2957a4fb9e8b2ff1645063bad0f2
5082 F20101124_AAANRR cecchini_m_Page_122thm.jpg
a26e4f680f0efb3fb43d1eb0c5077e2f
60dc4259441d829643d22304671e7db46f018ce0
49829 F20101124_AAAOUT cecchini_m_Page_123.pro
8267570f62805a04e7d9b16f3175a3f1
a8fdcdfe3f1492476b0430a89bd4453dfc4195d4
F20101124_AAAPBD cecchini_m_Page_128.txt
2e1f533f84fb5cdc97e78c1349ffb490
f86da6aed34766ae85373aa22584a8687334251b
2342 F20101124_AAAPAP cecchini_m_Page_111.txt
83ae0600449b39050c723f56e367dc22
17551a1f7c0ab19399e89c957f455ca4c77ac923
12696 F20101124_AAANSH cecchini_m_Page_038.QC.jpg
1cad8ded933cde75daabf4f69c5c0b08
a991e301e007d3311d5254b87ccf531f49474aa8
118995 F20101124_AAAOVJ cecchini_m_Page_142.pro
b5ae459a6f7c339e73b3e7a262a6e98e
58fd9cdc4fe16c7bde43725cc55a78a079a9130c
43912 F20101124_AAANRS cecchini_m_Page_040.pro
13114b8e6441f39e7c82c737ba035283
22fd563922a193196a656f097ecc0bb295ebce9f
27917 F20101124_AAAOUU cecchini_m_Page_124.pro
6f8362c1f8e77132f74fe2b03528d806
164c6fafe0dfa55c1fec6b3db39014b36caec1ed
873 F20101124_AAAPBE cecchini_m_Page_129.txt
dedf014549a158ccfaac81428be8235c
faf99f3754d21df811d890bd5b7e70d103dba7ae
2462 F20101124_AAAPAQ cecchini_m_Page_112.txt
daf29e8d5e6a85cff17071c31acdbf26
063157e9e611f59c16e776d20c4b16e61760c1d8
6720 F20101124_AAANSI cecchini_m_Page_094thm.jpg
526f943abf3650cbff9c1d88fadc9bc2
355bedb0c46de2f20a97f95556ca8ca316dc7de6
120530 F20101124_AAAOVK cecchini_m_Page_143.pro
c43b55aab92a13aeb331c499f41fe7b9
bd2c5e49052d02b052c139232ec2099cc5d47bb7
4169 F20101124_AAANRT cecchini_m_Page_146thm.jpg
3e162125c093e518c5d4ce1be056576c
25ce23a0858539239d09a8c854d54f68479a633a
24995 F20101124_AAAOUV cecchini_m_Page_125.pro
8a0bf292647cade845e9b3c7e4eb36d9
25319b150835500a06962f561c46aa0d5ed6f811
1299 F20101124_AAAPBF cecchini_m_Page_130.txt
f459c564f336f80e2ed04b97fb8f97e1
f9cbfbbc48c741873e26ee7a95aed77f2e99feba
2490 F20101124_AAAPAR cecchini_m_Page_113.txt
cb71acc495d65895091bf5300b9f0062
0bb8e59565372d4cdfaacc2355f2e65871ad35d7
80476 F20101124_AAANSJ cecchini_m_Page_139.jp2
f3de9903387cf268f76cbb2f126acc1f
88c632cc88107c3c5e1648ed7b1dbc78e675e4f3
124774 F20101124_AAAOVL cecchini_m_Page_144.pro
640873543c1c34961d7f26d25c0efac1
42bbb25dfec29d1c006a5d3fecc9e317e502e4ae
37756 F20101124_AAANRU cecchini_m_Page_038.jpg
7955c7866ea03801768d83d7852fca2c
0a144b53593c40086650eff02775cb1a6f031d26
25343 F20101124_AAAOUW cecchini_m_Page_126.pro
1057aaab83614da3a92380cbdd77e03b
c841600dce69cb93ae60c3a539d0d7fad12f2683
490 F20101124_AAAPBG cecchini_m_Page_132.txt
16eeb707ca526a50f0ff661d0cca8978
42118a6e52d6b8aae114e5e94bd5e16ff6ef4ded
1988 F20101124_AAAPAS cecchini_m_Page_114.txt
0606083205f21d6191ee1ba95142bc67
1abf381843033c272770012acb92ec73f67aedca
124900 F20101124_AAAOWA cecchini_m_Page_161.pro
4375b4903f3178c13364a594592a3fbc
36a8d43d08f787b260e1f592f041e7f8401c370b
F20101124_AAANSK cecchini_m_Page_117.tif
8230367af826153a0445782f9538366b
ab87986ecad10d759feaa0e504fcfc04701a05ab
64349 F20101124_AAAOVM cecchini_m_Page_145.pro
7b3201775425267ee2d6420a3b16fac2
d5044f1f4f05907e8ac771a07b660ace75978349
3139 F20101124_AAANRV cecchini_m_Page_173.txt
449078038dcd7cf6fde4e5c118381e35
c863986421cd883f1a83a20a2f129b59fe0d0fff
23947 F20101124_AAAOUX cecchini_m_Page_127.pro
19c68f1fdc36f799a2c922d3e3307df7
2a0c7647c5fc8db02c18c928ef3600bcef2d659f
1132 F20101124_AAAPBH cecchini_m_Page_133.txt
0056f9b18db21c4a17c5960e5dc5f736
7f07122d7d98e50cb7d5dc72c221889a9c52bf64
1568 F20101124_AAAPAT cecchini_m_Page_116.txt
bd84f1cea0edce4cde78c0882951c1b1
fc96a7788a1cae286ba6d514f86cd21c16c1c90d
122289 F20101124_AAAOWB cecchini_m_Page_162.pro
9db44750a9810d84844cc6c2fcc76582
d4b466cf4407cf51371e90072c3b5174dc5ec323
14435 F20101124_AAANRW cecchini_m_Page_167.QC.jpg
02b0407f338068a39bd642dd2f57f6cc
efc7a56dcc6a6486a371d05c12e50a51e43e2b93
21983 F20101124_AAAOUY cecchini_m_Page_128.pro
4e2218b49bd6d3fde82c6d9ca1dc65a9
8a5bfa4e821b75c765d9eb9d8c4e9f4c29fa4d35
1426 F20101124_AAAPBI cecchini_m_Page_134.txt
6114155e786104d44252acf3fc37a1cd
a85160880f5bbaf78e6925586e5ac84e57d0f694
2092 F20101124_AAAPAU cecchini_m_Page_117.txt
1d7b716f63862f68456eecff267977cf
8e8a67bc53b2ff576b349b62ea6c86875bf25c83
124257 F20101124_AAAOWC cecchini_m_Page_164.pro
46b03383dc05226bcf71d15a535ee16a
492747c37c01e57a1a97ba65d7853a90a2dad924
111140 F20101124_AAANTA cecchini_m_Page_076.jp2
cda4f69c13d93f96408a2900ad500817
b6cb53c9fa4c6defdea43d799a42c8ad66b8ed19
20656 F20101124_AAANSL cecchini_m_Page_015.QC.jpg
f651c46b9128f61466f145d5d913dfe5
85e15502bfebefa441ea17c31982694a4e0cda67
112903 F20101124_AAAOVN cecchini_m_Page_147.pro
bc1063e221e919890f65a15f087cbc37
cd3db709edf3ba16b1a4eaaa4dab422e7373444c
926 F20101124_AAANRX cecchini_m_Page_027.txt
09913d34cb3adce317b4e50907484708
1420b30148105c43d83338efb35d51cf1c236e21
20470 F20101124_AAAOUZ cecchini_m_Page_131.pro
5a5c1f945483f2bbfe6725fbc23a68e6
19a0100cbed78bd929c5b16fbea83c4cb7196458
1546 F20101124_AAAPBJ cecchini_m_Page_135.txt
4078fd64c819e393741f686a8c49969b
d3002b8e4637d56e5da05a28493f1871ca9907b6
1905 F20101124_AAAPAV cecchini_m_Page_118.txt
57b0b6a5747a5f7a3ad5aa1cfcf52c7a
5ef94c81ed063dd9b03570c51f845fcd5afd6256
125188 F20101124_AAAOWD cecchini_m_Page_165.pro
babb7af30f609e88730961fec7b151db
c989f77928d1b08db620988f84d862a9307b70aa
4843 F20101124_AAANTB cecchini_m_Page_108thm.jpg
2fa261d95accfa0f45c412c8560c5fb8
817844b07d2dbb7119cbcda28fce1b451c989104
19916 F20101124_AAANSM cecchini_m_Page_073.QC.jpg
519b57336221dbf325b7fa31d7a7490f
a4592540743f677a0f0906fb0651f72b6cc7cd60
115589 F20101124_AAAOVO cecchini_m_Page_148.pro
179c821da13a6dc055d73efe7f9e6eb3
3732f72239fd01a46c3b3271e1d160606607a03f
5719 F20101124_AAANRY cecchini_m_Page_025thm.jpg
7342e9d70b9e86b92cb830894f822b34
936874edf59881b98a7dc345dec41b05b1a4b710
503 F20101124_AAAPBK cecchini_m_Page_137.txt
37ae250db0b77edfdf8a72382f1247b9
5e6f7cd9bda65552c796b936dcdfa26a6ec7e55c
2046 F20101124_AAAPAW cecchini_m_Page_119.txt
6b997b4e0b979c5c2acdb469f41d79fe
0c51ee2f50bc5f744e30d126a561284f513d777e
121644 F20101124_AAAOWE cecchini_m_Page_166.pro
bb730c7d9c0b081a39dd1dcedb3ed807
6fa1ad7badc00a7c1f41f1a88bc6bfdae8f3f6aa
F20101124_AAANTC cecchini_m_Page_184.tif
09541edef80a552535a0b51791d63b95
8aa45d6a8319748ad179b12acf00c1a8e72e5afa
F20101124_AAANSN cecchini_m_Page_124.tif
00ab4c0b57954c3049535fe7ad50ba9c
fd8e2ffa3eca100372929a6dd4a9423618da76ae
56204 F20101124_AAAOVP cecchini_m_Page_150.pro
6e77f254084ec5e2d38ae9852ae2d5ef
8f12293b01b381e844d6dca4477ae4ad42dfdafd
F20101124_AAANRZ cecchini_m_Page_078.tif
c6dc575d5b1baaa80f4f62c5455af80f
997eb50fea86cac2b0083e6c28d9736fd33a93e6
1884 F20101124_AAAPBL cecchini_m_Page_139.txt
f6350896ce0251aa45652a8860a3d06b
8851bcc9b3188ba7177a3dbf97e69ac0e65aef1f
120580 F20101124_AAAOWF cecchini_m_Page_167.pro
1109ae472bd04c352674d4d5363178ab
47b78c3851daa46aa8325307cd926d3fbc3dda31
125396 F20101124_AAAOVQ cecchini_m_Page_151.pro
0524ef2e3ddc4fd5cd5148f912742d30
3d2b8501f21ca76e023e59a47a4a5fb9ae2129c5
F20101124_AAANTD cecchini_m_Page_148.tif
950c45352c7551da0cc26a9ab91062f5
5ef03b85e472b3c4d0cd05680bc7c38f014b4f38
16055 F20101124_AAANSO cecchini_m_Page_143.QC.jpg
153d0d7b7445bd9e70016dcbd1f668d1
d48eaaa93efd023a06fbb43b7bad5c334c222e31
5910 F20101124_AAAPCA cecchini_m_Page_154.txt
4a90b7cf4a2b61e2bf713f4b8ca78ac7
4a7695d706f91518cae6b5189a308e0785b7507c
75 F20101124_AAAPBM cecchini_m_Page_140.txt
09ecaefb692d05b5eb76bcd0b13d35c6
7e20cf1c075a70bcd5e1310d6298fa96608a4030
1705 F20101124_AAAPAX cecchini_m_Page_120.txt
566ec9cea4954db129af8601399f0558
fc10929e173b05340b1ffa8e73cc3d53a37ebe72
127046 F20101124_AAAOWG cecchini_m_Page_168.pro
df19cd1c5d4f195900bd6f8bb51b8be4
808fa0db6b4344ac8000d55aa4c964a4c3c1947d
121892 F20101124_AAAOVR cecchini_m_Page_152.pro
e07130191811cbc62f474eeb092be104
8231105a67737dceb33f245dad66bd9297171b9b
F20101124_AAANTE cecchini_m_Page_165.tif
245885d0bd8612cef9c74bfd8fde682b
838fad229225be4d70a1960f08c533846f7dd0af
14279 F20101124_AAANSP cecchini_m_Page_166.QC.jpg
fee93f13b8ec9724d5197955e21ec877
c2c467f9a7cfe860d912f73c5f2aacb78c7409f2
5909 F20101124_AAAPCB cecchini_m_Page_155.txt
e4422820af38306343151e2f4702b452
f5ac3573570d28ddd6f168150a79d73dcfd9fe56
50 F20101124_AAAPBN cecchini_m_Page_141.txt
8ffb14494070c20c909fff0768fbe8a5
b756d4fada927ad89dfe9eeec4abb047579957bc
1088 F20101124_AAAPAY cecchini_m_Page_121.txt
1c4c2eb6d8c1c97b48dd213f04f3bd87
0f39fa7facae9ddb18a1b8fdeed05173585b12f6
124905 F20101124_AAAOWH cecchini_m_Page_169.pro
bedccc29a905a0150daeeee14d039f2a
94d3570a7a0d186e6239740bce9e450481398f3e
126068 F20101124_AAAOVS cecchini_m_Page_153.pro
aa22df1e5fafa0b216ee3e9f38bb4322
584848ff732479c75153f8471687d69b4522480c
92854 F20101124_AAANTF cecchini_m_Page_006.jpg
c9c8e950d9521a8745ccf5edd388709a
0813f8c2b3fd3fde252d16a3fb17d9d7a513b2ba
52900 F20101124_AAANSQ cecchini_m_Page_010.pro
1517b74736d60c8fc5ff87dfd6becf4d
2efa85888a06a70c60f05eb9747c9012aa8180f4
5838 F20101124_AAAPCC cecchini_m_Page_156.txt
d4ceda30d45ba305c9f9e0aacbb3e223
57be59e6e933bb6f3fed43332f72aadf32d462b0
5572 F20101124_AAAPBO cecchini_m_Page_142.txt
159458ccac9f9085fcaddaf363bfc757
9914f0745c5734e902b38ac58ec4ecb98a29cf9c
1968 F20101124_AAAPAZ cecchini_m_Page_122.txt
881aa685d5e3718fbe72e3e899f77b34
de0dcb5b289e82ddf9c0dc075b5e2cc88c7cd6f0
124503 F20101124_AAAOWI cecchini_m_Page_170.pro
46ea62fc7bbec67d775a138b61c936c2
29534c8285cafc5efa721b6322788fbd025d4166
126794 F20101124_AAAOVT cecchini_m_Page_154.pro
359b2bbb891f6f9a652121f230fa338e
96e70284ca0fef2ec8cbd47e8d7e930872e07437
45957 F20101124_AAANTG cecchini_m_Page_112.pro
8d2667c8508ec9e2f6eb7ae19dbf2aca
8484d26f4857efbb6dd39820e519af526617e076
14644 F20101124_AAANSR cecchini_m_Page_168.QC.jpg
93ee0807be4af08dadea595bf2a55a22
5a9fbf0a1f36e9adc06c6d1a36b2da1ebefaca3b
5724 F20101124_AAAPCD cecchini_m_Page_157.txt
4aeb8db78e07bcfbcc818aa1df9390f8
0eeea3b64843b17103fee8b161f839ce5c189258
5635 F20101124_AAAPBP cecchini_m_Page_143.txt
7f3747aecf316f6246e6755d6602a1a4
cb85eebdbbcbb0a16d7c34d4617b22866efad8b2
123089 F20101124_AAAOWJ cecchini_m_Page_172.pro
ced84aed09ba26660bd299fef4058067
b2a3e9917af8b643899863cdfc4979af4459e78d
126210 F20101124_AAAOVU cecchini_m_Page_155.pro
7021a0d47943d43949d287841193229f
55444bb8f45ee6caa6844fc90c948353294de117
6149 F20101124_AAANTH cecchini_m_Page_140.jp2
5aac3b8b95a5880b992439b0156a1132
6e14bd02bed348dd8669b3334fb12be87678dbb3
14618 F20101124_AAANSS cecchini_m_Page_158.QC.jpg
345515f03df6f1fd20d3b480fb78d5c9
3cf049192ad09cf3c27b37a6256c626a0073f94b
5896 F20101124_AAAPCE cecchini_m_Page_158.txt
dc5e2a778ac056b24b7877ce57bbfc24
ab8e8873ec7de927d7bf5e15d252e7db5872fcce
5653 F20101124_AAAPBQ cecchini_m_Page_144.txt
a246921c075a3dce1f7c592fa5e44ea9
cffb0b8c88a7327ced118202b1ac7605519bf927
66187 F20101124_AAAOWK cecchini_m_Page_173.pro
ebb22e76ad8fd44b98486a17cdee3d94
b8a15edb45620264eeef0f3081a9e08c43e31be9
124727 F20101124_AAAOVV cecchini_m_Page_156.pro
a2a41f7741b3b5287a4c7876c8a3a61e
dacf203ecb8268c090ccfb7e14d52f3cb66d8efe
50753 F20101124_AAANTI cecchini_m_Page_171.jpg
a01ad686d21bd5be3df585c66c1e07fe
ecb16d461db2fbdf7ccfab581e2ceb6213f4ccbb
91745 F20101124_AAANST cecchini_m_Page_066.jp2
93a5939693dfde66deb415d5325494dc
14c9b9131b017156ca39e1804c2c55baea4b64ce
5907 F20101124_AAAPCF cecchini_m_Page_159.txt
6ba0dfeba6678dbc3cb364c71fb86cfe
91db9c02b15614443ba6b6a36f3a0f15599620cf
2898 F20101124_AAAPBR cecchini_m_Page_145.txt
9f9a7b92f2c14cb2a21e2e6f2ea47eb9
f04dcd734225e9ffa878c5f74815d60c713de4de
1161 F20101124_AAAOWL cecchini_m_Page_174.pro
ac01bebabcb856a7e3eb908448badf2f
ae40839012a721dc70e8ec11b767dbb2b497e875
120543 F20101124_AAAOVW cecchini_m_Page_157.pro
457655816562e3ea4dfa03ecd3417099
b88167bc33eab8e417379122f9ea8fc48d4d450c
19896 F20101124_AAANTJ cecchini_m_Page_012.QC.jpg
f9bb3797220395e992065a236c97f8b5
1c39b2663a511df427da47ba49a34ba236f631f0
2103 F20101124_AAANSU cecchini_m_Page_175.txt
52ecf0ae61248eb012684827d92fc331
feb0fdace86dfe9215d8dae83fd313cb1daf5f6f
5689 F20101124_AAAPCG cecchini_m_Page_160.txt
ee79eb6bb0799acc9603fc64dcf7fbcc
8e9671dd16682604719ce8da53460ecabda990e8
5181 F20101124_AAAPBS cecchini_m_Page_146.txt
4cf24d9fce6bc0c103f4e633624e90ec
22f851dc8322f23d7fa799be91b0bbd576da6f7e
2109 F20101124_AAAOXA cecchini_m_Page_008.txt
307677807f13544a0110aebd234a9889
992ec839a73ca4f2687cb29da33a979ec9d2f175
51146 F20101124_AAAOWM cecchini_m_Page_175.pro
9a5dffcf3c497db39e2d51f72cda4c32
821eb7b2b1e6bddc987ae751c415a401a3b0b53b
126661 F20101124_AAAOVX cecchini_m_Page_158.pro
8321ba0e17f5283724bdc55d2b559057
d2c4f62cae68243fee9d8b48b488a805002f6001
1847 F20101124_AAANTK cecchini_m_Page_115.txt
f63937560aa973068c1120a2f81f9f0b
8380468200cf70e0121d482546301b02720c4ae5
83321 F20101124_AAANSV cecchini_m_Page_046.jp2
01b6fea72cdbedd4cd534cae11f3d41d
0d3403bfb03205fa30f4feff02e9c17cfe842dd4
5859 F20101124_AAAPCH cecchini_m_Page_161.txt
803d17ea5b765737f15b6ed7c3cabef9
264e87061fd62381712f43b70504a5f85fef41b9
5004 F20101124_AAAPBT cecchini_m_Page_147.txt
3426482f518f269faf4902844a679232
f7a9408bf8eb90347fbb11aaf7edb47d4094bddb
2190 F20101124_AAAOXB cecchini_m_Page_009.txt
dda20b95cb69cd3c0cfdee4447305914
0e852de757244e95ae261df8252c74f1dd5e1ec9
55479 F20101124_AAAOWN cecchini_m_Page_176.pro
798ca70df3b350df036cca3e9833f53e
9cb67a689359cd2ec9009cfedc51d4bb9b973224
125685 F20101124_AAAOVY cecchini_m_Page_159.pro
f2dbc55ad87dd4d6a7e6da115b6ec249
d50998022532fd579be4ed1b6d8e1f39f908e653
4453 F20101124_AAANTL cecchini_m_Page_087.QC.jpg
57ef466e2b8825b2dd921351223b3f7f
a102143e0c910e3505df81574c81272b7915c4d4
5819 F20101124_AAANSW cecchini_m_Page_058thm.jpg
bb9673badb68396f50e5ffd65f1220b7
a4f0d221ede707b1632d5cdcb4f02a6ee8a2fa27
5777 F20101124_AAAPCI cecchini_m_Page_162.txt
443fee6566c931f6f0984daba0d4c8d7
2f975a0067c0011637e528e43e9bbfeae0297275
5132 F20101124_AAAPBU cecchini_m_Page_148.txt
38c3db60d8001c03641172ab6c12e143
bea3d80462f6459d5c87fe065563091aab84e083
2175 F20101124_AAAOXC cecchini_m_Page_010.txt
2e237cb60ceb169472066e3ccbfd85c5
da418dd54a005c22243d73d16fad31db77c637cc
123004 F20101124_AAAOVZ cecchini_m_Page_160.pro
95883a36d8ac86190ef8ba1e02e39d3a
19f4702bc5d28363e118a0ca20a9c4e4bc00c6a8
1218 F20101124_AAANSX cecchini_m_Page_067.txt
bd6ca7580b57bb08d836cd5579c10dcc
b1314ff655e41b245d73003824cc9d8b2eba2de4
5593 F20101124_AAANUA cecchini_m_Page_002.jp2
8151e5eb80ba9175ab0ffafd85b74e9f
5c0a147ec8d9e7cb54ef913dbd79e7948370a978
5833 F20101124_AAAPCJ cecchini_m_Page_163.txt
f9b65aea12b118fdac8be75e9974dfec
da00232837fef425acb320ca15b58dc715087ce9
5145 F20101124_AAAPBV cecchini_m_Page_149.txt
c9049fa7752b61e2defef10643a333aa
fa291fb118d558da2914d42ef592243f71f48041
192 F20101124_AAAOXD cecchini_m_Page_011.txt
311d079eb48c821755e4ebefd3f60ae1
12ccd52595f87e6b36b6b95ae6fc1a34eded8316
60335 F20101124_AAAOWO cecchini_m_Page_177.pro
002eb7c7a93153a7ff76c6f17facc9eb
002c896cfec716ba2d35b938f8e9a3d6d4d6fe60
626 F20101124_AAANSY cecchini_m_Page_099.txt
3437e74c79c2b5fb41c2fe44b687eae0
9303519455a697642c9473624590bef4e4963d9d
51012 F20101124_AAANUB cecchini_m_Page_159.jpg
9db329761c3264b846c875a706f16c52
3dfeee413fb9f7b0050d5fec8c8bd627415d634b
15235 F20101124_AAANTM cecchini_m_Page_010.QC.jpg
b756f2f399ef3d6451c3bba32842183c
b5e9ab6878dfbe4d9d3120eef8d52c571e128aa2
5845 F20101124_AAAPCK cecchini_m_Page_165.txt
40f16121d66eb1df0e3b119f6d370f36
4b3cabd8bf7a776a80977e5206456f62757a282a
2467 F20101124_AAAPBW cecchini_m_Page_150.txt
2fef812decef0ffa6e13ec91379b8cee
dd53b36fb3a847d431d92912a5541e82e78df897
1838 F20101124_AAAOXE cecchini_m_Page_012.txt
f299a8e70cbf11f2388ca4c6ce219bf3
a80d69aa5ef28aa444d0f79e68ddcec9ead1e3fe
57506 F20101124_AAAOWP cecchini_m_Page_178.pro
66709d574810f4f60bc91c6f374e6318
4623bddbaccce3a970e65bf20d8643e2d67169a8
56954 F20101124_AAANSZ cecchini_m_Page_179.pro
e5bcea0b3960406cfb00ee2af7a0feb4
779d7de910c3c2b2b40149b14a09e6d8f9cbcee9
19654 F20101124_AAANUC cecchini_m_Page_044.QC.jpg
b2819d5c80dfca14a8f74f12632a5d09
c8d294fa4cbfb48085fddd156e8965811e738fe5
68806 F20101124_AAANTN cecchini_m_Page_049.jp2
2c9993a3902eb199aa3008e9a8b5656a
ad8c39ef7ce59e9887a975d251560ec354d8246e
5764 F20101124_AAAPCL cecchini_m_Page_166.txt
ea3e5094a33280407ee19ddcc2ca057e
4ae1586fed18b987db65f5d85db32b59a729c49b
5917 F20101124_AAAPBX cecchini_m_Page_151.txt
65de2db165d4617e416d228f2aa63971
2889e1e9230805a44772189afb0bf01f212ca16d
1808 F20101124_AAAOXF cecchini_m_Page_013.txt
467a8a8a215c1f44efec26f638f34d08
2c9ece8284d7cd7197ede85666c83bc7a41b2b43
60422 F20101124_AAAOWQ cecchini_m_Page_180.pro
dcebbefbed33c2a69951535d29156423
3cde68b9cd76b6c41eeebd788c9d5e618dd8d231
F20101124_AAANUD cecchini_m_Page_153.tif
d4a9dfe0ef543acc971761905e975a33
bef8d81649cd482e57661136020f83a48d5bcade
1000 F20101124_AAANTO cecchini_m_Page_136.txt
4a778859276d285cf0e1fdb63dbf8833
40b4089107ea04ba8ca41388e1d97a6b4c7eff43
951 F20101124_AAAPDA cecchini_m_Page_184.txt
3c12f141464c67b819d50a05355afa05
1e22b2f8f5066aca8cd4e4c0f775c9c56bef6d04
F20101124_AAAPCM cecchini_m_Page_168.txt
cf218771b02438099e2ba4287feeb352
60e976ddf9680398490df6143e12c38e3cf7adcd
1969 F20101124_AAAOXG cecchini_m_Page_014.txt
7b645e15bb2cc80fbde19feaa71dfaa4
470bbf2edb8f1d3401a62bd65ffe2aac8f1960f3
51061 F20101124_AAAOWR cecchini_m_Page_181.pro
1d433ec6efa042c72b63cd51b2769548
e2a6bb200bc6e78d739091367887e8b728b2a06f
F20101124_AAANUE cecchini_m_Page_166.tif
399e8438ab9e9afda468846fe2e0d2d2
f391c8fad36a085dd8dd303470421e9ca4feb3a1
F20101124_AAANTP cecchini_m_Page_091.tif
94d3b4a67a59f2a8d64b4273d2f5a9ae
577e3c5b572437c6923adc2ad708f801f1f22d21
2429 F20101124_AAAPDB cecchini_m_Page_001thm.jpg
86702d236ff891527c24c670f59784c0
6a33a0ef84ccae7f4e4297ffc410f319c8eb28a5
5886 F20101124_AAAPCN cecchini_m_Page_169.txt
2f387226f765db03dc215466074848f5
a1c3487cfcfa9080729df2d73107751e4f380bb1
5787 F20101124_AAAPBY cecchini_m_Page_152.txt
ae621854763c5dc58858129103ea2ac1
bb0928b7313d7a3f9d78a6338b1b514ec173e18e
1775 F20101124_AAAOXH cecchini_m_Page_015.txt
7a9c72a8c955bc9f2a0f519187fa61f9
4989e0a8ebe8130c22f56e0e11a07ac556729524
61442 F20101124_AAAOWS cecchini_m_Page_182.pro
336c56ab0b5fb83acb2be7343cd1b70c
565790cdea8065063889d4d73b04cfde4de658d0
F20101124_AAANUF cecchini_m_Page_009.tif
00a4304343995b3e8d3bef781fc5e433
892472d403aeea400b10072e516427f7dfd9c550
4619 F20101124_AAANTQ cecchini_m_Page_077thm.jpg
d006ed46a6a9060d8fbabf66f8356d5f
1d34db70fad27f28dc747094389f18884793011f
641418 F20101124_AAAPDC cecchini_m.pdf
80c859e997dfa471eb78735b83c41532
aa6be1e0456f17c36491d9c5e23fa4af84a09322
BROKEN_LINK
Text_Data.zip
Text_Data.zip
Text_Data.zip
Text_Data.zip
5885 F20101124_AAAPCO cecchini_m_Page_170.txt
fd368595ed2efb44b70b4cddae360ca6
80f4ea898ca4480e11ecff6c3f6e48561eacdb4b
F20101124_AAAPBZ cecchini_m_Page_153.txt
ce4327389eee1e92d984744186254d38
f1ca44ef6abb83225fbbe1d5952e2357a1bf6435
950 F20101124_AAAOXI cecchini_m_Page_016.txt
303496666416427133181d770d3706e2
82813bc6c665140ae78fd1adbdee0935fb5bfba0
7607 F20101124_AAAOWT cecchini_m_Page_183.pro
037d88153e18d33a8d8df431a8095da3
7c4d74a8bb561c833aa27c56a88122c04cd94491
48510 F20101124_AAAOAA cecchini_m_Page_051.jpg
5d936e4c2f765aedf00aafe75a798ca6
39871bc53b84fefc46ae0b2721180f7ba27e38f3
3800 F20101124_AAANUG cecchini_m_Page_159thm.jpg
1dba15b034a97357087911ed75739f54
72c6fc8418d615c9cdd4d3df2697acb06e97ee5a
3802 F20101124_AAANTR cecchini_m_Page_153thm.jpg
c3123ec13b82202e73194e2361605ba4
587614453a8c5f49af818ac7237ab1a2c5dc7a9a
12400 F20101124_AAAPDD cecchini_m_Page_121.QC.jpg
3296ff8f430bc760bf9cb897a7e4536e
44e37f1e786f49eb3da37d55a23e93e5dd763771
5854 F20101124_AAAPCP cecchini_m_Page_171.txt
72e9123c237d9c211959f3fa2417ff7b
1d778c04d38d82f99f1ff929947db8c72a7d622b
1802 F20101124_AAAOXJ cecchini_m_Page_017.txt
d8d79e35c62b0782f96d6586338cb090
cc69dc2247ccfebbb7f3159eb5f439c07a24a483
22747 F20101124_AAAOWU cecchini_m_Page_184.pro
076cfe24683975cea7cdcaec1aa1e8b9
a15e97ec6e48062f506310da14e71a3aa16e3e27
50319 F20101124_AAAOAB cecchini_m_Page_052.jpg
b6ee2e157e31108206b531dc72fc7eab
a64f906558afc0ebc179710ae8c4766a3ee945bb
17206 F20101124_AAANUH cecchini_m_Page_060.jpg
ac145393ed8a7d323063dd0a476a3fc3
bf4b9773147552bb7133b8ea956b82bf4c7d5616
1728 F20101124_AAANTS cecchini_m_Page_095.txt
37ca08e996011eb4cbd0741d7bc9cad7
ea20044c25623106bf92ff5304b5309b75b4676f
8832 F20101124_AAAPDE cecchini_m_Page_007.QC.jpg
e2e096142bf634c65141caae5e10e530
5d88e927c8b3f722ddbd8e4c23ef8f8bf7b5d324
5801 F20101124_AAAPCQ cecchini_m_Page_172.txt
8d15bf959ef988b90d22effcf52bcc6c
431feff557125029aba5b36630d72f45afdd0da2
2036 F20101124_AAAOXK cecchini_m_Page_018.txt
f5046bc1e7efb7426383e2b7eaf9c917
777e6dc8356052d2b6b781b4247a4c09b9d6b247
469 F20101124_AAAOWV cecchini_m_Page_001.txt
ec9b2a7660f2bf66cd6ad4b2e4e98c34
17bcf1643c475de7b6826f9dd54a0406cc85be4b
71579 F20101124_AAAOAC cecchini_m_Page_053.jpg
44a49c7defe5f39c5759c3cbb4f94fbe
95e57b20763a8a2c0507367a82572dea5169d1ed
3750 F20101124_AAANUI cecchini_m_Page_166thm.jpg
eb191c70b8462853ae16c3c67b5d3244
bb5d1979ecd693cbc9bd96df279d32763054dea0
15864 F20101124_AAANTT cecchini_m_Page_049.QC.jpg
ec75b130387804c8c8e89465d5293ba6
6501a1a9b929da51e1788fa75f05a571b6bc20ec
16106 F20101124_AAAPDF cecchini_m_Page_142.QC.jpg
19cab65fe0a9ebdc371a8a1621224594
d069941d77f44886f2124ecc7bfd71986a0925a1
83 F20101124_AAAPCR cecchini_m_Page_174.txt
6e349624a051d58bf3acf3ea3d63e79e
5b220dfb1d8638d4b67bba332e6b8408feaa2b8b
1983 F20101124_AAAOXL cecchini_m_Page_019.txt
8e5a3fe1125e92a57e40c3113bd1cdfa
2b3a11df7b3088dd88f6e3e627696c0eeb9d9bc0
99 F20101124_AAAOWW cecchini_m_Page_002.txt
3f70f0c5b7c6d006b8e262e32679e117
9d75fdb532cd641259599f670fb43e13698046ee
33887 F20101124_AAAOAD cecchini_m_Page_054.jpg
45a0e2512c0365407816610218408b11
a4130722e5f7e786aec16e5972b48d0c6ae7384d
3786 F20101124_AAANUJ cecchini_m_Page_165thm.jpg
c6edd686df24ad6292b720c3b4fd620a
f883b1053daa1285d60a938b7467254b20aa4c6f
23131 F20101124_AAANTU cecchini_m_Page_121.pro
915bbcea8994890feb987606325529e4
fb6e037f58ec861a264be7ea7fcaf9f5aeb88fdd
6405 F20101124_AAAPDG cecchini_m_Page_053thm.jpg
56f16aa0c4b14858c9a0d8374cb79f40
aa43defcdda6f408a498c42954ae5626923f0181
2318 F20101124_AAAPCS cecchini_m_Page_176.txt
0603d8d726480e89acdd5c7db4d34817
383e5e6c0d589d182613d439d9ecc17676c9c92a
1837 F20101124_AAAOYA cecchini_m_Page_036.txt
f871a10c51e61fd0a14f63b365c04c10
65117018f6a5985db3419df4fe18e2de68c60ccd
1970 F20101124_AAAOXM cecchini_m_Page_020.txt
b4d717bc241b8a44db48796237d39d47
0a2fad8ec772e0a89fddb4ae2f5dcf4e78ca2f67
803 F20101124_AAAOWX cecchini_m_Page_004.txt
0a0f1dd0f05d7fe2c264e6ab45d7fa37
6aee6aabe343640d576bfeb88ad94cbe58641256
66384 F20101124_AAAOAE cecchini_m_Page_055.jpg
67b940fa24014f41bd66bfe6cd91d953
34fff220d5d07a2057b2c2e8a69f942628bba63e
159801 F20101124_AAANUK cecchini_m_Page_167.jp2
cb4bd9bc59dc7dae00b42973801b170b
c585ccb5090dc9f21a9d1c24018cbf9e10231bea
5591 F20101124_AAANTV cecchini_m_Page_031thm.jpg
3dffc654321983a212f575ec07ba13c4
dd2f2b0d15c8e6c55cde0aa349a16dd99bdfe9c8
F20101124_AAAPDH cecchini_m_Page_165.QC.jpg
dd4e1b201ef68e531d28fcceca3d51de
c2ae912e35bb723ae134a91e9e3630c3a9810357
2498 F20101124_AAAPCT cecchini_m_Page_177.txt
b02c19032dd9fa0a06cc13f914e62583
3df379afdca29f67878cff78d3ecc20cf072a858
1772 F20101124_AAAOYB cecchini_m_Page_037.txt
a0e919c07aec3c8c3039c0579b4e3895
a8d7f42e8cd6bc90ed57412a8e924dabd64737cf
F20101124_AAAOXN cecchini_m_Page_021.txt
1e9c44bcf70c30c3a64aa35de37aa076
b599db0d7698ddc1a05edbbf711eedeb673bd8ae
2764 F20101124_AAAOWY cecchini_m_Page_005.txt
7ff8c96e744cced2ee3ed1a0384108a0
61ff7d79ac3ed39c064d11639cccb64283e2d038
42981 F20101124_AAAOAF cecchini_m_Page_056.jpg
8ec65101b98353f675467508580ca885
34e1db226f2db1e9f2af4b009f0d1ef4baedfaea
140 F20101124_AAANUL cecchini_m_Page_138.txt
336538815f609ba7418fb9de8f3afb74
efbeab1f2a76a0c3fadf8945d53de545ce414c2b
3962 F20101124_AAANTW cecchini_m_Page_124thm.jpg
ec8d8aaae531697a7169993eec10e909
f788964ff64c8531f72a9f2c9fff42a61b1c7d9a
3805 F20101124_AAAPDI cecchini_m_Page_161thm.jpg
3fd3ff146855ebe10e0ed465fa45c570
8a5f70c1e529765c5b605846c581f0849f2cc8e9
2376 F20101124_AAAPCU cecchini_m_Page_178.txt
16e1e28c8606bb9a1c26f02408b21a92
56b1ba9c3f03a53388ee8e6b77c4dd7391e49e1b
912 F20101124_AAAOYC cecchini_m_Page_038.txt
bcac2071948b4ccfb94c76ad491b1b51
7fc64bff4388274df104d43a087a849f3ecdc458
1812 F20101124_AAAOXO cecchini_m_Page_022.txt
7f545fc00b8e8ce4f5d8bd4a92359954
b265251f7a48fa26d10c58ee6d8e177723d03b09
1012 F20101124_AAAOWZ cecchini_m_Page_007.txt
c20920ea45af1d0c65c6cd75bc4c9014
ff08c9b0bedc990b0b002ac10c371995835a1240
63370 F20101124_AAAOAG cecchini_m_Page_057.jpg
636699bf119373c5678e089a27954fb9
4f84ebbd40aa971ce3e765b56996c8dc8eb3aa35
62669 F20101124_AAANVA cecchini_m_Page_056.jp2
4617c2130fbbb09bf8ef7dd82763923f
b354c94177f1155d3f94c74cca98ac4709b96a9c
18389 F20101124_AAANUM cecchini_m_Page_080.jp2
3a502f5b37f748511b671095f3a83860
34b46daf8cba3512c1764021330691861db14da2
F20101124_AAANTX cecchini_m_Page_066.tif
a12450bd21201e955371022668efd0cd
48fd23c9739ebd9c9558341cb9358e0edf7edc5b
28182 F20101124_AAAPDJ cecchini_m_Page_177.QC.jpg
005a197ce746485cf9e432e90a395d06
a245dfee9504438bfc0037e84238f5bd96a45e24
2349 F20101124_AAAPCV cecchini_m_Page_179.txt
727c01aeb107b26b93d4c7fa65055f3a
fe428ea06a4ced24f0a21265af2cc3c9ad739c5c
1855 F20101124_AAAOYD cecchini_m_Page_039.txt
6e9d87e6a93bee200e0dda845f4ef11c
c4e8dca387cee2106838894cfaad2429cc447548
58365 F20101124_AAAOAH cecchini_m_Page_058.jpg
65147debdf07ca1df3c22c57d65c9922
77b25bc119018af9cb7b84a8c50c132201fc8961
6151 F20101124_AAANVB cecchini_m_Page_039thm.jpg
9c76bc895370ec41beb91b939cc7bd5d
4174ea6378ceb11e516864984ee3bedba9078702
1633 F20101124_AAANTY cecchini_m_Page_087thm.jpg
6b50f8109cbecd5251f127ecd5ccd08f
a03e20e93428c50ea2a9c7b26ed903606e63e456
19894 F20101124_AAAPDK cecchini_m_Page_078.QC.jpg
ca47088f544648b3da33fa8eaa176fdf
5afb92c7eb7f57745b1dc50db9270beb44c00d9b
2489 F20101124_AAAPCW cecchini_m_Page_180.txt
223c4ac20c26101a01b691f2c4b3e7a1
5baeef04aa0d9bc4bb704f0fecaf98c3172c50c7
1782 F20101124_AAAOYE cecchini_m_Page_040.txt
01cbcfed00b909ef378ec5aced9b4442
65452dd346f2efa036936c518d06af1b70e62c1d
1909 F20101124_AAAOXP cecchini_m_Page_023.txt
55b92fad1561d57898bf4fb48ff1980d
ece57fc436660bb571c450a2dd3df98d427055e1
50965 F20101124_AAAOAI cecchini_m_Page_059.jpg
2948a74d172a6fa41d82adbe5ff092aa
6bcfedc12375cae43e5f437830c71b4a4f54ec26
22617 F20101124_AAANVC cecchini_m_Page_038.pro
27adb2d1e3f45ff7faedbd293693cb74
cbf6ef51b51b89dd9883872c92285aad172fde35
1629 F20101124_AAANUN cecchini_m_Page_043.txt
dfc3fb41c6aeca02814d0d920c8d9636
7292481bcff12d157ecddde683453a0ecd81a3a2
50837 F20101124_AAANTZ cecchini_m_Page_033.pro
fb8e337a9da021bd5b17c95eec3a8895
093b32e4051af2b4d7ab9c4f516f6f4fe197e198
14008 F20101124_AAAPDL cecchini_m_Page_124.QC.jpg
d3fa0c2dccd9e4661722fd3ca6877531
e4d46a0ef6130e93aad1749f4bfdee04e70dbe78
2127 F20101124_AAAPCX cecchini_m_Page_181.txt
05e0326dbd263a9c03b07c1f69990f68
0b0f4824c50cba37b29896786e7159e72e32a613
1750 F20101124_AAAOYF cecchini_m_Page_041.txt
8eee760e64eff14aae98a4170f577685
286dd56fcfad4b3c5ccfffee55a27cf52112b6ea
2010 F20101124_AAAOXQ cecchini_m_Page_024.txt
8ddd25f3eb36aa44943abee84b132e42
3d7b3afa02444597bcc385af3ade26e7aa4c85ad
47741 F20101124_AAAOAJ cecchini_m_Page_061.jpg
4c0d6e8e8384072b3479e5e3e5b4b251
5776b93e5045bf89c6a5121e31042803653f7667
20741 F20101124_AAANVD cecchini_m_Page_112.QC.jpg
5164f31a14b27d01fd3ac3b05ee632b3
e966ec1f693699cb376f17f7f75a2a78613bb5b3
158680 F20101124_AAANUO cecchini_m_Page_164.jp2
033a327da84196e4b9cd6f1361d53a15
343d5853f1486184a87b0591b11c5734653e53d9
6252 F20101124_AAAPEA cecchini_m_Page_029thm.jpg
bb5ce82125d1ae9b582cc56379a96f36
d8346b34a6ec97d1114f2cf96ad9c639b07c400a
16174 F20101124_AAAPDM cecchini_m_Page_144.QC.jpg
7875fc819ec97cdd8ae3b32e76bfe1a0
0e94d91fedbccd017f6219d1932e7cea70e2c0dd
2531 F20101124_AAAPCY cecchini_m_Page_182.txt
4072425b0b3bc5545ae94449bdc4bd7f
89380a4dd6b399be936123337c48198ffba199e5
1821 F20101124_AAAOYG cecchini_m_Page_042.txt
efd37e4a440df0cf3366a5ed6b734558
73a2ef72caa5499d9ea7b7c52024b0a4cadc2b0b
1699 F20101124_AAAOXR cecchini_m_Page_025.txt
609a7006303a078c26e649b1696d8b31
5929722914da26049cebdf4abd635f102915f67b
5965 F20101124_AAANVE cecchini_m_Page_110thm.jpg
f7767d25dcc5fb2ee0faff6197925cf5
fb544a6d2ce417a8821e936c75b73ef558c060a7
22058 F20101124_AAANUP cecchini_m_Page_026.QC.jpg
1f00e1b7951e6ed05f865e106e6e87ed
00e69c3038ef3e9a499a555cc35df68aaf41acf0
47490 F20101124_AAAOAK cecchini_m_Page_062.jpg
238f41d342cf55ce3f34d16297b5fd4b
a4ab623d62974f0de3aa5f2db31f1afe45dcd9f9
6373 F20101124_AAAPEB cecchini_m_Page_023thm.jpg
1295fc97cf80a00e7def49b704f7f8f8
695e5939bdf9aa672872f13f9e19cd7a4d6d4d94
22895 F20101124_AAAPDN cecchini_m_Page_020.QC.jpg
816726515ef1086d0d9d413c0f6d793a
cc5f7682fbf36e34c904a9ab8d20189cb9d250cc
1733 F20101124_AAAOYH cecchini_m_Page_044.txt
54790c4ece1cd426f463494ada395e71
079fea32741a0f9af6f750f0b7155ce92ef8ec52
1887 F20101124_AAAOXS cecchini_m_Page_026.txt
af3e3f255321f18a17f381f3cdd55353
119142bfba74499a371e0a7faf0f3edb4d4a3e35
6108 F20101124_AAANVF cecchini_m_Page_047thm.jpg
d6eee1e47c9da76143d269e2b2bfdd22
e40eabb53f140356e8063313c799e957339bea9e
65488 F20101124_AAANUQ cecchini_m_Page_092.jpg
77ca6c3ae526ea1f7902d39ea6b750f4
cd004fa95b997a98dcdb3ba2ee4767ac7b06cd4d
64054 F20101124_AAAOAL cecchini_m_Page_063.jpg
c7bc6f389acf25fa01522dd7781d5616
e94bbce3b3b5cdd4a5a65c3a0ea4bce50480c33a
5786 F20101124_AAAPEC cecchini_m_Page_057thm.jpg
5b22169f3c0bee6671d10a9f4079abf2
12cdd15e58635a4e2158e57d6401b74254beada2
5798 F20101124_AAAPDO cecchini_m_Page_080.QC.jpg
0124c1b02c82d340f0394b319d7080fa
1799280c40523fa1f8addf9d6bcf6af5495df323
361 F20101124_AAAPCZ cecchini_m_Page_183.txt
166e3c978779fc1baf4fea4a5050da85
21e588a4f349bdc6722260dc015c2782e40892f3
1827 F20101124_AAAOYI cecchini_m_Page_045.txt
bb212c47b96083fa23c680b5efaa6ee0
ef4e3231d567303382da709be31978f5383f8017
1796 F20101124_AAAOXT cecchini_m_Page_028.txt
55197d13f654773ecabd766494a8cc65
b0a9b5baf85b180abeb3d07c5b11cc33c4462725
123069 F20101124_AAANVG cecchini_m_Page_163.pro
a0d76a00159371f8e8c59fd1f3ac6465
c8b336bdd642a6cacf05184f3a22c38b9d63008f
111155 F20101124_AAANUR cecchini_m_Page_117.jp2
bf8b5cf0c2315194f5c3438aca81a52c
98734a850f028f519af398680744cf3635a58db1
52400 F20101124_AAAOBA cecchini_m_Page_079.jpg
34c1bc4e0f2ad0aadfb96f6af23575f2
3bbcab5067b17a958905762695d9b5aa00a0fc07
38922 F20101124_AAAOAM cecchini_m_Page_064.jpg
515a3729e3fb24013dd8838f19a2b798
75e77d804e4b259f3996e7407e461459b5c9a9e4
3493 F20101124_AAAPED cecchini_m_Page_136thm.jpg
2b74e35657f7a61598cb281b85fa1dad
03cabeb661c25b30a8214bb4de33e8533cbbd02e
5986 F20101124_AAAPDP cecchini_m_Page_086thm.jpg
d9070b37594ef300030a67430f82cf53
e7d75b86e0106f788a5cd74ff49ea1a9c56b8088
1614 F20101124_AAAOYJ cecchini_m_Page_046.txt
b4350a4f7ab049a9841b830113647c58
4092adb8411df9695d0c6e58673eca9011410722
1914 F20101124_AAAOXU cecchini_m_Page_029.txt
eeddaa593068a0b570b096baaaf1a109
9ed02c4f2f23112c4bb5e95ed805578c4692c168
33171 F20101124_AAANVH cecchini_m_Page_007.jpg
435e3d04a92c00db0429b58ac58ca088
678dfe8af528de89873c1b5fda41a613b49eecaa
24937 F20101124_AAANUS cecchini_m_Page_065.jpg
93ebea429936d844050af9ddc0edce4a
d6086a5673068a6d4e2e53433147a43aff6ac09d
17014 F20101124_AAAOBB cecchini_m_Page_080.jpg
3a73ca9265ebfc3f1cfab149a64cb4e7
f32d019df2d14606722ead3ea3ae641e535138c3
61380 F20101124_AAAOAN cecchini_m_Page_066.jpg
526b5b0f706ffc5a82d53b5a1a1d6b18
514013075a9f4c073240b869ce80dd97b41da8d5
21467 F20101124_AAAPEE cecchini_m_Page_063.QC.jpg
17dbcbeca7755aa3f031541540b1840f
83e825027044f6af10c87e3b6f3d8215a0bba66e
6231 F20101124_AAAPDQ cecchini_m_Page_015thm.jpg
c89b0e2be015d3c5085241421f9627f5
2dda5b564cba2e805387af6250f77c335e90df7d
1727 F20101124_AAAOYK cecchini_m_Page_047.txt
1d9dfa18fcca33ec3acd131d82456fcd
d6ab3b8b8ca4f6354f0d548d7ddf705da1122b04
1743 F20101124_AAAOXV cecchini_m_Page_030.txt
28752d66f912077b22c077708d7cbd06
5e34e29433e43c1a8152e5c524ce16ac7e031172
1491 F20101124_AAANVI cecchini_m_Page_051.txt
37738bc285399ad9a533ad60dd281d5a
0d829d8d2aa6db6f3a6f68635fe6eb30207e2cfd
6591 F20101124_AAANUT cecchini_m_Page_020thm.jpg
26c8b0a848ecfa0218c95385af3a8add
9d6b6c75caaf3dfa79afee6922b45d0e5ca8cb2a
48547 F20101124_AAAOBC cecchini_m_Page_081.jpg
92f9ff227effc6f28a1dd07d3cf01d0f
f077801e4d65902c99aa3859e157348474b86874
46474 F20101124_AAAOAO cecchini_m_Page_067.jpg
27f3c0560dc7844acf40567bc1da7323
409cddabc5b339ccc22ff540e6c65cec16c746b2
14709 F20101124_AAAPEF cecchini_m_Page_162.QC.jpg
7267d904000fd8cb4fe7c6d71824c3c4
982f9d3aaed6d01ab1b4c5f36ef5acefe057b76f
21768 F20101124_AAAPDR cecchini_m_Page_021.QC.jpg
b7c8bd3a00ab9a4094fe7011e89d3cab
558b1f03483ade8eb5a859a8e12bfe37922a5048
1767 F20101124_AAAOYL cecchini_m_Page_048.txt
322f6311a35d7c25efa8ed3fc03feb3f
1a51509130b087798a2a49607635b9277a6b8b56
1860 F20101124_AAAOXW cecchini_m_Page_031.txt
9d50f4e1341ef3756d6be1b24b773965
05088d79bccf9d815af0ec34e88100695cc07fd9
1101 F20101124_AAANVJ cecchini_m_Page_125.txt
dc455d36671dc7806539d1d899897bcd
6074f874d345eefb02221eef5ebf1727fdba1eec
48393 F20101124_AAANUU cecchini_m_Page_029.pro
b073c98513a3738d65108dd172313608
46909b3a0b6f512213a256d2a36f40771dba76e5
39700 F20101124_AAAOBD cecchini_m_Page_082.jpg
00cf3ef59aafe1895c001610ed10338e
832510e06d15fbd7a614aa6f2b3c6b3c09249195
27330 F20101124_AAAOAP cecchini_m_Page_068.jpg
0d6555e212c85594164dc93e285b8ae7
5760822a50fd3f1e574a828f46b2eed340237049
5816 F20101124_AAAPEG cecchini_m_Page_085thm.jpg
53fad7b41184e6155a92fbb7b06e509e
ea047ea400d4e4f6e1afdc29fe7eb9a3e8b1757d
6515 F20101124_AAAPDS cecchini_m_Page_024thm.jpg
62cbd3e8a3bade95c0e69544aa881711
6441b89c7a04cc4300447d7b937791fc06674b30
1233 F20101124_AAAOZA cecchini_m_Page_064.txt
d268fe51b211ce658ecfe008aa60c040
44d62ffad877208469a7d4d4f54840ec5a7b58eb
1280 F20101124_AAAOYM cecchini_m_Page_049.txt
29109db04cb9608c1843b08118000946
15fca95ed54aec5fcdd60b11c2cb9897260e71ac
1964 F20101124_AAAOXX cecchini_m_Page_032.txt
aae2075e87d812b0faac8c3f50f409c4
6f08826a97766ea26d13657cde5e57724f6ba686
23105 F20101124_AAANVK cecchini_m_Page_033.QC.jpg
a9a6d93d4ac48985722bf21e1347f9a9
6b0be7a7313237e5bae31a253259fb5ad6225b56
11062 F20101124_AAANUV cecchini_m_Page_004.QC.jpg
eda5183e77d4118ea09e16745b61b804
90e5dbc89348eb5939cf3be00cee791b4ac1bc59
42082 F20101124_AAAOBE cecchini_m_Page_083.jpg
8e7032fa67068c6529f6fe233eedd932
bb151d33283735720c22fcaf0d47e605536ccfc6
39514 F20101124_AAAOAQ cecchini_m_Page_069.jpg
2babe0a9e3a6c6010cef2e15cee45f57
d07232eda1c1098ed14c2e876b0f90fd9dcb4706
20269 F20101124_AAAPEH cecchini_m_Page_048.QC.jpg
77c2a578152769aa406b9b6a56f3887d
8882beb2c69c2b65bc48f587b5aee034e6b762a9
4101 F20101124_AAAPDT cecchini_m_Page_064thm.jpg
f7f50e49b632895a6cd935f2fe3138b8
389ad4f005e3daa43875ee1be1e728376b5a7891
184 F20101124_AAAOZB cecchini_m_Page_065.txt
f1aa9c0d4b9f39f97841268294bf802e
9029465fa1b5963fecbf89670fc393152ffce127
1339 F20101124_AAAOYN cecchini_m_Page_050.txt
f99ab4a32a6534ab0855ef1aa98e00fa
75ce193f1bc6ccf36aa1c1ddbb54b3d31414acde
1658 F20101124_AAAOXY cecchini_m_Page_034.txt
d1d07b4569381ee0d76c93ba602e249d
e3602612c67e577c27f2f95e4921554eded8eb51
3424 F20101124_AAANVL cecchini_m_Page_140.QC.jpg
a9a1a5485244caec2ddc9c9c8fe5a50b
5c88e3f4b161706c413d31abb9373cca2e8337dc
3918 F20101124_AAANUW cecchini_m_Page_006.txt
0468137894beb73e73dbfb1d9c0b82ce
b9cc02f5fadcaa370b74af3e3c3a02d9e75b8c1e
57280 F20101124_AAAOBF cecchini_m_Page_084.jpg
2afff71aaf038e31ace998721d0a2707
31103fb1e1d7dd861af594c6ca9aa8a19cf92d31
48785 F20101124_AAAOAR cecchini_m_Page_070.jpg
85f09475d2549762845edfa469d7c0e5
e2120a6a4251bd5fa1b3755e2e12eeaf66eb4ee5
22897 F20101124_AAAPEI cecchini_m_Page_014.QC.jpg
5a6873fe0027bc2a943b3f83d5893ab4
cc98a3824281081625ea38ede5021e6d4345e866
6464 F20101124_AAAPDU cecchini_m_Page_091thm.jpg
e4b088f9e577f1753de415617004932b
43d892db3e314de223411d38365bbc9768b5413b
1712 F20101124_AAAOZC cecchini_m_Page_066.txt
c4fa1f2423d559a48f8ce3f6e51a7fd7
b31c64907c31c05960c0c9f889eff2bc450d6776
1352 F20101124_AAAOYO cecchini_m_Page_052.txt
16266feb1ba6402305513a9b7d9fe9ef
86a0f56a3408ad51a1e6a03609f5f9a4b417df98
1764 F20101124_AAAOXZ cecchini_m_Page_035.txt
0cda0e6434719490a88dc84fad35c49c
f5b960214184c74d93ac9c8d35a3a6c912b45818
5213 F20101124_AAANVM cecchini_m_Page_051thm.jpg
5cbad0beb5fe2573e7b0b3b2a5ed5cde
664135718f4f1c387943595efd4135565c2ecba7
51304 F20101124_AAANUX cecchini_m_Page_164.jpg
297ea21000de2d7378c8680664b3c667
146b88434d7ebb4c538d1a74d593953562e3180c
58085 F20101124_AAAOBG cecchini_m_Page_085.jpg
829284909b45c7dc159f37abf2161642
cf020b17747b1201db0ac664c4f95015827fb7d7
98375 F20101124_AAANWA cecchini_m_Page_021.jp2
2e22e33a3bf8717eccdfad4a05e8638d
b323ffe1777b92b604c03a8e32edab59813307e3
61285 F20101124_AAAOAS cecchini_m_Page_071.jpg
7ba01a755021b2ab601674ee7cba724e
baf64be6b71a2dadc5a401f01774b2b60b8c9e99
6664 F20101124_AAAPEJ cecchini_m_Page_181thm.jpg
19d9c9780f1a6cdc80fca6a06fec616f
1aad076af9492c1bcb399ee7a6ed2c7db9d596c4
21446 F20101124_AAAPDV cecchini_m_Page_035.QC.jpg
5286bea60687b895cb878e9325269130
e68e7377daab13216675dd60b8645a53749c51ca
246 F20101124_AAAOZD cecchini_m_Page_068.txt
50f7f45d9292549c327ae17d0eca2a60
ab2a14d09ac3c2f64aa69185ded0537decb5d027
2006 F20101124_AAAOYP cecchini_m_Page_053.txt
a6690489c47332e604b417e357ae278a
d89f796da16ac6f891675377c0d6efc8c5bab556
F20101124_AAANVN cecchini_m_Page_098.tif
a46eecb506c0aa99fc9f0d8d835fdfa9
9ac227fc01399705658864215017e2744439dce8
40439 F20101124_AAANUY cecchini_m_Page_025.pro
6832ac8956cdfb741b5cea03abda3cea
09efa887c96abe71eb9a88c6326b931ae37f8ba2
14874 F20101124_AAAOBH cecchini_m_Page_087.jpg
2fc05684f0ead855020da47816050b7e
cee6ecea1d89875730c4dbf18dc5f7099d4c7697
1051973 F20101124_AAANWB cecchini_m_Page_179.jp2
12005fb9405444d3bc5b9d561718686f
e2bb401197274ebeeb690e89dfa6accc365c3c19
64052 F20101124_AAAOAT cecchini_m_Page_072.jpg
6d1681a168a76f12f92c89c17ac39ad3
416c8370b3e73d694740a27811f6d5dbfff146c5
5248 F20101124_AAAPEK cecchini_m_Page_101thm.jpg
6cce8e00b1371db169f6649300dec6b7
7e83d8692db926df7b10274c56aaefa3e7453f4b
13890 F20101124_AAAPDW cecchini_m_Page_056.QC.jpg
361f8f8a10ace62bf9ab6cb57fefad4e
40dcc38ecb38a81b8007e8809b27e8391ff58d43
976 F20101124_AAAOZE cecchini_m_Page_069.txt
f92b2f37d993c3c9d2d47838d489a671
c592fb10d64ad8268b62a547970f2881ac2cdf39
13045 F20101124_AAANUZ cecchini_m_Page_125.QC.jpg
41ff9dbbad6eae5aefc1e7c715c09314
466e7917f3b87bf1a6eb329bb3233ca4ab06ee3d
62996 F20101124_AAAOBI cecchini_m_Page_088.jpg
1daab69bd9730aba8af5b30fcac3bf3a
c334d0c6bdddb726c2e5bde2f3601c30d32d747e
41803 F20101124_AAANWC cecchini_m_Page_128.jpg
ff1af279186a9c364647a4564e9f68a7
00909da5edb3f390dd4f5bdf3f3c6dee31b3eb7f
64485 F20101124_AAAOAU cecchini_m_Page_073.jpg
721be1dce701a001b168f7a824564c03
bd663b6e8b60baa7c425685a74cb18f06e94d708
23725 F20101124_AAAPEL cecchini_m_Page_089.QC.jpg
5027ff85164850ff7340ca7ce418a99e
0cc8d107ec42f5ff87551ffac7b4430850bfe42f
6564 F20101124_AAAPDX cecchini_m_Page_033thm.jpg
621a8e988d343f068c48514f9ce1eed5
68e488d350cf169f7ad159b427e89c024d1243af
1616 F20101124_AAAOZF cecchini_m_Page_070.txt
15f2fa63008cf5db23c3d8099b2cf5ba
c8ba83fa6428de79029a51f05f3777e902b126b0
831 F20101124_AAAOYQ cecchini_m_Page_054.txt
e586a9bdd82d08fe6ca766b61391c027
ba0f31c53ea268b5cc10bb555078184cce21fa37
14567 F20101124_AAANVO cecchini_m_Page_157.QC.jpg
7162651ee06d1cf7da8bbd9d17ff1f1c
c2dd5ddec0d224a5fd3fe76fe977bbdd4006a79c
84480 F20101124_AAAOBJ cecchini_m_Page_089.jpg
0ecf749d4ccc52f6b4dfbfa87b18b738
ed5af2afa8ebf93da66e0cd67dbe4becd0e074c7
F20101124_AAANWD cecchini_m_Page_121.tif
f1d01fde9a0f283ff17d751e191a971e
4b8f65a8116231de20419b1be826bfe5918a383c
56855 F20101124_AAAOAV cecchini_m_Page_074.jpg
518a71fe61ba3785c4ad5115e19e196c
71663ccadaf6f127fe3e298a3ac50b23856891f6
15869 F20101124_AAAPFA cecchini_m_Page_134.QC.jpg
d9c5d71c931d6c1cd0f08f45b6065fbf
e02756c489d2b0841c5ddbd99094c1f19640dea5
17485 F20101124_AAAPEM cecchini_m_Page_108.QC.jpg
69db2e2749bbd036de6183d7d1344cc7
d682b8f192ef8204a42ea1544fc075cb8a35ff77
14476 F20101124_AAAPDY cecchini_m_Page_156.QC.jpg
91319603d75adbd694af1242c90bb7ae
904fcab8940dc95d55f166ad1a352976bf517847
F20101124_AAAOZG cecchini_m_Page_071.txt
9e9b87c086fcfc12bb34a70b338d761d
1780d78953d772faefe262bc7745f584103de028
1878 F20101124_AAAOYR cecchini_m_Page_055.txt
9f1ea38ce5e74c7f467735a4b5078edd
17f8a55892c5d94167e3b477fed6e62cddef147c
F20101124_AAANVP cecchini_m_Page_016.tif
5b8649aa135ed46bf785719e3ec1f4ff
2f30201678d6b83660017b37e995f2228da0bbce
72483 F20101124_AAAOBK cecchini_m_Page_090.jpg
0ddfe09d1ae3c3968b0ed9513f4b4cc9
dcfbbb47a6629df03076de1baac73443a4f5eb7b
F20101124_AAANWE cecchini_m_Page_003.tif
2224e13226ec5dadf0fec51d6ff047b1
3ad42f1694cd946f2769f5ea143e4596e5ac83ff
53298 F20101124_AAAOAW cecchini_m_Page_075.jpg
af01200c67b14310148861958091f62a
9c68af2e1cbc6a64a2038e739c4cebfa60f36487
14764 F20101124_AAAPFB cecchini_m_Page_160.QC.jpg
4f9cf81ce4f598615fa0d70b338d0a33
a66b30fbb2ac37eadef967f21f73cbf631b21c51
15523 F20101124_AAAPEN cecchini_m_Page_067.QC.jpg
fcda2e25f561c03c10eac955ef01c37e
00e23f5b188d680a8d3855dedfb1a2afc201f7a1
21470 F20101124_AAAPDZ cecchini_m_Page_092.QC.jpg
aa4787afd3a659ee2ca11ebebc260777
4d2790040613a852531dac3c9e880f0559bc6a42
1799 F20101124_AAAOZH cecchini_m_Page_072.txt
98ea43d3fbabcaf90731f242cef2bc74
622c70e70d44ebaf1a3d54adb057c182328fb1bd
1292 F20101124_AAAOYS cecchini_m_Page_056.txt
43f300b4e9698a7a6574d014d8ef89bf
34304ac3f4fba7b9d4262d718e95a420526b1345
1809 F20101124_AAANVQ cecchini_m_Page_096.txt
2c5c6e59454666f9907fa6e947f770f9
90f0d72a554b55422ba270b1a254ffea21283844
77855 F20101124_AAAOBL cecchini_m_Page_091.jpg
a9e53458c6c837a8670172e099cb808c
3050a372b88865a824c01096bdee783d9c9c99b0
103026 F20101124_AAANWF cecchini_m_Page_026.jp2
afef9681604a2b44cc105edd9b944127
eb512d51e1d641e85b66ecb209b08919cb1f4a0f
73261 F20101124_AAAOAX cecchini_m_Page_076.jpg
4d546dc942c1b3ede9be77e3fe0f0d1e
4b832a264966d02824e1fb81ad6333a5be38fd4e
20480 F20101124_AAAPFC cecchini_m_Page_028.QC.jpg
551f80444739a7db0f7ae81a75cdd967
2f825bff2f2c5754c414ad1e72a279a3edc5a587
16005215 F20101124_AAAPEO text_data.zip
af443ce169a7a36a46fb7378749610db
1da24bc260782dd997969b701607aea656bbdc0d
1849 F20101124_AAAOZI cecchini_m_Page_073.txt
f3a04ee49c7983524de70526f1c3d6fa
84ae37d9c943c82e1f6da7a1b705f12388827f8a
2106 F20101124_AAAOYT cecchini_m_Page_057.txt
430da4a98ecdb5c8cd99b7b45b99888e
2d815b2a0455a659821791f0317018205f0f4336
17107 F20101124_AAANVR cecchini_m_Page_139.QC.jpg
ea71a1f4361b2b3d1f99a96a92506b4d
910cf734c51f8a7a76bd10b8f7beb390fbcb0af6
55483 F20101124_AAAOCA cecchini_m_Page_108.jpg
011f2ae53328d660d6c2277e3f33033c
6128caad88f5e8bdcd2eed689e12d209304772e7
71193 F20101124_AAAOBM cecchini_m_Page_093.jpg
fbf702a885983c949a3f9cb57132846b
72ae88b40a50981f6b2e23d2ca044b8803696510
1381 F20101124_AAANWG cecchini_m_Page_174thm.jpg
93b843147797aa41a1ece678b4072bd8
6f99cd9fdb88cd52f884b7509354b83ce9aed964
13922 F20101124_AAAPFD cecchini_m_Page_151.QC.jpg
ab7dbd72a26878e6e95dc8f834dd1e3b
d6149b937ffe98d2925c3f4c2abd65a33493bd62
1464 F20101124_AAAPEP cecchini_m_Page_138thm.jpg
a190e12a22b8d53ebd4503aaab8918f5
983d8af40df286c28b2faf72bbf7466cba233c81
1628 F20101124_AAAOZJ cecchini_m_Page_074.txt
3dec4125d780cc72825e781daa4264e0
ce7b20e20a16ff2e5f33e90e525e01a74947586e
1741 F20101124_AAAOYU cecchini_m_Page_058.txt
2b82e0c260df8c52c5a7c7241ca77f40
6cf370fd1132104336e38d7a67737a4558c309cb
17520 F20101124_AAANVS cecchini_m_Page_082.pro
ae38896a9bdb184ef17124a89f4eab81
39c21ee1c0c60009ea02dfc91e7a7f32ef4276e8
71939 F20101124_AAAOCB cecchini_m_Page_109.jpg
902438b2b24208564f07766753a0727d
0b0b9ed7a178802cbd6f9b0efdab14436289d3be
73386 F20101124_AAAOBN cecchini_m_Page_094.jpg
1d06bcda246bda20b65b28817e376c8a
ca735fb9bf3bf3095d27b90d5c6afe9fabcac641
62526 F20101124_AAANWH cecchini_m_Page_135.jpg
eee4aaf18f97b450ee1d2a9a8b3d967c
5f2ae732356c43256ab60f2fa1f8e2300b6e7a18
48796 F20101124_AAAOAY cecchini_m_Page_077.jpg
1fc9ba30e7117ed28fea4385d68cb13c
52329b53255fb10998597d8c287533ccf0fc3620
6366 F20101124_AAAPFE cecchini_m_Page_105thm.jpg
17e101331a5c6a616444f0fb77bc6b86
9bbc492bc9d77a61a8f750ef09e3b300da71fc3a
5050 F20101124_AAAPEQ cecchini_m_Page_062thm.jpg
c802f393f4f08ddb1f810a9e0bdd9bd3
54691c538307dfa362cd20033e55980954df6644
1148 F20101124_AAAOZK cecchini_m_Page_075.txt
6b3c79d8c38bcf9663895e0b875cc158
b781fbecef672b4830198a151814aa0fec660eb3
1370 F20101124_AAAOYV cecchini_m_Page_059.txt
cde88e46ea7163627b28cf04f6a78140
601391a8e93f0df3cec1d2c8ae4e28039f880df7
F20101124_AAANVT cecchini_m_Page_136.tif
26868c253396c1b54981aa96fea792fb
4ae68344535e9aeca484cf0d26517cb5ccf626f3
65806 F20101124_AAAOCC cecchini_m_Page_111.jpg
8ba3ee433cbf3439b055ff6a7b3868ea
5b3216c41e53d52e83aa0aed0350401fe02917cf
69923 F20101124_AAAOBO cecchini_m_Page_095.jpg
bc63b45081ceaa1cd35df940bc277291
8b567268e291d48ece5617b96576b385e60bcf6e
F20101124_AAANWI cecchini_m_Page_051.tif
7e6e8d7122cb1fb1e15ee0a1c1f84a98
33df7b6ceac412330c0a80ef24ea20ff2ff80625
62765 F20101124_AAAOAZ cecchini_m_Page_078.jpg
f6b61fab754a0f629976f87a528fb224
0f430707acc0a555a3ab41c0ffd17a3c2f6626a0
23592 F20101124_AAAPFF cecchini_m_Page_053.QC.jpg
bfd616d233c9a84f82da95dc40a10d18
604612c78bb8867aef948f85e9bebb675817c2bc
3811 F20101124_AAAPER cecchini_m_Page_162thm.jpg
4559b627ea54f1df52ebf2376e5be124
fd18cd9bafcbe81b9d3f28ae0a6223d72ba0a291
918 F20101124_AAAOZL cecchini_m_Page_077.txt
122a98a26c429bbd6841370eef036bce
910413f91f5067a360f1b8b53b461d77da320663
217 F20101124_AAAOYW cecchini_m_Page_060.txt
e9b4edcf7b57b1c22015564bf60c9da7
6d4a021e03f8fe0688143020b6051e33d3909ad9
90313 F20101124_AAANVU cecchini_m_Page_150.jp2
817578d701f4a687cd5d1af555a5b30b
7974933364ebde8186ebe9f3f886dc3084791589
63613 F20101124_AAAOCD cecchini_m_Page_112.jpg
60a046895f144494376096a5c1c78ce3
cbdcf4b8b052f085ea7821b850b69e73dd117f01
63383 F20101124_AAAOBP cecchini_m_Page_096.jpg
0b154664aaf9ea85018f8fb3e471bc5e
fa8494c249e1c405c46478669b21db8190d0868c
F20101124_AAANWJ cecchini_m_Page_097.txt
42f29c59dee2ff3173ce701a942d9479
ff9fee621b7c36c5dd3ba18107df37b6de8e4c5e
5789 F20101124_AAAPFG cecchini_m_Page_012thm.jpg
ca8758bb856e8fbf09a9de46feae89b0
81f04c6cd38cfd5442c770c0f7ef7296c7fb2dd6
5708 F20101124_AAAPES cecchini_m_Page_048thm.jpg
68f12c1385eb99f51197ce67ea05c8b6
e565305426b1d85b2bb335408e317439011b5dc4
F20101124_AAAOZM cecchini_m_Page_078.txt
6623cb59ce74db28b801dcfd1e2d0e8e
16aa1a7520ce3c15ca2672702ecf168b1fe47b62
1163 F20101124_AAAOYX cecchini_m_Page_061.txt
ed293caa9632afabf69d7eef5e293a0c
7a82501ddd2fe92bd6bf9be84ba0d8635ed31205
90246 F20101124_AAANVV cecchini_m_Page_057.jp2
bdc491d1eff2698ec5dccbe698fc4730
c9a2d6a6076f42b670182628cc9a17c8cfb4067a
66387 F20101124_AAAOCE cecchini_m_Page_113.jpg
b85b1de00d512237c62bbf68015fb4d0
fdac8d3cb3e4eefbc7397ec4c67d1ef68694338f
64135 F20101124_AAAOBQ cecchini_m_Page_097.jpg
16f9e4e82f1f11df98b15629af5264dc
ec89981b89abc16e9a408d5674eaecfc66c57676
F20101124_AAANWK cecchini_m_Page_170.tif
c913c30acde9940bc3a5d8070271ebb7
5f11c87283fbf57368ec0a72f54cb09c6dc2c7a4
19562 F20101124_AAAPFH cecchini_m_Page_071.QC.jpg
4ea80c1a3b1ed77ede21a6b8bf6ecb79
8e06415beefc02ddb6095970d3c635bcc6d88531
6017 F20101124_AAAPET cecchini_m_Page_072thm.jpg
e5e1affe94ba798891b7780611f4583d
491fae9db16bef3ad9b2fef684200ca955439967
1490 F20101124_AAAOZN cecchini_m_Page_079.txt
e978ebfe9d3ee0b84ca5971f08084a96
38f5eaf3bf5cad287b9414a472c63e94446fe864
1848 F20101124_AAAOYY cecchini_m_Page_062.txt
2f4f9c4760b26a49538b565a7b008e7e
6ca0ad6ba9d2f0defb3e02cf515f142de66786b0
20017 F20101124_AAANVW cecchini_m_Page_066.QC.jpg
240c5dab2f1ebc88141d60429a7f3ad1
edd353188fa8c0da46343fb105dc56373346c277
71485 F20101124_AAAOCF cecchini_m_Page_114.jpg
17c537072cdd385d930afd2f84b8a331
0a2e6bd54107f9d5fb602ecb4fee971453919779
55907 F20101124_AAAOBR cecchini_m_Page_098.jpg
4bf793cf5ad7a2ef67b0394b6f2826c9
44dbb4a338d7b235da1535f69afe0d5b097d2da9
51600 F20101124_AAANWL cecchini_m_Page_155.jpg
b5ace216b3d0d94fe5da318740ad1fbc
9b082f73beca0b1e577e4520c9857691c6536cac
14662 F20101124_AAAPFI cecchini_m_Page_130.QC.jpg
84edae84551c58f0a648d7ef772bdc02
e0a6f13a66d32154a9bbf6a52ae90b5083bc84fa
2660 F20101124_AAAPEU cecchini_m_Page_145thm.jpg
d7bab53c18b677ac9e03258e9812ccb4
ab88cd1038e765deff64dbcddb5e687bbdafde9e
318 F20101124_AAAOZO cecchini_m_Page_080.txt
c0b2e06c879cd9385931fe4817489b05
9705a092b423ccff122f5a553eee1d46b9d446bf
1907 F20101124_AAAOYZ cecchini_m_Page_063.txt
a0825d7b3802e1ba930066e7170e2396
8436657d39339d3baf98056ec2e449480b2c5013
12252 F20101124_AAANVX cecchini_m_Page_136.QC.jpg
cd468c828435feaadddad7f933356e1d
9459f0c50fe711e6d88812deef58bcde7b87cb10
65677 F20101124_AAAOCG cecchini_m_Page_115.jpg
aacb5b337a695cb5864803dffd4a330d
be6665120e1d1d5b61a8aab3d18d7b90c572b620
56704 F20101124_AAANXA cecchini_m_Page_128.jp2
778fa89837bb5dff8a213368916f7006
94f2438a390e3b582a44b429eba2eb8b636eaecf
26437 F20101124_AAAOBS cecchini_m_Page_099.jpg
7b0f27312147abfa7e91b3f6564cf01c
1df373bab1f024416a64d292c9a6e2036bd18763
42717 F20101124_AAANWM cecchini_m_Page_017.pro
21e1fe8308a7f306a996f1fcdf41fe29
a0a09ff52ca52168e1984b350bc02bdde96b394a
21204 F20101124_AAAPFJ cecchini_m_Page_090.QC.jpg
b6e3259b00432f6d1e08103f5a288b81
e4e2421ff732c3f34f0521994ca7ca7294e3abbb
21914 F20101124_AAAPEV cecchini_m_Page_036.QC.jpg
eec006edd2ddaf26dbc9072bfe54ccf2
577575aa4d38d64af5d4af75a453db564c157461
1258 F20101124_AAAOZP cecchini_m_Page_081.txt
eecd337ea500fe377b753adf4a638c7c
41b22a27825b87825af0adf8cc610514d7b033ba
124408 F20101124_AAANVY cecchini_m_Page_171.pro
c10b28a9a29f2eae9f45518778016dde
b020cfcc7885369886b8411dc92c79e5f2570fa0
57458 F20101124_AAAOCH cecchini_m_Page_116.jpg
54e5147d21c240886e0cdbfce5c6335d
4e20ce58975a80d1fb5785dd09e95b134e5d36b3
40721 F20101124_AAANXB cecchini_m_Page_066.pro
da2ca957ca7415e4f4fb8d22d5ec53db
239970a606cf8da12adc27bf405c6e93d0bb8d65
57541 F20101124_AAAOBT cecchini_m_Page_100.jpg
22897558ac2d9ba98833b422d20eecf6
80fd0e2ec24faf2a39be216986647de947669dad
F20101124_AAANWN cecchini_m_Page_103.tif
1fa5a4c754e79015f57fbfee9e567513
fc28050f8a7b41493c3ba293c79b357f6dca376d
3768 F20101124_AAAPFK cecchini_m_Page_156thm.jpg
146e51253bf2fdecb08427b6b9190918
05af41904ae82344557fcc2580e7b8c54ff6d930
20490 F20101124_AAAPEW cecchini_m_Page_057.QC.jpg
c9f613d00b535a61387cf6bf8875e236
4656e7fe41f1e12bd0870af08ff4f87ee0d10a72
711 F20101124_AAAOZQ cecchini_m_Page_082.txt
47eec6a6f7de53c295b90f186588ced6
8e91ac875b1e548d027e512179bc296a3e0d8b18
2027 F20101124_AAANVZ cecchini_m_Page_076.txt
d09fa99c23e7c2c3f826c1734c99af07
4495092bfc144bbfb3993531c33945b198f74b53
74572 F20101124_AAAOCI cecchini_m_Page_117.jpg
bffe90b48f567abef2bbb43a01003b00
11f1db3189fb7e09baa4adb3c47500f10faf953a
204 F20101124_AAANXC cecchini_m_Page_003.txt
6ce38c610274d8885867dfcaff6623dc
e7144a2b924841eac4059b093992384e0fa5076e
55812 F20101124_AAAOBU cecchini_m_Page_101.jpg
f18a5736067af6efea6ea04ed4215a98
dd20aa0ca49eb19467242f1f6e21a4d2cac6fa3d
114037 F20101124_AAANWO cecchini_m_Page_146.pro
b071ffb7e1168ff496f7fb35bfd85988
58b555fd9cb06a862eb1d36c413c23400ac8c5a4
18918 F20101124_AAAPFL cecchini_m_Page_084.QC.jpg
f33433ec563810bfaa196d35e75d346a
d40a7ca546bacb8ce4a7a11ec338887cac3c42cc
5688 F20101124_AAAPEX cecchini_m_Page_088thm.jpg
93bd3fb0175e429117cd1715b4bc051c
9d9b06c12ac0bc64fdfad9cfd21ceaa61e71948f
67742 F20101124_AAAOCJ cecchini_m_Page_118.jpg
c06085e5c016e7b604953101f738bbf4
4641820e015e96c6b812bfc34645e5a85f56e069
68376 F20101124_AAANXD cecchini_m_Page_035.jpg
7f7a1a4a7b16cf4fb59d1e5764c27e4c
c968b5c818cd1e98efe8e0d5cd5f75a28ae73b78
63539 F20101124_AAAOBV cecchini_m_Page_102.jpg
0c4ee99f3bcf12673cb6a19603eabba9
22e1585dd7edcb7272ed772a19836fb2c7713b2a
6403 F20101124_AAAPGA cecchini_m_Page_114thm.jpg
0778809835a402f2e5e223f59a3d933d
bbd4003adf03b23d7bc11380d9b14e013be85b59
6207 F20101124_AAAPFM cecchini_m_Page_021thm.jpg
1450181c1847c8aa27f29adbefb57804
65c9d60c515df571f93e001dee852da8ce5d87f0
14424 F20101124_AAAPEY cecchini_m_Page_155.QC.jpg
5291359f3311cdbb74054b6930fb6507
05488fd2cb1e14434a1c99c7915db70e3960dad7
1201 F20101124_AAAOZR cecchini_m_Page_083.txt
d435a2f89fc8f6632fdca7dc41c60cb1
eb73fa5ec515f5a00ca1d1e9ddb5f0963f84cf29
73145 F20101124_AAAOCK cecchini_m_Page_119.jpg
ef78dd175486a5c52ee00159ffaa7728
96906d53a6edbbafc34714b648114312b93a1a4d
1117 F20101124_AAANXE cecchini_m_Page_126.txt
8e5c3ee6db6b6fcc10a139417abaf020
08eff578368e9982efcfc37eb54b877d4b43214b
70399 F20101124_AAAOBW cecchini_m_Page_104.jpg
776e9b64e85ce9eaed648cf08a46d81a
a009b34d4a770641756638685e4224977e505f7a
3349 F20101124_AAANWP cecchini_m_Page_004thm.jpg
588f25f6322370dc966059dd430d6b7f
35833719eeab8348156948e780948de08b2e35e9
276807 F20101124_AAAPGB UFE0011430_00001.xml FULL
77680d37bea16d23f2a27ba5359c6ff2
0f28539c349de98ae17540261017ebd31e7410e8
9170 F20101124_AAAPFN cecchini_m_Page_150.QC.jpg
ad0584b98fa973356681bfb8b76809f7
a2415a37cf7c1ff4acc954c2ce0da6f32bef2911
20914 F20101124_AAAPEZ cecchini_m_Page_123.QC.jpg
59caaa8a37db55bba0d8402ecdbc79dd
f0ad3cae8affbbfe8c736dd772b3ebb447e1f126
1637 F20101124_AAAOZS cecchini_m_Page_084.txt
2a05639cd5dfa50c4d8a08869f9f22d6
a6a5bd657c8adb34dd204a9f430604b6c20d02b2
62079 F20101124_AAAOCL cecchini_m_Page_120.jpg
382738f2548eb7d4d1533cd69f9919e0
f5067d3a971c7f2a6c0ed0b750c307c049bd4ec5
F20101124_AAANXF cecchini_m_Page_149.tif
eabebfae8791c6bde7b36c5eacf96298
c1ac4d72971407b72c8353c677c0a3eb0a088e82
70424 F20101124_AAAOBX cecchini_m_Page_105.jpg
fcd2a5ecb88c2e7295495f23c3a69b57
69a6f2bbc297babf78017f0ed7d537bc8931707f
62624 F20101124_AAANWQ cecchini_m_Page_126.jp2
43119cc98acf0c2beb5b2ac4e0eac89e
e1bc48231f40462376293fac69be8a3cc61936f8
7458 F20101124_AAAPGC cecchini_m_Page_001.QC.jpg
6013af11ff4c4848fbadd657d77305a2
66eb03f27f6ebc31a46733166d18c9a9c965c605
19006 F20101124_AAAPFO cecchini_m_Page_041.QC.jpg
739d73a5f491a4343be1d5cb6d349c93
5763fa12000191babb4d117df729fa0dee877c80
1623 F20101124_AAAOZT cecchini_m_Page_085.txt
f7d1a49e166add1f25c306d36a5bfd33
440f862c11d6e88468248f1c4fc3a0a9f6155dc5
43144 F20101124_AAAOCM cecchini_m_Page_121.jpg
73be18ee85151c587086bcaf0703bb67
d68e46ea8581ce20decfc105eef89623a12c99e5
34445 F20101124_AAANXG cecchini_m_Page_051.pro
f787f66eb38998b5c9bcfbc65062de48
4e59de57541cb02680dcc80a4ac8c590f22c5f6c
67150 F20101124_AAAOBY cecchini_m_Page_106.jpg
e51750dfc990537ea69bedf752d59f66
dae6f16a63bd8e7c57cc3ed6cc38c0b4ff5b5558
F20101124_AAANWR cecchini_m_Page_107.tif
b225ef0f2f8592ec4463e4811ca1f929
362edc9548d82a5bc569a264f90a510f75624a3a
24887 F20101124_AAAODA cecchini_m_Page_137.jpg
c0c0d8d988e1b61ef105873c0badef61
e1acc8ef32eafc4baae247cb2aa8cfd2d83ca1d6
3250 F20101124_AAAPGD cecchini_m_Page_002.QC.jpg
8101823c409c719bedd1d3e0f60c9503
9f0222c7cdd5b2d82a348dcfe9cfed70bd665e2a
F20101124_AAAPFP cecchini_m_Page_158thm.jpg
8c9c82ad5cb5e5f3be11ab627ae10308
82c1abf7b1c840e6200e04139fb014f7f31cfc7c
F20101124_AAAOZU cecchini_m_Page_086.txt
d0991c0f12d6defe654147e836bf9180
61630c915419a914b4b3f0720957134bb06fedfb
83504 F20101124_AAAOCN cecchini_m_Page_122.jpg
a609eaba5a46a9fcdadd39a3b9a86534
1c80fc30657270305fd1d646615a48b959f2ba40
6288 F20101124_AAANXH cecchini_m_Page_118thm.jpg
140b15ce442e6415881be712faf9d976
87e22297cbe70f40646dfd16bbc454e134f4738a
F20101124_AAANWS cecchini_m_Page_164.txt
708a4a15ff0ac74d1cebf5efaa4498b8
42d709da75b35af2991bbf80369dff0a053c2951
12276 F20101124_AAAODB cecchini_m_Page_138.jpg
3b815730380e8796f86e4deb45a42610
2be02c320a5abb6fe8e91f345e5f3377d953ddd7
1379 F20101124_AAAPGE cecchini_m_Page_002thm.jpg
18227c50b533541873c85ce94edbcb42
c2c1a14f4ed17a90811f2c46c0a9492725c83eb2
17215 F20101124_AAAPFQ cecchini_m_Page_005.QC.jpg
a103c6fca93dd98d5d4f94c25c0f0453
0335aa125edd158d3a8ed578e81295183473d971
96 F20101124_AAAOZV cecchini_m_Page_087.txt
e0c32cb36f5ed21fa0072fceca1d3972
94ae129ec30c2282d984635db2e5b38908a0f6fc
82077 F20101124_AAAOCO cecchini_m_Page_123.jpg
b257e89fc2f1f374ba38c81cf5cb7440
dbe27272cc320feda41ed710f68ada8badd2625e
21023 F20101124_AAANXI cecchini_m_Page_030.QC.jpg
2bef1c0068f15082dc2a49fd48110210
007944adda24450c0cdd5106a950e6be100af82c
65723 F20101124_AAAOBZ cecchini_m_Page_107.jpg
f10a63a6994258c98b8e639e1ab78127
f06a1e1f69e80a7c95e4e14c3996e331300f18c2
110368 F20101124_AAANWT cecchini_m_Page_033.jp2
e57829e7c90c45f1b24eb2dd45843f08
139ef4c9ff72b7ca71c6b30e64fe3b8421e53af1
55983 F20101124_AAAODC cecchini_m_Page_139.jpg
3966cf268516f94bfdd5cde7aba65cf8
312ca4e0a4a737f8edc3b1190621cd4630bd7d72
3842 F20101124_AAAPGF cecchini_m_Page_003.QC.jpg
01362218751d12f70e12e0b27148e4b2
e30011a55d375e46445b37c277c7437223930621
16250 F20101124_AAAPFR cecchini_m_Page_070.QC.jpg
ff9301b3399cd4b289269a54129d9653
6fc1a5ccd05bf19782c1ebee3df24774a8c4f890
1815 F20101124_AAAOZW cecchini_m_Page_088.txt
71a0d3cae8c3ec530f9284e69d3263c4
1b9a55ef7bb8911e01e574cb31dfb3fdc917052d
49706 F20101124_AAAOCP cecchini_m_Page_124.jpg
a351da80865ba552aefb2fa47446a801
22f530576ab84f3107f901e07530ec1fe78c3d1a
24616 F20101124_AAANXJ cecchini_m_Page_094.QC.jpg
f8f83f516bb5f5968eeddb1c3516de22
3242aac50087afff8ac67081afd21b7a79772bea
117997 F20101124_AAANWU cecchini_m_Page_149.pro
8139d3426be3330c48d7ed357e76622a
759a4c4a26753abb125a041f8b42b6a24cba68bf
10670 F20101124_AAAODD cecchini_m_Page_140.jpg
570b4a8b2c43c56b0ef2aea1f918ebb5
96ddba1a8ecb2181f161fc845b0bf1966228bfe5
1612 F20101124_AAAPGG cecchini_m_Page_003thm.jpg
bb9e98a1499ccf683af34781fc43ca12
c0902186abbed9aca29319bc88e0e952c0d6f16a
5664 F20101124_AAAPFS cecchini_m_Page_113thm.jpg
99d2ae766249b4eb1fef5741bce85e1b
a6049654d507947ca4fc728c7aba965cb6dfd13f
2056 F20101124_AAAOZX cecchini_m_Page_089.txt
1b7bde002f5eba9fd229e810d9d473db
26d04be7b87e6befcffdc6f8aeee7ea0dcdf5c43
45019 F20101124_AAAOCQ cecchini_m_Page_125.jpg
50e7abafc0f78b6af692ccfa15e3c2b3
deaf5b3f2cd3b4e57aa41518e955faf9e81e795e
52423 F20101124_AAANXK cecchini_m_Page_168.jpg
d4ab0a5ad891f4256243656a98a4441a
4cec6a051f8841f4dbd06649c142c0b2b61b3fa1
62048 F20101124_AAANWV cecchini_m_Page_005.jpg
7d77600b290305013a160d58d684f8b5
0e000a8851c148adffdbc5791c3348480929dd89
9824 F20101124_AAAODE cecchini_m_Page_141.jpg
1172468564b9393e88639ff527e381a1
89fefcae39382bbf96028b7f4d5ae50aa023a1ce
4817 F20101124_AAAPGH cecchini_m_Page_005thm.jpg
83048ad183784e869deb200be6eb61fe
5a458528cbb2c68a71f5070857451d8d05dfadac
1338 F20101124_AAAPFT cecchini_m_Page_141thm.jpg
60647ef798bfd851f1b91d1eb9709ea4
edc82caedcaa82b97342cedf918b3428beff53b5
1856 F20101124_AAAOZY cecchini_m_Page_090.txt
c41990aad23f924e410a552b5270ea44
ea9d1941b09cad87dd2cba8a09e33bcd7b4533de
46342 F20101124_AAAOCR cecchini_m_Page_126.jpg
cc599e6fab74ff904bcace6bc7f64791
19b8e8cac7ae0d5f570c4040bde4965b5f3b2adc
160655 F20101124_AAANXL cecchini_m_Page_165.jp2
ce283e64b9be6ca16a4dde027758160d
7a5f8d7bebdedfd56aba34b61de39145e04ae913
63956 F20101124_AAANWW cecchini_m_Page_086.jpg
d222752ff6d59426865b3f5681164cc8
bb138f6eaa5b502a67efb13e77dfccb202828d11
55792 F20101124_AAAODF cecchini_m_Page_142.jpg
5362c012a7aa81f3b80ad9e55ae433f5
77db04ff72ab9f32580c350f717af3a6d5f8eaa9
23722 F20101124_AAAPGI cecchini_m_Page_006.QC.jpg
c2311e735d498cc8f60c3fc0282d5a36
dc352f6985f4ae1e8c06b858c8c71d0b9e13a4d6
24044 F20101124_AAAPFU cecchini_m_Page_119.QC.jpg
4a20712cf078735e5ffc431c32544ea3
6bdf45711142ac96db94b3061de4c1ebe0636a3f
2039 F20101124_AAAOZZ cecchini_m_Page_091.txt
2800f8806d49f7dcbf48ad4fbd72d1c5
b51063ad974e6cc2647df5f4b3b7e3f641371691
4149 F20101124_AAANYA cecchini_m_Page_147thm.jpg
da35ee86cb715c81089e9efc45f4979c
478ac2edaf5375690d218b54499aee38f618c81e
44415 F20101124_AAAOCS cecchini_m_Page_127.jpg
089abe70d1a945bfafc398a990af0cdc
3537952a0f216b1dc636c0f7bc9437793e354614
639289 F20101124_AAANXM cecchini_m_Page_061.jp2
5023125efb83883813cfd6cb73c7b24d
ef405d748d275081f0b4e7aae60878ec3811ce42
65603 F20101124_AAANWX cecchini_m_Page_103.jpg
16c9293eb691aede1fb6b261eac5f6bd
6304f5beea64f3c342267648a6245bb0f91c197f
55537 F20101124_AAAODG cecchini_m_Page_143.jpg
e0a9ff7b77242c433aae370af65aae36
cec5e162c6f4af8beb819e2a50e323e599f28f0b
5813 F20101124_AAAPGJ cecchini_m_Page_006thm.jpg
e184e616bcac30fb49867dd402196a29
fa93a5f105355cf84ef658329c1902e6eb83fe40
18980 F20101124_AAAPFV cecchini_m_Page_116.QC.jpg
c6f61e55e6c72f61255f1997babe5e72
b1103002add44efcfd1fcf13f6b8c4404d8806d9
97849 F20101124_AAANYB cecchini_m_Page_015.jp2
edbc39e36a48be959d42e696235fa3ca
4c95efb956dab9bbd0af495439fc8ac40469893c
38679 F20101124_AAAOCT cecchini_m_Page_129.jpg
6dca5399b9b61ffa244168582f7cb84c
cd3007f5e03151643c4e63ccf580294f39352c12
F20101124_AAANXN cecchini_m_Page_046.tif
f09be2355fd126de8da53331b94f8ed0
e83b724718c5cc17ba2bbbe55ff05f36ae592bbb
64223 F20101124_AAANWY cecchini_m_Page_110.jpg
b45dd70d2ad755e6b73872761e9e8d4a
dc1bffa40c8593551dcf0d88d2c8ac370cff334b
56465 F20101124_AAAODH cecchini_m_Page_144.jpg
84720beec6a0848ff543c46f5f328ab3
417f706a68ceb85e4e3a36975ae2c6e89fd0b7be
2640 F20101124_AAAPGK cecchini_m_Page_007thm.jpg
a19d7c81d9b0bed549d837f712a6ed26
d87aa1cacc4f39861e7ff03a226fa8013155aee9
3921 F20101124_AAAPFW cecchini_m_Page_151thm.jpg
8701942a38bb79f84446d37849c9ba6d
a07f8be3ed7e3d891cda4c3e5a003c71c1fd8a13
70248 F20101124_AAANYC cecchini_m_Page_062.jp2
43692e88ec8e6f852e45d4612863cb7b
e56b18e7225811d110d69d7f8f9d19bd3a8d6ba6
53348 F20101124_AAAOCU cecchini_m_Page_130.jpg
444895aa87f0fec652111f284313bcda
7ebe6660ad2aca4bcb87ca87f48cc8164363bf9f
61855 F20101124_AAANXO cecchini_m_Page_125.jp2
d5b7ecdea5f40517aad701384cf3b5d5
e1fc2d5495a22bf99f11bcadc818fdc0300cdd76
5064 F20101124_AAANWZ cecchini_m_Page_070thm.jpg
f9a71801920f141e058df2bda016fe3e
54939ee9307f0ac856dd9a7b5a2e02997f8244bb
31369 F20101124_AAAODI cecchini_m_Page_145.jpg
75fab6a760abf9e77cb836667ef0b019
649b6471d6f88a795aaf0643d4adf66d62677f73
22561 F20101124_AAAPGL cecchini_m_Page_008.QC.jpg
830b85421a296efbb66def06344af1b4
93db9ec8b7ceed68f6b1232be5ff3f5e06c2acd4
18751 F20101124_AAAPFX cecchini_m_Page_100.QC.jpg
8ada2962d7f02aa345f5f3e54f61bad9
9e3a5c27a9759920a5647e30817519b72ec89039
24479 F20101124_AAANYD cecchini_m_Page_061.pro
725fafb8790b94f30f6c9bc21bbb92a6
66e8496ad3f08b36203a9ced19363465720854fc
37796 F20101124_AAAOCV cecchini_m_Page_131.jpg
01ece86337e75b9f7732b356eafa0622
0c55c9ed34ddd9f6cf023e4dd973f731be559a28
2614 F20101124_AAANXP cecchini_m_Page_138.pro
5fc8228680bc914ad7f4de9fcc3c662b
b1fc61a8633cf6183c31e361efa4725500f16853
58547 F20101124_AAAODJ cecchini_m_Page_146.jpg
51b452077e84f73f8ae7aee2353ff82e
a3dac05d81a4ac4378182525fa0f7d13c698072d
6521 F20101124_AAAPHA cecchini_m_Page_018thm.jpg
7ff58af31269bd83c517fa351bba72ca
6ad18a3ae06736f007e4e19c2bf39061ac66e20d
5972 F20101124_AAAPGM cecchini_m_Page_008thm.jpg
bb5697803c2f62d47b5dee8651d1bf8b
148d48256d62ab67302d9d602869a9fec39b74f8
3772 F20101124_AAAPFY cecchini_m_Page_157thm.jpg
0a8a0a371c4f6d03783854919f8ecb45
e38f9703ee60284a1d9a4fa15eeef0cc334bed12
29848 F20101124_AAANYE cecchini_m_Page_130.pro
c8bc8e55778726ed6fcbaaac2e65bbfc
2be94d3522393f1799257fe9c10fbb1df951ee21
24412 F20101124_AAAOCW cecchini_m_Page_132.jpg
4fd191564f598d9d94cb8cdea842f205
89f2eea07060d3cb47ed955840e31bf2c6991cae
59167 F20101124_AAAODK cecchini_m_Page_147.jpg
69e644d26702ee132195ebc07306c0d8
ad4eeab8b8177f5e15fc774afc40e7aabd7dbac4
23312 F20101124_AAAPHB cecchini_m_Page_019.QC.jpg
4a186e8384157f908c88f22f4203d8f5
dad338c23afc7f6911e988ecc2b90218baee1eb1
25372 F20101124_AAAPGN cecchini_m_Page_009.QC.jpg
45dd4e04b6c2b221f91b63d2d5004e37
cbe8184e450e60fa5944f5ebc0b913426675beac
13620 F20101124_AAAPFZ cecchini_m_Page_064.QC.jpg
3286c46dd4159991640f32ca94124b2b
ce239d2c6271f36de5924b737b4ecee351f69608
213414 F20101124_AAANYF UFE0011430_00001.mets
e12d671feb55a528817f713a9f981788
7acbba0e3637a65ba70ab13c49e77fa231e72981
46579 F20101124_AAAOCX cecchini_m_Page_133.jpg
81fa7f61d08cbc5ff78684fe4c13b889
c09dc7f15c0b9ace1d41bb84b7a8ebbc73910379
2045 F20101124_AAANXQ cecchini_m_Page_080thm.jpg
d9c0b67be9cd2fde34f2ddc006027a83
569071556eaa6e99427cbcb75f1978a10bf853f7
59584 F20101124_AAAODL cecchini_m_Page_148.jpg
4f461d347f5385679fdd500099af642c
f7f10d2faeac6c2dc57810aad6dab0c334582743
6469 F20101124_AAAPHC cecchini_m_Page_019thm.jpg
2fbff61e94478935ed06290fef334b14
5012c7b1c8cfbe0332830a4c40c80082dde5121b
6572 F20101124_AAAPGO cecchini_m_Page_009thm.jpg
44d36f6beacec92539edc7c196bd3570
5d4171a2a24a86be58c61cb5f4882b5d5eee977c
56242 F20101124_AAAOCY cecchini_m_Page_134.jpg
3f7ed8293e156754906385708ba7e101
04581251f59b3fc6085ca1aea89c6ddabc83462a
47414 F20101124_AAANXR cecchini_m_Page_026.pro
8e5909ebd8991c3eb70b4baba6265f27
7332d2e555177e48fe794592b8630693cc9a1709
50347 F20101124_AAAOEA cecchini_m_Page_166.jpg
caa7b9b4625bc75174329cf2014db4c5
6ca6ded123737a9737390331807eaef1fec23066
61828 F20101124_AAAODM cecchini_m_Page_149.jpg
03dfeff192ad40f50cb1944bcd53bf9f
3263f209a8c346b6ee998750721856f79b1f3e24
6062 F20101124_AAAPHD cecchini_m_Page_022thm.jpg
6a772cbe805a443c4eb641baa7c7fd71
131bfbe1e5548ead0983cc0b0a8fe2a52732ed39
4107 F20101124_AAAPGP cecchini_m_Page_010thm.jpg
754167049d136394595144558b361084
5c239ad6331185fa7641437290ebcaf5f697e2b1
44049 F20101124_AAAOCZ cecchini_m_Page_136.jpg
4a612b0434c9e0db32c4d71cfcac4645
c80092b1f6f60d0349837f356193ef7240fc1a33
21424 F20101124_AAANXS cecchini_m_Page_022.QC.jpg
fa33856ea1ca014746d01753d911b45f
1970df4f7352fc24cf10412d1d38aefd78dac6a3
52034 F20101124_AAAOEB cecchini_m_Page_167.jpg
152e2c7562334b8e049eed0d667b2e03
542a56bc6876477ed99dbc62578fd7f97b3a2585
32703 F20101124_AAAODN cecchini_m_Page_150.jpg
f21d91f59ab76d0efe23e944ffcc7b0f
1a848bc13bb4e6a41c8bd2c77911ddd4b877dbbf
22609 F20101124_AAAPHE cecchini_m_Page_023.QC.jpg
59e701fa949421956e1e89894378089c
2ffd5d05f527f917d16090f97b5a6fd3fb0a59b9
3888 F20101124_AAAPGQ cecchini_m_Page_011.QC.jpg
0db8a48b8efe5824ade48b96ff5ed162
8ea7ff524595a670d5b5b6abc9486d113220fea2
23376 F20101124_AAANYI cecchini_m_Page_001.jpg
46e9916fb2c41fce55d09d7598b20fad
3df673c7aea459720a33dc4840f826b90ff5f49b
27971 F20101124_AAANXT cecchini_m_Page_083.pro
4e0721ba6a1c6be32f5447946e1507ef
930638e802c596cdb9abb0408e935d9f8ef506d8
51719 F20101124_AAAOEC cecchini_m_Page_169.jpg
92a1ab32977d237c6f5c13fbec7ce49b
7d4cab4b7235e5f6420aa0a96d8177961d1f24b9
49769 F20101124_AAAODO cecchini_m_Page_151.jpg
2ab62a9b0067219900a72dbea3b4d9fc
d0fa002c1618bae5f3e09893614d6bb48becd810
23440 F20101124_AAAPHF cecchini_m_Page_024.QC.jpg
be2184d43c7c096f6a1918877c8acd58
fefb877c9af0f1bafa46c6eb9f88fee5c3b13d1e
1615 F20101124_AAAPGR cecchini_m_Page_011thm.jpg
e9722ed357a56c363b88d4a215e260de
2b3b5c193f216997e5c60f7aba4936315cf5d1c8
10104 F20101124_AAANYJ cecchini_m_Page_002.jpg
f18e2232f6132071e490ee807e2bac5a
8b24a8328512148493367f103230a015fe0dfe81
46770 F20101124_AAANXU cecchini_m_Page_054.jp2
b0ff2a3a98865ad3b3042a7a50bcdd63
bab07da5d9ad3aefdc74234fbacde4be8ec41f0b
51389 F20101124_AAAOED cecchini_m_Page_170.jpg
45587acb84f27c48fb78dd70db1b84f1
e636f710deef80a10a9ddb5fe157d92e485ac4a9
52472 F20101124_AAAODP cecchini_m_Page_152.jpg
33c01d3675e299e1912047cf4b67aaf3
57c60403a9888feadcc4265d7bc67b778a48ada8
19274 F20101124_AAAPHG cecchini_m_Page_025.QC.jpg
78c377c841e2168833b82f59814fc7f1
5e1f680d4b78ce5a55c333758dfed27cc87649a0
20989 F20101124_AAAPGS cecchini_m_Page_013.QC.jpg
5e92b2019c7156de8ba7a1725a015e94
6f1044de14eac4263a11fe63a30ba4888bab6c66
96967 F20101124_AAANXV cecchini_m_Page_086.jp2
ee2c9f5977d73b4f004e2a6857d3a3ac
84641661d408df130d487739c737979f88c5058f
51454 F20101124_AAAOEE cecchini_m_Page_172.jpg
0ff925257649b15a9528f1bc65cf17cd
e0c7ec93dd26cddd3b288bf5ac9068f0b373d83c
51594 F20101124_AAAODQ cecchini_m_Page_153.jpg
183f8b2e0cbfe7bec2503b071bb14a67
1cc5112dd41ebeb6078114ff7e58d1f5f309bbc2
13187 F20101124_AAANYK cecchini_m_Page_003.jpg
d720381c01a900965abf5bd36dc09d81
2b35465ce0c5c6aecb4149500fa0ca8df8649f96
6119 F20101124_AAAPHH cecchini_m_Page_026thm.jpg
c98580346c4df5b9641520ec52f8c9ba
9f92b0e57a1d33ec49f745608cb373a75ddeed09
5847 F20101124_AAAPGT cecchini_m_Page_013thm.jpg
49d98a434623ab455bc5ad47a74898a1
f98ce51daaa47d9f6c5a40b030bfb539c13b1481
20835 F20101124_AAANXW cecchini_m_Page_129.pro
ef687564eb25aaf73950d058622b7ac5
56d3fbab9451375c409e2324be2a850164ffcc3e
29380 F20101124_AAAOEF cecchini_m_Page_173.jpg
6d93c0501c5895dbfc6cae138679ac9b
4e1d963183d4b1d00a68b3e65bcb9485c874ed64
51616 F20101124_AAAODR cecchini_m_Page_154.jpg
54e9949526061d023b9e06ebb05db709
13873de61212ec41a80f0ac085515e064e2fd603
33112 F20101124_AAANYL cecchini_m_Page_004.jpg
640f3a7136d803dbdcd3bb74e4d1a6d3
2f9e97c18b6018ae902bc6e63e5b3ae64d098d85
12632 F20101124_AAAPHI cecchini_m_Page_027.QC.jpg
384ca1d78e2a3c6476f39bd0af36c2f8
6725329b1d6ccf01ff9ecdad2c1cc98c04448ef0
6315 F20101124_AAAPGU cecchini_m_Page_014thm.jpg
6782206b3834b605899a00d4ae30648e
9762b957a1a141a95609818b3c6222f2e438c0eb
5710 F20101124_AAANXX cecchini_m_Page_100thm.jpg
050c6080959644ade1a6639a4c8dc6fe
b3b898e3f6b337fbb46f0864b818646a6f58ecfb
10062 F20101124_AAAOEG cecchini_m_Page_174.jpg
87a3e4b672ef811f36d196d210dcad40
9f2528ada165d5acb1f681874663b1d0ece0f141
66332 F20101124_AAANZA cecchini_m_Page_022.jpg
7d66e5bf221846be8195759d087de8da
893c2634dc0d04019f9e98a410a0ede2bd3a9b7f
51605 F20101124_AAAODS cecchini_m_Page_156.jpg
fe3847899722bcc35d43d336866faaff
037fd86cb1c4a571da86c815ef64d4bc2cabfc2e
75578 F20101124_AAANYM cecchini_m_Page_008.jpg
826e1f9f2afb677de9a6935addd287dc
5fe611bb6fc1ddb815638b8eb635d94ac445bdb3
3603 F20101124_AAAPHJ cecchini_m_Page_027thm.jpg
dac006b09be8d8a2939c5495a23ca6f4
e5d2ed7cb7d95ea2f01dd4c40a1d46bfc21d5683
12755 F20101124_AAAPGV cecchini_m_Page_016.QC.jpg
39232d3c505d6dda62ee25f70e21b674
24489de6da7bf15ca28eee4f9e208cbe287a4bf2
937551 F20101124_AAANXY cecchini_m_Page_035.jp2
1077d6b22ee4b8e34196cd31c006f4b5
ddaa02488fc1b907cc0121a4d8e2188c798a1a48
91809 F20101124_AAAOEH cecchini_m_Page_176.jpg
366219d11f8a87fc603377a41aa8a869
f17df74d1f6d2da1426e486d5b8b78d9233c918f
69194 F20101124_AAANZB cecchini_m_Page_023.jpg
ebd96bc43e0f863e9e213d36c6ed3449
bb63e16848d024242c0bb91331f30370a935a8ec
51771 F20101124_AAAODT cecchini_m_Page_157.jpg
e0f5dc112a0fe8468ec405bb606daede
81e80cfacbfb9fa7cc5b1bb56ac83c4a865a72b5
85081 F20101124_AAANYN cecchini_m_Page_009.jpg
358a8eb23998e43b132c7b8fa6aca79c
fc1797cd3bde437913a6378901ee4931d4174b26
5717 F20101124_AAAPHK cecchini_m_Page_028thm.jpg
840a6bf6a56f06af1efb7b74d354e928
e5726362ae5d70e673eeab166282e9f35820fc66
F20101124_AAAPGW cecchini_m_Page_016thm.jpg
5a2c5544ecf8980c963f1281d44e9b0a
1b75a6e0de8e216308c2cc3c9bd2b76f476e1235
88572 F20101124_AAANXZ cecchini_m_Page_058.jp2
7a5d59f01fa267e7508a53ff9f3eae1f
952781a0b239449a8a19e9a3fb9c026edb425cd6
99513 F20101124_AAAOEI cecchini_m_Page_177.jpg
c29fe0fd4b8bafa09d460da69618bafc
01bb4548a6d6188c1b436d37cbaa248084c06e4f
58752 F20101124_AAANZC cecchini_m_Page_025.jpg
8743c7e338fee0f5569e2b8fcbdb3a3f
8be4f77e7bc49ea5c77b50b0ba93a6cd12dad5cd
52979 F20101124_AAAODU cecchini_m_Page_158.jpg
7b895ec7cefd3464aa00303dde6ab34d
5d59da6d422192fe26277bfeecb5db051f7fc56b
52622 F20101124_AAANYO cecchini_m_Page_010.jpg
62e0df02e2469edce2a97105ab5ad24e
91d9610cdfd5fb8a6daf21d9e46d1b8511a0bab2
22225 F20101124_AAAPHL cecchini_m_Page_029.QC.jpg
17412e43772f3e98cbece31d79b7b037
034cede770b1be761f985e44de296f49460997ac
19784 F20101124_AAAPGX cecchini_m_Page_017.QC.jpg
a83de8f8f05cc183ef854685a421b342
e520bae12d4603b9bcdd86db627ec2a25741ca6b
93259 F20101124_AAAOEJ cecchini_m_Page_178.jpg
c5449d4314893c2f446e69bbfec1a975
0f1594dcd31fed490e36e11ac6870481aae479e9
67331 F20101124_AAANZD cecchini_m_Page_026.jpg
621bb757855eb56617922d3ade0fd45a
8e475e8b8f49ba4bed5b90617c36b91c9c3281df
53245 F20101124_AAAODV cecchini_m_Page_160.jpg
4c5869c6f3215449f5fb81fda00bbd85
2b1d9d6b2c69ac2d53d6b3c14857b86d503535c4
12189 F20101124_AAANYP cecchini_m_Page_011.jpg
6f07d3f451390c8f040d1f89693b86ce
269b46a9fcdbd6a3bf65329eed945fd326dec36b
20933 F20101124_AAAPIA cecchini_m_Page_042.QC.jpg
864ca545a8d5be40769891bc567ad15a
99d627762191db884e8c22713bcd3c6364a0dafa
6067 F20101124_AAAPHM cecchini_m_Page_030thm.jpg
b47d6d7d5541e0da9825c17073e71e39
569d0c3ae81ac1ebeaefb3b34cb15e52ddeaf32c
5899 F20101124_AAAPGY cecchini_m_Page_017thm.jpg
925858479f756420857d7b059fc48ca3
c6c5cd2d5fe0f77e02341d063c2268af84c74a87
95062 F20101124_AAAOEK cecchini_m_Page_179.jpg
927702171182052360a1215bb8874577
f0e96a96de1855e01e7382cc8a4c224dec097f86
38223 F20101124_AAANZE cecchini_m_Page_027.jpg
4e56ca6aff9eeb242660a7bfa165828a
91be3ffb36b9139f2a1e619ecf053474d765b412
51208 F20101124_AAAODW cecchini_m_Page_161.jpg
51ecb90009981610d0acad935b1b2235
2479b122a36191ed74c354807c939a8cfe5f2b9a
63978 F20101124_AAANYQ cecchini_m_Page_012.jpg
eb0e9ddc49c141fbb6c779d948851db1
5f325fc50808639a1c098a00599653483f709a76
6436 F20101124_AAAPIB cecchini_m_Page_042thm.jpg
7944866f07228ff3aea7fc747a308810
6ad7e8c769054bffe6b4bb75a898f443b42aab84
19363 F20101124_AAAPHN cecchini_m_Page_031.QC.jpg
60e766debd9ca97d292e4b6a116db615
703a1c8a585c7628c40c9f93eae8099c3a70946c
23804 F20101124_AAAPGZ cecchini_m_Page_018.QC.jpg
4bf385b3184445b1f4ef57a700a27fc5
b0430e2f44232a35f8b6c4fa149e89d493590baa
101654 F20101124_AAAOEL cecchini_m_Page_180.jpg
ddee7eba9d99232b74c8de455cc94695
0d33b4414d94f51df3c05c55a2c71922f8693e9d
62618 F20101124_AAANZF cecchini_m_Page_028.jpg
eb866b149d62c8cf45249aa4e73b3266
31bc30526502aa6f00ebd1db42c7600ffb6becd9
52791 F20101124_AAAODX cecchini_m_Page_162.jpg
00a95d7e268ece75f41a120ca8885d4d
ded050daea12c964ad82601b992d8564a85e7070
18337 F20101124_AAAPIC cecchini_m_Page_043.QC.jpg
e1ecfcd739fe6b952afcce50bd1e7a1a
131557ecb61cbda4de74db90bead238ac74ce496
23599 F20101124_AAAPHO cecchini_m_Page_032.QC.jpg
3dd1337b9d86035f6fb3c33988e7664e
cfe2e82d249f867c9bf78b9922eab52a7a60d727
94462 F20101124_AAAOFA cecchini_m_Page_012.jp2
6110a3828f95c6e6e45e8dc707c64cb8
c8a7a59fb957337bfef5485df4e46d9b4782cb7c
82418 F20101124_AAAOEM cecchini_m_Page_181.jpg
fbcf8651fd20c797b356dc02ae1ddec9
4fb710ce3b0c7e221bee7a2dee9220300bb6f783
68190 F20101124_AAANZG cecchini_m_Page_029.jpg
ef2e3a714dc6b418e01a362577932ece
3eb684ba3b0618fb87cb234d9455c29f8d17c150
52308 F20101124_AAAODY cecchini_m_Page_163.jpg
84eff63caaae5307184bcfb2495d3370
9bd7b1bda4e3daa35681fc65babb68197e62df89
64080 F20101124_AAANYR cecchini_m_Page_013.jpg
344cc1ec5f57dcc9401496d2091b8db1
ae00e2a2f2209310aea41e2d13d2f3b81c9194d2
5544 F20101124_AAAPID cecchini_m_Page_043thm.jpg
435bfc770d090ccd1145856092163ee2
1995a0e7f25fa16dff22d1a0c88c83763c8231aa
6425 F20101124_AAAPHP cecchini_m_Page_032thm.jpg
4c480862e7441993e1a55d21c350b19f
f3be3ef77b5c7e684b5d51e400c00ca9d856f249
97138 F20101124_AAAOFB cecchini_m_Page_013.jp2
c7d2303e60e673b8dcc69a81d978115a
657973d91806f87b9e6387e7cf709b80c41b3d76
100246 F20101124_AAAOEN cecchini_m_Page_182.jpg
383ccaea85756a096d4de786f3b56d73
37bd428b6fc017ad8d5cd1ecbb267b86950fa17f
61405 F20101124_AAANZH cecchini_m_Page_030.jpg
fcd534cee7f0e9cc522ecb98ab051111
6c0dcf4aaa2e068fd0fd37ce19b1b6b4c4534d4b
52057 F20101124_AAAODZ cecchini_m_Page_165.jpg
876fcb72d8e4edb0583788ac5fef0272
26b1f1195366d61eb4792ef9f05de330885b13b0
70768 F20101124_AAANYS cecchini_m_Page_014.jpg
e0ba6628a9c82a8ad9847d765c110feb
efe10fa1d9bd5205562ae866d7d6ef6cb4b55ebf
F20101124_AAAPIE cecchini_m_Page_044thm.jpg
9836c946da6cca076f36d70d3e2ac070
17e06cdeb58e272259adc6c9801eeab3ec4430cd
19125 F20101124_AAAPHQ cecchini_m_Page_034.QC.jpg
885958c57b8c7eedf33ee039868b25bc
6347a96b37b84ffb460ce304535d7959b472346b
107683 F20101124_AAAOFC cecchini_m_Page_014.jp2
ff22b66659f8a653134d052557d48706
17215113b09a54708fdf325127605002942302b8
18776 F20101124_AAAOEO cecchini_m_Page_183.jpg
f68eb7c0587d8087fbd18960ce2a8a4a
1afe43fbfb7e83846476bff6ef0e9d76f7e50981
57436 F20101124_AAANZI cecchini_m_Page_031.jpg
6f6b3fe819357d1402b0ea87a2f81f54
b687ec067ec7853eda1ebd16a98b4cac6395c4d3
64836 F20101124_AAANYT cecchini_m_Page_015.jpg
98a49bf4b75f5be41be7de86fbbafdfc
c3f8b425fd424c1c50f404c0035ef9778835e489
20551 F20101124_AAAPIF cecchini_m_Page_045.QC.jpg
87c71f9498375fd286952cb78e36b4cb
db184174d61cef5d6c5807a879c1479b093f9496
5842 F20101124_AAAPHR cecchini_m_Page_034thm.jpg
516f53f0e98265a09db08d559aa2cdfa
6c29612fba5c5abc52625c9d2157f419fc3b0d39
53751 F20101124_AAAOFD cecchini_m_Page_016.jp2
0a8a1022b79d19f9aef1987f61bdbff9
4972f2a879cbad69cf93379851d693bfa74dc106
38722 F20101124_AAAOEP cecchini_m_Page_184.jpg
2947373df983ed1a11115e389c284838
1b2f9bad83625e3df4fbfe714ce8d3195dfb3a61
71219 F20101124_AAANZJ cecchini_m_Page_032.jpg
bf3320673ef46b3c37ecaf2bdd8d23a2
b069e5f1b4ec0baf1504e564bbbc298a4e732212
38876 F20101124_AAANYU cecchini_m_Page_016.jpg
2b3cea3b7ea472c903873ee6f5e8fdb6
492c3446c301b380697644a1e58bdd4725930cef
6064 F20101124_AAAPIG cecchini_m_Page_045thm.jpg
eef47707af5940ffbdbe039f5ab4caca
9207c2108b4dbf0689f7ff7b9da3d0ed70478962
6359 F20101124_AAAPHS cecchini_m_Page_035thm.jpg
74088c0297bb5037eb68ca733de28b70
2b7f61c9be04125016d05a77fd9c344a51224474
92275 F20101124_AAAOFE cecchini_m_Page_017.jp2
f2a3cd5d146eb6c518f9adc217969010
7088fd697ad2170421fe9c9f2c14712c15bf51ce
24387 F20101124_AAAOEQ cecchini_m_Page_001.jp2
c7bf544ce28b63611b2dc0fee16ee41c
fb6e956842e2756a9eb830bec39b284ecc071627
71443 F20101124_AAANZK cecchini_m_Page_033.jpg
2450ba0d774ca602d2ff711d699ce876
508748bd99736e7212a3290d84ff3d5ed66dc512
61644 F20101124_AAANYV cecchini_m_Page_017.jpg
9c95a499320b103b1a195014ffa7c4be
33aeac7703dd7b4ecbe928b6c1171182310bb127
18992 F20101124_AAAPIH cecchini_m_Page_046.QC.jpg
d3cf59d972d3391b73fec7dc63f97744
1194b18daf2bd91e13bc04b9e940bed0e7562b83
6404 F20101124_AAAPHT cecchini_m_Page_036thm.jpg
11234208c199ec4832ed527443194608
911a2893299e8d14ebabe2c87d74346664e30906
111531 F20101124_AAAOFF cecchini_m_Page_018.jp2
fa093a1fa06fa1ca2995172ce4996bdd
57dd33db7e4ee376862cc4438802cc285511b1be
9958 F20101124_AAAOER cecchini_m_Page_003.jp2
9b9c1233bd690e4f05950c5f04e49b35
bb0c65b2812ae1d31d2f16ef0510ce0576a0099c
60603 F20101124_AAANZL cecchini_m_Page_034.jpg
2d127b4b828e9dde117e07d59bd68571
d394db646b3dff23fc4317bdc833f8de38acd468
72605 F20101124_AAANYW cecchini_m_Page_018.jpg
64e6c7b9a5f945d730e02d2ecd7d331e
f52f360fc632249f33ea5884ce64ddbf181bcc4a
5631 F20101124_AAAPII cecchini_m_Page_046thm.jpg
3e9d01361f59d6e8eb111ad99b045659
dcc24140ad8ab94abc3c4d80356022a614489428
21090 F20101124_AAAPHU cecchini_m_Page_037.QC.jpg
0d5655d64fc6836b614de83eca37de45
30fe02ef9ecc4f0cb9841c6c869427b2cf828bb0
108202 F20101124_AAAOFG cecchini_m_Page_019.jp2
3b95976b32ce8056ec4517397aae88d6
fbae1930258ae9654ad6c902b4fb56f2a95b6074
44014 F20101124_AAAOES cecchini_m_Page_004.jp2
1aa01744779066534a966bf28fbba813
a6aa5b2d12d40b7210fa1dd8023922b9a1707ef5
68034 F20101124_AAANZM cecchini_m_Page_036.jpg
35ed8ad535f7ce608af79dd1c94028f8
c83e37d997ee6d553f76870f86f33e64442f2b76
71289 F20101124_AAANYX cecchini_m_Page_019.jpg
1ce2da28bce95bf9016f6c86d91c8aad
15969d84e723144c14875bb954e40d6097d6f55c
20593 F20101124_AAAPIJ cecchini_m_Page_047.QC.jpg
0e072029c4041d10e87c88c25d33314a
2e0da4c97b687249cfff030541ebce4f6676f765
6011 F20101124_AAAPHV cecchini_m_Page_037thm.jpg
4cb30b5153630423ccc2c293e5ad290b
e4e3576aa3af48c69cb898daad225788ecc60428
F20101124_AAAOET cecchini_m_Page_005.jp2
8ed0261ad10db51bb13d5762c857659a
96192987e47d36c5b13a6f1ca2784c507cc90b71
65216 F20101124_AAANZN cecchini_m_Page_037.jpg
65edd1dca92007904a15252dd64d7ce9
31fc431db55b054defd578f6da18eab53321d325
70008 F20101124_AAANYY cecchini_m_Page_020.jpg
87375c77f8fcb41a5c5e83c43b05b4a8
bb85a1584a69e4d4e445e4f49ba62addaf896dde
106890 F20101124_AAAOFH cecchini_m_Page_020.jp2
1ab8b0946cf34f0e56cad12ce610857a
9082bc8bd98d6ff5132878ecb233cb74f3324133
4543 F20101124_AAAPIK cecchini_m_Page_049thm.jpg
e3168ccbd2161962a4d62c5f7046858f
7bee4eae6c257d33ede12b7390580f7852bb9537
20760 F20101124_AAAPHW cecchini_m_Page_039.QC.jpg
998b20904a43ef09d2c321a141589c22
9a9c6a80ba11da46a2c4b83349132b37d1e08b38
F20101124_AAAOEU cecchini_m_Page_006.jp2
38dd295e7a7fadd452b46caf31d2d9dd
ad2c1a6a20a69fd0d542824e302214ff97e0fd63
65364 F20101124_AAANZO cecchini_m_Page_039.jpg
803896208cc17c8246f33c883c3a954a
fbf3894344d338161efe7bed6ecc51ad02923d1f
66059 F20101124_AAANYZ cecchini_m_Page_021.jpg
62db3d223f7d197a554100692848d4d6
246fcf4cda626130daaca54542ff8a8724e1e7a2
99745 F20101124_AAAOFI cecchini_m_Page_022.jp2
dc98538c78da6ad9153b30c4399a292c
eaa3d9fa685931ad9e20cd69e6c3456d68663aa7
15180 F20101124_AAAPIL cecchini_m_Page_050.QC.jpg
e4103f0b163f71c901da724bbad59662
62f334fc293945ffc568b6f2e1d64ee2a900ec80
20588 F20101124_AAAPHX cecchini_m_Page_040.QC.jpg
00da579d41bf95a56f7f9a612ba66132
9499d3552e41948076ab1a32a9695d617ba5ed95
744296 F20101124_AAAOEV cecchini_m_Page_007.jp2
149f71122bac095fcf20584fde752f00
b7f9de9f10f9fd643dafbb6ec5a0c43bdde156e2
63473 F20101124_AAANZP cecchini_m_Page_040.jpg
ff23f3bfbe8c22fa497906a8a084a8ca
57bc64ad4bcb2b9efcdfb804ab884de391b9a8d0
103956 F20101124_AAAOFJ cecchini_m_Page_023.jp2
71b02acbfdd4939800b38e1ca60a5b77
2466a666ed7ccda555d6cbd40078bbc0a6cad9ed
15720 F20101124_AAAPJA cecchini_m_Page_061.QC.jpg
ebbf57bcd6c6a55a0cc1aaa2e73acefd
ebf2ae843589c71bd1950aa7cfdb6e56e0a0a508
4840 F20101124_AAAPIM cecchini_m_Page_050thm.jpg
e172b93ae0ac9eff5111d20038465fe2
1c9908eb80a0f9e52b3de79ca8d07e35abbe1df1
6105 F20101124_AAAPHY cecchini_m_Page_040thm.jpg
1be6ee92af60979706a3eea91b4d6e01
9b727fea336815a8cd0387424199fb22b01c7b29
1051979 F20101124_AAAOEW cecchini_m_Page_008.jp2
f8d52b26e892639462e89aba04f224d0
791ac090f5dc9c7958c84e1207f6c8ff8825c281
57088 F20101124_AAANZQ cecchini_m_Page_041.jpg
0a19ac949a30fdcf7cc540635280fc99
75914dfc3057fb282fdef5e96e1432467a8309f0
109251 F20101124_AAAOFK cecchini_m_Page_024.jp2
98175b4098a9be2f2e27b90a90d4ad41
d91dd6092cd8a83d182c1ecd1ca1e97fe0949b12
5116 F20101124_AAAPJB cecchini_m_Page_061thm.jpg
9d69d7480a66d4d01722684a19800d98
8b0d272ad052f3436ab414cdcb086cc0518cbb2e
16583 F20101124_AAAPIN cecchini_m_Page_051.QC.jpg
c4d64a813e4f49fb72417bff77208a19
505587e35fda4b2370d3837df531d73a89d919cd
F20101124_AAAPHZ cecchini_m_Page_041thm.jpg
4a3d3788fb88654db9901c7572900113
235b17a0de66db9c6803b7ae06e259853a27adbc
1051975 F20101124_AAAOEX cecchini_m_Page_009.jp2
427fb2916d84b79c1664a74c218d69a9
39879a15046cdfbb44962f48cfcbd03141bfa98f
67328 F20101124_AAANZR cecchini_m_Page_042.jpg
b2dde341ff528633e95cebba6ee9d58f
55afac09b47310686c6f2621d714247a22ef0f0d
88790 F20101124_AAAOFL cecchini_m_Page_025.jp2
83b32b18dddd76450e9f5552d509265e
1951c95c42cbef4aef98d55049dd77a1cc7e2fb9
14905 F20101124_AAAPJC cecchini_m_Page_062.QC.jpg
09b2ad02e0eb9e86154699738faa0e65
f2ad7fcfc4d7d9d2aa031a1221718c84b29fe39e
16551 F20101124_AAAPIO cecchini_m_Page_052.QC.jpg
bf765c199965c632c4753911cf16b7e5
0b12f182ffb9e09ddb08a13a7132c25e25df3567
F20101124_AAAOEY cecchini_m_Page_010.jp2
b7297b7098bae1d5f0a6d824c9381305
ab1dd0d7f1f1a1346b14a0ccb545a4c31ee76fd0
85479 F20101124_AAAOGA cecchini_m_Page_043.jp2
eb45b126fe3113a766c6f010a027812b
53316708c6113b09744d9d8e1042e5f52f518394
53496 F20101124_AAAOFM cecchini_m_Page_027.jp2
617b5e616ed98c2503a2862683ac5707
4c92df4e1f03e46b55df056c3f2b839894e88148
5964 F20101124_AAAPJD cecchini_m_Page_063thm.jpg
e8470b32743847a6cdf96780bbbf965a
612ac5e4ab01ed6a510fd3008b2599cc07f3e40a
4862 F20101124_AAAPIP cecchini_m_Page_052thm.jpg
67aba80357bb797a04da1fdcc8f4060a
cb0636004d1059f747506a9589e111eb8bc9eb5b
94541 F20101124_AAAOEZ cecchini_m_Page_011.jp2
3ba8c5644ec9b0643d13acf824d08426
1bc32bd40a39be816024842fabf672dcb935bd2c
55534 F20101124_AAANZS cecchini_m_Page_043.jpg
7be6bb2de646e3cbbe83a1f5cf9f78a4
64c288fe571f5cbf7007347e85d4a540dab996c4
89361 F20101124_AAAOGB cecchini_m_Page_044.jp2
9e884c541e76df4e9917ccebecd6dcce
c36c382433940157a377ba5dc9edc33781cc946d
93402 F20101124_AAAOFN cecchini_m_Page_028.jp2
3e2ad99ca1fb63c954174a5b144688e0
48879da392bcd99a98a874f668133cc15879af6b
8344 F20101124_AAAPJE cecchini_m_Page_065.QC.jpg
08e7c1e7501c6c211469fd9b147b9c44
d7e77b9f41ad1b06dcf570cf89734f4e43b9d5a7
10882 F20101124_AAAPIQ cecchini_m_Page_054.QC.jpg
3f7baa85bb62742d605be7d910d9cbfe
6ecb207eae4d42cd1904eebaf7fd504953a9a19a
59660 F20101124_AAANZT cecchini_m_Page_044.jpg
0125509bc9d7df40696d58724425ad6a
ebddbac7e7e309ae5d96e7bfee897613957d92a9
95328 F20101124_AAAOGC cecchini_m_Page_045.jp2
10fecbf4ab2bfd1d67f30592eec7d78e
bec476764577537518ab2670d92b722ab2de63bb
104771 F20101124_AAAOFO cecchini_m_Page_029.jp2
5ff92f508f8bb39721c0d4863686d169
a9644badd7d43de07a489e2fa7400a87cb2ce66e
2906 F20101124_AAAPJF cecchini_m_Page_065thm.jpg
51f73a356487f737ef7ea0928f871e84
29e26e62030163d7f115bd00a5ce459c96fbe87c
3447 F20101124_AAAPIR cecchini_m_Page_054thm.jpg
1243669759bcf6d701fb6ee7f81513fc
5e9098df2a99b40c580028f44ab12c3a18760e07
62850 F20101124_AAANZU cecchini_m_Page_045.jpg
669f9b88815f288820906fddf758714b
0af24e0015d39b5a59b6da530b71fe11b2976e4e
91144 F20101124_AAAOGD cecchini_m_Page_047.jp2
364ee373eef0892cf37afd79742359f8
29b07e32c7f9d193fe7686025584ade62ccd8479
93974 F20101124_AAAOFP cecchini_m_Page_030.jp2
934b3bca7de021c7df3b4a6b607fda19
5d41a17f67ee8c78b639a52252a417596116984a
5846 F20101124_AAAPJG cecchini_m_Page_066thm.jpg
fd8280decefe0a1d0f4f5e7ea1d09874
14c3ce32ba6ab8755cb1a4d9a177b1edd9f2ce8b
21717 F20101124_AAAPIS cecchini_m_Page_055.QC.jpg
aa7bfd586b744f382b809c7721d29052
cfeff73f18480adb47b4bdf2751806db45e52a71
57093 F20101124_AAANZV cecchini_m_Page_046.jpg
9dd33d218ffea36309de2f601735e6a1
3c3e34f2aeeb9b04c6b03d1b49790fbe5114071c
91001 F20101124_AAAOGE cecchini_m_Page_048.jp2
b09f0e4b6f0431d694dcc5978598468c
94626b3a82b21bd239e884e201303edbac43350d
86859 F20101124_AAAOFQ cecchini_m_Page_031.jp2
832d691fe97a7a1c163619430e0c3e80
5f82d2324006a735082158f39174bedc0cfb941a
4549 F20101124_AAAPJH cecchini_m_Page_067thm.jpg
a3fc784483fe6e00c645ae86b3186ca8
3e06ea3ee9bebd973dda67ea795674a5c7cfbe40
6058 F20101124_AAAPIT cecchini_m_Page_055thm.jpg
dbabcbf5e3c954e0e086ae679bf9fbda
3387ca3f46d45b8e1eeb79b8de2be84eb932f881
63516 F20101124_AAANZW cecchini_m_Page_047.jpg
fca86208916b60da5f0dfb8e46c20b18
2a5782f488c8f88e972c6a567e7a7e9cb793188c
70153 F20101124_AAAOGF cecchini_m_Page_050.jp2
f340742d0a54a7d013726a011a8a16a8
dd05226f98779e95f03acc505564ca3956f87a25
109690 F20101124_AAAOFR cecchini_m_Page_032.jp2
79f3603bac5074b8e0671faa4d4ff3cc
d7b45acfc8c0444147439a438a7a79a14bff0ef7
8865 F20101124_AAAPJI cecchini_m_Page_068.QC.jpg
db9eeb468a5ddd97543b4d0d6ed894e5
d781417b5dab995c9cda804dbb8f2eae31d4cb97
4734 F20101124_AAAPIU cecchini_m_Page_056thm.jpg
a086cb463599e8e78356e3b0746a18e4
47fbbdf56079d68067064a225febf16ef7909cc0
61291 F20101124_AAANZX cecchini_m_Page_048.jpg
3a0d12e9c3cb867cf18aaf4f4b8283ce
1ba2b27de317f5b1aebcfe68c9e4a04fd721c56e
72984 F20101124_AAAOGG cecchini_m_Page_051.jp2
18db651767ec6f554808d47fdd7b58f8
98a734a4761723b8acf2dc67f5c505511d39c01d
80222 F20101124_AAAOFS cecchini_m_Page_034.jp2
3663a05937abfdceea1b7bc6ea7806db
9f3a8317bb921a5d65e086b70791d7c1f3a9b11b
2952 F20101124_AAAPJJ cecchini_m_Page_068thm.jpg
0520bc154098f46d01a6855110acc6bf
6282a48976583d2218a723095dab9146b7007028
18647 F20101124_AAAPIV cecchini_m_Page_058.QC.jpg
eaecaae40e82820a135268e2c21fd974
a5025d22529ae0b3b8302e76c5085f2ef96e9015
48004 F20101124_AAANZY cecchini_m_Page_049.jpg
ffb3083d7ff86ff14d71c588b4284192
d43d9551b1e2e440bed0f6ee26d7fa4da00d6f12
73074 F20101124_AAAOGH cecchini_m_Page_052.jp2
9d5110a7ef9f0486e925c40c0caeaaae
8014843679817911f4e1dfa64ac8539dea622c9e
103145 F20101124_AAAOFT cecchini_m_Page_036.jp2
cdca5c53b3cb9c88625ea3c1f7ce6b55
88e029fe2e93b65fbf95a590a6ff6460a4f1fa72
12445 F20101124_AAAPJK cecchini_m_Page_069.QC.jpg
f51447edbfbdf805275a88ee61fb5c5f
5f926998382b4ef97b11526075ff02ce5d0f5d43
16853 F20101124_AAAPIW cecchini_m_Page_059.QC.jpg
35728cbc8afe169347286ef19679cb42
2de08084f966dbf08d5b68ab3d5ebd03d1d85647
48066 F20101124_AAANZZ cecchini_m_Page_050.jpg
5c1f88ca2ccb06c18e8e810c3ce1d89d
13affa391ce449202f4930aee609e1fe56936d6f
109337 F20101124_AAAOGI cecchini_m_Page_053.jp2
3002431856ce06ab55f9d9b077f66057
8e60e3cf7f5c951ff41a8d174e743d1220240846
97748 F20101124_AAAOFU cecchini_m_Page_037.jp2
46dc0d1dafc30f678ee49dfc0f4784b3
962e9c7a38bb9530c35326e13ace9d9984de2e20
4929 F20101124_AAAPKA cecchini_m_Page_081thm.jpg
792c5e70f717ccbd9d629abb215f9399
17b180a4780a817062f823915f210787234c9386
3878 F20101124_AAAPJL cecchini_m_Page_069thm.jpg
cac49ed0e196af65e141c3e6c8c33fa1
0c81bb6cf5f55a86a0e02418797c805afb4b6edf
4870 F20101124_AAAPIX cecchini_m_Page_059thm.jpg
8ddc04fcccd6a3b6b04b584dc3d636d2
e4d059259630d4e27c05f5346118766b8eeb35d7
99110 F20101124_AAAOGJ cecchini_m_Page_055.jp2
a0d97e89366f75bb08d89109065da38c
170ead22cafa86fa4a80e5c67887e30149224f0d
52373 F20101124_AAAOFV cecchini_m_Page_038.jp2
846dd6dc68d53fc3c4560fd14a6d7d4e
9ca582f8349a4dcb97c3fed5d817b2b07efb6132
6144 F20101124_AAAPJM cecchini_m_Page_071thm.jpg
ee7176999ecd5eaf707b3cfd84cefb71
e1714420ca15602ded6e10c41cdf70fcc08d7cd8
5784 F20101124_AAAPIY cecchini_m_Page_060.QC.jpg
771fa04ea2a2f66c656ce0ad57d5f752
c84018b95652fa6fb4e91a47ab069c5989b631fa
71521 F20101124_AAAOGK cecchini_m_Page_059.jp2
ba1d238efb6bc1d38a5f32864b708c1e
e0742ce4cc760190527d325812742b0d3e3b27c2
96462 F20101124_AAAOFW cecchini_m_Page_039.jp2
896ffd4a2b95f4c46f298f8fccea9956
2cda7a649dd1034eae99ecf04532199f24bdff12
13465 F20101124_AAAPKB cecchini_m_Page_082.QC.jpg
0b5aa25252e425a9425791f5c05c332e
e140e11620b450cf48197993d6e093a99402e399
21200 F20101124_AAAPJN cecchini_m_Page_072.QC.jpg
38b334c83c9328d712b1924e9a6c9d57
dc65f57e1c0f868e99881ef13f9209aa807fbd06
2163 F20101124_AAAPIZ cecchini_m_Page_060thm.jpg
559d8f1a2d75d74f195a9d9fd34d9c2d
ecf50f13fc075a13ca99e6d8a79ec35f746bfb8f
26132 F20101124_AAAOGL cecchini_m_Page_060.jp2
034926739b8cede1ed1008c062e8b649
e57d189b732b68e27f8c2e0f417e30ad2f097463
93901 F20101124_AAAOFX cecchini_m_Page_040.jp2
9bac595883ca57a5d09fbccc01430fb1
4c3e84a5dc0937786d3dd845531d36d716b2e0c8
4425 F20101124_AAAPKC cecchini_m_Page_082thm.jpg
6d6302c3d065c2a1c943a62712e3df44
0841e0604c07762f6cc4d131e2b2edc84630bde4
5823 F20101124_AAAPJO cecchini_m_Page_073thm.jpg
2903383a9549c6873a14f67a12c5cc67
2564f0b31160142724e3a0bd6230da09c48d8319
75906 F20101124_AAAOHA cecchini_m_Page_079.jp2
c355e9e236c3a668402ed4396ca2ca48
017cc3d51d4858a948530f87949ae3d3efa1f04c
92665 F20101124_AAAOGM cecchini_m_Page_063.jp2
904f0d268486830e6433ad5dc4188589
3e0e9e10b8df522fd72511a83faf790b21e019df
83224 F20101124_AAAOFY cecchini_m_Page_041.jp2
f71d8f2155d9ed79982f7e89d98cdf57
560de0dd66b4c66c55bda919ea9d6b346f8000e7
13613 F20101124_AAAPKD cecchini_m_Page_083.QC.jpg
50d1c8d06c807f73fac0b4134d7c4693
60c1478ef0c0b63aa4cc455554e986d4ce0ec810
19238 F20101124_AAAPJP cecchini_m_Page_074.QC.jpg
8a28544731ae46406d8b042c9fcf4fa1
a258741d47fff3c73a05931b2310236917c84331
642988 F20101124_AAAOHB cecchini_m_Page_081.jp2
b2c8a5b87670e616e3d97752cfff1da4
7a73b71cae71cb1315e88b59256fe3824ba0c73a
55449 F20101124_AAAOGN cecchini_m_Page_064.jp2
30d82ce3236d8ce6fffdf66c1b3bf59d
8be9ac3c714ff315582e43392d68bf87ba5c1d46
100593 F20101124_AAAOFZ cecchini_m_Page_042.jp2
33c283b12067aff7918f163470398064
5eecfe94ad1bc73e3b94194eb87c4245bac81a5a
4263 F20101124_AAAPKE cecchini_m_Page_083thm.jpg
7261928c1b38a29bb922d54211e5fdea
fd77235086379982d67a21afbfddf2888b501ce1
5663 F20101124_AAAPJQ cecchini_m_Page_074thm.jpg
098438c3a256a349638cb9e8be029e6d
9333289152a6dce4bc9e1f2f2d530e6daa02ed93
52082 F20101124_AAAOHC cecchini_m_Page_082.jp2
cf5a41179fe5e5d14695cd956aae53d6
1104483ccbfc595f56f67e6ae0a81eb5834f8c5a
362334 F20101124_AAAOGO cecchini_m_Page_065.jp2
db39c52e44e191df42437caec10ea0c2
b3d536fab32d27736c4743df0b2869f35ca9e306
5334 F20101124_AAAPKF cecchini_m_Page_084thm.jpg
08fdd563e79b72e24d47855561f4ac78
bd71376e1629518ba2a17f229428b5dcdaeacc35
16634 F20101124_AAAPJR cecchini_m_Page_075.QC.jpg
c5520d6ee4c33818d40e1c54ed6ec993
c6ddfe6132bbb77a401ec725baeee7499cc48e7a
60371 F20101124_AAAOHD cecchini_m_Page_083.jp2
932e5bf70d45b2b7b6cc950862b9b031
ccd6124b16f0bc8147f22ed36d3189c671b71c9d
68676 F20101124_AAAOGP cecchini_m_Page_067.jp2
bd28fb56e61ba90cffca3f338cec5109
64fe65a213a2744a743a6eb2bc9b73a244146157
18652 F20101124_AAAPKG cecchini_m_Page_085.QC.jpg
cf51cd43d70cfc8ec7a7dd0787688c4b
0b94caec59ca0c32d47449d63a875b9b04b63869
4872 F20101124_AAAPJS cecchini_m_Page_075thm.jpg
0525594581a8d068a230480c79c96dc0
17897ac3c1b75152d338dd57905f2a66da9923bd
84931 F20101124_AAAOHE cecchini_m_Page_084.jp2
57636f2021fd06312c9c039cf69be57c
b40550fc53ab2f5975950abc39b2f0d39acb0df8
278449 F20101124_AAAOGQ cecchini_m_Page_068.jp2
25300f04d658237db7b7f713d3832d6d
e10b257a4d5c3dd64de905639d33c7e4f82db31a
20932 F20101124_AAAPKH cecchini_m_Page_086.QC.jpg
40b7180cba9be2286f17aeb71f6284d1
64bb4184deb8e4b0a3b6cc4d86daacc4fd23ff44
23965 F20101124_AAAPJT cecchini_m_Page_076.QC.jpg
7c3d4bd0b5a8fa00f659fccc26bb26fc
bd9c8ac053d0ba8c5bc15a64d5d05b89fd565a9b
88108 F20101124_AAAOHF cecchini_m_Page_085.jp2
994ff4296dd6a5322f3e5e0e2b58f1f7
f53f1f2e7c225910e240ee5a67d1019d002f765c
54464 F20101124_AAAOGR cecchini_m_Page_069.jp2
c1789aa0745e114425b3b3f5bcc603f8
49d6cc6ee903343b61c2bd4f2554049be9fdf57c
19892 F20101124_AAAPKI cecchini_m_Page_088.QC.jpg
18bdc42df1802ea1ac96f07f663ef3a8
d2a7a1c060e86b3f3897aef3d339517f68cf17d5
6644 F20101124_AAAPJU cecchini_m_Page_076thm.jpg
da09746bdd62d6d17d96e6fc915e683f
0fb348992644dd863754dd1b8a3afc903747c0b3
262345 F20101124_AAAOHG cecchini_m_Page_087.jp2
24c55e37dc4a853d9222e798ce7e57d8
e36aceadd1e4695711a30277ddc543320cafc715
71402 F20101124_AAAOGS cecchini_m_Page_070.jp2
79265cbb447cf6f7a86b8498b0b5f244
17533b820af1f64f268776bdfe761baa092eefbd
5997 F20101124_AAAPKJ cecchini_m_Page_089thm.jpg
55fb17536a73b74cbea8b34738e1da61
204254bde2754b903331af3688a14f17882e288c
15865 F20101124_AAAPJV cecchini_m_Page_077.QC.jpg
9c25f1f7f045df2335c4c42f0500c044
15fc9503a3bd68e5e5964eaf106f3aec596cda98
92611 F20101124_AAAOHH cecchini_m_Page_088.jp2
4488a467a4700900734276a9bc34a4ad
38e8ab512f8d29c5c6c561710199a88ed296f61c
92746 F20101124_AAAOGT cecchini_m_Page_071.jp2
c00dae4cca78030caf7a97e7b0f60502
741c2788e3e59b21d8e514128e549ae902f41184
5683 F20101124_AAAPKK cecchini_m_Page_090thm.jpg
acf1a5ec1ca10af6e421f1e18d126f8a
004ac15fefed29487667abb0ba6c59402787b7a2
5952 F20101124_AAAPJW cecchini_m_Page_078thm.jpg
f8702131d6435e0cb411a0a95ae4524a
f2ab16dd194e536af2e0146daf7d37c7427677f5
121580 F20101124_AAAOHI cecchini_m_Page_089.jp2
a96c05c0abff7dbb533bda423745d34d
7e7e82b4ed76c6445d602cecb24bc365ac11d708
98313 F20101124_AAAOGU cecchini_m_Page_072.jp2
66c939e7d1e978666b1890d104e430d8
aae43ee10aedd0d3c0d23717ca32f314fbce68a8
20219 F20101124_AAAPLA cecchini_m_Page_102.QC.jpg
03795b8c37642b244a9fd691f7e4e15b
89819713ffb10fa5f34de29f772c153e66d81113
23762 F20101124_AAAPKL cecchini_m_Page_091.QC.jpg
2f4e141549cff74633d0e0558209193b
d5d64e6264eac49504568d9dcfb111a25e0afd4b
17281 F20101124_AAAPJX cecchini_m_Page_079.QC.jpg
0a3d7eb70732b40e85dc42413647998c
cd0c730d7f4e16073b0b40d5ba4245db7624a163
99121 F20101124_AAAOHJ cecchini_m_Page_090.jp2
51860a28dede079075825faf4f48f48d
8ab9453db20316e8ddccaa9d54758cf1a20007dc
860080 F20101124_AAAOGV cecchini_m_Page_073.jp2
7fa7a5d7f6018bf3d138cc8266236085
a00959d35d5142cdc35b0911fcdf7422b992534e
5545 F20101124_AAAPLB cecchini_m_Page_102thm.jpg
2c8fb3178ac8171f8383c67076828bbf
f4e719b02897569487a026a3444a14b84b476fa3
5958 F20101124_AAAPKM cecchini_m_Page_092thm.jpg
83b4a5e43e82f42b1a6d0c84720c8134
0c9bec371d9e8aafbfb63928b793686d1af3d5ef
5067 F20101124_AAAPJY cecchini_m_Page_079thm.jpg
01743cea68159fa8d3e511eaa36155eb
684d5f0a8dded08d0d81c0a4011ab923c09b41b3
113372 F20101124_AAAOHK cecchini_m_Page_091.jp2
16444db19cf6ed7e3fadb033523d83c3
81cefc469488d83a3bfa6cbbff01ac76420bc4fb
86387 F20101124_AAAOGW cecchini_m_Page_074.jp2
27800130c0619bec48ce533067b21ca3
20e19df440931413dab7db014e5a0612f6985282
23096 F20101124_AAAPKN cecchini_m_Page_093.QC.jpg
14d9ab67311a53518bae73da952372ac
4cede14f5c3be70fbb9e3b7594058bb88dcd0851
15303 F20101124_AAAPJZ cecchini_m_Page_081.QC.jpg
e5a074662675fe465eca0f687c1b2ffe
4f92064084bb2cd07b7862f9e479cb40f6dd5d16
96756 F20101124_AAAOHL cecchini_m_Page_092.jp2
b3170009f69f95a826ccd91bf2347eaf
19549c7fd74da22f94724d58335f5a2ec9bb8c18
690131 F20101124_AAAOGX cecchini_m_Page_075.jp2
00c687018d7275501b0f75629b3df1d2
2b9a421ea427dd7043ed36559ccf0120a3a1016b
19684 F20101124_AAAPLC cecchini_m_Page_103.QC.jpg
b88eed07045f43ef72e74f6cd1da39db
2e4639f8987d3353ccecbc42853d52b518bd4540
6638 F20101124_AAAPKO cecchini_m_Page_093thm.jpg
0b7cf21f87c8b9a6635cf3486f62dfe2
2d39b1e5f0f84daaa226768fb30236a8138a0ede
106702 F20101124_AAAOHM cecchini_m_Page_093.jp2
eebb5dfb26fa15d0a48c444a22c2a239
b9ea6e305bdf76e8daa8e86f5699e1af4dbc04cc
623016 F20101124_AAAOGY cecchini_m_Page_077.jp2
b7117beadc2f69505d4bb452e470672f
c2a4619359477227b8b96242b4bfa5c7205cc53d
833699 F20101124_AAAOIA cecchini_m_Page_107.jp2
7e175a84f1b086fae2881b36502a6691
4389df5810672ccd9270b016ebdec8b155975767
4869 F20101124_AAAPLD cecchini_m_Page_103thm.jpg
ae2907666323bc27196e125ae9792650
17d124bc7e68405415f32f453e2b6b60322f99ef
21993 F20101124_AAAPKP cecchini_m_Page_095.QC.jpg
3cbebb7c3dad0b54bb34cd0b97f3df58
2e481e0880794d2d2e0d6288fb6bc7883b272b7c
113270 F20101124_AAAOHN cecchini_m_Page_094.jp2
94d294ff71a19b358b23e8976eff4a32
ee2547980dd36591502a086b8e819e984313532f
94644 F20101124_AAAOGZ cecchini_m_Page_078.jp2
ed4a04ee0831fe71e65f786dfe4b3f40
402d9ce1f3c8bfbd905663baf75c9abee279c2b5
70134 F20101124_AAAOIB cecchini_m_Page_108.jp2
d502b9f4501a8956c1a1a87319ee38e0
1edab7e35397cae70d1b39af21c7d4e425f1a751
F20101124_AAAPLE cecchini_m_Page_104.QC.jpg
b0c71ef8c86c8068cb82bf2ef5df3e50
f88db31ba8be30e88033719b8b92c7e97bc99f7f
6209 F20101124_AAAPKQ cecchini_m_Page_095thm.jpg
cc40d8c45e68d14e15936e0edefef8fe
dafb33f32585fb758893f4f829f18f3793fb0b97
970473 F20101124_AAAOHO cecchini_m_Page_095.jp2
b243349a52544c708b975d47c1daf1a3
20fdaa9eb6b40739f79df87091876d9a17791a87
104094 F20101124_AAAOIC cecchini_m_Page_109.jp2
ec9f94056f7afa47169f31809a8dadba
8778bac56a592bad3d1f26c2be5d7a723f82e0b6
6291 F20101124_AAAPLF cecchini_m_Page_104thm.jpg
93d71afc152350790664a266aa7596f7
fc268caa76f576e1e48bde5ce6b055c624b3bda9
20864 F20101124_AAAPKR cecchini_m_Page_096.QC.jpg
c3268d2f8dca84d12bf7b6a1e9cb77b9
8dd3504d8e61b1427173db94ac46566b0e3a265e
92932 F20101124_AAAOHP cecchini_m_Page_096.jp2
f11dc603baba6aad866faf012bb18bfb
1d89e9d652d37eca3467d590c6c4ea5d958e3562
97055 F20101124_AAAOID cecchini_m_Page_110.jp2
fb5cfc5b7621890545794f07a3c78087
fe12a88163b201ca5253219c7c04f72314a06237
22809 F20101124_AAAPLG cecchini_m_Page_105.QC.jpg
9b5bd5c00bc9574cd4088a6a060db088
53f83044bd01606fbca55dbfe705dad80cde2316
5944 F20101124_AAAPKS cecchini_m_Page_096thm.jpg
1c772a2b1cbf604272062cb57a1c8953
7dad677f5266a8b08cd2fba7f5928f0afa22e74d
96975 F20101124_AAAOHQ cecchini_m_Page_097.jp2
f36967b7eb56f93d83b923b7296a7913
eec29c167a7cc9f81014eb4ac143fd490668a807
86156 F20101124_AAAOIE cecchini_m_Page_111.jp2
eaa3a4282cda9b549d234db4f94e0dcd
b253309f474393baafd0c66fb60ed4092b4bf39d
21379 F20101124_AAAPLH cecchini_m_Page_106.QC.jpg
a3f8c08c52cd374eff8de12052a3cfde
9cb2c186f2494d81f72df48edb8603b85636e200
20737 F20101124_AAAPKT cecchini_m_Page_097.QC.jpg
e046a08897051c12d34c39dfcb498d28
054ff8f410e750454b302bf4be0db1f1c9407f10
83291 F20101124_AAAOHR cecchini_m_Page_098.jp2
e146dfe0aea14d0a9b88d19f02d7e492
a81a4751a4f76cc81b62e69d55718005cb35d570
82743 F20101124_AAAOIF cecchini_m_Page_112.jp2
97d18e51ea6986dae833113ea909dfdb
e22cf7f00460c3fc5926bba2680b10d7fa5b29f6
5979 F20101124_AAAPLI cecchini_m_Page_106thm.jpg
d93f7bc58dfcde859338788c1468b481
9a4257f041060db07a41249ba15d1cd4c4c391ec
F20101124_AAAPKU cecchini_m_Page_097thm.jpg
2bcf0e9c8bbaaa435c75f15a2a5c89f3
89316163e684aecb399e09d383162e07efe6a8cf
35036 F20101124_AAAOHS cecchini_m_Page_099.jp2
1d89e7c9fffb401bbb25c0d2637056cf
ee62a60f45e4f68d8ef2cdadaa2a5a9af1f6b37d
83955 F20101124_AAAOIG cecchini_m_Page_113.jp2
d313168529c34223e39ec945a69ae69a
96268bd4724f089d81cd03eb3f5d3e42816d74cf
19583 F20101124_AAAPLJ cecchini_m_Page_107.QC.jpg
af093e963bf45597c401e08e562d67fa
6297aeeb6c188b3bd3dfd3e2e8627f6139f45b54
18545 F20101124_AAAPKV cecchini_m_Page_098.QC.jpg
daf32b677c4d94d511d69ef4814878b1
c2de78c5db06774b34644c94c97592ef8a20147c
85226 F20101124_AAAOHT cecchini_m_Page_100.jp2
c15de3340d0e8e8ec9a18ad5310a08fb
26a8544f3c6777aee6923775754f53ec8d8dd45d
107171 F20101124_AAAOIH cecchini_m_Page_114.jp2
bb6a6de78804f65d540388a5a28d682e
1587aaa4ba42dac89ee76bb9a5e9eae4916a4331
5592 F20101124_AAAPLK cecchini_m_Page_107thm.jpg
7f62442797668f64d2b3419641fedae0
6e614769ffd6962d4f4529c1f286477a0b06ab02
5341 F20101124_AAAPKW cecchini_m_Page_098thm.jpg
644c2a33de4338b417785513fe8271ec
22c07c9df62c548155bd49893b39635b5af9d9b8
79380 F20101124_AAAOHU cecchini_m_Page_101.jp2
77f43943e16bc991c7ae9535bad2ecbe
d6c9666c702478f265adebca2d67a804a44953d8
100259 F20101124_AAAOII cecchini_m_Page_115.jp2
bfb1155bce72b9920d991afec4285b3b
642def68edc7b8c65ff70253f7447db6af1220a4
20599 F20101124_AAAPMA cecchini_m_Page_120.QC.jpg
0e1a300c0cd41fd7f7b585dc04d9db1b
856e538567344ab896d24e2420f3f9a36856f1c2
23352 F20101124_AAAPLL cecchini_m_Page_109.QC.jpg
0d8618abfa6a179225cf3545db0204dd
e2a932c7c158a911d6696d69c22fcfcb2a31a8a6
8927 F20101124_AAAPKX cecchini_m_Page_099.QC.jpg
5878ef23adea6385fc60d5add23090bb
c91aba4b50318fc597aef393e3293511b677a000
78405 F20101124_AAAOHV cecchini_m_Page_102.jp2
c2ac5e65c72014aa6a32fe05ac2b1022
03b5950a4e855776ab6c858954afa4baeb47d73e
83168 F20101124_AAAOIJ cecchini_m_Page_116.jp2
526cf1aa37c25679384041a0a36a36ca
3ebfcb72f9428dba63c761ac1b9aba934960b4b5
F20101124_AAAPMB cecchini_m_Page_120thm.jpg
40e996a640da73fce045ef1d6f38a2e7
be7d88d1c05389c678ec05f0ad37c686fb5cd70a
6590 F20101124_AAAPLM cecchini_m_Page_109thm.jpg
fff2a23d6711fdc8b6127ef6cc3fb07a
8f1816680cc93b35e31b28b2d15d889849ddc100
2780 F20101124_AAAPKY cecchini_m_Page_099thm.jpg
1de9f86a33efc06825eefdc800cfbd87
ce44ca37a6c06475d1a1a76a1c8827ff054659ce
805863 F20101124_AAAOHW cecchini_m_Page_103.jp2
4ea0b43f47e7e0c9e649b91a21758d00
d6bbb533e071a527b6e2a367fb07bb9a68552a95
102782 F20101124_AAAOIK cecchini_m_Page_118.jp2
1d119c5de45fa1f2e7d1a919d2155c0f
f68942f81faa30e01f180a4396f6e63236b2dcde
21388 F20101124_AAAPMC cecchini_m_Page_122.QC.jpg
94960c08e6a830ad0a93aeafec06db30
f53c3c4073e37a96a60d3f23cfb501a4886856f6
20849 F20101124_AAAPLN cecchini_m_Page_110.QC.jpg
7f496d2b96dddb8c53f5d908614a1084
3c171fd603cd94fb39a99892bcad4c7ff9e509e3
17945 F20101124_AAAPKZ cecchini_m_Page_101.QC.jpg
2a6f5d26500a618b0589c14c2e0a6e43
e8699f066a57657a0e9540ceaa38437ff571280a
98705 F20101124_AAAOHX cecchini_m_Page_104.jp2
515fc5adb645df026aca537432359094
574632a4fefcbc380c849f15569caf308011ffb3
9126 F20101124_AAAOJA cecchini_m_Page_138.jp2
a0fc0e08adf05ed78fc37a5aeac3c4ef
f8e15a693d2585ad6dd0e1114c98473eb8200aa5
110193 F20101124_AAAOIL cecchini_m_Page_119.jp2
9a6fef68723b7ec940e66080c31fdaa5
1fdc17cd0f6c146deca40938fb5cbdf860a7bc96
21030 F20101124_AAAPLO cecchini_m_Page_111.QC.jpg
b4a5ae63d07e37cf71070c91502bce37
5ac4e3f188eee3d5ae78fba63c70ce19d7c71d28
105211 F20101124_AAAOHY cecchini_m_Page_105.jp2
ac842baba37a8ec3939f24f449478c69
09f0607ec7086816b6a3f8da4bd9660abda10674
91932 F20101124_AAAOIM cecchini_m_Page_120.jp2
fc7c95853bb92fe5ab802e4ccf632273
afe76837ed44fa2b62fb61ef17ea7fd0b1795561
4952 F20101124_AAAPMD cecchini_m_Page_123thm.jpg
83f5ab94ea210995eed6d8cdc6462ae1
eceaccaa30cc7b7db9a562c9b8a0b70d87759889
5769 F20101124_AAAPLP cecchini_m_Page_111thm.jpg
f8a90fcfae7b0fd0bab3d79fccfba02b
4afdc5f68f7ffade31c2abe6b100aba7cb1872c1
90690 F20101124_AAAOHZ cecchini_m_Page_106.jp2
b4ad2038050292a06b58d1f81fcc1afb
08b3faf839e90e32b142db5512cae971b034677b
4972 F20101124_AAAOJB cecchini_m_Page_141.jp2
c02c6de8ddb80bbcecfbf69493f20292
3c81c55a44d75ba3d492dbc8ac8977cec26ab407
57857 F20101124_AAAOIN cecchini_m_Page_121.jp2
fc56addff4f8b0802178815809baefa7
76c7215c234ebb2be5a1278ba6899962bf7d1dcc
3906 F20101124_AAAPME cecchini_m_Page_125thm.jpg
9016b1053520cef4b44c9551a145185f
eba4ea096620de37a6968f98f774c81ce35666c3
F20101124_AAAPLQ cecchini_m_Page_112thm.jpg
74d3ed6d71d42b28ab318a587377977e
9adfb734364bb1da070a9e886736a0b47d54ef4c
165029 F20101124_AAAOJC cecchini_m_Page_142.jp2
a003aae642f09f99fc5b3be6c3684fc2
25f318f2e1475519188d03cfdfe8d7bb83010f50
115098 F20101124_AAAOIO cecchini_m_Page_122.jp2
79bceb8f81af5ed672b0a321f5bd0f33
5f25f8447f93e857f022ec80c9e3d16474b839ae
13405 F20101124_AAAPMF cecchini_m_Page_126.QC.jpg
c541e3d2abddf33b91fdb6d17d76e1ab
5ef636ad8c220b4f8050bd84661cf66280588457
20707 F20101124_AAAPLR cecchini_m_Page_113.QC.jpg
0eefbe0f208fa2be9d3fefc8ee1519e1
ee22647014b56a9bcd6b066d34bdfbd38662cc0f
164865 F20101124_AAAOJD cecchini_m_Page_143.jp2
b50521a9a71f8098092e858089e6e0b3
427d08de8c54a009766da85547ae47acc7f64e2d
113640 F20101124_AAAOIP cecchini_m_Page_123.jp2
950b411f4d69e621768c554bb5bddd76
472824e42b7d3c677f1663a0cd6169e1fbb0e67b
3685 F20101124_AAAPMG cecchini_m_Page_126thm.jpg
cc50378ab4fcde34c6c11d8fad6a7884
12ce11ba41e56a578df5dc0d3039a29cfc928235
23146 F20101124_AAAPLS cecchini_m_Page_114.QC.jpg
f603164a071d5daf3ea98dbbc1bffa8a
80baa58fa984da1dfed555dea4d7c5bad251bd07
167770 F20101124_AAAOJE cecchini_m_Page_144.jp2
eb002a7c5f4883af1107d210e6d89d55
cd390f6ab3de2ade0e4d027157a18c71aa11295f
67274 F20101124_AAAOIQ cecchini_m_Page_124.jp2
f3fca6c76f5cf65f785d4c846a6821a6
a7de9390e5ac7e38fc86cf2599225ffd17702b50
12688 F20101124_AAAPMH cecchini_m_Page_127.QC.jpg
3176cbbf198c037b56202b6a41600b70
8fbc6a4cb5b7013728b2e89d60654d1e6ceb620b
21375 F20101124_AAAPLT cecchini_m_Page_115.QC.jpg
11594bc758733a84ba7d0f9f096e84a9
c87da9b21264125fff383c6f8a57dea53df5aa8e
88059 F20101124_AAAOJF cecchini_m_Page_145.jp2
1c352cb84bbf08800d28f1feced89891
88a029e397a7c9d55444c77e09ffe0edbbecabae
60044 F20101124_AAAOIR cecchini_m_Page_127.jp2
221eaac5faa7122b0d7f87754d51601d
d1fc1453004ec24932ece154a0292453ae480159
3741 F20101124_AAAPMI cecchini_m_Page_127thm.jpg
1357d8080870b14cead05efd62864368
bd902d3f1b26f8a03d812488a19712cd8413572a
6201 F20101124_AAAPLU cecchini_m_Page_115thm.jpg
5f696e25e67e1e504f2d5adefd82bc6e
7b8a093a5ae369e3ecc659eef82c2fcb826ef565
171783 F20101124_AAAOJG cecchini_m_Page_146.jp2
db71b75f00e6a0c4c0334883fa7e8b2e
7535420ac0040cbb5013cc77f9f8b95a15df63ad
51460 F20101124_AAAOIS cecchini_m_Page_129.jp2
d1bb84bc388ae300df109f04162e1835
357509bacf586aa2659d533de8a127876fb0b15b
12439 F20101124_AAAPMJ cecchini_m_Page_128.QC.jpg
d32d809075e89d7b910ff4a7d7168dad
f7ba3062f1ade78cb13c1166de67169b39a0e67a
5238 F20101124_AAAPLV cecchini_m_Page_116thm.jpg
47812e17149a5b9c74205a7e05a778a7
bddc11bd7cdf7e04572e49f66560e037c9cd01e5
175670 F20101124_AAAOJH cecchini_m_Page_147.jp2
23706375a0189d7c4c0e7115caf64354
d9542d1f3c874841025cd271123d446be88ae60a
75502 F20101124_AAAOIT cecchini_m_Page_130.jp2
379a11e61b329eed29373384f8982d05
c0a688cc42727b18d231bf35cd3c46ae1b67943e
3615 F20101124_AAAPMK cecchini_m_Page_128thm.jpg
0317393afc65e171cfdd6f33e536e4bf
51afaca8cbc42132e74735d8537060bf97b1d333
24091 F20101124_AAAPLW cecchini_m_Page_117.QC.jpg
b511631d4f74c59a31e089e3e3296197
2901d8865bc749a976ab13c7e4e48192f0e6839e
176975 F20101124_AAAOJI cecchini_m_Page_148.jp2
2281786d34784b39bead2449d23fd1f0
4f5b2a80a502226ab52da50b44fca1a6d0f5cb4d
50495 F20101124_AAAOIU cecchini_m_Page_131.jp2
fe9814fa6918ddfc1f24ef1fb4ac8a94
2df5d94ab4e3a0bb565814207a09a5fc58c775d5
3220 F20101124_AAAPNA cecchini_m_Page_141.QC.jpg
4903a611833deb134348180501726902
8998e8daa66fbc2ac91c9b76479a2ff9a9864aa3
11919 F20101124_AAAPML cecchini_m_Page_129.QC.jpg
ce38517fc15e9109ff231b09a087e2e4
7d9a42c45bf22b37957b45b674da6f379bf6d7a9
6581 F20101124_AAAPLX cecchini_m_Page_117thm.jpg
b106d904d1a867b0a9dfd03af1d0a309
97b328c69622165db1e9a6f2f6497d89eb2282e7
178922 F20101124_AAAOJJ cecchini_m_Page_149.jp2
704d009a2d7e877b094ef1897c2d643a
0f6de0984de6baf0c4fdf5e956f62d5cdc93c4c6
30959 F20101124_AAAOIV cecchini_m_Page_132.jp2
6f8be01a8ee68cae59d2bb7c4cb2c390
92756fdf8431af3ba1596f4fa63a164bae2023b4
4084 F20101124_AAAPNB cecchini_m_Page_142thm.jpg
724db3b7b2f761585e090c05bc27e4e1
bfdf26b6230688fc8a6e26382d2535b93d98e627
3513 F20101124_AAAPMM cecchini_m_Page_129thm.jpg
2c309594acd66280af78720fcaa771bf
7de59f6b8466a04251168655bd476a83dc3c0804
22408 F20101124_AAAPLY cecchini_m_Page_118.QC.jpg
6fc48f8ed699265431c98632f7cd060b
32d364d0099c454bbeec84a9e90a456dfc034563
151853 F20101124_AAAOJK cecchini_m_Page_151.jp2
cbaa5bb4792e4cc69152d0864d0efa9d
2387216ec0abae04d662f4d4762b03549b3ef0ca
63529 F20101124_AAAOIW cecchini_m_Page_133.jp2
5baed86cc0eb16fced31bf1fa757bd24
f7ab8792b2e7e62c086df081bfcc59e3acc5b9dd
4074 F20101124_AAAPNC cecchini_m_Page_143thm.jpg
5ef95c1217e6e934d1553625285b5aeb
4d866ae1091080eb8a5df6c08c120a3491fe52dd
4227 F20101124_AAAPMN cecchini_m_Page_130thm.jpg
fb312643dcf0d4e5822fbd8e786c5bfe
1dbb58b85dc2467d5862c1498a5efb8d9114e993
6480 F20101124_AAAPLZ cecchini_m_Page_119thm.jpg
ae9361130f45dc0c9390f90150c9eb27
64ec61d629eb2bdd6949bb3a5b7e192b2e2a7487
153327 F20101124_AAAOKA cecchini_m_Page_171.jp2
859c1aabf5a491320e7f50a241b2c539
91a5cc9b42795b91289197329b3a6afbb412ac90
160217 F20101124_AAAOJL cecchini_m_Page_152.jp2
7d9da02beaf792bcf59b7980ae42d3e3
6509a331ef79437c17a2b427413fbe1dc87c1b94
79141 F20101124_AAAOIX cecchini_m_Page_134.jp2
f7741fc7788ec9c1bbc5eda562f39689
3484f1a88413d717b6c599196ade770846ff8a8a
4116 F20101124_AAAPND cecchini_m_Page_144thm.jpg
a9476b9bcafae31fabf4ed6c76180c9f
6773471a9cbe7d7fcda83107c61a3e324561a54d
10706 F20101124_AAAPMO cecchini_m_Page_131.QC.jpg
43f67286f9329317970ef07114acf043
ce6a9fc77cf397a5ba5b81ab325e11da1ffe98d1
158314 F20101124_AAAOKB cecchini_m_Page_172.jp2
bf15ba83a8118093870c57c8c44a2392
24171520de7e2463bf8ee368197b53ce40b50186
158433 F20101124_AAAOJM cecchini_m_Page_153.jp2
a0b6fcb4f5cc8b9d794c5e59f36884aa
1dff6c699b697d6d473d5ab72a54704508f62b48
58006 F20101124_AAAOIY cecchini_m_Page_136.jp2
40f0a48cb3d0e68e00a35d830952eaae
8bd10c9e19510bc3bd1cb3a58c46865ae77d4310
3331 F20101124_AAAPMP cecchini_m_Page_131thm.jpg
abc59f08657f9e413717018ede09c389
e5cdc6de34f1a28a374aef56938d761d22274e35
156982 F20101124_AAAOJN cecchini_m_Page_154.jp2
70713cebc8a2cddb2c7d2ad07c077d48
fac3939bfd0400df0bda69c620c5044cfa6989e2
29720 F20101124_AAAOIZ cecchini_m_Page_137.jp2
c9d52855ade70823c798c19721b82458
7f25a18378e242fe8c8c4d9e483c610f0ebfc077
9680 F20101124_AAAPNE cecchini_m_Page_145.QC.jpg
bc1d17a18a7cd74d4986f6bbef4962f6
7245b84499489b0a7c1fedf173c7e6da679b365d
7244 F20101124_AAAPMQ cecchini_m_Page_132.QC.jpg
88dc83cd70fafcb7f5d84f610ad1c20e
6be08b1ace82b6350055c1bf49952caa8c3f12a8
84167 F20101124_AAAOKC cecchini_m_Page_173.jp2
fcc4c87c4c52544af524c7c798f1dc04
ad9bce52f6e9d737e111d998a135fc7813d95374
156994 F20101124_AAAOJO cecchini_m_Page_155.jp2
a76fc91e23f48788fd0187fd03924f13
784352c5a1da9563b18cc702d7a5094b6f06b916
15887 F20101124_AAAPNF cecchini_m_Page_146.QC.jpg
1a47e41796b631e5f3004ce1cae2e8ff
9ee0fdbdcccef735a9beb2351cba28f3e550b0cc
2457 F20101124_AAAPMR cecchini_m_Page_132thm.jpg
7fa305df041e3a6ce61ba872a7bfabfd
55355def4e84f74f4bd61edf966c46238a9921d7
45780 F20101124_AAAOKD cecchini_m_Page_174.jp2
08b004f1c1eb49282c65f2e9a37659c0
a6b193e2992859a0f6f9611f44a0824965ada127
159169 F20101124_AAAOJP cecchini_m_Page_156.jp2
a4e4a614eb464d792706f3ddca79befd
58cc0d25527776299f74d493b667f9bac54f41e2
15738 F20101124_AAAPNG cecchini_m_Page_147.QC.jpg
d92e0f102dfc20a9326a0f71ccffa08a
b045afaba82e5df26f5367d160fb06c15f963d07
4001 F20101124_AAAPMS cecchini_m_Page_133thm.jpg
ba69dd61aebd160afb908e51f3159ef2
3b144c3a85c938efaad6106ae6981d442cecdb49
165154 F20101124_AAAOJQ cecchini_m_Page_158.jp2
913ddd0cba308179d9a7fa45812c6928
d8b5a101cba34092949ec5e6f18dee11acee62cb
110061 F20101124_AAAOKE cecchini_m_Page_175.jp2
e597a2bb3d34260b8bcbc976f44ac9ed
6a74f92b8f5a6a5b2b0e75980d85bf3053f91180
15939 F20101124_AAAPNH cecchini_m_Page_148.QC.jpg
7002db6c5f19160aa364605c148b80da
b4f1309e1cf86f18b974eaf9172a55e83c9517d2
4345 F20101124_AAAPMT cecchini_m_Page_134thm.jpg
1abcb4b10605b481698ed2811aa915de
7b21515ce9af06ad40d60e36db54f72e359ac208
156127 F20101124_AAAOJR cecchini_m_Page_159.jp2
4eda9958ac01f912c068aa867bbe92b1
fea487e833e40539d49aa579c4b854c1b6fa0f66
1051969 F20101124_AAAOKF cecchini_m_Page_176.jp2
ef7a3f81de4cdfeaa9697029e9234514
1f20a79efaf3d4d52f40bcfc796a7928203a8008
4191 F20101124_AAAPNI cecchini_m_Page_148thm.jpg
c6b80b77a5dd299515b4a7b8583cb4af
d0883610bb4c786a38bac080c759753589b8b1c2
16992 F20101124_AAAPMU cecchini_m_Page_135.QC.jpg
aba4af4c8114e8788d509debe92045b3
406187d93aa948006afdd560bd7fa8bb946b3fd7
166794 F20101124_AAAOJS cecchini_m_Page_160.jp2
fdd598121070a57482e28b4bfec8afc1
e4f605c06b5b3d69971730cc140d750ca54e4849
F20101124_AAAOKG cecchini_m_Page_177.jp2
00bb276fed55b57f4da84585ed74e57f
5a311d90248728721367e157b2aef3612f31bf3b


Permanent Link: http://ufdc.ufl.edu/UFE0011430/00001

Material Information

Title: Quantifying the Risk of Financial Events Using Kernel Methods and Information Retrieval
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0011430:00001

Permanent Link: http://ufdc.ufl.edu/UFE0011430/00001

Material Information

Title: Quantifying the Risk of Financial Events Using Kernel Methods and Information Retrieval
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0011430:00001


This item has the following downloads:


Full Text












QUANTIFYING THE RISK OF FINANCIAL EVENTS USING KERNEL METHODS
AND INFORMATION RETRIEVAL















By

MARK CECCHINI


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2005





























Copyright 2005

by

Mark Cecchini

































This document is dedicated to Tara, Julian and Campbell, who were my inspiration in
pursuing and finishing a PhD.















ACKNOWLEDGMENTS

I would like to thank Tara and the rest of the Cecchinis for putting up with me

throughout this process. I would also like to thank our families for their support

throughout this endeavor. Without my committee there would be no dissertation. So, I

would like to acknowledge my Advisor Gary Koehler, who came up with the initial

research idea and has seen this research through from the beginning, Haldun Aytug, who

has been working on this project for three years, Praveen Pathak for his information

retrieval expertise and Gary McGill for helping me to understand the accounting

relevance of the work. Finally, I'd like to thank Karl Hackenbrack for his guidance in the

early stages of this work.
















TABLE OF CONTENTS

page

A C K N O W L E D G M E N T S ................................................................................................. iv

LIST OF TABLES ........ .................................................. ....... ........ viii

LIST OF FIGURES ................................. ...... ... ................. .x

L IST O F O B JE C T S .... ......................................... ............. .. .. ...... .............. xi

ABSTRACT ........ .............. ............. .. ...... .......... .......... xii

CHAPTER

1 INTRODUCTION AND MOTIVATION ......................................... ...............1

2 F IN A N C IA L E V E N T S ..................................................................... .....................5

2 .1 F rau d D election ................... .... ...................................... ........ ......... .. ....
2.2 B bankruptcy D election ............................................................ ............ 8
2.3 R estate ent D election ........................................................... ...............12

3 INFORMATION RETRIEVAL METHODOLOGIES............... ................ 16

3 .1 O v e rv iew ......................................................................................................... 1 6
3.2 V sector Space M odel .......................... .................... .. .. .. ...... ........... 18
3 .3 W o rd N et ............................................................................................................... 2 1
3.4 O ntology C reaction ........................ ................ .. .. ......... ..... .... 23

4 MACHINE LEARNING METHODOLOGIES ............... ............................... 27

4.1 Statistical L earning Theory........................................................ ............... 28
4.2 Support V ector M machines ............................................................................. 29
4.3 Kernel Methods .................... ....................................33
4.3.1 G general K ernel M ethods........................................ ......................... 34
4.3.2 D om ain Specific K ernels...................................... ......................... 40

5 THE FINANCIAL KERNEL .............................................................................. 43

6 THE ACCOUNTING ONTOLOGY AND CONVERSION OF DOCUMENTS
T O T E X T V E C T O R S ...................................................................... .....................54









6.1 The A accounting O ntology ................... ............ ............................... .... 54
6.1.1 Step 1: Determine Concepts and Novel Terms that are specific to the
accounting dom ain .............. .. .............. .............. .. ........ ...... ............ 55
6.1.2 Step 2: Merge Novel Terms with Concepts ...........................................61
6.1.3 Step 3: Add multi-word domain concepts to WordNet.............................64
6.2 Converting Text to a Vector via the Accounting Ontology..............................65

7 COMBINING QUANTITATIVE AND TEXT DATA ............................................69

8 RESEARCH QUESTIONS, METHODOLOGY AND DATA ..............................72

8 .1 H y p o th e se s ...................................................................................................... 7 2
8.2 R research M odel .......................... .............. ................. .... ....... 74
8 .3 D a ta se ts ........................................................................................................... 7 6
8 .3 .1 F rau d D ata .............................................................7 6
8.3.2 Bankruptcy D ata ................................................... .... ................. 77
8.3.3 R estate ent D ata .......................................................................... .... ... 79
8.4 The Ontology ................................. ............................... ........ 80
8.5 Data Gathering and Preprocessing...................... .... ......................... 82
8.5.1 Preprocessing-Quantitative Data..... .......... ...................................... 84
8.5.2 Preprocessing-Text D ata ........................................ ........................ 84

9 R E S U L T S .......................................................................... 8 8

9.1 F raud R results .......................................................................89
9.2 Discussion of Fraud Results ................................................... ......... ..........92
9.3 Bankruptcy Results ................................................. ...... .. ............ 94
9.4 Discussion of Bankruptcy Results............................................... .................. 97
9.5 Restatem ent Results ............................................. .... .... ............... ... 98
9.6 Discussion of Restatement Results .............. .............................................. 102
9.7 Support of H ypotheses................................................ ............................ 103

10 SUMMARY, CONCLUSION AND FUTURE RESEARCH...............................104

10 .1 Su m m ary ...................................... ............................................. 104
10.2 C conclusion ................................................................... ......... 105
10.3 Future R research ........................................ ................... ........ 106

APPENDIX

A ONTOLOGIES AND STOPLIST .................................. .............................. ...... 109

A 1 O ntologies ..................................................... ...... ......... ............... 109
A.1.1 GAAP, 300 Dimensions, 100 concepts, 100 2-grams, 100 3-grams.......109
A. 1.2 GAAP, 60 Dimensions, 40 concepts, 10 2-grams, 10 3-grams.............115
A .1.3 GAAP, 10 D im tensions, 10 concepts ......................................................117
A.1.4 10K, Bankruptcy, 100 Dimensions ....... ......... ....................117
A.1.5 10K, Bankruptcy, 50 Dimensions, 50 Concepts............... ...............119









A.1.6 10K, Bankruptcy, 25 Dimensions, 25 concepts..................................120
A. 1.7 10K, Fraud, 150 Dimensions, 50 concepts, 50 2-grams, 50 3-grams......121
A. 1.8 10K, Fraud, 50 Dim tensions, 50 concepts...............................................124
A. 1.9 10K, Fraud, 25 Dimensions, 25 concepts................. ..... ............. 125
A .2 S to p list ..................................................................................... 12 7

B QUANTITATIVE AND TEXT DATA..... .................. ...............128

B 1 Q uantitative D ata ...................................................... ... .. ......... .... 129
B.2 Text Data ........................... .............................162

LIST O F R EFEREN CE S ......... ................... ....................................... ............... 163

BIOGRAPHICAL SKETCH ................................... ............. 172
















LIST OF TABLES


Table pge

1 F financial K ernel V alidation ........................................................................... .... ... 51

2 Fraud Detection Results using Financial Kernel .............. ............ .....................89

3 Fraud Detection Results using Text Kernel, 300 Dim GAAP Ont............................90

4 Fraud Detection Results using Comb. Kernel, 300 Dim GAAP Ont..........................90

5 Fraud Detection Results using Text Kernel, 60 Dim GAAP Ont.............................90

6 Fraud Detection Results using Comb. Kernel, 60 Dim GAAP Ont............................90

7 Fraud Detection Results using Text Kernel, 10 Dim GAAP Ont .............................90

8 -Fraud Detection Results using Comb. Kernel, 10 Dim GAAP Ont............................91

9 Fraud Detection Results using Text Kernel, 150 Dim 10K Ont ................................91

10 Fraud Detection Results using Comb. Kernel, 150 Dim 10K Ont ..........................91

11 Fraud Detection Results using Text Kernel, 50 Dim 10K Ont ................................91

12 Fraud Detection Results using Comb. Kernel, 50 Dim 10K Ont ............................91

13 Fraud Detection Results using Text Kernel, 25 Dim 10K Ont ................................92

14 Fraud Detection Results using Comb. Kernel, 25 Dim 10K Ont ............................92

15 Bankruptcy Prediction Results using Financial Kernel ..........................................94

16 Bankruptcy Prediction Results using Text Kernel, 300 Dim GAAP Ont..................94

17 -Bankruptcy Prediction Results using Comb. Kernel, 300 Dim GAAP Ont. ............94

18 Bankruptcy Prediction Results using Text Kernel, 60 Dim GAAP Ont .................. 95

19 Bankruptcy Prediction Results using Combination Kernel, 60 Dim GAAP Ont......95

20 Bankruptcy Prediction Results using Text Kernel, 10 Dim GAAP Ont .................. 95









21 Bankruptcy Prediction Results using Combination Kernel, 10 Dim GAAP Ont .....95

22 Bankruptcy Prediction Results using Text Kernel, 100 Dim 10K Ont....................95

23 Bankruptcy Prediction Results using Combination Kernel, 100 Dim 10K Ont........96

24 Bankruptcy Prediction Results using Text Kernel, 50 Dim 10K Ont.......................96

25 Bankruptcy Prediction Results using Combination Kernel, 50 Dim 10K Ont..........96

26 Bankruptcy Prediction Results using Text Kernel, 25 Dim 10K Ont........................96

27 Bankruptcy Prediction Results using Text Kernel combined with Financial
Attributes, 25 D im 10K Ont ........................................ ............... ............... 97

28 Restatement (1,379 cases) Prediction Results using Financial Kernel ....................99

29 Restatement Prediction Results using Financial Kernel ........................................ 99

30 Restatement Prediction Results using Text Kernel, 300 Dim GAAP Ont ...............99

31 Restatement Prediction Results using Comb. Kernel, 300 Dim GAAP Ont.............99

32 Restatement Prediction Results using Text Kernel, 60 Dim GAAP Ont...............100

33 Restatement Prediction Results using Combination Kernel, 60 Dim GAAP Ont...100

34 Restatement Prediction Results using Text Kernel, 10 Dim GAAP Ont .................100

35 Restatement Prediction Results using Combination Kernel, 10 Dim GAAP Ont..100

36 Restatement Prediction Results using Text Kernel, 150 Dim 10K Ont.................00

37 Restatement Prediction Results using Combination Kernel, 150 Dim 10K Ont.....101

38 Restatement Prediction Results using Text Kernel, 50 Dim 10K Ont.....................101

39 Restatement Prediction Results using Combination Kernel, 50 Dim 10K Ont.......101

40 Restatement Prediction Results using Text Kernel, 25 Dim 10K Ont..................... 101

41 Restatement Prediction Results using Text Kernel combined with Financial
A attributes, 25 D im 10K O nt ..................................................................................101
















LIST OF FIGURES

Figure pge

1 Ontology Hierarchy ........................................ ... ...... .......... ...23

2 Basic Graph K ernel .............................. .............................. 38

3 Graph Kernel ...................... ................................ 40

4 T he F financial K ernel 1........................................................................ ...................4 8

6 U updated Financial K ernel .................................................. .............................. 53

7 Accounting Ontology Creation Process.................................... ....................... 56

8 WordNet Noun hierarchy with Domain Concepts.....................................................61

9 WordNet Noun hierarchy with Domain Concepts enriched with Novel Terms..........63

10 WordNet Noun Hierarchy with Domain Concepts, Novel Terms and Multi-Word
C o n c e p ts ............................................................................................................. 6 5

1 1 T ex t K ern el ................................................................6 9

12 Com bined K ernel ............................................... ........ .............. .. 70

13 T he D discovery P rocess...................................................................... ...................75

14 Fraud Features...................... ......... .. ..... ..... .. ............77

15 B bankruptcy F features ......................................................................... ................... 78

16 Fraud D ataset ................................................................... .. ............ 130

17 B bankruptcy D ataset................................................................................ ...... ... 134

18 R estate ent D ataset ........................................................................ ...................139








x

















LIST OF OBJECTS


Object


1 T ex t D ata ............. ................... .... ......... ................... ................ 16 2


page









Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

QUANTIFYING THE RISK OF FINANCIAL EVENTS USING KERNEL METHODS
AND INFORMATION RETRIEVAL

By

Mark Cecchini

August, 2005

Chair: Gary Koehler
Major Department: Decision and Information Sciences

A financial event is any happening which dramatically changes the value of a firm.

Examples of financial events are management fraud, bankruptcy, exceptional earnings

announcements, restatements, and changes in corporate structure. This dissertation

creates a method for timely detection of financial events using machine learning methods

to create a discriminant function. As there are a myriad of possible causes for any

financial event, the method created must be powerful. In order to increase the power of

current methods of detection text related to the company is analyzed together with

quantitative information on the company. The text variables are chosen based on an

automatically created accounting ontology. The quantitative variables are mapped to a

higher dimension which takes into account ratios and year-over-year changes. The

mapping is achieved via a kernel. Support vector machines use the kernel to perform the

learning task. The methodology is tested empirically on three datasets: management

fraud, bankruptcy, and financial restatements. The results show that the methodology is

competitive with the leading management fraud detection methods. The bankruptcy and

restatement results show promise.














CHAPTER 1
INTRODUCTION AND MOTIVATION

SAS 99, Consideration ofFraud in a Financial Statement Audit, establishes

external auditors' responsibility to plan and perform audits to provide a reasonable

assurance that the audited financial statements are free of material fraud. Recent events

highlight that failing to detect fraudulent financial reporting not only exposes the audit

firm to adverse legal consequences (e.g., the demise of Arthur Andersen LLP), but

exposes the audit profession to increased public and governmental scrutiny that can lead

to fundamental changes in the structure of the public accounting industry, accounting

firm conduct, and government oversight of the accounting profession (consider, for

example, the Sarbanes-Oxley Act of 2002 89 and subsequent actions of the SEC 92 and

NYSE 71). Research that helps auditors better assess the risk of material misstatement

during the planning phase of an audit will reduce instances of fraudulent reporting. Such

research is of interest to academics, standard setters, regulators, and audit firms.

Current research in accounting has examined methods to assess the risk of

fraudulent financial reporting. The methodologies are varied and usually combine some

behavioral and quantitative factors. For example, Loebbecke, Eining and Willingham 55

compiled an extensive list of company characteristics associated with fraudulent

reporting (called "red flags"). This list contains financial ratios and behavioral

characteristics of company management. Other methods scrutinize accounting entries that

are not easily verified by outside sources; these entries are called discretionary accruals.









Board composition and executive compensation are also used to model the type of

environment that is ripe for fraud.

This dissertation proposes a methodology that can estimate the likelihood of

fraudulent financial reporting. The resulting decision-aid has the potential to complement

the unaided auditor risk assessments envisioned in SAS 99. Our approach combines

novel aspects of the fraud assessment research in accounting with computational methods

and theory used in Information Retrieval (IR) and machine learning/datamining.

Machine learning uses computational techniques to automate the discovery of

patterns that may be difficult to find by normal analytic techniques. Machine learning

methodologies have been used in order to determine financial statement validity or,

somewhat related, the likelihood of bankruptcy and credit worthiness. There are many

models commonly used in machine learning with neural networks 66, linear discriminant

functions 34, logit functions 3, and decision trees 80 being popular choices. Attempts

have been made to recognize patterns in fraudulent companies using neural networks,

linear discriminant functions, logit functions, and decision trees. These studies utilized

quantitative data from financial statements and surveys from auditors. Unlike these

earlier studies, recent advances in machine learning theory consider generalization ability

and domain knowledge while the learning task is undertaken.

Existing work on fraud detection has left out a key component of and about the

company, text documents. In most public documents, the preponderance of information

is textual but most automated methods for detecting fraud are based on quantitative

information alone. So, either an expert has to distill the text to numbers, which is a

monumental task, or the text-based information is largely ignored. We hypothesize that









there is information hidden "between the lines" that is overlooked. Our approach can

incorporate textual materials like management discussion and analysis, news articles, and

so on.

An area of research called Information Retrieval (IR) can help us to make use of

the text. IR is often employed in library science and, more recently, in powerful Internet

search engines (such as Google 39). IR is used for varied purposes, including question

answering, document sorting, knowledge engineering, query expansion, and inferencing.

We use IR methodologies to cull the financial text down to numbers, which can be used

in conjunction with numerical attributes obtained from the financial statements to

automatically predict the likelihood of fraud.

What distinguishes the proposed approach from prior attempts to understand and

aid fraud-risk assessments are advances in machine learning theory, both through a

theory that addresses generalization errors and methods incorporating domain knowledge

while the learning task is undertaken, and in IR that enable computer programs to analyze

textual materials.

The methodologies we create can be generalized to other accounting issues, such as

the early detection of bankruptcy, detection of earnings management, early detection of

increased market value, and general industry stability. Each of these issues has the

potential to impact a company's value significantly shortly after a related first press

release or news item is made public. As a result of the speedy impact, these issues can be

called financial events. In this dissertation, we look at the early detection of bankruptcy,

together with the detection of management fraud.









The goal of this dissertation is discussed below. In the following chapter we

review financial events detection literature and summarize key concepts and results.

Chapter 3 summarizes relevant machine learning literature. Chapter 4 summarizes

relevant Information Retrieval literature. In Chapter 5 we develop a machine learning

methodology that handles quantitative financial data. In Chapter 6 we develop the IR

methodologies that enable us to utilize text for financial event detection. In Chapter 7 we

explain how we put the text data together with the quantitative data. We also extend the

methodology we create in Chapter 5 to include text. These methods are used to study

some actual data on which we ask a number of questions. The research model and

hypotheses are developed in Chapter 8 and tested. Chapter 9 explains the results along

with a conclusion and an explanation of future work.














CHAPTER 2
FINANCIAL EVENTS

As explained in the Introduction, a financial event is any event that significantly

alters the value of a company. One can think of such an event as one that raises or lowers

the value of the company. A partial list of possible events which lower the value of the

firm are as follows: civil or criminal litigation, bankruptcy, management fraud,

defalcations, restatements, earnings management and poor press. We focus on three such

events in particular, management fraud, bankruptcy and restatements. In Section 2.1 we

look at the fraud detection literature from accounting and machine learning. In section 2.2

we look at bankruptcy detection literature from those perspectives as well. In section 2.3

we look at the Restatements literature.

2.1 Fraud Detection

A key result in audit research was given by Loebbecke, Eining and Willingham 55.

They partitioned a large set of indicators into three main components: conditions,

motivation, and attitude. They find in 86% of the fraud cases at least one factor from

each component was present, indicating it is extremely rare for fraud to exist without all

three components existing simultaneously. Hackenbrack 41 finds the relative influence

of such components on auditor fraud-risk assessments varies systematically with auditor

experiences. This research influenced standard setting and much of the fraud assessment

research that has followed.

Bell and Carcello 9 developed a logistic regression model to estimate the likelihood

of fraudulent financial reporting. The significant risk factors considered were as follows:









weak internal control environment, rapid company growth, inadequate or inconsistent

relative profitability, management places undue emphasis on meeting earnings

projections, management lied to the auditors or was overly evasive, the ownership status

(public vs. private) of the entity, and an interaction term between a weak control

environment and an aggressive management attitude toward financial reporting. The

logistic regression model was tested on a sample of 77 fraud engagements and 305 non-

fraud engagements. The model scored better than auditing professionals in the detection

of fraud. The model performed equally as well as audit professionals for the non-fraud

portion. The authors suggest that the use of this model might be used to satisfy the SAS

82 requirements for assessing the risk of material misstatement due to fraud.

Hansen, McDonald, Messier, and Bell 42 develop a generalized qualitative-

response model to analyze management fraud. They use the same dataset of 77 fraud and

305 nonfraud cases as collected by Loebbecke, Eining, and Willingham. They first tested

the model with symmetric costs between type I and type II errors. Over 20 trials they got

an 89.3% predictive accuracy. They adjusted the model to allow for asymmetric costs

and the accuracy dropped to 85.5%; however, the type II errors decreased markedly. The

consideration of type I and II errors is important in fraud detection research. Minimizing

the type II error is minimizing the chance that the model will miss an actual fraud

company. When type II error is minimized, type I error will increase. In the case of

fraud detection, type I error is much less important than type II error.

In fraud detection, discretionary accruals are a cause for concern as discretionary

accruals have been known to be used to help "smooth" fluctuations in periodic income.

Accounts that are used in discretionary accruals, such as Bad Debts Expense, Inventory









and Accounts Receivable, are susceptible to "engineering" on the part of management.

By considering year-over-year changes in ratios, which include these accounts, a clearer

picture of the company emerges.

McNichols and Wilson 56 look at the provision for bad debts and consider how it

should be reported in the absence of earnings management. Earnings management is a

term that describes a spectrum of "cheating" that at a minimum is aggressive and not in

strict compliance with GAAP, and at maximum is management fraud. The research

found that firms use the provision for bad debts as an income smoothing method; in other

words, it is raised in times of high earnings and lowered in times of low earnings.

Ragothaman, Carpenter, and Buttars 82 developed an expert system to help

auditors in the planning stage of an audit. The system is designed to detect error potential

in order to determine if additional substantive testing is necessary for the auditors. The

expert system rules were developed using financial statement data. The expert system

methodology was rule induction. The system decides whether the firm is an "error" firm

or a "non-error" firm. If the firm is an "error" firm then the auditor should consider

additional substantive testing. A training sample of 55 firms (22 error firms and 33 non-

error firms) was used. A holdout sample of 37 firms was used. The training sample was

able to group 86.4% of errors correctly and 100% of non-error firms correctly. The

holdout sample classified 83.3% of error firms correctly and 92% of non-error firms.

This study was limited by the available data. The accounting literature on fraud detection

is covered at great length in Davia 25 and Rezaee 84.

Beneish 10 developed a probit model and considered several quantitative financial

variables for fraud detection. Five of the 8 variables involved year-over-year changes.









The study considered differing levels of relative error cost. At the 40:1 error cost ratio

(Type I:Type II) 76% of the manipulators were correctly identified. Also, descriptive

statistics showed that the Days Receivable Index and the Sales Growth Index were most

effective in separating the manipulators from the non-manipulators.

2.2 Bankruptcy Detection

Bankruptcy detection is a well-studied area. Many methodologies have been used

to solve this problem, including discriminant analysis, neural networks, fuzzy networks,

ID3 (a decision tree classification algorithm), logistic regression and genetic algorithms.

In this section we describe some major contributions to the literature.

In 1966 Beaver 8 showed the efficacy of financial ratios for detecting bankruptcy.

The study was performed on a dataset of 79 bankrupt and 79 nonbankrupt firms. Beaver

computed the mean values of fourteen financial ratios for all companies in the study for a

five year period prior to bankruptcy. Many of the ratios proved to be valuable to the

detection, because the mean values of the bankrupt companies were significantly

different than the mean value for the nonbankrupt companies.

Altman's paper in 1968 4 was a seminal work in bankruptcy detection. He

developed a discriminant analysis model using financial ratios. Using a paired-sample

approach, Altman compared twenty-two ratios for efficacy in bankruptcy prediction.

Five ratios stood out as they were able to accurately predict bankruptcy one year

preceding the event. The model predicted bankruptcy correctly 95% of the time and

nonbankruptcy correctly 80% of the time. The resulting function, dubbed the Altman Z-

Score, has been the benchmark for bankruptcy detection work ever since. The specific

ratios of the Altman Z-Score are as follows:

Working Capital/Total Assets (WC/TA)









Retained Earnings/Total Assets (RE/TA)

Earnings Before Interest and Taxes/Retained Earnings (EBIT/RE)

Book Value of Equity/Total Liabilities (BVE/TL).

The function has had several incarnations and its weights differ based on industry.

For the manufacturing industry it is

6.56(WC / TA) +3.26(RE / TA) +6.72(EBIT/RE) +1.05(BVE/TL) = Z -score.

The weights on each ratio indicate the ratio's relative importance for classification

of healthy and unhealthy companies. A score which is less than some threshold means the

company is likely in financial distress while a score greater than or equal to this threshold

means the company is likely safe from bankruptcy, at least for the short term. There is a

gray area around the threshold that can be construed as an area of concern. The

predictive accuracy of this discriminant analysis function is still competitive today for

sorting out healthy companies from unhealthy ones.

Altman et al. noted that the discriminant analysis technique had limitations, one

being its inability to handle time series 5. Bankruptcy is the sum product of many events.

A company which goes bankrupt is likely to have been in a deteriorating state for more

than one period. Year-over-year changes can capture this deterioration better than single

year measures.

Ohlson 72 was the first to utilize a logistic regression approach to bankruptcy

prediction. He identified four factors as statistically significant in affecting the

probability of failure within one year. The factors are as follows: the size of the

company, a measure of financial structure, a measure of performance, and a measure of

current liquidity. Another finding of the research was that the predictive powers of linear









transforms of a vector of ratios appear to be robust for estimating the probability of

bankruptcy.

Abdel-khalik and El-Sheshai 1 designed an experiment testing human judgment.

Decision makers (loan officers) were allowed to choose the information cues they use to

make their judgments. The information cues were used to determine whether a loan

would end in default. In comparison to mechanical models discriminantt analysis), loan

officers performed worse. The finding was that the choice of information cues is more

responsible for the lack of correct prediction than the processing of the cues.

Frydman, Altman and Kao 36 developed a recursive partitioning algorithm (RPA)

for bankruptcy classification. The RPA is a Bayesian procedure, with classification rules

derived in order to minimize the expected cost of misclassification. In most cases, the

RPA outperformed Altman's previous results via discriminant analysis.

Messier and Hansen 62 use inductive inference to analyze examples of bankrupt

companies and loan defaults to infer a set of general rules in the form of if-then-else

statements. The set of output statements is called a production system. The bankrupt

study used only the following ratios: current ratio, earnings to total tangible assets and

retained earnings to total tangible asset. The production system was 100% accurate on a

very small holdout set (12 bankrupt and 4 nonbankrupt). The study also used the

production system to detect potential loan defaults. The method was 100% accurate on

the training sample and 87.5% accurate on a validation sample. In both studies, the

production system used fewer ratios and was more accurate than the discriminant models

it was compared against.









It should be noted that Tsai and Koehler 104 tested the robustness of the results of

several papers using inductive learning, including the results of Messier and Hansen 62.

The authors determined the accuracy of the induced concepts when tested on the same or

similar domains. In the case of Messier and Hansen, their findings included a probability

of error on the learned concept of the bankruptcy sample. The probability that the error

of the learned concept exceeds 20% is 30.96%. This is due, in part, to the small sample

size. The study throws up a caution flag, warning readers that the true accuracy of

concepts learned by induction may not be revealed in studies of small sample size.

Tam and Kiang 101 use a back propagation neural network to predict bank

defaults. They compare their results with k nearest neighbor, discriminant analysis 34,

logistic regression 79 and ID3 80. When considering the year prior to bankruptcy, a

multilayer neural network got the best results. When considering two years prior to

bankruptcy, logistic regression performed the best.

Charalambous, Charitou, and Kaourou 17 compare the performance of three neural

network methods, namely Learning Vector Quantization, Radial Basis Function, and the

Feedforward network. They test their results on 139 matched pairs of bankrupt and

nonbankrupt U.S. firms. Their results indicate that Learning Vector Quantization gave

superior results over feedforward networks and logit analysis.

Piramuthu, Raghavan, and Shaw 77 develop a method of feature construction. The

method finds the features that are most pertinent to the classification problem and

discards the ones that are not. The "constructed" features are fed into a back propagation

neural network. The method was tested on Tam and Kiang's 101 bankruptcy data. The









network showed a significant improvement in both classification results and computation

time.

2.3 Restatement Detection

The literature covering restatements is found in conjunction with earnings

management literature as well as fraud literature. Each company for which the SEC

discovers fraud is forced to restate. All restatements, however, are not fraudulent.

Restatements can be made for various reasons, including stock splits, errors, accounting

irregularities, and fraud. Restatements may be voluntary or involuntary. For the

purposes of this research, restatements are defined as in General Accounting Office

report GAO-03-138 37. These restatements may be voluntary or involuntary and only

arise as a result of accounting irregularities. An accounting irregularity is fraudulent if

committed with intention and nonfraudulent if committed by mistake. Restatements can

be seen as a superset of fraud. The restatement literature specifically related to detection

is limited as compared to fraud and bankruptcy. The literature reviewed in this section

gives an overview of the research problems related to restatements.

Dechow et al. 26 evaluate the performance of competing models of earnings

management detection. The models tested are the Jones model, the Modified Jones

model, the DeAngelo model and the Industry model. These models are based on the

amount of discretionary accruals made by a company in a particular year. Discretionary

accruals are not readily observable based on publicly available reports. The models infer

the amount of discretionary accruals based on other inputs and total accruals. The results

show that all methods are accurate for detecting earnings management for extreme cases.

However, all methods gave poor results when faced with discretionary accruals which

were a small percentage of total assets (1% 5%). Earnings management is more likely









to occur at the 1% 5% levels, so the practical value of the methods used is brought into

question.

Koch and Wall develop an economic model of earnings management, which

elucidates the situations in which earnings management are most likely to occur, based on

executive compensation packages. The authors determine how accruals can be used to

manage reported earnings. In the paper the authors explain several earnings management

tactics. A partial list follows;

(1) The "Live for Today" strategy Managers minimize accrued expenses in

order to maximize profit.

(2) The "Occasional Big Bath" strategy managers attempt to meet earnings

targets whenever possible. If it looks impossible to meet targets then they

attempt to accrue a high amount of expenses in that period to allow for

meeting earnings targets the next.

(3) Miscellaneous Cookie Jar Reserves strategy This is defined as the usage

of unrealistic assumptions in the process of estimating accruals.

These methods can be readily detected after-the-fact. It is much more difficult to detect

these tactics as they are happening.

Abbot et al. 1 study the impact of the audit committee on the likelihood of

restatement. The authors find that an independent and active audit committee

significantly reduces the likelihood of restatement. An audit committee which contains at

least one member with financial expertise further reduces the likelihood of restatement.

This empirical study gives weight to the arguments for having an audit committee.









Feroz et al. 33 study the effect of Accounting and Auditing Enforcement Releases

on company valuation. A reporting violation leads to a 13% decline over a two day

period, on average. The study also finds that the companies which are in violation

substantially under perform the market in the years prior to the release, indicating that the

incentive to cheat is at least in part due economic pressures on the executives of the

company.

Hribar and Jenkins 43 study the effect of restatements on a firm's cost of equity

capital. The authors find that restatements lead to a decrease in expected future earnings

and increases in the firm's cost of equity capital. The increases were found to be between

7% and 12%. Over the long-term the rates remain higher than before the restatement by

at least 6%. Another finding of the work are that firms with greater leverage are

associated with larger increases in capital.

Kinney et al. 50 approach the problem from the auditor's perspective. They study

the correlation between restatements and the amount of non-audit services performed by

the auditor. This topic became especially interesting when the Sarbanes-Oxley Act of

2002 specifically forbade auditors to provide certain non-audit services to their clients.

The study found no significant positive correlation between financial information systems

services and restatements. There was a significant positive correlation between

unspecified non-audit service fees and restatements. This study supports the notion that

auditor independence can be compromised by non-audit consulting engagements with

audit clients.

Peasnell et al. 75 focus on the factors associated with low earnings quality by

looking at a sample of 47 firms which have been identified as having defective financial









statements. A positive correlation between defective financial statements and losses or

significant earnings decreases was found. Restating firms were less likely to increase

dividends, provide optimistic forecasts, and more likely to be involved in corporate

restructuring. Restating firms were also less likely to employ a Big 4 auditor and often

carried higher debt as a percentage of total assets as compared to nonrestating firms. The

study also found that firms which employed active audit committees were less likely to

have defective financial statements.

In this Chapter the literature on Financial Events was reviewed. Bankruptcy, fraud

and restatement research was reviewed. The next two chapters explain the

methodologies used for this research project. Chapter 3 reviews Information Retrieval

Methodologies and Chapter 4 reviews Machine Learning methodologies.














CHAPTER 3
INFORMATION RETRIEVAL METHODOLOGIES

This chapter presents a general overview of research in the area of IR. As this is an

enormous area of research, the main focus is on contributions to the field as they relate to

this dissertation. Specifically, we focus on methods of ontology creation and WordNet.

The sections are as follows: Section 4.1 provides a brief overview of general IR research.

Section 4.2 explains the Vector Space Model. Section 4.3 explains the basic concepts of

lexical databases with specific details about WordNet. Section 4.4 explains the

fundamentals of ontology creation.

3.1 Overview

"An Information Retrieval system does not inform (i.e., change the knowledge of)

the user on the subject of his inquiry. It merely informs on the existence (or non-

existence) and whereabouts of documents relating to his request" 105. This field of study

has exploded with the reality of massive amounts of text in an online environment the

Internet. The need to correctly choose the documents that are relevant to a keyword

search has become important to industry (in the form of search engines), decision

scientists, and computer scientists. There is much more to the field of IR than merely

document retrieval. Some of these are as follows.

Question answering systems take natural language questions as input, allowing the

user to avoid learning tedious query structures. In response, the system outputs a number

of short responses, designed to answer the specific question of the user. The goal of

question-answering is to give a more precise response to the user. Whereas normal









document retrieval outputs a list of documents, question-answering outputs small

passages from documents 74.

Query expansion is a research area which has grown tremendously as a result of the

internet. Query expansion is most commonly used by search engines as a means to

improve the accuracy of results of user queries. A user types a few words as a query, and

the system expands that query by adding words which will presumably give better results.

There are many methods of query expansion. Automatic query expansion uses machine

learning techniques to choose the best expanded query 64.

Inferencing systems are a generalization of query expansion. They can be used at

all levels of the IR process. They attempt to "infer" the meaning of a query and add

further detail. The inference is usually based on semantic relatedness of the words in the

query. Semantic relatedness can be determined by parsing a particular corpus as in the

case of latent semantic analysis 52, which uses statistical techniques to find co-

occurrences between words in a corpus, or it can be determined by using a lexical

reference system, such as WordNet.

Literature-based discovery uses IR techniques to discover hidden truths from a

particular domain. The basic idea is: parse a set of documents A related to a particular

subject and find a list of subjects that A refers to. Parse a second set of documents B

related to the subjects A refers to in order to find the subjects B refers to. The subjects B

refers to are called C. If some subjects in C are unexplored in relation to A, then they

may be worth looking at. The seminal work in this area is by Swanson 99. Using

Medline (a medical document repository) he was able to find previously unknown

connections between Raynaud's disease and fish oil. Those connections were tested









empirically by medical researchers. The results showed that fish oil can actually reduce

the symptoms of Raynaud's disease.

3.2 Vector Space Model

A primary goal of IR research is to relate relevant documents to user queries.

Using IR methods, one seeks to separate relevant textual documents from non-relevant

ones.

A powerful method in IR research is called the vector space model 18, 54, 88. This

approach begins by truncating all words in the document into word stems. Word stems

are the base form of words, without suffixes. Stemming is important because a computer

cannot see that "stymies" and "stymied" are basically the same thing. If we stem the two

words, they both become "stymie." This allows the computer to see the two as one word,

thus adding to the words importance (via word count) in the document. Then it

transforms the document into a vector by counting the frequency of each word in the

document. Various ways of normalizing these vectors are available. A key observation

is that these vectors are now quantitative representations of the textual parts of

documents.

Here is a more formal explanation of the vector space model 48. Each document in

the vector space model is represented by a vector of keywords as follows:

d W = (wjW,,w2, "Wn,j)'f

where n is the number of keywords and w, is the weight per keyword i in document j .

This characterization allows us to view the whole document collection as a matrix of

weights and is called the term-by-document matrix. The columns of this matrix are the

documents and the terms are the rows. A document is translated into a point in an n









dimensional vector space. For this method to be useful, the vectors must be normalized.

The dot product between normalized vectors gives the cosine of the angle between the

two vectors. When the vectors representing two documents are identical, they will have a

cosine of 1; when they are orthogonal, they will receive a cosine of 0. The similarity

measure between documents j and k is as follows:

n
Y W k

sim(d, dk) =
V 2 2
W,k


Finding a w that most accurately depicts the importance of the keywords in the

collection is very important to document classification. Sparck Jones 97 made a seminal

breakthrough on this problem with the TF-IDF function. The function stands for Term

Frequency, Inverse Document Frequency. The basic TF-IDF function is as follows:

N
w' = (tf), *log tf, is the frequency of term t, in document d N is the number of
n

documents in the collection and n is the number of documents where the term t occurs

at least once. The logic is as follows: for (tf), or term frequency, a word that occurs

more often in a document is more likely to be important to the classification of that

N
document. A measure of inverse document frequency, idf, is defined by (idf), = log .
n

The logic is that a word that occurs in all documents is not helpful in the classification of

the document (hence the inverse) and therefore gets a 0 value. A word that appears in

only one document is likely to be helpful in classifying that document and gets a value of

1.









Many researchers have attempted to improve upon the basic vector space model.

Improvements take the form of making the document vector more accurately depict the

document itself. Part-of-speech tagging is one such improvement. A part-of-speech

tagger reads a document in natural language and tags every word with a part-of-speech,

such as noun, verb, adjective and adverb. The tags are created using sentence structure.

All part-of-speech taggers are heuristics with no guaranteed accuracy. However, the

recent taggers have become so accurate that they only make a few mistakes on entire

corpuses 11.

Another improvement is word sense disambiguation (WSD). WSD is the attempt

to understand the actual definition of a word, in the context of a sentence. Often words

that are spelled identically have several meanings. In the basic vector space model, the

document vector would take all instances of the word "crane" and add them up. What if

one sentence read, "The crane is part of the animal kingdom" and another sentence read,

"The crane was the only thing that could move the 2 ton truck to safety"? Crane in the

first sense is referring to a bird whereas crane in the second sentence is referring to a

mechanical device. A word sense disambiguated vector would have two versions of the

word crane if both showed up in the corpus. This avoids some confusion that might arise

were we comparing the similarity between two documents, one which was about the bird

called a crane, and the other which was about the piece of equipment. How is WSD

accomplished? One method is to look at a previously hand-tagged corpus. One such

corpus is called SemCor 22. It is a corpus of documents, which are all tagged with

particular word meanings. Researchers use SemCor as a tool to learn WSD. For

example, take all sets of word pairs from a corpus and compare with SemCor, looking for









pairs that appear together often enough to be considered statistically significant. The

phrase "crane lifts beams" may show up in the corpus. It is possible to determine if the

noun "crane" and the verb "lifts" are found together often enough in SemCor to be

considered significant. If this co-occurrence pair is considered significant, then "crane"

will be given the particular sense number for which it was tagged in SemCor.

3.3 WordNet

A lexical reference system is one which allows a user to type a word in and get in

return that word's relationships with other words. "WordNet is an online lexical

reference system whose design is inspired by current psycholinguistic theories of human

lexical memory. English nouns, verbs, adjectives and adverbs are organized into

synonym sets, each representing one underlying lexical concept 22." The current version

of WordNet has 114,648 nouns, 11,306 verbs, 21,436 adjectives and 4669 adverbs in its

system today 22. WordNet is hand-crafted by linguists. The basic relation in WordNet is

called synonymy. Sets of synonyms (called synsets) form its basic building blocks. For

example, the word "history" is in the same block as the words past, past times,

yesteryear, and yore. Due to synonymy, WordNet would be much closer to a thesaurus

than a dictionary. Nouns are organized into a separate lexical hierarchy as are verbs,

adjectives and adverbs.

There are two main types of relations in WordNet, lexical relations and semantic

relations. Lexical relations are between words and semantic relations are between

concepts. A concept is another word for a synset. A relationship between concepts can

be hierarchical, as is the case of hyponyms and hypernyms. The hyponym/hypernym is a

relation on nouns. Nouns are separated from other parts of speech because their

relationships are considered different than the relationships between verbs and adjectives.









A hyponym/hypemym relationship is an "is a" relation. WordNet can be represented as a

tree. Starting at the top or root node, the concept is very general (as in the case of

"entity" or "psychological feature"). As you go down the tree, you encounter more fine-

grained concepts For example, a robin is a subordinate of the noun bird and bird is a

superordinate of robin. The subordinates are called hyponyms (is a kind of bird) and the

superordinates are called hypernyms (robin is a kind of). Modifiers, which are adverbs

and adjectives are connected similarly as are verbs. Hyponomy is only one of many

relations in WordNet. Below is a list of other WordNet relations with examples 94:

Relation Example Applicable POS

Has-Member Faculty Professor Noun

Member-of Copilot Crew Noun

Has-part Table Leg Noun

Part-of Course Meal Noun

Antonym Leader Follower Noun

Increase-Decrease Verb

Troponym Walk Stroll Verb

Entails Snore Sleep Verb

Traditional vector space model retrieval techniques focus on the amount of times a

word stem appears in a document without considering the context of the word. Consider

the following two sentences, "What are you eating?" "What's eating you?" The words

"what," "are" and "you" would most likely be stop words. (A stop word is any word that









is thought to have little impact on the classification of any document. Common stop

words are "the", "and", "but", "what", "are" and "you". The list of stop words is usually

determined by taking statistics on the document set. If a word appears too often it is said

to carry little weight. This word becomes a stop word. Stop words do not appear in the

document vector.) The two sentences above would have identical meaning in the vector

space model. The meaning of the two sentences are however, completely different.

Using concepts and contexts it is possible to create a lexical reference system that

interprets data specific to a particular area of interest.

3.4 Ontology Creation

Figure 1 67 shows that there are three types of ontologies. There are top

ontologies, upper domain ontologies, and specific domain ontologies. Top ontologies are

populated with general, abstract concepts. Upper domain ontologies are more

specialized, but still very general. Specific domain ontologies are populated with

concepts that are specific to a particular subject. Top ontologies for the English language

are relatively complete. Upper domain ontologies and specific domain ontologies are

still under construction 67.


lordl r of Ituoca epts

(ordrr afl 10 c oncqpt
SpI"r it DamAin On nti..I'
S(ordcr o I c cnc ptls) )




Figure 1 Ontology Hierarchy

WordNet is a top ontology. Many domain engineers attempt to make domain

specific ontologies using the backbone of top ontologies. Often a problem arises in that









there is a gap between the top ontology and the specific domain ontology. In this case, an

upper domain ontology is necessary. An upper domain ontology connects the top

ontology to the specific domain ontology. The upper domain ontology forms the root

nodes for the Domain Specific Ontologies.

Domain specific ontologies are usually created for a specific purpose and these are

very difficult to obtain. Navigli and Velardi explain "A domain ontology seeks to reduce

or eliminate conceptual and terminological confusion among the members of a user

community who need to share various kinds of electronic documents and information

68." Domain ontology creation is a new and active research area in IR. Here are some

papers which highlight the current state of the research.

Khan and Luo 49 construct ontologies using domain corpora and clustering

algorithms. The hierarchy is created using a self-organizing tree. WordNet is used to

find domain concepts. The concept hyponyms are added to the tree, under the concept.

This is a novel usage of WordNet and a completely automated method of ontology

construction. The method is tested on the Reuters 21578 text document corpus.

Navigli and Velardi 68 give a step-by-step method explaining the process of

obtaining ontology. Candidate terminology is extracted from a domain corpus and

filtered by contrastive corpora. The contrastive corpora are used to ignore candidate

terms which are in actuality part of the general domain. The word senses of domain

terminology are discovered via SemCor and WordNet. New domain specific

relationships are determined based on rule based machine learning techniques. These

relationships are used to determine multi-word terms which are domain specific. Finally,









the domain ontology is trimmed and pruned. This methodology was used to create a

tourism domain ontology.

Vossen 108 describes a methodology of extending WordNet to the technical

domain. The domain corpus is parsed into header and modifier structures. A header is a

noun or verb and a modifier is an adjective or adverb respectively. A header may have

more than one modifier, as in the example "inkjet printer technology". Here

"technology" is the head and "inkjet" and "printer" are modifiers. Salient multiword

terms are hierarchically organized creating a domain concept forest. A domain concept

forest is a set of concepts related to a specific domain together with relationships between

the concepts. The root node of each of the domain concepts is attached to a WordNet

concept. In the above example "technology" would be the root node. The result is a

domain concept forest attached to WordNet.

Buitelaar and Sacaleanu 15 create a method of ranking synsets by domain

relevance. The relevance of a synset is determined by its importance to the domain

corpus. The importance is determined by the amount of times the concept appears in the

corpus. A contrastive corpora is used to filter out concepts that are general, as in Navigli

and Velardi 68. A unique contribution of this research is the usage of hyponyms to

determine domain relevance. A hyponym is lower on the tree, therefore it is a

specialization of the concept. The authors look at how often a hyponym to a concept

appears in the document as part of the relevance measure. The result is an ordered list of

domain terms. The authors tested the methodology on the medical domain by parsing

medical journal abstracts.









Buitelaar and Sacaleanu 14 extend their work by adding words to domain concepts

based on lexico-syntactic patterns. The domain corpus is parsed to look at the syntax

patterns of seven word combinations. Each pattern is separately considered for

relevance. For all salient patterns, mutual information scores are given to co-occurrences

within the pattern. Novel terms from the domain which are not in WordNet are added to

WordNet concepts if it is determined that they are statistically significant. This

methodology is tested on the medical domain.

In this Chapter Information Retrieval Methodologies were explained. The specific

areas reviewed were the Vector Space Model, WordNet and Ontology creation. These

areas were chosen because of their relevance to the contributions of this work. In the

Chapter 4 Machine Learning Methodologies are reviewed.














CHAPTER 4
MACHINE LEARNING METHODOLOGIES

Most machine learning/datamining methods 66 start with a training set of data from

past cases illustrating positive and negative examples of the concept to be learned. This

is called supervised learning. For example, if we are trying to learn how to discriminate

between companies likely to default on loans in the coming year from those unlikely to

default, we would collect past cases of defaulting and non-defaulting companies as done

in studies such as 1 62. Such a training set consists of 1 observations and a classification

for each. That is, there are 1 pairs of the form z' (u',y') where u' E X c 91" represent

the n input attributes (the independent variables) with X called the instance space of all

possible companies, y' e {-1, +1 the classification (+1 means a positive example and -1

a negative example of the concept) for i = 1,..., and the sample S is

((u1, yl),...(u', y)) c (XxY)' 20. Unless otherwise stated, a vector is denoted by a

bold, lowercase letter. The superscript on the vector is reserved for the observation

number. An unbolded, subscripted, lowercase letter refers to the components of the

vector. The subscript represents the index of the component. In Chapter 5 we add a

second subscript to denote the year (or period). Typical approaches, such as neural

networks, logit, etc. start with a training set and try to fit the data as best as possible using

the concept structure chosen (i.e., a neural network, a logit function, etc. respectively).

This invariably leads to over-fitting. To ameliorate this, the training set is often broken

into two (or more) sets where part of the cases are used to fit a function and part to test









it's ability to predict on a data set not used for fitting. These approaches do help with

over-fitting but are largely ad hoc.

4.1 Statistical Learning Theory

Statistical learning theory 106 formally develops the goal of learning a function

from examples as that of minimizing a risk functional

R(c) = JL(z,g (z, c))dF(z)

over a e A where L( ) is a loss function, and g(z, a) is a set of target functions

parametrically defined by ca A (the family of functions we are investigating). In this

approach it is assumed that observations, z, are drawn randomly and independently

according to an unknown probability distribution F(z). Since F(z) is unknown, an

induction principle must be invoked. One common induction principle is to minimize the

number of misclassifications. Minimizing the number of misclassifications is directly

equivalent to minimizing the empirical risk with the loss function as a simple indicator

function. Other loss functions give different risk functions. For example, the classical

method for linear discriminant functions, developed by Fisher 34, is equivalent to

minimizing the probability of misclassification.

As is well known, empirical risk minimization often results in over-fitting. That is,

for small sample sizes, a small empirical risk does not guarantee a small overall risk.

This has been observed in many studies. For example, Eisenbeis 28 critiques studies

based on such over-fitting.

Statistical learning theory approaches this problem by using a structural risk

minimization principle 106. For an indicator loss function, it has been shown 106 that for

any ca A with a probability at least 1- r the bound









Rst (h, l1,) R ()
R (a) Rmp+(a)+ R
R () 2 Rstruct (h,1, )

holds where the structural risk Rstuct ( ) depends on the sample size, 1 ,the

confidence level, Tr and the capacity, h, of the target function. The Rs,,, expression is

as follows 20:

Sh, h(41n(21 /h)+4)- ln( / 4)
Rstmct (h,l, -)=1

The capacity, h, measures the expressiveness of the target class of functions. In

particular, for binary classification, h is the maximal number of points (k) that can be

separated into two classes in all possible 2k ways using functions in the target class of

functions. This measure is called the VC-dimension. For linear discriminant functions,

without additional assumptions, the VC-dimension is h = n + 1 107, 20. The empirical

risk is measured by a loss function on the set of examples ? as follows 91:


Remp (U) = L (x, g (x,, .a)).

Since we cannot directly minimize R (a) the structural risk minimization principle

instead tries to minimize Rbound (a). It is almost always the case that the smaller the VC-

dimension, the lower this bound.

4.2 Support Vector Machines

Support Vector Machines (SVM) are growing in popularity rapidly in part because

both theoreticians and applied scientists find them useful. SVMs incorporate ideas from

many fields of study including applied mathematics, operations research, machine

learning, and more. Based on Statistical Learning Theory, early research suggests that









SVMs have had good success with supervised learning. They have compared well with

other learning algorithms such as Neural Networks, k-Means, and Decision Trees 20.

Joachmis 46 used SVMs to categorize news stories. Pontil and Verri 78 used SVMs for

object recognition (independent of aspect). Cortes and Vapnik 23 tested SVMs on hand

written zip code identification, getting accuracy just shy of human error. Brown et al. 12

applied SVMs to the problem of classifying unseen genes with success.

Support vector machines determine a hyperplane in the feature space that best

separates positives from negative examples. Features are mappings of original attributes

(we discuss this shortly). The margin of an example (u', y') with respect to a hyperplane

(w, b) is A' = y'((w, u) + b) where w is a weight vector is and b is a bias term. The

margin about the hyperplane A is the minimum of the margin distribution with respect to

a training sample S. The VC-dimension is bounded by


h <1+min n,R2 ]


where R is the radius of a ball large enough to contain the input attribute space. If a

margin is large enough, the VC-dimension may be much smaller than n + 1. SVMs learn

by maximizing the margin which, in turn, minimizes the VC-dimension and, usually, the

bound of the risk functional.

This distinguishes them from other popular methods such as neural networks which

use heuristic methods to help find parameters that best generalize. In addition, and unlike

most methods, SVM learning is theoretically guaranteed to find the best such linear

concept, if the data are separable. Neural networks, decision trees, etc. do not carry this

guarantee leading to a plethora of heuristic approaches to find acceptable results. For









example, most decision tree induction use pruning algorithms that try to create the

smallest tree that produces an acceptable training error in the hopes that smaller trees

generalize better (this is the so called Occam's razor or minimum description length

principle) 80. Unfortunately, there is no guarantee that the tree produced minimizes

generalization error. SVMs also scale-up to very large data sets and have been applied to

problems involving text data, pictures, etc.

The SVM is formulated as a quadratic optimization problem with linear inequality

constraints. Below is the primal formulation assuming the data is separable.

min(w,w)

st

y'((w,u')+b)>1, i=1,...

(w, w) is minimized in the objective function in order to maximize A, thus potentially

minimizing the bound on the VC-dimension which was expressed above. This can be

explained as follows. We replace the functional margin with the geometric margin. The

geometric margin will equal the functional margin if the weight vector is a unit vector.

Thus we normalize the linear function y'(w, u) + b)] and A = because
w w w

the inequality will be tight at a support vector. In order to maximize A we merely

minimize w .

This problem has a dual formulation. The dual solution is useful as w is no longer

explicitly computed and the explicit usage of the data points is collapsed into a matrix of

inner products, allowing for higher, possibly infinite dimensional feature spaces. These









feature spaces are implicitly calculated by a kernel which we explain in great detail

below. The dual formulation is:

1-
max W(A)= A' 2- y'yAA' (u',uJ
1=1 z,j=1

st


y'A' = 0
1=1

A >0 i =1,...,

where A are the dual variables. w is no longer in the formulation and all data appears

inside the dot product, which is key to using kernels in the SVM.

A kernel is an implicit mapping q! of an input attribute space X onto a potentially

higher dimensional feature space F. The kernel improves the computational power of

the learning machine by implicitly allowing combinations and functions of the original

input variables. For example, if only price and earnings are inputs, a PE ratio would not

be explicitly considered by a linear learning mechanism. A kernel, properly chosen,

would allow many different relationships between variables to be simultaneously

examined, presumably including price divided by earnings. The PE measure is termed a

"feature" of the input variables. There are many powerful, generic kernels 20, 38 but

kernels can also be made to suit a specific application area as we do later in this study.

Some application areas are sensitive to periodic changes, making correct pattern

recognition more likely with the usage of time series analysis. Ruping 87 shows how to

extend a number of kernels to handle time series data. Jin, Lu, and Shi 45 show that the

right subset of attributes for a particular domain is important to time series classification

for knowledge discovery applications. Their methodology trimmed the attributes to









include only data pertinent to the domain's time series. Preliminary research suggests

that kernels which are constructed with the help of application specific information tend

to have better results 20.

4.3 Kernel Methods

A kernel is a central component of the SVM. Shawe-Taylor and Cristianini call it

the information bottleneck of the SVM 95. This is because all data input into a SVM goes

through the kernel function and ends up in the kernel matrix. The kernel matrix is a

matrix with entries Kj =< q(u'), ~(u)) >, where q is a mapping S: X -> R", and

u', ui E X. Often the dimension of the feature space is much larger than the attributes

space, and may even be infinite (ref. the Gaussian kernel in Section 4.3.1). Key to the

value of kernel methods is the ability to implicitly capture this feature space via a

mapping qS. The dual formulation expressed in Section 4.2 can be generalized to allow

the usage of kernels as follows:


maxW(A) = -' y'yJA'AJK(u',u')
1=1 I=1

The kernel function is an inner product between feature vectors and is denoted as

K(u, v) =< 0(u), 0(v) > where {u, v} e X. The feature vectors may not have to be

explicitly calculated if the kernel function can create a mapping implicitly. In Section

4.3.1 we show how a kernel can increase the dimension of the attribute space, thus

allowing for more unique features, without significantly increasing computational cost.

An alternative to using a kernel is to explicitly create all features deemed necessary for

classification as direct input to the SVM as attributes. However, this is both time

consuming and computationally costly. Creating a kernel unleashes the potentially









nonlinear power of the learning machine, allowing it to find patterns on the attributes that

were previously unknown. In Section 4.3.1 we explain the properties of general kernels.

In Section 4.3.2 we extend our explanation of kernels by considering domain specific

kernels. These kernels are designed with the structure of a particular domain in mind.

4.3.1 General Kernel Methods

As explained above, a kernel is evaluated within an inner product between

mappings of examples u', where examples are vectors of attributes from the instance

space X. There are many known kernels and the list is growing 214687. Two specific

kernels can be used to illustrate the nature and expressive power of these functions. The

polynomial kernel is:

K(u, v) (K(u, v)+ R)d

where K(u, v) is the normal inner product < u, v >, d is a positive integer and R is

fixed. Consider a set of examples S = ((u, y'),...(u', yL)) c (X x Y) each with four

attributes, u' = (u,,u,2, u,3, u,,)' and v' = (v,,,V,2,v ,,v,4)', with d=1 and R= 0.

K(u, v) = uv, + u2v + uv3 + 4V4.

and with R= 0 and d =2,

K(u, v) = (u1Vl +u2v2 +u3v3 +u4v4 )2.

While K(u, v) has four features, namely (u1,u2,u3,14)', K(u, v) has 10 features, namely

all monomials of degree 2, or (u1, u2, u, u, 2uu2,2u 1u3,2uu4, 2u12u3,2 u2u4, 2u3u4

Consider a d of arbitrary dimension with n attributes, the number of features is

n+d-atinaloplibeoeunra a and
d ). The computational complexity becomes unreasonable as n and d grow.









Due to the implicit mapping in the polynomial kernel (between examples via the inner

product), the monomials of degree d can be features of an SVM without their explicit

creation.

An even more powerful kernel is the Gaussian, which is defined as:

K(u, v) = exp(- u v /(2&2)), where is the 2-norm 91 and a is a positive

parameter.

An exponential function can be approximated by polynomials with positive

coefficients, making the Gaussian kernel a limit of the sum of polynomial kernels 95.

The features of the Gaussian can be best illustrated by considering the Taylor expansion

O 1
of the exponential function exp(x)= -x' 95. The features are all possible monomials


with no restriction on the degree. This feature space has infinitely many dimensions.

Now that it is obvious that kernels are a powerful tool, we will look at their

properties. To be useful in SVM work, a kernel function must have the following

minimum characteristics (Cristianini and Shawe-Taylor 20):

(1) the function must be symmetric (ie K(u, v) = K(v, u))

(2) the function must be positive semidefinite, and

3. the function must obey the Cauchy-Schwarz inequality.

#1 is easy to check. #2 is a little more complicated and it is usually determined by

studying a related square-symmetric matrix, A, and its eigen decomposition. Let

K(u, v) be a symmetric function on X. K(u, v) is a kernel function if and only if the

matrix A = (K(u, u,)),, is positive semi-definite (has non-negative eigenvalues) 20. #3

is satisfied as long as the function obeys the Cauchy-Schwarz inequality. The Cauchy-









Schwarz inequality as applied to kernels is defined by Cristianini and Shawe-Taylor 20

as:

K(u, v)2 = ((u), O(v)) < \(u)2 I(v) 2= < <(u), O(u)>


= (&(u), 0(u)) (O(v), O(v)) = K(u, u)K(v, v)

A kernel function often alters the dimensionality of the data, mapping it into feature

space. The inner product between all feature vectors is carried out using a kernel matrix.

A matrix formed by such inner products is called a Gram matrix. The Gram matrix has

some useful properties, for example it is positive semidefinite. Since all of the entries in

the Gram matrix are in the form of an inner product, we must be concerned with their

proper existence. An inner product space is a vector space endowed with an inner

product. The inner product is actually the metric used to determine the distance between

two points. The inner product space is enough structure to properly define each element

of the Gram matrix when considering the finite dimensional case. However, if we want

to take advantage of an infinite dimensional feature space (as in the Gaussian case) we

need the inner product to define a complete metric (defined below). If the inner product

defines a complete metric, then it is a Hilbert space 59. A complete metric is one in

which every Cauchy sequence is convergent. Consider all countable sequences of real

numbers. The Hilbert space is a subset of all countable sequences x = {x,, x,..., x ,...


such that x 2= x2 < o The inner product of sequences can be defined as

his infinite space is also called L 59 An important characteristic of
(x,y) = x, y,. This infinite space is also called L2 59. An important characteristic of
1=1









an Hilbert space is that it is isomorphic to R" in the finite case and L2 in the infinite

case.

A compelling property of kernel methods is the ability to form new kernels from

existing kernels. For example, one could take a polynomial and a gaussian kernel and

add them up to get the features from each. Kernels are also multiplicative. Cristianini

and Shawe-Taylor 20 show that the following functions of kernels are in fact kernels:

K(u, v) = K(u, v)+ K2(u, v)

K(u, v) aK, (u, v)

K(u,v)= K,(u, v)K2(u, v)

K(u, v) = K3 ((u), (v))

where K,, K2, and K3 are kernels, q: X -> R" and a > 0.

Until this point, we have looked at two kernels, the Polynomial and the Gaussian.

These kernels are very powerful, but offer little opportunity for crafting kernels that are

specific to a domain. The graph kernel is a general kernel which can be made domain

specific, as long as certain rules are followed. A powerful characteristic of the graph

kernel is its intuitive nature. The graphic representation allows us to better understand

how a kernel works. Before formulating this kernel, a simple example is useful.

Consider a graph G(A,E) with nodes a and edges e. See the Figure 2 below:












S & el e 2

se3 e4



e e5



Figure 2 Basic Graph Kernel

All e, in this graph are base kernels (for example, a polynomial kernel on a

component of the attribute space). To differentiate base kernels from general kernels, the

base kernels are denoted as K(u,,v) Any path from s to t is a feature. This feature is

arrived at via the product of all edges in the path between s and t. In general, all paths

from s to t create features. This allows the researcher to create his own kernel, by

choosing the structure of the graph.

Here is a more formal explanation of the graph kernel. It is based on a directed

graph G with a source vertex s of in-degree 0 and a sink vertex t of out degree 0. A

directed graph is one where the flow on each edge is in a single direction. Each edge is

labeled with a base kernel. It is assumed that this is a simple graph, meaning that there

are no directed loops. In general, loops are allowed but that makes proving that the

resulting mapping is, indeed, a kernel extremely complicated. Takimoto and Warmuth

100 proved that a directed, acyclical graph with base kernels on the edges is indeed a

kernel.









Shawe-Taylor and Cristianini 95 describe the kernel as follows: Let P, be the set

of directed paths from s to t for a path p = (aal ...ad). The product of the kernels

associated with the edges of p can be seen as follows:

d
K (u, v)=n K( (,)(,v) .
1=1

The graph kernel is the aggregation of all Kp (u, v) and can be seen as follows:

d
K(u,v)= K,(u,v)= n iK(,,-,,)(u,v)
PGPt P P .t 1-

Here is another example for clarification. Look at the Figure 3 below. It is a

slightly more complex version of the one above. The nodes are labeled for explanatory

purposes and the edges are labeled with the base kernel K(u, v) )= (u, v. If s = 1 and

t = 2, then there would be a single feature, u1. If s = 1 and t = 3, there would also be a

single feature uIu2, but the feature would be the product of the two base kernels on the

path p = (alazas). Three paths converge at node 5, specifically p, = (alazalas) and


P2 = (alaza) and p, = (alaza4as). Node 5 can be seen as a kernel which sums the

products of the base kernels on each path. If Node 5 were t, the output would be the sum

of all paths into node 5, p, + p2 + p, or u1u2u5 + u1u4 + u1u3u6. In general, at each node a

(except s), all paths from s to a are summed. The contribution of a path to the kernel is

based on the product of its edges.













U3V3 U 4V4 a55V5
sa>----------^



3 6V64 U7V7



Figure 3 Graph Kernel

4.3.2 Domain Specific Kernels

A kernel should have two properties for a particular application. First, it should

capture a similarity measure appropriate to the domain. The features that offer the most

information content for a particular domain need to be represented by the kernel. Second,

its evaluation should require significantly less computation than would be needed by

using the explicit feature mapping 95. The first point is key to the contribution of this

dissertation. General kernels are building blocks but the goal of a kernel method is to

determine patterns correctly and tuning a kernel to a specific domain best does this.

Much empirical research has been done where a dataset is tested using several kernels

and results are given as to which kernel performs better. This is ad hoc. It seems likely

that a kernel which is tuned to a domain will better capture the features necessary to

correctly classify instances in that domain.

Ultimately we will combine kernels that deal with quantitative financial

information and textual information. Below is a brief summary of some text-based

kernels.









Joachims 46 uses the polynomial and gaussian kernel for text categorization. He

compares several parameters for each kernel (d for polynomial and C for Gaussian).

He showed that the parameter that elicited the lowest estimated VC dimension was the

one with the best performance on the empirical tests. Thus, he has tailored general

kernels to the text domain. This is an early and simplistic example of domain specificity.

Cristianini et al. 21 develop a Latent Semantic Kernel designed to sort documents

into categories by keywords, which are automatically derived from the text corpus. The

kernel implicitly maps keywords into a "semantic" space, which allows documents which

share no keywords to be related. This is accomplished by analyzing co-occurrence

patterns. A co-occurrence pattern is where two terms which are often found in the same

document are considered related. The co-occurrence information is extracted using a

singular value decomposition of the term by document matrix. This paper illustrates the

usage of domain knowledge in the development of a kernel.

Another kernel adaptable to text problems is the string subsequence kernel 20. A

string is a finite set of characters from a set T. In the case of a subsequence kernel, T is

the alphabet. The goal of this kernel is to define the similarity between two documents

by calculating the number of subsequences these documents have in common. The

subsequences do not have to be contiguous. However, there is a penalty incorporated

into the function based on the distance between words of a subsequence.

Early researchers in kernel methods have given us several general forms with

which to work. Recent applications of kernel methods to domains include protein

folding, handwriting recognition, face recognition, image retrieval, and text retrieval.

Finding the right kernel for a particular problem has proven to be an ad hoc, yet









extremely important step. The real power of kernels is harnessing those general forms to

create kernels that are specific to these domains. This work has just begun. A domain

may be defined by more than one type of data, thus complicating matters. In the case of

the accounting domain, both quantitative and text attributes contain information on a

firm. In order to utilize the text data, we must first understand how to narrow down our

potential attributes, by looking at text specific to the domain of accounting.

The next Chapter utilizes the methodologies reviewed in this Chapter to create a

Financial Kernel. A review of Chapters 2 as well as this Chapter should give the reader

an understanding of the reasons for the particular design of the Financial Kernel in

Chapter 5.














CHAPTER 5
THE FINANCIAL KERNEL

Defining a domain specific kernel for finance entails looking to the finance and

accounting literature to see what attributes and features are often utilized for

classification. It also requires us to consider the kernels available and which ones would

fit our work the best. As this work focuses specifically on financial events the main

publications reviewed were in the realm of management fraud and bankruptcy, as seen in

Chapter 2. Without fail, most financial analyses look to ratios of items on the financial

statements. Models for earnings quality in accounting utilize ratios, such as the study by

Francis, LaFond, Olsson and Schipper 35. Loebbecke, Eining and Willingham 55 use

financial ratios as part of their management fraud model as well. All of the studies

detailed in Section 2.2 on Bankruptcy Detection use financial ratios.

McNichols and Wilson 56 used year-over-year changes in key account values to

help determine earnings management. Francis, LaFond, Olsson and Schipper 35 utilized

year-over-year changes extensively in their study on earnings quality. Beneish 10

utilized year-over-year changes to help determine management fraud. The majority of

the bankruptcy prediction methods which were reviewed in Section 2.2 show the

accuracy of their methodologies for the year of bankruptcy, the year prior to bankruptcy,

and sometimes further back. As years prior to bankruptcy increase, the predictive

accuracy of the models decreases. In general, the picture is not clear. However, a trend

may be emerging. This trend can be captured by year-over-year changes in key ratios. As

explained in Section 2.2 Altman 5 notes that a limitation on his discriminant analysis









function for bankruptcy detection was its lack of year-over-year changes. Year-over-year

changes in ratios are captured by this function:


U1 2 _U11

U 2
j2

where i, j = ...n are the attribute numbers and the second subscript is the year (or

period).

We created two kernels to handle ratios and year-over-year changes. The first

kernel utilizes the polynomial kernel structure on a mapping of the data to produce

inverses. Recall, the general polynomial kernel is K(u, v) = (K(u, v) + R)d where R is a

constant and d is the degree of the polynomial. We apply the polynomial kernel to a

mapping of the input attributes 0(u) i>i, where u = (ul,u2,..., u,)' and



u= U1, U2,...,Un,-, -,. "


Setting R to zero and d = 2, K(u, v) =< 0(u), q(v) >, gives all possible ratios of


individual attributes In addition, we get the following attributes: uj2 and This
Ui UlUi

can be seen in a simple example. Consider u = (u1,, 2, u3)' and v = (v1,, v2,v)', for all


111 ia111n
u,vGX. Ob(u)=\U,2 ,-,- and U(vU)= v, 1 1 1 V, ,
1, u2, u3 V v V2 V3










The function result is:

K(u, v) = (< (u), (v) >)2
=2ulu2vv + u2v2 +2+ +2 v2 +22uu3 1V3 +2u2U3V2V
1212 1 22 3 3
+2u1v 2uv 2 u 2u3 2uv3 2uv2 2
+ + + + + +
u3V3 u3v3 1,V,1 u22 u1V1 u1u212Vv

2 1 1 1
+ + + -
23223 12 222 2
u2u3V2V3 U1VI U2V2 U3v3

which gives the following feature vector:



,F 2 1 2 3 2 123 2' 2 2 2
11 2' u u ^ u ^ uu u 2 3 1
U2 U3 U3 U1 U2 U1 U1U2 U2U3 U 1 U2 U3
We validated this kernel on simulated data. We used the Altman Z-Score with

weights for the manufacturing industry (ref Ch. 2). We created attributes for each

variable in the Z-score. The attributes were TA, EBIT, RE, B.V.E., TL, WC, as defined

in Section 2.2. The attribute values were created using a normal distribution with means

and variances appropriate to the domain. When we created the variables we preserved

the structure of the balance sheet (i.e. TA = TL + B.V.E. + RE). Each example was input

into the Altman Z-Score function to obtain its score. The examples with scores were

sorted by score. The top 50% of scores were labeled with a +1 and the bottom 50% of

scores were labeled -1. We only input the attributes and labels into the SVM. We were

able to separate perfectly on the Altman Z score, but had problems rediscovering weights

from the actual function. We determined this is due to the fact that many extra features

are created by this kernel and are highly correlated with each other. This correlation is

due in part to the structure of the Altman Z score. Total Assets and Retained Earnings

are two of the six attributes used in creating the ratios of the Altman Z-score. Both of










these attributes are used in two different ratios. Our kernel creates (2n)d features, some

of which did not add a significant amount of information to the learning algorithm (i.e.


uj and 1).
uu

To add a time series representation for this kernel, we would have to represent the


U12 U11 21

following relationship: -=2 J 1 uu The left hand side of this function is
U,2 U/U,12
uj2

the year-over-year changes as explained above. The right side is a representation that can

be constructed by our kernel by dropping the constant. The attribute vector would double

in size as the second year would be concatenated onto the end. In order to get year-over-


year changes in this format '2-- we need d > 3. The number of features in the year-
2j1 12

over-year case would be at least (2(2n))3. Even a modest number of attributes causes a

huge explosion in features. The n = 4 case would generate 4,096 features.

The second kernel we created was built as a response to the problems we had with

the first, namely dimensionality explosion and unnecessary features. We design this

kernel with the goal of getting all the important features, including all possible intra-year

ratios and year-over-year ratios. However, we want to avoid the problem of unwanted

features.

For this we chose the graph kernel. As discussed already, the graph kernel is

extremely flexible, which makes it a natural choice when trying to construct specific

features. We exploit the research of Takimoto and Warmuth 100 to build this kernel. We









call this kernel the Financial Kernel, and denote it as KF (u, v). KF (u, v) is a directed

graph G e (A,E) with base kernels on all edges e and K(u, v) on a. The Financial

Kernel has as input n attributes per year for 2 years. The attributes vector is

u= (u11,...,ul,u22,..., un2)', where the first index is the attribute number and the second

index is the time period. See Figures 4 and 5 for an illustration of financial domain

kernel. Figure 4 illustrates one of n -1 graphs that make up the Financial Kernel. Each

of the n -1 graphs has a source node s, and a sink node t,. The graphs decrease in size

with n. The reason is that each graph carries information for attributes i through n.

Each path from source to sink is a feature. The number of features are equal to the

number of paths. All n 1 graphs from Figure 4 are brought together by the graph in

Figure 5. The paths from s to t make up all of the features in KF (u, v).

The kernels on e are base kernels. As defined in Chapter 4, a base kernel is a

kernel function on a vector component. We can have as many different kernels as there

are edges. For the creation of a financial kernel, we limited the base kernels to two

forms, one is the standard inner product kernel of K(u,, v,) =< u,, v, > the second is


K(u,,v,)=1
UIVI

































/ 1 1 1
1 1F


1 4(+1)1 ( 1)1 0(a+2)1 (n+2)1 nlnnl







Figure 4 The Financial Kernel 1














\ s! G,, -
92 t-





Figure 5 The Financial Kernel 2

According to Takimoto and Warmuth 100, in order to prove that K (u, v) is a

kernel, we need only have a directed graph without cycles and show that each edge e is a

valid kernel. (For details of their proof, see pg. 33 of Takimoto and Warmuth 100.)

Examination of Figures 4 and 5 clearly show that the graph is directed and free of cycles.

We need to show that both K(u,, v,) and K(u, v ) are kernels. K(u,, v,) is simply the

standard inner product kernel. K(u,, v ) can be shown to be a kernel as follows:

(1) f(u)= u ,i ...n

(2) f(v,)= v,',i =l...n


(3) f(u,)f(v,) = u ,1=-
uiv

(4) By Cristianini and Shawe-Taylor [2000] (pg. 42) 20 K(u, v,) f(u, )f(v,)


The features of the Financial Kernel are: b(u)= u, Uj2 ,2i,j = 1...n,i uJ He2 1 12

Here is a small example. In this example u = (u I,u21 ,U12, 22)' and


v= (v11,v21, 12, 22)'.







50


1111 u 22 22 u 11 V11 22V22
KF (u, v) + +
u21v21 u12v12 U21V21U12V12

which gives the following features:


U11 U 22 U11 U22
U21 U12 U21 U12


In general, for year 1 we get all ratios in the form of u. In year two we get all
Ui
Uu

ratios in the form of which is the inverse of year 1. We structure the ratios in this
U


form in order to get year-over-year changes of the form uu
UJIU12

The feature space we have constructed so far with intra-year ratios has the structure


u' and j. It is evident that with this kernel we get the feature or its inverse. In other
J 1 ,12


words, if the true feature is u- this mapping only gives the inverse. By constructing the
U1

features in this manner, we reduce the dimensionality necessary to get year-over-year

changes, but we lose a potentially important set of features in the process. For the year-


over-year changes all we need to do is get the product of the intra-year ratios u' and
UJu


SThe computational complexity of the Financial Kernel is 3n(n--- for n
u2 2 2

attributes and 2 periods. This is easy to see as each pair of attributes i, j are represented


U, I u uj2 1UU, I j n(n-1)
three times, U 2 UJ and the number of attribute pairs are --
u\ ul 2 u1u 22 2









We validate the financial kernel on simulated data to test the kernel's ability on

inputs of a known function. We take the Altman Z-Score and modify it slightly, to add a

time series component.

The function we create is:

6.56 ((CA CL)/ TA) + 3.26 (RE/TA), + 6.72* (EBIT / RE), +1.05 *(BE / TL), +


6.56 *((CA CL)/ TA), + 3.26 (RE/TA)2 + 6.72* (EBIT / RE)2 +1.05 (BE / TL)2 +


2 ((CA CL)/TA), *(TA (CA CL)) + 3 (RE/TA) (TA /RE)2 + 3* (EBIT/RE), (RE/ EBIT)2 +
(BE/TL), *(TL/BE), = score

The first and second rows of this function are year 1 and year 2 individual Altman

Z Scores. The third row is year-over-year changes in the ratios of the Altman Z-Score.

The weights on the year-over-year changes were chosen arbitrarily.

Our dataset contains 2,000 randomly generated examples labeled with the modified

Altman Z-score function. We divide the examples up by sorting the data on the score.

The threshold value for our modified Altman function is chosen as a midpoint between

the score of sorted item 1,000 and 1,001. Thus all of the top-half are labeled as +1 and all

of the bottom half as -1. We run experiments using the financial kernel, a polynomial

kernel of degree 2, a Gaussian kernel, and a linear kernel. The results are as follows:

Table 1 Financial Kernel Validation
SV Test on Train 10 fold cross validation
Linear 877 85% 84%
Polynomial (deg 2) 1998 75% 55%
Gaussian 1056 86% 86%
Financial Kernel 707 92% 91%

The results show that the Financial Kernel achieves superior results when using 10-

fold cross validation. The first column is the number of support vectors. A bound on










#SV
generalization error is The generalization error of the Financial Kernel is the


lowest of the listed kernels. The result is not quite as expected though. One would

expect the Financial Kernel to achieve perfect separation. The reason for the error is an

assumption we made when developing the kernel. The assumption was that we could

U U u U
represent both of the following ratios by only one of and In order to get
Ui U, Ui U,

U, U
perfect separation, we hypothesize that we need to have both and as features.
U U,

This has been easily achieved by adding a mirror image of Figure 4, with the components

being inverses of the components of Figure 4. Figure 6 shows the updated Financial

Kernel.

This Chapter detailed the development of the Financial Kernel, one of the two main

methodological contributions of this research. In Chapter 6 the development of the

Accounting Ontology is explained.


































n^l nl


1

a2Vn2


i2i2 E il l


Figure 6 Updated Financial Kernel


n2vn2


1

,IVnI














CHAPTER 6
THE ACCOUNTING ONTOLOGY AND CONVERSION OF DOCUMENTS TO
TEXT VECTORS

We describe a methodology for creating an accounting ontology in Section 6.1.

Section 6.2 describes how the ontology is used in conjunction with the vector space

model to turn accounting documents into text vectors.

6.1 The Accounting Ontology

The accounting ontology is built using an accounting corpus to represent the

accounting domain and general corpora to represent the general domain. The accounting

corpus is the US Master GAAP Guide 16. We chose this because it explains generally

accepted accounting principles in a fairly non-technical manner. It uses all the

terminology, but in more regular language than a legal publication. We get our general

corpora from the Text Research Collection, which is syndicated by TREC 103. This

collection includes material from the Associate Press, The Wall Street Journal, the

Congressional Record, and various newspapers. The Text Research Collection has been

used in many natural language processing applications and is often used to test IR

methodologies.

A domain specific ontology is created by a series of major steps, each with its own

series of minor steps. Figure 7 shows how the ontology is created. There are two classes

of corpora, the domain corpora and the general corpora. Both are part-of-speech tagged

and fed into the function that determine which concepts are germane to the accounting

domain, as described in Step 1 below. A set of concepts and other domain specific terms









called Novel Concepts are put through a process that uses the syntactic structure of the

accounting corpora, as described in Step 2 below. The result of this step is a WordNet

enriched with novel terms from the accounting domain. The final step in ontology

creation is to add new multi-word concepts to WordNet based on an algorithm that uses

the syntactic structure of domain concepts, as described in Step 3 below. The details of

each step are explained in the remainder of the section.

6.1.1 Step 1: Determine Concepts and Novel Terms that are specific to the
accounting domain

We start with a part-of-speech (POS) tagger used to tag the natural language text.

This puts additional structure on the individual words. The POS tagger used is a

derivative of the Brill tagger, called MontyTagger 110. The tagger is run on both the

accounting corpus and the general corpora. The POS tagged data is culled down to the

following form:



Word 1 #POS#WordCount

Word2#POS#WordCount

WordN#POS#WordCount





Word 1 #POS#WordCount

Word2#POS#WordCount







































Domain
Concepts
enriched with
Novel Terms
(Step 2)


Add Multi-
word
Domain
Concepts
(Step 3)


Figure 7 Accounting Ontology Creation Process











WordN#POS#WordCount





Word 1 #POS#WordCount

Word2#POS#WordCount

WordN#POS#WordCount





Word 1 #POS#WordCount

Word2#POS#WordCount

WordN#POS#WordCount



where Word#POS#WordCount is as follows:

Word Stemmed Word

POS Part of Speech of Word

WordCount Number of times a word appears in a document.

The word counts are run through a function in order to detect words that have the

highest amount of information for that particular domain. For example, when

considering the accounting domain versus a general domain, the word "defeasance" will

have a higher score for the accounting domain because it is specific to accounting, while

the word "balance" will have a lower score as it can be found equally in the accounting









and the general domain. The function used is a modification of the basic TF-IDF

function which includes WordNet Concepts.

N
Recall the basic TF-IDF function: w, = (tf) log where w is the weight of
n

term tj in document d tfj is the frequency of term t, in document d N is the number

of documents in the collection and n is the number of documents where the term t

N
occurs at least once 44 The inverse document frequency (idf), = log .
n

We modify the TF-IDF as follows:


(1) rlv(t I d) =log(tf,dlo N


(2) rlv(c d) = rlv(t I d)
tEC


(3) rlv(c, Id)= rlv(t d)T



(4) max rlv(c,+ d)= rlv(t d)
c I^c,+ C, )

Function (1) mimics the basic TF-IDF function, only the d stands for domain

instead of document in our research. rlv is the domain relevance of a term t on domain

d N is the number of domains. In function (2), we introduce c for concept, where

c = {t, ,..t.,t. }. This introduces the notion of a synset. By considering the relevance of

terms and their synonyms, we get a clearer understanding of the domain. Function (2)

sums up the relevance rlv for all terms t in the synset c. This is a concept relevance

score. Function (3) sharpens this, considering hyponyms. The c. is the concept,









unioned with all of its direct hyponym sets. Recall, the hyponym of a concept i is a

concept j which is a specific instance of concept i. For example, a bank account is an

account. Function (3) sums up the relevance rlv for all terms t in c and all of c's direct

hyponyms. Looking at the direct hyponyms gives us one more measure of a concept's

T
relevance. Function 3 adds an additional term T where T is the total number of terms


t, within a concept c which are found in the domain corpus. The c is the cardinality of

the concept. This set of functions was developed by Sacaleanu and Buitelaar 15.

We add a measure of word sense disambiguation in Function 4 by comparing the

domain frequency of various senses of a term t. In other words, consider concept c,

with terms (t, t, t3) and concept c, with terms (t3,t4, 5). Notice that t3 is in both c,

and c2. We determine which concept t3 actually belongs to by comparing the Function 3

scores of the two concepts. We choose the concept (c, or c2) which achieves the

maximum value in Function 3.

Here is an illustrative example. The noun "stocks" in WordNet has 17 different

senses (definitions). Listed below are 4 of the 17 senses.

1. stock -- (the capital raised by a corporation through the issue of shares entitling

holders to an ownership interest (equity); "he owns a controlling share of the company's

stock")

=> capital, working capital -- (assets available for use in the production of

further assets)

2. broth, stock-(liquid in which meat and vegetables are simmered; used as a basis

for e.g. soups or sauces; "she made gravy with a base of beef stock")









=> soup-(liquid food especially of meat or fish or vegetable stock often

containing pieces of solid food)

3. stock, inventory-(the merchandise that a shop has on hand; "they carried a vast

inventory of hardware")

=> merchandise, wares, product-(commodities offered for sale; "good business

depends on having good merchandise"; "that store offers a variety of products")

4. livestock, stock, farm animal-(not used technically; any animals kept for use or

profit)

=> placental, placental mammal, eutherian, eutherian mammal-(mammals

having a placenta; all mammals except monotremes and marsupials)

Senses 1 and 3 are much more likely to come up in an accounting context than

senses 2 and 4. In order to test which sense is the most likely sense in the context of a

document or corpus, we compare relevance scores, which include for Sense 1 "stock" and

its hyponyms "capital" and "working capital", with Sense 3 "stock", "inventory", and its

hyponyms "merchandise", "wears", and "product". The Sense with the highest relevance

becomes the candidate to be a domain specific concept.

All word sense disambiguated concepts are sorted based on score, and the highest

scoring concepts become domain specific concepts. Novel terms are those terms that

have high scores but do not fit into a WordNet category. These terms are very important

as they give us an opportunity to enrich WordNet with domain knowledge.

WordNet can be viewed as a hierarchical tree where the nodes are concepts and the

edges are relationships. Figure 8 shows a simplified WordNet tree after Step 1. In this

tree accounting domain concepts are filled in with the color gray. We also show that









there is a listing of novel terms which are highly important to the accounting domain but

cannot be matched to current WordNet concepts. These terms are the subject of Step 2.





Novel Term List
Term 1
Term 2
Domain
Concept Term n




0 Domain
Do n Concept
Domain
Concept

Figure 8 WordNet Noun hierarchy with Domain Concepts

6.1.2 Step 2: Merge Novel Terms with Concepts

In this step, we take the novel terms that were not matched to concepts and we

attempt to fit them into a domain concept. We use the methodology of Buitelaar and

Sacaleanu 14. This process is done using lexico-syntatic patterns. Consider the natural

text, before preprocessing. In such a text there are certain syntactic patterns that arise,

such as [determiner, adjective, noun, verb, noun]. A sentence with this structure would be

"The large crane eats breakfast." The "the" is a determiner, large is an adjective (ADJ),

crane is a noun (NN), eats is a verb (V) and breakfast is a noun (NN). We consider the

syntactic patterns that arise in 7-grams, that is, contiguous 7 word structures. We look for

patterns with three words to the left and three words to the right of a central word. This

central word will always be either a domain concept (which includes all constituent

terms) or a novel term. The basic idea is as follows: look for patterns where novel terms









and concepts appear together, often interchangeably. A visual representation of a 7-gram

is below:

[null, null, ADJ, Term/Concept, null, V, NN]


This 7-gram has words in the following sequence: two words on the left that are

not important to us (signified by null), an adjective, the term or concept, another

unimportant word, and then a verb and a noun. The parts-of-speech we are concerned

with are nouns, verbs and adjectives. All other parts-of-speech are considered "null".

In order to determine patterns which are populated with words that are related, we

use a mutual information score based on co-occurrence. The sore is used to determine

the semantic similarity of two-word pairs based on how often pairs of words are found

together relative to chance. The mutual information score MI is the following function:



MI(c, w) = log2 tG- N
Y fQ, )f(w)


where c is a concept, w is a word in the pattern, and N is the total number of words in

the pattern. MI is an approximation of the probabilistic mutual information function:

log2 P(x, y)
P(x)P(x)

The details of the derivation of MI(C,w) can be found in 14.

In order to determine if a novel term belongs inside a particular concept we have to

first decide whether the pattern is reliable. We assume a pattern is reliable if all the terms

of a concept are assigned back to the concept, using an unsupervised clustering algorithm

called k-nearest neighbor. Below is the data structure for the example pattern above:









[c,MI, ADJ, V, NN].

As "null" attributes are unimportant, we simply leave them out. For reliability

testing we expect the concept c in its representation above to be clustered together will all

term instances, represented as [t, ,MI,ADJ,V,NN]. If this is not the case, the pattern is

considered unreliable.

For all reliable patterns, we use the k-nearest neighbor to cluster the concepts as

seen above together with the novel terms (NT) in the following representation:

[NT, MI, ADJ, V, NN].

If a NT is clustered with a Concept, then we add the NT to the concept, thus enriching

WordNet. Figure 9 shows the WordNet tree after Step 2. This Figure updates Figure 7.

The domain concepts which were found in Step 1 are shaded gray. Figure 8 illustrates

that after Step 3 some of the domain concepts include novel terms, thus enriching

WordNet.







Domain
Concept




Concept Nove terms {Concept u NovelTerms}



Figure 9 WordNet Noun hierarchy with Domain Concepts enriched with Novel Terms









6.1.3 Step 3: Add multi-word domain concepts to WordNet

At this point, we have domain concepts enriched with novel terms. We would like

to extend WordNet, by adding new nodes. We do this using a slightly altered form of the

method described by Vossen 108. Vossen utilized the header-modifier relationship to

determine multi-word concepts. For our purposes, a header is a noun and a modifier is

one or more adjectives describing it. For example, "bank account" is a two-word

structure with account as the header and bank as the modifier. Vossen considers all

header-modifier structures, limiting the final set to the ones above a statistical threshold

for a particular domain. We already have our domain concepts from Steps 1 and 2, so we

consider only header-modifiers where the header is one of the domain concepts.

If an instance of the header-modifier structure is considered statistically significant,

then it is added as a node below the header in the tree. This means it becomes a hyponym

of the domain concept. The potential for more than one layer exists. Consider the

following phrases, "federal tax expense" and state tax expense." Both of these multiword

phrases are actually line items on an income statement. "Expense" is a term specific to

the accounting domain. A "tax expense" is a term that belongs below "tax" in a

hierarchy. There can be an additional set of nodes below "tax expense" called "federal

tax expense" and "state tax expense." There can be any number of modifiers for any

noun, although it is likely that the number of modifiers will be between one and three.

The WordNet tree takes on new nodes underneath domain concepts. The new nodes are

the header modifiers deemed significant to the domain. Figure 10 shows a simplified

representation of the tree after Step 3. The figure shows (as Figures 8 and 9) the domain

concepts as shaded light gray. Additional nodes are added below some domain concepts.









This is represented by nodes which are shaded dark gray. These new nodes are the

domain specific multi-word concepts, added in this step.






























Figure 10 WordNet Noun Hierarchy with Domain Concepts, Novel Terms and Multi-
Word Concepts

6.2 Converting Text to a Vector via the Accounting Ontology

Above we developed an accounting ontology methodology. Now we use this

ontology to aid in detection of financial events by using it as domain filter to get rid of

unwanted noise. Recall that the output of the Accounting Ontology is a set of concepts

specific to the accounting domain, as well as relationships between those concepts. The

process of getting a quantitative form of a text vector is as follows: We input the

company reports in natural language and use a Part-of-Speech tagger 110 as a









preprocessor. The preprocessed document is then parsed down to word counts labeled

with a Part-of-speech (e.g. Word#POS#WordCount). Each word is compared to all

domain concepts. If a word fits into one of the domain concepts, its word count is added

to the vector. The power of concepts rests in the fact that all words inside a concept will

count the same. For example, the word "liability" has the following words as synonyms,

"indebtedness", and "financial obligation". All three of these words are part of the same

concept. If one document has the word "liability", a word count is placed in the index

reserved for the "liability" concept. If another document has the word "indebtedness" or

"financial obligation", a word count is placed in the index reserved for the "liability"

concept. Below is a class of concepts:

wC1, c,..., cj
W ECj
i = 1,..., |I c I
j = 1,...,n

where ci are concepts, w, are words and I cj I is the size of the concept set. The filtering

process leaves us with only concepts ci that are specifically related to the accounting

domain.

We take this reduction a step further by considering the relationships between the

concepts in the Accounting Ontology. We do this by utilizing the tree structure of

WordNet. We need a measure to determine the similarity between nodes (or concepts).

There is a vast literature on similarity measures, so we choose an off-the-shelf measure

that has proven to be among the best. Based on the work of Budanitsky and Hirst 13, we

choose the Jiang and Conrath measure, which has been shown to be more accurate on the

Miller-Charles 63 set than competing similarity measures. We create a similarity matrix,










comparing each of the concepts, where sy is the similarity between concepts c, and ci

for i,j e {1,2,..., n Here is a view of this matrix:


S11 S12 1 SIn
S21 S22 S2n


SIn S2n "' Snn

We input the similarity matrix into an agglomerative clustering algorithm 47. This

algorithm clusters the most similar items and shrinks the matrix. This algorithm is

iterative, in each run concepts which are less similar are added to existing clusters, so we

choose a parameter k where k is the minimum level of similarity with which two

concepts can be clustered. The clustered concepts c are called super-concepts sc .

c <_ sc where is the size. In turn, the total number of super-concepts sc are less than

or equal to the total number of concepts c.

There are two goals to creating super-concepts:

(1) The super-concepts are designed to cluster concepts that are similar, therefore

financial documents which share accounting super-concepts are more likely to be similar.

(2) The super-concepts allow us to shrink the size of an undoubtedly large vector.

This can help us avoid overfitting on the empirical data, which is possible due the small

datasets available for fraud and bankruptcy. Below is a class of super-concepts:

{sc, sc,,..., sc, }
C1 SCk

j = 1,...,mk
k= ,...,s

where mk is the number of such concepts in super-concept k.






68


Chapter 6 explained the methods used to develop the Accounting Ontology. The

procedure for converting text to a vector of numbers was also explained. In the next

Chapter the method of combining the text data with the quantitative data is detailed.















CHAPTER 7
COMBINING QUANTITATIVE AND TEXT DATA

In this chapter we combine quantitative and textual financial data for subsequent

analyses. We turn the text into a numeric vector as discussed in Chapter 6, here we

concatenate the quantitative form of text to the vector of quantitative financial data.

Since we will be applying a kernel to this concatenated vector, we need to expand the

financial kernel developed in Chapter 5.

We concatenate the text and quantitative attribute vectors as a single, partitioned

vector u =(u11,u 21 ..., unl 12u,22...,' n2 I 2n+ ,2n+2,,...,u )'. The Financial Kernel is

applied to u1 ,..., UnI, u 2, U22, un2 and the text kernel is applied to u2n+ ,..., um where

these m n values are the quantitative representation of text. This is a two step process.

(1) We create a graph kernel K, (u, v) for the text. (2) We add the text graph to the

Financial Kernel graph.

(1) Text Graph: The text kernel is a linear kernel K, (u, v) =< u, v >. We

show K,(u, v) in graph form (Figure 11):



U2n+1 V+1









Figure 11 Text Kernel









This is a directed graph with no cycles and each edge e is a base kernel. This

graph is a kernel by Takimoto and Warmuth [2004] 100 (pg. 33).

(2) Add K, (u, v) to KF (u, v) to create a combined kernel Kc (u, v). See Figure

12 below:


Figure 12 Combined Kernel

The text graph TG is added to the Financial Kernel. The addition of TG does not

alter the fundamental structure of the Financial Kernel graph. The graph is still directed

and still contains no cycles. Thus K (u, v) is a kernel.

A simple example illustrates the Combined Kernel. There is an input of 2

quantitative attributes for both years, u11,u21,u12, u22 and 4 text attributes,










U5,u6,u7,u8 .The input vectors are u = (u1, u21 12,u22 u5 u5,u6,u78)' and


V = (11, 21, V12, 22 I V5, V6, V7, V8 ) .

/1 1 22v22 u 1v11u 22'v''22
Kc (u, v) = + + u 2 2222 + 55 +U66 +U77 +U8V8
u21 21 u12V12 u21 21u12V12

with the following features:


U1,1 22 U 11U 22U
.u21 u12 u21 12

Other kernels could be used in place of the linear kernel, giving additional features on the

text. For this study it is not necessary due to the extensive preprocessing steps used

during the creation of the text vector.

This Chapter explained the method used to combine text and quantitative data.

Chapter 7 is the final chapter in the methodology creation. The following three Chapters

delve into the empirical research, testing, results and a conclusion. Specifically, Chapter

8 details the research questions, the three datasets used for testing (management fraud,

bankruptcy, and restatements), and the ontologies created. Chapter 9 gives the results

from the tests on the datasets. Chapter 10 gives a summary, conclusion and explanation

of future research.














CHAPTER 8
RESEARCH QUESTIONS, METHODOLOGY AND DATA

This chapter explains the research hypotheses, the data gathering methodology,

accounting ontology creation and data preprocessing. In Section 8.1 the Hypotheses and

test mechanisms are articulated. Section 8.2 outlines the Research Model. Section 8.3

explains the methods used for gathering data for the Fraud, Bankruptcy and Restatement

datasets. Section 8.4 details the ontologies created and Section 8.5 explains data

preprocessing.

8.1 Hypotheses

The main contributions of this research are threefold. (1) We have developed a

financial kernel that operates on quantitative financial attributes. (2) We have developed

an accounting ontology to aid in using textual data in learning tasks. (3) We have

combined these two kernels to simultaneously analyze quantitative and text information.

These methods will be tested to for their effectiveness in early detection of financial

events. Our first testable hypothesis is as follows.



Hypothesis 1: A support vector machine using the Combined Kernel, which
includes the Financial Kernel for quantitative data and the Text Kernel for text data
detects financial events with greater accuracy than quantitative methods alone,
including the Financial Kernel.



A series of tests are run on the financial events data, using the Combination kernel.

All available data, both quantitative and text is used. We use 10-fold cross validation as









a method for estimating generalization error. We compare the classification accuracy of

our method with the other methods as explained in Chapter 2, including linear

discriminant functions, and logit functions.

The concepts in WordNet include semantic relationships between individual words.

Developing an ontology specific to the domain of accounting allows us to utilize these

relationships when creating the text vector. The basic vector space model does not take

these relationships into account. The expectation is that the ontology driven text vector

will provide a better representation of accounting-related documents than the basic vector

space model.



Hypothesis 2: A Support Vector Machine using data from a text vector filtered
through the accounting ontology will detect financial events with greater accuracy
than a Support Vector Machine using only the vector space model.



Two tests are run on the financial events data, using the combination kernel. One

test uses a vector created by filtering the text through the accounting ontology. The other

is run using a vector of word counts. The results of the tests' 10-fold cross validation are

reported and compared.

Comparing the classification accuracy of the text and quantitative data allows us to

effectively compare the "information content" in the numbers against that of the text.



Hypothesis 3: Text filtered through an accounting ontology will detect financial
events at least as accurately as compared to pure quantitative methods.









Two tests are run on the financial events data, one using only the quantitative data

which is fed into the Financial Kernel and the SVM. The other uses only the text data in

the form of the concept vector which is preprocessed using the accounting ontology. The

concept vector is fed into the Text Kernel and the SVM. The results of the tests' 10-fold

cross validation are compared.

8.2 Research Model

In this section, the Research Model is explained. Figure 13 shows the process we

use to study the efficacy of our approach. The empirical analysis is carried out to test our

methodology. Starting on the left of the figure, we gather our dataset, which consists of

companies that were shown to be fraudulent and/or bankrupt. We match the fraud and

bankrupt companies with nonfraud and nonbankrupt companies based on year, sector,

and total assets. Once we have chosen the companies in our dataset, we gather

quantitative data from financial statements and text data from the 10Ks. The financial

data is converted into a vector of attribute values. The text data is filtered through the

accounting ontology and turned into a numerical vector using the counts of the concepts

in the ontology. The text and financial vectors are concatenated and run through the

combination kernel. An SVM using the combination kernel is used to determine a

classifier to distinguish the companies as fraud/nonfraud and bankrupt/nonbankrupt. The

financial vector is similarly processed using the financial kernel to get classification

results for the quantitative data alone. We compare the quantitative results against the

results for the text-only case by feeding the text vector into the text kernel SVM.
































Figure 13 The Discovery Process









8.3 Datasets

The data gathering methods are described in this section for the Fraud, Bankruptcy

and Restatement datasets. Text and quantitative data are gathered for all companies in

the datasets.

8.3.1 Fraud Data

Gathering fraud data is a task which requires considerable time and effort. The

main data sources are the SEC Accounting and Auditing Enforcement Releases 93 as

well as the Accounting and Auditing Association Monograph by Palmrose 73. The set

was limited to fraud which occurred no earlier than 1993. The extracted financial data

consists of financial statement figures for two years. The text data set consists of the text

portion of annual reports (10Ks). As the fraud dataset required both text and quantitative

attributes, any company which was missing either the text or quantitative attributes was

deleted from the dataset. The quantitative dataset is shown in Figure 16 of Appendix B.

The attribute definitions are as follows:

Ticker Company ticker for stock market
Label fraudulent (-1) nonfraudulent (1)
Ind Industry Number
Year 1st year of data collection
Salesyr[l,2] Sales
ARyr[l,2] Accounts Receivable
INVyr[l,2] Inventory
TAyr[l,2] Total Assets
OAyr[l,2] Other Assets
CEyr[l,2] Capital Expenditures

The attributes were chosen based on their reported occurrence in cases of fraud. A

secondary reason for choosing these particular attributes was the likelihood of getting

reported data. This is in contrast to other highly reported fraud attributes, such as

Advertising Expense, Research and Development Expense and Allowance for Bad Debts.










The dimension of the feature space for the Financial Kernel in this experiment is

90. The features are listed in Figure 14. A "YOY" in front of a ratio means the year-

over-year change for that ratio. Here is a listing of the features:


Salesyrl/ARyrl
ARyrl/Salesyrl
Salesyr2/ARyr2
ARyr2/Salesyr2
YOYSalesyrl/ARyrl
YOYARyrl/Salesyrl
Salesyrl/INVyrl
INVyrl/Salesyrl
Salesyr2/INVyr2
INVyr2/Salesyr2
YOYSalesyrl/INVyrl
YOYINVyrl/Salesyrl
Salesyrl/TAyrl
TAyrl/Salesyrl
Salesyr2/TAyr2
TAyr2/Salesyr2
YOYSalesyrl/TAyrl
YOYTAyrl/Salesyrl
Salesyrl/OAyrl
OAyrl/Salesyrl
Salesyr2/OAyr2
OAyr2/Salesyr2
YOYSalesyrl/OAyrl
YOYOAyrl/Salesyrl
Salesyrl/CEyrl
CEyrl/Salesyrl
Salesyr2/CEyr2
CEyr2/Salesyr2
YOYSalesyrl/CEyrl
YOYCEyrl/Salesyrl
Figure 14 Fraud Features


ARyrl/INVyrl
INVyrl/ARyrl
ARyr2/INVyr2
INVyr2/ARyr2
YOYARyrl/INVyrl
YOYINVyrl/ARyrl
ARyrl/TAyrl
TAyrl/ARyrl
ARyr2/TAyr2
TAyr2/ARyr2
YOYARyrl/TAyrl
YOYTAyrl/ARyrl
ARyrl/OAyrl
OAyrl/ARyrl
ARyr2/OAyr2
OAyr2/ARyr2
YOYARyrl/OAyrl
YOYOAyrl/ARyrl
ARyrl/CEyrl
CEyrl/ARyrl
ARyr2/CEyr2
CEyr2/ARyr2
YOYARyrl/CEyrl
YOYCEyrl/ARyrl
INVyrl/TAyrl
TAyrl/INVyrl
INVyr2/TAyr2
TAyr2/INVyr2
YOYINVyrl/TAyrl
YOYTAyrl/INVyrl


INVyrl/OAyrl
OAyrl/INVyrl
INVyr2/OAyr2
OAyr2/INVyr2
YOYINVyrl/OAyrl
YOYOAyrl/INVyrl
INVyrl/CEyrl
CEyrl/INVyrl
INVyr2/CEyr2
CEyr2/INVyr2
YOYINVyrl/CEyrl
YOYCEyrl/INVyrl
TAyrl/OAyrl
OAyrl/TAyrl
TAyr2/OAyr2
OAyr2/TAyr2
YOYTAyrl/OAyrl
YOYOAyrl/TAyrl
TAyrl/CEyrl
CEyrl/TAyrl
TAyr2/CEyr2
CEyr2/TAyr2
YOYTAyrl/CEyrl
YOYCEyrl/TAyrl
OAyrl/CEyrl
CEyrl/OAyrl
OAyr2/CEyr2
CEyr2/OAyr2
YOYOAyrl/CEyrl
YOYCEyrl/OAyrl


8.3.2 Bankruptcy Data

The bankrupt companies were chosen using the Compustat Research database 19.

All chosen companies are from the Manufacturing sector (Industry codes 2000 3999).

The companies chosen were delisted between 1993 and 2002. A company is delisted

when it does not meet the minimal requirements of financial stability according to the










market (NYSE, NASDAQ, AMEX). The analysis is limited to post-1992 company years

due to the fact that the model requires the text of the 10Ks to be in electronic form.

Electronic 10Ks were not available until 1993.

Figure 17 in Appendix B shows the entire quantitative dataset. The attribute

definitions are as follows:

Label bankrupt (-1) nonbankrupt (1)
Ticker company ticker for stock market


Ind -


Industry Number


Year 1s year of data collection
TAyr[l,2] Total Assets
REyr[l,2] Retained Earnings
WCyr[l,2] Working Capital
EBITyr[1,2] Earnings before Interest and Taxes
SEyr[l,2] Stockholder's Equity
TLyr[l,2] Total Liabilities

These attributes chosen were the components of the Altman Z score for

manufacturing. The dimension of the feature space for the Financial Kernel in this

experiment is 90. The features are listed in Figure 15. A "YOY" in front of a ratio

means the year-over-year change for that ratio. Here is a listing of the features:


TAyrl/REyrl
REyrl/TAyrl
TAyr2/REyr2
REyr2/TAyr2
YOYTAyrl/REyrl
YOYREyrl/TAyrl
TAyrl/WCyrl
WCyrl/TAyrl
TAyr2/WCyr2
WCyr2/TAyr2
YOYTAyrl/WCyrl
YOYWCyrl/TAyrl
TAyrl/EBITyrl
EBITyrl/TAyrl
TAyr2/EBITyr2
EBITyr2/TAyr2


REyrl/WCyrl
WCyrl/REyrl
REyr2/WCyr2
WCyr2/REyr2
YOYREyrl/WCyrl
YOYWCyrl/REyrl
REyrl/EBITyrl
EBITyrl/REyrl
REyr2/EBITyr2
EBITyr2/REyr2
YOYREyrl/EBITyrl
YOYEBITyrl/REyrl
REyrl/SEyrl
SEyrl/REyrl
REyr2/SEyr2
SEyr2/REyr


Figure 15 Bankruptcy Features


WCyrl/SEyrl
SEyrl/WCyrl
WCyr2/SEyr2
SEyr2/WCyr2
YOYWCyrl/SEyrl
YOYSEyrl/WCyrl
WCyrl/TLyrl
TLyrl/WCyrl
WCyr2/TLyr2
TLyr2/WCyr2
YOYWCyrl/TLyrl
YOYTLyrl/WCyrl
EBITyrl/SEyrl
SEyrl/EBITyrl
EBITyr2/SEyr2
SEyr2/EBITyr2










YOYTAyrl/EBITyrl
YOYEBITyrl/TAyrl
TAyrl/SEyrl
SEyrl/TAyrl
TAyr2/SEyr2
SEyr2/TAyr2
YOYTAyrl/SEyrl
YOYSEyrl/TAyrl
TAyrl/TLyrl
TLyrl/TAyrl
TAyr2/TLyr2
TLyr2/TAyr2
YOYTAyrl/TLyrl
YOYTLyrl/TAyrl


YOYREyrl/SEyrl
YOYSEyrl/REyrl
REyrl/TLyrl
TLyrl/REyrl
REyr2/TLyr2
TLyr2/REyr2
YOYREyrl/TLyrl
YOYTLyrl/REyrl
WCyrl/EBITyrl
EBITyrl/WCyrl
WCyr2/EBITyr2
EBITyr2/WCyr2
YOYWCyrl/EBITyrl
YOYEBITyrl/WCyrl


YOYEBITyrl/SEyrl
YOYSEyrl/EBITyrl
EBITyrl/TLyrl
TLyrl/EBITyrl
EBITyr2/TLyr2
TLyr2/EBITyr2
YOYEBITyrl/TLyrl
YOYTLyrl/EBITyrl
SEyrl/TLyrl
TLyrl/SEyrl
SEyr2/TLyr2
TLyr2/SEyr2
YOYSEyrl/TLyrl
YOYTLyrl/SEyrl


Figure 15 Continued

8.3.3 Restatement Data

Restatements as defined in this research are annual reports by publicly traded

companies, which have been restated either voluntarily or involuntarily. Restatements

are a much more loosely defined dataset than that of bankruptcy or fraud. There is a

strong interest as to the underlying causes of restatements, which was a primary

motivation for the addition of this dataset. The restatements analyzed in this study were a

subset of all restatements of publicly traded companies for the years of 1997 2002

(details are explained below). The Restatement dataset was gathered using report code

GAO-03-138 37 by the General Accounting Office. The restatements in this report

involve accounting irregularities resulting in material misstatements of financial results.

Restatements can be seen as a superset which includes fraud and earnings management as

subsets. When a company is deemed to have committed fraudulent activity or managed

earnings, the SEC requires that the company restate its financial. The GAO report

includes an appendix which lists all restatements for the years between 1997 and 2002.

The restatement dataset is the largest of the datasets tested (800/1,379), (i.e. the

fraud dataset had 122 cases and the bankruptcy dataset had 156 cases). There were 919









restatements for publicly traded companies between the years of 1997 and 2002 37. The

quantitative dataset was 1379 companies, 690 of which were restatements and 689 of

which were non restatements. The smaller, 800 case dataset is a subset of the 1,379 case

dataset which includes text and quantitative attributes. The size 800 dataset is split

evenly between restatements and nonrestatements. The reduction from 919 to 690 was

due completely to the lack of quantitative data available for some of the companies in the

GAO report. The drop from 690 to 400 restatements for the combined dataset was due to

the inability to get 10K data for some of the GAO companies. This was due in part to the

GAOs inclusion of foreign companies and companies traded on Over The Counter

markets, both of which are not required to submit the same type of 10K. The quantitative

attributes for this dataset are as follows:

Ticker Company ticker for stock market
Label restatement (-1) nonrestatement (1)
Ind Industry Number
Year 1st year of data collection
Salesyr[l,2] Sales
ARyr[l,2] Accounts Receivable
INVyr[l,2] Inventory
TAyr[l,2] Total Assets
OAyr[l,2] Other Assets
CEyr[l,2] Capital Expenditures

The entire quantitative dataset is in Appendix B under the title of Figure 18. The

features for the Restatement Dataset are the same as the features in the Fraud Dataset,

under Figure 14.

8.4 The Ontology

The ontology is a three-level ontology composed of concepts, two-grams and three-

grams. The concepts may be one word or two word concepts. The two-grams and three-

grams are built on top of the concepts. The size of the ontology is determined at three









levels, the concept, two-gram, and three-gram level. A concept can have many children

at the two-gram and three-gram levels. A two-gram can have many children at the three-

gram level. A two-gram is always a direct child of a concept. A three-gram may be a

direct child of a concept or a two-gram.

Appendix A shows a 300 dimension ontology. This ontology was built using the

entire GAAP text [28] as the accounting corpus. The 300 dimensions include 100

concepts, 100 two-grams and 100 three-grams. Given the small number of examples in

the fraud and bankruptcy datasets, 300 dimensions was the largest ontology created. The

concepts are determined by the functions described in Chapter 6. The concepts chosen

for this ontology are the ones that had the highest scores as described in Chapter 6. The

two-grams and three-grams are chosen based on mutual information scores, using

respectively the Dice Coefficient and 113 [5]. Commonly accepted Mutual Information

scores are available for two and three-grams. Higher order n-grams do not have accepted

Mutual Information scores, therefore this analysis is limited to two and three-grams. An

ontology of 100 two-grams and 100 three-grams makes it feasible to have some concepts

with both children and grandchildren. The deeper the tree the more specific the ontology

gets. The effect is a more precise ontology. The prediction accuracy on the test datasets

ultimately determine which ontologies are the best for this particular project. The two-

grams and three-grams are preceded by their part-of-speech (n-noun, a-adjective, v-verb).

As seen in Appendix A, there are two numbers after the two and three-grams. The first is

the mutual information score and the second is the overall ranking of the n-gram's

importance as compared to all n-grams. The ranking is used to determine which two and









three-grams are used in the ontology. A two or three-gram is eligible for the ontology if

at least one of its component words is a concept in the ontology.

Ontology creation is an iterative process. The process must be refined based on the

actual results achieved. The 300 dimension GAAP ontology Appendix A was used in

conjunction with 10Ks of bankrupt and nonbankrupt companies (see Section 9.x for

further details). Due to the small size of the dataset the 300 dimension ontology appears

to be overfitting. Two additional GAAP ontologies were created having 60 and 10

dimensions, respectively. These ontologies are available in Appendix A.

Choosing an accounting text as the basis of the ontology has a major impact on the

results. GAAP was chosen because it is a general purpose text that covers all major

accounting topics and is written in natural language. A drawback of GAAP is its indirect

relationship to the MDNAs. A more direct accounting text would be the MDNAs. A set

of ontologies were created using the MDNAs from the bankrupt and nonbankrupt

companies as the accounting text. These ontologies are of the following dimensions, 150

(including 50 concepts, 50 2-grams, 50 3-grams), 50 (including 50 concepts), and 25

(including 25 concepts). All ontologies are available in Appendix A.

8.5 Data Gathering and Preprocessing

The financial information for bankrupt firms was gathered for two consecutive

years prior to delisting. In the event that the financial information was not available for

the two years directly prior to delisting, the latest two years of pre-delisted data were

taken instead. In the case of fraud the financial data was gathered for the first year of

fraud and the year prior to fraud, as reported by the SEC. For example, if the first year of

fraudulent activity was 2000, then data from 1999 and 2000 is gathered. In the case of

restatements, the restatements were gathered for the year of the restatement and the year









prior to the restatement. The fraudulent/bankrupt/restatement companies were matched

with nonfraudulent/nonbankrupt/nonrestatement companies based on industry, year and

total assets. A match was accepted if total assets of a

nonfraudulent/nonbankrupt/nonrestatement company were within 10% of the

fraudulent/bankrupt company for year one. If no company met this requirement, then the

company with the nearest total asset value was chosen. The Compustat Industrial

Annual database was used in conjunction with a script created using Perl to download the

quantitative financial data for all three datasets.

The 1OKs were gathered directly from www.sec.gov. There is one 10K per

company and the year of the 10K matches (in most instances) the last year of the

financial information. If the 10K was not available for the last year, then the 10K was

chosen as follows:

(1) The 10K for the year prior to the final year

(2) If (1) was not available, the year after the final year (as long as it is not past the

delist year in the bankruptcy case, the restatement year in the restatement case or

the fraud year in the fraud case).

If (1) and (2) were not available, both the company and its match company were deleted

from the analysis.

The text analysis was limited to the section entitled "Management's Discussion and

Analysis of Financial Condition and Results of Operations (MDNA)." The MDNA

section is a natural choice as it is the portion of the 10K which allows management to

explain the underlying causes of the company's financial condition. It also is a section

where forward-looking statements are allowed.









Using the Financial Kernel, the attributes are mapped to features, as explained in

Chapter 5. The total size of the attribute space is 12 for the fraud, bankruptcy and

restatement datasets. The attributes in the fraud and restatement datasets are described in

Section 8.2. The attributes in the bankruptcy dataset sets are described in Section 8.3.

The feature space is determined by the function


6(A 1/Y )


where A/ Y = the number of attributes per year.

8.5.1 Preprocessing-Quantitative Data

There are three issues to consider for quantitative attributes: missing data, "0-

valued" data, and scaling. Missing data is a common problem with publicly available

financial information. The method used to fill in the missing data for this paper is called

multiple imputation [81]. This method takes into account not only the statistics of the

missing variable over the entire dataset, but also the relationship between the missing

variable and the other variables in the example. The data is put through a multiple

imputation routine in the statistics package R 81. Quantitative attributes with a value of 0

is a problem in this analysis because of the extensive usage of ratios in the Financial

x
Kernel. A ratio of the form for any x is undefined. In order to avoid this problem, 0
0

data are given a value of .0001 and the entire dataset is scaled between 1 and -1.

8.5.2 Preprocessing-Text Data

The preprocessing of the text data involved the following steps:

(1) Making all text lowercase. This is done to avoid the problem that a computer

will see the same words as different if they are different cases. For example, the word









Asset, asset and ASSET would be considered three separate words. Making all letters

lowercase avoids this problem.

(2) Deleting stopwords. Stopwords are common words that add noise to text

analysis. Deleting these stopwords is a method of cleaning the text. The stopword list

used for preprocessing the ontology is the same stoplist used for preprocessing the

MDNAs. The stoplist is available in Appendix A.

(3) Part-of-speech tagging and stemming. Part-of-speech tagging assures that

matches between the MDNAs and the ontology will occur only for words with the same

spelling and part-of-speech. Stemming removes the suffixes from the words to facilitate

matching of concepts that are the same but used in different tenses.

(4) Concept-naming. For this step, all synsets from each concept from the ontology

are given a single, representative word. For example, the concept liability has three

synonyms; liability, indebtedness and financial obligation. The MDNA is searched for

all three words and each instance is replaced with a single representative word. This

allows for correct matches between the ontology (which was preprocessed with concept-

naming as well) and the MDNAs.

Simple counts of each component of the ontology are placed in vector form for

each company MDNA. The size of the ontology is a user-defined parameter. The size of

an ontology is limited to the top scoring concepts, two-grams and three-grams. The user

decides how many of each should be in the ontology. The main limitation is that only

two and three-grams that have an ontology concept as one of their components words can

be in the ontology.









Below is an example of one MDNA, (company name Fifth Dimension Inc., year

1996)

This is a sample of the raw text.

The Corporation spent $122,128 on capital additions during 1996 while recording

$124,611 of depreciation expense. A reduction in capital spending is projected

for 1997 while depreciation reserves are projected at slightly lower levels than in

1996.


This is the text after Steps (1) and (2).

corporation spent 122,128 capital additions 1996 recording 124,611 depreciation

expense. reduction capital spending projected 1997 depreciation reserves

projected slightly lower levels 1996.

This is the text after Steps (3) and (4).

null/JJ/null corporation/NN/corporation spent/VBD/spend 122/CD/122 ,/,/,

128/CD/128 capital/NN/capital additions/NNS/addition 1996/CD/1996

recording/NN/recording 124/CD/124 ,/,/, 611/CD/611

depreciation/NN/depreciation expense/NN/expense ././.

reduction/NN/reduction capital/NN/capital spending/NN/spending

projected/VBN/project 1997/CD/1997 depreciation/NN/depreciation

reserves/NNS/reserve projected/VBN/project slightly/RB/slightly lower/JJR/lower

levels/NNS/level 1996/CD/1996 ././.

The complete MDNA is available via a link in Appendix B.

The text vectors are created by totaling the number of times each ontology

component is encountered in the text of a company's MDNA. The text vectors are






87


normalized by dividing each vector component by the total word count of the company's

MDNA text. This normalization procedure assures that the importance of concepts to a

particular document is not diminished due to the difference in sizes between documents.

This Chapter gave the research questions along with detailed explanations of the

bankruptcy, fraud and restatement datasets. Data preprocessing was explained as well as

ontology creation. In the next Chapter test results are given on the three datasets along

with discussions on each.














CHAPTER 9
RESULTS

This chapter gives the results of the empirical tests. Each dataset is tested

individually and the results are listed in table format. Following the results for each

dataset is an discussion of the results. The format of the results is explained below.

The experiments are set up so that the hypotheses in Chapter 8 can be either

supported or negated. There are three main categories of tests. The quantitative data is

tested using a SVM with the Financial Kernel. The Text Kernel is tested using various

sizes and types of ontologies. The Combination Kernel is tested using various sizes and

types of ontologies as well. The results are given in tables 2 40. The table headings are

described as follows:

"SV" is the number of support vectors.

SV / is a rough measure of the generalizability of the "Test on Training"

results. Here ? is the number of examples in the dataset.

"C" is a user-defined parameter that defines the penalty for a mistake in the

quadratic optimization problem. Deciding on the right C is more of an art than a science.

After raising C to a certain point, the results will level off or decline. Results are given

for various values of C.

"Test on Training" is the test results of the examples used to train the SVM. The

number shown is the prediction accuracy of the model.

"10-fold Cross Validation" results are the average prediction accuracy of 10 SVM

runs where 10% of the examples are left out from training on each run and used for