• TABLE OF CONTENTS
HIDE
 Title Page
 Acknowledgement
 Table of Contents
 List of Tables
 List of Figures
 Abstract
 Introduction
 The audit process and professional...
 The experiment
 Results of the study
 Summary, conclusions, and suggested...
 Bibliography
 Appendix: The experimental...
 Biographical sketch














Title: Internal control information and audit program revision
CITATION PDF VIEWER THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00098633/00001
 Material Information
Title: Internal control information and audit program revision an empirical sutdy of human judgment
Physical Description: xi, 174 leaves : ill. ; 28 cm.
Language: English
Creator: Tabor, Richard Herbert, 1951-
Publication Date: 1980
Copyright Date: 1980
 Subjects
Subject: Auditing   ( lcsh )
Auditors   ( lcsh )
Accounting thesis Ph. D   ( lcsh )
Dissertations, Academic -- Accounting -- UF   ( lcsh )
Genre: bibliography   ( marcgt )
non-fiction   ( marcgt )
 Notes
Thesis: Thesis--University of Florida.
Bibliography: Bibliography: leaves 131-137.
General Note: Typescript.
General Note: Vita.
Statement of Responsibility: by Richard H. Tabor.
 Record Information
Bibliographic ID: UF00098633
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
Resource Identifier: alephbibnum - 000099222
oclc - 06959041
notis - AAL4673

Downloads

This item has the following downloads:

PDF ( 7 MBs ) ( PDF )


Table of Contents
    Title Page
        Page i
    Acknowledgement
        Page ii
    Table of Contents
        Page iii
        Page iv
        Page v
        Page vi
    List of Tables
        Page vii
        Page viii
    List of Figures
        Page ix
    Abstract
        Page x
        Page xi
    Introduction
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
        Page 8
        Page 9
        Page 10
        Page 11
    The audit process and professional judgment
        Page 12
        Page 13
        Page 14
        Page 15
        Page 16
        Page 17
        Page 18
        Page 19
        Page 20
        Page 21
        Page 22
        Page 23
        Page 24
        Page 25
        Page 26
        Page 27
        Page 28
        Page 29
        Page 30
        Page 31
        Page 32
        Page 33
    The experiment
        Page 34
        Page 35
        Page 36
        Page 37
        Page 38
        Page 39
        Page 40
        Page 41
        Page 42
        Page 43
        Page 44
        Page 45
        Page 46
        Page 47
        Page 48
        Page 49
        Page 50
        Page 51
        Page 52
        Page 53
        Page 54
        Page 55
        Page 56
        Page 57
        Page 58
        Page 59
        Page 60
        Page 61
        Page 62
        Page 63
        Page 64
    Results of the study
        Page 65
        Page 66
        Page 67
        Page 68
        Page 69
        Page 70
        Page 71
        Page 72
        Page 73
        Page 74
        Page 75
        Page 76
        Page 77
        Page 78
        Page 79
        Page 80
        Page 81
        Page 82
        Page 83
        Page 84
        Page 85
        Page 86
        Page 87
        Page 88
        Page 89
        Page 90
        Page 91
        Page 92
        Page 93
        Page 94
        Page 95
        Page 96
        Page 97
        Page 98
        Page 99
        Page 100
        Page 101
        Page 102
        Page 103
        Page 104
        Page 105
        Page 106
        Page 107
        Page 108
        Page 109
        Page 110
        Page 111
        Page 112
        Page 113
        Page 114
        Page 115
        Page 116
        Page 117
        Page 118
        Page 119
        Page 120
    Summary, conclusions, and suggested future research
        Page 121
        Page 122
        Page 123
        Page 124
        Page 125
        Page 126
        Page 127
        Page 128
        Page 129
        Page 130
    Bibliography
        Page 131
        Page 132
        Page 133
        Page 134
        Page 135
        Page 136
        Page 137
    Appendix: The experimental materials
        Page 138
        Page 139
        Page 140
        Page 141
        Page 142
        Page 143
        Page 144
        Page 145
        Page 146
        Page 147
        Page 148
        Page 149
        Page 150
        Page 151
        Page 152
        Page 153
        Page 154
        Page 155
        Page 156
        Page 157
        Page 158
        Page 159
        Page 160
        Page 161
        Page 162
        Page 163
        Page 164
        Page 165
        Page 166
        Page 167
        Page 168
        Page 169
        Page 170
        Page 171
        Page 172
        Page 173
    Biographical sketch
        Page 174
        Page 175
        Page 176
        Page 177
Full Text













INTERNAL CONTROL INFORMATION AND AUDIT PROGRAM REVISION:
AN EMPIRICAL STUDY OF HUMAN JUDGMENT














By

RICHARD H. TABOR


A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF
THE UNIVERSITY OF FLORIDA
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY







UNIVERSITY OF FLORDIA


1980
















ACKNOWLEDGEMENTS


I thank my supervisory committee, Dan Smith, Doug Snowball,

Bill Messier, and Joe Reitz, for their guidance and support. Addi-

tional thanks go to Gary Holstrum, Charlie Swan, and others who

contributed to the development of the materials used in this study.

I also am grateful to those auditors who gave of their valuable

time to participate in this study and to the four firms for allowing

their people to take part.

Finally, my thanks go to Teri Greene for contributing his

typing skills to this project and to the members of my family and

close friends who offered their support by continuously asking the

important question: "Aren't you finished yet?"



















TABLE OF CONTENTS




ACKNOWLEDGEMENTS . . . . . . . . .


LIST OF TABLES . . . . . . . . .


LIST OF FIGURES . . . . . . . . .


ABSTRACT . . . . . . . . . . .


CHAPTER I. INTRODUCTION . . . . . . .

Compliance and Substantive Tests . . . .

Auditor Judgment . . . . . . . .

Research Objectives . . . . . .

Judgment Consensus . . . . . .

Evaluation of Evidence . . . . .

Importance of Controls and Self-Insight .

Firm and Experience Effects . . . .

Research Approach . . . . . . .

Dissertation Organization . . . . .


CHAPTER II. THE AUDIT PROCESS AND PROFESSIONAL JUI

Introduction . . . . . . . . .

The Audit Process . . . . . . .

Internal Control . . . . . . . .

Statistical Sampling and Professional Judgment

Human Information Processing . . . . .


. . .








DGEN ..




. . . . .


iii











Psychological Literature

Consensus ..


Self-Insight . .

Auditing Literature . .

Most Relevant Prior Research .

Ashton Study, Replications

Joyce Study . . . .

Hock and Turner Study .

Summary . . . . . .


PADTEn TTT 'Tl. PD.TER l 'T


Extensions


I RI I X -M . . . . . .

Introduction . . . . . . . . . .

The Lens Model . . . . . . . . .

Specification of the Univariate Lens Model

The Multivariate Lens Model . . . . .

Analysis of Variance . . . . . .

The Experimental Design and Methodology . . .

The Task . . . . . . . . .

Task Selection . . . . . .

Task Development . . . . ..

Decision Process . . . . . .

Experimental Materials . . . . .

Cue Selection . . . . . . .

Experimental Design . . . . . . .

Description of the Auditor's Judgment Process

Judgment Consensus . . . . . . .

Self-Insight . . . . . . . .

Lack of Formal Hypotheses . . . . . .


. . . . . . . . 20


CIH


.











Administration of the Experiment . . . .

Pilot Studies . . . . . . .

Primary Study . . . . . . .

Limitations of the Experiment . . . . .

Selection of Subjects . . . . .

Audit Environment . . . . . .

Administration of the Experiment . . .

Explaining the Judgment Process . . .

Generalizability of Results . . .

Summary . . . . . . . . . .


CHAPTER IV. RESULTS OF THE STUDY . . . . .

Introduction . . . . . . . .

Preliminary Audit Program . . . ....

Comparison Among Firms . . . . .

Relative Importance of IACs . . . .

Audit Program Revision . . . . . .

Description of the Reliability/Sample Size

Judgment Consensus . . . . . .

Canonical Correlation . . . .

Pearson Product-Moment Correlation

Firm and Experience Effect ..

Cluster Analysis . . . . .

Self-Insight . . . . . . . .

Subjective Weights . . . . .

Self-Insight Index . . . . .

Additional Data . . . . . . .


. . . . 57


Judgment









. . .











Decision Making Approach . . . . . . 115

Questionnaire Results . . . . . . .. 116

Summary . . . . . . . . . . . . . 118


CHAPTER V. SiMMARY, CONCLUSIONS, AND SUGGESTED FUTURE RESEARCH 121

Summary of the Problem and Research Approach . . ... 121

Summary of the Results of the Study . . . . . ... 123

Description of Auditors' Judgments . . . . .. 123

The Extent of Judgment Consensus . . . . ... 124

The Degree of Self-Insight . . . . . . ... 125

Implications for the Auditing Profession . . . ... .126

Suggestions for Future Research. . . . . . . .128


BIBLIOGRAPHY . . . . . . . . . . . . . . 131


APPENDIX: THE EXPERIMENTAL MATERIALS . . . . . ... .138


BIOGRAPHICAL SKETCH . . . . . . . . . . . 174

















LIST OF TABLES


TABLE


3-1 FACTORS AND LEVELS USED IN THE STUDY . . . ... 52

3-2 EXPERIENCE LEVEL OF SUBJECTS BY FIRM . . . ... 60

4-1 MEAN RESPONSES BY FIRM FOR THE PRELIMINARY AUDIT
PROGRAM . . . . . . . . . . . 67

4-2 NUMBER OF AUDITORS USING VARIOUS CONFIDENCE LEVELS
FOR COMPLIANCE TESTING. . . . . . . 70

4-3 SUMMARY OF ANOVA RESULTS--RELIABILITY JUDGMENT ... . 72

4-4 SUMMARY OF ANOVA RESULTS--SAMPLE SIZE JUDGMENT ... . 77

4-5 SELECTED EXAMPLES OF DIFFERENCES IN THE IMPORTANCE
OF THE IACs TO THE VARIOUS AUDITORS (RELIABILITY
JUDGMENT) . . . . . . . . . . . 84

4-6 SELECTED EXAMPLES OF DIFFERENCES IN THE IMPORTANCE
OF THE IACs TO THE VARIOUS AUDITORS (SAMPLE SIZE
JUDGMENT) . . . . . . . . . . . 86

4-7 RELATIVE IMPORTANCE OF THE INTERNAL CONTROLS AND
THEIR TWO-FACTOR INTERACTIONS TO THE SUBJECTS'
RELIABILITY AND SAMPLE SIZE JUDGMENT . . ... 87

4-8 VALUE OF STATISTICAL WEIGHTS BY FIRM AND EXPERIENCE
LEVEL ..... . . . . . . . . . 90

4-9 DISTRIBUTION OF SUBJECTS BY TOTAL VARIANCE EXPLAINED . 92

4-10 JUDGMENT CONSENSUS AMONG AUDITORS BY FIRM AND
EXPERIENCE LEVEL (canonical correlations) ... . 97

4-11 JUDGMENT CONSENSUS AMONG AUDITORS BY FIRM AND
EXPERIENCE LEVEL (Pearson correlations) . . .. 98

4-12 MEAN RESPONSES BY FIRM AND EXPERIENCE LEVEL . . .. 101

4-13 SUMMARY OF CLUSTER ANALYSIS . . . . . ... .104












TABLE


4-14 VALUE OF SUBJECTIVE WEIGHTS BY AUDITOR . . . .. .107

4-15 VALUE OF SUBJECTIVE WEIGHTS BY FIRM AND EXPERIENCE
LEVEL . . . . . . . . . . . 109

4-16 SELF-INSIGHT INDICES FOR INDIVIDUAL AUDITORS . . .. .112

4-17 SELF-INSIGHT INDICES BY FIRM AND EXPERIENCE LEVEL . 114

4-18 SAMPLE SIZE DECISIONS--BY FIRM . . . . . ... 117











LIST OF FIGURES


FIGURE


2-1 The audit process .. . . . .. . . . 13

3-1 Diagram of the lens model showing the relationships
among the cues, criteria, and judge's responses . 36

3-2 Diagram of the lens model for the two criterion case
showing the relationships among the cues,
criteria, and judge's responses ...... . 40

3-3 The experimental decisions process . ... .. . . 46
















Abstract of Dissertation Presented to the Graduate Council
of the University of Florida in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy







INTERNAL CONTROL INFORMATION AND AUDIT PROGRAM REVISION:
AN EMPIRICAL STUDY OF HUMAN JUDGMENT

By

Richard H. Tabor

August 1980


Chairman: E. Dan Smith
Major Department: Accounting


This dissertation has provided additional evidence to facili-

tate a better understanding of the role of auditor judgment in the

evaluation of IAC information and the decision as to the extent of

subsequent audit procedures (substantive tests). A laboratory experi-

ment was conducted to gather data which were used to examine auditor

judgment about the reliability of an IAC system and the resulting

selection of a sample size for a substantive test. The participants

were 109 members of four "Big-Eight" accounting firms. A descriptive

approach was taken within the framework provided by the Brunswik lens

model to examine these decisions for a specific audit task.

Two key decision points within the audit process that require

the exercise of professional judgment were examined in this study.

These included: (1) the preliminary audit planning stage and (2) the









audit program revision stage. Examination of the responses from these

stages resulted in descriptions of the auditors' judgments and the

evaluation of auditor consensus and self-insight.

Responses of the auditors at both the preliminary audit stage

and at the audit program revision stage suggest that, within the con-

text of this study, the most important of three key controls in decid-

ing that "recorded sales are for valid transactions" is the "control

over physical shipment.' The controls for "shipment authorization" and

"credit approval" followed in importance in that order.

The extent of judgment consensus at the audit program revision

stage, as examined through the application of canonical correlation and

Pearson product-moment correlation, was found to be slightly lower than

in other audit studies. The complexity of the audit task was thought

to be a contributing factor to this finding. The nature of the audit

task also was thought to contribute to the relatively low self-insight

indices that were found. Firm differences and experience levels were

found to be significant in examining the importance of controls, the

extent of consensus, and the level of self-insight exhibited by the

auditors.

In summary, although other studies have examined similar deci-

sions made by auditors, some specific and significant limitations were

thought to exist in these studies. The present study sought to over-

come these limitations. Primarily, a more realistic setting was used,

with a greater degree of control over the significant factors impacting

upon the decisions of the auditors. This research has been presented

with the hope that it can in some way serve as a stepping stone for

future research in this vitally important area.
















CHAPTER I

INTRODUCTION



The demand for the services provided by external auditors has

increased in recent years as the complexities of the business world

have increased. The knowledge and expertise of the independent auditor

have resulted in a natural dependency on the auditor to provide infor-

mation or services desired by management, interested third parties, or

the general public.

As prescribed by the second standard of field work [AICPA,

1979a, Section 320], a primary area of interest and concern of the

auditor when engaged to perform an audit is the functioning of the

internal accounting controls (IACs). Internal control has been defined

as follows:


Internal control comprises the plan of organization
and all of the coordinate methods and measures adopted
within a business to safeguard its assets, check the
accuracy and reliability of its accounting data, pro-
mote operational efficiency, and encourage adherence
to prescribed managerial policies [AICPA, 1979a,
Section 320.09].


This definition incorporates both accounting and administrative

controls. However, the extent of the auditor's concern for internal

control depends primarily upon the type of engagement or audit objective.

Recent reports relating to the evaluation of internal control (i.e.,


See AICPA Professional Standards, Sections 320.09-320.13
for a discussion of administrative versus accounting controls.










The Foreign Corrupt Practices Act of 1977, Required Communication of

Material Weaknesses in Internal Accounting Control [AICPA, 1979a,

Section 323], and the Report of the Special Advisory Committee on

Internal Accounting Control [AICPA, 1979b]) have publicized the signi-

ficance of IAC evaluation to both management and external auditors.

The primary importance of IAC evaluation from the external

auditor's perspective lies in the impact of the evaluation of the audit

tests to be performed. As indicated in the AICPA Professional Stan-

dards:


There is to be a proper study and evaluation of the
existing internal control as a basis for reliance
thereon and for the determination of the resultant
extent of the tests to which auditing procedures
are to be restricted [AICPA, 1979a, Section 320.01]
(underlining added).

The present study examines this particular relationship within a

specific audit setting.

This chapter introduces the concepts of IAC evaluation, com-

pliance testing and substantive testing, followed by brief discussions

of the relationships among these concepts and the importance of audit

judgment to the audit process. The research objectives of the study

are presented and discussed; a brief description of the research

approach then is provided. Finally, the organization of the disserta-

tion is presented.


Compliance and Substantive Tests

The extent of reliance on IAC is determined by evaluation of

the results of compliance testing. Compliance testing is used to

provide reasonable assurance that the control procedures within a










company are operating as prescribed through preliminary evaluation.

The results of compliance testing provide the necessary information

to determine the nature, timing, and extent of substantive procedures.

Substantive testing includes those procedures necessary to

obtain sufficient competent evidential matter as prescribed in the

AICPA's third standard of field work [AICPA, 1979a, Section 330].

These procedures include: (a) tests of details of transactions and

balances and (b) analytical review procedures applied to financial

information. Therefore, it is a combination of compliance test pro-

cedures and substantive test procedures that constitutes the main

component of the audit program that is to be followed.


Auditor Judgment

As a result of the additional emphasis on generating IAC infor-

mation, we might expect to find a redistribution of resources by the

auditor with regard to the appropriate combination of audit procedures,

or higher audit fees for the additional work that would be required.

From a research perspective, there is an obvious need for additional

examination of the auditor's decisions that result from the exercise

of his professional judgment. Specifically, his incorporation of the

important inputs of preliminary IAC evaluation and compliance test

results into a decision regarding the optimal extent of additional

audit procedures (substantive tests) warrants examination.

Professional judgment is fundamental to the auditing profes-

sion and pervades the entire audit process. Many statements have

been made regarding the necessity to exercise "judgment" in evaluat-

ing both the qualitative and quantitative considerations in an audit










situation. Some statements found in the AICPA Professional Standards

that relate specifically to this study include the following:


In the observance of generally accepted auditing
standards, the independent auditor must exercise his
judgment in determining which auditing procedures are
necessary in the circumstances to afford a reasonable
basis for his opinion. His judgment is required to
be the informed judgment of a qualified professional
person [AICPA, 1979a, Section 110.04] (underlining
added).

...the purpose of tests of compliance with accounting
control procedures is to provide "a reasonable degree
of assurance that they are in use and are operating as
planned." What constitutes a "reasonable" degree of
assurance is a matter of auditing judgment,... [AICPA,
1979a, Section 320.60] (underlining added).

Although statistical sampling furnishes the auditor
with a measure of precision and reliability, statis-
tical techniques do not define for the auditor the
values of each required to provide audit satisfaction.
Specification of the precision and reliability neces-
sary in a given test is an auditing function and must
be based upon judgment in the same way as is the
decision as to audit satisfaction required when
statistical sampling is not used [AICPA, 1979a, Sec-
tion 320A.03] (underlining added).

The amount and kinds of evidential matter required to
support an informed opinion are matters for the auditor
to determine in the exercise of his professional judg-
ment after a careful study of the circumstances in the
particular case [AICPA, 1979a, Section 330.09] (under-
lining added).


The fact that professional judgment plays a major role in the

decision process of auditors has been recognized for many years. Mautz

[1959, p. 44] presented conclusions concerning auditor judgment as

follows: (1) judgment must play a major role in auditing, and (2)

auditors would do well to recognize this and acquaint themselves with


the process of judgment formation.










Although the following statement was made in reference to

accounting in general, it is particularly germane to the auditing proc-

ess and the specific purpose of this study:


Judgment is, of course, a vital part of any profes-
sional's work. In accounting it plays an important
role every step of the way. But that does not mean
that it is a mysterious process, undefinable and
inexplicable. We know that the processes that feed
professional judgment are varied and complex, yet it
does not follow that we cannot make some progress in
their analysis and description [Bernstein, 1967, p. 9].


Recently, there has been increased recognition that auditor judgment

must be further evaluated and its impact understood. A number of

recent research publications, e.g., Kennedy [1977], Gibbins [1977],

and the Journal of Accounting Research Supplement on Human Information

Processing [1976], have presented discussions of recent studies that

concern auditor judgment. This study also emphasizes the importance

of auditor judgment as recognized in the specific research objectives

discussed in the following section.


Research Objectives


The objective of this study is to examine and describe (within

a specific audit setting) the impact of internal control information

(including the specific results of compliance tests) on the auditor's

judgment concerning the reliability of the IAC system and on his

resulting decision as to the extent of substantive testing. This

objective is pursued through the use of a laboratory experiment which

utilizes auditors from four of the "Big-Eight" public accounting firms.

Questions relevant to this particular objective and addressed in this

study can be summarized as follows:










(1) To what extent do auditors reflect consensus in terms

of their specification of the reliability of a client's

internal controls (both prior to and following the

evaluation of compliance tests) and to what extent is

there consensus at the later point in the audit process

where the question as to the appropriate amount of sub-

stantive testing to perform is addressed?


(2) How do auditors respond to the results of compliance

testing in terms of deciding the appropriate amount of

substantive testing, when they are presented with results

that may confirm or disconfirm their original beliefs

about the existence and effectiveness of key controls?


(3) How important are the various key internal controls in

arriving at the appropriate amount of substantive testing

as determined by: 1) the confidence levels chosen for

compliance testing, 2) the subjective weights assigned by

auditors, and 3) weights determined from the analysis-of-

variance (ANOVA) technique? And, to what extent do

auditors exhibit self-insight in understanding the rela-

tive importance of the key internal controls?


(4) Do such factors as firm differences and experience levels

have a significant effect on the decisions made by

auditors?










Judgment Consensus


Consensus is reflected by agreement among auditors when using

the same data to reach a decision. Lack of consensus, or disagreement

among auditors, is thought by many to be costly. For example, Joyce

has stated this concern as follows:


Within firms, the existence of the review process sug-
gests that individual differences exist and are likely
to be costly if unresolved. The increasing concern
within the profession about "quality control" issues .
The existence of continuing education programs within
at least the major audit firms is further evidence of
their willingness to consume resources to restrict
judgment variance among their professional staff [1976,
p. 31].


This study is expected to provide additional evidence as to

the extent of agreement among auditors when provided with a particular

audit situation. Although the purpose of this study is not to provide

any information as to the costs of any differences that may be found,

additional information as to the source of such differences would be a

meaningful contribution. The study evaluates decisions made by the

auditors at both a preliminary decision point in the audit process

(prior to conducting compliance tests) and, more extensively, at the

audit program revision stage (after evaluation of compliance test

results).2 This analysis may provide insights into the point at which

disagreement among auditors begins to emerge.


See Chapter II above for a discussion of these various stages
of the audit process.










Evaluation of Evidence


The independent auditor's objective is to obtain sufficient

competent evidential matter to provide him with a reasonable basis for

forming an opinion under the circumstances. He is also expected to be

thorough in his search for evidential matter and objective in its

evaluation [AICPA, 1979a, Section 330.09-330.15]. However, very little

is known concerning the auditor's reaction to, and incorporation of,

the evidence he has gathered.

Of primary interest in this study is the auditor's reaction to

evidence provided by IAC information. The results of this study should

provide insights into the auditor's evaluation of information that has

been lacking in previous studies. This opportunity is enhanced by the

requirement that the auditor use a statistical sampling approach in

evaluating the evidence concerning the key IACs within a controlled

audit environment.


Importance of Controls and Self-Insight


The fact that the auditors will be presented with a specific

audit objective and the key controls to be considered in meeting the

objective provide an excellent opportunity to evaluate the relative

importance of the controls. There are three possible methods for

determining the relative importance of the controls. First, confi-

dence levels used in determining sample sizes for the compliance tests

can be used to infer the relative importance of the three IACs.

Second, the ANOVA technique provides a measure of importance from the

evaluation of the decisions made by the auditors after their evaluation










of the compliance test results. Finally, a post-experiment question

requires a subjective weighting of the relative importance of the

three key controls.

The indications of the relative importance of the controls

allow for additional evaluation of a measure of consensus and also for

a calculation of a self-insight index. This ability of the auditor to

estimate the relative importance he places on the cues (controls) in

making his judgments is considered most important within the auditing

profession. Joyce, in discussing the importance of self-insight to

auditors, made the following statement:


One of the implications of poor self-insight in situ-
ations where a considerable amount of professional
expertise is communicated verbally "on the job"
(between senior and junior accountants on an audit,
for example) is clear: A distorted representation
of one professional's decision behavior will be
transmitted to another professional [1976, p. 52].


This study will provide evidence that may aid the assessment

of the extent to which the lack of self-insight represents a signifi-

cant problem within the auditing profession.


Firm and Experience Effects


This study will also provide information as to the effect of

two key factors on the judgments made by the auditors. Of specific

interest are differences that may be attributable to differences in

the firm affiliation of auditors and/or differences between various

experience levels.











Research Approach

Past research concerning the audit process has been both of a

normative and a descriptive nature. This study will take a descriptive

approach. Support for this type of research is provided by Kaplan:


It seems difficult to make a central attack on the audit
process because so little is known about what actually
constitutes a good audit. This suggests that, rather
than start with research on normative models in an
attempt to improve existing audits, we devote signifi-
cant resources to developing descriptive models of what
auditors are actually doing now [1977, p. 9].


Further support for this approach is provided by recent calls for addi-

tional research efforts in the area of audit judgment by Kaplan [1977]

and Libby and Lewis [1977].

Recently, accounting researchers have adapted the "lens model"

approach from psychology to the examination of judgments made in

accounting. This particular approach appears most consistent with the

descriptive nature of this study. The appropriateness of the lens

model for a study such as this has been recognized by Libby and Lewis:


. this approach (lens model) is particularly useful
in studying the impact of information set variables on
decision rule form, stability or learning, cue usage,
and decision accuracy, reliability, and predictability
or in descriptive studies of these variables. This
method of modeling judgment provides a compromise
between the overly simplified approach of asking sub-
jects to describe the weights they place on information
and the more complex and expensive process tracing
models that have been used in the study of judgment
[1977, p. 248] (underlining added).

A more extensive discussion of the "lens model" methodology is


presented in Chapter III.










The "lens model" will be used in this study to examine the

auditors' decisions with respect to the evaluation of internal control

information. More specifically, the audit program revision stage of

the experiment is arranged in a 2 x3 x2 factorial design which results

in twelve case situations presented to each auditor. The analysis of

the responses is discussed in Chapter III. The contents of the remain-

ing chapters are summarized below.


Dissertation Organization

This dissertation is composed of five chapters. Chapter II

presents a more detailed discussion of the audit process and the

roles of statistical sampling and professional judgment at various

stages of the process. A better understanding and description of the

audit process enables a focus on the key decision points which are of

interest in this study. Also, the roles of statistical sampling and

professional judgment are evaluated and the complementary nature of

their relationship is established. Relevant literature reviews for

these areas are presented and discussed throughout Chapter II.

The general research design and methodologies used are described

in Chapter III along with the details of the experimental setting and

the materials utilized. Characteristics of this study representing

both strengths and limitations are also noted in this chapter.

Chapter IV presents the results of the study according to the

objectives presented previously. The final chapter presents implica-

tions of the results and some ideas for future research.

















CHAPTER II

THE AUDIT PROCESS AND PROFESSIONAL JUDGMENT



Introduction


The problem of interest has been broadly stated as a lack of

understanding of the impact of IAC information on other audit proce-

dures. The sequential nature of the auditing process and the fact

that professional judgment plays a major role in the evaluation of

information are both contributing factors to this problem. The pur-

pose of this study is to further describe the impact of IAC informa-

tion on the auditor's judgment concerning the reliability of the IAC

system and his resulting choice of the amount of substantive testing.

In this chapter the relevant components of the audit process

and the role of professional judgment are introduced and discussed.

Relevant literature in these areas will also be noted and in some

cases discussed.


The Audit Process


Figure 2-1 presents a diagrammatic representation of the audit

process. Many studies in auditing have dealt either with one of these

steps exclusively or various combinations of steps. This study is

concerned with the decisions required at the two key points represented

by Step 3 and Step 5, with primary interest in the responses of the

auditors at Step 5.






13


DATA STEP 1
COLLECTION




PRELIMINARY
REVIEW STEP 2
OF IAC




PRELIMINARY
AUDIT PROGRAM STEP 3
PLANNING




COMPLIANCE
TESTING AND STEP 4
EVALUATION




AUDIT PROGRAM STEP 5
REVISION




SUBSTANTIVE
TESTING




ADDITIONAL
DATA STEP 7
(if needed)




PAUDIT STEP 8
REPORT


Figure 2-1. The audit process.










Both of these two key decision points within the audit process

call for professional judgment on the part of the auditor. First, as a

result of internal control evaluation, the auditor must decide on the

degree of reliability he associates with existing controls in order to

develop the appropriate preliminary audit program (Step 3). This pro-

gram would include both compliance and substantive tests. Second, and
2
more importantly, the auditor may be required to adjust his planned

substantive tests (Step 5) as warranted by his evaluation of the results

of compliance testing (Step 4).

One study [Ashton, 1974] has particular relevance to the present

study since it deals specifically with the auditor's ability to assess

the internal control environment (Step 2) and the extent to which

auditors agree at this point in the audit process. Other studies

[Joyce, 1976; Mock and Turner, 1979] have expanded the decisions

required by their subjects to include some type of program planning

(Step 3), such as a specification of either man hours or sample sizes.3

The present study will extend to the audit program revision stage (Step

5) of the audit process.

The dynamic nature of the audit environment requires the

auditor to adjust his plans in response to additional information. The


IThis would include the completion of the internal control
questionnaire, walk-through of the system and its evaluation.

2The second decision point is more important because actual
allocation of the firm's resources results from this decision.

3These three studies will be discussed in greater detail later
in this chapter.










auditor will plan the nature, timing, and extent of the audit procedures
4
he deems most appropriate as a result of his knowledge about the client

and the specific objectives of the audit. The auditor is then required

to evaluate additional evidence to determine if a revised audit program

is appropriate. More specifically, additional information obtained

through the performance of compliance tests on internal controls and

the resulting evaluation of such tests must be considered.

The importance of analyzing the program revision stage of the

audit process was recognized by Joyce as follows:


The selection of audit program planning as a measure
of audit work to perform can be criticized on the
grounds that auditors may disagree on an audit program
yet end up performing the same tests and arriving at
the same opinion. Differences in the initial audit
programs might vanish as they are revised in view of
information collected as the audit progresses. I am
unaware of any empirical evidence to support or refute
this contention [1976, p. 35].


Although individuals and accounting firms may express differ-

ent audit philosophies and may prefer a different emphasis on audit

procedures, the essence of the audit process as reflected in Figure

2-1 is generally accepted. That is, evidence is gathered in a sequen-

tial process that allows for the nature, extent, and timing of such

evidence to be controlled in a manner that is deemed optimal. It is

the evidence gathering and evaluation process that ultimately results

in the decision as to the optimal allocation of resources within an

audit. Therefore, this decision should be a major concern of those

performing the audit. The importance of the recognition of the


This would include any information he has obtained personally
in either the current or previous years and the information made avail-
able to him from the working papers relating to his client.










sequential nature of the audit process has been highlighted by Wright

[1976]. A focus of this study will be on the sequential nature of

the audit process as the auditor uses his professional judgment in

making the decisions relative to Step 3 (preliminary audit program

planning) through Step 5 (audit program revision).

The importance of understanding the revision stage of the

audit process has been expressed by Mock and Turner as follows:


Understanding the auditor's decision making process
may lead to decision aids which will assist the auditor
in evaluating audit evidence. It certainly seems
unlikely that significant improvements will be forth-
coming without some general agreement on how auditors
act when faced with decisions on how much substantive
evidence is appropriate in different internal control
situations 11979, p. 277].


Internal Control

The importance of internal control evaluation and its impact

on the ultimate issuance of an audit opinion has led to many recent

research efforts relating specifically to internal control. For

example, Burns [1974], Burns and Loebbecke [1975], and many others

have discussed internal control evaluation from various perspectives.

Mathematical approaches to the analyzing and/or modeling systems of

internal control have been presented by Grimlund [1978], Bodnar

11975], Gushing [1974], Yu and Neter [1973], and others.

A certain level of internal control is expected and even

required to be present within a firm (see The Foreign Corrupt Practices

Act of 1977). The accumulation of additional audit evidence is

affected directly by the IAC environment. Of interest in this study

is the relationship between internal control evaluation and subsequent










compliance and substantive testing. The impact of IAC evaluation on

audit scope adjustments has been discussed by Morris and Anderson [1976]

and Smith [1972]. This type of analysis was greatly extended by Kinney

[1975a; 1975b]. Kinney presented a decision theory approach to exami-

nations of the relationship of IAC evaluation and compliance and

substantive tests that consider such relevant factors as costs of

sampling, costs of errors, and utility functions of the auditor. How-

ever, the nature of the decision theory approach extends beyond the

scope of the present study.

Within the context of the present study, the relationships

between the key factors of IAC evaluation and substantive testing can

be related to the concept of risk and summarized as follows:


(1-R) = (1-R) (1-R)
o c s

where, (l-R ): Total overall risk [R = overall reliability]
o o
(1-R ): Risk of substantive tests [R = reliability
s of substantive tests] s

(1-R ): Risk of internal control [R = reliability
of internal control]

The use of statistical sampling has been advocated as one way of

controlling for overall audit risk [Roberts, 1978; Warren, 1979].


Statistical Sampling and Professional Judgment


The nature of the evaluation process performed by auditors and

the role that the concept of risk has within the decision process has

led to an increasing appreciation by auditors of statistical sampling

techniques.










Elliot and Rogers [1972] discussed in a very basic fashion the

use of statistical sampling in auditing. Arkin [1976] analyzed the use

of statistics in auditing with particular emphasis upon the area of

internal control compliance. Other discussions of statistical sampling

in auditing include Loebbecke and Neter 11975] and Teitlebaum and

Robinson [1975].

Regardless of the extent to which statistical sampling tech-

niques are employed, the importance of professional judgment is not

diminished. The use of statistical sampling for planning compliance

and substantive tests would require judgments such as the following:

1) expected error rates in the population, 2) desired level of confi-

dence, 3) desired precision level, 4) standard deviation of population

items, and 5) the amount considered material. The role of judgment in

statistical sampling has been evaluated by Boer [1974]. Uecker and

Kinney [1977] reported results of a study to investigate the type and

severity of errors that practicing CPAs make in the judgmental evalu-

ation of statistical sampling outcomes. Results indicated that the

problem of subjectively evaluating sample outcomes may be significant.

In their study, seventy-four percent of the CPAs made at least one

serious error of judgment and fifty-six percent made at least two such

errors.

The role of professional judgment becomes paramount as statis-

tical sampling techniques must be integrated with various stages of

the audit process. The most obvious integration occurs within internal

control assessment. That is, the auditor must decide the extent to

which he will rely upon internal controls in planning the nature,

timing, and extent of compliance and substantive tests. Due to the










difficulty in dealing with all of these planning variables, this study

will emphasize the decisions concerned with the extent of testing.

Given the role of professional judgment within the audit

process in determining the extent of audit work to perform, the com-

mitment of a firm's resources could vary considerably with variations

in auditor judgments. In the absence of an operational normative

model to rely upon, a strict evaluation of such judgments as being

"correct" or "optimal" is not possible. The appropriate evaluation

criteria then becomes the degree of consensus among auditors when

presented with the same audit environment and audit evidence. Einhorn

[1974] suggests that consensus among experts is one of several neces-

sary, although not sufficient, conditions for the existence of pro-

fessional expertise.

Although the actual cost associated with a lack of consensus

among auditors can not be measured directly, various forms of evidence

are available that suggest the auditing profession strives for consen-

sus. Such factors include the following: 1) the administration of a

uniform CPA exam, 2) issuance of standards to follow in the practice

of auditing, and 3) the use of training schools by individual firms to

guide the learning process of their employees. Given the actions taken

to strive for consensus by the auditing profession, an inference can

be made that the cost of such actions must be less than the alternative

cost of not having a certain degree of consensus.

A major question of interest then becomes, "where within the

audit process should this consensus be reached?" Should consensus

exist at every step of the audit process, or is agreement as to the










opinion that is to be issued the only point at which consensus is

important? The answer to this question seems to be that while the

issuance of the opinion is a key point of consensus, firms are also

interested in the optimal utilization of their available resources as

they seek to reduce the probability of making an incorrect decision.

The lack of consensus at decision points associated with the commitment

of a firm's resources could result in a less than optimal use of such

resources. The topic of consensus and other topics in the human

information processing area that are relevant to this study are dis-

cussed in the following section.


Human Information Processing

Psychological Literature


The area of human information processing (HIP) has been

examined extensively in the psychological literature. Three broad

approaches to the study of HIP include: 1) Bayesian, 2) regression,

and 3) cognitive complexity. A number of authors have discussed the

relative merits of the different approaches to study HIP [Slovic and

Lichtenstein, 1971; Slovic, Fischhoff, and Lichtenstein, 1977] with

comments concerning the appropriate use of each of these approaches.

A Bayesian approach is most concerned with the evaluation of

an "optimal" manner to combine information, or "how men should think."


The auditor is concerned with controlling the probability of
accepting the financial statements as being "fairly presented" when,
in fact, they are not. The cost of this Type II(3) error could be
very large.










Thus, it provides a normative model for the evaluation of probabilities

expressed by judges or subjects. A cognitive complexity approach to

the evaluation of HIP requires that individuals be put into complex

situations. The processing of information is then evaluated as deci-

sions are made to determine the "style" used.

This study is not concerned with the ability of the subject to

"optimally" process information (Bayesian approach) or the style used

(cognitive complexity approach). Therefore, the regression approach

is considered most appropriate for use in this study, with the "lens

model" considered the most applicable "representative design" within

the regression approach.

The lens model6 is based on the assumption that man makes

inferences about the state of an uncertain environment on the basis of

uncertain information. Therefore, the outcomes that result from an

interaction between man and his uncertain environment are most cer-

tainly probabilistic. The lens model specifies a relationship between

the probabilistic outcomes of the environment and the prediction or

judgment by an individual of a particular state of that environment.

Analysis of the information represented on the "right-hand-

side" of the lens model (the actual and predicted subjects' responses)

can result in the capturing of the decision policy of the subjects.

The correlation of the actual responses between the subjects is con-

sidered an appropriate measure of consensus (agreement). Also, the

mathematical modeling of the judgment process allows for the evaluation


6For a more complete presentation and explanation of the lens
model refer to Chapter III, Brunswik 119491, or Slovic and Lichtenstein
[1971].










of the relative importance of the various cues (factors). Self-

insight indices can then be obtained by correlating the mathematical

weightings of the cues with the subjective weightings provided by

the subjects.

The following sections present a discussion of the evaluation

criteria of consensus and self-insight that were mentioned above.

These particular areas have been investigated in the psychological

literature and have the most relevance to this study.


Consensus. Consensus is the extent of agreement among judges

using the same information at the same point in time. Many psycho-

logical studies have indicated that consensus among judges is very

poor (e.g., Goldberg, 1968): most accounting studies have indicated

a relatively high degree of consensus.

The extent of consensus found in most studies is thought to

be a function of the judgment task and the specific environment. How-

ever, most realistic tasks have yielded results which indicate a lack

of consensus among persons considered professionals with respect to

the judgmental task. Various opinions exist as to the source of such

differences. For example, Ashton and Kramer state the following:


Some psychologists (e.g., Brehmer, 1976; Dawes, 1975;
Edwards, 1971) have argued recently that certain formal
or statistical characteristics of the judgment task,
and not the substantive or content characteristics of
the task nor the particular subjects in the task, are
the most important determinants of human information
processing and decision making [1980].


Self-insight. Self-insight is the ability of the judge to

determine the relative weights of the various factors he uses in his

judgment process. Studies have indicated that judges exhibit poor










insight into their own weightings of factors [Slovic, 1969; Slovic,

Fleissman, and Bauman, 1972]. Researchers have found differences

between the subjective weights provided by judges and those computed

by a mathematical model of their judgmental policies. For example,

the results from psychological studies have indicated that judges over-

estimate the less important factors and underestimate the more important

factors (e.g., Slovic, 1969).

Of additional interest is the finding that non-linear models

of the judgment process contribute little as compared to linear models

[Goldberg, 1968; Slovic, 1969; Slovic and Lichtenstein, 1971; Dawes

and Corrigan, 1974]. The implication is that a simple linear model

will satisfactorily approximate the manner in which an individual

processes information. The adequacy of the linear model also will be

examined in the current study.


Auditing Literature


Auditing researchers also have developed an interest in HIP,

the applicability of the lens model approach, and the resulting model

of the judgment process. The probabilistic relationship between man

and his uncertain environment, as reflected in the lens model, is

certainly present in the audit environment.

The auditor is required to evaluate the uncertain audit

environment to ascertain that either "acceptable" or "unacceptable"

situations exist. He must then make decisions concerning specific

combinations and the extent of audit procedures that are most appro-

priate given his current beliefs about the state of the environment.










Within the specific judgmental context of this study, the true

state of the environment is that either a specific control objective is

"acceptably" achieved, or it is not. Of interest in this study is the

extent of evidence required by the auditors to determine whether an

objective has been "acceptably" achieved. Because the "true" state of

the environment cannot be determined,7 the analysis of this study is

limited to the "right-hand-side" of the lens model where the auditor's

judgment and his use of cues can be evaluated. A better understanding

of this approach is obtained through a discussion of some recent studies

in auditing.


Most Relevant Prior Research

Various studies have evaluated the extent of consensus among

auditors at different points in the audit process. Results indicate

that the degree of consensus among auditors apparently varies according

to the particular stage in the audit process or according to the type

of question addressed. Ashton [1974] and others who have replicated

and extended his original study, Joyce [1976], and Mock and Turner

11979], provide a basis for summarizing the judgment literature in

accounting. These studies will be discussed separately, with particu-

lar emphasis on their relationship to the present study.


Ashton Study, Replications and Extensions


Ashton was most concerned with the extent of inconsistency

shown by auditors in their evaluation of internal control. By


7For relevant discussions of the importance of the 'true" or
criterion value in the lens model analysis, see Casey [1976] and Ashton
[1976].










presenting sixty-three practicing auditors with thirty-two cases (each

a different combination of Yes/No responses to six questions found on

an internal control questionnaire for payroll), judgments of internal

control strength were elicited. These judgments were made on a scale

from one (extremely weak) to six (adequate to strong).

Ashton used a descriptive analysis-of-variance (ANOVA) tech-

nique to evaluate two types of consistency. These are: 1) consensus,

which refers to consistency across auditors at the same point in time,

and 2) stability, which refers to consistency over time for the same

data. By correlating the judgments of each auditor on the cases, both

with the other auditors and with later administrations for the same

auditor, Ashton found that the judgments exhibited a "fairly high level"

of consensus (r= .70), stability (r= .81), and self-insight (r=.89).

By addressing the very crucial question of "to what extent do

auditors agree in their assessment of the strength of an internal

control questionnaire?" Ashton's study represents a major contribution

to auditing research. Ashton chose to replicate his study with the

use of upper division auditing students at The University of Texas at

Austin [Ashton and Kramer, 1980]. These results were inconclusive

with respect to whether the auditing students were good surrogates

for practicing auditors. However, the results did indicate a similar

high degree of consensus on the part of the students (r =.66), with

auditors also showing a higher degree of self-insight (r=.89) as

compared to the students (r = .77).










Ashton and Brown [1980] extended Ashton's earlier work through

the addition of two cues to the six that had been used in the original

study. The subjects used in this study were 31 practicing auditors

from four offices of seven of the "Big-Eight" public accounting firms.

A similar consensus index was found (r = .67), with self-insight indices

averaging .86. It should be noted that this lower average self-insight

index was obtained from auditors who were generally less experienced

than those who participated in the original study, but who were of

course more experienced than the students tested in the Ashton and

Kramer study.

Hamilton and Wright [1977] also used the payroll environment

to evaluate audit judgment in a modified replication of Ashton's

original study. Seventeen practicing auditors from the same city were

presented with five internal control questions relative to payroll.

The average values of consensus and self-insight were .66 and .87,

respectively. Interesting results with respect to experience levels

showed that the group with more experience had both greater insight

(.93 to .84) and greater consensus (.78 to .62). These particular

findings are consistent with the results of the Ashton and Ashton and

Kramer studies.

The most recent extension of Ashton's study of internal control

relative to payroll was provided by Reckers and Taylor [1979]. Thirty

practicing auditors were presented with a completed payroll question-

naire containing responses to approximately 36 questions: these

responses were varied to form 5 different cases. In analyzing the

subjects' responses as to the reliability of the internal control

system (0-100 percent), an average inter-rater correlation of .155










was obtained. This increased slightly (to .357) for those participants

with more than seven-and-one-half years' experience. These results

suggest that the level of consensus is reduced as greater realism is

introduced into the audit task. When auditing professors were pre-

sented with the same task similar results were obtained (r= .28).

However, Ashton [1979b] expresses sone concern about the cases used

and assumptions made by Reckers and Taylor.

The studies discussed above provided a meaningful evaluation

of audit judgment within the payroll environment and with respect to

the strength of an internal control environment. However, because of

their nature, extensions of these studies appear warranted. A natural

progression of auditor judgment research would involve a shift in

focus to a different point in the audit process. This shift appears

necessary since the conclusion that auditors have exhibited a "fairly

high level" of consensus relative to the strength of internal control

says nothing as to the extent of their agreement at a later point in

the audit process; or more specifically, as to how this level of

agreement extends to decisions about the extent of other audit proce-

dures to perform. Joyce provided such an extension.


Joyce Study


Joyce [1976] extended the research in the area of auditor

judgment by focusing upon the decision regarding the extent of audit

work (planned man-hours) rather than upon the assessments of the

quality of internal control. This represented an analysis of auditor

judgment for a different question at a later point in the audit

process.










Thirty-five practicing auditors were presented with a set of

sixteen systematically varied combinations of stimulus information

related to an audit program for accounts receivable. The auditors

were asked to indicate the planned extent (in man-hours) of five audit

procedures.

Some conclusions resulting from Joyce's use of correlation

measures and ANOVA (or MANOVA) can be summarized as follows:


(1) There was considerable disagreement concerning how the
independent variables (factors used in determining the
extent of audit procedures) should be weighted.

(2) There was little consensus (r= .37) among auditors con-
cerning how much time should be planned for the audit
procedures.9

(3) Main effects accounted for virtually all the reliable
judgment variance.

(4) The level of consensus decreased as experience
increased.


The stimulus information provided by Joyce was presented as

having been developed from various sources such as an internal control

questionnaire and analytical review data (i.e., significant ratios and

turnovers). An appropriate adjustment to this experimental environment

would be to provide the auditors with a more realistic setting and

information as to the results of compliance tests on the internal


Joyce actually performed two experiments, with the second
experiment being a full replication of the 25 factorial conducted as
Experiment 1. Joyce was therefore able to assess the impact of two-
way and three-way interactions.

Note that although Joyce's subjects made multiple judgments
on each case, the consensus measure was calculated by summing the
audit hours on each case and then correlating the subjects' judgment.










control information. Mock and Turner performed a study with these

characteristics.


Mock and Turner Study


Mock and Turner [1979] sought empirical evidence about the

effect of changes in internal controls on auditors' decisions on the

extent of substantive testing. Seventy-one audit seniors and two audit

supervisors (all from a single firm) were given information on improve-

ments in internal controls from the previous year and asked to adjust

the sample sizes for four specific audit procedures from the planned

audit program. The improved internal controls were evidenced primarily

by a general change in compliance test results of specific controls for

the current year as compared to the previous year.

Some of the major results of the study include the following:


(1) Subjects reduced their judgmental samples in every case
except one. (The subjects were dealing only with improve-
ments in controls and the exception was probably due to
the fact that the planned sample for this case was
knowingly set quite low.)

(2) Subjects reacting to strong controls consistently recom-
mended smaller sample sizes for planned substantive tests
than did subjects reacting to fair controls.

(3) There was considerable variability in auditors' decisions.
(Results showed both wide ranges and large coefficients
of variation for all control procedures with sample size
decisions for strong controls varying much less than for
fair controls.)

(4) No evidence was found of a relationship between the sub-
jects' backgrounds and their judgments.










(5) Possible effects of anchoring0 were found. Those who
had anchors recommended smaller sample sizes and men-
tioned in rationale memos that they gave attention to
the previous sample sizes.


The present study is intended to be an important extension of

the study by Mock and Turner. The decision to build upon and to extend

their study is, to some extent, based upon their use of a realistic

setting. Accordingly, some specific characteristics of Mock and Tur-

ner's experimental environment and important extensions and differences

relative to this study are provided below. These particular character-

istics of Mock and Turner's study may be summarized as follows:


(1) They provided general evidence about changing internal
control strengths (resulting from compliance tests) on
a year-to-year basis.


The present study provides more specific information pertaining

to the results of compliance testing of key internal controls as com-

pared to the preliminary evaluation of the current period. These

specific results will then serve as a basis for adjusting the planned

extent of audit procedures for the current period. This restriction

will serve to reduce the possibility of incorporating confounding

variables into the experimental setting, through superior control over

the factors being considered by the auditor.


(2) They required the subjects to deal with a "perceived"
audit objective in a broad audit setting (i.e., the
Revenue Cycle).


10"Anchoring" is the process of adjusting from initial values
or starting points to yield final estimates. See Tversky and Kahneman
[1974] for a discussion of this heuristic principle.










This study also uses the Revenue (Sales) Cycle as the experi-

mental environment. However, a specific audit objective is provided

to the subjects and the internal control information is then keyed to

this particular objective. This will allow the auditors to evaluate

the evidence provided to them on a commonn" basis and therefore make

the comparison of their decisions more valid.


(3) The subjects were told the degree of reliance to place
on internal controls for either fair or strong controls.


The subjects in this study are provided with the specific

results of compliance tests (noncompliance rates for the sample sizes

selected by the subjects) and must decide their degree of reliance on

the key internal controls after receiving these results. This adjust-

ment incorporates additional realism into the setting and provides

greater assurance as to the message that is being provided to the

auditor. The auditor can then interpret this message as he desires.


(4) The subjects were presented only with improvements in
internal controls and asked to adjust a planned sample
size.

This study provides various combinations of "confirming" and "discon-

firming" evidence as to the existence and effectiveness of the key

internal controls as compared to certain expectations. The subjects

also are asked to adjust their own planned sample size for a substan-

tive test rather than to adjust a planned sample size that has been

provided to them. As this study is most concerned with the final

sample size decision, it was felt that providing a planned sample size

would influence the auditors' decisions. Again, a confounding factor

would be introduced because it would not be clear what effect this










would have on each auditor. It was considered that an inferior

measure of consensus among auditors would be obtained if a planned

sample size was provided.


(5) The subjects were presented with only one combination
of controls to use in planning a substantive program.


The subjects in this study are provided with twelve combinations

of compliance test results for three key internal controls that are

considered most important in meeting the specific objective. This is

necessary to allow for the desired analysis relative to consensus and

the weightings of the key variables. The single case approach taken

by Mock and Turner did not allow for a within-subject evaluation.


(6) The subjects were required to make "judgmental" sample
size decisions.

All sample sizes in this study are chosen from statistical

sampling tables provided to the subjects. For compliance tests,

tables were provided that gave the subjects various confidence levels

and desired precision levels from which to choose. For the substan-

tive test, a table that was derived from a difference estimation

approach was provided; in using this table it is necessary for the

subject to decide upon a confidence level and an amount for material-

ity. This approach provides additional information as to which

factors were considered most important and/or altered as the auditor

moved from case to case.

The above discussion provides a comparison of this study with

the most recent study of Mock and Turner. Perhaps the most obvious

intention of this study is to provide the auditor with a more compact

audit environment and specific audit objective to a greater degree










than in previous studies. At the same time, the desire to present a

realistic setting is also of concern.


Summary

The review of the psychology and auditing literature presented

in this chapter provides additional support for the contention that

the application of professional judgment, both in general and within

the auditing context, deserves further investigation. Support is also

provided for the importance of the evaluation of internal accounting

controls and their role within the audit process.

Various deficiencies in previous auditor judgment studies were

presented and discussed. General characteristics of this study were

presented as significant improvements over prior studies. Specific

details of the experiment that was conducted are presented in Chapter

III in conjunction with a presentation of the methodology and experi-

mental design.

















CHAPTER III

THE EXPERIMENT



Introduction

The purpose of this chapter is to present the methodology and

experimental design used in this study, along with the details of the

experiment. First, in order to provide a better understanding of the

descriptive approach used in this study, a brief description of the

Brunswik lens model and the analysis of variance model is presented.

A discussion of the specific task, the experimental design, and the

methodology follow in the next section. Finally, the steps in the

administration of this study and some expectations as to results are

presented, followed by several recognized limitations of the study.


The Lens Model

The lens model as developed and described by Egon Brunswik

[1943, 1949] has become an acceptable and popular model to use in

human information processing (HIP) research. Because of the descrip-

tive nature of this study, the lens model provides the most appro-

priate framework for the description and examination of the auditor's

judgment process.

The lens model is considered a significant development within

a "correlational" paradigm of the regression approach to the study of










human information processing [Slovic and Lichtenstein, 1971]. Bruns-

wik's model emphasizes "the probabilistic interrelations between

organismic and environmental components of the judgment situation"

[Slovic and Lichtenstein, 1971, p. 655] as opposed to a focus on the

judge (information processor).


Specification of the Univariate Lens Model


As shown in Figure 3-1, the univariate lens model utilizes

both environmental and subject response data. The left side of the

model represents the environmental system, while a subject's (decision

maker's) actual and predicted responses are represented on the right

side. Correlational measures are indicated for the relationships

within and between those systems- The variables shown in the lens

model are defined as follows:


X. = the variables found in the information set which
S serve as cues to the subject.

Y = the criterion value or "distal variable." This
e value represents the actual result of an environ-
mental event.

Y = the predicted criterion value. This prediction
e results from taking the "optimal" linear combination
of the cues, as accomplished by minimizing the squared
deviations between Y and Y .
e e
k
Therefore, Y = Z b X.,
i=l e

where b. represents the relative weightings of the cues.
le

Y = The subject's response. This value represents his
S judgment as to the criterion value.

Y = the predicted response of the subject. This prediction
s results from taking the "optimal" linear combination of
















<>4
42J
4-J 4

o
~3

C1CO

0




- Ia.I 4
(1) ci) Ci




co to o t
lw44
ca)
w -0
"| a LI r I
-4
> 00 (O 14


/j'. -4i i 9






a)) ci)
i.I -r 41







rc X C W c ou
Li






/~~L fI \ I





\. / / t e .0-
4J i4j
C ci'







~:L / I *
-4 .'-
w 00
tC a)a





x On c io




Q) 0ii)
-I~
\I hI / /" OJ* ^




m x x U Q)
4j -
c) ci) 0F ala)













\I caj c' Li C co
C Li TIa ) 'fi c C
ci .. Li 4Ji^ i-

0 > rr U IV
---- ---- X----- C U--
4j


-i 4-: 4.'
C$4 U
w r

0.0













-- Wi
*"~ 2.
01~,-L CCp 0r ec

ei





0c 0










cci
$4
> Li
L~ U10
mm dmcii)




cU m

0-' '-4 i 0
C m,4
c, 4
04-4 P




cci

to










the cues, as accomplished by minimizing the squared devia-
tions between Y and Y .
s s
k
Therefore, Y = Z b. X.,
s i=1 is I

where bis represents the relative importance of the
cues to the subject.


The "correlational" nature of the lens model is reflected by

the six correlations that can be calculated among both the actual and

predicted environmental and subject response variables. Within the

environmental system (left side) of the model, two correlational

measures can be calculated between the criterion value (Y ) and the
e
cues (X.), and between the criterion value (Y ) and the predicted

criterion value (Y ). The correlations are explained as follows:


r. = the correlation across stimuli between cue Xi and
le Y This correlation reflects the relevance of the
ith information source (cue) in the environment.

Y Y = the correlation between Ye and Ye. This correlation
ee represents the degree to which the (linearly)
weighted combination of cues serves to predict the
state of Ye and is referred to as "environmental
predictability."


Two correlational measures can also be calculated within the

subject's system (right side of the lens model). The subject's

response (Y ) can be correlated with the cues (X.) and also with the
S 1

predicted subject response (Ys). Explanations of these relationships

are as follows:


r. = the correlation across stimuli between cue Xi and
S Ys. This correlation represents the relevance of
the ith information source (cue to the subject).

r = the correlation between Ys and Yg. This correlation
s s represents the extent to which the subject's judgments










can be predicted by a linear combination of cue
values and is referred to as "response linearity."

An evaluation of the subject's performance can be obtained

through correlations between the left side and right side of the model.

These two measures are known as the "achievement index" and the "match-

ing index" and are explained as follows:


r = the correlation between Ye and Ys. This correlation
e s represents the subject's ability to predict an out-
come and is known as an "achievement index."

r = the correlation between Ye and Ys. This correlation
is between the two regression equation model esti-
mates and is known as a "matching index."

The use of the lens model in the current internal control

evaluation study requires the presentation of a set of relevant

accounting information cues (X.) as a basis for the auditor (subject)

to exercise his judgment in deciding the extent of substantive test-

ing to perform (Y ). The actual decision that should be made using

the information cues and the appropriate loss function is represented

by Ye. Because an implied loss function is necessary to decide an

"optimal" value for Ye, this value is not known within the context of

the current study.

The lack of a true criterion value (Y ) is not unusual for
e
real-world judgment situations. As a result, the current study

concentrates on the right side of the lens model as the framework for

analysis. However, the emphasis on the auditor's responses (Ys) and

his predicted responses (Y ) does provide the necessary relationships

to properly pursue the objectives of the study. It could be argued that

by obtaining a certain level of consensus among the auditors in this










study, an appropriate value for Y could be inferred for the specific
e
internal control environment that was predicted.


The Multivariate Lens Model


The existence of more than one criterion value within the

context of a single judgmental situation would require some adjust-

ment to the univariate model just presented. Within the current study,

the auditor must implicitly decide upon the reliability of the internal

control system prior to selecting an appropriate sample size for the

substantive test. Figure 3-2 presents the lens model representation

of a two criterion value situation.

The multivariate lens model allows for the simultaneous evalu-

ation of a judgment process when there is more than one criterion

value. An obvious difficulty associated with the use of the univariate

model in such a situation has been summarized as follows: "The

researcher would be left with a large number of indices and measures

which would require some sort of aggregation in order to interpret

exactly what was occurring in the entire system" [Castellan, 1972,

p. 244].

Representation of the two criterion value case requires only

minor changes to vector notation as follows:1


Y = the vector (Y e' e2).

Y = the vector (Y ,Y ).

X = the vector (X1,X2,...,Xk)



1The following is taken from a more complete discussion of a
multiple criterion case of the lens model discussed by Castellan
[1972].













































SOURCE: Messier, W. F., Jr., "An Examination of Expert Judgment in the Materiality/Disclosure Decision,"
Ph.D. dissertation, Indiana University, 1979.

Figure 3-2. Diagram of the lens model for the two criterion case showing
the relationships among the cues, criteria, and judge's responses.










The multivariate lens model for the two criterion case is then presented

as


X* = (YY s,x),


where X* is a representation of the complete system.


The appropriate correlational measures then become multivariate

and can be obtained from the use of canonical correlation. However,

the multivariate analysis does not affect the situation where a true

criterion value is lacking. Therefore, in the current study, appro-

priate canonical correlations are found relative to the right side of

the lens model.


Analysis of Variance


Strict adherence to the lens model framework discussed above

would not allow for the incorporation of a nonlinear model. The

analysis of variance (ANOVA) model enables the examination of both

linear and configural processing by subjects. The extent to which a

cue contributes to a subject's response can be determined through an

ANOVA model, as can the significance of the contribution of two or

more cues when considered simultaneously. Also, a multivariate

analysis of variance (MANOVA) is possible for situations having more

than one dependent variable.

The ANOVA (or MANOVA) framework offers some other distinct

advantages over a strict adherence to the lens model approach. The

use of the factorial design and the orthogonality obtained are specific

advantages. The fact that the cues (Xi) can be discrete or categori-

cal rather than continuous offers another advantage for most studies.










And finally, given the desire to examine the auditor's ability to

understand his decision process, the calculation of a self-insight

index is most important. This is possible through the calculation of

omega-squared indices which provide an indication of the proportion

of total variance accounted for by a particular cue or combination of

cues. Therefore, the framework of the current study is expanded to

incorporate the ANOVA model to provide additional important analyses.


The Experimental Design and Methodology

The Task


Task selection. The judgment process used by auditors in

routine audit situations is best examined through the use of a specific

audit task. The experimental task chosen for this study requires the

subject to determine the reliability to be attached to a set of inter-

nal controls of a hypothetical company, and then to use the available

information about these controls in determining the extent of substan-

tive testing to be performed to meet a specific audit objective (e.g.,

to satisfy himself that recorded sales are for valid transactions).

As discussed in Chapter II, previous studies dealing with

auditor judgment have dealt with a broader audit environment and there-

fore a somewhat different audit task. The experimental task used in

this study was chosen because:


(1) The task is most representative of the type of judgments
that auditors must make at different points within the
entire audit process.










(2) It incorporates an area (internal control evaluation)
that has become a most important topic within the
auditing profession.2

(3) It represents a very important portion of the audit
process because of the commitment of the firm's resources
and the risks associated with incorrect decisions.

(4) The task can be represented to subjects in a setting that
allows for the best combination of a realistic situation
with the necessary control for variables not of specific
interest to the experimenter.

(5) It deals with a specific objective within a particular
cycle of the business. (A cycle approach to auditing is
considered an acceptable and desirable method within the
profession.)

Task development. This study focuses on auditors' decisions

concerning the reliability of an IAC system and sample sizes for com-

pliance and substantive tests. These decisions are made with respect

to a specific audit objective (e.g., satisfaction that recorded sales

are for valid transactions) and three key IACs that can be summarized

as follows: (1) proper approval for credit, (2) authorization for

shipment, and (3) control over physical shipment of goods. Since

each of these internal controls could or could not exist, there are

eight (2x2x2) possible combinations that could be presented. However,

for practical reasons, a number of these "scenarios" were not included

in the experiment.

First, since the study is concerned with the effect of compli-

ance test results, the situation where none of the three controls

exist was not considered appropriate. This situation also appears to

be "unrealistic" in terms of the absence of all three of the key controls.


See, for example, the AICPA report by the Commission on Audi-
tor's Responsibility [1978] and The Foreign Corrupt Practices Act of
1977.









The objective of providing settings that would be acceptable as realistic

was a major consideration in the selection of scenarios. For example,

most companies have procedures to evaluate credit, and it is certainly

realistic for a company to have either control over shipment authoriza-

tion or control over physical shipment without necessarily having both

controls. Another consideration in reducing the eight possible scenar-

ios was the amount of tine that would have been required to include

all of them in the experiment.

As a result of considering the above factors, it was decided

that the inclusion of two scenarios would be the most appropriate.

Therefore, Scenario #1 includes the existence of all three key internal

controls and Scenario #2 includes IC-1 (credit approval) and IC-3

(control over physical shipment). These scenarios provide the neces-

sary information and elicit the appropriate responses for the prelimi-

nary audit program stage.

Within each scenario, there are a given number of "cases."

These cases provide the opportunity to present compliance test results

and elicit responses at the audit program revision stage. The cases

are the possible combinations of compliance test results that could

occur within a given scenario assuming that a compliance test result

will either confirm or disconfirm the auditor's prior belief about

the effectiveness of the control. For example, in Scenario #1 where

all three of the controls exist, there are eight case situations

(2x2x2). In Scenario #2, where two of the three controls exist, there


3The factors were considered in conjunction with references to
auditing texts and audit manuals, along with discussions with faculty
members and auditors involved in pilot testing.










are four case situations (2xlx2). Given that no compliance tests are

performed if a control is not present, and that possible results of

compliance testing will be either confirming or disconfirming, the

experimental task at the audit program revision stage consists of

twelve case situations.


Decision process. The decision process followed by the auditor

in this study is summarized in flowchart format in Figure 3-3 and is

discussed in this section. The experimental task requires four

responses by the auditor at various stages of the audit process. First,

the auditor is asked the degree of reliability he associates with the

existing internal controls. This response is made after reviewing

information that is common to all case situations, i.e., background

information of the company, flowchart of the sales cycle, specific

audit concern, etc., and after being presented with the combination of

key controls that exist for a particular scenario, e.g., all three

controls exist. This first elicitation of reliability (R1) is required

for each of the two scenarios and can be represented as follows:


R1 = f(Bl, II, 12, 13),

where RI = a measure of reliability taken from a 7-point scale;

B1 = unchanging information, common to all cases;

Ii = internal control #1;

12 = internal control #2; and

13 internal control #3.


Recall that II, 12, and 13 are dichotomous variables where either:

(i) the control exists or (ii) the control does not exist.









[(Relates to entire experiment) I (Relates to scenarios)- I


LtEED


KEY

1 : Given

Requires a decision


Figure 3-3. The experimental decision process.










At this point, the auditor also is asked for a preliminary
4
audit program (containing planned sample sizes for both compliance

tests and a substantive test5 relating to the specific objective).

After receiving specific results in the form of noncompliance rates

for the compliance tests, the auditor is asked to indicate the degree

of reliability he associates with the existing internal controls.

This second elicitation of reliability (R2) is made for all 12 cases

and can be represented as follows:


R2 = f (BI, II, 12, 13, RI, Cl, C2, C3, PP),


where R2 = a measure of reliability taken from a 7-point scale
(after observing results of compliance tests);

Cl =result of compliance test for Il;

C2 = result of compliance test for 12, if performed;

C3 = result of compliance test for 13; and

PP = preliminary program as planned by auditor prior to
receiving results of compliance testing.

Cl, C2, and C3 are trichotomous variables where either: (i) a compli-

ance test was not performed because the control does not exist, (ii)

the compliance test resulted in confirming evidence, or (iii) the

compliance test resulted in disconfirming evidence.


In the context of this study, "preliminary audit program"
implies an initial audit program that will possibly be adjusted for
the substantive test as a result of the evaluation of compliance test
results. At that time a "final audit program" is determined.

The confirmation of accounts receivable was chosen for use in
this study because of its most direct relationship with the sales
account and its familiarity to the subjects.










The fourth elicitation, a final sample size for the substantive

test (FS), also is required for all 12 cases and can be represented as

follows:


FS = f(BI, II, 12, 13, RI, Cl, C2, C3, PP, R2).

Experimental materials. The materials provided to the sub-

jects are presented in Appendix A and included the following:


(1) a set of instructions to explain the purpose of the study,
the setting in which the subject would operate, the
factors that would be varied, and the specific decisions
that were to be required of each subject;

(2) background information to describe the company to be
audited and additional information that was gathered
during preliminary audit work (i.e., the existence of
the key internal controls and other controls that may
impact on the attainment of the specific objective);

(3) a flowchart representation of the sales cycle of the
hypothetical company, with representations of the three
key internal controls;

(4) a statement of the specific audit objective to be con-
sidered and a summary description of the controls of
interest and the corresponding preliminary audit program
(containing procedures for both compliance testing and
the appropriate substantive test);

(5) specific information relating to each of the 12 individual
cases (this includes, for example, the extent to which the
three key internal controls exist and the results from
performing the desired amount of compliance testing);

(6) tables for determining the sample sizes for both compli-
ance tests and the substantive test using a statistical
sampling approach, along with tables for interpreting
the results of compliance tests which were presented in
the form of noncompliance rates for specific attributes;
and


(7) a post-experiment questionnaire.










Cue Selection


The decision to present an audit situation with a specific

objective was based primarily on the desire to maximize control over

cues provided to the subjects. The audit environment is recognized as

very complex with many interrelated components. To say that the three

key controls highlighted in this experiment, in conjunction with the

accounts receivable confirmation results, are the only important cues

in deciding whether "recorded sales are for valid transactions" is of

course not true. However, as a result of reviewing auditing textbooks,

audit manuals from "Big-Eight" accounting firms, and discussing this

particular audit situation with those auditors participating in the

pilot studies, the three key controls were considered most appropriate

in deciding the number of accounts receivable confirmations to send

relative to this particular objective.

Other information that could be deemed important in meeting the

specific objective was provided. It was indicated to subjects that the

review procedures revealed no problems relating to monthly statements,

customer complaints, and prenumbering documents. It also was stated

that normal review procedures would be followed in such related areas

as inventory cut-off, review of aged receivables, uncollectibles, etc.

Appendix A includes the information given the participants concerning

these areas and the primary cues to be used in determining the reli-

ability of the system and the resulting extent of substantive testing

for this particular audit objective.










Experimental Design


The experiment was conducted using a 2x3x2 factorial design

(fixed effects model)6 and was performed by 109 subjects. A fac-

torial design offers advantages in the evaluation of the cues (factors)

used in the experiment. Specifically, factorial designs allow for the

evaluation of a combined effect of two or more variables (factors)

when used simultaneously. The more complete information that results

has been described by Winer:


Information obtained from factorial experiments is
more complete than that obtained from a series of
single factor experiments, in the sense that factorial
experiments permit the evaluation of interaction
effects. An interaction effect is an effect attribut-
able to the combination of variables above and beyond
that which can be predicted from the variables con-
sidered singly [1971, p. 309].


The impact of interaction effects will vary from study to study. How-

ever, it is unusual for any higher order interactions greater than

those of two factors to contribute significantly to the model (e.g.,

Goldberg, 1968). Accordingly, the focus of this experiment upon main

effects and two-factor interactions should not constitute a serious

limitation.

As discussed previously, two broad scenarios were presented to

the subjects. Scenario #1 indicates that all three controls (factors)

exist and each control has dichotomous values, either: 1) a


6See Winer [1971, p. 312] for a discussion of the fixed effects
(factors) model.

There were a total of 119 packages distributed. Ten subjects
were not included in the analysis because they submitted incomplete
packages or responses that indicated a lack of understanding of the
requirements of the study.










2% noncompliance rate that implies confirming evidence, or 2) an 8%

noncompliance rate that implies disconfirming evidence.


Scenario #2 presents the situation where IC-2 does not exist

and therefore will have no compliance testing performed. This allows

for a third value (level) for IC-2 compared to only two values for IC-1

and IC-3. Therefore, the experiment has three factors (IC-1, IC-2,

and IC-3) with two possible levels for IC-1 and IC-3 and three possible

levels for IC-2. This results in the 2x3x2 factorial design with the

three factors and their levels as shown in Table 3-1.

The design allows for the estimation of all main effects and

two factor interactions with the 11 degrees of freedom available. The

degrees of freedom for individual analysis are used as follows:


effect d.f.
main 4
two factor interaction 5
higher-order (error) 2

total 11

As indicated, the mean square from the higher order interactions was

combined to form the error term.

The 12 cases, representing the 12 cells of the factorial

design, were presented in a random order within each scenario, with

Scenario #1 always presented first. Approximately one-third of the


8This percentage was given as the "expected" noncompliance
rate.

This percentage was developed through discussions with
auditors and references to other audit situations. It provides an
achieved precision level that is obviously unacceptable and therefore
represents "unexpected" or disconfirmingg" evidence.










TABLE 3-1

FACTORS AND LEVELS USED IN THE STUDY


Factors


Level


Results of
Compliance Tests


1. IC-1: Credit Approval 1 2% noncompliance rate

2 8% noncompliance rate



2. IC-2: Shipment Authoritation 1 2% compliance rate

2 8% noncompliance rate

3 Not performed



3. IC-3: Physical Shipment 1 2% noncompliance rate

2 8% noncompliance rate










subjects reviewed the cases in one randomized order and the remaining

two-thirds in a different randomized order.10


Description of the Auditor's Judgment Process


Univariate analysis of variance (ANOVA) was applied to the

reliability and sample size responses for the 12 factor combinations

of each subject. Also, ANOVA was applied to the responses by groups

according to firm and experience levels of the subjects.

F ratios and levels of significance were computed for each main

effect and two-factor interaction. As discussed previously, significant

interaction effects are indicative of configural processing and are not

predominant in human information processing research findings.

The omega-squared statistic was calculated for both reliability

and sample size responses to determine the proportion of variance

accounted for by each main effect and two-factor interaction. The

omega-squared value indicates the proportion of total variance accounted

for by a particular factor.


Judgment Consensus


Of major interest in this study is the degree of consensus or

agreement among subjects when responding to the same data. The degree

of consensus was evaluated at the audit program revision stage using

Pearson product-moment correlation, canonical correlation, and cluster

analysis.


10This imbalance resulted from the unexpectedly large number
of participants provided by one of the firms.










Pearson correlation measures were calculated among subjects on

their reliability decisions and then on the sample size decisions for

the 12 cases. This resulted in two measures of the strength of the

linear relationship of the responses between each pair of subjects.

Canonical correlation analysis can consider the reliability

and sample size resonses simultaneously, enabling the derivation of a

linear combination for each subject such that the correlation between

each pair of subjects is maximized. This correlation measure is the

canonical correlation developed and discussed for the 5,886 possible

combinations of subjects in this study.

The cluster analysis was performed both on the reliability and

sample size responses individually and combined. The cluster program

used a hierarchical clustering technique [Johnson, 1967] which begins

by forming a cluster for each subject in the analysis. Using a Euclid-

ean distance measure, the two closest clusters then are combined into

one cluster, the two closest of the new set then are combined into a

single cluster, and so on.


Self-Insight

Another area of relative importance in this study is the

extent of self-insight possessed by the subjects. That is, does the

auditor have a good "feel" for the judgment process he employed in

arriving at his decisions? This question specifically addresses the

auditor's ability to understand and express the relative weights or

importance of the cues used to arrive at his decisions.










To allow for the calculation of a self-insight index, a post-

experiment questionnaire asked each subject to allocate 100 points to

the three key internal controls so as to reflect the relative importance

of each to his decisions. A self-insight index was calculated following

the approach employed by Slovic [1969], Ashton [1973], and Messier

[1979]. The subjective weights elicited from the subjects were corre-

lated with adjusted objective weights calculated from the omega-squared

values (discussed earlier). The omega-squared values were adjusted by

normalizing the values to 100 for the main effects. The original

omega-squared value for a main effect was increased by the omega-

squared values for any interaction terms containing that particular main

effect. For example, the omega-squared values for the interaction terms

(IC-1)(IC-2) and (IC-1)(IC-3) were added to the omega-squared value for

the main effect of IC-1. The adjusted omega-squared values for the

three internal controls were summed and then each value was divided by

that sum. The results are the adjusted objective weights for the three

internal controls which sum to 100 and can be correlated with the sub-

jective weights given by the subjects to provide a self-insight index

for each subject.


Lack of Formal Hypotheses

The overall objective of this study as given previously is to

describe and analyze the auditor's judgment as reflected in a routine

audit situation. It should be stressed that due to the emphasis on

the descriptive nature of this study there are no formal hypotheses to

be tested. However, previous studies concerning auditor judgment










provide some basis for expectations. Accordingly, expected results of

the current study are discussed below.

The extent of auditor consensus relative to some recent judgment

studies was discussed in Chapter II. The current study is characterized

by a similar audit setting requiring similar decisions. Evidence con-

cerning consensus is provided at the preliminary audit program stage,

with more extensive evidence provided at the audit program revision

stage. Previous studies have indicated various degrees of consensus

among auditors. Due to the nature of the audit setting and the audit

task, some lack of consensus is expected in this study. However, the

existence of some consensus is more likely to appear as the auditors

are grouped according to firm affiliation or experience level.

More specifically, it is anticipated that moving from level 1

(confirming evidence) to level 2 disconfirmingg evidence) will cause

the degree of reliability of the controls to decrease and the sample

size for the substantive test to increase. However, the movement to

level 3 (control does not exist) from level 2 for factor 2 will not

necessarily have the same effect. The question becomes: Which situa-

tion is considered "stronger," to have a control that is ineffective in

terms of allowing an unacceptable noncompliance rate or for the control

not to exist? The control may not exist because it is not needed. For

this reason, it is expected that there will be a lack of consensus on

this point.

With regard to the decision models developed from the auditors'

responses, it is expected that the main effect variables will account

for most of the variance when combined in a linear fashion. Two-factor










interactions are expected to account for some variance, with higher-

order interactions being insignificant.

Disagreement among the auditors is expected with respect to

the relative importance of the three controls (variables). An a priori

subjective ranking of the relative importance of the controls within

the specific setting of this study is possible, but was not attempted

because of the subjective nature of such a ranking. Whatever the

relative importance of the variables attached by the auditors, pre-

vious accounting studies have indicated a high degree of self-insight

by the participants. A similar degree of self-insight is expected in

this study.


Administration of the Experiment


Pilot Studies


Several pilot studies were conducted to provide essential feed-

back on the experimental materials. Two initial sessions were conducted

with individual auditors who went through the experiment, followed by a

discussion of their comments and suggestions.

The first participant was a member of a local CPA firm who had

previously been with a "Big-Eight" firm and has a specific interest in

statistical sampling. His comments dealt mostly with clarifications

with respect to the representations on the flowchart and the use of the

statistical tables provided for the various decisions.

The second participant was chosen primarily because of his

affiliation with one of the firms that would be participating in the

primary study. His experience level and training also were comparable










to those participants in the primary study. Some changes as to wording

and clarity were made as a result of his comments. However, his major

contribution was that he felt members of his firm at his experience

level would feel "comfortable" with the audit situation presented by

the experimental materials.

The final pilot study was conducted in early August, 1979, with

three members of one of the primary study firms. Their major concern

was that other audit procedures important in this experimental setting

must be accounted for in some manner. Their specific recommendations

in this area along with some additional changes in wording were incor-

porated into the materials.

A final revision in the experimental materials was made as the

result of discussion with a "Big-Eight" partner with a strong research

background. His comments led to some key changes in the wording of

the specific objective, a change in one of the key controls, and other

important changes necessary for additional "realism" and clarity. The

basic structure and organization of the experimental materials and the

response modes, however, remained unchanged from earlier pilot studies.


Primary Study


The primary study was conducted with four public accounting

firms in August and September, 1979. The study was conducted in the

offices of three of the participating firms and in a conference room

of a hotel where a seminar was being conducted by the other firm. The

experimenter was in charge and present for the duration of all adminis-

trations of the experiment. The same set of instructions were read to

all participants with no additional comments made or questions answered










during the approximately 70 minutes of each administration. Although

there was no time limit given, participants were told to expect to

spend 50-60 minutes going through the experiment and actually spent

anywhere from 30 to 75 minutes with a mean of about 55 minutes.

The four firms that participated were all members of the "Big-

Eight." Three of the firms provided participants from a single office

in the same city. The other firm provided subjects from various

offices who were brought together through their participation in a

statistical sampling seminar. The experience levels of those partici-

pating varied somewhat from firm to firm and are summarized in Table

3-2 according to the experience classifications used in Chapter IV to

analyze the results of the study. Also, the number of participants

from each firm varied somewhat due to the fact that one criterion for

choosing subjects was their availability. The major difference was

with respect to Firm #4, where 75 usable responses were obtained as

compared to 12, 10, and 12 for Firms 1-3 respectively.


Limitations of the Experiment


There are certain inherent limitations associated with behav-

ioral studies. In addition to the problems with the development and

conduction of the experiment itself, other shortcomings arise relative

to the analysis of the information generated. Specific limitations of

this study are discussed below and include:


1. the selection of subjects;
2. the audit environment;
3. the administration of the experiment;
4. the ability to explain the judgment process; and
5. the generalizability of results.









TABLE 3-2

EXPERIENCE LEVEL OF SUBJECTS BY FIRM

Experience Level

E-l (1 or 2 yrs) E-2 (? 3 yrs)


Firm #1


Firm #2


Firm Firm #3


Firm #4


Total











Firm #1


Firm #2


Firm Firm #3


Firm #4


Total


12 12


2 8 10


11 1 12


39 34 73


52 55 107






Experience Level

X-i (1-3 yrs) X-2 (2 4 yrs) Total


3 9 12


6 4 10


12 12


64 9 73


85 22 107


NOTE: Subjects 100-4 and 102-4 could not be classified as to experi-
ence and are not included in this table.


Total










Selection of Subjects


Studies which require the use of auditors are restricted by

the availability of participants. To assemble a group of auditors

together for the sake of performing a task such as that required in

this study constitutes a substantial intrusion upon their time and

therefore is costly to their firms. For this reason, availability

becomes a major consideration in determining the participants from

each firm. Also, those responsible for the selection of participants

from each firm may want their "best" people to represent them in such

a study. On the other hand, the "best" people may be out on assign-

ment while the others participate in the "academic exercise" in the

office. Whatever the method of selection of participants, it must

be recognized as "non-random" and in some cases possibly biased.

Whether the results of such a sample represent the cross-section of

senior auditors from "Big-Eight" accounting firms is open for dis-

cussion. This limitation is therefore noted with the belief that the

impact of such a shortcoming on implications of this study is negli-

gible.


Audit Environment


The experimental materials were developed with the goal of

presenting the auditors with a realistic setting in which they would

have little difficulty responding to the questions presented. A

"self-contained" audit situation was sought to the extent possible in

order to better control for extraneous variables. However, this

results in the sacrifice of a certain amount of realism due to the

dynamic and inter-related nature of the audit process.










The resulting audit situation is admittedly lacking in some

"realism" characteristics. It was felt that any further attempts at

realism would have introduced confounding factors that serve to

reduce the internal validity of the experiment. The problem associ-

ated with creating a realistic audit setting is discussed in the

following section.


Administration of the Experiment


Although the experiment was conducted in a controlled setting

by the experimenter, some problems are still recognized. Whether an

auditor will make "meaningful" audit decisions in a "contrived" audit

situation with no reward or penalty structure must always be of concern.

However, positive feedback was received on the materials and the con-

duct of the experiment with no apparent motivational problems among

those participating.

The situation in which seventy-five participants from one firm

performed the study in a single room posed some administrative problems.

More participants were given the same order of cases as a result of

duplication problems and there was not the total lack of communication

with others that was assured with the small groups in the other firms.

These shortcomings did not seem to create any significant problems in

the current study.


Explaining the Judgment Process


To explain the judgment process is a most admirable research

goal. This section serves as a reminder that it was not the purpose

of this study to explain the judgment process of auditors within the










context of internal control evaluation. There is no attempt to infer

any understanding of the judgment process through the evaluation of

any statistical techniques such as ANOVA or MANOVA. However, the

results of the application of ANOVA procedures are presented as part

of the results of this study in an attempt to describe the judgment

process.


Generalizability of Results


A non-randomly chosen group of auditors was presented a

specific audit situation and asked to evaluate a hypothetical company

and respond to a set of questions regarding the extent of audit work

to perform. The participants spent approximately sixty minutes in

the experiment with no anticipation of reward or fear of penalty.

For these reasons, generalizing from the results of this study to

other audit situations for these auditors or to other auditors in the

same audit situation is not justified.

The inability to generalize the results of this study does not

distract from the specific benefits derived and the implications for

future research that will be discussed in Chapter V


Summary


This chapter has introduced the framework in which the current

study was conducted The lens model was presented along with ANOVA as

most appropriate for the analysis of the judgment process of auditors.

The dependence upon correlational measures was also emphasized.

Within this described framework, the experimental task was discussed,










both as to support for its selection and to the specific experimental

materials.

The selection of a 2x3x2 factorial design was discussed in

conjunction with the various possible levels for each factor (cue).

The analysis techniques that were used within the described framework

and to fulfill the objectives of this study were presented. These

included the use of correlation (both univariate and multivariate),

ANOVA (or MANOVA), cluster analysis, and "self-insight" indices (as

developed from omega-squared values).

Without the statement of formal hypotheses, some expected

results of this study were presented. While some of these results

were inferred from reference to similar previous studies, others were

inferred from an evaluation of the professional literature and the

practice of auditing. The procedures followed in both the pilot studies

and primary study were presented along with information concerning the

participants involved.

Finally, certain limitations of the experiment were presented

and discussed. None of the limitations are expected to have a signifi-

cant impact on the results of the study which are presented in Chapter

IV.
















CHAPTER IV

RESULTS OF THE STUDY



Introduction


The results of the study are presented and discussed in this

chapter. As noted in Chapter III, decisions were required by the

auditors at both the preliminary audit program stage and at the audit

program revision stage. Due to the structure of the study, the analy-

sis of responses from the audit program revision stage is emphasized.

This chapter first will present a discussion of the responses relating

to the formulation of a preliminary audit program. From examining these

preliminary decisions, some comments are made relating to a comparison

of the participating firms and to the relative importance of the three

IACs used in the study. Finally, and in much greater detail, an analy-

sis of the responses from the audit program revision stage is presented.

The additional data gathered from the twelve case situations

presented at the audit program revision stage allow for a more extensive

analysis. First, results of the descriptive analysis of the judgment

process of the auditors required in their IAC reliability and substan-

tive test sample size decisions are presented. Second, the extent of

judgment consensus is evaluated. Finally, the extent of self-insight

exhibited by the auditors is discussed. Although some interpretations

of the results in these areas are presented within this chapter, the pri-

mary evaluation of results and discussion of implications are provided









in Chapter V. The final sections of the present chapter present summa-

ries of additional data gathered in the post-experiment questionnaire.


Preliminary Audit Program


For the preliminary audit program stage, the auditors in the

study responded to questions on: (1) the reliability of the IAC system

based only on the knowledge of the existence (nonexistence) of the three

key internal controls, (2) the desired sample sizes for the compliance

tests to be performed on the existing controls, and (3) the preliminary

sample size for the substantive test. The limited amount of data gath-

ered from these questions restricts the analysis both within this stage

of the audit process and the comparison with the data gathered at the

audit program revision stage. However, some interesting relationships

can be established from examining responses from the preliminary audit

program stage.

Summaries of responses for the preliminary audit program are pre-

sented in Table 4-1 and Table 4-2. Table 4-1 provides the mean responses

by firm for the reliability and sample size (compliance and substantive

test) decisions. Using the data gathered at this point in the audit

process, some specific observations can be made regarding the partici-

pants and the three key internal accounting controls used in this study.


Comparison Among Firms

The preliminary responses shown in Table 4-1 provide a basis

for considering the responses of Firm #1 and Firm #4 to be similar,

while the auditors' resonses from Firm #2 and Firm #3 could also be

considered similar to one another. In addition, the preliminary

responses to the substantive test sample size questions are the basis










TABLE 4-1

MEAN RESPONSES BY FIRM FOR
THE PRELIMINARY AUDIT PROGRAM





SCENARIO #1 Existence of IC-1, IC-2, and IC-3


Firm #1 Firm #2 Firm #3 Firm #4 Combined


5.00

77.08

93.75

118.75

63.33


5.80

122.50

137.50

145.00

84.10


5.00

106.25

129.17

179.17

74.67


5.11

80.67

99.67

107.93

46.83


5.15

86.93

105.73

120.37

55.13


SCENARIO #2 Existence of IC-1 and IC-3


Firm #1 Firm #2 Firm #3


4.33 4.30 4.27

77.08 97.50 143.75

125.00 140.00 181.25

78.83 130.20 110.08


Reliability

CT: IC-1

CT: IC-2

CT: IC-3

Sub. Test


Reliability

CT: IC-1

CT: IC-3

Sub. Test


Firm #4


4.06

88.85

128.37

69.81


Combined


4.14

94.44

134.94

80.78










for considering Firm #2 and Firm #3 to be more conservative than Firm

#1 and Firm #4..

The preliminary reliability responses for the two scenarios

are as expected in that the reliability level decreased as IC-2 was

eliminated as an existing control. On the average, when moving to

Scenario #2, the reliability level decreased by 1.1 (on the 7-point

scale); however, for Firm #2 the reliability level decreased by 1.5.

Possible explanations are that either Firm #2 is more conservative

than the other firms, or that the auditors from Firm #2 perceived IC-2

to be more important than the auditors from the other firms. Evidence

of greater importance being attributed to IC-2 is provided by the sub-

sequent analysis of the responses of the auditors (i.e., see Table 4-2

and Table 4-15). In addition, indications of the conservativeness of

Firm #2 are found in the auditors' responses to the preliminary deci-

sion as to the sample size for the substantive test. For Scenario #1,

auditors from Firm #2 suggested a preliminary sample size for the sub-

stantive test of 84.10, as compared to a mean response from the auditors

of the other firms of 52.20. Similarly, when responding to the same

question for Scenario #2, auditors from Firm #2 had a mean response of

130.20 as compared to a mean response of 75.78 for all other auditors.

The above discussion provides evidence of some differences

among the firms at the preliminary audit program stage. While some

differences exist among the firms in the selection of sample sizes for

the compliance and substantive tests, an indication of agreement as to

the relative importance of the three IACs is evident.










Relative Importance of IACs


Table 4-2 summarizes the number of auditors who used each of

the various confidence levels in testing the three key IACs. Generally,

controls that are felt to be of greater importance are tested at a

higher confidence level than those controls of lesser importance.

Therefore, IC-3 (control over physical shipment) appears to be the most

important control across all firms. Although members of Firm #2 indi-

cated the same selection of confidence levels for IC-1 and IC-3 for

Scenario #1, the selections for Scenario #2 indicated slightly higher

confidence levels for IC-3. IC-2 (shipment authorization) seems to

follow IC-3 in importance, with IC-1 (credit approval) considered the

least important. As following sections will indicate, these responses

are consistent with the more extensively analyzed responses at the

audit program revision stage.


Audit Program Revision

The objectives of this study are pursued primarily through the

examination of the responses at the audit program revision stage. The

twelve case situations presented to the auditors included the results

of compliance testing. These tests assumed the use of the suggested

sample sizes that were elicited in the preliminary audit program. Due

to the number of cases, the data generated from the reliability and

substantive test sample size responses allow for a more extensive

evaluation and analysis than that from the preliminary audit program


Any statements concerning the relative importance of these
three controls are made only within the context of the current study
and should not be considered generalizable.











TABLE 4-2


NUMBER OF AUDITORS USING VARIOUS CONFIDENCE LEVELS FOR COMPLIANCE TESTING


SCENARIO #1 Compliance Testing of IC-1, IC-2, and IC-3


Firm #1
IC-1 IC-2 IC-3


Firm #2
IC-1 IC-2 IC-3
1 0 1
8 9 8
0 0 0
1 1 1
10 10 10


I


Firm #3 Firm #4
C-1 IC-2 IC-3 IC-1 IC-2 IC-3
5 4 2 33 27 18
5 6 7 38 45 49
1 1 2 2 3 7
1 1 1 0 0 1
12 12 12 73a 75 75


SCENARIO #2 Compliance Testing of IC-1 and IC-3


Confidence
Level
90 %
95 %
97.5%
99 %
Total


Firm #1
IC-1 IC-3
9 1
3 9
0 2
0 0
12 12


Firm #2
IC-1 IC-3
4 2
6 7
0 0
0 1
10 10


Firm #3
IC-1 IC-3
3 1
5 6
1 1
3 4
12 12


Firm #4
IC-1 IC-3
27 14
41 52
5 6
1 3
74a 75


Some auditors indicated zero sample sizes for the compliance testing of IC-1.


Confidence
Level
90 %
95 %
97.5%
99 %
Total










stage. The analysis that follows will include an examination of

responses made by the individual auditors as well as a comparison of

responses among the firms and the various experience levels.


Description of the Reliability/Sample Size Judgment

The experimental cases were arranged in a factorial design

(2x3x2) which permitted analysis-of-variance (ANOVA) computations for

each auditor participating in the study. ANOVA was used to evaluate

the relative importance of the key internal controls provided for this

specific reliabilty/sample size judgment.

The ANOVA analysis was performed individually for both the

reliability decisions and the sample size decisions. Therefore, an

ANOVA model was constructed for each subject corresponding to their

reliability decisions and to their sample size decisions. The omega-

squared values for each main effect and two-factor interaction are

presented by subject (auditor)2 in Table 4-3 (reliability judgment)

and Table 4-4 (sample size judgment).3 It will be recalled that the

omega-squared value indicates the proportion of total variance

accounted for by a particular IAC or combination of controls.

As is indicated in Tables 4-3 and 4-4, tests for significant

factors could not be conducted for seven auditors with respect to the


2The auditors are numbered from 1-1 to 109-4, with the second
number representing the firm affiliation of the auditor.

ANOVA was originally conducted using the 3-way interaction to
form the error term. With a mean-square-error of zero for 24 and 19
auditors on the reliability and sample size judgments, respectively, a
second ANOVA was conducted for these auditors using 2-way and 3-way
interactions to form the error term. The omega-squared calculations
resulting from this second ANOVA are noted with an asterisk (*) in
Tables 4-3 and 4-4, and are included in all further analyses unless
otherwise noted.










TABLE 4-3


SUMMARY OF ANOVA RESULTS RELIABILITY JUDGMENT


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
+1 1
*2 1 0.0 (.999) 8.8 (.088) 70.2 (.001) 79.0
*3 1 0.0 (.530) 37.0 (.005) 44.2 (.001) 81.2
*4 1 0.0 (.999) 38.5 (.001) 54.5 (.001) 93.0
5 1 40.0 (.008) 44.0 (.015) 8.0 (.038) 0.0 (.500) 0.0 (.423) 4.0 (.125) 96.0
6 1 70.2 (.010) 0.0 (.500) 10.6 (.057) 0.0 (.500) 10.6 (.057) 0.0 (.500) 91.4
7 1 0.0 (.423) 4.0 (.250) 80.0 (.012) 0.0 (.500) 0.0 (.423) 4.0 (.250) 88.0
8 1 4.5 (.184) 18.2 (.125) 53.0 (.207) 9.1 (.200) 0.0 (.423) 0.0 (.999) 84.8
*9 1 0.0 (.999) 8.8 (.088) 71.9 (.001) 80.7
110 1
11 1 0.0 (.423) 0.0 (.500) 93.3 (.006) 0.0 (.500) 0.0 (.423) 0.0 (.500) 93.3
12 1 0.0 (.243) 4.0 (.250) 80.0 (.012) 0.0 (.500) 0.0 (.423) 4.0 (.250) 88.0
*13 2 0.0 (.999) 32.5 (.001) 59.8 (.001) 92.3
14 2 0.0 (.622) 44.7 (.058) 37.9 (.035) 0.0 (.750) 2.9 (.225) 0.0 (.500) 85.5
15 2 15.6 (.147) 38.6 (.136) 15.6 (.147) 0.0 (.750) 0.0 (.999) 0.0 (.750) 69.8
16 2 1.7 (.184) 54.2 (.020) 35.6 (.015) 0.0 (.500) 1.7 (.184) 0.0 (.500) 93.2
17 2 0.0 (.423) 0.0 (.500) 66.7 (.038) 0.0 (.500) 0.0 (.423) 0.0 (.500) 66.7
*18 2 9.3 (.001) 83.0 (.001) 3.9 (.007) 96.2
19 2 0.0 (.423) 21.1 (.039) 52.6 (.008) 0.0 (.500) 0.0 (.423) 21.1 (.039) 94.8
20 2 58.6 (.010) 23.7 (.046) 8.9 (.057) 0.0 (.500) 1.8 (.184) 0.0 (.500) 93.0
21 2 26.7 (.012) 28.0 (.023) 40.0 (.008) 1.3 (.250) 0.0 (.423) 0.0 (.500) 96.0
*22 2 0.0 (.999) 41.6 (.001) 48.6 (.001) 90.2
23 3 1.1 (.368) 0.0 (.500) 70.1 (.044) 0.0 (.750) 0.0 (.999) 0.0 (.750) 71.2

Note: The values shown in this table are omega-squared values (significance levels).
-i ANOVA was not possible for these subjects due to the lack of variability in their responses.
* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-3 continued


FACTORS
Subi Firm IC-1 IC-2 IC-3 IC-I x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
24 3 2.3 (.272) 39.5 (.077) 34.6 (.046) 0.0 (.999) 0.0 (.667) 7.2 (.250) 83.6
*25 3 2.3 (.104) 57.5 (.001) 28.7 (.001) 88.5
26 3 37.3 (.015) 23.7 (.046) 20.7 (.027) 7.1 (.125) 1.8 (.184) 2.3 (.250) 92.9
*27 3 0.0 (.999) 41.6 (.001) 48.6 (.001) 90.2
28 3 1.0 (.375) 17.1 (.219) 54.3 (.053) 0.0 (.700) 0.0 (.742) 0.0 (.875) 72.4
29 3 26.4 (.044) 15.6 (.125) 42.0 (.029) 0.0 (.750) 0.0 (.999) 3.5 (.300) 87.5
30 3 2.5 (.184) 33.0 (.046) 52.1 (.015) 0.0 (.500) 2.5 (.184) 0.0 (.500) 90.1
t31 3
32 3 5.0 (.038) 0.8 (.250) 90.9 (.002) 0.0 (.500) 0.0 (.423) 0.8 (.250) 97.5
33 3 4.5 (.225) 32.9 (.107) 34.3 (.056) 0.0 (.500) 4.5 (.225) 0.0 (.750) 76.2
*34 3 6.7 (.033) 27.9 (.004) 51.9 (.001) 86.5
*35 4 19.0 (.014) 38.0 (.008) 19.0 (.014) 76.0
36 4 11.7 (.015) 6.7 (.050) 74.3 (.003) 0.0 (.500) 2.8 (.057) 2.2 (.125) 97.7
37 4 17.4 (.027) 23.9 (.039) 31.3 (.015) 7.9 (.100) 1.5 (.184) 11.9 (.071) 93.9
38 4 7.4 (.096) 66.7 (.026) 7.4 (.096) 3.7 (.250) 0.0 (.423) 3.7 (.250) 88.9
39 4 32.4 (.020) 27.0 (.046) 32.4 (.020) 0.0 (.500) 0.0 (.423) 0.0 (.500) 91.8
40 4 2.1 (.184) 8.5 (.125) 70.2 (.010) 0.0 (.500) 2.1 (.184) 8.5 (.125) 91.4
*41 4 1.8 (.033) 3.7 (.021) 90.8 (.001) 96.3
42 4 0.0 (.423) 15.4 (.071) 76.9 (.008) 0.0 (.500) 0.0 (.423) 0.0 (.500) 92.3
43 4 10.8 (.020) 16.2 (.026) 64.9 (.033) 5.4 (.071) 0.0 (.423) 0.0 (.500) 97.3
44 4 15.2 (.094) 0.0 (.500) 66.4 (.026) 0.0 (.875) 2.4 (.270) 0.0 (.875) 84.0
45 4 5.2 (.100) 23.1 (.050) 51.3 (.012) 2.5 (.250) 0.0 (.423) 10.2 (.100) 92.3
46 4 5.5 (.118) 4.7 (.219) 81.1 (.011) 0.0 (.500) 0.0 (.742) 0.0 (.875) 91.3


Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.








TABLE 4-3 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
47 4 3.7 (.184) 0.0 (.500) 77.8 (.015) 0.0 (.500) 3.7 (.184) 0.0 (.500) 85.2
48 4 6.8 (.147) 29.3 (.089) 50.8 (.029) 0.0 (.750) 0.0 (.999) 0.0 (.750) 86.9
49 4 8.0 (.038) 8.0 (.071) 74.4 (.004) 4.0 (.125) 0.0 (.423) 1.3 (.250) 96.0
50 4 13.3 (.038) 33.3 (.031) 44.5 (.012) 0.0 (.500) 0.0 (.423) 2.2 (.250) 93.3
51 4 0.0 (.423) 5.9 (.125) 82.4 (.006) 0.0 (.500) 0.0 (.423) 5.9 (.125) 94.2
52 4 6.9 (.038) 0.0 (.500) 82.8 (.003) 0.0 (.500) 6.9 (.038) 0.0 (.500) 96.6
53 4 0.0 (.423) 21.0 (.100) 63.2 (.020) 0.0 (.500) 0.0 (.423) 0.0 (.500) 84.2
54 4 0.0 (.423) 14.2 (.250) 28.8 (.096) 0.0 (.500) 0.0 (.423) 14.2 (.250) 57.2
55 4 44.4 (.020) 33.3 (.050) 0.0 (.423) 11.1 (.125) 0.0 (.423) 0.0 (.500) 88.8
56 4 3.1 (.368) 6.3 (.429) 18.7 (.225) 0.0 (.750) 0.0 (.999) 0.0 (.999) 28.1
57 4 5.7 (.270) 37.7 (.159) 18.3 (.152) 0.0 (.875) 0.0 (.529) 0.0 (.700) 61.7
58 4 35.1 (.035) 32.4 (.071) 9.9 (.102) 0.0 (.500) 2.7 (.225) 3.6 (.300) 83.7
59 4 9.5 (.102) 40.0 (.058) 33.9 (.035) 0.0 (.750) 2.6 (.225) 0.0 (.500) 86.0
*60 4 1.6 (.228) 17.6 (.041) 56.0 (.001) 75.2
61 4 3.7 (.270) 34.6 (.125) 12.0 (.152) 0.0 (.500) 3.7 (.270) 11.5 (.250) 65.5
62 4 0.0 (.999) 63.2 (.050) 7.6 (.147) 0.0 (.500) 0.0 (.999) 11.7 (.188) 82.5
t63 4
64 4 0.0 (.423) 3.7 (.125) 88.9 (.003) 0.0 (.500) 0.0 (.423) 3.7 (.125) 96.3
65 4 52.4 (.010) 25.4 (.039) 7.9 (.057) 0.0 (.500) 7.9 (.057) 0.0 (.500) 93.6
66 4 2.7 (.100) 12.0 (.050) 74.7 (.004) 1.3 (.250) 0.0 (.423) 5.3 (.100) 96.0
67 4 8.2 (.038) 75.3 (.009) 2.8 (.100) 0.0 (.500) 8.2 (.038) 1.4 (.250) 95.9
68 4 0.0 (.423) 73.0 (.006) 10.8 (.020) 0.0 (.500) 0.0 (.423) 13.5 (.031) 97.3
69 4 6.2 (.038) 28.9 (.017) 57.8 (.004) 3.1 (.125) 0.0 (.423) 1.0 (.250) 97.0


Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-3 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-I x IC-3 IC-2 x IC-3 Total
t70 4
71 4 7.0 (.225) 23.3 (.188) 25 5 (.102) 0.0 (.500) 7.0 (.225) 0.0 (.750) 62.8
72 4 3.5 (.270) 0.0 (.500) 74.1 (.034) 0.0 (.875) 0.0 (.529) 0.0 (.875) 77.6
73 4 9.6 (.100) 57.1 (.039) 9.6 (.100) 4.7 (.250) 0.0 (.423) 4.7 (.250) 85.7
74 4 8.5 (.057) 20.3 (.050) 55.9 (.010) 0.0 (.500) 1.7 (.184) 6.8 (.125) 93.2
75 4 0.0 (.423) 4.1 (.100) 92.8 (.003) 0.0 (.500) 0.0 (.423) 0.0 (.500) 96.9
76 4 31.2 (.046) 25.9 (.100) 31.2 (.046) 0.0 (.999) 0.0 (.667) 0.0 (.999) 88.3
*77 4 0.0 (.999) 7.1 (.088) 77.3 (.001) 84.4
78 4 0.0 (.423) 2.6 (.125) 94.0 (.002) 0.0 (.500) 0.0 (.423) 0.8 (.250) 97.4
79 4 15.5 (.096) 7.6 (.250) 46.2 (.038) 7.6 (.250) 0.0 (.423) 0.0 (.500) 76.9
80 4 0.0 (.423) 17.8 (.036) 76.7 (.004) 0.0 (.500) 0.0 (.423) 1.4 (.250) 95.9
81 4 12.6 (.149) 39.8 (.115) 19.4 (.111) 0.0 (.594) 0.0 (.691) 0.0 (.679) 71.8
82 4 39.2 (.012) 29.4 (.031) 23.5 (.020) 1.9 (.250) 0.0 (.423) 0.0 (.500) 94.0
83 4 0.0 (.493) 15.8 (.250) 47.6 (.069) 0.0 (.813) 0.0 (.808) 0.0 (.500) 63.4
*84 4 3.9 (.104) 29.7 (.008) 47.6 (.001) 81.2
85 4 71.3 (.030) 0.0 (.875) 13.7 (.118) 0.0 (.700) 0.0 (.742) 0.0 (.875) 85.0
t86 4
87 4 1.0 (.272) 4.6 (.200) 84.2 (.009) 0.0 (.999) 0.0 (.667) 3.1 (.250) 92.9
88 4 36.4 (.038) 9.1 (.200) 36.4 (.038) 0.0 (.500) 0.0 (.423) 0.0 (.500) 81.9
89 4 3.3 (.057) 16.6 (.025) 70.7 (.003) 2.6 (.125) 0.7 (.184) 3.5 (.100) 97.4
90 4 7.0 (.057) 45.1 (.020) 29.6 (.015) 0.0 (.500) 1.4 (.184) 11.3 (.071) 94.4
91 4 13.0 (.118) 1.2 (.438) 67.7 (.030) 0.0 (.875) 0.0 (.742) 0.0 (.700) 81.9
92 4 0.0 (.742) 0.0 (.875) 4.9 (.375) 0.0 (.875) 0.0 (.742) 0.0 (.700) 4.9

Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-3 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
93 4 12.8 (.057) 20.5 (.071) 53.8 (.015) 0.0 (.500) 2.6 (.184) 0.0 (.500) 89.7
94 4 10.9 (.038) 7.3 (.100) 76.4 (.006) 0.0 (.500) 0.0 (.423) 0.0 (.500) 94.6
95 4 0.0 (.574) 0.0 (.900) 67.5 (.057) 0.0 (.692) 0.0 (.423) 0.0 (.750) 67.5
96 4 16.2 (.094) 20.5 (.140) 16.2 (.094) 3.4 (.350) 2.6 (.270) 17.1 (.159) 76.0
97 4 42.0 (.029) 15.6 (.125) 26.4 (.044) 0.0 (.750) 0.0 (.999) 3.5 (.300) 87.5
98 4 9.6 (.096) 42.9 (.050) 9.6 (.096) 4.7 (.250) 0.0 (.423) 19.0 (.100) 85.8
99 4 10.9 (.038) 7.3 (.100) 76.4 (.006) 0.0 (.500) 0.0 (.423) 0.0 (.500) 94.6
*100 4 53.6 (.001) 4.9 (.088) 29.7 (.001) 88.2
101 4 44.4 (.020) 0.0 (.500) 22.2 (.038) 0.0 (.500) 22.2 (.038) 0.0 (.500) 88.8
102 4 51.3 (.012) 38.5 (.031) 0.0 (.423) 2.5 (.250) 0.0 (.423) 0.0 (.500) 92.3
103 4 49.1 (.020) 19.5 (.088) 11.5 (.074) 4.2 (.250) 0.0 (.999) 4.2 (.250) 88.5
104 4 1.3 (.270) 0.0 (.500) 80.0 (.012) 6.8 (.184) 1.3 (.270) 0.0 (.700) 89.4
t105 4
106 4 4.9 (.057) 7.8 (.071) 82.5 (.004) 0.0 (.500) 1.0 (.184) 0.0 (.500) 96.2
*107 4 0.0 (.999) 3.1 (.088) 90.2 (.001) 93.3
108 4 19.2 (.189) 0.0 (.700) 32.0 (.138) 0.0 (.875) 0.0 (.580) 0.0 (.750) 51.2
109 4 3.4 (.074) 87.3 (.007) 3.4 (.074) 1.2 (.250) 0.0 (.999) 1.2 (.250) 96.5

COMPOSITE 4.5 (.001 8.1 (.001) 32.3 (.001) 0.0 (.493) 0.1 (.038) 0.4 (.003) 45.4

Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.











TABLE 4-4


SUMMARY OF ANOVA RESULTS SAMPLE SIZE JUDGMENT


IC-2
0.0 (.500)
2.5 (.250)
40.2 (.001)
11.8 (.196)
58.8 (.005)
4.0 (.316)
15.5 (.008)
9.1 (.209)
5.2 (.088)


0.0
0.0
0.0
5.7
29.7
65.5
0.0
10.7
0.0


0.0
3.0
0.0
0.5
1.7
0.0
13.0
0.0
40.2
32.9
0.0
8.6


IC-3
63.9 (.038)
87.7 (.007)
53.0 (.001)
52.2 (.033)
1.3 (.081)
3.0 (.240)
61.4 (.001)
58.7 (.026)
83.5 (.001)


87.7 (.007)
20.3 (.007)
42.0 (.039)
0.5 (.309)
55.9 (.010)
55.6 (.052)
13.0 (.116)
63.1 (.001)
3.2 (.288)
32.9 (.007)
54.8 (.001)
59.1 (.031)


FACTORS
IC-1 x IC-2
0.0 (.500)
0.0 (.500)

0.0 (.500)
7.8 (.032)
4.0 (.316)
0.0 (.500)
6.0 (.262)



0.0 (.500)
5.9 (.045)
0.0 (.500)
0.0 (.734)
0.0 (.500)
0.0 (.500)
3.9 (.345)
0.0 (.500)
6.3 (.329)
2.1 (.155)

0.0 (.553)


IC-1 x IC-3
0.0 (.423)
0.0 (.423)


(.184)
(.358)
(.240)
(.423)
(.659)



(.423)
(.423)
(.423)
(.535)
(.184)
(.423)
(.201)
(.423)
(.805)
(.057)


IC-2 x IC-3
0.0 (.500)
2.5 (.250)


1.7
0.7
0.0
22.4
0.0



2.5
1.3
18.3
0.0
0.0
0.0
3.9
11.2
7.1
2.1


(.423)
(.423)
(.999)
(.184)
(.004)
(.025)
(.423)
(.111)
(.999)


(.423)
(.044)
(.423)
(.309)
(.184)
(.A23)
(.116)
(.423)
(.064)
(.007)
(.999)
(.146)


Subi Firm


(.250)
(.004)
(.146)
(.012)
(.031)
(.288)
(.097)
(.001)
(.281)
(.020)
(.001)
(.250)


IC-1


0.0 (.416) 1.9 (.404)


2.5
67.8
17.8
93.8
33.9
9.8
35.5
25.5
9.4
23.4
42.5
7.8


Total
63.9
92.7
93.2
77.1
98.4
79.5
99.3
84.5
88.7


92.7
98.3
78.1
94.8
93.2
65.4
74.7
99.8
66.2
97.0
97.3
77.5


(.409)
(.210)
(.500)
(.006)
(.908)



(.250)
(.158)
(.143)
(.734)
(.500)
(.837)
(.345)
(.002)
(.315)
(.155)


Note: The values shown in this table are omega-squared values (significance levels).
i ANOVA was not possible for these subjects due to the lack of variability in their responses.
* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-4 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
24 3 1.0 (.300) 26.0 (.071) 43.0 (.024) 0.0 (.929) 0.0 (.808) 20.0 (.089) 90.0
25 3 0.5 (.369) 41.2 (.066) 45.4 (.032) 0.0 (.737) 0.0 (.869) 0.0 (.803) 87.1
26 3 18.6 (.168) 19.3 (.262) 18.6 (.168) 0.0 (.791) 0.0 (.747) 0.0 (.791) 56.5
27 3 0.0 (.423) 63.6 (.008) 26.2 (.010) 0.0 (.500) 0.0 (.423) 6.9 (.068) 96.7
28 3 1.2 (.356) 19.2 (.189) 48.5 (.052) 0.0 (.541) 0.0 (.721) 0.0 (.518) 68.9
29 3 38.2 (.008) 32.6 (.017) 26.0 (.011) 0.0 (.500) 0.0 (.667) 0.0 (.500) 96.8
30 3 14.4 (.023) 25.1 (.027) 52.6 (.007) 1.1 (.279) 0.8 (.208) 1.7 (.229) 95.7
31 3 30.6 (.124) 7.7 (.370) 3.3 (.332) 0.0 (.587) 1.1 (.387) 0.0 (.643) 42.7
32 3 12.4 (.026) 8.1 (.074) 63.2 (.005) 0.0 (.500) 4.1 (.070) 8.1 (.074) 95.9
33 3 0.0 (.480) 46.5 (.153) 15.2 (.185) 0.0 (.943) 0.0 (.629) 0.0 (.880) 61.7
34 3 6.3 (.012) 37.6 (.004) 45.1 (.002) 0.0 (.500) 6.3 (.012) 3.9 (.036) 99.2
35 4 16.4 (.078) 43.8 (.063) 9.6 (.117) 11.5 (.177) 1.9 (.278) 0.0 (.765) 83.2
36 4 30.6 (.006) 3.2 (.093) 63.0 (.003) 0.6 (.284) 0.0 (.742) 0.6 (.269) 98.0
37 4 0.0 (.423) 64.5 (.014) 20.4 (.021) 0.0 (.500) 0.0 (.423) 9.7 (.079) 94.6
38 4 15.1 (.166) 35.8 (.161) 15.1 (.166) 0.0 (.828) 0.0 (.716) 0.0 (.828) 66.0
*39 4 49.8 (.001) 0.0 (.999) 49.8 (.001) 99.6
40 4 0.0 (.762) 1.0 (.472) 45.7 (.076) 13.2 (.283) 0.0 (.762) 0.0 (.651) 59.9
41 4 0.4 (.369) 0.0 (.500) 88.9 (.014) 0.0 (.735) 0.0 (.860) 0.0 (.735) 89.3
42 4 0.0 (.423) 6.3 (.107) 82.3 (.005) 0.0 (.500) 0.0 (.423) 6.3 (.107) 94.9
43 4 15.2 (.039) 36.7 (.033) 33.6 (.019) 1.8 (.294) 1.7 (.199) 3.2 (.226) 92.2
44 4 43.0 (.025) 4.5 (.258) 16.9 (.060) 4.5 (.258) 16.9 (.020) 0.0 (.500) 85.8
45 4 5.3 (.197) 39.4 (.085) 21.3 (.076) 3.5 (.348) 0.0 (.751) 8.1 (.250) 77.6
46 4 6.4 (.022) 4.5 (.058) 86.3 (.002) 0.8 (.214) 0.2 (.283) 0.1 (.454) 98.3


Note: The values shown in this table are omega-squared values (significance levels).

I ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-4 continued


FACTORS
Subj Firm IC-I IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
t47 4
48 4 9.9 (.105) 16.1 (.130) 46.2 (.028) 2.8 (.333) 4.2 (.184) 4.0 (.293) 83.2
49 4 0.0 (.493) 6.8 (.110) 80.6 (.006) 3.3 (.184) 3.2 (.111) 0.5 (.401) 94.4
50 4 0.4 (.368) 28.5 (.075) 59.0 (.020) 0.0 (.750) 0.0 (.999) 0.0 (.500) 87.9
51 4 14.2 (.041) 0.0 (.500) 71.3 (.009) 1.7 (.300) 3.4 (.130) 1.7 (.300) 92.3
*52 4 3.7 (.033) 0.0 (.999) 90.1 (.001) 93.8
53 4 0.0 (.831) 26.3 (.151) 48.8 (.051) 0.0 (.749) 0.0 (.485) 0.0 (.608) 75.1
54 4 1.5 (.369) 59.3 (.117) 1.5 (.369) 0.0 (.743) 0.0 (.903) 0.0 (.743) 62.3
55 4 0.0 (.732) 40.1 (.109) 17.1 (.117) 9.4 (.271) 7.1 (.201) 0.0 (.878) 73.7
56 4 2.6 (.368) 0.0 (.500) 33.3 (.147) 0.0 (.750) 0.0 (.999) 0.0 (.750) 35.9
57 4 1.0 (.394) 7.3 (.389) 26.5 (.151) 0.0 (.684) 1.0 (.394) 0.0 (.623) 35.8
58 4 14.1 (.068) 36.7 (.056) 26.4 (.040) 0.0 (.629) 0.0 (.866) 11.0 (.149) 88.2
59 4 2.5 (.208) 20.1 (.087) 63.5 (.016) 0.0 (.500) 2.5 (.208) 0.0 (.731) 88.6
60 4 7.3 (.219) 19.6 (.205) 36.1 (.077) 0.0 (.649) 4.2 (.275) 0.0 (.792) 67.2
61 4 1.3 (.337) 62.1 (.066) 7.0 (.183) 0.0 (.500) 1.3 (.337) 0.1 (.495) 71.8
62 4 0.0 (.423) 80.0 (.013) 7.6 (.062) 0.0 (.500) 0.0 (.423) 5.8 (.138) 93.4
t63 4
*64 4 0.0 (.999) 5.2 (.088) 83.5 (.001) 88.7
65 4 25.5 (.122) 8.5 (.338) 25.5 (.122) 0.0 (.658) 0.0 (.860) 0.0 (.658) 59.5
66 4 1.1 (.321) 13.0 (.160) 67.1 (.022) 0.0 (.871) 0.0 (.926) 4.6 (.286) 85.8
67 4 17.2 (.047) 54.8 (.032) 11.6 (.067) 0.9 (.404) 0.0 (.726) 5.1 (.210) 89.6
68 4 0.0 (.423) 66.7 (.031) 12.2 (.073) 0.0 (.500) 0.0 (.423) 8.1 (.174) 87.0
69 4 0.8 (.333) 19.5 (.107) 66.1 (.019) 0.0 (.824) 0.0 (.873) 1.0 (.418) 87.4

Note: The values shown in this table are omega-squared values (significance levels).
t ANOVA was not possible for these subjects due to the lack of variability in their responses.
* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-4 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1 x IC-2 IC-1 x IC-3 IC-2 x IC-3 Total
t70 4
71 4 6.4 (.233) 24.5 (.179) 6.4 (.233) 0.0 (.500) 6.4 (.233) 15.5 (.234) 59.2
72 4 0.0 (.423) 0.0 (.500) 66.7 (.038) 0.0 (.500) 0.0 (.423) 0.0 (.500) 66.7
73 4 25.4 (.054) 17.5 (.132) 25.4 (.054) 4.6 (.289) 3.8 (.206) 4.6 (.289) 81.3
74 4 7.7 (.005) 24.7 (.003) 59.2 (.001) 0.0 (.500) 7.7 (.005) 0.1 (.265) 99.4
*75 4 0.0 (.999) 7.1 (.088) 77.5 (.001) 84.6
76 4 30.9 (.068) 14.8 (.202) 30.9 (.068) 0.0 (.851) 1.5 (.336) 0.0 (.851) 78.1
77 4 0.0 (.423) 39.1 (.018) 54.5 (.007) 0.0 (.500) 0.0 (.423) 1.9 (.216) 95.5
t78 4
79 4 9.1 (.184) 0.0 (.500) 45.5 (.057) 0.0 (.500) 9.1 (.184) 0.0 (.500) 63.7
*80 4 0.0 (.999) 25.8 (.038) 42.0 (.006) 67.8
81 4 13.3 (.077) 29.1 (.074) 23.4 (.048) 10.0 (.168) 0.0 (.935) 10.3 (.166) 86.1
82 4 40.9 (.006) 27.7 (.017) 20.2 (.012) 0.0 (.500) 0.3 (.270) 7.8 (.057) 96.9
83 4 8.4 (.159) 37.0 (.096) 32.3 (.058) 0.0 (.744) 0.0 (.966) 1.1 (.445) 78.8
84 4 2.8 (.050) 3.5 (.077) 78.4 (.002) 0.6 (.253) 6.5 (.023) 6.2 (.047) 98.0
85 4 26.6 (.102) 6.4 (.347) 5.0 (.262) 7.0 (.337) 4.3 (.277) 0.3 (.489) 49.6
t86 4
87 4 6.6 (.124) 12.5 (.138) 54.8 (.021) 0.0 (.737) 8.4 (.104) 5.0 (.243) 87.3
88 4 41.6 (.001) 12.3 (.007) 36.7 (.001) 3.4 (.025) 5.1 (.009) 0.3 (.181) 99.4
89 4 1.5 (.029) 37.1 (.003) 51.5 (.001) 3.0 (.029) 0.0 (.423) 6.4 (.014) 99.5
90 4 9.6 (.116) 48.6 (.057) 22.0 (.060) 2.4 (.362) 0.0 (.711) 0.2 (.488) 82.8
91 4 26.4 (.020) 0.0 (.500) 65.9 (.008) 0.0 (.500) 1.1 (.222) 0.0 (.500) 93.4
92 4 0.2 (.368) 44.7 (.026) 20.1 (.028) 0.0 (.750) 0.0 (.999) 29.1 (.039) 94.1


Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.










TABLE 4-4 continued


FACTORS
Subj Firm IC-1 IC-2 IC-3 IC-1x IC-2 IC- x IC-3 IC-2 x IC-3 Total
93 4 8.1 (.022) 39.5 (.009) 45.1 (.004) 2.3 (.123) 4.6 (.060) 0.0 (.500) 97.6
94 4 2.9 (.243) 7.2 (.243) 63.1 (.025) 7.5 (.239) 0.0 (.438) 0.0 (.591) 80.7
95 4 0.1 (.417) 0.9 (.461) 69.3 (.035) 0.0 (.516) 0.0 (.442) 0.0 (.559) 70.3
96 4 0.0 (.434) 0.0 (.527) 70.7 (.044) 0.0 (.671) 0.0 (.675) 0.0 (.816) 70.7
97 4 21.3 (.045) 24.7 (.074) 40.7 (.025) 0.6 (.436) 0.0 (.616) 0.5 (.449) 87.8
98 4 22.2 (.096) 33.3 (.125) 0.0 (.423) 11.1 (.250) 0.0 (.423) 0.0 (.500) 66.6
*99 4 3.3 (.209) 36.0 (.030) 18.0 (.043) 57.3
*100 4 48.8 (.001) 0.0 (.999) 48.8 (.001) 97.6
101 4 68.4 (.020) 6.0 (.247) 0.0 (.664) 9.8 (.188) 0.9 (.335) 0.0 (.700) 85.1
102 4 22.5 (.001) 52.3 (.001) 0.0 (.423) 24.9 (.002) 0.0 (.423) 0.0 (.500) 99.7
103 4 39.5 (.098) 4.2 (.415) 1.7 (.368) 5.0 (.402) 0.0 (.999) 0.0 (.750) 50.4
*104 4 0.0 (.999) 58.5 (.008) 7.7 (.104) 66.2
1-105 4
106 4 5.2 (.155) 3.8 (.289) 76.5 (.016) 0.5 (.455) 0.0 (.900) 0.0 (.534) 86.0
107 4 0.0 (.721) 13.6 (.258) 35.4 (.081) 0.0 (.500) 0.0 (.721) 13.6 (.258) 62.6
108 4 16.7 (.161) 0.0 (.571) 41.2 (.085) 0.0 (.500) 0.0 (.684) 0.0 (.734) 57.9
109 4 10.8 (.272) 6.3 (.422) 18.9 (.216) 0.0 (.999) 0.0 (.667) 0.0 (.925) 36.0

COMPOSITE 1.5 (.001) 3.0 (.001) 6.3 (.001) 0.1 (.267) 0.0 (.487) 0.0 (.423) 10.9


Note: The values shown in this table are omega-squared values (significance levels).

t ANOVA was not possible for these subjects due to the lack of variability in their responses.

* These subjects required the use of 2-way and 3-way interactions to form the error term.










reliability judgments and for eight auditors with respect to the sample

size judgments. An examination of the responses of these auditors

revealed a lack of variability in their responses. In fourteen of the

fifteen cases, the auditor's responses fluctuated between only two

numbers. For example, auditor number 10 from Firm #1 responded with

either a 4 or 2 for the reliability judgment and either 80 or 107 for

the sample size judgment. The following discussion of individual ANOVA

results is based on the 102 usable results from the reliability judg-

ment and 101 usable results from the sample size judgments.

Results from the individual ANOVAs (Tables 4-3 and 4-4) indi-

cate that IC-3 (control over physical shipment) accounted for most of

the variance in their judgments. Specifically, 65 of 102 auditors for

the reliability judgment and 57 of 101 for the sample size judgment

indicated greatest reliance on IC-3. Auditors whose responses indi-

cated IC-2 (shipment authorization) accounted for most of the variance

numbered 21 and 28 for reliability and sample size, respectively,

whereas the result for IC-1 (credit approval) indicated 13 and 10

auditors for reliability and sample size. The number of auditors whose

responses indicated the highest proportion of variance was attributed

equally to IC-1 and IC-3 were 3 and 6 for the reliability and sample

size judgments, respectively. Results also indicated that IC-3 was

significant (ai .05) for 73 auditors on the reliability judgment and

for 62 auditors on the sample size judgment. Similarly, IC-2 and IC-1

were significant for 42 and 34 auditors, respectively, on the relia-

bility judgment and for 31 and 27 auditors, respectively, on the sample

size judgment.










The fact that IC-3 accounted for most of the variance in the

auditors' judgments did not preclude IC-1 and IC-2 from accounting for

much of the variance in many of the auditors' judgments. For example,

in the reliability and sample size judgments, IC-3 accounted for the

least amount of variance for 17 and 19 auditors, respectively. There-

fore, there is considerable disagreement among some auditors as to the

relative importance of the IACs evaluated in this study. This point

can be substantiated with respect to the reliability judgment by

examining Table 4-5 where omega-squared values for six of the auditors

are presented.

The auditors were selected for Table 4-5 as examples of the

variance attributed to each of the IACs and their interaction terms.

The results are shown in this manner to indicate the relative impor-

tance of IC-1, IC-2, and IC-3 to auditors 6-1, 109-4, and 11-1,

respectively, and the relative importance of certain interaction terms

to auditors 37-4, 101-4, and 19-2. As indicated, auditor number 6-1

relied heavily on IC-1 and to some extent on IC-3 but to no signifi-

cant extent on IC-2. In sharp contrast, auditors 109-4 and 11-1 relied

almost exclusively on IC-2 and IC-3, respectively, when making the

reliability judgment.

Referring again to Table 4-3, the importance of interaction

effects can be seen. There were 15 auditors who had at least one sig-

nificant (a f.10) interaction term in their reliability judgments.

Three examples shown in Table 4-5 include auditors number 37-4, 101-4

and 19-2 who reflect significant interactions for IC- xIC-2, IC-1 x

IC-3, and IC-2 xIC-3, respectively. As can be seen, auditor 37-4

relied on all three IACs and showed two significant interactions,












TABLE 4-5

SELECTED EXAMPLES OF DIFFERENCES IN THE
IMPORTANCE OF THE IACs TO THE VARIOUS AUDITORS
(RELIABILITY JUDGMENT)


Auditor Percentage of Total Variance in Judgment
Number Accounted For by IACs:

IC-1 IC-2 IC-3 IC-lxIC-2 IC-lxIC-3 IC-2xIC-3

6-1 70.2 nsa 10.6 ns 10.6 ns

109-4 3.4 87.3 3.4 ns ns ns

11-1 ns ns 93.3 ns ns ns

37-4 17.4 23.9 31.3 7.9 ns 11.9

101-4 44.4 ns 22.2 ns 22.2 ns

19-2 ns 21.1 52.6 ns ns 21.1


aNot significant at the .10 level; all others significant at a< .10.











while the results for auditor 101-4 indicate no significant reliance

on IC-2 and auditor 19-2 indicates none on IC-1 for the reliability

judgment. However, both auditors 101-4 (22.2%) and 19-2 (21.1%) have

significant interaction terms which account for a large percentage of

total variance. Table 4-5, therefore, provides some examples of the

variation in responses to the reliability judgments.

Table 4-6 provides a similar analysis for the sample size

judgment. A selection of auditors provides results that again indicate

the differences that exist with respect to the reliance on the IACs

when making the sample size judgment. IC-1 accounted for 68.4% of

auditor 101-4's variance, IC-2 accounted for 93.8% of auditor 15-2's

variance, and 88.9% of the variance in auditor 41-4's sample size

judgment came from IC-3. For all of these auditors, the other IACs

and interactions were not significant.

Interaction effects were also significant for many auditors

in the sample size judgment. Nineteen auditors had at least one sig-

nificant (aS .10) interaction term, with five auditors having two

significant interaction terms. As shown in Table 4-6, auditors 88-4

and 32-3 each had two significant interaction terms along with signif-

icant main effects for all three IACs. Auditor 92-4 shows a signifi-

cant interaction for IC-2 x IC-3, which also accounts for a substantial

percentage of his total variance (29.1%). Additional sample items

from Tables 4-3 and 4-4 could provide a similar basis for the com-

parisons discussed above and summarized in Tables 4-5 and 4-6.

The relative importance of the IACs and their two-factor inter-

actions are summarized in Table 4-7. The importance of IC-3 within





















TABLE 4-6

SELECTED EXAMPLES OF DIFFERENCES IN THE
IMPORTANCE OF THE IACs TO THE VARIOUS AUDITORS

(SAMPLE SIZE JUDGMENT)


Auditor Percentage of Total Variance in Judgment
Number Accounted For by IACs:

IC-1 IC-2 IC-3 IC-1 xIC-2 IC-1 x IC-3 IC-2 x IC-3

a
101-4 68.4 ns ns ns ns ns

15-2 ns 93.8 ns ns ns ns

41-4 ns ns 88.9 ns ns ns

88-4 41.6 12.3 36.7 3.4 5.1 ns

32-3 12.4 8.1 63.2 ns 4.1 8.1

92-4 ns 44.7 20.1 ns ns 29.1




Net significant at the .10 level; all others significant at a .10.











TABLE 4-7

RELATIVE IMPORTANCE OF THE INTERNAL CONTROLS
AND THEIR TWO-FACTOR INTERACTIONS TO THE SUBJECTS'
RELIABILITY AND SAMPLE SIZE JUDGMENT


Number of Subjects
For Whom Significantt

Reliability Sample Size


Average Proportion of Total Variance
Accounted For by Internal Control Variable
or Interaction

Significant Ones Onlyt All Participants

Reliability Sample Size Reliability* Sample Size**


IC-1 34(49) 28(35) 29.1(22.9) 24.9(25.0) 12.2 11.4

IC-2 42(62) 29(49) 36.7(30.6) 43.5(26.1) 22.4 23.3

IC-3 74(89) 61(78) 57.4(51.3) 52.7(47.9) 46.5 40.7

IC-1 x IC-2 0(2) 5(5) 0(6.7) 9.0(9.0) 1.1 2.0

IC-1 x IC-3 3(6) 4(8) 12.4(9.8) 6.4(6.6) 1.3 1.5

IC-2x IC-3 2(8) 6(11) 17.3(12.0) 13.2(12.0) 2.8 3.2


tAt the .05 level of significance.

()At the .10 level of significance.


* From 102 subjects for which ANOVA analysis was possible.

**From 101 subjects for which ANOVA analysis was possible.


Interaction










the specific audit setting used in this study is again evident, with

evidence for IC-2 being considered more important than IC-1. The

relative importance of the controls is maintained whether evaluating

the reliability judgment or the sample size judgment and also for

either participants with significant controls and interactions or for

all participants.

The fact that the least important control (IC-1) accounts for

24.9% of the variance in significant sample size judgments and 11.4%

of the variance in all sample size judgments is an indication that

none of the three controls was disregarded completely. The dependence,

to a certain extent, on all three controls is reflected in the relative

importance of the interaction terms. For significant interactions, the

average variance was 14.9% for the reliability judgment and 9.5% for

the sample size judgment. Comparisons to other studies should be made

with caution due to the specific nature of the audit task and the

inclusion of IACs known for their importance in making the required

judgments. However, other judgment studies in auditing have shown

average variances for significant interaction terms of 3.3% [Joyce,

1976], 5% and 5.1% [Messier, 1979], and 6.4% [Ashton, 1974].

An evaluation of the variances for the interaction terms

across all participants indicates average variances of 1-3%, which may

be considered "normal" for this type of task. An analysis of the

information pertaining to the interaction terms indicates that few of

the auditors had at least one significant (aS .05) interaction term

(5 of 85 for reliability judgment; 10 of 90 for sample size judgment)


4The weightings of the interaction terms were possible for 80
and 95 auditors, respectively, for the reliability and sample size
judgments as previously indicated.










although average variances were high for the significant interactions.

It therefore appears that some auditors processed information in a

highly configural manner, although the number was few.5

The various firms and experience levels of the auditors may be

suggested as factors underlying differences in responses. Table 4-8

provides a summary of the omega-squared values by firm and experience

level. Two classification schemes are used in presenting information

as to experience. First, a comparison is made between those auditors

with 1-2 years of experience and those with 3 or more years. Secondly,

those with 3 years of experience are grouped with those that have 1-2

years and contrasted with the auditors with 4 or more years of experi-

ence. The values that are shown in the table are averages that were

computed by summing the omega-squared values for a particular firm or

experience level and dividing by the number of auditors in that cate-

gory. The total of 102 auditors for the reliability judgment and 101

auditors for the sample size judgment for which the ANOVA analysis was

possible are reduced to 100 and 99, respectively, for the experience

classification due to the inability to properly classify two of the

auditors.

IC-3 appears to be recognized as the most important control

for the reliability decision irrespective of the firm or experience

level. However, for Firm #2 the difference between IC-3 and IC-2 is

negligible. The relative importance of IC-3 is maintained for the

sample size judgment across all classifications except for Firm #2,

where the average omega-squared value of IC-2 (35.9) was greater than


5See, for example, auditors 88-4 and 89-4 in Table 4-4.




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs