USING INDUCTIVE INFERENCE OF
PAST PERFORMANCE TO BUILD STRATEGIC
COGNITIVE ADVERSARY MODELS
STEVEN MICHAEL WALCZAK
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
I would like to thank the faculty and staff of the Computer and Information
Sciences Department. Several staff members proved to be invaluable in helping me
cope with the university system. I specifically recognize the assistance of Ms. Jeffie
Woodham, Ms. Marlene Hughes, and Ms. Cheryl Coleman. Ms. Tammera Reedy
and Ms. Judy Longfellow were especially helpful in getting various figures and papers
printed and delivered to their final destinations. Professor Joseph Wilson provided
me with insight into the vision based problems encountered while performing my
research. Other faculty members including George Logothetis, Richard
Newman-Wolfe, Gerhard Ritter, and Ravi Varadarajan encouraged me to openly
discuss research issues in computer science and to enjoy being a student.
Two computer and information science graduate students helped significantly
during the early stages of my research. I would like to thank USA Captain
Christopher Wagner and USN Lieutenant Richard Fiola for there assistance in
uncovering the adversarial modeling techniques used by the armed services.
I would like to give special thanks to my committee members--Manuel
Bermudez, Walter Cunningham, Douglas Dankel, and Ira Fischler. Dr. Cunningham
has been a tremendous source of knowledge and help in establishing the techniques
used by the adversary modeling methodology to infer adversary playing styles. I owe
a great debt of gratitude to my committee chairman, Paul Fishwick. He spent many
tireless hours reviewing and analyzing my research and encouraging me.
My greatest human source of aid, comfort, and encouragement during my
graduate studies has been my wife. Karen gave three years of her life to work and
get us both through graduate school. She has my undying love.
I thank God for giving me the wisdom, intelligence, and motivation to pursue
my doctoral degree. The knowledge of His presence in my life has been the single
most important factor in my ability to complete graduate school. I hope and pray
that I will be faithful and wise enough to hear His calling in my life, that I may
choose to follow His path.
TABLE OF CONTENTS
LIST OF TABLES ............
LIST OF FIGURES ...........
............................ .... vii
1 INTRODUCTION .......
Problem Statement .....................
Contribution of the Dissertation ...........
Outline of the Dissertation ...............
2 BACKGROUND AND RELATED WORK .....
Tree Search ..........................
Athletic Adversarial Domains .............
Education Domain .....................
Adversarial Game Domains ...............
Law Enforcement Domains ...............
Military Domains ......................
Human Adversary Modeling Efforts
Computer Planning in Military Domains
Political and Economic Domains ...........
Religion Domain ...................................
Chess Domain--Our Application Area ...................
Human Use of Adversary Modelin in Chess .........
Importance of Pawn Structures and Chess Openings ...
Computer Chess and Artificial Intelligence .......... 38
Justification for Chess Domain Application .......... 43
3 ADVERSARY MODELING METHODOLOGY ............. 45
Knowledge Acquisition of Chunks ................... 46
Defining the Chunks to be Acquired ............... 46
Acquiring Geometric Chunks .................... 50
Acquiring Textual Chunks ....................... 58
Induction to Learn Repeated Chunk Patterns .............. 62
Acquiring an Adversary's Playing Style ................... 66
Application of the Adversary Model .................... 68
4 IA M ............................................... 75
Overview of IAM's Execution ......................... 75
Knowledge Acquisition and Inductive Learning Stage 76
Predicting Adversary Actions Stage ................ 82
Integrating IAM's Knowledge into a Chess Program ... 86
IAM's Performance Results ........................... 88
General Performance Results .................... 88
Performance with Respect to Time ................ 99
Performance with Respect to Textual Chunk Size .... 102
Performance with Respect to Geometric Chunk Size 104
The Effect of Domain Specific Knowledge .......... 107
Repeatability of the Demonstrated Results ......... 108
5 RESU LTS .......................................... 110
Reducing Opening Book Size ........................ 111
Revealing New Search Paths in the Game Tree ........... 113
Reducing Search Complexity Through Automatic Pruning ... 116
Summary of Research Contributions ................... 122
6 CONCLUSIONS AND FUTURE RESEARCH .............. 124
Conclusions ...................................... 124
Future Research Directions .......................... 125
Acquire More Geometric Chunks ................ 125
Increase the Knowledge Contained in Each Chunk .. 126
Place More Adversarial Knowledge in the Adversary
M odel ............................... 128
APPENDIX ............................. .......... ........ 132
REFERENCES ............................................. 135
BIOGRAPHICAL SKETCH ................................... 144
LIST OF TABLES
1. IAM Performance Measures for TEST1 ......................... 91
2. Effect of Likelihood on Prediction Ratios ........................ 96
3. IAM Performance Measures for TEST2 ......................... 97
4. Performance with Responsive Modification ....................... 98
5. Effect of Recency on IAM Performance for TEST1 ................ 100
6. Effect of Aging/Forgetting on IAM's Performance ................. 101
7. Performance (TEST1) with 10 Opening Moves .................... 103
8. Performance (TEST1) with 15 Opening Moves .................... 103
9. IAM Performance (TEST1) with Five-by-Five Chunks .............. 105
10. IAM Performance (TEST1) with Six-by-Six Chunks ................ 105
11. Effect of Domain Specific Knowledge .......................... 108
12. Ratio of Won and Lost Games for White and Black ............... 111
13. Statistical Analysis of Soviet Chess Weaknesses ................... 130
LIST OF FIGURES
Logic flow diagram. ..........................
Overview of the physical system .................
Sample Minimax tree. ........................
Sample cognitive chunk. .......................
Computer chess program ratings. .................
Chunks acquired for Botvinnik. .................
Internal representation of a board position. ........
Convolution templates .......................
Pseudo-code to perform convolution. .............
Step functions ..............................
Pseudo-code to perform step function ............
Pseudo-code to perform chunk coalescing. .........
Chunks on a chess board and their convolution values.
Chunk acquisition in Go.......................
Flowchart to collect textual move knowledge .......
The effect of noise on chunks. ..................
Four identical chunks. ........................
Fuzzy logic used by the heuristic rules ............
19. An example of a game record--the 1948 Hague-Moscow Tournament. .. 77
20. Chunk occurrence in three different games ....................... 79
21. Pseudo-code to perform induction on geometric chunks ............. 81
22. Predictions from four chunks. ................................ 86
23. A game from the 1963 Botvinnik versus Petrosian match ............ 89
24. Another game from the 1963 Botvinnik versus Petrosian match ....... 90
25. Pseudo-code to suggest possible adversary moves. ................. 94
26. Chunk acquired with 5x5 size and corresponding smaller chunk ....... 106
27. Hypothetical game/search tree. ............................... 114
28. Prediction of Botvinnik's next move. .......................... 116
29. Standard search tree and the IAM improved search tree ............ 118
30. Search tree obtained from Correct Piece identification. .............. 121
31. Knight template and partial Rook template. ...................... 127
Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
USING INDUCTIVE INFERENCE OF
PAST PERFORMANCE TO BUILD STRATEGIC
COGNITIVE ADVERSARY MODELS
Steven Michael Walczak
Chairman: Paul A. Fishwick
Major Department: Computer and Information Sciences
To perform optimally in adversarial domains, artificial intelligence programs
must be able to identically evaluate domain situations and develop plans as the
adversary. Knowledge about an adversary's evaluation criteria and objectives is
difficult to obtain since an adversary will guard this information in order to promote
a favorable outcome for himself. We present an adversary modeling methodology
which is capable of acquiring the evaluation criteria of an adversary through
observations of the adversary's past performances in the domain.
Our adversary modeling methodology uses induction to capture perceptual
chunks that are repeatedly displayed by a specific adversary. The acquired chunks
are then used to predict the adversary's actions during future encounters. Pattern
recognition is used to identify perceptual chunks in the current adversarial domain
situation which are analogous to chunks that have been previously learned from the
We have implemented our adversary modeling methodology in the program
IAM, Inductive Adversary Modeler, which has been applied to the domain of chess.
When high quality knowledge is available about an adversary's past performances,
the adversary modeling methodology accurately predicts ten percent of an adversary's
future action plans. The ability to accurately predict an adversary's move choices
reduces the complexity of the game tree search which enhances the playing ability
of current chess programs.
Adversarial situations surround us in varied forms on a daily basis. Examples
of adversarial situations include vying for a promotion at our place of employment,
Monday night's football game, and military conflicts throughout the world. These
adversarial situations are characterized by a minimum of two actors each of whom
is attempting to realize a goal that conflicts with the goal of the opposing actor. The
actors can be individuals or groups of individuals such as a football team players that
are cooperating to achieve the goal of the actor.
When humans are faced with the problem solving task of realizing a goal
against the desires of an adversary, they employ a variety of techniques that enable
them to outperform their adversary and accomplish their goal. The standard
cognitive problem solving methods of difference reduction and working backwards
are insufficient in adversarial domains. Human competitors use deductive reasoning
and analogy to previous domain situations to realize their goals in adversarial
domains. The use of analogy to gain a strategic advantage over an adversary
necessitates the studying of a specific adversary's previous performance, preferably
in situations analogous to the upcoming contest.
Human competitors performing in adversarial domains study the past
performances of their adversaries to identify strengths and weaknesses which can be
exploited respectively to gain a strategic or tactical advantage. This information is
preserved by the competitor as a model of the likely actions an adversary will
perform in particular situations. The model of the adversary's probable actions is
then exploited when an appropriate situation occurs during the course of the
competition. Such situations arise when the current game state--situation in the
adversarial domain--is analogous to a game state which occurred in the adversary's
1.1 Problem Statement
Current artificial intelligence application efforts in adversarial domains are not
performed optimally. A factor which keeps adversary-based programs from
performing at expert levels is the lack of knowledge about the potential adversary.
Lenat and Feigenbaum (1987) claim that domain specific knowledge is required to
solve difficult problems and that the performance of intelligent programs improves
as additional domain knowledge is utilized.
Acquiring knowledge for programs in adversarial domains is hindered by the
fact that adversaries are unwilling to reveal their tactical and strategic goals, since
this would be counter-productive to the attainment of their goals. Candland (1980)
and Bolc (1987) have claimed that knowledge about any domain is available from a
wide variety of sources. Lenat (1982) proposes the use of the observable actions of
the adversary while performing in the adversarial domain. The proper selection of
adversarial domain situations to observe guarantees the acquisition of high quality
For the domain of chess, lack of knowledge about an adversary's tactics and
strategies is analogous to searching the game tree to a depth of only one ply. All
moves are considered as winning moves and the game proceeds in a random manner.
To acquire knowledge for use in adversarial domain application programs, we will
emulate the traits of human experts performing in adversarial domains, by analyzing
the previous performances of a particular adversary in relevant domain situations.
1.2 Contribution of the Dissertation
Adversarial domain programs require knowledge about an adversary's tactics
and strategies to continue to improve their performance. We need to consider two
separate factors that affect the strategic decisions an adversary will make. An
adversary's strategic decisions are affected by the plans or scripts that the adversary
feels will satisfy the current strategic goal. Before the adversary can select a plan of
action however, the domain situation must be evaluated to determine which plans are
appropriate to that particular situation.
Current programs performing tactical and strategic planning for adversarial
domains rely heavily on tree structures and search techniques such as iterative
deepening and alpha-beta pruning. The ability to accurately predict the choices that
will be made by an adversary in specific domain situations reduces computational
complexity and increases the depth of search. The increased depth of search that our
research enables is directly responsible for the augmented tactical ability of these
search based programs.
The purpose of our research is to develop a generalized method for modeling
the strategic decision making process of an adversary. The information flow diagram
for our adversary modeling methodology is shown in Figure 1. Our adversary
modeling methodology makes use of knowledge which is currently not used by
programs operating in adversarial domains. By increasing the knowledge available
to tactical and strategic planners, we will improve the overall performance of these
adversarial domain applications.
Repeated Patterns Adversar|y I|
Sct ion Predictor
Current Adversar ial Analogous Situation
Domain Situation Detector
Figure 1: Logic flow diagram.
Our adversary modeling methodology uses psychological principles of
organization and evaluation such as the Gestalt principles of organization (Anderson,
by the Adversary -
S Knowledge Base
to Acquire Patterns noun bg
-I Adversar ies
to Identify & Learn I
1980, and Pomerantz, 1986) to emulate the primary evaluation criteria used by an
adversary. Kodratoff and Michalski (1990) note that recent machine learning efforts
have focused on building cognitive learning architectures. We intend for the
adversary modeling methodology to be applicable across all adversarial domains and
hence, we include as little domain specific knowledge as possible. This means that
we are content to acquire the primitive elements that an adversary uses in evaluating
a domain situation without embellishing those elements with further domain specific
The psychological principles of organization are applied to a collection of
previous adversary performances in the domain (e.g., chess games played by the
adversary in prior tournaments and matches). We acquire the cognitive chunks used
by an adversary to evaluate the current domain situation. The acquired cognitive
chunks of the adversary help us to view the domain the same as our adversary and
to avoid influencing domain evaluations with personal biases.
We now have a large collection of chunks that have been displayed by the
adversary. The chunks are either textual patterns corresponding to the adversary's
verbal memory or geometric patterns corresponding to the adversary's visual memory.
The chunks we have acquired contain patterns that have been used only once and
may have occurred by chance. We use induction to eliminate these chance chunks
and create a valid and useful adversary model. The inductive phase identifies chunks
which are repeated in two or more games. The repetition of patterns is used to
indicate the probability that an adversary will use the acquired chunk in future
evaluations of domain situations. Because we are using induction as our learning
method, the adversary model created will be used as a heuristic guide instead of an
Following the inductive learning, the collection of chunks that have been
repeatedly displayed by the adversary are stored in the adversary model. We
consider these chunks to be the primitive elements used by an adversary in evaluating
current domain situations. The chunks of the adversary model are then used to
predict adversary decisions when analogous situations occur in the adversarial
The knowledge obtained by the adversary modeling methodology is highly
reliable, because it comes directly from the adversary. A certain level of care must
be taken to guarantee that the adversary is performing normally in the domain and
not providing misinformation to the modeling methodology. We verify the adversary
modeling methodology by implementing the methodology for the adversarial domain
of chess. The implementation is called IAM, for Inductive Adversary Modeler. Our
justification for selecting the chess domain is given in section 2.10.6.
Berliner and Ebeling (1988) note that speed is a critical factor for evaluation
functions in chess as well as other adversarial domains. The knowledge that we are
learning about an adversary is available as historical records prior to the current
performance of the adversarial domain program. Therefore, the adversary modeling
methodology's knowledge acquisition can be performed prior to its use by the domain
program. This preprocessing effectively eliminates any time or complexity costs
generated by the adversary modeling methodology from being added to the current
program's evaluation function cost (Aho, 1974). The adversary model acts as a
heuristic coach to an existing domain program by predicting which moves an
adversary will make, as shown in Figure 2. The only part of the adversary model that
impacts the time order of the existing evaluation function is the analogous situation
detector that was shown in Figure 1.
Game Plaging Program
Figure 2: Overview of the physical system.
1.3 Outline of the Dissertation
We have presented a general overview of the problem of the lack of
knowledge about adversaries available to intelligent programs operating in
adversarial domains. Our solution to this problem has been briefly presented in this
In Chapter 2, we present background material on induction and psychology
that affects our research. The research that has already been performed in several
of the adversarial domains including the domain of chess is discussed. Finally, we
present the reasons for selecting the domain of chess as our application domain.
In Chapter 3, we detail our research by first discussing the implementation
method and then the operation of the adversary modeling methodology. Each of the
acquisition and induction steps that have been briefly described in this chapter are
presented in detail.
Chapter 4 contains an example of the adversary modeling methodology in
action and provides quantitative performance results of our application program,
IAM. Our results and their impact on artificial intelligence search methods are
provided in Chapter 5. We discuss our conclusions and future research directions in
Chapter 6. The Appendix contains a glossary of chess and artificial intelligence
terms to aid the reader in understanding some of the domain descriptions that we
BACKGROUND AND RELATED WORK
In this chapter, we review the theoretical and applied work on induction. We
also describe the theoretical work on game tree search to emphasize our contribution
to this research area. Finally, we describe related work in the various adversarial
domains that have led to our solution with a detailed review of the domain of chess
which serves as our application domain. For each of the adversarial domains
presented, we describe the use of adversary modeling by humans and any applied
research that affects the implementation of our adversary modeling methodology.
Mankind has been using induction to perform problem solving and learning
tasks for many millennia. Rosch (1973) describes the way humans form concepts and
categories about their natural world. Polya (1973), in the early 1950s, formalized the
concept of induction. Induction is the process of discovering general laws or concepts
from observations and combinations of particular instances.
An example of induction for a natural world concept would be the
generalization that all ravens are black. This generalization is based on the
observation of a limited number of instances of a raven. Without seeing all the
possible examples or instances of raven, we are confident to induce that the color of
a raven is the same as the color of the instances we have seen--black. The strength
of our belief increases as more and more examples of black ravens are observed.
However, only one instance of a nonblack raven needs to be identified to invalidate
From the above example we can see that inductive inference is learning in the
limit (Daley & Smith, 1986). We need to see sufficient observations of the right type
to form a correct generalized concept. Since we cannot guarantee that the right type
of observations are being provided to an inductive inference system, we must rely on
the use of a large number of examples of category instances to acquire the desired
concept. This means that the adversary modeling methodology we have developed
should perform more accurately as we provide greater numbers of examples of an
adversary's domain performance.
Since Polya's formalization of the concept of induction, theoretical research
on induction has primarily focused on the formation of concepts or classifications.
Angluin and Smith (1983) and Case and Smith (1983) present rigorous discussions
of the mathematical principles of induction. This theoretical background provides
a solid base for the use of inductive inference as a computational tool for learning
classes of objects or ideas.
The extraction of general principles from specific examples is a major research
goal in machine learning (Anderson, 1987). Induction has been applied to many
domains. Examples of applied inductive inference algorithms are Michalski's
INDUCE and Quinlan's ID3 (Michalski et al., 1983 and 1986). Each of these
example applications attempts to classify instances of a domain problem (e.g.,
soybean diseases for the INDUCE program) into maximally-specific groups that will
identify all known instances as well as accounting for unknown instances.
Prior to the theoretical discussions of inductive inference mentioned above,
Blum and Blum (1975) had already demonstrated that the ability to play games, or
at least to make technically correct moves, can be learned through inductive
inference. Induction has not been utilized in game domains due to the lack of high
quality play produced by Blums' algorithm. Our adversary modeling methodology
focuses on a narrower slice of the game domains, namely predicting adversary move
choices and playing style, and is able to learn high quality knowledge.
Adversarial domains present several specialized problems to the standard
inductive inference algorithms like ID3. Langley (1985) has noted that feedback is
an essential element of most learning methods. Inductive inference algorithms use
feedback to verify acquired concepts and to identify concepts that need to be
modified. In adversarial domains, feedback on the performance of an algorithm is
a not attainable until after the current domain situation has ended. Because of the
inherent time delay for feedback in adversarial domains, we substitute probabilistic
reasoning so that our confidence in the current adversary model increases as
additional examples of the same type are observed.
Another problem in adversarial domains is the dependence of the domain on
time. Actions occurring in the domain are temporally related. The inductive
inference applications cited above belong to the class of hierarchical or structure
induction methods. Structure induction methods are used to produce classification
systems. Muggleton (1990) proposes an inductive method called sequence induction
which has a time effect relationship to the domain. Sequence induction uses a series
of domain descriptions that are altered by actions operating on the domain to
produce control systems.
Mitchell et al. (1986) refer to a process of mechanized induction which
receives domain events as input and produces concepts for predicting future events.
For adversarial domains, sequence induction provides a method to produce
evaluation criteria for predicting an adversary's strategic decisions.
2.2 Tree Search
Programs in the adversarial domains of games invariably use trees as their
knowledge representation scheme. Finding solutions or move choices in game trees
involves searching the tree for the optimal finishing value. Von Neumann and
Morgenstern (1953) published the Minimax algorithm for backing the final game
values up through the game tree to the current node. Minimaxing attempts to
account for the effects of an adversary in the game domain by alternately raising the
minimum possible value or maximum possible value at each level of the tree.
If an exhaustive search can reach all of the leaf nodes of the tree, then the
true finishing value of the game can be raised to the current node and an optimal
game will ensue. However, games like chess, Go, and Othello have tree
representations that hold 1040 nodes or more. Game trees of this size prohibit an
exhaustive search. Instead, game programs will search down to a preselected depth
and apply an evaluation function that will determine a best guess for the true
finishing value obtainable from that node.
Evaluation functions use domain knowledge to predict the outcome of the
game from a specific position. All domain specific knowledge is contained in the
evaluation function. The quality of knowledge has a direct effect on the playing
ability of the game program (Berliner & Goetsch, 1984, and Lee & Mahajan, 1988).
Northwestern University's CHESS4.6 was able to defeat other chess programs which
searched several ply (levels) deeper into the game tree based on the strength of its
evaluation function (Elithorn & Banerji, 1984, and Wilkins, 1980).
The primary problem faced by search algorithms using tree structures is the
horizon effect. The horizon effect occurs when the solution to a search problem lies
at a depth in the tree that is beyond the current search depth. This is a realistic
problem for complex adversarial domains. In chess, the KRKN endgame, king and
rook versus king and knight, can require a twenty-seven move solution or a game tree
search depth of fifty-two ply (Bratko & Michie, 1980). Similarly, the KBBKN
endgame has solutions of sixty-six moves or one hundred thirty-two ply search depth
(Roycroft, 1988). Current chess programs search between eight and fourteen ply
before selecting a move.
Two different schools of thought exist concerning methods for continuing to
improve game program performance. The first school believes that the depth of the
search is the sole criterion affecting the playing ability of game programs.
Adelson-Velsky et al. (1988) state that increasing the depth of search of a game tree
increases the quality and reliability of game play. Berliner and Ebeling (1988)
specifically state that a deep tree search will outperform a shallower search which
uses more knowledge. Various techniques are used to increase the depth of search
for a program. Limiting the breadth of search and pruning of undesirable branches
through alpha-beta pruning and iterative deepening permit a deeper search for a
static number of evaluated tree nodes (Barr & Feigenbaum, 1981). Specialized
hardware can also be used to increase the speed and thereby the depth of search
An example of alpha-beta pruning is shown in Figure 3. The four leaf nodes
which have their branches cut by a diagonal line can each be pruned using alpha-beta
pruning. Once the three value is elevated to the left subtree level two node, then the
level one node is guaranteed a minimum value of three. Since each of the first leaf
nodes below the center and right subtrees have lower values than three, the
remaining leaf nodes do not need to be evaluated. This is called an alpha cutoff.
A beta cutoff is similar except that the cutoff is performed based on a guaranteed
maximum value for a Min node.
The other method for improving the performance of game programs is to
increase the knowledge that is available to them. We have already seen that better
quality knowledge enabled CHESS4.6 to outperform other programs that searched
several ply deeper into the game tree. The use of additional knowledge has the same
effect as searching deeper into the game tree. The projection ability of certain types
Figure 3: Sample game tree with alpha-beta pruning.
of knowledge in the chess domain is nine ply for tactical knowledge, twenty-five ply
for positional knowledge, and forty-five ply for strategic knowledge (in Elithorn &
Banerji, 1984). Current game programs use mostly tactical knowledge along with
some positional knowledge.
For adversarial domains, the use of domain knowledge can be dangerous by
leading us into a false sense of security. Samuel (1959) warns that programs using
the Minimax algorithm must take into consideration the intent of the adversary.
Cecchini (in Crookal et al., 1987) claims that the actual intent of the adversary will
require programs to use different heuristics in order to win. Human tree search is
goal directed and the direction and method of search may change as new information
becomes available (Frey, 1983). This last statement means that the preceding events
in a game and tournament affect the future strategic decisions of the adversary.
Our adversary modeling methodology follows the school of thought which
believes that more knowledge is required to continue improving the performance of
game programs. To avoid the complications of adversarial intent, our heuristics are
based on the primitive psychological elements that are used by an adversary to
evaluate the domain. Application of the adversary modeling heuristics occurs when
the current game board pattern is analogous to a previously induced situation. The
use of pattern recognition to increase search depths has already been implemented
by Samuel (1959 and 1967) in the form of signature tables and in the Russian chess
program KAISSA (Adelson-Velsky et al., 1975) to perform cutoffs or pruning.
The theoretical increase in search depth which we have achieved through the
adversary modeling methodology is detailed in Chapter 5. Several authors (Findler
& Meltzer, 1971, and Utgoff, 1986) have noted that the economy of a search
heuristic can be measured by the reduction in search effort and complexity which
produces a deeper search.
2.3 Athletic Adversarial Domains
The modeling of an adversary by humans is well documented in athletic
domains. Coaches of baseball (Aliston & Weiskopf, 1984), football (Riggins &
Winter, 1984), and other sports use scouting reports and films of their opponents to
study their opponent's strategies. This enables the coaches of athletic teams or
individuals to develop counter-strategies to foil the goals of the adversary and
promote their own goals.
In a personal communication, Owen J. Holyoak of the Exercise and Sport
Sciences Department at the University of Florida stated that the amount of
preparation performed by athletic team coaches corresponds directly to the perceived
threat of the adversary. When the technique of studying an adversary's past
performance is not fully used, surprising results can occur. At the 1982 homecoming
football game for the University of Florida (UF), the opponent was West Texas State
University (WTS). Although WTS was considered to be an underdog and was not
expected to score any points, they obtained a fourteen point lead early in the game
by using some plays that UF had not anticipated. The lack of preparation on UF's
part was responsible for the surprising start of the 1982 homecoming game.
2.4 Education Domain
The artificial intelligence domain of intelligent tutoring systems (ITSs) or
intelligent computer-aided instruction has already made successful use of models of
human thinking. These models are based on the availability of knowledge to the
human and that human's mental set at a particular point in time. The people being
modeled in ITS programs are the students who are attempting to learn. The student
models are used to predict the sources of error in student thinking which cause
incorrect responses to test items produced by the ITS.
Woolf (1984) has demonstrated an ITS which uses a model of the probable
knowledge currently held by the student for discourse planning. This model adapts
its understanding of the student's knowledge during the lesson to capture changes in
the student's cognitive state precipitated by the acquisition of new knowledge. Our
adversary modeling methodology operates similarly by constructing a model of the
probable evaluation mechanisms known to an adversary to predict future strategic
decisions of the adversary. Other examples of this modeling approach as used in
ITSs can be found in Sleeman and Brown (1982).
2.5 Adversarial Game Domains
Because of their finite size, games provide us with a convenient abstraction
of more difficult real world problems. The former Japanese use of the game of Go
to train their military leaders (Reiss Games, 1974) demonstrates the practical
extension of a game domain to a military domain. Pritchard (1973) elaborates
further by claiming that Go is guerilla warfare on a grand scale.
We will be using two person zero-sum perfect information games (Zagare,
1984) to demonstrate the adversary modeling methodology. Zero-sum games are
always adversarial in nature, since the wins of one side are equivalent to the losses
of the other side. Our selection of perfect information games was made to permit
an algorithmic evaluation of the results. The domain of chess which we will use for
our application of the adversary modeling methodology is reviewed in section 2.10.
Other types of games, such as games of chance, that do not fit our selection
of zero-sum perfect information games can also benefit from the adversary modeling
methodology. Our contribution which reduces the complexity of search is usable by
any game program that utilizes game tree search and heuristics to solve domain
problems. Berliner's BKG backgammon program (in Levy, 1988) is an example of
a game of chance which uses heuristic methods and game tree search to select
Christensen and Korf (1986) claim that heuristic methods for one person
puzzles are equivalent to heuristic methods for two person games. The adversary
modeling methodology is capable of predicting moves in one person games that will
reduce the complexity of the current game position to one which has a known
solution. Other research using induction of previous games has been performed for
one person games. PREFER (Utgoff & Saxena, 1987) solves the 8-puzzle by
performing induction on previous 8-puzzle solutions.
Samuel (1959 and 1967) was the first researcher to use machine learning to
create an intelligent game playing program. Samuel's checker playing program uses
rote learning techniques to learn how to play checkers. This program is capable of
imitating the style of an adversary by mimicking the winning moves of the adversary
presented in the set of training instances. Programs which rely on rote learning tend
to become inflexible and require an equally large number of retraining instances to
adapt to a different style of play. The method of mimicking an adversary has been
shown to fail for the game of Go (Pritchard, 1973) and should not be considered as
a generalized method of strategic planning against adversaries.
Samuel developed signature tables to reduce search complexity. Signature
tables are used to remember the evaluation function value for specific positions.
When identical positions are identified through a pattern matching mechanism, the
signature tables are used to retrieve the corresponding node value. Wilcox (1985)
has also applied pattern recognition techniques to the game of Go to construct
NEMESIS, a strong Go playing program.
IAGO (Rosenbloom, 1982) uses the standard search techniques of alpha-beta
pruning and iterative deepening to play championship-level Othello. Rosenbloom
points out that the heuristics for IAGO need to change during different phases of the
game to capture shifts in the strategic importance of various game principles. For
example, in the middle game of Othello mobility is a critical factor, but in the end
game portion of Othello the quantity of stable pieces is the primary concern.
Berliner (1979) notes similar shifts in strategic reasoning for the domain of chess.
Our adversary modeling methodology compensates for the strategic shifts at various
points in the game by assigning time durations to the domain evaluation chunks that
are acquired for each adversary.
GINA (DeJong & Schultz, 1988) tries to learn to play Othello by using
experience gained from previous games. This approach is very similar to the one
used by Samuel for his checkers program. GINA attempts to exploit the weaknesses
of a specific opponent. Several of GINA's implementation features are applicable
across adversarial domains. These features are:
Serve as an external coach to an existing program in the domain.
Only use observable behavior to learn about an adversary. For GINA,
the observable behavior is the moves made by an adversary in prior
Do not use feedback to modify learning, except for the results of the
games that have been studied.
The best example of strategic planning based on an adversary's modus
operandi is Waterman's (1970) poker player. Waterman has implemented a draw
poker playing program which analyzes an opponent's style of play to decide when a
bet is appropriate. This program learns over a period of several hands which
heuristics to use against a particular adversary. We expand upon Waterman's idea
by not limiting the program to a static number of predefined heuristics.
2.6 Law Enforcement Domains
Police departments and the Federal Bureau of Investigation each make use
of adversary modeling. As an example, a police detective will try to predict when,
where, and to whom a serial killer will perform his next crime. This is done
primarily by analyzing the criminal's previous performances of crimes. Similar
techniques are being considered for predicting the outcome of possible terrorist
activities in the RISK project sponsored by Lawrence Livermore National
The key to successful drug enforcement is intelligence gathering and
information processing (Witkin, 1990). Raw data is voluminous and must be
converted to a useable form. Our adversary modeling methodology is capable of
finding patterns in raw data that correspond to resource movement problems (e.g.,
the actual drugs and the monetary compensations) and can be used by law
enforcement officials to increase their effectiveness.
2.7 Military Domains
Devastating results occur on the battlefield when sufficient preparation against
an adversary is not performed. Due to different sociological, geographical, and
educational backgrounds, potential adversaries will have unique behavioral rules
(Bond, 1986 and Boring, 1945). These behavioral rules govern varying beliefs and
goals concerning war and the use of nuclear weapons (McMillen, 1984 and
Sokolovskiy, 1968). Dunnigan (1982) and Ermarth (1978) state that if we assume
that an enemy will act strictly according to our own beliefs, then we will produce a
tactical blindness. Tactical blindness produces the inability to predict or fathom the
plans of our adversaries. Historic examples of the tactical and strategic blindness
produced by conscripting our own beliefs onto an adversary are the bombing of Pearl
Harbor by the Japanese in 1941, the defeat of the United States tactical forces at the
Chosin Reservoir during the Korean War (Wallace, 1990), and the Tet Offensive of
the North Vietnamese in 1968. Many soldier's lives might have been saved early in
World War II if the United States soldiers had understood the Japanese attitude
toward surrender (Boring, 1945).
2.7.1 Human Adversary Modeling Efforts in Military Domains
Military leaders (Gruner, 1988, and Ryan, 1988) claim that victory is often
contingent upon knowing your enemy. This means that military planners must be
able to evaluate the current situation using the same criteria as their adversary. A
vital component of intelligence information is briefings on an adversary's operational
characteristics or tactics (Gruner, 1988). Robert E. Lee, who is often cited as the
greatest military strategist (B. Davis, 1956), is a practical example of the power
accompanying detailed knowledge about an adversary. Lee was educated at West
Point, where all of the Union commanders were also trained, and served along side
many of the adversaries he would face in the Civil War during his tenure as an
officer in the United States Army and Cavalry (Snow, 1867). Lee's detailed
knowledge of the training and tactics of his adversaries enabled him to frequently
outperform better equipped and larger forces.
The United States military makes use of this concept in training our soldiers.
Each service branch maintains a cadre of personnel who are trained in Soviet-style
war tactics and strategy (Robbins, 1988). The training centers for the Army,
Air Force, and Navy are the National Training Center, Red Flag, and the Top Gun
school respectively. The purpose of these Soviet analogous fighting forces is to
provide a realistic training environment to better prepare the military for the
possibility of an actual war against Soviet forces or forces that use Soviet-style tactics.
A historic example of adversary modeling for military purposes occurred
during World War II. A detailed psychological profile of Hitler was constructed by
Langer (Langer, 1972) for the Allied forces. This model of Hitler was constructed
solely from second-hand information. The psychological model of Hitler accurately
predicted the method of his ultimate demise via suicide.
2.7.2 Computer Planning in Military Domains
Military games and strategic and tactical planners make use of varying levels
of adversary modeling, ranging from none in typical multi-player games and training
simulators to moderately advanced models in certain strategic planning programs.
Failing to account for the actions of an adversary severely inhibits strategic planning.
Military planning tools must account for the probable actions of an adversary based
on the adversary's intentions and strategic style, otherwise tactical blindness will
The tactical troop movement planner TACPLAN (Andriole et al., 1986) is
typical of most military planning tools. TACPLAN only accounts for static elements
of the domain such as the effect of terrain on equipment movement. Knowledge
about adversaries is nonexistent. The next step towards an adversary model is
programs similar to ARES (Young & Lehner, 1986) which acknowledges the
presence of an adversary. ARES is also a tactical planner; however, the plans
generated by ARES attempt to account for the possible blocking actions of an
adversary. Although this approach claims to account for the actions of an adversary,
the actual outcome is not that different from the plans generated by programs like
TACPLAN. The lack of difference between the two approaches is caused because
the actions of an adversary are prescribed in advance which is similar to only
permitting a chess adversary to make forward moves. Infinitely many strategic
choices can be made by an adversary that will not be covered by the prescribed
action choices that are permitted by the tactical planning algorithm.
The POLITICS system (Carbonell, 1981) proceeds in the development plan
by introducing goals that an adversary is trying to accomplish, such as blocking the
current goal of the planning program. The adversarial goals used in POLITICS are
statically defined and are based on preconceived notions of an adversary's intent.
Basing the actions of an adversary on static predefined goals runs the risk of creating
tactical blindness mentioned in the introduction to section 2.7. The actual goals of
an adversary, much less how the adversary intends to accomplish those goals, may not
be known during the contest or may be contrary to the expected goals (Crookall et
al., 1987 and Narendra & Thathachar, 1989). POLITICS does provide a good
example of a program in which a thinking adversary attempts to achieve specific
TM (Erickson & Zytkow, 1988) is another goal based system for making
tactical decisions. The program BOGEY is an external learning program which
generates large numbers of simulations of tactical engagements with specific goals.
BOGEY, like GINA in section 2.5, is another example of the utility of coaching
programs that support existing domain specific programs through an external learning
The MARK I and MARK II systems (Davis, 1988a and 1988b and Davis et
al., 1986) embody the mind set of an adversary in DEWT, Deterrence Escalation
control and War Termination, models. Currently the MARK systems have two
decision models, East and West. While MARK I and MARK II attempt to capture
differences in the mind sets between adversaries, the models are statically defined
like the goals of the POLITICS program. These static mind sets are based on
predefined conceptions of an adversary's intentions which may not be accurate and
will not account for the modification over time to decision heuristics used by an
adversary. One of the contributions that artificial intelligence can provide to military
planning is reasoning that accounts for the temporal relevance of actions (Bonasso,
The use of multiple models to generate goals for different adversaries
demonstrated by the MARK systems is a reasonable approach to separate the varying
evaluation criteria used by various adversaries. An improvement to the MARK
systems is accomplished by acquiring the evaluation criteria and traits of particular
adversaries in specific situations along with the temporal relevance for each of the
evaluation criteria of an adversary.
We mentioned in section 1.2 that geometric or visual relationships between
domain resources are used by the inductive learning mechanism of the adversary
modeling methodology. This means that military domains are required to represent
the physical resources of the domain in a geometric manner. The services are
trained to use standardized formations such as the column, echelon, line, and wedge
formations (Department of the Army, 1986). Typical board wargames make use of
a hexagonal map for plotting the movement of resources. By using such map
representations, the use of specific formations by a particular commander can be
acquired by our adversary modeling methodology.
2.8 Political and Economic Domains
Top level policy decisions made by government officials are also affected by
prior knowledge about an adversary's goals and strategies (Cimbala, 1987).
Government negotiators attempt to discover the limitations faced by an adversary to
accomplish the optimal outcome of an agreement or treaty between the two
adversaries. The use of knowledge of the political mind set for a geographic region
permits negotiators to interpret information from the proper perspective (Cimbala,
The political use of adversary modeling can also be extended to corporate
politics. We can acquire patterns from adversarial actions such as corporate
takeovers or mergers. This knowledge enables corporate entities to be better
defended or prepared for such adversarial actions.
Other domains that are concerned with the geometric relationships of
elements in the domain can benefit directly from the geometric pattern acquisition
phase of the adversary modeling methodology. Examples of such domains include
city planning and architecture. For the city planning domain, patterns of streets from
other cities which have desirable and equivalent traffic patterns can be acquired for
use in the current city. The primary benefits to these domains will be economic
savings and time savings for the reduced complexity of the design phase for
2.9 Religion Domain
The importance of knowing the intentions and strategic style of an adversary
has been recorded throughout history. An early example of these writings comes
from the first century A.D. when the apostle Paul (1 Corinthians 9:19-22) writes:
that I might win the more. And to the Jews I became as
a Jew, that I might win Jews; to those who are under the
Law, as under the Law to those without law, as
without law ... .To the weak I became weak, that I
might win the weak.
Paul is stating that he will model the lifestyle of the people he is attempting to win.
The training centers for the military at the National Training Center, Red Flag, and
the Top Gun schools are modern examples of the approach used by Paul.
2.10 Chess Domain--Our Application Area
We mentioned in section 2.5 that games like chess can be thought of as
abstractions of actual military contests. Although the outcome of these games may
be viewed as trivial compared to the outcomes of military conflicts, personal
reputation, and financial prosperity are the stakes of such games for expert game
In the remainder of this section, we demonstrate the use of adversary
modeling by humans in the chess domain and the philosophical and psychological
foundations that enable the adversary modeling methodology to operate in the chess
domain. Finally, we present a detailed background of past and current computer
2.10.1 Human Use of Adversary Modeling in Chess
Before playing for the World Chess Championship against Capablanca who
was considered to be invincible and was appropriately called "The Chess Machine",
Alekhine studied and analyzed Capablanca's prior games (Schonberg, 1973). From
his study, Alekhine was able to determine a weakness in Capablanca's play that he
was able to exploit to become the new World Chess Champion. Other chess masters
including the World Chess Champions Botvinnik, Tal, and Kasparov prepare for
chess matches by rigorously studying the previous games of their opponents
Nunn (Nunn & Griffiths, 1987) describes a game against Ost-Hansen in which
both players based their game strategy on expectations of their adversary's playing
style. These expectations were formed from prior experience against the adversary.
Although no one was able to capitalize upon the information, it has been noted that
prior to his World Championship match against Spassky, Fischer varied his standard
opening for white from e4, pawn to King four, only three times during tournament
level play (Evans, 1970).
Several of the chess grandmasters, including Lasker and Tal (Horowitz, 1973
and Schonberg, 1973), have been described as psychological players. These players
would intentionally make the move that would be most disturbing to their opponents,
even though such moves were frequently not the best tactical choice. Levy (1984)
has described his style of play against computer chess programs similarly, stating that
his move choices are often made to confuse the computer.
The advent of intelligent programs in various game domains, including chess,
has caused humans expert game players to consider the computer as an adversary.
Subsequently, human competitors who will face an intelligent computer program as
an adversary have created models of adversary strategy and playing style that
represent the computer game programs. Certain human competitors are considered
to be specialists against computer styles of play.
A famous example of this specialization is the series of bets which David Levy
(1984) has had with the computer chess community. In these bets, Levy claims that
no computer chess program will be able to beat him. The first bet which had a ten
year duration was won by Levy, however, DEEP THOUGHT has defeated Levy in
the latest of the bets. A recent discussion on the rec.games.chess news network by
Feng-Hsiung Hsu indicates an increase in the number of human opponents of
computer chess programs using "anti-computer" playing styles against programs
entered in major chess tournaments.
When Mike Valvo defeated DEEP THOUGHT, the reigning computer chess
program champion for the past three years, in a two game postal chess match via
electronic mail, it was noted (Newborn & Kopec, 1989) that Valvo had observed
DEEP THOUGHT play a number of games and had acquired the playing style of
DEEP THOUGHT. The chess program did not have a similar opportunity. Berliner
(1988) claims that various chess players have asked for the previous game records of
HITECH, the 1989 computer chess program co-champion, so that they might be
prepared for a possible match.
2.10.2 Importance of Pawn Structures and Chess Openings
We pay special attention to pawn formations during our chunk learning
process. Philidor has stated that the win or loss of a game depends on the quality
of the pawn positions during the game (Pachman, 1975). The strength and weakness
of pawn formations is of paramount importance (Lasker, 1973). Pawn formations can
be used to reveal information concerning positional advantages (Reinfeld, 1987).
Furthermore, three of the six criteria for determining the character of a position
given by Capablanca relate to the effectiveness of the pawns in the game (Pachman,
Pawn formations contain information about the positional nature of the chess
game (Soltis, 1976). The geometric patterns of pawns displayed by a specific
adversary can be used to identify his playing style. By definition we can recognize
open and closed board positions solely from the absence or presence of long pawn
structures. The general playing style of the adversary is indicated by the predominant
style displayed by the adversary in the studied games. Adversaries will tend to
remain with the particular style that they feel most comfortable in playing. Chigorin
played only open style games (Pachman, 1975). Our own analysis of the games of
Botvinnik and Spassky has indicated that each of these chess grandmasters prefers
a particular style, closed and open respectively.
Openings also play an important role in determining the positional advantage
of the game. Current computer chess programs utilize a book or database of
opening moves which have a duration from two moves to more than ten moves. The
evaluation algorithms which enable chess programs to estimate the strength of the
current game position are not used until the book is exited. Typically, exits from
opening books occur only when an adversary makes a move that is not contained in
the book or the current opening line of play is exhausted. This means that current
computer chess programs can find themselves from ten to twenty percent of the way
through a game, for a game lasting fifty moves, before they start to calculate the
value of the current position.
The opening sequences of play that are used as the opening book for current
chess programs are stored verbatim. Mednis (1990) has demonstrated that particular
opening formations can be achieved through a wide variety of the actual move order.
A statistical analysis of the opening move sequences displayed by an adversary
accounts for opening move order variations by finding the upper bound, lower bound,
and mean time of specific moves.
Selecting an opening line of play is usually performed at random or in
response to the adversary's choice of opening move. By studying the playing style
and previous games of an adversary, specific opening lines that are well known to an
adversary are identified. This knowledge can be used by a chess program to emulate
human chess experts by choosing openings that are not as well known to the
adversary. Nunn (Nunn & Griffiths, 1987) avoids the Pelikan variation of the Sicilian
Defense in one of his games against Anthony because Nunn knows from experience
that Anthony is a Pelikan specialist. Other examples of selecting opening lines of
play based on the prior performance of an adversary are given by Nunn.
Acquisition of the opening sequence knowledge held by the adversary will
enable current chess computer programs to select lines of play which are strategically
advantageous. Holding (1985) supports our perspective by stating that we should use
information about players playing style to disrupt their normal play.
Dennett (1987) states that current chess programs concentrate their time on
branches of a game tree which represent the adversary's best response as viewed by
the program. While this would appear to be a reasonable approach, Dennett
elaborates by stating that a rational adversary might have a higher ulterior motive
than avoiding defeat in chess. For example, a young Soviet chess player with a rating
of master is performing well in a European chess tournament. The young Soviet has
acquired a score of 8.5 points and is about to face the tournament champion, a fellow
Soviet grandmaster who has scored 11 points thus far. The grandmaster and young
master both believe that the grandmaster will win the final game. However, no other
player in the tournament has a score greater than 9 points. The Soviet grandmaster
realizes that an outcome of a draw with himself will greatly raise the young Soviet
master's chess rating without damaging his own rating and might improve the young
master's confidence and future playing ability. Therefore, the Soviet grandmaster
who has already won the tournament by virtue of his previous victories offers the
younger master-level player a draw midway through the game.
Another example of the problems faced by computer programs in dealing with
the intentionality of human adversaries is provided by Levy (1984) who claims that
he will intentionally choose inferior moves when playing against a computer. The
inferior move choices are selected by Levy because he believes that computer chess
programs have greater difficulty in playing a winning game against such an irrational
style of play.
The Levy example above demonstrates one of the traits of human adversaries.
Humans, especially when competing against a machine, attempt to mislead their
adversary about their intentions and capabilities. We must be able to distinguish
between relevant information and misinformation about an adversary's playing style.
Incorporating the detection of misinformation from an adversary into the inductive
learning mechanism adds an additional heuristic element. The addition of another
heuristic element reduces the probability of acquiring a valid adversary model.
The effect of an adversary's misinformation attempts is negated by selecting
previous domain performances that limit the possibility of misinformation. By only
using the previous games of an adversary from tournament play in which the
adversary has something at stake, misinformation is eliminated. For our application
in the chess domain we use the World Championship Match games of adversaries
taken from the compendium by Gelo (1988).
Chase and Simon (1973) have extended the research performed by de Groot
which claims that the depth of game tree search and the total number of moves
considered is approximately the same for both novices and chess masters. However,
the ability to recall chess positions after a five second presentation of the positions
is markedly different. Chess masters are able to reconstruct chess positions involving
twenty or more pieces with complete accuracy. Chess novices are only able to place
four or five of the pieces correctly. The difference between the masters and the
novices disappears when the subjects are asked to reproduce random patterns of
chess pieces. Each group can only place three or four of the pieces from the random
board configurations. The apparent decrease in ability by the master level players
is attributed to the fact that the random configurations of pieces contained no
inherent domain knowledge.
Based on Miller's (1956) hypothesis, the chess masters are forming higher
level chunks composed of groups of pieces from the game positions. These chunks
contain domain specific knowledge from the prior experience of the chess masters.
Chase and Simon (1988) report that human chess masters store from 10,000 to
100,000 chess patterns.
The skill of chess masters increases as more patterns are acquired. Novice
chess players move away from slow deductive reasoning towards the fast perceptual
processing used by masters as more and more patterns are learned (Chase & Simon,
1973). The chunking theory of learning (Laird et al., 1986) supports this viewpoint
in stating that performance improves via the acquisition of knowledge about patterns
and chunks. Richard Seltzer, the father of fourteen year old chess master Bobby
Seltzer, in a personal communication relates that Bobby's increase in skill is reflected
by an improved perception of the board position and subsequent position evaluation.
Bratko et al. (1986) state that natural positional moves are closely related to
the chunk structures of a game position. Specific chunks cause chess players to
generate tactical and strategic plans. The chunk displayed in Figure 4 will cause an
adversary to immediately consider plans for a back-row mate with either the queen
or one of the rooks. Back-row mate plans result from the knowledge contained in
the chunk concerning the opponent's king's limited mobility.
Figure 4: Sample cognitive chunk.
Humans recall chess chunks through familiarity with common board
configurations. Simon and Gilmartin (1973) developed MAPP which reconstructs
chess positions by simulating human eye movements recorded in their study. Simon
and Gilmartin's research shows that humans tend to group or chunk together pieces
which share geometric proximity. We will use geometric proximity as the primary
filter for selecting chess pieces to be formed into chunks. Proximity is one of the
Gestalt principles of organization (Anderson, 1980) along with similarity, good
continuation, and symmetry. We will also utilize the Gestalt principles of similarity
and good continuation to construct chunks. Chase and Simon (1973) found that
proximity and piece color, or similarity, were the two strongest factors affecting chunk
pattern recall by their chess master subjects.
The significant use of patterns of game pieces by chess masters has led us to
make the following three part hypothesis:
Chess masters acquire several thousand patterns of pieces that are
used to evaluate a game position.
Chess masters will continue to use a specific strategy while winning
games and tournaments.
Chess masters will tend to reduce the complexity of a game position
by moving board positions that do not contain familiar patterns into
positions that do contain familiar patterns.
We present the general hypothesis for all adversarial domains in Chapter 3. This
version of the hypothesis is the chess domain translation from the general hypothesis
and is presented here to examine the underlying foundations. The first part of our
hypothesis is a direct result from the research of Chase and Simon examined above.
The second part of our hypothesis is supported by the Reinforcement Theory
of psychology and is an extension of the results from various psychological
experiments (e.g., the water jug problem (Anderson, 1980) and the 1023-choice task
(Laird et al., 1986)) which indicate that human subjects will continue to use a learned
strategy for solving a class of problems even when a more efficient strategy would
also solve a specific problem.
While the third part of our hypothesis is intuitively appealing, direct support
comes from the research of Horgan et al. (1989) and Saariluoma (1984) which
demonstrates the use of chunking and analogy to similar positions by chess masters
in solving chess problems. The chunking of several pieces into a known pattern
produces a cognitive economy while evaluating complex board positions.
The search complexity of game trees is reduced as a result of the application
of our hypothesis. The strategic and tactical decisions of an adversary can be
accurately predicted because the adversary will tend to recreate familiar patterns to
reduce the cognitive complexity of evaluating the current game position. A practical
example of the use of familiar patterns is demonstrated in the 1978 Budapest
tournament game between Pinter and Bronstein (Roycroft, 1988). This game
resulted in a KNKBB endgame. Chess literature had previously claimed that a
position known as the Kling and Horowitz position (Kb6Nb7) will result in a draw
for the KNKBB endgame. Pinter's knowledge of the Kling and Horowitz position
led him to consistently maneuver to create this position. Pinter succeeded in forming
the Kling and Horowitz position three different times in three different corners of
the board. However, Pinter's haste to create the chunk caused him to make inferior
moves nearly thirty percent of the time according to the 250,000,000 position BBN
database (Roycroft, 1988) which exhaustively solves the KNKBB endgame in an
optimal number of moves.
Computer chess programs need to simulate the cognitive economy of chess
masters to continue to improve in performance. Lenat et al. (1979) state that
intelligent systems must perform expectation filtering, using predictions to filter
unsurprising data. Such a filtering process is provided in adversarial domains by the
ability to predict adversary moves in specific situations.
2.10.5 Computer Chess and Artificial Intelligence
The first chess "computer" was the automaton constructed by Torres y
Quevedo circa 1890 (Bell, 1978). Torres y Quevedo's chess machine was capable of
accurately playing certain variations of the KRK, King and Rook versus King,
Since Claude Shannon's (1950) paper on the requirements for a chess playing
program, many artificial intelligence researchers have devoted themselves to the
creation of master level game playing programs. Shannon defines two main
strategies for developing chess playing programs. These strategies are the type A
strategy which generates all possible moves to a certain depth of the game tree and
then uses an evaluation function to choose the best move and the type B strategy
which uses heuristics for suggesting moves without an exhaustive search.
The type A strategy chess programs dominate the type B strategy competitors.
Northwestern University's type A chess program CHESS4.X (Frey, 1983), were X is
a version number, reigned during the 1970s as the North American computer chess
program champion and was the World Computer Chess program champion from
1977 until 1980 (Kopec & Newborn, 1987). The three type A programs BELLE,
HITECH, and DEEP THOUGHT have each set the high water mark for chess
program playing strength at USCF ratings of 2200, 2400, and 2500 respectively.
These three programs have also been the North American computer chess program
champions from 1985 to the present (Newborn & Kopec, 1989).
The chart in Figure 5 shows the progress in playing strength of chess programs
with respect to search speeds. The bottom axis is logarithmic which means that the
linear progression in strength of chess programs during the 1970s was accomplished
by an exponential increase in search speeds. Recent progress depicted by the dashed
\ N C"
v w n
\ i 's &
\~ 8 ^
\5r .T 1S
^u I Il
A ^ P *
\ ~~ 5N m
curve reflects the hyper-exponential increase in search speeds required to maintain
the linear growth in playing ability. BELLE, HITECH, and DEEP THOUGHT each
use specialized hybrid architectures to achieve the dramatic increases in search speed
depicted in Figure 5 (Ebeling, 1986). HITECH uses a VLSI architecture with fine
grained parallelism to perform pattern matching which is cognitively similar to the
pattern recognition performed by human chess masters. DEEP THOUGHT
(Newborn & Kopec, 1989) uses special VLSI circuitry to achieve a search rate of
720,000 position nodes per second.
Although type A strategy chess programs are the best performing chess
programs, their performance against grandmaster rated human players has been
abysmal. In an exhibition round robin tournament played in October 1989 at Boston,
four human grandmasters crushed four of the top computer chess programs, including
HITECH and DEEP THOUGHT (Hamilton, 1990), with a score of 14.5 to 1.5.
World Chess Champion Gary Kasparov defeated DEEP THOUGHT in a two game
match played in New York prior to the Boston tournament. Kasparov commented
that DEEP THOUGHT lacked experience and was unable to analyze the reasons for
some of the moves he made (Geitner, 1989).
Berliner and Goetsch (1984) state that the performance of chess programs is
proportional to the amount of knowledge used by the program's evaluation function.
Bratko and Michie (1980) demonstrated the need to use domain specific knowledge
to solve certain domain tasks by having CHESS4.5 and the AL1 program try to solve
the KRKN endgame. CHESS4.5 was unable to solve two of the three chess
problems used in Bratko and Michie's research. However, AL1 which uses a type
B heuristic method with domain knowledge was able to solve all of the endgame
Type B strategies vary in their approaches to playing chess. The first heuristic
method uses plans or goals that have been constructed from domain specific
expertise to solve chess problems. AL1 is a goal based chess program which solves
KRKN endgames. LEBL (Tadepalli, 1989) uses goals during King-Pawn endgames
to reduce search complexity. Wilkins (1980 and 1982) also uses domain specific
knowledge to produce plans which his program PARADISE tries to achieve. Goal
based planning allows PARADISE to accurately play quiescent middle game
positions. The use of goals to achieve higher playing performance in chess programs
simulates the cognitive method of means-ends analysis used by humans in certain
problem solving tasks (Anderson, 1980).
Another heuristic approach which is similar to the cognitive methods analyzed
by Chase and Simon, described in section 2.10.4, simulates the human cognitive
process of chunking. This heuristic approach tries to identify common configurations
or chunks and then uses these chunks to identify analogous solutions to chess
problems. Quinlan's ID3 program (in Michalski et al., 1983) tries to inductively
classify equivalent board configurations in the KRKN endgame. The ID3 induced
patterns are used to determine if the King-Rook side has a winning position within
a specified number of moves. Currently the program has solved the KRKN endgame
for all positions which can be won in three ply. Campbell's (1988) CHUNKER also
takes advantage of board configurations to produce chunks which are used to solve
Each of the heuristic methods described above have been implemented on
endgame or middle game portions of chess problems and deal primarily with tactical
solutions to chess problems. The knowledge available to chess programs can be
significantly augmented by extending the research of Quinlan and Campbell in the
Collect chunks from complete chess games instead of just the endgame
Use the chunks of pieces to predict an adversary's tactical and strategic
2.10.6 Justification for Chess Domain Application
,' Shapiro (1987) presents several reasons for choosing chess as an experimental
1. The game constitutes a fully defined and well-formalized domain.
2. The game is sufficiently complex to challenge the highest levels of
human intellectual capacity.
3. A large body of knowledge has been accumulated and is available in
4. A generally accepted numerical scale of performance is available so
that increases in performance can be rated.
Our application domain needs to be sufficiently complex so that exhaustive search
is not a feasible solution. Typical chess game trees consist of 1043 nodes (Berliner
& Goetsch, 1984) which prohibits exhaustive search solutions given current
technological capabilities. Because chess is a well understood domain and there is
an accepted numerical rating scale, performance improvements contributed by the
adversary modeling methodology can be evaluated.
ADVERSARY MODELING METHODOLOGY
In this chapter, we examine in detail each of the functions required to create
an adversary model from the adversary modeling methodology. The foundation for
our adversary modeling methodology is the three part hypothesis:
Relevant patterns and plans are learned by the adversary from
practical experience in the domain.
The strategic and tactical actions of an adversary are repeated as long
as the actions produce a positive result.
An adversary tends to reduce the complexity of a situation to enable
a more precise strategic evaluation of the situation by using the plans
and patterns already acquired in the domain.
Evidence supporting this hypothesis and the specific chess domain translation are
presented in section 2.10.4.
To accurately predict the strategic and tactical plans of an adversary, we must
first be able to perceive and evaluate the domain the same as our adversary. This
means that we need to acquire the evaluation criteria used by our adversary. The
first and third parts of our foundational hypothesis indicate that all competitors in
adversarial domains make use of patterns that have been previously acquired in the
domain for their evaluation process.
3.1 Knowledge Acquisition of Chunks
For chess, the patterns involve both the geometric patterns of pieces and the
textual patterns related to the actual moves that are executed. The geometric
patterns correspond to chunks that are stored in visual memory, while the textual
patterns correspond to chunks that are stored in the adversary's verbal memory.
3.1.1 Defining the Chunks to be Acquired
We rely on the Gestalt principles of organization and the research of Chase
and Simon described in section 2.10.4 to construct our definition of the elements that
are to be contained in a geometric chunk. Because we are trying to find chunks of
pieces that are used to reduce the complexity of a situation, our chunks necessarily
have a minimum of two pieces. Although chunks need only two pieces to satisfy our
definition, chunks are generally required to contain a minimum of three pieces to be
used in our predictions of probable adversary moves. Chase and Simon (1973)
indicate that the chunks used by chess masters contain three to four pieces. Two
piece chunks are strategically significant in the opening game segment of chess due
to the minimal number of pieces that have moved from their starting positions and
in the end game segment due to the previous elimination of pieces. However, two
piece chunks in middle game positions often occur without effect from the strategy
or cognitive simplification efforts of the adversary. The chunking mechanism
emphasizes chunks which contain a minimum of three pieces.
Based on the principal of proximity, chunks contain only pieces which are
adjacent to another piece. Pieces contained in a chunk are all of the same color to
satisfy the principle of similarity. Requiring chunks to be of a single color, namely
the pieces played by the adversary, restricts the contents of the chunk to be under
direct control of the adversary. An adversary cannot control the move choices of an
opponent except for inter-locked pawn chains and therefore cannot use a chunk with
mixed colors of pieces for long range strategic planning.
Research performed by Church and Church (Holding, 1985) demonstrates that
human chess players have difficulty in processing information from pieces on
diagonals. From this research we only chunk pieces by proximity which are adjacent
along either the horizontal axis or vertical axis of the playing board. Other
adversarial game domains like checkers can rotate the major axes to include the
diagonals and exclude the horizontal and vertical axes.
The Gestalt principle of good continuation or meaningfulness is used to chunk
pieces which are capable of affecting the strategic meaning of a chunk from the
diagonals. The bishops, queens, and king are all included in a chunk if they are
diagonally adjacent to another piece in the chunk. Meaningfulness is also used to
include pawns which are diagonally behind, to reflect the movement capabilities of
the pawn piece, a piece in a chunk.
Furthermore, we need to define the maximum size of each chunk to build an
efficient knowledge representation structure for storing the chunks. Humans
compose complex chunks hierarchically from smaller, less complex chunks. We use
a maximum chunk size of a four-by-four, or sixteen square, board area. All of the
chunks collected by IAM from a twelve game analysis of M. Botvinnik are shown in
Figure 6. From these chunks we can see that a sixteen square area is sufficient to
hold all of the chunks found. We increased the maximum size of defined chunks to
five-by-five and six-by-six to measure the affect of a larger chunk size. Our research
indicated that only one additional chunk was captured during an analysis of twenty
games played by Botvinnik (see Section 4.2.4, Figure 22). This marginal increase in
acquired chunks does not merit the increased cost in space requirements for larger
chunk definitions. The sixteen square size chunk is the minimal implementation that
also permits the use of a chunk to acquire additional knowledge we deemed useful,
including the pieces remaining at the end of each game.
Because of the significance of pawn formations in the analysis of chess
positions, we acquire a second type of geometric chunk which is aimed at acquiring
pawn formations. As noted above, pawn formations are stable and an adversary can
rely on the presence of an opponent's pawns in a chunk. Our similarity principle is
modified to collect all similar pieces, i.e. pawns, instead of similar colors. The
proximity and meaningfulness principles remain the same for pawn chunks. Because
pawn structures can extend over a large area of the game board, we increase the size
of the maximum defined chunk from a sixteen square area to a thirty-six square area.
Support for increasing the maximum chunk size for the pawn formation chunks was
found when several pawn structures were acquired during our research which would
have exceeded the sixteen square area used in the standard geometric chunk
Figure 6: Chunks acquired for Botvinnik.
We have emulated the cognitive perceptual processes of human chess players
to define the composition of geometric chunks. Our definition of the elements which
are viable chunk members acquires chunks which are easily perceived and which
contain the maximum amount of knowledge concerning the current game situation.
Our definition of textual chunks is less complicated. Human chess players
remember verbatim dozens of opening lines of play that they have experienced. We
emulate this rote learning by chunking the first five moves made by the adversary
verbatim from the game records of the studied games. Our research indicates that
chess masters tend to leave an opening line of play between the fifth and tenth move
of the game. We are interested in acquiring knowledge that is high in quality and
reliability and therefore we rote learn only the first five moves. One of our
experiments considered the effect of increasing the textual chunk size to ten and
fifteen moves and the results of this experiment are detailed in Chapter 4.
3.1.2 Acquiring Geometric Chunks
From our background study presented in Chapter 2, we note the significance
of perceptual processing in the development of chess expertise. We take advantage
of this aspect of expert chess performance by viewing the game board and pieces as
an image to be processed into chunks. By viewing the game board as an image, we
consider via analogy previously defined computer vision techniques for processing
images and collecting information.
The convolution operator is used in the image algebra (Ritter et al., 1988) of
computer vision to collect information from the area surrounding a specific pixel,
which is analogous the proximity principle we use for defining chunks. For games,
a homomorphism can be drawn between pixels in an image and pieces on a game
board. The meaningfulness principle that we use in our chunk definition requires us
to capture knowledge from different areas surrounding a piece, depending on the
current piece being analyzed. The technique of varying templates enables us to alter
the summing template used by the convolution operator to correspond to an
individual piece. This permits us to collect information about the proximal pieces
that is dependent upon the central piece.
We maintain an internal representation of the game board in an eight-by-eight
array with subscripts 'i' and 'j'. The first subscript denotes the column and the
second subscript denotes the row. Our internal representation is similar to the one
described by Shannon (1950) with negative values assigned to non-adversary pieces
and the numbers one through six representing the pawns, knights, bishops, rooks,
queens, and kings respectively. Unoccupied squares are denoted by the value zero.
A board position and its corresponding internal representation are demonstrated in
Figure 7. The piece value at each board location is denoted by the algebraic symbol
i= 1 2 3 4 5 6 7 8
Figure 7: Internal representation of a board position.
The three different templates used in our implementation of the convolution
algorithm are shown in Figure 8. Each template can be thought of as a three-by-
three array with the center location denoted by the subscript (0,0). Other template
subscripts range from negative one to one. The general template in Figure 8 is used
for bishops, queens, and the king with the other pieces being convolved by the
appropriately named template. The pseudo-code in Figure 9 demonstrates how the
convolution operation is performed. When an adversary's piece is detected, the
appropriate template is loaded into a copy of the current template--TP. The values
depicted in the templates were a proximal piece is located are summed and stored
in the return array Y using an inclusive or operator--G. We have defined the
predicate function, ON_BOARD, to verify that the array subscripts 'i + M' and 'j + N'
correspond to a valid board location. The return array Y is an eight-by-eight copy
of the internal board representation with all values initially set to zero. This
-4 -2 B 8 8 -4 -6 8
8 -3 -5 -1 -3 8 -1 -1
-1 -1 8 a -1 8 8 8
8a B B 8 -i a a
8 1 1 1 8 a -z2
1 8 2 8 2 1 8
8 8 8 8 8 1 3 1
48 5 4 8 6 B
4 8 16 4 8 16 D 8 0
2 32 2 32 2 32
1 128 64 0 128 0 0 128 0
General Pawn Rook, Knight
Figure 8: Convolution templates.
Case Xj of
3, 5, 6 : TP the general template ;; (See Figure 8)
1 : TP the pawn template
2, 4 : TP +- the rook-knight template
otherwise Return ;; Non-adversary piece, no need to convolve
For N = -1 to 1 do
For M = -1 to 1 do
If ONBOARD(i+N, j+M) and X(i+N), +m) <> 0
Yij +- Y,j TPN,M
Y(i+N),(j+M) Y(i+N),(j+M) TP-N,-M
Figure 9: Pseudo-code to perform convolution.
effectively captures all pieces satisfying our chunk definition qualities and returns
their relational position to the center piece location in Y.
The internal representation of the game board is updated after each move
that is read from the game record. For example, if the move 'al-cl' is read to move
the rook, in the lower-left corner of the board in Figure 7, two squares to the right,
then the value at X3,1 would be set to four and the value of X1,1 would be changed
from four to zero. Currently we use a modified algebraic notation for recognizing
the moves played during the game. Other notations can be easily translated to
algebraic notation for use by the IAM program. The chunks we desire to capture are
the ones that have just been formed by the adversary, which improves our capability
for predicting definite adversary actions. Therefore, we collect chunks following the
The appropriate step function shown in Figure 10 is applied to the board
representation X and is stored in an eight-by-eight array Y. Pseudo-code which
reduces the complexity of the ensuing board search by removing all pieces that do
not satisfy the appropriate similarity constraint is shown in Figure 11. We use the
step functions to increase the speed of the search algorithm which locates pieces that
can be chunked together. The convolution operation shown in Figure 9 is then
performed at the site of the current board search. If any neighbors are identified as
belonging to the chunk, then the convolution operator is applied iteratively to all
related neighbors. This acquires the chunk as soon as the left-most and bottom-most
piece, with respect to the al square, is found.
Chunks of pieces which have never been moved by the adversary do not
represent any significant new geometric relational or strategic knowledge. To reduce
the total number of chunks being stored, we discard chunks which have greater than
half of their pieces still in their original starting positions. To further reduce the
storage requirements of the adversary modeling methodology we take advantage of
Pawn Formation Chunks
Figure 10: Step functions.
For I = 1 to 8 do
For J = 1 to 8 do
If gathering standard geometric chunks AND XI, > 0
Then Yj, 1
Else If gathering pawn formation chunks AND ABS(X,) = 1
Then YI, 1
Else Y,4 4- 0
For I = 1 to 8 do
For J = 1 to 8 do
If Y, = 1 Then perform convolution
Figure 11: Pseudo-code to perform step function.
chunk durations. If a chunk is created on the tenth move of a game and persists
until the twentieth move, then only one chunk is created instead of ten chunks. The
times of a chunk's inception and subsequent dissolution are stored with each chunk.
We also use the time tags associated with each chunk to eliminate storing a chunk
which is dissolved by the adversary and then recreated at a later time during the
same game. Pseudo-code to perform the chunk comparisons, which produce the
storage savings just mentioned, is shown in Figure 12. The variable 'C' represents
the double linked list of chunk records with the subscript 'I' denoting the position in
the list and the variable 'D' represents the most recent chunk that has been identified
for the current game.
For I = 1 to the total number of chunks created for this game do
If C, = D
If D.start_time < CI.start_time
Then CI.starttime ,- D.start_time
If D.end_time > C,.end_time
Then C,.end_time D.endtime
Exit the loop
Figure 12: Pseudo-code to perform chunk coalescing.
An example of the chunks identified by the convolution operator and the
convolution values returned is shown in Figure 13 where the chunks have been
acquired for the white player. The position shown in Figure 13 is taken from the
fifth game of the 1951 World Championship match between Botvinnik and Bronstein
after white's eighteenth move. The three chunks acquired by the convolution
8 33 162 2
16 8 8 8 224 195 8
120 174 7
8 32 2 28
B 32 2 28
a-I- E 1 L
Blanks = 8
Figure 13: Chunks on a chess board and their convolution values.
algorithm are outlined in the figure. The pawn structure in the middle-left of the
board could become a strong factor in the endgame and possibly create a passed
pawn. This knowledge is available from the geometric relationship of the pieces in
the chunk. The other two chunks are a standard offensive configuration (i.e., the
queen and rook) and a standard defensive configuration (i.e., the fianchettoed bishop
and castled king) that are found in many chess games.
The generality of the geometric chunk acquisition technique is demonstrated
by applying the convolution operator to the adversarial domain of the Go game.
Because only one type of piece is used, the general template from Figure 8 is the
only template used in performing the convolution. The results of the convolution
algorithm applied to a section of the Go game board are shown in Figure 14. This
chunk represents the Go formation called eyes. Eyes are an important defensive
strategic feature in Go. Other Go formations such as ladders, knots, false eyes, and
tiger's mouths are also acquired by the adversary modeling methodology's
3.1.3 Acquiring Textual Chunks
The background research presented in Chapter 2 indicates that chess masters
make use of their verbal memory to recall specific opening sequences of play. Our
adversary modeling methodology acquires this facet of expert chess performance in
addition to the geometric piece chunks.
Chess masters can recall verbatim particular opening sequences of play, which
indicates the use of rote learning. We emulate the cognitive process of rote learning
The heavU black line Values in squares correspond
represents the edge to pieces located at game
of the game board. board intersections
Figure 14: Chunk acquisition in Go.
to acquire textual chunks. Each move from a previously played game is read in
textually, in modified algebraic notation, from the game record used as input. The
algebraic notation is then translated to update the current board representation
maintained by the adversary modeling methodology application. Prior to translating
the textual notation used to record the moves of an adversary's game, specific types
of moves are saved in the "move" section of the adversary knowledge base verbatim.
Due to the availability of transpositions which lead to the same board position
through several different move sequences, we store each of the first five moves
performed by an adversary individually. A statistical representation of the frequency
that a move occurs and the average game turn is generated and stored with each
move. The statistical representation includes: the number of times a specific move
168 35 34 98 138
has been observed, the mean move number at which the move has occurred, and the
color of the pieces being played.
From the statistical knowledge of each of the moves displayed by the
adversary, the adversary model predicts opening sequences of play based on the
average over-all performance of an adversary. The knowledge of piece color is
crucial because chess masters use different modes of play for each of the two colors.
As the white player, a chess master can choose which opening sequence to play and
has greater control over the position of the pieces following the opening. However,
as the black player, a chess master plays a responsive mode instead of the controlling
mode of white. The opening moves of black are made as responses to the white
player's move choices. Therefore, if an identical move is made by an adversary as
both a white piece and black piece player, then the two moves are considered as
different moves and are stored separately.
Other types of moves are useful for predicting the playing style of an
adversary. We save the number of pawn moves that occur during the first ten moves
of each game and the number of attacking moves made during the first ten and
twenty moves of each game to aid in inferring the playing style of an adversary. Any
other move which can be identified textually is also a candidate for the textual move
knowledge of the adversary model. A flowchart for the process of collecting textual-
based knowledge is shown in Figure 15. The frequency and direction of castling
moves are also acquired to assist in prediction of possible castle moves during an
adversary's future games.
To demonstrate the flowchart, assume that the move c4--pawn to queen's
bishop four--is made by the adversary on his fourth game turn. Since the move
occurs during the first five moves of the game, the move will be saved into the
textual chunk knowledge base. If the move already exists in the textual chunk
knowledge base, then the number of times this move has been observed is
incremented and the statistics regarding the time of the move are updated to reflect
the new knowledge. Because a pawn--the queen's bishop pawn--is being moved, we
increment the count of pawn moves made during the first ten moves of the game.
We save the total number of games analyzed, the number of games the
adversary played as white, the number of games won by the adversary, the number
of games won as white, and the average length of all analyzed games. This
information is used by the adversary model to determine the frequency of a specific
move and the likelihood that the move will be repeated by the adversary.
Other adversarial domains can benefit from the textual knowledge acquisition
performed by the adversary modeling methodology. For example, the acquisition of
textual patterns will capture standard operating procedures and protocols, such as
how to accept an enemy's surrender, that have been learned by an adversary in a
3.2 Induction to Learn Repeated Chunk Patterns
We have acquired the geometric chunks, both standard and pawn structure
chunks, through the computer vision technique of applying convolutions to the board
image. Our goal with the adversary modeling methodology is to learn the chunks
which are used by the adversary to evaluate domain situations. We propose that the
chunks which are commonly used by an adversary can be identified through their
repeated use across several games.
Induction is used to identify groups of chunks from the collection of chunks
acquired from each of the analyzed games. Each group which contains a minimum
of two chunks, indicating the observation of the chunk in two separate games, is
identified as a cognitive chunk used by the adversary in domain evaluations. Chunks
that are found only in single games may occur by chance and are therefore not used
to predict the tactical and strategic decisions of an adversary. Because of our
inductive premise, we require that a minimum of two adversary games be studied
prior to performing the induction.
The induction algorithm requires chunks to be identical before being grouped
together. Our induction algorithm is noise intolerant since even small variations in
a chunk can have a large effect on the strategic evaluation value of a chunk. An
example of three chunks which only differ by the location or value of a single piece
and which have significantly different strategic meanings is shown in Figure 16. The
first chunk shows a strong pawn formation, while the second chunk shows two pawns
being attacked by an opponent's pawn, and the third chunk shows a pawn formation
with a double pawn which may become a liability towards the end of the game. The
noise intolerance of the induction algorithm is not a real problem since most
adversarial domains keep very accurate records of the past performances of domain
i sij I iil SIui
Figure 16: The effect of noise on chunks.
Chunks are considered identical when each has the same number and type of
pieces arranged in the same geometric configurations. The strategic significance of
a chunk is usually invariant with respect to left and right alignment, so chunks are
reflected through a vertical axis of the game board while trying to identify identical
chunks. When the adversary is playing the black pieces, we reflect the pieces through
the mid-board horizontal axis to group chunks that occur while the adversary is
playing either color of pieces. Chunks considered to be identical, as shown in Figure
17, are grouped together and stored in the knowledge base that forms the adversary
Figure 17: Four identical chunks.
We use a frame based system to store the chunks found by the induction
algorithm in the adversary model's knowledge base. The slots in each frame are the
actual chunk pieces and relative positions of the induced chunks, the number of times
this chunk has been observed, the earliest move number when the chunk was created,
the latest move number when the chunk was dissolved, the outcome (win or loss) of
the games in which the chunk occurred, and the color of the adversary's pieces.
Although we have not implemented this slot, chunks which are observed frequently
can have a "best move" slot added which would model the scripts used by chess
masters for certain configurations.
The adversary model's knowledge base of patterns known to an adversary can
be constructed in parts. At least two games need to be processed by the convolution
operator to acquire potential chunks. After the potential chunks have been acquired,
the induction learning mechanism processes the potential chunks of each game by
first checking to see if the chunk already exists in the adversary model's knowledge
base. If the chunk already exists in the knowledge base then the appropriate slots
are updated to reflect the new information associated with the new potential chunk.
Should the chunk not exist in the current knowledge base, then the chunk is
compared against the chunks of every other game that has just been processed by the
convolution operation. Any identical chunks are grouped and stored in the
knowledge base as a new pattern known to the adversary.
The induction learning mechanism is applied separately to general chunks
containing different pieces of the same color and pawn structure chunks containing
only pawns of both colors. Following the induction procedure, the adversary model
has a knowledge base which identifies all the recurring textual and geometric
patterns from the previous games of a particular adversary. Multiple adversaries can
be modelled by the adversary modeling methodology with each adversary having a
separate knowledge base. The adversary model's knowledge bases of specific
adversaries are used to infer the playing style of an adversary and to predict the
tactical and strategic movement choices of an adversary.
3.3 Acquiring an Adversary's Playing Style
Knowing the general playing style of an adversary enables chess programs to
select moves that will place the adversary in a disadvantageous strategic position.
The general playing style of an adversary is inferred from the collection of patterns
contained in the adversary model knowledge base. We use a rule-based inference
engine to heuristically determine the playing style of an adversary.
Currently, we infer two different styles of play. The inference engine indicates
that an adversary prefers open positions, closed positions, or has no obvious
preference. The closed position style of play is further used to imply strong
positional play by the adversary. Likewise, an open position playing style implies a
strong tactical player.
The rule base heuristics address the following domain specific factors which
indicate an adversary's playing style preference:
The presence and relative size of pawn structures.
The number of pawn moves made during the opening sequence.
The presence of pawn chains, or inter-locked pawn structures.
The number of opening moves that are attacking--move a piece across
the mid-board horizontal axis.
Each rule is evaluated to determine if a specific playing style is demonstrated. The
rules are not competitive, but instead they lend support to the adversary model's
belief that a particular playing style is preferred by the adversary by adjusting the
likelihood value attached to each playing style. The playing style with the greatest
likelihood is then inferred as the general playing style of the adversary.
8 4 18
0 1.25 1.75 3.8
Pawn Structures of 5 Pawns
0 1.5 2.0 5.0+
8 1.5 3.8 18.8+
Attack ing Moves
Figure 18: Fuzzy logic used by the heuristic rules.
The heuristic rules use fuzzy logic during the evaluation process. Graphs of
the fuzzy values for each of the heuristic rules is shown in Figure 18, with the fuzzy
regions shaded. Heuristic rules that have a value in the fuzzy areas which lie close
to the division between the two styles of play currently inferred do not support either
hypothesis. Our decision to use fuzzy logic prevents us from assigning a playing style
to an adversary that is based on statistically insignificant data with respect to the
fuzzy set values.
The heuristic rules used to infer an adversary's playing style are not
competitive, new heuristic rules can be added at any location of the rule base. As
new domain specific knowledge becomes available that will infer additional playing
styles or support the current playing style inferences, the current rule base can be
The determination of an adversary's playing style preference has been added
to the generalized adversary modeling methodology to increase the adversary model's
performance in the chess domain. The playing style inference engine makes use of
domain specific knowledge in addition to the textual and geometric patterns and for
this reason is not considered as a main part of the adversary modeling methodology.
The pattern acquisition, induction learning mechanism, and adversary action
prediction tool each use only minimal amounts of domain specific knowledge so the
adversary model implementation can be used in other adversarial domains with little
or no modifications.
3.4 Application of the Adversary Model
The adversary model consists of a knowledge base of textual and geometric,
or visual, patterns known to the adversary. The third part of our hypothesis
presented at the beginning of this chapter indicates how we are going to use this
knowledge. An adversary will attempt to reduce the complexity of evaluating the
current domain situation by maneuvering the board position to contain one or more
of the pattern chunks with which he is familiar.
The adversary model also contains two processes which utilize the knowledge
about patterns known by an adversary. The first process detects analogous situations
in the domain and the second process attempts to predict the most probable
adversary action. We define an analogous domain situation to exist when a chunk
in the knowledge base is almost complete in the domain. The second process then
predicts the necessary move to complete the chunk.
Currently a move window size of one is used when detecting analogous
domain situations. For chess, this means that the chunk must be able to have all of
the pieces of the chunk in their correct relational positions in exactly one move.
Board positions that are considered to be analogous to chunks in the knowledge base
may only have at most two pieces out of position, but are normally required to only
have one piece out of position. An extension to the move window size has been
implemented to permit the adversary model to eliminate interfering pieces when a
castle is the required move to complete a chunk.
We detect the analogous situations we have defined for the chess domain by
performing pattern recognition on the game board. The board representation of the
adversary's game currently in progress is searched prior to each move to be made by
the adversary until a piece belonging to the adversary is located. Next all of the
chunks in the adversary model's knowledge base which can fit into the current board
location are compared against the current board pattern. Inconsistencies between
the chunks in the knowledge base and the current board configuration are counted.
An inconsistency exists if a board location corresponding to the location of a piece
in a knowledge base chunk is empty or contains a different piece. If only one
inconsistency exists, or two if the missing pieces are the king and rook, then the
analogous situation detection process checks to see if any viable move can complete
Chunks in the knowledge base occupy a square area of the game board
defined by the pieces farthest to the left, top, right, and bottom of the chunk. The
square areas of chunks frequently contain blank or non-occupied squares. These
squares serve as wild cards so that the actual square on the game board may be
blank or contain any other piece. This is done to simulate the effect of hierarchically
composing larger chunks from smaller chunks.
Each chunk in the knowledge base which can be completed suggests the
appropriate move to complete the chunk as the adversary's next move choice.
Several chunks can each have analogous situations for any particular turn of the
current game. The collection of possible moves that enable an adversary to complete
a chunk are evaluated by the adversary move prediction process to select the most
likely adversary move.
The adversary move prediction process uses a heuristic rule-based inference
engine to evaluate the collection of suggested moves that will complete chunks. The
inference engine is similar to the one described above for analyzing the playing style
of an adversary. An initial probability value is generated for each move by dividing
the number of times the chunk suggesting the move has been observed by the total
number of prior adversary games analyzed. The heuristic rules then add or subtract
from the base probability of each suggested move. The move with the highest
probability is then used to predict the adversary's next move. If two or more moves
have nearly equal probabilities, then the inference engine makes multiple move
Computer chess programs can use the probability value associated with each
move as a measure of belief. When the adversary model infers a probability for a
move that is less than some specified value, the chess program can choose to ignore
the adversary model's prediction.
The heuristics used in the prediction inference engine attempt to follow
several psychological principles of cognitive organization and economy. The general
description of the purpose of our heuristics follows.
Large chunks are favored over small chunks. Chunk size is dependent on the
number of pieces contained in a chunk.
Chunks containing major pieces are favored over chunks containing only
Favor moves which have been suggested by more than one chunk.
Reduce the probability of chunks that have only been observed in lost games.
Increase the probability of chunks whose move suggestion causes a gain in
material advantage. (This heuristic is only used in quiescent positions due to
the volatile nature of material advantage in non-quiescent positions.)
Chunks which have occurred while using the same color pieces as the current
game are favored over chunks that have only been observed for the opposite
Adjust the probability to account for temporal discrepancies. If the current
game move number is outside of the move range defined by the two slots for
the chunk's time of inception and dissolution, then reduce the probability
proportional to the distance between the current move and the chunk's
Eliminate smaller chunks that attempt to borrow pieces from an existing
Reduce the probability of chunks containing only two pieces during the
middle game segment.
If the adversary has just dissolved a chunk, then do not recreate the chunk
Reduce the probability of chunks that have a maximum dissolution game turn
that is within the time range covered by the textual opening move knowledge
These heuristics attempt to find the move suggestion which has the greatest number
of situational factors in common with the current game condition, such as the color
of the pieces and the time or game turn within the game. Additionally, the heuristics
promote the moves that will create the largest possible chunk and therefore afford
the greatest cognitive economy to the chess player. The heuristic which supports
moves suggested by multiple chunks is simulating the construction of hierarchically
complex chunks from smaller chunks. Our general description of the heuristic rules
indicates a minimal usage of domain specific knowledge which augments our
inter-domain application capabilities.
The role of the adversary model as a coach needs to be remembered.
Predictions of probable adversary actions are only made when the current game
situation resembles domain situations which have already been learned inductively.
After analyzing nine and then twelve games of Botvinnik the resultant adversary
models were used to predict the actions of Botvinnik in a new game which had not
been previously analyzed. For each of models' predictions, over forty percent of the
new game's board configurations had no similarities to the chunks in the knowledge
Geometric chunk patterns are uncommon during the opening game segment
of a chess game. The textual knowledge base of patterns known to the adversary is
used to supplement the lack of knowledge in the geometric chunk pattern knowledge
base. When a computer chess program equipped with the adversary model is playing
the white pieces, the statistical analysis of opening moves displayed by the adversary
is used to select an opening sequence of play which is unfamiliar to the adversary.
This grants a strategic advantage to the computer chess program with the first move
of the game. Predictions for the opening moves of the adversary are made by
selecting the move with the highest probability value. Probability values are assigned
by using the statistical values stored with each opening move displayed by the
adversary to attain a base probability corresponding to the frequency each move has
been observed. This base probability is then modified to account for any difference
between the current time of the game and the mean game turn of the knowledge
base textual move pattern.
Moves which have already been made in the current game cause the identical
move in the knowledge base to be rejected. Additionally, when the adversary is
playing the black pieces, we implemented "response code" in the prediction process
which heuristically favors textual move patterns that have previously followed the
current move just made by the white player. The effect of the response code is
examined in Chapter 4.
In this chapter we present a detailed examination of our implementation of
our adversary modeling methodology for the domain of chess. Our program is
named IAM, an acronym for Inductive Adversary Modeler. We first present several
detailed examples that demonstrate IAM's functionality. Next we review the
effectiveness of IAM by examining the predictive capabilities of our adversary model
while performing against an actual opponent.
4.1 Overview of IAM's Execution
IAM is defined in two stages. The first stage performs the knowledge
acquisition of chunks and the learning by induction phases of the adversary modeling
methodology which establish the knowledge base of geometric and textual chunks
contained in the adversary model. The second stage applies the knowledge in the
knowledge base to a game in progress to predict probable adversary moves and to
identify the adversary's playing style.
IAM has several global data structures which facilitate the algorithmic design
of the program. Chunks, both textual and geometric, and the suggested moves of the
adversary model are stored in dynamic data structures. The form of these structures
and the pointers to them are among the global data structures. Additionally,
application constants such as the size of geometric chunks and pawn structure chunks,
the number of opening moves to acquire, the internal representation of the chess
board, and various constants used by the heuristics of the two inference engines are
stored as global variables. The use of these global structures allows us to easily
change the amount, size, or type of knowledge that is to be acquired and provides
a means for fine tuning the heuristics used in predicting an adversary's move choices.
4.1.1 Knowledge Acquisition and Inductive Learning Stage
Input to IAM, for the knowledge acquisition and learning stage, consists of
two or three game records of the adversary's previous performances. The records are
written in algebraic notation as shown for the sample game in Figure 19. The first
line of the game record indicates the color of pieces played by the adversary, the
length of the game, and the game result, either win or loss, for the adversary. The
remaining lines are the actual moves of a particular game.
We use only the previous games of an adversary which have resulted in a win
or loss. Drawn games are often influenced by factors, like the state of health of the
adversary, that do not impart any knowledge about the strategic tendencies or
evaluation criteria of an adversary. A good example of this comes from the 1957
World Chess Championship match between Botvinnik and Smyslov. Three of the last
four games in the match ended in a draw with an average game length of thirteen
turns. Botvinnik was down by three points in this match and the first draw was
probably offered by Botvinnik so that he could rest and prepare for the next game.
The final two draws were offered and accepted because the result of the match was
Botvinnik vs. Keres (Moscow, May 4/5, 1948)
1. d4 d5 21. Nb4 Rd8 41. Kd3 Ne6
2. Nf3 Bf5 22. Qf5 Rd6 42. Nd5 Kc6
3. c4 e6 23. Rfcl Rxc6 43. h4 Nd8
4. cxd5 exd5 24. Rxc6 Rd8 44. Nf4 Kd6
5. Qb3 Nc6 25. Rxb6 cxb6 45. Nh5 Ne6
6. Bg5 Be7 26. Nc6 Qc7 46. Ke3 Ke7
7. Bxe7 Ngxe7 27. Nxd8 Qxd8 47. d5 Nc5
8. e3 Qd6 28. Qc2 Qc7 48. Nxg7 Kd6
9. Nbd2 0-0 29. Qxc7 Nxc7 49. Ne6 Nd7
10. Rcl a5 30. Nbl Kf8 50. Kd4 Ne5
11. a3 Rfc8 31. Kfl Ke7 51. Ng7 Nc4
12. Bd3 a4 32. Ke2 Kd6 52. Nf5+ Kc7
13. Qc2 Bxd3 33. Kd3 Kc6 53. Kc3 Kd7
14. Qxd3 Nd8 34. Nc3 Ne8 54. g4 Ne5
15. 0-0 Ne6 35. Na2 f6 55. g5 fxg5
16. Rc3 b5 36. f3 Nc7 56. hxg5 Nf3
17. Qc2 Rcb8 37. Nb4+ Kd6 57. Kb4 Nxg5
18. Nel Nc8 38. e4 dxe4+ 58. e5 h5
19. Rc6 Qe7 39. fxe4 Ne6 59. e6+ Kd8
20. Nd3 Nb6 40. Ke3 Nc7 60. Kxb5
Figure 19: An example of a game record--the 1948 Hague-Moscow Tournament.
a forgone conclusion, similar to our example in Chapter 2 of the young Soviet chess
master and the older Soviet grandmaster.
Additional knowledge can be inferred from the win/loss result of each game
played by an adversary. When the induction procedure detects a series of lost games
by the adversary, then the possibility that the adversary will be altering his
established strategy is inferred. The adversary move prediction inference engine
decreases the probability of chunks observed only in lost games from predicting the
next adversary move. Likewise, an overall winning performance by an adversary
reinforces the likelihood of chunks being repeated in future games.
Each move of every game is read in as input separately. The textual chunking
mechanism then identifies if the current move has any of the qualifications for being
saved in the textual portion of the knowledge base. If so, a dynamic data structure
for holding textual moves is created and the move is stored.
Next, the textual move is translated and the internal board representation is
updated. After each move made by the adversary, potential chunks for each game
are identified by the convolution operator we discussed in Chapter 3. Chunks which
are identical to another chunk in the same game are coalesced into a single chunk
(see the pseudo-code in Figure 12). This step, for the twelve games played by
Botvinnik in the 1948 Hague-Moscow Championship tournament, reduced the 699
chunks identified by the convolution operator to 272 potential chunks that the
inductive learning algorithm must process. Chunks from different games are
considered unique at this point in processing.
Pawn structure chunks are collected by the convolution operator separately
from the standard geometric piece chunks. Because the occurrence of identical pawn
structures during different segments of a game yields knowledge about an adversary's
playing style, pawn structure chunks are not coalesced.
After the individual geometric chunks have been collected for each game, the
induction algorithm processes the chunks to find patterns which are duplicated across
game boundaries. The induction algorithm found twenty-eight unique chunks, shown
in Figure 6, from the 272 potential chunks identified for Botvinnik in the
Hague-Moscow tournament. Each of the twenty-eight chunks induced was found in
at least two games. For example, the partial board representations for the three
games which produced one of the twenty-eight chunks are shown in Figure 20.
Botv inn ik
occurrence in three different games.
The induction algorithm, with pseudo-code shown in Figure 21, verifies if a
potential or induced chunk already exists in the adversary pattern knowledge base
and updates the knowledge base appropriately. When a chunk is found to be
repeated in multiple games, we use the function REMOVE to eliminate multiple
instances of the chunk from the list of potential chunks produced from our
convolution algorithm. The induced chunks which do not exist in the current
knowledge base are then written into new frames, created by the function NEW, in
the knowledge base by the function WRITE.
The potential chunks collected by the convolution operator which are not
grouped into a valid chunk to be saved in the knowledge base are written to a
temporary file. This temporary file serves as a database of non-repeating chunks
which is used to identify chunk repetitions that occur outside of the current games
being induced. All chunks in this temporary database are used by the induction
algorithm to group chunks that have not been previously grouped. The database is
deleted after all games from a specific time period or tournament have been
analyzed. The deletion of the database forces the induced chunks to have a temporal
relevance, or recency, to the potential chunks being analyzed.
The textual move chunks are then analyzed to gather the statistics to be used
by the prediction stage of IAM. The statistics that are gathered include the mean
game turn that a move was executed, the number of times a particular move was
observed, the results of the games in which the move was observed, and the color of
the pieces being played. This statistical analysis requires that only one entry be made
For I = 1 to number of chunks found for the current games do
Flag_1 FALSE ;; indicates when a chunk has been repeated
Flag_2 ,- FALSE ;; indicates when a chunk is duplicated in the KB
For J = I+1 to number of chunks found for the current games do
;; First check this chunk against the chunks that were just found
If C, = Cj
Increment number of times C, has been observed
If Ci.start_time > Cj.starttime
Then Ci.starttime ,- Cj.start_time
If Ci.end time < Cj.endtime
Then Ci.endtime Cj.endtime
For J = 1 to number of chunks currently in the chunk Knowledge Base do
;; Next check this chunk against all chunks already in the Knowledge Base
If CI = Cj
Then Begin ;; Update KB knowledge
Increment number of times Cj has been observed
If C1.starttime < Cj.starttime
Then Cj.starttime 4- Ci.starttime
If Ci.endtime > Cj.end_time
Then Cj.endtime C- Ci.endtime
If Flag_l = TRUE AND Flag_2 = FALSE
Figure 21: Pseudo-code to perform induction on geometric chunks.
in the knowledge base for each specific move. Additional information concerning the
total number of games analyzed, number of games played as white, number of games
won, and number of games won as white is stored with the textual move patterns.
The pawn structure chunks are also analyzed statistically at the same time as
the textual move patterns. The statistical analysis gathers knowledge that is used by
the adversary playing style inference engine. This knowledge is about the presence,
quantity, and quality of pawn structures and inter-locked pawn chains. The quality
of a pawn structure is determined by the number of pawns involved in the structure.
The knowledge from this analysis is stored with the textual moves in the adversary
patterns knowledge base.
Following the induction on geometric chunks and the statistical analysis of
pawn structures and rote learned textual chunks, the knowledge base of patterns
known by an adversary is available for use by the prediction stage of the adversary
model. The adversary pattern knowledge base is constructed incrementally. As new
games of an adversary become available the chunk knowledge from those games can
be quickly incorporated into the existing knowledge base.
4.1.2 Predicting Adversary Actions Stage
After an adversary patterns knowledge base has been constructed, the
adversary model then attempts to predict adversary actions in relevant domain
situations. Simulations of a game played by an adversary are used to verify the
prediction capabilities of the adversary modeling methodology. Each of the
simulated games used to represent a future contest against an adversary has never
been analyzed by IAM.
Prior to each of these simulated games, the preferred playing style of the
adversary is inferred from the chunks in the knowledge base. We had IAM study
twelve games of two different chess grandmasters who display preferences for the two
different playing styles currently inferred by IAM. The two players were Botvinnik,
who prefers closed positions, and Spassky, who prefers open positions. By analyzing
the presence and frequency of pawn structures and the frequency of pawn and
attacking moves, IAM correctly identified the playing style preference of each
adversary. Knowledge about the playing style of an adversary is then used by the
current domain program which the adversary model is coaching to select specific
moves that manipulate the game board away from the adversary's preferred style.
During the opening portion of the simulated game, IAM relies exclusively on
the textual move knowledge base which contains the statistical analysis of opening
patterns displayed by the adversary. Prior to the adversary's move, IAM predicts the
adversary's move from the moves currently in the knowledge base. Following the
actual move made by the adversary, the corresponding move in the knowledge base
is eliminated from further consideration. For an adversary playing the white pieces,
IAM has had accurate predictions ranging from forty percent to one hundred percent,
with a mean prediction accuracy of seventy-five percent.
IAM's opening sequence predictions are less accurate for an adversary playing
the black pieces. This is because the adversary is playing responsively to his
opponent's moves. Accuracy for opening move predictions against a black piece
playing adversary ranged from zero to twenty percent.
By modifying the textual chunk learning algorithm to remember the white
player's move which preceded an adversary's black move, we were able to
significantly increase IAM's opening sequence prediction accuracy to range between
forty and one hundred percent, with a mean of sixty percent. This modification
produces results which are similar to IAM's performance against the adversary while
playing the white pieces.
The number of textual chunks stored in the knowledge base increases
proportionately, since identical moves by the adversary which follow a different white
move are now considered as unique moves. The average increase in the knowledge
base size is forty-five percent. For example, after analyzing twenty-two games of
Botvinnik with the original textual chunking algorithm, thirty-one opening move
textual chunks were produced. With the response code modification forty-seven
opening move textual chunks were saved to the knowledge base. The actual textual
chunks are also larger to contain the adversary's opponent's move. Because of the
relatively small number of chunks in the knowledge base for a specific opponent, the
increase in prediction accuracy warrants the small increase in size to the knowledge
All move predictions following the opening sequence are made primarily from
the contents of the geometric piece chunk knowledge base. Before each move to be
made by the adversary, the current board representation is searched to find pieces
belonging to the adversary. This is done because each of the chunks contained in the
knowledge base consists of pieces belonging to the adversary.
Once an adversary's piece has been located on the board representation, each
chunk in the knowledge base is compared against the board representation to find
identical matches and analogous chunks, which have one or two pieces out of place.
Because chunks occupy a specific square area of the game board, chunks which are
to large to fit into the current board location are ignored to reduce processing time.
The detection of analogous board positions is performed via pattern matching. When
an analogous board position has been identified, IAM then checks to see if a legal
chess move would complete the analogous chunk. An example of this process is
shown in Figure 22. Four of the chunks shown in Figure 6 were used to identify
analogous positions on the current game board which is prior to Botvinnik's
thirteenth move from the game displayed in Figure 19. The corresponding moves
that would produce the chunks are also displayed in the figure.
The collection of moves suggested by the analogous chunks is then processed
by an inference engine to heuristically determine the most probable move. The
inference engine predicted the castling move for the Figure 22 board position. This
prediction represents the result of applying the various psychologically based
heuristics concerning chunk size and relevance to the current domain. The queen
being en prise is domain specific information and is therefore not considered by the
move prediction inference engine. However, the actual move made by Botvinnik,
Qc2 or b3-c2, is one of the moves suggested by an analogous chunk and reviewed by
the inference engine. An experiment to augment the adversary pattern knowledge
base with a minimal amount of domain specific knowledge is presented in section 4.2.
Figure 22: Predictions from four chunks.
The process of searching the current board configuration to identify analogous
positions is repeated before each move to be made by the adversary. Existing chess
programs can use IAM's predictive capabilities by having IAM search the possible
board configuration at a specific node of the game tree and finding analogous chunk
situations in the proposed board configuration. IAM will then predict the most
probable adversary move that corresponds to our domain simplification hypothesis
of Chapter 3.
4.1.3 Integrating IAM's Knowledge into a Chess Program
Our induction based adversary modeling methodology produces knowledge
which can be incorporated into current chess programs by two methods. Knowledge
about the specific opening sequences preferred by an adversary can be used to
manually optimize a chess program's opening book. Optimization of the opening
book entails augmenting the opening book with opening sequences that are unknown
to the adversary and adjusting the preferences of the chess program to select these
new opening sequences.
Current chess programs use a move generator to limit the breadth of their
game trees. Move generators suggest a number of moves, typically five, which are
considered as the only moves available from a specific position. We describe the
reduction of tree complexity by the adversary modeling methodology in Chapter 5.
IAM predicts the move that an adversary will execute next from a particular board
position. These predictions can be used by chess programs to further limit the search
of their game trees.
A chess program would have the adversary model make predictions of the
next move in the game tree while simultaneously generating the five best moves with
its move generator. When predictions from the adversary model are available, the
chess program would use the predictions to expand its current game tree. Otherwise,
the chess program would continue to use the moves supplied by its move generator.
Combinations of adversary model predicted moves and move generator suggested
moves can be used by heuristically ordering the moves suggested by both methods
and expanding the game tree with the first five moves from the ordered move list.
4.2 IAM's Performance Results
In this section we describe several experiments that have been performed with
IAM to test the overall capabilities of the adversary modeling methodology. Each
experiment is described preceding the analysis of the results.
The games of the adversary Botvinnik are all taken from his World
Championship matches from the 1948 Hague-Moscow tournament through his 1961
match against Tal. The two games which are used to simulate Botvinnik's
performance in a future match, come from Botvinnik's 1963 match against Petrosian,
and are shown in Figure 23 and Figure 24. The game TEST1 is the first game from
the Petrosian match resulting in a win or loss in which Botvinnik played the white
pieces. TEST2 is the next game from the match, following TEST1, in which
Botvinnik played the black pieces.
4.2.1 General Performance Results
The purpose of our first experiment is to determine the base performance
results of IAM so that we can compare these results against other experimental
results. We anticipate that the predictive capabilities of IAM will increase as more
chunks are learned. The records of every game played by Botvinnik from his World
Championship Matches, prior to 1963, are used as input.
126.96.36.199 Performance versus the adversary as the white player
TEST1 is simulated for IAM following the construction of the adversary
model's knowledge base for each of the tournaments of Botvinnik's career. The
length in game turns of TEST1 is fifty-seven moves. The performance of IAM in this
experiment is shown in Table 1.
TEST1--Botvinnik playing White
1. d4 d5 21. Re2 Nb6 41. Nc5 Bf5
2. c4 e6 22. Rhel Nc4 42. Kg3 a4
3. Nc3 Be7 23. Bxc4 Rxc4 43. Kf4 a3
4. cxd5 exd5 24. Rd2 Re8 44. Ke5 Rb4
5. Bf4 c6 25. Re3 a6 45. Nd3 Rb5
6. e3 Bf5 26. b3 Rc6 46. Kd6 Kf7
7. g4 Be6 27. Na4 b6 47. Kc6 Bxd3
8. h3 Nf6 28. Nb2 a5 48. Rxd3 Rb2
9. Bd3 c5 29. Nd3 f6 49. Rxa3 Rg2
10. Nf3 Nc6 30. h4 Bf7 50. Kxd5 Rxg5+
11. Kfl 0-0 31. Rxe8+ Be6 51. Kc6 h5
12. Kg2 cxd4 32. Qe3 Bf7 52. d5 Rg2
13. Nxd4 Nxd4 33. g5 Be6 53. d6 Rc2+
14. exd4 Nd7 34. Nf4 Bf7 54. Kd7 h4
15. Qc2 Nf6 35. Nd3 Be6 55. f4 Rf2
16. f3 Rc8 36. gxf6 Qxf6 56. Kc8 Rxf4
17. Be5 Bd6 37. Qg5 Qxg5+ 57. Ra7+
18. Rael Bxe5 38. hxg5 a4
19. Rxe5 g6 39. bxa4 Rc4 1:0
20. Qf2 Nd7 40. a5 bxa5
Figure 23: A game from
the 1963 Botvinnik versus Petrosian match.
The first column, #G, is the number of games currently analyzed to produce
the adversary pattern knowledge base. The #Ch and #M columns are the number
of geometric chunks and textual opening move chunks respectively. The first number
in the #M column represents the total number of chunks and the second number is
the chunks pertaining to white moves, for TEST1, or black moves, for TEST2. The
next three columns are the number of predictions made by IAM (#P), the number
of predictions which exactly matched the adversary's ensuing move (#C), and the