Using inductive inference of past performance to build strategic cognitive adversary models

MISSING IMAGE

Material Information

Title:
Using inductive inference of past performance to build strategic cognitive adversary models
Physical Description:
xi, 144 leaves : ill. ; 29 cm.
Language:
English
Creator:
Walczak, Steven Michael
Publication Date:

Subjects

Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 1990.
Bibliography:
Includes bibliographical references (leaves 135-143).
Statement of Responsibility:
by Steven Michael Walczak.
General Note:
Typescript.
General Note:
Vita.

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001677231
notis - AHY9134
oclc - 24880978
System ID:
AA00003762:00001

Full Text








USING INDUCTIVE INFERENCE OF
PAST PERFORMANCE TO BUILD STRATEGIC
COGNITIVE ADVERSARY MODELS












BY

STEVEN MICHAEL WALCZAK


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


1990











ACKNOWLEDGMENTS


I would like to thank the faculty and staff of the Computer and Information

Sciences Department. Several staff members proved to be invaluable in helping me

cope with the university system. I specifically recognize the assistance of Ms. Jeffie

Woodham, Ms. Marlene Hughes, and Ms. Cheryl Coleman. Ms. Tammera Reedy

and Ms. Judy Longfellow were especially helpful in getting various figures and papers

printed and delivered to their final destinations. Professor Joseph Wilson provided

me with insight into the vision based problems encountered while performing my

research. Other faculty members including George Logothetis, Richard

Newman-Wolfe, Gerhard Ritter, and Ravi Varadarajan encouraged me to openly

discuss research issues in computer science and to enjoy being a student.

Two computer and information science graduate students helped significantly

during the early stages of my research. I would like to thank USA Captain

Christopher Wagner and USN Lieutenant Richard Fiola for there assistance in

uncovering the adversarial modeling techniques used by the armed services.

I would like to give special thanks to my committee members--Manuel

Bermudez, Walter Cunningham, Douglas Dankel, and Ira Fischler. Dr. Cunningham

has been a tremendous source of knowledge and help in establishing the techniques

used by the adversary modeling methodology to infer adversary playing styles. I owe






a great debt of gratitude to my committee chairman, Paul Fishwick. He spent many

tireless hours reviewing and analyzing my research and encouraging me.

My greatest human source of aid, comfort, and encouragement during my

graduate studies has been my wife. Karen gave three years of her life to work and

get us both through graduate school. She has my undying love.

I thank God for giving me the wisdom, intelligence, and motivation to pursue

my doctoral degree. The knowledge of His presence in my life has been the single

most important factor in my ability to complete graduate school. I hope and pray

that I will be faithful and wise enough to hear His calling in my life, that I may

choose to follow His path.












TABLE OF CONTENTS



ACKNOWLEDGEMENTS ...................


LIST OF TABLES ............

LIST OF FIGURES ...........

ABSTRACT ................

CHAPTERS


............................ .... vii


1 INTRODUCTION .......


Problem Statement .....................
Contribution of the Dissertation ...........
Outline of the Dissertation ...............

2 BACKGROUND AND RELATED WORK .....

Induction .............................
Tree Search ..........................
Athletic Adversarial Domains .............
Education Domain .....................
Adversarial Game Domains ...............
Law Enforcement Domains ...............
Military Domains ......................
Human Adversary Modeling Efforts
Domains ...................
Computer Planning in Military Domains
Political and Economic Domains ...........


Military

. .


Religion Domain ...................................
Chess Domain--Our Application Area ...................
Human Use of Adversary Modelin in Chess .........
Importance of Pawn Structures and Chess Openings ...
Philosophy ..................................
Psychology ..................................






Computer Chess and Artificial Intelligence .......... 38
Justification for Chess Domain Application .......... 43

3 ADVERSARY MODELING METHODOLOGY ............. 45

Knowledge Acquisition of Chunks ................... 46
Defining the Chunks to be Acquired ............... 46
Acquiring Geometric Chunks .................... 50
Acquiring Textual Chunks ....................... 58
Induction to Learn Repeated Chunk Patterns .............. 62
Acquiring an Adversary's Playing Style ................... 66
Application of the Adversary Model .................... 68

4 IA M ............................................... 75

Overview of IAM's Execution ......................... 75
Knowledge Acquisition and Inductive Learning Stage 76
Predicting Adversary Actions Stage ................ 82
Integrating IAM's Knowledge into a Chess Program ... 86
IAM's Performance Results ........................... 88
General Performance Results .................... 88
Performance with Respect to Time ................ 99
Performance with Respect to Textual Chunk Size .... 102
Performance with Respect to Geometric Chunk Size 104
The Effect of Domain Specific Knowledge .......... 107
Repeatability of the Demonstrated Results ......... 108

5 RESU LTS .......................................... 110

Reducing Opening Book Size ........................ 111
Revealing New Search Paths in the Game Tree ........... 113
Reducing Search Complexity Through Automatic Pruning ... 116
Summary of Research Contributions ................... 122

6 CONCLUSIONS AND FUTURE RESEARCH .............. 124

Conclusions ...................................... 124
Future Research Directions .......................... 125
Acquire More Geometric Chunks ................ 125
Increase the Knowledge Contained in Each Chunk .. 126
Place More Adversarial Knowledge in the Adversary
M odel ............................... 128

APPENDIX ............................. .......... ........ 132






REFERENCES ............................................. 135

BIOGRAPHICAL SKETCH ................................... 144












LIST OF TABLES


1. IAM Performance Measures for TEST1 ......................... 91

2. Effect of Likelihood on Prediction Ratios ........................ 96

3. IAM Performance Measures for TEST2 ......................... 97

4. Performance with Responsive Modification ....................... 98

5. Effect of Recency on IAM Performance for TEST1 ................ 100

6. Effect of Aging/Forgetting on IAM's Performance ................. 101

7. Performance (TEST1) with 10 Opening Moves .................... 103

8. Performance (TEST1) with 15 Opening Moves .................... 103

9. IAM Performance (TEST1) with Five-by-Five Chunks .............. 105

10. IAM Performance (TEST1) with Six-by-Six Chunks ................ 105

11. Effect of Domain Specific Knowledge .......................... 108

12. Ratio of Won and Lost Games for White and Black ............... 111

13. Statistical Analysis of Soviet Chess Weaknesses ................... 130











LIST OF FIGURES



Logic flow diagram. ..........................

Overview of the physical system .................

Sample Minimax tree. ........................

Sample cognitive chunk. .......................

Computer chess program ratings. .................

Chunks acquired for Botvinnik. .................

Internal representation of a board position. ........

Convolution templates .......................

Pseudo-code to perform convolution. .............

Step functions ..............................

Pseudo-code to perform step function ............

Pseudo-code to perform chunk coalescing. .........

Chunks on a chess board and their convolution values.

Chunk acquisition in Go.......................

Flowchart to collect textual move knowledge .......

The effect of noise on chunks. ..................

Four identical chunks. ........................

Fuzzy logic used by the heuristic rules ............

viii






19. An example of a game record--the 1948 Hague-Moscow Tournament. .. 77

20. Chunk occurrence in three different games ....................... 79

21. Pseudo-code to perform induction on geometric chunks ............. 81

22. Predictions from four chunks. ................................ 86

23. A game from the 1963 Botvinnik versus Petrosian match ............ 89

24. Another game from the 1963 Botvinnik versus Petrosian match ....... 90

25. Pseudo-code to suggest possible adversary moves. ................. 94

26. Chunk acquired with 5x5 size and corresponding smaller chunk ....... 106

27. Hypothetical game/search tree. ............................... 114

28. Prediction of Botvinnik's next move. .......................... 116

29. Standard search tree and the IAM improved search tree ............ 118

30. Search tree obtained from Correct Piece identification. .............. 121

31. Knight template and partial Rook template. ...................... 127











Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

USING INDUCTIVE INFERENCE OF
PAST PERFORMANCE TO BUILD STRATEGIC
COGNITIVE ADVERSARY MODELS

By

Steven Michael Walczak

December 1990


Chairman: Paul A. Fishwick
Major Department: Computer and Information Sciences

To perform optimally in adversarial domains, artificial intelligence programs

must be able to identically evaluate domain situations and develop plans as the

adversary. Knowledge about an adversary's evaluation criteria and objectives is

difficult to obtain since an adversary will guard this information in order to promote

a favorable outcome for himself. We present an adversary modeling methodology

which is capable of acquiring the evaluation criteria of an adversary through

observations of the adversary's past performances in the domain.

Our adversary modeling methodology uses induction to capture perceptual

chunks that are repeatedly displayed by a specific adversary. The acquired chunks

are then used to predict the adversary's actions during future encounters. Pattern

recognition is used to identify perceptual chunks in the current adversarial domain






situation which are analogous to chunks that have been previously learned from the

current adversary.

We have implemented our adversary modeling methodology in the program

IAM, Inductive Adversary Modeler, which has been applied to the domain of chess.

When high quality knowledge is available about an adversary's past performances,

the adversary modeling methodology accurately predicts ten percent of an adversary's

future action plans. The ability to accurately predict an adversary's move choices

reduces the complexity of the game tree search which enhances the playing ability

of current chess programs.











CHAPTER 1
INTRODUCTION

Adversarial situations surround us in varied forms on a daily basis. Examples

of adversarial situations include vying for a promotion at our place of employment,

Monday night's football game, and military conflicts throughout the world. These

adversarial situations are characterized by a minimum of two actors each of whom

is attempting to realize a goal that conflicts with the goal of the opposing actor. The

actors can be individuals or groups of individuals such as a football team players that

are cooperating to achieve the goal of the actor.

When humans are faced with the problem solving task of realizing a goal

against the desires of an adversary, they employ a variety of techniques that enable

them to outperform their adversary and accomplish their goal. The standard

cognitive problem solving methods of difference reduction and working backwards

are insufficient in adversarial domains. Human competitors use deductive reasoning

and analogy to previous domain situations to realize their goals in adversarial

domains. The use of analogy to gain a strategic advantage over an adversary

necessitates the studying of a specific adversary's previous performance, preferably

in situations analogous to the upcoming contest.

Human competitors performing in adversarial domains study the past

performances of their adversaries to identify strengths and weaknesses which can be






2

exploited respectively to gain a strategic or tactical advantage. This information is

preserved by the competitor as a model of the likely actions an adversary will

perform in particular situations. The model of the adversary's probable actions is

then exploited when an appropriate situation occurs during the course of the

competition. Such situations arise when the current game state--situation in the

adversarial domain--is analogous to a game state which occurred in the adversary's

past.


1.1 Problem Statement


Current artificial intelligence application efforts in adversarial domains are not

performed optimally. A factor which keeps adversary-based programs from

performing at expert levels is the lack of knowledge about the potential adversary.

Lenat and Feigenbaum (1987) claim that domain specific knowledge is required to

solve difficult problems and that the performance of intelligent programs improves

as additional domain knowledge is utilized.

Acquiring knowledge for programs in adversarial domains is hindered by the

fact that adversaries are unwilling to reveal their tactical and strategic goals, since

this would be counter-productive to the attainment of their goals. Candland (1980)

and Bolc (1987) have claimed that knowledge about any domain is available from a

wide variety of sources. Lenat (1982) proposes the use of the observable actions of

the adversary while performing in the adversarial domain. The proper selection of

adversarial domain situations to observe guarantees the acquisition of high quality

knowledge.






3
For the domain of chess, lack of knowledge about an adversary's tactics and

strategies is analogous to searching the game tree to a depth of only one ply. All

moves are considered as winning moves and the game proceeds in a random manner.

To acquire knowledge for use in adversarial domain application programs, we will

emulate the traits of human experts performing in adversarial domains, by analyzing

the previous performances of a particular adversary in relevant domain situations.


1.2 Contribution of the Dissertation


Adversarial domain programs require knowledge about an adversary's tactics

and strategies to continue to improve their performance. We need to consider two

separate factors that affect the strategic decisions an adversary will make. An

adversary's strategic decisions are affected by the plans or scripts that the adversary

feels will satisfy the current strategic goal. Before the adversary can select a plan of

action however, the domain situation must be evaluated to determine which plans are

appropriate to that particular situation.

Current programs performing tactical and strategic planning for adversarial

domains rely heavily on tree structures and search techniques such as iterative

deepening and alpha-beta pruning. The ability to accurately predict the choices that

will be made by an adversary in specific domain situations reduces computational

complexity and increases the depth of search. The increased depth of search that our

research enables is directly responsible for the augmented tactical ability of these

search based programs.






4

The purpose of our research is to develop a generalized method for modeling

the strategic decision making process of an adversary. The information flow diagram

for our adversary modeling methodology is shown in Figure 1. Our adversary

modeling methodology makes use of knowledge which is currently not used by

programs operating in adversarial domains. By increasing the knowledge available

to tactical and strategic planners, we will improve the overall performance of these

adversarial domain applications.


Repeated Patterns Adversar|y I|
*Adversary

Sct ion Predictor

Current Adversar ial Analogous Situation
Domain Situation Detector



Figure 1: Logic flow diagram.

Our adversary modeling methodology uses psychological principles of

organization and evaluation such as the Gestalt principles of organization (Anderson,


Record of
Previous Performances


ADVERSARY nODEL
by the Adversary -
S Knowledge Base
of Patterns
Knowledge Acquisition
to Acquire Patterns noun bg
-I Adversar ies

Inductive Inference
to Identify & Learn I


I


I






5
1980, and Pomerantz, 1986) to emulate the primary evaluation criteria used by an

adversary. Kodratoff and Michalski (1990) note that recent machine learning efforts

have focused on building cognitive learning architectures. We intend for the

adversary modeling methodology to be applicable across all adversarial domains and

hence, we include as little domain specific knowledge as possible. This means that

we are content to acquire the primitive elements that an adversary uses in evaluating

a domain situation without embellishing those elements with further domain specific

knowledge.

The psychological principles of organization are applied to a collection of

previous adversary performances in the domain (e.g., chess games played by the

adversary in prior tournaments and matches). We acquire the cognitive chunks used

by an adversary to evaluate the current domain situation. The acquired cognitive

chunks of the adversary help us to view the domain the same as our adversary and

to avoid influencing domain evaluations with personal biases.

We now have a large collection of chunks that have been displayed by the

adversary. The chunks are either textual patterns corresponding to the adversary's

verbal memory or geometric patterns corresponding to the adversary's visual memory.

The chunks we have acquired contain patterns that have been used only once and

may have occurred by chance. We use induction to eliminate these chance chunks

and create a valid and useful adversary model. The inductive phase identifies chunks

which are repeated in two or more games. The repetition of patterns is used to

indicate the probability that an adversary will use the acquired chunk in future






6

evaluations of domain situations. Because we are using induction as our learning

method, the adversary model created will be used as a heuristic guide instead of an

algorithmic solution.

Following the inductive learning, the collection of chunks that have been

repeatedly displayed by the adversary are stored in the adversary model. We

consider these chunks to be the primitive elements used by an adversary in evaluating

current domain situations. The chunks of the adversary model are then used to

predict adversary decisions when analogous situations occur in the adversarial

domain.

The knowledge obtained by the adversary modeling methodology is highly

reliable, because it comes directly from the adversary. A certain level of care must

be taken to guarantee that the adversary is performing normally in the domain and

not providing misinformation to the modeling methodology. We verify the adversary

modeling methodology by implementing the methodology for the adversarial domain

of chess. The implementation is called IAM, for Inductive Adversary Modeler. Our

justification for selecting the chess domain is given in section 2.10.6.

Berliner and Ebeling (1988) note that speed is a critical factor for evaluation

functions in chess as well as other adversarial domains. The knowledge that we are

learning about an adversary is available as historical records prior to the current

performance of the adversarial domain program. Therefore, the adversary modeling

methodology's knowledge acquisition can be performed prior to its use by the domain

program. This preprocessing effectively eliminates any time or complexity costs






7
generated by the adversary modeling methodology from being added to the current

program's evaluation function cost (Aho, 1974). The adversary model acts as a

heuristic coach to an existing domain program by predicting which moves an

adversary will make, as shown in Figure 2. The only part of the adversary model that

impacts the time order of the existing evaluation function is the analogous situation

detector that was shown in Figure 1.


Current
Game


Strategic


I
Advice J

Existing
Game Plaging Program

[__-


Figure 2: Overview of the physical system.


1.3 Outline of the Dissertation


We have presented a general overview of the problem of the lack of

knowledge about adversaries available to intelligent programs operating in






8

adversarial domains. Our solution to this problem has been briefly presented in this

chapter.

In Chapter 2, we present background material on induction and psychology

that affects our research. The research that has already been performed in several

of the adversarial domains including the domain of chess is discussed. Finally, we

present the reasons for selecting the domain of chess as our application domain.

In Chapter 3, we detail our research by first discussing the implementation

method and then the operation of the adversary modeling methodology. Each of the

acquisition and induction steps that have been briefly described in this chapter are

presented in detail.

Chapter 4 contains an example of the adversary modeling methodology in

action and provides quantitative performance results of our application program,

IAM. Our results and their impact on artificial intelligence search methods are

provided in Chapter 5. We discuss our conclusions and future research directions in

Chapter 6. The Appendix contains a glossary of chess and artificial intelligence

terms to aid the reader in understanding some of the domain descriptions that we

have presented.











CHAPTER 2
BACKGROUND AND RELATED WORK

In this chapter, we review the theoretical and applied work on induction. We

also describe the theoretical work on game tree search to emphasize our contribution

to this research area. Finally, we describe related work in the various adversarial

domains that have led to our solution with a detailed review of the domain of chess

which serves as our application domain. For each of the adversarial domains

presented, we describe the use of adversary modeling by humans and any applied

research that affects the implementation of our adversary modeling methodology.


2.1 Induction


Mankind has been using induction to perform problem solving and learning

tasks for many millennia. Rosch (1973) describes the way humans form concepts and

categories about their natural world. Polya (1973), in the early 1950s, formalized the

concept of induction. Induction is the process of discovering general laws or concepts

from observations and combinations of particular instances.

An example of induction for a natural world concept would be the

generalization that all ravens are black. This generalization is based on the

observation of a limited number of instances of a raven. Without seeing all the

possible examples or instances of raven, we are confident to induce that the color of






10
a raven is the same as the color of the instances we have seen--black. The strength

of our belief increases as more and more examples of black ravens are observed.

However, only one instance of a nonblack raven needs to be identified to invalidate

our induction.

From the above example we can see that inductive inference is learning in the

limit (Daley & Smith, 1986). We need to see sufficient observations of the right type

to form a correct generalized concept. Since we cannot guarantee that the right type

of observations are being provided to an inductive inference system, we must rely on

the use of a large number of examples of category instances to acquire the desired

concept. This means that the adversary modeling methodology we have developed

should perform more accurately as we provide greater numbers of examples of an

adversary's domain performance.

Since Polya's formalization of the concept of induction, theoretical research

on induction has primarily focused on the formation of concepts or classifications.

Angluin and Smith (1983) and Case and Smith (1983) present rigorous discussions

of the mathematical principles of induction. This theoretical background provides

a solid base for the use of inductive inference as a computational tool for learning

classes of objects or ideas.

The extraction of general principles from specific examples is a major research

goal in machine learning (Anderson, 1987). Induction has been applied to many

domains. Examples of applied inductive inference algorithms are Michalski's

INDUCE and Quinlan's ID3 (Michalski et al., 1983 and 1986). Each of these






11
example applications attempts to classify instances of a domain problem (e.g.,

soybean diseases for the INDUCE program) into maximally-specific groups that will

identify all known instances as well as accounting for unknown instances.

Prior to the theoretical discussions of inductive inference mentioned above,

Blum and Blum (1975) had already demonstrated that the ability to play games, or

at least to make technically correct moves, can be learned through inductive

inference. Induction has not been utilized in game domains due to the lack of high

quality play produced by Blums' algorithm. Our adversary modeling methodology

focuses on a narrower slice of the game domains, namely predicting adversary move

choices and playing style, and is able to learn high quality knowledge.

Adversarial domains present several specialized problems to the standard

inductive inference algorithms like ID3. Langley (1985) has noted that feedback is

an essential element of most learning methods. Inductive inference algorithms use

feedback to verify acquired concepts and to identify concepts that need to be

modified. In adversarial domains, feedback on the performance of an algorithm is

a not attainable until after the current domain situation has ended. Because of the

inherent time delay for feedback in adversarial domains, we substitute probabilistic

reasoning so that our confidence in the current adversary model increases as

additional examples of the same type are observed.

Another problem in adversarial domains is the dependence of the domain on

time. Actions occurring in the domain are temporally related. The inductive

inference applications cited above belong to the class of hierarchical or structure






12
induction methods. Structure induction methods are used to produce classification

systems. Muggleton (1990) proposes an inductive method called sequence induction

which has a time effect relationship to the domain. Sequence induction uses a series

of domain descriptions that are altered by actions operating on the domain to

produce control systems.

Mitchell et al. (1986) refer to a process of mechanized induction which

receives domain events as input and produces concepts for predicting future events.

For adversarial domains, sequence induction provides a method to produce

evaluation criteria for predicting an adversary's strategic decisions.


2.2 Tree Search


Programs in the adversarial domains of games invariably use trees as their

knowledge representation scheme. Finding solutions or move choices in game trees

involves searching the tree for the optimal finishing value. Von Neumann and

Morgenstern (1953) published the Minimax algorithm for backing the final game

values up through the game tree to the current node. Minimaxing attempts to

account for the effects of an adversary in the game domain by alternately raising the

minimum possible value or maximum possible value at each level of the tree.

If an exhaustive search can reach all of the leaf nodes of the tree, then the

true finishing value of the game can be raised to the current node and an optimal

game will ensue. However, games like chess, Go, and Othello have tree

representations that hold 1040 nodes or more. Game trees of this size prohibit an






13

exhaustive search. Instead, game programs will search down to a preselected depth

and apply an evaluation function that will determine a best guess for the true

finishing value obtainable from that node.

Evaluation functions use domain knowledge to predict the outcome of the

game from a specific position. All domain specific knowledge is contained in the

evaluation function. The quality of knowledge has a direct effect on the playing

ability of the game program (Berliner & Goetsch, 1984, and Lee & Mahajan, 1988).

Northwestern University's CHESS4.6 was able to defeat other chess programs which

searched several ply (levels) deeper into the game tree based on the strength of its

evaluation function (Elithorn & Banerji, 1984, and Wilkins, 1980).

The primary problem faced by search algorithms using tree structures is the

horizon effect. The horizon effect occurs when the solution to a search problem lies

at a depth in the tree that is beyond the current search depth. This is a realistic

problem for complex adversarial domains. In chess, the KRKN endgame, king and

rook versus king and knight, can require a twenty-seven move solution or a game tree

search depth of fifty-two ply (Bratko & Michie, 1980). Similarly, the KBBKN

endgame has solutions of sixty-six moves or one hundred thirty-two ply search depth

(Roycroft, 1988). Current chess programs search between eight and fourteen ply

before selecting a move.

Two different schools of thought exist concerning methods for continuing to

improve game program performance. The first school believes that the depth of the

search is the sole criterion affecting the playing ability of game programs.






14

Adelson-Velsky et al. (1988) state that increasing the depth of search of a game tree

increases the quality and reliability of game play. Berliner and Ebeling (1988)

specifically state that a deep tree search will outperform a shallower search which

uses more knowledge. Various techniques are used to increase the depth of search

for a program. Limiting the breadth of search and pruning of undesirable branches

through alpha-beta pruning and iterative deepening permit a deeper search for a

static number of evaluated tree nodes (Barr & Feigenbaum, 1981). Specialized

hardware can also be used to increase the speed and thereby the depth of search

(Ebeling, 1986).

An example of alpha-beta pruning is shown in Figure 3. The four leaf nodes

which have their branches cut by a diagonal line can each be pruned using alpha-beta

pruning. Once the three value is elevated to the left subtree level two node, then the

level one node is guaranteed a minimum value of three. Since each of the first leaf

nodes below the center and right subtrees have lower values than three, the

remaining leaf nodes do not need to be evaluated. This is called an alpha cutoff.

A beta cutoff is similar except that the cutoff is performed based on a guaranteed

maximum value for a Min node.

The other method for improving the performance of game programs is to

increase the knowledge that is available to them. We have already seen that better

quality knowledge enabled CHESS4.6 to outperform other programs that searched

several ply deeper into the game tree. The use of additional knowledge has the same

effect as searching deeper into the game tree. The projection ability of certain types




























Figure 3: Sample game tree with alpha-beta pruning.

of knowledge in the chess domain is nine ply for tactical knowledge, twenty-five ply

for positional knowledge, and forty-five ply for strategic knowledge (in Elithorn &

Banerji, 1984). Current game programs use mostly tactical knowledge along with

some positional knowledge.

For adversarial domains, the use of domain knowledge can be dangerous by

leading us into a false sense of security. Samuel (1959) warns that programs using

the Minimax algorithm must take into consideration the intent of the adversary.

Cecchini (in Crookal et al., 1987) claims that the actual intent of the adversary will

require programs to use different heuristics in order to win. Human tree search is

goal directed and the direction and method of search may change as new information


LEVEL

1





23


.(






16

becomes available (Frey, 1983). This last statement means that the preceding events

in a game and tournament affect the future strategic decisions of the adversary.

Our adversary modeling methodology follows the school of thought which

believes that more knowledge is required to continue improving the performance of

game programs. To avoid the complications of adversarial intent, our heuristics are

based on the primitive psychological elements that are used by an adversary to

evaluate the domain. Application of the adversary modeling heuristics occurs when

the current game board pattern is analogous to a previously induced situation. The

use of pattern recognition to increase search depths has already been implemented

by Samuel (1959 and 1967) in the form of signature tables and in the Russian chess

program KAISSA (Adelson-Velsky et al., 1975) to perform cutoffs or pruning.

The theoretical increase in search depth which we have achieved through the

adversary modeling methodology is detailed in Chapter 5. Several authors (Findler

& Meltzer, 1971, and Utgoff, 1986) have noted that the economy of a search

heuristic can be measured by the reduction in search effort and complexity which

produces a deeper search.


2.3 Athletic Adversarial Domains


The modeling of an adversary by humans is well documented in athletic

domains. Coaches of baseball (Aliston & Weiskopf, 1984), football (Riggins &

Winter, 1984), and other sports use scouting reports and films of their opponents to

study their opponent's strategies. This enables the coaches of athletic teams or






17

individuals to develop counter-strategies to foil the goals of the adversary and

promote their own goals.

In a personal communication, Owen J. Holyoak of the Exercise and Sport

Sciences Department at the University of Florida stated that the amount of

preparation performed by athletic team coaches corresponds directly to the perceived

threat of the adversary. When the technique of studying an adversary's past

performance is not fully used, surprising results can occur. At the 1982 homecoming

football game for the University of Florida (UF), the opponent was West Texas State

University (WTS). Although WTS was considered to be an underdog and was not

expected to score any points, they obtained a fourteen point lead early in the game

by using some plays that UF had not anticipated. The lack of preparation on UF's

part was responsible for the surprising start of the 1982 homecoming game.


2.4 Education Domain


The artificial intelligence domain of intelligent tutoring systems (ITSs) or

intelligent computer-aided instruction has already made successful use of models of

human thinking. These models are based on the availability of knowledge to the

human and that human's mental set at a particular point in time. The people being

modeled in ITS programs are the students who are attempting to learn. The student

models are used to predict the sources of error in student thinking which cause

incorrect responses to test items produced by the ITS.






18

Woolf (1984) has demonstrated an ITS which uses a model of the probable

knowledge currently held by the student for discourse planning. This model adapts

its understanding of the student's knowledge during the lesson to capture changes in

the student's cognitive state precipitated by the acquisition of new knowledge. Our

adversary modeling methodology operates similarly by constructing a model of the

probable evaluation mechanisms known to an adversary to predict future strategic

decisions of the adversary. Other examples of this modeling approach as used in

ITSs can be found in Sleeman and Brown (1982).


2.5 Adversarial Game Domains


Because of their finite size, games provide us with a convenient abstraction

of more difficult real world problems. The former Japanese use of the game of Go

to train their military leaders (Reiss Games, 1974) demonstrates the practical

extension of a game domain to a military domain. Pritchard (1973) elaborates

further by claiming that Go is guerilla warfare on a grand scale.

We will be using two person zero-sum perfect information games (Zagare,

1984) to demonstrate the adversary modeling methodology. Zero-sum games are

always adversarial in nature, since the wins of one side are equivalent to the losses

of the other side. Our selection of perfect information games was made to permit

an algorithmic evaluation of the results. The domain of chess which we will use for

our application of the adversary modeling methodology is reviewed in section 2.10.






19

Other types of games, such as games of chance, that do not fit our selection

of zero-sum perfect information games can also benefit from the adversary modeling

methodology. Our contribution which reduces the complexity of search is usable by

any game program that utilizes game tree search and heuristics to solve domain

problems. Berliner's BKG backgammon program (in Levy, 1988) is an example of

a game of chance which uses heuristic methods and game tree search to select

between moves.

Christensen and Korf (1986) claim that heuristic methods for one person

puzzles are equivalent to heuristic methods for two person games. The adversary

modeling methodology is capable of predicting moves in one person games that will

reduce the complexity of the current game position to one which has a known

solution. Other research using induction of previous games has been performed for

one person games. PREFER (Utgoff & Saxena, 1987) solves the 8-puzzle by

performing induction on previous 8-puzzle solutions.

Samuel (1959 and 1967) was the first researcher to use machine learning to

create an intelligent game playing program. Samuel's checker playing program uses

rote learning techniques to learn how to play checkers. This program is capable of

imitating the style of an adversary by mimicking the winning moves of the adversary

presented in the set of training instances. Programs which rely on rote learning tend

to become inflexible and require an equally large number of retraining instances to

adapt to a different style of play. The method of mimicking an adversary has been

shown to fail for the game of Go (Pritchard, 1973) and should not be considered as

a generalized method of strategic planning against adversaries.






20

Samuel developed signature tables to reduce search complexity. Signature

tables are used to remember the evaluation function value for specific positions.

When identical positions are identified through a pattern matching mechanism, the

signature tables are used to retrieve the corresponding node value. Wilcox (1985)

has also applied pattern recognition techniques to the game of Go to construct

NEMESIS, a strong Go playing program.

IAGO (Rosenbloom, 1982) uses the standard search techniques of alpha-beta

pruning and iterative deepening to play championship-level Othello. Rosenbloom

points out that the heuristics for IAGO need to change during different phases of the

game to capture shifts in the strategic importance of various game principles. For

example, in the middle game of Othello mobility is a critical factor, but in the end

game portion of Othello the quantity of stable pieces is the primary concern.

Berliner (1979) notes similar shifts in strategic reasoning for the domain of chess.

Our adversary modeling methodology compensates for the strategic shifts at various

points in the game by assigning time durations to the domain evaluation chunks that

are acquired for each adversary.

GINA (DeJong & Schultz, 1988) tries to learn to play Othello by using

experience gained from previous games. This approach is very similar to the one

used by Samuel for his checkers program. GINA attempts to exploit the weaknesses

of a specific opponent. Several of GINA's implementation features are applicable

across adversarial domains. These features are:






21

Serve as an external coach to an existing program in the domain.

Only use observable behavior to learn about an adversary. For GINA,

the observable behavior is the moves made by an adversary in prior

games.

Do not use feedback to modify learning, except for the results of the

games that have been studied.

The best example of strategic planning based on an adversary's modus

operandi is Waterman's (1970) poker player. Waterman has implemented a draw

poker playing program which analyzes an opponent's style of play to decide when a

bet is appropriate. This program learns over a period of several hands which

heuristics to use against a particular adversary. We expand upon Waterman's idea

by not limiting the program to a static number of predefined heuristics.


2.6 Law Enforcement Domains


Police departments and the Federal Bureau of Investigation each make use

of adversary modeling. As an example, a police detective will try to predict when,

where, and to whom a serial killer will perform his next crime. This is done

primarily by analyzing the criminal's previous performances of crimes. Similar

techniques are being considered for predicting the outcome of possible terrorist

activities in the RISK project sponsored by Lawrence Livermore National

Laboratories.






22

The key to successful drug enforcement is intelligence gathering and

information processing (Witkin, 1990). Raw data is voluminous and must be

converted to a useable form. Our adversary modeling methodology is capable of

finding patterns in raw data that correspond to resource movement problems (e.g.,

the actual drugs and the monetary compensations) and can be used by law

enforcement officials to increase their effectiveness.


2.7 Military Domains


Devastating results occur on the battlefield when sufficient preparation against

an adversary is not performed. Due to different sociological, geographical, and

educational backgrounds, potential adversaries will have unique behavioral rules

(Bond, 1986 and Boring, 1945). These behavioral rules govern varying beliefs and

goals concerning war and the use of nuclear weapons (McMillen, 1984 and

Sokolovskiy, 1968). Dunnigan (1982) and Ermarth (1978) state that if we assume

that an enemy will act strictly according to our own beliefs, then we will produce a

tactical blindness. Tactical blindness produces the inability to predict or fathom the

plans of our adversaries. Historic examples of the tactical and strategic blindness

produced by conscripting our own beliefs onto an adversary are the bombing of Pearl

Harbor by the Japanese in 1941, the defeat of the United States tactical forces at the

Chosin Reservoir during the Korean War (Wallace, 1990), and the Tet Offensive of

the North Vietnamese in 1968. Many soldier's lives might have been saved early in

World War II if the United States soldiers had understood the Japanese attitude

toward surrender (Boring, 1945).






23

2.7.1 Human Adversary Modeling Efforts in Military Domains

Military leaders (Gruner, 1988, and Ryan, 1988) claim that victory is often

contingent upon knowing your enemy. This means that military planners must be

able to evaluate the current situation using the same criteria as their adversary. A

vital component of intelligence information is briefings on an adversary's operational

characteristics or tactics (Gruner, 1988). Robert E. Lee, who is often cited as the

greatest military strategist (B. Davis, 1956), is a practical example of the power

accompanying detailed knowledge about an adversary. Lee was educated at West

Point, where all of the Union commanders were also trained, and served along side

many of the adversaries he would face in the Civil War during his tenure as an

officer in the United States Army and Cavalry (Snow, 1867). Lee's detailed

knowledge of the training and tactics of his adversaries enabled him to frequently

outperform better equipped and larger forces.

The United States military makes use of this concept in training our soldiers.

Each service branch maintains a cadre of personnel who are trained in Soviet-style

war tactics and strategy (Robbins, 1988). The training centers for the Army,

Air Force, and Navy are the National Training Center, Red Flag, and the Top Gun

school respectively. The purpose of these Soviet analogous fighting forces is to

provide a realistic training environment to better prepare the military for the

possibility of an actual war against Soviet forces or forces that use Soviet-style tactics.

A historic example of adversary modeling for military purposes occurred

during World War II. A detailed psychological profile of Hitler was constructed by






24

Langer (Langer, 1972) for the Allied forces. This model of Hitler was constructed

solely from second-hand information. The psychological model of Hitler accurately

predicted the method of his ultimate demise via suicide.

2.7.2 Computer Planning in Military Domains

Military games and strategic and tactical planners make use of varying levels

of adversary modeling, ranging from none in typical multi-player games and training

simulators to moderately advanced models in certain strategic planning programs.

Failing to account for the actions of an adversary severely inhibits strategic planning.

Military planning tools must account for the probable actions of an adversary based

on the adversary's intentions and strategic style, otherwise tactical blindness will

result.

The tactical troop movement planner TACPLAN (Andriole et al., 1986) is

typical of most military planning tools. TACPLAN only accounts for static elements

of the domain such as the effect of terrain on equipment movement. Knowledge

about adversaries is nonexistent. The next step towards an adversary model is

programs similar to ARES (Young & Lehner, 1986) which acknowledges the

presence of an adversary. ARES is also a tactical planner; however, the plans

generated by ARES attempt to account for the possible blocking actions of an

adversary. Although this approach claims to account for the actions of an adversary,

the actual outcome is not that different from the plans generated by programs like

TACPLAN. The lack of difference between the two approaches is caused because

the actions of an adversary are prescribed in advance which is similar to only






25

permitting a chess adversary to make forward moves. Infinitely many strategic

choices can be made by an adversary that will not be covered by the prescribed

action choices that are permitted by the tactical planning algorithm.

The POLITICS system (Carbonell, 1981) proceeds in the development plan

by introducing goals that an adversary is trying to accomplish, such as blocking the

current goal of the planning program. The adversarial goals used in POLITICS are

statically defined and are based on preconceived notions of an adversary's intent.

Basing the actions of an adversary on static predefined goals runs the risk of creating

tactical blindness mentioned in the introduction to section 2.7. The actual goals of

an adversary, much less how the adversary intends to accomplish those goals, may not

be known during the contest or may be contrary to the expected goals (Crookall et

al., 1987 and Narendra & Thathachar, 1989). POLITICS does provide a good

example of a program in which a thinking adversary attempts to achieve specific

goals.

TM (Erickson & Zytkow, 1988) is another goal based system for making

tactical decisions. The program BOGEY is an external learning program which

generates large numbers of simulations of tactical engagements with specific goals.

BOGEY, like GINA in section 2.5, is another example of the utility of coaching

programs that support existing domain specific programs through an external learning

function.

The MARK I and MARK II systems (Davis, 1988a and 1988b and Davis et

al., 1986) embody the mind set of an adversary in DEWT, Deterrence Escalation






26

control and War Termination, models. Currently the MARK systems have two

decision models, East and West. While MARK I and MARK II attempt to capture

differences in the mind sets between adversaries, the models are statically defined

like the goals of the POLITICS program. These static mind sets are based on

predefined conceptions of an adversary's intentions which may not be accurate and

will not account for the modification over time to decision heuristics used by an

adversary. One of the contributions that artificial intelligence can provide to military

planning is reasoning that accounts for the temporal relevance of actions (Bonasso,

1988).

The use of multiple models to generate goals for different adversaries

demonstrated by the MARK systems is a reasonable approach to separate the varying

evaluation criteria used by various adversaries. An improvement to the MARK

systems is accomplished by acquiring the evaluation criteria and traits of particular

adversaries in specific situations along with the temporal relevance for each of the

evaluation criteria of an adversary.

We mentioned in section 1.2 that geometric or visual relationships between

domain resources are used by the inductive learning mechanism of the adversary

modeling methodology. This means that military domains are required to represent

the physical resources of the domain in a geometric manner. The services are

trained to use standardized formations such as the column, echelon, line, and wedge

formations (Department of the Army, 1986). Typical board wargames make use of

a hexagonal map for plotting the movement of resources. By using such map






27

representations, the use of specific formations by a particular commander can be

acquired by our adversary modeling methodology.


2.8 Political and Economic Domains


Top level policy decisions made by government officials are also affected by

prior knowledge about an adversary's goals and strategies (Cimbala, 1987).

Government negotiators attempt to discover the limitations faced by an adversary to

accomplish the optimal outcome of an agreement or treaty between the two

adversaries. The use of knowledge of the political mind set for a geographic region

permits negotiators to interpret information from the proper perspective (Cimbala,

1987).

The political use of adversary modeling can also be extended to corporate

politics. We can acquire patterns from adversarial actions such as corporate

takeovers or mergers. This knowledge enables corporate entities to be better

defended or prepared for such adversarial actions.

Other domains that are concerned with the geometric relationships of

elements in the domain can benefit directly from the geometric pattern acquisition

phase of the adversary modeling methodology. Examples of such domains include

city planning and architecture. For the city planning domain, patterns of streets from

other cities which have desirable and equivalent traffic patterns can be acquired for

use in the current city. The primary benefits to these domains will be economic

savings and time savings for the reduced complexity of the design phase for

individual projects.







28

2.9 Religion Domain


The importance of knowing the intentions and strategic style of an adversary

has been recorded throughout history. An early example of these writings comes

from the first century A.D. when the apostle Paul (1 Corinthians 9:19-22) writes:

that I might win the more. And to the Jews I became as
a Jew, that I might win Jews; to those who are under the
Law, as under the Law to those without law, as
without law ... .To the weak I became weak, that I
might win the weak.

Paul is stating that he will model the lifestyle of the people he is attempting to win.

The training centers for the military at the National Training Center, Red Flag, and

the Top Gun schools are modern examples of the approach used by Paul.


2.10 Chess Domain--Our Application Area


We mentioned in section 2.5 that games like chess can be thought of as

abstractions of actual military contests. Although the outcome of these games may

be viewed as trivial compared to the outcomes of military conflicts, personal

reputation, and financial prosperity are the stakes of such games for expert game

players.

In the remainder of this section, we demonstrate the use of adversary

modeling by humans in the chess domain and the philosophical and psychological

foundations that enable the adversary modeling methodology to operate in the chess

domain. Finally, we present a detailed background of past and current computer

chess programs.






29
2.10.1 Human Use of Adversary Modeling in Chess

Before playing for the World Chess Championship against Capablanca who

was considered to be invincible and was appropriately called "The Chess Machine",

Alekhine studied and analyzed Capablanca's prior games (Schonberg, 1973). From

his study, Alekhine was able to determine a weakness in Capablanca's play that he

was able to exploit to become the new World Chess Champion. Other chess masters

including the World Chess Champions Botvinnik, Tal, and Kasparov prepare for

chess matches by rigorously studying the previous games of their opponents

(Horowitz, 1973).

Nunn (Nunn & Griffiths, 1987) describes a game against Ost-Hansen in which

both players based their game strategy on expectations of their adversary's playing

style. These expectations were formed from prior experience against the adversary.

Although no one was able to capitalize upon the information, it has been noted that

prior to his World Championship match against Spassky, Fischer varied his standard

opening for white from e4, pawn to King four, only three times during tournament

level play (Evans, 1970).

Several of the chess grandmasters, including Lasker and Tal (Horowitz, 1973

and Schonberg, 1973), have been described as psychological players. These players

would intentionally make the move that would be most disturbing to their opponents,

even though such moves were frequently not the best tactical choice. Levy (1984)

has described his style of play against computer chess programs similarly, stating that

his move choices are often made to confuse the computer.






30

The advent of intelligent programs in various game domains, including chess,

has caused humans expert game players to consider the computer as an adversary.

Subsequently, human competitors who will face an intelligent computer program as

an adversary have created models of adversary strategy and playing style that

represent the computer game programs. Certain human competitors are considered

to be specialists against computer styles of play.

A famous example of this specialization is the series of bets which David Levy

(1984) has had with the computer chess community. In these bets, Levy claims that

no computer chess program will be able to beat him. The first bet which had a ten

year duration was won by Levy, however, DEEP THOUGHT has defeated Levy in

the latest of the bets. A recent discussion on the rec.games.chess news network by

Feng-Hsiung Hsu indicates an increase in the number of human opponents of

computer chess programs using "anti-computer" playing styles against programs

entered in major chess tournaments.

When Mike Valvo defeated DEEP THOUGHT, the reigning computer chess

program champion for the past three years, in a two game postal chess match via

electronic mail, it was noted (Newborn & Kopec, 1989) that Valvo had observed

DEEP THOUGHT play a number of games and had acquired the playing style of

DEEP THOUGHT. The chess program did not have a similar opportunity. Berliner

(1988) claims that various chess players have asked for the previous game records of

HITECH, the 1989 computer chess program co-champion, so that they might be

prepared for a possible match.






31
2.10.2 Importance of Pawn Structures and Chess Openings

We pay special attention to pawn formations during our chunk learning

process. Philidor has stated that the win or loss of a game depends on the quality

of the pawn positions during the game (Pachman, 1975). The strength and weakness

of pawn formations is of paramount importance (Lasker, 1973). Pawn formations can

be used to reveal information concerning positional advantages (Reinfeld, 1987).

Furthermore, three of the six criteria for determining the character of a position

given by Capablanca relate to the effectiveness of the pawns in the game (Pachman,

1975).

Pawn formations contain information about the positional nature of the chess

game (Soltis, 1976). The geometric patterns of pawns displayed by a specific

adversary can be used to identify his playing style. By definition we can recognize

open and closed board positions solely from the absence or presence of long pawn

structures. The general playing style of the adversary is indicated by the predominant

style displayed by the adversary in the studied games. Adversaries will tend to

remain with the particular style that they feel most comfortable in playing. Chigorin

played only open style games (Pachman, 1975). Our own analysis of the games of

Botvinnik and Spassky has indicated that each of these chess grandmasters prefers

a particular style, closed and open respectively.

Openings also play an important role in determining the positional advantage

of the game. Current computer chess programs utilize a book or database of

opening moves which have a duration from two moves to more than ten moves. The

evaluation algorithms which enable chess programs to estimate the strength of the






32

current game position are not used until the book is exited. Typically, exits from

opening books occur only when an adversary makes a move that is not contained in

the book or the current opening line of play is exhausted. This means that current

computer chess programs can find themselves from ten to twenty percent of the way

through a game, for a game lasting fifty moves, before they start to calculate the

value of the current position.

The opening sequences of play that are used as the opening book for current

chess programs are stored verbatim. Mednis (1990) has demonstrated that particular

opening formations can be achieved through a wide variety of the actual move order.

A statistical analysis of the opening move sequences displayed by an adversary

accounts for opening move order variations by finding the upper bound, lower bound,

and mean time of specific moves.

Selecting an opening line of play is usually performed at random or in

response to the adversary's choice of opening move. By studying the playing style

and previous games of an adversary, specific opening lines that are well known to an

adversary are identified. This knowledge can be used by a chess program to emulate

human chess experts by choosing openings that are not as well known to the

adversary. Nunn (Nunn & Griffiths, 1987) avoids the Pelikan variation of the Sicilian

Defense in one of his games against Anthony because Nunn knows from experience

that Anthony is a Pelikan specialist. Other examples of selecting opening lines of

play based on the prior performance of an adversary are given by Nunn.

Acquisition of the opening sequence knowledge held by the adversary will

enable current chess computer programs to select lines of play which are strategically






33
advantageous. Holding (1985) supports our perspective by stating that we should use

information about players playing style to disrupt their normal play.

2.10.3 Philosophy

Dennett (1987) states that current chess programs concentrate their time on

branches of a game tree which represent the adversary's best response as viewed by

the program. While this would appear to be a reasonable approach, Dennett

elaborates by stating that a rational adversary might have a higher ulterior motive

than avoiding defeat in chess. For example, a young Soviet chess player with a rating

of master is performing well in a European chess tournament. The young Soviet has

acquired a score of 8.5 points and is about to face the tournament champion, a fellow

Soviet grandmaster who has scored 11 points thus far. The grandmaster and young

master both believe that the grandmaster will win the final game. However, no other

player in the tournament has a score greater than 9 points. The Soviet grandmaster

realizes that an outcome of a draw with himself will greatly raise the young Soviet

master's chess rating without damaging his own rating and might improve the young

master's confidence and future playing ability. Therefore, the Soviet grandmaster

who has already won the tournament by virtue of his previous victories offers the

younger master-level player a draw midway through the game.

Another example of the problems faced by computer programs in dealing with

the intentionality of human adversaries is provided by Levy (1984) who claims that

he will intentionally choose inferior moves when playing against a computer. The

inferior move choices are selected by Levy because he believes that computer chess






34

programs have greater difficulty in playing a winning game against such an irrational

style of play.

The Levy example above demonstrates one of the traits of human adversaries.

Humans, especially when competing against a machine, attempt to mislead their

adversary about their intentions and capabilities. We must be able to distinguish

between relevant information and misinformation about an adversary's playing style.

Incorporating the detection of misinformation from an adversary into the inductive

learning mechanism adds an additional heuristic element. The addition of another

heuristic element reduces the probability of acquiring a valid adversary model.

The effect of an adversary's misinformation attempts is negated by selecting

previous domain performances that limit the possibility of misinformation. By only

using the previous games of an adversary from tournament play in which the

adversary has something at stake, misinformation is eliminated. For our application

in the chess domain we use the World Championship Match games of adversaries

taken from the compendium by Gelo (1988).

2.10.4 Psychology

Chase and Simon (1973) have extended the research performed by de Groot

which claims that the depth of game tree search and the total number of moves

considered is approximately the same for both novices and chess masters. However,

the ability to recall chess positions after a five second presentation of the positions

is markedly different. Chess masters are able to reconstruct chess positions involving

twenty or more pieces with complete accuracy. Chess novices are only able to place






35

four or five of the pieces correctly. The difference between the masters and the

novices disappears when the subjects are asked to reproduce random patterns of

chess pieces. Each group can only place three or four of the pieces from the random

board configurations. The apparent decrease in ability by the master level players

is attributed to the fact that the random configurations of pieces contained no

inherent domain knowledge.

Based on Miller's (1956) hypothesis, the chess masters are forming higher

level chunks composed of groups of pieces from the game positions. These chunks

contain domain specific knowledge from the prior experience of the chess masters.

Chase and Simon (1988) report that human chess masters store from 10,000 to

100,000 chess patterns.

The skill of chess masters increases as more patterns are acquired. Novice

chess players move away from slow deductive reasoning towards the fast perceptual

processing used by masters as more and more patterns are learned (Chase & Simon,

1973). The chunking theory of learning (Laird et al., 1986) supports this viewpoint

in stating that performance improves via the acquisition of knowledge about patterns

and chunks. Richard Seltzer, the father of fourteen year old chess master Bobby

Seltzer, in a personal communication relates that Bobby's increase in skill is reflected

by an improved perception of the board position and subsequent position evaluation.

Bratko et al. (1986) state that natural positional moves are closely related to

the chunk structures of a game position. Specific chunks cause chess players to

generate tactical and strategic plans. The chunk displayed in Figure 4 will cause an






36
adversary to immediately consider plans for a back-row mate with either the queen

or one of the rooks. Back-row mate plans result from the knowledge contained in

the chunk concerning the opponent's king's limited mobility.





Ili .





Figure 4: Sample cognitive chunk.

Humans recall chess chunks through familiarity with common board

configurations. Simon and Gilmartin (1973) developed MAPP which reconstructs

chess positions by simulating human eye movements recorded in their study. Simon

and Gilmartin's research shows that humans tend to group or chunk together pieces

which share geometric proximity. We will use geometric proximity as the primary

filter for selecting chess pieces to be formed into chunks. Proximity is one of the

Gestalt principles of organization (Anderson, 1980) along with similarity, good

continuation, and symmetry. We will also utilize the Gestalt principles of similarity

and good continuation to construct chunks. Chase and Simon (1973) found that

proximity and piece color, or similarity, were the two strongest factors affecting chunk

pattern recall by their chess master subjects.

The significant use of patterns of game pieces by chess masters has led us to

make the following three part hypothesis:






37
Chess masters acquire several thousand patterns of pieces that are

used to evaluate a game position.

Chess masters will continue to use a specific strategy while winning

games and tournaments.

Chess masters will tend to reduce the complexity of a game position

by moving board positions that do not contain familiar patterns into

positions that do contain familiar patterns.

We present the general hypothesis for all adversarial domains in Chapter 3. This

version of the hypothesis is the chess domain translation from the general hypothesis

and is presented here to examine the underlying foundations. The first part of our

hypothesis is a direct result from the research of Chase and Simon examined above.

The second part of our hypothesis is supported by the Reinforcement Theory

of psychology and is an extension of the results from various psychological

experiments (e.g., the water jug problem (Anderson, 1980) and the 1023-choice task

(Laird et al., 1986)) which indicate that human subjects will continue to use a learned

strategy for solving a class of problems even when a more efficient strategy would

also solve a specific problem.

While the third part of our hypothesis is intuitively appealing, direct support

comes from the research of Horgan et al. (1989) and Saariluoma (1984) which

demonstrates the use of chunking and analogy to similar positions by chess masters

in solving chess problems. The chunking of several pieces into a known pattern

produces a cognitive economy while evaluating complex board positions.







38

The search complexity of game trees is reduced as a result of the application

of our hypothesis. The strategic and tactical decisions of an adversary can be

accurately predicted because the adversary will tend to recreate familiar patterns to

reduce the cognitive complexity of evaluating the current game position. A practical

example of the use of familiar patterns is demonstrated in the 1978 Budapest

tournament game between Pinter and Bronstein (Roycroft, 1988). This game

resulted in a KNKBB endgame. Chess literature had previously claimed that a

position known as the Kling and Horowitz position (Kb6Nb7) will result in a draw

for the KNKBB endgame. Pinter's knowledge of the Kling and Horowitz position

led him to consistently maneuver to create this position. Pinter succeeded in forming

the Kling and Horowitz position three different times in three different corners of

the board. However, Pinter's haste to create the chunk caused him to make inferior

moves nearly thirty percent of the time according to the 250,000,000 position BBN

database (Roycroft, 1988) which exhaustively solves the KNKBB endgame in an

optimal number of moves.

Computer chess programs need to simulate the cognitive economy of chess

masters to continue to improve in performance. Lenat et al. (1979) state that

intelligent systems must perform expectation filtering, using predictions to filter

unsurprising data. Such a filtering process is provided in adversarial domains by the

ability to predict adversary moves in specific situations.

2.10.5 Computer Chess and Artificial Intelligence

The first chess "computer" was the automaton constructed by Torres y

Quevedo circa 1890 (Bell, 1978). Torres y Quevedo's chess machine was capable of






39

accurately playing certain variations of the KRK, King and Rook versus King,

endgame.

Since Claude Shannon's (1950) paper on the requirements for a chess playing

program, many artificial intelligence researchers have devoted themselves to the

creation of master level game playing programs. Shannon defines two main

strategies for developing chess playing programs. These strategies are the type A

strategy which generates all possible moves to a certain depth of the game tree and

then uses an evaluation function to choose the best move and the type B strategy

which uses heuristics for suggesting moves without an exhaustive search.

The type A strategy chess programs dominate the type B strategy competitors.

Northwestern University's type A chess program CHESS4.X (Frey, 1983), were X is

a version number, reigned during the 1970s as the North American computer chess

program champion and was the World Computer Chess program champion from

1977 until 1980 (Kopec & Newborn, 1987). The three type A programs BELLE,

HITECH, and DEEP THOUGHT have each set the high water mark for chess

program playing strength at USCF ratings of 2200, 2400, and 2500 respectively.

These three programs have also been the North American computer chess program

champions from 1985 to the present (Newborn & Kopec, 1989).

The chart in Figure 5 shows the progress in playing strength of chess programs

with respect to search speeds. The bottom axis is logarithmic which means that the

linear progression in strength of chess programs during the 1970s was accomplished

by an exponential increase in search speeds. Recent progress depicted by the dashed







40


I





\ N

























'4;
I N
B -
















6I IC
L **

iS
'4 0






\ N C"
v w n

\ i 's &
\~ 8 ^
\5r .T 1S
^u I Il
A ^ P *
\ ~~ 5N m






41

curve reflects the hyper-exponential increase in search speeds required to maintain

the linear growth in playing ability. BELLE, HITECH, and DEEP THOUGHT each

use specialized hybrid architectures to achieve the dramatic increases in search speed

depicted in Figure 5 (Ebeling, 1986). HITECH uses a VLSI architecture with fine

grained parallelism to perform pattern matching which is cognitively similar to the

pattern recognition performed by human chess masters. DEEP THOUGHT

(Newborn & Kopec, 1989) uses special VLSI circuitry to achieve a search rate of

720,000 position nodes per second.

Although type A strategy chess programs are the best performing chess

programs, their performance against grandmaster rated human players has been

abysmal. In an exhibition round robin tournament played in October 1989 at Boston,

four human grandmasters crushed four of the top computer chess programs, including

HITECH and DEEP THOUGHT (Hamilton, 1990), with a score of 14.5 to 1.5.

World Chess Champion Gary Kasparov defeated DEEP THOUGHT in a two game

match played in New York prior to the Boston tournament. Kasparov commented

that DEEP THOUGHT lacked experience and was unable to analyze the reasons for

some of the moves he made (Geitner, 1989).

Berliner and Goetsch (1984) state that the performance of chess programs is

proportional to the amount of knowledge used by the program's evaluation function.

Bratko and Michie (1980) demonstrated the need to use domain specific knowledge

to solve certain domain tasks by having CHESS4.5 and the AL1 program try to solve

the KRKN endgame. CHESS4.5 was unable to solve two of the three chess






42

problems used in Bratko and Michie's research. However, AL1 which uses a type

B heuristic method with domain knowledge was able to solve all of the endgame

problems.

Type B strategies vary in their approaches to playing chess. The first heuristic

method uses plans or goals that have been constructed from domain specific

expertise to solve chess problems. AL1 is a goal based chess program which solves

KRKN endgames. LEBL (Tadepalli, 1989) uses goals during King-Pawn endgames

to reduce search complexity. Wilkins (1980 and 1982) also uses domain specific

knowledge to produce plans which his program PARADISE tries to achieve. Goal

based planning allows PARADISE to accurately play quiescent middle game

positions. The use of goals to achieve higher playing performance in chess programs

simulates the cognitive method of means-ends analysis used by humans in certain

problem solving tasks (Anderson, 1980).

Another heuristic approach which is similar to the cognitive methods analyzed

by Chase and Simon, described in section 2.10.4, simulates the human cognitive

process of chunking. This heuristic approach tries to identify common configurations

or chunks and then uses these chunks to identify analogous solutions to chess

problems. Quinlan's ID3 program (in Michalski et al., 1983) tries to inductively

classify equivalent board configurations in the KRKN endgame. The ID3 induced

patterns are used to determine if the King-Rook side has a winning position within

a specified number of moves. Currently the program has solved the KRKN endgame

for all positions which can be won in three ply. Campbell's (1988) CHUNKER also






43

takes advantage of board configurations to produce chunks which are used to solve

King-Pawn endgames.

Each of the heuristic methods described above have been implemented on

endgame or middle game portions of chess problems and deal primarily with tactical

solutions to chess problems. The knowledge available to chess programs can be

significantly augmented by extending the research of Quinlan and Campbell in the

following ways:

Collect chunks from complete chess games instead of just the endgame

segment.

Use the chunks of pieces to predict an adversary's tactical and strategic

movement decisions.

2.10.6 Justification for Chess Domain Application

,' Shapiro (1987) presents several reasons for choosing chess as an experimental

test-bench.

1. The game constitutes a fully defined and well-formalized domain.

2. The game is sufficiently complex to challenge the highest levels of

human intellectual capacity.

3. A large body of knowledge has been accumulated and is available in

literature form.

4. A generally accepted numerical scale of performance is available so

that increases in performance can be rated.

Our application domain needs to be sufficiently complex so that exhaustive search






44
is not a feasible solution. Typical chess game trees consist of 1043 nodes (Berliner

& Goetsch, 1984) which prohibits exhaustive search solutions given current

technological capabilities. Because chess is a well understood domain and there is

an accepted numerical rating scale, performance improvements contributed by the

adversary modeling methodology can be evaluated.












CHAPTER 3
ADVERSARY MODELING METHODOLOGY

In this chapter, we examine in detail each of the functions required to create

an adversary model from the adversary modeling methodology. The foundation for

our adversary modeling methodology is the three part hypothesis:

Relevant patterns and plans are learned by the adversary from

practical experience in the domain.

The strategic and tactical actions of an adversary are repeated as long

as the actions produce a positive result.

An adversary tends to reduce the complexity of a situation to enable

a more precise strategic evaluation of the situation by using the plans

and patterns already acquired in the domain.

Evidence supporting this hypothesis and the specific chess domain translation are

presented in section 2.10.4.

To accurately predict the strategic and tactical plans of an adversary, we must

first be able to perceive and evaluate the domain the same as our adversary. This

means that we need to acquire the evaluation criteria used by our adversary. The

first and third parts of our foundational hypothesis indicate that all competitors in

adversarial domains make use of patterns that have been previously acquired in the

domain for their evaluation process.







46

3.1 Knowledge Acquisition of Chunks


For chess, the patterns involve both the geometric patterns of pieces and the

textual patterns related to the actual moves that are executed. The geometric

patterns correspond to chunks that are stored in visual memory, while the textual

patterns correspond to chunks that are stored in the adversary's verbal memory.

3.1.1 Defining the Chunks to be Acquired

We rely on the Gestalt principles of organization and the research of Chase

and Simon described in section 2.10.4 to construct our definition of the elements that

are to be contained in a geometric chunk. Because we are trying to find chunks of

pieces that are used to reduce the complexity of a situation, our chunks necessarily

have a minimum of two pieces. Although chunks need only two pieces to satisfy our

definition, chunks are generally required to contain a minimum of three pieces to be

used in our predictions of probable adversary moves. Chase and Simon (1973)

indicate that the chunks used by chess masters contain three to four pieces. Two

piece chunks are strategically significant in the opening game segment of chess due

to the minimal number of pieces that have moved from their starting positions and

in the end game segment due to the previous elimination of pieces. However, two

piece chunks in middle game positions often occur without effect from the strategy

or cognitive simplification efforts of the adversary. The chunking mechanism

emphasizes chunks which contain a minimum of three pieces.

Based on the principal of proximity, chunks contain only pieces which are

adjacent to another piece. Pieces contained in a chunk are all of the same color to






47
satisfy the principle of similarity. Requiring chunks to be of a single color, namely

the pieces played by the adversary, restricts the contents of the chunk to be under

direct control of the adversary. An adversary cannot control the move choices of an

opponent except for inter-locked pawn chains and therefore cannot use a chunk with

mixed colors of pieces for long range strategic planning.

Research performed by Church and Church (Holding, 1985) demonstrates that

human chess players have difficulty in processing information from pieces on

diagonals. From this research we only chunk pieces by proximity which are adjacent

along either the horizontal axis or vertical axis of the playing board. Other

adversarial game domains like checkers can rotate the major axes to include the

diagonals and exclude the horizontal and vertical axes.

The Gestalt principle of good continuation or meaningfulness is used to chunk

pieces which are capable of affecting the strategic meaning of a chunk from the

diagonals. The bishops, queens, and king are all included in a chunk if they are

diagonally adjacent to another piece in the chunk. Meaningfulness is also used to

include pawns which are diagonally behind, to reflect the movement capabilities of

the pawn piece, a piece in a chunk.

Furthermore, we need to define the maximum size of each chunk to build an

efficient knowledge representation structure for storing the chunks. Humans

compose complex chunks hierarchically from smaller, less complex chunks. We use

a maximum chunk size of a four-by-four, or sixteen square, board area. All of the

chunks collected by IAM from a twelve game analysis of M. Botvinnik are shown in







48

Figure 6. From these chunks we can see that a sixteen square area is sufficient to

hold all of the chunks found. We increased the maximum size of defined chunks to

five-by-five and six-by-six to measure the affect of a larger chunk size. Our research

indicated that only one additional chunk was captured during an analysis of twenty

games played by Botvinnik (see Section 4.2.4, Figure 22). This marginal increase in

acquired chunks does not merit the increased cost in space requirements for larger

chunk definitions. The sixteen square size chunk is the minimal implementation that

also permits the use of a chunk to acquire additional knowledge we deemed useful,

including the pieces remaining at the end of each game.

Because of the significance of pawn formations in the analysis of chess

positions, we acquire a second type of geometric chunk which is aimed at acquiring

pawn formations. As noted above, pawn formations are stable and an adversary can

rely on the presence of an opponent's pawns in a chunk. Our similarity principle is

modified to collect all similar pieces, i.e. pawns, instead of similar colors. The

proximity and meaningfulness principles remain the same for pawn chunks. Because

pawn structures can extend over a large area of the game board, we increase the size

of the maximum defined chunk from a sixteen square area to a thirty-six square area.

Support for increasing the maximum chunk size for the pawn formation chunks was

found when several pawn structures were acquired during our research which would

have exceeded the sixteen square area used in the standard geometric chunk

definition.








a I

aII

I aaaI
al
onl I

a11






aa8


a L


a


Figure 6: Chunks acquired for Botvinnik.


w


a

a_






_a _
aa



Ag ac

















Figure 6--continued.

We have emulated the cognitive perceptual processes of human chess players

to define the composition of geometric chunks. Our definition of the elements which

are viable chunk members acquires chunks which are easily perceived and which

contain the maximum amount of knowledge concerning the current game situation.

Our definition of textual chunks is less complicated. Human chess players

remember verbatim dozens of opening lines of play that they have experienced. We

emulate this rote learning by chunking the first five moves made by the adversary

verbatim from the game records of the studied games. Our research indicates that

chess masters tend to leave an opening line of play between the fifth and tenth move

of the game. We are interested in acquiring knowledge that is high in quality and

reliability and therefore we rote learn only the first five moves. One of our

experiments considered the effect of increasing the textual chunk size to ten and

fifteen moves and the results of this experiment are detailed in Chapter 4.

3.1.2 Acquiring Geometric Chunks

From our background study presented in Chapter 2, we note the significance

of perceptual processing in the development of chess expertise. We take advantage

of this aspect of expert chess performance by viewing the game board and pieces as






51
an image to be processed into chunks. By viewing the game board as an image, we

consider via analogy previously defined computer vision techniques for processing

images and collecting information.

The convolution operator is used in the image algebra (Ritter et al., 1988) of

computer vision to collect information from the area surrounding a specific pixel,

which is analogous the proximity principle we use for defining chunks. For games,

a homomorphism can be drawn between pixels in an image and pieces on a game

board. The meaningfulness principle that we use in our chunk definition requires us

to capture knowledge from different areas surrounding a piece, depending on the

current piece being analyzed. The technique of varying templates enables us to alter

the summing template used by the convolution operator to correspond to an

individual piece. This permits us to collect information about the proximal pieces

that is dependent upon the central piece.

We maintain an internal representation of the game board in an eight-by-eight

array with subscripts 'i' and 'j'. The first subscript denotes the column and the

second subscript denotes the row. Our internal representation is similar to the one

described by Shannon (1950) with negative values assigned to non-adversary pieces

and the numbers one through six representing the pawns, knights, bishops, rooks,

queens, and kings respectively. Unoccupied squares are denoted by the value zero.

A board position and its corresponding internal representation are demonstrated in

Figure 7. The piece value at each board location is denoted by the algebraic symbol

Xij.


























i= 1 2 3 4 5 6 7 8

Figure 7: Internal representation of a board position.

The three different templates used in our implementation of the convolution

algorithm are shown in Figure 8. Each template can be thought of as a three-by-

three array with the center location denoted by the subscript (0,0). Other template

subscripts range from negative one to one. The general template in Figure 8 is used

for bishops, queens, and the king with the other pieces being convolved by the

appropriately named template. The pseudo-code in Figure 9 demonstrates how the

convolution operation is performed. When an adversary's piece is detected, the

appropriate template is loaded into a copy of the current template--TP. The values

depicted in the templates were a proximal piece is located are summed and stored

in the return array Y using an inclusive or operator--G. We have defined the

predicate function, ON_BOARD, to verify that the array subscripts 'i + M' and 'j + N'

correspond to a valid board location. The return array Y is an eight-by-eight copy

of the internal board representation with all values initially set to zero. This


j=8

7




4
3

B 2


-4 -2 B 8 8 -4 -6 8

8 -3 -5 -1 -3 8 -1 -1
-1 -1 8 a -1 8 8 8

8a B B 8 -i a a
8 1 1 1 8 a -z2
1 8 2 8 2 1 8
8 8 8 8 8 1 3 1

48 5 4 8 6 B










4 8 16 4 8 16 D 8 0
2 32 2 32 2 32
1 128 64 0 128 0 0 128 0

General Pawn Rook, Knight
Figure 8: Convolution templates.


Case Xj of
3, 5, 6 : TP the general template ;; (See Figure 8)
1 : TP the pawn template
2, 4 : TP +- the rook-knight template
otherwise Return ;; Non-adversary piece, no need to convolve
End Case
For N = -1 to 1 do
For M = -1 to 1 do
If ONBOARD(i+N, j+M) and X(i+N), +m) <> 0
Then Begin
Yij +- Y,j TPN,M
Y(i+N),(j+M) Y(i+N),(j+M) TP-N,-M
End Begin

Figure 9: Pseudo-code to perform convolution.

effectively captures all pieces satisfying our chunk definition qualities and returns

their relational position to the center piece location in Y.

The internal representation of the game board is updated after each move

that is read from the game record. For example, if the move 'al-cl' is read to move

the rook, in the lower-left corner of the board in Figure 7, two squares to the right,






54

then the value at X3,1 would be set to four and the value of X1,1 would be changed

from four to zero. Currently we use a modified algebraic notation for recognizing

the moves played during the game. Other notations can be easily translated to

algebraic notation for use by the IAM program. The chunks we desire to capture are

the ones that have just been formed by the adversary, which improves our capability

for predicting definite adversary actions. Therefore, we collect chunks following the

adversary's move.

The appropriate step function shown in Figure 10 is applied to the board

representation X and is stored in an eight-by-eight array Y. Pseudo-code which

reduces the complexity of the ensuing board search by removing all pieces that do

not satisfy the appropriate similarity constraint is shown in Figure 11. We use the

step functions to increase the speed of the search algorithm which locates pieces that

can be chunked together. The convolution operation shown in Figure 9 is then

performed at the site of the current board search. If any neighbors are identified as

belonging to the chunk, then the convolution operator is applied iteratively to all

related neighbors. This acquires the chunk as soon as the left-most and bottom-most

piece, with respect to the al square, is found.

Chunks of pieces which have never been moved by the adversary do not

represent any significant new geometric relational or strategic knowledge. To reduce

the total number of chunks being stored, we discard chunks which have greater than

half of their pieces still in their original starting positions. To further reduce the

storage requirements of the adversary modeling methodology we take advantage of













1- -




a


to


Standard

Geometric Chunks


Piece
Values
8
-1 1

P4

*$
wM


Pawn Formation Chunks


Figure 10: Step functions.


For I = 1 to 8 do

For J = 1 to 8 do

If gathering standard geometric chunks AND XI, > 0

Then Yj, 1
Else If gathering pawn formation chunks AND ABS(X,) = 1

Then YI, 1

Else Y,4 4- 0
For I = 1 to 8 do
For J = 1 to 8 do

If Y, = 1 Then perform convolution

Figure 11: Pseudo-code to perform step function.

chunk durations. If a chunk is created on the tenth move of a game and persists

until the twentieth move, then only one chunk is created instead of ten chunks. The


Piece Values


I






56

times of a chunk's inception and subsequent dissolution are stored with each chunk.

We also use the time tags associated with each chunk to eliminate storing a chunk

which is dissolved by the adversary and then recreated at a later time during the

same game. Pseudo-code to perform the chunk comparisons, which produce the

storage savings just mentioned, is shown in Figure 12. The variable 'C' represents

the double linked list of chunk records with the subscript 'I' denoting the position in

the list and the variable 'D' represents the most recent chunk that has been identified

for the current game.


For I = 1 to the total number of chunks created for this game do
If C, = D
Then Begin
If D.start_time < CI.start_time
Then CI.starttime ,- D.start_time
If D.end_time > C,.end_time
Then C,.end_time D.endtime
D -0
Exit the loop
End Begin
Figure 12: Pseudo-code to perform chunk coalescing.

An example of the chunks identified by the convolution operator and the

convolution values returned is shown in Figure 13 where the chunks have been

acquired for the white player. The position shown in Figure 13 is taken from the

fifth game of the 1951 World Championship match between Botvinnik and Bronstein

after white's eighteenth move. The three chunks acquired by the convolution



































8 33 162 2

16 8 8 8 224 195 8

120 174 7
8 32 2 28
B 32 2 28
a-I- E 1 L


Blanks = 8


Figure 13: Chunks on a chess board and their convolution values.






58

algorithm are outlined in the figure. The pawn structure in the middle-left of the

board could become a strong factor in the endgame and possibly create a passed

pawn. This knowledge is available from the geometric relationship of the pieces in

the chunk. The other two chunks are a standard offensive configuration (i.e., the

queen and rook) and a standard defensive configuration (i.e., the fianchettoed bishop

and castled king) that are found in many chess games.

The generality of the geometric chunk acquisition technique is demonstrated

by applying the convolution operator to the adversarial domain of the Go game.

Because only one type of piece is used, the general template from Figure 8 is the

only template used in performing the convolution. The results of the convolution

algorithm applied to a section of the Go game board are shown in Figure 14. This

chunk represents the Go formation called eyes. Eyes are an important defensive

strategic feature in Go. Other Go formations such as ladders, knots, false eyes, and

tiger's mouths are also acquired by the adversary modeling methodology's

convolution algorithm.

3.1.3 Acquiring Textual Chunks

The background research presented in Chapter 2 indicates that chess masters

make use of their verbal memory to recall specific opening sequences of play. Our

adversary modeling methodology acquires this facet of expert chess performance in

addition to the geometric piece chunks.

Chess masters can recall verbatim particular opening sequences of play, which

indicates the use of rote learning. We emulate the cognitive process of rote learning






















The heavU black line Values in squares correspond
represents the edge to pieces located at game
of the game board. board intersections

Figure 14: Chunk acquisition in Go.

to acquire textual chunks. Each move from a previously played game is read in

textually, in modified algebraic notation, from the game record used as input. The

algebraic notation is then translated to update the current board representation

maintained by the adversary modeling methodology application. Prior to translating

the textual notation used to record the moves of an adversary's game, specific types

of moves are saved in the "move" section of the adversary knowledge base verbatim.

Due to the availability of transpositions which lead to the same board position

through several different move sequences, we store each of the first five moves

performed by an adversary individually. A statistical representation of the frequency

that a move occurs and the average game turn is generated and stored with each

move. The statistical representation includes: the number of times a specific move


168 35 34 98 138

24 12






60

has been observed, the mean move number at which the move has occurred, and the

color of the pieces being played.

From the statistical knowledge of each of the moves displayed by the

adversary, the adversary model predicts opening sequences of play based on the

average over-all performance of an adversary. The knowledge of piece color is

crucial because chess masters use different modes of play for each of the two colors.

As the white player, a chess master can choose which opening sequence to play and

has greater control over the position of the pieces following the opening. However,

as the black player, a chess master plays a responsive mode instead of the controlling

mode of white. The opening moves of black are made as responses to the white

player's move choices. Therefore, if an identical move is made by an adversary as

both a white piece and black piece player, then the two moves are considered as

different moves and are stored separately.

Other types of moves are useful for predicting the playing style of an

adversary. We save the number of pawn moves that occur during the first ten moves

of each game and the number of attacking moves made during the first ten and

twenty moves of each game to aid in inferring the playing style of an adversary. Any

other move which can be identified textually is also a candidate for the textual move

knowledge of the adversary model. A flowchart for the process of collecting textual-

based knowledge is shown in Figure 15. The frequency and direction of castling

moves are also acquired to assist in prediction of possible castle moves during an

adversary's future games.


































To demonstrate the flowchart, assume that the move c4--pawn to queen's

bishop four--is made by the adversary on his fourth game turn. Since the move

occurs during the first five moves of the game, the move will be saved into the

textual chunk knowledge base. If the move already exists in the textual chunk

knowledge base, then the number of times this move has been observed is

incremented and the statistics regarding the time of the move are updated to reflect

the new knowledge. Because a pawn--the queen's bishop pawn--is being moved, we

increment the count of pawn moves made during the first ten moves of the game.

We save the total number of games analyzed, the number of games the

adversary played as white, the number of games won by the adversary, the number






62

of games won as white, and the average length of all analyzed games. This

information is used by the adversary model to determine the frequency of a specific

move and the likelihood that the move will be repeated by the adversary.

Other adversarial domains can benefit from the textual knowledge acquisition

performed by the adversary modeling methodology. For example, the acquisition of

textual patterns will capture standard operating procedures and protocols, such as

how to accept an enemy's surrender, that have been learned by an adversary in a

military domain.


3.2 Induction to Learn Repeated Chunk Patterns


We have acquired the geometric chunks, both standard and pawn structure

chunks, through the computer vision technique of applying convolutions to the board

image. Our goal with the adversary modeling methodology is to learn the chunks

which are used by the adversary to evaluate domain situations. We propose that the

chunks which are commonly used by an adversary can be identified through their

repeated use across several games.

Induction is used to identify groups of chunks from the collection of chunks

acquired from each of the analyzed games. Each group which contains a minimum

of two chunks, indicating the observation of the chunk in two separate games, is

identified as a cognitive chunk used by the adversary in domain evaluations. Chunks

that are found only in single games may occur by chance and are therefore not used

to predict the tactical and strategic decisions of an adversary. Because of our






63 .
inductive premise, we require that a minimum of two adversary games be studied

prior to performing the induction.

The induction algorithm requires chunks to be identical before being grouped

together. Our induction algorithm is noise intolerant since even small variations in

a chunk can have a large effect on the strategic evaluation value of a chunk. An

example of three chunks which only differ by the location or value of a single piece

and which have significantly different strategic meanings is shown in Figure 16. The

first chunk shows a strong pawn formation, while the second chunk shows two pawns

being attacked by an opponent's pawn, and the third chunk shows a pawn formation

with a double pawn which may become a liability towards the end of the game. The

noise intolerance of the induction algorithm is not a real problem since most

adversarial domains keep very accurate records of the past performances of domain

entities.




i sij I iil SIui





Figure 16: The effect of noise on chunks.

Chunks are considered identical when each has the same number and type of

pieces arranged in the same geometric configurations. The strategic significance of

a chunk is usually invariant with respect to left and right alignment, so chunks are

reflected through a vertical axis of the game board while trying to identify identical






64

chunks. When the adversary is playing the black pieces, we reflect the pieces through

the mid-board horizontal axis to group chunks that occur while the adversary is

playing either color of pieces. Chunks considered to be identical, as shown in Figure

17, are grouped together and stored in the knowledge base that forms the adversary

model.


Figure 17: Four identical chunks.

We use a frame based system to store the chunks found by the induction

algorithm in the adversary model's knowledge base. The slots in each frame are the

actual chunk pieces and relative positions of the induced chunks, the number of times

this chunk has been observed, the earliest move number when the chunk was created,

the latest move number when the chunk was dissolved, the outcome (win or loss) of






65
the games in which the chunk occurred, and the color of the adversary's pieces.

Although we have not implemented this slot, chunks which are observed frequently

can have a "best move" slot added which would model the scripts used by chess

masters for certain configurations.

The adversary model's knowledge base of patterns known to an adversary can

be constructed in parts. At least two games need to be processed by the convolution

operator to acquire potential chunks. After the potential chunks have been acquired,

the induction learning mechanism processes the potential chunks of each game by

first checking to see if the chunk already exists in the adversary model's knowledge

base. If the chunk already exists in the knowledge base then the appropriate slots

are updated to reflect the new information associated with the new potential chunk.

Should the chunk not exist in the current knowledge base, then the chunk is

compared against the chunks of every other game that has just been processed by the

convolution operation. Any identical chunks are grouped and stored in the

knowledge base as a new pattern known to the adversary.

The induction learning mechanism is applied separately to general chunks

containing different pieces of the same color and pawn structure chunks containing

only pawns of both colors. Following the induction procedure, the adversary model

has a knowledge base which identifies all the recurring textual and geometric

patterns from the previous games of a particular adversary. Multiple adversaries can

be modelled by the adversary modeling methodology with each adversary having a

separate knowledge base. The adversary model's knowledge bases of specific






66

adversaries are used to infer the playing style of an adversary and to predict the

tactical and strategic movement choices of an adversary.


3.3 Acquiring an Adversary's Playing Style


Knowing the general playing style of an adversary enables chess programs to

select moves that will place the adversary in a disadvantageous strategic position.

The general playing style of an adversary is inferred from the collection of patterns

contained in the adversary model knowledge base. We use a rule-based inference

engine to heuristically determine the playing style of an adversary.

Currently, we infer two different styles of play. The inference engine indicates

that an adversary prefers open positions, closed positions, or has no obvious

preference. The closed position style of play is further used to imply strong

positional play by the adversary. Likewise, an open position playing style implies a

strong tactical player.

The rule base heuristics address the following domain specific factors which

indicate an adversary's playing style preference:

The presence and relative size of pawn structures.

The number of pawn moves made during the opening sequence.

The presence of pawn chains, or inter-locked pawn structures.

The number of opening moves that are attacking--move a piece across

the mid-board horizontal axis.

Each rule is evaluated to determine if a specific playing style is demonstrated. The







67

rules are not competitive, but instead they lend support to the adversary model's

belief that a particular playing style is preferred by the adversary by adjusting the

likelihood value attached to each playing style. The playing style with the greatest

likelihood is then inferred as the general playing style of the adversary.


Pawn Moves








8 4 18









0 1.25 1.75 3.8
Pawn Chains


Xe3

Opei
StgJi

Clos'
StJ I


Pawn Structures of 5 Pawns
1.0i -


ed
.e


8.8


0 1.5 2.0 5.0+









8 1.5 3.8 18.8+
Attack ing Moves


Figure 18: Fuzzy logic used by the heuristic rules.

The heuristic rules use fuzzy logic during the evaluation process. Graphs of

the fuzzy values for each of the heuristic rules is shown in Figure 18, with the fuzzy

regions shaded. Heuristic rules that have a value in the fuzzy areas which lie close

to the division between the two styles of play currently inferred do not support either

hypothesis. Our decision to use fuzzy logic prevents us from assigning a playing style

to an adversary that is based on statistically insignificant data with respect to the

fuzzy set values.


1.8


1.0






8.8






68

The heuristic rules used to infer an adversary's playing style are not

competitive, new heuristic rules can be added at any location of the rule base. As

new domain specific knowledge becomes available that will infer additional playing

styles or support the current playing style inferences, the current rule base can be

easily augmented.

The determination of an adversary's playing style preference has been added

to the generalized adversary modeling methodology to increase the adversary model's

performance in the chess domain. The playing style inference engine makes use of

domain specific knowledge in addition to the textual and geometric patterns and for

this reason is not considered as a main part of the adversary modeling methodology.

The pattern acquisition, induction learning mechanism, and adversary action

prediction tool each use only minimal amounts of domain specific knowledge so the

adversary model implementation can be used in other adversarial domains with little

or no modifications.


3.4 Application of the Adversary Model


The adversary model consists of a knowledge base of textual and geometric,

or visual, patterns known to the adversary. The third part of our hypothesis

presented at the beginning of this chapter indicates how we are going to use this

knowledge. An adversary will attempt to reduce the complexity of evaluating the

current domain situation by maneuvering the board position to contain one or more

of the pattern chunks with which he is familiar.






69
The adversary model also contains two processes which utilize the knowledge

about patterns known by an adversary. The first process detects analogous situations

in the domain and the second process attempts to predict the most probable

adversary action. We define an analogous domain situation to exist when a chunk

in the knowledge base is almost complete in the domain. The second process then

predicts the necessary move to complete the chunk.

Currently a move window size of one is used when detecting analogous

domain situations. For chess, this means that the chunk must be able to have all of

the pieces of the chunk in their correct relational positions in exactly one move.

Board positions that are considered to be analogous to chunks in the knowledge base

may only have at most two pieces out of position, but are normally required to only

have one piece out of position. An extension to the move window size has been

implemented to permit the adversary model to eliminate interfering pieces when a

castle is the required move to complete a chunk.

We detect the analogous situations we have defined for the chess domain by

performing pattern recognition on the game board. The board representation of the

adversary's game currently in progress is searched prior to each move to be made by

the adversary until a piece belonging to the adversary is located. Next all of the

chunks in the adversary model's knowledge base which can fit into the current board

location are compared against the current board pattern. Inconsistencies between

the chunks in the knowledge base and the current board configuration are counted.

An inconsistency exists if a board location corresponding to the location of a piece






70

in a knowledge base chunk is empty or contains a different piece. If only one

inconsistency exists, or two if the missing pieces are the king and rook, then the

analogous situation detection process checks to see if any viable move can complete

the chunk.

Chunks in the knowledge base occupy a square area of the game board

defined by the pieces farthest to the left, top, right, and bottom of the chunk. The

square areas of chunks frequently contain blank or non-occupied squares. These

squares serve as wild cards so that the actual square on the game board may be

blank or contain any other piece. This is done to simulate the effect of hierarchically

composing larger chunks from smaller chunks.

Each chunk in the knowledge base which can be completed suggests the

appropriate move to complete the chunk as the adversary's next move choice.

Several chunks can each have analogous situations for any particular turn of the

current game. The collection of possible moves that enable an adversary to complete

a chunk are evaluated by the adversary move prediction process to select the most

likely adversary move.

The adversary move prediction process uses a heuristic rule-based inference

engine to evaluate the collection of suggested moves that will complete chunks. The

inference engine is similar to the one described above for analyzing the playing style

of an adversary. An initial probability value is generated for each move by dividing

the number of times the chunk suggesting the move has been observed by the total

number of prior adversary games analyzed. The heuristic rules then add or subtract






71
from the base probability of each suggested move. The move with the highest

probability is then used to predict the adversary's next move. If two or more moves

have nearly equal probabilities, then the inference engine makes multiple move

predictions.

Computer chess programs can use the probability value associated with each

move as a measure of belief. When the adversary model infers a probability for a

move that is less than some specified value, the chess program can choose to ignore

the adversary model's prediction.

The heuristics used in the prediction inference engine attempt to follow

several psychological principles of cognitive organization and economy. The general

description of the purpose of our heuristics follows.

Large chunks are favored over small chunks. Chunk size is dependent on the

number of pieces contained in a chunk.

Chunks containing major pieces are favored over chunks containing only

pawns.

Favor moves which have been suggested by more than one chunk.

Reduce the probability of chunks that have only been observed in lost games.

Increase the probability of chunks whose move suggestion causes a gain in

material advantage. (This heuristic is only used in quiescent positions due to

the volatile nature of material advantage in non-quiescent positions.)

Chunks which have occurred while using the same color pieces as the current

game are favored over chunks that have only been observed for the opposite

color.






72

Adjust the probability to account for temporal discrepancies. If the current

game move number is outside of the move range defined by the two slots for

the chunk's time of inception and dissolution, then reduce the probability

proportional to the distance between the current move and the chunk's

temporal range.

Eliminate smaller chunks that attempt to borrow pieces from an existing

larger chunk.

Reduce the probability of chunks containing only two pieces during the

middle game segment.

If the adversary has just dissolved a chunk, then do not recreate the chunk

immediately.

Reduce the probability of chunks that have a maximum dissolution game turn

that is within the time range covered by the textual opening move knowledge

base.

These heuristics attempt to find the move suggestion which has the greatest number

of situational factors in common with the current game condition, such as the color

of the pieces and the time or game turn within the game. Additionally, the heuristics

promote the moves that will create the largest possible chunk and therefore afford

the greatest cognitive economy to the chess player. The heuristic which supports

moves suggested by multiple chunks is simulating the construction of hierarchically

complex chunks from smaller chunks. Our general description of the heuristic rules

indicates a minimal usage of domain specific knowledge which augments our

inter-domain application capabilities.






73
The role of the adversary model as a coach needs to be remembered.

Predictions of probable adversary actions are only made when the current game

situation resembles domain situations which have already been learned inductively.

After analyzing nine and then twelve games of Botvinnik the resultant adversary

models were used to predict the actions of Botvinnik in a new game which had not

been previously analyzed. For each of models' predictions, over forty percent of the

new game's board configurations had no similarities to the chunks in the knowledge

base.

Geometric chunk patterns are uncommon during the opening game segment

of a chess game. The textual knowledge base of patterns known to the adversary is

used to supplement the lack of knowledge in the geometric chunk pattern knowledge

base. When a computer chess program equipped with the adversary model is playing

the white pieces, the statistical analysis of opening moves displayed by the adversary

is used to select an opening sequence of play which is unfamiliar to the adversary.

This grants a strategic advantage to the computer chess program with the first move

of the game. Predictions for the opening moves of the adversary are made by

selecting the move with the highest probability value. Probability values are assigned

by using the statistical values stored with each opening move displayed by the

adversary to attain a base probability corresponding to the frequency each move has

been observed. This base probability is then modified to account for any difference

between the current time of the game and the mean game turn of the knowledge

base textual move pattern.






74

Moves which have already been made in the current game cause the identical

move in the knowledge base to be rejected. Additionally, when the adversary is

playing the black pieces, we implemented "response code" in the prediction process

which heuristically favors textual move patterns that have previously followed the

current move just made by the white player. The effect of the response code is

examined in Chapter 4.











CHAPTER 4
IAM

In this chapter we present a detailed examination of our implementation of

our adversary modeling methodology for the domain of chess. Our program is

named IAM, an acronym for Inductive Adversary Modeler. We first present several

detailed examples that demonstrate IAM's functionality. Next we review the

effectiveness of IAM by examining the predictive capabilities of our adversary model

while performing against an actual opponent.


4.1 Overview of IAM's Execution


IAM is defined in two stages. The first stage performs the knowledge

acquisition of chunks and the learning by induction phases of the adversary modeling

methodology which establish the knowledge base of geometric and textual chunks

contained in the adversary model. The second stage applies the knowledge in the

knowledge base to a game in progress to predict probable adversary moves and to

identify the adversary's playing style.

IAM has several global data structures which facilitate the algorithmic design

of the program. Chunks, both textual and geometric, and the suggested moves of the

adversary model are stored in dynamic data structures. The form of these structures

and the pointers to them are among the global data structures. Additionally,






76

application constants such as the size of geometric chunks and pawn structure chunks,

the number of opening moves to acquire, the internal representation of the chess

board, and various constants used by the heuristics of the two inference engines are

stored as global variables. The use of these global structures allows us to easily

change the amount, size, or type of knowledge that is to be acquired and provides

a means for fine tuning the heuristics used in predicting an adversary's move choices.

4.1.1 Knowledge Acquisition and Inductive Learning Stage

Input to IAM, for the knowledge acquisition and learning stage, consists of

two or three game records of the adversary's previous performances. The records are

written in algebraic notation as shown for the sample game in Figure 19. The first

line of the game record indicates the color of pieces played by the adversary, the

length of the game, and the game result, either win or loss, for the adversary. The

remaining lines are the actual moves of a particular game.

We use only the previous games of an adversary which have resulted in a win

or loss. Drawn games are often influenced by factors, like the state of health of the

adversary, that do not impart any knowledge about the strategic tendencies or

evaluation criteria of an adversary. A good example of this comes from the 1957

World Chess Championship match between Botvinnik and Smyslov. Three of the last

four games in the match ended in a draw with an average game length of thirteen

turns. Botvinnik was down by three points in this match and the first draw was

probably offered by Botvinnik so that he could rest and prepare for the next game.

The final two draws were offered and accepted because the result of the match was









Botvinnik vs. Keres (Moscow, May 4/5, 1948)

1601
1. d4 d5 21. Nb4 Rd8 41. Kd3 Ne6
2. Nf3 Bf5 22. Qf5 Rd6 42. Nd5 Kc6
3. c4 e6 23. Rfcl Rxc6 43. h4 Nd8
4. cxd5 exd5 24. Rxc6 Rd8 44. Nf4 Kd6
5. Qb3 Nc6 25. Rxb6 cxb6 45. Nh5 Ne6
6. Bg5 Be7 26. Nc6 Qc7 46. Ke3 Ke7
7. Bxe7 Ngxe7 27. Nxd8 Qxd8 47. d5 Nc5
8. e3 Qd6 28. Qc2 Qc7 48. Nxg7 Kd6
9. Nbd2 0-0 29. Qxc7 Nxc7 49. Ne6 Nd7
10. Rcl a5 30. Nbl Kf8 50. Kd4 Ne5
11. a3 Rfc8 31. Kfl Ke7 51. Ng7 Nc4
12. Bd3 a4 32. Ke2 Kd6 52. Nf5+ Kc7
13. Qc2 Bxd3 33. Kd3 Kc6 53. Kc3 Kd7
14. Qxd3 Nd8 34. Nc3 Ne8 54. g4 Ne5
15. 0-0 Ne6 35. Na2 f6 55. g5 fxg5
16. Rc3 b5 36. f3 Nc7 56. hxg5 Nf3
17. Qc2 Rcb8 37. Nb4+ Kd6 57. Kb4 Nxg5
18. Nel Nc8 38. e4 dxe4+ 58. e5 h5
19. Rc6 Qe7 39. fxe4 Ne6 59. e6+ Kd8
20. Nd3 Nb6 40. Ke3 Nc7 60. Kxb5
1:0
Figure 19: An example of a game record--the 1948 Hague-Moscow Tournament.


a forgone conclusion, similar to our example in Chapter 2 of the young Soviet chess

master and the older Soviet grandmaster.

Additional knowledge can be inferred from the win/loss result of each game

played by an adversary. When the induction procedure detects a series of lost games

by the adversary, then the possibility that the adversary will be altering his

established strategy is inferred. The adversary move prediction inference engine

decreases the probability of chunks observed only in lost games from predicting the

next adversary move. Likewise, an overall winning performance by an adversary

reinforces the likelihood of chunks being repeated in future games.






78

Each move of every game is read in as input separately. The textual chunking

mechanism then identifies if the current move has any of the qualifications for being

saved in the textual portion of the knowledge base. If so, a dynamic data structure

for holding textual moves is created and the move is stored.

Next, the textual move is translated and the internal board representation is

updated. After each move made by the adversary, potential chunks for each game

are identified by the convolution operator we discussed in Chapter 3. Chunks which

are identical to another chunk in the same game are coalesced into a single chunk

(see the pseudo-code in Figure 12). This step, for the twelve games played by

Botvinnik in the 1948 Hague-Moscow Championship tournament, reduced the 699

chunks identified by the convolution operator to 272 potential chunks that the

inductive learning algorithm must process. Chunks from different games are

considered unique at this point in processing.

Pawn structure chunks are collected by the convolution operator separately

from the standard geometric piece chunks. Because the occurrence of identical pawn

structures during different segments of a game yields knowledge about an adversary's

playing style, pawn structure chunks are not coalesced.

After the individual geometric chunks have been collected for each game, the

induction algorithm processes the chunks to find patterns which are duplicated across

game boundaries. The induction algorithm found twenty-eight unique chunks, shown

in Figure 6, from the 272 potential chunks identified for Botvinnik in the

Hague-Moscow tournament. Each of the twenty-eight chunks induced was found in






79

at least two games. For example, the partial board representations for the three

games which produced one of the twenty-eight chunks are shown in Figure 20.


Botv inn ik

vs.

Reshevsky


Xeres

VS.

Botvinnik


Botv innik

vs.

Euue


occurrence in three different games.






80

The induction algorithm, with pseudo-code shown in Figure 21, verifies if a

potential or induced chunk already exists in the adversary pattern knowledge base

and updates the knowledge base appropriately. When a chunk is found to be

repeated in multiple games, we use the function REMOVE to eliminate multiple

instances of the chunk from the list of potential chunks produced from our

convolution algorithm. The induced chunks which do not exist in the current

knowledge base are then written into new frames, created by the function NEW, in

the knowledge base by the function WRITE.

The potential chunks collected by the convolution operator which are not

grouped into a valid chunk to be saved in the knowledge base are written to a

temporary file. This temporary file serves as a database of non-repeating chunks

which is used to identify chunk repetitions that occur outside of the current games

being induced. All chunks in this temporary database are used by the induction

algorithm to group chunks that have not been previously grouped. The database is

deleted after all games from a specific time period or tournament have been

analyzed. The deletion of the database forces the induced chunks to have a temporal

relevance, or recency, to the potential chunks being analyzed.

The textual move chunks are then analyzed to gather the statistics to be used

by the prediction stage of IAM. The statistics that are gathered include the mean

game turn that a move was executed, the number of times a particular move was

observed, the results of the games in which the move was observed, and the color of

the pieces being played. This statistical analysis requires that only one entry be made









For I = 1 to number of chunks found for the current games do
Begin
Flag_1 FALSE ;; indicates when a chunk has been repeated
Flag_2 ,- FALSE ;; indicates when a chunk is duplicated in the KB
For J = I+1 to number of chunks found for the current games do
;; First check this chunk against the chunks that were just found
If C, = Cj
Then Begin
Increment number of times C, has been observed
If Ci.start_time > Cj.starttime
Then Ci.starttime ,- Cj.start_time
If Ci.end time < Cj.endtime
Then Ci.endtime Cj.endtime
REMOVE(Cj)
Flag_l TRUE
End Begin
For J = 1 to number of chunks currently in the chunk Knowledge Base do
;; Next check this chunk against all chunks already in the Knowledge Base
If CI = Cj
Then Begin ;; Update KB knowledge
Increment number of times Cj has been observed
If C1.starttime < Cj.starttime
Then Cj.starttime 4- Ci.starttime
If Ci.endtime > Cj.end_time
Then Cj.endtime C- Ci.endtime
Flag_2 TRUE
End Begin
If Flag_l = TRUE AND Flag_2 = FALSE
Then Begin
NEW(geometric_frame, F)
F C,
WRITE(F, Geometric_Chunk_Knowledge_Base)
End Begin
End Begin
Figure 21: Pseudo-code to perform induction on geometric chunks.






82

in the knowledge base for each specific move. Additional information concerning the

total number of games analyzed, number of games played as white, number of games

won, and number of games won as white is stored with the textual move patterns.

The pawn structure chunks are also analyzed statistically at the same time as

the textual move patterns. The statistical analysis gathers knowledge that is used by

the adversary playing style inference engine. This knowledge is about the presence,

quantity, and quality of pawn structures and inter-locked pawn chains. The quality

of a pawn structure is determined by the number of pawns involved in the structure.

The knowledge from this analysis is stored with the textual moves in the adversary

patterns knowledge base.

Following the induction on geometric chunks and the statistical analysis of

pawn structures and rote learned textual chunks, the knowledge base of patterns

known by an adversary is available for use by the prediction stage of the adversary

model. The adversary pattern knowledge base is constructed incrementally. As new

games of an adversary become available the chunk knowledge from those games can

be quickly incorporated into the existing knowledge base.

4.1.2 Predicting Adversary Actions Stage

After an adversary patterns knowledge base has been constructed, the

adversary model then attempts to predict adversary actions in relevant domain

situations. Simulations of a game played by an adversary are used to verify the

prediction capabilities of the adversary modeling methodology. Each of the

simulated games used to represent a future contest against an adversary has never

been analyzed by IAM.






83
Prior to each of these simulated games, the preferred playing style of the

adversary is inferred from the chunks in the knowledge base. We had IAM study

twelve games of two different chess grandmasters who display preferences for the two

different playing styles currently inferred by IAM. The two players were Botvinnik,

who prefers closed positions, and Spassky, who prefers open positions. By analyzing

the presence and frequency of pawn structures and the frequency of pawn and

attacking moves, IAM correctly identified the playing style preference of each

adversary. Knowledge about the playing style of an adversary is then used by the

current domain program which the adversary model is coaching to select specific

moves that manipulate the game board away from the adversary's preferred style.

During the opening portion of the simulated game, IAM relies exclusively on

the textual move knowledge base which contains the statistical analysis of opening

patterns displayed by the adversary. Prior to the adversary's move, IAM predicts the

adversary's move from the moves currently in the knowledge base. Following the

actual move made by the adversary, the corresponding move in the knowledge base

is eliminated from further consideration. For an adversary playing the white pieces,

IAM has had accurate predictions ranging from forty percent to one hundred percent,

with a mean prediction accuracy of seventy-five percent.

IAM's opening sequence predictions are less accurate for an adversary playing

the black pieces. This is because the adversary is playing responsively to his

opponent's moves. Accuracy for opening move predictions against a black piece

playing adversary ranged from zero to twenty percent.






84

By modifying the textual chunk learning algorithm to remember the white

player's move which preceded an adversary's black move, we were able to

significantly increase IAM's opening sequence prediction accuracy to range between

forty and one hundred percent, with a mean of sixty percent. This modification

produces results which are similar to IAM's performance against the adversary while

playing the white pieces.

The number of textual chunks stored in the knowledge base increases

proportionately, since identical moves by the adversary which follow a different white

move are now considered as unique moves. The average increase in the knowledge

base size is forty-five percent. For example, after analyzing twenty-two games of

Botvinnik with the original textual chunking algorithm, thirty-one opening move

textual chunks were produced. With the response code modification forty-seven

opening move textual chunks were saved to the knowledge base. The actual textual

chunks are also larger to contain the adversary's opponent's move. Because of the

relatively small number of chunks in the knowledge base for a specific opponent, the

increase in prediction accuracy warrants the small increase in size to the knowledge

base.

All move predictions following the opening sequence are made primarily from

the contents of the geometric piece chunk knowledge base. Before each move to be

made by the adversary, the current board representation is searched to find pieces

belonging to the adversary. This is done because each of the chunks contained in the

knowledge base consists of pieces belonging to the adversary.






85
Once an adversary's piece has been located on the board representation, each

chunk in the knowledge base is compared against the board representation to find

identical matches and analogous chunks, which have one or two pieces out of place.

Because chunks occupy a specific square area of the game board, chunks which are

to large to fit into the current board location are ignored to reduce processing time.

The detection of analogous board positions is performed via pattern matching. When

an analogous board position has been identified, IAM then checks to see if a legal

chess move would complete the analogous chunk. An example of this process is

shown in Figure 22. Four of the chunks shown in Figure 6 were used to identify

analogous positions on the current game board which is prior to Botvinnik's

thirteenth move from the game displayed in Figure 19. The corresponding moves

that would produce the chunks are also displayed in the figure.

The collection of moves suggested by the analogous chunks is then processed

by an inference engine to heuristically determine the most probable move. The

inference engine predicted the castling move for the Figure 22 board position. This

prediction represents the result of applying the various psychologically based

heuristics concerning chunk size and relevance to the current domain. The queen

being en prise is domain specific information and is therefore not considered by the

move prediction inference engine. However, the actual move made by Botvinnik,

Qc2 or b3-c2, is one of the moves suggested by an analogous chunk and reviewed by

the inference engine. An experiment to augment the adversary pattern knowledge

base with a minimal amount of domain specific knowledge is presented in section 4.2.































Figure 22: Predictions from four chunks.


The process of searching the current board configuration to identify analogous

positions is repeated before each move to be made by the adversary. Existing chess

programs can use IAM's predictive capabilities by having IAM search the possible

board configuration at a specific node of the game tree and finding analogous chunk

situations in the proposed board configuration. IAM will then predict the most

probable adversary move that corresponds to our domain simplification hypothesis

of Chapter 3.

4.1.3 Integrating IAM's Knowledge into a Chess Program

Our induction based adversary modeling methodology produces knowledge

which can be incorporated into current chess programs by two methods. Knowledge






87
about the specific opening sequences preferred by an adversary can be used to

manually optimize a chess program's opening book. Optimization of the opening

book entails augmenting the opening book with opening sequences that are unknown

to the adversary and adjusting the preferences of the chess program to select these

new opening sequences.

Current chess programs use a move generator to limit the breadth of their

game trees. Move generators suggest a number of moves, typically five, which are

considered as the only moves available from a specific position. We describe the

reduction of tree complexity by the adversary modeling methodology in Chapter 5.

IAM predicts the move that an adversary will execute next from a particular board

position. These predictions can be used by chess programs to further limit the search

of their game trees.

A chess program would have the adversary model make predictions of the

next move in the game tree while simultaneously generating the five best moves with

its move generator. When predictions from the adversary model are available, the

chess program would use the predictions to expand its current game tree. Otherwise,

the chess program would continue to use the moves supplied by its move generator.

Combinations of adversary model predicted moves and move generator suggested

moves can be used by heuristically ordering the moves suggested by both methods

and expanding the game tree with the first five moves from the ordered move list.






88

4.2 IAM's Performance Results


In this section we describe several experiments that have been performed with

IAM to test the overall capabilities of the adversary modeling methodology. Each

experiment is described preceding the analysis of the results.

The games of the adversary Botvinnik are all taken from his World

Championship matches from the 1948 Hague-Moscow tournament through his 1961

match against Tal. The two games which are used to simulate Botvinnik's

performance in a future match, come from Botvinnik's 1963 match against Petrosian,

and are shown in Figure 23 and Figure 24. The game TEST1 is the first game from

the Petrosian match resulting in a win or loss in which Botvinnik played the white

pieces. TEST2 is the next game from the match, following TEST1, in which

Botvinnik played the black pieces.

4.2.1 General Performance Results

The purpose of our first experiment is to determine the base performance

results of IAM so that we can compare these results against other experimental

results. We anticipate that the predictive capabilities of IAM will increase as more

chunks are learned. The records of every game played by Botvinnik from his World

Championship Matches, prior to 1963, are used as input.

4.2.1.1 Performance versus the adversary as the white player

TEST1 is simulated for IAM following the construction of the adversary

model's knowledge base for each of the tournaments of Botvinnik's career. The






89

length in game turns of TEST1 is fifty-seven moves. The performance of IAM in this

experiment is shown in Table 1.


TEST1--Botvinnik playing White

1. d4 d5 21. Re2 Nb6 41. Nc5 Bf5
2. c4 e6 22. Rhel Nc4 42. Kg3 a4
3. Nc3 Be7 23. Bxc4 Rxc4 43. Kf4 a3
4. cxd5 exd5 24. Rd2 Re8 44. Ke5 Rb4
5. Bf4 c6 25. Re3 a6 45. Nd3 Rb5
6. e3 Bf5 26. b3 Rc6 46. Kd6 Kf7
7. g4 Be6 27. Na4 b6 47. Kc6 Bxd3
8. h3 Nf6 28. Nb2 a5 48. Rxd3 Rb2
9. Bd3 c5 29. Nd3 f6 49. Rxa3 Rg2
10. Nf3 Nc6 30. h4 Bf7 50. Kxd5 Rxg5+
11. Kfl 0-0 31. Rxe8+ Be6 51. Kc6 h5
12. Kg2 cxd4 32. Qe3 Bf7 52. d5 Rg2
13. Nxd4 Nxd4 33. g5 Be6 53. d6 Rc2+
14. exd4 Nd7 34. Nf4 Bf7 54. Kd7 h4
15. Qc2 Nf6 35. Nd3 Be6 55. f4 Rf2
16. f3 Rc8 36. gxf6 Qxf6 56. Kc8 Rxf4
17. Be5 Bd6 37. Qg5 Qxg5+ 57. Ra7+
18. Rael Bxe5 38. hxg5 a4
19. Rxe5 g6 39. bxa4 Rc4 1:0
20. Qf2 Nd7 40. a5 bxa5


Figure 23: A game from


the 1963 Botvinnik versus Petrosian match.


The first column, #G, is the number of games currently analyzed to produce

the adversary pattern knowledge base. The #Ch and #M columns are the number

of geometric chunks and textual opening move chunks respectively. The first number

in the #M column represents the total number of chunks and the second number is

the chunks pertaining to white moves, for TEST1, or black moves, for TEST2. The

next three columns are the number of predictions made by IAM (#P), the number

of predictions which exactly matched the adversary's ensuing move (#C), and the