• TABLE OF CONTENTS
HIDE
 Title Page
 Dedication
 Acknowledgement
 Table of Contents
 Abstract
 Introduction
 Sensor data fusion: Managing...
 Intelligent fully autonomous mobile...
 The proposed hybrid control...
 Experimental setup and impleme...
 Experimental results
 Summary and conclusions
 Introduction to clips
 Reference
 Biographical sketch
 Copyright






Intelligent autonomous systems
CITATION SEARCH THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00082166/00001
 Material Information
Title: Intelligent autonomous systems controlling reactive behaviors with consistent world modeling and reasoning
Physical Description: Book
Creator: Bou-Ghannam, Akram A.
Publisher: Akram A. Bou-Ghannam
Publication Date: 1991
 Record Information
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
Resource Identifier: oclc - 25222039
alephbibnum - 001693355
System ID: UF00082166:00001

Table of Contents
    Title Page
        Page i
        Page ii
    Dedication
        Page iii
    Acknowledgement
        Page iv
    Table of Contents
        Page v
        Page vi
    Abstract
        Page vii
        Page viii
        Page ix
    Introduction
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
        Page 8
        Page 9
        Page 10
        Page 11
    Sensor data fusion: Managing uncertainty
        Page 12
        Page 13
        Page 14
        Page 15
        Page 16
        Page 17
        Page 18
        Page 19
        Page 20
        Page 21
        Page 22
        Page 23
        Page 24
        Page 25
        Page 26
        Page 27
        Page 28
        Page 29
        Page 30
        Page 31
        Page 32
        Page 33
        Page 34
        Page 35
        Page 36
        Page 37
        Page 38
        Page 39
    Intelligent fully autonomous mobile robots
        Page 40
        Page 41
        Page 42
        Page 43
        Page 44
        Page 45
        Page 46
        Page 47
        Page 48
        Page 49
        Page 50
        Page 51
        Page 52
        Page 53
        Page 54
        Page 55
        Page 56
        Page 57
        Page 58
        Page 59
        Page 60
        Page 61
        Page 62
        Page 63
        Page 64
        Page 65
        Page 66
        Page 67
    The proposed hybrid control architecture
        Page 68
        Page 69
        Page 70
        Page 71
        Page 72
        Page 73
        Page 74
        Page 75
        Page 76
        Page 77
        Page 78
        Page 79
        Page 80
        Page 81
        Page 82
        Page 83
        Page 84
        Page 85
        Page 86
        Page 87
        Page 88
        Page 89
        Page 90
        Page 91
        Page 92
    Experimental setup and implementation
        Page 93
        Page 94
        Page 95
        Page 96
        Page 97
        Page 98
        Page 99
        Page 100
        Page 101
        Page 102
        Page 103
        Page 104
        Page 105
        Page 106
        Page 107
        Page 108
        Page 109
        Page 110
        Page 111
        Page 112
        Page 113
        Page 114
        Page 115
        Page 116
        Page 117
        Page 118
        Page 119
    Experimental results
        Page 120
        Page 121
        Page 122
        Page 123
        Page 124
        Page 125
        Page 126
        Page 127
        Page 128
        Page 129
        Page 130
        Page 131
    Summary and conclusions
        Page 132
        Page 133
        Page 134
        Page 135
        Page 136
        Page 137
        Page 138
    Introduction to clips
        Page 139
        Page 140
        Page 141
        Page 142
        Page 143
        Page 144
    Reference
        Page 145
        Page 146
        Page 147
        Page 148
        Page 149
        Page 150
        Page 151
        Page 152
    Biographical sketch
        Page 153
        Page 154
        Page 155
        Page 156
    Copyright
        Copyright
Full Text











INTELLIGENT AUTONOMOUS SYSTEMS:
CONTROLLING REACTIVE BEHAVIORS WITH
CONSISTENT WORLD MODELING AND REASONING
















By

AKRAM A. BOU-GHANNAM


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY


UNIVERSITY OF FLORIDA


1991







































(C) Copyright 1991 by Akram A. Bou-Ghannam. All rights
reserved.



































To my daughter Stephanie

Through her eyes I see peace on earth,
and receive my motivation.
















ACKNOWLEDGEMENTS

It takes lots of special people to make a dissertation.

First and foremost, I would like to thank my wife Nada and

my daughter Stephanie, for giving me lots of love and moral

support. I am also very grateful to the IBM Corporation for

the financial support and wonderful treatment. My IBM

managers and colleagues at the Boca Raton site were

extremely helpful and understanding. I could always count on

their support whenever I needed parts, equipment, technical

material, or attended technical conferences. Special thanks

go to the following IBMers: Rit VanDuren, Jerry Merckel, Tom

Kludt, Barabra Britt, Vic Moore, Rick Mendelson, Steve

Henderson, and Sharon Amato.

Within the university, I would like to thank my advisor

and all my committee members responsible for the

intellectual stimulation that shaped this dissertation.

Special thanks go to Dr. Carl Crane for sponsoring and

encouraging the cooperation between the robotics groups in

electrical, mechanical, and nuclear engineering. This

fruitful cooperation allowed me to put my ideas to practice

and run live experiments. Guido Reuter worked closely with

me on the "map builder" implementation and deserves much

thanks. Also, thanks go to Tom Heywood for his help in the

system setup and robot control.









TABLE OF CONTENTS


ACKNOWLEDGEMENTS ......................................... iv

ABSTRACT ................................................. vii

CHAPTER 1: INTRODUCTION .................................. 1
1.1 Philosophical Underpinnings and Overview .............7
1.2 Contributions of the Research ........................11

CHAPTER 2: SENSOR DATA FUSION: MANAGING UNCERTAINTY .....12
2.1 Motivation ...........................................12
2.2 Sensors and the Sensing Process ......................14
2.3 Classification of Sensor Data ........................15
2.4 Levels of Abstraction of Sensor Data ................. 18
2.5 Sensor Data Fusion Techniques ........................20
2.5.1 Correlation or Consistency Checking ...........22
2.5.2 Fusion at Lower Levels of Abstraction ........ 24
2.5.3 Fusion at Middle and High Levels of
Abstraction .........................................27
2.5.3.1 Bayesian Probability Theory ........... 28
2.5.3.2 Certainty Theory ........................29
2.5.3.3 Fuzzy Set Theory ........................31
2.5.3.4 Belief Theory ...........................32
2.5.3.5 Nonmonotonic Reasoning ................. 35
2.5.3.6 Theory of endorsements ................. 36
2.6 Implementation Examples ..............................37

CHAPTER 3: INTELLIGENT FULLY AUTONOMOUS MOBILE ROBOTS .... 40
3.1 Behavior-Based Approaches to Robot Autonomy ..........43
3.1.1 Lessons from animal behavior .................. 43
3.1.2 Current behavior-based approaches to robot
autonomy ............................................48
3.1.3 Limitations of the subsumption architecture ...50
3.2 Control Architecture: Behavior-Based vs.
Traditional Control ...................................... 52
3.3 Issues in World Model Construction ................... 55
3.3.1 Position Referencing for a Mobile Robot .......56
3.3.2 World Model Representation .................... 57
3.3.3 Managing World Model Inconsistencies ..........59
3.4 Direction of Proposed Research .......................62

CHAPTER 4: THE PROPOSED HYBRID CONTROL ARCHITECTURE ...... 68
4.1 The Planning Module: A Knowledge-Based Approach ......72
4.2 Lower-Level Behaviors ................................75
4.2.1 Avoid Obstacles Behavior ......................78
4.2.2 Random Wander Behavior ........................79
4.2.3 Boundary Following Behavior ................... 80
4.2.4 Target-Nav Behavior ........................... 81
4.3 Arbitration Network ..................................82
4.4 Map Builder: A Distributed Knowledge-Based
Approach ................................................. 87











CHAPTER 5: EXPERIMENTAL SETUP AND IMPLEMENTATION .........93
5.1 Implementation of the Planning Module ................97
5.2 Implementation of the Map Builder .................... 00
5.2.1 The EOU Knowledge Source ......................102
5.2.2 The Filter-Raw Knowledge Source ...............107
5.2.3 The 2-D Line Finding Knowledge Source .........107
5.2.4 The Consistency Knowledge Sources .............112
5.2.4.1 The Match/Merge Knowledge Source .......113
5.2.4.2 The Re-reference Knowledge Source ......117

CHAPTER 6: EXPERIMENTAL RESULTS ..........................120
6.1 Results From A Live Experimental Run ................. 120
6.2 Discussion of Results ................................128

CHAPTER 7: SUMMARY AND CONCLUSIONS .......................132

APPENDIX: INTRODUCTION TO CLIPS ..........................139

REFERENCES ..................................................145

BIOGRAPHICAL SKETCH ...................................... 153










Abstract of Dissertation Presented to the Graduate School of
the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

INTELLIGENT AUTONOMOUS SYSTEMS:
CONTROLLING REACTIVE BEHAVIORS WITH
CONSISTENT WORLD MODELING AND REASONING

By

Akram Bou-Ghannam

August 1991

Chairman: Dr. Keith L. Doty
Major Department: Electrical Engineering



Based on the philosophical view of reflexive behaviors

and cognitive modules working in a complementary fashion,

this research proposes a hybrid decomposition of the control

architecture for an intelligent, fully autonomous mobile

robot. This architecture follows a parallel distributed

decomposition and supports a hierarchy of control with

lower-level reflexive type behaviors working in parallel

with higher-level planning and map building modules. The

behavior-based component of the system provides the basic

instinctive competence for the robot while the cognitive

part is characterized by knowledge representations and a

reasoning mechanism which performs higher machine

intelligence functions such as planning. The interface

between the two components utilizes motivated behaviors

implemented as part of the behavior-based system. A

motivated behavior is one whose response is dictated mainly

by the internal state (or the motivation state) of the










robot. Thus, the cognitive planning activity can execute its

plans by merely setting the motivation state of the robot

and letting the behavior-based subsystem worry about the

details of plan execution. The goal of such a hybrid

architecture is to gain the real-time performance of a

behavior-based system without losing the effectiveness of a

general purpose world model and planner. We view world

models as essential to intelligent interaction with the

environment, providing a "bigger picture" for the robot when

reactive behaviors encounter difficulty.

Another contribution of this research is the Sensory

Knowledge Integrator proposed as the underlying model for

the map builder. This proposed framework follows a

distributed knowledge-based approach to the fusion of sensor

data from the various sensors of a multi-sensor system in

order to provide a consistent interpretation of the

environment being observed. Within the various distributed

knowledge sources of the Sensory Knowledge Integrator, we

tackle the problems of building and maintaining a consistent

model of the world and robot position referencing.

We describe a live experimental run of our robot under

hybrid control in an unknown and unstructured lab

environment. This experiment demonstrated the validity of

the proposed hybrid control architecture and the Sensory

Knowledge Integrator for the task of mapping the

environment. Results of the emergent robot behavior and










different map representations of the environment are

presented and discussed.














CHAPTER 1
INTRODUCTION




Organisms live in a dynamic environment and tailor

their actions based on their internal state and on the

perceived state of the external environment. This

interaction with the environment becomes more complex as one

ascends the ladder of hierarchy of organisms, starting with

the simplest ones that follow a stimulus-response type of

interaction where actions are a direct response to the

sensory information, and ending with humans that are endowed

with intelligence. Intelligence enables humans to reason

with symbols, to make models of the world and to make plans

to favorably alter the relationship between themselves and

the environment. Given the ability to reason does not mean

that people are devoid of the primitive instinctive type of

behaviors. As a matter of fact, reflexive responses account

for most of what people do when walking, eating, talking,

etc.... Less than half the brain is dedicated to higher-

level thinking [Albus 81].

Relative to the development of a machine (robot) which

exhibits various degrees of autonomous behavior, which of

the following two approaches is most appropriate: 1) Should

one design machines to mimic human intelligence with

symbolic reasoning and symbolic models of the world?, 2)










Should one design machines that mimic "insect intelligence"

with no central brain and symbolic models of the world? This

deep difference in philosophy currently divides the

artificial intelligence community into two camps: 1) The

"traditionalists," constituting the majority of researchers

who have long assumed that robots, just like humans, should

have models of their world and should reason about the next

action based on the models and current sensor data [Ayache

87] [Crowley 85] [Giralt 84b] [Kriegman 89] [Shafer 86]. 2)

The behavior-based camp of researchers [Brooks 86a] [Connell

89] [Payton 86] [Anderson 88] [Agre 90] who avoid symbolic

representations and reasoning, and advocate the endowment of

a robot with a set of low-level behaviors that react to the

immediacy of sensory information in a noncognitive manner.

The main idea is that "the world is its own best model", and

complex behaviors emerge from the interaction of the simple

low-level behaviors as they respond to the various stimuli

provided by the real world. The number of researchers in

this camp is small but growing rapidly. In the next

paragraphs we discuss the characteristics of each approach,

starting with the behavior-based one.

Ethological observations of animal behavior [Gould 82]

[Manning 79][McFarland 87] provide the inspirational basis

for the behavior-based approach in robotics. The observation

is that animals use instinctive behaviors rather than

"reasoning" to survive in their ever-changing environment.

Apparently, their actions are the resultant of various










reactions to external stimuli. These reactions are

apparently due to animal instinct and are not a consequence

of sophisticated processing or reasoning. Braitemberg

[Braitemberg 84] elaborates on the observation that animal

behavior could be an outcome of simple primitive behaviors

and such behavior could be reproduced in mobile robots whose

motors are driven directly by the output of the appropriate

sensors. Rodney Brooks' introduction of the subsumption

architecture [Brooks 86a] has given the behavior-based

approach a great push forward and has forced researchers in

the robotics community to reexamine their methods and basic

philosophy of robot control architecture. For example,

researchers have always taken for granted that a robot needs

to model its environment. Now, alerted by the main thesis of

the behavior-based approach of no global internal model and

no global planning activity, they ask questions of why and

for what specific tasks does one need to model the

environment. Brooks' subsumption architecture uses a menu of

primitive behaviors such as avoid obstacles, wander, track

prey, etc., each acting as an individual intelligence that

competes for control of the robot. There is no central brain

that chooses and combines these simple behaviors, instead,

the robot sensors and what they detect at that particular

moment determine the winning behavior. All other behaviors

at that point are temporarily subsumed. Surprisingly, the

conduct of Brooks' brainless "insect" robots often seems

clever. The simple responses end up working together in










surprisingly complex ways. These "insects" never consult a

map or make plans; instead, their action is a direct

response to sensory information. The payoff for eliminating

symbolic models of the environment and the central planner

is speed. Real time operation becomes possible since the

computational burden is distributed and greatly reduced.

Another advantage of the subsumption architecture is its

modularity and flexibility. In principle, more behaviors may

easily be added until the desired level of competence is

reached. A drawback of the behavior-based approach is that

one cannot simply tell the various behaviors how to achieve

a goal. Instead, in an environment which has the expected

properties, one must find an interaction loop between the

system and that environment which will converge towards the

desired goal [Maes 90]. Thus, the designer of a behavior-

based system has to "pre-wire" the arbitration strategy or

the priorities of the various behaviors. This inflexibility,

coupled with the inability to handle explicitly specified

goals, makes it hard for such behavior-based systems to be

useful for different types of missions over a wide range of

domains. Additionally, an efficient behavior to assure

reaching a specified goal cannot always be guaranteed. So,

it is possible for a robot using the behavior-based approach

to take a certain pathway many times over, even though

traversing this pathway might not be desirable for many

reasons. For example, the path might lead the robot away

from the target or into danger. This is possible because the










robot does not build or have a map of its environment. In

effect, the robot does not remember what it has seen or

where it has been. Anderson and Donath [Anderson 88]

describe some cyclical behavior exhibited by a reflexive

behavior-based robot and attribute such behavior to the lack

of internal state within each behavior. They also report

that this cyclic behavior was observed by Culberston

[Culberston 63]. Brooks and Connell [Brooks 86b] have also

observed cyclical behavior in their wandering and wall

following behaviors. To avoid such problems, later work by

Mataric [Mataric 89] a member of Brooks' group experimented

with map building and use under the subsumption

architecture.

The traditional approach to robot control architecture

is derived from the standard AI model of human cognition

proposed by Newell and Simon in the mid-fifties. It follows

the Deliberative Thinking paradigm where intelligent tasks

can be implemented by a reasoning process operating on a

symbolic internal model. Thus, it emphasizes cognition or a

central planner with a model or map of the environment as

essential to robot intelligence. Sensory confirmation of

that model is equally important. Such symbolic systems

demand from the sensor systems complete descriptions of the

world in symbolic form. Action, in this case, is not a

direct result of sensor data but rather is the outcome of a

series of stages of sensing, modelling, and then planning. A

desirable feature of such systems is the general ability to










handle explicit high-level user specific goals. Given a set

of goals and constraints, the planning module advances the

overall mission by deciding the robot's next move based on

an analysis of the local model of the environment

(constructed from current sensor data) and the existing

global model. The global or world model is obtained either

directly from the user, if the robot is operating in a known

environment, or it is autonomously constructed over time

from the various local models when operating in an unknown

environment. The world model representation employed in the

traditional approach is general purpose and, thus, useful

for a variety of situations and planning tasks. Without such

a general purpose model, features critical to plan execution

may not be discovered. But, a general purpose world model

puts some unrealistic demands on the perception task and has

the disadvantage of an unavoidable delay in the sensor to

actuator loop. Such delay is due to the computational

bottleneck caused by cognition and the generation of a

symbolic model of the world. Lack of real time operation

(speed) and inflexibility are the two major complaints from

the behavior-based camp about the cognitive approach. These

claims are supported by the fact that the few autonomous

mobile robot projects implemented with the traditional

approach suffer from slow response times and inflexibility

when operating in complex dynamic environments. The response

of the traditionalists is that while reflexive behavior can

keep a robot from crashing into a wall, a higher-level










intelligence is needed to decide whether to turn left or

right when the robot comes to an intersection.



1.1 Philosophical Underpinnings and Overview

Our goal is to develop a general purpose robot that is

useful for a variety of tasks (explicitly stated by a user)

in various types of dynamically changing environments. The

philosophical view of our research is that such a goal could

only be accomplished by combining the two approaches

mentioned above, and that these two approaches complement

each other just as reflexive responses and higher-level

thought complement each other in human beings. For example,

while one does not think about how to articulate the joints

in one's legs when walking down a sidewalk (the reflexive

behaviors take care of the walking function), higher-level

thinking and planning is needed when one, for example,

remembers that the sidewalk is not passable further down due

to construction noticed earlier. At this moment one has to

plan a suitable alternative route to the destination.

Adhering to this philosophical view of reflexive behaviors

and cognitive modules working in a complementary fashion

where the advantages of one approach compensates for the

limitations of the other, this research proposes a hybrid

decomposition of the control architecture for an intelligent

fully autonomous mobile robot. This architecture follows a

parallel distributed decomposition and supports a hierarchy

of control with lower-level reflexive type behaviors working










in parallel with higher-level planning and map building

modules. Thus, our architecture includes a cognitive

component and a behavior-based component. The cognitive

component is characterized by knowledge representations and

a reasoning mechanism which performs higher mental functions

such as planning. The behavior-based component system hosts

the cognitive component and provides the basic instinctive

competence for the robot. The level of competence of the

behavior-based component determines the degree of complexity

of the planner in the cognitive component. Thus, the higher

the competence level of the behavior-based system, the

simpler the planning activity. Once the behavior-based

system is built to the desired level of competence, it can

then host the cognitive part. The interface between the two

components utilizes motivated behaviors implemented as part

of the behavior-based system. We define a motivated behavior

as one whose response is driven mainly by the associated

'motivation' or internal state of the robot. This is

analogous to motivated behavior exhibited by animals. For

example, the motivated behavior of feeding depends on the

internal motivation state of hunger in addition to the

presence of the external stimulus of food. Utilizing

motivated behaviors, the cognitive planning activity can

thus execute its plans by merely setting the motivation

state of the robot and letting the behavior-based subsystem

worry about the details of plan execution. In our approach,

the arbitration of the responses of lower-level behaviors is










partly hardwired in the behavior-based system, and partly

incorporated into a set of production rules as part of the

planning module of the cognitive system. These rules are

driven by the goals of the robot, and the current situation

facts provided by the world model and the status of the

behavior-based system. In addition, in the behavior-based

system, we use superposition in a potential force field

formulation (similar to [Arkin 87] and [Khatib 85]) to

combine the responses of the various complementary behaviors

that are active at any one time. The goal for the hybrid

architecture is to gain the real-time performance of a

behavior-based system without loosing the general goal

handling capability of a general purpose world model and

planner. We view world models as essential to intelligent

interaction with the environment, providing a "bigger

picture" for the robot when reflexive behaviors encounter

difficulty.

In our framework we tackle the behavior fusion problem

with the lower-level behaviors, while higher-level modules

such as the map builder tackle the sensor fusion problem in

attempting to build a general purpose representation.

Theoretical concepts and mathematical tools for sensor data

fusion are presented in chapter 2; issues in designing

intelligent fully autonomous mobile robots are presented in

chapter 3, while details of the workings of our proposed

architecture are explained in chapter 4. This architecture

is implemented and tested in a dynamic, unknown, and










unstructured environment in our lab for controlling a K2A

Cybermotion mobile robot. Chapter 5 covers the experimental

setup and implementation issues, while chapter 6 presents

and discusses the results obtained. We share the belief that

a world representation and sensory confirmation of that

representation are essential to the intelligence of an

autonomous mobile robot. Thus, the map builder is an

important part of the hybrid control architecture. We

propose, also in chapter 4, a distributed knowledge-based

framework called the Sensory Knowledge Integrator (SKI) as

the underlying model for the map builder. The SKI framework

organizes the domain knowledge needed to describe the

environment being observed into data-driven and model-driven

knowledge sources, and provides a strategy for applying that

knowledge. The theoretical concepts of the SKI model are

presented in section 4.3, while the implementation of the

map builder is discussed in chapter 5. The results of

implementing the various knowledge sources of the map

builder are also presented in chapter 5. These results show

two types of representations: an occupancy grid

representation and a 2-D line representation generated from

sonar sensor data. Results of position correction or re-

referencing of the robot are also presented. Chapter 7

concludes this dissertation and discusses limitations and

future research trends.









1.2 Contributions of the Research

We see the contributions of this work as:

1. The development and implementation of a hybrid control

architecture that combines both traditional and behavior-

based approaches.

2. The development and implementation of the Sensory

Knowledge Integrator framework which provides a parallel

distributed model for sensor data fusion and consistent

world modeling.

3. The development and implementation of a new approach for

consistent world modeling. This approach involves the

interactive use of the occupancy grid and the 2-D line

representations for filtering out unsupported raw input data

points to the line-finder knowledge source and thus

providing a better 2-D line representation.

4. A fast algorithm for position referencing of a mobile

platform using the 2-D line representation.

5. Addressing the important question of combining both the

behavior-based and the traditional approach and whether it

provides better performance.














CHAPTER 2
SENSOR DATA FUSION: MANAGING UNCERTAINTY




In this chapter we concentrate on sensor fusion, an

essential issue of the traditional approach (discussed in

chapter 1) concerned with generating meaningful and

consistent interpretations (in terms of symbolic models) of

the environment being observed. We discuss sensor data

fusion techniques and methods for managing uncertainty at

various levels of abstraction, and illustrate the advantages

and disadvantages of each. Some of the techniques presented

in this chapter are used in the various knowledge sources of

the map builder module, an essential part of our hybrid

architecture. The implementation of the map builder

(including the sensor fusion techniques used) is presented

in detail in chapter 5.



2.1 Motivation

The increasing demand for robot systems to perform

complex tasks such as autonomous operation and flexible

manufacturing has spurred research aimed at upgrading robot

intelligence and capabilities. These complex tasks are often

associated with unstructured environments where information

is often uncertain, incomplete, or in error. This problem

calls for the development of a robot system capable of using










many different sources of sensory information in order to

overcome the limitations of single sensory robot systems.

Single sensory robot systems are limited in their ability to

resolve and interpret unknown environments, since they are

only capable of supplying partial information. The need for

multi-sensor robot systems is evident in the literature:

[Giralt 84a], [Durrant-Whyte 86a], [Henderson 84],

[Ruokangas 86], [Flynn 88], [Luo 88], [Mitiche 86], [Shafer

86]. The motivation is to obtain from a set of several

different and/or similar sensors, information that would be

impossible or impractical to obtain from any one of the

sensors alone. This is often possible since different

sensors are sensitive to different properties of the

environment. Thus, each sensor type offers unique attributes

and contextual information in interpreting the environment.

The goal of a multi-sensor system is to combine information

from the various sensors, with a priori knowledge about the

environment, the sensors, the task, etc., into a meaningful

and consistent interpretation of the environment. In this

manner, the system maintains an internal description of the

world which represents its "best guess" about the external

world.

Sensor data fusion combines information from various

sensors into one representative set of data that provides a

more accurate description of the observed environment (an

improved world model) than the description provided by any

of the sensors acting alone. The objective is to reduce










uncertainty about the observed environment. However, in

addition to the fusion of information from multiple sensory

sources, the problem of generating an accurate world model

representation involves the fusion of sensory data with

object models and a priori knowledge about the environment.



2.2 Sensors and the Sensing Process

The field of robotic sensor design is rapidly growing

and undergoing a great deal of research activity. A variety

of sensors are available for robotic applications. These

include TV cameras, infrared cameras, ranging devices such

as acoustic, infrared, and laser range finders, touch

sensors, proximity sensors, force/torque sensors,

temperature sensor, etc. An assessment of robotic sensors is

presented in [Nitzan 81]. Nitzan defines sensing as "the

translation of relevant physical properties of surface and

volume elements into the information required for a given

application." [Nitzan 81, p. 2]. Thus, physical properties

such as optical, mechanical, electrical, magnetic, and

temperature properties are translated by the sensing process

into the information required for the specific application.

For example, a parts inspection application might require

information about dimensions, weights, defects labeling,

etc.. The basic steps of sensing are shown in the block

diagram of figure 1.1 (from [Nitzan 81]).












PHYSIC Tandducig Preprocessi I IMPROVED Interpreing REQUIRED Application
PROPERTIES SIGNAL SIGNAL INFORMATION I

0 PmcQ ---- Now
I I Sen ing



Figure 2.1 Block diagram of sensing steps.
From [Nitzan 81].



In this chapter we are concerned with the sensing

process where information from a variety of sensors is

combined and analyzed to form a consistent interpretation of

the observed environment. As we will discuss later, the

interpretation process is complex and involves processing of

sensor data at various levels of abstraction using domain

specific knowledge.



2.3 Classification of Sensor Data

The fusion technique of choice depends on the level of

abstraction and on the classification of the sensor data. In

multi-sensor systems, data from the various sensors are

dynamically exchanged. The use of these data in the fusion

or integration process falls under one of the following

classes:



Competitive. In this case the sensors' information is

redundant. This occurs when the observations of the

sensors) intersect; that is, they supply the same type of










information about the same feature or property of the same.

object. The following sensor configurations and scenarios

produce competitive information interaction:

a) Two or more sensors of the same type measuring the

value of the same feature of an object. For example, two

sonar sensors measuring the depth of an object from a fixed

frame of reference.

b) Different sensors measuring the value of a specific

feature. For example, depth information could also be

provided using stereo vision as well as a sonar range

finder. Another example of how different sensing modalities

produce the same feature, is the generation of edge features

of an object from either intensity images or range images.

c) A single sensor measuring the same feature at

different times. For example, a sonar sensor continuously

acquiring depth measurements from the same position and

viewpoint.

d) A single sensor measuring the same feature from a

different viewpoint or operating configuration or state.

e) Sensors measuring different features but, when

transformed to a common description, the information becomes

competitive. For example, the speed of a mobile robot could

be measured by using a shaft encoder, or it could be deduced

from dead-reckoning information from fixed external beacons.



Complementary. In this case the observations of the

sensors are not overlapping, i.e., the sensors supply











different information about the same or different feature.

The measurements are added (set union) to the total

environment description without concern for conflict. A good

example of complementary sensor data fusion is Flynn's

combining of sonar and IR sensor data [Flynn 88]. The sonar

can measure the distance to an object but has poor angular

resolution, while the IR sensor has good angular resolution

but is not able to measure the distance accurately. By using

both sensors to scan a room, and combining their information

in a complementary manner where the advantages of one sensor

compensates for the disadvantages of the other, the robot

was able to build a better map of the room.



Cooperative. This occurs when one sensor's information

is used to guide the search for another's new observations.

In other words, one sensor relies on another for information

prior to observations. For example, the guiding of a tactile

sensor by initial visual inspection [Allen 88].



Independent. In this case one sensor or another is used

independently for a particular task. Here fusion is not

performed, but the system as a whole employs more than one

sensor for a particular task and uses one particular sensor

type at a time while the others are completely ignored, even

though they may be functional. For example, in an

environment where the lighting conditions are very poor, a

mobile robot may depend solely on a sonar sensor for










obstacle avoidance, while in good lighting conditions both

vision and sonar sensing modalities could be employed.

Independent sensor data interaction occurs in the natural

world where redundant sensors are abundant. For example,

pigeons have more than four independent orientation sensing

systems, that do not seem to be combined but rather,

depending on the environmental conditions, the data from one

sensory subsystem tends to dominate [Kriethen 1983].



2.4 Levels of Abstraction of Sensor Data

Levels of abstraction of sensor data are application

and task dependent. These levels of abstraction vary from

the signal level (lowest level) where the raw response of

the sensor is present, to the symbolic level (highest level)

where symbolic descriptions of the environment exist for use

by the planning subsystem. This model is based upon

psychological theories of human perceptual system that

suggest a collection of processors that are hierarchically

structured and modular [Fodor 83]. These processors create a

series of successively more abstract representations of the

world ranging from the low-level transducer outputs to the

highly abstract representations available to the cognitive

system. Thus, in order to bridge the wide gap between raw

sensory data and understanding of what those data mean, a

variety of intermediate representations are used. These

representations make various kinds of knowledge explicit and

expose various kinds of constraints [Winston 84]. For










complicated problem domains such as the problem of

understanding and interpreting a robot's environment based

mainly on data from its sensors, it becomes important to be

able to work on small pieces of the problem separately, and

then combine the partial solutions at the end into a

complete problem solution. This task of understanding the

environment is thus accomplished at various levels of

analysis or abstraction. Thus, sensor data exist at the

various levels of knowledge abstraction in the solution

space and appropriate fusion techniques are applied at the

various levels for an improved solution. Though the specific

abstraction levels are task dependent, they could be

generalized as follows:



Signal level. This level contains data that are close

to the signal or unprocessed physical data level. At this

level data are usually contaminated with random noise and

are generally probabilistic in nature. Therefore statistical

inferencing techniques are appropriate for data fusion on

this level.



Feature level. Data at this level consist of

environmental/object features derived from the signal. The

various features describe the object or solution. Often,

incomplete descriptions of the features must be used. This

calls for a type of reasoning that is subjective and based










on the body of evidence associated with each feature. This

type of reasoning is called evidential reasoning.



Symbol level. At this level symbolic descriptions of

the environment exist and propositions about the environment

are either true or false. Therefore, logical (Boolean)

reasoning about these descriptions is most appropriate.



2.5 Sensor Data Fusion Techniques

A variety of techniques for combining sensor data have

been proposed. Most approaches concentrate on Bayesian or

statistical combination techniques [Richardson 88], [Luo

88], [Porrill 88], [Ayache 88], [Durrant-Whyte 86a]. Some

researchers followed a heuristic approach [Flynn 88], [Allen

88]. Garvey et al, [Garvey 82] proposed evidential reasoning

as a combination technique and claims it is more general

than either Bayesian or Boolean approaches. The fusion

technique of choice depends on the classification of the

sensor data involved, and on the level of abstraction of the

data. For example, at the signal level, data is generally

probabilistic in nature and hence a probabilistic approach

is most appropriate. At higher levels such as the symbolic

feature level, a Boolean approach is generally appropriate.

Sensor data classification and levels of abstraction have

been discussed in the previous sections. For complementary

data the sensors' data are not overlapping and there is no

concern for conflict. In this case, at any level of










abstraction, the technique of choice is to simply add the

sensor descriptions to the total environment description.

Similarly, for cooperative sensor data there is no concern

for conflict since the data of one sensor guides the

observations of the other. In the case of competitive sensor

data when two or more sensors provide information about the

same property value of the same object, a fusion technique

is called for. But, before choosing a fusion technique to

combine the data, how does one determine if the data is

truly competitive? That is, how does one determine if the

data represent the same physical entity? This correlation or

consistency check is discussed in the next section. The

choice of fusion technique in the competitive case depends

on the level of abstraction of the data. For example, at the

raw data level the problem becomes that of estimating a

parameter x from the observations of the sensors involved.

This could be resolved by either using a deterministic or

nonrandom approach (like the least-squares method for

example), or by using a probabilistic or random approach

(like the minimum mean squared error method). Dealing with

uncertain information is still a problem at higher levels of

abstraction and a variety of methods have been proposed. In

the following sections we will discuss fusion techniques in

more detail.










2.5.1 Correlation or Consistency Checking

To determine whether sensor data, or features derived

from that data could be classified as competitive, a

consistency check is performed on the data. This is a common

difficulty in robotic perception where often the correlation

between perceived or observed data and model data has to be

determined. The well known problem of correlating between

what the robot actually sees and what it expects to see is

an appropriate example. Following is an example that

illustrates one form of consistency checking:

Let pei and pej be parametric primitive vectors

estimated by sensors i, and j respectively. We desire to

test the competitive hypothesis Ho that these estimates are
for the same primitive p of an object. Let 8,ei = pei pei be

the estimate of 8iJ = pi pi where pi and pi are the

corresponding true parameter vectors, then Ho: 5ij = 0. We

want to test Ho vs. Hi: 6ij <> 0. Let the corresponding

estimation errors be represented by ei = p Pei, and eJ = p
- pei. Define eij = 8ij eij, it follows that eij = pi pi -

Pei + Pej. Then, under Ho we have


eij /Ho = P pei (p Pej) = ei ei,



and the covariance of the error under Ho is given by



C /Ho = E[ (eiJ /Ho) (eiJ /Ho)T]

= E[ (ei ei) (ei eJ)T]










= E[ (ei) (ei)T] E[ (ei) (e)T] -

E[ (ei) (ei) T] + E [ (eJ) (ei) T]

= Ci Cii Cii + Ci.



If the errors ei and ei are independent then Cij = Cii = 0,

and

C /Ho = Ci + CJ.



For Gaussian estimation errors the test of Ho vs. H1 is

as follows: Accept Ho if


d = (BeiJ)T C-1 (8eiJ) 9,8

where 0 is a threshold such that: p{ d>0 /Ho} = a, and a is

a small number such as 0.05 for example.



If Ho is accepted, then pei and pej are competitive and

can thus be fused to obtain Peij the combined estimate of p.

Using the standard Kalman filter equations, and letting the

prior mean of p be Pei, we obtain the following combined

estimate and corresponding error covariance:



peiJ = Pei + K(pel Pei)

Cov = Ci K Ci,

where K is the Kalman filter gain, and for independent

errors ei and ei is given by


K = Ci(Ci + CJ)-1












2.5.2 Fusion at Lower Levels of Abstraction

At the lower levels of abstraction where the sensor

data is close to the signal level, the data contains signal

components random in nature, and hence probabilistic

reasoning or inference is most appropriate. Most of the

literature on sensor data fusion involves problems at this

level with the majority of the researchers adopting

probabilistic (Bayesian) inference. Next we will briefly

discuss the commonly used assumption, advantages, and

disadvantages of statistical sensor fusion techniques.

Most researchers treat the sensor fusion problem, at

this level of abstraction, as an optimal estimation problem.

This problem is briefly stated as follows: Given a system of

interest (e.g. the environment of a robot) represented by an

n-dimensional state vector x, and the measured quantities

(output of sensor systems) represented by the m-dimensional

measurement vector z, what is the best way to estimate the

value of x given z according to some specified optimality

criterion. A general measurement model is given by



z = h(x, v)



where h(x, v) is an m-dimensional vector which represents

the ideal operation of the sensor system, and v represents

the m-dimensional random noise or error vector.

Unfortunately, most problems with this general measurement










model have not been solved in a practical sense [Richardson

88]. Instead, the following measurement model with additive

noise is usually considered:



z = h(x) + v



Moreover, it is commonly assumed that x and v are

statistically independent, and that the probability density

functions f(x) and f(v) are known a priori. In addition,

most methods often assume f(x) and f(v) to be Gaussian with

the following statistical properties:



E[v] = 0, E[vvT] = R

E[x] = Ex, E[(x-E) (x-Ex)T] = M



where M and R are the state and noise covariance matrices

respectively. The above assumptions and equations are

sometimes written as:


x N(Ex, M), v N(0, R), and Cov(x, v) = 0



The measurement model is further simplified when the

function h(x) is linear. In this case, when the measurement

model is both linear and Gaussian, a closed form solution to

the estimation problem is obtained. The linear measurement

model is represented as










z = Hx + v



This is the measurement equation for the standard Kalman

filter. Given the above assumptions, the optimal estimate of

x, given z measurements, xopt(z) is determined by minimizing

a loss or risk function. This risk function is the

optimality criterion which provides a measure of the

"goodness" of the estimate. A typical loss function is the

mean squared error given by:



L(x, xopt) = (x Xopt)T W (x X)

where W is a symmetric positive definite weighting matrix.



Now let us consider two sensor systems producing two

sets of measurements zl and z2 of the state x. We are

interested in estimating x based on zl and z2, that is,

computing xopt(zl, z2) In the general case, one can not
compute xopt(zl, z2) based on the separate estimates xopt(zl)

and xopt(z2) of sensors 1 and 2 respectively [Richardson 88].

However, this is possible in the special case of a linear

Gaussian measurement model. For more information on

estimation in multi-sensor system the reader is referred to

Willner et al. [Willner 76] and Richardson and Marsh

[Richardson 88].

The major limitations of the above mentioned methods

stem from the assumption that f(x) is known a priori. This

requires a large number of experiments to be performed on











the sensor system in order to establish a model for f(x) and

f(v). Unfortunately, this is usually not done because it is

impractical or impossible. In such a case, the initial value

of Ex is usually set to zero, and the initial value of M is

set to a large multiple of the identity matrix indicating

our lack of knowledge of prior observations.



2.5.3 Fusion at Middle and Hiah Levels of Abstraction

At intermediate and high levels of abstraction,

features derived from lower level sensory data are present.

These features are normally associated with some degree of

uncertainty. It is the task of the multi-sensor system to

apply domain knowledge to these features in order to produce

valid interpretations about the environment. Thus, the basic

methodology involves the application of symbolic reasoning

and artificial intelligence techniques to aid the

interpretation task. Moreover, because "knowledge is power",

a powerful multi-sensor perception system must rely on

extensive amounts of knowledge about both the domain and the

problem solving strategy effective in that domain

[Feigenbaum 77].

Uncertainty results from the use of inadequate

knowledge as well as from attempts to reason with missing or

unreliable data. For example, in a speech understanding

system, the two sources of uncertainty are: 1) noise in the

speech waveform (sensor noise and variability), and 2) the

application of incomplete and imprecise theories of speech










[Newell 75]. Several methods for managing uncertainty have

been proposed. These include the use of Bayesian probability

theory, certainty theory (developed at Stanford and employed

in the MYCIN system [Buchannan 84]), fuzzy set theory [Zadeh

83], the Dempster/Shafer theory of evidence, nonmonotonic

reasoning, and theory of endorsements. Ng and Abramson [Ng

90] provide a good reference that introduces and compares

these methods.



2.5.3.1 Bayesian Probability Theory

Bayes theorem, a very important result of probability

theory, allows the computation of the probability of a

hypothesis based on some evidence, given only the

probabilities with which the evidence follows from the

hypothesis. Let


P(Hi/E) = the probability that Hi is true given evidence E
P(E/Hi) = the probability of observing evidence E when Hi is
true
P(Hi) = the probability that Hi is true
n = the number of possible hypothesis



Then, the theorem states that:


P(E/Hi) P(Hi)
P(Hi/E)=
n
S(P(E/HK) *P(HK)
k=l










In using Bayes's theorem, two major assumptions are

required: first that all the prior probabilities P(E/Hk) and

P(Hk) are known; second that all P(E/Hk) are independent.

These assumptions are difficult or impossible to meet in

many practical domains. In such situations, more heuristic

approaches are used. Another problem with statistical

methods in general is that these methods can not distinguish

between the lack of belief and disbelief. This stems from

the observation that in traditional probability theory the

sum of confidence for a certain hypothesis and confidence

against the same hypothesis must add to 1. However, often

one might have a certain degree of confidence that a certain

hypothesis is true, yet have no knowledge about it being not

true. Certainty theory attempts to overcome this limitation.



2.5.3.2 Certainty Theory

Certainty theory splits the confidence for and the

confidence against a certain hypothesis by defining the

following two measures:



MB(H/E) is the measure of belief of a hypothesis H given
evidence E, with 0
MD(H/E) is the measure of disbelief of a hypothesis H given
evidence E, with 0


These two measures are tied together with the certainty

factor:


CF(H/E) = MB(H/E) MD(H/E)












The certainty factor approaches 1 as the evidence for a

hypothesis becomes stronger, with 1 indicating absolute

truth. As the evidence against the hypothesis gets stronger

the certainty factor approaches -1, with -1 indicating

absolute denial. A certainty factor around 0 indicates that

there is little evidence for or against the hypothesis. To

combine the certainty factors of different hypothesis, the

following rules apply:



CF(H1 AND H2) = MIN[CF(H1), CF(H2)]

CF(H1 OR H2) = MAX[CF(H1), CF(H2)]



Another problem is how to compute the certainty factor of a

conclusion based on uncertain premises. That is, if P

implies Q with a certainty factor of CF1, and CF(P) is

given, then CF(Q) = CF(P) CF1. Also how to combine the

evidence when two or more rules produce the same result.

Assume that result Q produced by rule R1 has a certainty

factor CF(R1) and that rule R2 also produced Q with a

certainty factor CF(R2), then the resulting certainty factor

of Q, CF(Q), is calculated as follows:



1. When CF(R1) and CF(R2) are positive,

CF(Q) = CF(R1) + CF(R2) CF(R1)*CF(R2)

2. When CF(R1) and CF(R2) are negative,

CF(Q) = CF(R1) + CF(R2) + CF(R1)*CF(R2)










3. Otherwise,

CF(R1) + CF(R2)
CF(Q) =
1- MIN[ICF(R1)I, ICF(R2)I]




Although certainty theory solves many of the problems

presented by an uncertain world (as in MYCIN), the meaning

of the certainty measures and how they are generated, is not

well defined. The assignment of numeric certainty measures

based on human terms such as "it is very likely that" is not

well defined and considered by some as ad hoc.



2.5.3.3 Fuzzy Set Theory

Fuzzy set theory is yet another approach for dealing

with uncertainty. The main idea is that often information is

vague rather than random and hence a possibility theory must

be proposed as a measure of vagueness just as probability

theory measures randomness. The lack of precision is

expressed quantitatively by the notion of a fuzzy set. This

notion introduces a set membership function that takes on

real values between 0 and 1 and measures the degree to which

a set member belongs to the fuzzy set. To illustrate, let I

be the set of positive integers, and A be the fuzzy subset

of I that represents the fuzzy set of small integers. A

possibility distribution that defines the fuzzy membership

of various integer values in the set of small integers could

be characterized by:












mA(1)=l, mA(2)=l, mA(3)=0.9, mA(4)=0.7, . ,mA(30)=0.001



where mA(i) is the membership function that measures the

degree to which i (a positive integer) belongs to A.

To illustrate some of the combination rules of fuzzy

set theory, assume that both A and B are propositions with

C=A@B denoting the proposition that C is the combination of

A and B. Then, for



1. Conjunction: A AND B, is given by

m[A*B] (a,b)=min[mA(a),mB(b)].

2. Disjunction: A OR B, is given by

m[A+B] (a,b)=max[mA(a) ,mB(b)].

3. Implication: IF A THEN B, is given by

m[A/B] (a,b)=min[l, (l-mA(a)+mB(b))]



For more details on possibility theory including evidence

propagation and truth quantification rules the reader is

.referred to [Zadeh 78], while [Cheeseman 86] provides a

comparison between fuzzy and probabilistic reasoning.



2.5.3.4 Belief Theory

Belief theory, developed by Dempster and Shafer [Shafer

76] as an alternative to the theory of probability, makes a

fundamental distinction between uncertainty and ignorance.

As mentioned above, in probability theory the extent of










knowledge about a belief B is expressed in a single

probability number P(B). In cases where the prior

probabilities are not known, the choice of P(B) may not be

justified. Belief theory proposes belief functions where

each function distributes a unit of belief across a set of

propositions (called the "frame of discernment") for which

it has direct evidence, in proportion to the weight of that

evidence as it bears on each proposition. The frame of
discernment (0) is defined as an exhaustive set of mutually

exclusive propositions about the domain. The role of 0 in

belief theory resembles that of the sample space (U) in

probability theory except that in belief theory the number
of possible hypothesis is 12e1 while in probability theory it

is jQ. The basic probability assignment is a function m that

maps the power set of 0 into numbers between 0 and 1, that

is:


m: 20 ---> [0, 1]


If A is a subset of 0, then m satisfies:



1. m(0) = 0, where 0 is the null hypothesis.

2. X m(A) =1
AC 8



A belief function of a proposition A, BF(A) measures

the total amount of belief in A, and is defined as:











BF(A) = X m(B)
BCA



And satisfies the following:


1. BF (0)=0

2. BF ()=1

3. BF(A) + BF(~A) < 1



The Dempster/Shafer theory is based on Shafer's

representation of belief and Dempster's rule of combination.

The Shafer representation expresses the belief in a

proposition A by the evidential interval [BF(A), p(A)],

where BF(A) denotes the support for a proposition and sets a

minimum value for its likelihood, while p(A) denotes the

plausibility of that proposition and establishes its maximum

likelihood. P(A) is equivalent to 1-BF(~A), the degree with

which one fails to doubt 'A'. The uncertainty of 'A',

u(A)=p(A)-BF(A), is thus implicitly represented in the

interval [BF(A), p(A)]. The Dempster's rule of combination

is a method for integrating distinct bodies of evidence. To

combine the belief of two knowledge sources, suppose for

example that knowledge source 1 (KS1) commits exactly ml(A)

as a portion of its belief for proposition 'A', while KS2

commits m2(B) to proposition 'B'. Note that both 'A' and 'B'
are subsets of ) the frame of discernment. If we are










interested in computing the evidence for proposition C = A
n B, then


BF(C) = [1/(1-k)][ X ml(A)*m2(B)]

ArAB=C

u(C) = [l/(l-k)] [ml(8)*m2(0)]

where
k = X ml(A)*m2(B)

ArCB=0


The added complexity of the Dempster/Shafer theory

increases the computational cost. In addition, the

assumptions of independence required in the Bayesian

approach still apply here. Another criticism of this theory

is that it produces weaker conclusions due to the fact that

it avoids the assignment of stronger probability values, and

hence stronger conclusions may not be justified.



2.5.3.5 Nonmonotonic Reasoning

While all of the methods mentioned above use a numeric

model of uncertainty, nonmonotonic reasoning uses a non-

numeric approach. In this case the system starts by making

reasonable assumptions using the current uncertain

information, and proceeds with its reasoning as if the

assumptions were true. If at a later time these assumptions

were found to be false (by leading to an impossible

conclusion, for example), then the system must change these










assumptions and all the conclusions derived from them. Thus,

in contrast to the inference strategies discussed above

where knowledge can only be added (monotonic) and axioms do

not change, in nonmonotonic reasoning systems knowledge can

also be retracted. Truth Maintenance Systems [Doyle 79],

[deKleer 86], implement nonmonotonic reasoning. The argument

for nonmonotonic reasoning is that nonmonotonicity is an

important feature of human problem solving and reasoning. In

addition, numeric approaches to uncertainty do not consider

the problem of changing data, that is, what to do if a piece

of uncertain information is later found to be true or false.



2.5.3.6 Theory of endorsements

Cohen's theory of endorsements [Cohen 85] is yet

another qualitative approach to managing uncertainty. The

basic philosophy of this theory is to make explicit the

knowledge about uncertainty and evidence. The motivation for

this theory stems from the limitation of the numerical

approaches which summarize all supporting and opposing

evidence into a single number. The semantics of this number

that represent knowledge about uncertain information is

often unclear. Thus, the basic idea is that knowledge about

uncertain situations should influence system behavior.

Hence, if a required piece of evidence is lacking, an

endorsement-based system allocates resources to the

resolution task whose execution will provide the most

information for reducing the uncertainty. The system










represents all reasons for believing or disbelieving a

hypothesis in structures called endorsements. These

endorsements are associated with propositions and inference

rules. The system uses endorsements to decide whether a

proposition at hand is certain enough by back chaining and

determining if its sub-goals are well endorsed in order to

assert it. Cohen describes five classes of endorsements:


Rule endorsements.
Data endorsements.
Task endorsements.
Conclusion endorsements.
Resolution endorsements.


The main problem with this recently developed theory is

the exponential growth of the body of endorsements when

asserting a proposition based on endorsements of its sub-

goals and their associated sub-goals and so on. Thus, a

simple rule could lead to large bodies of endorsements after

a few inferences.



2.6 Implementation Examples

In this section we show how some of the techniques

described in this chapter are used in our research. As

mentioned in the introduction at the beginning of this

chapter, some of these techniques are used in the various

knowledge sources of the map builder module. The

implementation of the map builder is discussed in detail in

section 5.2. Here, we highlight with some examples the










implementation in our work of some of the sensor fusion

techniques described in this chapter.

The first example uses the consistency checking

techniques of section 2.5.1 to match observed and model line

parameters. A line parameter vector consists of orientation,

collinearity, and overlap variables with associated

uncertainties. The normal distance between the two parameter

vectors is calculated as described in section 2.5.1 and

compared to a threshold for a consistency check. Section

5.2.4.1 illustrates the details of the matching operation.

If the match is successful, the line parameter vectors are

now merged using the estimation techniques described in

section 2.5.1 also. These techniques use the standard Kalman

filter equations. The merged estimate with reduced

uncertainty is then compared to the observed lines to

determine the error in robot position and orientation.

Section 5.2.4.2 details such operations.

Another example illustrates a symbolic uncertainty

management technique similar to the theory of endorsements

presented in section 2.5.3.6, and to uncertainty management

within the schema system proposed in [Draper 88]. Such a

technique is used for conflict resolution when no match

exists between observed features (what the sensors are

actually seeing) and model features (what the sensors should

be seeing). To resolve the inconsistencies, we propose a

knowledge-based approach within the framework of the Sensory

Knowledge Integrator. Using a priori domain-dependent and









domain-independent knowledge, the system reasons about the

conflict and generates resolution tasks to resolve it. These

tasks utilize symbolic endorsements which constitute a

symbolic record of the object-specific evidence supporting

or denying the presence of an object instance. By

accumulating endorsements from a priori expectations about

the object, and from sub-part and co-occurrence support for

the object, the system deals with reasons for believing or

disbelieving a certain hypothesis. To illustrate this

approach we give the following example: Suppose for example

that the system's model of a coffee mug includes a handle.

So, the absence of a handle in a particular view of the mug

reduces the confidence rating of the mug hypothesis. Rather

than just lower the numeric confidence value, the "mug

detector" knowledge source also records the absence of the

handle. This is a source of negative support weakening the

confidence in that hypothesis. The system then takes steps

to remove this negative evidence, invoking another behavior,

for example the "curiosity" behavior, to scan the object

from a variety of view points to account for the missing

piece of evidence. If a hypothesis is subsequently posted

for the handle, the mug hypothesis regains its higher

confidence. The system thus arrives at more reliable

conclusions by reasoning about the sources of uncertainty.

The symbolic representation of uncertainty facilitates this.














CHAPTER 3
INTELLIGENT FULLY AUTONOMOUS MOBILE ROBOTS




One of the main goals of robotics research is the

development of autonomous robots. Autonomous robots are

desirable in many applications especially those where human

intervention is difficult. This chapter gives an overview

and analysis of the research issues associated with the

field of intelligent autonomous systems, and based on this

analysis presents the directions of the proposed research.

Traditional autonomous mobile robot projects [Crowley 85]

[Shafer 86] [Ayache 88] use some of the sensor fusion

techniques presented in chapter 2 in an attempt to build a

complete and accurate model of the environment. However,

despite the positive attributes of completeness and detail

of a global world model, some researchers [Brooks 86a] [

Connell 89] argue the need for its existence. The task of

constructing this world model may conflict with the need to

provide timely information about the environment. Hence, a

tradeoff exists between immediacy and assimilation [Payton

86]. For control purposes, immediacy considerations give a

higher value to sensor data that can be used to effect

action more quickly. This is because in many real time

situations the time between receiving sensor data and acting

on it is very critical. The disadvantage of immediacy is the










difficulty to obtain information or features critical to

plan execution from sensor data that has not undergone

sufficient assimilation. In addition, the extracted data may

be inconsistent or in error. To effect immediacy, Brooks

[Brooks 86a] proposed a parallel behavior-based

decomposition of the control system of a mobile robot. His

approach deviates from the traditional serial decomposition

approach which allows for greater sensor data assimilation.

The traditional approach decomposes the sensor to actuator

path into a few large processing modules in series, figure

3.la. In the behavior-based approach, figure 3.1b, the path

is divided into many small parallel modules each with its

own specialized task and complete path from the sensor to

the actuator. A general characteristic difference between

the two approaches is that the parallel approach requires

behavior fusion, while the traditional approach requires

sensor fusion. Section 3.2 compares these two approaches and

illustrates the advantages and disadvantages of each.

In this chapter we discuss the important issues

concerning the design of intelligent autonomous agents that

are capable of interacting with a dynamic environment. These

issues include consistent world modeling and control

architectures. We present a brief survey of the current

research in the field and highlight the approaches and

methodologies used by the various researchers in













SENSOR


O
-J
0


cr
S ACTUATORS

0
I


a. Serial decomposition.


REASON ABOUT BEHAVIOR OF OBJECTS

PLAN CHANGES TO THE WORLD

IDENTIFY OBJECTS

SENO MONITOR CHANGES
SENSO_
BUILD MAPS

EXPLORE

WANDER

AVOID OBJECTS


_CTUATORS


b. Parallel decomposition.




Figure 3.1 Control architectures, from [Brooks 86]:
a. Serial vs. b. Parallel decomposition.










tackling the important issues. On the issue of autonomous

robot control architecture, researchers are split between

the behavior-based decomposition and the traditional

decomposition. A brief survey of current research in the

behavior-based approach is presented in section 3.1.2, while

a survey of research in the traditional approach is embedded

in section 3.3 on world model construction issues since such

research involves the classical problems of sensor data

fusion, consistent world modeling, and robot position

referencing. In section 3.2 we discuss and compare the two

approaches, listing the advantages and limitations of each.

Fundamental to the traditional approach is the issue of

consistent world modeling which is presented in section 3.3.

Finally, section 3.4 discusses the directions of this

research.



3.1 Behavior-Based Approaches to Robot Autonomy

In this section we begin by tracing the basis of the

behavior-based approach to concepts in animal behavior, then

we provide a survey of current behavior-based approaches to

robot autonomy, and finally discuss the limitations of the

subsumption architecture [Brooks 86a], and reactive systems

in general.



3.1.1 Lessons from animal behavior

In designing autonomous mobile robots, valuable

insights can be obtained and lessons learned from nature.










Nature provides us with a variety of examples of animal

behavior as they successfully interact with their

environment in order to survive. Animals survive by

possessing the ability to feed, avoid predators, reproduce,

etc.. It is believed that animals survive due to a

combination of inherited instinctive responses to certain

environmental situations, and the ability to adapt to new

situations. Ethologists who study animal behavior in their

natural habitat, view animal behavior as largely a result of

the innate responses to certain environmental stimuli.

Behavioral psychologists, on the other hand, study animal

behavior under controlled laboratory settings, and believe

that animal behavior is mainly learned and not an innate

response. Observations and experiments by [Manning 79]

support the existence of both learned and innate behaviors

in animals. Animals with a short life span and small body

size such as insects seem to depend mostly on innate

behaviors for interacting with the environment, while

animals with a longer life span and larger body size

(capable of supporting large amounts of brain tissue

required for learning capacity) seem to develop learned

behavior.

Reflexive behavior is perhaps the simplest form of

animal behavior. A reflexive behavior is defined as having a

stereotyped response triggered by a certain class of

environmental stimuli. The intensity and duration of the

response of a reflexive behavior depends only on the










intensity and duration of the stimulus. Reflexive responses

allow the animal to quickly adjusts to sudden environmental

changes, and thus provides the animal with protective

behavior, postural control, and gait adaptation to uneven

terrain. Such reflexive responses are believed to be

instinctive and not learned since they have been observed in

animals which have been isolated from birth. Other reactive

types of behaviors include orientation responses where an

animal is oriented towards or away from some environmental

agent, and fixed-action patterns which are extended, largely

stereotyped responses to sensory stimulus [Beer 90].

The behaviors mentioned above are by no means solely

dependent on external stimuli. The internal state of the

animal plays an important role in the initiation,

maintenance, and modulation of a given behavior. Motivated

behavior are those governed primarily by the internal state

of the animal with no simple or rigid dependence on external

stimuli. For example, the behavior of feeding does not only

depend on the presence of food (external stimuli) but also

upon the state of hunger (the internal motivational

variable). Thus a behavior exhibited by an animal at a

certain moment enjoys the highest motivational potential

along with the proper combination of external stimuli

detected at that moment. The motivational potential of a

motivated behavior varies with the level of arousal and

satiation. In addition, such behaviors can occur in the










complete absence of any external stimuli, and can greatly

outlast any external stimulus[Beer 90].

Given a diverse set of sensory information about the

environment, and a diverse behavioral repertoire, how does

an animal select which information to respond to, and

properly coordinate its many possible actions into a

coherent behavior needed for its long term survival? The

answer to the first part lies in the fact that many

different animals have sense organs specialized in detecting

certain environmental features. For example, [Anderson 90]

reports observations by [Lettvin 70] about the frog's visual

system as specialized in detecting movements of small, dark

circular objects at close range, while it is unable to

detect stationary food objects or large moving objects.

Other animals that posses a variety of specialized sensory

detectors can actively select a subset of these detectors to

initiate the response of the animal. For example,

experiments and observations by Lorenz reported by [Anderson

90], indicate that the herring gull can detect attributes of

shape, size, color, and pattern of speckles of its eggs.

Moreover, depending on the task performed by the gull,

certain attributes become important while others become

unimportant. For example, when herring gulls steal the eggs

of other herring gulls, the shape and size of the egg are

very important. But when retrieving its own eggs as they

roll from the nest during incubation, shape becomes

unimportant while attributes such as color and pattern of











speckles gain importance. Based on these observations Lorenz

formulated his idea of the "innate releasing mechanism" that

actively selects a subset of the available sensory

information to trigger the appropriate response. This leads

us to the second part of the question posed at the beginning

of this paragraph: How do animals handle behavior conflict

where certain stimuli and motivations cause the tendency to

simultaneously perform more than one activity? The

observation is that the various behaviors of an animal

exhibit a certain organization or hierarchy [Manning 79].

Some behaviors take precedence over others, while others are

mutually exclusive. Internal state of the animal and

environmental conditions determine the switches between

behaviors. Such switches may not be all or nothing switches,

and the relationship between behaviors may be non-

hierarchical. That is, behaviors can partially overlap

making it sometimes difficult to identify direct switches

between them [Beer 90]. Survival dictates that whatever

behavioral organization exists in animals, it must support

adaptive behavior, whereby, based on past interactions with

the environment, aspects of future behavior are modified.

Anderson and Donath [Anderson 90] summarize some of the

important observations in their research regarding animal

behavior as a model for autonomous robot control:


a) To some degree all animals possess a set of innate behaviors
which allow the animal to respond to different situations.
b) The type of behavior exhibited at any given time is the result
of some internal switching mechanism.










c) Complex behavior can occur as the result of the sequential
application of different sets of primitive behaviors with the
consequence of a given behavior acting as a mechanism which
triggers the next one.
d) Simple reflex types of behavior occur independent of
environmental factors and provide the animal with a set of
protective behaviors.
e) Activation of more complex types of behavior typically depend
upon external and internal constraints.
f) Animals typically only respond to a small subset of the total
amount of sensory information available to them at any given time.
Animals have developed specialized types of detectors which allow
them to detect specific events.
g) Behavior is often organized hierarchically with complex
behavioral patterns resulting from the integration of simpler
behavioral patterns.
h) Conflicting behaviors can occur in animals. These will require
either a method of arbitration between such behaviors or the
activation of alternate behaviors. Pages 151-152.



3.1.2 Current behavior-based approaches to robot autonomy

[Brooks 86a] follows a behavior-based decomposition and

proposes the famous subsumption architecture for behavior

arbitration. The main idea is that higher-level layers or

behaviors override (subsume) lower-level ones by inhibiting

their outputs or suppressing their inputs. His subsumption

architecture have been used in a variety of robots [Brooks

90], and proved robust in dynamic environments. Most of his

robots are designed to be "artificial insects" with a

deliberate avoidance of map or model building. Brooks' idea

is that "the world is it's own best model", and intelligent

action is the outcome of many simple behaviors working

concurrently and coordinated through the context of the

world. There is no explicit representation of goals or

plans, rather, the goals are implicitly designed into the

system by the pre-determined interactions between behaviors










through the environment. The next section discusses in

detail the advantages and limitations of the subsumption

architecture.

[Payton 86] also follows a behavior-based

decomposition, and describes a collection of reflexive

strategies in a hierarchy of control, all competing for the

control of the vehicle. The winning behavior is determined

by a winner take all arbitration mechanism. Later work by

[Payton 90] describes methods of compiling world knowledge

such as mission constraints, maps, landmarks, etc.. into a

form of "internalized plans" that would have maximal utility

for guiding the action of the vehicle. He proposes a

gradient description to implicitly represent the

"internalized plans". Using this representational technique,

a priori knowledge such as a map can be treated by the

behavior-based system as if it were sensor data.

[Arkin 87] proposes a schema-based approach to the

navigation of a mobile robot. His motor schemas are

processes that run concurrently and independently, each

operating in conjunction with its associated perceptual

schemas. No arbitration mechanism is required, instead the

outputs of the various motor schemas are mapped into a

potential field and combined to produce the resultant

heading and velocity of the robot. Arkin demonstrates that

strategies for path following and obstacle avoidance can be

implemented with potential field methods by assigning










repulsive fields around observed obstacles, and by

appropriately adjusting the strength of the fields.



3.1.3 Limitations of the subsumption architecture

At first glance the subsumption architecture appears

modular. Theoretically, a new layer could simply be added on

top of the previous layers to achieve a new level of

competence. In reality, upper layers or behaviors interfere

with the internal states of lower-level layers, and thus can

not be designed independently. In fact, the whole controller

must be redesigned when even small changes to lower-level

behaviors are implemented [Hartley 91]. The reason for the

lack of modularity is that in the subsumption architecture,

the behavior arbitration mechanism is not separated from the

actual stimulus/response behavioral function. Moreover, the

arbitration strategy is further complicated by the use of

timeouts, or temporal ordering of behaviors [Anderson 90].

Another limitation of the subsumption architecture stems

from its rigid hierarchy of behaviors. In such a hierarchy,

a behavior is either higher or lower than another, with the

higher behavior inhibiting or suppressing the one below it.

For many non-trivial real life applications, such hierarchy

can not be established. Many behaviors are mutually

exclusive and not hierarchically organized. For example, in

our implementation the "target-nav" behavior which guides

the robot towards a specified location, is not higher or

lower than the "boundary-following" behavior. Rather, the










two are mutually exclusive. In addition, it is possible that

lower-level behaviors need to inhibit higher-level ones.

Examples of such situations are numerous in biological

system. In the subsumption architecture, only higher-level

behaviors inhibit lower ones.

In general, the subsumption architecture suffers the

same limitations of behavior-based systems. Such systems

require some method of arbitration such as the subsumption

architecture. The subsumption architecture is specifically

implemented as part of behavior-based systems motivated by

the desire of their designers to produce artificial insects.

Hence, such systems avoid world modeling and the explicit

representation of goals. Instead, as mentioned earlier, the

driving philosophy is that "the world is its own best

model", and goals are implicitly designed within the system

by the designer establishing, a priori, the interactions

between behaviors through the environment. This is a serious

limitation of such behavior-based systems since the designer

is required to predict the best action for the system to

take under all situations. Obviously, there is a limit to

how far one can forsee the various interactions and

constraints in order to precompile the optimum arbitration

strategy [Maes 90].









3.2 Control Architecture: Behavior-Based vs. Traditional
Control

As mentioned before, the control architecture of an

autonomous intelligent mobile robot can be modeled as either

a serial or a parallel decomposition of the perception-

action control path, figure 3.1. The more traditional

approaches [Crowley 85] [Kriegman 89] [Moravec 85] are

serial in nature where the control path is decomposed into a

few modules in series such as: 1) Sensing, 2) Modelling, 3)

Planning, 4) Actuation. In the parallel decomposition

approach [Brooks 86a] [Connell 89] [Payton 86] multiple

parallel control paths or layers exist such that the

perceptual load is distributed. Each layer performs a

specialized goal or behavior and processes data and issues

control commands in a manner specific to its own goals. In

[Brooks 86a] these layers of control correspond to levels of

competence or behaviors with the lower layers achieving

simple tasks such as avoiding obstacles and the higher

layers incrementally achieving more complex behaviors such

as identifying objects and planning changes to the world. In

this hierarchical set of layers, each layer is independent

from the others in performing its own tasks even though

higher level layers may influence lower level layers by

inhibiting their output or suppressing their input. Unlike

the traditional approach which has the disadvantage of

imposing an unavoidable delay in the sensor to actuator

loop, the layered approach enjoys direct perception to

action through concurrency where individual layers can be










working on individual goals concurrently. In this case

perception is distributed and customized to sensor-set/task-

set pair of each layer. This eliminates the need for the

robot to make an early decision on which goals to pursue.

Another advantage of the parallel approach is flexibility.

Since modules or layers are independent, and each having its

own specialized behavior and goals, it is possible that each

may have its own specialized interface to the sensors and

actuators. Flexibility stems from the fact that the

interface specification of a module is part of that module's

design and does not affect the other modules' interfaces.

This contradicts the traditional serial approach where a

modification to a module's interface might require

modification of at least the previous and the following

modules if not the whole system. Moreover, all of the

modules in the serial approach must be complete and working

before the system is operational, while on the other hand, a

behavior-based system can still produce useful behavior

before all the modules are complete. The main disadvantage

of a behavior-based approach is the possibility of

exhibiting cyclical behavior patterns by the robot due to

the lack of memory within some behaviors, that is, such

behaviors do not remember previous events and base their

decisions solely on the latest sensor stimuli. This prevents

the robot from responding to events which happen over

several time periods and could cause cyclical behavior. In

our initial simulation of a robot wandering with a fixed










control algorithm, our robot exhibited cyclical behavior.

Anderson and Donath [Anderson 88] presented an approach

based upon the use of multiple primitive reflexive

behaviors, and came to the conclusion that "... cyclical

behavior may indicate that an essential characteristic of a

truly autonomous robot is the possession of memory and

reactive behavior (i.e., the ability to react to events

which occur over a number of intervals of time and the

ability to alter the behavior depending upon the previous

behavior)." [Anderson 88, p. 205]. Brooks and Connell

[Brooks 86b] have also observed cyclical behavior in their

wandering and wall following behaviors.

The majority of mobile robot projects follow somewhat

the traditional approach. In this approach, world

representation and sensory confirmation of that

representation are essential to the intelligence of an

autonomous mobile robot. A composite world representation is

generated through the integration of the various local

representations which are themselves formed by the fusion of

data from the multiple sensors onboard. Fusion at various

levels of abstraction is performed in order to produce the

representation useful to the planning subsystem. Unlike the

behavior-based approach where each behavior task employes

its own specialized representation, the representation

employed in the traditional approach is general purpose and

thus is useful for a variety of situations and planning

tasks. In contrast, the behavior-based approach employs a










variety of specialized representations (each derived from a

small portion of the sensors data) for use by a number of

concurrent planning tasks resulting in many distinct,

possibly conflicting, behaviors. Thus, the need to perform

"behavior fusion" arises in this case as opposed to sensor

fusion in the traditional approach. Behavior fusion

sacrifices the generality obtained by sensor fusion in order

to achieve immediate vehicle response, while sensor fusion

sacrifices immediacy for generality. The immediacy versus

assimilation tradeoff issue is adequately presented by

[Payton 86].



3.3 Issues in World Model Construction

In this section we examine the issues of constructing a

world model of an autonomous mobile robot. The environment

of a mobile robot is often unstructured and contains objects

either as obstacles to avoid or as items to be examined or

manipulated. In the traditional approach, a mobile robot

must build and use models of its environment. This model

must be accurate and must remain consistent as the robot

explores new areas or revisits old ones [Chatila 85].

Handling inconsistencies in world model construction of a

multi-sensor system, is one of the main problems tackled by

the proposed research. In order to construct an accurate and

consistent model of the environment, the robot must be able

to correctly determine its position and orientation. This is

a difficult task given that sensors are imprecise.












3.3.1 Position Referencing for a Mobile Robot

A mobile robot can achieve position referencing by any

of the following methods:



Trajectory Integration Referencing. Uses odometeric

devices such as shaft encoders without external reference.

These methods are prone to errors (due to wheel slippage)

that are cumulative and cause position drift.



Absolute position referencing. Uses fixed known

external beacons throughout the environment. The more

external beacons we place at known absolute positions in the

environment, the more structured this environment becomes.

In this case the errors in the robot's position and

orientation are related to the beacon system measurement

accuracy.



Relative position referencing. Is performed with

respect to objects with characteristic features whose

positions in the environment are known with good accuracy.

This method is very desirable yet it introduces considerable

complexity. A challenging task in this case is for the robot

to define its own references.











3.3.2 World Model Representation

Various world model representations have been proposed.

The choice of an adequate representation depends on the

domain (indoor, outdoor, factory environment, etc..), the

task (navigation, manipulation, identification, etc..), and

the sensors used. Crowley [Crowley 87, 85] suggested a

representation in terms of line segments in a 2-D floor plan

world. Kak et al [Kak 87] also used a model represented as a

2-D line drawing of the expected scene. Chatila and Laumond

[Chatila 85] used three types of models: geometrical,

topological, and a semantic model. The geometrical model is

a 2-D model containing the position in absolute coordinates

of the ground projection of vertices of polyhedral objects.

The topological model is represented by a connectivity graph

of places where a place is defined as an area that is a

functional unit such as a workstation or a topological unit

such as a room or a corridor. The semantic model is a

symbolic model containing information about objects, space

properties, and relationships. Kent et al [Kent 87] proposed

a representation of the world that consists of both a

spatial and an object or feature-based representation. The

spatial representation classifies the world space as

occupied, empty, or unknown, and explicitly represents

spatial relationships between objects. The feature-based

representation associates each object with the set of

features that verifies its identity. Elfs [Elfs 89]

described the occupancy grid representation which employes a










2-D or 3-D tessellation of space into cells, where each cell

stores a probabilistic estimate of its state. The state

associated with a cell is defined as a discrete random

variable with two states, occupied and empty.

This research adopts a representation similar to that

proposed by Kent but the spatially-indexed representation

employs a 2-D tessellation of space where each cell in the

grid, not only contains its state of occupied, empty, or

unknown, but it also contains information such as what

object the cell (if occupied) belongs to and whether it is a

boundary point or an edge point, etc.. This representation

is useful for navigation in computing free paths, and for

determining the identity of objects or features in a given

location. The object-indexed representation is linked to the

spatial representation and contains entries such as the

object's name, vertices, bounding edges, and other

discriminating features. This representation is suited to

responding to inquiries about objects or features by name or

by description. In addition to the two representations

mentioned above, we also construct a 2-D line representation

from sonar data. All these representations are implemented

under the proposed Sensory Knowledge Integrator framework

described in section 4.5.1. We describe the implementation

of the spatially-indexed representation in section 5.2.1,

while section 5.2.3 describes the implementation of the 2-D

line representation. This representation is used for







59


position referencing of the robot as described in section

5.2.4.



3.3.3 Managina World Model Inconsistencies

At any point in time the robot has a global and a local

model of its environment. The local model is robot-centered

and represents the environment as perceived at the moment by

the various sensors onboard. The robot is first faced with

the challenge of constructing the local model by integrating

information from the various sensor systems with a priori

knowledge about the environment. Next it must update the

global model using the local model and its position and

orientation information. The difficult problem is to

maintain a consistent model given inaccuracies in position

measurements and in sensor data and its processing elements.

Inconsistencies between the global and the local models must

be resolved in order to avoid model degradation. Resolving

these inconsistencies should improve the accuracy of the

position and orientation information.

A variety of methods have been proposed for model

consistency checks. Crowley [Crowley 85] defines a function

named CORRESPOND that is called for every line in the sensor

model (most recent sensor data described in terms of line

segments) to be matched with every line segment in the

composite local model (an integration of recent sensor

models from different viewing angles). This function tests

the correspondence of two line segments by checking: (1) If










the angle between the 2 segments is less than a certain

threshold, (2) If the perpendicular distance from the

midpoint of one segment to the next is less than a

determined threshold, and (3) If one segment passes through

a bounding box (tolerance) around the other. The outcome of

this test gives five types of correspondence. In a later

paper Crowley [Crowley 87] uses the normal distribution to

represent spatial uncertainty, and the normal distance as a

measure for matching the model parametric primitives (ex.

lines, edge segments) to the observed ones. He defines the

function SIMILAR which returns a true if the normal distance

between the two primitives is less than a certain threshold,

else it returns a false. The function CORRESPOND now

consists of a simple set of attribute tests using the

function SIMILAR.

Andress and Kak [Andress 87] define a COLLINEARITY and

a NONCOLLINEARITY function as a measure of compatibility and

incompatibility respectively between a model edge segment

and an observed one. These compatibility measures form the

initial "basic probability assignment" for the observed

segments. The Dempster-Shafer formalism is used for belief

update in the face of new evidence.

Our approach to consistent world modeling is similar to

that of [Crowley 87]. It is implemented as part of the

Sensory Knowledge Integrator framework. Section 5.2.4 of

chapter five details the implementation of our method. The

parameters of observed and model lines are matched using the










correlation and consistency checking techniques described in

section 2.5.1 of chapter 2. A successful match indicates

that all orientation, collinearity, and overlap tests have

been satisfied. Next, we merge the parameters of the two

matched lines using estimation techniques also described in

section 2,5.1. These techniques use the standard Kalman

filter equations. The merged estimate with reduced

uncertainty is then compared to the observed lines to

determine the error in robot position and orientation.

Section 5.2.4.2 details such operations.

In case no match exists between a model feature (what

the sensors should be seeing) and an observed feature (what

the sensors are actually seeing), then a conflict exists and

must be resolved. In our implementation we did not encounter

such conflicts and we did not setup our experiments in order

to obtain a conflict. Instead, in what follows, we propose

how inconsistencies could be approached under the Sensory

Knowledge Integrator framework. We propose a knowledge-based

approach for resolving the inconsistencies where the system

generates resolution tasks to resolve the conflict, and

reasons about the conflict using a priori domain-dependent

and domain-independent knowledge. At higher levels of

abstraction, the conflict resolution task utilizes symbolic

endorsements which constitutes a symbolic record of the

object-specific evidence supporting or denying the presence

of an object instance. Thus, we deal with reasons for

believing or disbelieving a certain hypothesis by











accumulating endorsements from a priori expectations about

the object, and from sub-part and co-occurrence support for

the object.



3.4 Direction of Proposed Research

As mentioned in chapter 1, this research follows a

hybrid (behavior-based and cognitive) approach to the

problem of controlling an autonomous mobile robot. The goal

is to enable autonomous operation in a dynamic, unknown, and

unstructured indoor environment. The robot knows about

certain objects to expect in the environment, but does not

have a map of it. The robot has considerable general

information about the structure of the environment, but

cannot assume that such information is complete. It

constructs a model of the environment using onboard

sensors. We aim to develop a general purpose robot useful

for a variety of explicitly stated user tasks. The tasks

could either be general or specific including such tasks as

"do not crash or fall down", "build a map", "locate all

green drums" etc..

Unlike most behavior-based approaches which avoid

modelling of the world and the use of world knowledge, our

view is that while world models are unnecessary for low-

level actions such as wandering around while avoiding

obstacles, they are essential to intelligent interaction

with the world. General, efficient, and flexible navigation

of a mobile robot require world models. The world models










provide a "bigger picture" for the robot when

reflexive/reactive behaviors encounter difficulty. Such

difficulties include trap situations due to local minima

causing cyclic behavior, oscillations in narrow passages or

in the presence of obstacles, and inability to pass between

closely spaced obstacles [Koren 91][Brooks 86b] [Anderson

88]. Trap situations are caused by various topologies of the

environment and by various obstacle configurations such as a

U-shaped configuration. [Arkin 90] gives a vivid example of

the difficulties encountered by reactive control as

resembling the "fly-at-the-window" problem. This situation

arises when the insect expends all of its energy trying to

go towards the sunlight outside through the glass of the

window. Our work dynamically builds a map of the environment

within the Sensory Knowledge Integrator framework. As we

will see in section 4.3.1, this framework contains a variety

of knowledge sources including different types of "trap

detector" knowledge sources that use map data to look for

traps. This information is used by the planning module to

avoid or recover from traps. Map data is used to reconfigure

the lower-level behaviors and not to replace the function of

these behaviors. In this manner the environment is more

efficiently explored while the robot still enjoys the real-

time operation of the reflexive behaviors. Additionally, the

construction of a general purpose world model makes use of

the available world knowledge. For example, the Sensory

Knowledge Integrator, the underlying framework of our map










builder module, exploits a priori knowledge about the

environment such as objects to be encountered or

manipulated. The a priori knowledge gives the robot an idea

about its relationship to the world and allows it to

efficiently use its resources.

The architecture for planning and control of a mobile

robot should be a synthesis of the traditional serial

approach and the parallel behavior-based approach. This

architecture consists of a hierarchy of control in which

lower level modules perform "reflexive" tasks while higher

level modules perform tasks requiring greater processing of

sensor data. We call our architecture a hybrid one because

it includes a cognitive component and a behavior-based

component, figure 3.2. A robot controlled by such a hybrid

architecture gains the real-time performance of a behavior-

based system while maintaining the effectiveness and goal

handling capabilities of a planner with a general purpose

world model. The basic instinctive competence for the robot

such as avoiding obstacles, maintaining balance, wandering,

moving forward, etc..are provided by the behavior-based

component of the system, while the cognitive part performs

higher mental functions such as planning. The higher the

competence level of the behavior-based system, the simpler

the planning activity. Motivated behaviors implemented as

part of the behavior-based system, and the associated

motivation state, form the interface between the two

components. A motivated behavior is triggered mainly by the











































associated motivation state. By merely setting the

motivation state of the robot, the cognitive module

activates selected motivated behaviors in order to bias the

response of the behavior-based system towards achieving the

desired goals. The details of plan execution are left to the

behavior-based subsystem. The motivation state consists of a

variety of representations, each associated with the

corresponding motivation-driven behavior. It is the means of

communication between the cognitive and the behavior-based

subsystems, and could be thought of as a collection of










virtual sensor data. The "target-nav" behavior, discussed in

section 4.2.4 of our implementation, is an example of a

motivation-driven behavior. Its motivation state is

represented as a location that is set by the planning

module. In addition to its virtual sensor data input, it

also acquires data from the real position shaft encoder

sensors in order to generate a vector heading towards the

target. The arbitration of the various behaviors in the

behavior-based system competing for control of the robot

actuators, is partly hardwired in the behavior-based system,

and partly encoded in a flexible arbitration strategy in the

cognitive system. The flexible strategy changes during the

operation of the robot depending on the current situation

and the task at hand. In section 4.3 of the next chapter, we

discuss various arbitration strategies including the one

used in our implementation.

As mentioned earlier, we view world models as essential

to intelligent interaction with the environment, providing a

"bigger picture" for the robot when reflexive behaviors

encounter difficulty. Thus, the research proposed in this

paper adopts a knowledge-based approach to the problem of

constructing an accurate and consistent world model of the

environment of an autonomous mobile robot. We propose to

construct this model within the framework of the Sensory

Knowledge Integrator proposed in [Bou-Ghannam 90a,b] and

described in chapter 4. A more accurate model is obtained

not only through combining information from multiple sensory










sources, but also by combining this sensory data with a

priori knowledge about the domain and the problem solving

strategy effective in that domain. Thus, multiple sensory

sources provide the added advantages of redundancy and

compensation (where the advantages of one sensor compensates

for the disadvantages or limitations of the other) while

domain knowledge is needed to compensate for the

inadequacies of low-level processing, as well as to generate

reasonable assumptions for the interpretations of features

derived from lower-level sensory data. In addition we

propose to achieve a consistent model by making explicit the

knowledge about the inconsistencies or conflict, and using

this knowledge to reason about the conflict. This is similar

to the theory of endorsement proposed by Cohen [Cohen 85]

where resolution tasks are generated, and positive and

negative endorsements are accumulated in order to resolve

the conflict.















CHAPTER 4
THE PROPOSED HYBRID CONTROL ARCHITECTURE




We propose an architecture that is a synthesis of the

parallel decomposition and the traditional serial

decomposition of a mobile robot control system. Both of

these architectures were discussed and compared earlier.

Figure 4.1 depicts the general framework of our hybrid

control architecture while figure 4.2 shows a block diagram

of a specific implementation of the proposed architecture.

The functions of the various blocks in the figures will

become clear as we proceed through the chapter. The hybrid

architecture supports a hierarchy of control in which the

various lower level modules (such as "avoid-obstacles",

"target-nav", "follow-wall", etc..) perform reflexive

actions or behaviors providing a direct perception to action

link, while the higher level modules (such as the map

builder and the planning modules) perform tasks requiring

greater processing of sensor data such as modelling. It is

important to emphasize parallelism or concurrency in the

hybrid architecture. For example, in figure 4.2, the

planning module, the map builder, and the lower-level

behaviors are all running concurrently. The lower-level

behaviors constitute the behavior-based subsystem and

provide the basic instinctive competence, while the higher-


68










level modules provide the cognitive function. Note that

action takes place only through the behavior-based

subsystem. When needed, the cognitive (planning) module

effects action by reconfiguring the behavior-based system.

Reconfiguration involves arbitration, and changing the

motivation state of the behavior-based subsystem, as will

become clear later in this chapter. We call this

decomposition a hybrid parallel/serial decomposition because

even though the whole system follows a parallel layered

decomposition, the higher level modules (namely the map

builder and planning modules) follow somewhat the

traditional serial approach of sensing, modelling, and

planning before task execution or actuation occurs. However,

unlike the traditional approach, the planning module does

not have to wait on the map builder for a highly processed

representation of the environment before it can effect

action. Instead, based on its current knowledge and the

status provided by the lower level behaviors, the planning

module can select from a variety of lower level behaviors or

actions. Thus, unlike the subsumption architecture proposed

by [Brooks 86a] where any behavior subsumes the function of

all the behaviors below it (by inhibiting their outputs or

suppressing their inputs), in our implementation the

arbitration strategy is incorporated in a set of rules in

the planning module. In what follows we discuss the

individual blocks of our proposed architecture starting with

the planning module.




























Actuators


Competence level 0


Figure 4.1 A general framework of the proposed hybrid
control architecture.
















































Figure 4.2 A specific implementation of the hybrid control
architecture.













71










4.1 The Planning Module: A Knowledge-Based Approach

The planning module performs reasoning and task

planning to accomplish user specific tasks, such as locate

all green drums in a warehouse for example. In order to

accomplished such tasks, the planning module performs

various tasks including map-based planning (such as route

planning) and behavior reconfiguration. For example, in our

implementation of the planning module, discussed in detail

in section 5.1, the task is to efficiently map the

environment without crashing into things. Thus our planning

module performs some simple map-based planning functions,

but deals mainly with behavior arbitration. The arbitration

strategy that effects the behavior reconfiguration is

embedded into a set of production rules in the planning

module. In determining the current behavior arbitration, the

arbitration rules utilize knowledge about the goals, the

environment, the individual behaviors, the current situation

status, etc.., and thus create a flexibile, goal-driven

arbitration strategy that is useful for a variety of

missions in different types of domains. We will learn more

about arbitration and the arbitration network in section

4.3. It is important to note that behavior reconfiguration

is only used when the behavior-based system encounters

difficulties such as a trap situation as described in

section 3.4. Various trap situation analogous to the "fly-

at-the-window" situation can be detected in the map builder

module, and recovered from or avoided using heuristic rules










in the planning module. Reconfiguration is accomplished by

changing the motivation state of the motivation-driven

behaviors, and by performing some behavior arbitration.

Thus, the planning module selects (enables and disables)

from a variety of behaviors (such as avoid-obstacles,

target-nav, follow wall, etc..) the appropriate set of

behaviors at the appropriate time for the task at hand. For

example, in our implementation given the task of mapping the

environment, when the 2-D line representation becomes

available, the corner detector knowledge source in the map

builder examines this representation and posts a corner

hypothesis on the hypothesis panel, which in turn causes a

specific rule of the arbitration strategy rules in the

planning module to fire selecting as a result the target-nav

behavior with the location of the corner to be investigated

as its target. Thus, the behavior selection is based upon

status inputs from the various behaviors and from the map

builder module which, in time, will also provide a high-

level spatially-indexed and object-indexed representation of

the environment. In addition, the planning module uses a

priori knowledge about the task, the environment, and the

problem solving strategy effective in that domain. For

example, if the robot's task is to locate all green drums of

a certain size, a priori knowledge (such as "drums of this

size are usually located on the floor in corners") can

greatly help the task of searching for the drums. This

knowledge about the task and objects in the environment,










coupled with knowledge about the environment (such as in an

indoor environment, corners are the intersection of two

walls, etc..) can be brought to bear to improve the

efficiency of the search. Other types of a priori knowledge

include self knowledge such as the diameter and height of

the robot, the physical arrangement of the sensors onboard

the robot, and types and characteristics of sensors used.

For example, knowing the diameter of the robot, helps in

making the decision of whether to let the robot venture into

a narrow pathway. Note that having a priori knowledge about

the environment does not mean that the environment is known,

instead it means that the robot knows about certain objects

to expect in the environment (for example, walls, doors and

corners in an indoor environment), but does not have a map

of it.

One type of reasoning performed by the planning module

include the detection and prevention of cyclic behavior

patterns exhibited by the robot when driven by the lower-

level reflexive behaviors. For example, if in the wander-

while-avoiding-obstacles behavior, the robot gets stuck in a

cyclic pattern (similar to the fly-at-the-window situation)

giving no new information to the map builder, the map

builder forwards its status to the planning module including

the location of the centers of various frontiers that form

the boundary between empty and unknown space. The planning

module enables the target-nav behavior to choose one of

these centers as its target. When this target is reached










with the robot oriented towards the yet unknown area, the

planning module might enable the wander behavior again to

allow discovery of unknown areas. The actual implementation

of the planning module is presented in chapter 5 and follows

a knowledge-based approach. We use CLIPS, a knowledge-based

system shell, as an implementation tool to represent and

reason with the knowledge used to accomplish the tasks of

the planning module. A review of CLIPS is given in the

appendix.



4.2 Lower-Level Behaviors

As mentioned earlier, the behavior-based subsystem

provides the robot with the basic instinctive competence

which are taken for granted by the planning module. The

subsystem consists of a collection of modular, task-

achieving units called behaviors that react directly to

sensory data, each producing a specific response to detected

stimuli. These behaviors are independently running modules

that perform specific tasks based on the latest associated

sensor data. Each of these modules is associated with a set

of sensors needed to perform the specialized behavior of the

module. Sensor data is channelled directly to the individual

behavior allowing for "immediate" reaction. Thus each module

constructs from its sensor data a specialized local

representation necessary to effect the behavior of the

module. The responses from multiple behaviors compete for

the control of the robot actuators, and the winner is










determined by an arbitration network. The behaviors

comprising our behavior-based subsystem are reactive,

characterized by a rigid stimulus-response relationship with

the environment. The response of a reactive behavior is

deterministic and strictly depends on the sensory stimulus

(external environment and internal state). It is not

generated by a cognitive process with representational

structures. The model of our reactive behavior is given in

figure 4.3. The function F(Si) represents the deterministic
stimulus-response function, while the threshold T is

compared to F(Si) before an output response is triggered. In

our implementation, the various thresholds are

experimentally determined. In general, the threshold

represents one the important parameters for behavior

adaptation and learning. Thus, adhering to the biological

analogy, the threshold can be set by other behaviors

depending on the environmental context and the internal

state. We further divide our reactive behaviors into

reflexive and motivation-driven behaviors. The motivation-

driven behaviors are partly triggered by the motivation

state of the robot (set by the planning module in our

implementation, but could theoretically be set by other

behaviors), while the response of a reflexive behavior is

driven only by external stimuli. Reflexive behaviors

constitute the protective instincts for the robot such as











Stimuli
S.
I


Figure 4.3 Model of a reactive behavior.


Sonar


2
/r) e


Robot


Figure 4.4 Sonar sensor repulsive force model.











avoiding obstacles and maintaining balance, while motivated

behaviors execute goal-driven tasks triggered by the

associated motivation state set by the planning module in an

effort to bias the response of the behavior-based system

towards achieving the overall mission. Such motivated tasks

include moving to a specified target location as in the

"target-nav" behavior.

In addition to providing the specific task-achieving

functionality, the behaviors also serve as abstraction

devices by providing status information to the planning

module. Such status include variables representing error

status variables, and operating conditions variables. The

error status include variables indicating errors such as

robot communications error, or sonar data error. The

operating conditions variables represent a behavior's

operating conditions such as "target-reached" or "break-

detected" for example.

Some of the behaviors we are interested in include

"avoid-obstacles", "wander", "target-nav", "boundary-

follower", and "path follower".



4.2.1 Avoid obstacles behavior

This behavior uses sonar information from a ring of 12

sonar sensors placed around the robot. Each sonar hit (range

data) is modeled as the site of a repulsive force whose

magnitude decays as the square of range reading (distance to

the obstacle) of that sonar [Khatib 85], figure 4.4. The










avoid behavior determines the resultant repulsive force

acting on the robot by summing the forces sensed by each

sonar sensor, as follows:


12
Fres = (l/ri)2.ei
i=l

The magnitude of the repulsive force is then compared to an

experimentally determined threshold value. If the magnitude

exceeds the threshold value then the repulsive force

represents the response of the avoid behavior as a vector

heading for the robot. This threshold is fixed

(experimentally determined and normalized to outputs of

other behaviors) in our implementation, but within an

adaptive behavior-based system it is not fixed and

constitutes a variable parameter which is adjusted by the

outputs of other behaviors as the situation demands. For

example, a "charge-battery" behavior may tune down the

effect of the "avoid-obstacles" behavior by raising its

threshold as the robot approaches the battery charging

station, allowing the robot to dock and connect to the

charging pole.



4.2.2 Random wander behavior

This behavior generates a random heading for the robot.

This behavior, coupled with the avoid obstacles behavior,

form an emergent behavior with the following functionality:

"wander at random without bumping into things". Such an










emergent behavior is used in totally unknown environments

for random wandering. This is also useful for leading the

robot out of trap situations.



4.2.3 Boundary following behavior

This behavior uses sonar scan data to determine the

direction for safe wandering or the nav-vector as we call

it. In this behavior mode the robot keeps moving forward as

long as the range of the forward pointing sonar is larger

than a certain threshold, and the range values of the rest

of the sonar sensors are also larger than their respective

thresholds. The range data is dynamically read from a set of

12 sonar sensors onboard forming a ring around the robot.

When the conditions for a clear (no obstacle in the way)

forward motion are no longer satisfied, this behavior

determines the next clear nav-vector from the latest sonar

scan. Usually, many nav-vectors are valid and the one

closest to a forward direction is chosen, thus minimizing

the degree of turns for the robot. This also has the effect

of following the boundary of the indoor environment such as

walls. The algorithm involved is straight forward, fast and

utilizes sonar range data directly with minimal processing

of that data. This behavior could also be used as a

generalized wandering behavior for map building. Note that

while this behavior is active, the map builder is busy

(concurrently) assimilating sensor data from the various

locations visited by the robot.












4.2.4 Taraet-Nay Behavior

This behavior is a location attraction behavior that

generates as its response a vector proportional to the

vector between the current location of the robot and the

specified target location. Robot position is provided by

wheel encoder sensors, while the target position is provided

by the planning module as part of the motivation state of

the robot. So, this is a motivation-driven behavior, the

motivation being to reach the target location. A robot R at

(xr, Yr) is attracted to a target T at (xt, Yt) by the

following heading vector:


V = RT = (xt Xr)i + (Yt Yr)J



The magnitude of this vector decreases as the robot

approaches the target, and the target-reached flag is set

when the magnitude drops below a certain threshold. In

addition, our implementation normalizes the magnitude and

sets an experimentally determined threshold value as its

maximum amplitude. As a result, the avoid-obstacles behavior

gains higher priority as the robot encounters obstacles on

its path to the target. This behavior coupled with the avoid

obstacles behavior attempt to navigate the robot towards the

target without bumping into obstacles. The robot actually

tries to go around obstacles towards the target. Results of

our experimentation show the robot successfully navigating











around obstacles placed in its path. Keep in mind that some

topologies of the environment will trap the robot in

unproductive cyclic maneuvers when operating under the

target-nav behavior.



4.3 Arbitration Network

Behavior-based systems require some mechanism for

combining or arbitrating the outputs of the different

behaviors. In the case of a behavior-based mobile robot,

each behavior generates a heading for the robot, but the

robot can only accept one heading at a time. The resultant

or winning heading is determined by the arbitration

mechanism. One method of arbitration is the subsumption

architecture [Brooks 86a] where higher-level behaviors

subsume lower-level ones by suppressing their input or

inhibiting their outputs. Arbitration, however, can be

achieved in many ways. The simplest method uses a priority

list where the behavior with the highest priority on the

list gets control of the actuators. Another method

implements a strategy where once a certain behavior gets

control, it remains in control as long as it is active

regardless of other behaviors. Once the stimulus for

activating the controlling behavior disappears, the behavior

releases control of the actuators. Other strategies involves

combining the outputs of the various behaviors in some kind

of formulation, such as the potential field method [Arkin

87].










Our approach for behavior arbitration implements simple

binary all or nothing switches in the arbitration network,

with the arbitration control strategy incorporated into a

set of production rules, figure 4.5. In our implementation,

the rules reside in the planning module running under CLIPS,

a knowledge-based systems shell. CLIPS implementation of the

arbitration strategy is discussed in section 5.1. We believe

that by encapsulating the arbitration strategy in a set of

rules under the control of the cognitive system, a robot

useful for a wide range of missions can be created. The

rules incorporate knowledge about the goals, the individual

behaviors, the environment, and the current situation

status. Note that the real-time performance of reactive

control is still maintained, since the behavior-based system

is always enabled. The arbitration rules in the cognitive

system reconfigure the arbitration switches of the behavior-

based system depending on the goal at hand, and the current

situation. For example, in our experiments, we initially

configure the behavior-based system into an "explore" mode

allowing the robot to wander around without bumping into

things. After the initial configuration, the cognitive

system leaves the behavior-based system alone. Later, when a

knowledge source in the map builder module discovers large

boundaries between the empty and the unknown areas of the

occupancy grid representation being constructed, and that

such boundaries are not being traversed by the robot, this

information is made available to the arbitration rules in










the planning module and brought to bear on the behavior-

based system configuration by activating the target-nav

behavior with the center of the discovered boundaries as

target location. When the robot arrives at its target and

crosses the boundary, it is now reconfigured to discover the

new unknown areas.

Adhering to the biological analogy, the switches in an

arbitration network of a behavior-based system are not

purely binary all or nothing switches, but are active

elements with adjustable gains and thresholds, as modeled in

figure 4.6 by the operational amplifier circuit. The

programmable gain of the switch varies from a minimum to a

maximum allowable value. When the gain is either 0 or 1,

then a simple binary switch is obtained. The threshold is

another adjustable value above which the response of the

input behavior will produce an output. The programmable gain

and threshold constitute the basic parameters for adaptive

behavior and learning. The best values for such parameters

are not readily apparent, but can be fine tuned by some

adaptive learning algorithm or neural network. [Hartley 91]

proposes the use of "Genetic Algorithms" to accomplish such

tasks. Using the switch model of figure 4.6, the

implementation of an arbitration network involves the

knowledge of which behavior affects (adjusts the parameters

of) which other behavior, by how much, and in which

situations or context. The wiring of such an arbitration

network depends upon the goals or the desired competence of









Arbitration
An Arbitration Control
Rules


Behavior-n output I


Behavior-0 output I


Aritration Network
Arbitration Network


4-


a,
0
CD
C/


Actuators
-----BOB-


Figure 4.5 Arbitration by
superposition.


production rules and


Programmable
Threshold


Figure 4.6 Model of an arbitration network switch.










the robot. In addition, within a single system many

arbitration networks may exist, each servicing a sub-group

of behaviors that determine a certain competence level as

shown in figure 4.1. In our view, each competence level is

developed separately, and the thresholds and gains

determined experimentally (by using a neural network, for

example). The behaviors shown in figure 4.7 give a simple

example of one competence level, the "explore" competence

level used by the robot to explore unknown environments for

map building tasks and local navigation. As shown in figure

4.7, this level includes an "avoid", a "random-wander", a

"boundary-following", and a "target-nav" behavior. The

latter three behaviors are mutually exclusive while either

of the three is complementary to the "avoid" behavior. One

arbitration scheme for this subset of behaviors could give

the "target-nav" the highest priority of the three behaviors

with the "boundary-following" behavior having a higher

priority than the "random-wander" behavior. Thus, for

example, once the "boundary-following" behavior is

triggered, it inhibits the "random-wander" behavior causing

the output of the "boundary-following" behavior to be

combined (superimposed) with the output of the "avoid"

behavior which has the highest priority.























Figure 4.7 Behaviors of the explore competence level.


4.4 Man Builder: A Distributed Knowledge-Based Approach

The map builder generates two linked representations of

the world: a spatially indexed and an object or feature

indexed representation. The spatially indexed representation

consists of a 2-D tessellation of space where each cell in

the grid contains information about its state whether

occupied, empty, or unknown, in addition to information such

as what object the cell (if occupied) belongs to and whether

it is a boundary point or an edge point, etc... This

representation is useful for navigation in computing free

path, and for determining the identity of objects or

features in a given location. The object-indexed

representation is linked to the spatial representation and

contains entries such as the object's name, vertices,

bounding edges, and other discriminating features. This

representation is suited to responding to inquiries about

objects or features by name or by description.










We follow a knowledge-based approach as a framework for

the map builder since the goal of a multi-sensor system such

as the map builder, is not only to combine information from

multiple sensory sources, but also to combine this sensory

data with a priori domain-dependent and domain independent

knowledge. Thus, a powerful multi-sensor system must rely on

extensive amounts of knowledge about both the domain and the

problem solving strategy effective in that domain

[Feigenbaum 77]. This knowledge is needed to compensate for

the inadequacies of low-level processing, as well as to

generate reasonable assumptions for the interpretations of

features derived from lower level sensory data. Domain-

dependent knowledge consists of knowledge about domain

objects and ways of recognizing them. This involves semantic

descriptions of objects, semantic relationships between

objects, the use of interpretation context, experimentally

derived classification functions, and knowledge about the

task and the sensor. Domain-independent knowledge involves

general principles such as perspective distortion,

occlusion, varying points of view, etc..

The challenge of multi-sensory perception requires a

flexible inference strategy that supports both forward and

backward chaining. This flexibility allows the system to

dynamically alternate between data-driven and model-driven

strategies as the situation requires. The blackboard

framework allows for this flexibility [Nii 86a,b]. In a

blackboard framework forward chaining and backward chaining










steps can be arbitrarily interleaved. In addition, the many

knowledge sources have continual access to the current state

of the blackboard, and thus, can contribute

opportunistically by applying the right knowledge at the

right time.

Our proposed Sensory Knowledge Integrator [Bou-Ghannam

90b] follows a blackboard framework. The Sensory Knowledge

Integrator is described in detail in a technical report[Bou-

Ghannam 90a]. Its highlights will be discussed in the

remainder of this section.

An intelligent multi-sensor system maintains an

internal description of the world which represents its "best

guess" about the external world. This world model is built

using sensory input and a priori knowledge about the

environment, the sensors, and the task. Thus, the problem of

constructing a world model representation involves two types

of information fusion: 1) Fusion of information from

multiple sensory sources, and 2) Fusion of sensory data with

a priori knowledge and object models [Kent 87]. This

representation of the world consists of both a spatial and

an object or feature-based representation. The spatial

representation classifies the world space as occupied,

empty, or unknown, and explicitly represents spatial

relationships between objects. The feature-based

representation associates each object with the set of

features that verifies the identity of the object.










In previous work [Bou-Ghannam 90a,b] we introduced the

Sensory Knowledge Integrator (SKI), a knowledge-based

framework for sensor data fusion. What follows in this

section is a brief review of SKI, figure 4.8. SKI organizes

the domain knowledge and provides a strategy for applying

that knowledge. This knowledge, needed to describe the

environment being observed in a meaningful manner, is

embedded in data-driven and model-driven knowledge sources

at various levels of abstraction. These knowledge sources

are modular and independent emphasizing parallelism in the

SKI model. The knowledge sources use algorithmic procedures

or heuristic rules to transform information (observations

and hypothesis) at one level of abstraction into information

at the same or other levels. Processed sensory data in the

observations database cause the execution of the data-driven

knowledge sources while model data in the hypothesis

database cause the execution of model-driven knowledge

sources. This execution produces new data (on the

observations or hypothesis database) which in turn cause the

execution of new knowledge and the production of new data

until a high level description of the environment under

observation is incrementally reached. These high level

descriptions comprise the robot's local world model that is

continually updated by new sensor observations.

Data in the observations database range from intensity

and depth arrays at lower levels of abstraction, to lines,

edges, regions, surfaces, at intermediate levels, to objects






91



and their relationships at higher levels of abstraction. The

partitioning of the data into application dependent

hierarchies or levels of abstraction is essential because it

makes it easy to modularize the knowledge base. Thus,

certain knowledge sources become associated with a certain

level of data abstraction and could only be triggered by

data on that level of abstraction. This eliminates the need

to match all the knowledge sources to all the data. The

hypothesis database contains hypothesized high level goals

with hypothesized sub-goals derived from these goals in a

backward (top-down) reasoning scheme. The sub-goals are

matched to facts at lower levels of abstraction to assert

the validity of the hypothesis.

The control module handles the conflict resolution

among knowledge sources and thus determines what knowledge

source or group of knowledge sources to apply next. The

control module monitors the changes in the observations/

hypothesis database along with the potential contributions

of the related knowledge sources in the knowledge base and

determines the next processing steps or actions to pursue.

In other words, the control module determines the focus of

attention of the system. It contains knowledge about the

"big picture" of the solution space and, hence, can resolve

conflict among knowledge sources triggered by the current

situation data. We implement the control module in CLIPS.

To clarify the theoretical concepts introduced above,

section 5.2 of the next chapter discusses the implementation


r




University of Florida Home Page
© 2004 - 2011 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated May 24, 2011 - Version 3.0.0 - mvs