The effects of sign symbol function, computer experience, and context on the interpretation of pictorial sign symbols us...

MISSING IMAGE

Material Information

Title:
The effects of sign symbol function, computer experience, and context on the interpretation of pictorial sign symbols used in a human-computer interface
Physical Description:
xi, 107 leaves : ill. ; 29 cm.
Language:
English
Creator:
Kamper, Robert J., 1949-
Publication Date:

Subjects

Subjects / Keywords:
Visual communication -- Data processing   ( lcsh )
User interfaces (Computer systems)   ( lcsh )
Communication -- Graphic methods   ( lcsh )
Signs and symbols -- Data processing   ( lcsh )
Curriculum and Instruction thesis Ph. D
Dissertations, Academic -- Curriculum and Instruction -- UF
Genre:
bibliography   ( marcgt )
non-fiction   ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 1993.
Bibliography:
Includes bibliographical references (leaves 99-106).
Statement of Responsibility:
by Robert J. Kamper.
General Note:
Typescript.
General Note:
Vita.

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001937417
oclc - 30899860
notis - AKB3542
sobekcm - AA00006097_00001
System ID:
AA00006097:00001

Full Text










THE EFFECTS OF SIGN SYMBOL FUNCTION,
COMPUTER EXPERIENCE, AND CONTEXT
ON THE INTERPRETATION OF PICTORIAL SIGN SYMBOLS
USED IN A HUMAN-COMPUTER INTERFACE














By

ROBERT J. KAMPER


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


1993






























Copyright 1993

by

Robert J. Kamper






























This dissertation is dedicated to my parents, Charles and Beatrice Kamper, to

my wife, Sondra M. Kamper, and to my children, Molly Kamper, Katherine Kamper,

and Michael Hoffmann.













ACKNOWLEDGEMENTS


I would like to extend thanks to the many people who have contributed to my

studies at the University of Florida and the completion of this dissertation project,

both directly and indirectly.

Thanks are expressed to my chairperson and the members of my doctoral

committee. Thanks are extended to Lee Mullally for his personification of the

principles and ideals that inspired me to pursue this degree. Thanks are also extended

to Jeff Hurt and Roy Bolduc for their enthusiasm and good humor and to David

Miller and Sandra Damico for their guidance. Thanks are extended to all members of

the committee for their patience and professionalism. I am privileged to have had

them on the committee.

Thanks are also extended to Sebastian Foti and Bob Melczarek for their

assistance in conducting the pilot project. I am deeply indebted to Dr. Wayne Wolfe

of the Office of Instructional Resources, and Nancy Sorkin and David Gruber of the

Faculty Support Center for their participation in both the formative and final stages of

this project.

Special thanks go to John Tyler for his criticism and review of early drafts of

this material and for his assistance in the development of the stimuli.








Finally, my greatest appreciation and thanks are reserved for my friend and

spouse, Sondra M. Kamper, without whose cooperation and support this endeavor

would not have occurred.















TABLE OF CONTENTS


ACKNOWLEDGEMENTS ...

LIST OF TABLES ........


. . . . . . . . iv

. . . . . . . . viii


LIST OF FIGURES ........................................ ix

ABSTRACT .......................................... x

CHAPTER 1 INTRODUCTION .............................. 1

Statement of the Problem ..................... ......... 1
Need for the Study .................. ................ 5
Definition of Terms .................................. 10
Limitations ......... ............................. 13
H ypotheses ....................................... 14
Sum m ary ........................................ 15

CHAPTER 2 REVIEW OF RELATED LITERATURE ................ 16

Knowlton's Taxonomy ................................ 18
Educational Media .................................... 27
Signs and Symbols, Pictograms and Pictographs ............... 31
Pictorial Sign Symbols in the Human-Computer Interface .......... 35
Computer Experience, Literacy, and Knowledge ............... 52
Summary ................... ..................... 56

CHAPTER 3 METHODOLOGY ............................... 58


Research Design ..........
Population and Sample ......
Materials and Measures .....
Procedure ..............
Summary ...............


.............



.............
.............








CHAPTER 4 RESULTS AND ANALYSIS ...........

Introduction ..........................
R results . . . . . . . .
A analysis ........... .... .. ...... .. ...

CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS


Introduction ..................
Findings ....................
Discussion ...................
Implications ..................
Recommendations for Future Research .
Summary ....................


REFERENCE LIST ......................................

BIOGRAPHICAL SKETCH ................................


............
............
............


I














LIST OF TABLES


Table page
3-1. Research Design ................................... 61

3-2. Expert Ratings of Pictorial Sign Symbols .................. 66

4-1. Analysis of Variance Summary Table-Between Subjects Effects ...... 75

4-2. Analysis of Variance Summary Table-Within Subjects Effects ....... 76

4-3. Means of Pictorial Sign Symbols by Sequence of Viewing .......... 79

4-4. Repeated Measures Analysis of Variance

Contrast Between Realistic Sign Symbols and Sequence . ... 81

4-5. Repeated Measures Analysis of Variance

Contrast Between Analogical Sign Symbols and Sequence ...... 82

4-6. Repeated Measures Analysis of Variance

Contrast Between Logical Sign Symbols and Sequence ........ 83

4-7. Means by Computer Experience Type ...................... 85














LIST OF FIGURES


Figure page
1-1. Example of a Realistic Picture ............................. 10

1-2. Example of an Analogical Picture ..... ......... ..... 11

1-3. Example of a Logical Picture ........................... 12

4-1. Means of Pictorial Sign Symbols by Sequence of Viewing ........ 80

4-2. Means by Type of Computer Experience ................... 84













Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

THE EFFECTS OF SIGN SYMBOL FUNCTION,
COMPUTER EXPERIENCE, AND CONTEXT
ON THE INTERPRETATION OF PICTORIAL SIGN SYMBOLS
USED IN A HUMAN-COMPUTER INTERFACE

By

Robert J. Kamper

August 1993


Chairperson: Lee J. Mullally, Ph.D.
Major Department: Instruction and Curriculum

This study was conducted to determine whether types of computer-related

experience, viewing context and sequence affect the ability to interpret different

functional types of pictorial sign symbols used in human-computer interfaces. Groups

of teacher education students viewed and attempted to interpret sets of pictorial sign

symbols representing common word processing menu commands, constructed

according to Knowlton's taxonomy of functional picture types. The statistical analysis

used a four-way ANOVA based on a 3 x 2 x 3 x 3 factorial design with repeated

measures. Factors measured included computer experience type (three levels),

context (two levels), pictorial sign symbol (PSS) type (three levels), and sequence of

viewing (three levels).








Main effects were observed for computer experience type, pictorial sign

symbol type, and sequence. A significant interaction was observed between the

sequence and pictorial sign symbol variables. There was no statistically significant

effect due to context as operationalized for this study.

Analogical PSSs were more effectively interpreted by subjects with all types of

previous experience, and logical PSSs were the least well interpreted by all subjects.

Realistic PSSs were interpreted more accurately during the second and third sequence

of viewing positions than during the initial presentation. Prior experience with a

graphic user interface positively affected the ability to interpret PSSs.

This study generated evidence that may be useful to those engaged in

designing visual messages within human-computer interfaces for teacher education

students. The results of this study provide support for the systematic use of

Knowlton's taxonomy in the design and evaluation of pictorial sign symbols, and the

limitations of this study indicate appropriate directions for further research. This

provides empirical information that should be useful to designers of computer

systems, designers of computer-based instructional systems, designers of educational

media, and educators using computers as tools to improve learning.













CHAPTER 1
INTRODUCTION



Designers of human-computer interfaces increasingly use pictorial sign

symbols called "icons" to communicate messages visually to computer users. Are

there differences within individuals in the ability to interpret these symbols due to the

functional type of sign symbol used? Do people with different levels of computer-

related experience differ in their abilities to interpret pictorial sign symbols

representing functions of a computer system? Does the context within which pictorial

sign symbols are viewed affect interpretation? This study was designed to find

answers to these questions.


Statement of the Problem


Many changes have occurred over the last century in the types of educational

media used in the schools. Books, globes, and maps were typical visual aids used

prior to 1900 (Reiser, 1987). The development of film in the early part of the

century created interest in and resulted in research into visual education (Hoban &

Van Ormer, 1950). Investigations into the educational effects of television

accompanied its widespread distribution during the 1950s.










During the latter third of the century, the development of computers and

information processing technology stimulated further investigations. In recent years,

investigations in educational media research have shifted from gross media

comparisons of the effectiveness of technological hardware devices to issues related to

instructional messages and learner characteristics (Clark, 1989). Some researchers

now view media as vehicles for the delivery of instructional messages to learners

(Clark, 1983). Educational media involve the symbol systems used, the messages

conveyed, the learning tasks required, and the characteristics of the learner (Salomon,

1974). Researchers investigating educational media should look at the symbol

systems and the conventions of visual symbols and symbolic codes used in media

(Clark & Salomon, 1986; Levie, 1978).

Learners need to be able to understand the symbol systems used in the human-

computer interface. Human-computer interfaces frequently include pictorial symbols

in addition to textual information. Media and psychological researchers have

produced a knowledge base and provided many guidelines for the creation of

communicative visual displays (Fleming, 1989; Fleming & Levie, 1978). Media

research has typically involved pictures with text. Researchers have not fully

investigated learning from pictures alone, which remains a topic of continuing interest

(Gagne, 1986).

The growing use of technology has also increased the need for communication

and instruction across cultural and linguistic barriers (Frutiger, 1989; Kolers, 1969;

Mangan, 1978; Modley, 1966; Wildbur, 1989). One approach to this problem has










been the use of pictorial sign symbols in human-computer interfaces (Apple Human

Interface Guidelines, 1987; Lodding, 1983; Nadin, 1988). Terms used to describe

pictorial sign symbols include graphic icons, icons, iconic signs, pictographs,

pictographic symbols, and picture symbols. For the sake of brevity, pictorial sign

symbols will be called "PSSs" throughout this paper.

Designers use PSSs to make the human-computer interface more easily

understood, learned, and used by novices and experts. One highly successful

approach to designing the human computer interface has been the use of a "collective

metaphor," such as the office desktop. Systems that use a collective metaphor

represent underlying objects or functions of the computer by incorporating PSSs into

the human-computer interface. The Xerox Star office automation system (Smith,

Irby, Kimball, Verplank, & Harslem, 1982) is the classic example of this type of

interface. Unfortunately, haphazard and inconsistent imitations based on marketing

pressures instead of the users' needs in the specific context have followed (Gittens,

1986; Nadin, 1988).

Investigators have often studied expert and technical users. As access to

information through computer systems continues to expand, the focus on computer

interfaces will move away from technical issues toward the users and the system

interface (Smithson & Hirschheim, 1990). Many issues related to user interfaces

await further investigation. Proponents of the use of graphics in the human-computer

interface recognize that PSSs are not a panacea; there are several problems associated

with using graphics (Shneiderman, 1987). Among these problems are (a) the visual










content can confuse as easily as it can clarify, (b) the meanings of graphic icons may

not be clear to all users, (c) the graphic representation when understood, may mislead

the user, (d) graphic icons may take up excessive space, and (e) manipulation of

graphic objects may be less efficient than keyboard input for some situations.

Computer users who are novices in the use of a system actively engage in

exploration and discovery in attempting to learn how to use a software application

program (Carroll & Mack, 1984). In this situation, the human-computer interface

becomes an educational medium that provides cues and feedback to the individual

learner seeking to achieve unique learning goals. Messages to and from the

underlying program, and the user's responses to the program create an interactive

learning dialogue. In the graphical user interface, PSSs frequently convey messages

between the system and the user. The first three items on Shneiderman's list of

problems with the use of graphics focus on the problem of encoding and decoding

messages. Concerns about display space and efficient input modes are secondary

issues.

Norberg (1966) noted a continuing need for educational media research that

explores the potential of iconic signs, or PSSs, when the problem involved "helping

the learner creatively construct a response" (p. 45), such as that described by Carroll

and Mack (1984). Until recently, most computer-based educational materials used

digital symbols (written text) to convey messages to learners. Media researchers have

not fully explored the area of PSSs dissociated from text. Conversely, human-

computer interface design has involved research into the effects of PSSs both with and










without accompanying text. The issues of encoding and decoding the messages

delivered through PSSs to computer users are primary problems in the use of graphics

in the human-computer interface (Shneiderman, 1987).

How does the functional type of picture affect the ability to interpret PSSs?

How does prior computer-related experience affect the ability to interpret PSSs? How

does the viewing context affect the ability to interpret PSSs? Are there interactions

between the functional type of sign symbol, prior computer-related experience, and

context that affect the ability of the learner to interpret the PSSs used?


Need for the Study


Pictures and pictograms that closely resembled the designated concepts

preceded and eventually developed into arbitrary symbol systems (Frutiger, 1989;

Knowlton, 1964). Ironically, the growing use of technology, made possible by the

use of arbitrary symbols in mathematics and science, has expanded the need for

universally understood non-arbitrary sign symbols in many areas of communication

and education (Frutiger, 1989). PSSs are increasingly used as a language-free

medium for communication in an attempt to meet this need (Easterby, 1970). The

increase in international transportation and trade during the latter half of this century

has expanded the move for standardization in the use of PSSs (Dreyfuss, 1966;

Medley, 1966).

The incorporation of PSSs or icons into graphical human-computer interfaces

has occurred primarily during the last decade (Israelski et al., 1989; Lodding, 1983;










Smith et al., 1982). Most human-computer interfaces had been text based before the

introduction of the Xerox Star office automation system, the Apple Lisa, and the

Apple Macintosh (Nadin, 1988).

If, as Olson and Bruner (1974) have suggested, intelligence is skill in the

symbolic systems employed in the media used in instruction, then literacy in a

medium depends on the ability to use the symbols employed in that medium

effectively. The ability to use computers and to learn through computer-based

instruction has been highly dependent on the ability to use written language. This has

led to concern that the increased use of technology and computer-based instruction

will strengthen rather than mitigate the literacy bias of schools (Clark & Salomon,

1986).

How will the trend toward human-computer interfaces featuring PSSs affect

learning and instruction? Will the current bias toward those skilled in abstract symbol

manipulation continue, or will the graphic user interface ameliorate the advantages of

written and verbal linguistic skills? How will the type of graphic objects used to

represent the functions and commands of the computer system affect the ability of

varied users to understand and use the system effectively? This study was conducted

to provide data relating PSS functional types to the ability of individuals to interpret

intended messages. This should provide empirical information useful to designers of

computer systems and software, designers of computer-based instructional systems,

and educators using computers as tools to improve learning.










The factors involved in investigating the symbol systems used in the human-

computer interface require a multidisciplinary approach. Theory and research from

the fields of computer science, instructional design, psychology, human factors,

sociology, communications, graphic design, and linguistics contribute to an

understanding of the basic issues involved in the design and evaluation of the human-

computer interface (Nadin, 1988; Shneiderman, 1987). Interface designers have used

semiotics, the theory of signs, as a bridge to link these disparate disciplines (Nadin,

1988).

Educational media researchers have frequently addressed the role of pictures

with text instead of the interpretation of PSSs per se, because pictures typically appear

in a verbal context (Knowlton, 1966; Levie, 1978). Studies of pictures used in direct

instruction have resulted in guidelines for the use of pictures to facilitate the

achievement of specific learning objectives (Dwyer, 1978; Kemp & Smellie, 1989).

The role of pictures in paired-associate learning has also been widely investigated.

This has led to the general conclusion that pictures are superior to words, and words

with pictures are superior to words alone in recognition and recall tasks (Pressley,

1977). Much of this information can be applied to the design of the human-computer

interface.

James Q. Knowlton (1964, 1966) created a taxonomy of pictures based on the

symbolic sign functions of pictures. Within this taxonomy there are three types of

pictures; realistic pictures, analogical pictures, and logical pictures. There has been

little research into the usefulness of this taxonomy. Researchers in this area have










reported positive results (Alesandrini, 1989; Hurt, 1987) but have not specifically

addressed the use of PSSs in the human-computer interface. Knowlton's taxonomy

was applied to PSSs as used in the human-computer interface in this study.

Beginning with the introduction of the Apple Macintosh computer in 1984,

graphic user interfaces featuring PSSs, commonly called icons, have become widely

available. PSSs are often thought to transcend the limitations of verbal languages, but

in actuality are not universally interpretable (Kolers, 1969; Mangan, 1978; Modley,

1966; Sukaviriya & Moran, 1990). Graphic user interface designers serving users

across national and cultural boundaries need to consider cultural differences in the

meanings inferred from PSSs. The ideal would be to create PSSs that do not rely on

verbal labels, and to incorporate symbolic conventions that are globally understood.

Researchers have investigated questions regarding the design of instructional

messages, pictorial symbols, sign systems, and the human-computer interface for

decades (Collins, 1982; Dreyfuss, 1966; Easterby, 1970; Fleming & Levie, 1978;

Kolers, 1969; Zwaga & Easterby, 1984). These researchers have generated many

guidelines for the design of pictorial symbols (Apple Human Interface Guidelines,

1987; Fleming & Levie, 1978, Gittens, 1986; International Business Machines

Corporation, 1991; Kemp & Smellie, 1989; Lodding, 1983). Designing these

symbols still appears to be more of an art than a science (Nadin, 1988). Recent

research results show that even when designers follow guidelines, the assumptions

underlying the design of pictorial symbols are not always supported (Baecker, Small,










& Mander, 1991; Blankenberger & Hahn, 1991). Much work remains to be done to

clarify the issues involved (Blankenberger & Hahn, 1991).

A person must have attained a concept before a symbol that refers to the

concept can be interpreted correctly (Knowlton, 1964, 1966; Levie, 1978). If PSSs

contribute to ease of learning, then what types of computer-related concepts must the

learner bring to the human--computer interface? Results of recent studies indicate

that prior experience has an influence on the ability to interpret the meanings of PSSs

used in the human-computer interface (Baecker, Small, & Mander, 1991).

Researchers and designers have recognized that different levels of computer users,

from naive through expert, may prefer different styles of dialogue with the computer

system (Shneiderman, 1987; Smithson & Hirschheim, 1990).

The meaning of a symbol may vary depending on its context (Mangan, 1978;

Ossner, 1990; Zwaga & Easterby, 1984). Specificity of context can affect the ability

to interpret a pictorial symbol (Cahill, 1975; Rogers, 1986). Spatial context appears

to have an effect on interpretative ability even when general conceptual knowledge is

high, in some contexts (Cahill, 1975). How will differing contexts affect the

interpretation of PSSs with varied functional types?

This study was designed to investigate the relationship of PSS type to the

ability to interpret typical messages employed within human-computer interfaces. The

results of previous studies have not supported theoretical assumptions and guidelines

commonly used in the design of PSSs for human-computer interfaces (Blankenberger

& Hahn, 1991). Some investigations into the improvement of the interface have








10

revealed that some widely used PSSs fail to communicate clearly (Baecker, Small, &

Mander, 1991). This study used Knowlton's taxonomy of pictures (Knowlton, 1966)

to categorize PSSs. Subjects' type of prior computer experience was used as a

blocking variable. The viewing context of PSSs was manipulated as a treatment

condition.


Definition of Terms


The following terms are used in this proposal as defined below and as

described by Knowlton (1966). These terms will be used to classify PSSs used in the

study.

A realistic picture represents

something in the world of visual

perception and refers to the type of

object portrayed. For example, a

graphic representation of a disk drive

may be used to refer to a computer

storage device. The graphic

representation resembles the appearance

of a computer storage device (disk Figure 1-1 Example of a Realistic Picture
of a computer storage device (disk

drive) and refers to the type of object portrayed. A PSS used to convey the concept

"Eject Disk" with a realistic picture could represent a disk in a disk drive and indicate









the direction of movement. See Figure 1-1 for an example of a realistic sign

symbolizing the command to eject a disk.

An analogical picture refers to

something by representing something

else in the visual world as an example

that is "like" the referent. The picture

refers to something other than itself to

communicate this. For example, a

graphic representation of a filing cabinet

may be used to refer to a computer
Figure 1-2 Example of an Analogical
storage device. The storage of data in Picture

electronic form on a computer storage device is analogous to the storage of data on

paper in an office filing cabinet. The picture of the filing cabinet does not look like a

computer storage device, but it is analogous to the computer storage device. A PSS

used to convey the concept "Eject Disk" with an analogical picture could represent a

crumpled piece of paper being thrown into a trash can. See Figure 1-2 for an

example of an analogical sign symbolizing the command to eject a disk. This

particular analogy may not be as clear as the filing cabinet analogy and may have

negative connotations. However, it is very similar to an analogical picture commonly

used to refer to the "Eject Disk" function in a commercially available microcomputer

operating system.










A logical picture is one in which

the elements are represented in an

arbitrary fashion. The graphic

representation refers to something other

than itself but does not resemble it.

There are three types of logical pictures.

These include pictures whose purpose is

to represent the relationships between
Figure 1-3 Example of a Logical Picture
elements, pictures whose purpose is to

illustrate or represent a theory, and pictures whose referent is an idea. An example

of the first type of logical picture is a road map. A road map illustrates the

relationships between cities and towns, but the small circles, squares, or stars used to

indicate cities bear no relationship to the actual city boundary lines. The relationships

of the cities on the map, and the contours of the lines representing roads connecting

them, do resemble the actual relationships. An example of the second type of logical

picture is a picture of the structure of an atom. A representation of an atom, with a

nucleus with electrons circling around it, does not resemble an actual atom. The

intention is to communicate visually a theory about atoms. An example of the third

type of logical picture, which refers to an idea, might be the placement of a sequence

of numbers in a grid with equal rows and columns to illustrate the concept of a

number squared. A graphic representation of a geometric square may be used to

refer to a computer storage device. The "Eject Disk" concept could be symbolized by










placing the square within an open-ended box and indicating the direction of

movement. See Figure 1-3 for an example of a logical picture symbolizing the

command to eject a disk. This could be considered an example of a logical picture

signifying relationships, if there is no intention to represent the actual physical

appearance of a computer storage device. A graphic representation of the directory

"tree" (file structure of a computer storage device) would be an example of a PSS that

refers to an idea. Although the logical arrangement of the data and files is

isomorphic with the tree or directory structure, in reality the files and data may be

non-contiguous, scattered in clusters over the physical surface of the disk.


Limitations


The interpretation of results from this study is subject to the following

limitations and assumptions. Any conclusions are limited to the population

represented by the sample. Generalizations to other populations can only be made

with much caution and subject to replication of results. The results of the measures

used in the study assume that subjects answered all questions independently, honestly

and to the best of their abilities.

The symbol sets used in this study were constructed to avoid possible prior

exposure to or experience with these specific sets. However, some correct

interpretations and incorrect interpretations may have been due to prior experience

with highly similar PSSs or symbol conventions. Prior computer experience was

modeled into the design of the study as a blocking variable to minimize this threat to








14
internal validity. The results of this study are limited to the variables as defined and

manipulated. Caution should be used in generalizing any results of this study beyond

the specific PSS sets used in this study.


Hypotheses


This study was conducted to find answers to questions related to the

interpretation of PSSs used in human-computer interfaces. Several assumptions about

the possible relationships among the variables were made based on theory and

previous research findings. Prior experience with a graphic computer interface was

assumed to have a positive effect on the ability to interpret PSSs, regardless of the

sign symbol type. Cues provided by a highly specific context were expected to have

a positive effect on the ability to interpret PSSs, regardless of the sign symbol type.

Therefore, the research questions were stated by the following null hypotheses:

Hypothesis 1. There will be no significant differences within subjects in the

ability to interpret different types of PSSs.

Hypothesis 2. There will be no significant differences in the ability to

interpret PSSs between subjects viewing PSSs within a high context condition and

subjects viewing PSSs within a low context condition.

Hypothesis 3. There will be no significant differences in the ability to

interpret PSSs between subjects with different types of computer experience.

Hypothesis 4. There will be no significant differences in the ability to

interpret PSSs within subjects due to the sequence in which PSSs are viewed.










Hypothesis 5. There will be no significant differences in the ability to

interpret PSSs due to interactions among the functional types of PSS viewed, the type

of computer experience, the viewing context conditions, and the sequence of viewing.


Summary


Learners using computers need to understand the symbol systems used in the

human-computer interface. The growing need for communication and instruction

across cultural and linguistic barriers has fostered the use of PSSs in human-computer

interfaces to convey information to users. A broad knowledge base exists for the

creation of communicative visual displays. Much is known about learning from

pictures with text, but educational media researchers have not fully explored learning

from pictures alone. The trend toward the use of PSSs in human-computer interfaces

has generated many investigations, but theoretical assumptions have not always been

supported by the results. Comprehensive prescriptive guidelines for the design and

implementation of PSSs have not yet been established.

The following chapter includes a review of selected theory and research

concerning what is known and what remains undetermined about the interpretation

and comprehension of PSSs, with particular emphasis on graphic computer interfaces.

The general literature review is followed by a discussion of the specific studies that

led to the questions addressed by the current study.













CHAPTER 2
REVIEW OF RELATED LITERATURE


Research into the use of pictorial sign symbols (PSSs) in the human-computer

interface has lacked a unified theoretical basis. Writers in the educational media

literature have recycled the basic principles of instructional message design developed

over years of research (Aspillaga, 1992; Grabinger, 1989; Hannafin & Hooper, 1989;

Morrison, Ross, O'Dell, Schultz, & Higginbotham-Wheat, 1989), or incorporated

results from human factors research, human-computer interface theories, and graphic

arts principles (Lucas, 1991; Reilly & Roach, 1986). Although much of this material

is useful, none of it has focused on the symbol systems used within the interface.

Experts have proposed taxonomies for PSSs icons used in the human-computer

interface (Gittens, 1986; Lodding, 1983), but have not systematically investigated the

implications for application. Similar taxonomies, using categories such as

"representational," "abstract," and "arbitrary" (Lodding, 1983); "associative,"

"literal," and "abstract" (Gittens, 1986); "concrete object," "concrete analogy," and

"abstract symbol" (Rogers, 1986); and "iconic," indexicall," and "symbolic" (Nadin,

1988) have been used to classify human-computer interface icons in similar ways, but

there has been no consensus of terms or definitions. This lack of consistency makes

it difficult to generalize the results from one study to other situations, groups, or








17

symbol sets. Nor have research results clearly supported the theoretical assumptions

used for design (Blankenberger & Hahn, 1991).

Mihai Nadin (1988) argued for the use of semiotics, the theory of signs, in the

design of the human-computer interface, as was done in the improvement of the

original Macintosh interface. Jervell and Olsen (1985) argued that semiotic analysis

of icon designs indicated that these PSSs were not precise in meaning and that context

would affect interpretation. The continuing trend towards the use of PSSs in the

human-computer interface suggests that it might be useful to investigate whether some

PSSs are more effective in conveying messages, and whether there are any systematic

effects of context on interpretation.

Knowlton's (1966) semiotic-based taxonomy, using the functional categories of

"realistic," analogicall," and "logical" to classify pictures, is the theoretical basis for

the primary variables manipulated in this study. Therefore, this review of related

literature will begin with an exposition of James Q. Knowlton's semiotics-based

taxonomy of pictures. Selected research related to the use of pictures in educational

media follows. Human-computer interface designers have drawn upon human factors

research involving PSSs used in traffic control, public information, and instrument

display panels. A review of pertinent results in this area precedes selected research

on human-computer interaction. Finally, the discussion focuses on specific studies

related to the issues and questions addressed in this study.










Knowlton's Taxonomy


The use of signs is a predominantly human trait. No other animal deals with

signs with equal complexity and elaboration; signs are basic to human communication,

society, civilization, and science. A sign is, in common language, something that

stands for or refers to something for someone (Morris, 1938).

The "something" that acts as a sign is called the designatum, object, or sign

vehicle. The "something" that a sign stands for or refers to is the referent. The

"someone" to whom the sign vehicle represents the designatum is the interpreter; the

interpretant is the effect on the interpreter (Knowlton, 1966; Morris, 1938; Nadin,

1988). Knowlton classified signs produced with the intent to communicate as symbols

(Knowlton, 1964, 1966). Pictorial sign symbols are pictures, graphic visual displays,

or visual-iconic representations produced with the intent to communicate by acting as

sign vehicles.

For example, a picture of an electronic hand calculator within the visual

display of a human-computer interface may refer the user to a program that emulates

a hand calculator or to a mathematical calculation function; the pictorial representation

chosen will influence the user's interpretation (Nadin, 1988). A program that

emulates the capacity of a simple hand calculator may visually display a

representation of the calculator on the screen. The visual display of a calculator is

the sign vehicle. The underlying function of the computer system is the object or

designatum. The interpretant, or effect produced on the interpreter will depend on

the prior knowledge and experience of the interpreter, as well as the particular context










within which it appears. Within a consistent desktop office environment metaphor,

this symbol might be interpreted as a function that emulates a hand calculator. Within

a word processing or spreadsheet application, a calculator icon might be interpreted as

a mathematical equation editor, a recalculation function button, or a feature providing

access to built-in math and financial functions. Whether the interpreter correctly

infers the referent of the PSS may depend on interaction between the prior knowledge

and experience of the interpreter and the context within which the sign is perceived.

The "something" that a sign refers to does not have to be a tangible or even a

real object-the referent or designatum is not a thing, but a category or class of

objects (Knowlton, 1966; Morris, 1938). Sign categories that refer to referent

categories are defined by their attributes, the relevant cues that allow discrimination

among categories (Knowlton, 1966). Knowlton identified two types of attributes.

Formal attributes refer to the perceptual qualities of a sign or an object, such as color,

size, and weight. Criterial attributes refer to those features used as keys for sorting

and discriminating between classes. Criterial attributes depend upon the context

within which a sign is perceived. Formal attributes such as size and shape would be

non-criterial for separating limes and lemons. Color would be a criterial attribute for

this task.

Signs can be categorized along the continuum of generality and specificity as

equivalence and identity categories (Knowlton, 1966). Equivalence categories consist

of broad or generic terms such as "animal," "man," "statue" or "house." The

members of this type of class of items are perceived as separate entities that share a








20

common attribute or set of attributes. An equivalence category would be "statues" or

"houses." Elements of this category would be seen as different entities, sharing a

common definition. The identity category consists of items such as "The Statue of

Liberty" or "The White House." Elements of this category would be considered as

different aspects of the same entity.

Pictorial sign symbols used in a human-computer interface are often classified

according to their relative position along a continuum of concreteness and abstractness

(Nadin, 1988, p. 67). Concreteness used in this sense indicates a high level of

iconicity. Concrete images bear a high visual resemblance to the thing or object

depicted, while abstract images have less detail. Concrete images tend to be iconic,

and their interpretation is based on recognition. Abstract images tend to be more

symbolic, and a greater amount of convention is involved in their interpretation.

Each of the items signifying "The Statue of Liberty," for instance, can be

considered media for transmitting a concept, or shared meaning. The medium,

whether photograph, drawing, silhouette, spoken, or written word, is the sign vehicle

that delivers the message to the recipient. The shared meaning (conventional response

and interpretation) is the sign, or sign symbol, in Knowlton's taxonomy. We do not

usually make a verbal distinction between the medium and the message by

differentiating between the sign vehicle and the sign symbol (Knowlton, 1966). For

this study, the nature of the sign vehicles used is indicated by use of the term pictorial

sign symbol.








21

An iconic sign is one that resembles that for which it stands (Knowlton, 1966;

Nadin, 1988). A digital sign bears no resemblance to its referent, and is arbitrary,

determined by convention. Examples of digital signs include words, numbers, sign

languages, and codes (Knowlton, 1966). A pictorial representation of a calculator

would be an iconic sign. Representations of the symbols used to represent

mathematical calculations for addition, subtraction, multiplication, and division, or the

word "calculator" would be digital signs.

Knowlton noted that sign vehicles high in iconicity will generally depict

identity categories, while single words typically depict equivalence categories. In

order to broaden the message conveyed and interpreted by a picture, the picture

would be stripped of detail until the criterial attributes alone remained. Examples of

the continuum of concreteness to abstraction are provided by the evolution of

characters in a pictographic/ideographic writing system such as Chinese (Knowlton,

1966), the evolution of a pictorial representation of a calculator (Nadin, 1988, p. 71),

and the evolution of the phonetic character "A" (Frutiger, 1989, p. 145). Pictures that

are concrete, realistic, and detailed representations tend to be interpreted as individual

entities within the identity category, while pictures stripped of detail or local features

tend to be interpreted within the equivalency category.

The aspects of Knowlton's taxonomy described above are common concepts in

the field of semiotics. There are two aspects of Knowlton's taxonomy that do offer a

unique scheme for categorization. First is the specification of the varied functions of

pictures as visual-iconic signs into three different categories. The second aspect is that










this classification of pictures by their functional meanings is largely independent of

the physical attributes of the pictures.

Visual classification schemes may discuss physical characteristics such as size,

shape, and color (e.g., Gittens, 1986). If the issue of interest is the interpretation of

the meanings of symbols, however, a taxonomy based on physical attributes is

inadequate. Knowlton classified pictures as visual-iconic signs according to the

function of the picture as realistic, analogical, or logical, independently of the

physical attributes of the symbol. To classify a PSS within Knowlton's taxonomy,

one must consider the verbal context within which the PSS appears.

The parts of a visual-iconic sign include the elements, their pattern of

arrangement, and their order of connection (Knowlton, 1966). For example, in a

diagram of a circuit board, the elements are the memory chips, transistors, resistors,

capacitors, and other items. The pattern of arrangement would be the spatial layout

of these items in the diagram. The order of connection would be the sequence in

which these elements are connected. Knowlton considered a visual representation to

be a visual-iconic sign if the representation of at least one of the three categories of

parts was nonarbitrary. The types of representation are iconic, analogical, and

arbitrary. Pictures with iconic representation of elements are categorized as realistic

pictures. Pictures with iconic elements that use real world objects to refer to other

objects, actions, or concepts are categorized as analogical pictures. The picture

element may be a representation of a concrete object, but it is used to refer to








23

something else. Pictures with arbitrary representation of elements are categorized as

logical pictures.

Knowlton's taxonomy is based on the function of the PSS in communicating its

meaning, instead of its physical characteristics alone. An iconic representation of a

hand calculator that refers to a computer program that faithfully replicates the

appearance and functions of a hand calculator could be considered a realistic picture.

It conveys its meaning by directly resembling its referent. An iconic representation of

a hand calculator that refers to a spreadsheet function such as "recalculate" would

probably be considered an analogical picture. Although it is an iconic representation

of a concrete object, it conveys meaning through analogy. The analogical picture

uses the real world object (calculator) to refer to something other than itself (the

process of recalculating the values of all the cells within the program data) to

communicate this. An iconic representation of a mathematical square root symbol

that refers to a group of built-in functions would be considered a logical picture,

because the symbol is arbitrary and does not "resemble" a square root.

The critical difference in Knowlton's taxonomy is the classification of PSSs by

their functions instead of solely by their physical attributes. Knowlton (1966) noted

an illustration of two lumberjacks working together to support or move a felled tree.

The picture of the lumberjacks and the trees is meant to depict through analogy how

muscles are attached to the bones of the skeleton and work together to move in

different directions. Although the physical attributes of the picture might classify it as








24
a realistic depiction of lumberjacks, the function of the sign vehicle in this context is

such that it is categorized as an analogical picture.

Another detail in Knowlton's taxonomy that other schema lack is the

identification of elements, pattern of arrangement, and order of connection in

determining to which category a picture belongs. Assignment of a picture to a

realistic, analogical, or logical sign symbol category depends on the functional

representation of the elements of the picture. The pattern of arrangement and the

order of connection of the elements may or may not be congruent with the function of

the elements. For example, arrows are often used within PSSs. An arrow may be

used in a sign symbol as an element or as part of the pattern of arrangement or order

of connection. An arrow that is itself a PSS intended to communicate the message

"Advance to next screen" is an element, and is an example of a logical picture. An

arrow that is part of a PSS portraying a piece of paper being thrown into a trash can

is an arbitrary portrayal of the pattern of arrangement (the paper is outside the trash

can, traveling on a trajectory that will end inside the trash); the symbol itself is an

analogical picture for the "delete" function. An arrow that is part of a PSS consisting

of a rectangular area, a document, and an arrow pointing from the document to the

rectangular area would represent the order of connection, or the flow of data between

the elements. A bi-directional arrow in such a display might indicate a bi-directional

connection between the document and the rectangular block.

Semiotics theory developed by C.S. Peirce identifies three kinds of signs:

iconic, indexical, and symbolic (Berger, 1989; Nadin, 1988). An iconic sign










signifies by resemblance to the thing for which it stands. This is consistent with

Knowlton's taxonomy. An indexical sign is a sign that signifies through a logical

connection to the referent (Berger, 1989) or through a representation of something

causally influenced by the referent (Nadin, 1988). A symbolic sign signifies through

convention and must be learned (Berger, 1989; Nadin, 1988). In the example of the

calculator, an indexical sign symbol could be a picture of a paper printout ribbon,

showing the results of the calculations. A symbolic sign symbol could be a picture of

the conventional mathematical symbols used to represent addition, subtraction,

multiplication, and other functions performed by a calculator, according to Nadin.

Nadin presented a table of varied PSSs representing a hand calculator with

iconic, indexical, and symbolic representations on one axis and a continuum of

concreteness-abstractness on the other axis (Nadin, 1988, p. 67). Further analysis

reveals that the example of an indexical sign provided by Nadin does not differentiate

between the iconic and indexical representations, but represents a calculator by

synecdoche, in much the same fashion that a handset represents a telephone.

Realistic, analogical, and logical pictures can be used effectively in instruction,

but there has been no systematic investigation using Knowlton's taxonomy

(Alesandrini, 1989). Alesandrini discussed the use of learner-generated realistic,

analogical, and logical computer graphics to enhance learning of science topics, but

did not address the design of the human-computer interface. Hurt (1987) has shown

that the type of information learned and recalled from text can be affected by the

functional type of picture accompanying the text. This suggests that an analysis of the








26

type of PSSs used in the human-computer interface could be used to align instruction

with the type of sign presented by the computer. If the print function is represented

by a realistic PSS of a printer, then instruction should be most effective if it

emphasizes that this symbol is used to send information to the printer to be printed

out on paper. If an editing function is represented by analogical PSSs of scissors and

a paste jar, then instruction should be most effective if it emphasizes that moving and

relocating information within the computer file is analogous to cutting out a part of a

printed text document with scissors and pasting it in a different location within the

overall document. If the online help function is represented by a logical PSS of a

question mark, then instruction should be most effective if it emphasizes the link

between the question mark, an arbitrary symbol used to represent inquiry,

inquisitiveness, or confusion, and the process of seeking information, directions, or

help by asking questions. These assumptions could be tested empirically. This

study's purpose is to investigate the effects of functional types on the interpretation of

PSS, not to confirm the generalizability of previous research to the task of learning to

use a computer.

To summarize, the classification of a picture according to Knowlton's

taxonomy depends on the intended function of the sign as a realistic, analogical, or

logical sign symbol, and not solely on the physical attributes of the picture

(Knowlton, 1966). The classification also depends on the signification of the elements

as iconic, analogical, or arbitrary, and not their patterns of arrangement or order of

connection. This classification was chosen for this study due to the potential to










describe, explain, and predict more fully the interpretation of PSSs used in human-

computer interfaces. The commonly found bipolar schema of concrete-abstract and

realistic-symbolic visual iconic signs are compatible with, but do not describe the

functions of PSSs as comprehensively as Knowlton's taxonomy. Knowlton's

taxonomy will be used later in this review to analyze specific research findings related

to the use of PSSs in the human-computer interface. At this point the review will

turn to a consideration of the educational media literature as it relates to the study

topic.


Educational Media


Interest in the use of PSSs has been evident in the educational media and

instructional design field for several decades (Knowlton, 1964, 1966; Levie, 1987;

Norberg, 1966). Overall, questions have shifted from the study of stimuli within the

behaviorist paradigm to the exploration of learner characteristics and internal

cognitive processes (Clark & Salomon, 1986; Olson & Bruner, 1974; Salomon,

1974).

Many of the principles derived from psychological research and applied to

educational media (Fleming & Levie, 1978) also apply to the design of the human-

computer interface and PSSs used in the interface. For example, Fleming and Levie

made the following generalizations from basic research in the behavioral sciences:

1. Material that is meaningful to the learner is acquired and retained more


easily.










2. Concreteness facilitates learning and remembering.

3. Pictures are more easily remembered than words, and concrete words are

more easily remembered than abstract words.

4. Combined text and graphics should be congruent or redundant, so that the

learner receives the same message regardless of the channel of communication

attended to.

5. Messages to the novice within a domain should begin with the more

concrete and move to the more abstract as the learner increases knowledge.

6. Cues that are familiar or direct attention to relationships can facilitate

learning.

7. Discriminations are learned most readily by progressing from maximal

differences to finer differences.

These principles or generalizations, based on a variety of studies, are among

those applicable to the analysis of the human-computer interface. They are restated

below as guidelines for designing PSSs for human-computer interactions.

1. Use images that are familiar to and meaningful to the user/learner.

2. Use concrete images rather than abstract images.

3. Use pictures rather than words when possible, and use concrete words

rather than abstract words.

4. When combining text with a graphic, both text and graphic should convey

the same message.










5. Pictorial sign symbol sets for novices should use more concrete images;

pictorial sign symbol sets for experts may use more abstract symbols and images.

6. Include familiar or directional cues to facilitate interpretation of pictorial

sign symbols.

7. Menu and command options should use images that are maximally different.

Simply adapting guidelines established with print-based media may not result

in the same effect on learning using computer screens (Hannafin & Hooper, 1989).

Without systematic research directed towards screen design issues, instructional

designers need to be cautious in the application of intuition, common sense, or rules

developed with print media to a more interactive interface. The results of recent

research have supported and contradicted guidelines and assumptions. Specific issues

will be noted later within this review.

The educational media literature regarding computer screen design typically

addresses recommendations regarding textual issues (Grabinger, 1989; Morrison et

al., 1989). Recommendations include admonitions to use lower case text and ragged

right margins for ease of readability, use appropriate amounts of white space between

lines, and provide a visual structure for the learner (Aspillaga, 1992). Though

previous research involving typographic features has informed suggested guidelines

for computer-based instruction, the results do not always meet expectations. For

instance, when provided sufficient contextual support, some subjects prefer a

computer screen that has more, rather than fewer words, as suggested by previous

research (Morrison et al., 1989).








30
Recently, attention has moved toward the visual design of the human-computer

interface (Lucas, 1991; Reilly & Roach, 1986) and toward the use of visuals with or

without computer displays (Frascara, 1984). Writers in this field caution against

adherence to simplified rules (Hannafin & Hooper, 1989). Greater attention is needed

for the relationship between task attributes and the capabilities of learners (Hannafin

& Hooper, 1989). There are many topics to be explored in this area, including icons

or PSSs; future research should focus on smaller subtasks rather than overall learning

(Grabinger, 1989).

Inquiries into the human-computer interface may use information produced

through studies involving familiar media. For instance, Beck (1991) relates the

results of studies involving text and arrow cues in pictorial and non-pictorial

materials. In one study, combinatorial cues (text and arrows) benefitted both above

and below average reading ability groups. This suggests that in addition to

identifying labels, arrows used to portray a pattern of arrangement or order of

connection as parts of PSSs in the human-computer interface could benefit viewers

with varied levels of experience related to computers.

The knowledge gleaned from educational media research often applies to the

design of the human-computer interface. Designers of computer-assisted instruction

have been urged to follow guidelines developed for text design in developing screen

displays (Duin, 1988). With purely textual interfaces, this may be appropriate.

Designers need to be careful in extending pictorial-related research to computers,

since recent results have occasionally appeared to contradict previous findings.








31

The use of PSSs in the human-computer interface often differs from traditional

uses of pictures in instruction. Traditionally, pictures accompany text; the

instructional message is often embodied in text and merely supplemented by the visual

display. PSSs in the human-computer interface are sometimes displayed without text

labels, are used as commands to enable the user to perform a task, and are messages

(instructional or informational) in themselves. In this respect, these PSSs are related

to pictographic and public information symbols. The literature in this area should

contribute to an understanding of PSSs used in human-computer interfaces.


Signs and Symbols. Pictograms and Pictographs


In a preliterate or illiterate culture, pictorial signs served to locate services in

towns, such as barbers' or bakers' shops, to the stranger or traveler. In the modem

world of global transportation, many people find themselves in locations where they

are linguistically illiterate, and must rely on PSSs to locate baggage, money

exchanges, or restrooms (Modley, 1966). International trade, travel, and

communication have created a need for standardization of symbols and proposals for

universal graphic symbol systems (Collins, 1982; Dreyfuss, 1966; Easterby, 1970;

Modley, 1966).

The modern attempt to develop this visual mode of communication began with

traffic signs, public information signs, and instrument controls before the development

of graphical user interfaces for computers. Kolers (1969) recommended that PSSs use

realistic pictures of technological objects, rather than analogical pictures of naturally










occurring objects, on the assumption that different cultures may interpret natural

objects differently, but new technology is used consistently across cultures. For

example, a picture of a telephone would be universally understood to anyone familiar

with the use of a telephone, while a picture of a penguin might elicit different

interpretations from a stock person in a grocery store and an ornithologist in a zoo.

Using the symbol set designed by Dreyfuss (1966) for Deere & Co., Cahill

(1975) tested whether the visual context of the symbol and prior experience of the

viewer affected the interpretability of sign symbols. Thirty mechanical engineering

majors were presented with a set of ten symbols used to label parts of heavy

automotive machinery and asked to name the part or function each represented. Half

of the subjects were provided with a numbered drawing indicating the positions of the

symbols and the corresponding instruments within a cab interior. The results

supported the view that context contributed to the meanings inferred from symbols.

Prior experience with the same type of machinery also made some difference.

Cahill made no distinctions regarding the functions of the pictorial symbols.

Analysis of the stimuli using Knowlton's taxonomy yields interesting results, but no

specific pattern. The symbol for the choke function was a realistic sign symbol of a

butterfly valve, the symbol for the "engage" function was a logical sign symbol.

Neither of these two symbols were interpreted correctly. A realistic sign symbol for

the hand brake, an analogical sign symbol for the car horn, and a logical sign symbol

for the turn signals were all interpreted correctly regardless of prior experience.

These findings run counter to general guidelines that PSSs for concrete objects are










more easily interpreted than abstract concepts, as well as Koler's guideline that

technological objects should be interpreted more consistently than other types of

representations.

PSSs used in automotive controls were the materials for a study conducted by

Green and Pew (1978). In this study, subjects were drawn from a pool and did not

represent a special population. Given an imaginary scenario to provide a context and

provided with the complete set of 19 symbols, the subjects were asked to point to a

specific symbol. Only six of the symbols met an acceptance criterion set at 75%

correct responses. Of these, four were realistic sign symbols for the seat belt, front

and rear hoods, ventilating fan, and the battery representing the charging circuit. The

analogical sign symbol of a bugle referring to the car horn, and a logical sign symbol

consisting of the mnemonic letter "P" with lines representing light rays emanating

from it (signifying the parking lights) were also easily comprehended. A sex effect

(males interpreted symbols better than females) and a sex x technical training

interaction (male students majoring in engineering interpreted symbols better than

males majoring in non-technical disciplines) was observed. The sex effect was

spurious, as the higher overall performance of males was attributed to those with

technical training.

Mackett-Stout and Dewar (1981) studied different public information signs

intending to convey the same message. However, there was no systematic variation

of symbols to provide specific categorical exemplars for each message. They

concluded that abstract symbols are much more difficult to comprehend than iconic








34

pictographs. In contradiction to their conclusion, however, the logical sign symbol of

a question mark representing the concept of "Information" was chosen as the best

overall symbol for that message.

The use of pictographic materials to communicate instructions for equipment

operation was the subject of experiments conducted with an artificial piece of

equipment (Marcel & Barnard, 1979) to explore how people comprehend and use

pictorial instructions. The task given to subjects was to view a set of pictorial

instructions and then either to describe the instructions verbally so someone at the

other end of a telephone could follow them correctly or to perform the instructions on

the equipment. Instructions that showed the entire apparatus were easier to

comprehend than those that showed only the relevant parts to be acted on at a

particular step. The attempt to focus attention by excluding irrelevant parts was

overridden by the loss of the overall frame of reference and visual-spatial

relationships of the parts. Fewer relational misinterpretations occurred when the

subjects actually operated the equipment. Marcel and Barnard noted that people

tended to interact with the equipment in a problem-solving manner, drawing

inferences from the results of the interactions, instead of reading the instructions first

and then using the instructions to guide their actions. They also indicated that people

tended to operate on the instructions in a piecemeal or syntactic fashion and only

grasped the underlying semantics of the instructions when asked to verbalize the

instructions. Marcel and Barnard suggest that instructions be designed based on

questions that users have for particular purposes rather than on a task analysis of how








35
the equipment works, and concluded that the intelligibility of such instructions cannot

be assessed outside the appropriate context.

In summary, studies into the use of PSSs in the form of pictographs for

language-free communication have not yielded conclusive recommendations for

general application. The evidence indicates that both an individual's prior knowledge

and experience in a domain, and the context within which the symbols appear affect

comprehension. The focus of this paper now shifts to the human-computer interface.


Pictorial Sign Symbols in the Human-Computer Interface


The shift in concern from sheer technological power to human factors concerns

of ease of use and learning occurred in computer science as tremendous technological

advances made dramatic cost reductions possible (Shneiderman, 1984). The

expansion of computer users beyond the technically trained to include novices and

non-technically trained people has been a major force towards the concern for human

factors issues in the human-computer interface. Increasing dependence on interactive

computer systems for organizational operations and communications as well as critical

needs for error-free operation in medicine, air-traffic control, emergency services,

and space and military applications have also been important. In addition to computer

science, the disciplines of cognitive and educational psychology, instructional design,

graphic design, technical writing, and human factors are all involved in the discovery

of solutions to problems of human performance in interaction with computer systems










(Shneiderman, 1987). Systems involving direct manipulation of PSSs are a

manifestation of this effort.

The use of PSSs or icons, as they are usually referred to when used in the

human-computer interface, grew out of human factors research in man-machine

interfaces. The term "icon" as originally used in the computer interface was an

analogy to religious icons, which were considered to embody the properties of the

saints they represented (Perry & Voelcker, 1989). Icons in the computer interface

were treated as objects that contained all the properties and attributes of the programs

and data represented, and could therefore be acted on as if they were the actual

programs or data. For example, the act of dragging a file into a trash can on the

computer screen represented the act of deleting that file. A more recent development

has been the use of PSSs to represent commands or functions or menus. Choosing

(through pointing and clicking) a command would carry out that command on any

selected items on the screen. Although the functions of these graphics may vary, the

term "icon" has become commonplace, and is used in the references cited below.

Throughout this section, the term "icon" will be used interchangeably with the term

"pictorial sign symbol" to avoid redundancy.

The application of design principles to a computerized office system led to the

creation of the Xerox "Star" system (Smith et al., 1982). Human factors testing of

the Star interface icons included paper and pencil naming tests, timed tests to measure

recognizability and distinguishability, and ratings tests to obtain subjects' opinions

(Bewley, Roberts, Schroit, & Verplank, 1987). The icons tested were all objects,










such as document, file, and printer; icons representing commands were not

incorporated into the desktop metaphor. In Knowlton's taxonomy, the entire desktop

display and its contents would be considered as analogical sign symbols. Subsequent

graphical user interfaces have been criticized for imitation of the desktop metaphor

without adequate definition of intended users and appropriate interface features

(Gittens, 1986; Nadin, 1988).

Lodding (1983) proposed three characteristics of an optimal icon. First, the

viewer should be able to infer its meaning on the first encounter. Second, when

selecting from a menu or set of choices, only one icon should be appropriate; and

third, the icon should not convey any negative connotations.

An example of an interface that goes beyond the desktop metaphor is that of

the "book house" in which users may search for books by interacting with icons as an

alternative to the traditional keyboard based command dialog system interface seen in

typical library automation systems (Pejtersen & Goodstein, 1990). The metaphor

chosen for this database was that of a "book house." After a user chooses to interact

with a mouse input device, a screen appears that shows three rooms with

representations of two children, an adult and a child, and two adults, respectively,

standing in front of the rooms. At this point the user can choose to work with the

children's book database, the total database, or the adult books database by clicking

the mouse in the appropriate area. Upon entering a room, representations of people

performing various search strategies are shown on the display to indicate the search

strategy alternatives available. The types of searches available are analytical search,










search by analogy, browsing in pictures, and browsing in book descriptions. The

analytical search provides indices to the user, in the form of an open book. The

search by analogy allows the user to select a book title and then searches for books

that are similar to the one chosen. The picture browse search allows the user to pick

PSSs that appear interesting, and then presents titles within that topic. The book

description browse provides random selected book descriptions from the data. With

each of these types of searches, Boolean logic can be applied.

Testing for meaningfulness of icons used in this interface involved trained

librarians, adult users, and children. The pictorial symbols were each displayed for

25 seconds, followed by a list of several intended subject areas. The subjects then

identified the concept communicated by the icon from the list of intended subject

areas. Although specific data are not reported, the authors indicate that some icons

were interpreted strongly as only one of the terms provided, others were interpreted

in a more varied manner. Icons that directly represented the object signified were

easily interpreted. Children's, and to a lesser degree, women's responses were more

widely distributed than men's responses. Due to these results, the search terms used

in the children's book database for the same PSS are different from those used in the

adult book database (Pejtersen & Goodstein, 1990). Caution must be used in drawing

any conclusions from this report, as it provides examples only, and lacks any

evaluative data from actual use. Another limitation in drawing conclusions from this

and other studies is that direct representation of a concrete object, as often described,

does not tell us whether the visual representation functions as a realistic or an










analogical sign symbol. It is interesting to see the possibilities, potential, and

problems in applying a complete metaphor involving PSSs (other than the desktop

metaphor) to the human-computer interface for an information processing application.

Apple Computer has made a strong effort to incorporate PSSs into its

Macintosh desktop interface metaphor in a consistent manner (Apple Human Interface

Guidelines, 1987; Nadin, 1988). Recent studies conducted by the Apple Human

Interface Group (Baecker, Small, & Mander, 1991) indicate that some currently used

icons are less than optimal. Although the main focus of their study was to explore

the use of animated icons, several static icons already in use were consistently

misinterpreted by the subjects, notably the "paint bucket" tool. This tool's function is

to fill an enclosed area with a selected pattern. Clear differences were seen between

experienced and novice users; however, the limited number of subjects and

exploratory nature of the study limit conclusions to be drawn from this study. The

paint bucket tool, interpreted incorrectly by all novice subjects, is a "concrete" image

in most classifications. Generally, concrete images and messages are recommended

for novices (Fleming & Levie, 1978). Although it is an iconic representation of a

paint bucket, its function is analogical, in that the computer function that it refers to

produces an effect that is analogous to the effect produced by pouring paint onto a

surface. In Knowlton's taxonomy the paint bucket tool would be classified as an

analogical picture. An adequate theoretical model to describe, explain, and predict

the meaningfulness of PSSs would help to deal with data that apparently contradicts

earlier findings.










Researchers into the human-computer interface have often used technically

trained subjects (Guastello, Traut, & Korienek, 1989; Israelski et al., 1989; Tullis,

1981), which can lead to concerns about the validity and generalizability of

experimental results to other populations, especially novices. Even with technical

experts, different studies have produced disparate results. Some populations may

prefer text codes to pictorial symbols or may prefer newly created symbols to industry

standard conventions (Guastello, Traut, & Korienek, 1989). On the other hand,

studies have also found technical experts both preferred and improved performance

with a graphic interface through direct manipulation of icons (Israelski et al., 1989;

Tullis, 1981). Contrary results have been reported; under some task conditions, icons

in the human-computer interface provided no performance advantages (Landsdale,

1988).

Some investigations into aspects of the human-computer interface have

provided support for the use of principles and guidelines from the field of

instructional message design. These will be briefly noted below.

Material that is meaningful to the learner is acquired and retained more easily

(Fleming & Levie, 1978). Icons that are meaningful to users are recalled more

effectively and improve performance in the human-computer interface (Landsdale,

1988).

Landsdale coded icons by shape, location on screen, and color and provided

subjects with a simulated filing task for information storage and retrieval. Subjects

had to assign job advertisements to an employment database using icons, then










retrieved the advertisements following a one hour delay. Iconic methods alone did

not automatically improve performance. Meaningfulness of icons and representational

shapes improved both recall and performance. For instance, an advertisement related

to educational television would be assigned for filing under an icon of a person

wearing a mortarboard cap and gown. In a meaningless condition, the icon would be

arbitrarily assigned by the system to a shape that was not compatible with the

advertisement. Landsdale also found that subjects' generation of the encoding icons

used for filing improved performance. For example, under this condition the subject

might choose to file an advertisement relating to educational television under an icon

representing tragedy-comedy masks (for entertainment) instead of the mortarboard

icon.

Pictures are more easily remembered than words (Fleming & Levie, 1978).

Menu selections with graphics are more accurately searched for information than text

menus (Muter & Mayson, 1986).

Muter and Mayson contrasted menu selection using a linear text menu, a

graphic menu with the same text items with related graphics spread across the screen,

and a control menu with the text items in the same location as in the graphics screen,

but minus the graphics. For instance, a menu item titled "2. Furniture" was

presented in the center of the screen with an outline drawing of a table and chairs. In

the control condition the graphic would be absent. In the text condition it would

appear second in order in the list of menu options. Slides of the screens were

displayed and subjects were asked to respond to goal oriented questions or commands








42

related to the menu items. For example, "Look for a new sofa" should have resulted

in a response of "2. Furniture." Though there was no difference in speed overall, the

graphics menus resulted in fewer errors.

Combined text and graphics should be congruent or redundant, so the learner

receives the same message regardless of the channel of communication attended to

(Fleming & Levie, 1978). Mixed modality icons (text and graphics) are more

meaningful and preferred by users within a content domain (Guastello, Traut, &

Korienek, 1989).

A survey study providing several alternatives to represent system controls and

displays in an automated building heating, ventilation, and air-conditioning system

was conducted by Guastello and associates (1989). Their data supported the

hypothesis that dual-modality icons would be more meaningful than either modality by

itself. Longer verbal abbreviations were preferred over shorter industry standard

abbreviations. Perhaps due to the technical background of the population surveyed,

verbal icons were sometimes preferred over pictorial if the mixed modality icon was

not available.

Discriminations are learned most readily by progressing from maximal

differences to finer differences (Fleming & Levie, 1978). Menu navigation is

facilitated by maximal distinctiveness among text choices (Schwartz & Norman,

1986). Visual distinctiveness of PSSs aids in locating specific items (Arend, Muthig,

& Wandmacher, 1987).










Schwartz and Norman (1986) modified an existing menu used by the

Compuserve time-sharing system. Menu modifications consisted primarily of adding

the words "General" or "Specific" to the beginning of a menu item and the word

"Information" at the end of the item. This "rather trivial" manipulation resulted in

significantly modified performance. Subjects using the modified menus to locate

information within the system performed better than those using the original menu

along the dimensions of time per search, number of frames covered during search,

and number of searches quit. Subjects with written documentation accompanying the

original menu quit searching more often; the presence of descriptions of the menu

items served to discourage subjects rather than facilitate searches.

Noting that research such as that described above dealt only with text items

and not with PSSs, Arend, Muthig, and Wandmucher (1987) investigated commands

using a word set, an abstract icon set, and a representational icon set. Icons in the

abstract set would be categorized as logical pictures in Knowlton's taxonomy. These

items differed greatly in overall shape or global features. The representational icon

set would be categorized as analogical pictures, in that the primary element was a

representation of a sheet of paper. In this set, local features differed while the global

feature of the sheet of paper remained constant. Menus featuring icons with global

differences were searched more quickly than text or local difference icons. Increasing

menu size from 6 to 12 items did not significantly increase search time for icons with

global differences, but did for text and icons with local differences. The abstract










icons used in this experiment were retrieved more quickly regardless of menu size

without any difference in error rates.

Although some principles for instructional message design have been supported

by research involving human-computer interfaces, studies have also yielded

contradictory results. This evidence will be noted next.

Concreteness facilitates learning and remembering. Messages to the novice

within a domain should begin with the more concrete and move to the more abstract

as the learner increases knowledge (Fleming & Levie, 1978). The assumption that

icons should represent concrete objects is not supported in some human-computer

interface experiments (Arend, Muthig, & Wandmacher, 1987; Baecker, Small, &

Mander, 1991; Blankenberger & Hahn, 1991).

As noted above, Arend, Muthig, and Wandmacher found that the more

concrete and representational icons were not searched more quickly or accurately than

were those that were more abstract in representation. Blankenberger and Hahn found

that when screen placement was fixed, the design of the menu items, whether pictorial

or text, made no significant difference after learning. However, Blankenberger and

Hahn also found that the closer articulatoryy distance" (Hutchins, Hollan, & Norman,

1986) of representational icons assisted in learning to associate icons with their

respective commands more effectively than meaningless arbitrary figures.

These studies indicate that users can adapt to varied types of PSSs within the

human-computer interface and, given training, there is little difference in search and

selection performance after items and locations have been memorized. The studies








45

cited have typically involved investigations of PSSs contrasted or in combination with

text, and have explored differences between visual and verbal memory processes.

Generally, the primary focus has not been to determine the functional intent or initial

interpretation of the meaning of the PSS. As noted earlier, only recently have icons

been used as commands in addition to representing data or programs. Therefore, the

discussion will focus next on specific studies that have explored the construction of

PSSs for use as commands and will explore the rationale for addressing this as an

instructional message design problem using Knowlton's taxonomy.

Gabriele Rohr (Rohr, 1984, 1986; Rohr & Keppel, 1984) conducted a series of

studies to investigate the relationship of types of commands to types of pictorial

symbols to determine whether commands could be successfully represented and

understood using pictorial symbols rather than through verbal commands alone.

Types of commands included those that dealt with manipulating text, processing files,

and controlling internal states. Text manipulation commands included items such as

input, insert, move, and delete. File processing commands included items such as

save, send, erase, and print. State control commands included select, help, and back.

Icon command sets were developed by showing pictorial and abstract elements

singly and in combinations to subjects. Pictorial symbols used included iconic

representations of sheets of paper, pencil, a hand, a tear in the paper, an envelope, a

hand stamp, and a box. Abstract symbols included such elements as rectangles,

slashes, crosses, arrows, points, letters, and numbers. A group of ten subjects with

no prior data or word processing experience rated each item in relation to the amount








46
of functionality, manipulative information, or pictorial quality contained in each item.

They also rated how well icons represented text processing functions.

The arrow was the only icon element rated highly for functionality and

manipulative information. File processing commands were found to be best

represented using pictorial symbols alone or in combination with abstract symbols

such as arrows. For text manipulation and state control, abstract symbols indicating

actions were chosen. Based on the ratings provided by these subjects, three sets of

icons were constructed using the elements identified.

These icon command sets included two pictorial sets and one word set. The

two pictorial sets were varied systematically to include or exclude functional

elements. For example, the papersheet pictorial symbol was selected as symbolizing

"file." This was combined with other symbols, such as the "tear" symbolizing

"erase" and "delete" to create all file processing commands. Two sets of file

processing commands were created that included either an arrow (functional symbol)

symbolizing an action, or lacking an arrow (non-functional symbol). Text editing and

state control icons were created using similar techniques, except that text editing

commands showed shaded rectangles denoting cursor placement actions for the

functional symbol set. The state control icons used arrows to indicate the selected

state. These sets were incorporated into a prototype text editing system (Keppel &

Rohr, 1984) that differed only in the icon sets displayed for the commands.

Following minimal instruction, subjects with no prior text processing experience were

given tasks to perform using the system.










Results of these studies indicated a superiority of visual symbols over verbal

commands in the initial learning stage, as indicated by significantly more help

requests from those learning the verbal command group. The second part of the

experiment showed no difference between groups in the number of help requests. In

addition to number of help requests, Rohr also measured time in editing. Functional

symbol groups took longer to learn, but were superior to both non-functional symbol

and verbal command groups during the second part of the experiments. Rohr

indicated that the visual-spatial information encoded in the functional text

manipulation symbols helped to reduce mental memory workload by presenting visual

cues regarding the operation of the commands.

Rohr also conducted experiments using paper and pencil testing as a control,

after providing instruction using the help screens from the iconic command version

and the word command version (Rohr, 1986). This time, the functional

representation icons differed only with spatial-relations tasks. There was a learning

effect, as the functional icons took longer to learn, but were associated with faster

performance during the second measurement of task performance. Rohr concluded

that the nature of the task determined the utility of the icon commands. Subjects

performed tasks that involved spatial relationships more effectively using the

functional icon commands after learning. File handling iconic commands had no

advantage over word commands. Visual information was advantageous when the task

involved spatial functions, by reducing memory workload and reducing the complexity









of the task. With more highly abstract functions, visual concepts increased

complexity and were not recommended.

An analysis of the iconic commands used by Rohr in this study, using

Knowlton' s taxonomy, shows that the file handling commands were analogical

pictures showing objects and places, and that the text editing commands were logical

pictures. Since Rohr developed the icon sets before implementation, it cannot be said

that analogical pictures are better for a certain type of command than either realistic

pictures or logical pictures. Analysis of individual PSSs reveals that the pattern of

arrangement or order of connection, which Rohr considered spatial functions, were

consistently represented using arrows. Due to the findings by Rohr, the PSS sets

constructed for this study include arrows as components of realistic, analogical, and

logical pictures alike. The addition of arrows or functional indicators would seem to

provide additional information that would be difficult to infer without the arrows.

And, in the small sample used to identify meaningful elements, the arrow was the one

item that was meaningful to all viewers in relation to function.

Building on Rohr's work, and using a classification system with some elements

similar to Knowlton's taxonomy, Rogers (1986) investigated the relationship between

type of representation and the initial comprehension of icon sets used to represent

word processing command operations. Rogers' symbols were classified by type as

"abstract symbol," "concrete analogy associated with action," and "concrete object

operated on." These simple sets were used to create an additional three combined

symbol sets.








49
Rogers provided subjects with a verbal context appropriate to word processing

tasks. The full icon set was provided with descriptions of the commands included.

The subjects' task was to match the icons to the command descriptions. Subjects

were advised to cross out each icon after having matched it to a description. Main

effects by command set were found, as well as interactions between symbol type and

command type. The conclusion was made that icons representing concrete objects to

be acted on were superior for use as commands.

However, Rogers' icons include abstract symbols as elements and as indicators

of pattern or order of connection without making a distinction between the different

functions. Rogers' definition of concrete objects and concrete analogies did not

consider the analogical nature of the desktop metaphor and its elements. For

example, a representation of a sheet of paper with a small arrow at the bottom was

considered a concrete object. A finger pointing downward was considered an

analogy. An arrow pointing downward was considered an abstract symbol. The

encoded message within each of these symbols was the command "Go to the bottom

of the text." The combinatorial symbols all contained a representation of a sheet of

paper as a concrete object. The arrow pointing downward was added as an abstract

symbol, and the hand pointing downward was added as a concrete analogy. The

combination of all three classifications in one picture was achieved by using the sheet

of paper, the hand pointing downward as the concrete analogy, and the addition of

motion cues in the form of two lines next to the hand.










Rogers concluded that the use of depictions of concrete objects operated on

was the most direct and meaningful way to represent abstract referents. Icons that

depicted command operations by her definition of a concrete analogy were the least

effective. Although she referred in her introduction to Rohr's (1984) finding that

manipulative control actions such as "insert text" were best represented by more

abstract symbols, she did not discuss or explain this apparent discrepancy with Rohr's

study. One possible explanation is that the symbol sets used by Rohr and Rogers

were not equivalent.

Knowlton's taxonomy provides a framework for examining Rogers' sets of

PSSs that may explain the results. The category "concrete objects operated on,"

which included the best understood symbols, includes analogical pictures when

categorized by Knowlton's schema. Rogers' "concrete analogy" symbols are

analogical pictures in Knowlton's taxonomy, and her "abstract symbol" category

appears similar to Knowlton's logical pictures classification. Of the symbols included

in her article (Rogers, 1986), a depiction of a blank computer screen representing the

"quit" command is the only command that might be considered as a realistic sign

symbol within Knowlton's taxonomy. How would one classify Rogers' other

examples more parsimoniously?

For example, consider the PSSs used by Rogers for the command "go to

bottom of text." The text displayed on the screen and kept in electronic form in the

computer's memory is represented by analogy as a sheet of paper with text on it.

Although the picture might be considered to be more concrete than abstract in its










depiction of a sheet of paper, it still functions as an analogy. Since this sheet of

paper is the main element of the picture, the PSS should be classified as an analogical

picture. The addition of arrows or pointing fingers denoting the goal of the command

"go to bottom of text" indicate only the arrangement or connection of the current

cursor location and the destination cursor location and do not change the functional

classification within the taxonomy. The "concrete object" picture used to denote this

command actually includes an arrow pointing at the bottom of the text and could be

interpreted as equivalent to Rogers' "concrete object plus abstract symbols"

classification. In addition, including abstract (logical) components within the symbols

proved superior to the simple icon set. This might be explained both by Rohr's

results, in which text manipulations were more effectively represented by more

abstract visuals, and by instructional message design guidelines, which suggest that

concreteness facilitates learning and remembering, concrete pictures are more easily

remembered than words, and that cues that are familiar to the learner or that direct

attention to relationships can facilitate learning. Although Rohr and Rogers provided

additional information about the interpretation of PSSs used in human-computer

interfaces, the constructs used by Rogers were poorly defined. Rohr's primary

purpose was to develop graphic menu commands rather than to test or develop theory.

This study was designed to explore questions left unanswered by both Rohr's

and Rogers' studies regarding the initial comprehension and interpretation of PSSs

used in the human-computer interface. In this investigation, Knowlton's taxonomy

was used systematically to create sets of PSSs, so each command was represented by








52

an example from each functional type of picture. This counterbalancing was not done

in Rohr's studies. Knowlton's taxonomy offers a more concise scheme for

classification than does Rogers' categories. As discussed above, Rogers' schema is

not as comprehensive as Knowlton's in describing and explaining all possible

combinations. The inclusion of arrows as a component within PSSs aids in

comprehension; in this study, the pattern of arrangement and order of connection

within a PSS, where appropriate, were represented consistently by arrows across

functional types.


Computer Experience. Literacy, and Knowledge


Researchers have used experts and novices (Baecker, Small, & Mander, 1991;

Cahill, 1975) as categories in investigating interpretation of PSSs. Cahill found that

subjects with specific experience with the type of equipment represented were able to

interpret PSSs that referred to that equipment's controls more successfully than

subjects with equal technical knowledge but lacking the specific experiential context.

The study conducted by Baecker and associates distinguished between subjects with no

experience and subjects with prior experience with the software program used in the

study. Computer experience and knowledge will vary among individuals, and can be

expected to have an effect on the interpretation of PSSs used in the human-computer

interface.

For the purposes of this study, two nationally standardized instruments found

in the Educational Testing Service Catalog were reviewed for suitability. These








53
measures are the Computer Aptitude, Literacy, and Interest Profile (CALIP) (Poplin,

Drew, & Gable, 1984), and the Standardized Test of Computer Literacy (Montag,

Simonson, & Maurer, 1984).

The CALIP is designed to assess computer-related abilities of individuals

between 12-60 years of age. The CALIP is a standardized test battery composed of

four aptitude subtests, one interest subtest, and one literacy subtest. The four aptitude

subtests (Estimation, Graphic Patterns, Logical Structure, and Series) are primarily

visual in nature and do not require much reading. Standard scores from the four

aptitude subtests are combined and a Computer Aptitude Quotient (CAQ) is derived

from a conversion table. The CALIP can be administered individually or in groups,

and requires approximately one hour for completion.

The CALIP was standardized on a national sample of 1,236 children,

adolescents, and adults. Levinson (1986) notes that three of the four aptitude subtests

are highly correlated with each other, and may measure the same construct, which

may be nonverbal problem-solving ability. Validity studies reported in the manual

indicate that the CALIP can differentiate between expert programmers and random

others, between those who have written intermediate/advanced programs and random

others, and between those who know two or more computer languages and those who

know one or none. Significant differences are also reported between those who have

had two or more computer courses versus those who have had one or no courses

(Poplin et al., 1984). Levinson (1986) suggests that the CALIP can differentiate

between individuals with greater and lesser computer-related experience, but not










necessarily between individuals with greater or lesser aptitude when experience is

equal. Internal consistency reliability coefficients are above .85 for a population

roughly equivalent to that sampled for this project. Specifically, the Spearman-Brown

split-half reliability coefficients for the subtests ranged from .86 to .95 for the age

groups between 15 and 30 years old. The coefficient alphas ranged from .85 to .93.

The Standardized Test of Computer Literacy (STCL) was developed over a

two-year period at Iowa State University in Ames, Iowa (Montag et al., 1984). The

STCL includes a competency based test of general computer literacy, comprised of

three parts, and a Computer Anxiety Index (CAIN). The three sections of the STCL

include competencies related to computer systems, computer applications, and

computer programming. Reliability estimates for the STCL are .64, .75, and .69 for

the three sections, with an estimated reliability of .86 for the total STCL. These

estimates were based on a pilot test of 152 college students who had just completed a

semester long, three credit, college level computer literacy course. Normative data

were obtained from 341 students in six states, with an estimated reliability of .87

(Simonson, Maurer, Montag-Torardi & Whitaker, 1987). The CAIN was found to

have an internal consistency reliability estimate of .94 and a test-retest reliability

estimate of .90.

The CALIP and the STCL do not appear to be equivalent to each other. The

only area of overlap between the two is in the assessment of computer literacy. The

CALIP includes a 30-item subtest of computer literacy that includes questions

intended to discriminate between those with high and low levels of knowledge and










experience. One question, related to types of floppy disks, refers to disks that are

single sided or eight inches wide. This reveals the period during which the

instrument was developed, and suggests that this particular subtest may not be

currently valid. The emphasis on graphic materials in the estimation and graphic

patterns subtests, and on symbolic data in the logical structures and series subtests

suggest that these subtests would not reflect specific computer course curricula. A

computer experience survey is included with the CALIP, but is not part of the scored

items. The STCL, on the other hand, uses 80 questions to measure 70 specific

competencies related to general computer literacy in the areas of systems,

applications, and programming. A single question about a technological anachronism

is less likely to affect the overall results. The STCL was normed on students who

had completed a semester course in computer literacy, and takes 1.5 hours to

administer. The scope of the STCL is much wider than was needed for this study.

Since the purpose of this study was to investigate the effects of functional type

on individuals' ability to interpret the meanings of PSSs, the CALIP was considered

to be more closely correlated to the nonverbal, visual nature of the stimuli than the

competencies measured by the STCL. Use of the CALIP could confound the effect of

the sign symbols with non-verbal problem solving ability. This would make the

CALIP an unsuitable choice for this study.

Because no suitable nationally standardized instrument was practical for this

study, an instrument was developed locally to classify subjects by type of computer

related experience. This instrument consists of two parts. The first part assesses the









breadth and depth of computer experiences. The second part of the instrument

consists of a self-assessment of expertise and experience. Together, these two

sections are used to arrive at a composite category. This instrument was tested in a

pilot project to develop reliability estimates prior to the implementation of the

experimental study. A discussion of the computer experience survey instrument is

included in the next chapter.


Summary


Pictorial sign symbols used in a human-computer interface may be viewed as

non-verbal instructional messages. Selected materials from the theoretical and

research literature of several disciplines have been reviewed to place the issues related

to the initial interpretation of these PSSs in perspective. Theoretical assumptions

regarding the design of these messages have not been supported in some situations,

generally in regard to the level of concreteness or abstraction of the PSS. A

semiotics-based approach has been suggested and has been used for dealing with

PSSs, but an adequate taxonomy for the criterial attributes of these symbols within the

human-computer interface has not yet emerged.

Knowlton's taxonomy and its usefulness for human-computer interface design

has not been systematically investigated. The distinction of the realistic, analogical,

and logical functions, and the separation of pictures into elements, patterns, and order

of connection provides a coherent structure for investigating systematic differences in

the interpretation of PSSs. Knowlton's taxonomy has been used in this review to










analyze and explain apparently contradictory findings in recent studies in this area.

As a result, this study was conducted to investigate the relationship of the functional

type of PSS, viewer's prior experience, and viewing context to the ability of

individuals to interpret PSSs used in a human-computer interface. Level of

experience and context were assumed to have a positive effect on the ability of

individuals to interpret PSSs.

To test these assumptions and determine the relationships stated above, the

following questions were raised. Can messages encoded as realistic, analogical, and

logical pictures be interpreted equally well? Does an individual's level of computer-

related experience affect the ability to interpret PSSs? Does the context within which

a PSS appears affect interpretation? Are there interactions between varied levels of

experience, context, and the functional type of PSS that affect interpretation? The

next chapter will describe the methodology, design, and instrumentation used to

answer these questions.













CHAPTER 3
METHODOLOGY



This study was designed to determine whether there are differences between

people with different types of computer-related experience in their abilities to interpret

pictorial sign symbols (PSSs) representing features of a computer system due to the

functional type of picture used as a sign symbol. In addition, the study explored

whether there are any differences in interpretation due to general and specific contexts

within which interpretations are made, when there are no textual cues to the meanings

of the visual symbols. Sets of PSSs representing common word processing menu

commands were presented to groups of teacher education students, who interpreted

the intended messages. This chapter includes a description of the research design and

the methodology used to gather evidence to support the reliability and validity of the

measures used in the conduct of the experiment.


Research Design


The design used for this study is a mixed within-subject or repeated measures

factorial design (Keppel & Zedek, 1989) using a 3 x 2 x 3 x 3 split plot, as shown in

Table 3-1. Factor A (computer experience) has three levels: (a) little or no computer

experience, (b) text-only computer experience, and (c) graphic interface computer










experience. Factor B (context) has two levels: (a) low context (non-specific verbal

context), and (b) high context (specific verbal context of computer application type

and menu names provided). Factor C (sign symbol type) has three levels: (a) realistic

sign symbols, (b) analogical sign symbols, and (c) logical sign symbols. Factor D

(sequence) has three levels: 1, 2, and 3, representing the order of presentation.

Each subject was categorized under a level of variable A (computer experience)

independently of assignment to the context condition. Computer experience was not

manipulated within the experiment but was used as a blocking variable. Each subject

was randomly assigned to one level of the context variable. Each subject received all

levels of variable C (PSS types) and all levels of variable C were represented at each

level of variable D. Therefore, computer experience type and context condition were

between subjects factors, while PSS type and sequence represented within subjects

factors.


Population and Sample


The population for this study was defined as undergraduate teacher education

students. The sample for this study was recruited from students enrolled in

introductory educational media courses at a college of education in a four year

university. This population was of interest for two reasons: (a) computer competence

is now considered a basic skill, and (b) many teachers and computer coordinators in

the public school system have little training in computer studies and have mediocre

computer skills, according to nationally collected data (Martinez & Mead, 1988).








60

Due to curricular adjustments made in recent years, current teacher education students

may have higher levels of knowledge about computers and greater educational

computing experience than teachers already in the field. One of the purposes of this

study was to determine whether these experiences are related to the ability to interpret

PSSs similar to those used in graphic human-computer interfaces.

Students were recruited from four intact sections of introductory educational

media courses and randomly assigned to two context conditions. Of 122 students

enrolled in the sections sampled, 108 attended on the days the study was conducted,

and all present participated. The sample, therefore, represented 89% of the

population at this location. Due to the nature of the study, written informed consent

was not required. Participants were advised of the right to refuse to answer any of

the questions on the survey form, and several chose to omit age, sex, and ethnic

information. Based on those providing information, 89% were female and 11% were

males, ranging from 19 to 42 years old, with an average age of 22 years. The sample

group responses to ethnic origin categories were 91% White, 3.7% Black, and 3.7%

Hispanic. The remainder listed Other or omitted ethnic origin. The sample group

had completed an average of 14.47 years of schooling. Over half (53.8%) reported

having obtained an AA or AS degree. The number of computer-related courses

reported ranged from none to three courses, with an average of 1.33. Those

reporting having had none, one, two, and three computer-related courses were 17.6%,

43.5%, 27.8%, and 11.1%.










Table 3-1 Research Design


Within-Subjects Factors Between-Subjects Factors

Little or No Text-Oriented Graphic
Computer Computer Interface
Experience Experience Computer
Only Experience
Context Context Context
Sequence Sign Type low high low high low high
1 Realistic n=12 n=10 n=9 n=6 n=33 n=38
Analogical n=12 n=10 n=9 n=6 n=33 n=38

Logical n=12 n=10 n=9 n=6 n=33 n=38
2 Realistic n=12 n=10 n=9 n=6 n=33 n=38
Analogical n=12 n=10 n=9 n=6 n=33 n=38
Logical n=12 n=10 n=9 n=6 n=33 n=38
3 Realistic n=12 n=10 n=9 n=6 n=33 n=38
Analogical n=12 n=10 n=9 n=6 n=33 n=38

Logical n=12 n=10 n=9 n=6 n=33 n=38


Materials and Measures


The PSSs and the scoring procedure used were developed locally for this

study. The validity of theoretical constructs and materials is a primary concern.

Reliable instrumentation and measurements are essential to meaningful interpretation

of data. Therefore, this section includes a comprehensive discussion of the process of










materials and instrumentation development, beginning with the PSSs used before

proceeding to evidence gathered in support of the reliability of the measurements.

This section describes the process of materials development and the steps taken

to ensure the content relevance and construct validity of the PSSs generated for use in

the study. Messick (1989) has pointed out the need for systematic attempts to

document the consensus of multiple judges to deal with irrelevant variance during test

development stage. Among the actions recommended are the use of content

specialists' (a) judgements of whether or not each item reflects the content defined by

the domain, (b) ratings of items for the extent to which it reflects the domain facet it

was intended to measure, and (c) matching of items to the domains which the items

best represent. Each of these methods was employed to ensure the content

representativeness and construct validity of the PSSs used.

First, the primary investigator examined word processing software menus and

identified common word processing menu commands, features, and options. The

application software reviewed included (a) WordPerfect, Word and Edit for the DOS

operating system, (b) WordPerfect, Word, and Write for the Windows graphical user

interface, (c) Word and MacWrite for the Macintosh operating system, and (d)

Pagemaker, a page layout program with highly similar Windows and Macintosh

system versions. The Edit and Write software applications represent basic word

processors with simple features. WordPerfect, Word, and MacWrite are full-featured

application programs that together represent the most widely used word processing

software applications on the DOS, Windows, and Macintosh system platforms.








63
The investigator selected a set of 21 of the most common menu and command

concepts from the software applications surveyed. These options were available on

basic and full-featured software alike and were represented by identical or highly

similar wording on all programs. For example, "File Open" and "File Save"

commands are represented by the same words in all programs. The menu commands

"Align," "Alignment," "Justify," and "Justification" represent a single concept with

different, but similar titles.

Once the pool of command concepts had been identified, three PSSs

constructed to meet the definitions of realistic, analogical, and logical pictures

according to Knowlton's taxonomy were drawn for each command. An effort was

made to avoid exact copies of icons used in currently available software, but the use

of similar representations was not ruled out. A public domain image editing program

for the Macintosh was used to generate and save the pictures in machine readable file

formats. The pictures were each drawn within a one-inch frame and displayed on the

computer video display screen prior to printing.

Expert review of representation of taxonomy categories. The 63 pictures were

printed and copied to card stock paper and cut into one-inch squares. The paper

squares were then shuffled and mixed before presentation to two educational media

experts familiar with Knowlton's taxonomy, who independently sorted the pictures

into the three categories. Full agreement on the functional types of the pictures was

attained following discussion of picture referents and interpretations. The expert








64

reviewers provided comments and suggestions regarding the interpretation and design

of the pictures.

Expert matching of sign symbols to domain content represented. The set of 21

menu command concepts and the 21 groups of three PSSs were printed and pasted on

separate sets of 4" x 6" index cards. Both groups of 21 cards were shuffled and

presented to each of two computer usage experts. Both subjects achieved 100%

accuracy in matching the sets of three PSSs with the corresponding written menu

commands.

The two educational media experts were then provided with the individual sets

of 21 PSSs by functional type. Each set of pictures was randomized. The assigned

task was to match each individual PSS with the intended menu and command. The

sequence of presentation of each full set of pictures was randomized and cycled. The

experts achieved more than 80% success in matching the PSSs to the intended menus

and commands. Misinterpretations and crossed interpretations of the intended

messages were noted and used in the decision process to reduce the pictures to the

experimental set. For example, the PSSs for "File New" and "File Open" were

misinterpreted in some instances. The "File New" command was eventually

eliminated from the sets used in the study, and both "New" and "Open" were

accepted as correct interpretations of the intended message of the "File Open"

symbol.

Expert ratings of sign symbols regarding domain content representation.

Finally, each group of 21 sign symbols was printed on a sheet of letter size paper and








65
organized in a menu sequence consistent with that found in the software applications

reviewed. A panel of six experts evaluated how well each of the PSSs communicated

or represented the intended menu command, using a five-point Likert scale. These

results are shown in Table 3-2. Using these ratings, the three highest ranked symbols

from each of four groups of related concepts were selected for inclusion in the

experimental set of 36 pictures representing 12 concepts. The 36 PSSs were randomly

assigned to three groups so each group contained an equal number of realistic,

analogical, and logical pictures. Overhead transparencies were constructed so each

symbol was equal in size. Each overhead transparency consisted of three PSSs

representing commands typically found on the same menu. Four overhead

transparencies were used in each set, representing four different menus.


Procedure


The experimental setting was the classroom within which the classes normally

met. The classroom contains student desks, tables, chalkboards, and a video

projection screen. All groups viewed the materials under a standard presentation

protocol. Experimental materials were projected against a wall designed for and

normally used for projected visuals, so the materials were readily visible to all

subjects. Materials were presented to subjects in intact sections. Subjects within each

section were randomly assigned to either a low context or a high context condition.

The context condition was manipulated experimentally through random assignment of

differential instructions to subjects.










Table 3-2 Expert Ratings of Pictorial Sign Symbols

Variable N Mean SD SE

File New 18 3.28 1.18 .28

File Open 18 3.33 1.28 .30

File Save 18 3.78 .88 .21

File Print 18 4.00 .97 .23

File Exit 18 3.17 1.10 .26

Edit Undo 18 3.44 1.15 .27

Edit Cut 18 4.22 .88 .21

Edit Copy 18 4.11 .83 .20

Edit Paste 18 3.5 1.20 .28

Justify Left 18 4.72 .46 .11

Justify Center 18 4.56 .62 .15

Justify Right 18 4.72 .46 .11

Justify Full 18 4.44 .78 .18

Single Space 18 4.39 .85 .20

1.5 Line Space 18 4.44 .86 .20

Double Space 18 4.44 .86 .20





Prior to presentation of the PSSs, verbal informed consent procedures were

followed and subjects were requested to complete a locally developed instrument










assessing computer experience and a self-assessment of experience and expertise.

Following presentation of the stimuli and collection of the data, each subject was

coded to one of three computer experience categories. The categories of experience

were (a) novice, (b) text-only, and (c) graphic interface experience.

Subjects assigned to the low context condition were given the following

explanation: "You will be shown some pictures, commonly called "icons," which are

intended to represent commands, features, or functions of a microcomputer system.

These pictures will be shown in groups. Each group of pictures will be shown for the

same amount of time. Indicate whether you recognize an icon from having seen or

used it in a software program before by checking either Y for "yes" or N for "no."

Describe or give a name for each numbered picture and describe the computer feature

that you think it stands for. Please be as specific as possible."

Subjects assigned to the high context condition were given the following

explanation: "You will be shown some pictures which are intended to represent

commands, features, or functions of a microcomputer system. These pictures will be

shown in groups. Each group of pictures will be shown for the same amount of time.

Each individual picture stands for a feature that can be found within a word

processing program. Imagine that you have been given a marked up, edited version

of a term paper that you need to rewrite. The word processor commands and features

are indicated only by these pictures; there are no written labels. The groups of

pictures you see are the word processor menu options (File, Edit, Format) available

to you on the computer screen. Each numbered picture is intended to convey to you








68

the name or function of the feature it represents. Each picture will be shown for the

same amount of time. Indicate whether you recognize an icon from having seen or

used it in a software program before by checking either Y for "yes" or N for "no."

Describe or name each picture and describe the computer command, function, or

feature that you think it stands for. Please be as specific as possible."

PSSs were presented to each section following the same procedure. All

subjects viewed PSSs within each of three sign symbol types: realistic, analogical, and

logical pictures. These symbols were randomly assigned to three groups of symbol

sets so each process or function was represented within each group. Subjects viewed

one of three groups of sign symbols at each measurement. The order of viewing the

symbol sets was cycled to defend against the possibility of treatment interactions due

to the sequence of presentation. Since the viewer should be able to infer an icon's

meaning on the first encounter (Lodding, 1983), all three sets of pictures were

presented in one session of less than one hour. The symbols were presented in a

standard fashion, so each symbol was viewed for the same amount of time. Three

symbols, numbered from one to three, were presented simultaneously for one minute

on each overhead transparency.

The instrument used for recording subjects' responses had two forms. The

forms differed for high context and low context conditions solely in the written

instructions provided to the subject. The main portion of the form consisted of a

numbered table. Each numbered row was used to respond to an individual PSS.

Subjects were asked to identify, by checking a box, symbols that were recognized








69
from prior experience. For all symbols, subjects were asked to provide a name and a

brief description of the function or feature to which the symbol referred.

Scoring was made on the basis of the number of correct interpretations of the

PSSs used in the study. Thus, each subject's score could range from zero to 12 per

occasion, and the total score for each could range from zero to 36. A subject's score

for a specific set of PSSs also could range from zero to 12.

Reliability of scoring. A pilot study was conducted using 18 graduate students

enrolled in a course in educational television design and production to test the

presentation protocol and evaluate the reliability of the scoring protocol. The

assessment measures and protocol are described below.

The investigator developed a survey of computer experience following a

review of currently available computer literacy instruments and the nature of

Knowlton's taxonomy. The survey of computer experience asked subjects to check

off hardware and software items with which they have had experience. The items

were grouped into IBM PC compatible (text based) interfaces, Windows and

Macintosh (graphics) interfaces, and other hardware and software. The pattern of

items checked was used by judges following a written protocol to assign each subject

to one of the three computer experience categories. In addition, a five-item self-

assessment survey was created using a five point scale. The first two items asked

subjects to indicate how often and for how long they have used computers. The last

three items asked subjects to assess themselves, from novice through expert levels,

regarding overall computer experience and experience related to text and graphic








70
interfaces. The judges used this assessment to assign each subject to one of the three

computer experience categories. Finally, a composite category was assigned based on

the categories to which each subject had been assigned for the two separate

instruments.

Response scoring was conducted independently of categorization of experience

type. Each judge was provided with an answer key providing the intended menus and

commands in the order presented. Incorrect responses and responses that indicated

the subject was unable to guess a meaning were scored as zero points. Each correct

response was scored as one point. An answer was considered correct if the command

name or brief description matched or was highly similar to that on the answer key.

Each partially correct response was scored as one-half point. An answer was

considered partially correct if the subject correctly identified the intended menu but

did not specify the command function. For example, if the "File Open" PSS had

been displayed and the response had been "File," the answer would be scored as

partially correct.

For the pilot project, the judges reached a consensus of 94% in categorizing

subjects to computer experience categories. Correlation of judges' scoring of subjects

in the pilot project was high, with a coefficient alpha of .995 reported.

Due to the high level of agreement on the pilot project for both the

categorization and the scoring instruments, one judge performed the categorization

and scoring of the study data. A random sampling of 20 subjects was independently

categorized and scored by each of the other two judges used for the pilot study.










Written instructions were provided for the categorization and scoring tasks, with no

retraining of judges. The level of consensus for the categorization task was .85 and a

coefficient alpha of .977 was calculated for the scoring task. The lower level of

consensus regarding the categorization of experience was expected, as some of the

participants in the study were enrolled in an introductory course in instructional

computing. The concurrent exposure in class and laboratory to the Macintosh

graphical user interface may have influenced subjects' responses to the computer

experience survey and the self-assessment instrument. The graduate students sampled

in the pilot study were more clearly experienced or not experienced with the different

types of user interfaces. This evidence provides additional support for the reliability

of the instruments and instructions developed for categorization of subjects and

scoring of subjects' responses.

The following null hypotheses were derived from the research design for

testing using a four-way analysis of variance with repeated measures statistical

procedure.

Hypothesis 1. There will be no significant differences within subjects in the

ability to interpret different types of PSSs.

Hypothesis 2. There will be no significant differences in the ability to

interpret PSSs between subjects viewing PSSs within a given context and subjects

viewing PSSs with no specific context.

Hypothesis 3. There will be no significant differences in the ability to

interpret PSSs between subjects with different levels of computer experience.










Hypothesis 4. There will be no significant differences in the ability to

interpret PSSs within subjects due to the sequence in which PSSs are viewed.

Hypothesis 5. There will be no significant differences in the ability to

interpret PSSs due to interactions among the functional type of PSS viewed, the type

of computer experience, the viewing context conditions, and the sequence of viewing.


Summary


The use of PSSs within the human-computer interface has created a need for

comprehensive guidelines for the construction of these non-verbal messages. This

study was conducted to provide data relating PSS functional types to the ability of

individuals to interpret the intended messages. Recent research results were

incorporated into the design of the study to determine whether there are differences in

the ability to interpret PSSs due to differences in functional type based on Knowlton's

taxonomy. This study also was designed to determine whether differences in context

and computer related experience affect the interpretation of PSSs and whether there

are any interactions among these variables. Information to be gained from this study

has implications for and could be useful to designers of computer systems, designers

of computer-based instructional systems, designers of educational media, and

educators using computers as tools to improve learning.

To produce interpretable results, the investigator relied upon evidence gathered

to support the content representativeness and construct validity of the materials used

as experimental stimuli. The investigator developed instruments for categorizing








73
subjects by computer experience type and for scoring interpretation responses. A

limited pilot project was conducted to assess the reliability of these instruments across

multiple judges. The research design used an analysis of variance with two between-

subjects and two within-subjects factors to take advantage of the increased power of

repeated measures. Steps were taken in the implementation of the research design to

defend against the possibility of differential carryover or practice effects. The study

was conducted and the resulting data are reported and analyzed in the next chapter.













CHAPTER 4
RESULTS AND ANALYSIS


Introduction


This study was designed to determine whether the functional type of pictorial

sign symbol, as defined by James Q. Knowlton's taxonomy (Knowlton, 1964, 1966),

affects the interpretation of pictorial sign symbols. Computer experience was used as

a between-subjects attribute variable with three levels (novice, text-only, and

graphics). Viewing context was manipulated as a between-subjects experimental

condition with two levels (low context and high context). Pictorial sign symbol type

was used as a within-subjects factor with three levels (realistic, analogical, and logical

pictorial sign symbols). Sequence was used as a within-subjects factor with three

levels. A 3 x 2 x 3 x 3 ANOVA design was used for the study. Statistical analyses

of the data were conducted using the General Linear Models (GLM) procedure of the

SAS statistical analysis software program. The results of this study are reported

below. Statistically significant effects related to pictorial sign symbol type, computer

experience, sequence of viewing, and the interaction between pictorial sign symbol

type and sequence were found. Implications of these results are discussed in the final

chapter.










Results


The experiment was conducted and data were collected as described in the

previous chapter. The data were analyzed using a four-way ANOVA model including

repeated measures on the sign symbol type factor. These results are presented as

source tables in Table 4-1 and Table 4-2 and are discussed separately below. The

report of results begins with a review of the null hypotheses tested.

Table 4-1 Analysis of Variance Summary Table-Between Subjects Effects

Source df SS MS F Pr>F

Experience 2 9.61 34.31 12.00 .0001

Context 1 2.91 2.91 1.00 .3188

Experience x Context 2 1.33 .37 0.23 .7951

Error 102 295.95 2.90





Hypothesis 1 stated that there would be no significant differences in the ability

to interpret different types of pictorial sign symbols. The analysis of variance

procedure produced an F value of 52.28 (df 2, 204) that was statistically significant at

the .05 alpha level (see Table 4-2). Therefore, the null hypothesis was rejected. The

functional type of sign symbol viewed was related to the ability to interpret the

symbol.

Hypothesis 2 stated that there would be no significant differences between

subjects viewing pictorial sign symbols within a high context condition and subjects








Table 4-2 Analysis of Variance Summary Table-Within Subjects Effects

Source df SS MS F Pr>F

Sequence 2 4.96 2.48 4.53 .0119

Sequence x Experience 4 3.50 .88 1.60 .1763

Sequence x Context 2 .23 .11 1.97 .1003

Error 204 111.88 .55



Sign 2 57.79 28.90 52.28 .0001

Sign x Experience 4 3.90 .97 1.76 .1375

Sign x Context 2 .34 .17 .31 .7345

Error 204 112.75 .55



Sequence x Sign 4 6.33 1.58 4.03 .0032

Sequence x Sign x Experience 8 2.40 .30 .76 .6353

Sequence x Sign x Context 4 .40 .10 .26 .9053

Sequence x Sign x Experience x Context 8 3.45 .43 1.10 .3623

Error 408 160.11 .39





viewing pictorial sign symbols within a low context condition in the interpretation of

the pictorial sign symbols. The statistical analysis produced an F ratio of 1.00 (df 1,

102). This value is not significant at the .05 alpha level (see Table 4-1). The null









hypothesis was not rejected. Context, as operationalized for this study, was not

significantly related to the interpretation of pictorial sign symbols.

Hypothesis 3 stated that there would be no significant differences in the ability

to interpret between subjects with different types of computer experience. The

analysis produced an F value of 12.00 (df 2, 102) that was significant at the .05 alpha

level (see Table 4-1). The null hypothesis was rejected. The type of computer

related experience, as categorized for this study, was related to the ability of subjects

to interpret the pictorial sign symbols used in this study.

Hypothesis 4 stated that there would be no significant differences within

subjects in the ability to interpret associated with the sequence in which pictorial sign

symbols were viewed. The observed F value of 4.53 (df 2, 204) was statistically

significant at the .05 alpha level (see Table 4-2). The null hypothesis was rejected.

The ability to interpret pictorial sign symbols was related to the sequence in which the

subjects viewed the groups of symbols.

Hypothesis 5 stated that there would be no significant differences between

subjects with differing types of experience viewing pictorial sign symbols under

differing context conditions and in different orders of viewing in the ability to

interpret different functional types of pictorial sign symbols. Stated more simply, this

hypothesis stated that there would be no interactions between the variables measured.

For the Sequence x Pictorial Sign Symbol type interaction, the observed F value of

4.03 (df 4, 408) was statistically significant at the .05 alpha level (see Table 4-2).








78

The null hypothesis was rejected. The ability to interpret different functional types of

pictorial sign symbols was related to the sequence in which the symbols were viewed.

Although the tests of hypotheses resulted in significant main effects for

pictorial sign symbol type, sequence of viewing, and computer experience, these

results are of secondary interest to the effect of the observed interaction. The

following section will review the salient aspects of the analysis of these data.


Analysis


Main effects were found for pictorial sign symbol type, computer experience

type, and sequence of viewing variables. An interaction between the pictorial sign

symbol and sequence factors was observed. For all analyses, the Greenhouse-Geisser

Epsilon statistic was non-significant, indicating that the assumption of homogeneity of

variances was not violated. For all significant effects, the G-G adjustment did not

affect the statistical significance of the observed F value.

To determine the cell comparisons of interest, statistics were calculated and

cell means were plotted. Means and standard deviations for the pictorial sign symbol

types at the different sequence levels are included in Table 4-3, and these means are

plotted in Figure 4-1. Single degree of freedom contrasts were used to compare the

cell means across adjacent sequence positions within each sign symbol type.

Significant differences were found between sequence I and sequence 2, and

between sequence 2 and sequence 3 for realistic pictorial sign symbols (see

Table 4-4). There were no significant differences by sequence between means for









Table 4-3 Means of Pictorial Sign Symbols by Sequence of Viewing

Variable N M Std Dev Std Err


Sequence Level One

Realistic 108 .60 .65 .06

Analogical 108 1.36 1.03 .10

Logical 108 .32 .50 .05

Sequence Level Two

Realistic 108 .79 .83 .08

Analogical 108 1.20 1.17 .11

Logical 108 .59 .71 .07

Sequence Level Three

Realistic 108 1.13 1.06 .10

Analogical 108 1.43 1.14 .11

Logical 108 .59 .75 .07





analogical pictorial sign symbols (see Table 4-5). A significant difference was found

between sequence 1 and sequence 2, but not between sequence 2 and sequence 3, for

logical pictorial sign symbols (see Table 4-6). Main effects were found for the

pictorial sign symbols type, sequence of viewing, and type of computer experience

variables.









80




2.00

1.75

1.50 3

1.25 1 3

0 1.00
) 0.7
0.75
0.6-- 0.59 0.59
0.50

0.25

0.00
1 2 3
Sequence

SRealistic Analogical -+- Logical



Figure 4-1 Means of Pictorial Sign Symbols by Sequence of Viewing


Due to the more complex interaction of the sign symbols type and sequence

variables, tables for the main effects of sign symbol type and sequence are not

presented here. From a possible range from zero to 12 correct interpretations for

each, overall means for logical, realistic, and analogical PSSs were 1.51, 2.51, and

3.99 respectively. The contrast comparisons between the PSS types were all

statistically significant (p > .0001), indicating that each of the functional types of

pictures within the Knowlton taxonomy was different from the others.

The means for the first, second, and third viewing sequence levels were 2.29,

2.58, and 3.14. The contrast comparisons between adjacent groups were not










Table 4-4 Repeated Measures Analysis of Variance
Contrast Between Realistic Sign Symbols and Sequence


SS


Source df

Contrast of Sequence One vs Sequence Two

Mean 1

Experience Type 2

Context 1

Experience Type x Context 2

Error 102



Contrast of Sequence Two vs Sequence Three

Mean 1

Experience Type 2

Context 1

Experience Type x Context 2

Error 102


MS F Pr>F


3.42

.17

.52

.82

.63





4.06

1.42

.03

.95

.67


5.40

.26

.82

1.30







6.10

2.13

.04

1.42


.0221

.7686

.3662

.2769







.0152

.1239

.8360

.2453


statistically significant, but the comparison between the first and third sequence

positions was statistically significant (p > .0068), indicating that subjects' ability to

interpret the PSSs was greater for the third group viewed than for the initial exposure

group. These main effects for sequence and sign symbol type variables should be

considered in the context of the Sequence x Pictorial Sign Symbol interaction.


3.42

.33

.52

1.64

64.48





4.06

2.84

.03

1.90

67.90










Table 4-5 Repeated Measures Analysis of Variance
Contrast Between Analogical Sign Symbols and Sequence

Source df SS

Contrast of Sequence One vs Sequence Two

Mean 1 1.83

Experience Type 2 .03

Context 1 .17

Experience Type x Context 2 6.75

Error 102 173.27


Contrast of Sequence Two vs Sequence Three

Mean 1

Experience Type 2

Context 1

Experience Type x Context 2

Error 102


.51

5.62

.31

3.62

136.60


MS F Pr>F


1.83

.01

.17

3.38

1.70





.51

2.81

.31

1.81

1.34


1.08

.01

.10

1.99







.38

2.10

.23

1.35


.3020

.9915

.7518

.1421







.5390

.1278

.6312

.2637


In addition to the effects of sign symbol type and sequence, a main effect due

to the type of previous computer experience was observed. This effect was consistent

across sequence and pictorial sign symbol type. Statistics are shown in Table 4-7 and

are graphed in Figure 1-2. Tukey's HSD statistic was calculated at an










Table 4-6 Repeated Measures Analysis of Variance
Contrast Between Logical Sign Symbols and Sequence

Source df SS

Contrast of Sequence One vs Sequence Two

Mean 1 3.72

Experience Type 2 .27

Context 1 .00

Experience Type x Context 2 .03

Error 102 41.41


Contrast of Sequence Two vs Sequence Three

Mean 1

Experience Type 2

Context 1

Experience Type x Context 2

Error 102


MS F Pr>F


3.72

.13

.00

.01

.41


.09

.20

.03

1.08

32.23


9.15

.33

.00

.03


.0031

.7187

.9617

.9690


.27 .6018

.32 .7297

.09 .7610

1.72 .1850


alpha level of .05, producing a minimum significant mean difference of 1.18,

exceeded by each of the pairwise contrasts between means.

To summarize, the analyses of data collected from this study resulted in the

following findings. Significant differences in the ability of subjects to interpret

pictorial sign symbols were found related to the interaction between the sign symbol










84




12.00

11.00

10.00

9.00

8.00

7.00

0 6.00 5.63

5.00

4.00

3.00

2.00

1.00

0.00
Novice Text-Only Graphics
Computer Experience Type



Figure 4-2 Means by Type of Computer Experience


type viewed and the sequence of viewing. Significant differences were found in the


ability of subjects to interpret pictorial sign symbols associated with the taxonomic

classifications of these symbols into realistic, analogical, and logical categories.

Significant differences were found in the ability of subjects to interpret pictorial sign


symbols related to subjects' type of computer experience as operationalized for this

study into novice, text-only, and graphic interface experience categories. Significant

differences were found in the ability of subjects to interpret pictorial sign symbols

related to the sequence in which the symbols were viewed. The ability of subjects to










Table 4-7 Means by Computer Experience

Category of Computer Experience

Novice

Text-Only

Graphics


Type

n

22

15

71


M Std Dev

3.98 2.76

5.63 4.06

9.76 5.74


interpret pictorial sign symbols was not significantly affected by the viewing context

as operationalized for this study.

This study was designed to determine whether Knowlton's taxonomy of the

functions of pictures could be applied to the design of pictorial sign symbols used to

represent commands in the human-computer interface. How do these results affect

the knowledge base used by instructional message designers, human-computer

interface designers, and developers of educational software programs? These

questions are addressed in the next chapter.


Std Err

.59

1.05

.68













CHAPTER 5
CONCLUSIONS AND RECOMMENDATIONS



Introduction


The purpose of this study was to investigate the relationship of the functional

type of picture, as defined by Knowlton's taxonomy (Knowlton, 1964, 1966), to the

ability of subjects with different types of experience to interpret pictorial sign symbols

(PSSs) representing commands used in a graphic human-computer interface, under

varying context conditions. The research design utilized a 3 x 2 x 3 x 3 factorial

design with repeated measures. Factors measured included computer experience type

(three levels), context (two levels), pictorial sign symbol type (three levels), and

sequence of viewing (three levels).

To implement this study, teacher education students were chosen as the

population of interest from which the sample was taken. Materials were developed to

represent word processing software commands in each of the three functional types of

pictures, using expert opinions systematically to substantiate the construct validity of

the materials used. Instruments for categorizing subjects into computer experience

categories and for scoring interpretations were developed and pilot tested to

substantiate the reliability of measurements. The results of this study in terms of the

research hypotheses are presented next.











Findings


The study was implemented, data were collected, and the data were analyzed

in terms of the stated hypotheses. The research questions had been stated by the

following null hypotheses:

Hypothesis 1. There will be no significant differences within subjects in the

ability to interpret different types of pictorial sign symbols. This hypothesis was

rejected.

Hypothesis 2. There will be no significant differences in the ability to

interpret PSSs between subjects viewing pictorial sign symbols within high context

and low context conditions. This hypothesis was not rejected.

Hypothesis 3. There will be no significant differences in the ability to

interpret PSSs between subjects with different types of computer experience. This

hypothesis was rejected.

Hypothesis 4. There will be no significant differences in the ability to

interpret PSSs within subjects related to the sequence in which pictorial sign symbols

are viewed. This hypothesis was rejected.

Hypothesis 5. There will be no significant differences in the ability to

interpret PSSs due to interactions among the functional type of PSS viewed, the type

of computer experience, the viewing context conditions, and the sequence of viewing.

This hypothesis was rejected.










Specifically, an interaction was found between pictorial sign symbol type and

the sequence of viewing. Main effects were found for computer experience type, for

pictorial sign symbol type and sequence of viewing. The viewing context, as

operationalized for this study, did not result in a statistically significant effect on the

interpretation of sign symbols. These results are interpreted in the next section.


Discussion


The most interesting result of this study was the differential effect of sequence

on the interpretation of the different types of PSSs. As shown in Table 4-1, each of

the sign symbol types displayed a different pattern of correct interpretations over

sequence order, creating an ordinal interaction.

Specifically, realistic PSSs were interpreted poorly during the first sequence,

but each subsequent viewing group resulted in a statistically significant positive

difference. Analogical PSSs were interpreted correctly at a higher level than other

PSSs, but the average number of correct interpretations varied little and did not

increase or decrease significantly over time. Logical PSSs were initially interpreted

at a relatively low level of correct interpretations and increased significantly between

the first and second sequence. However, the average number of correct

interpretations remained the same for the second and third sequence positions.

The interpretation of these data must take into account that the population used

for this study, teacher education students, was homogenous, consisting primarily of

young adult white females with little computer science background. The general










application of these results to other populations is not recommended. The nature of

the task domain (word processing) also should be considered before attempting to

generalize these results to other tasks involving human-computer interaction.

A main effect due to sequence was observed, although the research design was

structured to reduce the possibility of practice effects. This suggests that the repeated

exposure to the twelve concepts represented by the 36 PSSs may have influenced the

decoding of the concepts and produced the higher average of correct interpretations on

the subsequent exposures. The sequence effect was significant for realistic PSSs,

indicating that as more examples were viewed, subjects were able to decode and

interpret the realistic symbols designed for this study more effectively.

The analogical PSSs were interpreted at a consistently high level. One

possible explanation may be that for those with graphic interface experience, the high

level of interpretation was in fact a high level of recognition. The capacity of humans

to recognize previously viewed pictures has been well established (Shepard, 1967;

Standing, Conezio, & Haber, 1970). Although the PSSs used were not intended to

duplicate currently used icons, the similarity between the study materials and

commercially available software icons indicates that this is a plausible explanation.

Another possibility is that the basic metaphors used for the word processing

commands represented as analogical PSSs were particularly appropriate to the domain

represented. This could account for the high level of interpretation among subjects

who were not categorized as having graphic interface experience. Although the icons

may not have been recognized from previous use, the pictorial sign symbol may have