• TABLE OF CONTENTS
HIDE
 Cover
 Table of Contents
 Executive summary
 Methodology
 Initial resource impressions
 Scenario-based testing
 Findings and recommendations
 Conclusions
 Appendices






Title: Usability Test Report for From the Air: Florida Photography
CITATION PDF VIEWER THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00102994/00001
 Material Information
Title: Usability Test Report for From the Air: Florida Photography
Physical Description: Archival
Language: English
Creator: Ochoa, Marilyn N.
Publisher: UF Libraries
Place of Publication: Gainesville, FL
Publication Date: September 30, 2010
Copyright Date: 2010
 Record Information
Bibliographic ID: UF00102994
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.

Downloads

This item has the following downloads:

PDF ( PDF )


Table of Contents
    Cover
        Page 1
    Table of Contents
        Page 2
    Executive summary
        Page 3
    Methodology
        Page 4
        Page 5
        Page 6
    Initial resource impressions
        Page 7
        Page 8
    Scenario-based testing
        Page 9
        Page 10
    Findings and recommendations
        Page 11
        Page 12
        Page 13
        Page 14
        Page 15
        Page 16
    Conclusions
        Page 17
    Appendices
        Page 18
        Page 19
        Page 20
Full Text






Usability Test Report for From the Air: Florida Photoguraphy


Date of Report:
Period of Testing:

Prepared for:


September 30, 2010
May 13, 2010-September 14, 2010

Carol McAuliffe
Map Librarian
Map and Imagery Library, Spatial Information Services Unit
Government Documents Department

Laurie Taylor
Interim Head
Digital Library Center
University of Florida


Prepared by:


Marilyn N. Ochoa
Usability Manager, From The Air Grants
Assistant Head Librarian, Education Library
University of Florida


Julio Perez-Downes
Assessment Team Member













Table of Contents


Executive Summary........................ ...............3.................

Methodology ................. ...............4.................

Who and Where We Tested

What data we collected


Initial Resource Impressions ............... ...............6............ ....

Scenario-Based Testing ..... ...............8.........

Findings and Recommendations............... .............

Resource Identification


Flights by County

Known Location Searches
By Address
By Name

"Select a Point" or "Select an Area" Search

Aerial Tile Identification

Search Modification

Aerial Download

Conclusion....................... ...............14...........


Appendices....................... ...............15...........

Appendix A, Informed Consent

Appendix B, Post-Test Discussion












Executive Summary

The University of Florida Digital Collections (UFDC) is a research tool that enables a user to
find resources held at the University of Florida (UF), learn about the physical collections and use
online full content of that resource. The University of Florida Digital Library Center (DLC)
developed this digital library to house the digital resources (e.g. monographs and serials, maps
and other images) and to provide open access to remote and local users. The online aerials
collection, Aerial Photography: Florida, is a sub-collection of the UFDC that is used for
education, historical evaluation, general planning purposes and aerial photograph preservation.

The Aerials Collection was previously searchable through a GIS interface
(http://smathersntl13.uffib.ufl. edu/fta2/viewer.htm) that runs on the ArcGIS server. In an effort to
increase use and accessibility by middle and high school students and the general public, a more
user-friendly search interface was developed that incorporated a new Google Maps-based search
option (http://ufdc.ufl. edu/aerials). Google Maps allows searching by address, point, or area to
access the aerial photographs that cover a selected point or area. The new Google Maps search
option offers more flexibility for location searching and that functionality may be integrated
throughout UFDC. Because the option may impact other sub-collections of UFDC, user testing
was written as a requirement of the From the Air: the photographic record ofFlorida 's lands
Phase III grant.

As with other evaluations of UFDC sub-collections, such as the Florida Digital Newspaper
Library and Digital Library of the Caribbean, user testing provides feedback on the ease of use of
a specific sub-collection and specific functionality that may impact function throughout UFDC.
This particular report calls for few recommendations that may impact location searching within
UFDC. However, actual changes implemented are subject to the limitations of the technical
capability of the open source Greenstone Digital Library System currently used for its metadata
storage, retrieval, and search engine.

In early spring 2010, the Aerials Usability Manager worked with the UFDC programmer to
remove extraneous information and search features of the initial Aerials Homepage using Google
Maps. From the resulting toned-down interface, she developed and administered usability
testing to groups of middle school children and their teachers in late Spring/Fall 2010 and to a
group of university participants in summer 2010. The testing examined how well the Google
Maps location search allows for ease of use, navigability and learnability. Key areas of concern
were identified prior to testing:

Is the main aerials homepage intuitive?
Are the multiple search options usable and intuitive?
Are users able to understand the relationship between the Google Maps search interface
and the aerials themselves based on the interface text and search options?
Can users properly modify a search to find more precise locations?
Can users download aerials from the aerial page view?
Can users locate aerials for a particular date or range of dates?











The testing resulted in feedback with the following overall findings:

The Aerials Google Maps interface, while an improvement over the GIS-ARC interface,
is not intuitive
Users who are not familiar with aerials often do not understand the relationship between
the Google Maps result pages and the aerials page view
The search options on the Map Search interface need clarification and additional
instructional tools
More self-paced educational tools are needed

The following report will provide a summary of findings, including:

User difficulties and frustrations with the resource
Significant usability findings and recommendations

Methodology

Prior to testing, the Assessment Team Usability manager submitted a University of Florida
Institutional Review Board Protocol (IRB). After submitting the documentation and the survey
instruments that focused only on testing of middle school and high school participants, the
manager was informed by the IRB Office that the protocol was expedited. In order to test,
parents/guardians of minor participants were required to sign a copy of the approved informed
consent document (see Appendix A, Informed Consent).

The Team determined that testing of university participants would supplement and provide a
complete look at aerials searching than the IRB-covered participants alone.

Minor changes occurred to the resource between the testing administered at Roosevelt Middle
School, The University of Florida, and P.K. Yonge Developmental Research School, including:

1. Deletion of the "Flights By County" Tab
2. Deletion of list of years available in "Flights by County"
3. Addition of filtering "Narrow Results" tool

These changes seemed to have no effect on the level of usability of the resource, as this report
will demonstrate through statistical and analytical evidence.

Who and Where We Tested
The individual usability test was administered to three groups:
1. Roosevelt Middle School in West Palm Beach, FL (henceforth known as Grozy One):
141 participants, May 13, 2010
2. University of Florida in Gainesville, FL (henceforth known as Grozy Two): five
participants, August 11, 2010
3. P.K. Yonge Developmental Research High School in Gainesville FL (henceforth known
as Grozy Three): 24 participants, September 14, 2010





















Group Two-University of Florida

Undergraduate 1
Graduate 3

Staff 1


Group One-Roosevelt Middle

Faculty 3
Students 138

TOTAL 141


Group Three-PK Yonge

Faculty 0
Students 24

TOTAL 24


No online search experience 0
Novice user 0
Profieient user 1
Highly experienced user 2
Expert 12
No Response 5
TOTAL 5


No online search experience 3
Novice user 11
Profieient user 51
Highly experienced user 43
Expert 20
No Response 13
TOTAL 131


No online search experience 0
Novice user 1
Profieient user 8
Highly experienced user 11
Expert 14
TOTAL 24


Testing observation and discussion, in all cases, was conducted in a computing environment.

Faculty and student participants from Group One and Group Three were recruited by email to
personal contacts of the Library's grant manager at the University of Florida prior to the award
of the grant. Participants from Group Two were frequent library users who responded to an open
invitation by the Usability Manager. The participants reported the following profie
characteristics:


33 TOTAL


Number of Groups


5 Number of Groups


Self-reported Web Usage and Skills
Searching skill levels and frequency of web usage are all self-reported values gathered from pre-test
questionnaires.
Group One-Roosevelt Middle Group Two-University of Florida


Group Three-PK Yonge High


N= 5


















































Library Database like Factiva or
Lexi sNexi s 3
Google 103
Another WWW search engine 76


Use of truncation and wildcards 11
Use of Boolean operators 9
Simplenavigaton of resource 73
Saving items to a basket 93
Help screens 46
Consistent navigation 53
Ability to search seific fields 87
Ability to retrieve a variety of
formats (e.. images, PDF, text) 92


Frequency of use of Google Maps or
Another Map Service-Group One

4%lll


11f)8lVf


Frequency of use of World Wide Web
Group One


I GROUP 1 N=141 I



Frequency of use of World Wide Web Frequency of use of Google Maps or
Group Three Another Map Service-Group Three










GROUP 3 N=24


+ For Group Two participants, 100% of the respondents reported using the World Wide Web and
Google Maps or another map service daily.

+ Among all groups (N=170) participants reported the following preferences with regards to online
fearues and preferred resources used to locate information:
Resources Used to Locate Information Important features of online resources as
reported by all participating Groups









What data we collected
The assessment team members collected data that would address the maj or elements of the
Aerials Photography Collection including homepage design, navigation, ease of use of searching
features, the result pages, and the aerials view.

Initial Resource Impressions

At the beginning of each scenario-based test session, participants were allowed to preview the
resource. Participants were instructed to answer the following questions:

What do you think this resource will provide?
What years do these maps cover?
Why might you use this resource?

Among participants of Group One, the most common responses (N=109) indicated that the
resource will provide useful information to locate addresses, find directions to places, old
pictures, as well as information about an area and how to get there. A small number of
participants (N=29) reported that the resource could provide information on aerial photography,
history, and information on historical places based on homepage text and access to maps using
the Google Maps interface.

Similar to Group One, participants from Group Two reviewed the interface of the Google Maps
view and used the features within it, such as the map, satellite and terrain features. Among
Participants of Group Two, three participants indicated that the resource provides information on
aerial photography, geographical information that can be useful for boundary disputes, and
historical information with regards to changes in landscape. Two participants believed that the
resource can be used to find directions to places, information with regards to roads and about
different maps using the Google Maps view.

Among Group Three participants, the most common response (N=20) indicated that the resource
will provide information about maps, the landscape of Florida, history of the changes in the
topography of Florida, and information about aerial photography (students seem to report mainly
the description provided on the homepage of the resource). Only two students reported that the
resource would provide directions to places and maps of specific locations. Two students
refrained from answering this question.

The overall assumption among the maj ority of participants regarding resource content
emphasized that the function of Google Maps is to cover directions to places, specific known
addresses, and an overall view of an area.










What are your initial impressions of this resource?
Group One Group Two Group Three
* Self-reported initial Excitement about the existence Self-reported initial
impressions of the of the resource impressions of the
resource by Group One A resource that seems fairly resource by Group
were not available due to easy to use Three were not
time constraint An improvement over the aerial available due to time
database (GIS) originally used constraint

What did you like about this resource?
Group One Group Two Group Three
* Participants did not report Ability to use searching options Participants did not
anything specific that they such as year and county report anything
liked about the resource Variety of years available specific that they
Instructions improve usability liked about the
of resource resource

What did you dislike about this resource?

Group One Group Two Group Three
* Resource needs to have Search cannot be easily The Resource is rather
additional instructions for modified complicated and
students to follow Unclear instructions difficult to utilize
Resource can be mistaken to be
Google Maps
Instructions are not readily
provided on the map search
page
Search instructions lead users
to believe that resource is
limited to searching only
addresses
Aerial photography is not
smle to understand










What type of information would you expect to find on this site?
Group One Group Two Group Three
* Directions to places Directions to places Information about the
* Pictures of locations such as Information about Florida landscape of Florida
houses hi story History of the changes
* Images of the world To locate a place in the topography of
* Maps with directions to To find better information the state of Florida
places of interest and directions related to Information about
* Information getting to an address aerial photography
* Stuff Aerial photography
* Aerial Information Knowledge about geography,
* Information useful for boundary lines and
proj ects geographical features
* University of Florida
information
* Historical information about
Florida and the world




Scenario-Based Testing

The Scenario-Based questions were utilized to test the navigation intuitiveness through the
Aerial Photography Collection. Participants were presented with scenarios in which they were to
provide answers with specific steps to reach the desired results. Participants of Group One
(N=138) were broken down into groups of five people each, (N=33 small groups for Group
One), while participants from Group Three (N=24) were broken down into groups of two people
each (N=12 small groups for Group Three). Participants from Group Two were tested
individually.

Limited assistance was provided to the participants by the Assessment Team when necessary.
Participants were asked to complete simple tasks such as determining the years of photography
available for Palm Beach County and reporting the number of flights taken during specific years.
The questions were presented in order on a list format to ensure that participants explored most
features of the resource. Difficulties experienced by participants while using the resource are
addressed in the Initial Resource Impression and the Findings and Recommendations section of
this document. All questions used in the testing can be found in the following section of this
document.











Question Function/Feature Tested
1) How many years of aerial photographs are available for Palm Homepage
Beach County?
la. How many flights were taken in 1953? Item Searching
*Tab Navigation
lb. Who is the creator of those images? Tab Navigation
2) What was located at the 1900 N Australian Ave., West Palm Beach in Map Search Feature
1953? Tab Navigation
*Recognition of Results
Page/Aerial Photograph
2a. Download the item Link Location/Navigation
3) Was the Palm Beach Mall in existence in 1953? Map Search Feature
Tab Navigation
Recognition of Results
Page/Aerial Photograph
3a. What structures appear in the 1968 aerials at that location? Tab Navigation
*Navigation of the Results
Page/Aerial Photographs
4) How many tiles cover the Palm Beach Mall in 1968? Tile Identification
4a. What are the tile numbers that include a picture of the Results Page Navigation
Mall's location? Tile Identification
4b. Modify your search to cover Clear Lake to discover Resource Navigation
what tiles include images of the Lake? Tile Recognition
4c. How many tiles cover Clear Lake in 1968 Tile Recognition
4d. What are the tile numbers? Results Page Navigation
*Tile Identification

































Findings and Recommendations

The following findings and recommendations provide information from user testing and the
individual post-test discussion that concluded the testing of Group Two participants. The full
comments of the post-test discussion are located in Appendix B.

Participants from Groups thought the resource was overall difficult and cumbersome to utilize.
The site requires some modification to increase ease of use. The following are specific issues and
recommendations:

Participants had trouble locating the answer to the question "What year do these maps
cover?" as most of them reported years such as the 2000s, an answer that is consistent
with images from Google Maps rather than an Aerial picture provided by the database
Participants who provided answers located on the website seemed to have a difficult time
distinguishing between the years documented by the photos and the years available by
county, as both answers lead participants to believe that either one of the dates available
could be the appropriate answer for the years of aerial photography available
Some respondents reached a reasonable level of success with completing the scenario-
based questions. However, most participants were unable to easily navigate the Aerials
Website. The small groups that comprised Group One (N=33 small groups) refrained
from answering the questions, possibly due to the level of difficulty of the task or lack of
a user-friendly interface. Evidence of the difficulty of usage of the resource can be
observed as students explicitly stated that they either could not locate the answer or found
the answer but reported that it was difficult to locate. Very few participants from the
Groups tested provided the steps taken to complete the task, and out of those that did,
most of them were able to locate the right answer and complete the task successfully


* Homepage Navigation: describes the ability of the user to navigate the Homepage to locate
basic information regarding the resource.

* Item Searching: question prompts user to locate specific items in the collection.
* Tab Navigation: tests intuitiveness and usability of the navigation tabs located on the resource.
* Map Search Feature: tests the usability and intuitiveness of the "Map It" feature and the
usability of the Google Maps interface for locating and Einding an aerial view of an address in
question.
* Recognition of Results Page/Aerial Photograph: tests whether users can access the aerial
photographs and the results page pertinent to the information they are trying to locate.
* Link Location/Navigation: tests the intuitiveness and ease of recognition of links available to
users in the resource.
* Navigation of the Results Page/Aerial Photographs: tests the intuitiveness of the navigation
feature of the results page, including whether users are able to recognize that the results page
includes Aerial Photographs.
* Tile Identification: provides information with regards to user's familiarity and recognition that
the Tiles are Aerial Photographs.
* Resource Navigation: tests whether or not users can successfully navigate the entire resource to
perform tasks, such as modifying a search to include additional landmarks.












Resource Identification
Participants were asked to provide the years covered in the collection. Most participants from
Group One and Two were able to readily identify the answer located on the Aerials Homepage.
The maj ority of participants from Group One and a sizable minority of Group Three participants
often reported the documented years (193 7-1990) instead of the current available years (193 7-
1970), as the information reported on the homepage seems to be easily interchangeable.

Comments/Supportinn Evidence
Fifty two students from Group One believed that the Images provided by Google Maps were
Aerial Photographs based on the initial impression of the resource. One participant from Group
Two could not identify aerial photographs. Four participants from Group Two could not readily
locate the instructional/help tools. Seven participants from Group Three could not report the
current available years of aerial photography due to lack of clarity of information on the
Homepage.

Recommendations
Create new tab on the Homepage labeled "Help" to provide users with information on
terminology, images of the expected results and instructions on what is contained in the
resource
*Provide a more thorough distinction on the Homepage and on the Map Search page
between Aerial photographs and Google Maps images
Clarify the actual years available of Aerial photography by providing the information in
bulleted format

Flights by County
Participants were asked to report the years of Aerial Photography available for Palm Beach
County. Most participants from Group One and Three were able to successfully report the
answers to some of the required tasks. Participants had difficult time determining which tab to
use (i.e. Map versus Flights by County Searches). Participants from Group One were exposed to
the resource prior to changes, such as the elimination of the table format of the number of years
of aerial photography.

Comments/Supportinn Evidence
Among participants from Group One, 70 students reported the wrong answer to the years
available for Palm Beach County. Three participants from Group Two were partially successful
in searching/locating the required information. Participants used features of website such as
links, tabs, and quick-links available under the "Narrow Search" options. Two participants in
Group Two attempted to utilize the "Narrow Results" box on the "Flights by County" screen.
Filtering options provided in the "Narrow Results" box were inconsistent with the results on the
Flights by County Tab (i.e. years did not match). Two participants from Group Three mistakenly
accessed the GIS search system as a means to complete this task, reporting the wrong answer.

Recommendations
*Use the brief view as default









* Develop an instructional tools for Flights by County searches (create a new tab on the
Homepage labeled "Help" that will readily provide users with instructions on how to
distinguish between search options)
* Clarify on the Homepage the use of GIS as an alternate tool to search for Aerials

Known Location Searches


By Address. Participants were asked to report what was located at 1900 N Australian Ave., West
Palm Beach in 1953. Participants from Group Two utilized the zoom features of Google Maps to
report an answer of a "building" based on the current images provided by Google Maps; they did
use the aerial view to answer the question. One participant used the "Select an area" option on
the search map but was unable to select the area without considerable problems (e.g. click-drag
issue). Six Group Three groups performed the search by typing the address in the address bar on
the "Map It" tab, while the other six groups did not complete the task. The answer, if the search
is performed appropriately, is that a field (or "nothing") was located at that address in 1953.

Comments/Supportinn evidence
Two Group One groups reported that they could not find the answer, while 11 groups reported
the wrong answer. Among participants from Group Two, two participants provided the right
answer to the question by searching the address and using the aerial. Three participants from
Group Two did not reach the aerial, remaining in the Google Maps view. Five of the six Group
Three groups utilized the "Map It" tab as intended but did not reach the aerials, as they reported
an answer that is consistent with the images of Google Maps (namely, Lincoln Park). The lack of
success in completing this task can be attributed to the fact that users are not aware that they
must click the "Search" button after they perform the address seach.

Recommendations
* Provide directions on using the known location search by address, including the need to use
the "Search" button located on the map to access the actual aerial of interest
* Clarify the difference between flight lines and all volume-related flights (see brief view for
example)
* Continue to use map view as default since it includes years in result title, but change Google
Maps image to aerial image
* Consider the removal of the "Press to" button on the Google Maps interface in the Map It
view immediately after a search for an address has been performed

By Name. Participants were asked to report whether the Palm Beach mall was in existence in
1953. The correct answer indicated by the aerial photograph is that the mall was not in existence.
Detailed steps taken to find the answer were not provided by the participants from Group One or
Group Three. Among Group Two, all participants were uncertain in how to proceed, as the
address bar specifically says 'type an address.' One participant used the navigation links to go
back to the home page, clicked on map Search and attempted to search the address of the mall,
eventually deciding to type in Palm Beach mall on the address bar and successfully accessed the
aerial picture. Once result came up, the participant used the zoom tool to try to locate the mall.










Three participants proceeded to search for the mall by accessing the flights by county link,
locating Palm Beach County and choosing the aerials of 1953. Participants proceeded to click on
available aerials and used zoom tools to locate the mall by scanning the picture and/or comparing
satellite images provided on the map search page to the aerials from 1953.

One participant used the map search page and attempted to type in Palm Beach mall in the
address bar. The participant expressed concern in typing the name of the mall as the address bar
indicates the user to type in an address instead of a landmark and/or location. Once aerials were
located, the participant proceeded to browse through all available pictures of 1953 by using the
zoom tool.

Among Group Three, only two groups successfully completed the task out of the possible
twelve; one group reported that they utilized the "Map It" feature to determine whether or not the
Palm Beach mall was in existence in 1953.

Comments/Supportinn Evidence
Most participants from Group One and Three provided no answer to the question. Out of those
who provided answers from Group One, only 30.3% of small groups provided the right answers,
while 6.1% provided the wrong answer. Three participants from Group Two reported that the
mall was in existence in 1953. Furthermore, participants from Group Two expressed confusion
over the use of "Find Address" and "Search" buttons on the map search page, as the functions of
each of the buttons are not explained. Out of the participants from Group Three who provided
answers, only 16.7% were able to complete the task and provide the right answer.

Recommendations
* Address bar suggestions should be modified to include "Type an Address or Name of a
Landmark"

"Select a Point" or "Select an Area" Search

Very few users attempted to use this search option. Participants attempted to use this search
option to modify their search. Participants expressed confusion over the use of the options and
the difficulty level in selecting an area accurately. This selection refers to choosing a particular
area from the "red box" that overlaps the entire map.

Comments/Supportinn Evidence
Participants from Group Two commented that the features were difficult to use due to lack of
instructions and changes in the text of the feature when it is used.

Recommendations
* Improve the sensitivity of the Select a point" or Select an area" options
* Further explanation of the Select a point" or Select an area" is necessary, as it will allow
users to utilize the Map Search feature in a more intuitive way









Aerial Tile Identification

With the use of Google Maps, the search result page is able to provide the outline of the aerial
photograph so that area of coverage is readily seen.

Once participants located results page, they were asked to identify the tiles and numbers that
cover the Palm Beach Mall in 1968. The correct answer is six tiles. Detailed steps were not
provided by any participants from Group One and GroupThree.

Two participants from Group Two counted the number of tiles reported by the resource, one
counted the tile numbers reported on the "Map button" located on the map screen while another
participant reported the number located on the menu with the tile numbers on the top left side of
the screen. One participant attempted to identify the tile numbers by navigating back to the home
page, clicking in Flights by County, choosing 1968, and browsing through most of the available
tiles to locate the mall.

One participant used the back button on the internet browser to come up to the Google Maps
image with the available tiles for the entire Palm Beach County and used the zoom feature to
locate the mall and determine the number of tiles that included the mall. Participant clicked on
each tile that included the mall and looked at aerials. Another participant did not know what a
tile was. Participant attempted to go back to the aerials Home Page, clicked on GIS site to try and
locate answer. Participant followed to go back to home page, clicked on Map Search tab and
typed in Palm Beach mall, examined image provided by Google Maps and tried to locate tiles.
Participant readily recognized that the Google image was not an aerial; however, they were not
able to locate the aerials and/or the Tiles. Participant reported that it was difficult to find because
the Map search function readily calls for an address.

The great maj ority of participants from Group Three were not able to complete the task at hand
due to time constraints. Only one group out of the possible twelve was able to complete the task.

Comments/Supportinn Evidence
Two participants in Group Two did not know what a Tile was or how to locate it. Furthermore,
only three groups (out of thirty three) from Group One readily understood what a Tile was, as
observed by their answers to the task. Among Group Three groups, only one group (out of
twelve) was able to identify the tiles and successfully completed the task at hand.


Recommendations
* Clearly define a tile on the result page (e.g. by use of a legend)
* Create a help section that describes the differing parts of the aerial view, including tile types,
and the relationship between Google Maps and an aerial
* Make tile numbers larger on the result page
* Highlight the brief view box that shows the aerial photographs









Search Modification
Participants were asked to modify their search to include Clear Lake. Successful participants
from Group One recognized tiles, the results page, and were able to navigate the resource.
Among Group Two, one participant utilized the "Modify Search" link within the results page to
modify the search to include Clear Lake. Three participants did not utilize the Modify Search
option within the results page to include Clear Lake in their search. Two of the three participants
used the zoom tool in the "Map It" tab to locate the number of tiles that cover both Clear Lake
and the Palm Beach Mall; using this method resulted in the apparent retrieval of tiles that
matched which did not actually include the location desired.

The third participant utilized the "Thumbnails" tab, selected flight corresponding to Palm Beach
Mall and Clear Lake, counted and identified the number of tiles that included both landmarks.
Another participant did not know how to modify the search to include Clear Lake. The
participant browsed through the aerials available for 1968, used the "find" feature available in
web-browsers to try to locate Clear Lake, and searched through citations and thumbnails in an
unsuccessful attempt to modify the search.

Comments/Supportinn Evidence
One group out of the thirty three participating groups from the Group One sample was able to
successfully modify their search and provided the right answers to the task at hand. Furthermore,
three participants from Group Two and all participating groups from Group Three were not able
to modify their search.

Recommendations
* Make the "Modify Search" option more visible
* Provide instructions on how to modify searches on the results page
* Deemphasize the "Map It" tab after results have been retrieved to prevent users from
inaccurately counting those as correct results
* In the modified search, only highlight the tiles with the desired location

Aerial Download
Among the new functionalities of the Google Maps search is ease of downloading aerials in
JPEG2000 file format. Once a participant has located the requested aerial, the download option is
available for all tiles in two locations on the aerial result view. The hyperlink for downloading is
nested in the functions bar located at the top of the aerial, along with a download tab also
available for use. The question sought to see how intuitive it would be for a user to locate it.

Comments/Supportinn Evidence
Most participants from Group One did not provide an answer to the download feature with
regards to the location of the link or how to download the item. All participants from Group Two
could find the download option; participants clicked on the download tab located at the top of the
results page or on the 'Download This Tab' hyperlink. All participants from Group Three were
unable to download the Aerial due to technological difficulties with the computers or because
"they did not know how to or could not do it," as reported by them.










Recommendations
Indicate in the Homepage of the FAQ tab the available option to download items
Make the Download link more visible to users by changing the size and/or location


Conclusion


Participants were generally not successful in navigating/locating resource features. Some of the
features of the resource that seemed more difficult to participants included:
Map Search and use of the Address Search Bar
Identification of an Aerial Picture and Tiles
Modification of Search

In general, participants from both groups had a difficult time identifying and utilizing aerials.
Participants attempted to utilize the aerial photography as interactive resources such as Google
Maps. After being exposed to the resource and asked to complete the tasks, participants self-
reported that the resource, while originally thought to be easy to use and interactive, is rather
cumbersome and difficult to use. Evidence of this finding is reflected on the answers provided by
participants with regards to the usability of the Aerials Resource. Some of the comments
provided by the participants include:

Resource is not user-friendly
Resource is not intuitive
Resource is not easy to utilize and navigate
Resource lacks appropriate instructions with regards to its usability
Resource lacks appropriate orderliness for participants of all ages to easily follow

Overall, participants were not successful at navigating the resource and thought that it was
difficult to utilize.











Appendix A, Informed Consent

W UNIVERSITY of



George A, Smaathers Libraries 1500 Normran Hall
Education Library PO Box 117016
Gainesvlle, FL
32611-7016

352-273-2780
352-3924789 Faxt
www.uilib. udRlU/ educ/

Dear Parent/Gunulian.

I am a facukty member in the ieorge A. Smothers Libraries at the University ofFlorida, conducting research on the
Aerial Photography, Florida H~istorical oollction (Aerial collection) which provkles access to Florida aerial
photographs documenting changes in Florida's land use from 1937 to 1975. The purpose ofthis study is to examine
information discovery and retrieval using this specialincd online resource. Thbe results of the study may include
development of more clic~tive resomres with intuitive home~page, sarch interfacces practical scarcit capabilities, and
effetbive tresult pages. With an improved interfooe, we hope to ensure tweeder access and incr~eaed use by mkidice and
high school sandents and the general public. These munlts may not direcly help your child today, bto may benefit
future students. With your permission, I would like to ask your child to volunteer for this reseesch.

Studemts participating in this research winl be adred to complete a pre-rest questionnaire about their experience with
using the World Wide Web. The research involves a full class group exercise, srcenari-bas~ed tasks exercise to
evainate search behavior, and a grup focus/discussion session. The first group exercise will consist of participants
reviewing and commenting on the resource. The scenario based testing completed bry small groups will consist of
structured exercises using the Acrials collection. A follow-up full participant focus session will be used to review the
tasks excacise and will gather additional infomration from the participants including feedback; about their experience
using the Aerials collectionl. 111is research is scheduled during a school period.

The children will not be asked to write their names on their pre-test questionnaires- their identity will be kept
confidential to the extent provided by law. ResuHs will only be reported in the form of group data. Participation or
non- participation In this study winl not affect the children's grdes or placement In any programs.

You and your child have the right to withdraw consent for your chikbs participation at any time without consequence.
There use no known risks or immediate benefits to the participants. No compensation is offered for participation.
Group results of this study will be available upon reqluet. If you have any questions about this teseaseh protocol,
pha~se contact me at (352) 273-2627. Questions or conourns about your child's rights as research participant may be
directed to the IRBO2 office, University of Florida. Box 1 12250, Gainesvilic. FL 32611 (352}392-0433.

Marilyn N. Ochoa



I have read the procedure described above. I voluntarily give my consent for my child, ,to
participate in Marilyn N, Othos' study of the Aerial Photography: Florida Historical collection. I have receive a
copy ofthis description.


Paren / Guardian Date Approved by
University of Florida
Institutional Reviewv Board 02
2d Parent / Witness Dae Protocol # 213U08
For Use Through 03-22-2011
The Foutrdation for Thbe Garer Nation
ca, qualOppommtly nditulo~n





























Searching options such as year and Search cannot be easily modified
county Unclear Instructions
Variety of years available Similarities with Google Maps
Instructions improve usability of resource Instructions are not readily provided on
the map search page
Instructions lead users to believe that
resource is limited to searching only
addresses
Aerial Photography is not simple to
understand

* Did you find anything particularly easy to complete? Did you find anything confusing?

Easy Confusing
Typing in address, as the search option Finding structures and landmarks
recognizes anything Lack of information in certain links
Finding aerials by year and county located on the resource
Finding addresses on the Map Search
Looking for a seific destination

*Was this resource easy or difficult to use ?
a. On a numbered scale from one to five, with five meaning very difficult to use,
participants rated the resource an average of 3.5, indicating that participants consider the
resource difficult to use in general.

* What do you wish you could do m ithr this resource ? Common answers included:
a. Search for specific landmarks like highways (an Atlas)
b. Perform research, find addresses.
c. Nothing, as I am not interested in the resource


Appendix B, Post-Test Discussion

Some of the recommendations in the previous section are based on the answers in this section. Upon
completion of the Usability tests, the Assessment Team encouraged a facilitated discussion to gather
further information about the participants and their reactions to the resource. The following
questions were addressed during the facilitated discussion:

* What were your initial impressions? Did your attitude about the resource change as you used
it?
a. Attitude changed from excited to frustrated
b. In general, participants thought resource would be easy to use
c. Participants concluded after using the resource that it is rather complicated and
cumbersome to utilize.

* What do you like about the resource? Why? What do you dislike about the resource? Why?


Liked


Disliked











* What could be changed to make the Map search interface easier to use ? What do you think
about having multiple search options on the Map search?

a. Making navigation simpler
b. Comparing current addresses with previous addresses
c. Have the option to search by Highway, location, etc.
d. Explicitly state directions instead of having to search for them on the resource.
e. In general, participants agree that having multiple search options on the Map search
would be helpful.

* [Review of resultpages] What do you think of the current initial result view ? Which of the
views (briefJ table, thumbnail or nzap) is easiest to understand?

a. Out of Group two, which provided answers to this question,
i. Two participants thought Thumbnails are easier to understand
ii. Two participants thought the map view was easier to understand
iii. One participant thought the brief view was easier to understand

* What could be changed to make getting to the aerial you want easier? What nzodifications, if
any, do you think the resultpages need? Common answers include:

a. Provide help screens while utilizing the resource
b. Provide more information in thumbnail images
c. Provide a marker to denote specific landmarks like highways.
d. Provide the option to perform a more specific search
e. More search options, such as city, neighborhood, etc.
f. Provide a regional labeling of the aerials.

* Did you use help tools throughout the site? Was the legend useful? The terminology page?

a. Most participants used the help tools through the site but found them not explicit/useful
enough to make their experience with the resource easier.
b. Participants found the legend useful.
c. Most participants did not locate the terminology page, as it is not readily available.




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs